Optional LLM: setup and verbs

Everything here is off by default. Paperful’s PDF loop (run / attach / lint / fix-metadata / dedupe) never needs a model. Turn the LLM on only for the five verbs below, and only after paperful doctor shows the LLM row green.

Design rules (see architecture § LLM layer):

  • Local first. Default provider is Ollama on 127.0.0.1:11434. Nothing leaves the machine unless you opt into a remote endpoint, and then every command prints a yellow egress notice.

  • Disk first, apply second. Each verb writes under state/ before any Zotero write; Zotero writes go through the same library adapter as attach.

  • Grounded. Prompts only see item metadata plus text extracted from a PDF already on disk (out/ or state/pdf-cache/). Nothing is fetched from the web for a model call.

  • No silent defaults. API keys live in the environment, never in config.toml; browser_agent is never in the default source list. run appends it only when [llm].enabled and the extra is installed.

1. Install

Ollama (default, local)

# macOS
brew install ollama && ollama serve          # or the Ollama.app menu-bar daemon
# Linux
curl -fsSL https://ollama.com/install.sh | sh

ollama pull qwen2.5:7b        # title / identity / summaries
ollama pull qwen2.5:14b       # recommended floor for `recover` (browsing agent)

No Python extra is needed for Ollama: paperful talks HTTP to the daemon.

Verb

Works well with

Notes

fix-metadata titles, lint identity, summarize, synthesize

7B–12B instruct models (qwen2.5:7b, gemma3:12b, llama3.1:8b)

JSON-capable instruct tags; thinking models are fine but slower. summarize and synthesize set Ollama num_ctx from the prompt, capped by [llm].max_num_ctx

recover

14B+ (qwen2.5:14b, qwen3:14b, larger)

Small models loop on cookie banners and publisher menus; doctor warns under ~10B by tag name

LiteLLM (paid / OpenAI-compatible APIs)

uv sync --extra llm            # installs litellm
export OPENAI_API_KEY=...      # or ANTHROPIC_API_KEY, etc. — provider-specific env var

[llm]
enabled = true
provider = "litellm"
model = "openai/gpt-4.1-mini"          # LiteLLM model id (provider/model)
# api_base = "https://my-proxy.example/v1"   # optional OpenAI-compatible base

Rules: ollama/… model ids are rejected on the LiteLLM path (use provider = "ollama" instead); api_base must be http(s). With LiteLLM, title/abstract/PDF excerpts (and, for recover, page text) leave your machine.

Browser agent (recover / last run lane)

uv sync --extra browser-agent  # browser-use + ollama client; needs Python >= 3.11
uv run paperful session login scholar   # the agent reuses this Chromium profile

Paperful core supports Python 3.10; the extra is marked python_version >= "3.11" so uv sync on 3.10 simply skips it and recover exits 1 with a hint. If your .python-version is 3.10, run uv python pin 3.12 && uv sync --extra browser-agent.

2. Configure

[llm]
enabled = true
provider = "ollama"                 # ollama | litellm
model = "qwen2.5:7b"
base_url = "http://127.0.0.1:11434"
allow_remote = false                # true only for a non-loopback Ollama host
timeout_s = 120
# max_num_ctx = 32768              # cap for the Ollama context window on summarize / synthesize

[fix_metadata]
llm_title = false                   # verb B: grounded title proposals

[lint]
llm_pdf_match = false               # verb C: pdf_identity_mismatch finding
llm_pdf_match_min_confidence = 0.6

[summarize]                         # verb D
prompt_template = "default"         # or a path, e.g. "prompts/summary.md"
max_context_chars = 24000
tag = "paperful-summary"
# dest = "both"                    # disk | zotero | both

[synthesize]                        # verb E
prompt_template = "default"
max_context_chars = 24000
tag = "paperful-report"
# dest = "both"                    # disk | zotero | both
# timeout_s = 300                  # default is max([llm].timeout_s, 300)

[browser_agent]                     # last run lane + recover --item
max_steps = 20
max_wall_s = 300
# during_run = true                 # false keeps recover as --item only
# model = "qwen2.5:14b"             # override [llm].model for browsing only

Full key table: Configuration § LLM.

3. Check

uv run paperful doctor

Row

Green

Amber

LLM

daemon reachable and model tag present (Ollama), or paperful[llm] importable (LiteLLM)

disabled is also green; amber = unreachable, model not pulled, extra missing, non-loopback URL without allow_remote, ollama/… id on LiteLLM

browser-agent extra

browser-use importable and agent model ≥ 10B by name

extra missing (uv sync --extra browser-agent), or model tag looks small

doctor never sends a paid completion: Ollama is probed via /api/tags, LiteLLM by import only. The TTY guide prints the fix for each amber row.

4. Verbs

A. recover — browser-agent PDF recovery

On run, when [llm].enabled and paperful[browser-agent] are available, paperful appends browser_agent after Scholar / EZProxy / htmlpdf. The agent fires only if one of those vault lanes was tried and failed (not merely skipped as inapplicable). Playwright releases the session profile first. [browser_agent].during_run = false turns that auto-lane off. Sci-Hub, when opted in, stays after recover.

paperful recover --item still targets named keys without waiting for other lanes:

uv run paperful recover --item ABCD1234 --dry-run   # shows the start URL only
uv run paperful recover --item ABCD1234             # runs the agent, attaches on success
uv run paperful recover --item K1 --item K2 --no-attach

  • Start URL is https://doi.org/<DOI> when the item has a DOI, else its URL.

  • One item at a time, on the session vault profile. Never in DEFAULT_SOURCES.

  • Hard CAPTCHAs are not solved: the item ends as captcha and is retried on a later run / recover. Timeouts / no download → not_found.

  • Access blocks (403 / “Request blocked” / paywall with no free PDF) are instructed as immediate stop — the agent must not open search engines or support/help pages. If it navigates to Google/Bing/etc. or a support/contact path anyway, paperful force-stops that attempt.

  • As soon as a valid PDF lands in the recover download folder (size stable across two polls), paperful calls agent.stop() so the step budget does not keep running after the click already succeeded.

  • Success lands like any other source: PDF under out/, manifest line with source = "browser_agent", attach through the normal path. A dedicated recover writes state/runs/<stamp>-recover.json; auto-recover on run is counted in that run’s report (command = "run").

  • Prints a disclaimer: you are responsible for publisher terms; page content goes to your configured LLM.

Do not run two commands concurrently against the same vault: both would open the same Chromium profile.

B. Grounded title proposals (fix-metadata)

With [fix_metadata].llm_title = true, items flagged title_all_caps, title_html, or title_filename get a proposed clean title grounded in the abstract and first two PDF pages. ALL CAPS titles already get a deterministic Title Case patch without the LLM; the model can still override when enabled. Proposals that share no content words with the grounding are dropped. They land in state/metadata-patches.jsonl with source = "llm_title" and are written only on fix-metadata --apply.

C. PDF identity check (lint)

With [lint].llm_pdf_match = true, lint asks the model whether the first two pages match the record (title, author surnames, year, DOI). A false, or a true below llm_pdf_match_min_confidence, becomes the finding pdf_identity_mismatch (detail carries the model’s reason and confidence). Nothing is deleted or re-attached; you decide. Model or extraction failure is silent (no finding).

D. summarize — grounded summary note

uv run paperful summarize --item ABCD1234                 # disk HTML + tagged child note
uv run paperful summarize -C BBNJ --to disk               # HTML only; Zotero tree stays clean
uv run paperful summarize -C BBNJ --year-from 2023 -T journalArticle --limit 5
uv run paperful summarize --item ABCD1234 --prompt prompts/one-liner.md

  • Full PDF text is extracted (pdftotext, then pypdf) and reduced to max_context_chars as head + detected headings + tail.

  • The builtin prompt asks for Objective / Methods / Key findings / Limitations in simple HTML. Local models often answer in Markdown anyway; paperful converts headings, bullets, bold, and code fences so the Zotero note renders cleanly.

  • Every summary ends with a provenance footer: paperful · <model> · <UTC date> · prompt <sha8> (prefixed remote LLM when the provider is not loopback).

  • --to disk|zotero|both (default both, or [summarize].dest) chooses the write. disk is state/summaries/<key>.html. zotero is one child note tagged [summarize].tag, updated on re-run rather than duplicated. --to disk leaves the Zotero tree clean. --apply still means “this run must write the note” and exits 1 together with --to disk.

  • Ollama receives num_ctx sized from the prompt (about 3 characters per token, plus reply headroom, rounded up, capped by [llm].max_num_ctx). The model tag has to actually support that window. If Ollama logs a truncation warning, lower max_context_chars or raise max_num_ctx.

  • If [lint].llm_pdf_match is on and the item is flagged, summarize refuses it; pass --force to override.

E. synthesize — summary of summaries

uv run paperful synthesize -C BBNJ --year-from 2021 --year-to 2026 -T journalArticle
uv run paperful synthesize -C BBNJ --dry-run          # counts and chunk plan, no model call
uv run paperful synthesize -C BBNJ --to disk          # state/reports/ only
uv run paperful synthesize --library --to zotero --report-collection BBNJ
uv run paperful synthesize -C BBNJ --force            # ignore the up-to-date sidecar

  • Reads summary notes already produced by summarize. A file under state/summaries/<key>.html wins; otherwise the tagged child note is read. PDFs are not opened again. Items with neither are listed under Not included and are not sent to the model.

  • Notes are packed under [synthesize].max_context_chars. One chunk is one completion. Several chunks are synthesised in batches, then combined. If that still overflows, paperful reduces again, at most three times, then exits 1 and asks you to narrow the scope or raise the budget.

  • The builtin prompt asks for Corpus / Themes / Points of agreement / Disagreements and tensions / Gaps and open questions / Suggested reading order, citing only [Surname Year]. --prompt FILE overrides it; its SHA is stamped in the footer.

  • After the model text, paperful appends a Sources list (key, DOI, and the source note’s model and date), the Not included list, and any [Surname Year] token that matches no source.

  • Disk output is state/reports/<slug>.html plus a <slug>.json sidecar (paperful.synthesis.v1) of source hashes. A later run with the same hashes and the same destination skips the model; --force regenerates.

  • Zotero output is one standalone note in each collection you named with -C (or in --report-collection), tagged paperful-report and paperful-report:<slug>. Re-runs update that note. --library or --item with no collection cannot file a note: pass --report-collection or --to disk.

  • A run report lands at state/runs/<stamp>-synthesize.json and does not replace state/last-run.json.

Custom prompt file example (prompts/one-liner.md):

Write one paragraph (max 120 words) stating the research question, the
approach, and the single most important finding. Plain HTML <p> only.
Use only the metadata and document text provided.

The file’s SHA-256 is stamped in the footer so you can tell which prompt produced which note.

5. Docker

The build-local image does not include litellm or browser-use; the browser_agent lane (run auto-recover and paperful recover) is host-only (it needs the headed-login vault anyway). B/C/D work inside the container against a host Ollama:

[llm]
enabled = true
base_url = "http://host.docker.internal:11434"
allow_remote = true      # required: host.docker.internal is not loopback

and start Ollama bound to all interfaces on the host (OLLAMA_HOST=0.0.0.0 ollama serve). See Docker.

6. Troubleshooting

Symptom

Cause / fix

llm.enabled is false in config.toml

Set [llm].enabled = true

Ollama unreachable

ollama serve not running, wrong base_url, or firewall

model 'x' not in Ollama tags

ollama pull x; tags must match (e.g. qwen2.5:7b, not qwen2.5)

Ollama URL host … is not local

Set allow_remote = true knowingly (egress notice will print)

LiteLLM not installed

uv sync --extra llm

model 'ollama/…' must use llm.provider = "ollama"

Switch provider or model id

recover requires Python 3.11+

uv python pin 3.12 && uv sync --extra browser-agent

browser-use is not installed

uv sync --extra browser-agent

session vault not ready

paperful session login scholar (headed, on the host)

recover ends not_found quickly

Model too small for browsing; try a 14B+ tag via [browser_agent].model

recover never starts during run

Extra missing, [llm].enabled false, or [browser_agent].during_run = false

Summary note shows raw ##

Update paperful (0.5+ converts Markdown); re-run summarize

could not extract PDF text

Scanned PDF. Run paperful ocr -C … then --apply, and summarize again

Report seems to ignore half the notes

Ollama truncated the prompt. Lower [synthesize].max_context_chars or raise [llm].max_num_ctx, and confirm the model supports that window

Not included list is long

Those items have no summary yet. Run summarize for them, then synthesize again