Skip to content

Run your first paper

Turn a research paper into structured JSON: standalone facts, equations, visual findings, and named entities you can use in RAG or a scientific knowledge graph. To see the output first, browse the published results without an API key.

Requirements

  • Python 3.11+ and uv.
  • A GROBID service. The Docker command below starts one locally.
  • An OpenAI-compatible Chat Completions endpoint with image support, and its API key if required.

GROBID receives the PDF; the model endpoint receives page images and scientific context. Model serving and GROBID run separately from Distils.

Install Distils

Clone the repository and install its dependencies:

sh
git clone https://github.com/maureille/distils.git
cd distils
uv sync

Alternatively, install from the repository root with python -m pip install -e . using Python 3.11+. The commands below use uv.

Start GROBID

In a separate terminal, run:

sh
docker run --rm --init --ulimit core=0 \
  -p 127.0.0.1:8070:8070 grobid/grobid:0.9.1-crf

On Apple Silicon, add --platform linux/amd64 after docker run. See the GROBID Docker guide for service setup.

Configure the model

From the Distils repository root:

sh
cp config.example.json config.local.json

Edit config.local.json. Replace base_url with your Chat Completions API root and model with an image-capable model ID served by that endpoint:

json
{
  "base_url": "https://your-endpoint.example.com/v1",
  "model": "your-vision-model-id",
  "api_key_env": "OPENAI_API_KEY",
  "workers": 1,
  "grobid_url": "http://127.0.0.1:8070"
}

The client appends /chat/completions to the API root. Set OPENAI_API_KEY in your environment to the endpoint's API key. Change api_key_env if you use another environment variable. For a server that needs no bearer authentication, set api_key_env to null; no Authorization header is sent unless you configure one explicitly.

The example configurations include OpenRouter and OrcaRouter setups. See Configuration for custom authentication, provider options, and optional cost reporting.

Download and process one paper

sh
uv run python examples/download.py
uv run distils-render --input examples/pdfs/2305.13245v3.pdf --run-id quick
uv run distils-run --config config.local.json --run runs/quick

The downloader fetches the seven-page GQA example and verifies its checksum. The renderer prepares the page images; the runner processes them in reading order.

Inspect the result

Open these files:

FileContents
runs/quick/documents/2305.13245v3/document.jsonExtracted content, metadata, glossary, and page records
runs/quick/report.jsonCompletion, costs when available, and timing statistics
runs/quick/artifacts/2305.13245v3/Original model responses, GROBID output, and checkpoints

See the output format for the complete run directory and record contract. A complete document means every required step finished; it does not verify scientific accuracy.

Use your own papers

Pass a PDF or a nested folder of PDFs to the renderer:

sh
uv run distils-render --input /path/to/your/papers --run-id my-corpus
uv run distils-run --config config.local.json --run runs/my-corpus

Rendering creates a new run. Defaults are 200 DPI, a maximum image edge of 2,000 pixels, and JPEG quality 92. Use uv run distils-render --help for rendering options.

After a service interruption, repeat the same runner command to resume saved progress. See Failures & recovery for the requirements and retry policy.

Released under the Apache License 2.0.