Apprentice docs: cut LLM cost without losing quality

Use this when you already have a CSV of inputs and correct outputs for one repeatable LLM job. The steps below create a task, upload the rows, run prompt optimization, and print the score change.

No account, no key: pip install 'runapprentice[optimize]' then apprentice optimize <task> --local --data golden.csv runs the same optimization on your machine with your own OpenAI key. Add [local] for MLX fine-tuning on Apple silicon. No golden dataset yet? An API key also lets Apprentice generate a starter set and route rows to a subject-matter expert for review, and tracks every run in the console.

Install

Inside a project, where your code also imports the SDK:

pip install runapprentice

Prefer the CLI as a standalone tool? uv installs it isolated from your projects, or runs it once without installing anything:

uv tool install runapprentice              # permanent, isolated `apprentice` command
uvx --from runapprentice apprentice --help # one-off run, nothing installed

LangChain support is optional. Install it only if you plan to capture from a LangChain app or pass a LangChain prompt:

pip install "runapprentice[langchain]"

Set your key

export APPRENTICE_API_KEY="ap_live_..."

Create the API key once in the console. The client reads it from the environment, so you can leave it out of your code. It talks to the hosted Apprentice API by default; you only set base_url (or APPRENTICE_BASE_URL) to point at a self-hosted backend.

Run it

import os
from runapprentice import Apprentice

client = Apprentice(api_key=os.environ["APPRENTICE_API_KEY"])

client.tasks.create("invoice-json", metric="json_f1")
client.datasets.upload(
    "invoice-json",
    path="golden.csv",
    input_col="input",
    output_col="output",
    prompt="Extract the invoice fields as a JSON object.",
)

report = client.optimize("invoice-json").wait().report()
print(report.baseline_score, "->", report.optimized_score)

# Run it: messages() rebuilds the messages the score was measured with.
best = client.prompts.get("invoice-json")
messages = best.messages(input=invoice_text)

golden.csv needs an input column and an output column. For a JSON task, the output is the exact JSON you want back.

What you get

report.baseline_score and report.optimized_score are field-level scores on rows the optimizer held out, so treat the result as proof for this dataset, not a universal benchmark. report.optimized_prompt is the rewritten instruction. Send it with client.prompts.get(task).messages(...) rather than pasting it: on its own it has no slot for your input, so a hand-rolled call can drop the input without erroring.

If optimized_score does not beat baseline_score, that is a real result, not a failure of the tool. Add cleaner verified rows, or pick a metric that fits the task, then run again. We never report a gain that is not there.

JSON extraction, end to end

The same flow with a real dataset and the optimized prompt pulled back into code.

Capture from LangChain

Build the dataset from your live traffic instead of a CSV.

Model replacement comes after this first prompt win. It is described on pages marked Building until the public flow ships.

Get started

How-to guides

Concepts

Reference

Quickstart

Install

Set your key

Run it

What you get

Next

JSON extraction, end to end

Capture from LangChain

​Install

​Set your key

​Run it

​What you get

​Next

JSON extraction, end to end

Capture from LangChain

Install

Set your key

Run it

What you get

Next