This week thinking machines released Inkling, an open-weights model built for customization and available for fine-tuning through its Tinker API. I've wanted to learn how to fine-tune my own models for a while, and this felt like as good a time as any to start. So I built an agent to do it for me.

Meet RL Tutor, an agent that helps me learn and post-train my own models. I don't want a chatbot that knows LoRA. I want an agent that helps me study the field, build datasets, run experiments, and record what we learn.

My end goal is to fine-tune Inkling, but I'm starting with the Qwen MoE as a first step. RL Tutor plugs into my claude code and codex via MCP.

It's important that I use rl-tutor through cc and codex because that's where I write most of my code.

RL Tutor operates in two modes:

1. Study mode. RL tutor maintains a Markdown wiki inspired by Andrej Karpathy's LLM knowledge bases. Drop a link and it can convert a raw source into knowledge; ask a question and it answers from the wiki.
2. Build mode. When a Tinker key is set, RL tutor gets tools for creating datasets and running fine-tunes. It is designed to compare the tuned checkpoint with the base model and file the experiment back into its wiki.

How it works

I use RL Tutor entirely through my coding agents, because that's where I'm writing most of my code. For example, here's rl-tutor adding a doc to its wiki:

Add https://thinkingmachines.ai/news/introducing-inkling/ to the wiki

The artifacts, datasets, reports are stored locally. I can inspect everything in my editor or in Obsidian:

As the wiki grows, it becomes the context for the next experiment. When an experiment finishes, or fails, the report becomes another source the tutor can learn from.

Current status

RL Tutor is a work in progress. The code is available at ashpreetbedi/rl-tutor.

I've tested the study loop and completed two small, paid Tinker runs. The first run trained but exposed a sampling bug. The second completed end to end and produced clean outputs.

That proves the training, checkpoint, comparison, and reporting pipeline can run. It does not prove the fine-tuned model learned the behavior or is usable in any shape or form.

Try it with me

To try RL Tutor with me, clone the repo, setup env + credentials, and run the app.

# Clone and cd
git clone https://github.com/ashpreetbedi/rl-tutor && cd rl-tutor
 
# Setup env
uv venv --python 3.12 .venv && source .venv/bin/activate && uv sync --locked -q
 
cp example.env .env
# Export the following keys or set in your .env
export OPENAI_API_KEY="sk-..."
export TINKER_API_KEY="..."
export PARALLEL_API_KEY="..."
 
# Run RL Tutor on AgentOS
python app.py

Your RL Tutor is live at localhost:7777. Access it using:

• Chat UI. Open os.agno.com, connect to http://localhost:7777, and chat with the tutor, browse its sessions, memory, and runs.
• REST API at localhost:7777/docs, because it's FastAPI.
• MCP server. Add it to Claude Code and the tutor becomes available in your coding agent

Add RL Tutor to your coding agents

This is KEY for me. Open another terminal, and run

uvx agno connect --url http://localhost:7777 --server-name rl-tutor

Then open (or restart) claude code (or whichever coding agent you use) and ask: "can you access the rl-tutor mcp?"

First training run

Ask claude code to prepare our first training run:

ask rl-tutor to teach me how I would train my first model with Tinker. This is a
walkthrough only: do not call any Tinker tools or create any files. Use the
canonical 30-example haiku experiment and give me five numbered steps covering
the Tinker key, dataset design, saving the dataset, running the fine-tune, and
inspecting the report. End with a separate prompt I can send later if I choose
to run it, then stop without training anything.

The first experiment is intentionally small: 30 examples that teach a model to answer in haiku, followed by a base-versus-tuned comparison. Once the walkthrough makes sense, I can run the fine-tune using:

Run the haiku fine-tune now. Create and save a strong 30-example dataset, wait for the saved confirmation, then run the finetune.

BTW a completed report only means the pipeline completed. It does not prove that the model learned the behavior.

What's next

I'll continue to learn and explore, turning what I learn into datasets, running small experiments, comparing the results, and filing what happened back into the wiki. After that, I'll keep the experiments small and change one variable at a time—dataset quality, epochs, learning rate, or model. The long-term path still points toward Inkling, DPO, and RL, but I want to earn the steps.

If you try RL Tutor, tell me what worked, what broke, and what confused you. Open an issue or send me a note. I'll keep updating the repo as I learn.

Thanks for reading!

Ashpreet

• Also posted on X