ksg91/toam-100m-v0.1

TOAM-100M v0.1

Superseded: a newer checkpoint, TOAM-100M v0.2, is available. It completes the pretraining run (10B tokens) and scores higher on every benchmark reported here. This repository is kept unchanged for reproducibility; only this card text has been updated.

TOAM-100M is a 102M-parameter model for tool use and agent control. You give it tool schemas and a request, and it returns the next action as JSON. It is trained from scratch (not fine-tuned from another model) and loads as a standard LlamaForCausalLM.

Parameters	BFCL v4 Live AST	Median latency	Peak process RAM
102.5M	22.9 %	202 ms	1.66 GB

Compared with FunctionGemma-270M, on the same harness and GPU (details):

• 38 % of its parameters
• 86 % of its BFCL Live accuracy (22.9 vs 26.6 %)
• 3.6× lower median latency per request (202 vs 731 ms)
• but only 52 % of its BFCL Non-live accuracy (22.8 vs 43.6 %). See limitations.

Research release. TOAM is shared as a research artefact and proof of work. It explores how much tool-use and agent-control ability a very small model trained from scratch can achieve, and documents the process openly so others can learn from it, reproduce it and build on it. It is not a state-of-the-art model and has not been validated for production use. Please do not use it for safety-critical, medical, legal, financial or other high-stakes decisions, or let it trigger irreversible actions without human review. Treat its outputs as suggestions, and validate every tool call before it runs. Feedback and failure reports are very welcome and directly shape future versions.

Example

tools = [{"type": "function", "function": {
    "name": "get_weather", "description": "Get the current weather for a city.",
    "parameters": {"type": "object", "properties": {"city": {"type": "string"}}, "required": ["city"]}}}]

agent.decide(tools, "What's the weather in Pune right now?")

{"type": "tool_call", "tool": "get_weather", "arguments": {"city": "Pune"}}

The model can also call several tools at once, ask the user for a missing value or for confirmation, say that no tool fits, or escalate.

Why TOAM?

A general-purpose LLM spends much of its capacity storing knowledge about the world. An agent controller can get the knowledge it needs at run time from tool schemas, tool results, state and context. TOAM tests how small a model can be when its main job is to decide the next action, not to know everything itself. The goal is useful capability per unit of memory, latency and energy, not the highest benchmark score.

Intended use

• Picking a tool and filling its arguments in a small or local agent
• Routing requests in an agent pipeline (call, ask, confirm, refuse, escalate)
• Embedded, on-device or CPU-only agent controllers where a large model is too heavy
• Research on small agentic language models

Out of scope: TOAM-100M is not a chatbot, assistant or knowledge model. It knows little about the world, writes poor free text, and should not answer questions or be used without a tool menu.

Usage

TOAM uses its own raw prompt and action format. It is not a chat model: do not apply a chat template (there is none, and a generic one will give garbage). tiny_agent.py in this repository builds the prompt, runs greedy decoding and parses the action:

from tiny_agent import TinyAgent

agent = TinyAgent("ksg91/toam-100m-v0.1")
agent.decide(tools, "What's the weather in Pune right now?")
# {'type': 'tool_call', 'tool': 'get_weather', 'arguments': {'city': 'Pune'}}

Optional inputs: policy (plain-text rules, e.g. "Confirm with the user before any booking or payment."), history (earlier tool calls and their results), state (e.g. {"today": "2026-09-28"} so relative dates resolve), and a multi-turn conversation as the goal ("User: ...\nAssistant: ...\nUser: ...").

Actions: tool_call, tool_calls (parallel), ask_user (a required value is missing, confirmation, or no tool fits), escalate, and the multi-step control actions finish / retry / wait.

The context window is 2,048 tokens, so keep menus to about 10 tools with short descriptions.

Raw prompt format

This is what tiny_agent.format_prompt(...) produces for the example above. The model writes the action after <|action|> and then stops at EOS:

<|system|>
You are an agent controller. Choose the correct next action.
<|tools|>
[{"name":"get_weather","description":"Get the current weather for a city.","parameters":{"properties":{"city":{"type":"string"}},"required":["city"]}}]
<|goal|>
What's the weather in Pune right now?
<|history|>
[]
<|state|>
{"step":0}
<|observation|>
null
<|action|>

The <|...|> tags are special tokens and must be tokenized as such. Always build the tool list with format_prompt, because it renders schemas the way the model saw them in training.

GGUF (llama.cpp, LM Studio, Ollama)

File	Precision	Size
toam-100m-v0.1-f16.gguf	F16	206 MB
toam-100m-v0.1-q8_0.gguf	Q8_0	110 MB

Send the prompt from format_prompt(...) as raw text (a completion, not a chat) with greedy decoding. llama-server parses the special tokens by default:

llama-server -m toam-100m-v0.1-q8_0.gguf -c 2048
curl http://127.0.0.1:8080/completion -d '{"prompt": "<prompt from format_prompt>", "temperature": 0, "n_predict": 256}'

Equivalence check (llama.cpp build b11223, against the transformers model): identical tokenization on 39/39 prompts, and identical greedy outputs on 35/35 held-out prompts for both F16 and Q8_0. A BF16 GGUF changed 1 of 35 outputs and is not shipped. On CPU only (16 threads, llama-server), a request takes about 290 ms at the median.

Weights

model.safetensors is stored in FP16 (205 MB). Training used FP32 master weights. On 100 held-out prompts, FP16 gives the same greedy outputs as FP32 (checked on CPU). tiny_agent.py runs in BF16 on CUDA and FP32 on CPU by default; pass dtype="float32" for exact FP32 behaviour.

Benchmarks

1. Independent benchmark: BFCL v4, single-turn

BFCL prompts and functions were not written by us. Fine-tuning data was decontaminated against BFCL. The pretraining corpus was not (see Training). Both models were run locally with the same harness and scored with bfcl_eval's own AST checker. This is not an official leaderboard submission. Our FunctionGemma-270M numbers agree with the ones on its model card, which is a check that the harness is correct.

Model	Params	Non-live AST	Live AST	Irrelevance	Latency p50 / p95	Peak VRAM	Peak process RAM
TOAM-100M v0.1	102.5M	22.8 %	22.9 %	68.1 %	202 / 509 ms	626 MB	1.66 GB
FunctionGemma-270M	268.1M	43.6 %	26.6 %	72.2 %	731 / 1996 ms	578 MB	2.06 GB

All numbers in this table were measured by us. Irrelevance means the model correctly makes no call when no tool fits. 38 of the ~3,600 prompts are longer than TOAM's 2,048-token context and count as failures.

Benchmark environment

Hardware	NVIDIA RTX 5080 (16 GB), Windows 11, GPU otherwise idle
Software	PyTorch 2.14 (CUDA 13.0), transformers 5.17
Backend	TOAM: plain PyTorch decoding loop, no KV cache. FunctionGemma: transformers generate, KV cache
Precision	BF16 autocast for both
Batch size	1 (one request at a time)
Decoding	Greedy. Max new tokens 256 (TOAM) / 512 (FunctionGemma). Median output 32 / 37 tokens
Latency	Wall time per request, tokenization to decoded text
Memory	Peak VRAM = torch.cuda.max_memory_allocated. Peak process RAM = peak working set of the process, which includes the PyTorch/CUDA runtime, not just the model

Latency depends heavily on the inference stack. These numbers compare two models in one harness, not optimised deployments. TOAM's peak VRAM is higher than its size suggests because this run kept FP32 weights under BF16 autocast. BFCL was run on the training checkpoint, whose greedy outputs match this repository's FP32 export on 40/40 held-out prompts.

2. Internal held-out evaluation

These tests use tools and services that never appeared in any training data. For the held-out servers that includes pretraining. The tests are in-house, and the requests look more like the training data than BFCL's do.

Test	Tool correct	Exact action
84 real-world MCP tool servers held out from all training	78.2 %	61.9 %
Human-written dialogues (Schema-Guided Dialogue) on services absent from training	83.9 % (right action type)	–

The SGD dialogue format itself (for other services) was in training.

3. Close to the training distribution

These are new rows, but they come from the same generators or sources as the training data. Read them as a sanity check, not as a measure of generalisation.

Test	Tool correct	Exact action
Unseen tools from training-split servers	63.3 %	50.3 %
Development-split servers	76.8 %	66.7 %
Held-out slice of the public fine-tuning datasets	73.9 %	53.9 %
Multi-step episodes from 4 held-out simulated-world families	55.0 %	43.3 %

Limitations

• Pretrained on 3.25B tokens only, so its language understanding is limited. It can bind a value to the wrong field when a request has several text values (e.g. email recipient vs. subject).
• Parallel requests ("weather in Paris and in Berlin") often produce a single call.
• Weak on maths/science-style functions, which make up much of BFCL Non-live.
• The questions it asks can be clumsy ("Which to should I use?").
• English only. Context is 2,048 tokens.
• Always validate the returned arguments before running a tool, and require confirmation for anything irreversible.

Training

• Architecture: Llama-style decoder (RMSNorm, SwiGLU, RoPE, grouped-query attention with 12 query / 4 KV heads, d=768, 12 layers, tied embeddings), 32k byte-level BPE tokenizer, 2,048-token context. 102.5M parameters. Trained from scratch.
• Pretraining: 3.25B tokens sampled from a 19.6B-token corpus of public data: web and educational text (roughly 60 % of what the model saw), code (about 25 %), structured data and API specifications (JSON/YAML, OpenAPI, JSON Schema; about 12 %) and encyclopedic text (about 5 %). The corpus was decontaminated against the 84 held-out tool servers (matching tool names, identifiers and description shingles) before tokenization. It was not decontaminated against BFCL.
• Fine-tuning: one epoch on about 137k rows. About 55 % are in-house data generated from real-world MCP tool schemas (single-step requests and multi-step episodes in simulated tool worlds). About 45 % are public function-calling and human-written dialogue/intent data, with 20 % typo/noise augmentation. Every label is checked automatically: calls must be schema-valid, and every argument must be traceable to the request, the state or a schema default (otherwise the right action is to ask the user). Rows overlapping BFCL (function names, 8-gram overlap) or the held-out tools are removed.

The in-house training data, its generators and the exact mixture are not released.

Versions

Each checkpoint is published as its own repository, and existing weights are never replaced.

Version	Date	Notes
v0.1	2026-09	First release. Early checkpoint: pretraining stopped at 3.25B tokens with an early learning-rate decay

Next: the next TOAM checkpoint will extend pretraining toward ~10B tokens and use lessons from v0.1. Its pipeline and results will be documented when it ships.

Acknowledgements

Fine-tuning data includes openly licensed datasets from Salesforce and Argilla (APIGen / xLAM), Team-ACE (ToolACE), NVIDIA (When2Call), Google (Schema-Guided Dialogue, CC BY-SA 4.0) and Amazon (MASSIVE, CC BY 4.0). Pretraining data includes FineWeb-Edu, Cosmopedia (SmolLM corpus), CodeParrot, StarCoderData, CodeSearchNet, Wikipedia, the APIs.guru OpenAPI directory and SchemaStore. Evaluation uses the Berkeley Function-Calling Leaderboard (BFCL v4).

License

Apache-2.0 (weights and inference code). The training datasets keep their own licenses.