joycx/Qwen3.8-27B-Velocity

New: Velocity with the Sharp chat template at xhigh keeps its accuracy and thinks 14–26% less on HMMT, C-Eval and IFBench. Results below.

Qwen3.8-27B · Velocity 1.0

Velocity is a LoRA for Qwen3.8-27B that keeps its answers and cuts its thinking: 28–43% fewer reasoning tokens per answer across maths, knowledge, instruction-following and coding, at close to stock accuracy, and well ahead of stock on LiveCodeBench.

• Quantized builds (llama.cpp): Qwen3.8-27B-Velocity-GGUF
• Adapter: rank 32 · alpha 64 · 0.87 GB · PEFT

Results

All runs use Qwen's chat template at its default reasoning effort, xhigh. Stock is Qwen3.8-27B without the LoRA. The accuracy difference is in percentage points; thinking is the number of reasoning tokens per answer.

Benchmark	Velocity accuracy	Stock accuracy	Difference	Velocity thinking	Stock thinking	Thinking saved
AIME 2026	95.0%	98.7%	−3.7 pts	12.7k	22.0k	43% fewer
HMMT Nov 2025	98.3%	99.3%	−1.0 pts	14.8k	22.0k	33% fewer
C-Eval	90.0%	90.0%	same	0.89k	1.49k	41% fewer
IFBench	72.3%	73.5%	−1.2 pts	4.9k	8.1k	39% fewer
LiveCodeBench v6	75.7%	63.7%	+12.0 pts	11.9k	16.5k	28% fewer

• Same questions for both. Velocity: AIME 2026 and HMMT Nov 2025 30 problems × 2 seeds, C-Eval 1,346 × 1, IFBench 300 × 1, LiveCodeBench v6 175 × 2. Stock is each question's average over its logged runs: UkisAI's 5 published stock seeds, and our own 2 seeds on LiveCodeBench.
• Same settings for both: temperature 1.0, top-p 0.95, top-k 20; full precision (BF16, LoRA applied at runtime in vLLM 0.29.0); no system prompt.
• Token budget per answer, the same for every model and taken from UkisAI's published configs: AIME and HMMT 250,000, IFBench 81,920, C-Eval 16,384, LiveCodeBench v6 32,768.
• Closeness to training: AIME, HMMT, C-Eval and LiveCodeBench test the same kinds of task Velocity was trained on (none of their questions); IFBench tests nothing like its training.

How often the budget ran out

An answer that runs out of budget usually scores as wrong, so this shows how much of each score is the budget.

	Velocity 1.0	Swift 1.5	Stock
AIME 2026 (250k)	0 of 60	n/a	0 of 150
HMMT Nov 2025 (250k)	0 of 60	n/a	0 of 150
C-Eval (16k)	4 of 1,346 (0.3%)	n/a	134 of 6,730 (2.0%)
IFBench (82k)	0 of 300	n/a	4 of 1,500 (0.3%)
LiveCodeBench v6 (32k)	61 of 350 (17%)	83 of 350 (24%)	118 of 350 (34%)

• LiveCodeBench is where the budget matters. Budget hits are 61 of Velocity's 85 misses, 83 of Swift 1.5's 98 and 118 of stock's 127. Much of Velocity's +12 points comes from finishing inside 32k more often.
• Maths and IFBench are not budget-limited. Nothing hit the budget, though Velocity used more than 32k tokens on 7 of 60 AIME answers, 9 of 60 HMMT answers and 7 of 300 IFBench answers. A 32k budget would have cost accuracy there.
• Swift 1.5's budget hits are known only for our LiveCodeBench run; UkisAI's card does not publish them.

Above 32k

• No runaway answers: on a loop test (one AIME problem × 10 seeds, 65,536-token budget) Velocity solved 10/10 with no repetition loops and no budget hits, median 10.7k tokens against stock's 24.8k.
• LiveCodeBench at 98,304 tokens (an earlier checkpoint): on 20 hard problems stock never solved at 32k, stock solved 15 of the 17 it finished and Velocity's supervised stage (stage 2 below) 12 of 13. Those problems are mostly budget-bound, not out of reach. This test predates the DPO stages and was not repeated on Velocity 1.0.

Compared with Swift 1.5

Swift 1.5 (UkisAI) is another Qwen3.8-27B tune that cuts thinking. Each cell is accuracy / thinking tokens per answer / thinking vs stock.

	Velocity 1.0	Swift 1.5	Stock
AIME 2026	95.0% / 12.7k / −43%	96.0% / 13.2k / −40%	98.7% / 22.0k
HMMT Nov 2025	98.3% / 14.8k / −33%	97.3% / 15.0k / −32%	99.3% / 22.0k
C-Eval	90.0% / 0.89k / −41%	90.9% / 0.82k / −45%	90.0% / 1.49k
IFBench	72.3% / 4.9k / −39%	72.1% / 5.0k / −38.5%	73.5% / 8.1k
LiveCodeBench v6	75.7% / 11.9k / −28%	72.0% / 13.4k / −19%	63.7% / 16.5k

• All at xhigh, full precision (BF16), on the same questions, with the budgets above. Stock is the same baseline as above.
• Swift 1.5 on AIME, HMMT, C-Eval and IFBench: UkisAI's published numbers (5 seeds, against the same stock runs). On LiveCodeBench v6 we ran Swift 1.5 ourselves, on the same questions and seeds as Velocity and stock.
• Swift 1.5's token figures are UkisAI's mean total tokens per answer (thinking plus answer); on these benchmarks the two are within about 5%.
• UkisAI publish 81.7% for Swift 1.5 on LiveCodeBench v6 with their own setup; our run uses a 32,768-token budget per answer, the same for every column.
• On maths, knowledge and instruction-following the two are within about a point of each other at about the same thinking cut; on LiveCodeBench Velocity is ahead in both accuracy and thinking saved.

With the Sharp chat template

The Sharp chat template by peculiar-ragdoll (built on froggeric's Qwen-Fixed-Chat-Templates, Apache-2.0) adds a short "answer directly, no filler" instruction to the system message; for single-turn prompts it is otherwise identical to Qwen's template. Same model, questions, budgets and settings as above. Each cell is accuracy / thinking tokens per answer / thinking vs stock.

	Velocity + Sharp, xhigh	Velocity + Sharp, medium	Velocity, Qwen template, xhigh	Stock
AIME 2026	96.7% / 13.6k / −38%	88.3% / 7.2k / −67%	95.0% / 12.7k / −43%	98.7% / 22.0k
HMMT Nov 2025	95.0% / 12.7k / −42%	81.7% / 10.2k / −54%	98.3% / 14.8k / −33%	99.3% / 22.0k
C-Eval	90.7% / 0.66k / −55%	90.0% / 0.50k / −66%	90.0% / 0.89k / −41%	90.0% / 1.49k
IFBench	71.3% / 4.0k / −50%	59.3% / 3.6k / −55%	72.3% / 4.9k / −39%	73.5% / 8.1k
LiveCodeBench v6	74.9% / 11.7k / −29%	77.1% / 10.5k / −36%	75.7% / 11.9k / −28%	63.7% / 16.5k

• Sharp at xhigh matches Qwen's template within noise (HMMT −3.3 points is 2 of 60 answers) and thinks 14–26% less on HMMT, C-Eval and IFBench; on AIME it thinks 7% more and on LiveCodeBench about the same.
• Sharp at medium (the Sharp template's own default) cuts the most, but loses 7–17 points on maths and 13 on IFBench. Use it for coding and short questions, not hard maths.
• Budget hits: no maths or IFBench answer hit the budget in either run; LiveCodeBench 61 of 350 at xhigh and 49 of 350 at medium; C-Eval 3 and 0 of 1,346.
• To use it, load its chat_template.jinja and pass reasoning_effort: "xhigh".

How it was trained

One LoRA (rank 32, alpha 64) on the attention (q/k/v/o), linear-attention (in_proj_qkv, in_proj_z, out_proj) and MLP (gate/up/down) projections of the 64 language layers, trained in five stages, each starting from the previous one:

1. Supervised warm-up (SFT): stock Qwen3.8-27B's own published answers (UkisAI's 5-seed runs at xhigh, 32k budget) on the 880 older LiveCodeBench problems (v1–v5). Problems stock solves on at least 4 of 5 seeds contribute their shortest correct answer verbatim; on the others, the thinking is cut right after the first draft that passes the problem's hidden tests, plus a short check and "Now final.", and only that thinking is trained (no answer). 388 rows, 4.3M tokens, 55 steps, learning rate 1e-4 (cosine).

2. Supervised continuation: 754 new rows of the same kind, 7.6M tokens, 95 steps, learning rate 5e-5.

3. DPO round 1: 375 length pairs from the same stock answers (the shorter correct answer preferred, at least 1.5× and 2,000 tokens shorter), NLL anchor 0.01, 94 steps, stage 2 as reference.

4. DPO round 2: 932 pairs: 630 MMLU-Pro pairs from stock's published answers (length and right-vs-wrong) and 302 coding pairs from the round-1 adapter's own answers. 233 steps.

5. DPO round 3 (on-policy): the round-2 adapter answered each training prompt 4 times (vLLM, Qwen's template, xhigh, 32k budget); the answers were graded; and two kinds of preference pairs were built:

   • length pairs: a shorter correct answer is preferred over a longer correct one (teaches brevity without changing the answer);
   • correctness pairs: a correct answer is preferred over a wrong one (keeps accuracy).

   892 pairs (MMLU-Pro 576, maths 164, code 152), one epoch, 223 steps.

All DPO rounds: β 0.1, learning rate 1e-5 (5 warm-up steps, cosine), 4 pairs per step, sequences up to 32,768 tokens, the previous stage as reference, and a small NLL anchor on the preferred answer (0.05 in rounds 2 and 3).

Round 3's training prompts (no evaluation question was used in any stage):

Source	Prompts × samples	Graded by
MMLU-Pro	700 × 4	answer letter
LiveCodeBench, problems older than v6	300 × 4	hidden test cases
DeepScaleR (integer-answer maths)	400 × 4	exact answer; 13-gram de-duplicated against the eval sets

Hardware: the supervised stages ran on one NVIDIA B200; each DPO round on one NVIDIA RTX PRO 6000 (96 GB).

Use

Transformers + PEFT

from transformers import AutoModelForImageTextToText, AutoTokenizer
from peft import PeftModel

base = "Qwen/Qwen3.8-27B"
model = AutoModelForImageTextToText.from_pretrained(base, dtype="bfloat16", device_map="auto")
model = PeftModel.from_pretrained(model, "joycx/Qwen3.8-27B-Velocity")
tokenizer = AutoTokenizer.from_pretrained(base)

vLLM

vllm serve Qwen/Qwen3.8-27B --enable-lora --max-lora-rank 32 --lora-modules velocity=joycx/Qwen3.8-27B-Velocity

llama.cpp: use the merged builds in Qwen3.8-27B-Velocity-GGUF.

Limitations

• About 1–4 points below stock on the hardest maths (AIME, HMMT) in exchange for a third less thinking.
• The LiveCodeBench gain is partly a budget effect (see "How often the budget ran out").
• Trained on single-turn maths, knowledge and coding prompts; multi-turn agentic use was only piloted.
• Evaluate on your own tasks before relying on it.

License

Copyright 2026 joycx. Velocity is licensed under CC BY-NC-SA 4.0.

Summary (the legal code governs): you may copy, share and adapt it for non-commercial purposes; selling it or using it for commercial advantage is not permitted. If you share it or a modified version, credit it as "Velocity by joycx (https://huggingface.co/joycx/Qwen3.8-27B-Velocity), CC BY-NC-SA 4.0", say what you changed, and share adaptations under the same or a compatible license. Provided as-is, without warranties of any kind; see section 5 of the license.

Velocity is a derivative of Qwen3.8-27B, Copyright 2026 Alibaba Cloud, licensed under Apache 2.0 (copy in LICENSE-Qwen3.8-27B). Changes: a LoRA adapter trained on these weights (this repository contains only the adapter). CC BY-NC-SA 4.0 applies to Velocity's contribution only; it does not restrict Qwen's own weights.