---
license: apache-2.0
base_model: rohit267/Qwen3.8-9B-heretic-uncensored
language:
- en
library_name: mlx
pipeline_tag: text-generation
tags:
- mlx
- apple-silicon
- mtplx
- speculative-decoding
- mtp
- qwen3_5
- qwen3.8
- heretic
- uncensored
- 4-bit
---

# Qwen3.8-9B Heretic Uncensored - 4-bit MLX + MTP (MTPLX)

Fast local build of [rohit267/Qwen3.8-9B-heretic-uncensored](https://huggingface.co/rohit267/Qwen3.8-9B-heretic-uncensored)
for Apple Silicon, tuned for native-MTP speculative decoding with [MTPLX](https://mtplx.com).

Measured **21.25 -> 40.05 decode tok/s (+88%)** on an M5 24GB MacBook, **lossless**
(exact speculative verify - the emitted tokens match the target model's own decoding).

## Contents

| File | What it is |
| --- | --- |
| `model.safetensors` (+ index) | 4-bit affine MLX trunk, group size 64 (4.5 bpw, ~5.0 GB) |
| `mtp.safetensors` | Native MTP head sidecar (Qwen3.5-9B lineage) |
| `mtp-adapter-r64.npz` | Trained C4 LoRA MTP adapter (rank 64) that lifts draft acceptance on this fine-tune |
| `mtplx_runtime.json` | MTPLX runtime contract (depth, hidden variant, sampler) |

The adapter is **merged at load time**; it is not baked into `mtp.safetensors`, so the
base sidecar stays reusable.

## How it was built

1. Convert the BF16 source to 4-bit MLX: `mlx_lm.convert --q-bits 4 --q-group-size 64`.
2. Graft the Qwen3.5-9B MTP sidecar, then train a rank-64 C4 LoRA adapter on hidden
   states captured from **this** 4-bit trunk (coding + math + long-code calibration).
   Training the adapter on the final trunk is required - an adapter calibrated on a
   different trunk pairs wrongly.
3. Serve with a tape-replay GDN verify kernel at draft depth 2.

Full lever study (every quantization / depth / adapter-rank / verify-core tested):
see the autoresearch repo linked below.

## Run it

Requires macOS 14+, Apple Silicon, and MTPLX (`brew install youssofal/mtplx/mtplx`).

```bash
mtplx serve \
  --model . \
  --model-id qwen-heretic \
  --mtp --depth 2 \
  --profile sustained \
  --verify-strategy capture_commit \
  --verify-core linear-gdn-from-conv-tape \
  --mtp-adapter mtp-adapter-r64.npz --merge-mtp-adapter \
  --host 127.0.0.1 --port 8080
```

Serves OpenAI `/v1/chat/completions` and Anthropic `/v1/messages` (Claude Code / OMP compatible).

## Notes / caveats

- **Apple Silicon + MLX only.** The MTP fast path is MTPLX-specific.
- **Lossy trunk, lossless decode.** 4-bit quantization changes the weights; the
  speculative decode itself is exact-verify, so output matches the 4-bit target model.
- This fine-tune **always opens with a `<think>` block**; budget tokens accordingly.
- Draft acceptance on the benchmark suite is ~0.71 at depth 2; depth 3+ does not pay
  on this hybrid (GatedDeltaNet) 9B. ~40 tok/s is the honest lossless ceiling on this
  model / this hardware.

## License

Apache-2.0, inherited from the Qwen3.5-9B base.