Instructions to use 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M

Use Docker

docker model run hf.co/0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M

Ollama
How to use 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF with Ollama:
```
ollama run hf.co/0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M
```
Unsloth Desktop

How to use 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF with Docker Model Runner:
```
docker model run hf.co/0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M
```

Lemonade

How to use 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

RVN Qwen3.8-Flash-Next Abliterated Uncensored — GGUF

Split GGUF builds of 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored.

This repository currently publishes 18 verified quant families as 144 GGUF shards. Five IQ families were produced with the V6 calibration importance matrix; the remaining families are standard core quantizations.

Scope

Text/main model only.
NextN/MTP speculative-draft tensors excluded.
Vision projector excluded.
Produced from the verified F16 V6 main checkpoint.
Files are split below the Hugging Face per-file limit.
A recent llama.cpp build with qwen4exp support is required.

Available quant families

Each family contains eight shards. Sizes below are the exact combined GGUF payload size; runtime memory also depends on context length, KV-cache format, offload configuration, and backend. “Yes” in the importance-matrix column means that llama-quantize was invoked with the V6 calibration importance matrix.

Quant	Combined size	Shards	Importance matrix
`Q2_K`	80,447,450,592 bytes (74.92 GiB)	8	No
`IQ3_XS`	86,027,649,280 bytes (80.12 GiB)	8	Yes
`Q3_K_S`	88,755,339,232 bytes (82.66 GiB)	8	No
`IQ3_S`	88,882,766,080 bytes (82.78 GiB)	8	Yes
`IQ3_M`	89,540,819,200 bytes (83.39 GiB)	8	Yes
`Q3_K_M`	94,137,094,112 bytes (87.67 GiB)	8	No
`Q3_K_L`	96,656,400,352 bytes (90.02 GiB)	8	No
`IQ4_XS`	97,473,155,840 bytes (90.78 GiB)	8	Yes
`Q4_0`	99,959,519,712 bytes (93.09 GiB)	8	No
`IQ4_NL`	100,079,450,880 bytes (93.21 GiB)	8	Yes
`Q4_1`	110,973,453,792 bytes (103.35 GiB)	8	No
`Q4_K_S`	111,769,915,872 bytes (104.09 GiB)	8	No
`Q4_K_M`	119,150,723,552 bytes (110.97 GiB)	8	No
`Q5_0`	121,987,387,872 bytes (113.61 GiB)	8	No
`Q5_K_S`	127,728,766,432 bytes (118.96 GiB)	8	No
`Q5_1`	133,001,321,952 bytes (123.87 GiB)	8	No
`Q5_K_M`	134,106,668,512 bytes (124.90 GiB)	8	No
`Q6_K`	167,639,839,712 bytes (156.13 GiB)	8	No

Matched Q6_K benchmark

This is a quant-matched proxy, not a BF16-vs-BF16 benchmark:

Official base: Qwen/Qwen3.8-Flash-Next-GGUF, Q6_K, revision 158fc825df3eaa6c22d3c57a5927a5adf1c7cda7.
RVN: this release’s verified Q6_K family.
Runtime: llama.cpp build 10721, commit eaf937655.
Hardware: 4× NVIDIA RTX PRO 6000 Blackwell Workstation Edition; two GPUs per model, layer split 1/1, full GPU offload.
Warm throughput: two role-swapped runs × five repetitions = 10 samples per model; -fa 1, -b 512, -ub 512, 16 CPU threads per model.

Warm throughput

Values are mean tokens/s ± sample standard deviation.

Test	Official base Q6_K	RVN V6 Q6_K	Mean delta
`pp512`	2,810.019 ± 104.550	2,810.162 ± 99.701	+0.005%
`tg128`	111.6978 ± 0.6588	118.7456 ± 0.4427	+6.310%

Prompt processing was effectively unchanged in this run. RVN decoded 6.31% faster in this runtime, but the benchmark does not establish that the model edit caused the difference.

WikiText-2 perplexity

Full WikiText-2 test corpus: 150 complete 2,048-token chunks, context 2,048, batch/ubatch 512. Lower is better.

Model	PPL	Reported uncertainty
Official base Q6_K	4.5239	±0.02589
RVN V6 Q6_K	4.5016	±0.02529

The absolute V6-minus-base delta was -0.0223 (-0.493%). This is a coarse sanity check against gross language-model damage only. Abliteration damage can appear in instruction following and reasoning without materially changing raw LM perplexity, so this result is not evidence that capability was preserved.

Matched downstream evaluation

The same official-base and RVN V6 Q6_K artifacts were evaluated with lm-eval-harness 0.4.13 through two local llama.cpp build 10721 chat-completion servers. Each model used two RTX PRO 6000 Blackwell GPUs with full offload, layer split 1/1, four concurrent 4,096-token slots, batch size 1, seed 0, the model chat template, and reasoning disabled. Every reported task used its complete test set; no --limit was applied. Values are score ± the harness-reported standard error where available. Delta is V6 minus base in percentage points.

Task / metric	n	Official base Q6_K	RVN V6 Q6_K	Delta
IFEval prompt-level strict accuracy	541	83.73 ± 1.59	84.66 ± 1.55	+0.92 pp
IFEval instruction-level strict accuracy	541	88.85	89.45	+0.60 pp
GSM8K CoT (8-shot), flexible extract	1,319	89.31 ± 0.85	88.48 ± 0.88	−0.83 pp
GSM8K CoT (8-shot), strict match	1,319	88.10 ± 0.89	86.73 ± 0.93	−1.36 pp
BBH logical deduction, five objects (0-shot), flexible extract	250	34.00 ± 3.00	38.00 ± 3.08	+4.00 pp

The valid tasks move in both directions: V6 is slightly higher on IFEval and this BBH subset and slightly lower on GSM8K. Each delta is small relative to the reported uncertainty. These results provide a narrower and more relevant check than raw LM perplexity, but they still support only no gross regression on the tested tasks and configuration, not universal capability preservation.

arc_challenge_chat was also run on all 1,172 examples, but its score is intentionally omitted. With this chat-API/task combination, the filter retained the full generated string (for example, The best answer is C) while the target was the single letter (C), producing an artifact 0.0 exact-match score for both models. A corrected scorer is required before that task can be interpreted.

Benchmark limitations

Four concurrent CPU llama-quantize jobs were active during the throughput runs. GPU-pair roles were swapped, but a clean idle-host replication remains pending.
These measurements compare Q6_K artifacts only; they are not evidence for BF16 throughput or BF16 perplexity.
Raw LM perplexity does not measure instruction following or reasoning; the matched downstream results above must be interpreted separately and remain task-bounded.
Throughput is runtime-, hardware-, offload-, context-, and build-dependent.
WikiText-2 corpus SHA-256: d790b833ef8cf03a90db7bf1271b7520b83c45ce07ba3c1a9699df81e239eca0.
llama-bench binary SHA-256: cb17fad0f47bf6af15e008a58cae2af5b2fd5733a4cd5f198610a0b1942b32b2.

Download

Download one family rather than cloning the entire repository:

hf download 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF \
  --include "*Q4_K_M*" \
  --local-dir ./RVN-Qwen3.8-Flash-Next-Q4_K_M

For split models, load the first shard; llama.cpp discovers the remaining shards automatically:

llama-cli \
  -m RVN-Qwen3.8-Flash-Next-Q4_K_M-00001-of-00008.gguf \
  -ngl 99 -c 8192 --single-turn \
  -p "Explain why the sky appears blue in one concise paragraph."

Adjust GPU offload and context settings to the available hardware. The combined file size is not a complete runtime-memory estimate.

Verification policy

A quant family is published only after all of the following pass:

exact eight-shard inventory and byte accounting;
GGUF metadata and tensor census;
unique tensor-name and expected tensor-count checks;
confirmation that no NextN/MTP tensors are present;
selected-tensor dequantization and non-finite checks where supported;
fresh llama.cpp load and deterministic generation canary;
SHA-256 inventory for every shard;
remote LFS size/hash verification after upload.

Behavioral evaluation, lineage, limitations, and the constrained-surgery methodology are documented on the parent model card.

License

The upstream Qwen Community License 1.0 applies. This card does not replace or modify that license.

Downloads last month: 21,903

GGUF

Model size

177B params

Architecture

qwen4exp

Hardware compatibility

2-bit

3-bit

4-bit

5-bit

6-bit

Model tree for 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF

Base model

Qwen/Qwen3.8-Flash-Next

Finetuned

0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored

Quantized

(3)

this model

Collection including 0bserverx/RVN-Qwen3.8-Flash-Next-Abliterated-Uncensored-GGUF

Uncensored / Abliterated GGUF Models

Collection

GGUF quants of abliterated (uncensored) models for local llama.cpp inference. Curated by 0bserverx. • 4 items • Updated Sep 1 • 4