Qwythos-9B-v2 Fixes AI Repetition Without Hurting Reasoning

Overview Qwythos-9B-v2 is a 9-billion-parameter text generation model developed by empero-ai that fixes repetitive output problems in the base Qwythos while preserving its reasoning capabilities. Built on the Qwen3.5-9B architecture (a hybrid combining 3:1 Gated-DeltaNet linear-attention with full attention blocks), the model ships with a native multi-token-prediction head, 1,048,576-token context via YaRN rope-scaling, and intentionally unrestricted capabilities for research and technical work. The key innovation is FTPO (Final-Token Preference Optimization), a targeted fine-tuning method that eliminates the looping/repetition problem that plagued v1 (dropping from 6.7% to 0% looping under greedy decoding) without degrading knowledge or reasoning across benchmarks. It runs on the Hugging Face transformers library with bfloat16 weights in safetensors format and uses the Qwen3.5 native ChatML-style tokenizer. Best use cases Long-context reasoning and analysis: The model excels at problems requiring extended chain-of-thought over long documents or codebases, thanks to its 1M-token context window and preserved reasoning performance (83.8% MMLU with chain-of-thought, 93.6% GSM8K). If you need to reason over entire research papers, multiple source files, or complex multi-step math problems without truncation, this model's context length and reasoning traces make it substantially stronger than smaller-context alternatives. The looping fix ensures that reasoning stays coherent across long generations without artificial repetition penalties. Code generation and technical problem-solving: With 77.4% pass@1 on HumanEval and the expand-around-center algorithm example demonstrating correct algorithm selection and explanation, the model generates working code with sound explanations. The training on Claude traces means it tends to explain why an approach works, not just generate syntax, making it useful when you need both functional code and teachable output. The 1M-token context is particularly valuable for whole-project refactoring or multi-file code understanding. Uncensored technical and clinical documentation: The model does not over-refuse on legitimate technical topics like naloxone mechanisms, pharmacology, chemistry, biology, and cybersecurity research. Unlike many open-source models that collapse into refusals on sensitive topics, Qwythos-9B-v2 engages directly with research-level questions while maintaining 0% refusal rate. This makes it suitable for educational materials, clinical summaries, and security research documentation where accurate information is more important than content filtering. Greedy and low-temperature decoding without penalty workarounds: Because looping is trained out, you can use greedy decoding or low-temperature sampling (0.6) without relying on repetition_penalty as a load-bearing safety mechanism. This is valuable in production serving where you want deterministic or stable outputs without the quality degradation that repetition penalties can introduce. The model generates clean, non-repetitive output at temperature 0.6 directly. Speculative decoding and efficient serving: The native multi-token-prediction head (restored in v2) enables speculative decoding setups, allowing faster token generation through draft-model acceleration. For serving at scale, this architectural compatibility with modern inference optimization techniques makes it more efficient than models without MTP support. Limitations Looping fix does not improve absolute capability. This is explicitly a hygiene release. The model scores identically to base Qwythos on reasoning benchmarks; the win is robustness and cleaner behavior, not higher raw scores. If you need maximum performance on a specific benchmark, you should measure directly rather than expecting v2 to outperform the base. HumanEval performance regressed slightly. At 77.4% pass@1, it scores below the raw Qwen3.5-9B base (81.7%) and below the original Qwythos (79.9%). This is a small but measurable cost of the fine-tuning process. For pure code generation speed and correctness on standard benchmarks, the base Qwen3.5-9B may be preferable if looping is not a concern in your use case. MTP head not co-trained with fine-tuned weights. The multi-token-prediction module was restored from the base Qwen3.5-9B without retraining alongside the FTPO fine-tuning. This means speculative-decoding acceptance rates may be modest compared to models where MTP and main weights are jointly optimized. If you rely heavily on speculative decoding for speed, expect less dramatic acceleration than with architectures where both components were fine-tuned together. GPQA-diamond performance dropped. The model scores 49.0% on GPQA-diamond compared to the base Qwythos at 52.0%, suggesting the fine-tuning may have slightly degraded performance on hard knowledge questions. This is within eval noise but worth noting if your use case is highly specialized science reasoning. YaRN static scaling carries short-context trade-off. The 1M-token context is enabled via static YaRN scaling (4× the native 262,144 window). This scaling carries a measurable short-context trade-off; if you work primarily with short prompts, you may see slightly degraded performance compared to models without rope-scaling. The maintainers recommend scaling the rope factor to the context length you actually use if short sequences matter. Intentionally uncensored—requires responsible deployment. The model will engage with topics others refuse (pharmacology synthesis, cybersecurity exploits, offensive code). It is suitable for research but must be deployed within applicable law and ethical guidelines. This is not a limitation of capability but a requirement for responsible use. Evaluations from internal harness, not official benchmarks. The reported numbers (MMLU, ARC, GSM8K, etc.) come from Empero AI's internal evaluation framework with specified sample sizes (n=500 for most, n=198 for GPQA). They are not from standard leaderboard evaluations, making direct comparison with other reported scores harder. Use these for relative comparison within Empero's models but add official-harness numbers if you need apples-to-apples comparisons with other model cards. Only 67 downloads. Adoption is minimal, suggesting limited real-world testing outside Empero AI. Community feedback and known failure modes may be sparse. How it compares vs. Qwythos-9B-Claude-Mythos-5-1M : Choose Qwythos-9B-v2 if you experience looping or repetition problems with the base Qwythos, or if you need greedy decoding without repetition penalties. The base Qwythos is the direct predecessor and v2 is meant to be a drop-in improvement—same reasoning capability, cleaner behavior. Stick with base Qwythos only if you have already tuned parameters and do not want to retune. vs. Qwythos-9B-Claude-Mythos-5-1M-GGUF : Choose the GGUF quantized version if you need to run the model on consumer hardware or CPU with minimal VRAM, or within llama.cpp, Ollama, or similar engines. Qwythos-9B-v2 in full bfloat16 is for deployments with GPUs or where maximum precision matters; GGUF trades precision for speed and memory. The GGUF models are quantized variants of the base Qwythos, not v2, so if you want the looping fix in quantized form, you would need to quantize v2 yourself. vs. Qwable-9B-Claude-Fable-5 : Choose Qwythos-9B-v2 for general reasoning, math, and analysis tasks. Choose Qwable-9B-Claude-Fable-5 if your workload is agentic coding, tool-use, or multi-turn reasoning with external function calls—it is a distillation-style fine-tune targeting that specific pattern. Both are 9B, but Qwythos focuses on chain-of-thought reasoning while Qwable focuses on agent behavior and tool integration. Technical specifications Architecture: Qwen3.5-9B hybrid combining 3:1 Gated-DeltaNet linear-attention layers with full attention blocks, natively multimodal-capable (though text-only in practice), with restored native multi-token-prediction (MTP) head. Parameters: 9 billion, stored in bfloat16 precision in safetensors format. Context window: 1,048,576 tokens using YaRN rope-scaling at 4× the native 262,144-token window. Static scaling applies uniformly; dynamic scaling can be configured for specific context lengths. Training data and method: Fine-tuned on ~2,000 preference tuples via FTPO (Final-Token Preference Optimization). Preference pairs were auto-mined by eliciting looping at low temperature and extracting the loop-starting token versus coherent top-k alternatives from the model's own distribution. The base model was Qwythos-9B-Claude-Mythos-5-1M . Training hyperparameters: LoRA rank (r) = 256, alpha (α) = 128 Learning rate = 1.5e-5 1 epoch with early stopping at chosen_win ≥ 0.30 All attention projections, MLP projections, and lm_head trained (LoRA applied across these) Early stopping criterion: chosen-win margin of at least 0.30 Tokenizer: Qwen3.5 native tokenizer using ChatML-style chat templates. Library support: Runs on Hugging Face transformers library (tested with AutoModelForImageTextToText and AutoTokenizer ). Compatible with vLLM for serving (requires --trust-remote-code ). Inference optimization: Native multi-token-prediction head enables speculative-decoding acceleration in compatible serving stacks. License: Apache-2.0 (permissive, allows commercial use with attribution). Model inputs and outputs Inputs Text prompts in ChatML format (applied automatically via tok.apply_chat_template() ) Single-turn or multi-turn conversation history passed as list of {"role": "user"/"assistant", "content": "..."} dicts Sampling parameters: temperature (recommended 0.6), top_p (0.95), top_k (20), repetition_penalty (1.05, now optional), max_new_tokens (up to 16384 for reasoning tasks) Compatible with greedy decoding (temperature=0, no sampling) without degradation Outputs Generated text tokens decoded to UTF-8 string Chain-of-thought reasoning interleaved with final answers when given reasoning-style prompts Code with explanations when prompted for implementation tasks Unfiltered technical/clinical content on sensitive topics (no safety refusals applied) Getting started from transformers import AutoModelForImageTextToText, AutoTokenizer model_id = "empero-ai/Qwythos-9B-v2" tok = AutoTokenizer.from_pretrained(model_id) model = AutoModelForImageTextToText.from_pretrained( model_id, dtype="bfloat16", device_map="auto" ) # Single-turn example: reasoning task messages = [{"role": "user", "content": "A snail is at the bottom of a 10-meter well. Each day it climbs 3 meters, but each night it slips back 2 meters. How many days until it escapes?"}] text = tok.apply_chat_template(messages, tokenize=False, add_generation_prompt=True) inputs = tok(text, return_tensors="pt").to(model.device) out = model.generate( **inputs, max_new_tokens=2048, do_sample=True, temperature=0.6, top_p=0.95, top_k=20, repetition_penalty=1.05 ) response = tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True) print(response) For serving at scale, vLLM is supported: python -m vllm.entrypoints.openai.api_server \ --model empero-ai/Qwythos-9B-v2 \ --trust-remote-code \ --limit-mm-per-prompt '{"image":0,"video":0}' Frequently asked questions Q: Does Qwythos-9B-v2 fix the looping problem from the base Qwythos? A: Yes. Looping under greedy decoding dropped from 6.7% to 0.0%, and at temperature 0.6 from 1.3% to 0.7%. This was the primary goal of the v2 release. The fix uses FTPO to target the exact token that starts a loop and train the model to prefer coherent continuations instead. Q: Can I use this model commercially? A: Yes. The Apache-2.0 license permits commercial use, modification, and distribution provided you include a copy of the license and copyright notice. No royalties or special permissions are required. Q: What hardware do I need to run Qwythos-9B-v2? A: The model is 9B parameters in bfloat16, requiring approximately 18 GB VRAM for inference. An NVIDIA A100, RTX 4090, or similar GPU is practical for single-user serving. For multi-GPU serving or cost-sensitive deployments, quantized versions (e.g., GGUF) or smaller context windows can reduce memory needs. CPU inference is slow but possible with quantization. Q: How does v2 compare to the base Qwythos for reasoning tasks? A: They score identically on MMLU, GSM8K, ARC, and GPQA within evaluation noise—v2 is not a capability upgrade. The difference is that v2 stays coherent under greedy or low-temperature decoding without needing repetition_penalty as a workaround. If you do not have looping issues with the base, there is no reasoning performance gain. Q: Can I fine-tune this model further? A: Yes. The model is distributed in safetensors format and is compatible with LoRA fine-tuning via the peft library, full-parameter fine-tuning via transformers , and other standard PyTorch-based training pipelines. The base was fine-tuned with LoRA (r=256, α=128), so LoRA stacking is possible. Q: Does this model refuse to answer sensitive questions? A: No. The model is intentionally uncensored and will engage with pharmacology, biology, chemistry, cybersecurity, and clinical topics without refusal (0% refusal rate). It is suitable for research, education, and technical documentation but must be deployed responsibly and within applicable law. Q: How does the 1M-token context affect short prompts? A: YaRN static rope-scaling carries a small short-context trade-off—the model may perform slightly worse on short sequences compared to models without scaling. If your workload is primarily short prompts, you can reduce the YaRN factor dynamically or use a base model without scaling. For most mixed workloads, the trade-off is negligible. Q: Is the multi-token-prediction head trained with the fine-tuned weights? A: No. The MTP head was restored from the base Qwen3.5-9B without retraining during FTPO. This means speculative-decoding acceptance rates may be modest. If speculative decoding is critical for your deployment, expect less dramatic speed gains than with models where MTP and main weights were jointly optimized. This is a simplified guide to an AI model called Qwythos-9B-v2 maintained by empero-ai. If you like these kinds of analysis, join AIModels.fyi or follow us on Twitter.