swift-qwen3.8-rtx3090

My coding agent runs against a local model, not an API. The machine is a single RTX 3090, and the stack that makes 27B parameters usable on it is syv’s HyperQwen. It pins and patches vLLM, keeps the body at 4-bit weights, and requantises the heads and the MTP draft module so the whole thing fits in 24 GB with a 114k context.

Stock Qwen3.8-27B reasons too much for my use. On two open-ended code tasks it spent the full 8,192-token thinking budget and never wrote any code. Swift, a community finetune tuned for shorter reasoning, finished the same tasks in 2,500 to 7,100 thinking tokens and produced working code. No decode-speed trick closes a gap like that, so I switched.

This page follows the work in build order: base Qwen, Swift 1.0, the fast variant, then the Swift 1.5 rebuild in late 2026. The current build and the current published model are the Swift 1.5 ones; every Swift 1.0 number below is history that the 1.5 section measures against.

The syv stack will serve Swift, but as a third-party checkpoint, and those get a lower tier of support than base Qwen. Base Qwen has a “fast variant” with an int4 GPTQ lm_head and draft module, plus a draft vocabulary counted over the model’s own outputs. That variant is worth about 15 percent. Third-party checkpoints get int8 heads and base Qwen’s draft vocabulary. Swift ran fine that way. I wanted to know how much of the fast variant’s gain I could get back by rebuilding it for Swift.

Body, heads and drafter

A checkpoint in this stack has three parts, and the fast variant leaves the biggest one alone.

The body is the stack of transformer layers in the middle, and it holds most of the 27B parameters. Each layer reads a hidden state of 5,120 numbers per token and writes an updated one. Swift’s body is a 4-bit AWQ export. I never change it.

The heads sit at either end of the body, with one row per token in the 248k vocabulary. Strictly, only the lm_head is a head, but the syv stack quantises the two tables together and calls them the heads, so I do too. The embeddings turn a token into its first 5,120 numbers. The lm_head turns the last hidden state into a score for every token. At 248k by 5,120, each table is about 1.3 billion parameters, and the AWQ export leaves both unquantised. Swift’s int8 build quantises them to int8.

Only one of them is worth shrinking further. vLLM reads the whole lm_head on every decode step, so the fast variant takes it to int4. A token only reads its own row of the embeddings, so they stay at int8.

The drafter is the MTP module. It is one extra transformer layer plus an input projection, eight linears in all. The projection mixes two inputs, the body’s last hidden state and the embedding of the previous drafted token. The drafter runs once per guess, feeding each drafted token back in to guess the next. To score those guesses it uses a draft head, a slice of the lm_head with rows only for the tokens in the draft vocabulary. For Swift 1.0 that is 25,879 rows.

The draft vocabulary, the draft head and the MTP weights exist only for speculative decoding, so I call them the speculative artefacts. None of them are newly trained. The vocabulary is a count over Swift’s outputs, and the MTP weights are Swift’s own, requantised against its hidden states. The full model then checks the whole block of drafted tokens in one forward pass through the embeddings, body and lm_head, and keeps them up to the first one it disagrees with. A bad artefact makes the model slower, never wrong.

Speculative artefacts depend on the model

The MTP drafter scores a 40,960-row slice of lm_head instead of the full 248k vocabulary. It can never draft a token outside that slice. Every miss is a guaranteed rejection, and a rejection ends the speculation chain, so misses cost more than their share of tokens.

The shipped slice was counted over base Qwen’s outputs. Against what Swift actually generates, it covered 96.7 percent of tokens. That sounds fine until you look at where the other 3.3 percent sits. It is code and tool-call syntax, which is most of what my agent emits.

GPTQ calibration has the same problem. The Hessian you quantise against comes from the hidden states that feed the layer. States captured from base Qwen are the wrong calibration for a finetune that writes differently.

So I redid the fast-variant recipe with every input derived from Swift. The syv repo already had the recipe as scripts under drafter/, and I ran them unmodified. The work was feeding them the right corpus and handling a checkpoint layout they did not expect. Swift’s AWQ export splits lm_head, embeddings and MTP across separate shards, and upstream’s quantisation scripts assumed base Qwen’s single-shard layout.

Corpus, outputs, vocabulary

Upstream built its draft vocabulary from a Danish-heavy chat mix. That suits the people who wrote it and says little about a Rust coding agent. I wrote a generator that emits 5,500 prompts weighted by my workload. 35 percent is Rust implementation work, 16 percent TypeScript, 12 percent debugging from symptoms, 10 percent code edits and 8 percent tool-use transcripts. The rest is architecture questions, reasoning puzzles and general chat. About 60 percent of prompts run with thinking on, because that is how I use the model day to day. The generator is seeded, so the same config always produces the same corpus.

Running 3,072 of those prompts through Swift took about 1.7 GPU-hours and produced 4.2 million output tokens. I stopped there because held-out coverage had flattened. A vocabulary built from the first quarter of the data already covered 99.17 percent of held-out outputs, and the full run only reached 99.81.

The vocabulary is a frequency count over 90 percent of those outputs, with special tokens forced in and everything else ranked by count. The other 10 percent, 417k tokens, is the held-out set. Against it, coverage went from 96.7 to 99.81 percent, and to 99.86 on code prompts.

The overlap surprised me more. Only 18,701 of base Qwen’s 40,960 ids appear in Swift’s list at all. Swift 1.0’s corpus only ever emitted 25,879 distinct tokens, so the right list is shorter than the slot count. Any check that demands exactly 40,960 rows is wrong for this model, which mattered when I came to write the verifier.

GPTQ heads

With outputs in hand, the rest is upstream’s recipe. Capture hidden states for every token, dump the MTP Hessians, then GPTQ the lm_head and the eight MTP linears to int4 against them. I judged the lm_head by KL divergence from the bf16 head on held-out states. Round-to-nearest int4 scored 0.00707. GPTQ on Swift’s own Hessian scored 0.00234, lower than the 0.0029 that base Qwen’s fast variant shipped at on its own states.

The bug that earned a verifier

The fast directory shares body shards with the int8 directory through hardlinks, so a 15.2 GB variant costs almost nothing extra on disk. The catch is that vLLM reads every key of every shard it opens, not only the tensors the index assigns to that shard.

My first build hardlinked model-nonquant.safetensors, which still carried the int8 MTP tensors from the int8 build. The new int4 tensors lived in model_extra_tensors.safetensors. vLLM loaded both, and the boot died on a shape assertion that named neither file. Every existence check passed, because every mapped tensor existed in the right shard exactly once. Nothing looked at the other shards. Had the shapes matched, whichever copy loaded last would have won, and I would have served the wrong weights without an error.

Upstream’s verify.sh couldn’t help. It hard-required an int8 lm_head, so it rejected every int4 fast variant, base Qwen’s included. Until I had something better, my fast directory booted with VERIFY=0.

So I wrote a structural verifier before making the fast variant my default boot target. It reads safetensors headers directly and needs no torch. It checks that packed and scale shapes match the bit widths the config declares, that the draft head has one row per id in the draft vocabulary, and that every index-mapped tensor lives in exactly one shard, the one the index names. That last check catches the collision before anything boots. The repo ships a stdlib-only selftest that builds toy model directories, corrupts each one in a specific way, and asserts the verifier rejects all of them. The stale-tensor collision is one of the fixtures.

Results

All five configurations ran the same night with four repetitions each, and the table shows medians. Decode is tokens per second. Acceptance is the fraction of drafted tokens the target kept, read from vLLM’s own speculative-decoding counters. Tok/step is tokens produced per forward pass.

variant	decode	acceptance	tok/step	VRAM MiB
Swift int8, MTP long	94.0	0.630	2.89	22085
Swift-fast, MTP long	98.4	0.660	2.98	22225
Qwen fast, MTP long	98.2	0.634	2.90	22191
Swift-fast, DFlash2	156.4	0.385	3.70	23081
Qwen fast, DFlash2	167.4	0.387	3.71	23007

MTP with the long context is my daily profile. There, Swift-fast decodes 4.7 percent faster than Swift’s int8 build and matches base Qwen’s fast variant, 98.4 against 98.2 tokens per second, with higher acceptance. A separate quote benchmark asks the model to reproduce 8,000 tokens of its own context verbatim. Swift-fast drafts it at 0.998 acceptance and 3.99 tokens per step against a ceiling of 4, up from 0.962 on the int8 build.

The DFlash2 profile is faster on paper, and I still don’t use it as the default. It caps context at 46k tokens, and my agent’s wall time depends on how many reasoning tokens a task takes and whether it finishes, more than on raw decode. In my agent-latency runs, Swift-fast and Qwen-fast finished the same share of tasks, and long agent sessions need the 114k context more than the extra speed.

The new vocabulary helped DFlash2 anyway. On an earlier night, Swift’s DFlash2 acceptance trailed Qwen’s, 0.367 against 0.387. It now sits at 0.385, which is noise. The remaining gap of 156 against 167 tokens per second comes from the AWQ body reading about 0.24 GiB more weight bytes per step than Qwen’s AutoRound body. No drafter fixes that. Requantising the body would, but it means a 55 GB source download and more GPU hours to win back 7 percent on a profile I don’t run daily. I skipped it.

Quality held. On a nine-task battery with mechanical checks, the fast variant passed 8 of 9, the same set as the int8 build. The failure is a hard combinatorics problem that both models get right on some samples and wrong on others. I saw no regressions in how reasoning terminates.

Splitting the work

Two of the fixes weren’t specific to Swift, so I sent them upstream. PR 122 makes verify.sh accept the int4 lm_head layout and checks the packed shape against the declared bit width. PR 123 adds the duplicate-tensor scan. syv merged both.

The target-specific pipeline stayed in my repo, swift-qwen3.8-rtx3090. A workload mix is an opinion about one person’s usage, and HyperQwen is deliberately model-agnostic. The repo runs upstream’s scripts rather than forking them, and adds the corpus generator, vocabulary builder, head quantisation and verifier around them.

Publishing the weights

The finished model is on the Hugging Face Hub as liamwh/Swift-Qwen3.8-27B-W4A16-syv-fast. It serves directly on the syv stack with no preparation step. The model card carries both licences the Swift Open License requires of derivatives, and traces the provenance from Alibaba through UkisAI and TheUnderscore to this pipeline.

The upload was much smaller than the model. The AWQ body shards are byte-for-byte identical to TheUnderscore’s checkpoint, so Xet deduplication only transferred what I changed: the int4 heads, the MTP tensors, the extra tensors and the model card. About 2 GB of upload published a 15.2 GB model.

The pipeline repo still contains no weights. If you want to run Swift-fast, download the published model. If you want to rebuild it, or build the same thing for another Qwen3.8-family finetune, use the pipeline with a checkpoint you’re licensed to use. Edit the workload mix first. The draft vocabulary is only as good as the corpus behind it.

Swift 1.5

UkisAI shipped Swift 1.5 in late September 2026. It is not a new architecture. It is Swift 1.0 with more post-training, aimed at coding and long-horizon agentic work. UkisAI’s tokenizer and chat template files are byte-identical to 1.0’s (I hashed them, the hashes are in the pipeline repo’s docs/provenance.md), and the original Qwen3.8 MTP head is still in the box. All fifteen MTP tensors in the official AWQ export are bit-identical to base Qwen’s. I checked that before anything else, because the whole fast variant depends on it.

The rebuild ran the same pipeline on the same terms. Same seeded corpus, same 3,072 prompts, same 90/10 split. The one structural difference: UkisAI’s AWQ ships the MTP module as its own shard instead of TheUnderscore’s model-nonquant.safetensors, so the assembly script needed a small generalisation. The finished directories come out the same size as the 1.0 ones. The two fast directories differ by 1.6 MB of safetensors, and the two int8 directories by about 1 KB.

The vocabulary result was the surprise. A list rebuilt from 4.31M of Swift 1.5’s own output tokens has 25,285 ids and covers 99.79% of held-out 1.5 output. The Swift 1.0 list, with 25,879 ids, covers 99.81%. The 1.0 list transferred almost perfectly. The two lists share 21,494 ids out of a 29,670 union (Jaccard 0.724). Each covers more than 99.7% of the same held-out tokens, so the ids that only one list has account for about 0.2% of them at most. Base Qwen’s 40k list still misses 3.5%.

That is narrower than saying the output distribution didn’t change. Coverage only asks whether a token is in the list. It says nothing about how likely each token is. What the numbers support is that the set of tokens Swift draws on moved a lot from Qwen to Swift and hardly at all from 1.0 to 1.5.

The new heads quantise slightly better than 1.0’s: lm_head

KL 0.00221 against 0.00234, on Hessians from 1.5's own hidden states. MTP relative errors land where upstream's shipped variant does.

Swift 1.5, measured

My first pass at this was wrong in ways worth describing. I ran each configuration once, in a fixed order, and read a 3% speed loss for 1.5 and an 18% gain from the fast variant off it. Neither survived a controlled rerun. The 1.5 int8 leg had run first. The 1.0 comparison leg had run last. The same 1.0 build had already moved 6% between two nights.

So I reran it properly. Twenty-five container boots, one configuration per boot, in an interleaved order, on one RTX 3090 at its default 350 W limit (my earlier write-ups said 250 W, which does not match the card). Same pinned stack, same prompts, fixed seeds. Each boot gets one discarded warm-up pass, then the six-prompt suite four times at temperature 0.7, then four times at temperature 0. The 1.0 and 1.5 builds went 1.0, 1.5, 1.5, 1.0 and then 1.5, 1.0, 1.0, 1.5, so drift and order cancel. Tokens per step and steps per second come from vLLM’s own counters, pooled over each build’s boots.

	Swift 1.0 fast	Swift 1.5 fast
speculation off, tok/s	52.29	52.24
MTP, T=0.7, tok/s	104.6	103.6
acceptance	0.668	0.657
tokens per step	3.005	2.971
verify steps per second	34.8	34.9
MTP, T=0, tok/s	108.6	104.7
acceptance	0.663	0.636
tokens per step	2.988	2.909
verify steps per second	36.3	36.0

The speculation-off rows are four boots, two per build. Everything else is eight boots per build, each with its own published draft list.

Raw decode is the same. With speculation off, 1.5 runs at 52.24 tok/s against 52.29 for 1.0, and at temperature 0 at 52.60 against 52.66. Prompt by prompt the difference is under 0.1% and all six prompts are level. Whatever UkisAI’s extra post-training changed, it did not change how fast one forward pass runs, which is what the identical file sizes already suggested.

With speculation on, 1.5 is a little slower, and the size depends on how you sample. At temperature 0.7 it is about 1% behind, and prompt by prompt the mean is -0.5% with a 95% interval from -3.1% to +2.1%, which does not establish a difference. Under greedy decoding it is 3.5% behind, and prompt by prompt -3.6% (-9.7% to +1.5%), most of it from one prompt that runs 17% slower.

Almost all of that is yield, not step time. Verify steps per second differ by +0.2% at 0.7 and -0.9% at 0. Tokens per step fall 1.1% and 2.6%. The drafter simply gets fewer tokens accepted on what 1.5 writes.

I would not call 1.5 slower on this evidence. It has the same raw speed, and its speculative throughput sits between level and a few percent behind, inside a spread that the same build shows against itself. Across boots the 1.0 build alone pooled anywhere from 99.2 to 107.0 tok/s. The prompts differ in speed far more than the builds do, so the only fair comparisons are prompt by prompt and between adjacent boots.

The fast variant’s gain over int8 heads on 1.5 also shrank. The int8 leg, this time run ninth, measured 98.6 tok/s against 103.6 for the fast build, about 5%, and 102.0 against 104.7 at temperature 0. The 18% I reported first did not reproduce. Against plain decoding the fast variant is 1.98 times faster, and that figure holds.

VRAM is the peak on a card that also runs my desktop, which moves by about 1 GiB on its own, so I stopped reporting it.

The quote benchmark was measuring the list

The 1.0 build reproduced an 8,000-token document verbatim at 0.998 acceptance. Swift 1.5 managed 0.927 with its own list. I first read that as 1.5’s post-training taking something away. The data say otherwise. Every leg reproduced the document exactly, so the target side is the same, and three different models behind base Qwen’s list (Swift 1.0 int8, Qwen fast and Swift 1.5 int8) logged identical counts: 18,873 tokens drafted, 18,162 accepted, 6,291 steps. Acceptance here follows the list and nothing else. In the new session the same 1.5 list scored 0.927 on the 1.0 target too, and the 1.0 list scored 0.997 and 0.998 on both targets.

A token outside the list can never be drafted, so I tokenised the document and counted. Base Qwen’s list misses 232 of its 8,151 tokens, the 1.5 list misses 236, and the 1.0 list misses 10. The 1.5 list’s misses are four tokens, logger, .getLogger, (__ and __), 56 appearances each. The document is synthetic Python, and none of the four tokens appeared in the 90% of 1.5’s output that I counted into the list. Aggregate held-out coverage of 99.8% hid the problem completely. A few repeated misses can wreck a speculation chain while barely moving a coverage number.

Which draft list to ship

That raised the question I had skipped. Is the list I rebuilt for 1.5 actually the best one for 1.5? I built three variants of the 1.5 fast directory that differ only in the draft list: 1.5’s own, the 1.0 list, and the union of both. Each draft head is rebuilt from 1.5’s own int4 lm_head, so nothing but the id set changes. As a control, rebuilding the shipped list reproduced the shipped head tensor for tensor. I ran the three lists in mirrored order, two boots each, and ran the same three on the 1.0 target to separate model from vocabulary.

1.5 target, draft list	ids	T=0.7 tok/s	T=0 tok/s	T=0 acceptance	T=0 tokens/step	quote acceptance
Swift 1.5	25,285	104.3	107.2	0.636	2.909	0.927
Swift 1.0	25,879	103.6	108.4	0.644	2.933	0.998
union	29,670	106.2	108.6	0.646	2.937	0.998

The union beat the 1.5 list in all four paired boots, by 1.8% at 0.7 and 1.3% at 0. Prompt by prompt that is +1.6% (-0.9% to +4.2%) and +0.6% (-1.7% to +2.5%), with 5 and 4 of 6 prompts faster. The intervals include zero, so this is a consistent small gain, not a proven one. The 1.0 list was not consistent, 1.3% slower at 0.7 and 0.9% faster at 0. The bigger draft head cost nothing I could measure. Steps per second went up slightly, 35.5 against 35.1.

The same table answers the question I started with. At temperature 0 the list moves acceptance by about one point, 0.636 to 0.646 on the 1.5 target. Switching from the 1.5 to the 1.0 target with the same list moves it by two to three points. So most of 1.5’s lower acceptance belongs to the model and not to my vocabulary. Why 1.5’s text is harder for the fixed drafter to predict, I can measure but not explain.

I adopted the union. It was faster in every paired boot, it removes the quote cliff, and it costs about 10 MB. The Hugging Face repo now serves it on main, and the tag vocab-1.5-only keeps the revision I published first.

Quality

On the nine-task quality battery, 1.5-fast passes 8 of 9, the same score as both 1.0 builds. The failure is new: an open TypeScript task where the model spent all 8,192 thinking tokens without writing code. 1.0 passed that task. One sample at temperature 0.7 neither establishes a regression nor rules one out, so my small battery did not establish a clear regression, and the task goes on the watch list.

The upgrade call

I run Swift 1.5 as my daily model. These are my reasons, and I keep them apart from UkisAI’s.

What I measured is narrow. Raw decode speed is unchanged. Speculative throughput is level to a few percent lower, depending on how I sample. The cost of that is small next to the run-to-run spread of the setup.

What UkisAI reports is theirs, not mine. On their model card, against base Qwen3.8-27B, Swift 1.5 scores 81.71% on LiveCodeBench v6 (76.76% for the base) and 72.13% on Terminal-Bench 2.1 (69.21%). Mean thinking tokens drop 24.5% on LiveCodeBench and 16.3% on Terminal-Bench. Those are comparisons with the base model, run on their own evals and settings, and I have not reproduced them. My agent’s wall clock depends on how many reasoning tokens a task takes and whether it finishes, and both live on the model side. If those claims hold for my work, a sub-5% change in speculative throughput does not matter. If a future workload makes raw decode the constraint, the 1.0 build is a model directory swap away, and the pipeline that built both is on GitHub.

What I tried next

This project optimises the model artefacts themselves: the draft vocabulary, the GPTQ heads, the verifier, the published weights. After shipping the Swift 1.5 fast build I went the other way and asked what a much larger model would do on the same card. Swift 1.5 Flash Next is a 125B-class mixture-of-experts model, far too large for 24 GB of VRAM, and strata-swift-flash-next-rtx3090 is the project that runs it on the same card in Zeus, my workstation, by keeping the expert weights in host RAM.

The two are complementary rather than versions of one implementation. Here I change what the model is; there I change how a much bigger one is served and integrated, with the Nix-managed service, GPU arbitration and Open WebUI wiring that make it the machine’s default backend. The techniques even rhyme: MTP speculation on both sides, but this pipeline quantises and calibrates the artefacts while Strata caches experts between RAM and VRAM.

I have not benchmarked the two against each other on matched tasks, so I make no claim that Flash Next writes better code or finishes agent work faster than this 27B build. The Strata write-up records what I did measure.