Running Kokoro TTS on an iPhone: MLX, ONNX and the iOS limits nobody documents

Choosing a TTS engine for audiobooks: the voice showdown

Before any of the architecture, we had to pick a voice, and a thirty-second sample cannot pick one. The failures that matter in a ten-hour book are cumulative: prosody that goes samey after an hour, artifacts where one chunk of audio joins the next, invented names mangled a hundred times over. So we built a showdown page for sitting with a voice.

It rendered long passages from Project Gutenberg through every engine behind one contract, each passage chosen to stress something: quoted dialogue, hard proper nouns (Queequeg, Pequod, Bildad), heteronyms, dense exposition as a fatigue test, and plain narrative as the baseline hour. On the page, the number keys switch engine mid-sentence, landing at the same place in the text rather than the same second, because once durations diverge a shared clock drops you into a different sentence. Clicking a word jumps there, and every take gets a star rating and notes that follow the voice across passages.

In the showdown: Kokoro in all 28 of its English voices; Resemble AI's Chatterbox, Chatterbox Turbo and Chatterbox Nano, including zero-shot clones from reference voices we screened out of the LibriTTS-R corpus; Speechify's Simba; ElevenLabs v3; and Apple's system voices. We also ran every engine two ways, a sentence per call and paragraphs packed together, because chunking turned out to be a variable and not a detail: Kokoro leaves 0.90 s of dead air between sentences fed one at a time, and 0.16 s when they arrive together.

The ear threw out Chatterbox. The contract threw out most of the rest, because a reader that highlights each word as it is spoken needs two things from an engine at once: word-level timings, and a way to correct a pronunciation that does not break those timings.

engine       word timings           pronunciation fix     timings survive it
Kokoro       exact, native          inline phonemes       yes
Simba        native, char offsets   <sub alias> only      no, needs a remap
ElevenLabs   forced alignment call  none needed           n/a
Chatterbox   none                   none                  n/a

ElevenLabs read every hard case right from context with no help at all, and it became the correctness reference. Kokoro was the only engine with both properties. Its override channel is Misaki's inline markup, [word](/phonemes/), which is stripped before tokenising, so the model is steered while the timings still index the clean text.

That still left the voices every iPhone already has. Apple's system voices are what almost every read-aloud app ships with, and a Mac's say command could not stand in for them. So the last round ran on the phone, in the app, on an iPhone 16 Pro Max: every English voice iOS lets a third-party app use (Ava and Zoe at Premium quality, Evan and Nathan at Enhanced; Siri's voices are off limits to apps) against Kokoro running on the same chip.

Kokoro was a major step up. That audition is the reason Lector exists at all: a reader built on AVSpeechSynthesizer would be one more of many. So from that point the voice was not negotiable and the architecture had to bend around it. Everything below is the bending.

Architecture 1: Kokoro on AWS Lambda, and what cloud TTS costs

The first version, in April, was the conventional one: a React and Capacitor front end, and an AWS backend with Cognito, DynamoDB for reading progress, and Kokoro's ONNX build running in a container Lambda that generated a chapter of audio at a time.

It worked, and it put a network round trip between the reader and every chapter. Audio for the next chapter needed a connection, a chapter-sized Lambda invocation on CPU, and a wait, and every hour of listening was an hour of server-side inference that we were paying to run. That operating cost grows with every chapter anyone listens to, forever, and it never goes away while the service is up. And a book is something you carry onto a plane or into a basement; its voice shouldn't depend on a server being reachable.

Hosting our own model was the cheap version of the cloud. The alternative is renting a premium voice API, and those bill by the character. A novel runs about 550,000 characters, roughly 100,000 words or nearly ten hours of audio. At the published prices when we checked in September 2026, generating one audiobook costs:

provider     $ per 1M characters   one novel, one voice
ElevenLabs   $91 to $165           $91 to $165 (depending on plan)
Cartesia     $31 to $42            $17 to $18 (on the larger plans)

That is per book, per voice, and it is paid again every time a pronunciation fix means re-rendering. The best-sounding voices in the world are not the problem; the meter is. On the phone, generating the same novel costs us nothing.

Architecture 2: a local Kokoro server on a Mac

So the audio moved to a single Python process on a Mac: one machine, one person's library, Kokoro generating on demand, and the phone reaching it over Tailscale. The quality was right and there was no hosting bill. But the install was now "run a server," and the person this was built for is not expected to own a terminal.

Architecture 3: Kokoro in the browser, phonemes from a Lambda

Next attempt: a static site that runs the speech model in the browser, on the device. One piece would not come along. Kokoro does not read letters, it reads phonemes, and the grapheme-to-phoneme (G2P) engine it was trained against, Misaki, exists in Python, Rust and Swift but not JavaScript. The Rust port is not output-compatible with Kokoro's 114-symbol vocabulary. So we built a small Lambda that took deduplicated words and returned phonemes, with the sentences themselves never leaving the phone.

We abandoned it before it shipped. A web page on iOS cannot hold a 300 MB model in memory reliably, and it cannot keep playing audio when the screen locks. The second one is fatal for a book you listen to while doing the dishes. Both of those are things a native app can do, and MisakiSwift meant the phoneme service was unnecessary there too.

Architecture 4: a WKWebView shell, and MLX noise on the iPhone

The quickest native app is one that reuses everything. The reader UI already existed as a web page talking HTTP to the Python server, so the first iOS build was a thin Swift shell: a loopback HTTP server on 127.0.0.1, answering the same routes the Python server did, serving the unmodified page into a WKWebView. Kokoro ran in Swift on mlx-swift, Apple's machine learning framework, through the KokoroSwift port. (A WKURLSchemeHandler would have avoided the socket, but WebKit's media stack does not load audio through custom schemes reliably, and seeking needs byte-range support.)

The app built, signed, installed, and rendered books. Every macOS test passed. On the phone, it squealed.

Measure the phone, not the Mac

Same code, same weights, same phonemes, correct speech on the Mac and noise on the A18. We pulled the device's own WAV off the phone and compared spectra: correct speech had a spectral centroid near 2,945 Hz; the phone's output sat at 6,796 Hz with 53% of its energy above 8 kHz, clipping at full scale. We filed it as "MLX is broken on the A18" and routed around it, which turned out to be the most expensive sentence in the project.

The route around was ONNX Runtime. This time, before building anything on top, we proved it on the device. A reference script ran the model on the Mac and emitted the exact input token IDs and style vector as a Swift fixture, so the phone ran identical inputs and anything that differed was the runtime. The verdict was a Whisper transcript rather than a spectrum, because a spectral fingerprint cannot tell correct speech from fluent-sounding wrong speech.

Kokoro ONNX on iPhone 16 Pro Max
model             provider   speed          size
fp32              CPU        2.8x realtime  326 MB   <- ship this
fp32              CoreML     2.2x realtime  +4.6 s to build the session
q8f16 (quantised) CPU        1.9x realtime   86 MB
q8f16             CoreML     fails to build a plan
fp16              any        all NaN

0% word error rate on the phone. Both of our assumptions going in were wrong: neither quantising nor the CoreML execution provider was faster.

iOS kills the app in the background: "exceeding limit of 50% cpu over 180 seconds"

With working audio, the app started dying whenever the screen locked. Twice around 1 GB of memory, both times partway through generating. iOS kills without warning, and our guard read os_proc_available_memory, which claimed 2,372 MB free at the instant of death.

So we cut memory. The quantised model became the default, passes got shorter, caches got trimmed on the way into the background. Peak memory under lock fell from 1,004 MB to 873 to 742, and the app kept dying, at 64 seconds and then around 170. Each time the peak dropped, it died at a lower number. That is not the shape of a memory ceiling.

The crash report on the device said what it was. Not a jetsam memory kill; a CPU resource event:

Event:            cpu usage
CPU:              90 seconds cpu time over 105 seconds (86% cpu average), exceeding limit of 50% cpu over 180 seconds
CPU limit:        90s
Limit duration:   180s

iOS terminates a background app that averages more than 50% of one core across 180 seconds, charged across every thread in the process. The memory figures were just where the footprint happened to be when the window expired. And Kokoro on ONNX Runtime cost 1.3 to 1.5 CPU-seconds per second of audio in every configuration we could find:

Kokoro ONNX, iPhone 16 Pro Max
configuration          speed      CPU-s per audio-s
default (~3 threads)   1.4-1.6x   1.5
thread spinning off    1.4x       1.3
two threads            1.1x       1.4
one thread             0.2x       4.4-4.7
CoreML provider        0.9x       1.7-2.25

Keeping up under lock needs 0.5. Nothing was within 2.6x of it. (Turning off ONNX Runtime's spin-waiting worker threads was the one free win: spin-waiting is CPU time spent on nothing, and CPU time is exactly what the limit is charged in.)

Architecture 5: render ahead in the foreground, only play under lock

If generation cannot happen under lock, then generation has to happen before the lock, and nothing generated can ever be thrown away. That became the core decision record for the app:

• Every sentence Kokoro speaks is persisted, one AAC file per sentence plus a per-chapter JSON index of word timings, and never synthesised twice. A file per sentence rather than per chapter because an M4A cannot be appended once it is finalised, and the appendable alternative (raw ADTS) carries no gapless metadata. A crash loses at most one sentence.
• The app renders ahead whenever it is in front, walking forward from where you are. Move your place and the renderer follows you; nothing already made is discarded.
• Locked, the app only plays files. What is in flight finishes, and the reader stops cleanly at the end of what exists rather than waiting for iOS to kill it.
• Playback is AVAudioEngine, scheduling PCM buffers on an AVAudioPlayerNode, so a join between sentences is sample-accurate and the player clock is precise enough for word highlighting.

A small detail that surprised us: writing those sentence files through AVAudioFile reserves about 26 KB of free space in every file. Harmless on a chapter, roughly double the size of a store made of sentences. AVAssetWriter doesn't, and reading back through AVAudioFile still honours the encoder's priming and padding, so sentences decode sample-exact.

The same week, the web page went too. The native player, text view and library replaced it screen by screen, and once nothing was listening the loopback server, the page, and the local-networking exception in the app's Info.plist were deleted.

The real MLX bug on the A18: Neural Accelerator misdetection in mlx 0.30.1

Then invented names in a fantasy novel started coming out as nonsense on the phone, and correctly on the Mac. Misaki's fallback for a word it does not know is a small BART model, and it runs on MLX. We had moved synthesis off MLX, but never checked the other model sitting on it. One probe on the device, three minutes of work:

MisakiSwift G2P fallback, same build
Georgiana   mac ʤˌɔɹʤiˈɑnə   phone ɡˈɛɡəwɪŋɡ
Pirrip      mac pˈɪɹəp       phone fˈɪsᵊnɹˌɛt
Marlowe     mac mˈɑɹlO       phone məbˈɑlliət

Same symptom, second model, so this time we went looking for a mechanism instead of a detour. It was upstream and specific. mlx 0.30.1, which mlx-swift 0.30.2 bundles, decided in is_nax_available() that a device had Apple's Neural Accelerator matmul hardware whenever its GPU generation was 17 or higher on iOS 26.2 or later. The A18 is generation 17 and has no such hardware, so MLX ran NAX kernels the chip does not support and got wrong numbers back. Full-precision models were not spared: MLX_ENABLE_TF32 defaults to 1, and the matmul path takes the NAX route for fp32 whenever TF32 is on. ml-explore/mlx#3083 had already fixed it by requiring generation 18 on phones.

We verified it both ways before changing anything: the same installed build, launched with MLX_ENABLE_TF32=0 in its environment, produced the Mac's phonemes; launched normally, garbage. Pinning mlx-swift 0.30.6, the first 0.30.x release to carry the fix, cured the names, and almost certainly the original squeal too. Our by-ear pronunciation tables for a couple of books, it turned out, had been a workaround for this bug that nobody had diagnosed.

MLX vs ONNX Runtime for Kokoro on iPhone: benchmarks

With both runtimes producing correct audio on the same phone for the first time, we could finally compare them properly. Same phonemes, same audio lengths, scored by Whisper:

Kokoro-82M fp32, iPhone 16 Pro Max (A18 Pro)
chars   MLX (GPU)                  ONNX Runtime (CPU)
  59    5.1x realtime, 0.5 CPU-s   2.2x, 5.1 CPU-s
 119    3.8x,          1.1 CPU-s   2.4x, 9.3 CPU-s
 238    4.2x,          2.0 CPU-s   2.3x, 17.2 CPU-s
 293    4.1x,          2.5 CPU-s   2.3x, 22.3 CPU-s

Identical transcripts, about twice the speed, and a tenth of the CPU. When you render a book in the foreground, CPU is battery and heat, so MLX became the engine and ONNX was deleted. That took 392 MB of models and voices with it, and the built app went from 804 MB to 398 MB.

Kokoro's vocoder memory on MLX: 2.4 GB to 872 MB

A single MLX pass peaked at 1.4 GB for four seconds of audio and grew with length, which forced short passes and put an audible seam inside long sentences. A stage-by-stage memory trace put all of it in the vocoder. KokoroSwift evaluated it as one lazy graph, so the last upsampling stage held every intermediate of nine dilated convolutions at once (about 700 MB), and MLX's transposed convolution unfolded a column matrix that grew with the audio (about 850 MB at fourteen seconds).

We patched it to force evaluation after each residual pair and to run the transposed convolution in overlapping time chunks with overlap-add: the same operation, checked against the single call to within 1e-5. A 238-character pass went from 2.4 GB to 872 MB peak at the same speed, and on the phone, memory stayed flat through 293 characters, so whole sentences render in one pass again.

Metal crash on screen lock: iOS revokes GPU access in the background

MLX comes in well under the background CPU limit, so rendering under lock should have been back on the table. It isn't: iOS withholds the GPU from a backgrounded app. A pass started under lock sat for 198 seconds and finished the instant the screen unlocked. Worse, a pass in flight at the moment of locking crashed the app, once at 1.5 GB and once at 534 MB, so again not memory. The crash reports showed a Metal command buffer completing with an error after the GPU was revoked, and MLX's completion handler throwing on Metal's own dispatch thread, which aborts the process.

The fix is a rule: no GPU work may be issued once the app has resigned active. The Kokoro port asks a hook before every eval, the hook trips at willResignActive (the earliest signal iOS gives on a lock; didEnterBackground and the GPU revocation are a breath behind it), and the interrupted sentence becomes the first one rendered after unlock. Verified on the phone: locked mid-pass, a clean stop, 135 seconds of already-rendered audio played out under lock at 3% of a core, and no crash report.

So the render-ahead design survived the engine change, but now it stands on two legs instead of one. ONNX cannot render under lock because of the CPU limit. MLX cannot because of the GPU.

Where it ended up

Lector today is a native SwiftUI app with no server and no network access, which also means no inference fleet for us to host and pay for: the compute runs on the phone. It imports EPUBs, renders Kokoro audio sentence by sentence while you read, keeps every sentence, and highlights each word as it's spoken. Half-precision weights (164 MB instead of 327) run on a four-year-old iPhone 13 (A15, 4 GB) at about 1.8x realtime, peaking around 430 MB for whole chapters.

Kokoro question intonation is flat: blind listening tests

"We fixed everything" would be the more flattering sentence and the less true one. Kokoro reads every question flat.

We didn't take that on impression. A second audition page rendered pairs of sentences that differ only in the final mark ("You're coming with us?" against "You're coming with us.") through the reader's own engine, grouped by how much work intonation has to do: declarative questions where the mark is the only cue, inverted yes/no questions, wh-questions as the control (those fall in natural speech anyway), and one-word questions with nothing to build a contour on.

The page has a blind mode, which is the part that matters. It plays one clip at random without saying which it is, you call question or statement, and it keeps score against a 50% chance line. It exists because knowing which clip you're hearing is exactly what stops you from telling whether a twelve-hertz rise is audible. Try it yourself:

Blind, the Kokoro pairs were indistinguishable. And "Did you hear what she told me?", about as unambiguous as a question gets, did not sound like one in any of the 22 voices we ship.

The same sentences through ElevenLabs were easy to tell apart, which proved flat questions are particular to Kokoro rather than inherent to text-to-speech. It also exposed a trap in our own measure: we had been scoring pitch at the end of the sentence, which is a rise detector. ElevenLabs doesn't stick a rise on the end of a statement; it performs the whole sentence differently, with different emphasis and phrasing throughout, and it reads a question as a question even with a full stop, because it reads the sentence rather than the mark. On nine of twelve pairs, two takes of the same string moved its final pitch further than the question mark did.

Bending Kokoro's pitch curve doesn't make a question

Kokoro is built on StyleTTS2, which predicts a pitch (F0) curve before the decoder turns it into audio, so we tried to fake the question there and judged the fakes by ear. Ramping the predicted pitch up over the last few hundred milliseconds sounded like a statement with a rise stuck on the end, which is what it was. Cancelling the pitch declination across the whole sentence moved pitch far more and still didn't read as a question.

Two rejections in two shapes is the finding: pitch is not the mechanism. A question is decided upstream, in the style vector and the duration predictor, before there's any pitch curve to bend. The question mark does reach the model, as vocab token 6, and the Python kokoro package on the same weights produces the same flat contour; the issue is reported upstream against the ONNX model, the WebGPU demo and the official Hugging Face Space.

Last, we surveyed twenty other on-device models against Lector's constraints. Nothing else combines Kokoro's size, speed on a phone, pronunciation overrides, and word timings that survive them. The best evidence we found that questions are solvable at all belongs to a 4.4-billion-parameter model, more than fifty times Kokoro's size. The cloud voice that gets questions right, ElevenLabs, is the one billing $91 or more per novel, which is where we started.

What we'd tell anyone putting a model on a phone

FAQ

Why does MLX produce garbage or noise on an iPhone 16 but correct output on a Mac?

mlx 0.30.1 (bundled by mlx-swift 0.30.2) enables Neural Accelerator (NAX) matmul kernels on iOS 26.2 or later whenever the GPU generation is 17 or higher. The A18 is generation 17 and has no NAX hardware, so those kernels return wrong numbers, and fp32 models are affected too because MLX_ENABLE_TF32 defaults to on. It was fixed in ml-explore/mlx#3083; mlx-swift 0.30.6 is the first 0.30.x release with the fix. Launching with MLX_ENABLE_TF32=0 confirms the diagnosis.

Why does iOS kill my app in the background at a different memory figure every time?

It may not be memory. iOS terminates a background app that averages more than 50% of one CPU core over 180 seconds, charged across all threads. The crash report records it as a cpu usage event, "exceeding limit of 50% cpu over 180 seconds". The memory footprint at the moment of death is just wherever it happened to be when the window expired.

Can an iOS app run MLX or Metal GPU work while the screen is locked?

No. iOS withholds the GPU from a backgrounded app. Work submitted after the lock waits until the app returns to the foreground, and a Metal command buffer in flight at the moment of locking completes with an error, which MLX's completion handler turns into a process abort. Stop issuing GPU work at willResignActive, which arrives before the GPU is revoked.

How fast is Kokoro TTS on an iPhone?

On an iPhone 16 Pro Max, Kokoro-82M on MLX renders at about 4 to 5 times realtime using roughly 0.1 CPU-seconds per second of audio. ONNX Runtime on the CPU renders at about 2.3 to 2.8 times realtime and costs 1.3 to 1.5 CPU-seconds per second of audio. Half-precision MLX weights ran at about 1.8 times realtime on an iPhone 13.

Can Kokoro TTS read questions with rising intonation?

Not in our testing. In blind listening tests Kokoro v1.0 question and statement pairs were indistinguishable in every voice, including plainly interrogative sentences, and bending the predicted pitch curve did not fix it. The question mark reaches the model as a token but does not produce question intonation.

Kyle Twogood is the founder of Magrathea Software. He’s been building production software since 1997.