Garlic LLM

A 9-billion-parameter AI running on a PS4. Offline.

Garlic LLM is a homebrew app that runs Qwen3.5-9B on the console's own GPU. No PC and no internet while it runs.

Download Source code Support

Characteristics
ModelQwen3.5-9B
Quantizationcustom Q4 and Q3
Chat speed31–39 tokens/s
Code speedup to 43 tokens/s
Context window16K tokens
Voice conversation7 languages
Dictation25 languages
Network needednone

Tested on a base/Slim PS4, firmware 11.52.

Figure 1. Recorded on the console: the model writes a turtle program, runs it, then answers a spoken question. The demo runs at temperature 0 so it can be reproduced; the chat's default is 0.7.

What it does

Text chat

The 9B model answers at 31–39 tokens per second, and up to 43 when it writes code. Speculative decoding (MTP) and custom 4-bit kernels make that possible on this hardware. The context window holds 16K tokens.

This paragraph arrives at 35 tokens per second, the middle of the range measured on the console. Each chunk is one token: a word, part of a word or a punctuation mark. A short reply of 60 tokens is on screen in under two seconds. A full answer of 300 tokens takes under nine seconds. Sixteen thousand tokens, the size of the context window, would take close to eight minutes. Code comes out a little faster, at up to 43 tokens per second. Nothing in this box is model output; it only shows the pace.

Screenshot of the chat: an answer with two short lists on why the sky is blue and sunsets are red.
Figure 2. A chat answer with two short lists.
Screenshot of the voice mode: a halo around the garlic mark and a spoken answer about the tallest mountain in the solar system.
Figure 3. Voice mode during a spoken answer.

Voice conversation

Speak to it and it answers aloud. Chatterbox TTS runs on the same GPU, in seven languages: Spanish, English, Portuguese, Russian, Italian, French and German.

Screenshot of dictation: a bar labelled Your message with a transcribed question, ready to send.
Figure 4. A dictated question, transcribed and ready to send.

Dictation

Hold a button and talk instead of typing. Parakeet turns speech into text on the console, in 25 European languages.

Screenshot of the Python mode: a rainbow spiral drawn by a turtle program the model wrote.
Figure 5. A drawing made by a program the model wrote and ran.

Python mode

Ask for a program. It writes MicroPython and runs it on the console itself, so the code is tested where it was written.

Screenshot of the Think mode: a counting problem answered after the model thought for 26 seconds.
Figure 6. A counting problem answered after 26 seconds of thinking.

Think mode

For harder questions, switch on Think and the model reasons before it answers. It is off by default, because most questions do not need it.

Requirements and install

Requirements
ConsolePS4 base or Slim PS4 Pro not tested yet
Firmware11.52 the version it was tested on
HomebrewGoldHEN
Free spaceabout 6 GB 12 GB if you copy the package to the console first
Networknone no PC or internet while it runs

Install in three steps

  1. Download the package from Hugging Face. It is about 5.7 GB and has the models inside.
  2. Copy it to the console and install it like any other homebrew package.
  3. Open Garlic LLM from the home screen. From here on it needs no PC and no connection.
Download
PackageGarlic LLM, app and models in one file
Size5.7 GB
Get it fromHugging Face

To update, download the new package and install it over the old one. The models are inside, so there is nothing else to copy. The README has the full instructions and troubleshooting.

How it works

A PS4 gives homebrew no compute API and no machine-learning library, so Garlic LLM talks to the GPU directly. The model's operations (matrix multiplies over 4-bit weights, attention, normalisation) are compute kernels built for the console's GCN architecture. The app dispatches them by writing PM4 command buffers by hand, the low-level packets a graphics driver would normally generate for you.

Speculative decoding with the model's MTP head drafts tokens ahead and the full model verifies them, which is how hardware from 2013 reaches 31–39 tokens per second. Everything is compiled with the open-source OpenOrbis toolchain.

  1. PromptTyped with the controller or dictated.
  2. TokenizerRuns on the CPU.
  3. PM4 command bufferWritten by the app, packet by packet.
  4. GCN compute kernelsRun on the GPU over 4-bit weights.
  5. TokensShown as text or spoken aloud.
Figure 7. One generation step. Stages 3 to 5 repeat until the answer ends.

Support the project

Garlic LLM is free and open source. If you want to see more of it (more consoles tested, faster kernels), you can help fund the work.

Or buy the guide, Nine Billion Parameters on a PS4: 147 pages on the reasoning the code cannot show. The app and the source stay free under GPLv3; the guide helps pay for the work.

GitHub Sponsors Ko-fi The guide

Or send crypto

  • BitcoinBitcoin network only

    1Fs8Htdgcy3K2D4bX1GfDi1Ty2M2sqyuhc

  • USDT (Tether)Tron network (TRC20) only

    TViJDfyo4mKEKsyDnBXpDqGFANTcyQ42aG

Questions

Does it need internet?

No. The model, the voices and the speech recognition are inside the app and run on the console. Nothing is sent anywhere.

Does it run on a PS4 Pro?

Not tested yet. It was developed and tested on a base/Slim console with firmware 11.52. If you try it on a Pro, a report on GitHub is welcome.

And on a PS5?

No. Garlic LLM is written for the PS4's GPU and system software.

Can it harm my console?

It is an ordinary homebrew app and contains no exploit. It does need GoldHEN, which you install at your own risk.

Why “Garlic”?

On the PS4, “Garlic” is the name of the memory path the GPU reads directly. Its sibling is called “Onion”. The model's weights live on the Garlic side, so the name was already there.