Colibri

1. What is Colibri?

Colibri is a program (an "inference engine") that runs very large AI language models, hundreds of billions of parameters in size, on an ordinary personal computer. Models like these normally need a rack of servers with expensive graphics cards. Colibri gets around that by keeping the parts used on every step in RAM and leaving the rest (the "experts") on your drive, pulling them in only when needed. The price you pay is speed: how fast it answers depends heavily on how fast your drive is.

It replaces renting a paid AI through an API: the model runs entirely on your machine and needs no account. Colibri is free and open source (Apache-2.0 license), written in C.

Plainly: this is a tool for engineers and researchers, not a "click install and go" app. The developers describe it as a research platform and say outright that they make no promise about speed. You will type a few commands and download a model that weighs tens to hundreds of gigabytes. This guide takes the lightest route a newcomer can realistically follow: the 35B Qwen3.6 model, about 20 GB.

A good fit if:

• You are curious and want to run a large AI model on your own computer, independent of any service.
• Your machine has at least 24 GB of RAM and a few tens of gigabytes free on an SSD, and you are fine typing commands.
• You like experimenting and can live with slow replies.

Not a fit if:

• You want an assistant as fast as ChatGPT. On an ordinary machine Colibri may produce only a few tokens per second, or fewer (a token is a small chunk of text; the AI writes one chunk at a time).
• Your computer has 8 GB of RAM or very little free disk space.
• You only have a phone. Colibri runs on Windows, macOS and Linux.

2. What you need

Computer: Windows 10/11, macOS or Linux. Prebuilt downloads exist for Linux (x86-64), Windows (x86-64) and Apple Silicon Macs (arm64). Intel Macs and ARM Linux machines have to compile from source, which is beyond a beginner guide.

RAM and disk (from the project's README, for the model this guide uses):

Model	Disk space	RAM	Graphics card
Qwen3.6-35B-A3B (recommended for a first try)	about 20 GB	24 GB	Optional
GLM-5.2 (the project's flagship model)	about 372 GB	16 GB minimum	Optional

A graphics card only makes things faster; it is not required. A fast NVMe SSD matters most for speed. The Qwen3.6 page itself suggests about 30 GB of RAM if you want it to run comfortably.

Software to install first:

• Python 3 (a programming language; here it only runs the coli launcher, while the engine itself is C and needs nothing else). Download from https://www.python.org/downloads/ . On Windows, tick "Add python.exe to PATH" during setup.
• The Hugging Face hf tool (for downloading the model from Hugging Face, a site where people share AI models). Installation is in Step 2.

No account, no API key, no payment. The only real cost is disk space and bandwidth for roughly 20 GB of downloads.

3. Step-by-step installation

Step 1: Download the prebuilt Colibri

1. Open https://github.com/JustVugg/colibri/releases/latest . At the time of writing the newest version is v1.12.1 (24 September 2026).
2. Scroll to "Assets" and download the file for your machine:
   • Windows: colibri-v1.12.1-windows-x86_64.zip
   • Apple Silicon Mac (M1 or later): colibri-v1.12.1-macos-arm64.tar.gz
   • Linux: colibri-v1.12.1-linux-x86_64.tar.gz
3. Unpack it into its own folder, for example colibri.

On Windows, right-click the zip and choose Extract. On macOS and Linux, run this in the folder holding the download:

mkdir colibri && tar xzf colibri-v1.12.1-linux-x86_64.tar.gz -C colibri && cd colibri

(On macOS, use the file name colibri-v1.12.1-macos-arm64.tar.gz instead.)

Inside you will find colibri (or colibri.exe), several sibling engines and a file called coli. On Windows there is also coli.cmd.

Note: on Windows, do not double-click the .exe files. They are engines; without a model they flash a window and close. The file to run is coli.cmd.

Step 2: Install Python and the model downloader

Install Python 3 using the link in section 2. Then open Terminal (macOS/Linux) or PowerShell (Windows) and install the hf tool:

pip install -U "huggingface_hub"

Check it:

hf --help

What success looks like: a list of commands such as download and auth. If it says hf was not found, see section 5.

On Windows there is another route from Hugging Face's own documentation:

powershell -ExecutionPolicy ByPass -c "irm https://hf.co/cli/install.ps1 | iex"

Note that this installer also adds a "skill" for AI coding assistants. If you do not want that, use the pip command above.

Step 3: Download the model (about 20 GB)

Pick a folder on a drive with free space, ideally a fast SSD. Use the model already converted for Colibri, the "gs64" build the project recommends.

Linux or macOS:

hf download Kreuzzelg/qwen36-35b-a3b-colibri-i4-gs64 --local-dir ~/Models/qwen36_i4_gs64

Windows (saving to drive D as an example):

hf download Kreuzzelg/qwen36-35b-a3b-colibri-i4-gs64 --local-dir D:\Models\qwen36_i4_gs64

What success looks like: a progress bar runs, and when it finishes the folder holds the model's files. Expect anywhere from tens of minutes to a few hours depending on your connection. If it is interrupted, run the same command again.

Model page: https://huggingface.co/Kreuzzelg/qwen36-35b-a3b-colibri-i4-gs64

Step 4: Check that the machine is ready

Open Terminal or PowerShell and go to the colibri folder you unpacked in Step 1.

Windows:

coli.cmd doctor --model D:\Models\qwen36_i4_gs64

macOS / Linux:

python3 coli doctor --model ~/Models/qwen36_i4_gs64

What success looks like: the tool reports what is ready and names anything missing (model files, permissions and so on). This command only reads; it changes nothing on your machine.

Step 5 (optional): Build from source

You only need this on an Intel Mac, on ARM Linux, or if you want a build tuned to your exact chip. It needs gcc (or clang) with OpenMP, git and make. On Ubuntu/Debian:

sudo apt update
sudo apt install -y build-essential git python3

Then fetch the code and build the Qwen3.6 engine:

git clone https://github.com/JustVugg/colibri
cd colibri
make -C c qwen36

The official per-OS guide is at https://github.com/JustVugg/colibri/blob/main/docs/quickstart.md

4. First use

From the colibri folder, start a chat with the model you just downloaded.

Windows:

coli.cmd chat --model D:\Models\qwen36_i4_gs64

macOS / Linux:

python3 coli chat --model ~/Models/qwen36_i4_gs64

What success looks like: the program loads the always-needed weights into RAM (the first launch takes a while) and then shows a prompt. Try a short message such as "Hello, who are you?" and press Enter. Text appears a few words at a time. On an ordinary drive, slow output is normal, because the model's experts are being read from disk. The longer you use it, the more Colibri remembers which parts you use most, and it speeds up.

Web interface. If you prefer a chat window in the browser:

Windows:

coli.cmd web --model D:\Models\qwen36_i4_gs64

macOS / Linux:

python3 coli web --model ~/Models/qwen36_i4_gs64

This opens a dashboard in your browser with a chat box and speed figures. The default address is http://127.0.0.1:8000 , reachable only from your own machine.

A few more handy commands, run the same way:

You want to	Command
See where the model will live (RAM, disk, graphics card)	coli plan --model <model folder>
Run an OpenAI-compatible API server with no browser	coli serve --model <model folder>
Measure and save the fastest profile for this machine	coli tune --model <model folder>

On Windows write coli.cmd in place of coli; on macOS/Linux write python3 coli. Where <model folder> appears outside a code block, replace it with your real path.

5. Common problems and fixes

Symptom	Cause	Fix
On Windows, double-clicking an .exe flashes a window and closes	The .exe files are only engines; they need the launcher to tell them which model to load	Run coli.cmd from PowerShell, or double-click coli.cmd; see section 4
python or python3 is not found	Python is not installed, or "Add python.exe to PATH" was not ticked	Reinstall Python from https://www.python.org/downloads/ with that box ticked; open a new command window and retry
hf command not found	The folder holding the command is not on PATH, or the window was opened before installing	Close and reopen the terminal; if it still fails, try python -m pip install -U "huggingface_hub" and open a new window, or follow Hugging Face's install guide
"engine is not built" when running coli	coli was run outside the unpacked folder, so it cannot find the engine beside it	Go into the unpacked colibri folder and run it again. If you cloned the source, run make -C c qwen36 first
The machine freezes or the program dies while loading	Not enough RAM: the Qwen3.6 model needs about 24 GB	Close other applications; on a machine with under 24 GB of RAM this model is not a fit
The download stops with "no space left"	The drive lacks about 20 GB	Free up space, or point --local-dir at another drive
Text appears very slowly	Slow drive: the model reads data from disk as it writes	Copy the model to an NVMe SSD. The project's own documentation says your drive sets the speed
coli doctor reports something missing	Wrong model folder, or the download did not finish	Check the path after --model and run the hf download command again to finish

If port 8000 is already taken when you run coli web, you may have another Colibri open. Close it, or add --port 8001 to the command to use a different port.

If you are still stuck, the project has an issue tracker at https://github.com/JustVugg/colibri/issues and a Discord community listed in the README.

6. Uninstalling and updating

Updating

• Prebuilt version: download the new release from https://github.com/JustVugg/colibri/releases/latest , unpack it into a new folder and use it as before. The model you downloaded stays as is; no need to fetch it again.
• Built from source: go into the cloned folder and run git pull, then rebuild with make -C c qwen36.
• The hf tool: run hf update.

Uninstalling

• Delete the unpacked colibri folder. Colibri installs nothing into your system.
• To get the disk space back, delete the model folder (for example ~/Models/qwen36_i4_gs64 or D:\Models\qwen36_i4_gs64).
• To remove the downloader: pip uninstall huggingface_hub.

7. Frequently asked questions

Does Colibri cost anything? No. Colibri is free, open-source software (Apache-2.0). The project lists the Qwen3.6 model under the Apache 2.0 license as well. Your only costs are disk space, electricity and the bandwidth to download.

Does it need the internet? Only to download the program and the model. Once those are on your machine, chatting runs locally.

Are my chats sent anywhere? The engine runs on your own machine, and the web server by default listens only on 127.0.0.1 (reachable from your machine alone). For anything more detailed, read the source code or the project's official documentation.

Will it run on a weak machine? This Qwen3.6 build needs about 24 GB of RAM. For a machine with less, the project has a smaller model, OLMoE (8 GB of RAM, about 7 GB of disk), but you must convert it yourself with a bundled Python tool, which is for people already comfortable programming. See the project README for details.

Do I need a graphics card? No. The README states plainly that none of the supported models requires a GPU; if you have one it runs faster.