moeshift

Memory efficient and high performance Mixture-of-Experts inference for Apple Silicon

Automatically checked · GitHub checked 2026-11-14

moeshift

Overview

MoEShift

Run a local language model that’s bigger than your Mac’s memory.

On Apple Silicon, MoEShift keeps the full model on your SSD and reads the parts it needs as it works. That leaves more memory for the editor, browser and other apps you use alongside your local assistant.

On a MacBook Pro with an M5 Pro and 24 GB of memory, Qwen 3.8 Flash Next (125B) generated about 15 tokens per second on a short prompt and 13 on a long prompt. MoEShift’s reported peak memory use was about 8.5 GB, with Safari, Teams and IntelliJ open.

Use MoEShift with OpenCode or another OpenAI-compatible client.

Qwen 3.8 Flash Next answering in OpenCode on a Mac running MoEShift

Requirements

You need an Apple Silicon Mac with at least 10 GB of RAM, a fast SSD and Homebrew. The benchmarks below used Macs with 24 GB and 32 GB of memory.

Model in the setup wizardFree SSD space needed
Qwen 3.8 Flash Next (125B)120 GB
Qwen3.6-35B-A3B (35B)61 GB

Disk space includes both the downloaded model and the files MoEShift prepares for streaming.

Quick start

brew install davide-troiani/moeshift/moeshift
moeshift

On first launch, choose a model. MoEShift downloads it, prepares the files and starts a localhost server that handles one inference request at a time.

Configure your coding agent or API client with the OpenAI-compatible base URL http://127.0.0.1:8080/v1 and the model ID for your preset.

ModelModel ID
Flash Next (125B)qwen-3.8-flash-next-rco-gsq-iq3-xxs
Qwen3.6 (35B)qwen-3.6-35b-a3b-q6-xl

Tested models and benchmarks

Qwen 3.8 Flash Next on M5 Pro

The 125B model (GSQ-RCO IQ3_XXS) ran on a MacBook Pro with 24 GB of unified memory, using the internal SSD and default MoEShift settings. The Mac was plugged in, with the user's usual apps open.

MeasurementShort promptLong prompt
Input tokens726,060
Prompt processing-374.15 tok/s
Text generation14.69 tok/s13.14 tok/s
Wall time for a 512-token reply36.71 s55.13 s

Each column shows the median of three runs after one warmup, with prompt caching disabled. Peak process memory was measured separately at approximately 8.5 GB. Source and test conditions

Qwen3.6-35B-A3B on M5

The 35B model (UD-Q6_K_XL) ran on a MacBook Pro with an M5 chip and 32 GB of unified memory, using the internal SSD and default MoEShift settings. The Mac was plugged in, with a code editor open and a video playing in the background.

MeasurementShort promptLong prompt
Input tokens726,060
Prompt processing-473.75 tok/s
Text generation16.07 tok/s14.94 tok/s
Wall time for a 512-token reply33.64 s47.09 s

Each column shows the median of three runs after one warmup. Combined resident memory (RSS) for the router and worker peaked at approximately 7.7 GB. System swap stayed at 0 MB throughout the run. Source and test conditions

Browse community benchmarks

Configuration

To adjust memory use or conversation length, edit the JSON configuration and restart MoEShift. Configuration reference

The two presets above support SSD streaming. Other GGUF models load directly and have different memory requirements. Supported models · Architecture

Contributing

MoEShift is in alpha. If you hit a problem during setup or while using a model, open an issue. For fixes and pull requests, see the contributing guide.

Share a result from your Mac

Running a Mac with less than 24 GB? Tell us which model you tried, what else was open and how your Mac felt while it answered. Someone with the same Mac could use that information before downloading the model. Run the benchmark and share your result.

Credits and license

MoEShift builds on tc-mb/llama.cpp-omni, llama.cpp and ggml.

The setup wizard uses Qwen model files published by ISTA-DASLab and Unsloth.

Original Git history and copyright notices are retained. See LICENSE and the bundled third-party licenses. Model weights retain their own licenses.