Run DeepSeek V4 Flash on Mac With Palm-Infra

The first problem I hit was linking. On an 8GB M1 Mac mini, Palm-Infra’s current build detected i8mm, then failed on missing symbols. Its documented NEON fallback built cleanly; four targeted DeepSeek and SSD tests passed. That Mac is outside the project’s 48GB demonstration, so I did not run the 159.6GB checkpoint.

This DeepSeek V4 Flash Mac guide separates my build verification from the maintainers’ M5 Pro results. A compile is not a model run.

Confirm the Supported Mac Configuration

Chip, Memory, Storage, Runtime, and Model Format

The pinned Palm-Infra README supports Apple Silicon, without a Mac-by-Mac matrix. Large-MoE documentation demonstrates a 48GB Mac; the performance row uses M5 Pro. That is not proof that every DeepSeek V4 Flash Mac or MacBook LLM setup behaves alike.

Gate	Verified requirement or boundary
Chip	Apple Silicon; M5 Pro is the published benchmark host
Unified memory	48GB demonstrated; 16 GiB is assigned to the expert cache
Storage	159.6GB source download, plus a separate .mollm output and free working space
Runtime	DeepSeek path is CPU-only with SSD offload
Model input	Official Hugging Face directory with config.json, tokenizer.json, index, and 46 safetensors shards
Precision	Native FP8 E4M3 for dense/shared experts and MXFP4 for routed experts

The converter defaults to 16,384 tokens. Palm-Infra does not demonstrate DeepSeek’s full advertised window on Mac, so I would not raise --n-ctx without measuring KV memory and failures.

Experimental Features and Known Limitations

DeepSeek support is experimental. Metal exists elsewhere in the runtime, but this path is CPU inference. The SSD-offload protocol keeps dense weights in RAM, fetches routed experts with asynchronous pread, and pins the macOS cache by default.

The reported 9.52 prefill and 5.71 decode tokens per second used four threads, 16 GiB cache, pp256 + tg64, and three warmups on M5 Pro. It is one provisional process, not a five-process median. No GitHub release existed on September 8, 2026, so pin the commit.

Install Palm-Infra and Prepare the Model

Build the Supported Runtime

I verified this build at commit eacc4ac6cffe4dd87ecb3d88b7c660f38702343c on arm64 macOS:

git clone https://github.com/TencentYoutuResearch/Palm-Infra.git
cd Palm-Infra
git checkout eacc4ac6cffe4dd87ecb3d88b7c660f38702343c
python3 -m venv .venv
source .venv/bin/activate
python -m pip install cmake ninja numpy safetensors huggingface_hub
cmake -G Ninja -B build_i8mm -DCMAKE_BUILD_TYPE=Release
cmake --build build_i8mm -j

My older toolchain failed after AUTO enabled i8mm. This documented fallback built:

cmake -G Ninja -B build_neon -DCMAKE_BUILD_TYPE=Release -DMOLLM_ARM_I8MM=OFF
cmake --build build_neon -j

This verifies build recovery, not 8GB M1 inference.

Obtain the Approved DeepSeek V4 Flash Files

Use the official DeepSeek V4 Flash repository, not a derivative. A verified hf download --dry-run found 73 files totaling 159.6GB; the repository is public, ungated, and MIT-licensed.

hf download deepseek-ai/DeepSeek-V4-Flash \
  --revision 60d8d70770c6776ff598c94bb586a859a38244f1 \
  --local-dir /Volumes/ModelSSD/DeepSeek-V4-Flash

python models/deepseek_v4.py \
  /Volumes/ModelSSD/DeepSeek-V4-Flash \
  /Volumes/ModelSSD/deepseek_v4_flash_native.mollm \
  --prefill-seq-len 32 --n-ctx 16384

Conversion preserves checkpoint precision; do not add W4. Source and output coexist, while current docs give no exact output size. Check df -h /Volumes/ModelSSD first.

Run One Reproducible Inference Test

Freeze Prompt, Context, Precision, and Generation Settings

Start with Palm-Infra’s synthetic shape: 256 dummy prompt tokens, 64 generated tokens, 16,384 context, four threads, three warmups, native FP8/MXFP4, greedy decoding, and 16 GiB cache. Dummy tokens remove tokenizer drift.

mkdir -p runs
/usr/bin/time -l ./build_i8mm/mollm_bench \
  --device cpu \
  --package /Volumes/ModelSSD/deepseek_v4_flash_native.mollm \
  --prompt-tokens 256 --max-new-tokens 64 --n-ctx 16384 \
  --warmup 3 --threads 4 --temperature 0 --seed 42 \
  --ssd-cache-mb 16384 --ssd-io-workers 8 \
  --trace runs/dsv4-trace.json --output kv \
  2>&1 | tee runs/dsv4-eacc4ac6.log

For the fallback build, use build_neon; do not compare it directly with M5 Pro.

Capture Tokens per Second, Memory, Storage, and Errors

Keep KV output, trace, stderr, /usr/bin/time -l maximum RSS, du -h package size, and df -h capacity. The trace separates prefill, decode, routing, cache waits, and SSD reads. Run five fresh processes; report all results and the median. Record macOS, chip, memory, power mode, SSD, filesystem, and commit. One clean run is a smoke test, not a Mac performance claim.

Decide Whether Local Execution Fits

Privacy and Offline Benefits

After download, this Palm-Infra DeepSeek path can run prompts without a hosted service, reducing egress and supporting offline experiments. It creates no security boundary: prompts, traces, shell history, weights, and output remain under local account and disk controls.

Operational Limits Versus a Hosted API

DeepSeek V4 Flash local execution trades provider dependency for a large artifact, conversion time, SSD traffic, thermal variance, and manual recovery. A hosted API adds network and retention questions but supplies remote capacity and service operations. A DeepSeek V4 Flash Mac can fit reproducible Apple Silicon inference research. This experimental, release-less runtime is not evidence of a production SLA. This is where my data ends.

FAQ

Can managed Macs install the required build toolchain?

Only if policy permits Command Line Tools, Python environments, downloaded executables, and external volumes. Palm-Infra documents dependencies, not MDM exceptions. Have IT approve the commit, packages, and storage path.

Who owns downloaded model files on shared workstations?

MIT grants broad use rights but does not decide employment ownership. Employer device, IP, and asset policies control the local copy. Record its revision, license, custodian, owner, and approved users.

Does external SSD use affect device warranty or support?

Connecting an SSD does not automatically void Mac coverage. Apple’s Mac warranty excludes the third-party device and damage caused by a component that fails specifications. Check the SSD vendor’s warranty separately.

Can the runtime resume after a laptop sleep cycle?

Palm-Infra documents neither checkpointed generation nor sleep recovery. Keep the Mac awake during recorded runs. After sleep, confirm the volume is mounted; restart when state or I/O is uncertain.

How should teams remove model data during employee offboarding?

Revoke access, stop processes, then remove source shards, .mollm, Hugging Face cache, prompts, traces, logs, and backups under a ticket. Follow NIST SP 800-88 Rev. 2; file deletion may not satisfy SSD sanitization. Let IT choose approved cryptographic erase or reprovisioning.

Previous posts: