K2 Horizon Edge Models: Choose a Device Tier
K2 Horizon edge models can be mapped to device tiers without treating parameter count as a complete capability ranking.
Put an offline maintenance assistant on a field tablet, and the first problem is not reasoning. The OS, weights, KV cache, and thermal limit all want the same memory. I would choose K2 Horizon edge models by device budget, one narrow task, and a fallback route. This is not a three-model leaderboard. I have no matched edge-hardware results, so there is no latency winner here.
Start With the Device Constraint
Do not select the checkpoint first. Record usable memory after OS load, sustained power, acceptable latency, and required context. “Fits once” is not “runs for a shift.” Demos show the ceiling. Production shows the floor.
Memory, power, latency, and context budget
Context headlines can mislead. The 0.9B technical appendix at revision 2e1ac41 reports 1,078,285,824 parameters, 2.16GB of BF16 files, and 131,072 tokens with YaRN. Its BF16 KV-cache estimates rise from roughly 0.44GiB at 8,192 tokens to 7GiB at the limit, before overhead. A compact model becomes a large memory job when context is unlocked.
I would reserve 25–30% memory headroom and cap context to the smallest accepted input. Record cold-start and warm p50/p95 latency separately. On battery hardware, add joules per task and temperature after repeated runs.
Define one on-device task and fallback route
Use one task: read a 3KB maintenance alert, select one of four playbooks, and return route, confidence, and an evidence span. Reject invented values and malformed JSON. Low confidence, missing evidence, or two parse failures route to the next local tier or a server. Fallback is part of the product.
Map Compact Checkpoints to Tiers
These tiers describe deployment burden, not universal capability. Model names use core-count shorthand; embeddings increase the stored count.
| Device tier | Checkpoint facts checked September 15, 2026 | Starting role |
|---|---|---|
| Entry | 0.9B label; 1.078B actual; 2.16GB BF16; 131,072 context | Short routing with fallback |
| Mid | 3.78B core; 5.06B with embeddings; 10.1GB BF16; 524,288 context | More local reasoning if cooling allows |
| Upper local | 6.95B core; 9.00B with embeddings; 18GB BF16; 524,288 context | Workstation-class edge |
Entry-tier reasoning with the smallest model
K2 Horizon 0.9B is the sensible first build for a constrained device because it leaves the most room for application and cache. Its card lists English and Chinese and provides K2 parsers. Keep the task short, make output enumerable, and escalate ambiguity. I would not summarize an entire archive merely because 128K appears on the card.
Mid-tier and higher-capacity local routes
The K2 Horizon 3.7B model card at revision adcaf26 describes 3.78B core parameters, but embeddings lift the full count to 5.06B and BF16 files to 10.1GB. K2 Horizon 3.7B is mid tier only when weights, cache, and runtime fit together.
The K2 Horizon 7B model card at revision ff325e2 reports 6.95B core and 9.00B with embeddings, producing 18GB of BF16 files. K2 Horizon 7B is an upper local route, not a phone guarantee. It must sustain the task without swapping or throttling.
Validate One Edge Workflow
Lock quantization, runtime, and input set
Before running further, I narrowed the variables. Pin model SHA, weight hash, tokenizer, template, runtime commit, threads, backend, context, sampling, and power mode. Reuse 50 maintenance alerts, including malformed and ambiguous cases.
The official K2 quantization collection lists GGUF for all three and FP8 for 7B. Yet the compact GGUF cards say tensors remain BF16. GGUF is a container here, not proof of four-bit memory. Revalidate every quantization.
Measure task success, thermal behavior, and latency
Score playbook selection, valid fields, grounded evidence, fallback correctness, and prohibited output. Record peak memory, cold and warm latency, p95 after 30 minutes, energy per accepted result, temperature, and throttling. Report success with fallback rate. Escalating half the workload moves the cost; it does not remove it.
Limits and Trade-Offs
Model size does not predict every capability
Size is a capacity signal, not an acceptance result. The 0.9B vocabulary has 64,256 tokens; 3.7B and 7B use 250,624. Training and evaluation coverage also differ. I would ignore cross-benchmark headlines and grade only the maintenance fixture. That keeps K2 Horizon edge AI selection tied to the deployed job.
Edge results depend heavily on runtime support
The cards document Transformers, vLLM, and SGLang with remote code or K2-specific parsers. For llama.cpp, official GGUF cards still point to the IFM K2 Horizon fork while upstream support is pending. K2 Horizon local models may fit memory yet fail on tokenization, templates, or kernels. Test the shipped executable.
FAQ
Do compact K2 Horizon models support multilingual prompts?
Not equally. The 0.9B card tags English and Chinese; current 3.7B and 7B cards list English. Broad multilingual claims need language-specific tests.
Are safety evaluations available for the smallest checkpoints?
I did not find a dedicated compact-model safety scorecard in the current linked materials. The 0.9B appendix warns about inaccurate, biased, or unsafe text; earlier official 3.7B and 7B card revisions included similar safety warnings.
Can K2 Horizon edge models process images?
No. These are text-generation checkpoints, with no documented image interface, vision encoder, or image processor.
Does IFM publish mobile demo applications?
I found no official IFM Android or iOS demo. Device claims are positioning, not a downloadable reference implementation.
Is commercial support available for edge deployments?
The checkpoints use Apache-2.0, but these materials publish no model-specific edge support plan or SLA. Commercial rights and paid engineering support are separate; confirm support before delivery.
Conclusion
Choose K2 Horizon edge models by headroom: 0.9B for entry, 3.7B for a memory-capable mid tier, and 7B for upper local hardware. These are starting positions, not performance conclusions. Run the fixed workflow under sustained heat and keep escalation visible. My next action is 50 inputs on the real device, smallest checkpoint first. The conclusion is valid only when that run survives.
Previous posts:
