🧠 Private AI OS · BitNet.cpp · Offline-first

Your AI. On your device.
Zero cloud rent.

A one-command stack — BitNet.cpp 1.58-bit LLM + Qdrant vector search + Node API — that runs a capable AI OS on ordinary CPUs. Chat, RAG over your files, and a voice agent, all local, all yours.

Install in 3 commands See the 4 wedge use cases
6.17×
CPU speedup (x86)
82.2%
Energy saved
0
Cloud calls
4 GB
Disk footprint
4 highest-ROI wedges

Where Local AI wins economically

Frequent inference · privacy-sensitive · unreliable connectivity · GPU-cost bottlenecked — the four axes where BitNet flips the maths.

🧑‍💼
ROI 10 / 10

Private AI OS

For SMEs · Pros · Gov

A private assistant that indexes your files, emails, notes, contracts, procedures — and never sends a byte to the cloud.

  • Files stay on the device
  • Faster than a round trip
  • Air-gapped-friendly
📡
ROI 9.8 / 10

Offline Edge AI

For Field · Fleets · Ships

Runs on laptops, tablets, industrial computers, vehicles — anywhere connectivity is expensive, brittle, or forbidden.

  • Works without internet
  • No per-token cloud bill
  • x86 + ARM native
🧠
ROI 9.5 / 10

Local RAG · Digital Memory

For Consultants · Legal · Research

Ask across 10 years of your documents. BitNet's dedicated embedding models make CPU semantic search economically viable.

  • Qdrant vector store included
  • PDF · Markdown · text ingestion
  • Source-cited answers
🎙️
ROI 9 / 10

Voice-First Agent

For Accessibility · Ops · Meetings

Hands-free capture, transcription, and action — powered by VibeASR.cpp (Microsoft's BitNet-quantised speech recogniser).

  • Real-time ASR on CPU
  • Dictate → task → done
  • No cloud transcription fees
Why BitNet.cpp

The economics of 1.58-bit inference

Microsoft's official inference framework for 1-bit LLMs. Ternary weights, lossless quality, dramatic CPU speedups.

6.17×
Faster on x86
vs. full-precision baseline
5.07×
Faster on ARM
Apple Silicon · Raspberry Pi 5
82.2%
Less energy (x86)
70% on ARM
100B
Params on 1 CPU
Human-reading speed
2.4B
Default model
BitNet-b1.58-2B-4T
0.6B
Embedding model
1.42–2.28× vs. F16

Source: Bitnet.cpp technical report · microsoft/BitNet

Architecture

6 layers, one Docker stack

Runtime · Memory · Retrieval · Agents · Connectors · Policy. Everything runs on your device.

Clow Local AI OS — 6-layer architecture User accesses the OS through chat, voice, or REST API. Requests hit an agent orchestrator that coordinates a BitNet LLM, Qdrant vector search, and connectors to Microsoft 365, email, and files. Everything runs locally under a policy layer. L6 · INTERFACE Chat UI · Voice · REST API · CLI L5 · AGENT ORCHESTRATOR Plan → Retrieve → Reason → Act → Audit L4a · RUNTIME BitNet.cpp · 1.58-bit LLM CPU · i2_s quant L3 · MEMORY / RAG Qdrant + BitNet-embed Semantic search on CPU L4c · VOICE (opt-in) VibeASR.cpp RTF < 1 · multilingual L2 · CONNECTORS (opt-in) Files · M365 · IMAP · Notion · GitHub · Postgres L1 · POLICY & AUDIT Role-based access · Encryption at rest · Immutable audit log · Local-only enforcement
Runtime Memory + Connectors Voice (optional)
Install

Standalone on your device — 3 commands

Docker is the fastest path (no local toolchain). Native scripts are also provided for full control.

Fastest path — one file, one command

Requirements: Docker 24+ · 8 GB RAM · 4 GB free disk · any x86_64 or ARM64 host.

git clone https://github.com/dnzengou/clow.git && cd clow/local-ai cp .env.example .env docker compose up -d

Then open http://localhost:8080 for the chat UI, or curl http://localhost:8080/health.

Native install — full control

Requirements: macOS 12+ or any modern Linux · git · conda · cmake ≥ 3.22 · clang ≥ 18.

curl -fsSL https://raw.githubusercontent.com/dnzengou/clow/main/local-ai/install.sh | bash cd ~/clow-local-ai ./start.sh

The script clones BitNet, sets up a conda env, downloads the 2.4B model (~2 GB), builds i2_s kernels, and boots the Node API on :8080.

Native install — Windows 11

Requirements: Visual Studio 2022 with Desktop-dev C++, Clang, CMake, MSBuild-LLVM workloads · conda · run from a Developer PowerShell for VS 2022.

iwr -useb https://raw.githubusercontent.com/dnzengou/clow/main/local-ai/install.ps1 | iex cd $HOME\clow-local-ai .\start.ps1

First run compiles BitNet.cpp — expect 30–60 min on modern CPUs. Subsequent runs boot in seconds.

Raspberry Pi 5 (ARM64) — the "$100 PicoClaw"

Requirements: Pi 5 with 8 GB RAM · Raspberry Pi OS 64-bit · Docker installed.

curl -fsSL get.docker.com | sh git clone https://github.com/dnzengou/clow.git && cd clow/local-ai BITNET_MODEL=BitNet-b1.58-2B-4T THREADS=4 docker compose up -d

ARM64 wheels use the TL1 kernel for extra speed. Pairs naturally with PicoClaw — "$10 hardware, private AI brain."

Full docs, examples, and troubleshooting: local-ai/README.md

What ships in the box

Batteries-included — but replaceable

💬

Chat endpoint

POST /chat — OpenAI-compatible interface, streamed tokens, system prompts, session memory.

🔎

RAG endpoints

POST /ingest to add a document · POST /search for semantic search · POST /ask for cited answers.

🎙️

Voice bridge (opt-in)

POST /transcribe — WAV/OGG in, text out. Uses VibeASR.cpp when the container is running.

📊

Prometheus metrics

GET /metrics — tokens/sec, latency, cache hits. Hook up any Grafana or the built-in Clow ARM dashboard.

Get early access + the deploy walkthrough

Drop your email. You'll get the private-beta invite, the enterprise-connector roadmap, and a 10-min video that walks through the full stack.

No spam. One email when the private beta opens. Unsubscribe with one click.
✅ Copied