How to Use Gemma 4 to Set Up OpenClaw for Free

Google's Gemma 4 is one of the most capable open-weight models available today, and when paired with the right local tooling, it becomes a free, privacy-first alternative to paid AI coding assistants like Claude or GPT-4. In this guide, you'll learn how to set up OpenClaw — an open-source Claude-compatible AI assistant — powered entirely by Gemma 4 running on your own machine.

No API keys. No monthly fees. No data leaving your computer.

What you need

Step 1: Install Ollama

Ollama is the simplest way to run open-weight models locally. It handles model downloads, hardware acceleration, and exposes a local API endpoint.

macOS / Linux:

`curl -fsSL https://ollama.com/install.sh | sh`

Windows: Download the installer from ollama.com and run it.

Verify the install:

`ollama --version`

Step 2: Pull Gemma 4

Once Ollama is running, pull the Gemma 4 model. Google offers several sizes — start with the 12B parameter version for a good balance of speed and capability:

`ollama pull gemma4:12b`

For machines with more VRAM or RAM, use the full 27B version:

`ollama pull gemma4:27b`

The download is around 7–16GB depending on the size. Once complete, test it:

`ollama run gemma4:12b "Explain the difference between REST and GraphQL in one paragraph."`

You should get a sharp, well-reasoned response within a few seconds.

Step 3: Install Open WebUI (your Claude-like interface)

Open WebUI gives you a polished chat interface — similar to Claude.ai — that connects directly to your local Ollama instance.

Install via Docker (recommended):

`docker run -d -p 3000:3000 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main`

Or install without Docker using pip:

`pip install open-webui && open-webui serve`

Then open your browser to: `http://localhost:3000`

Create a local admin account (credentials stay on your machine), select gemma4:12b from the model dropdown, and you're live.

Step 4: Connect to Claude Code (optional — AI coding agent)

If you use Claude Code (Anthropic's CLI agent), you can point it to your local Gemma 4 instance via Ollama's OpenAI-compatible API endpoint.

Set the following environment variables before running Claude Code:

`export ANTHROPIC_BASE_URL=http://localhost:11434/v1`

`export ANTHROPIC_API_KEY=ollama`

Then launch Claude Code as normal:

`claude`

Claude Code will now route requests to your local Gemma 4 model. This works best for code review, refactoring, and general Q&A tasks. Complex multi-step agentic workflows may need the larger 27B model.

Note: Compatibility varies by task. Gemma 4 handles coding exceptionally well. For long agentic workflows, you may see differences from Claude Sonnet or Opus.

Step 5: Optimize for your hardware

Enable GPU acceleration (NVIDIA)

Ollama auto-detects CUDA. Verify it's being used:

`ollama ps`

Look for GPU in the output. If missing, install the CUDA toolkit and restart Ollama.

Enable GPU acceleration (Apple Silicon)

Ollama uses Metal automatically on M1/M2/M3/M4 Macs. No extra setup needed — it's fast.

Tune context window

For coding tasks with large files, increase the context window in Open WebUI:

What you can do with this setup

Bonus: Add web search with Perplexica

Want your local AI to search the web like Perplexity? Install Perplexica alongside Open WebUI:

`git clone https://github.com/ItzCrazyKns/Perplexica.git`

`cd Perplexica && cp sample.config.toml config.toml`

Edit `config.toml` to point `OLLAMA_API_URL` to `http://localhost:11434`, then run:

`docker compose up -d`

Open `http://localhost:3001` to get a full Perplexity-like research interface powered by Gemma 4.

Total cost: $0

Wrapping up

In about 20 minutes you've got a fully functional, privacy-first AI assistant running on your own hardware — no subscriptions, no data sharing, no rate limits. Gemma 4's strong coding and reasoning performance makes it one of the best free options available today for developers and power users who want Claude-quality output without the cost.

If you'd rather skip the DIY setup and have a production-grade AI agent built and deployed for your business in 14 days, see how our AI agents work or explore agents that handle payments and bookings.

Frequently asked questions

What hardware do I actually need to run Gemma 4 well?

For the 12B model, 16GB of unified memory (Apple Silicon) or 12GB+ VRAM (NVIDIA) gives you fast, responsive output. The 27B model wants 32GB+ unified memory or a 24GB GPU like an RTX 3090/4090. CPU-only works but expect 2–10 tokens/sec on the 12B.

Is Gemma 4 really free for commercial use?

Yes. Gemma is released under the Gemma Terms of Use (Apache 2.0–style permissions for most cases), which allows commercial use, fine-tuning, and redistribution. Always review the latest license text before shipping a product built on it.

Can I run this without Docker?

Yes. Use the `pip install open-webui && open-webui serve` path in Step 3. Docker is just the cleanest way to isolate the install and survive OS upgrades.

Why is my model running on CPU instead of GPU?

Run `ollama ps` — if you don't see "GPU" in the output, the runtime didn't detect your accelerator. On NVIDIA, install the latest CUDA drivers and restart Ollama. On Apple Silicon, make sure you're using the native (not Rosetta) Ollama build.

How do I make responses faster?

Three levers: pull a smaller quantization (`gemma4:12b-q4_K_M`), reduce `num_ctx` to the smallest size your task needs, and close other GPU-heavy apps. For pure speed, the 12B at 4-bit quantization is hard to beat.

Can multiple people on my network use the same Ollama instance?

Yes. Set `OLLAMA_HOST=0.0.0.0:11434` before starting Ollama, then point Open WebUI (or any teammate's client) at `http://your-machine-ip:11434`. Put it behind a VPN or reverse proxy with auth — never expose it raw to the public internet.

How does this compare to Claude Sonnet or GPT-5 in real use?

For code generation, refactoring, explanations, and writing, Gemma 4 (especially 27B) is competitive. For long agentic workflows, deep reasoning, and tool use, frontier hosted models still pull ahead. For most daily developer tasks, the gap is smaller than the price difference suggests.

Will my data stay private?

Yes. Ollama and Open WebUI run entirely on your machine. No prompts, files, or completions leave your hardware unless you explicitly add a plugin (like web search) that calls out.

How do I update Gemma 4 when a new version drops?

`ollama pull gemma4:12b` re-fetches the latest tag. To see what you have locally, run `ollama list`. To free disk, `ollama rm <model>`.

When should I stop DIY and have an agent built for me?

DIY is great for personal coding help and experimentation. If you need an agent that handles customer conversations, books appointments, processes payments, and integrates with your CRM with SLAs and uptime — that's where a purpose-built deployed agent saves weeks of glue code and ongoing maintenance.