openclaw/docs/LLM_CHECKER_GUIDE.md

llm-checker - Smart Model Selection for Jarvis

Why We Needed This

Problem: Vi valgte models manuelt baseret på gætterier om hardware compatibility.

Solution: llm-checker giver data-drevet model selection baseret på faktisk hardware.

Installation

sudo npm install -g llm-checker

Key Commands

1. Hardware Detection

llm-checker hw-detect

Output for vores system:

• CPU: Intel i5-12500H (16 cores, AVX2)
• RAM: 15GB
• GPU: None (CPU-only)
• Tier: MEDIUM LOW
• Max model size: 9GB

2. Model Recommendations

# Full analysis (slow - scrapes all Ollama models)
llm-checker recommend

# Quick mode with cache
llm-checker recommend --use-cache

# Smart recommendations with scoring
llm-checker smart-recommend

3. Installed Models Ranking

llm-checker installed

Ranks your current Ollama models by:

• Hardware compatibility
• Performance (tokens/sec estimate)
• Memory requirements
• Use-case fit

4. Search Models

llm-checker search <query>

# Examples
llm-checker search "code"
llm-checker search "fast"
llm-checker search "small"

5. AI-Powered Selection

llm-checker ai-check
llm-checker ai-run

Uses meta-analysis to pick the best model for your task.

Integration with Jarvis

Current State

Ollama models installed:

• qwen2.5:0.5b (397 MB) - Very small, fast, good for simple tasks

Cloud models (primary):

• gemini-3-flash-preview (Google)
• gemini-3-pro-preview (Google)
• qwen3-235b (NVIDIA)
• deepseek-v3.2 (NVIDIA)
• claude-sonnet-4-5 (Anthropic)
• gpt-5-mini (GitHub Copilot)

Recommended Strategy

Use llm-checker to:

1. Find optimal local models for CPU-only hardware

   • Target: 0.5B - 3B parameter models
   • Max size: ~9GB
   • Focus: Fast inference on AVX2

2. Benchmark installed models

   llm-checker installed --benchmark
   

3. Cost optimization

   • Simple queries → Local (qwen2.5:0.5b)
   • Complex queries → Cloud (gemini/nvidia/copilot)
   • Code generation → Local (codellama:7b-code)

4. Monthly check

   • New models released constantly
   • Run llm-checker smart-recommend monthly
   • Update Ollama models based on recommendations

Jarvis Automation

Added to tooling:

• /home/alex/clawd/tools/jarvis-model-optimizer.sh
• Runs hardware detection
• Logs installed models
• Suggests optimization opportunities

Future enhancement:

• Weekly cron job to check for better models
• Auto-install high-scoring models under 3GB
• Benchmark new models before adding to delegation

Model Selection Matrix

Based on llm-checker tier: MEDIUM LOW

Use Case	Recommended Model	Size	Speed
Simple chat	qwen2.5:0.5b	397MB	⚡⚡⚡ Fast
Code completion	deepseek-coder:1.3b	~1GB	⚡⚡ Fast
Summarization	phi3:3.8b	~3GB	⚡ Medium
Complex reasoning	Use cloud (Gemini Pro)	-	☁️ Cloud
Heavy code	Use cloud (Qwen3 235B)	-	☁️ Cloud

Common Issues

Ollama crashes/hangs

• Cause: Model too large for RAM
• Fix: Use llm-checker to find smaller models
• Setting: OLLAMA_NUM_PARALLEL=1 OLLAMA_MAX_QUEUE=1

Slow inference

• Check: CPU-only mode (no GPU)
• Fix: Use quantized models (Q4_0, Q4_K_M)
• Verify: llm-checker installed --benchmark

Out of memory

• Cause: Model size > available RAM
• Fix: Stick to < 3B param models
• Tool: llm-checker hw-detect shows max size

Next Steps

1. Run full analysis:

   llm-checker recommend --use-cache > /tmp/llm-recommendations.txt
   

2. Install top 3 recommended models:

   # Based on recommendations
   ollama pull <model-name>
   

3. Benchmark them:

   llm-checker installed --benchmark
   

4. Update delegation rules:

   • Add best local model to jarvis-delegate.sh
   • Route simple tasks to local
   • Keep complex tasks on cloud

5. Monitor costs:

   • Track API spend vs local usage
   • Optimize delegation based on perf/cost

Why We Didn't Use It Before

1. Not proactive - Didn't know it existed
2. Cloud-first mindset - Focused on API providers
3. Ollama instability - Avoided local inference
4. No automation - Manual model selection

Jarvis fix: Proactively suggest tools like this based on infrastructure needs.

Resources

• GitHub: https://github.com/Pavelevich/llm-checker
• npm: https://www.npmjs.com/package/llm-checker
• MCP support: Built-in Model Context Protocol integration

---

"Now we have data-driven model selection, sir." 🎯