# llm-checker - Smart Model Selection for Jarvis

## Why We Needed This

**Problem:** Vi valgte models manuelt baseret på gætterier om hardware compatibility.

**Solution:** `llm-checker` giver data-drevet model selection baseret på faktisk hardware.

## Installation

```bash
sudo npm install -g llm-checker
```

## Key Commands

### 1. Hardware Detection

```bash
llm-checker hw-detect
```

**Output for vores system:**

- CPU: Intel i5-12500H (16 cores, AVX2)
- RAM: 15GB
- GPU: None (CPU-only)
- Tier: MEDIUM LOW
- Max model size: 9GB

### 2. Model Recommendations

```bash
# Full analysis (slow - scrapes all Ollama models)
llm-checker recommend

# Quick mode with cache
llm-checker recommend --use-cache

# Smart recommendations with scoring
llm-checker smart-recommend
```

### 3. Installed Models Ranking

```bash
llm-checker installed
```

Ranks your current Ollama models by:

- Hardware compatibility
- Performance (tokens/sec estimate)
- Memory requirements
- Use-case fit

### 4. Search Models

```bash
llm-checker search <query>

# Examples
llm-checker search "code"
llm-checker search "fast"
llm-checker search "small"
```

### 5. AI-Powered Selection

```bash
llm-checker ai-check
llm-checker ai-run
```

Uses meta-analysis to pick the best model for your task.

## Integration with Jarvis

### Current State

**Ollama models installed:**

- qwen2.5:0.5b (397 MB) - Very small, fast, good for simple tasks

**Cloud models (primary):**

- gemini-3-flash-preview (Google)
- gemini-3-pro-preview (Google)
- qwen3-235b (NVIDIA)
- deepseek-v3.2 (NVIDIA)
- claude-sonnet-4-5 (Anthropic)
- gpt-5-mini (GitHub Copilot)

### Recommended Strategy

**Use llm-checker to:**

1. **Find optimal local models** for CPU-only hardware
   - Target: 0.5B - 3B parameter models
   - Max size: ~9GB
   - Focus: Fast inference on AVX2

2. **Benchmark installed models**

   ```bash
   llm-checker installed --benchmark
   ```

3. **Cost optimization**
   - Simple queries → Local (qwen2.5:0.5b)
   - Complex queries → Cloud (gemini/nvidia/copilot)
   - Code generation → Local (codellama:7b-code)

4. **Monthly check**
   - New models released constantly
   - Run `llm-checker smart-recommend` monthly
   - Update Ollama models based on recommendations

### Jarvis Automation

Added to tooling:

- `/home/alex/clawd/tools/jarvis-model-optimizer.sh`
- Runs hardware detection
- Logs installed models
- Suggests optimization opportunities

**Future enhancement:**

- Weekly cron job to check for better models
- Auto-install high-scoring models under 3GB
- Benchmark new models before adding to delegation

## Model Selection Matrix

Based on `llm-checker` tier: **MEDIUM LOW**

| Use Case          | Recommended Model      | Size  | Speed       |
| ----------------- | ---------------------- | ----- | ----------- |
| Simple chat       | qwen2.5:0.5b           | 397MB | ⚡⚡⚡ Fast |
| Code completion   | deepseek-coder:1.3b    | ~1GB  | ⚡⚡ Fast   |
| Summarization     | phi3:3.8b              | ~3GB  | ⚡ Medium   |
| Complex reasoning | Use cloud (Gemini Pro) | -     | ☁️ Cloud    |
| Heavy code        | Use cloud (Qwen3 235B) | -     | ☁️ Cloud    |

## Common Issues

### Ollama crashes/hangs

- **Cause:** Model too large for RAM
- **Fix:** Use llm-checker to find smaller models
- **Setting:** `OLLAMA_NUM_PARALLEL=1 OLLAMA_MAX_QUEUE=1`

### Slow inference

- **Check:** CPU-only mode (no GPU)
- **Fix:** Use quantized models (Q4_0, Q4_K_M)
- **Verify:** `llm-checker installed --benchmark`

### Out of memory

- **Cause:** Model size > available RAM
- **Fix:** Stick to < 3B param models
- **Tool:** `llm-checker hw-detect` shows max size

## Next Steps

1. **Run full analysis:**

   ```bash
   llm-checker recommend --use-cache > /tmp/llm-recommendations.txt
   ```

2. **Install top 3 recommended models:**

   ```bash
   # Based on recommendations
   ollama pull <model-name>
   ```

3. **Benchmark them:**

   ```bash
   llm-checker installed --benchmark
   ```

4. **Update delegation rules:**
   - Add best local model to `jarvis-delegate.sh`
   - Route simple tasks to local
   - Keep complex tasks on cloud

5. **Monitor costs:**
   - Track API spend vs local usage
   - Optimize delegation based on perf/cost

## Why We Didn't Use It Before

1. **Not proactive** - Didn't know it existed
2. **Cloud-first mindset** - Focused on API providers
3. **Ollama instability** - Avoided local inference
4. **No automation** - Manual model selection

**Jarvis fix:** Proactively suggest tools like this based on infrastructure needs.

## Resources

- GitHub: https://github.com/Pavelevich/llm-checker
- npm: https://www.npmjs.com/package/llm-checker
- MCP support: Built-in Model Context Protocol integration

---

**"Now we have data-driven model selection, sir." 🎯**