Clawd BotandClaude Opus 4.6
ca9b510922
- Reset master to upstream/main (16,697 commits) - Overlay 2,271 local-only files (skills, tools, workspace, configs, apps) - Restore IDENTITY.md and USER.md templates - Build verified, gateway running, Discord working Co-Authored-By: Claude Opus 4.6 <[email protected]>
3.7 KiB
3.7 KiB
Voice Surface Quick Reference
Quick Start
cd tools/voice-surface
pnpm install # First time only
./run.sh # Start server
Access: http://127.0.0.1:3030/?token=<shown-on-start>
How to Use
- Open the URL in your browser
- Grant microphone permissions
- Hold the button (or spacebar) to talk
- Release to process
Architecture Flow
Browser (MediaRecorder)
↓ WebSocket (chunked base64)
Server buffers audio
↓ Save to /tmp/voice-surface/input-*.webm
ffmpeg converts to WAV (if available)
↓ /tmp/voice-surface/converted-*.wav
whisper-cli OR openclaw media audio
↓ transcript text
pnpm openclaw agent --local --json --session-id <id> --message <text>
↓ agent reply text
edge-tts OR openclaw tts
↓ audio/mpeg
Server → Browser (base64)
↓
Browser plays audio
Components
Required
- Node.js 22+
- pnpm
- OpenClaw (agent + optional media/tts tools)
Optional (auto-detected, fallback to openclaw)
- ffmpeg - audio format conversion
- whisper-cli - local STT
- edge-tts - fast TTS
Configuration
Environment variables:
BIND_HOST=192.168.1.220 # Default: 127.0.0.1
PORT=3030 # Default: 3030
AUTH_TOKEN=secret123 # Auto-generated if not set
Security
- Token required for all connections
- Default bind:
127.0.0.1(localhost only) - LAN access: set
BIND_HOST=192.168.1.220 - Always use token - it's in the URL and WebSocket upgrade
Systemd Service
# Install
./install-service.sh
# Manage
sudo systemctl start voice-surface@alex
sudo systemctl stop voice-surface@alex
sudo systemctl enable voice-surface@alex # Auto-start on boot
sudo systemctl status voice-surface@alex
sudo journalctl -u voice-surface@alex -f # Logs
Troubleshooting
| Issue | Solution |
|---|---|
| "Missing token" | Check URL includes ?token=... |
| "Microphone denied" | Grant permissions in browser |
| "No speech detected" | Speak clearly, hold button longer |
| WebSocket disconnect | Check firewall, verify server running |
| "ffmpeg not available" | sudo apt install ffmpeg (works without) |
| Agent fails | Ensure OpenClaw model config set up |
Testing
./test.sh # Run component smoke tests
Files
tools/voice-surface/
├── src/
│ └── server.ts # WebSocket server + pipeline
├── public/
│ └── index.html # Push-to-talk UI
├── package.json
├── README.md # Full documentation
├── QUICKREF.md # This file
├── run.sh # Start script
├── test.sh # Smoke tests
├── install-service.sh # Systemd installer
└── [email protected] # Systemd template
WebSocket API
Client → Server:
{"type": "audio-chunk", "data": "<base64>"}
{"type": "audio-end"}
Server → Client:
{"type": "status", "message": "Processing..."}
{"type": "transcript", "text": "user said this"}
{"type": "reply", "text": "agent replied this"}
{"type": "audio", "data": "<base64>", "mimeType": "audio/mpeg"}
{"type": "error", "message": "what went wrong"}
Performance Notes
- Latency: ~2-5s total (depends on STT/LLM/TTS speed)
- Chunking: Full utterance (not streaming)
- Temp files:
/tmp/voice-surface/(auto-cleanup) - Session: Persistent per-server instance
Keyboard Shortcuts
- Spacebar: Push-to-talk (hold while talking)
- Works in addition to mouse/touch on button