# Voice Surface

Local voice interface for OpenClaw - push-to-talk web UI with full speech-to-text and text-to-speech pipeline.

## Features

- 🎤 **Push-to-talk** browser interface (mouse, touch, or spacebar)
- 🔊 **Full audio pipeline**: STT → Agent → TTS
- 🔒 **Owner-only**: token-based authentication
- 🚀 **Local-first**: runs entirely on your machine
- 🎯 **Simple & robust**: minimal dependencies, straightforward flow

## Architecture

1. **Frontend**: Static HTML page with MediaRecorder + WebSocket
2. **Backend**: Node.js (tsx) WebSocket server
3. **Pipeline**:
   - Capture audio from browser
   - Convert to WAV (ffmpeg, if available)
   - Transcribe with whisper-cli or `openclaw media audio`
   - Process with `openclaw agent --local --json`
   - Synthesize reply with edge-tts or `openclaw tts`
   - Stream audio back to browser

## Prerequisites

### Required

- Node.js 22+
- pnpm (for running OpenClaw commands)

### Optional (improves performance)

- **whisper.cpp/whisper-cli** - for local STT (fallback: `openclaw media audio`)
- **ffmpeg** - for audio conversion (fallback: use original format)
- **edge-tts** - for TTS (fallback: `openclaw tts`)

### Installing Optional Dependencies

**whisper.cpp** (recommended for best STT performance):

```bash
# Clone and build whisper.cpp
git clone https://github.com/ggerganov/whisper.cpp.git
cd whisper.cpp
make

# Download a model (base recommended for speed/quality balance)
bash ./models/download-ggml-model.sh base

# Add to PATH or create symlink
sudo ln -s $(pwd)/main /usr/local/bin/whisper-cli
```

**ffmpeg** (for audio conversion):

```bash
# Ubuntu/Debian
sudo apt install ffmpeg

# Arch/CachyOS
sudo pacman -S ffmpeg
```

**edge-tts** (for TTS):

```bash
pip install edge-tts
# or
pipx install edge-tts
```

## Installation

```bash
cd tools/voice-surface
pnpm install
```

## Configuration

Environment variables (optional):

```bash
# Bind address (default: 127.0.0.1)
export BIND_HOST=192.168.1.220

# Port (default: 3030)
export PORT=3030

# Auth token (auto-generated if not set)
export AUTH_TOKEN=your-secret-token
```

## Usage

### Option 1: Run directly

```bash
cd tools/voice-surface
pnpm start
```

Or use the convenience script:

```bash
./tools/voice-surface/run.sh
```

### Option 2: Install as systemd service (recommended for persistent use)

```bash
cd tools/voice-surface
./install-service.sh
```

This will:

- Install the service for your user
- Prompt for an auth token (or auto-generate one)
- Set up auto-restart on failure

After installation:

```bash
# Start the service
sudo systemctl start voice-surface@$(whoami)

# Enable auto-start on boot
sudo systemctl enable voice-surface@$(whoami)

# Check status
sudo systemctl status voice-surface@$(whoami)

# View logs
sudo journalctl -u voice-surface@$(whoami) -f
```

### Accessing the interface

The server will display the access URL with authentication token:

```
🚀 Voice Surface running at:
   http://127.0.0.1:3030/?token=abc123xyz789
```

### Access the interface

1. Open the URL in your browser
2. Grant microphone permissions when prompted
3. **Hold** the button (or spacebar) to talk
4. **Release** to send and process

### Security Notes

- Server binds to `127.0.0.1` by default (localhost only)
- Set `BIND_HOST=192.168.1.220` to allow LAN access
- **Always use the token** - it's required for all connections
- Token is displayed on startup if not explicitly set

## Troubleshooting

### "Microphone access denied"

- Check browser permissions
- HTTPS not required for localhost

### "No speech detected"

- Speak clearly and hold button while talking
- Check microphone levels in system settings
- Ensure you're holding the button long enough

### "whisper-cli failed"

- Check `whisper-cli` is in PATH: `which whisper-cli`
- Server will auto-fallback to `openclaw media audio`

### "ffmpeg not available"

- Install ffmpeg (see prerequisites)
- Server will work without it (may be less reliable)

### WebSocket connection issues

- Verify token in URL
- Check firewall settings if binding to LAN address
- Check server logs for errors

## Development

The server uses:

- **tsx** for TypeScript execution
- **ws** for WebSocket handling
- **fluent-ffmpeg** for audio conversion (optional)

File structure:

```
tools/voice-surface/
├── src/
│   └── server.ts          # WebSocket server + audio pipeline
├── public/
│   └── index.html         # Push-to-talk UI
├── package.json
├── README.md
└── run.sh                 # Convenience launcher
```

## API Protocol

WebSocket messages (JSON):

**Client → Server:**

```json
{ "type": "audio-chunk", "data": "<base64>" }
{ "type": "audio-end" }
```

**Server → Client:**

```json
{ "type": "status", "message": "Processing..." }
{ "type": "transcript", "text": "..." }
{ "type": "reply", "text": "..." }
{ "type": "audio", "data": "<base64>", "mimeType": "audio/mpeg" }
{ "type": "error", "message": "..." }
```

## License

Same as OpenClaw (see repository root).