Add local copy of OpenClaw docs for offline search
This commit is contained in:
@@ -0,0 +1,40 @@
|
||||
---
|
||||
summary: "RPC protocol notes for onboarding wizard and config schema"
|
||||
read_when: "Changing onboarding wizard steps or config schema endpoints"
|
||||
title: "Onboarding and Config Protocol"
|
||||
---
|
||||
|
||||
# Onboarding + Config Protocol
|
||||
|
||||
Purpose: shared onboarding + config surfaces across CLI, macOS app, and Web UI.
|
||||
|
||||
## Components
|
||||
|
||||
- Wizard engine (shared session + prompts + onboarding state).
|
||||
- CLI onboarding uses the same wizard flow as the UI clients.
|
||||
- Gateway RPC exposes wizard + config schema endpoints.
|
||||
- macOS onboarding uses the wizard step model.
|
||||
- Web UI renders config forms from JSON Schema + UI hints.
|
||||
|
||||
## Gateway RPC
|
||||
|
||||
- `wizard.start` params: `{ mode?: "local"|"remote", workspace?: string }`
|
||||
- `wizard.next` params: `{ sessionId, answer?: { stepId, value? } }`
|
||||
- `wizard.cancel` params: `{ sessionId }`
|
||||
- `wizard.status` params: `{ sessionId }`
|
||||
- `config.schema` params: `{}`
|
||||
|
||||
Responses (shape)
|
||||
|
||||
- Wizard: `{ sessionId, done, step?, status?, error? }`
|
||||
- Config schema: `{ schema, uiHints, version, generatedAt }`
|
||||
|
||||
## UI Hints
|
||||
|
||||
- `uiHints` keyed by path; optional metadata (label/help/group/order/advanced/sensitive/placeholder).
|
||||
- Sensitive fields render as password inputs; no redaction layer.
|
||||
- Unsupported schema nodes fall back to the raw JSON editor.
|
||||
|
||||
## Notes
|
||||
|
||||
- This doc is the single place to track protocol refactors for onboarding/config.
|
||||
@@ -0,0 +1,229 @@
|
||||
---
|
||||
summary: "Plan: isolate browser act:evaluate from Playwright queue using CDP, with end-to-end deadlines and safer ref resolution"
|
||||
owner: "openclaw"
|
||||
status: "draft"
|
||||
last_updated: "2026-02-10"
|
||||
title: "Browser Evaluate CDP Refactor"
|
||||
---
|
||||
|
||||
# Browser Evaluate CDP Refactor Plan
|
||||
|
||||
## Context
|
||||
|
||||
`act:evaluate` executes user provided JavaScript in the page. Today it runs via Playwright
|
||||
(`page.evaluate` or `locator.evaluate`). Playwright serializes CDP commands per page, so a
|
||||
stuck or long running evaluate can block the page command queue and make every later action
|
||||
on that tab look "stuck".
|
||||
|
||||
PR #13498 adds a pragmatic safety net (bounded evaluate, abort propagation, and best-effort
|
||||
recovery). This document describes a larger refactor that makes `act:evaluate` inherently
|
||||
isolated from Playwright so a stuck evaluate cannot wedge normal Playwright operations.
|
||||
|
||||
## Goals
|
||||
|
||||
- `act:evaluate` cannot permanently block later browser actions on the same tab.
|
||||
- Timeouts are single source of truth end to end so a caller can rely on a budget.
|
||||
- Abort and timeout are treated the same way across HTTP and in-process dispatch.
|
||||
- Element targeting for evaluate is supported without switching everything off Playwright.
|
||||
- Maintain backward compatibility for existing callers and payloads.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Replace all browser actions (click, type, wait, etc.) with CDP implementations.
|
||||
- Remove the existing safety net introduced in PR #13498 (it remains a useful fallback).
|
||||
- Introduce new unsafe capabilities beyond the existing `browser.evaluateEnabled` gate.
|
||||
- Add process isolation (worker process/thread) for evaluate. If we still see hard to recover
|
||||
stuck states after this refactor, that is a follow-up idea.
|
||||
|
||||
## Current Architecture (Why It Gets Stuck)
|
||||
|
||||
At a high level:
|
||||
|
||||
- Callers send `act:evaluate` to the browser control service.
|
||||
- The route handler calls into Playwright to execute the JavaScript.
|
||||
- Playwright serializes page commands, so an evaluate that never finishes blocks the queue.
|
||||
- A stuck queue means later click/type/wait operations on the tab can appear to hang.
|
||||
|
||||
## Proposed Architecture
|
||||
|
||||
### 1. Deadline Propagation
|
||||
|
||||
Introduce a single budget concept and derive everything from it:
|
||||
|
||||
- Caller sets `timeoutMs` (or a deadline in the future).
|
||||
- The outer request timeout, route handler logic, and the execution budget inside the page
|
||||
all use the same budget, with small headroom where needed for serialization overhead.
|
||||
- Abort is propagated as an `AbortSignal` everywhere so cancellation is consistent.
|
||||
|
||||
Implementation direction:
|
||||
|
||||
- Add a small helper (for example `createBudget({ timeoutMs, signal })`) that returns:
|
||||
- `signal`: the linked AbortSignal
|
||||
- `deadlineAtMs`: absolute deadline
|
||||
- `remainingMs()`: remaining budget for child operations
|
||||
- Use this helper in:
|
||||
- `src/browser/client-fetch.ts` (HTTP and in-process dispatch)
|
||||
- `src/node-host/runner.ts` (proxy path)
|
||||
- browser action implementations (Playwright and CDP)
|
||||
|
||||
### 2. Separate Evaluate Engine (CDP Path)
|
||||
|
||||
Add a CDP based evaluate implementation that does not share Playwright's per page command
|
||||
queue. The key property is that the evaluate transport is a separate WebSocket connection
|
||||
and a separate CDP session attached to the target.
|
||||
|
||||
Implementation direction:
|
||||
|
||||
- New module, for example `src/browser/cdp-evaluate.ts`, that:
|
||||
- Connects to the configured CDP endpoint (browser level socket).
|
||||
- Uses `Target.attachToTarget({ targetId, flatten: true })` to get a `sessionId`.
|
||||
- Runs either:
|
||||
- `Runtime.evaluate` for page level evaluate, or
|
||||
- `DOM.resolveNode` plus `Runtime.callFunctionOn` for element evaluate.
|
||||
- On timeout or abort:
|
||||
- Sends `Runtime.terminateExecution` best-effort for the session.
|
||||
- Closes the WebSocket and returns a clear error.
|
||||
|
||||
Notes:
|
||||
|
||||
- This still executes JavaScript in the page, so termination can have side effects. The win
|
||||
is that it does not wedge the Playwright queue, and it is cancelable at the transport
|
||||
layer by killing the CDP session.
|
||||
|
||||
### 3. Ref Story (Element Targeting Without A Full Rewrite)
|
||||
|
||||
The hard part is element targeting. CDP needs a DOM handle or `backendDOMNodeId`, while
|
||||
today most browser actions use Playwright locators based on refs from snapshots.
|
||||
|
||||
Recommended approach: keep existing refs, but attach an optional CDP resolvable id.
|
||||
|
||||
#### 3.1 Extend Stored Ref Info
|
||||
|
||||
Extend the stored role ref metadata to optionally include a CDP id:
|
||||
|
||||
- Today: `{ role, name, nth }`
|
||||
- Proposed: `{ role, name, nth, backendDOMNodeId?: number }`
|
||||
|
||||
This keeps all existing Playwright based actions working and allows CDP evaluate to accept
|
||||
the same `ref` value when the `backendDOMNodeId` is available.
|
||||
|
||||
#### 3.2 Populate backendDOMNodeId At Snapshot Time
|
||||
|
||||
When producing a role snapshot:
|
||||
|
||||
1. Generate the existing role ref map as today (role, name, nth).
|
||||
2. Fetch the AX tree via CDP (`Accessibility.getFullAXTree`) and compute a parallel map of
|
||||
`(role, name, nth) -> backendDOMNodeId` using the same duplicate handling rules.
|
||||
3. Merge the id back into the stored ref info for the current tab.
|
||||
|
||||
If mapping fails for a ref, leave `backendDOMNodeId` undefined. This makes the feature
|
||||
best-effort and safe to roll out.
|
||||
|
||||
#### 3.3 Evaluate Behavior With Ref
|
||||
|
||||
In `act:evaluate`:
|
||||
|
||||
- If `ref` is present and has `backendDOMNodeId`, run element evaluate via CDP.
|
||||
- If `ref` is present but has no `backendDOMNodeId`, fall back to the Playwright path (with
|
||||
the safety net).
|
||||
|
||||
Optional escape hatch:
|
||||
|
||||
- Extend the request shape to accept `backendDOMNodeId` directly for advanced callers (and
|
||||
for debugging), while keeping `ref` as the primary interface.
|
||||
|
||||
### 4. Keep A Last Resort Recovery Path
|
||||
|
||||
Even with CDP evaluate, there are other ways to wedge a tab or a connection. Keep the
|
||||
existing recovery mechanisms (terminate execution + disconnect Playwright) as a last resort
|
||||
for:
|
||||
|
||||
- legacy callers
|
||||
- environments where CDP attach is blocked
|
||||
- unexpected Playwright edge cases
|
||||
|
||||
## Implementation Plan (Single Iteration)
|
||||
|
||||
### Deliverables
|
||||
|
||||
- A CDP based evaluate engine that runs outside the Playwright per-page command queue.
|
||||
- A single end-to-end timeout/abort budget used consistently by callers and handlers.
|
||||
- Ref metadata that can optionally carry `backendDOMNodeId` for element evaluate.
|
||||
- `act:evaluate` prefers the CDP engine when possible and falls back to Playwright when not.
|
||||
- Tests that prove a stuck evaluate does not wedge later actions.
|
||||
- Logs/metrics that make failures and fallbacks visible.
|
||||
|
||||
### Implementation Checklist
|
||||
|
||||
1. Add a shared "budget" helper to link `timeoutMs` + upstream `AbortSignal` into:
|
||||
- a single `AbortSignal`
|
||||
- an absolute deadline
|
||||
- a `remainingMs()` helper for downstream operations
|
||||
2. Update all caller paths to use that helper so `timeoutMs` means the same thing everywhere:
|
||||
- `src/browser/client-fetch.ts` (HTTP and in-process dispatch)
|
||||
- `src/node-host/runner.ts` (node proxy path)
|
||||
- CLI wrappers that call `/act` (add `--timeout-ms` to `browser evaluate`)
|
||||
3. Implement `src/browser/cdp-evaluate.ts`:
|
||||
- connect to the browser-level CDP socket
|
||||
- `Target.attachToTarget` to get a `sessionId`
|
||||
- run `Runtime.evaluate` for page evaluate
|
||||
- run `DOM.resolveNode` + `Runtime.callFunctionOn` for element evaluate
|
||||
- on timeout/abort: best-effort `Runtime.terminateExecution` then close the socket
|
||||
4. Extend stored role ref metadata to optionally include `backendDOMNodeId`:
|
||||
- keep existing `{ role, name, nth }` behavior for Playwright actions
|
||||
- add `backendDOMNodeId?: number` for CDP element targeting
|
||||
5. Populate `backendDOMNodeId` during snapshot creation (best-effort):
|
||||
- fetch AX tree via CDP (`Accessibility.getFullAXTree`)
|
||||
- compute `(role, name, nth) -> backendDOMNodeId` and merge into the stored ref map
|
||||
- if mapping is ambiguous or missing, leave the id undefined
|
||||
6. Update `act:evaluate` routing:
|
||||
- if no `ref`: always use CDP evaluate
|
||||
- if `ref` resolves to a `backendDOMNodeId`: use CDP element evaluate
|
||||
- otherwise: fall back to Playwright evaluate (still bounded and abortable)
|
||||
7. Keep the existing "last resort" recovery path as a fallback, not the default path.
|
||||
8. Add tests:
|
||||
- stuck evaluate times out within budget and the next click/type succeeds
|
||||
- abort cancels evaluate (client disconnect or timeout) and unblocks subsequent actions
|
||||
- mapping failures cleanly fall back to Playwright
|
||||
9. Add observability:
|
||||
- evaluate duration and timeout counters
|
||||
- terminateExecution usage
|
||||
- fallback rate (CDP -> Playwright) and reasons
|
||||
|
||||
### Acceptance Criteria
|
||||
|
||||
- A deliberately hung `act:evaluate` returns within the caller budget and does not wedge the
|
||||
tab for later actions.
|
||||
- `timeoutMs` behaves consistently across CLI, agent tool, node proxy, and in-process calls.
|
||||
- If `ref` can be mapped to `backendDOMNodeId`, element evaluate uses CDP; otherwise the
|
||||
fallback path is still bounded and recoverable.
|
||||
|
||||
## Testing Plan
|
||||
|
||||
- Unit tests:
|
||||
- `(role, name, nth)` matching logic between role refs and AX tree nodes.
|
||||
- Budget helper behavior (headroom, remaining time math).
|
||||
- Integration tests:
|
||||
- CDP evaluate timeout returns within budget and does not block the next action.
|
||||
- Abort cancels evaluate and triggers termination best-effort.
|
||||
- Contract tests:
|
||||
- Ensure `BrowserActRequest` and `BrowserActResponse` remain compatible.
|
||||
|
||||
## Risks And Mitigations
|
||||
|
||||
- Mapping is imperfect:
|
||||
- Mitigation: best-effort mapping, fallback to Playwright evaluate, and add debug tooling.
|
||||
- `Runtime.terminateExecution` has side effects:
|
||||
- Mitigation: only use on timeout/abort and document the behavior in errors.
|
||||
- Extra overhead:
|
||||
- Mitigation: only fetch AX tree when snapshots are requested, cache per target, and keep
|
||||
CDP session short lived.
|
||||
- Extension relay limitations:
|
||||
- Mitigation: use browser level attach APIs when per page sockets are not available, and
|
||||
keep the current Playwright path as fallback.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Should the new engine be configurable as `playwright`, `cdp`, or `auto`?
|
||||
- Do we want to expose a new "nodeRef" format for advanced users, or keep `ref` only?
|
||||
- How should frame snapshots and selector scoped snapshots participate in AX mapping?
|
||||
@@ -0,0 +1,63 @@
|
||||
---
|
||||
summary: "Harden cron.add input handling, align schemas, and improve cron UI/agent tooling"
|
||||
owner: "openclaw"
|
||||
status: "complete"
|
||||
last_updated: "2026-01-05"
|
||||
title: "Cron Add Hardening"
|
||||
---
|
||||
|
||||
# Cron Add Hardening & Schema Alignment
|
||||
|
||||
## Context
|
||||
|
||||
Recent gateway logs show repeated `cron.add` failures with invalid parameters (missing `sessionTarget`, `wakeMode`, `payload`, and malformed `schedule`). This indicates that at least one client (likely the agent tool call path) is sending wrapped or partially specified job payloads. Separately, there is drift between cron provider enums in TypeScript, gateway schema, CLI flags, and UI form types, plus a UI mismatch for `cron.status` (expects `jobCount` while gateway returns `jobs`).
|
||||
|
||||
## Goals
|
||||
|
||||
- Stop `cron.add` INVALID_REQUEST spam by normalizing common wrapper payloads and inferring missing `kind` fields.
|
||||
- Align cron provider lists across gateway schema, cron types, CLI docs, and UI forms.
|
||||
- Make agent cron tool schema explicit so the LLM produces correct job payloads.
|
||||
- Fix the Control UI cron status job count display.
|
||||
- Add tests to cover normalization and tool behavior.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Change cron scheduling semantics or job execution behavior.
|
||||
- Add new schedule kinds or cron expression parsing.
|
||||
- Overhaul the UI/UX for cron beyond the necessary field fixes.
|
||||
|
||||
## Findings (current gaps)
|
||||
|
||||
- `CronPayloadSchema` in gateway excludes `signal` + `imessage`, while TS types include them.
|
||||
- Control UI CronStatus expects `jobCount`, but gateway returns `jobs`.
|
||||
- Agent cron tool schema allows arbitrary `job` objects, enabling malformed inputs.
|
||||
- Gateway strictly validates `cron.add` with no normalization, so wrapped payloads fail.
|
||||
|
||||
## What changed
|
||||
|
||||
- `cron.add` and `cron.update` now normalize common wrapper shapes and infer missing `kind` fields.
|
||||
- Agent cron tool schema matches the gateway schema, which reduces invalid payloads.
|
||||
- Provider enums are aligned across gateway, CLI, UI, and macOS picker.
|
||||
- Control UI uses the gateway’s `jobs` count field for status.
|
||||
|
||||
## Current behavior
|
||||
|
||||
- **Normalization:** wrapped `data`/`job` payloads are unwrapped; `schedule.kind` and `payload.kind` are inferred when safe.
|
||||
- **Defaults:** safe defaults are applied for `wakeMode` and `sessionTarget` when missing.
|
||||
- **Providers:** Discord/Slack/Signal/iMessage are now consistently surfaced across CLI/UI.
|
||||
|
||||
See [Cron jobs](/automation/cron-jobs) for the normalized shape and examples.
|
||||
|
||||
## Verification
|
||||
|
||||
- Watch gateway logs for reduced `cron.add` INVALID_REQUEST errors.
|
||||
- Confirm Control UI cron status shows job count after refresh.
|
||||
|
||||
## Optional Follow-ups
|
||||
|
||||
- Manual Control UI smoke: add a cron job per provider + verify status job count.
|
||||
|
||||
## Open Questions
|
||||
|
||||
- Should `cron.add` accept explicit `state` from clients (currently disallowed by schema)?
|
||||
- Should we allow `webchat` as an explicit delivery provider (currently filtered in delivery resolution)?
|
||||
@@ -0,0 +1,40 @@
|
||||
---
|
||||
summary: "Telegram allowlist hardening: prefix + whitespace normalization"
|
||||
read_when:
|
||||
- Reviewing historical Telegram allowlist changes
|
||||
title: "Telegram Allowlist Hardening"
|
||||
---
|
||||
|
||||
# Telegram Allowlist Hardening
|
||||
|
||||
**Date**: 2026-01-05
|
||||
**Status**: Complete
|
||||
**PR**: #216
|
||||
|
||||
## Summary
|
||||
|
||||
Telegram allowlists now accept `telegram:` and `tg:` prefixes case-insensitively, and tolerate
|
||||
accidental whitespace. This aligns inbound allowlist checks with outbound send normalization.
|
||||
|
||||
## What changed
|
||||
|
||||
- Prefixes `telegram:` and `tg:` are treated the same (case-insensitive).
|
||||
- Allowlist entries are trimmed; empty entries are ignored.
|
||||
|
||||
## Examples
|
||||
|
||||
All of these are accepted for the same ID:
|
||||
|
||||
- `telegram:123456`
|
||||
- `TG:123456`
|
||||
- `tg:123456`
|
||||
|
||||
## Why it matters
|
||||
|
||||
Copy/paste from logs or chat IDs often includes prefixes and whitespace. Normalizing avoids
|
||||
false negatives when deciding whether to respond in DMs or groups.
|
||||
|
||||
## Related docs
|
||||
|
||||
- [Group Chats](/channels/groups)
|
||||
- [Telegram Provider](/channels/telegram)
|
||||
@@ -0,0 +1,123 @@
|
||||
---
|
||||
summary: "Plan: Add OpenResponses /v1/responses endpoint and deprecate chat completions cleanly"
|
||||
owner: "openclaw"
|
||||
status: "draft"
|
||||
last_updated: "2026-01-19"
|
||||
title: "OpenResponses Gateway Plan"
|
||||
---
|
||||
|
||||
# OpenResponses Gateway Integration Plan
|
||||
|
||||
## Context
|
||||
|
||||
OpenClaw Gateway currently exposes a minimal OpenAI-compatible Chat Completions endpoint at
|
||||
`/v1/chat/completions` (see [OpenAI Chat Completions](/gateway/openai-http-api)).
|
||||
|
||||
Open Responses is an open inference standard based on the OpenAI Responses API. It is designed
|
||||
for agentic workflows and uses item-based inputs plus semantic streaming events. The OpenResponses
|
||||
spec defines `/v1/responses`, not `/v1/chat/completions`.
|
||||
|
||||
## Goals
|
||||
|
||||
- Add a `/v1/responses` endpoint that adheres to OpenResponses semantics.
|
||||
- Keep Chat Completions as a compatibility layer that is easy to disable and eventually remove.
|
||||
- Standardize validation and parsing with isolated, reusable schemas.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- Full OpenResponses feature parity in the first pass (images, files, hosted tools).
|
||||
- Replacing internal agent execution logic or tool orchestration.
|
||||
- Changing the existing `/v1/chat/completions` behavior during the first phase.
|
||||
|
||||
## Research Summary
|
||||
|
||||
Sources: OpenResponses OpenAPI, OpenResponses specification site, and the Hugging Face blog post.
|
||||
|
||||
Key points extracted:
|
||||
|
||||
- `POST /v1/responses` accepts `CreateResponseBody` fields like `model`, `input` (string or
|
||||
`ItemParam[]`), `instructions`, `tools`, `tool_choice`, `stream`, `max_output_tokens`, and
|
||||
`max_tool_calls`.
|
||||
- `ItemParam` is a discriminated union of:
|
||||
- `message` items with roles `system`, `developer`, `user`, `assistant`
|
||||
- `function_call` and `function_call_output`
|
||||
- `reasoning`
|
||||
- `item_reference`
|
||||
- Successful responses return a `ResponseResource` with `object: "response"`, `status`, and
|
||||
`output` items.
|
||||
- Streaming uses semantic events such as:
|
||||
- `response.created`, `response.in_progress`, `response.completed`, `response.failed`
|
||||
- `response.output_item.added`, `response.output_item.done`
|
||||
- `response.content_part.added`, `response.content_part.done`
|
||||
- `response.output_text.delta`, `response.output_text.done`
|
||||
- The spec requires:
|
||||
- `Content-Type: text/event-stream`
|
||||
- `event:` must match the JSON `type` field
|
||||
- terminal event must be literal `[DONE]`
|
||||
- Reasoning items may expose `content`, `encrypted_content`, and `summary`.
|
||||
- HF examples include `OpenResponses-Version: latest` in requests (optional header).
|
||||
|
||||
## Proposed Architecture
|
||||
|
||||
- Add `src/gateway/open-responses.schema.ts` containing Zod schemas only (no gateway imports).
|
||||
- Add `src/gateway/openresponses-http.ts` (or `open-responses-http.ts`) for `/v1/responses`.
|
||||
- Keep `src/gateway/openai-http.ts` intact as a legacy compatibility adapter.
|
||||
- Add config `gateway.http.endpoints.responses.enabled` (default `false`).
|
||||
- Keep `gateway.http.endpoints.chatCompletions.enabled` independent; allow both endpoints to be
|
||||
toggled separately.
|
||||
- Emit a startup warning when Chat Completions is enabled to signal legacy status.
|
||||
|
||||
## Deprecation Path for Chat Completions
|
||||
|
||||
- Maintain strict module boundaries: no shared schema types between responses and chat completions.
|
||||
- Make Chat Completions opt-in by config so it can be disabled without code changes.
|
||||
- Update docs to label Chat Completions as legacy once `/v1/responses` is stable.
|
||||
- Optional future step: map Chat Completions requests to the Responses handler for a simpler
|
||||
removal path.
|
||||
|
||||
## Phase 1 Support Subset
|
||||
|
||||
- Accept `input` as string or `ItemParam[]` with message roles and `function_call_output`.
|
||||
- Extract system and developer messages into `extraSystemPrompt`.
|
||||
- Use the most recent `user` or `function_call_output` as the current message for agent runs.
|
||||
- Reject unsupported content parts (image/file) with `invalid_request_error`.
|
||||
- Return a single assistant message with `output_text` content.
|
||||
- Return `usage` with zeroed values until token accounting is wired.
|
||||
|
||||
## Validation Strategy (No SDK)
|
||||
|
||||
- Implement Zod schemas for the supported subset of:
|
||||
- `CreateResponseBody`
|
||||
- `ItemParam` + message content part unions
|
||||
- `ResponseResource`
|
||||
- Streaming event shapes used by the gateway
|
||||
- Keep schemas in a single, isolated module to avoid drift and allow future codegen.
|
||||
|
||||
## Streaming Implementation (Phase 1)
|
||||
|
||||
- SSE lines with both `event:` and `data:`.
|
||||
- Required sequence (minimum viable):
|
||||
- `response.created`
|
||||
- `response.output_item.added`
|
||||
- `response.content_part.added`
|
||||
- `response.output_text.delta` (repeat as needed)
|
||||
- `response.output_text.done`
|
||||
- `response.content_part.done`
|
||||
- `response.completed`
|
||||
- `[DONE]`
|
||||
|
||||
## Tests and Verification Plan
|
||||
|
||||
- Add e2e coverage for `/v1/responses`:
|
||||
- Auth required
|
||||
- Non-stream response shape
|
||||
- Stream event ordering and `[DONE]`
|
||||
- Session routing with headers and `user`
|
||||
- Keep `src/gateway/openai-http.e2e.test.ts` unchanged.
|
||||
- Manual: curl to `/v1/responses` with `stream: true` and verify event ordering and terminal
|
||||
`[DONE]`.
|
||||
|
||||
## Doc Updates (Follow-up)
|
||||
|
||||
- Add a new docs page for `/v1/responses` usage and examples.
|
||||
- Update `/gateway/openai-http-api` with a legacy note and pointer to `/v1/responses`.
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
summary: "Exploration: model config, auth profiles, and fallback behavior"
|
||||
read_when:
|
||||
- Exploring future model selection + auth profile ideas
|
||||
title: "Model Config Exploration"
|
||||
---
|
||||
|
||||
# Model Config (Exploration)
|
||||
|
||||
This document captures **ideas** for future model configuration. It is not a
|
||||
shipping spec. For current behavior, see:
|
||||
|
||||
- [Models](/concepts/models)
|
||||
- [Model failover](/concepts/model-failover)
|
||||
- [OAuth + profiles](/concepts/oauth)
|
||||
|
||||
## Motivation
|
||||
|
||||
Operators want:
|
||||
|
||||
- Multiple auth profiles per provider (personal vs work).
|
||||
- Simple `/model` selection with predictable fallbacks.
|
||||
- Clear separation between text models and image-capable models.
|
||||
|
||||
## Possible direction (high level)
|
||||
|
||||
- Keep model selection simple: `provider/model` with optional aliases.
|
||||
- Let providers have multiple auth profiles, with an explicit order.
|
||||
- Use a global fallback list so all sessions fail over consistently.
|
||||
- Only override image routing when explicitly configured.
|
||||
|
||||
## Open questions
|
||||
|
||||
- Should profile rotation be per-provider or per-model?
|
||||
- How should the UI surface profile selection for a session?
|
||||
- What is the safest migration path from legacy config keys?
|
||||
@@ -0,0 +1,228 @@
|
||||
---
|
||||
summary: "Research notes: offline memory system for Clawd workspaces (Markdown source-of-truth + derived index)"
|
||||
read_when:
|
||||
- Designing workspace memory (~/.openclaw/workspace) beyond daily Markdown logs
|
||||
- Deciding: standalone CLI vs deep OpenClaw integration
|
||||
- Adding offline recall + reflection (retain/recall/reflect)
|
||||
title: "Workspace Memory Research"
|
||||
---
|
||||
|
||||
# Workspace Memory v2 (offline): research notes
|
||||
|
||||
Target: Clawd-style workspace (`agents.defaults.workspace`, default `~/.openclaw/workspace`) where “memory” is stored as one Markdown file per day (`memory/YYYY-MM-DD.md`) plus a small set of stable files (e.g. `memory.md`, `SOUL.md`).
|
||||
|
||||
This doc proposes an **offline-first** memory architecture that keeps Markdown as the canonical, reviewable source of truth, but adds **structured recall** (search, entity summaries, confidence updates) via a derived index.
|
||||
|
||||
## Why change?
|
||||
|
||||
The current setup (one file per day) is excellent for:
|
||||
|
||||
- “append-only” journaling
|
||||
- human editing
|
||||
- git-backed durability + auditability
|
||||
- low-friction capture (“just write it down”)
|
||||
|
||||
It’s weak for:
|
||||
|
||||
- high-recall retrieval (“what did we decide about X?”, “last time we tried Y?”)
|
||||
- entity-centric answers (“tell me about Alice / The Castle / warelay”) without rereading many files
|
||||
- opinion/preference stability (and evidence when it changes)
|
||||
- time constraints (“what was true during Nov 2025?”) and conflict resolution
|
||||
|
||||
## Design goals
|
||||
|
||||
- **Offline**: works without network; can run on laptop/Castle; no cloud dependency.
|
||||
- **Explainable**: retrieved items should be attributable (file + location) and separable from inference.
|
||||
- **Low ceremony**: daily logging stays Markdown, no heavy schema work.
|
||||
- **Incremental**: v1 is useful with FTS only; semantic/vector and graphs are optional upgrades.
|
||||
- **Agent-friendly**: makes “recall within token budgets” easy (return small bundles of facts).
|
||||
|
||||
## North star model (Hindsight × Letta)
|
||||
|
||||
Two pieces to blend:
|
||||
|
||||
1. **Letta/MemGPT-style control loop**
|
||||
|
||||
- keep a small “core” always in context (persona + key user facts)
|
||||
- everything else is out-of-context and retrieved via tools
|
||||
- memory writes are explicit tool calls (append/replace/insert), persisted, then re-injected next turn
|
||||
|
||||
2. **Hindsight-style memory substrate**
|
||||
|
||||
- separate what’s observed vs what’s believed vs what’s summarized
|
||||
- support retain/recall/reflect
|
||||
- confidence-bearing opinions that can evolve with evidence
|
||||
- entity-aware retrieval + temporal queries (even without full knowledge graphs)
|
||||
|
||||
## Proposed architecture (Markdown source-of-truth + derived index)
|
||||
|
||||
### Canonical store (git-friendly)
|
||||
|
||||
Keep `~/.openclaw/workspace` as canonical human-readable memory.
|
||||
|
||||
Suggested workspace layout:
|
||||
|
||||
```
|
||||
~/.openclaw/workspace/
|
||||
memory.md # small: durable facts + preferences (core-ish)
|
||||
memory/
|
||||
YYYY-MM-DD.md # daily log (append; narrative)
|
||||
bank/ # “typed” memory pages (stable, reviewable)
|
||||
world.md # objective facts about the world
|
||||
experience.md # what the agent did (first-person)
|
||||
opinions.md # subjective prefs/judgments + confidence + evidence pointers
|
||||
entities/
|
||||
Peter.md
|
||||
The-Castle.md
|
||||
warelay.md
|
||||
...
|
||||
```
|
||||
|
||||
Notes:
|
||||
|
||||
- **Daily log stays daily log**. No need to turn it into JSON.
|
||||
- The `bank/` files are **curated**, produced by reflection jobs, and can still be edited by hand.
|
||||
- `memory.md` remains “small + core-ish”: the things you want Clawd to see every session.
|
||||
|
||||
### Derived store (machine recall)
|
||||
|
||||
Add a derived index under the workspace (not necessarily git tracked):
|
||||
|
||||
```
|
||||
~/.openclaw/workspace/.memory/index.sqlite
|
||||
```
|
||||
|
||||
Back it with:
|
||||
|
||||
- SQLite schema for facts + entity links + opinion metadata
|
||||
- SQLite **FTS5** for lexical recall (fast, tiny, offline)
|
||||
- optional embeddings table for semantic recall (still offline)
|
||||
|
||||
The index is always **rebuildable from Markdown**.
|
||||
|
||||
## Retain / Recall / Reflect (operational loop)
|
||||
|
||||
### Retain: normalize daily logs into “facts”
|
||||
|
||||
Hindsight’s key insight that matters here: store **narrative, self-contained facts**, not tiny snippets.
|
||||
|
||||
Practical rule for `memory/YYYY-MM-DD.md`:
|
||||
|
||||
- at end of day (or during), add a `## Retain` section with 2–5 bullets that are:
|
||||
- narrative (cross-turn context preserved)
|
||||
- self-contained (standalone makes sense later)
|
||||
- tagged with type + entity mentions
|
||||
|
||||
Example:
|
||||
|
||||
```
|
||||
## Retain
|
||||
- W @Peter: Currently in Marrakech (Nov 27–Dec 1, 2025) for Andy’s birthday.
|
||||
- B @warelay: I fixed the Baileys WS crash by wrapping connection.update handlers in try/catch (see memory/2025-11-27.md).
|
||||
- O(c=0.95) @Peter: Prefers concise replies (<1500 chars) on WhatsApp; long content goes into files.
|
||||
```
|
||||
|
||||
Minimal parsing:
|
||||
|
||||
- Type prefix: `W` (world), `B` (experience/biographical), `O` (opinion), `S` (observation/summary; usually generated)
|
||||
- Entities: `@Peter`, `@warelay`, etc (slugs map to `bank/entities/*.md`)
|
||||
- Opinion confidence: `O(c=0.0..1.0)` optional
|
||||
|
||||
If you don’t want authors to think about it: the reflect job can infer these bullets from the rest of the log, but having an explicit `## Retain` section is the easiest “quality lever”.
|
||||
|
||||
### Recall: queries over the derived index
|
||||
|
||||
Recall should support:
|
||||
|
||||
- **lexical**: “find exact terms / names / commands” (FTS5)
|
||||
- **entity**: “tell me about X” (entity pages + entity-linked facts)
|
||||
- **temporal**: “what happened around Nov 27” / “since last week”
|
||||
- **opinion**: “what does Peter prefer?” (with confidence + evidence)
|
||||
|
||||
Return format should be agent-friendly and cite sources:
|
||||
|
||||
- `kind` (`world|experience|opinion|observation`)
|
||||
- `timestamp` (source day, or extracted time range if present)
|
||||
- `entities` (`["Peter","warelay"]`)
|
||||
- `content` (the narrative fact)
|
||||
- `source` (`memory/2025-11-27.md#L12` etc)
|
||||
|
||||
### Reflect: produce stable pages + update beliefs
|
||||
|
||||
Reflection is a scheduled job (daily or heartbeat `ultrathink`) that:
|
||||
|
||||
- updates `bank/entities/*.md` from recent facts (entity summaries)
|
||||
- updates `bank/opinions.md` confidence based on reinforcement/contradiction
|
||||
- optionally proposes edits to `memory.md` (“core-ish” durable facts)
|
||||
|
||||
Opinion evolution (simple, explainable):
|
||||
|
||||
- each opinion has:
|
||||
- statement
|
||||
- confidence `c ∈ [0,1]`
|
||||
- last_updated
|
||||
- evidence links (supporting + contradicting fact IDs)
|
||||
- when new facts arrive:
|
||||
- find candidate opinions by entity overlap + similarity (FTS first, embeddings later)
|
||||
- update confidence by small deltas; big jumps require strong contradiction + repeated evidence
|
||||
|
||||
## CLI integration: standalone vs deep integration
|
||||
|
||||
Recommendation: **deep integration in OpenClaw**, but keep a separable core library.
|
||||
|
||||
### Why integrate into OpenClaw?
|
||||
|
||||
- OpenClaw already knows:
|
||||
- the workspace path (`agents.defaults.workspace`)
|
||||
- the session model + heartbeats
|
||||
- logging + troubleshooting patterns
|
||||
- You want the agent itself to call the tools:
|
||||
- `openclaw memory recall "…" --k 25 --since 30d`
|
||||
- `openclaw memory reflect --since 7d`
|
||||
|
||||
### Why still split a library?
|
||||
|
||||
- keep memory logic testable without gateway/runtime
|
||||
- reuse from other contexts (local scripts, future desktop app, etc.)
|
||||
|
||||
Shape:
|
||||
The memory tooling is intended to be a small CLI + library layer, but this is exploratory only.
|
||||
|
||||
## “S-Collide” / SuCo: when to use it (research)
|
||||
|
||||
If “S-Collide” refers to **SuCo (Subspace Collision)**: it’s an ANN retrieval approach that targets strong recall/latency tradeoffs by using learned/structured collisions in subspaces (paper: arXiv 2411.14754, 2024).
|
||||
|
||||
Pragmatic take for `~/.openclaw/workspace`:
|
||||
|
||||
- **don’t start** with SuCo.
|
||||
- start with SQLite FTS + (optional) simple embeddings; you’ll get most UX wins immediately.
|
||||
- consider SuCo/HNSW/ScaNN-class solutions only once:
|
||||
- corpus is big (tens/hundreds of thousands of chunks)
|
||||
- brute-force embedding search becomes too slow
|
||||
- recall quality is meaningfully bottlenecked by lexical search
|
||||
|
||||
Offline-friendly alternatives (in increasing complexity):
|
||||
|
||||
- SQLite FTS5 + metadata filters (zero ML)
|
||||
- Embeddings + brute force (works surprisingly far if chunk count is low)
|
||||
- HNSW index (common, robust; needs a library binding)
|
||||
- SuCo (research-grade; attractive if there’s a solid implementation you can embed)
|
||||
|
||||
Open question:
|
||||
|
||||
- what’s the **best** offline embedding model for “personal assistant memory” on your machines (laptop + desktop)?
|
||||
- if you already have Ollama: embed with a local model; otherwise ship a small embedding model in the toolchain.
|
||||
|
||||
## Smallest useful pilot
|
||||
|
||||
If you want a minimal, still-useful version:
|
||||
|
||||
- Add `bank/` entity pages and a `## Retain` section in daily logs.
|
||||
- Use SQLite FTS for recall with citations (path + line numbers).
|
||||
- Add embeddings only if recall quality or scale demands it.
|
||||
|
||||
## References
|
||||
|
||||
- Letta / MemGPT concepts: “core memory blocks” + “archival memory” + tool-driven self-editing memory.
|
||||
- Hindsight Technical Report: “retain / recall / reflect”, four-network memory, narrative fact extraction, opinion confidence evolution.
|
||||
- SuCo: arXiv 2411.14754 (2024): “Subspace Collision” approximate nearest neighbor retrieval.
|
||||
Reference in New Issue
Block a user