feat(onboarding): contextual first-touch hints for /busy and /verbose (#16046)

Instead of a blocking first-run questionnaire, show a one-time hint the first
time the user hits each behavior fork:

1. First message while the agent is working — appends a hint to the busy-ack
   explaining the /busy queue vs /busy interrupt knob, phrased to match the
   mode that was just applied (don't tell a queue-mode user to switch to
   queue).

2. First tool that runs for >= 30s in the noisiest progress mode
   (tool_progress: all) — prints a hint about /verbose to cycle display
   modes (all -> new -> off -> verbose). Gated on /verbose actually being
   usable on the surface: always shown on CLI; on gateway only shown when
   display.tool_progress_command is enabled.

Each hint is latched in config.yaml under onboarding.seen.<flag>, so it
fires exactly once per install across CLI, gateway, and cron, then never
again. Users can wipe the section to re-see hints.

New:
- agent/onboarding.py — is_seen / mark_seen / hint strings, shared by
  both CLI and gateway.
- onboarding.seen in DEFAULT_CONFIG (hermes_cli/config.py) and in
  load_cli_config defaults (cli.py). No _config_version bump — deep
  merge handles new keys.

Wired:
- gateway/run.py: _handle_active_session_busy_message appends the hint
  after building the ack.  progress_callback tracks tool.completed
  duration and queues the tool-progress hint into the progress bubble.
- cli.py: CLI input loop appends the busy-input hint on the first busy
  Enter; _on_tool_progress appends the tool-progress hint on the first
  >=30s tool completion.  In-memory CLI_CONFIG is also updated so
  subsequent fires in the same process are suppressed immediately.

All writes go through atomic_yaml_write and are wrapped in try/except
so onboarding can never break the input/busy-ack paths.
This commit is contained in:
Teknium 2026-04-26 06:06:27 -07:00 committed by GitHub
parent 4bda9dcade
commit 83c1c201f6
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
8 changed files with 548 additions and 1 deletions

144
agent/onboarding.py Normal file
View File

@ -0,0 +1,144 @@
"""
Contextual first-touch onboarding hints.
Instead of blocking first-run questionnaires, show a one-time hint the *first*
time a user hits a behavior fork message-while-running, first long-running
tool, etc. Each hint is shown once per install (tracked in ``config.yaml`` under
``onboarding.seen.<flag>``) and then never again.
Keep this module tiny and dependency-free so both the CLI and gateway can import
it without pulling in heavy modules.
"""
from __future__ import annotations
import logging
from pathlib import Path
from typing import Any, Mapping, Optional
logger = logging.getLogger(__name__)
# -------------------------------------------------------------------------
# Flag names (stable — used as config.yaml keys under onboarding.seen)
# -------------------------------------------------------------------------
BUSY_INPUT_FLAG = "busy_input_prompt"
TOOL_PROGRESS_FLAG = "tool_progress_prompt"
# -------------------------------------------------------------------------
# Hint content
# -------------------------------------------------------------------------
def busy_input_hint_gateway(mode: str) -> str:
"""Hint shown the first time a user messages while the agent is busy.
``mode`` is the effective busy_input_mode that was just applied, so the
message matches reality ("I just interrupted…" vs "I just queued…").
"""
if mode == "queue":
return (
"💡 First-time tip — I queued your message instead of interrupting. "
"Send `/busy interrupt` to make new messages stop the current task "
"immediately, or `/busy status` to check. This notice won't appear again."
)
return (
"💡 First-time tip — I just interrupted my current task to answer you. "
"Send `/busy queue` to queue follow-ups for after the current task instead, "
"or `/busy status` to check. This notice won't appear again."
)
def busy_input_hint_cli(mode: str) -> str:
"""CLI version of the busy-input hint (plain text, no markdown)."""
if mode == "queue":
return (
"(tip) Your message was queued for the next turn. "
"Use /busy interrupt to make Enter stop the current run instead. "
"This tip only shows once."
)
return (
"(tip) Your message interrupted the current run. "
"Use /busy queue to queue messages for the next turn instead. "
"This tip only shows once."
)
def tool_progress_hint_gateway() -> str:
return (
"💡 First-time tip — that tool took a while and I'm streaming every step. "
"If the progress messages feel noisy, send `/verbose` to cycle modes "
"(all → new → off). This notice won't appear again."
)
def tool_progress_hint_cli() -> str:
return (
"(tip) That tool ran for a while. Use /verbose to cycle tool-progress "
"display modes (all -> new -> off -> verbose). This tip only shows once."
)
# -------------------------------------------------------------------------
# State read / write
# -------------------------------------------------------------------------
def _get_seen_dict(config: Mapping[str, Any]) -> Mapping[str, Any]:
onboarding = config.get("onboarding") if isinstance(config, Mapping) else None
if not isinstance(onboarding, Mapping):
return {}
seen = onboarding.get("seen")
return seen if isinstance(seen, Mapping) else {}
def is_seen(config: Mapping[str, Any], flag: str) -> bool:
"""Return True if the user has already been shown this first-touch hint."""
return bool(_get_seen_dict(config).get(flag))
def mark_seen(config_path: Path, flag: str) -> bool:
"""Persist ``onboarding.seen.<flag> = True`` to ``config_path``.
Uses the atomic YAML writer so a concurrent process can't observe a
partially-written file. Returns True on success, False on any error
(including the config file being absent onboarding is best-effort).
"""
try:
import yaml
from utils import atomic_yaml_write
except Exception as e: # pragma: no cover — dependency issue
logger.debug("onboarding: failed to import yaml/utils: %s", e)
return False
try:
cfg: dict = {}
if config_path.exists():
with open(config_path, encoding="utf-8") as f:
cfg = yaml.safe_load(f) or {}
if not isinstance(cfg.get("onboarding"), dict):
cfg["onboarding"] = {}
seen = cfg["onboarding"].get("seen")
if not isinstance(seen, dict):
seen = {}
cfg["onboarding"]["seen"] = seen
if seen.get(flag) is True:
return True # already marked — nothing to do
seen[flag] = True
atomic_yaml_write(config_path, cfg)
return True
except Exception as e:
logger.debug("onboarding: failed to mark flag %s: %s", flag, e)
return False
__all__ = [
"BUSY_INPUT_FLAG",
"TOOL_PROGRESS_FLAG",
"busy_input_hint_gateway",
"busy_input_hint_cli",
"tool_progress_hint_gateway",
"tool_progress_hint_cli",
"is_seen",
"mark_seen",
]

48
cli.py
View File

@ -417,6 +417,11 @@ def load_cli_config() -> Dict[str, Any]:
"base_url": "", # Direct OpenAI-compatible endpoint for subagents
"api_key": "", # API key for delegation.base_url (falls back to OPENAI_API_KEY)
},
"onboarding": {
# First-touch hint flags (see agent/onboarding.py). Each hint is
# shown once per install then latched here.
"seen": {},
},
}
# Track whether the config file explicitly set terminal config.
@ -7412,6 +7417,31 @@ class HermesCLI:
_cprint(f" {line}")
except Exception:
pass
# First-touch onboarding: on the first tool in this process
# that takes longer than the threshold while we're in the
# noisiest progress mode, print a one-time hint about
# /verbose. Latched on self so it fires at most once per
# process; persisted to config.yaml so it never fires again
# across processes either.
try:
if (
not getattr(self, "_long_tool_hint_fired", False)
and self.tool_progress_mode == "all"
and duration >= 30.0
):
from agent.onboarding import (
TOOL_PROGRESS_FLAG,
is_seen,
mark_seen,
tool_progress_hint_cli,
)
if not is_seen(CLI_CONFIG, TOOL_PROGRESS_FLAG):
self._long_tool_hint_fired = True
_cprint(f" {_DIM}{tool_progress_hint_cli()}{_RST}")
mark_seen(_hermes_home / "config.yaml", TOOL_PROGRESS_FLAG)
CLI_CONFIG.setdefault("onboarding", {}).setdefault("seen", {})[TOOL_PROGRESS_FLAG] = True
except Exception:
pass
self._invalidate()
return
if event_type != "tool.started":
@ -9295,6 +9325,24 @@ class HermesCLI:
f"agent_running={self._agent_running}\n")
except Exception:
pass
# First-touch onboarding: on the very first busy-while-running
# event for this install, print a one-line tip explaining the
# /busy knob. Flag persists to config.yaml and never fires
# again. Guarded for exceptions so onboarding can't break
# the input loop.
try:
from agent.onboarding import (
BUSY_INPUT_FLAG,
busy_input_hint_cli,
is_seen,
mark_seen,
)
if not is_seen(CLI_CONFIG, BUSY_INPUT_FLAG):
_cprint(f" {_DIM}{busy_input_hint_cli(self.busy_input_mode)}{_RST}")
mark_seen(_hermes_home / "config.yaml", BUSY_INPUT_FLAG)
CLI_CONFIG.setdefault("onboarding", {}).setdefault("seen", {})[BUSY_INPUT_FLAG] = True
except Exception:
pass
else:
self._pending_input.put(payload)
event.app.current_buffer.reset(append_to_history=True)

View File

@ -1630,6 +1630,27 @@ class GatewayRunner:
f"I'll respond to your message shortly."
)
# First-touch onboarding: the very first time a user sends a message
# while the agent is busy, append a one-time hint explaining the
# queue/interrupt knob. Flag is persisted to config.yaml so it never
# fires again on this install.
try:
from agent.onboarding import (
BUSY_INPUT_FLAG,
busy_input_hint_gateway,
is_seen,
mark_seen,
)
_user_cfg = _load_gateway_config()
if not is_seen(_user_cfg, BUSY_INPUT_FLAG):
message = (
f"{message}\n\n"
f"{busy_input_hint_gateway('queue' if is_queue_mode else 'interrupt')}"
)
mark_seen(_hermes_home / "config.yaml", BUSY_INPUT_FLAG)
except Exception as _onb_err:
logger.debug("Failed to apply busy-input onboarding hint: %s", _onb_err)
thread_meta = {"thread_id": event.source.thread_id} if event.source.thread_id else None
try:
await adapter._send_with_retry(
@ -9411,12 +9432,42 @@ class GatewayRunner:
last_tool = [None] # Mutable container for tracking in closure
last_progress_msg = [None] # Track last message for dedup
repeat_count = [0] # How many times the same message repeated
# First-touch onboarding latch: fires at most once per run, even if
# several tools exceed the threshold.
long_tool_hint_fired = [False]
_LONG_TOOL_THRESHOLD_S = 30.0
def progress_callback(event_type: str, tool_name: str = None, preview: str = None, args: dict = None, **kwargs):
"""Callback invoked by agent on tool lifecycle events."""
if not progress_queue or not _run_still_current():
return
# First-touch onboarding: the first time a tool takes longer than
# _LONG_TOOL_THRESHOLD_S during a run that's streaming every tool
# (progress_mode == "all"), append a one-time hint suggesting
# /verbose. We only fire when (a) the user hasn't seen the hint
# before and (b) /verbose is actually usable on this platform
# (gateway gate must be open). The CLI has its own trigger.
if event_type == "tool.completed" and not long_tool_hint_fired[0]:
try:
duration = kwargs.get("duration") or 0
if duration >= _LONG_TOOL_THRESHOLD_S and progress_mode == "all":
from agent.onboarding import (
TOOL_PROGRESS_FLAG,
is_seen,
mark_seen,
tool_progress_hint_gateway,
)
_cfg = _load_gateway_config()
gate_on = bool(_cfg.get("display", {}).get("tool_progress_command", False))
if gate_on and not is_seen(_cfg, TOOL_PROGRESS_FLAG):
long_tool_hint_fired[0] = True
progress_queue.put(tool_progress_hint_gateway())
mark_seen(_hermes_home / "config.yaml", TOOL_PROGRESS_FLAG)
except Exception as _hint_err:
logger.debug("tool-progress onboarding hint failed: %s", _hint_err)
return
# Only act on tool.started events (ignore tool.completed, reasoning.available, etc.)
if event_type not in ("tool.started",):
return

View File

@ -1016,6 +1016,13 @@ DEFAULT_CONFIG = {
"min_interval_hours": 24,
},
# Contextual first-touch onboarding hints (see agent/onboarding.py).
# Each hint is shown once per install and then latched here so it
# never fires again. Users can wipe the section to re-see all hints.
"onboarding": {
"seen": {},
},
# Config schema version - bump this when adding new required fields
"_config_version": 22,
}

View File

@ -0,0 +1,164 @@
"""Tests for agent/onboarding.py — contextual first-touch hint helpers."""
from __future__ import annotations
import yaml
import pytest
from agent.onboarding import (
BUSY_INPUT_FLAG,
TOOL_PROGRESS_FLAG,
busy_input_hint_cli,
busy_input_hint_gateway,
is_seen,
mark_seen,
tool_progress_hint_cli,
tool_progress_hint_gateway,
)
class TestIsSeen:
def test_empty_config_unseen(self):
assert is_seen({}, BUSY_INPUT_FLAG) is False
def test_missing_onboarding_unseen(self):
assert is_seen({"display": {}}, BUSY_INPUT_FLAG) is False
def test_onboarding_not_dict_unseen(self):
assert is_seen({"onboarding": "nope"}, BUSY_INPUT_FLAG) is False
def test_seen_dict_missing_flag(self):
assert is_seen({"onboarding": {"seen": {}}}, BUSY_INPUT_FLAG) is False
def test_seen_flag_true(self):
cfg = {"onboarding": {"seen": {BUSY_INPUT_FLAG: True}}}
assert is_seen(cfg, BUSY_INPUT_FLAG) is True
def test_seen_flag_falsy(self):
cfg = {"onboarding": {"seen": {BUSY_INPUT_FLAG: False}}}
assert is_seen(cfg, BUSY_INPUT_FLAG) is False
def test_other_flags_isolated(self):
cfg = {"onboarding": {"seen": {BUSY_INPUT_FLAG: True}}}
assert is_seen(cfg, TOOL_PROGRESS_FLAG) is False
class TestMarkSeen:
def test_creates_missing_file_and_sets_flag(self, tmp_path):
cfg_path = tmp_path / "config.yaml"
assert mark_seen(cfg_path, BUSY_INPUT_FLAG) is True
loaded = yaml.safe_load(cfg_path.read_text())
assert loaded["onboarding"]["seen"][BUSY_INPUT_FLAG] is True
def test_preserves_other_config(self, tmp_path):
cfg_path = tmp_path / "config.yaml"
cfg_path.write_text(yaml.safe_dump({
"model": {"default": "claude-sonnet-4.6"},
"display": {"skin": "default"},
}))
assert mark_seen(cfg_path, BUSY_INPUT_FLAG) is True
loaded = yaml.safe_load(cfg_path.read_text())
assert loaded["model"]["default"] == "claude-sonnet-4.6"
assert loaded["display"]["skin"] == "default"
assert loaded["onboarding"]["seen"][BUSY_INPUT_FLAG] is True
def test_preserves_other_seen_flags(self, tmp_path):
cfg_path = tmp_path / "config.yaml"
cfg_path.write_text(yaml.safe_dump({
"onboarding": {"seen": {TOOL_PROGRESS_FLAG: True}},
}))
assert mark_seen(cfg_path, BUSY_INPUT_FLAG) is True
loaded = yaml.safe_load(cfg_path.read_text())
assert loaded["onboarding"]["seen"][TOOL_PROGRESS_FLAG] is True
assert loaded["onboarding"]["seen"][BUSY_INPUT_FLAG] is True
def test_idempotent(self, tmp_path):
cfg_path = tmp_path / "config.yaml"
mark_seen(cfg_path, BUSY_INPUT_FLAG)
first = cfg_path.read_text()
# Second call must be a no-op on-disk content (file may be touched,
# but the YAML contents should be identical).
mark_seen(cfg_path, BUSY_INPUT_FLAG)
second = cfg_path.read_text()
assert yaml.safe_load(first) == yaml.safe_load(second)
def test_handles_non_dict_onboarding(self, tmp_path):
cfg_path = tmp_path / "config.yaml"
cfg_path.write_text(yaml.safe_dump({"onboarding": "corrupted"}))
assert mark_seen(cfg_path, BUSY_INPUT_FLAG) is True
loaded = yaml.safe_load(cfg_path.read_text())
assert loaded["onboarding"]["seen"][BUSY_INPUT_FLAG] is True
def test_handles_non_dict_seen(self, tmp_path):
cfg_path = tmp_path / "config.yaml"
cfg_path.write_text(yaml.safe_dump({"onboarding": {"seen": "corrupted"}}))
assert mark_seen(cfg_path, BUSY_INPUT_FLAG) is True
loaded = yaml.safe_load(cfg_path.read_text())
assert loaded["onboarding"]["seen"][BUSY_INPUT_FLAG] is True
class TestHintMessages:
def test_busy_input_hint_gateway_interrupt(self):
msg = busy_input_hint_gateway("interrupt")
assert "/busy queue" in msg
assert "interrupted" in msg.lower()
def test_busy_input_hint_gateway_queue(self):
msg = busy_input_hint_gateway("queue")
assert "/busy interrupt" in msg
assert "queued" in msg.lower()
def test_busy_input_hint_cli_interrupt(self):
msg = busy_input_hint_cli("interrupt")
assert "/busy queue" in msg
def test_busy_input_hint_cli_queue(self):
msg = busy_input_hint_cli("queue")
assert "/busy interrupt" in msg
def test_tool_progress_hints_mention_verbose(self):
assert "/verbose" in tool_progress_hint_gateway()
assert "/verbose" in tool_progress_hint_cli()
def test_hints_are_not_empty(self):
for hint in (
busy_input_hint_gateway("queue"),
busy_input_hint_gateway("interrupt"),
busy_input_hint_cli("queue"),
busy_input_hint_cli("interrupt"),
tool_progress_hint_gateway(),
tool_progress_hint_cli(),
):
assert hint.strip()
class TestRoundTrip:
"""After mark_seen, is_seen on the re-loaded config must return True."""
def test_mark_then_is_seen(self, tmp_path):
cfg_path = tmp_path / "config.yaml"
assert mark_seen(cfg_path, BUSY_INPUT_FLAG) is True
loaded = yaml.safe_load(cfg_path.read_text())
assert is_seen(loaded, BUSY_INPUT_FLAG) is True
assert is_seen(loaded, TOOL_PROGRESS_FLAG) is False
def test_mark_both_flags_independently(self, tmp_path):
cfg_path = tmp_path / "config.yaml"
mark_seen(cfg_path, BUSY_INPUT_FLAG)
mark_seen(cfg_path, TOOL_PROGRESS_FLAG)
loaded = yaml.safe_load(cfg_path.read_text())
assert is_seen(loaded, BUSY_INPUT_FLAG) is True
assert is_seen(loaded, TOOL_PROGRESS_FLAG) is True

View File

@ -349,3 +349,121 @@ class TestBusySessionAck:
result = await runner._handle_active_session_busy_message(event, sk)
assert result is False # not handled, let default path try
class TestBusySessionOnboardingHint:
"""First-touch hint appended to the busy-ack the first time it fires."""
@pytest.mark.asyncio
async def test_first_busy_ack_appends_interrupt_hint(self, tmp_path, monkeypatch):
"""First busy-while-running message gets an extra hint about /busy."""
import gateway.run as _gr
monkeypatch.setattr(_gr, "_hermes_home", tmp_path)
# mark_seen imports utils.atomic_yaml_write; make sure it resolves
# against a writable dir by pointing _hermes_home at tmp_path.
monkeypatch.setattr(_gr, "_load_gateway_config", lambda: {})
runner, _sentinel = _make_runner()
runner._busy_input_mode = "interrupt"
adapter = _make_adapter()
event = _make_event(text="ping")
sk = build_session_key(event.source)
agent = MagicMock()
agent.get_activity_summary.return_value = {
"api_call_count": 3, "max_iterations": 60,
"current_tool": None, "last_activity_ts": time.time(),
"last_activity_desc": "api", "seconds_since_activity": 0.1,
}
runner._running_agents[sk] = agent
runner._running_agents_ts[sk] = time.time() - 5
runner.adapters[event.source.platform] = adapter
await runner._handle_active_session_busy_message(event, sk)
call_kwargs = adapter._send_with_retry.call_args
content = call_kwargs.kwargs.get("content", "")
# Normal ack body
assert "Interrupting" in content
# First-touch hint appended
assert "First-time tip" in content
assert "/busy queue" in content
# The flag is now persisted to tmp_path/config.yaml
import yaml
cfg = yaml.safe_load((tmp_path / "config.yaml").read_text())
assert cfg["onboarding"]["seen"]["busy_input_prompt"] is True
@pytest.mark.asyncio
async def test_second_busy_ack_omits_hint(self, tmp_path, monkeypatch):
"""Once the flag is marked, the hint never appears again."""
import gateway.run as _gr
import yaml
monkeypatch.setattr(_gr, "_hermes_home", tmp_path)
# Pre-populate the config so is_seen() returns True from the start.
(tmp_path / "config.yaml").write_text(yaml.safe_dump({
"onboarding": {"seen": {"busy_input_prompt": True}},
}))
monkeypatch.setattr(
_gr, "_load_gateway_config",
lambda: yaml.safe_load((tmp_path / "config.yaml").read_text()),
)
runner, _sentinel = _make_runner()
runner._busy_input_mode = "interrupt"
adapter = _make_adapter()
event = _make_event(text="ping again")
sk = build_session_key(event.source)
agent = MagicMock()
agent.get_activity_summary.return_value = {
"api_call_count": 3, "max_iterations": 60,
"current_tool": None, "last_activity_ts": time.time(),
"last_activity_desc": "api", "seconds_since_activity": 0.1,
}
runner._running_agents[sk] = agent
runner._running_agents_ts[sk] = time.time() - 5
runner.adapters[event.source.platform] = adapter
await runner._handle_active_session_busy_message(event, sk)
call_kwargs = adapter._send_with_retry.call_args
content = call_kwargs.kwargs.get("content", "")
assert "Interrupting" in content
assert "First-time tip" not in content
assert "/busy queue" not in content
@pytest.mark.asyncio
async def test_queue_mode_hint_points_to_interrupt(self, tmp_path, monkeypatch):
"""In queue mode the hint should suggest /busy interrupt, not /busy queue."""
import gateway.run as _gr
monkeypatch.setattr(_gr, "_hermes_home", tmp_path)
monkeypatch.setattr(_gr, "_load_gateway_config", lambda: {})
runner, _sentinel = _make_runner()
runner._busy_input_mode = "queue"
adapter = _make_adapter()
event = _make_event(text="queue me")
sk = build_session_key(event.source)
runner.adapters[event.source.platform] = adapter
agent = MagicMock()
runner._running_agents[sk] = agent
with patch("gateway.run.merge_pending_message_event"):
await runner._handle_active_session_busy_message(event, sk)
content = adapter._send_with_retry.call_args.kwargs.get("content", "")
assert "Queued for the next turn" in content
assert "First-time tip" in content
assert "/busy interrupt" in content
# Must NOT tell the user to /busy queue when they're already on queue.
assert "/busy queue" not in content

View File

@ -242,6 +242,10 @@ You can also change it inside the CLI:
/busy status
```
:::tip First-touch hint
The very first time you press Enter while Hermes is working, Hermes prints a one-line reminder explaining the `/busy` knob (`"(tip) Your message interrupted the current run…"`). It only fires once per install — a flag in `config.yaml` under `onboarding.seen.busy_input_prompt` latches it. Delete that key to see the tip again.
:::
### Suspending to Background
On Unix systems, press **`Ctrl+Z`** to suspend Hermes to the background — just like any terminal process. The shell prints a confirmation:

View File

@ -219,6 +219,17 @@ Send any message while the agent is working to interrupt it. Key behaviors:
- **Multiple messages are combined** — messages sent during interruption are joined into one prompt
- **`/stop` command** — interrupts without queuing a follow-up message
### Queue vs interrupt (busy-input mode)
By default, messaging a busy agent interrupts it. To switch the whole install so follow-ups queue behind the current task instead, set:
```yaml
display:
busy_input_mode: queue # default: interrupt
```
The first time you message a busy agent on any platform, Hermes appends a one-line reminder to the busy-ack explaining the knob (`"💡 First-time tip — …"`). The reminder fires once per install — a flag under `onboarding.seen.busy_input_prompt` latches it. Delete that key to see the tip again.
## Tool Progress Notifications
Control how much tool activity is displayed in `~/.hermes/config.yaml`: