molecule-ai/molecule-ai-plugin-molecule-skill-llm-judge
34
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
8 Commits 3 Branches 1 Tag 40 KiB
Python 100%
main
Go to file
security-auditor 43b347797f
All checks were successful
CI / validate (push) Successful in 1m19s
Details
ci: re-trigger after orchestrator recreated runners 1-8 (CONFIG_FILE env) 
Per saved memory feedback_act_runner_needs_config_file_env: runners 1-8
were spawned without -e CONFIG_FILE=/config.yaml; act_runner fell back
to /data/config.yaml and ignored runner.envs the whole time. Orchestrator
recreated 1-8 with full proper env. All 16 now uniform with
AGENT_TOOLSDIRECTORY + RUNNER_TOOL_CACHE + GITHUB_SERVER_URL + GH_HOST.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-07 02:51:19 -07:00
.github/workflows fix(ci): lowercase 'molecule-ai/' in cross-repo workflow refs 2026-05-07 00:59:29 -07:00
.molecule-ci/scripts import from local vendored copy (2026-05-06) 2026-05-06 13:53:35 -07:00
adapters import from local vendored copy (2026-05-06) 2026-05-06 13:53:35 -07:00
runbooks docs(install): migrate git clone URL to git.moleculesai.app (#37)\n\nAnonymous git-clone refs in CLAUDE.md, runbooks/local-dev-setup.md migrated github.com/Molecule-AI \u2192 git.moleculesai.app/molecule-ai. Public repo, no auth-shape change. Same pattern as the other plugin-* sweeps in the #37 series.\n\nRefs: molecule-ai/internal#37, molecule-ai/internal#38, molecule-ai/internal#42 2026-05-07 00:01:17 -07:00
skills/llm-judge import from local vendored copy (2026-05-06) 2026-05-06 13:53:35 -07:00
.gitignore import from local vendored copy (2026-05-06) 2026-05-06 13:53:35 -07:00
CLAUDE.md docs(install): migrate git clone URL to git.moleculesai.app (#37)\n\nAnonymous git-clone refs in CLAUDE.md, runbooks/local-dev-setup.md migrated github.com/Molecule-AI \u2192 git.moleculesai.app/molecule-ai. Public repo, no auth-shape change. Same pattern as the other plugin-* sweeps in the #37 series.\n\nRefs: molecule-ai/internal#37, molecule-ai/internal#38, molecule-ai/internal#42 2026-05-07 00:01:17 -07:00
known-issues.md import from local vendored copy (2026-05-06) 2026-05-06 13:53:35 -07:00
plugin.yaml import from local vendored copy (2026-05-06) 2026-05-06 13:53:35 -07:00
README.md import from local vendored copy (2026-05-06) 2026-05-06 13:53:35 -07:00

README.md

molecule-skill-llm-judge — LLM-as-Judge Gate

Plugin for Claude Code. Scores whether an agent's deliverable (a PR, a delegation result, a generated config) actually addresses the original request — the failure mode unit tests miss.

The problem it solves

Unit tests verify the code ran. They don't verify it did the right thing for the customer's actual request. An agent can implement the wrong solution perfectly.

When to use

After an agent (PM, Dev Lead, QA, etc.) produces a deliverable:

  • A PR opened in response to an issue
  • A delegation result (A2A message/send response)
  • A generated config or template
  • A code review they posted

Trigger: "Agent came back with 'done' — before we believe them."

What it does

  1. Presents the original request and the agent's deliverable to an LLM judge
  2. Scores: does the deliverable actually address the request?
  3. Reports: passes, partial, or fails — with evidence

Installation

In org template (org.yaml)

plugins:
  - molecule-skill-llm-judge

From URL

github://Molecule-AI/molecule-ai-plugin-molecule-skill-llm-judge

License

Business Source License 1.1 — © Molecule AI.