fix(test): de-flake schedules cron UPDATE assertion (Handlers Postgres Integration BP-required red) #3157

Merged

core-devops merged 1 commits from fix/schedules-cron-next-run-wallclock-race into main 2026-06-22 05:59:20 +00:00

Conversation 0 Commits 1 Files Changed 1

+15 -2

core-devops commented 2026-06-22 05:54:49 +00:00

Member

Copy Link

Copy Source

Red being fixed

Handlers Postgres Integration / Handlers Postgres Integration (BP-required) = FAILURE on main HEAD adc2e9ff (04:46Z) — a genuine current failure, not stale (sibling CI / all-required went green at 04:49 but this required context stayed red).

Named mechanism (no flaky-dismiss)

Single failing test: TestIntegration_Schedules_CRUDRunHistoryHealth_RoundTrip, line 211:

UPDATE cron: next_run_at should have moved (orig=2026-06-23 03:00:00 UTC new=2026-06-22 05:00:00 UTC)

The assertion newNextRun.After(origNextRun) assumed the recomputed 5am next-run is always strictly later than the 3am next-run. But 0 3 * * * and 0 5 * * * are daily crons whose next occurrence wraps every 24h. When the test runs in the 03:00-05:00 UTC window, the 3am schedule has already rolled to tomorrow while the 5am is still today — so the 5am next-run is earlier and the ordering inverts.

Proven deterministically against scheduler.ComputeNextRun: the old assertion fails at exactly UTC hours 3 and 4 (the 04:46Z run window) and passes the other 22 hours. This is a time-of-day race — not testcontainers/Postgres infra, not migration-048's local constraint.

Fix

Replace the brittle ordering check with the wall-clock-independent invariants that always hold after a cron UPDATE: next_run_at (a) actually changed, and (b) lands at exactly 05:00:00 UTC, proving the new cron was applied. Preserves the test's intent without the daily-wrap bug.

Verification

• go vet ./internal/handlers/ clean.
• Standalone proof over all 24 UTC hours: OLD assertion fails at hours [3,4]; NEW invariants hold every hour.
• Integration test requires INTEGRATION_DB_URL (real Postgres, booted in CI) — will run green in the Handlers Postgres Integration job.

Generated with Claude Code

## Red being fixed `Handlers Postgres Integration / Handlers Postgres Integration` (BP-required) = FAILURE on main HEAD `adc2e9ff` (04:46Z) — a genuine current failure, not stale (sibling `CI / all-required` went green at 04:49 but this required context stayed red). ## Named mechanism (no flaky-dismiss) Single failing test: `TestIntegration_Schedules_CRUDRunHistoryHealth_RoundTrip`, line 211: ``` UPDATE cron: next_run_at should have moved (orig=2026-06-23 03:00:00 UTC new=2026-06-22 05:00:00 UTC) ``` The assertion `newNextRun.After(origNextRun)` assumed the recomputed **5am** next-run is always strictly later than the **3am** next-run. But `0 3 * * *` and `0 5 * * *` are daily crons whose next occurrence wraps every 24h. When the test runs in the **03:00-05:00 UTC** window, the 3am schedule has already rolled to *tomorrow* while the 5am is still *today* — so the 5am next-run is **earlier** and the ordering inverts. Proven deterministically against `scheduler.ComputeNextRun`: the old assertion fails at **exactly UTC hours 3 and 4** (the 04:46Z run window) and passes the other 22 hours. This is a time-of-day race — **not** testcontainers/Postgres infra, **not** migration-048's `local` constraint. ## Fix Replace the brittle ordering check with the wall-clock-independent invariants that always hold after a cron UPDATE: next_run_at (a) actually changed, and (b) lands at exactly **05:00:00 UTC**, proving the new cron was applied. Preserves the test's intent without the daily-wrap bug. ## Verification - `go vet ./internal/handlers/` clean. - Standalone proof over all 24 UTC hours: OLD assertion fails at hours [3,4]; NEW invariants hold every hour. - Integration test requires `INTEGRATION_DB_URL` (real Postgres, booted in CI) — will run green in the Handlers Postgres Integration job. Generated with Claude Code

core-devops added 1 commit 2026-06-22 05:54:49 +00:00

fix(test): de-flake schedules cron UPDATE assertion — daily next-run wrap is wall-clock-dependent ... fefb516bec

TestIntegration_Schedules_CRUDRunHistoryHealth_RoundTrip (Handlers Postgres
Integration, BP-required) failed on main HEAD adc2e9ff at 04:46Z with:

  UPDATE cron: next_run_at should have moved
  (orig=2026-06-23 03:00:00 UTC new=2026-06-22 05:00:00 UTC)

Root cause (named, not flaky-dismiss): the assertion `newNextRun.After(origNextRun)`
assumed the recomputed 5am next-run is always strictly later than the 3am next-run.
But "0 3 * * *" and "0 5 * * *" are daily crons whose next occurrence wraps every
24h. Whenever the test runs in the 03:00-05:00 UTC window the 3am schedule has
already rolled to *tomorrow* while the 5am schedule is still *today*, so the 5am
next-run is EARLIER than the 3am next-run and the ordering inverts. Verified
deterministically against scheduler.ComputeNextRun: the old assertion fails at
exactly UTC hours 3 and 4 (the 04:46Z run window), and passes the other 22 hours.
This is a time-of-day race, not infra/testcontainers and not migration-048.

Fix: assert the wall-clock-independent invariants that always hold after a cron
UPDATE — next_run_at (a) actually changed and (b) lands at exactly 05:00:00 UTC,
proving the new cron was applied. Preserves the test's intent (UPDATE recomputes
next_run_at from the new cron) without the daily-wrap ordering bug.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

molecule-code-reviewer approved these changes 2026-06-22 05:55:37 +00:00

molecule-code-reviewer left a comment

Member

Copy Link

Copy Source

APPROVE — Code-review axes (correctness / clarity / scope).

Verified the named mechanism: 0 3 * * * vs 0 5 * * * are daily crons; their next-occurrence wraps every 24h, so in the 03:00-05:00 UTC window the 3am next-run is tomorrow while 5am is today, inverting new.After(orig). The 04:46Z failure falls squarely in that window.

The replacement is correct and altitude-appropriate:

• newNextRun.Equal(origNextRun) proves recompute happened (3am != 5am always).
• Hour==5 && Minute==0 && Second==0 (UTC) proves the new cron was applied — robfig/cron minute-parser yields exact minute boundaries, and the schedule's timezone is UTC (CREATE set it; this PATCH only changes cron_expr), so 05:00:00 UTC is exact.
• Test-only change, no production code touched; Case 4 reads next_run_at independently afterward, unaffected.

Deterministic — no residual wall-clock dependence. Intent ("UPDATE recomputes next_run_at from the new cron") preserved.

**APPROVE** — Code-review axes (correctness / clarity / scope). Verified the named mechanism: `0 3 * * *` vs `0 5 * * *` are daily crons; their next-occurrence wraps every 24h, so in the 03:00-05:00 UTC window the 3am next-run is tomorrow while 5am is today, inverting `new.After(orig)`. The 04:46Z failure falls squarely in that window. The replacement is correct and altitude-appropriate: - `newNextRun.Equal(origNextRun)` proves recompute happened (3am != 5am always). - `Hour==5 && Minute==0 && Second==0 (UTC)` proves the *new* cron was applied — robfig/cron minute-parser yields exact minute boundaries, and the schedule's timezone is UTC (CREATE set it; this PATCH only changes cron_expr), so 05:00:00 UTC is exact. - Test-only change, no production code touched; Case 4 reads next_run_at independently afterward, unaffected. Deterministic — no residual wall-clock dependence. Intent ("UPDATE recomputes next_run_at from the new cron") preserved.

core-security approved these changes 2026-06-22 05:55:40 +00:00

core-security left a comment

Member

Copy Link

Copy Source

APPROVE — Security review.

No security surface: change is confined to a single assertion in an integration test (schedules_integration_test.go). No production code, no auth/IDOR/input-handling paths, no secrets, no migrations, no dependency changes. The IDOR (Case 6) and invalid-timezone-rejection (Case 5) coverage in the same test is untouched and still gates the handler. No new attack surface or test-coverage regression. Fix correctly removes a wall-clock-dependent false-red without weakening what the test asserts.

**APPROVE** — Security review. No security surface: change is confined to a single assertion in an integration test (`schedules_integration_test.go`). No production code, no auth/IDOR/input-handling paths, no secrets, no migrations, no dependency changes. The IDOR (Case 6) and invalid-timezone-rejection (Case 5) coverage in the same test is untouched and still gates the handler. No new attack surface or test-coverage regression. Fix correctly removes a wall-clock-dependent false-red without weakening what the test asserts.

core-devops scheduled this pull request to auto merge when all checks succeed 2026-06-22 05:57:15 +00:00

agent-reviewer-cr2 approved these changes 2026-06-22 05:59:07 +00:00

agent-reviewer-cr2 left a comment

Member

Copy Link

Copy Source

5-axis current-head review clean. This is test-only in schedules_integration_test.go. The old assertion compared daily cron next-run ordering and failed during the 03:00-05:00 UTC wrap window. The replacement checks wall-clock-independent invariants: next_run_at is recomputed and the new value lands at exactly 05:00:00 UTC for cron '0 5 * * *'. No production logic, auth, or performance surface changed.

5-axis current-head review clean. This is test-only in schedules_integration_test.go. The old assertion compared daily cron next-run ordering and failed during the 03:00-05:00 UTC wrap window. The replacement checks wall-clock-independent invariants: next_run_at is recomputed and the new value lands at exactly 05:00:00 UTC for cron '0 5 * * *'. No production logic, auth, or performance surface changed.

core-devops merged commit b7e865974d into main 2026-06-22 05:59:20 +00:00

core-devops referenced this issue from a commit 2026-06-22 05:59:21 +00:00

fix(test): de-flake schedules cron UPDATE assertion (Handlers Postgres Integration BP-required red) (#3157)

Sign in to join this conversation.

Reviewers

No Reviewers

molecule-code-reviewer

core-security

agent-reviewer-cr2

Labels

Clear labels

area/ci

CI/CD pipeline issues

do-not-auto-merge

Opt out of autonomous merge-queue merging

kind/infrastructure

Infrastructure-related issues

merge-queue

Ready for serialized Gitea merge queue

merge-queue-hold

Temporarily hold PR in merge queue

platform/go

Go platform test issues

release-blocker

Blocks the staging→main promotion / a release

release-test

security

test-label-sre

tier:high

High risk per dev-sop §SOP-6 — ceo only, 24h cooldown

tier:low

Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve

tier:medium

Medium risk per dev-sop §SOP-6 — managers/ceo can approve

triage-test

test

wip

Work in progress — do not auto-merge

No Label

Milestone

No items

No Milestone

Projects

Clear projects

No project

Assignees

Clear assignees

agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer agent-reviewer-1 agent-reviewer-cr2 app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) godwin hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 hongming-personal infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)

No Assignees

4 Participants

Notifications

Subscribe

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#3157