From a306a97dd3a4e50c3f2b4ce7ec52e358eb8796ee Mon Sep 17 00:00:00 2001 From: Hongming Wang Date: Sun, 3 May 2026 17:28:45 -0700 Subject: [PATCH] ci(synth-e2e): move cron off :00 to dodge GH scheduler drops MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit GitHub Actions scheduler de-prioritises :00 cron firings under load. Empirical 2026-05-03: the canary's cron was '0,20,40 * * * *' but actual firings landed at :08, :03, :01, :03 — :20 and :40 silently dropped. Detection latency degraded from claimed 20 min to actual ~60 min worst case. Move to '10,30,50 * * * *': - :10/:30/:50 sit 10 min off the top-of-hour load peak - Still 5 min from :15 sweep-cf-orphans and :45 sweep-cf-tunnels (the original constraint that kept us off :15/:45) - Same 20-min cadence; only the phase changes No code change beyond the cron expression + comment refresh. Co-Authored-By: Claude Opus 4.7 (1M context) --- .github/workflows/continuous-synth-e2e.yml | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-) diff --git a/.github/workflows/continuous-synth-e2e.yml b/.github/workflows/continuous-synth-e2e.yml index 924684e8..ba9633a9 100644 --- a/.github/workflows/continuous-synth-e2e.yml +++ b/.github/workflows/continuous-synth-e2e.yml @@ -32,10 +32,20 @@ name: Continuous synthetic E2E (staging) on: schedule: - # Every 20 minutes, on the :00 :20 :40. Offsets the existing :15 - # sweep-cf-orphans and :45 sweep-cf-tunnels so the three - # operations don't all hit Cloudflare/AWS at the same minute. - - cron: '0,20,40 * * * *' + # Every 20 minutes, on :10 :30 :50. Two constraints: + # 1. Stay off the top-of-hour. GitHub Actions scheduler drops + # :00 firings under high load (own docs: + # https://docs.github.com/en/actions/using-workflows/events-that-trigger-workflows#schedule). + # Empirical 2026-05-03: cron was '0,20,40 * * * *' but actual + # firings landed at :08, :03, :01, :03 with :20 + :40 silently + # dropped — only the :00-region run survived. Detection + # latency degraded from claimed 20 min to actual ~60 min. + # :10/:30/:50 sit far enough from :00 that GH-load skips + # stop dropping us. + # 2. Avoid colliding with the existing :15 sweep-cf-orphans + # and :45 sweep-cf-tunnels — both hit the CF API and we + # don't want to fight for rate-limit tokens. + - cron: '10,30,50 * * * *' workflow_dispatch: inputs: runtime: