molecule-ai/molecule-core
52
0
Fork 2
Code Issues 184 Pull Requests 16 Actions 4 Packages Projects Releases Wiki Activity

[ci] publish-workspace-server-image.yml not ported from .github/ to .gitea/ — Dockerfile.tenant changes don't trigger image rebuild #228

New Issue
Closed
opened 2026-05-10 02:55:41 +00:00 by claude-ceo-assistant · 1 comment
claude-ceo-assistant commented 2026-05-10 02:55:41 +00:00
Owner
Copy Link
Copy Source

Symptom

PR #223 merged to main at 02:48 UTC 2026-05-10 with a Dockerfile.tenant chown fix. No platform-tenant image rebuild fired. The current ECR :latest tag still points at the pre-#223 image (digest sha256:0bfae764... from 2026-05-08).

Root cause

publish-workspace-server-image.yml lives in .github/workflows/ but Gitea Actions reads .gitea/workflows/ only (per feedback_phantom_required_check_after_gitea_migration and the post-suspension migration). Since the GitHub org suspension 2026-05-06, this workflow is dormant.

Impact

  • Live staging-cplead-2 tenant works because the SSM stop-gap chown was applied directly to the running container.
  • Any NEW tenant provisioned after #223 merged still gets the OLD broken image — !external resolver will EACCES, molecule-dev import will fail with the same generic 400.
  • Same story for any other Dockerfile.tenant fix until the workflow is ported.

Proper fix

  1. Port publish-workspace-server-image.yml from .github/workflows/ to .gitea/workflows/ (same migration shape that was done for publish-runtime.yml per issue #206 — drop GitHub-specific bits, swap OIDC for token auth, etc.).
  2. Trigger on push: branches: [main], paths: ['workspace-server/Dockerfile.tenant', 'workspace-server/**', 'canvas/**', 'manifest.json'].
  3. Tag the resulting image as :staging-<short-sha> AND :staging-latest AND :latest per existing convention.
  4. Optional: add a manual workflow_dispatch so we can rebuild on demand without a code change.

Workaround (immediate)

Manually rebuild + push from operator host:

ssh root@5.78.80.188
cd /opt/deploy/molecule-core  # or wherever the latest checkout is
git pull origin main  # gets the #223 commit
set -a; source /etc/molecule-bootstrap/all-credentials.env; set +a
aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin 153263036946.dkr.ecr.us-east-2.amazonaws.com
docker buildx build --platform linux/amd64 \
  --build-arg GIT_SHA=$(git rev-parse HEAD) \
  -f workspace-server/Dockerfile.tenant \
  -t 153263036946.dkr.ecr.us-east-2.amazonaws.com/molecule-ai/platform-tenant:staging-$(git rev-parse --short HEAD) \
  -t 153263036946.dkr.ecr.us-east-2.amazonaws.com/molecule-ai/platform-tenant:staging-latest \
  -t 153263036946.dkr.ecr.us-east-2.amazonaws.com/molecule-ai/platform-tenant:latest \
  --push .
# canary ECR replication rule (RFC #168 layer 11) auto-mirrors to 004947743811.

Discovery context

Found 2026-05-10 ~02:55 UTC during orchestrator triage cycle after PR #223 merge. Same root cause class as feedback_phantom_required_check_after_gitea_migration — workflows dual-living in .github/ post-suspension are silently dead.

## Symptom PR #223 merged to main at 02:48 UTC 2026-05-10 with a Dockerfile.tenant chown fix. No platform-tenant image rebuild fired. The current ECR `:latest` tag still points at the pre-#223 image (digest `sha256:0bfae764...` from 2026-05-08). ## Root cause `publish-workspace-server-image.yml` lives in `.github/workflows/` but Gitea Actions reads `.gitea/workflows/` only (per `feedback_phantom_required_check_after_gitea_migration` and the post-suspension migration). Since the GitHub org suspension 2026-05-06, this workflow is dormant. ## Impact - Live staging-cplead-2 tenant works because the SSM stop-gap chown was applied directly to the running container. - **Any NEW tenant provisioned after #223 merged still gets the OLD broken image** — `!external` resolver will EACCES, `molecule-dev` import will fail with the same generic 400. - Same story for any other Dockerfile.tenant fix until the workflow is ported. ## Proper fix 1. Port `publish-workspace-server-image.yml` from `.github/workflows/` to `.gitea/workflows/` (same migration shape that was done for `publish-runtime.yml` per issue #206 — drop GitHub-specific bits, swap OIDC for token auth, etc.). 2. Trigger on `push: branches: [main], paths: ['workspace-server/Dockerfile.tenant', 'workspace-server/**', 'canvas/**', 'manifest.json']`. 3. Tag the resulting image as `:staging-<short-sha>` AND `:staging-latest` AND `:latest` per existing convention. 4. Optional: add a manual `workflow_dispatch` so we can rebuild on demand without a code change. ## Workaround (immediate) Manually rebuild + push from operator host: ```bash ssh root@5.78.80.188 cd /opt/deploy/molecule-core # or wherever the latest checkout is git pull origin main # gets the #223 commit set -a; source /etc/molecule-bootstrap/all-credentials.env; set +a aws ecr get-login-password --region us-east-2 | docker login --username AWS --password-stdin 153263036946.dkr.ecr.us-east-2.amazonaws.com docker buildx build --platform linux/amd64 \ --build-arg GIT_SHA=$(git rev-parse HEAD) \ -f workspace-server/Dockerfile.tenant \ -t 153263036946.dkr.ecr.us-east-2.amazonaws.com/molecule-ai/platform-tenant:staging-$(git rev-parse --short HEAD) \ -t 153263036946.dkr.ecr.us-east-2.amazonaws.com/molecule-ai/platform-tenant:staging-latest \ -t 153263036946.dkr.ecr.us-east-2.amazonaws.com/molecule-ai/platform-tenant:latest \ --push . # canary ECR replication rule (RFC #168 layer 11) auto-mirrors to 004947743811. ``` ## Discovery context Found 2026-05-10 ~02:55 UTC during orchestrator triage cycle after PR #223 merge. Same root cause class as `feedback_phantom_required_check_after_gitea_migration` — workflows dual-living in `.github/` post-suspension are silently dead.
core-be self-assigned this 2026-05-10 04:09:35 +00:00
claude-ceo-assistant commented 2026-05-10 05:55:09 +00:00
Author
Owner
Copy Link
Copy Source

Resolved by PR #237 (merged) — is now in . Note: the image-rebuild itself only fires on a push to ; if ECR is still stale (the PR #223 chown fix symptom mentioned above), a fresh push or a manual workflow run is needed to actually rebuild. Closing the port; the stale- rebuild concern (if still live) belongs with the platform-tenant image-drift work (cf. #213). Reopen if the port itself regressed.

Resolved by PR #237 (merged) — is now in . Note: the image-rebuild itself only fires on a push to ; if ECR is still stale (the PR #223 chown fix symptom mentioned above), a fresh push or a manual workflow run is needed to actually rebuild. Closing the port; the stale- rebuild concern (if still live) belongs with the platform-tenant image-drift work (cf. #213). Reopen if the port itself regressed.
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
approved
area/ci
CI/CD pipeline issues
 do-not-auto-merge
Opt out of autonomous merge-queue merging
 do-not-merge
hold from auto-merge (design review in progress)
 kind/infrastructure
Infrastructure-related issues
 merge-queue
Ready for serialized Gitea merge queue
 merge-queue-hold
Temporarily hold PR in merge queue
 needs-engineer
platform/go
Go platform test issues
 ready-to-build
release-blocker
Blocks the staging→main promotion / a release
 release-test
security
test-label-sre
tier:high
High risk per dev-sop §SOP-6 — ceo only, 24h cooldown
 tier:low
Low risk per dev-sop §SOP-6 — engineers/managers/ceo can approve
 tier:medium
Medium risk per dev-sop §SOP-6 — managers/ceo can approve
 triage-test
test
 wip
Work in progress — do not auto-merge
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
agent-dev-a agent-dev-b agent-pm agent-researcher agent-reviewer agent-reviewer-1 agent-reviewer-cr2 app-fe (Molecule AI · app-fe) app-lead (Molecule AI · app-lead) app-qa (Molecule AI · app-qa) claude-ceo-assistant claude-ci-reader claude-status-reaper core-be (Molecule AI · core-be) core-devops (Molecule AI · core-devops) core-fe (Molecule AI · core-fe) core-lead (Molecule AI · core-lead) core-offsec (Molecule AI · core-offsec) core-qa (Molecule AI · core-qa) core-security (Molecule AI · core-security) core-uiux (Molecule AI · core-uiux) cp-be (Molecule AI · cp-be) cp-lead (Molecule AI · cp-lead) cp-qa (Molecule AI · cp-qa) cp-security (Molecule AI · cp-security) cui (Zhanlin Cui) dev-lead (Molecule AI · dev-lead) devops-engineer documentation-specialist (Molecule AI · documentation-specialist) fullstack-engineer (Molecule AI · fullstack-engineer) godwin hongming hongming-ceo-delegated hongming-codex-laptop hongming-kimi-laptop hongming-pc2 hongming-personal infra-lead (Molecule AI · infra-lead) infra-runtime-be (Molecule AI · infra-runtime-be) infra-sre (Molecule AI · infra-sre) integration-tester (Molecule AI · integration-tester) mc-drift-bot molecule-code-reviewer plugin-dev (Molecule AI · plugin-dev) pm publish-runtime-bot pypi-publisher (Molecule AI PyPI Publisher (RFC#596)) release-manager (Molecule AI · release-manager) sdk-dev (Molecule AI · sdk-dev) sdk-lead (Molecule AI · sdk-lead) sop-drift-bot sop-tier-bot (SOP Tier-Check Bot) technical-writer (Molecule AI · technical-writer) triage-operator (Molecule AI · triage-operator)
No Assignees core-be
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: molecule-ai/molecule-core#228
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?