Skip to content

fix(observability): inject TRA support-file listener synchronously (SDK-7121) - #1159

Closed
osho-20 wants to merge 3 commits into
masterfrom
fix/sdk-7121-observability-support-file-race
Closed

fix(observability): inject TRA support-file listener synchronously (SDK-7121)#1159
osho-20 wants to merge 3 commits into
masterfrom
fix/sdk-7121-observability-support-file-race

Conversation

@osho-20

@osho-20 osho-20 commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator

Summary

The new Automate dashboard (TRA / Test Observability) received no test events for Cypress runs on a customer's DEV pipeline, while the old Automate dashboard worked and the same suite/CLI/packages worked on another environment. Ref: SDK-7121.

Root cause

setEventListeners (bin/testObservability/helper/helper.js) injects the observability plugin into the user's Cypress support file:

require('browserstack-cypress-cli/bin/testObservability/cypress');

It did this inside an async glob(pattern, {}, cb) callback and returned immediately without awaiting it. In bin/commands/runs.js, the caller proceeds synchronously to md5 hashing (checkUploadedMd5) and zip archiving (archiver.archive) right after the call. So the injection raced the archive:

  • Race won (fast FS / UAT): instrumented suite is zipped → TRA works.
  • Race lost (slower CI / customer DEV): the archive reads the un-instrumented support file → remote runner has no o11y plugin → zero TRA events.

Md5 caching makes a lost race sticky: once a non-instrumented zip is cached under the original-content md5, every subsequent run logs "Skipping zip upload since BrowserStack already has your test suite..." and reuses the bad zip. The old Automate dashboard is unaffected because it does not depend on the injected plugin — exactly the reported asymmetry.

Reproduction

Calling setEventListeners and reading the support file synchronously afterwards (as md5/archive do):

before fix:  injected SYNCHRONOUSLY? false   (archive can ship un-instrumented suite)
after  fix:  injected SYNCHRONOUSLY? true

Fix

  • setEventListeners → use glob.sync so the support-file writes complete before the function returns.
  • Applied the same fix to the identical race in setAccessibilityEventListeners' glob-pattern branch (bin/accessibility-automation/helper.js).

Testing

  • New regression test test/unit/bin/testObservability/setEventListeners.js — asserts the observability require is present synchronously after the call. Fails on unfixed code, passes with the fix.
  • Existing accessibility-automation/cypress unit tests: green (6/6).
  • commands/runs.js unit file shows the same pre-existing standalone failures with and without this change (unrelated setUsageReportingFlag assertions).

Release

  • patch — bug fix
  • Release note: Fix Cypress Test Observability (new Automate dashboard) receiving no test events on some CI pipelines due to a race between support-file instrumentation and test-suite archiving/caching.

🤖 Generated with Claude Code

osho-20 and others added 3 commits July 27, 2026 18:46
… [SDK-7081]

getCiInfo() checked Jenkins first on mere presence of JENKINS_URL/
JENKINS_HOME, so a self-hosted GitHub Actions runner with leftover
Jenkins env vars was classified as Jenkins with build_url undefined.
obs-api force-disables the TRA Re-run control for builds without a CI
build_url, greying the button. Move the explicit GITHUB_ACTIONS check
ahead of the presence-only providers.

Also return after resolve({}) in getGitMetaData when no git directory
is found, instead of falling through.

Adds getCiInfo unit coverage: GHA detection, the Jenkins-shadowing
regression, pure Jenkins, and no-CI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…DK-7121)

setEventListeners deferred the support-file write to an async glob callback
while runs.js proceeded synchronously to md5 hashing and zip archiving. The
injection raced the archive: a lost race shipped an un-instrumented suite, and
md5 caching ("Skipping zip upload...") made the bad zip sticky, so the new
Automate dashboard (TRA) received zero test events while the old dashboard
(independent of the injected plugin) kept working. This is why the symptom was
environment-specific — a slower CI pipeline loses the race.

Switch setEventListeners (and the identical race in the accessibility
setAccessibilityEventListeners glob branch) to glob.sync so the writes complete
before the caller reads the files. Adds a regression test asserting the
observability require is present synchronously after the call returns.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@osho-20
osho-20 requested a review from a team as a code owner July 30, 2026 06:41
@osho-20
osho-20 requested review from anish353 and rounak610 July 30, 2026 06:41
@osho-20 osho-20 changed the title Fix/sdk 7121 observability support file race fix(observability): inject TRA support-file listener synchronously (SDK-7121) Jul 30, 2026
@osho-20

osho-20 commented Jul 30, 2026

Copy link
Copy Markdown
Collaborator Author

Superseded by #1160 — this PR carried an unrelated SDK-7081 delta because its branch was cut from pre_prod. #1160 is a clean 3-file diff against master.

@osho-20 osho-20 closed this Jul 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants