Browse-test failures — team brief
2026-05-22 · run #26236942007 · target .raklet.net

What this is: deep dive on each of the 18 failures from the first full browse-smoke matrix run after the test-env deploy, with current status and concrete next steps per test.
Who should read: anyone owning browse-tests, payment-regression, or the test fixture orgs.
Author: Claude session, validated each diagnosis live via $B against general-test-account.raklet.net.

Scoreboard

18
failures in run #26236942007
9
addressed across PR #13851 + #13854 (merged or open)
9
remaining (3 categories below)
1
blocked on a product bug (RAK-349)
TestSuiteStatusOwner
09-contact-subscriptions-crudPMFIXED in #13851
11-contact-payments-crudPMFIXED in #13851
12-contact-subscription-remindersPMFIXED in #13851
14-contact-donations-crudPMFIXED in #13851
28-social-settings-mutationARFIXED in #13851
34-messages-email-draft-lifecycleARFIXED in #13851
36-messages-sender-identitiesARFIXED in #13851
24-settings-fields-mutationARFIXED in #13854 (open)
25-events-clone-online-eventARFIXED in #13854 (open)
26-events-clone-venue-eventARFIXED in #13854 (open)
22-application-form-create-custom-fieldARTest authorsee § Category A
28-directory-form-requirementsARTest author + manual cleanupsee § Category A
35-messages-templateARTest authorsee § Category B
06-fundraising-campaign-crudPMNeeds Stripe-sandbox passwordsee § Category C
07-fundraising-admin-donation-crudPMNeeds Stripe-sandbox passwordsee § Category C
15-application-form-invalid-email…PMNeeds Stripe-sandbox passwordsee § Category C
16-application-form-auto-approve-quick-registerPMProduct fix RAK-349see § Category D

Open sections

  1. Category A — Test-org data accumulation (AR 22 / 24 / 28-directory)
  2. Category B — Test design / infrastructure (AR 25 / 26 / 35)
  3. Category C — Sandbox credentials needed (PM 06 / 07 / 15)
  4. Category D — Product bug (PM 16)
  5. Team recommendations

Category A — Test-org data accumulation

These look like "selector drift" in the failure messages but the screenshots reveal a different story: the general-test-account org on .raklet.net has accumulated test fixtures across many runs and is hitting plan limits / pagination windows. Cleanup helpers either don't run or don't cover everything that was created.

AR 28 — 28-directory-form-requirements

Test author + manual cleanup
Failure
Timed out after 60000ms waiting for selector to become visible: input[ng-model='Data.Directory.Name'], input[name='Name']
Screenshot
Modal appears with text "Need More Directories? You have reached the 10 directories limit on your Premium-50K plan. You can upgrade your plan to unlock 100 directories…"
Live check
Confirmed via $B — exactly 10 directories on the org, all named amk-dir-search-* / am6-dir-* (test leftovers).

Why cleanup didn't catch up

Remove-DirectoriesByNamePrefix in browse-tests/helpers/directory.ps1 matches the current run's prefix only (am6-dir-form-…). Earlier runs used different prefixes (am6-dir-search-…, amk-dir-search-…) so the cleanup never removes them. Each run leaves all earlier-prefix orphans behind, ratcheting up the count until the plan limit hits.

Code path

Start-DirectoryCreate clicks a[ng-click*="checkForLimitAndRedirect"]. That handler checks orgs.RemainingDirectoryCount server-side and, when zero, opens the upgrade modal instead of navigating to the create form. The test then waits 60s for an input that never appears.

Fix options

  1. One-time manual cleanup: log into general-test-account.raklet.net and delete the 10 test directories. Tests pass once. Will regress when the next test fails before its own cleanup.
  2. Broaden the cleanup prefix: nuke any directory matching am[3-9]-dir-* / amk-dir-* / browse-dir-*. Lossy but self-healing.
  3. Setup-step cleanup: add a pre-test "ensure under N directories" step that prunes the oldest test directories until the count is within budget.
  4. Detect-and-skip: when the upgrade modal appears, dismiss it and exit 0 # SKIP with a clear message.

Recommendation

(2) + (3) combined. Broader cleanup is correct; setup-step makes it idempotent.

AR 22 — 22-application-form-create-custom-field

Test author
Failure
page.evaluate: Error: custom field missing — the test creates a custom field on the application form, then asserts it appears in the row repeater for ApplicationFormSettings.Fields.
Screenshot
Shows the Billing page (Subscription / Invoices / Payouts / Settings tabs) with a spinner — NOT the application form fields. The test got redirected away from where it was looking.
Probable root cause
Same plan-limit family as AR 28. The org has accumulated test fields (settings/fields lists 174 rows of custom fields) and either the application form has its own field limit, or some related entity hit a limit that triggered a billing-redirect.

What's needed

Re-run a focused dispatch with the JS error captured in full + walk through the create flow in $B to see exactly where the billing redirect comes from. Not isolated yet.

AR 24 — 24-settings-fields-mutation

FIXED in PR #13854
Failure
field not listed: am3-field-browse20260521161410-add-update
Screenshot
A very long list of custom fields (most of them named am3-field-browse*…, browse-field-*, etc.). The page is paginated/scrollable and the search finds nothing because the new field is appended at the bottom of a 174-row list.
Live check
Confirmed via $Bsettings/fields/ on general-test-account shows 174 custom-field repeats.

Fix shipped (PR #13854)

Switched the assertion to read the field from AngularJS scope ($rootScope walk to Data.Fields then match by Name) instead of DOM row scan — same pattern as my 25-tags fix in #13844.


Category B — Test design / infrastructure

These are test architecture issues that can't be fixed with a selector tweak.

AR 25 + AR 26 — events-clone-{online,venue}-event

FIXED in PR #13854
Failure
Event 'browse-clone-online-20260521162315-47396316' not found in events list
Live check
  • Events list is server-side paginated to 9 items in Data.EventsFuture.Data.
  • Test's New-EventFormData defaulted to StartDate = 01/01/2021 and EndDate = 03/03/2027 — a 2021-start event sorts below the visible window after years of test accumulation.
  • The previous Wait-EventVisibleInList clicked Load More once before throwing; filtered view of browse-clone-online already has 27+ leftover events.
  • Test's ?name=… URL filter does work server-side but returns only the first 9 matches.

Fix shipped (PR #13854)

  1. Today as default StartDate in New-EventFormData. New events sort to the top of the default list. No test assertion checks the date, so this is safe.
  2. Loop Load More aggressively in Wait-EventVisibleInList (up to 25 iterations per attempt, re-navigate between outer attempts). Verified live: filtered view of browse-clone-online goes 9 → 11 on one Load More click; repeated clicks reveal all matching events.

Followup for the team

Once #13854 lands and the next dispatch confirms the fix, please prune accumulated browse-clone-* events on the org (currently 27+ from prior failed runs) so Load More iterations stay short.

AR 35 — 35-messages-template

Test author
Failure
Draft email 'am6-msg-tpl-browse20260521171158 Email Template Title Here' was not listed. (PR #13851's poll-with-reload helps timing races; this is NOT a timing race — the title is never persisted)
Screenshot
The drafts list shows two newly-created rows with subject "Empty" at 05:09 PM and 05:10 PM (matches the test run window). The templates page redirected to drafts after save, and the row is there but with no title.

Root cause

New-EmailTemplate in browse-tests/helpers/messages.ps1 uses Invoke-Browse fill input[ng-model='Email.newEmailName'] then saveStep('name') + saveEmail. The fill IS using Invoke-Browse fill (which dispatches events correctly — unlike the contact-add bug already fixed in #13851), but the save persists with no name.

Likely sub-causes

Fix options

  1. $B walkthrough of New-EmailTemplate step by step, capturing modal state + Angular scope at each step (same pattern that found the contact-add bug). Investigation needs an end-to-end re-run after each tweak.
  2. Replace fill+saveStep with an Angular service call via injector: messagesService.createTemplate({Name: title, …}).

Recommendation

(1). Likely reveals another ng-model-event-not-firing bug, fixable with the same setVal shim pattern.


Category C — Sandbox credentials needed (PM 06 / 07 / 15)

These require a Stripe sandbox sub-org. I cannot validate fixes without the password.

Awaiting CEO

Credentials surfaced in Test.Extensions/Config.cs:

StandartPassword = "rakletrocks2022#" is the convention for raklet.test+*@gmail.com fixture accounts but I haven't confirmed it for this specific account. Actual CI values live in GH secrets BROWSE_TEST_STRIPE_ACCOUNT / BROWSE_TEST_STRIPE_ACCOUNT_EMAIL — write-only.

What I need: the password for that fixture user. With it I'll add the trio to browse-tests/local.settings.json (gitignored) and drive each failing flow via $B.

PM 06 — 06-fundraising-campaign-crud

Needs password
Failure
browse failed: fill #Goal 25 — Element not found or not interactable within timeout
Test flow
06-fundraising-campaign-crud.ps1 lines 90-95: navigates to /manager/donations/campaigns/, clicks Add Campaign, waits for #Name, fills #Name, then fills #Goal — and #Goal doesn't render in time.

Most likely root causes

PM 07 — 07-fundraising-admin-donation-crud

Needs password
Failure
browse failed: click div[id*='personSearch'] .ui-select-match, div.modal-content [ng-click*='select.activate'] — Selector matched multiple elements
Screenshot
"Add Donation" modal with Member, Amount, Payment Method, Reference Number, Campaign, Payment Date fields. The Member field is empty.

Root cause

The modal has multiple ui-select widgets (likely both Member AND Campaign use ui-select). Both match the broad fallback div.modal-content [ng-click*='select.activate'].

Likely fix (can be written blind, low risk)

Narrow to .modal.in div[id^='personSearch'] .ui-select-match (exact id^= and visible-modal-only). Confidence is high but needs a sandbox-credentialed re-run to confirm.

PM 15 — 15-application-form-invalid-email-stripe-standard-destination

Needs password
Failure
Timed out after 30000ms waiting for selector to become visible: #memberEmailMessage, .form-group.text-danger
Code path
helpers/application-form-payment.ps1 drives a public application form with a duplicate email, expects an inline error at #memberEmailMessage or .form-group.text-danger.

Most likely root cause

Error display selector drift on the public apply page. The error might now render in .invalid-feedback, [class*='Error'], or via a growl instead of inline.


Category D — Product bug (PM 16)

PM 16 — 16-application-form-auto-approve-quick-register

Blocked on RAK-349
Failure
Auto-approve application '…' did not appear in /manager/membership/applications/ within 60s.
Linear
RAK-349 — apply form required-field validation regression. The submitted application either never completes or never reaches the applications inbox.

This is a real product bug

The test is correctly catching a real product regression. Per memory the issue is High priority. Once the RAK-349 fix ships, this test should pass without any test-side change.


Team recommendations

  1. One-time data cleanup on general-test-account.raklet.net: prune accumulated test directories (10/10 against plan limit), application-form custom fields, settings custom fields (174+), and old test events (browse-clone-*, am[3-9]-* prefixes). This unblocks AR 22 / 24 / 28-directory immediately and gives a clean baseline.
  2. Adopt the broader-prefix cleanup pattern in Remove-DirectoriesByNamePrefix and the equivalent helpers in directory.ps1, messages.ps1, events.ps1. Match any am[3-9]-*, amk-*, browse-*, am6-* test prefix — not just the current run's prefix.
  3. PR #13854 covers AR 24 (scope-walk) + AR 25/26 (events-clone date + Load More loop) — please review.
  4. Prioritize RAK-349 — it's blocking a meaningful payment-regression test.
  5. Share the Stripe sandbox password so I can finish PM 06 / 07 / 15 on a follow-up branch.
  6. Long-term test design: consider rebuilding the most-accumulation-prone fixtures into ephemeral sub-orgs per CI run (or at least per day) so the shared general-test-account org doesn't keep ratcheting up plan-limit pressure.

Appendix — Validation method

Every fix in PR #13851 and PR #13854 was validated live by logging into general-test-account.raklet.net via $B (the gstack browse daemon) and walking through the failing flow manually before pushing the fix. No blind helper fixes per the existing feedback_no_blind_helper_fixes memory rule (broke tests 1-5 last time).

Sample probe (AngularJS scope-walk used in PR #13854):

(function(){
  var injector = angular.element(document.body).injector();
  if (!injector) {
    var els = document.querySelectorAll('[ng-app],[data-ng-app]');
    if (els.length) injector = angular.element(els[0]).injector();
  }
  var rs = injector.get('$rootScope');
  function walk(s, depth) {
    if (!s || depth > 10) return null;
    if (s.Data && s.Data.Fields) return s;
    var c = s.$$childHead;
    while (c) {
      var r = walk(c, depth + 1);
      if (r) return r;
      c = c.$$nextSibling;
    }
    return null;
  }
  var scope = walk(rs, 0);
  return scope ? scope.Data.Fields.length : 'not found';
})()