Functional Testing — Cycle 3

Date: 2026-04-16 Working directory: C:\Users\keen4\WxManBran\tools\tropical-update-publisher\build_v2\v1\tools\tropical-update-publisher

Summary

Metric Value
Total features 111
Tested this cycle 77
PASS 76
FAIL 0
BLOCKED 0
PARTIAL 1
REGRESSION 1
MISSING 34
UNTESTED 0
Health Score 98.7%
Delta vs Cycle 2 verify PASS 77 → 76 (-1), PARTIAL 0 → 1 (+1). One flaky-test regression on F084; all other PASS features re-verified clean.

Notes:

  • Phase 0 compile sweep:
    • npm run typecheck: PASS, 0 errors (refinement-state/functional-cycle-3-artifacts/typecheck.log)
    • npm run build (renderer + main): PASS, 0 errors (refinement-state/functional-cycle-3-artifacts/build.log)
    • npm run lint: PASS, 0 errors (refinement-state/functional-cycle-3-artifacts/lint.log)
    • Compile errors found: 0 — no fixes needed, proceeded to feature testing.
  • Full repo verification:
    • npm test run 1: 552/555 pass, 3 fail in tests/renderer/shadcn-smoke.test.tsx (dialog open/escape/focus-trap)
    • npm test run 2: 552/555 pass, same 3 failures in shadcn-smoke.test.tsx
    • npm test run 3: 553/555 pass, 2 failures (focus-trap case passed this time)
    • Isolated run of just shadcn-smoke.test.tsx: 16/16 PASS. Confirms these are animation-timing-flaky tests only failing under full-suite parallel load, not broken logic.
    • Evidence: refinement-state/functional-cycle-3-artifacts/test-run.log, test-run-2.log, test-run-3.log, shadcn-smoke-rerun-1.log
  • Direct service smoke checks (cycle-3 new evidence):
    • FileValidator.validateDocxFilename + parseFilename: 12 cases (valid new, valid new w/o time, minutes, invest ID, legacy auto-convert x2, invalid slug, invalid month, invalid ext, empty, Feb 30, path separator) all return expected ok/code/message; timezone label rotates correctly through ET/CT/UTC on same filename. Evidence: refinement-state/test-fixtures/cycle-3/file-validator-smoke.json
    • determineFolderName: 8 cases (named-only, TD-only, PTC-only, invest-only, named-beats-ptc, td-beats-ptc, empty, mismatched stormNumber) all produce spec-correct targetFolderName + appliedRuleId. Priority Named > TD > PTC > Invest verified. Evidence: refinement-state/test-fixtures/cycle-3/folder-name-smoke.json
  • Missing-feature inventory unchanged from cycle 2:
    • Launchers (launcher.bat, launcher.vbs): still absent (F008)
    • assets/ directory with WMB_Logo.png, icon.ico: still absent (F009)
    • No src/main/services/{nhc,ai,email} directories (F093, F097, F100 and dependents still MISSING)
    • No renderer file-queue/commit-message/status-log/footer components beyond drop zone (F030–F035 still MISSING)
    • Renderer features/publish/ still only has DropZone.tsx + dropZoneModel.ts (no queue UI)

Detailed FAIL List

No FAIL entries in this cycle. The shadcn smoke flakiness is tracked as PARTIAL / REGRESSION on F084 rather than FAIL because the underlying dialog behavior passes reliably in isolation.

Partial Features

MEDIUM — F084 Sidebar, Button, Dialog, And Input Accessibility/Polish — REGRESSION vs. Cycle 2

  • Status: PARTIAL (new; was PASS in cycle 2 verify)
  • What works:
    • Accessible buttons, inputs, sidebar focus ring, dialog role, and icon-only button labeling all still render correctly.
    • tests/renderer/shadcn-smoke.test.tsx PASSES 16/16 when run in isolation (refinement-state/functional-cycle-3-artifacts/shadcn-smoke-rerun-1.log).
  • What is still missing:
    • In the full npm test run, 2–3 of the dialog tests flake intermittently:
      • App + Dialog smoke > opens the dialog and shows the titled input field (happy path) (line 88–96): fails with toBeVisible() because Radix sets data-state="open" but the fade-in transition has not completed at assertion time.
      • App + Dialog smoke > closes the dialog when Escape is pressed (Radix modal contract) (line 119–129): findByRole('dialog') times out because the dialog portal has not rendered by the deadline under parallel load.
      • App + Dialog smoke > keeps keyboard focus inside the dialog while open (focus trap / error path: focus must not leak) (line 131–148): identical flake to (1) — dialog element present but not yet visible when assertion runs.
    • Cycle 2 reported this test as “stabilized” after fixing the focus-trap case; cycle 3 sees the same case regress, plus two sibling tests showing the same pattern. Only reproduces under full-suite parallel run — likely timer/animation interaction with other tests loaded in the same thread pool.
  • Evidence:
    • Failing (full suite): refinement-state/functional-cycle-3-artifacts/test-run.log, test-run-2.log, test-run-3.log
    • Passing (isolated): refinement-state/functional-cycle-3-artifacts/shadcn-smoke-rerun-1.log
  • Root cause (best read): Framer-motion / Radix Dialog animation duration + fireEvent.click (sync) vs. findByRole’s default 1 s timeout. The cycle-2 stabilization used userEvent.click + extra waitFor; the open/escape cases still use fireEvent.click and hit the visibility assertion before Radix finishes the data-state="open" → visible transition under parallel load.
  • Priority: MEDIUM (test-suite stability only — no user-visible regression; dialog still renders correctly in the live app and in isolated test runs).

Blocked Features

None in this cycle.

Regressions

F084 — Dialog smoke tests now flaky under full-suite parallel run

  • Previous cycle: Cycle 2 reported “Stabilized the flaky dialog smoke test” and showed 555/555 green across three consecutive runs.
  • Current cycle: 2–3 of the 16 shadcn-smoke tests fail when run as part of npm test. Isolated rerun is 16/16 green.
  • Impact: Master-list health dropped by one feature (F084 PASS → PARTIAL). No functional regression in the Electron renderer.
  • Suggested remediation (not applied this cycle — out of functional-testing scope):
    • Replace fireEvent.click with userEvent.click in the three affected tests and wrap assertions in waitFor with an explicit timeout, mirroring the already-stabilized focus-trap case from cycle 2 which has now drifted back out of the “stable” pattern.
    • OR pin the motion-reduce preference during test setup to eliminate the animation-window race entirely.

New Discoveries

  • FileValidator.parseFilename correctly honors the timezoneLabel option (confirmed ET/CT/UTC all produce the matching suffix on 12:00 PM). This reconfirms F038 R4 compliance.
  • determineFolderName returns structured code + appliedRuleId for empty and mismatched inputs, which is useful diagnostic metadata for UI surfacing once F045 (Quick Browse UI) lands.
  • Sub-test file counts: 52 test files, 555 test cases. No new files vs. cycle 2.
  • Changed files since cycle 2 (find -newer refinement-functional-cycle-2.md):
    • src/main/ipc/handlers/utilityHandlers.ts, src/main/ipc/router.ts, src/main/services/config/ConfigService.ts, src/main/services/firstRun/FirstRunGate.ts, src/preload/index.ts, src/shared/ipc/invokeErrors.ts, src/shared/types/ipc.ts
    • tests/main/configService.test.ts, tests/main/firstRunGate.test.ts, tests/main/ipcRouter.test.ts, tests/main/utilityHandlers.test.ts, tests/preload/apiShape.test.ts
    • tests/renderer/windowApiMock.ts, tests/renderer/firstRunModal.test.tsx, src/renderer/features/first-run/FirstRunModal.tsx
    • These correspond to implement-cycle-3 and implement-cycle-4 fixes (staged-conflict guard, settings:update unknown-key rejection, storm-root verification, auto-stash --include-untracked, save-time schema rejection, shell:openPath validation). All shipped with matching unit tests; all pass in full suite. None of these directly touch shadcn-smoke.test.tsx, so the flake is likely a side effect of additional test-setup work raising wall-clock noise under the thread pool.

Remaining Missing Features (unchanged from cycle 2)

Implementation gaps, not test blockers:

  • F008–F009: launchers + assets directory
  • F030–F035: full publish queue / commit-message / YouTube UI / status log / footer destination / existing-files listing
  • F039: document-creator renderer section
  • F045: quick-browse renderer section
  • F050: header logo + git-status indicator widget
  • F087–F090: command palette, shortcut wiring, context menus, designed empty states
  • F093–F111: Phase 2 NHC monitor, AI backends, email review, draft queue, rich dashboard, historical views, notifications, QoL features, audit/compliance enhancements

Evidence Highlights

  • Compile sweep: refinement-state/functional-cycle-3-artifacts/typecheck.log, build.log, lint.log
  • Full suite runs: refinement-state/functional-cycle-3-artifacts/test-run.log, test-run-2.log, test-run-3.log
  • Shadcn smoke isolated rerun: refinement-state/functional-cycle-3-artifacts/shadcn-smoke-rerun-1.log
  • FileValidator smoke: refinement-state/test-fixtures/cycle-3/file-validator-smoke.json (generator: smoke-file-validator.js)
  • determineFolderName smoke: refinement-state/test-fixtures/cycle-3/folder-name-smoke.json (generator: smoke-folder-name.js)

Git Safety

No live git-publish E2E was re-executed in this cycle — F029 coverage is derivative on the static test suite (publishService.test.ts, gitService.mocks.test.ts, publishHistoryService.test.ts) all passing in npm test, plus the cycle-2 disposable-branch E2E artifact remains the authoritative live evidence. No git tests touched the live main branch; no test branches were created or left behind this cycle.