Skip to content

qa-test-003.md

Latest commit

 

History

History
29 lines (22 loc) · 2.13 KB

qa-test-003.md

File metadata and controls

29 lines (22 loc) · 2.13 KB

QA Test Report: AI Personal Finance Advisor (Issue 003)

Functional Tests

  • Webhook Connection: PASS. Tested locally via Meta Webhook UI.
  • User Creation: PASS. Sending a text message correctly provisions a new user with status: active and weekly_goal: 10000.
  • Expense Logging: PASS. Valid numbers extracted and stored in Supabase logs.
  • Daily Nudge Cron: PASS (Verified logic flow).
  • Weekly Summary Cron: PASS (Verified logic flow and N+1 query fix).

Edge Cases

  • Message without numbers ("I bought lunch"): PASS. Caught successfully. Replies with "I didn't catch an amount..."
  • Non-Text Messages (Images, Audio): PASS. Caught and prevented crash. Replies with "I'm just a simple text bot!"
  • Zero Spend ("I spent 0 today"): PASS. Caught explicitly and rewards user directly.

Failure Scenarios

  • Meta API Downtime: FAIL-SAFE. If Meta rejects the message, fetch throws/logs an error but the backend does not crash.
  • Supabase Downtime: FAIL-FAST. If DB is unreachable during an incoming webhook, the API returns a 500. Meta will automatically retry webhooks that do not return a 2xx response. Because we don't have idempotency keys, a retry might result in double logging if the DB comes back up. (Acceptable risk for MVP given Supabase uptime).

Performance Risks

  • Cron Timeout Limits: PASS. Sending messages concurrently using Promise.allSettled is now highly performant and won't sequentialize API latency. Memory limits are well above what is needed for the 20-50 user MVP bracket.
  • Database Query Limits: PASS. Consolidated the N+1 logs query into a single in() query.

UX Issues

  • Missing Onboarding Explanation: Currently, when a user first messages the bot, they get "Got it! Logged X" or "Didn't catch an amount". They don't actually get a welcome message explaining what the bot does or what time it will nudge them. This is slightly abrasive but technically functional.

Final QA Verdict

Pass

The system is highly resilient for an MVP. The local end-to-end tests confirm behavioral edge cases are padded. The application is ready to advance to Metrics Planning.