153 lines
8.5 KiB
Markdown
153 lines
8.5 KiB
Markdown
# DocFast QA Test Results - February 14, 2026
|
||
|
||
## CRITICAL ISSUE FOUND
|
||
|
||
### 🚨 Browser Signup Flow Hangs
|
||
**Severity:** HIGH
|
||
**What I tested:** Clicked "Get Free API Key" button → filled email → clicked submit button
|
||
**Expected:** API key displays on page
|
||
**Actual:** Form submission hangs indefinitely, never returns API key
|
||
**Impact:** Users cannot sign up through the website interface
|
||
|
||
**Details:**
|
||
- Free signup button found and clickable ✅
|
||
- Email input field present and functional ✅
|
||
- Submit button found and clickable ✅
|
||
- Form submission **HANGS** - never completes ❌
|
||
- Test emails used: qa-test-1771089216449@example.com, qa-test-1771089267524@example.com
|
||
- Browser: Playwright/Chromium
|
||
- No console errors detected during page load
|
||
|
||
**Workaround:** Direct API call works fine: `POST /v1/signup/free`
|
||
|
||
---
|
||
|
||
## ALL OTHER TESTS PASSED ✅
|
||
|
||
### Browser Tests
|
||
- ✅ **Page Load:** https://docfast.dev loads with zero console errors
|
||
- ✅ **Pro Checkout:** "Get Started" button redirects to Stripe checkout successfully
|
||
- URL: https://checkout.stripe.com/c/pay/cs_live_a1k5WSEbRffDzpO7CjRSZqhAwl8uJUSAHtnuvIGH33LIC5lrOEr19gJpmX
|
||
|
||
### API Tests
|
||
- ✅ **Direct Signup:** `POST /v1/signup/free` returns valid API key instantly
|
||
- Test key: `df_free_538b4086765c6fdc68e77071ade8c67641cdabebdb9a399f`
|
||
- ✅ **HTML to PDF:** Generated valid 7149-byte PDF from `<h1>Test</h1>`
|
||
- ✅ **Documentation:** `/docs` endpoint returns comprehensive, real documentation with examples
|
||
- ✅ **Error Handling:**
|
||
- Bad API key: `{"error":"Invalid API key"}` ✅
|
||
- Missing html param: `{"error":"Missing 'html' field"}` ✅
|
||
- Wrong content-type: `{"error":"Unsupported Content-Type. Use application/json."}` ✅
|
||
|
||
### Security Verification (All Fixed Correctly)
|
||
- ✅ **CORS on Signup:** `Access-Control-Allow-Origin: https://docfast.dev` (NOT "*") - SECURE
|
||
- ✅ **CORS on API:** `Access-Control-Allow-Origin: *` (allows public API access) - CORRECT
|
||
- ✅ **SSRF Protection:** `{"error":"URL resolves to private/reserved IP"}` when testing 169.254.169.254 - BLOCKED
|
||
- ✅ **Stripe Webhook Forgery:** `{"error":"Missing webhook secret or signature"}` - PROTECTED
|
||
- ✅ **Security Headers:** Comprehensive CSP, HSTS, X-Frame-Options, etc.
|
||
|
||
### Response Headers Analysis
|
||
- Content-Security-Policy: Properly restrictive ✅
|
||
- Strict-Transport-Security: 1 year max-age with subdomains ✅
|
||
- X-Content-Type-Options: nosniff ✅
|
||
- X-Frame-Options: SAMEORIGIN ✅
|
||
- Rate limiting headers present ✅
|
||
|
||
## Summary
|
||
**1 Critical Issue:** Browser signup form hangs (while API signup works)
|
||
**12 Security Tests:** ALL PASSED
|
||
**Core Functionality:** API works perfectly
|
||
**Documentation Quality:** Excellent, comprehensive examples
|
||
|
||
## Recommendation
|
||
**URGENT:** Fix the browser signup form JavaScript issue. The backend works fine, so this is likely a frontend form submission or error handling bug preventing the API key from displaying after successful creation.
|
||
### BUG-012: Email signup for free tier serves no purpose
|
||
- **Found by:** Human (investor)
|
||
- **Date:** 2026-02-14
|
||
- **Severity:** MEDIUM (product design)
|
||
- **Description:** Free tier requires email but it's never verified. Either verify it (send confirmation email with the key) or remove the requirement. Collecting unverified emails is pointless and adds friction. Consider: if we verify, we have a real contact list for marketing. If we don't need email, just give the key instantly without asking.
|
||
- **Status:** Open — needs product decision from CEO
|
||
|
||
### BUG-013: Pro users — how do they get their API key?
|
||
- **Found by:** Human (investor)
|
||
- **Date:** 2026-02-14
|
||
- **Severity:** HIGH (broken flow)
|
||
- **Description:** After a Pro user pays via Stripe checkout, how do they receive their API key? Is it shown on the success page? Emailed? This flow needs to be clear and tested end-to-end: pay → get key → use key.
|
||
- **Status:** Open
|
||
|
||
### BUG-014: No way to recover or reset API key
|
||
- **Found by:** Human (investor)
|
||
- **Date:** 2026-02-14
|
||
- **Severity:** HIGH (missing feature)
|
||
- **Description:** If a user loses their API key, there's no way to get it again or reset it. Need a key recovery/reset mechanism — e.g. enter your email → get a new key (if email is verified), or a dashboard where users can see/rotate their key.
|
||
- **Status:** Open
|
||
|
||
### BUG-015: Need database instead of file-based storage
|
||
- **Found by:** Human (investor)
|
||
- **Date:** 2026-02-14
|
||
- **Severity:** HIGH (scalability)
|
||
- **Description:** Current storage is file-based JSON. Need a proper database (SQLite minimum, PostgreSQL for scale) for users, API keys, usage tracking. File-based won't survive concurrent requests or growth.
|
||
- **Status:** Open
|
||
|
||
### BUG-016: No production data backup strategy
|
||
- **Found by:** Human (investor)
|
||
- **Date:** 2026-02-14
|
||
- **Severity:** HIGH (data safety)
|
||
- **Description:** No backup mechanism for production data. If the server dies, all user data is lost. Need automated backups (daily minimum) to object storage or similar.
|
||
- **Status:** Open
|
||
|
||
### BUG-017: Unknown request throughput — need benchmarking
|
||
- **Found by:** Human (investor)
|
||
- **Date:** 2026-02-14
|
||
- **Severity:** MEDIUM (capacity planning)
|
||
- **Description:** We don't know how many requests/second the server can handle. Need load testing to determine real capacity, especially for PDF generation (Puppeteer is heavy). This informs pricing tiers and rate limits.
|
||
- **Status:** Open
|
||
|
||
### BUG-018: Pro account rate limits not data-backed
|
||
- **Found by:** Human (investor)
|
||
- **Date:** 2026-02-14
|
||
- **Severity:** HIGH (pricing integrity)
|
||
- **Description:** Pro tier promises 10,000 PDFs/month but rate limits need to be based on actual server capacity (from BUG-017 benchmarking). Can't promise what we can't deliver.
|
||
- **Status:** Open — depends on BUG-017
|
||
|
||
### BUG-019: Custom templates — feature or lie?
|
||
- **Found by:** Human (investor)
|
||
- **Date:** 2026-02-14
|
||
- **Severity:** HIGH (trust)
|
||
- **Description:** Pro plan landing page lists "Custom templates" as a feature but there's no way to upload or create custom templates. Either build the feature or remove the claim. Research what competitors offer for custom templates before deciding.
|
||
- **Status:** Open — CEO needs to research competitors and decide
|
||
|
||
---
|
||
|
||
## QA Run — 2026-02-14 17:29 UTC (Post-Merge Validation)
|
||
|
||
**Context:** UI/UX dev + backend dev simultaneous changes. Testing for merge conflicts and regressions.
|
||
|
||
### ✅ ALL 12 TESTS PASSED
|
||
|
||
| # | Test | Result |
|
||
|---|------|--------|
|
||
| 1 | Page load — zero console errors | ✅ PASS (0 errors) |
|
||
| 2 | Signup flow — no email, instant key | ✅ PASS (modal → Generate → key displayed with save warning + copy btn) |
|
||
| 3 | Pro checkout → Stripe | ✅ PASS (redirects to checkout.stripe.com) |
|
||
| 4 | Desktop visual quality | ✅ PASS (professional, polished, no layout issues) |
|
||
| 5 | Mobile responsiveness (375×812) | ✅ PASS (proper single-column, no overflow) |
|
||
| 6 | API signup with empty body | ✅ PASS (returns df_free_* key) |
|
||
| 7 | HTML→PDF conversion | ✅ PASS (200, application/pdf) |
|
||
| 8 | PDF validity | ✅ PASS (8109 bytes, PDF 1.4, 1 page) |
|
||
| 9 | /docs page | ✅ PASS (HTTP 200) |
|
||
| 10 | Error handling (bad key + missing params) | ✅ PASS (proper error messages) |
|
||
| 11 | CORS — evil.com blocked | ✅ PASS (Access-Control-Allow-Origin: https://docfast.dev only) |
|
||
| 12 | SSRF — metadata endpoint blocked | ✅ PASS ("URL resolves to private/reserved IP") |
|
||
|
||
### 📝 Notes
|
||
- **BUG-012 fix confirmed:** No email form. Two-click flow: "Get Free API Key" opens modal → "Generate API Key →" creates key instantly.
|
||
- **No merge conflicts detected:** Both devs' changes appear cleanly integrated.
|
||
- **Signup flow UX note (not a bug):** The landing page button says "Get Free API Key" but opens a modal with another button "Generate API Key →". This is a 2-click flow, not instant. Acceptable UX but worth noting — the task spec said "instantly request a key" which implies 1 click.
|
||
|
||
### BUG-020: Free tier too generous and no accountability
|
||
- **Found by:** Human (investor)
|
||
- **Date:** 2026-02-14
|
||
- **Severity:** HIGH (business model risk)
|
||
- **Description:** Free keys with no email = no accountability. 4 keys/IP/hour × 100 PDFs each = 400 free PDFs/IP/hour. Anyone can abuse this with zero consequences. Need: (1) Require email + verification (proves real person, gives us a contact for marketing/upsell), (2) One key per verified email, (3) Much stricter rate limiting. Free tier should be enough to evaluate the product, not enough to run a business on.
|
||
- **Status:** Open — CEO must redesign free tier signup flow
|