Release Testing Checklist - Thunderbird for Android

Purpose

Ensure consistent, repeatable pre-release validation across core areas: installation/upgrade, UI/UX, functionality regression, performance, connectivity, and security/privacy.
Provide a short “Smoke” run and a full “Release” run.

When to use

Test environments

Devices: At least one low-end and one mid/high-end device; 1 emulator
OS versions: Last 3 major Android versions supported + Lowest supported (API 21)
Network: Wi‑Fi, LTE/3G, Offline/Airplane; captive portal if available
Accounts: At least 2 IMAP accounts (different providers), 1 POP3 if supported; one account with many messages/folders
Locales: English + one non‑English (RTL if possible)
Themes: Light and Dark

Data prerequisites

Test mailboxes with folders: Inbox, Sent, Drafts, Trash, Spam, Archive, custom folders
A few threads with 50+ messages for performance/scroll tests
At least one account configured with push (if supported)

How to record results

Quick Smoke (15–25 min)

Install/Launch
- Fresh install: App installs and launches without crash
- First‑run: Onboarding visible, no ANRs
Account Setup (1 IMAP)
- Manual or auto‑config completes; inbox loads
UI Smoke
- Navigate: Account list → Inbox → Message → Back works
- Compose: Create draft, send to self, appears in Sent and then Inbox
Sync/Refresh
- Pull‑to‑refresh fetches new mail; background sync triggers at expected times
Notifications
- Receive mail notification; tap opens message; mark as read behavior correct
Basic Actions
- Read, star/flag, delete to Trash; undo/restore if offered
Performance quick check
- App cold start acceptable, scrolling in Inbox smooth

Top-level verification

Fresh Install — App installs and runs correctly on a fresh install
- Installs and launches without crash
- Permissions requested only when needed; rationale shown
- First‑run experience shows expected screens/toggles
Upgrade to Production — App upgrades correctly to the latest production version (only for releases)
- Upgrade to latest production build preserves accounts, settings, and local cache
- No duplicate notifications or migration errors on first launch
Upgrade to Beta — App upgrades correctly to the latest beta version
- Upgrade to latest beta build preserves accounts, settings, and local cache
- Feature flags and migrations behave correctly when upgrading to beta
- No duplicate notifications or migration errors on first launch
Backup/Restore
- Backup 1 account setting, delete account from app, restore from backup; account usable
- Backup all accounts, uninstall/reinstall, restore settings; accounts usable
Settings Import via QR code (only Thunderbird)
- From first-run flow, choose “Import settings”; scan a valid QR; settings are imported/applied or account is added/usable
- From Settings > Import, choose “Scan QR code”; scan a valid QR; settings are imported/applied or account is added/usable
- Try an invalid or expired QR; a clear error is shown and the app does not crash

Add accounts
- Auto‑config works for common providers; fallback to manual works
- Validation errors are clear (username/password/server settings)
Multiple accounts
- Switch accounts; titles/subtitles update correctly
- Unified Inbox visibility matches setting and number of accounts
Delete account
- Deleting current account transitions correctly (Unified Inbox if enabled and >1; else default account)
- No crash during or after deletion; capability checks safe
Special folders
- Drafts/Sent/Archive/Trash/Spam configured/created as needed

Receiving
- New message appears after push/refresh; badge/notification updates
Reading
- Open message; images loading policy obeyed; external images prompt if applicable
Compose/Reply/Forward
- From identity correct; quoted content; signatures; attachments; send succeeds
Drafts
- Auto‑save draft; edit and send later
Move/Copy
- Move/copy messages between folders; operation succeeds and reflects in UI
Search
- Local search by sender/subject/body; filters; results correct; remote search if supported
Attachments
- Download, open with external app, share; large attachment handling

Navigation
- Drawer/tabs/back behavior consistent; deep linking from notifications
Message List view
- Threaded/flat messages shown; selection mode; swipe actions (customizable to check archive/delete)
Message view
- Rendering of HTML/plain text; long messages; quoted text expansion; link handling
- Image loading per settings; attachment previews
Theming and localization
- Light/Dark theme correctness; typography and icons;
- Non‑English locale strings fit and are translated; RTL layout alignment
Accessibility
- TalkBack announcements; content descriptions present
- Focus order logical; dynamic type/Font scale respected

Startup performance
- Cold start time acceptable on low‑end device
Scrolling and list operations
- Smooth scrolling in Inbox with 1000+ items; no jank during load more
Sync performance
- Time to first sync for a fresh account reasonable; no repeated retries
Resource usage
- No abnormal CPU usage when idle; memory stable during long scrolling; cache does not balloon unexpectedly

Network transitions
- Wi‑Fi → Cellular → Offline: app handles gracefully; shows appropriate banners/status
Offline usage
- Read previously synced messages; queued actions (send/move) execute on reconnect

New mail
- Shows correct sender/subject; grouping for multiple messages
- Actions: Mark read, Delete, Archive function correctly and reflect in app
Quiet hours/Do Not Disturb
- Respected according to settings

General settings
- Unified Inbox toggle behavior: only shows when >1 accounts, correct fallback behavior
- Sort options persist per account/unified; changes reflected immediately
Account settings
- Outgoing/incoming server edits persist; re‑auth flows; OAuth if applicable

Permissions
- No unexpected permission prompts; revoking a permission shows clear error states
Sensitive logging
- Disabled by default; enabling shows warnings; logs redact personal data where possible
Certificates (if applicable)
- Certificate errors surfaced with clear options; pinning remembered

Check changes in this release
- Perform smoke tests around changed screens/use cases
Logs, crashes, and ANRs
- No new crashes/ANRs; logging remains appropriate and doesn’t expose sensitive data
Quick smoke around changed screens
- Basic open/close, refresh/sync, action buttons, and navigation work

K. Known Issues Verification

Validate that previously documented known issues still match behavior and are noted in release notes