🧪 Testing Guide

This document outlines the testing practices and guidelines for the Thunderbird for Android project.

Key Testing Principles:

Follow the Arrange-Act-Assert (AAA) pattern
Use descriptive test names
Prefer fake implementations over mocks
Name the object under test as testSubject
Use AssertK for assertions

🏗️ Test Structure

🔄 Arrange-Act-Assert Pattern

Tests in this project should follow the Arrange-Act-Assert (AAA) pattern:

Arrange: Set up the test conditions and inputs
Act: Perform the action being tested
Assert: Verify the expected outcomes

Example:

@Test
fun `example test using AAA pattern`() {
    // Arrange
    val input = "test input"
    val expectedOutput = "expected result"
    val testSubject = SystemUnderTest()

    // Act
    val result = testSubject.processInput(input)

    // Assert
    assertThat(result).isEqualTo(expectedOutput)
}

Use comments to clearly separate these sections in your tests:

// Arrange
// Act
// Assert

📝 Test Naming

Use descriptive test names that clearly indicate what is being tested. For JVM tests, use backticks:

@Test
fun `method should return expected result when given valid input`() {
    // Test implementation
}

Note: Android instrumentation tests do not support backticks in test names. For these tests, use camelCase instead:

@Test
fun methodShouldReturnExpectedResultWhenGivenValidInput() {
    // Test implementation
}

💻 Test Implementation

🎭 Fakes over Mocks

In this project, we prefer using fake implementations over mocks:

✅ Preferred: Create fake/test implementations of interfaces or classes
❌ Avoid: Using mocking libraries to create mock objects

Fakes provide better test reliability and are more maintainable in the long run. They also make tests more readable and less prone to breaking when implementation details change.

Mocks can lead to brittle tests that are tightly coupled to the implementation details, making them harder to maintain. They also negatively impact test performance, particularly during test initialization. Which can quickly become overwhelming when an excessive number of tests includes mock implementations.

Example of a fake implementation:

// Interface
interface DataRepository {
    fun getData(): List<String>
}

// Fake implementation for testing
class FakeDataRepository(
    // Allow passing initial data during construction
    initialData: List<String> = emptyList()
) : DataRepository {
    // Mutable property to allow changing data between tests
    var dataToReturn = initialData

    override fun getData(): List<String> {
        return dataToReturn
    }
}

// In test
@Test
fun `processor should transform data correctly`() {
    // Arrange
    val fakeRepo = FakeDataRepository(listOf("item1", "item2"))
    val testSubject = DataProcessor(fakeRepo)

    // Act
    val result = testSubject.process()

    // Assert
    assertThat(result).containsExactly("ITEM1", "ITEM2")
}

📋 Naming Conventions

When writing tests, use the following naming conventions:

Name the object under test as testSubject (not “sut” or other abbreviations)
Name fake implementations with a “Fake” prefix (e.g., FakeDataRepository)
Use descriptive variable names that clearly indicate their purpose

✅ Assertions

Use AssertK for assertions in tests:

@Test
fun `example test`() {
    // Arrange
    val list = listOf("apple", "banana")

    // Act
    val result = list.contains("apple")

    // Assert
    assertThat(result).isTrue()
    assertThat(list).contains("banana")
}

Note: You’ll need to import the appropriate AssertK assertions:

assertk.assertThat for the base assertion function
Functions from the assertk.assertions namespace for specific assertion types (e.g., import assertk.assertions.isEqualTo, import assertk.assertions.contains, import assertk.assertions.isTrue, etc.)

🧩 Test Types

This section describes the different types of tests we use in the project. Each type serves a specific purpose in our testing strategy, and together they help ensure the quality and reliability of our codebase.

🔬 Unit Tests

Unit tests verify that individual components work correctly in isolation.

What to Test:

Single units of functionality
Individual methods or functions
Classes in isolation
Business logic
Edge cases and error handling

Key Characteristics:

Independent (no external dependencies)
No reliance on external resources
Uses fake implementations for dependencies

Frameworks:

JUnit 4
AssertK for assertions
Robolectric (for Android framework classes)

Location:

Tests should be in the same module as the code being tested
Should be in the src/test directory or src/{platformTarget}Test for Kotlin Multiplatform
Tests should be in the same package as the code being tested

Contributor Expectations:

✅ All new code should be covered by unit tests
✅ Add tests that reproduce bugs when fixing issues
✅ Follow the AAA pattern (Arrange-Act-Assert)
✅ Use descriptive test names
✅ Prefer fake implementations over mocks

🔌 Integration Tests

Integration tests verify that components work correctly together.

What to Test:

Interactions between components
Communication between layers
Data flow across multiple units
Component integration points

Key Characteristics:

Tests multiple components together
May use real implementations when appropriate
Focuses on component boundaries

Frameworks:

JUnit 4 (for tests in src/test)
AssertK for assertions
Robolectric (for Android framework classes in src/test)
Espresso (for UI testing in src/androidTest)

Location:

Preferably in the src/test or src/commonTest, src/{platformTarget}Test for Kotlin Multiplatform
Only use src/androidTest, when there’s a specific need for Android dependencies

Why prefer test over androidTest:

JUnit tests run faster (on JVM instead of emulator/device)
Easier to set up and maintain
Better integration with CI/CD pipelines
Lower resource requirements
Faster feedback during development

When to use androidTest:

When testing functionality that depends on Android-specific APIs that are not available with Robolectric
When tests need to interact with the Android framework directly

Contributor Expectations:

✅ Add tests for features involving multiple components
✅ Focus on critical paths and user flows
✅ Be mindful of test execution time
✅ Follow the AAA pattern
✅ Use descriptive test names

📱 UI Tests

UI tests verify the application from a user’s perspective.

What to Test:

User interface behavior
UI component interactions
Complete user flows
Screen transitions
Input handling and validation

Key Characteristics:

Tests from user perspective
Verifies visual elements and interactions
Covers end-to-end scenarios

Frameworks:

Espresso for Android UI testing
Compose UI testing for Jetpack Compose
JUnit 4 as the test runner

Location:

In the src/test directory for Compose UI tests
In the src/androidTest directory for Espresso tests

Contributor Expectations:

✅ Add tests for new UI components and screens
✅ Focus on critical user flows
✅ Consider different device configurations
✅ Test both positive and negative scenarios
✅ Follow the AAA pattern
✅ Use descriptive test names

📸 Screenshot Tests

⚠️ Work in Progress ⚠️

Screenshot tests verify the visual appearance of UI components.

What to Test:

Visual appearance of UI components
Layout correctness
Visual regressions
Theme and style application

Key Characteristics:

Captures visual snapshots
Compares against reference images (golden images)
Detects unintended visual changes

Frameworks:

JUnit 4 as the test runner
Compose UI testing
Screenshot comparison tools (TBD)

Location:

Same module as the code being tested
In the src/test directory

Contributor Expectations:

✅ Add tests for new Composable UI components
✅ Verify correct rendering in different states
✅ Update reference screenshots for intentional changes

🚫 Test Types We Don’t Currently Have

This section helps contributors understand our testing strategy and future plans.

End-to-End Tests ✨

Full system tests verifying complete user journeys
Tests across multiple screens and features
Validates entire application workflows

Performance Tests ⚡

Measures startup time, memory usage, responsiveness
Validates app performance under various conditions
Identifies performance bottlenecks

Accessibility Tests ♿

Verifies proper content descriptions
Checks contrast ratios and keyboard navigation
Ensures app is usable by people with disabilities

Localization Tests 🌐

Verifies correct translation display
Tests right-to-left language support
Validates date, time, and number formatting

Manual Test Scripts 📝

Manual testing by QA team for exploratory testing
Ensures repeatable test execution
Documents expected behavior for manual tests

🏃 Running Tests

Quick commands to run tests in the project.

Run all tests:

./gradlew test

Run tests for a specific module:

./gradlew :module-name:test

Run Android instrumentation tests:

./gradlew connectedAndroidTest

Run tests with coverage:

./gradlew testDebugUnitTestCoverage

📊 Code Coverage

⚠️ Work in Progress ⚠️

This section is currently under development and will be updated with specific code coverage rules and guidelines.

Code coverage helps us understand how much of our codebase is being tested. While we don’t currently have strict requirements, we aim for high coverage in critical components.

Current Approach:

Focus on critical business logic
Prioritize user-facing features
No strict percentage requirements
Quality of tests over quantity

Future Guidelines (Coming Soon):

Code coverage targets by component type
Coverage report generation instructions
Interpretation guidelines
Exemptions for generated/simple code
CI/CD integration details

Remember: High code coverage doesn’t guarantee high-quality tests. Focus on writing meaningful tests that verify correct behavior, not just increasing coverage numbers.

Keyboard shortcuts

Thunderbird for Android - Developer Documentation