This Is a Test: Practical Ways to Write Reliable Automated Tests
Tests should be boring: fast, clear, and trustworthy. If yours aren’t, this guide shows concrete steps to fix them with patterns, code examples, and CI setup you can copy.
Goals
- Make tests deterministic and fast
- Reduce flakiness from time, randomness, and I/O
- Cover the right layers (unit, integration, e2e) without bloat
- Ship with confidence via CI
1) Keep a minimal plan (the test “pyramid”)
- Unit tests: 70–80%. Pure logic, no network or filesystem. Milliseconds.
- Integration tests: 15–25%. Real DB or external systems via containers.
- E2E: 5–10%. Critical user journeys, run on CI nightly or pre-release.
Tip: If you’re writing an E2E test to check a pure function, stop and move that logic to a unit test.
2) Patterns that make tests reliable
- Arrange–Act–Assert (AAA): Setup → Execute → Verify.
- Name tests as behavior: "it slugifies spaces" not "test_slug_2".
- Control time: Freeze or inject a clock.
- Control randomness: Inject RNG or seed it.
- Replace network with fakes/mocks; prefer fakes for complex behavior.
- Test Data Builders: Create minimal, clear data for each case.
- Property-based tests: Great for parsers, encoders, mathy code.
- Table-driven tests: Many inputs/outputs, one loop.
3) Example: Python (pytest)
3.1 Unit test with AAA and a builder
# app/slug.py
import re
def slugify(title: str) -> str:
s = title.strip().lower()
s = re.sub(r"[^a-z0-9\s-]", "", s)
s = re.sub(r"\s+", "-", s)
s = re.sub(r"-+", "-", s)
return s.strip("-")
# tests/test_slug.py
import pytest
from app.slug import slugify
class TitleBuilder:
def __init__(self):
self._title = "This is a test"
def with_chars(self, s):
self._title = s
return self
def build(self):
return self._title
def test_slugify_basic():
# Arrange
title = TitleBuilder().build()
# Act
result = slugify(title)
# Assert
assert result == "this-is-a-test"
@pytest.mark.parametrize("raw,expected", [
(" Hello, World! ", "hello-world"),
("Dashes -- and spaces", "dashes-and-spaces"),
("Ünicode ✓", "nicode"),
])
def test_slugify_table(raw, expected):
assert slugify(raw) == expected
3.2 Control time and randomness
# app/token.py
import time, random
def issue_token(user_id: str, clock=time, rng=random):
ts = int(clock.time())
nonce = rng.randint(1000, 9999)
return f"{user_id}-{ts}-{nonce}"
# tests/test_token.py
import types
from app.token import issue_token
class FixedClock:
def time(self):
return 1_700_000_000
class FixedRng:
def randint(self, a, b):
return 4242
def test_issue_token_is_deterministic():
tok = issue_token("alice", clock=FixedClock(), rng=FixedRng())
assert tok == "alice-1700000000-4242"
4) Example: JavaScript (Jest)
4.1 Mocking fetch and timers
// src/userClient.js
export async function getUser(id, fetchImpl = fetch) {
const res = await fetchImpl(`/api/users/${id}`);
if (!res.ok) throw new Error("User fetch failed");
return res.json();
}
// __tests__/userClient.test.js
import { getUser } from "../src/userClient";
test("gets user via fetch", async () => {
const fakeFetch = jest.fn().mockResolvedValue({
ok: true,
json: async () => ({ id: 1, name: "Alice" })
});
const user = await getUser(1, fakeFetch);
expect(fakeFetch).toHaveBeenCalledWith("/api/users/1");
expect(user).toEqual({ id: 1, name: "Alice" });
});
4.2 Property-ish table checks
import { slugify } from "../src/slugify"; // similar to Python version
test.each([
["This is a test", "this-is-a-test"],
["Hello, World!", "hello-world"],
[" spaces \t and\nlines ", "spaces-and-lines"],
])("slugify(%p) => %p", (input, expected) => {
expect(slugify(input)).toBe(expected);
});
5) Example: Go (table-driven)
// slug/slug.go
package slug
import (
"regexp"
"strings"
)
var reBad = regexp.MustCompile(`[^a-z0-9\s-]`)
var reSpace = regexp.MustCompile(`\s+`)
var reDash = regexp.MustCompile(`-+`)
func Slugify(s string) string {
s = strings.ToLower(strings.TrimSpace(s))
s = reBad.ReplaceAllString(s, "")
s = reSpace.ReplaceAllString(s, "-")
s = reDash.ReplaceAllString(s, "-")
return strings.Trim(s, "-")
}
// slug/slug_test.go
package slug
import "testing"
func TestSlugify(t *testing.T) {
cases := []struct{ in, want string }{
{"This is a test", "this-is-a-test"},
{"Dashes -- and spaces", "dashes-and-spaces"},
{" Hello, World! ", "hello-world"},
}
for _, c := range cases {
if got := Slugify(c.in); got != c.want {
t.Fatalf("Slugify(%q) = %q, want %q", c.in, got, c.want)
}
}
}
6) Fast, deterministic integration tests with containers
Use Testcontainers or a Docker service in CI to spin real dependencies quickly.
- Scope: repository layer, migrations, event publishing.
- Don’t hit third-party SaaS in CI; use fakes or local emulators.
Python example with pytest + Testcontainers (Postgres):
# tests/it/test_users_pg.py
import psycopg2
from testcontainers.postgres import PostgresContainer
def test_create_user_in_pg():
with PostgresContainer("postgres:15") as pg:
conn = psycopg2.connect(pg.get_connection_url())
cur = conn.cursor()
cur.execute("create table users(id serial primary key, name text not null)")
cur.execute("insert into users(name) values('Alice') returning id")
user_id = cur.fetchone()[0]
conn.commit()
cur.execute("select name from users where id=%s", (user_id,))
assert cur.fetchone()[0] == "Alice"
7) CI you can copy (GitHub Actions)
Matrix across languages; run unit tests first; integration tests behind a flag.
name: ci
on:
push:
branches: [ main ]
pull_request:
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
language: [python, node, go]
services:
postgres:
image: postgres:15
env:
POSTGRES_PASSWORD: postgres
ports: ["5432:5432"]
options: >-
--health-cmd "pg_isready -U postgres" \
--health-interval 10s --health-timeout 5s --health-retries 5
steps:
- uses: actions/checkout@v4
- name: Setup Python
if: matrix.language == 'python'
uses: actions/setup-python@v5
with:
python-version: '3.12'
- name: Install + Test (Python)
if: matrix.language == 'python'
run: |
pip install -U pip pytest pytest-cov psycopg2-binary
pytest -q --maxfail=1 --disable-warnings --cov=app
- name: Setup Node
if: matrix.language == 'node'
uses: actions/setup-node@v4
with:
node-version: '20'
cache: 'npm'
- name: Install + Test (Node)
if: matrix.language == 'node'
run: |
npm ci
npm test -- --ci
- name: Setup Go
if: matrix.language == 'go'
uses: actions/setup-go@v5
with:
go-version: '1.22'
- name: Test (Go)
if: matrix.language == 'go'
run: go test ./... -v
Tips:
- Cache dependencies (pip cache, npm cache, Go mod cache) for speed.
- Mark slow tests and run them in a separate job or nightly workflow.
8) Flaky test triage playbook
- Reproduce locally: run test 50–200 times with a fixed seed.
- Stabilize time: freeze or inject time.
- Stabilize async: await promises; flush timers; add proper awaits/conditions.
- Stabilize concurrency: avoid sleeps; use signals/latches; increase timeouts sparingly.
- Isolate I/O: unique temp dirs, unique DB schemas per test.
- Last resort: quarantine label + ticket; never silently retry in CI without tracking.
9) Coverage the healthy way
- Use thresholds to catch drops, not to chase 100%.
- Prefer branch coverage for risky modules.
- Review uncovered lines; either test or justify with comments.
Commands:
- pytest:
pytest --cov=app --cov-report=term-missing - jest:
jest --coverage - go:
go test ./... -coverprofile=cover.out && go tool cover -func=cover.out
10) A short checklist
- Names tell the behavior
- AAA structure
- Deterministic time/random
- Pure unit tests run < 1s per 100 tests
- Integration via containers, not shared dev DBs
- Flaky tests blocked from merging until fixed
- Coverage enforced with sensible thresholds
This is a test — and with these steps, it’s a reliable one.