Most teams discover the hard way that login and recovery flows are not really front-end tests. They are plumbing tests. The UI is just the last mile. The real problem is whether your application can reliably send a code, receive a message, correlate it to the right user, and tolerate delays, retries, and duplicate deliveries without turning your CI pipeline into a slot machine.

That is why a good sms and email verification harness matters. It gives your automation a controlled way to request OTPs, password reset links, magic links, and account verification messages, then observe those messages without a human opening an inbox or phone. If you build it carefully, it becomes a reusable test service, not a brittle pile of sleeps and ad hoc parsing.

This guide walks through a practical setup using Twilio messaging for SMS, Mailgun documentation for email delivery and testing hooks, and IMAP polling for mailbox retrieval. The emphasis is on reliability, not novelty. You want tests that fail for product reasons, not because a code took 11 seconds to arrive and your script checked at 10.

What this harness should do

A verification harness has a narrower job than a full test email or SMS platform. It should:

  • Trigger a message by exercising the real application flow
  • Capture the message through an automation-friendly channel
  • Extract the token, link, or one-time code deterministically
  • Associate the message with the correct test user and test run
  • Expose useful diagnostics when delivery breaks

The hardest part is not sending a message. It is proving you got the right message, in the right account, within the expected window, and that the message content matches what the product promised.

A good harness should also make failure modes obvious. Did the app fail to enqueue the message? Did the provider accept it but not deliver it? Did the provider deliver it but the inbox poller missed it? Or did the message arrive and the regex was too fragile?

Architecture: separate transport, retrieval, and assertion

Treat SMS and email as two transport paths with a shared verification contract.

Core components

  1. Test trigger
    • Your UI or API test starts the login, signup, or recovery flow.
    • Example: request a password reset for a specific test account.
  2. Message sink
    • SMS via Twilio.
    • Email via Mailgun, plus a mailbox that can be read over IMAP.
  3. Retrieval worker
    • Polls for incoming messages.
    • Normalizes content into a common schema.
  4. Parser and correlator
    • Extracts code or link from the body.
    • Matches by recipient, timestamp, subject, and sometimes custom headers.
  5. Assertion layer
    • Verifies delivery happened within a timeout.
    • Verifies message format, expiry, and token usability.

This separation matters because different parts fail differently. If you combine them into one test helper, you end up debugging delivery, parsing, and UI timing in a single stack trace, which is a convenient way to waste an afternoon.

Deciding how to handle SMS verification

SMS is usually the simpler transport to reason about and the harder one to make deterministic. Message delivery can be delayed, carrier routing can vary, and test numbers often have special provider behavior.

Twilio SMS testing patterns

Using Twilio for SMS tests usually means one of three patterns:

  • Real outbound SMS to test numbers, then polling Twilio logs or your own webhook endpoint
  • Twilio test credentials and simulator-like paths where supported for limited validation
  • Webhook-driven message capture with your application sending to Twilio and your harness observing the event lifecycle

For end-to-end verification flows, the most useful pattern is often real outbound SMS to dedicated test numbers, because it exercises your application, your provider integration, and message formatting together. Twilio’s messaging docs explain delivery events, status callbacks, and the shape of the API, which is important when you need to distinguish accepted, queued, sent, delivered, and failed states.

Practical SMS harness design

A robust SMS harness usually stores three things per run:

  • The test user identity
  • The target phone number
  • A provider correlation ID, such as the Twilio message SID

When your app requests the SMS, capture the SID if possible and store it with the test run. Then poll for delivery using either:

  • Your application’s own webhook receiver, or
  • A provider-side query endpoint, if your integration exposes one in a safe test environment

A minimal flow looks like this:

typescript

async function waitForSmsCode(fetchMessages: () => Promise<string[]>, timeoutMs = 60000) {
  const started = Date.now();
  while (Date.now() - started < timeoutMs) {
    const messages = await fetchMessages();
    const match = messages.find((m) => /verification code/i.test(m));
    if (match) {
      const code = match.match(/\b(\d{6})\b/)?.[1];
      if (code) return code;
    }
    await new Promise((r) => setTimeout(r, 3000));
  }
  throw new Error("Timed out waiting for SMS verification code");
}

That snippet is intentionally simple. In real code, you should do better than regex-only parsing. Use message metadata, filter by recipient, and reject stale messages from earlier runs.

Common SMS failure modes

  • Old code picked up first: a previous OTP still sits in the provider history or callback queue.
  • Retry collisions: the app sends a second code before the first one is consumed.
  • Formatting drift: the body text changes and the parser no longer recognizes it.
  • Carrier delay: the code arrives after the test timeout.
  • Region or number-type issues: some test numbers behave differently from production numbers.

The fix is not just a longer sleep. The fix is to correlate message identity, reduce ambiguity in test data, and assert content structure separately from delivery.

Deciding how to handle email verification

Email is more flexible than SMS, but the retrieval problem is more annoying. You often have multiple paths to the same inbox, and email clients happily preserve old messages, thread content, and filter artifacts that make automated polling less obvious than it sounds.

Mailgun for message generation and inspection

Mailgun is often used in two ways in a verification harness:

  • As the application’s email delivery provider
  • As a source of delivery events or test route behavior in non-production environments

If your app sends verification messages through Mailgun, you can leverage provider logs and events to understand whether a message was accepted and handed off. That is useful for diagnosing whether the failure happened before the message left your app or after it left your app.

For test harnesses, Mailgun is most valuable when your app sends real messages to a dedicated mailbox that the harness can poll, and when you configure message metadata or custom headers that help identify test runs. A stable subject line, plus a unique run identifier in a header or body, will save you a lot of grief later.

IMAP polling as the retrieval layer

For mailbox retrieval, IMAP remains the boring but practical choice. The current IMAP base specification is RFC 9051, and the behavior that matters here is simple: connect securely, search for messages, fetch headers and bodies, and mark what you have processed so you do not re-read the same verification email forever.

A common pattern is to create one dedicated inbox per environment, then poll it using a service account. Your poller should search by:

  • Recipient address
  • Subject prefix or exact subject
  • Date window
  • A unique test-run token embedded in the body or headers

A basic IMAP polling loop in Python might look like this:

import imaplib
import email
import time

def wait_for_verification_email(host, user, password, subject_token, timeout=60): end = time.time() + timeout while time.time() < end: with imaplib.IMAP4_SSL(host) as M: M.login(user, password) M.select(‘INBOX’) typ, data = M.search(None, f’(SUBJECT “{subject_token}”)’) ids = data[0].split() if ids: typ, msg_data = M.fetch(ids[-1], ‘(RFC822)’) raw = msg_data[0][1] msg = email.message_from_bytes(raw) return msg time.sleep(3) raise TimeoutError(‘Timed out waiting for verification email’)

This is enough for a proof of concept, but not enough for a production-grade harness. You still need parsing, deduplication, and state tracking.

Common email failure modes

  • Threading confusion: the new verification mail is grouped with an old thread and your parser reads the wrong body.
  • HTML-only content: the code is visible in the HTML part, but your fetch logic only reads plain text, or vice versa.
  • Quarantining and spam filtering: the mailbox never sees the message because the provider or inbox rules intercepted it.
  • Polling race: the message is fetched before the body is fully indexed, especially with external mailbox providers.
  • Stale state: a prior verification email remains unread and gets selected again.

To avoid these issues, keep verification messages boring. Use simple subject lines, include plain text bodies, and add a machine-readable marker that your harness can search for.

A shared message schema keeps the system sane

Do not write separate ad hoc parsers for every test. Normalize inbound messages into a common internal representation.

A practical schema might include:

  • transport: sms or email
  • recipient
  • sender
  • subject
  • body_text
  • body_html
  • provider_message_id
  • received_at
  • run_id
  • extracted_code
  • extracted_link

Once normalized, all your tests can ask the same question: did the expected message arrive, and does it contain the expected payload?

If you do not normalize early, every test becomes its own parser, and every parser becomes someone’s future incident.

Building correlation into the test flow

The best harnesses are designed around correlation, not search.

Add a unique run token

Generate a token at test start and include it in the user identity or message content. Examples:

  • Append +run12345 to a test email alias, if your mail routing supports it
  • Include a hidden header, like X-Test-Run-Id
  • Add the token to the application’s verification request metadata

If your app controls the template, put the token in the message body in a way that users will not notice but your harness can reliably detect.

Use dedicated test accounts

A shared inbox with multiple parallel runs is where test reliability goes to die. If possible:

  • Use one email address per run, or at least per parallel worker
  • Use one SMS number per environment or worker pool
  • Reset state between runs, including already-used codes

Make verification codes single-use in tests too

If the production system expects one-time use, test that behavior. A harness that only checks message arrival misses a whole class of bugs, such as replay acceptance or delayed invalidation. After you extract the code or link, submit it once, then assert it cannot be reused.

How to test the full flow without manual inbox checks

A robust end-to-end test often looks like this:

  1. Create a test user with a known email or phone number
  2. Trigger signup, login, or recovery
  3. Poll the message sink until the expected message appears
  4. Extract code or link
  5. Complete the flow in the app
  6. Assert success state, then attempt reuse or expiration checks

The key is to keep the waiting isolated from UI navigation. Your browser test should ask a harness service for the token, rather than reading a mailbox directly from the test script. That makes it easier to swap providers and easier to debug timeouts.

Example Playwright flow:

import { test, expect } from '@playwright/test';

test('password reset completes with verification email', async ({ page }) => {
  await page.goto('/forgot-password');
  await page.getByLabel('Email').fill('qa+run123@example.com');
  await page.getByRole('button', { name: 'Send reset link' }).click();

const link = await getResetLinkForRun(‘run123’); await page.goto(link); await page.getByLabel(‘New password’).fill(‘Str0ngPassw0rd!’); await page.getByRole(‘button’, { name: ‘Reset password’ }).click();

await expect(page.getByText(‘Password updated’)).toBeVisible(); });

The important part is getResetLinkForRun. That is the harness boundary. It should know how to poll Mailgun, IMAP, or whatever retrieval layer you use, but the UI test should not care.

CI and environment design

A verification harness is only useful if CI can run it predictably.

Keep external dependencies explicit

Use dedicated test credentials and make them obvious in environment variables. A CI job should not silently depend on a personal inbox or a developer’s phone.

name: verification-flows
on: [push]
jobs:
  test:
    runs-on: ubuntu-latest
    env:
      TEST_EMAIL_INBOX: qa-verification@example.com
      IMAP_HOST: imap.example.com
      TWILIO_SID: $
      MAILGUN_API_KEY: $
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm test -- --grep verification

Timeouts should reflect the transport, not optimism

SMS and email are asynchronous. If your team chooses a 10-second timeout because the UI test suite feels slow, you are encoding wishful thinking into your pipeline. Use a timeout based on realistic delivery expectations and the business cost of waiting.

A practical pattern is:

  • Short polling interval, such as 2 to 5 seconds
  • Moderate overall timeout, such as 45 to 90 seconds for non-critical verification tests
  • Clear failure messages that show recipient, run ID, and what the harness saw last

Separate smoke and deep verification tests

Not every pipeline run needs to prove every corner case. Split the suite into:

  • Smoke verification: one happy path for SMS and one for email
  • Contract checks: subject, code format, link shape, header markers
  • Negative checks: expired code, reused token, wrong recipient, delayed delivery

That keeps the most fragile tests from blocking every commit while still protecting the flows that matter.

Make the harness observable

A verification harness is infrastructure, so instrument it like infrastructure.

Log the following for each retrieval attempt:

  • run ID
  • recipient
  • transport
  • timestamp
  • provider message ID or IMAP UID
  • parse result
  • retry count

When a test fails, the first question is not usually “did the app work?” It is “where did the message go?” Observability should answer that quickly.

A simple rule helps:

If you cannot tell whether a failure came from delivery, retrieval, or parsing, the harness is too opaque to trust.

Security and compliance considerations

Verification flows often contain sensitive data, so test infrastructure deserves the same discipline as production, just smaller.

Do not reuse production secrets

Use separate credentials, separate sender identities, and separate inboxes for testing. If a test mailbox is compromised, you want the blast radius to stop there.

Minimize data retention

Verification messages often include links or codes that should expire quickly. Your harness should discard content once the assertion is complete, or at least redact it from long-term logs.

Protect inbox credentials

IMAP credentials are secrets. Store them in your CI secret manager, rotate them, and restrict access to the few jobs that actually need them.

Watch for PII in logs

It is easy to accidentally log email addresses, phone numbers, or reset links. That is useful during debugging and annoying forever afterward. Redact aggressively in shared logs.

When to keep this custom, and when not to

A custom sms and email verification harness is justified when your product has real end-to-end flows, real delivery dependencies, and multiple environments that need automation. It is especially useful when your test stack needs to prove that the application, provider integration, and mailbox behavior all line up.

Custom code is also a good fit when:

  • You need provider-specific assertions
  • You have complex message templates
  • You must support multiple inbox backends
  • You want tight CI integration with your existing test stack

The tradeoff is maintenance. Every parser, every timeout, and every provider edge case becomes your team’s problem. That is why the architecture should be small, explicit, and boring. The more behavior you can centralize into one harness, the less test debt leaks into every login test.

A practical checklist before you ship the harness

  • Dedicated test numbers and inboxes are configured
  • Run IDs are generated and attached to each request
  • SMS and email are routed through known providers
  • Polling handles retries and eventual consistency
  • Parsers read both plain text and HTML when needed
  • Tests fail with actionable diagnostics
  • Logs redact secrets and token values
  • Reuse and expiration behavior is covered
  • CI secrets are isolated from production credentials

Final thought

The best verification harnesses are not glamorous. They do one job, they do it repeatedly, and they make failure legible. Twilio gives you a well-documented SMS transport path, Mailgun gives you a structured email delivery layer, and IMAP gives you a standard mailbox interface that almost every environment can understand. Put those pieces together with correlation IDs, disciplined polling, and clean message parsing, and you get a system your team can actually trust.

That is the real goal of a sms and email verification harness: not just seeing the code arrive, but knowing exactly why it arrived, when it arrived, and whether the rest of the workflow behaved correctly once it did.