Skip to content

16. Cognito testing strategy

Date: 2026-03-17

Status

Accepted

Context

The proxy authenticates via AWS Cognito using OIDC/PKCE for browser login and JWT validation for API access. We need a testing strategy that covers both the Cognito-specific adapter behavior and the OIDC PKCE flow without depending on a real AWS account.

Tools evaluated

We evaluated five tools for emulating Cognito in tests:

Tool Type Cognito API OIDC endpoints PKCE S256 JWKS Cost Verdict
LocalStack Community Docker container No (Pro only) N/A N/A N/A Free cognito-idp returns InternalFailure: API not yet implemented or pro feature
LocalStack Pro Docker + license Yes Likely (not documented) Unknown Yes Paid Not verifiable without license
cognito-local Docker/npm Partial (SDK APIs only) No No Not documented Free Emulates SDK APIs only (AdminInitiateAuth, SignUp), no OAuth2/OIDC endpoints
moto Python in-process mock Yes (broad coverage) No No Yes (static keys) Free Implements SDK APIs + JWKS, but zero OAuth2 endpoints (/oauth2/authorize, /oauth2/token not implemented)
magnito Docker (Next.js) Partial (~30%) Yes (/oauth2/authorize, /oauth2/token) Yes (S256 validated) Yes (/.well-known/jwks.json) Free Full OAuth2/OIDC flow with PKCE S256 validation. Social sign-in emulation (Google/Apple/Amazon/Facebook)

Detailed findings

LocalStack Community (spike executed)

docker run -d -p 4566:4566 localstack/localstack
curl -X POST http://localhost:4566/ -H "X-Amz-Target: AWSCognitoIdentityProviderService.CreateUserPool" ...
→ {"__type": "InternalFailure", "message": "API for service 'cognito-idp' not yet implemented or pro feature"}

Cognito is a Pro-only feature in LocalStack. The Community edition does not support any Cognito operations.

cognito-local

jagregory/cognito-local emulates Cognito's SDK APIs (CreateUserPool, AdminInitiateAuth, SignUp) but does not implement any OAuth2/OIDC endpoints. No /oauth2/authorize, no /oauth2/token, no PKCE support. Only USER_PASSWORD_AUTH flow is supported.

moto

Source code inspection of moto==5.1.3 (moto/cognitoidp/) reveals: - Implemented: CreateUserPool, CreateUserPoolClient, AdminInitiateAuth, InitiateAuth, JWKS endpoint with static keys, JWT generation via joserfc with correct issuer format - Not implemented: Zero references to oauth2, authorize, token, code_challenge, code_verifier, or PKCE in the entire cognitoidp module - JWKS: Static key pair in resources/jwks-public.json.gz, tokens signed with matching private key - Token issuer: Correct format https://cognito-idp.{region}.amazonaws.com/{pool_id}

Moto is excellent for testing Cognito SDK interactions but cannot test the OIDC browser flow.

magnito

frourios/magnito is a Cognito emulator that implements the OAuth2/OIDC endpoints:

  • /oauth2/authorize: Accepts code_challenge and code_challenge_method (both plain and S256)
  • /oauth2/token: Accepts grant_type=authorization_code with code_verifier
  • PKCE S256 validation: assert(user.codeChallenge === createHash('sha256').update(codeVerifier).digest('base64url'))
  • JWKS: Dynamic key generation at /{userPoolId}/.well-known/jwks.json
  • Social sign-in: Emulates Google, Apple, Amazon, Facebook providers
  • Limitations: ~30% Cognito API coverage, CreateUserPoolDomain not implemented, requires Docker

pytest-iam (already in use)

pytest-dev/pytest-iam provides a real OIDC provider (Canaille + Authlib) in a thread: - PKCE S256: CodeChallenge(required=True) via Authlib's RFC 7636 implementation - Full OIDC flow: authorize → code → token exchange → userinfo - No Docker required: Pure Python, runs in-process - Limitation: Not Cognito-specific (generic OIDC provider)

Decision

Adopt a four-tier testing strategy:

Tier 1: Unit tests with pytest-httpx (existing)

Mock HTTP responses for Cognito-specific adapter behavior:

What it tests Tool Examples
OidcTokenValidator JWKS fetch + caching pytest-httpx Valid/expired/wrong-issuer tokens, JWKS refresh, unknown kid
CognitoTokenValidator delegation pytest-httpx Thin wrapper behavior
CognitoOIDCClient token exchange pytest-httpx Success, error, network failure
CognitoClaimsMapper claim extraction Pure unit test cognito:groups, scopes

Tier 2: Integration tests with pytest-iam (existing)

Test the full OIDC PKCE flow against a real OIDC provider:

What it tests Tool Examples
PKCE S256 end-to-end pytest-iam Generate challenge → authorize → exchange with verifier
Wrong verifier rejected pytest-iam Provider rejects tampered code_verifier
Replay prevention pytest-iam Same state used twice → rejected
Token validity pytest-iam Access token works at userinfo endpoint

Tier 3: Cognito-specific integration with moto (new)

Test Cognito SDK interactions and JWT verification:

What it tests Tool Examples
Token issuer format matches CognitoSettings.get_issuer() moto Verify https://cognito-idp.{region}.amazonaws.com/{pool_id}
JWKS endpoint returns verifiable keys moto Fetch JWKS, verify token signature
AdminInitiateAuth token validation moto Generate token via moto, validate with OidcTokenValidator
CognitoSettings derived URLs moto JWKS URI and issuer match moto's endpoints
cognito:groups in moto tokens moto Verify CognitoClaimsMapper extracts groups from moto-generated tokens

Tier 4: E2E tests with magnito + testcontainers (new)

Test the full Cognito OIDC PKCE flow against a real Cognito emulator running in Docker:

What it tests Tool Examples
Full PKCE S256 flow via Cognito endpoints magnito + testcontainers Authorize → code → token exchange with verifier
Wrong verifier rejected by Cognito magnito magnito rejects tampered code_verifier
Refresh token flow magnito Exchange refresh token for new access/id tokens
Token verification via magnito JWKS magnito + CognitoTokenValidator ID token signed by magnito verifiable by our adapter
Session cookie roundtrip magnito Tokens stored in and retrieved from HMAC-signed cookie
Invalid refresh token rejected magnito magnito rejects invalid refresh token

Tests are marked @pytest.mark.e2e and require Docker. Run with uv run pytest -m e2e.

Why both Tier 3 (moto) and Tier 4 (magnito)?

Concern moto (Tier 3) magnito (Tier 4)
Cognito SDK APIs Broad coverage ~30%
OAuth2/OIDC endpoints None Full (/oauth2/authorize, /oauth2/token)
PKCE S256 validation Not implemented Validated server-side
Docker required No Yes
CI speed Fast (in-process) Slower (container startup)
Use case JWT format + claim names + JWKS verification Full browser OIDC flow

Both tiers are needed: moto verifies Cognito-specific JWT behavior (issuer, claims, JWKS) without Docker, while magnito tests the complete OIDC authorization flow with real PKCE validation.

Future consideration: Tier 5 (staging)

When pre-production validation is needed, smoke tests against a real AWS Cognito User Pool in a staging environment. Not yet planned.

Consequences

  • Tiers 1-3 require no Docker — run in-process, fast CI feedback
  • Tier 4 requires Docker — runs as a separate CI job (e2e), depends on the quality job passing first
  • testcontainers manages magnito container lifecycle automatically (start/stop)
  • Tests marked @pytest.mark.e2e can be skipped locally with pytest -m "not e2e"
  • moto added as a dev dependency for Tier 3 Cognito-specific tests
  • pytest-iam remains the primary tool for standard OIDC PKCE flow testing (Tier 2)
  • pytest-httpx remains for adapter-level unit tests (Tier 1)
  • The four-tier combination covers: unit isolation, OIDC standard compliance, Cognito-specific JWT quirks, and full Cognito OIDC flow with PKCE

References