Showing posts with label launchdarkly. Show all posts
Showing posts with label launchdarkly. Show all posts

Wednesday, April 15, 2026

Feature Flags in Production: Gradual Rollouts, A/B Testing, and Kill Switches

Hero: Feature flag rollout percentages increasing from 1% to 100% with metrics tracking

Every major tech company ships features to a subset of users before rolling them out to everyone. Facebook rolls out changes to 1% of traffic first. Stripe tests payment flow changes with 5% of merchants before general availability. GitHub ships dark mode to beta users months before the official launch.

The mechanism behind all of this: feature flags. A feature flag is a conditional in your code that controls whether a feature is active — evaluated at runtime, configurable without deploying new code.

In 2026, feature flags have expanded beyond simple on/off toggles into a platform for gradual rollouts, targeted experiments, operational kill switches, and progressive delivery. This guide covers how to implement them correctly, which tools to use, and the patterns that make them powerful.

The Problem: Deploy and Pray

The traditional deploy model is binary: code ships to all users simultaneously. This creates several failure modes:

Big-bang releases: The "release day" model where a feature that took 3 months to build ships to 100% of users at once. When something breaks, you roll back the entire deployment — including unrelated changes.

Long-lived feature branches: Teams isolate features in branches to avoid shipping half-finished work. Branches diverge from main for weeks. Merging becomes painful. Integration issues surface late.

No experimentation infrastructure: Measuring whether a change actually improves user behavior requires A/B testing infrastructure most teams don't have.

Feature flags solve all three: features are deployed (to 0% of users) long before release, mainline development continues without branches, and experiments can be run with proper statistical controls.

graph LR subgraph "Traditional deploy" A[Code merged] --> B[All users] B --> C{Bug?} C -- Yes --> D[Rollback entire deploy] end subgraph "Feature flags" E[Code deployed] --> F[0% rollout] F --> G[1% → internal team] G --> H[5% → beta users] H --> I[25% → gradual] I --> J[100% → complete] J --> K{Bug?} K -- Yes --> L[Toggle flag off in 1s] end style D fill:#ef4444,color:#fff style L fill:#22c55e,color:#fff

How It Works: Anatomy of a Feature Flag

A feature flag evaluation has three parts:

  1. Flag definition: Name, type (boolean, string, number), default value, targeting rules
  2. Context: Information about the current request — user ID, company, region, plan tier
  3. Evaluation: Rules evaluated against context → returns variant
# Simplified flag evaluation logic
def evaluate_flag(flag_name: str, context: dict) -> bool | str:
    flag = get_flag_definition(flag_name)

    # Check targeting rules in order
    for rule in flag.targeting_rules:
        if rule.matches(context):
            return rule.variant  # Return the matched variant

    # No rules matched — return default
    return flag.default_variant

The evaluation happens in milliseconds, in-process (the SDK has a local copy of flag definitions cached from the flag service). The flag service doesn't sit in the critical path of every request.

Flag Types

Type Use Case Example
Boolean Feature on/off new_checkout_flow: true/false
String A/B variants homepage_hero: "control"/"variant_a"/"variant_b"
Number Gradual rollout % ai_summary_rollout: 0.25 (25% of users)
JSON Complex configuration rate_limits: {"free": 100, "pro": 1000}

Implementation: OpenFeature Standard

OpenFeature is a CNCF standard that decouples your feature flag code from the specific vendor. You write against the OpenFeature SDK; you swap providers without changing application code.

# pip install openfeature-sdk openfeature-provider-launchdarkly
from openfeature import api
from openfeature.evaluation_context import EvaluationContext
from openfeature.provider.launchdarkly import LaunchDarklyProvider  # or any other provider

# Initialize with your provider (done once at startup)
api.set_provider(LaunchDarklyProvider(sdk_key="sdk-your-key-here"))

client = api.get_client()

# In your request handler
def checkout_handler(request):
    # Build context from request
    ctx = EvaluationContext(
        targeting_key=str(request.user.id),
        attributes={
            "plan": request.user.plan,          # "free" / "pro" / "enterprise"
            "region": request.headers.get("CF-IPCountry", "US"),
            "email": request.user.email,        # For beta cohorts
            "company_id": str(request.user.company_id),
            "user_age_days": (datetime.now() - request.user.created_at).days,
        }
    )

    # Boolean flag — is the new checkout enabled for this user?
    use_new_checkout = client.get_boolean_value(
        "new-checkout-flow",
        default_value=False,
        evaluation_context=ctx,
    )

    # String flag — which pricing experiment variant?
    pricing_variant = client.get_string_value(
        "pricing-page-experiment",
        default_value="control",
        evaluation_context=ctx,
    )

    if use_new_checkout:
        return new_checkout_view(request, pricing_variant)
    else:
        return legacy_checkout_view(request)

Gradual Rollout with LaunchDarkly

LaunchDarkly is the market leader in 2026. Configuration is in their dashboard, but also exportable as JSON:

{
  "key": "new-checkout-flow",
  "kind": "boolean",
  "variations": [false, true],
  "rules": [
    {
      "description": "Internal team always sees new checkout",
      "clauses": [{"attribute": "email", "op": "endsWith", "values": ["@mycompany.com"]}],
      "variation": 1
    },
    {
      "description": "Enterprise customers excluded (revenue risk)",
      "clauses": [{"attribute": "plan", "op": "in", "values": ["enterprise"]}],
      "variation": 0
    }
  ],
  "fallthrough": {
    "rollout": {
      "variations": [
        {"variation": 0, "weight": 80000},
        {"variation": 1, "weight": 20000}
      ]
    }
  },
  "offVariation": 0,
  "on": true
}

This flag configuration: always shows new checkout to @mycompany.com users, never shows it to enterprise, rolls it out to 20% of everyone else.

Self-Hosted: Unleash

For teams that can't send user data to a SaaS vendor (GDPR, security policies), Unleash is the best open-source alternative:

# docker-compose.yml for Unleash
version: '3'
services:
  unleash:
    image: unleashorg/unleash-server:latest
    ports:
      - "4242:4242"
    environment:
      DATABASE_URL: postgres://unleash:password@db/unleash
      UNLEASH_DEFAULT_ADMIN_USERNAME: admin
      UNLEASH_DEFAULT_ADMIN_PASSWORD: changeme
    depends_on:
      - db
  db:
    image: postgres:16
    environment:
      POSTGRES_DB: unleash
      POSTGRES_USER: unleash
      POSTGRES_PASSWORD: password
# Python SDK for Unleash
from UnleashClient import UnleashClient

client = UnleashClient(
    url="http://localhost:4242/api",
    app_name="my-app",
    custom_headers={"Authorization": "Bearer your-api-key"}
)
client.initialize_client()

# Evaluate with context
context = {
    "userId": str(user.id),
    "properties": {
        "plan": user.plan,
        "region": user.region,
    }
}

if client.is_enabled("new-checkout-flow", context):
    return new_checkout()
else:
    return legacy_checkout()

Four Patterns That Make Feature Flags Powerful

Pattern 1: The Kill Switch

The most operationally valuable flag type. A kill switch is a boolean flag that's ON in production — until something breaks. Then you turn it OFF in 5 seconds without a deploy.

# Kill switch for a new payment processor
@app.route('/api/payments', methods=['POST'])
def process_payment():
    if not client.get_boolean_value("new-payment-processor", default_value=False, ctx=ctx):
        # Old processor
        return legacy_payment_processor.charge(request.json)

    try:
        return new_payment_processor.charge(request.json)
    except NewProcessorException as e:
        # Automatic fallback + alert
        alert_pagerduty(f"New payment processor failed: {e}")
        return legacy_payment_processor.charge(request.json)

When the new processor has issues, ops turns off the flag. No deploy. No rollback. No 3am war room. Just a toggle.

Pattern 2: Ring Deployment

Deploy to progressively larger rings of users, validating metrics at each stage:

flowchart LR A["Ring 0\nInternal (0.1%)"] --> B["Ring 1\nBeta users (1%)"] B --> C["Ring 2\nFree tier (10%)"] C --> D["Ring 3\nPro tier (50%)"] D --> E["Ring 4\nAll users (100%)"] A -.->|"Monitor:\nerror rate\nlatency\nbusiness metrics"| A B -.->|"Monitor 24hrs"| B C -.->|"Monitor 48hrs"| C D -.->|"Monitor 72hrs"| D style A fill:#3b82f6,color:#fff style E fill:#22c55e,color:#fff
# Ring deployment configuration
rings = [
    {"name": "internal", "targeting": {"email": {"endsWith": "@mycompany.com"}}, "weight": 100},
    {"name": "beta", "targeting": {"properties.beta_user": True}, "weight": 100},
    {"name": "free_10_percent", "targeting": {"plan": "free"}, "weight": 10},
    {"name": "pro_50_percent", "targeting": {"plan": "pro"}, "weight": 50},
    {"name": "all_users", "targeting": None, "weight": 100},
]

# Move to next ring after validating metrics
def advance_ring(flag_name: str, current_ring: int) -> bool:
    metrics = get_feature_metrics(flag_name, hours=24)

    if metrics.error_rate_increase > 0.01:  # 1% error rate increase
        alert(f"Flag {flag_name}: error rate elevated, holding at ring {current_ring}")
        return False

    if metrics.p99_latency_increase_ms > 50:  # 50ms p99 latency increase
        alert(f"Flag {flag_name}: latency elevated, holding at ring {current_ring}")
        return False

    return True  # Safe to advance

Pattern 3: Experiment Flags with Statistical Significance

Feature flags become A/B testing infrastructure when you add metric tracking and significance testing:

import scipy.stats as stats
import numpy as np

def evaluate_experiment(flag_key: str, metric_name: str, min_sample: int = 1000) -> dict:
    """
    Check if an experiment has reached statistical significance.
    Returns: variant recommendation and confidence level.
    """
    control_data = get_metric_data(flag_key, variant="control", metric=metric_name)
    treatment_data = get_metric_data(flag_key, variant="treatment", metric=metric_name)

    if min(len(control_data), len(treatment_data)) < min_sample:
        return {"status": "insufficient_data", "samples": len(control_data) + len(treatment_data)}

    # Two-sample t-test for continuous metrics (e.g., conversion rate, revenue)
    t_stat, p_value = stats.ttest_ind(control_data, treatment_data)

    control_mean = np.mean(control_data)
    treatment_mean = np.mean(treatment_data)
    lift = (treatment_mean - control_mean) / control_mean * 100

    return {
        "status": "significant" if p_value < 0.05 else "not_significant",
        "p_value": round(p_value, 4),
        "lift_percent": round(lift, 2),
        "control_mean": round(control_mean, 4),
        "treatment_mean": round(treatment_mean, 4),
        "recommendation": "ship" if (p_value < 0.05 and lift > 0) else "rollback" if (p_value < 0.05 and lift < 0) else "continue",
        "samples": len(control_data) + len(treatment_data),
    }

# Usage:
result = evaluate_experiment("checkout-redesign", "conversion_rate")
# → {"status": "significant", "lift_percent": 3.4, "p_value": 0.012, "recommendation": "ship"}

Pattern 4: Operational Configuration Flags

Flags aren't just for features. Use them for runtime configuration that operations may need to adjust under load:

# Rate limit configuration that ops can adjust without a deploy
rate_config = client.get_object_value(
    "api-rate-limits",
    default_value={"free": 100, "pro": 1000, "enterprise": 10000},
    evaluation_context=ctx,
)

# Under attack, ops sets: {"free": 10, "pro": 100, "enterprise": 1000}
# 10× reduction across the board, in 30 seconds, without a deploy

if request.rate_count > rate_config[user.plan]:
    return Response(status=429, headers={"Retry-After": "60"})

Cost and Latency: What Flag Evaluation Actually Costs

The operational concern teams often raise: "Won't feature flag evaluation add latency?" The answer, when implemented correctly: no.

Modern flag SDKs use a streaming architecture. On startup, the SDK downloads all flag definitions and stores them in memory. Flag evaluation happens entirely in-process — no network call, no database lookup. The SDK subscribes to a server-sent event stream and updates its local cache when flags change.

Evaluation time: sub-millisecond. Typically 50-200 microseconds, including context evaluation and rule matching.

The only performance concern is the initial SDK initialization (100-500ms to download and cache all flags). Don't evaluate flags before initialization completes — use the async initialization pattern with defaults.

# Benchmark flag evaluation latency
import time
import statistics

latencies = []
for _ in range(10000):
    start = time.perf_counter()
    client.get_boolean_value("new-feature", default_value=False, evaluation_context=ctx)
    latencies.append((time.perf_counter() - start) * 1000)

print(f"p50: {statistics.median(latencies):.3f}ms")  # → 0.041ms
print(f"p99: {statistics.quantiles(latencies, n=100)[98]:.3f}ms")  # → 0.128ms

The LaunchDarkly and Unleash SDKs both benchmark at under 0.2ms p99 for flag evaluation. For 99.9% of applications, feature flag evaluation is not in your performance budget.

Managing Technical Debt: Flag Lifecycle

The danger of feature flags is accumulating hundreds of stale flags in your codebase. Each flag is a branch in your logic — too many, and the code becomes impossible to reason about.

# Flag with built-in expiry tracking
@flag_lifecycle(
    flag_key="new-checkout-flow",
    expected_ship_date="2026-06-01",
    owner="team-checkout",
    jira_ticket="ENG-4521"
)
def checkout_handler(request):
    if client.get_boolean_value("new-checkout-flow", default_value=False, ctx=ctx):
        ...

Enforce flag retirement:
1. Set a ticket at flag creation: Create the cleanup ticket before the flag goes live
2. Alert on old flags: Monitor for flags > 90 days old that haven't been cleaned up
3. Regular flag reviews: Quarterly audit of all flags — is each one still needed?

-- Query to find stale flags (LaunchDarkly stores flag metadata)
SELECT flag_key, created_date, last_modified, owner
FROM feature_flags
WHERE last_modified < NOW() - INTERVAL '90 days'
  AND is_permanent = false
ORDER BY last_modified ASC;

Server-Side vs Client-Side Flags

Feature flags can be evaluated in two places:

Server-side flags: Evaluated in your backend. The client never sees the flag state — it only receives the feature or doesn't. No flag state exposed in client-side JavaScript. Good for: security-sensitive features, anything involving backend logic, pricing experiments.

Client-side flags: Evaluated in the browser or mobile app. The SDK downloads flag definitions and evaluates them locally. Enables UI personalization without a server round trip. Risk: flag rules are visible in client-side JavaScript — don't use for features you want to hide from users who inspect network traffic.

// Client-side SDK (LaunchDarkly Browser SDK)
import { LDClient, initialize } from 'launchdarkly-js-client-sdk';

const user = {
  kind: 'user',
  key: currentUser.id,
  plan: currentUser.plan,
  email: currentUser.email,
};

const client: LDClient = initialize('client-side-sdk-key', user);

await client.waitForInitialization();

// Evaluate a flag — happens locally, no server call
const showNewNav = client.variation('new-navigation', false);
if (showNewNav) {
  renderNewNavigation();
}

// React hook pattern
import { useLDClient, useFlags } from 'launchdarkly-react-client-sdk';

function Navigation() {
  const { 'new-navigation': showNewNav } = useFlags();
  return showNewNav ? <NewNavigation /> : <LegacyNavigation />;
}

For the backend equivalent:

# Server-side: flag evaluated in Python, result passed to template
def homepage_view(request):
    ctx = EvaluationContext(
        targeting_key=str(request.user.id),
        attributes={"plan": request.user.plan}
    )
    show_new_nav = client.get_boolean_value("new-navigation", default_value=False, evaluation_context=ctx)

    return render(request, "homepage.html", {
        "show_new_nav": show_new_nav,
        # Flag state is in template context, not exposed as JS to client
    })

The hybrid pattern: Use server-side flags for feature gating; use client-side for UI personalization where the latency of a server round-trip would be noticeable (navigation, layout).

Flag Targeting: Beyond Percentage Rollouts

Percentage rollouts are the most common targeting strategy, but several others are more appropriate in specific situations:

# Targeting strategies and when to use them

# 1. User segment: specific users by attribute
#    Use for: beta cohorts, VIP customers, internal team
flag_config = {
    "rules": [
        {"clauses": [{"attribute": "email", "op": "endsWith", "values": ["@mycompany.com"]}], "variation": 1},
        {"clauses": [{"attribute": "beta_opt_in", "op": "in", "values": [True]}], "variation": 1},
    ]
}

# 2. Sticky bucketing: same user always gets same variant
#    Default behavior in most SDKs — user hash is consistent
#    Critical for A/B tests: users shouldn't switch groups mid-experiment

# 3. Time-based: flag automatically turns off after a date
#    Use for: temporary maintenance banners, holiday promotions
from datetime import datetime

def time_gated_flag(flag_name: str, end_date: datetime) -> bool:
    if datetime.now() > end_date:
        return False
    return client.get_boolean_value(flag_name, default_value=False, evaluation_context=ctx)

# 4. Dependency: flag only active if another flag is active
#    Use for: progressive feature builds
def dependent_flag(parent_flag: str, child_flag: str) -> bool:
    if not client.get_boolean_value(parent_flag, default_value=False, evaluation_context=ctx):
        return False
    return client.get_boolean_value(child_flag, default_value=False, evaluation_context=ctx)

# 5. Context-based: target by request properties
#    Use for: region-specific features, mobile vs web
ctx_with_request = EvaluationContext(
    targeting_key=str(user.id),
    attributes={
        "region": request.headers.get("CF-IPCountry", "US"),
        "platform": request.headers.get("X-Platform", "web"),  # "ios", "android", "web"
        "app_version": request.headers.get("X-App-Version", "0.0.0"),
    }
)

Production Considerations

SDK Initialization and Fallbacks

The flag SDK must not block application startup or add request latency. Initialize asynchronously, provide defaults, cache aggressively:

# Async initialization — don't block startup
async def startup():
    await flag_client.initialize()  # Fetches flags from service, caches locally

# During startup failure — default_value is your safety net
# Never let flag evaluation throw exceptions into your business logic
try:
    enabled = client.get_boolean_value("new-feature", default_value=False, ctx=ctx)
except Exception as e:
    log.error(f"Flag evaluation failed: {e}")
    enabled = False  # Fail safe

SDKs Are Local — Flag Changes Are Near-Instant

Modern flag SDKs stream flag updates via server-sent events. Changes in the LaunchDarkly or Unleash dashboard propagate to all running SDK instances in 1-2 seconds. You don't need a deploy, a restart, or an API call — the change propagates automatically.

This is what makes kill switches operationally effective: you flip the flag, and within seconds, traffic shifts.

Feature Flags in CI/CD: Testing Against Real Flag States

Testing with feature flags requires understanding which variant your tests should run against. Two strategies:

Test both variants: Run your integration test suite against each variant. This ensures neither path regresses during a rollout.

# pytest parameterization over flag variants
import pytest
from unittest.mock import patch

@pytest.fixture(params=["control", "treatment"])
def checkout_variant(request):
    """Run each test against both checkout variants."""
    with patch.object(flag_client, 'get_string_value', return_value=request.param):
        yield request.param

def test_checkout_completes(client, checkout_variant):
    """Both variants should complete checkout successfully."""
    response = client.post('/api/checkout', json={"items": [{"id": 1, "qty": 1}]})
    assert response.status_code == 200
    assert response.json()["order_id"] is not None
    # Test passes regardless of which variant — both paths must work

Flag overrides in staging: Set specific users to specific variants in staging environments for deterministic testing.

# LaunchDarkly: staging environment flag overrides
# Individual user targeting (by ID) overrides all rules
user_targets:
  - variation: 1  # Treatment variant
    values:
      - user_id_of_test_account_1
      - user_id_of_qa_bot
      - user_id_of_automated_test_user

This ensures your QA environment always sees the new variant, while developers can test the control variant by using their personal accounts.

The Flag-Deployment Dependency

One subtle CI/CD consideration: the flag must exist in the flag service before the code that references it is deployed. If you deploy code that calls client.get_boolean_value("new-feature", ...) before creating the flag in LaunchDarkly/Unleash, the SDK returns the default value. That's fine if your default value is the safe path (default_value=False for disabled features).

The workflow:
1. Create flag in flag service (off by default, 0% rollout)
2. Deploy code (all users get default value = old behavior)
3. Enable flag for internal team (0.1% → validate)
4. Gradual rollout (1% → 10% → 50% → 100%)
5. Clean up flag (remove from code + delete from flag service)

Never deploy code that requires a flag to be already enabled on deploy. Always code for the default-value path to be safe.

Conclusion

Feature flags are one of the highest-leverage tools in modern software delivery. They separate deployment from release, enable safe experimentation, and give operations teams the ability to respond to incidents in seconds rather than minutes.

The key practices:
- OpenFeature standard decouples your code from vendor lock-in
- Ring deployments validate changes incrementally before full rollout
- Kill switches are the most operationally valuable flag type — add them proactively
- Statistical significance testing turns experiments into data, not opinions
- Flag lifecycle management prevents the codebase from becoming an unmaintainable branch forest

Start simple: add a kill switch to your next risky feature. Measure the reduction in rollback frequency and incident duration. The ROI makes itself obvious quickly.

The broader shift feature flags enable is cultural: deployment becomes routine rather than an event. When you can deploy code to production with zero users seeing it, and gradually roll it out with metrics validation at each step, the fear of shipping disappears. Teams ship more often, in smaller increments, with more confidence. That's the real value of the pattern — not just the kill switch, but the deployment culture it enables.


Sources

About the Author

Toc Am

Founder of AmtocSoft. Writing practical deep-dives on AI engineering, cloud architecture, and developer tooling. Previously built backend systems at scale. Reviews every post published under this byline.

LinkedIn X / Twitter

Published: 2026-05-16 · Written with AI assistance, reviewed by Toc Am.

Get These In Your Inbox

Weekly deep-dives on AI engineering, no fluff. Join the newsletter →

Subscribe (free)

Or grab the book ($39, ~100 pages) · Buy me a coffee

Buy Me a Coffee · 🔔 YouTube · 💼 LinkedIn · 🐦 X/Twitter

Sunday, April 12, 2026

Feature Flags in 2026: Progressive Delivery, Kill Switches, and Gradual Rollouts

Feature Flags Overview

Introduction

Deploying software used to feel like jumping off a cliff. You merged a branch, pressed deploy, held your breath, and watched error rates either stay flat or spike into the red. If something went wrong, the options were grim: roll back the entire release, hotfix under pressure, or scramble to isolate the bad code while half your user base hit a broken experience.

Feature flags change this equation entirely. Instead of treating deployment as a single, irreversible moment, flags decouple the act of shipping code from the act of releasing features. You can deploy to production continuously — every day, multiple times a day — while individual features remain hidden behind a flag, visible only to the users or environments you choose.

This is the foundation of progressive delivery: the practice of gradually exposing new features to increasing segments of your audience, with the ability to halt, revert, or adjust at any point. It is one of the most powerful patterns in modern software engineering, and in 2026 it has become an expectation rather than a luxury at companies of any serious scale.

This post goes deep on how feature flags actually work: the evaluation model, the different flag types, how to implement percentage rollouts and user targeting in Node.js and TypeScript, how to choose between self-hosted solutions like Unleash and Flagsmith versus SaaS platforms like LaunchDarkly and Statsig, and how to manage the technical debt that stale flags inevitably accumulate. By the end, you will have enough to build a production-grade flag system from scratch or evaluate which managed solution is the right fit for your team.


The Problem: Deployment Is Not the Same as Release

Most engineering teams have felt the pain of big-bang releases. A feature takes three weeks to build, lives on a long-lived branch, and gets merged the day before launch. The diff is enormous. Review is surface-level because the deadline is imminent. Testing is rushed. And then it ships to 100% of users simultaneously.

If the feature causes a performance regression, all users experience it. If there is a logic bug that only manifests at scale, you discover it in production on the worst possible day. If business requirements change mid-sprint, half the code is already shipped and the other half is in flight — untangling it is miserable.

The same problem appears at a subtler level with database migrations, API versioning, and infrastructure changes. You want to test a new query plan against real production traffic, but only a fraction of it. You want to gradually shift users from a legacy payment processor to a new one. You want to run an A/B test on a checkout flow without spinning up a separate experiment platform.

Feature flags solve all of these scenarios with a single, consistent abstraction: a named conditional that controls whether a code path is active.

if (flagClient.isEnabled('new-checkout-flow', userContext)) {
  return newCheckout(cart);
}
return legacyCheckout(cart);

That single conditional is doing enormous work. It lets you:

  • Deploy the new checkout to production without any users seeing it (flag is off globally)
  • Enable it for internal employees first to catch obvious bugs
  • Roll it to 5% of users, then 20%, then 50%, monitoring metrics at each stage
  • Kill the feature instantly if error rates climb, with no deployment required
  • Permanently remove the flag and the old code path once confidence is established

This is the model. The rest of this post is about doing it well.


How Feature Flags Work

Feature Flag System Architecture

Flag Types

Not all flags are the same. The two primary categories are boolean flags and multivariate flags.

Boolean flags are the simplest form: a feature is either on or off. They are appropriate for kill switches, gradual rollouts, and simple A/B tests. The evaluation result is true or false.

Multivariate flags return one of N values, where N is greater than two. The value can be a string, a number, a JSON object, or an enumeration. Common uses include:

  • String variants: 'control', 'variant-a', 'variant-b' — for multi-arm experiments
  • Number variants: returning a timeout value (500, 1000, 2000 ms) to test performance thresholds
  • JSON variants: returning an entire configuration object, so a single flag controls multiple related settings simultaneously
// Boolean flag
const showNewDashboard = flagClient.getBoolVariation('new-dashboard', context, false);

// String multivariate flag
const checkoutVariant = flagClient.getStringVariation('checkout-flow', context, 'control');
// Returns 'control' | 'single-page' | 'multi-step' | 'guided'

// Number multivariate flag
const timeoutMs = flagClient.getNumberVariation('api-timeout-ms', context, 3000);

// JSON multivariate flag
const searchConfig = flagClient.getJsonVariation('search-config', context, defaultConfig);
// Returns { algorithm: 'bm25', maxResults: 10, enableFuzzy: false }

Multivariate flags are especially powerful because they remove the need for a proliferation of related boolean flags. Instead of use-new-search, enable-fuzzy-search, increase-result-count, you have one search-config flag that returns a coherent configuration object.

The Evaluation Model

Feature flag evaluation is where the real power lives. A flag is not just a stored boolean value — it is a set of rules evaluated against a user context object at runtime.

flowchart TD A([User Request]) --> B[Build User Context\nuser_id, email, plan, region, beta] B --> C{Flag SDK\nLocal Cache} C -->|Cache hit| D[Load Flag Rules] C -->|Cache miss| E[Fetch from\nFlag Service] E --> F[Update Local Cache\nTTL: 30s] F --> D D --> G{Rule 1:\nIs user in\ntarget segment?} G -->|Yes| H[Return\nVariant A] G -->|No| I{Rule 2:\nIs user in\n10% rollout bucket?} I -->|Yes| J[Return\nVariant B] I -->|No| K{Rule 3:\nDefault rule} K --> L[Return\nFallback Value] H --> M([Code Path A]) J --> N([Code Path B]) L --> O([Default Code Path])

The context object is the key to targeting. It typically includes:

interface UserContext {
  key: string;           // stable user ID for consistent bucketing
  email?: string;        // for email-domain targeting
  plan?: string;         // 'free' | 'pro' | 'enterprise'
  region?: string;       // 'us-east-1' | 'eu-west-1'
  betaUser?: boolean;    // explicit opt-in segment
  customAttributes?: Record<string, string | number | boolean>;
}

Rules are evaluated in priority order. The first rule to match determines the result. Common rule types:

  • Individual targeting: user_id IN ['user-abc123', 'user-def456'] — useful for internal QA
  • Segment targeting: plan == 'enterprise' — enable features for paying customers first
  • Percentage rollout: hash the user key into a 0–99 bucket, enable if bucket < threshold
  • Default rule: the fallback that applies when no other rule matches

The percentage rollout mechanism deserves attention because it must be consistent and sticky. The same user must always land in the same bucket, regardless of when or where the evaluation happens. This is achieved by hashing the user key (not a random value):

function getBucketValue(userKey: string, flagKey: string): number {
  // Include flagKey in the hash to prevent identical rollouts across flags
  const input = `${flagKey}.${userKey}`;
  const hash = murmur3(input); // or any fast, consistent hash
  return (hash >>> 0) % 100; // 0–99
}

function isUserInRollout(userKey: string, flagKey: string, percentage: number): boolean {
  return getBucketValue(userKey, flagKey) < percentage;
}

This ensures that a user who is in the 10% rollout for a flag stays in that rollout consistently, and that their rollout status for one flag is independent of their status for another.


Implementation Guide

Building a Minimal Flag Client in TypeScript

Let's build a feature flag client from scratch to understand exactly what the production SDKs are doing under the hood. This is not a production replacement — it is a learning tool and a useful starting point for teams that want to self-host without a full platform.

// types.ts
export interface FlagRule {
  type: 'individual' | 'segment' | 'percentage' | 'default';
  attribute?: string;
  operator?: 'eq' | 'in' | 'lt' | 'gt' | 'contains';
  values?: (string | number | boolean)[];
  percentage?: number;
  variant: string;
}

export interface FlagDefinition {
  key: string;
  enabled: boolean;
  variants: Record<string, string | number | boolean | object>;
  rules: FlagRule[];
  defaultVariant: string;
}

export interface EvaluationContext {
  key: string;
  [attribute: string]: string | number | boolean | undefined;
}
// flag-client.ts
import { createHash } from 'crypto';
import type { FlagDefinition, EvaluationContext, FlagRule } from './types';

export class FeatureFlagClient {
  private flags: Map<string, FlagDefinition> = new Map();
  private cacheExpiry: number = 0;
  private readonly cacheTtlMs: number;
  private readonly flagsEndpoint: string;

  constructor(opts: { endpoint: string; cacheTtlMs?: number }) {
    this.flagsEndpoint = opts.endpoint;
    this.cacheTtlMs = opts.cacheTtlMs ?? 30_000;
  }

  async initialize(): Promise<void> {
    await this.refresh();
  }

  private async refresh(): Promise<void> {
    const res = await fetch(this.flagsEndpoint);
    if (!res.ok) throw new Error(`Flag fetch failed: ${res.status}`);
    const definitions: FlagDefinition[] = await res.json();
    this.flags.clear();
    for (const flag of definitions) {
      this.flags.set(flag.key, flag);
    }
    this.cacheExpiry = Date.now() + this.cacheTtlMs;
  }

  private async ensureFresh(): Promise<void> {
    if (Date.now() > this.cacheExpiry) {
      await this.refresh();
    }
  }

  private getBucket(userKey: string, flagKey: string): number {
    const input = `${flagKey}.${userKey}`;
    const hash = createHash('sha256').update(input).digest();
    // Use first 4 bytes as a uint32
    const value = hash.readUInt32BE(0);
    return value % 100;
  }

  private evaluateRule(rule: FlagRule, context: EvaluationContext): boolean {
    if (rule.type === 'default') return true;

    if (rule.type === 'percentage') {
      const bucket = this.getBucket(context.key, rule.variant);
      return bucket < (rule.percentage ?? 0);
    }

    if (rule.type === 'individual' || rule.type === 'segment') {
      const attribute = rule.attribute ?? 'key';
      const contextValue = context[attribute];
      if (contextValue === undefined) return false;

      switch (rule.operator) {
        case 'eq':
          return contextValue === rule.values?.[0];
        case 'in':
          return rule.values?.includes(contextValue as string | number | boolean) ?? false;
        case 'contains':
          return typeof contextValue === 'string' &&
            typeof rule.values?.[0] === 'string' &&
            contextValue.includes(rule.values[0]);
        case 'lt':
          return typeof contextValue === 'number' &&
            typeof rule.values?.[0] === 'number' &&
            contextValue < rule.values[0];
        case 'gt':
          return typeof contextValue === 'number' &&
            typeof rule.values?.[0] === 'number' &&
            contextValue > rule.values[0];
        default:
          return false;
      }
    }

    return false;
  }

  async evaluate<T>(
    flagKey: string,
    context: EvaluationContext,
    defaultValue: T
  ): Promise<T> {
    await this.ensureFresh();

    const flag = this.flags.get(flagKey);
    if (!flag || !flag.enabled) return defaultValue;

    for (const rule of flag.rules) {
      if (this.evaluateRule(rule, context)) {
        const variant = flag.variants[rule.variant];
        return (variant as T) ?? defaultValue;
      }
    }

    const defaultVariant = flag.variants[flag.defaultVariant];
    return (defaultVariant as T) ?? defaultValue;
  }

  async getBoolVariation(
    flagKey: string,
    context: EvaluationContext,
    defaultValue: boolean
  ): Promise<boolean> {
    return this.evaluate<boolean>(flagKey, context, defaultValue);
  }

  async getStringVariation(
    flagKey: string,
    context: EvaluationContext,
    defaultValue: string
  ): Promise<string> {
    return this.evaluate<string>(flagKey, context, defaultValue);
  }
}

Using the Client in an Express API

// app.ts
import express from 'express';
import { FeatureFlagClient } from './flag-client';
import type { EvaluationContext } from './types';

const app = express();
const flags = new FeatureFlagClient({
  endpoint: 'https://flags.internal.mycompany.com/api/flags',
  cacheTtlMs: 15_000,
});

// Initialize once at startup — client handles refresh internally
await flags.initialize();

app.get('/api/recommendations', async (req, res) => {
  const userId = req.headers['x-user-id'] as string;
  const userPlan = req.headers['x-user-plan'] as string;

  const context: EvaluationContext = {
    key: userId,
    plan: userPlan,
    region: process.env.AWS_REGION ?? 'us-east-1',
  };

  // Check flag — returns false if flag is undefined or user is not in rollout
  const useNewRecommendationEngine = await flags.getBoolVariation(
    'new-recommendation-engine',
    context,
    false
  );

  if (useNewRecommendationEngine) {
    const results = await newRecommendationEngine(userId);
    return res.json({ results, engine: 'v2' });
  }

  const results = await legacyRecommendationEngine(userId);
  return res.json({ results, engine: 'v1' });
});

Kill Switch Pattern

Kill switches are boolean flags that default to true and turn a feature off when triggered. This is the opposite of a rollout flag. The naming convention matters for clarity:

// Kill switch: defaults to ON, turned OFF in an emergency
const featureKilled = await flags.getBoolVariation(
  'kill-new-payment-processor',
  context,
  false  // default: not killed
);

if (featureKilled) {
  return legacyPaymentProcessor.charge(amount);
}

return newPaymentProcessor.charge(amount);

Kill switches should be pre-created for every high-risk feature before deployment. When a production incident occurs, the last thing you want to do is create a new flag, configure it, and deploy it. The flag should already exist and be toggleable in seconds.

Choosing a Rollout Strategy

flowchart TD A([New Feature Ready\nfor Release]) --> B{Is it a\nhigh-risk change?} B -->|Yes - DB migration,\npayment flow, infra| C[Start at 1%\nCanary Rollout] B -->|No - UI change,\nnon-critical feature| D[Start at 10%\nStandard Rollout] C --> E[Monitor for 30 min:\nError rate, latency, logs] D --> F[Monitor for 15 min:\nError rate, latency] E --> G{Metrics\nclean?} F --> G G -->|No — spike detected| H[Kill switch:\nSet flag to 0%] H --> I[Investigate &\nFix Bug] I --> C G -->|Yes| J[Increase to 25%] J --> K[Monitor 30 min] K --> L{Still clean?} L -->|No| H L -->|Yes| M[Increase to 50%] M --> N[Monitor 1 hour] N --> O{Still clean?} O -->|No| H O -->|Yes| P[Increase to 100%] P --> Q[Full Release:\nPlan flag cleanup\nin next sprint]

Comparison and Tradeoffs

Self-Hosted vs SaaS

The build-vs-buy decision for feature flags is not trivial. Here is a practical comparison across the most relevant dimensions.

Self-hosted vs SaaS Flag Platforms
Dimension Unleash (self-hosted) Flagsmith (self-hosted) LaunchDarkly (SaaS) Statsig (SaaS)
Evaluation location SDK-side (local) SDK-side (local) SDK-side (local) SDK-side + edge
Data residency Full control Full control US/EU options US primary
Pricing Free (OSS) + Enterprise Free (OSS) + Cloud From $20k/yr (enterprise) Usage-based
Latency < 1ms (local eval) < 1ms (local eval) < 1ms (local eval) < 1ms (local eval)
Analytics / experiments Basic Basic Full A/B + stats Advanced stats engine
Ops burden High (DB, servers, HA) Medium None None
Audit logs Yes (Enterprise) Yes Yes Yes
Edge / CDN flags No No LaunchDarkly Edge Statsig on Vercel/Cloudflare
Best for Cost-sensitive, data-sovereign Mid-size, mixed cloud Enterprise, full experiments Experiment-heavy products

Choose self-hosted if:
- You have GDPR or data sovereignty requirements that prevent user data leaving your infrastructure
- You are running at a scale where SaaS per-seat or per-MAU pricing becomes expensive (> 10M MAUs)
- You have the DevOps capacity to run and maintain the service reliably

Choose SaaS if:
- You want to ship feature flag infrastructure in days, not months
- You need advanced experimentation capabilities (sequential testing, CUPED variance reduction)
- Your team is small and cannot afford the ops burden of a self-hosted system

Flag Evaluation Performance

One common concern is whether flag evaluation adds latency to request paths. The answer is: it should not, and here is why.

All production-grade feature flag SDKs (LaunchDarkly, Unleash, Flagsmith, Statsig) use a local evaluation model. The SDK downloads a snapshot of all flag rules at startup and on a polling interval (typically 30 seconds). Evaluation is then done entirely in-memory, against local data, with no network call required per evaluation.

The cost of a flag evaluation in this model is roughly:

  • Hash computation: ~1 microsecond
  • Rule traversal: ~5–20 microseconds for a typical ruleset

Compared to a database query (1–10ms) or an external API call (10–200ms), flag evaluation is effectively zero-cost.

The one exception is streaming-based updates. SDKs like LaunchDarkly maintain a persistent SSE connection to receive flag changes in real time (< 200ms propagation), rather than waiting for the next polling interval. This is important for kill switches where a 30-second delay is too long.


Production Considerations

Flag Lifecycle and Stale Flag Debt

Feature flags accumulate. A team that ships two features per week can have 100 flags after a year. Without discipline, these become permanent conditionals in the codebase — dead code paths that no one dares remove, flags that are always on but never cleaned up, and rules that reference segments that no longer exist.

Stale flags are a form of technical debt that compounds over time:

  1. Readability: Code with many flag conditionals is harder to reason about. What code path actually runs in production?
  2. Testing burden: Every combination of flag states is theoretically a different code path to test.
  3. Performance: Even with local evaluation, a ruleset with 500 flags is slower to download and parse than one with 50.

The solution is a flag lifecycle policy enforced through tooling:

// Flag definition with mandatory expiry metadata
interface FlagDefinition {
  key: string;
  enabled: boolean;
  variants: Record<string, unknown>;
  rules: FlagRule[];
  defaultVariant: string;
  metadata: {
    createdAt: string;           // ISO date
    owner: string;               // team or individual
    expiresAt: string;           // ISO date — mandatory
    jiraTicket?: string;         // cleanup ticket
    type: 'release' | 'experiment' | 'kill-switch' | 'ops';
  };
}

Recommended expiry windows:

Flag Type Max Lifetime
Release flag 2 weeks after 100% rollout
Experiment flag Duration of experiment + 1 week
Kill switch Indefinite (but reviewed quarterly)
Ops flag Indefinite (but reviewed quarterly)

Integrate expiry checks into your CI pipeline so that a PR adding a flag without an expiry date fails the build. And track overdue flag cleanup in your sprint velocity — it is real work.

Monitoring and Observability

Flag evaluations should be emitted as metrics and structured logs:

// Instrumented evaluation wrapper
async function evaluateWithMetrics<T>(
  client: FeatureFlagClient,
  flagKey: string,
  context: EvaluationContext,
  defaultValue: T
): Promise<T> {
  const start = performance.now();
  let variant: string | undefined;
  let error: Error | undefined;

  try {
    const result = await client.evaluate<T>(flagKey, context, defaultValue);
    variant = JSON.stringify(result);
    return result;
  } catch (err) {
    error = err as Error;
    return defaultValue;
  } finally {
    const durationMs = performance.now() - start;

    metrics.histogram('feature_flag.evaluation_duration_ms', durationMs, {
      flag: flagKey,
    });

    logger.info('feature_flag.evaluated', {
      flag: flagKey,
      userKey: context.key,
      variant,
      durationMs,
      error: error?.message,
    });
  }
}

Set up dashboards that show:

  • Evaluation volume by flag (which flags are hottest?)
  • Error rate per flag variant (is variant B causing more 5xx than variant A?)
  • Flag propagation latency (how quickly do changes reach production SDKs?)
  • Stale evaluation warnings (flags evaluated after their expiry date)

Trunk-Based Development and Feature Flags

Feature flags are the enabling technology for trunk-based development (TBD) — the practice where all developers commit directly to main at least once per day. With TBD, there are no long-lived feature branches, which eliminates the merge conflict tax and the integration risk of big-bang merges.

The pattern is straightforward: every in-flight feature is wrapped in a flag from day one. Developers commit incomplete code behind a flag that is globally disabled. The code ships to production but is invisible. Once the feature is complete and the flag is enabled, it becomes live — with full rollout control.

// Day 1 commit — feature is incomplete but ships behind a flag
async function handleSearch(query: string, userId: string): Promise<SearchResult[]> {
  const context: EvaluationContext = { key: userId };

  const useNewSearchEngine = await flags.getBoolVariation(
    'new-search-engine',    // globally off — safe to ship incomplete
    context,
    false
  );

  if (useNewSearchEngine) {
    // TODO: This is a stub — full implementation in next commit
    return newSearchEngine.query(query);
  }

  return legacySearch(query);
}

This model has a profound effect on team dynamics. Developers stop hoarding code on local branches. Code review happens in smaller, more reviewable chunks. And the main branch is always deployable, because every in-flight feature is safely dark.

Progressive Delivery Lifecycle

gantt title Progressive Delivery Lifecycle — Feature X dateFormat YYYY-MM-DD axisFormat %b %d section Development Feature flagged (globally off) :done, dev1, 2026-04-01, 2026-04-07 Code review + merge to main :done, dev2, 2026-04-07, 2026-04-08 section Validation Internal team (0% public) :done, val1, 2026-04-08, 2026-04-09 Beta users / employees (5%) :done, val2, 2026-04-09, 2026-04-10 Canary rollout (10%) :done, val3, 2026-04-10, 2026-04-11 section Rollout 25% rollout + metrics review :active, roll1, 2026-04-11, 2026-04-12 50% rollout + A/B analysis : roll2, 2026-04-12, 2026-04-13 100% full release : roll3, 2026-04-13, 2026-04-14 section Cleanup Flag removal ticket created :crit, cln1, 2026-04-14, 2026-04-14 Dead code removed, flag deleted : cln2, 2026-04-14, 2026-04-21 section Contingency Rollback path (kill switch ready) :crit, rb1, 2026-04-08, 2026-04-14

The Gantt diagram above represents the ideal lifecycle. Notice two things: the rollback path (kill switch) is available throughout the entire rollout period, and the cleanup phase is scheduled immediately after full release — not as an afterthought.


Conclusion

Feature flags have matured from a scrappy workaround into a first-class engineering practice. In 2026, progressive delivery is table stakes for any team that cares about deployment safety and velocity.

The key ideas to take forward:

Decouple deployment from release. Shipping code and exposing features to users are two separate events. Feature flags give you control over when the second event happens, independently of the first.

Use the right flag type for the job. Boolean flags for simple rollouts and kill switches. Multivariate flags for experiments and configuration management. Avoid proliferating multiple related booleans when a single JSON flag captures the intent more clearly.

Build with the evaluation model in mind. User context, rule priority, and consistent hashing for percentage bucketing are the core mechanics. Understand them whether you are using an SDK or rolling your own.

Pick your platform based on your constraints. Self-hosted solutions (Unleash, Flagsmith) give you data sovereignty and cost control at the price of operational overhead. SaaS solutions (LaunchDarkly, Statsig) give you speed and advanced experimentation at a cost that scales with your user base.

Treat stale flags as debt, not decoration. Every flag that outlives its purpose is a cognitive tax on every developer who reads that code path. Build expiry enforcement into your process from the start.

Combine flags with trunk-based development. The two practices amplify each other. Trunk-based development eliminates merge risk. Feature flags eliminate release risk. Together, they enable the continuous delivery culture that high-performing engineering organizations depend on.

The most important step is to start. Pick one high-risk feature that is about to ship, wrap it in a flag, and do your first percentage rollout. Watch what you can do when deployment stops being a cliff jump and becomes a dial you turn.


Next up: We will look at how Statsig's experiment platform goes beyond basic A/B testing — sequential testing, CUPED variance reduction, and multi-armed bandits for adaptive rollouts.


Sources

About the Author

Toc Am

Founder of AmtocSoft. Writing practical deep-dives on AI engineering, cloud architecture, and developer tooling. Previously built backend systems at scale. Reviews every post published under this byline.

LinkedIn X / Twitter

Published: 2026-04-14 · Written with AI assistance, reviewed by Toc Am.

Get These In Your Inbox

Weekly deep-dives on AI engineering, no fluff. Join the newsletter →

Subscribe (free)

Or grab the book ($39, ~100 pages) · Buy me a coffee

Buy Me a Coffee · 🔔 YouTube · 💼 LinkedIn · 🐦 X/Twitter

Bigger Is Not the Same as Better. The Job That Moved Is the Phone, Not the Lab.

Bigger is a plan. The phone is the receipt. The brief for this cycle is a question: does bigger always mean better in AI? The 2026 answer i...