SE

CI/CD and deployment

Must-know concept1.5 hIntermediate

Blue-green, canary, rolling deployments.

Continuous integration keeps main green. Continuous delivery makes deploys boring. The three deploy shapes — rolling, blue-green, canary — each trade speed against safety. Pick by the risk profile of what you're shipping.

The big idea

A change should go from "code committed" to "running in production" with no manual steps you couldn't automate. The pipeline is your safety net; treat it like product code.

Commit
CItest + lint + build
Artifactimage / binary
Deployrolling / blue-green / canary
Observe + rollback

Continuous integration

Every push runs the same set of checks. The goal isn't "tests pass" — it's "main is always in a deployable state."

# .github/workflows/ci.yml — illustrative
name: ci
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with: { node-version: 20, cache: npm }
      - run: npm ci
      - run: npm run lint
      - run: npm run typecheck
      - run: npm test -- --coverage
      - run: npm run build

A useful checklist:

Fast
Under 10 minutes ideally. Slow CI = developers skip it.
Deterministic
Same input → same output. Flaky tests get quarantined or fixed, not retried.
Hermetic
No "works on my laptop" — pin tool versions, freeze deps with a lockfile.
Required to merge
A green check on the PR; nothing reaches main without it.

Three deploy shapes

AttributeStrategyTrade-off
RollingReplace instances N at a time over minutes.Cheap and simple; mixed-version state during the deploy; slow to roll back.
Blue-greenStand up a full new fleet, switch traffic atomically, keep the old as hot standby.Instant rollback; double compute cost during deploy; tricky for stateful services.
CanaryRoute 1% → 5% → 25% → 100% to the new version, watching metrics at each step.Catches problems on a fraction of users; complex to set up; best for high-risk releases.

Canary in practice

The hard part is the automated decision: when do you promote the next step?

  1. Define success criteria

    Latency p99 not worse than baseline by X%; error rate not above baseline by Y%.

  2. Run for a bake period

    Long enough to catch hourly patterns, short enough that the deploy isn't all day. 10–30 minutes per stage is common.

  3. Promote or roll back automatically

    Don't make the on-call human-judge each step at 3am.

  4. Always be able to abort

    A one-click "send 100% back to old version" is mandatory.

Feature flags

The other deploy lever: ship the code dark, enable the feature later with a flag.

if (flags.isEnabled('checkout_v2', { userId })) {
  return runNewCheckout(...);
}
return runOldCheckout(...);

You can:

  • Roll out by % of users.
  • Roll out by region or cohort.
  • Kill a buggy feature in seconds without a deploy.

Cost: technical debt — old branches must be removed once the flag is fully on. Track flag age; delete ruthlessly.

Rollback discipline

Three rules:

  1. Always be deployable. Roll forward usually beats roll back, but you must be able to do either fast.
  2. Roll back what you can revert. Reverting an app version is easy; reverting a migration is hard. Migrate in expand-contract steps.
  3. Practise rollbacks. Schedule game days where you roll back a fake change. The first time you do it in anger is not when you want to learn.

In practice

The pipeline is product code. Review changes to it, version it, write tests for it. The team that invests an extra week in a fast, reliable CI/CD pipeline saves it back in the first week.

Key takeaways

  • CI keeps main always deployable; CD turns deploys into a non-event.
  • Pipelines must be fast, deterministic, hermetic, and required to merge.
  • Rolling is cheap; blue-green gives instant rollback; canary catches issues before everyone sees them.
  • Migrate schemas in two deploys; never combine a column rename with a code change.
  • Feature flags decouple deploy from release — but track their age and delete ruthlessly.

Checkpoint questions

Use these to test whether the lesson is clear enough to explain without rereading.

  1. 1What checks should run before code is allowed to deploy?
  2. 2How do rolling, blue-green, and canary deployments reduce risk differently?
  3. 3What signal would make you stop or roll back a deployment?
  4. 4Which deployment step should be automated first for a small team?

References

External resources for going deeper after the lesson above.