Safer Publishing Workflows for SES Templates

Most teams have a more rigorous process for deploying a typo fix in their marketing site than for changing the password reset email that 50,000 customers will see this month.

That sentence sounds harsh until you check it. The marketing site goes through a PR with a Vercel preview, two reviewers, and a "deploy preview" check. The password reset template goes through someone clicking save in the SES Console.

The asymmetry exists because tooling for application deploys has been productized into oblivion — every CI vendor has thought about preview environments, gated rollouts, and rollback. Tooling for SES template deploys hasn't. Most teams build something or live with the gap.

This post is about what a real publishing workflow for SES looks like and how to assemble it from primitives you already have.

Why "publish on save" is the wrong default

The Console workflow is publish-on-save. Edit the template, click save, the change is live for the next send. There is no preview, no canary, no review gate, no rollback button.

This is fine for the demo template you're building during your AWS evaluation. It's not fine for production.

The failure modes that publish-on-save creates:

No rollback affordance. If the new template is bad, the only "rollback" is editing it back to the previous version — which you don't have, because the save overwrote it. Some teams keep the previous HTML in a ticket; most don't.

No preview. A template that renders fine in the Console preview can render badly in actual email clients. Outlook strips CSS. Gmail folds long lines. Apple Mail respects dark mode. The Console preview lies.

No review. The save button is the same button anyone with UpdateTemplate permission can press, alone, with no second pair of eyes.

No environment promotion. Production is the first place the change is seen. Staging, if it exists, isn't on the critical path.

No audit beyond CloudTrail. The why and the who are absent.

A real workflow fixes all of these without making every typo fix into a four-hour ceremony.

The shape of a good workflow

A publishing workflow that treats templates like production has six elements. You don't have to implement all of them on day one, but eventually you want all of them.

1. Source of truth outside SES. Git, IaC, or a control layer. Templates live somewhere with history, diffs, and access control.
2. Pre-publish review. A second human looks at the change before it goes to customers.
3. Pre-publish preview. The change is rendered in real email clients (or a representative subset) before publish.
4. Environment promotion. The change is published to staging first, then promoted to production.
5. Optional canary. For high-risk changes, the production publish is gated by sending to a small subset first.
6. One-click rollback. The previous version is preserved and can be restored quickly.

Below is each one, concretely.

Pre-publish review

The cheapest, highest-impact intervention. Every template change goes through a review surface before it ships.

If you're using Git, this is a PR. Review configuration:

# .github/CODEOWNERS
templates/                @engineering-leads @lifecycle
templates/security_*.html @security
templates/billing_*.html  @finance @engineering-leads

Different templates get different reviewers based on what they are. Password reset and MFA emails get security review. Billing emails get finance review. Marketing-tinged transactional emails (welcome, trial expiry) get lifecycle review.

The review checklist a reviewer should mentally run through:

• Does the change preserve every template variable that the application code expects? {{customer_name}} becoming {{ customer_name }} is a silent break in some Handlebars-style templates and not in others — know which behavior your engine has.
• Does the change preserve the language and tone that the customer is used to? Sudden stylistic changes confuse trust.
• Are links going to expected destinations?
• Does the subject line still respect the deliverability rules — no all-caps, no excessive punctuation?
• Is there a plain-text version, and does it match the HTML?

Make the checklist a PR template comment so reviewers don't have to remember it.

Pre-publish preview

A Console preview shows you what the template looks like in a sandbox-rendered iframe. That iframe is not Outlook, not Gmail, and not Apple Mail Dark Mode.

Real preview options:

Litmus, Email on Acid, or similar. Send the rendered template to dozens of real email clients and screenshot each one. Integrate this into CI on PRs that touch templates. The cost is per-render; for low-volume teams it's affordable.

Self-hosted with a small client matrix. A CI step that renders the template via Mailpit or a similar local mail server, then pulls it back through headless browser screenshots emulating a couple of clients. Cheaper, less coverage.

Send to a seed list. A staging environment fires the templated email to a small internal seed list every time staging picks up the change. Reviewers eyeball the seed inbox before the production publish.

The seed list approach is underrated. It costs nothing, requires no third-party tooling, and reveals the exact thing you care about: what the email looks like in your team's actual inboxes. The downside is it doesn't fan out across many clients — but for teams whose customers are mostly on Gmail or Outlook, the seed inbox covers the cases that matter.

Environment promotion

Production should never be the first SES environment to see a template change. Staging gets it first, even if staging just means a different AWS account or region with the same template namespace.

The promotion flow:

1. Change merged to main.
2. CI publishes to staging SES.
3. Automated checks run: render checks, link checks, variable presence checks, seed-list send.
4. A human approves promotion to production.
5. CI publishes to production SES.

The human approval can be lightweight — a Slack /approve command, a button on a CI job, a label on the PR. It exists so that no template change lands in production without somebody having seen the staging behavior.

Some teams skip the human approval and rely on automated checks alone. This is fine if your automated checks are comprehensive. They usually aren't. The human approval is cheap insurance against the failure modes your checks don't cover.

Canary releases for high-stakes templates

For most templates, a binary publish — the new version goes live for everyone — is fine. For some, it isn't.

Templates that warrant canary:

• Templates with significant logic changes (new variables, new conditional blocks).
• Templates that just had a bug fix where you're not 100% sure of the fix.
• Templates for high-volume sends where a bad version could blast tens of thousands of customers in an hour.

The pattern from the versioning post — blue/green template names with an application-layer pointer — is the foundation here. The publish writes the new version into the inactive slot. The application reads a routing key that determines what fraction of sends go to each slot. Start at 1%, ramp to 10%, ramp to 100% over an hour or a day, depending on volume and risk.

def template_for_send(name: str, user_id: str) -> str:
    routing = ssm.get_parameter(Name=f"/ses/templates/{name}/routing")["Parameter"]["Value"]
    config = json.loads(routing)  # {"green": 0.1, "blue": 0.9}

    bucket = (hash(user_id) % 100) / 100
    return "green" if bucket < config["green"] else "blue"

The canary observable is bounce rate, complaint rate, and customer support tickets correlated to the new version. If any of those spike during the ramp, flip the routing back to 100% blue and investigate.

Canary is overhead. Not every template needs it. The template that needs it is the one whose failure mode is "we just emailed 30,000 customers something incorrect about their account."

Rollback playbooks

A rollback procedure you've never tested isn't a procedure. The day you need to roll back a template, you don't want to be reading documentation.

Rollback components:

Pre-publish snapshot. Before publishing the new version, capture the current production version somewhere. Git captures it via commit history. IaC captures it via state. A control layer captures it natively. The point is that the moment of publish is also the moment "the previous version" is archived.

One-step revert. From the snapshot to live SES, with one command or one click. For Git, this is git revert <sha> and a CI redeploy. Five minutes. For a control layer, it's a button that publishes the previous version. Subsecond.

Logged. The rollback is itself an auditable event. Who rolled back, when, why, what version was restored.

Practiced. Quarterly, deliberately, on a non-critical template. The muscle memory matters. The first time you do a rollback in anger should not be the first time you do a rollback.

Write the runbook now. Put it next to your incident runbooks. Test it.

Publish-time notifications and after-action

Every template publish — staging or production — should generate a notification. Slack, email, whatever channel your team watches.

The minimum notification:

Template password_reset published to production by Maria (PR #1234). Diff: subject changed; one variable added.

Why notifications matter: they create a passive review surface. Even if nobody is paying close attention to staging, somebody will notice the production notification. Bad publishes are caught faster when more people are passively aware.

After-action habit: once a month, review the publish log. Were any rollbacks triggered? Were there publishes that, in retrospect, should have gone through canary but didn't? Were there publishes that landed on a Friday afternoon that nobody approved? Use the data to refine the workflow.

A reference workflow, end-to-end

For a B2B SaaS with two engineers, a marketer who edits transactional copy, and three environments:

1. Source of truth. Templates live in a Git repo, one JSON file per template, with HTML and text bodies as separate files referenced by path.
2. Editing surfaces. Engineers edit via PR. The marketer edits via a control layer UI that produces PRs (or directly produces an audit-tracked change set).
3. Review. PRs reviewed by CODEOWNERS-defined reviewers. Security templates require security review. Billing templates require finance review. All other templates require any engineer.
4. Pre-publish checks. CI on the PR runs: variable validation, render check, link check, send-to-seed-list-staging.
5. Merge. Merge to main triggers CI publish to staging SES.
6. Promotion. A staging-stable label on the PR or a CI job approval triggers production publish.
7. Production publish. For low-risk templates, direct publish. For high-risk templates, blue/green with canary ramp.
8. Notification. Slack post on every publish.
9. Rollback. git revert + CI redeploy for low-risk; routing flip for high-risk.
10. Audit. Each publish records who, what, when, why, and the content diff. Retained 24 months.

Most of this is achievable with Git, GitHub Actions, and a small amount of custom code. Some of it — the marketer-friendly UI, the role-based review enforcement that goes beyond CODEOWNERS, the audit log retention with content diffs — is where teams either build it or buy a control layer.

Where Sovy fits

Sovy implements this workflow as a managed product. Drafts are first-class. Reviews are role-aware. Promotion across environments is a deliberate action, not a CI artifact. Rollback is one click and is itself logged. The marketer who wants to fix a typo in the renewal email gets a UI; the engineer who wants Git in the loop gets webhooks and an API. The audit log captures every step.

The argument for Sovy is the argument for any control layer: you can build this yourself, but you'll spend three quarters doing it well, and the maintenance burden is permanent. If transactional email is core to your product, building is reasonable. If it's important but not differentiating, buying lets your engineers go work on the thing that is.

If you're going to build it: start with Git as source of truth, add a publish CI job, add a staging environment, add a seed-list check, and only then worry about canary. Most of the safety comes from the first three steps.

---

Sovy is a control layer for Amazon SES templates with built-in review, approval, and rollback workflows. If your team is shipping templates with publish-on-save and you're feeling the gap, we'd like to hear from you.