Should you build agent email yourself with SES, parsing, and storage?

A capable backend engineer can wire Amazon SES, a Lambda, and an object store in a week. That stack will send mail, catch a webhook, and persist a JSON blob. It is a useful demo. It is not an inbox an AI agent can own for months. The distance between “we received MIME” and “a role can hold a conversation, isolate tenants, and stay deliverable to Google, Yahoo, and Microsoft” is where most teams lose a quarter.

What the week-one stack actually covers

SES is a sending fabric. Receipt rules or a catch-all MX can dump inbound bytes somewhere you control. A parser can extract text. A queue can retry a 500. None of that gives the agent an address it owns, a thread a human can reply into, or a policy that stops outbound mail until a person says yes.

The prototype also tends to share one domain, one IAM role, and one table. That is fine for a spike. It fails when two customers’ agents must never see each other’s threads, or when a compromised key should not dump every mailbox at once.

The work that takes months

Threading is the first silent failure. Replies arrive with mutated subjects, missing In-Reply-To headers, or forwards from a new address. You write heuristics, then a fallback search, then a comment that begs the next engineer not to “simplify” the matcher. Body retention follows: agents need original HTML, stripped text, attachments, and enough history to answer what was already promised without rehydrating MIME from a short-lived log.

Per-tenant isolation is a product requirement, not a table prefix. Each inbox needs its own identity, credentials, and blast radius. Mailbox providers expect SPF, DKIM, DMARC, complaint handling, and a reputation story. Warm-up, bounce classification, and list hygiene are yearly operations, not a launch checklist item you tick once.

Parsing is never finished. Multipart messages, nested forwards, calendar invites, and scanned PDFs all look like “email” until an agent has to act. Retries need idempotency so the same inbound event does not create two tickets and two outbound replies. Attachment extraction, virus scanning, and size limits become part of the product surface whether you planned for them or not.

Yearly upkeep is the real bill

Someone owns blocklist monitoring. Someone rotates keys. Someone updates parsers when a provider changes MIME. Someone answers why a thread forked. Deliverability is a relationship with mailbox providers, not a library version bump. If your company does not want an email platform team, you are still buying that team—either on payroll or as continuous firefighting.

When building is the right call

Build when email is the product, you already operate mail infrastructure, and you can staff deliverability. Build when residency rules, custom routing, or unusual protocols make a hosted inbox the wrong primitive. Do not build because SES looked cheaper on a spreadsheet that omitted engineering time, on-call, and reputation work.

When Agent Inbox is the right call

Agent Inbox treats the mailbox as the API object. One call provisions an address people can reply to. SDKs and MCP give the agent a way to list threads, read bodies, and send. Forwarding into Neuro OS brings a human into the loop when the thread leaves the role’s authority. Outbound defaults to Ask.

from agentinbox import AgentInbox
client = AgentInbox()
inbox = client.inboxes.create(username="hello", domain="agentinbox.space")

The useful distinction is email API versus inbox API: a send pipe versus a mailbox the agent owns. If the job is a conversation—not a one-way receipt—start from the inbox, not from a receipt rule.

Choose build when you intend to run a mail platform. Choose Agent Inbox when the product is the agent role, and email is how that role stays reachable.

Agent Inbox gives each role a mailbox people can reply to, with forwarding into Neuro OS when a human must see the thread. Outbound mail defaults to Ask. Run the role on Neuro OS. To scope the first inbox, get started.

Explore Neuro OS →