Outbox Pattern for Reliable Microservices Messaging in .NET

The sweet shop order book

Imagine a busy sweet shop in your town during Diwali. The shopkeeper, Ramesh, takes orders all day. When a customer asks for two boxes of laddoos, Ramesh does two things. First, he writes the order in his order book. Second, he is supposed to call the kitchen at the back to start making the sweets.

Now think about what can go wrong. Ramesh writes the order in the book, but just as he picks up the phone to call the kitchen, the phone line goes dead. The order is written, but the kitchen never hears about it. The customer waits and waits, and no laddoos come.

Or the opposite happens. Ramesh shouts the order to the kitchen first, the kitchen starts cooking, but then a crowd rushes in and he forgets to write it in the book. Now the kitchen made sweets nobody paid for.

The smart fix is simple. Ramesh only writes in the order book. He never calls the kitchen himself. Instead, a helper boy walks to the kitchen every few minutes, reads the new lines in the order book, and tells the kitchen. The order book is the single source of truth. As long as the order is written down, it will reach the kitchen, even if the helper boy is slow.

This is exactly the Outbox Pattern. Your app writes the order (the data) and a small note (the message) into the same book (the database). A background helper reads the notes later and delivers them to the kitchen (the message broker). Nothing gets lost.

The real problem: the dual-write trap

In microservices, one service often needs to tell other services that something happened. When an Orders service saves a new order, the Shipping service and the Billing service need to know. The usual way is to publish a message to a broker like RabbitMQ, Azure Service Bus, or Kafka.

So your code does two writes:

1. Save the order to the database.
2. Publish an OrderCreated message to the broker.

These are two different systems. A database and a broker cannot share one transaction. This is called the dual-write problem. Here is the danger.

If the app crashes, or the broker is down, right after the database save but before the publish, then the order exists but no one is told. The customer is charged later but nothing ships. This is a silent, painful bug. It does not throw an obvious error in the happy path, so it often hides until production.

The table below shows the four ways a naive dual write can play out.

Only the first row is truly safe. The Outbox Pattern turns every row into a safe one.

The core idea: write the message into the database

Here is the trick that makes it all work. A database can write to two tables in one transaction. That gives you atomicity for free.

So instead of publishing to the broker directly, you write the message as a row into an outbox table inside the same transaction that saves your order. Either both the order and the message are saved, or neither is. There is no in-between.

The order data and the outbox message live in the same database, protected by the same commit. After the commit, a separate background worker (our helper boy) reads unprocessed outbox rows and publishes them to the broker. When it succeeds, it marks the row as processed.

Notice how the broker is now out of the critical path. Your user-facing request finishes the moment the transaction commits. If the broker is down for ten minutes, the messages simply wait safely in the table.

Building the outbox table and message

Let's build a small version by hand so you understand every moving part. First, the message we store. We keep it simple: an id, the type name, the content as JSON, when it was created, and when it was processed.

public class OutboxMessage
{
    public Guid Id { get; set; }
    public string Type { get; set; } = default!;     // e.g. "OrderCreated"
    public string Content { get; set; } = default!;  // JSON payload
    public DateTime OccurredOnUtc { get; set; }
    public DateTime? ProcessedOnUtc { get; set; }     // null = not yet sent
    public string? Error { get; set; }                // last failure, if any
}

A null value in ProcessedOnUtc means the message is still waiting. That is the flag the worker looks for. We also keep an Error column so a poisoned message leaves a trail instead of failing silently.

Saving data and the message together

Now the important part. When we create an order, we add the order and the outbox row to the same DbContext, then call SaveChanges once. Entity Framework Core wraps a single SaveChanges call in one transaction, so both inserts commit together.

public async Task<Guid> CreateOrderAsync(CreateOrder cmd, CancellationToken ct)
{
    var order = new Order(cmd.CustomerId, cmd.Items);
 
    var message = new OutboxMessage
    {
        Id = Guid.NewGuid(),
        Type = nameof(OrderCreated),
        Content = JsonSerializer.Serialize(
            new OrderCreated(order.Id, order.CustomerId, order.Total)),
        OccurredOnUtc = DateTime.UtcNow
    };
 
    _db.Orders.Add(order);
    _db.OutboxMessages.Add(message);
 
    // One SaveChanges = one transaction = both rows, or neither.
    await _db.SaveChangesAsync(ct);
 
    return order.Id;
}

There is no broker call here at all. The method does pure database work. That is what makes it bulletproof. If SaveChangesAsync throws, nothing is saved and the caller gets a clean error. If it succeeds, both rows are safely on disk.

A neat upgrade is to use an EF Core SaveChangesInterceptor. It can collect domain events from your entities and turn them into outbox rows automatically, so your business code never has to remember to add the message. Many teams use this so the outbox becomes invisible plumbing.

The background publisher (the helper boy)

Next we need the worker that reads unprocessed rows and publishes them. In ASP.NET Core this is a BackgroundService that runs on a timer.

public class OutboxProcessor(IServiceProvider services, IBus bus)
    : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken stopToken)
    {
        while (!stopToken.IsCancellationRequested)
        {
            using var scope = services.CreateScope();
            var db = scope.ServiceProvider.GetRequiredService<AppDbContext>();
 
            var messages = await db.OutboxMessages
                .Where(m => m.ProcessedOnUtc == null)
                .OrderBy(m => m.OccurredOnUtc)
                .Take(20)
                .ToListAsync(stopToken);
 
            foreach (var msg in messages)
            {
                try
                {
                    await bus.PublishRawAsync(msg.Type, msg.Content, stopToken);
                    msg.ProcessedOnUtc = DateTime.UtcNow;  // mark done
                }
                catch (Exception ex)
                {
                    msg.Error = ex.Message;  // leave it for the next loop
                }
            }
 
            await db.SaveChangesAsync(stopToken);
            await Task.Delay(TimeSpan.FromSeconds(2), stopToken);
        }
    }
}

Read this carefully, because the order of operations matters. We publish first, then mark the row processed. If the worker crashes right after publishing but before saving the ProcessedOnUtc value, the row stays unprocessed. On the next loop it gets published again. That is why the Outbox gives at-least-once delivery, not exactly-once.

At-least-once, and why duplicates are fine

Because a message can be sent more than once, your consumers must handle duplicates. A consumer that runs twice for the same OrderCreated should not ship two parcels.

The fix is to make consumers idempotent. Each message carries a unique id. The consumer remembers which ids it has already handled (often in an inbox table) and ignores repeats. This pairs beautifully with the Inbox Pattern.

Think of it like a teacher marking attendance. If a student's name is called twice by mistake, the teacher does not mark them present twice. The name is the unique id. The Outbox guarantees the name is always called; the idempotent consumer makes sure double-calling causes no harm.

Concurrency: don't let two workers fight

If you run more than one instance of your service for high availability, you might have two outbox workers reading the same rows at once. Both could publish the same message, doubling your duplicates, or worse, fight over the same rows.

Two common fixes:

PostgreSQL's SELECT ... FOR UPDATE SKIP LOCKED is a clean tool here. Each worker grabs a batch of rows, locks them so no other worker can touch the same rows, and skips any rows already locked. SQL Server has a similar trick using READPAST and UPDLOCK hints. Either way, the goal is the same: each message is claimed by exactly one worker per pass.

Polling versus Change Data Capture

The worker above uses polling: it asks the table "any new rows?" every two seconds. Polling is easy to understand and easy to build. The cost is a tiny constant load on the database and a small delay equal to the poll interval.

For very high scale, there is a smarter way called Change Data Capture (CDC). Instead of asking the table, a tool like Debezium reads the database's transaction log directly. The moment a row is committed to the outbox table, CDC streams it out to the broker. No polling, lower latency, and the publishing concern leaves your application entirely.

Start with polling. It covers the vast majority of apps and is far simpler to operate. Reach for CDC only when polling load or latency becomes a measured problem.

Keeping the table tidy

The outbox table grows forever if you never clean it. Processed rows pile up and slow down your queries. Add a small cleanup job that deletes rows where ProcessedOnUtc is older than, say, a few days. Keep them around long enough to debug issues, then remove them.

Two indexes keep the worker fast:

• An index on ProcessedOnUtc (or a filtered index where it is null) so finding unprocessed rows is instant.
• An index on OccurredOnUtc so ordering is cheap.

Without these, the WHERE ProcessedOnUtc == null scan gets slower as the table grows, and your latency creeps up over time.

Should you build it or use a library?

Building the outbox by hand, as we did, is a great way to learn and is fine for small systems. For production, well-tested libraries save you from subtle bugs around locking, retries, and ordering.

• MassTransit has a first-class transactional outbox. Note that MassTransit moved to a commercial license for new major versions starting in 2025, so review the pricing and terms before you adopt it.
• Wolverine also ships a durable outbox and is a strong open-source option.
• NServiceBus has had a mature outbox for many years in the commercial space.

If you are on .NET 10 (the current LTS) with C# 14, all of these work well. Pick based on your team's budget, support needs, and how much you want to own yourself. There is no shame in a hand-rolled outbox of a hundred lines if it fits your scale.

A full picture

Here is how the whole flow looks once everything is in place, from the user's click to the other services hearing about it.

The user gets a fast, reliable response the moment the transaction commits. Everything after that happens safely in the background, and no message is ever lost, even if the broker has a bad day.

References and further reading

• Pattern: Transactional Outbox — microservices.io
• Transactional Outbox with Azure Cosmos DB — Microsoft Learn
• Outbox Pattern for Reliable Microservices Messaging — Milan Jovanović
• Transactional Outbox Pattern — AWS Prescriptive Guidance

Quick recap

• Sending data to a database and a message to a broker is two writes that cannot share a transaction. This is the dual-write problem, and a crash in between loses messages or creates ghost events.
• The Outbox Pattern writes the message as a row into an outbox table inside the same transaction as your data, so they can never disagree.
• A background worker reads unprocessed rows, publishes them to the broker, then marks them processed. Publish first, mark second.
• This gives at-least-once delivery, so make consumers idempotent to handle duplicates safely, often with the Inbox Pattern.
• Use row locking (like FOR UPDATE SKIP LOCKED) or a single leader so multiple workers don't fight over rows.
• Start with polling; move to Change Data Capture only when scale demands it. Add indexes and a cleanup job to keep the table fast.
• Libraries like MassTransit (now commercial), Wolverine, and NServiceBus provide a battle-tested outbox if you'd rather not build your own.