<?xml version="1.0" encoding="utf-8"?>
<rss version="2.0" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/">
    <channel>
        <title>Milan's .NET Weekly</title>
        <link>https://milanjovanovic.tech</link>
        <description>Every Saturday morning I will send you 1 actionable tip on .NET that you can easily implement.</description>
        <lastBuildDate>Mon, 06 Jul 2026 15:59:55 GMT</lastBuildDate>
        <docs>https://validator.w3.org/feed/docs/rss2.html</docs>
        <generator>Feed for Milan's .NET Weekly</generator>
        <image>
            <title>Milan's .NET Weekly</title>
            <url>https://milanjovanovic.tech/profile.png</url>
            <link>https://milanjovanovic.tech</link>
        </image>
        <copyright>All rights reserved 2026, Milan Jovanović</copyright>
        <item>
            <title><![CDATA[Build Your Own VPN With Tailscale]]></title>
            <link>https://milanjovanovic.tech/blog/build-your-own-vpn-with-tailscale</link>
            <guid isPermaLink="false">build-your-own-vpn-with-tailscale</guid>
            <pubDate>Sat, 04 Jul 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[Most of what runs on your servers was never meant to be public: databases, dashboards, admin panels, internal APIs. Tailscale connects your machines into one private, encrypted network so they can talk to each other (and to you) while the public internet sees nothing. Here's what it is and how to wire up applications across multiple VPSs with zero open ports.]]></description>
            <content:encoded><![CDATA[<p>Right now, a bot is scanning the internet for your database.
Not yours in particular. Every database, on every server with a public IP, around the clock.
The moment something on your VPS gets a public port, you're in that game whether you meant to join or not.</p>
<p>And most of what runs on a server was never built to play it.
The database.
The admin panel.
The metrics dashboard.
The internal API only your other services call.</p>
<p>None of that is meant for the public.
Yet the standard tutorial hands each one a public port anyway, and with it a second job: certificates, login pages, IP allowlists, and bots probing around the clock.</p>
<p>There's a better default: keep those services private, reachable only by you and by each other, and expose nothing.
This issue shows you how, using <a href="https://tailscale.com"><strong>Tailscale</strong></a>.</p>
<h2>What Tailscale Actually Is</h2>
<p>A <strong>VPN</strong> (virtual private network) is an encrypted tunnel between machines over the public internet.
Traffic inside it stays private, even across networks you don't control.</p>
<p><strong>Tailscale</strong> uses that idea to connect <em>your own</em> machines (PCs, servers, phones) into a single private network only your devices can see, called a <strong>tailnet</strong>.
It runs on <a href="https://www.wireguard.com/"><strong>WireGuard</strong></a>, a modern, heavily audited encryption protocol, and manages all the keys and addresses for you.</p>
<p>The shape is what sets it apart from an old-school VPN.
A traditional VPN is <strong>hub and spoke</strong>: every device dials one central server, and all traffic funnels through it.
Tailscale builds a <strong>mesh</strong>, where every device connects <em>directly</em> to every other device over its own encrypted tunnel.</p>
<p><img src="/blogs/mnw_201/hub_and_spoke_vs_mesh.png" alt="A traditional hub-and-spoke VPN where all traffic flows through one central server, next to a Tailscale mesh where every device has a direct encrypted tunnel to every other device"></p>
<p>No server sits in the traffic path.
Your app-to-database traffic takes the shortest route between the boxes, and there's nothing to babysit.</p>
<p>How do devices find each other with no server in the middle?
Tailscale splits the job: a <strong>coordination server</strong> keeps a directory of your devices and their keys (a phone book), while your <strong>actual data</strong> flows directly between devices, encrypted end to end.</p>
<p><img src="/blogs/mnw_201/coordination_vs_data_plane.png" alt="Tailscale"></p>
<p>The key property: devices only ever dial <em>outward</em>, to the coordination server and to each other.
Nothing has to connect <em>inward</em>, which is what lets us close every firewall port next.</p>
<h2>Connecting Two VPSs With Zero Open Ports</h2>
<p>Installing Tailscale is two commands per machine (plus the app on your PC):</p>
<pre><code class="language-bash">curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up
</code></pre>
<p>Open the login URL it prints, approve the device, and it joins your tailnet with a <strong>stable private IP</strong> (in the <code>100.x.y.z</code> range) and a name that never change, on any network.</p>
<p>Here's the setup we'll build:</p>
<ul>
<li><code>vps-app</code> (<code>100.64.0.2</code>) runs a .NET API in <a href="https://www.docker.com">Docker</a>.</li>
<li><code>vps-data</code> (<code>100.64.0.3</code>) runs <a href="https://www.postgresql.org">Postgres</a> and <a href="https://grafana.com">Grafana</a> in Docker.</li>
<li>The API queries Postgres across the boxes, you reach everything from your PC, and the internet sees none of it.</li>
</ul>
<p><img src="/blogs/mnw_201/private_apps_over_tailnet.png" alt="A PC, an API VPS, and a data VPS connected in one tailnet: the PC calls the API by tailnet name, the API reaches Postgres on the other VPS by tailnet IP, and a public internet node is blocked with no inbound ports open"></p>
<p><strong>Close the firewall.</strong>
Because Tailscale only dials outward, your cloud firewall needs no inbound rules to reach these boxes.
Confirm SSH works over the tailnet, then delete the public port-22 rule so SSH stops existing as far as the internet is concerned.</p>
<p><strong>Bind services to the tailnet IP.</strong>
This is the step that makes &quot;zero open ports&quot; true.
Publishing a Docker port the usual way (<code>-p 5432:5432</code>) binds it to <code>0.0.0.0</code>, every interface.
Publish private services on the <strong>tailnet IP only</strong> instead. On <code>vps-data</code>:</p>
<pre><code class="language-yaml">services:
  postgres:
    image: postgres:18
    restart: unless-stopped
    environment:
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    ports:
      - '100.64.0.3:5432:5432'   # tailnet IP, not 0.0.0.0
    volumes:
      - pgdata:/var/lib/postgresql/data

  grafana:
    image: grafana/grafana:12.1.0
    restart: unless-stopped
    ports:
      - '100.64.0.3:3000:3000'
    volumes:
      - grafana:/var/lib/grafana

volumes:
  pgdata:
  grafana:
</code></pre>
<p>Now Postgres has no public endpoint at any layer, yet every device on your tailnet reaches it.</p>
<p><strong>Wire the app to the other box</strong> with an ordinary connection string pointed at the data box's stable tailnet IP:</p>
<pre><code class="language-yaml">services:
  api:
    image: ghcr.io/milanjovanovic/api:latest
    restart: unless-stopped
    ports:
      - '100.64.0.2:8080:8080'
    environment:
      ConnectionStrings__AppDb: 'Host=100.64.0.3;Port=5432;Database=app;Username=app;Password=${POSTGRES_PASSWORD}'
</code></pre>
<p>Then from your PC, on any network:</p>
<pre><code class="language-bash">curl http://vps-app:8080/health
psql -h vps-data -p 5432 -U app app
</code></pre>
<p>Look back at everything this setup let you skip.
WireGuard encrypts every byte, so TLS certificates never entered the picture.
Grafana went up without a reverse proxy or a login page, and you reached each service by its tailnet name instead of a DNS record.
The services run, but the internet can't see them.</p>
<h2>The Payoff</h2>
<p>Once your machines share one private network, every internal service (databases, queues, dashboards, admin panels, service-to-service APIs) stops being a public endpoint you have to defend.
It becomes a private address you simply connect to.</p>
<p>This is the exact pattern I run for <a href="https://katabench.com"><strong>Katabench</strong></a>, the coding platform I'm building.
One reverse proxy on 80 and 443 is public, because users load the app through it.
Everything else (the <strong>deployment panel</strong>, Postgres, the message queue, Grafana, all the telemetry) lives on the tailnet with no public hostname at all.</p>
<p>The mental model that makes it stick: <strong>a public hostname is something a service has to earn</strong>, and only when an outside party genuinely must reach it.
Everything else stays private by default.</p>
<p>Fifteen minutes of setup, zero open ports, and your infrastructure disappears from the public internet without losing an ounce of convenience.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_201.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Getting Started With NATS JetStream in .NET]]></title>
            <link>https://milanjovanovic.tech/blog/getting-started-with-nats-jetstream-in-dotnet</link>
            <guid isPermaLink="false">getting-started-with-nats-jetstream-in-dotnet</guid>
            <pubDate>Sat, 27 Jun 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[NATS almost never comes up when .NET developers talk about message queues, and that's a shame. It's a tiny, fast, single-binary broker, and JetStream adds durable, at-least-once delivery on top. Here's why it's worth a look and how to wire it into an ASP.NET Core app.]]></description>
            <content:encoded><![CDATA[<p>When .NET developers need a message queue, they reach for <a href="event-driven-architecture-in-dotnet-with-rabbitmq"><strong>RabbitMQ</strong></a>, <a href="messaging-made-easy-with-azure-service-bus"><strong>Azure Service Bus</strong></a>, or a Postgres table.</p>
<p>NATS almost never comes up.
That's a shame: it's quietly become one of my favorite tools for this.</p>
<p>NATS is a messaging system written in Go that runs as a single binary with no external dependencies.
<strong>JetStream</strong>, its durable layer, turns it into a real queue with at-least-once delivery.
And the <a href="https://github.com/nats-io/nats.net"><strong>.NET client</strong></a> is a pleasure to work with.</p>
<h2>Core NATS vs JetStream</h2>
<p>NATS has two layers, and the difference matters.</p>
<p><strong>Core <a href="https://nats.io">NATS</a></strong> is fire-and-forget pub/sub.
You publish to a subject, and whoever is subscribed at that moment gets it.
If no one is listening, the message is gone, which suits live notifications but not a work queue.</p>
<p><strong><a href="https://docs.nats.io/nats-concepts/jetstream">JetStream</a></strong> is the persistence layer on top.
It captures messages published to a subject into a stream on disk, so a consumer can read them later, even after a restart.
That persistence is what turns a subject into a durable queue.</p>
<p><img src="/blogs/mnw_200/core_nats_vs_jetstream.png" alt="Core NATS drops a message when no subscriber is online; JetStream persists it to a file-backed stream and delivers it later"></p>
<h2>Why It's Worth a Look</h2>
<p>A few things stood out coming from the usual brokers:</p>
<ul>
<li><strong>Tiny.</strong> The official server image is about <strong>18 MB</strong>, a single Go binary with no ZooKeeper or Erlang to babysit.</li>
<li><strong>Fast.</strong> Core NATS pushes <strong>millions</strong> of small messages per second on a single node.
JetStream adds disk persistence, so it's slower, but still comfortably in the <strong>hundreds of thousands</strong> per second.</li>
<li><strong>Cheap to run.</strong> A server idles in tens of megabytes of RAM, so it runs right next to your app.</li>
<li><strong>Flexible per stream.</strong> Each <a href="https://docs.nats.io/nats-concepts/jetstream/streams">stream</a> sets its own storage and retention, so one server can host a cache and a strict work queue side by side.</li>
</ul>
<h2>Set It Up</h2>
<p>You need the server and two NuGet packages.</p>
<p>Run the server with JetStream enabled.
<code>-js</code> turns it on, and <code>-sd</code> points it at a directory so streams survive a restart:</p>
<pre><code class="language-yaml"># docker-compose.yml
nats:
  image: nats:2.14-alpine
  command: ['-js', '-sd', '/data']
  ports: ['4222:4222']
  volumes:
    - nats-data:/data
  restart: unless-stopped
</code></pre>
<p>Add the client and its dependency-injection integration:</p>
<pre><code class="language-bash">dotnet add package NATS.Net
dotnet add package NATS.Extensions.Microsoft.DependencyInjection
</code></pre>
<p>Then wire it into <code>Program.cs</code>.
<code>AddNatsClient</code> registers one multiplexed, self-reconnecting connection, and the next line exposes a JetStream context to inject anywhere:</p>
<pre><code class="language-csharp">// Program.cs
builder.Services.AddNatsClient(nats =&gt;
    nats.ConfigureOptions(opts =&gt; opts with { Url = &quot;nats://localhost:4222&quot; }));

builder.Services.AddSingleton(sp =&gt;
    sp.GetRequiredService&lt;INatsConnection&gt;().CreateJetStreamContext());
</code></pre>
<h2>Publish a Job</h2>
<p>With the JetStream context in DI, a Minimal API endpoint publishes in one call.
<code>Job</code> is a plain record, and <code>NATS.Net</code> serializes it to JSON for you, so you work with typed messages, no extra setup.
<code>EnsureSuccess</code> throws if the stream didn't store the message:</p>
<pre><code class="language-csharp">app.MapPost(&quot;/jobs&quot;, async (CreateJob request, INatsJSContext js, CancellationToken ct) =&gt;
{
    var job = new Job(Guid.NewGuid(), request.Payload);

    PubAckResponse ack = await js.PublishAsync(&quot;jobs.work&quot;, job, cancellationToken: ct);
    ack.EnsureSuccess();

    return Results.Accepted($&quot;/jobs/{job.Id}&quot;);
});
</code></pre>
<p><img src="/blogs/mnw_200/nats_job_pipeline.png" alt="A producer publishes to a work-queue stream, and a pool of workers competes on one durable pull consumer"></p>
<h2>Process Jobs in a Worker</h2>
<p>A <code>BackgroundService</code> is the natural home for the consumer.
It creates the stream and durable consumer on startup, then pulls messages in a loop.
Every running instance shares the <code>workers</code> consumer, so they compete for jobs and each runs once:</p>
<pre><code class="language-csharp">public class JobWorker(INatsJSContext js) : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken ct)
    {
        await js.CreateStreamAsync(new StreamConfig(&quot;JOBS&quot;, [&quot;jobs.work&quot;])
        {
            Retention = StreamConfigRetention.Workqueue, // a queue: acked messages are removed
            Storage   = StreamConfigStorage.File         // durable: survives a restart
        }, ct);

        var consumer = await js.CreateOrUpdateConsumerAsync(&quot;JOBS&quot;, new ConsumerConfig(&quot;workers&quot;)
        {
            AckPolicy  = ConsumerConfigAckPolicy.Explicit,
            AckWait    = TimeSpan.FromSeconds(30), // must exceed your worst-case processing time
            MaxDeliver = 5                         // drop a poison message after 5 tries
        }, ct);

        await foreach (var msg in consumer.ConsumeAsync&lt;Job&gt;(cancellationToken: ct))
        {
            await ProcessAsync(msg.Data, ct);          // the side effect
            await msg.AckAsync(cancellationToken: ct); // then ack
        }
    }
}
</code></pre>
<p>Register it with <code>builder.Services.AddHostedService&lt;JobWorker&gt;()</code>.
The worker is a singleton, so resolve scoped dependencies like <code>DbContext</code> through <code>IServiceScopeFactory</code>.</p>
<p>Two stream settings shape how the queue behaves.</p>
<p><code>Storage</code> is <code>File</code> (on disk, survives restarts) or <code>Memory</code> (faster, but gone on restart).</p>
<p><code>Retention</code> controls when a message leaves the stream:</p>
<ul>
<li><code>Limits</code> (the default) keeps every message until it hits an age, size, or count limit. The stream is a replayable log, and reading a message doesn't remove it.</li>
<li><code>Workqueue</code> drops a message the moment a consumer acks it, so the stream itself is the queue. Messages are delivered in publish order, oldest first (FIFO).</li>
<li><code>Interest</code> keeps a message only while a consumer still needs it, then drops it once every interested consumer acks.</li>
</ul>
<p>For a job queue: <code>Workqueue</code> on <code>File</code>, as in the worker above.</p>
<h2>Acknowledge After the Side Effect</h2>
<p>Look closely at the worker loop: it processes first, then acks.
That order is the rule that makes JetStream reliable, and most quickstarts skip it.</p>
<p><strong>Acknowledge the message <em>after</em> the side effect, never before.</strong></p>
<p>JetStream gives you at-least-once delivery.
If a worker runs a job and crashes before acking, JetStream redelivers it.
But ack before the work is finished, and a crash leaves the job marked done with nothing to show for it.</p>
<p><img src="/blogs/mnw_200/ack_after_side_effect.png" alt="A worker fetches a job, runs it, persists the result, and only then acks; a crash before the ack causes a redelivery"></p>
<p>The flip side is that a job can run more than once, so your handler has to be idempotent.
The usual fix is to track the messages you've already handled and skip duplicates, in the same transaction as the side effect.
I covered the full pattern in <a href="the-idempotent-consumer-pattern-in-dotnet-and-why-you-need-it"><strong>The Idempotent Consumer Pattern in .NET</strong></a>.
At-least-once delivery only holds up when the handler reading the stream is idempotent.</p>
<h2>Summary</h2>
<p>NATS JetStream gives you a durable, at-least-once work queue from a single 18 MB binary, and it slots into an <a href="http://ASP.NET">ASP.NET</a> Core app cleanly: publish from an endpoint, process in a <code>BackgroundService</code>, ack after the work is done.</p>
<p>I went in skeptical, half-expecting to miss RabbitMQ.
It won me over: easy to operate, no surprises, and it clusters with Raft-based replication when a bigger load calls for it.
It's now the first thing I reach for when I need a queue and don't want to think much about the broker.</p>
<p>If you haven't tried it, spin up the container and publish a message.
That's all there is to getting started.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_200.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[The Modular Monolith Boundary I Couldn't Take Back]]></title>
            <link>https://milanjovanovic.tech/blog/the-modular-monolith-boundary-i-couldnt-take-back</link>
            <guid isPermaLink="false">the-modular-monolith-boundary-i-couldnt-take-back</guid>
            <pubDate>Sat, 20 Jun 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[We built the system exactly the way a modular monolith is supposed to be built, and that's how I ended up with two modules I couldn't pull apart a year later. Here's how a reasonable boundary decision turns into one you can't undo, and the early signals I walked right past.]]></description>
            <content:encoded><![CDATA[<p>We built the system the way you're supposed to: a <a href="what-is-a-modular-monolith"><strong>modular monolith</strong></a>, one module per business domain, modules talking to each other through events so nothing was tightly coupled.</p>
<p>The system let dealers and customers build an order together on a showroom floor.
So giving that order-building collaboration its own module, separate from the product catalog, was an easy decision to defend.
They were two distinct domains, and separating them into modules is textbook.</p>
<p>It took about a year, and a piece of business context none of us had at the start, for that decision to quietly stop being reversible.</p>
<h2>The Setup</h2>
<p>This was the <a href="what-rewriting-a-40-year-old-project-taught-me-about-software-development"><strong>40-year-old system I wrote about rewriting</strong></a>.
The backend is a manufacturing ERP, with the dealer ordering tool layered on top.</p>
<p>Two areas felt obviously distinct.
The <strong>Catalog</strong> module owned products, configurations, options, and pricing.
The <strong>Collaboration</strong> module owned the back-and-forth of building an order: drafts, comments, approvals, revisions.
It looked like a clean separation: different responsibilities, different parts of the team.</p>
<p><img src="/blogs/mnw_199/clean_boundary_day_one.png" alt="Day one: Catalog and Collaboration as two cleanly separated modules, linked only by a single asynchronous events flow"></p>
<p>Splitting them was the deliberate call, not a reflex, and every choice that followed held up on its own.</p>
<h2>The Slow Snowball</h2>
<p>The trouble started as ordinary feature work.</p>
<p>A collaboration screen needed product options, so it read from the catalog.
Then it needed live pricing too.
Then a requirement landed: an order had to reflect catalog changes immediately.
None of them looked like an architecture decision.</p>
<p>But every one of them added a thread between <code>Collaboration</code> and <code>Catalog</code>.
The two modules I had carefully separated were weaving themselves back together, one feature at a time.
The boundary was still there in the folder structure.
It had stopped being there in any way that mattered.</p>
<p><img src="/blogs/mnw_199/coupling_a_year_later.png" alt="Catalog and Collaboration a year later, linked by events, two reads, and a synchronous price check"></p>
<h2>The Assumption That Aged Badly</h2>
<p>The deeper problem was the communication style, and it's the decision I'd still defend hardest.
Across the whole rewrite, modules <a href="modular-monolith-communication-patterns"><strong>communicated asynchronously</strong></a>, through events.
It kept modules decoupled during a high-stakes migration and let us replace the legacy system one piece at a time.
Catalog publishes an event, Collaboration reacts when it gets to it, and the two stay independent.</p>
<p>That held up right until the business needed an order to be correct <em>now</em>.
A dealer changes a configuration, and the price has to be right the instant they hit save.
Eventual consistency had been the right assumption for every requirement we knew about.
The requirement that broke it didn't exist yet when we drew the boundary.</p>
<p><img src="/blogs/mnw_199/eventual_vs_immediate_consistency.png" alt="Eventual consistency on day one versus the later need for immediate consistency between Collaboration and Catalog"></p>
<p>You can't bolt immediate consistency onto an asynchronous boundary.
So we did what everyone does: synchronous calls, shared transactions, and workarounds to paper over the gap.
Fix by fix, the system was telling us that, given the consistency the business now needed, these two belonged in one module.</p>
<h2>The Signals That Didn't Look Like Signals</h2>
<p>In hindsight, the signals were all there.
At the time, not one of them looked like a signal.</p>
<ul>
<li><strong>Collaboration constantly read Catalog's data.</strong>
I took it for ordinary cross-module traffic, when it was really the boundary telling me the two belonged together.</li>
<li><strong>Nearly every new Collaboration feature reached into Catalog.</strong>
That feels like healthy growth, right up until you notice it's merge pressure.</li>
<li><strong>We kept adding event handlers just to keep the two in sync.</strong>
It passed for good event-driven design, and it was actually the coupling I wanted to avoid, wearing a disguise.</li>
<li><strong>Then came the first &quot;this has to be correct immediately&quot; hotfix.</strong>
Easy to wave off as a one-off, except it was the eventual-consistency assumption starting to crack.</li>
</ul>
<p>Any one of these is invisible.
Together, over a year, they're the whole story.</p>
<h2>What I'd Tell My Past Self</h2>
<p><strong>A module boundary is a guess, so treat it like one.</strong>
You're guessing - so keep testing the guess instead of filing it away as settled.</p>
<p><strong>Start with fewer, coarser modules.</strong>
You can always split a module once the seam is obvious.
When you're unsure, <a href="how-to-keep-your-data-boundaries-intact-in-a-modular-monolith"><strong>keep things together</strong></a> and let a module earn its independence.
It's the same instinct as waiting for the <a href="dry-is-the-most-misunderstood-rule-in-programming"><strong>third repetition before you extract an abstraction</strong></a>.</p>
<p><img src="/blogs/mnw_199/split_is_cheap_merge_is_expensive.png" alt="Splitting one coarse module into two is cheap and low risk; merging two modules back into one is expensive and risky"></p>
<p><strong>Let consistency draw your boundaries.</strong>
Eventual consistency across a boundary is a bet that they never will.
Make that bet on purpose, not by default.</p>
<p><strong>Watch the cheap decisions, not the expensive ones.</strong>
We pour deliberation into decisions that are easy to reverse and wave through the ones that quietly aren't.
A module boundary feels cheap, because on the day you draw it, it is.
The cost shows up later, compounding, which is exactly why nobody's watching when it does.</p>
<h2>The Part That's Uncomfortable</h2>
<p>None of this means modular monoliths are a trap, or that async messaging is a mistake.
I'd build it as a modular monolith again tomorrow.
Every call was sound for the context we had, and that context was incomplete in a way nobody could see yet.</p>
<p>The lesson is smaller and harder to sit with: the boundaries you draw earliest are the ones most likely to be invalidated by what you learn later, and the least likely to be revisited once they are.
The door wasn't one-way when I walked through it.
It became one-way behind me, one reasonable feature at a time.</p>
<h2>Summary</h2>
<ul>
<li>A module boundary is a <strong>guess</strong> made when you know the least, so treat it as provisional, not settled.</li>
<li>Prefer <strong>fewer, coarser modules</strong>. Splitting one later is cheap - merging two back together is the expensive, risky direction.</li>
<li>Let <strong>consistency</strong> draw your boundaries. If two things have to be correct at the same instant, they belong on the same side of the line.</li>
</ul>
<p>If you want a structured way to design module boundaries, and the judgment for when to keep modules together or split them apart, that's exactly what I teach in <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a>.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_199.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Building Dapr Workflows in .NET With Aspire]]></title>
            <link>https://milanjovanovic.tech/blog/building-dapr-workflows-in-dotnet-with-aspire</link>
            <guid isPermaLink="false">building-dapr-workflows-in-dotnet-with-aspire</guid>
            <pubDate>Sat, 13 Jun 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[Long-running business processes are hard to get right. Dapr Workflow lets you write them as plain C# code that survives crashes and restarts, and Aspire makes the whole thing run with a single command. Here's how to wire the two together.]]></description>
            <content:encoded><![CDATA[<p>Most real business processes don't finish in a single request.</p>
<p>An order gets placed, inventory gets checked, a payment gets charged, stock gets reserved, and the customer gets notified.
Each step can fail, time out, or need a retry.
And the whole thing has to survive a process restart without losing its place or charging someone twice.</p>
<p>We usually solve this with a pile of queues, a state table, and a lot of defensive code to track where each process is.
It works, but the business logic ends up scattered across handlers and database rows, and nobody can read the flow top to bottom anymore.</p>
<p><a href="introduction-to-dapr-for-dotnet-developers"><strong>Dapr</strong></a> (a graduated <a href="https://www.cncf.io/">CNCF</a> project) has a building block for exactly this: <strong>Workflow</strong>.
You write the process as ordinary C# code, and Dapr makes it durable.
If the host crashes halfway through, the workflow picks up right where it left off.</p>
<p>In this article, we'll build a small Dapr Workflow, run it with <a href="dotnet-aspire-a-game-changer-for-cloud-native-development"><strong>.NET Aspire</strong></a>, and inspect its state with the <a href="https://docs.diagrid.io/develop/local-development/dev-dashboard/?utm_source=milanjovanovic&amp;utm_medium=referral&amp;utm_campaign=workflows"><strong>Diagrid Dev Dashboard</strong></a>.
If you'd rather learn this hands-on, there's also a free <a href="https://www.diagrid.io/dapr-university/dapr-workflows-dotnet-aspire?utm_source=milanjovanovic&amp;utm_medium=referral&amp;utm_campaign=workflows"><strong>Dapr University track</strong></a> built around this exact stack.</p>
<p>Let's dive in.</p>
<h2>What Dapr Workflow Actually Is</h2>
<p>You define a <strong>workflow</strong> that orchestrates a process, and <strong>activities</strong> that do the actual work (call a database, hit an API, send an email).
This is <a href="orchestration-vs-choreography"><strong>orchestration</strong></a> rather than choreography: one place drives the process instead of services reacting to each other's events.</p>
<p>The definitions live in your app; the <strong>workflow engine</strong> that executes them runs in the Dapr sidecar:</p>
<figure>
  ![Diagram of a workflow app containing workflow and activity definitions, communicating over the Dapr API (HTTP/gRPC) with the Dapr Workflow engine running in the sidecar.](/blogs/mnw_198/workflow-overview.png)
  <figcaption>
    Source: [Dapr - Workflows
    overview](https://docs.dapr.io/developing-applications/building-blocks/workflow/workflow-overview/)
  </figcaption>
</figure>
<p>The key idea is <strong>durable execution</strong>.
Dapr records every step to a state store, so the workflow can be replayed from history at any time.
A crash, a deployment, or a scale-out event doesn't lose progress, and a workflow can run for seconds or for months.</p>
<p>One rule follows from this: <strong>workflow code must be deterministic</strong>.
No <code>DateTime.Now</code>, no random values, no I/O - anything non-deterministic goes into an <em>activity</em>.
Even logging is affected: use <code>context.CreateReplaySafeLogger&lt;T&gt;()</code> inside a workflow, or every replay will repeat your log lines.</p>
<p>Under the hood, this all runs on Dapr actors, which is why the state store must support actors. More on that in a moment.</p>
<h2>Building the Workflow</h2>
<p>The quickest starting point is the <a href="https://aspire.dev/"><strong>Aspire CLI's</strong></a> starter template:</p>
<pre><code class="language-bash">aspire new aspire-starter -n OrderProcessing
</code></pre>
<p>It generates the app host, an API service, and a <code>ServiceDefaults</code> project that wires up OpenTelemetry and health checks (that's where the <code>AddServiceDefaults</code> call comes from later).
If you're on Claude Code, the <a href="https://github.com/diagrid-labs/dapr-skills"><strong>Dapr Skills</strong></a> plugin can scaffold the entire workflow project from a prompt and review it for determinism mistakes.
Everything we build here is also in a <a href="https://github.com/m-jovanovic/dapr-workflows-with-aspire"><strong>working sample on my GitHub</strong></a>, so you can clone it and follow along.</p>
<p>The API service needs the <code>Dapr.Workflow</code> package:</p>
<pre><code class="language-xml">&lt;PackageReference Include=&quot;Dapr.Workflow&quot; Version=&quot;1.18.1&quot; /&gt;
</code></pre>
<p>A workflow derives from <code>Workflow&lt;TInput, TOutput&gt;</code> and reads top to bottom like a normal method, even though every step is durably persisted:</p>
<pre><code class="language-csharp">using Dapr.Workflow;

namespace OrderApi.Workflows;

internal sealed class OrderProcessingWorkflow : Workflow&lt;OrderPayload, OrderResult&gt;
{
    public override async Task&lt;OrderResult&gt; RunAsync(
        WorkflowContext context,
        OrderPayload order)
    {
        // 1. Check inventory
        var inventory = await context.CallActivityAsync&lt;InventoryResult&gt;(
            nameof(CheckInventoryActivity),
            order);

        if (!inventory.InStock)
        {
            return new OrderResult(order.OrderId, &quot;Rejected: out of stock&quot;);
        }

        // 2. Charge the customer
        await context.CallActivityAsync(
            nameof(ProcessPaymentActivity),
            new PaymentRequest(order.OrderId, order.TotalAmount));

        // 3. Reserve the stock
        await context.CallActivityAsync(
            nameof(UpdateInventoryActivity),
            order);

        // 4. Notify the customer
        await context.CallActivityAsync(
            nameof(NotifyCustomerActivity),
            order.CustomerId);

        return new OrderResult(order.OrderId, &quot;Completed&quot;);
    }
}
</code></pre>
<p><code>CallActivityAsync</code> doesn't invoke the activity directly.
It schedules the work with the workflow engine, which records the result once the activity completes.
If the process dies after the payment step, Dapr replays the workflow, feeds it the recorded results for completed steps, and resumes at the inventory update.
The customer never gets charged twice.</p>
<p>This is the <strong>task chaining</strong> pattern.
Dapr Workflow also supports fan-out/fan-in, external events, timers, and child workflows - all in plain C# (fan-out is just <code>Select</code> plus <code>Task.WhenAll</code>).
If you want to build the richer patterns hands-on, the free <a href="https://www.diagrid.io/dapr-university/dapr-workflows-dotnet-aspire?utm_source=milanjovanovic&amp;utm_medium=referral&amp;utm_campaign=workflows"><strong>Build Dapr Workflows in .NET with Aspire</strong></a> track has you fan out to parallel activities and aggregate the results.</p>
<p>One production caveat: the recorded history is tied to the shape of your code, so changing a workflow while instances are in flight breaks their replay.
That's solved with <a href="https://www.diagrid.io/blog/how-to-version-net-workflows?utm_source=milanjovanovic&amp;utm_medium=referral&amp;utm_campaign=workflows"><strong>workflow versioning</strong></a>; we're staying on version one here.</p>
<p>An activity is where the real work happens, and the only place you're allowed to be non-deterministic.
It derives from <code>WorkflowActivity&lt;TInput, TOutput&gt;</code> and supports constructor injection:</p>
<pre><code class="language-csharp">using Dapr.Workflow;

namespace OrderApi.Activities;

internal sealed class CheckInventoryActivity(IInventoryService inventory)
    : WorkflowActivity&lt;OrderPayload, InventoryResult&gt;
{
    public override async Task&lt;InventoryResult&gt; RunAsync(
        WorkflowActivityContext context,
        OrderPayload order)
    {
        bool inStock = await inventory.HasStockAsync(order.ProductId, order.Quantity);

        return new InventoryResult(inStock);
    }
}
</code></pre>
<p>The other activities follow the same shape: charge the card, decrement stock, send the confirmation email.
Each one is isolated, so Dapr can retry a failed activity without re-running the whole workflow.
And since every input and output gets serialized to the state store, simple JSON-friendly records are the right tool for these types.</p>
<h2>Starting Workflows Over HTTP</h2>
<p>Register the workflow and its activities in the API service's <code>Program.cs</code>:</p>
<pre><code class="language-csharp">builder.Services.AddDaprWorkflow(options =&gt;
{
    options.RegisterWorkflow&lt;OrderProcessingWorkflow&gt;();

    options.RegisterActivity&lt;CheckInventoryActivity&gt;();
    options.RegisterActivity&lt;ProcessPaymentActivity&gt;();
    options.RegisterActivity&lt;UpdateInventoryActivity&gt;();
    options.RegisterActivity&lt;NotifyCustomerActivity&gt;();
});
</code></pre>
<p>This also registers a <code>DaprWorkflowClient</code> we can use to start and query workflow instances:</p>
<pre><code class="language-csharp">app.MapPost(&quot;/orders&quot;, async (
    OrderPayload order,
    DaprWorkflowClient workflowClient) =&gt;
{
    string instanceId = await workflowClient.ScheduleNewWorkflowAsync(
        name: nameof(OrderProcessingWorkflow),
        instanceId: order.OrderId,
        input: order);

    return Results.Accepted($&quot;/orders/{instanceId}&quot;, new { instanceId });
});

app.MapGet(&quot;/orders/{instanceId}&quot;, async (
    string instanceId,
    DaprWorkflowClient workflowClient) =&gt;
{
    WorkflowState? state = await workflowClient.GetWorkflowStateAsync(instanceId);

    if (state is null || !state.Exists)
    {
        return Results.NotFound();
    }

    return Results.Ok(new
    {
        RuntimeStatus = state.RuntimeStatus.ToString(),
        Output = state.ReadOutputAs&lt;OrderResult&gt;()
    });
});
</code></pre>
<p><code>ScheduleNewWorkflowAsync</code> returns immediately and the workflow runs in the background.
It's the same idea as <a href="how-to-scale-long-running-api-requests"><strong>scaling long-running API requests</strong></a>: return <code>202 Accepted</code> and let the client poll for status.
Two SDK quirks worth knowing: <code>GetWorkflowStateAsync</code> returns <code>null</code> for an instance it has never seen, and <code>RuntimeStatus</code> is an enum that serializes as a bare number without the <code>ToString()</code>.</p>
<h2>Running Everything With Aspire</h2>
<p>Here's where Aspire earns its keep.
A Dapr Workflow needs a sidecar and a state store running alongside the app, and Aspire orchestrates all of it from one place.</p>
<figure>
  ![Architecture diagram: Aspire locally manages the workflow app and the Dapr sidecar with its workflow engine, which reads and writes workflow state to a state store. The Diagrid Dev Dashboard connects to the same state store to visualize workflow executions.](/blogs/mnw_198/workflow-app-aspire.png)
  <figcaption>
    Source: [Dapr
    University](https://www.diagrid.io/dapr-university/dapr-workflows-dotnet-aspire?utm_source=milanjovanovic&utm_medium=referral&utm_campaign=workflows)
  </figcaption>
</figure>
<p>The app host needs two packages.
The Dapr integration lives in the <a href="https://learn.microsoft.com/en-us/dotnet/aspire/community-toolkit/overview"><strong>Aspire Community Toolkit</strong></a> these days; the original <code>Aspire.Hosting.Dapr</code> package is deprecated.</p>
<pre><code class="language-xml">&lt;PackageReference Include=&quot;CommunityToolkit.Aspire.Hosting.Dapr&quot; Version=&quot;13.0.0&quot; /&gt;
&lt;PackageReference Include=&quot;Aspire.Hosting.Valkey&quot; Version=&quot;13.4.3&quot; /&gt;
</code></pre>
<p>Then the app host:</p>
<pre><code class="language-csharp">using CommunityToolkit.Aspire.Hosting.Dapr;

var builder = DistributedApplication.CreateBuilder(args);

builder.AddDapr();

// Pin the password. Aspire generates a random one on every run otherwise,
// and the Dapr component file below has to know it.
var statePassword = builder.AddParameter(
    &quot;statestore-password&quot;, &quot;state-store-123&quot;, secret: true);

// Valkey (a Redis fork) as the workflow state store
var statestore = builder
    .AddValkey(&quot;statestore&quot;, 16379, statePassword)
    .WithDataVolume();

builder.AddProject&lt;Projects.OrderApi&gt;(&quot;order-api&quot;)
    .WithDaprSidecar(new DaprSidecarOptions
    {
        ResourcesPaths = [&quot;./Resources&quot;]
    })
    .WaitFor(statestore);

builder.Build().Run();
</code></pre>
<p><code>WithDaprSidecar</code> runs a Dapr sidecar next to <code>order-api</code>, and <code>ResourcesPaths</code> points it at the Dapr component files (relative paths resolve against the app host directory).</p>
<p>The one component the workflow needs is a state store - a <code>statestore.yaml</code> in the app host's <code>Resources</code> folder:</p>
<pre><code class="language-yaml">apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: workflowstore
spec:
  type: state.redis
  version: v1
  metadata:
    - name: redisHost
      value: 'localhost:16379'
    - name: redisPassword
      value: 'state-store-123'
    - name: actorStateStore
      value: 'true'
</code></pre>
<p>That <code>actorStateStore: &quot;true&quot;</code> line is the one people forget.
Dapr Workflow runs on top of actors, so without it, workflows won't run.
Notice the application never sees any of this: swapping Valkey for Postgres means editing this YAML file, not your C# code.</p>
<p>Install the <a href="https://docs.dapr.io/getting-started/install-dapr-cli/"><strong>Dapr CLI</strong></a> and run <code>dapr init</code> once (that's where the sidecar binary comes from), then start everything with a single command:</p>
<pre><code class="language-bash">aspire run
</code></pre>
<p>Aspire spins up Valkey, the Dapr sidecar, and the API, with logs, traces, and health in one dashboard.
Grab the API's port from there and post an order:</p>
<pre><code class="language-bash">curl -X POST http://localhost:5555/orders \
  -H &quot;Content-Type: application/json&quot; \
  -d '{&quot;orderId&quot;:&quot;order-001&quot;,&quot;customerId&quot;:&quot;cust-42&quot;,&quot;productId&quot;:&quot;pro-plan&quot;,&quot;quantity&quot;:2,&quot;totalAmount&quot;:49.99}'
</code></pre>
<p>Poll the status endpoint and you'll see the workflow march through its activities (if the first request returns a <code>500</code>, give the sidecar a few more seconds to connect to the placement service):</p>
<pre><code class="language-bash">curl http://localhost:5555/orders/order-001
</code></pre>
<pre><code class="language-json">{
  &quot;runtimeStatus&quot;: &quot;Completed&quot;,
  &quot;output&quot;: {
    &quot;orderId&quot;: &quot;order-001&quot;,
    &quot;status&quot;: &quot;Completed&quot;
  }
}
</code></pre>
<p>Each activity shows up as a span in the distributed trace:</p>
<p><img src="/blogs/mnw_198/aspire_distributed_trace_order_creation.png" alt="Aspire dashboard distributed trace for POST /orders, showing the workflow orchestration span and individual spans for the CheckInventory, ProcessPayment, UpdateInventory, and NotifyCustomer activities."></p>
<h2>Inspecting Workflow State Locally</h2>
<p>The Aspire dashboard shows you the request flow, but not the <em>workflow's</em> internal state: which step it's on, what each activity returned, and the full execution history.
For that, there's the <a href="https://docs.diagrid.io/develop/local-development/dev-dashboard/?utm_source=milanjovanovic&amp;utm_medium=referral&amp;utm_campaign=workflows"><strong>Diagrid Dev Dashboard</strong></a>: a free, local-only tool that reads your workflow state store (Redis-compatible, Postgres, or SQLite) and visualizes every instance.
It comes from Diagrid, the company founded by the creators of the Dapr OSS project, which also provides enterprise Dapr support.</p>
<p>Since the whole point of this setup is that one command starts everything, let's add it to the app host:</p>
<pre><code class="language-csharp">builder.AddContainer(&quot;diagrid-dashboard&quot;, &quot;ghcr.io/diagridio/diagrid-dashboard:latest&quot;)
    .WithBindMount(&quot;./Resources&quot;, &quot;/app/components&quot;)
    .WithEnvironment(&quot;COMPONENT_FILE&quot;, &quot;/app/components/dashboard-store.yaml&quot;)
    .WithEnvironment(&quot;APP_ID&quot;, &quot;diagrid-dashboard&quot;)
    .WithHttpEndpoint(targetPort: 8080)
    .WaitFor(statestore);
</code></pre>
<p>Why a second component file? Networking.
The sidecar runs as a host process, so <code>localhost:16379</code> works for it.
The dashboard runs in a container, where <code>localhost</code> means the container itself, so its <code>dashboard-store.yaml</code> reaches the host through <code>host.docker.internal</code>
(on Linux without Docker Desktop, use the bridge gateway IP instead):</p>
<pre><code class="language-yaml">apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: dashboardstore
spec:
  type: state.redis
  version: v1
  metadata:
    - name: redisHost
      value: 'host.docker.internal:16379'
    - name: redisPassword
      value: 'state-store-123'
    - name: actorStateStore
      value: 'true'
scopes:
  - diagrid-dashboard
</code></pre>
<p>The <code>scopes</code> field keeps the API's sidecar from picking up this component, since both files sit in the same <code>Resources</code> folder.</p>
<p>Run <code>aspire run</code> again and open the dashboard's endpoint from the Aspire resources view.
Every workflow instance is listed with its status, app ID, and duration:</p>
<p><img src="/blogs/mnw_198/diagrid_dashboard_workflows.png" alt="Diagrid Dev Dashboard listing OrderProcessingWorkflow executions with their status, instance ID, app ID, and start and end times."></p>
<p>Clicking an instance shows the exact input the workflow received and the output it produced:</p>
<p><img src="/blogs/mnw_198/diagrid_dashboard_workflow_instance.png" alt="Workflow Execution Details page showing a running OrderProcessingWorkflow instance with its status and input payload."></p>
<p>Below that sits the full execution history - the ground truth of what your workflow actually did.
Expand a <code>TaskScheduled</code> event to see an activity's input, or a <code>TaskCompleted</code> event to see its input and output:</p>
<p><img src="/blogs/mnw_198/diagrid_dashboard_workflow_history.png" alt="Execution History table showing TaskScheduled and TaskCompleted events for each activity in the order processing workflow."></p>
<p>Being able to see the workflow's actual state, not just guess at it from logs, is what makes local workflow development feel sane.
There's also a <a href="https://www.nuget.org/packages/Diagrid.Aspire.Hosting.Dashboard"><code>Diagrid.Aspire.Hosting.Dashboard</code></a> package that wraps this container setup into a single <code>AddDiagridDashboard</code> call.</p>
<h2>Summary</h2>
<p>Dapr Workflow gives you durable execution for long-running processes without dragging a heavy orchestration engine into your code:</p>
<ul>
<li>The process is <strong>plain C# code</strong> that reads top to bottom.</li>
<li>Dapr makes it <strong>fault-tolerant</strong>, replaying from the state store so a crash never loses progress.</li>
<li>The orchestration stays <strong>deterministic</strong>; the side effects live in activities.</li>
<li><strong>Aspire</strong> runs the sidecar, the state store, and the dashboard with one command.</li>
<li>The <strong>Diagrid Dev Dashboard</strong> shows you exactly what each instance is doing.</li>
</ul>
<p>You can grab the complete <a href="https://github.com/m-jovanovic/dapr-workflows-with-aspire"><strong>source code for this article</strong></a> on my GitHub, including Aspire integration tests for the workflow.</p>
<p>If you want to go deeper, the free <a href="https://www.diagrid.io/dapr-university/dapr-workflows-dotnet-aspire?utm_source=milanjovanovic&amp;utm_medium=referral&amp;utm_campaign=workflows"><strong>Build Dapr Workflows in .NET with Aspire</strong></a> track on Dapr University is the natural next step.
You'll build a fan-out/fan-in workflow on this exact stack in a hosted sandbox, with nothing to install.
The <a href="https://www.diagrid.io/dapr-university/dapr-workflow?utm_source=milanjovanovic&amp;utm_medium=referral&amp;utm_campaign=workflows"><strong>Dapr Workflow track</strong></a> covers the remaining patterns in standalone examples.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_198.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[DRY Is the Most Misunderstood Rule in Programming]]></title>
            <link>https://milanjovanovic.tech/blog/dry-is-the-most-misunderstood-rule-in-programming</link>
            <guid isPermaLink="false">dry-is-the-most-misunderstood-rule-in-programming</guid>
            <pubDate>Sat, 06 Jun 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[DRY was never about code that looks the same. It's about knowledge. Most of the tangled abstractions I've had to unwind started as a well-meaning attempt to remove duplication that was never really duplication. Here's how I think about it now, and the rule I use instead.]]></description>
            <content:encoded><![CDATA[<p>Every developer learns DRY early, and almost everyone learns it wrong.</p>
<p>Don't Repeat Yourself.
See two pieces of code that look the same, extract a method, delete the duplicate.
I did this for years and wrote some of the worst code I've ever had to maintain:</p>
<ul>
<li>A shared helper that grew a new boolean parameter every sprint.</li>
<li>A base class nobody dared touch because six unrelated features inherited from it.</li>
<li>A &quot;common&quot; module two independent parts of the system both depended on, so neither could change without the other.</li>
</ul>
<p>Every one started as an innocent attempt to not repeat myself.</p>
<h2>What DRY Actually Says</h2>
<p>Here's the part most people skip.
The original definition, from Andy Hunt and Dave Thomas in <a href="https://en.wikipedia.org/wiki/The_Pragmatic_Programmer"><em>The Pragmatic Programmer</em></a>,
says nothing about code:</p>
<blockquote>
<p>Every piece of <strong>knowledge</strong> must have a single, unambiguous, authoritative representation within a system.</p>
</blockquote>
<p>It's about knowledge. A single <em>fact</em> about your domain, like a tax rule or the format of an invoice number,
should live in exactly one place.
When that fact changes, you change it once instead of hunting for seven copies.</p>
<h2>The Mistake: Deduplicating Code, Not Knowledge</h2>
<p>Two pieces of code can look identical and represent completely different knowledge.</p>
<p>Say you validate two addresses.
One is a customer's shipping address, the other is a warehouse address.
Today the rules are identical:</p>
<pre><code class="language-csharp">public bool IsValid(Address address) =&gt;
    !string.IsNullOrWhiteSpace(address.Street) &amp;&amp;
    !string.IsNullOrWhiteSpace(address.City) &amp;&amp;
    !string.IsNullOrWhiteSpace(address.PostalCode);
</code></pre>
<p>The DRY reflex says extract one validator and call it from both places.
But these are different concepts that happen to share rules this week.
The day the warehouse needs a loading-dock code,
you're back in the shared method bolting on a flag to keep the other caller working:</p>
<pre><code class="language-csharp">public bool IsValid(Address address, bool requireDockCode = false) =&gt;
    !string.IsNullOrWhiteSpace(address.Street) &amp;&amp;
    !string.IsNullOrWhiteSpace(address.City) &amp;&amp;
    !string.IsNullOrWhiteSpace(address.PostalCode) &amp;&amp;
    (!requireDockCode || !string.IsNullOrWhiteSpace(address.DockCode));
</code></pre>
<p>That boolean is the tell.
The first time a shared method grows a flag so one caller behaves differently, you didn't have duplication.
You had two things that looked alike and glued them together.
Give it a year and the signature has three more flags,
each one a place where the two concepts were never actually the same.</p>
<h2>The Wrong Abstraction Costs More Than Duplication</h2>
<p><strong>Duplication is far cheaper than the wrong abstraction</strong>.</p>
<p>Copy-paste is visible and local.
You can see both copies, and if they drift apart, that was always allowed.
The wrong abstraction is invisible and global.
Every caller depends on one shape and bends it to fit, the flags pile up,
and you end up afraid to touch a method you no longer understand.
I've spent more time deleting bad abstractions than I ever saved writing them.</p>
<p>This is the <a href="the-real-cost-of-abstractions-in-dotnet"><strong>hidden coupling cost I wrote about in the abstractions piece</strong></a>,
and DRY-by-reflex is one of the most common ways it sneaks in.</p>
<h2>Where It Hurts Most: Across Boundaries</h2>
<p>Inside one class, a bad helper is annoying. Across module boundaries, it's structural damage.</p>
<p>Picture a <a href="what-is-a-modular-monolith"><strong>modular monolith</strong></a> with a <code>Billing</code> module and a <code>Shipping</code> module.
Both have an <code>Order</code>.
A well-meaning engineer notices the two classes share fields and pulls them into a shared type both modules reference:</p>
<pre><code class="language-csharp">// Shared.Orders, referenced by both Billing and Shipping
public class Order
{
    public Guid Id { get; set; }
    public string CustomerName { get; set; }
    public decimal Total { get; set; }
    // ...whatever either module happens to need
}
</code></pre>
<p>Now Billing and Shipping can't evolve independently.
A change to billing's order forces a recompile, re-test, and redeploy of shipping.
You took two bounded contexts that were supposed to be decoupled and welded them together to save a few properties.</p>
<p>Two modules each owning their own <code>Order</code> is the whole point of <a href="how-to-keep-your-data-boundaries-intact-in-a-modular-monolith"><strong>keeping data inside its boundaries</strong></a>. The shapes are allowed to be similar, modeling the same real-world thing from two points of view that drift over time. It's the same reason <a href="vertical-slice-architecture-is-easier-than-you-think"><strong>vertical slices</strong></a>
tolerate a little repetition, so each slice can change on its own.</p>
<h2>The Rule I Use: Wait for the Third Time</h2>
<p>I don't deduplicate the second time I see something.
I wait for the third, and I ask one question: if this rule changes, do both copies have to change together?</p>
<ul>
<li><strong>Yes</strong> - it's real duplication, the same fact written in several places. Extract it, and that's DRY doing its job.</li>
<li><strong>No</strong> - the resemblance is a coincidence. Leave it alone, and coupling them will cost you later.</li>
</ul>
<p>Let the code repeat until the right abstraction becomes obvious,
because good abstractions are discovered from concrete cases, not guessed up front.
Some people call this AHA, for &quot;Avoid Hasty Abstractions.&quot;</p>
<p>A practical tell: extract when you can name the concept.
A real domain name like <code>Money</code>, <code>TaxRate</code>, or <code>InvoiceNumber</code> is probably knowledge worth a <a href="value-objects-in-dotnet-ddd-fundamentals"><strong>value object</strong></a>.
If the best name you can find is <code>Helper</code>, <code>Utils</code>, or <code>ProcessData</code>, you're abstracting shape, not knowledge.</p>
<h2>When DRY Is Right</h2>
<p>Applied correctly, DRY is invaluable.
A business rule belongs in exactly one place.
Watch what happens when &quot;an order over $1,000 needs manager approval&quot; gets copy-pasted across three services:</p>
<pre><code class="language-csharp">// OrderService
if (order.Total &gt; 1000) { /* require approval */ }

// CheckoutService
if (order.Total &gt; 1000m) { /* require approval */ }

// AdminController - someone bumped the limit here, and only here
if (order.Total &gt; 5000) { /* require approval */ }
</code></pre>
<p>You will eventually update two of them and ship a bug.
That drifted third copy is exactly how it happens.
Push the rule into the <a href="refactoring-from-an-anemic-domain-model-to-a-rich-domain-model"><strong>domain model</strong></a> where it has one home:</p>
<pre><code class="language-csharp">public bool RequiresManagerApproval() =&gt; Total &gt; 1000;
</code></pre>
<p>That's the single authoritative representation DRY is actually about.</p>
<h2>Summary</h2>
<ul>
<li>DRY is about <strong>knowledge</strong>, not code that looks alike.</li>
<li>The wrong abstraction costs more than the duplication it replaced, and it's harder to undo.</li>
<li>Wait for the third occurrence. Extract only when both copies encode the same fact and must change together.</li>
</ul>
<p>The next time you're about to delete a duplicate, don't ask whether the code looks the same.
Ask whether it <em>means</em> the same.
That one question will save you more maintenance pain than DRY ever saved you typing.</p>
<p>If you want to see how I draw these boundaries in a real system,
with independent modules and abstractions that earn their place, that's the heart of <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture</strong></a>.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_197.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Union Types Are Finally Coming to C#]]></title>
            <link>https://milanjovanovic.tech/blog/union-types-are-finally-coming-to-csharp</link>
            <guid isPermaLink="false">union-types-are-finally-coming-to-csharp</guid>
            <pubDate>Sat, 30 May 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[For years we faked union types with marker interfaces, base classes, and the OneOf library. C# 15 finally bakes them into the language - and here's a quick tour of what they look like and why I think they're a big deal.]]></description>
            <content:encoded><![CDATA[<p>Every backend developer eventually hits the same wall: a method that can return <em>one of several things</em>.</p>
<p>A parse that either gives you a number or an error.
A lookup that returns a value or &quot;not found&quot;.
An operation that succeeds or fails.
In C#, we've never had a clean way to model &quot;this is an <code>A</code> <strong>or</strong> a <code>B</code>&quot;.
So we faked it - with marker interfaces, abstract base classes, tuples, nullable returns, exceptions,
or the excellent <a href="https://github.com/mcintyre321/OneOf"><strong>OneOf</strong></a> library.</p>
<p>C# 15 (shipping with .NET 11) finally adds <strong>union types</strong> to the language.
I've wanted this for years, so let me give you a quick tour.</p>
<p>Let's dive in.</p>
<h2>The Problem</h2>
<p>Say a method can return a user or fail because they don't exist. Today you'd reach for something like this:</p>
<pre><code class="language-csharp">// Throw for the &quot;failure&quot; case - control flow via exceptions
public User GetUser(int id) =&gt;
    _users.TryGetValue(id, out var user)
        ? user
        : throw new UserNotFoundException(id);
</code></pre>
<p>The signature says it returns a <code>User</code>, but that's a lie - it might throw instead. The caller has no way to know that without reading the body. The other usual workarounds (a bool <code>TryGet</code> with an <code>out</code> parameter, a custom <code>Result</code> class with nullable fields, or a <code>OneOf&lt;User, NotFound&gt;</code>) all add ceremony to express one simple idea.</p>
<p>What you actually want is a <strong>closed set</strong> of types. That's exactly what a union is.</p>
<h2>Declaring a Union</h2>
<p>The syntax is delightfully small. You list a name and the case types:</p>
<pre><code class="language-csharp">public union Result&lt;T&gt;(T, Exception);
</code></pre>
<p>That's it. A <code>Result&lt;T&gt;</code> is now <em>either</em> a <code>T</code> <em>or</em> an <code>Exception</code> - and nothing else. The types don't even need to be related, which is the whole point.</p>
<p>Here's a more concrete example with unrelated record types:</p>
<pre><code class="language-csharp">public record CreditCard(string Last4, string Brand);
public record PayPal(string Email);
public record BankTransfer(string Iban);

public union PaymentMethod(CreditCard, PayPal, BankTransfer);
</code></pre>
<h2>Creating Values</h2>
<p>There's an implicit conversion from each case type, so you just assign the value directly:</p>
<pre><code class="language-csharp">PaymentMethod method = new CreditCard(&quot;4242&quot;, &quot;Visa&quot;);
</code></pre>
<p>Try to assign a type that isn't in the set, and it's a <strong>compile error</strong>. The set is closed.</p>
<h2>Consuming a Union</h2>
<p>This is where it shines. Pattern matching just works, and the compiler checks the inner value for you:</p>
<pre><code class="language-csharp">string Describe(PaymentMethod method) =&gt; method switch
{
    CreditCard card  =&gt; $&quot;{card.Brand} ending {card.Last4}&quot;,
    PayPal paypal    =&gt; $&quot;PayPal ({paypal.Email})&quot;,
    BankTransfer ach =&gt; $&quot;Bank transfer to {ach.Iban}&quot;,
}; // No `_` or `default` needed
</code></pre>
<p>Notice there's <strong>no discard <code>_</code> and no <code>default</code> arm</strong>. Because the union is closed, the compiler knows all three cases are covered. Forget one, and you get a warning at compile time:</p>
<pre><code>warning CS8509: The switch expression does not handle all possible values
of its input type (it is not exhaustive). For example, the pattern 'BankTransfer'
is not covered.
</code></pre>
<p>That exhaustiveness check is the feature I'm most excited about. Add a new case to the union later, and the compiler points you at every <code>switch</code> you forgot to update.</p>
<h2>Back to The Problem</h2>
<p>Remember our lying <code>GetUser</code> method from earlier? Let's fix it with a union.</p>
<p>First, declare what the method can actually return - a <code>User</code> or a <code>NotFound</code>:</p>
<pre><code class="language-csharp">public record NotFound(int Id);

public union UserResult(User, NotFound);
</code></pre>
<p>Now the signature tells the truth, and there are no exceptions for control flow:</p>
<pre><code class="language-csharp">public UserResult GetUser(int id) =&gt;
    _users.TryGetValue(id, out var user)
        ? user
        : new NotFound(id);
</code></pre>
<p>And the caller has to handle both outcomes - the compiler won't let them forget:</p>
<pre><code class="language-csharp">IResult response = GetUser(42) switch
{
    User user      =&gt; Results.Ok(user),
    NotFound found =&gt; Results.NotFound($&quot;No user with id {found.Id}&quot;),
};
</code></pre>
<p>That's the whole pitch. The return type <em>tells you the truth</em>: here are exactly the shapes you'll get back, and you can't ignore one by accident. No more reading the method body to discover what it might throw.</p>
<h2>A Few Caveats</h2>
<p>This is still a <strong>preview/experimental</strong> feature. A few things to keep in mind:</p>
<ul>
<li>It targets <strong>C# 15 / .NET 11</strong>, and the syntax may still change before release. Try it on .NET 11 Preview 4 or later.</li>
<li>Under the hood, a union is compiled to a <code>struct</code> that boxes value-type cases and stores the contents as a single <code>object?</code>. There's a non-boxing path for performance-sensitive code, but the default is simple.</li>
<li>This is a <em>type</em> union (an <code>A</code> or a <code>B</code>), not a full discriminated union with named cases yet. It covers the vast majority of what I reach for OneOf for today.</li>
</ul>
<h2>Summary</h2>
<p>Union types close a gap that's been open in C# for a very long time.</p>
<ul>
<li>Declare a <strong>closed set</strong> of types with <code>public union Name(A, B, C);</code>.</li>
<li>Assign case values <strong>directly</strong> - implicit conversions handle the rest.</li>
<li><strong>Pattern match</strong> with full compiler-checked exhaustiveness, no <code>default</code> arm required.</li>
<li>Model results, options, and &quot;one of these&quot; returns <strong>without</strong> marker interfaces, base classes, or extra libraries.</li>
</ul>
<p>It's a small syntax with a big payoff: your method signatures finally tell the truth about what they return, and the compiler keeps every <code>switch</code> honest.</p>
<p>I'll explore this feature more in the future, but I wanted to share this quick tour now that it's available in preview.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_196.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How to Scale Long-Running API Requests]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-scale-long-running-api-requests</link>
            <guid isPermaLink="false">how-to-scale-long-running-api-requests</guid>
            <pubDate>Sat, 23 May 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[When a single API call takes minutes to finish, it punishes both your users and your server. Here's the progression I walk through to turn long-running endpoints into something responsive, scalable, and operable - and when to reach for managed cloud services instead.]]></description>
            <content:encoded><![CDATA[<p>Every system I've worked on eventually grows an endpoint that takes minutes to finish (or longer).
A report that aggregates years of data. A bulk import. A workflow that fans out to three external services and a database before it can answer.</p>
<p>You end up with two problems at once.
Your users sit on a spinner for several minutes, and your API holds that request open the entire time - burning a thread, a connection, and a slot in your concurrency budget.
A small traffic spike on that one endpoint could potentially take the rest of the API down with it.</p>
<p>I want to walk through the progression I use to fix this.
It's the same path the diagram below traces, from &quot;the request just blocks&quot; to a fully decoupled,
queue-backed worker pool - the shape I usually call an <a href="building-async-apis-in-aspnetcore-the-right-way"><strong>async API</strong></a>.</p>
<p>Depending on your requirements, you might stop at any point along the way - but I want to make sure you understand the full path and the trade-offs at each step.</p>
<p><img src="/blogs/mnw_195/scaling_long_running_requests.png" alt="System design progression for scaling long-running API requests, from synchronous request to queue-based competing consumers."></p>
<h2>Step 0: The Naive Version</h2>
<p>A user sends a request. The application server does the work. The work takes five minutes. The connection stays open the whole time.</p>
<p>There is nothing <em>wrong</em> with this approach - it's just paying for correctness with availability.
The user experience is bad, and the blast radius is large: every long request you accept is a request you <em>can't</em> accept somewhere else.</p>
<p>The first realization you need to internalize is that <strong>the response time and the work duration don't have to be the same thing</strong>.</p>
<div className="centered">
  ![Diagram of a blocking API request, where the user sends a request and waits for the work to finish before getting a response.](/blogs/mnw_195/blocking_api_request.png)
</div>
<h2>Step 1: Accept the Work, Don't Do It</h2>
<p>The first move is to stop doing the work inside the request.</p>
<p>I add a <code>jobs</code> table that represents the work I <em>intend</em> to do. The API endpoint now does three things:</p>
<ol>
<li>Validate the request.</li>
<li>Insert a row into <code>jobs</code> with status <code>Pending</code>.</li>
<li>Return <code>202 Accepted</code> with a job ID.</li>
</ol>
<p>A <a href="running-background-tasks-in-asp-net-core"><strong>background processor</strong></a> running inside the same API picks up <code>Pending</code> rows and works through them.
The client either polls a <code>GET /jobs/{id}</code> endpoint or - better - I push updates via <a href="adding-real-time-functionality-to-dotnet-applications-with-signalr"><strong>SignalR</strong></a>,
<a href="server-sent-events-in-aspnetcore-and-dotnet-10"><strong>Server-Sent Events</strong></a>, or email when the job is done.</p>
<p><img src="/blogs/mnw_195/202_accepted_background_processor.png" alt="Diagram of an API request that accepts work and returns 202, with a background processor that picks up pending jobs and processes them asynchronously."></p>
<p>This already buys you a lot.
The endpoint returns in milliseconds, the user gets a job ID they can track, and a spike of incoming requests just becomes a spike of rows in a table.
That table is cheap to write to.</p>
<p>But there's a ceiling here, and it's easy to hit.</p>
<h2>Step 2: Decouple the Worker From the API</h2>
<p>The background processor in Step 1 still lives inside your API process.
It competes for the same CPU, memory, and connection pool as your real endpoints.
If processing gets heavy or slow, your API starts feeling it - the very thing you were trying to avoid.</p>
<p>The fix is to pull the background processor out into its own deployable, and put a queue between the two.</p>
<p>The API now publishes a message to the queue (and optionally writes the job row for tracking).
A pool of background workers consumes from the queue and does the actual work - the same shape I covered in <a href="event-driven-architecture-in-dotnet-with-rabbitmq"><strong>event-driven architecture with RabbitMQ</strong></a>.
This is the <strong>competing consumers</strong> pattern, and it gives you something the previous step couldn't: <strong>independent scaling</strong>.</p>
<p><img src="/blogs/mnw_195/scaling_long_running_requests_final.png" alt="System design progression for scaling long-running API requests, from synchronous request to queue-based competing consumers."></p>
<p>Three things change once you cross this line:</p>
<ul>
<li><strong>The queue absorbs spikes.</strong> Your API can keep accepting work at a constant rate while the workers drain at their own pace.</li>
<li><strong>You scale workers separately from the API.</strong> More throughput on background jobs doesn't mean more API instances.</li>
<li><strong>Failures become normal.</strong> A worker crash is just a message that goes back on the queue, not a 500 to your user.</li>
</ul>
<p>You also get retryability, pause/resume, structured error handling, and a <strong>dead-letter queue</strong> for poison messages - effectively for free, because the queue infrastructure already provides them.</p>
<h2>What This Costs You</h2>
<p>I'd be lying if I said this was a free upgrade.</p>
<p>You're now running a queue, a worker fleet, and a notification path. That's more moving parts to deploy, monitor, and alert on.
Your &quot;is this done yet?&quot; semantics are no longer obvious from the HTTP response - the client has to ask, or you have to tell them.
And every job needs to be <a href="the-idempotent-consumer-pattern-in-dotnet-and-why-you-need-it"><strong>idempotent</strong></a>, because at-least-once delivery means your workers <em>will</em> see duplicates.</p>
<p>If you only have one slow endpoint and modest traffic, this is overkill.
A simple &quot;fire-and-forget with status polling&quot; inside the same process is fine.
Don't reach for a queue until the pain justifies it.</p>
<h2>When I'd Use a Cloud Service Instead</h2>
<p>You don't always need to assemble this from parts. A few alternatives I would consider:</p>
<ul>
<li><a href="complete-guide-to-amazon-sqs-and-amazon-sns-with-masstransit"><strong>AWS SQS</strong></a> <strong>+ Lambda</strong> or <a href="messaging-made-easy-with-azure-service-bus"><strong>Azure Service Bus</strong></a> <strong>+ Azure Functions</strong> when I want the worker pool to scale to zero and I don't want to manage hosts.</li>
<li><strong>Azure Durable Functions</strong> or <strong>AWS Step Functions</strong> when the work is a multi-step workflow with timers, retries, and human approvals. Orchestration is what they're good at.</li>
<li><a href="https://temporal.io/"><strong>Temporal</strong></a> when the workflow is long-lived (hours, days) and I need first-class durable execution, versioning, and visibility across runs.</li>
</ul>
<p>The trade-off is the usual one: less operational work, more vendor coupling, and a pricing model you need to model carefully when throughput grows.</p>
<h2>Summary</h2>
<p>The progression is simple, and it generalizes:</p>
<ol>
<li><strong>Don't do slow work inside the request.</strong> Accept it, persist it, return <code>202</code>.</li>
<li><strong>Don't run workers inside the API.</strong> Put a queue in between and scale the two sides independently.</li>
<li><strong>Tell the user when it's done.</strong> Polling is fine, push is better.</li>
</ol>
<p>This isn't a microservices argument. It's a separation between <em>accepting work</em> and <em>doing work</em> - two concerns that have very different scaling profiles and very different failure modes.</p>
<p>If you want the full implementation walkthrough, the <a href="https://youtu.be/U40HzU_KkDY"><strong>video version is here</strong></a>.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_195.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[When Your Use Case Half-Succeeds: Designing for Partial Failure in .NET]]></title>
            <link>https://milanjovanovic.tech/blog/when-your-use-case-half-succeeds-designing-for-partial-failure-in-dotnet</link>
            <guid isPermaLink="false">when-your-use-case-half-succeeds-designing-for-partial-failure-in-dotnet</guid>
            <pubDate>Sat, 16 May 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[A use case isn't a transaction. The moment it touches more than one system, you are dealing with partial failure. Here's how I classify side effects and design use cases that fail loudly and recover safely.]]></description>
            <content:encoded><![CDATA[<p>One of the recurring bugs I've chased over the years is the &quot;duplicate charge&quot; support ticket.</p>
<p>The customer was charged once, but our system thought the payment had failed.
The payment provider had taken the money, the order had rolled back to a draft state, and the user had received a &quot;Payment failed, please retry&quot; email on top of it.
A single user action left every subsystem with a different opinion about what actually happened.</p>
<p>A use case looks like a transaction because it sits behind a single method call.
But the moment it touches more than one system, you are dealing with <strong>partial failure</strong>.</p>
<p>Here's how I think about it:</p>
<ul>
<li>The three categories every side effect falls into</li>
<li>How to design use cases that fail loudly and recover safely</li>
<li>When to reach for the <a href="implementing-the-outbox-pattern"><strong>Outbox pattern</strong></a> and when not to</li>
</ul>
<p>Let's dive in.</p>
<h2>The Code That Looks Fine</h2>
<p>Here's a typical &quot;place an order&quot; use case.
I've written this exact shape a hundred times, and so have you.</p>
<pre><code class="language-csharp">internal sealed class PlaceOrder(
    IOrderRepository orders,
    IPaymentService payments,
    IEmailService emails,
    IUnitOfWork unitOfWork)
{
    public async Task&lt;Result&gt; ExecuteAsync(PlaceOrderRequest request, CancellationToken ct)
    {
        var order = Order.Create(request.CustomerId, request.Items);
        orders.Insert(order);

        await payments.ChargeAsync(order.Id, order.Total, ct);

        await emails.SendOrderConfirmationAsync(order.Id, ct);

        await unitOfWork.SaveChangesAsync(ct);
        return Result.Success();
    }
}
</code></pre>
<p>There are three side effects in this method, sitting behind what looks like a single transactional boundary, with no coordination between them.</p>
<p>If <code>SaveChangesAsync</code> throws <em>after</em> <code>ChargeAsync</code> succeeded, you've taken the customer's money and lost the order.
If <code>SendOrderConfirmationAsync</code> throws, the order saves and the charge goes through, but no email is sent.
And if you naively retry, you double-charge.</p>
<p>The use case &quot;works&quot; until it doesn't, and when it doesn't, it tends to fail in a different way every time.</p>
<h2>Three Categories of Side Effect</h2>
<p>Before you write a single line of recovery code, classify every side effect into one of three buckets:</p>
<ol>
<li><strong>Transactional</strong> - lives inside your database transaction. Inserts, updates, <a href="how-to-use-domain-events-to-build-loosely-coupled-systems"><strong>domain events</strong></a> dispatched in-process.</li>
<li><strong>External and reversible</strong> - an API call you can compensate for. Charge → refund. Reserve inventory → release.</li>
<li><strong>External and irreversible</strong> - sent emails, posted webhooks, SMS messages. Once they're out, they're out.</li>
</ol>
<p>The category determines the strategy. There is no single &quot;handle errors properly&quot; rule that covers all three.</p>
<h2>Strategy 1: Pull Transactional Work to the End</h2>
<p>The first move is mechanical.
Anything transactional should commit <em>last</em>, after all external calls have either succeeded or been explicitly tolerated.</p>
<pre><code class="language-csharp">public async Task&lt;Result&gt; ExecuteAsync(PlaceOrderRequest request, CancellationToken ct)
{
    var order = Order.Create(request.CustomerId, request.Items);

    var charge = await payments.ChargeAsync(order.Id, order.Total, ct);
    if (charge.IsFailure) return charge;
    order.MarkPaid(charge.Value.TransactionId);

    orders.Insert(order);

    await unitOfWork.SaveChangesAsync(ct);
    return Result.Success();
}
</code></pre>
<p>You can't always do this - sometimes you need a database ID before calling the external service.
That's fine.
The point isn't ordering for its own sake. It's making sure that if you commit, you've already done the work the commit promises.</p>
<h2>Strategy 2: Move Irreversible Side Effects Outside the Use Case</h2>
<p>This is where the <a href="scaling-the-outbox-pattern"><strong>Outbox pattern</strong></a> earns its keep.</p>
<p>Instead of sending the email directly, raise an <code>OrderPlaced</code> <a href="how-to-use-domain-events-to-build-loosely-coupled-systems"><strong>domain event</strong></a> and let an outbox dispatcher pick it up <em>after</em> the transaction commits.</p>
<pre><code class="language-csharp">public async Task&lt;Result&gt; ExecuteAsync(PlaceOrderRequest request, CancellationToken ct)
{
    var order = Order.Create(request.CustomerId, request.Items);

    var charge = await payments.ChargeAsync(order.Id, order.Total, ct);
    if (charge.IsFailure) return charge;
    order.MarkPaid(charge.Value.TransactionId);

    orders.Insert(order);
    order.Raise(new OrderPlacedEvent(order.Id));
    await unitOfWork.SaveChangesAsync(ct);

    return Result.Success();
}
</code></pre>
<p>The email is no longer the use case's problem.
If the transaction commits, the event commits with it inside the same write.
If it doesn't, the event never escapes the database and no email is ever sent.
A separate worker turns events into emails, with its own retries and its own <a href="the-idempotent-consumer-pattern-in-dotnet-and-why-you-need-it"><strong>idempotency</strong></a> guarantees.</p>
<h2>Strategy 3: Make External Calls Idempotent or Compensable</h2>
<p>The payment call is the dangerous one.
If it succeeds and your transaction rolls back, you've taken money you can't account for.</p>
<p>What you do <em>not</em> do is silently swallow the failure:</p>
<pre><code class="language-csharp">try
{
    await payments.ChargeAsync(order.Id, order.Total, ct);
}
catch
{
    // shrug
}
</code></pre>
<p>The symptom disappears from your logs, the money stays gone, and the next time the user retries you charge them again.</p>
<p>There are two approaches I actually use, and they compose well together.</p>
<h3>Approach A: Idempotency Keys</h3>
<p>Most serious payment providers (Stripe, Adyen, Braintree) let you attach an <strong>idempotency key</strong> to a charge.
A retry with the same key is a no-op on their side and returns the original result.
The natural key here is the order ID:</p>
<pre><code class="language-csharp">var charge = await payments.ChargeAsync(
    new ChargeRequest
    {
        OrderId = order.Id,
        Amount = order.Total,
        IdempotencyKey = order.Id.ToString()
    },
    ct);
</code></pre>
<p>Now it's safe to retry the use case.
If the previous attempt charged the customer and crashed before committing, the next attempt gets the <em>same</em> charge back from the provider instead of creating a new one, and the order finally gets persisted.</p>
<h3>Approach B: Compensate via a Domain Event</h3>
<p>Idempotency keys only help when you can replay with the same inputs.
Sometimes you can't - the user gave up, the request was cancelled, or the failure is permanent.</p>
<p>In that case, the money is real and needs to come back.
Make the failure itself a first-class event and refund out-of-band:</p>
<pre><code class="language-csharp">public async Task&lt;Result&gt; ExecuteAsync(PlaceOrderRequest request, CancellationToken ct)
{
    var order = Order.Create(request.CustomerId, request.Items);

    var charge = await payments.ChargeAsync(order.Id, order.Total, ct);
    if (charge.IsFailure) return charge;
    order.MarkPaid(charge.Value.TransactionId);

    try
    {
        orders.Insert(order);
        order.Raise(new OrderPlacedEvent(order.Id));
        await unitOfWork.SaveChangesAsync(ct);
    }
    catch (Exception ex)
    {
        await outbox.PublishAsync(
            new PaymentFailedEvent(
                order.Id,
                charge.Value.TransactionId,
                order.Total,
                Reason: ex.Message),
            ct);
        throw;
    }

    return Result.Success();
}
</code></pre>
<p>A background consumer subscribes to <code>PaymentFailedEvent</code> and issues the refund, using the transaction ID as its own idempotency key.
This turns a scary cross-process compensation into a normal, observable, retryable <a href="idempotent-consumer-handling-duplicate-messages"><strong>message handler</strong></a>.</p>
<p>In practice, I use Approach A for transient failures and Approach B for permanent ones. They aren't mutually exclusive.</p>
<h2>When the Saga Pattern Wins Instead</h2>
<p>The strategies above work when one use case coordinates a small number of side effects in a single service.
Once the work spans multiple services and needs to survive process restarts, you're in <a href="implementing-the-saga-pattern-with-masstransit"><strong>saga</strong></a> territory.</p>
<p>The rule I use: if you can fit the recovery logic in your head, a well-designed use case is enough. If you can't, reach for a saga.</p>
<h2>Summary</h2>
<p>A use case is a unit of <em>intent</em>, not a unit of <em>atomicity</em>.</p>
<ul>
<li><strong>Transactional</strong> work commits with the database, last.</li>
<li><strong>Irreversible</strong> work goes through the <a href="implementing-the-outbox-pattern"><strong>Outbox pattern</strong></a>, not the use case.</li>
<li><strong>External, reversible</strong> work uses idempotency keys first, compensating events second.</li>
<li><strong>Never</strong> swallow failures to make the use case &quot;look successful&quot;.</li>
</ul>
<p>Most of the production bugs I've debugged in event-driven systems come down to a use case that lied about whether it succeeded.
Stop lying, and the system gets a lot easier to reason about.</p>
<p>If you want to see this kind of thinking applied across a full system, that's exactly what I build inside <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a>.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_194.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[API Versioning Should Be Your Last Resort]]></title>
            <link>https://milanjovanovic.tech/blog/api-versioning-should-be-your-last-resort</link>
            <guid isPermaLink="false">api-versioning-should-be-your-last-resort</guid>
            <pubDate>Sat, 09 May 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[Most teams reach for v2 too early because they don't have a contract evolution strategy. Here's the API change management approach I prefer: evolve contracts safely, deprecate deliberately, and version only when the old and new worlds can't coexist.]]></description>
            <content:encoded><![CDATA[<p>I've written before about implementing <a href="api-versioning-in-aspnetcore"><strong>API versioning</strong></a> in <a href="http://ASP.NET">ASP.NET</a> Core.</p>
<p>But the more important question isn't <em>how</em> to version an API.
It's <em>when</em>.</p>
<p>Every API team eventually reaches for the same escape hatch:</p>
<blockquote>
<p>Just create <code>v2</code>.</p>
</blockquote>
<p>It sounds responsible.
Except now you maintain two APIs, two sets of docs, two behaviors, and a migration project clients will postpone for as long as possible.</p>
<p>I also touched on this briefly in <a href="the-5-most-common-rest-api-design-mistakes-and-how-to-avoid-them"><strong>my REST API design mistakes article</strong></a>, but I want to make the point more directly today:</p>
<p><strong>Versioning is a compatibility tool. It is not a design strategy.</strong></p>
<p>Most API changes do not require a new version.
They require better change management.</p>
<p>And that distinction matters.</p>
<p>If you treat every contract change as a versioning problem, you end up cloning APIs.
If you treat it as a change management problem, you start asking better questions:</p>
<ul>
<li>Can I add instead of replace?</li>
<li>Can old and new behavior coexist for a while?</li>
<li>Can I introduce a new operation instead of mutating an old one?</li>
<li>Can I deprecate this safely with telemetry and a migration path?</li>
</ul>
<p>That mindset leads to APIs that age much better.</p>
<h2>What Actually Breaks Clients?</h2>
<p>Breaking changes usually aren't about the URL alone.</p>
<p>Clients break when you:</p>
<ul>
<li>Remove or rename fields</li>
<li>Change the meaning of existing data</li>
<li>Tighten request validation</li>
<li>Change pagination or error formats</li>
<li>Assume enum-like values are closed forever</li>
</ul>
<p>This breaks a client just as surely as deleting an endpoint:</p>
<pre><code class="language-json">// Before
{ &quot;total&quot;: 100 }

// After
{ &quot;total&quot;: { &quot;amount&quot;: 100, &quot;currency&quot;: &quot;USD&quot; } }
</code></pre>
<p>You didn't change the path.
You didn't rename the endpoint.
You still broke clients.</p>
<p>So instead of asking, &quot;Should this be v2?&quot;, ask, &quot;Can the old and new contract safely coexist?&quot;</p>
<p>I'll use a simple <code>orders</code> API as the running example for the rest of the article.</p>
<h2>The Compatibility Rules</h2>
<p>When I want an API to age well, I keep four rules in mind:</p>
<ul>
<li>Keep existing fields and behavior in place</li>
<li>Don't turn optional request data into required data</li>
<li>Don't change what an existing operation does</li>
<li>Make anything new additive and optional by default</li>
</ul>
<p>These map directly onto the four rules from Z. Nemec's <a href="https://medium.com/good-api/api-change-management-2fe5bba32e9b">API Change Management</a> article: don't take anything away, don't change processing rules, don't make optional things required, and anything you add must be optional.</p>
<p>If you follow those rules, many &quot;versioning problems&quot; turn back into ordinary contract evolution.</p>
<h2>1. Add, Don't Replace</h2>
<p>The safest change is usually an additive one.</p>
<p>Let's say your original <code>GET /orders/{id}</code> response looked like this:</p>
<pre><code class="language-json">{
  &quot;id&quot;: &quot;ord_123&quot;,
  &quot;status&quot;: &quot;paid&quot;,
  &quot;total&quot;: 100
}
</code></pre>
<p>Instead of replacing <code>total</code>, add a new field:</p>
<pre><code class="language-json">{
  &quot;id&quot;: &quot;ord_123&quot;,
  &quot;status&quot;: &quot;paid&quot;,
  &quot;total&quot;: 100,
  &quot;totalMoney&quot;: {
    &quot;amount&quot;: 100,
    &quot;currency&quot;: &quot;USD&quot;
  }
}
</code></pre>
<p>Existing clients keep using <code>total</code>.
New clients can migrate to <code>totalMoney</code>.
You mark the old field as deprecated and remove it only after a real migration window.</p>
<p>The same idea applies beyond fields.
If you need richer semantics, don't mutate a field into a different shape.
Add a new field, a new link, or a new operation that carries the new meaning explicitly.</p>
<p>Sometimes an ugly contract is the price of compatibility.</p>
<h2>2. Make Clients Tolerant Readers</h2>
<p>A well-behaved client should not explode because the server added a field it doesn't understand.</p>
<p>If the response evolves from this:</p>
<pre><code class="language-json">{
  &quot;id&quot;: &quot;ord_123&quot;,
  &quot;status&quot;: &quot;paid&quot;
}
</code></pre>
<p>to this:</p>
<pre><code class="language-json">{
  &quot;id&quot;: &quot;ord_123&quot;,
  &quot;status&quot;: &quot;paid&quot;,
  &quot;estimatedDeliveryDate&quot;: &quot;2026-05-29&quot;
}
</code></pre>
<p>older clients should ignore the extra property and keep working.</p>
<p>In .NET, <code>System.Text.Json</code> helps because unknown properties are ignored by default.
The real risk is usually overly strict generated SDKs or contract tests that assert exact JSON equality.</p>
<p>This is one of the most common self-inflicted problems I see.
Teams say they want backward compatibility, then generate client models that reject any unexpected field in the response.</p>
<p>That is not a compatibility strategy.
That is a trap.</p>
<p>Your server should be free to add optional data.
Your clients should be resilient enough to ignore what they don't understand.</p>
<h2>3. Don't Change What an Existing Operation Does</h2>
<p>Fields and shapes get most of the attention in compatibility discussions, but the most dangerous breaking changes hide in <strong>behavior</strong>.</p>
<p>The URL is the same.
The request body is the same.
The response shape is the same.
What the operation <em>does</em> on the server is different.</p>
<p>Take <code>DELETE /orders/{id}</code>.</p>
<p>When the API shipped, that endpoint was a soft delete.
The order moved into an <code>archived</code> state, stayed in the database, still showed up in audit reports, and could be restored by support.</p>
<p>The contract that clients built on wasn't just the HTTP verb and the path.
It was the full behavior:</p>
<ul>
<li>The order is recoverable</li>
<li>Related invoices and shipments are untouched</li>
<li>Audit history is preserved</li>
<li>The same call is safe to retry</li>
</ul>
<p>Months later, the team decides soft-delete is messy.
The &quot;fix&quot; turns <code>DELETE /orders/{id}</code> into a hard delete:</p>
<ul>
<li>The order row is gone</li>
<li>Related invoices cascade or get orphaned</li>
<li>Audit history loses references</li>
<li>Retrying after a network blip can wipe the wrong record</li>
</ul>
<p>No client noticed at code-review time.
The SDK call still compiles.
The response is still <code>204 No Content</code>.
A support tool that used to call <code>DELETE</code> and then &quot;undo&quot; it now silently destroys data.</p>
<p>This is exactly the kind of change Z. Nemec's rules call out:
<strong>you must not change the processing rules of an existing operation.</strong>
Once clients have integrated, the behavior <em>is</em> the contract, even if it was never written down anywhere.</p>
<p>The same pattern shows up in subtler ways:</p>
<ul>
<li><code>POST /orders</code> used to be idempotent on a client-supplied key, then quietly stops being</li>
<li><code>POST /orders/{id}/cancel</code> used to refund automatically, then stops issuing refunds because &quot;refunds should be a separate call&quot;</li>
<li><code>PUT /orders/{id}</code> used to be a full replace, then becomes a partial merge</li>
<li>A webhook used to fire once per order, now fires per line item</li>
</ul>
<p>Each of these keeps the URL, the verb, and the JSON shape stable.
Each one breaks every existing integration in a way that won't show up in a schema diff.</p>
<p>The safe move is the same as before: <strong>add, don't mutate.</strong></p>
<p>If you want a hard delete, expose it as a new operation and leave the old one alone:</p>
<pre><code class="language-http">DELETE /orders/{id}            # still soft-delete, unchanged
DELETE /orders/{id}?purge=true # new, opt-in hard delete
</code></pre>
<p>Or introduce a new resource entirely (<code>DELETE /orders/{id}/purge</code>) so the destructive behavior has its own name and its own permissions.</p>
<p>The rule is simple: <strong>once an operation ships, its behavior is part of the contract.</strong>
You can add new operations next to it.
You can deprecate it.
You cannot quietly change what it does.</p>
<h2>4. Be Very Careful With Validation</h2>
<p>This one is underrated.</p>
<p>There are two flavors of the same mistake:</p>
<ul>
<li>Taking an existing optional field and making it required</li>
<li>Adding a brand-new field and making it required from day one</li>
</ul>
<p>Both break older clients in exactly the same way.
The endpoint path doesn't move, but requests that used to succeed now get rejected.</p>
<p>Here's a simple example using <code>POST /orders</code>.</p>
<p>Yesterday this request was valid:</p>
<pre><code class="language-json">{
  &quot;customerId&quot;: &quot;cus_123&quot;,
  &quot;currency&quot;: &quot;USD&quot;
}
</code></pre>
<p>Today the API requires a country for tax calculation:</p>
<pre><code class="language-json">{
  &quot;customerId&quot;: &quot;cus_123&quot;,
  &quot;currency&quot;: &quot;USD&quot;,
  &quot;country&quot;: &quot;US&quot;
}
</code></pre>
<p>Whether <code>country</code> was previously optional or didn't exist at all, the result is the same: every existing integration starts failing at runtime.</p>
<p>A safer path is to accept missing values for older clients, infer defaults where possible, or introduce a new operation for the stricter workflow.</p>
<p>For example:</p>
<ul>
<li>Accept missing <code>country</code> during a transition window</li>
<li>Infer it from an existing billing profile if you can</li>
<li>Add a new <code>POST /checkout-sessions</code> flow that requires the richer request model</li>
</ul>
<p>Response changes usually get careful design review.
Request validation changes deserve the same scrutiny.
And the underlying rule is the one that catches both flavors: <strong>anything you add to the contract has to be optional, and anything that was optional has to stay optional.</strong></p>
<h2>A New Operation Is Often Cheaper Than a New Version</h2>
<p>Sometimes the use case really did change enough that piling more flags and optional parameters onto an existing endpoint becomes confusing.</p>
<p>This is what a bad evolution path looks like:</p>
<pre><code class="language-http">POST /orders?validateOnly=true&amp;includeTaxEstimate=true&amp;reserveInventory=true
</code></pre>
<p>At that point you don't have one clean operation.
You have multiple workflows hiding behind one endpoint.</p>
<p>That's when I prefer a new operation or resource over a whole API version.</p>
<pre><code class="language-http">POST /orders
POST /orders/quote
POST /checkout-sessions
</code></pre>
<p>This keeps the old contract stable while giving the new behavior a clean home.</p>
<p><code>POST /orders</code> stays the simple &quot;place an order&quot; endpoint.
<code>POST /orders/quote</code> becomes the &quot;tell me what this would cost&quot; operation.
<code>POST /checkout-sessions</code> can support a richer, more guided flow without contaminating the original contract.</p>
<p>That is usually much cheaper than creating <code>/v2/orders</code> and dragging the rest of your API along with it.</p>
<h2>Deprecate Like You Mean It</h2>
<p>This is the missing half of API change management.</p>
<p>Most deprecations are fake.
They exist in docs, but nothing operational happens.</p>
<p>A real deprecation process should include four things:</p>
<ol>
<li>Mark the old field or endpoint as deprecated in your OpenAPI description.</li>
<li>Signal the deprecation at runtime.</li>
<li>Give consumers a migration path.</li>
<li>Measure actual usage before removing anything.</li>
</ol>
<p>If you're on HTTP, runtime signaling can be as simple as response headers like these:</p>
<pre><code class="language-http">Deprecation: true
Sunset: Wed, 31 Dec 2026 23:59:59 GMT
Link: &lt;https://docs.example.com/migrations/orders-total&gt;; rel=&quot;deprecation&quot;
</code></pre>
<p>Now the deprecation is visible in the docs, visible in live traffic, and connected to an actual migration guide.</p>
<p>And this is where telemetry matters.
If you don't know which clients still use the deprecated field or endpoint, you are not managing change.
You are guessing.</p>
<p>Track usage by client ID, API key, tenant, or application name.
Then wait until usage is effectively gone before removing anything.</p>
<h2>When Versioning Is Actually The Right Call</h2>
<p>I am not anti-versioning.</p>
<p>Version when the old and new semantics cannot coexist safely.
Version when the resource model changed fundamentally.
Version when compatibility rules would force you into a contract nobody can reason about.</p>
<p>In those cases, version deliberately.</p>
<p>And deliberate versioning means choosing the smallest break you can justify.</p>
<p>Sometimes that's a new endpoint shape.
Sometimes it's a representation variant.
Sometimes, especially for public APIs, it's straightforward URL versioning because it is explicit and easy to communicate.</p>
<p>The key is not which mechanism you pick.
The key is that you reached for it because coexistence failed, not because it was the first idea on the table.</p>
<p>And if you do version, pair it with an actual deprecation process:</p>
<ul>
<li>Mark old fields or endpoints as deprecated</li>
<li>Communicate a removal date</li>
<li>Give clients migration examples</li>
<li>Monitor usage before removing anything</li>
</ul>
<p>The real work is not creating <code>v2</code>.
The real work is getting consumers off <code>v1</code>.</p>
<h2>Takeaway</h2>
<p>The best API version is often the one you never have to create.</p>
<p>If you want a simple decision rule, use this:</p>
<ol>
<li>Can I add instead of replace?</li>
<li>Can old and new contracts coexist during a migration window?</li>
<li>Can I introduce a new operation instead of mutating an old one?</li>
<li>Can I deprecate the old shape with docs, headers, and telemetry?</li>
</ol>
<p>If the answer is yes, you probably don't need a new version.</p>
<p>If the answer is no, and the old and new worlds genuinely cannot live side by side, version deliberately.</p>
<p>That's the real point.</p>
<p>Design contracts to evolve.
Treat clients as long-lived integrations, not just today's code.
And reserve versioning for the cases where compatibility truly runs out.</p>
<p>If you want to go deeper on designing and evolving HTTP APIs, check out <a href="/pragmatic-rest-apis"><strong>Pragmatic REST APIs</strong></a>.
It's where I cover the patterns, trade-offs, and implementation details I use when building APIs that need to survive real clients and real change.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_193.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[What Invariants Are (and Why a Domain Model Is the Best Place to Enforce Them)]]></title>
            <link>https://milanjovanovic.tech/blog/what-invariants-are-and-why-a-domain-model-is-the-best-place-to-enforce-them</link>
            <guid isPermaLink="false">what-invariants-are-and-why-a-domain-model-is-the-best-place-to-enforce-them</guid>
            <pubDate>Sat, 02 May 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[Most 'DDD-ish' code I review enforces business rules everywhere except where it should: in the model itself. The same rule ends up duplicated across handlers, validators, and controllers, and each copy drifts a little over time. Here's how I think about invariants, and why an always-valid domain model is the cleanest way to enforce them.]]></description>
            <content:encoded><![CDATA[<p>A lot of the &quot;DDD-ish&quot; .NET code I review scatters business rules across handlers, validators,
and controllers, and barely puts any on the domain model itself.</p>
<p>Each copy of the same rule drifts a little over time, and whether a given object is <em>valid</em> starts to depend on which path the caller took to reach it.</p>
<p>You can absolutely build a working system this way.
I've shipped plenty of procedural code with enough <code>if</code>-checks to keep things in line.
But there's a cleaner way to think about it, and it starts with a single idea.</p>
<h2>What Is an Invariant?</h2>
<p>An <strong>invariant</strong> is a rule about an object that must hold true for as long as the object exists.</p>
<p>Not just when you save it, or when a validator happens to run.
The rule has to hold every time you touch the object, no matter how it got into memory.</p>
<p>A few examples:</p>
<ul>
<li>A <code>Course</code> always has a non-empty title.</li>
<li>An <code>Order</code> total always equals the sum of its line items.</li>
<li>A <code>Subscription</code> is in exactly one state: <code>Trial</code>, <code>Active</code>, <code>PastDue</code>, or <code>Canceled</code>.</li>
<li>A published course has at least one lesson.</li>
</ul>
<p>None of these mention validation, persistence, or HTTP.
They're statements about the <em>domain</em>, and they should be true regardless of how the object got loaded.</p>
<h2>Where Procedural Code Goes Wrong</h2>
<p>Take a simple <code>Course</code> written the way most CRUD-ish .NET apps still write it:</p>
<pre><code class="language-csharp">public class Course
{
    public string Title { get; set; }
    public CourseStatus Status { get; set; }
    public DateTime? PublishedOn { get; set; }
    public decimal Price { get; set; }
}
</code></pre>
<p>There's no constructor and every property has a public setter, so the class is willing to accept any combination of values.</p>
<p>To keep the data correct, the rules end up scattered across the application:</p>
<ul>
<li><code>CreateCourseValidator</code> checks the title isn't empty.</li>
<li><code>PublishCourseHandler</code> sets <code>Status</code> and <code>PublishedOn</code>, and remembers to check the course isn't already published.</li>
<li><code>ChangePriceHandler</code> checks the course isn't archived.</li>
<li>A new endpoint shows up, someone copies an existing handler, and the archive check quietly goes missing.</li>
</ul>
<p>Every rule lives in a place that just happens to be on the path the request took.
Nothing on the <code>Course</code> itself prevents it from drifting into an invalid state.</p>
<p>That's the real cost of an <a href="refactoring-from-an-anemic-domain-model-to-a-rich-domain-model"><strong>anemic model</strong></a>.
It isn't that there's no behavior on the class.
It's that the class makes no promises, so every caller has to enforce the rules itself.</p>
<h2>Always-Valid: The Model as the Source of Truth</h2>
<p>The shift I want to make is simple: the model never accepts an invalid state.</p>
<p>If you're holding a <code>Course</code> reference, you can trust it.
You don't need an <code>if (course.Title is null)</code> somewhere down the call stack, you don't need a parallel validator double-checking, and you don't need to hope the handler remembered the right guard.</p>
<p>Three moves get you there.</p>
<h3>1. Block construction of invalid objects</h3>
<p>A <code>Course</code> without a title shouldn't exist, so the easiest fix is to make it impossible to construct.</p>
<pre><code class="language-csharp">public class Course
{
    private Course(CourseId id, string title, Money price)
    {
        Id = id;
        Title = title;
        Price = price;
        Status = CourseStatus.Draft;
    }

    public static Result&lt;Course&gt; Create(string title, Money price)
    {
        if (string.IsNullOrWhiteSpace(title))
        {
            return CourseErrors.TitleRequired;
        }

        return new Course(CourseId.New(), title, price);
    }
}
</code></pre>
<p>A private constructor with a static factory gives you a single, well-known place where a <code>Course</code> can come into existence, and that's where the validation runs.
From that point on, any code holding a <code>Course</code> reference can assume it has a valid title.</p>
<p><a href="value-objects-in-dotnet-ddd-fundamentals"><strong>Value objects</strong></a> like <code>Money</code> apply the same idea at a smaller scope, so a <code>Money</code> can't be negative or be missing its currency by the time you're using it.</p>
<h3>2. Encapsulate state transitions</h3>
<p>Once construction is locked down, the next leak is state changes.
The class should control how it changes, instead of leaving that up to whoever has a reference to it.
No setters, and every change goes through a method that knows the rules.</p>
<pre><code class="language-csharp">public Result Publish(IDateTimeProvider clock)
{
    if (Status != CourseStatus.Draft)
    {
        return CourseErrors.AlreadyPublished;
    }

    if (_lessons.Count == 0)
    {
        return CourseErrors.CannotPublishWithoutLessons;
    }

    Status = CourseStatus.Published;
    PublishedOn = clock.UtcNow;
    return Result.Success();
}
</code></pre>
<p>The handler doesn't need to know whether the course was already published, and it doesn't need to remember to check for empty lessons.
It calls <code>Publish</code> and propagates whatever result comes back.
The rule lives next to the state it protects, in one place.</p>
<h3>3. Encapsulate the aggregate</h3>
<p>Some rules span multiple entities inside the same boundary.
The aggregate root is the right place to enforce those, because it's the transactional boundary.</p>
<p>Take this rule: a published course must have at least one lesson, and lessons can't be removed once it's published.</p>
<p>The wrong way is to expose <code>Lessons</code> as a mutable collection and rely on the application service to remember the rule everywhere it's used.
The right way is to keep the collection private and force every change through the root:</p>
<pre><code class="language-csharp">public sealed class Course
{
    private readonly List&lt;Lesson&gt; _lessons = [];
    public IReadOnlyCollection&lt;Lesson&gt; Lessons =&gt; _lessons.AsReadOnly();

    public Result RemoveLesson(LessonId id)
    {
        if (Status == CourseStatus.Published)
        {
            return CourseErrors.CannotModifyPublishedLessons;
        }

        var lesson = _lessons.FirstOrDefault(l =&gt; l.Id == id);
        if (lesson is null)
        {
            return CourseErrors.LessonNotFound;
        }

        _lessons.Remove(lesson);
        return Result.Success();
    }
}
</code></pre>
<p>When a rule needs to span <em>two</em> aggregates instead of one, that's a different problem, and I'd reach for a <a href="how-to-use-domain-events-to-build-loosely-coupled-systems"><strong>domain event</strong></a> rather than letting one aggregate reach into another.</p>
<h2>What You Actually Get</h2>
<p>You can write the same system procedurally, and it can work well.
The thing you give up is <strong>trust</strong>.</p>
<p>In a procedural system, every caller shares responsibility for not breaking the rules.
In an always-valid model, that responsibility lives on the domain model.
The difference compounds over time:</p>
<ul>
<li>Validators don't drift, because there's nothing to duplicate.</li>
<li>Code reviews focus on behavior instead of &quot;did we forget a check?&quot;.</li>
<li>New endpoints can't accidentally bypass a rule that lives on the entity.</li>
<li>Tests stop covering scenarios that aren't even expressible.</li>
</ul>
<p>The model goes from being a passive data carrier to being the smallest, sharpest place where the business rules live.</p>
<p>Fundamentally, this is about <strong>encapsulation</strong>.
The model encapsulates the rules that govern its state, and the rest of the system interacts with it through a well-defined interface.
That leads to cleaner code, fewer bugs, and a more maintainable system overall.</p>
<h2>Summary</h2>
<p>An invariant is a rule that must always hold true while the object exists, and the cleanest place to enforce it is on the object itself.</p>
<ul>
<li><strong>Construction invariants</strong> belong in a private constructor behind a factory.</li>
<li><strong>State transition invariants</strong> belong in methods that own the state they change.</li>
<li><strong>Aggregate-wide invariants</strong> belong on the root, with child entities accessed only through it.</li>
</ul>
<p>The tradeoff is that you give up the freedom to write procedural code that could be easier to understand in the short term,
but you get a model that you can trust to always be valid.</p>
<p>If you want to go deeper into modeling aggregates, value objects, and rich behavior across a real system,
I think you will enjoy <a href="/pragmatic-domain-driven-design"><strong>Pragmatic Domain-Driven Design</strong></a>.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_192.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[The Test Pyramid Is a Lie (and What I Do Instead)]]></title>
            <link>https://milanjovanovic.tech/blog/the-test-pyramid-is-a-lie-and-what-i-do-instead</link>
            <guid isPermaLink="false">the-test-pyramid-is-a-lie-and-what-i-do-instead</guid>
            <pubDate>Sat, 25 Apr 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[The test pyramid made sense when integration tests meant a shared database server and a 20-minute build. It doesn't match how I build .NET systems in 2026. Most of my real bugs live at the seams, and unit tests don't catch any of them. Here's the testing shape I actually ship with.]]></description>
            <content:encoded><![CDATA[<p>I'll be honest. For years, my projects didn't look like the <strong>test pyramid</strong>.</p>
<p>A wide base of unit tests, a narrow middle of integration tests, a tiny sliver of end-to-end tests at the top.
I'd nod along in conference talks, then go back to my own code and do something different.</p>
<p>A thin layer of unit tests for the things worth unit-testing.
A thick slab of integration tests against real PostgreSQL, real RabbitMQ, real HTTP.
A handful of end-to-end tests for the flows that would get me fired if they broke.</p>
<p>And I shipped with more confidence that way, not less.
Here's why, and here's the shape I actually use.</p>
<h2>Where the Pyramid Comes From</h2>
<p>The pyramid was popularized by Mike Cohn in 2009, when integration tests meant a shared database server, flaky CI, and 20-minute builds.
Unit tests with mocks were the pragmatic compromise.</p>
<p><img src="/blogs/mnw_191/test_pyramid.png" alt="The classic test pyramid, with a wide base of unit tests, a narrow middle of integration tests, and a tiny sliver of end-to-end tests at the top."></p>
<p>That world is gone.
With <a href="testcontainers-integration-testing-using-docker-in-dotnet"><strong>Testcontainers</strong></a>, I can spin up PostgreSQL, Redis, and RabbitMQ in a fresh container per test class in a few seconds.
The <a href="https://learn.microsoft.com/en-us/dotnet/aspire/testing/overview"><strong>Aspire test host</strong></a> takes this further by wiring up your entire application graph.
The argument that real dependencies are too expensive mostly doesn't apply anymore.</p>
<p>But the advice didn't update.</p>
<h2>The Bug That Convinced Me</h2>
<p>A few years ago I had a service with <strong>94% unit test coverage</strong>. All green.</p>
<p>A user reported that deleting an account didn't actually delete their data.
The bug was three lines long:</p>
<pre><code class="language-csharp">public async Task Handle(DeleteAccountCommand command, CancellationToken ct)
{
    var account = await _repository.GetByIdAsync(command.AccountId, ct);
    account.MarkAsDeleted();
    // Missing: await _unitOfWork.SaveChangesAsync(ct);
}
</code></pre>
<p>The honest diagnosis is that the test for this case was never written.
You can absolutely verify <code>SaveChangesAsync</code> was called with a mock.
But in a codebase where that test is <em>one of hundreds</em> of handler tests, each with its own mock setup and its own verification list, it's the kind of assertion people forget.
I forgot.</p>
<p>A single integration test against a real database would have caught it without anyone having to remember.
That's the point: the fewer invariants your test style forces you to remember, the fewer bugs slip through.</p>
<p>That was the last week I took the test pyramid seriously.</p>
<h2>What Unit Tests Are Actually Good At</h2>
<p>I still write unit tests. Just not many.</p>
<p>They earn their keep when the logic is non-trivial, pure (no I/O, no time, no randomness), and hard to exercise end-to-end.
That's a specific set of code: <a href="value-objects-in-dotnet-ddd-fundamentals"><strong>value objects</strong></a> and <a href="refactoring-from-an-anemic-domain-model-to-a-rich-domain-model"><strong>rich domain models</strong></a>, pricing and tax calculations, parsers, mappers, serializers.</p>
<p>Notice what's <strong>not</strong> on that list: application services, handlers, controllers, repositories, infrastructure.
Those live at the seams, and the seams are where real bugs live.</p>
<h2>What I Actually Write Instead</h2>
<p>Here's the shape I've settled on for a typical .NET service or <a href="what-is-a-modular-monolith"><strong>modular monolith</strong></a>.</p>
<h3>Layer 1: A thin base of unit tests</h3>
<p>Maybe 15-25% of the test count. All domain logic.
No mocks of collaborators. If a unit test needs a mock, I usually pull the test up to the integration layer instead.</p>
<pre><code class="language-csharp">[Fact]
public void Confirm_WhenPending_TransitionsToConfirmed()
{
    var order = Order.Create(CustomerId.New(), Money.Usd(100));

    order.Confirm();

    order.Status.Should().Be(OrderStatus.Confirmed);
    order.DomainEvents.Should().ContainSingle(e =&gt; e is OrderConfirmedEvent);
}
</code></pre>
<p>No container or mocks.
Microseconds per test.
This is what unit tests are for.</p>
<h3>Layer 2: A thick middle of integration tests</h3>
<p>The majority. Maybe 60-70% of the suite.
Every <a href="cqrs-pattern-with-mediatr"><strong>command and query handler</strong></a>, every HTTP endpoint, every message consumer gets a test that runs against <strong>real infrastructure</strong> inside Testcontainers.
In a <a href="testing-modular-monoliths-system-integration-testing"><strong>modular monolith</strong></a>, this is where you verify that modules talk to each other correctly across their public APIs.</p>
<pre><code class="language-csharp">public class DeleteAccountTests(IntegrationTestWebAppFactory factory)
    : BaseIntegrationTest(factory)
{
    [Fact]
    public async Task DeleteAccount_WhenAccountExists_MarksAccountAsDeleted()
    {
        var account = await CreateAccountAsync();

        var response = await HttpClient.DeleteAsync($&quot;/accounts/{account.Id}&quot;);

        response.StatusCode.Should().Be(HttpStatusCode.NoContent);

        var stored = await DbContext.Accounts
            .IgnoreQueryFilters()
            .SingleAsync(a =&gt; a.Id == account.Id);

        stored.IsDeleted.Should().BeTrue();
    }
}
</code></pre>
<p>That test exercises the HTTP layer, routing, model binding, authorization, the handler, the unit of work, EF Core, and PostgreSQL.
It proves the thing you actually care about: <strong>when I call this endpoint, the row changes.</strong>
And it does it without anyone having to remember to assert <code>SaveChangesAsync</code> was called.</p>
<h3>Layer 3: A small cap of end-to-end tests</h3>
<p>Under 10%. Only the flows where a silent failure would be a commercial or compliance problem.
Signup, payment, refund, password reset, <a href="how-to-implement-two-factor-authentication-in-aspnetcore"><strong>two-factor enrollment</strong></a>.
They're slow and occasionally flaky, but they catch the one failure mode everything else misses: <em>the system as a whole still works</em>.</p>
<h3>Layer 0: Architecture and contract tests</h3>
<p>Often forgotten, but they're part of the suite.
<a href="5-architecture-tests-you-should-add-to-your-dotnet-projects"><strong>Architecture tests</strong></a> enforce layering and module boundaries.
Contract tests verify that message schemas and API shapes don't drift.
They run in milliseconds and catch the &quot;six months from now, someone will break this without realizing&quot; kind of bug.</p>
<p>The shape that comes out of this is closer to Kent C. Dodds' <strong>testing trophy</strong> than a pyramid.
The fat middle is deliberate. That's where my confidence comes from.</p>
<h2>The Usual Objections</h2>
<p><strong>&quot;Integration tests are slow.&quot;</strong>
My typical integration suite runs in 2-4 minutes in CI with Testcontainers reuse and test-class parallelization.
Slower than unit tests, yes. Faster than finding the bug in production.</p>
<p><strong>&quot;Mocks are fine if you're disciplined.&quot;</strong>
Maybe. But every large codebase I've audited that leaned heavily on mocks had the same pathology: tests that pass after a refactor even though the refactor broke production.
That's not discipline failing. That's the tool being pointed in the wrong direction.</p>
<h2>Summary</h2>
<p>The test pyramid was good advice for 2009 infrastructure and poor advice for 2026 infrastructure.
Testcontainers and Aspire changed the economics, and the fastest feedback loop that still tells you the truth is now an integration test against real dependencies.
Unit tests still belong on pure domain logic. Everything at the seams belongs in the integration layer.</p>
<p>If you want to see how I wire this into a real system, with the <a href="testing-modular-monoliths-system-integration-testing"><strong>integration test harness</strong></a>, module boundaries, and the full <a href="/pragmatic-clean-architecture"><strong>Clean Architecture</strong></a> setup, check out <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture</strong></a>.
It's the same approach I use on my own projects.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_191.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Why I Switched to Primary Constructors for DI in C#]]></title>
            <link>https://milanjovanovic.tech/blog/why-i-switched-to-primary-constructors-for-di-in-csharp</link>
            <guid isPermaLink="false">why-i-switched-to-primary-constructors-for-di-in-csharp</guid>
            <pubDate>Sat, 18 Apr 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[I resisted primary constructors for a while. They felt like a shortcut that would cost me later. But after using them across several projects, I'm sold, with one important caveat. Here's what convinced me to switch, and the one pitfall you need to watch out for.]]></description>
            <content:encoded><![CDATA[<p>I'll be honest. I resisted primary constructors for a while.</p>
<p>When C# 12 extended them from <code>record</code> types to regular classes and structs, my first reaction was skepticism.
An implicit mutable capture instead of explicit <code>readonly</code> fields?
That felt like trading safety for convenience.</p>
<p>But after using them across several projects, I changed my mind.
The boilerplate they eliminate in DI service classes is significant, and the pitfall I was worried about is manageable once you know about it.</p>
<p>Here's what convinced me to switch, and the one thing you need to watch out for.</p>
<h2>What Changed My Mind</h2>
<p>Here's what my service classes used to look like:</p>
<pre><code class="language-csharp">public class OrderService
{
    private readonly IOrderRepository _orderRepository;
    private readonly ILogger&lt;OrderService&gt; _logger;

    public OrderService(
        IOrderRepository orderRepository,
        ILogger&lt;OrderService&gt; logger)
    {
        _orderRepository = orderRepository;
        _logger = logger;
    }

    public async Task&lt;Order?&gt; GetOrderAsync(Guid id)
    {
        _logger.LogInformation(&quot;Fetching order {OrderId}&quot;, id);

        return await _orderRepository.GetByIdAsync(id);
    }
}
</code></pre>
<p>And here's what they look like now:</p>
<pre><code class="language-csharp">public class OrderService(
    IOrderRepository orderRepository,
    ILogger&lt;OrderService&gt; logger)
{
    public async Task&lt;Order?&gt; GetOrderAsync(Guid id)
    {
        logger.LogInformation(&quot;Fetching order {OrderId}&quot;, id);

        return await orderRepository.GetByIdAsync(id);
    }
}
</code></pre>
<p>The field declarations, the constructor body, the assignments. All gone.
The parameters are captured and available throughout the class body.</p>
<p>This is the most common use case for primary constructors: <a href="improving-aspnetcore-dependency-injection-with-scrutor"><strong>dependency injection</strong></a> in service classes.
You declare what you need, and use it directly.</p>
<h2>Where I Use Them Most: DI Service Classes</h2>
<p>The place where primary constructors sold me is <a href="http://ASP.NET">ASP.NET</a> Core service classes.
This is where I spend most of my time, and the boilerplate savings add up fast.</p>
<p>Here's a more realistic example from a checkout flow:</p>
<pre><code class="language-csharp">public class CheckoutService(
    IPaymentProcessor paymentProcessor,
    IOrderRepository orderRepository,
    ILogger&lt;CheckoutService&gt; logger,
    IOptions&lt;CheckoutOptions&gt; options)
{
    public async Task&lt;CheckoutResult&gt; ProcessAsync(
        Cart cart,
        CancellationToken ct = default)
    {
        var settings = options.Value;

        if (cart.Total &lt; settings.MinimumOrderAmount)
        {
            logger.LogWarning(&quot;Order below minimum: {Total}&quot;, cart.Total);
            return CheckoutResult.BelowMinimum;
        }

        var order = Order.Create(cart);

        await paymentProcessor.ChargeAsync(order, ct);
        await orderRepository.SaveAsync(order, ct);

        logger.LogInformation(&quot;Checkout complete for order {OrderId}&quot;, order.Id);

        return CheckoutResult.Success;
    }
}
</code></pre>
<p>Four dependencies, zero boilerplate.
The class reads top-to-bottom without any noise.</p>
<p>This pattern works well because service classes typically don't need to validate or transform their dependencies.
The DI container provides them, and you use them.
Primary constructors are a perfect fit for this.</p>
<h2>Entity Construction (With a Caveat)</h2>
<p>I also started using primary constructors for <a href="refactoring-from-an-anemic-domain-model-to-a-rich-domain-model"><strong>domain entities</strong></a> and <a href="value-objects-in-dotnet-ddd-fundamentals"><strong>value objects</strong></a> where you want to enforce required parameters at construction time:</p>
<pre><code class="language-csharp">public class Order(Guid customerId, Money total)
{
    public Guid Id { get; } = Guid.NewGuid();
    public Guid CustomerId { get; } = customerId;
    public Money Total { get; } = total;
    public OrderStatus Status { get; private set; } = OrderStatus.Pending;
    public DateTime CreatedAt { get; } = DateTime.UtcNow;

    public void Confirm()
    {
        if (Status != OrderStatus.Pending)
        {
            throw new InvalidOperationException(
                $&quot;Cannot confirm order in {Status} status.&quot;);
        }

        Status = OrderStatus.Confirmed;
    }
}
</code></pre>
<p>There's no way to create an <code>Order</code> without a <code>customerId</code> and <code>total</code>.
The primary constructor makes this constraint visible at the type declaration level.</p>
<p>Notice the key difference from the service class pattern: here, I'm assigning primary constructor parameters to <strong>properties with initializers</strong> (<code>= customerId</code>).
This is important, and it leads to the biggest pitfall.</p>
<h2>The Pitfall That Almost Stopped Me</h2>
<p>This was the reason I held off for so long.</p>
<p><strong>Primary constructor parameters are not <code>readonly</code> fields.</strong></p>
<p>When you use a primary constructor parameter directly in the class body (like we did in the service class), the compiler captures it as a <strong>mutable variable</strong>.
There's no <code>readonly</code> backing field generated behind the scenes.</p>
<p>This means you can accidentally reassign a parameter:</p>
<pre><code class="language-csharp">public class OrderService(
    IOrderRepository orderRepository,
    ILogger&lt;OrderService&gt; logger)
{
    public async Task&lt;Order?&gt; GetOrderAsync(Guid id)
    {
        logger.LogInformation(&quot;Fetching order {OrderId}&quot;, id);

        return await orderRepository.GetByIdAsync(id);
    }

    public void SomeOtherMethod()
    {
        // This compiles. No warning. No error.
        orderRepository = null!;
        logger = null!;
    }
}
</code></pre>
<p>This compiles without any warning.</p>
<p>With a traditional constructor and <code>private readonly</code> fields, the compiler would stop you immediately.
With primary constructors, it stays silent.</p>
<p><strong>If you need immutability guarantees</strong>, explicitly assign the parameter to a <code>readonly</code> field:</p>
<pre><code class="language-csharp">public class OrderService(
    IOrderRepository orderRepository,
    ILogger&lt;OrderService&gt; logger)
{
    private readonly IOrderRepository _orderRepository = orderRepository;
    private readonly ILogger&lt;OrderService&gt; _logger = logger;

    public async Task&lt;Order?&gt; GetOrderAsync(Guid id)
    {
        _logger.LogInformation(&quot;Fetching order {OrderId}&quot;, id);

        return await _orderRepository.GetByIdAsync(id);
    }
}
</code></pre>
<p>But now you've lost most of the benefit of primary constructors.
You're back to field declarations and assignments, just with a different syntax.</p>
<p>In practice, I've never actually hit this bug in a DI service class.
You're unlikely to accidentally reassign <code>logger</code> in the middle of a method.
But it <strong>can</strong> bite you in entity classes or value types where immutability actually matters.
That's the one place where I still stay cautious.</p>
<h2>Where I Still Use Traditional Constructors</h2>
<p>I haven't switched everything over.
Here are the cases where I stick with the traditional approach:</p>
<p><strong>Complex validation logic.</strong> If you need to validate parameters before assigning them, you need a constructor body:</p>
<pre><code class="language-csharp">public class EmailAddress
{
    private readonly string _value;

    public EmailAddress(string value)
    {
        if (string.IsNullOrWhiteSpace(value) || !value.Contains('@'))
        {
            throw new ArgumentException(
                &quot;Invalid email address.&quot;, nameof(value));
        }

        _value = value;
    }
}
</code></pre>
<p>Primary constructors don't give you a place to put validation logic before the class body runs.</p>
<p><strong>Multiple constructor overloads.</strong> Primary constructors support one constructor signature.
If you need overloads, you'll have to chain secondary constructors with <code>this(...)</code>, which gets messy fast.</p>
<p><strong>Too many parameters.</strong> Once you hit 5+ dependencies, the primary constructor line becomes hard to read.
At that point, your class probably has too many responsibilities, and <a href="5-awesome-csharp-refactoring-tips"><strong>refactoring</strong></a> it is a better solution than formatting tricks.</p>
<h2>Summary</h2>
<p>Here's what I've settled on after using primary constructors across several projects:</p>
<ul>
<li>I use <strong>primary constructors</strong> for all my <strong>DI service classes</strong>. The boilerplate savings are worth it.</li>
<li>They're useful for <strong>entity construction</strong> when you want to enforce required parameters at the type level.</li>
<li>Primary constructor parameters are <strong>captured as mutable variables</strong>, not <code>readonly</code> fields. This is the one thing you need to know.</li>
<li>I'm not afraid of the mutable capture pitfall in service classes, because it's unlikely to cause real bugs in that context.</li>
<li>I stick with <strong>traditional constructors</strong> for validation-heavy types, multiple overloads, or classes with too many dependencies</li>
</ul>
<p>The switch was worth it.
My service classes are shorter, easier to scan, and the pitfall is manageable with the right tooling.</p>
<p>That's all for today.
See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_190.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Implementing the Saga Pattern With Wolverine]]></title>
            <link>https://milanjovanovic.tech/blog/implementing-the-saga-pattern-with-wolverine</link>
            <guid isPermaLink="false">implementing-the-saga-pattern-with-wolverine</guid>
            <pubDate>Sat, 11 Apr 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[Long-running business processes don't fit neatly into a single request. Wolverine's Saga support gives you a convention-based approach to orchestrating workflows in .NET - with built-in timeout handling and compensation. Here's how to implement a user onboarding saga.]]></description>
            <content:encoded><![CDATA[<p>Long-running business processes don't fit neatly into a single request.</p>
<p>Think about user onboarding: you register the user, send a verification email, wait for them to verify, and then send a welcome email.
Each step depends on the previous one.
If the user never verifies, you need a way to handle that.</p>
<p>The <a href="implementing-the-saga-pattern-with-masstransit"><strong>Saga pattern</strong></a> breaks this into a sequence of steps, each with its own message and handler.
If a step fails or times out, the saga runs <a href="https://en.wikipedia.org/wiki/Compensating_transaction">compensation logic</a> instead of leaving the system in a broken state.</p>
<p>I've covered sagas with <a href="implementing-the-saga-pattern-with-masstransit"><strong>MassTransit</strong></a> and <a href="implementing-the-saga-pattern-with-rebus-and-rabbitmq"><strong>Rebus</strong></a> before.
Both work well, but their <a href="https://en.wikipedia.org/wiki/Finite-state_machine">state machine</a> DSLs come with a fair amount of ceremony.
Since <a href="mediatr-and-masstransit-going-commercial-what-this-means-for-you">MassTransit moved to a commercial license</a>, more teams have been exploring Wolverine as an alternative.</p>
<p><a href="https://wolverinefx.net/"><strong>Wolverine</strong></a> takes a <a href="https://wolverinefx.net/guide/durability/sagas">different approach</a> - you write a class that extends <code>Saga</code>, define <code>Handle</code> methods for each message type, and <a href="https://wolverinefx.net/guide/handlers/cascading">cascade new messages</a> from return values.
Wolverine handles routing, persistence, and correlation automatically.</p>
<h2>Configuring Wolverine</h2>
<p>We need <a href="event-driven-architecture-in-dotnet-with-rabbitmq"><strong>RabbitMQ</strong></a> for message transport and <a href="https://www.postgresql.org/">PostgreSQL</a> for durable saga state and messaging.</p>
<pre><code class="language-csharp">var connectionString = builder.Configuration.GetConnectionString(&quot;user-mgmt&quot;);

builder.Host.UseWolverine(options =&gt;
{
    options.UseRabbitMqUsingNamedConnection(&quot;rmq&quot;)
        .AutoProvision()
        .UseConventionalRouting();

    options.Policies.DisableConventionalLocalRouting();

    options.PersistMessagesWithPostgresql(connectionString!);
});
</code></pre>
<ul>
<li><code>AutoProvision</code> creates RabbitMQ exchanges and queues automatically</li>
<li><code>UseConventionalRouting</code> routes messages to queues based on message type names</li>
<li><code>DisableConventionalLocalRouting</code> forces all messages through RabbitMQ instead of in-process handling</li>
<li><a href="https://wolverinefx.net/guide/durability/postgresql"><code>PersistMessagesWithPostgresql</code></a> stores saga state and messages in PostgreSQL. Wolverine uses <a href="https://wolverinefx.net/guide/durability/sagas#lightweight-saga-storage"><strong>lightweight saga storage</strong></a> to create a table per saga type, and the <a href="https://wolverinefx.net/guide/durability/"><strong>durable messaging</strong></a> infrastructure ensures nothing is lost if the process crashes</li>
</ul>
<p>Wolverine gives you <strong>three ways to persist saga state</strong>.
<a href="https://wolverinefx.net/guide/durability/sagas#lightweight-saga-storage"><strong>Lightweight storage</strong></a> (what we're using) serializes saga state as JSON in a per-saga table with zero ORM config.
<a href="https://wolverinefx.net/guide/durability/marten/sagas"><strong>Marten</strong></a> stores sagas as <a href="https://martendb.io/">Marten</a> documents with <a href="https://en.wikipedia.org/wiki/Optimistic_concurrency_control">optimistic concurrency</a> and strong-typed IDs.
<a href="https://wolverinefx.net/guide/durability/efcore/sagas"><strong>EF Core</strong></a> maps sagas into a flat, queryable table and lets you commit saga state with other data in a single transaction.
If you just need saga state management, lightweight storage is the simplest path.</p>
<p>Required packages:</p>
<pre><code class="language-xml">&lt;PackageReference Include=&quot;WolverineFx&quot; Version=&quot;5.16.2&quot; /&gt;
&lt;PackageReference Include=&quot;WolverineFx.Postgresql&quot; Version=&quot;5.16.2&quot; /&gt;
&lt;PackageReference Include=&quot;WolverineFx.RabbitMQ&quot; Version=&quot;5.16.2&quot; /&gt;
</code></pre>
<h2>The Saga Messages</h2>
<p>Before building the saga, let's define all the messages it will work with:</p>
<pre><code class="language-csharp">public record SendVerificationEmail(Guid UserId, string Email);
public record VerificationEmailSent(Guid Id);

public record VerifyUserEmail(Guid Id);

public record SendWelcomeEmail(Guid UserId, string Email, string FirstName);
public record WelcomeEmailSent(Guid Id);

public record OnboardingTimedOut(Guid Id) : TimeoutMessage(5.Minutes());
</code></pre>
<p><code>OnboardingTimedOut</code> extends Wolverine's <a href="https://wolverinefx.net/guide/durability/sagas#timeout-messages"><code>TimeoutMessage</code></a>, which automatically schedules a delayed delivery.
When the saga starts, Wolverine will deliver this message after 5 minutes.
If the user hasn't verified by then, the saga compensates.</p>
<h2>The Saga State Diagram</h2>
<p>Here's how the saga transitions between states:</p>
<div className="centered">
  ![Saga pattern state diagram showing message flow from broker to consumer to database and processor.](/blogs/mnw_189/saga_pattern_state_diagram.png)
</div>
<h2>Building the Saga</h2>
<p>Here's the complete saga class:</p>
<pre><code class="language-csharp">public class UserOnboardingSaga : Saga
{
    public Guid Id { get; set; }
    public string Email { get; set; } = string.Empty;
    public string FirstName { get; set; } = string.Empty;
    public string LastName { get; set; } = string.Empty;
    public bool IsVerificationEmailSent { get; set; }
    public bool IsEmailVerified { get; set; }
    public bool IsWelcomeEmailSent { get; set; }
    public DateTime StartedAt { get; set; }

    // Step 1: Start the saga when UserRegistered is published
    public static (
        UserOnboardingSaga,
        SendVerificationEmail,
        OnboardingTimedOut) Start(
            UserRegistered @event,
            ILogger&lt;UserOnboardingSaga&gt; logger)
    {
        logger.LogInformation(
            &quot;Starting onboarding for user {UserId}&quot;, @event.Id);

        var saga = new UserOnboardingSaga
        {
            Id = @event.Id,
            Email = @event.Email,
            FirstName = @event.FirstName,
            LastName = @event.LastName,
        };

        return (
            saga,
            new SendVerificationEmail(saga.Id, saga.Email),
            new OnboardingTimedOut(saga.Id));
    }

    // Step 2: Verification email was sent
    public void Handle(
        VerificationEmailSent @event,
        ILogger&lt;UserOnboardingSaga&gt; logger)
    {
        logger.LogInformation(
            &quot;Verification email sent for user {UserId}&quot;, Id);

        IsVerificationEmailSent = true;
    }

    // Step 3: User verified their email
    public SendWelcomeEmail Handle(
        VerifyUserEmail command,
        ILogger&lt;UserOnboardingSaga&gt; logger)
    {
        logger.LogInformation(&quot;Email verified for user {UserId}&quot;, Id);

        IsEmailVerified = true;

        return new SendWelcomeEmail(Id, Email, FirstName);
    }

    // Step 4: Welcome email sent - onboarding complete
    public void Handle(
        WelcomeEmailSent @event,
        ILogger&lt;UserOnboardingSaga&gt; logger)
    {
        logger.LogInformation(&quot;Onboarding complete for user {UserId}&quot;, Id);

        IsWelcomeEmailSent = true;

        MarkCompleted();
    }

    // Compensation: timeout handler
    public void Handle(
        OnboardingTimedOut timeout,
        ILogger&lt;UserOnboardingSaga&gt; logger)
    {
        if (IsEmailVerified)
        {
            logger.LogInformation(
                &quot;Timeout ignored - email already verified for user {UserId}&quot;,
                Id);
            return;
        }

        logger.LogWarning(
            &quot;Onboarding timed out for user {UserId} - email not verified&quot;,
            Id);

        MarkCompleted();
    }

    // NotFound: messages arriving for completed/deleted sagas
    public static void NotFound(
        VerifyUserEmail command,
        ILogger&lt;UserOnboardingSaga&gt; logger)
    {
        logger.LogWarning(
            &quot;Verify email received but saga {Id} no longer exists&quot;,
            command.Id);
    }

    public static void NotFound(
        OnboardingTimedOut timeout,
        ILogger&lt;UserOnboardingSaga&gt; logger)
    {
        logger.LogInformation(
            &quot;Timeout received for already-completed saga {Id}&quot;,
            timeout.Id);
    }
}
</code></pre>
<p>A few things worth calling out.</p>
<p><strong>Starting the saga.</strong> <code>Start</code> is a static factory that returns a tuple: the saga instance, a <code>SendVerificationEmail</code> command, and a <a href="https://wolverinefx.net/guide/messaging/message-bus#scheduling-message-delivery-or-execution">scheduled</a> <code>OnboardingTimedOut</code> message. Wolverine persists the saga and delivers the messages for you.</p>
<p><strong>Handling messages.</strong> Wolverine <a href="https://wolverinefx.net/guide/durability/sagas#saga-message-identity"><strong>correlates messages</strong></a> to the correct saga instance by looking for a <code>[SagaIdentity]</code> attribute, then <code>{SagaTypeName}Id</code>, then <code>Id</code>. Return <code>void</code> to update state silently, or return a message to cascade a new command.</p>
<blockquote>
<p><strong>Warning:</strong> Do not call <code>IMessageBus.InvokeAsync()</code> within a saga handler to execute a command on that same saga. You'll be acting on stale or missing data. Use cascading messages (return values) for subsequent work.</p>
</blockquote>
<p><strong>Completing the saga.</strong> <code>MarkCompleted()</code> tells Wolverine to delete the saga state from PostgreSQL.</p>
<p><strong>Concurrency.</strong> Wolverine applies <a href="https://en.wikipedia.org/wiki/Optimistic_concurrency_control">optimistic concurrency control</a> to saga state by default. If two messages for the same saga arrive at the same time, one succeeds and the other retries automatically.</p>
<p><strong>Timeout and compensation.</strong> <code>OnboardingTimedOut</code> fires 5 minutes after the saga started. If the user verified, we ignore it. Otherwise, we compensate and end the saga. This is the key advantage over fire-and-forget workflows.</p>
<p><strong>NotFound handlers.</strong> Static <a href="https://wolverinefx.net/guide/durability/sagas#when-sagas-are-not-found"><code>NotFound</code> methods</a> handle messages for sagas that no longer exist. You <strong>must</strong> have one for any message type that could arrive after the saga is deleted. The timeout <code>NotFound</code> handler matters most: in the happy path, the saga completes before the timeout fires.</p>
<h2>The Sequence Flow</h2>
<p>Here's the happy path where the user verifies before the timeout:</p>
<p><img src="/blogs/mnw_189/saga_pattern_sequence_diagram.png" alt="Saga pattern sequence diagram showing message flow from broker to consumer to database and processor."></p>
<p>If the user never verifies, the <code>VerifyUserEmail</code> message never arrives.
After 5 minutes, <code>OnboardingTimedOut</code> fires and the saga compensates.</p>
<h2>Summary</h2>
<p>Wolverine's <code>Saga</code> base class gives you a convention-driven way to implement long-running workflows:</p>
<ul>
<li><strong><code>Start</code> methods</strong> create and initialize the saga from a triggering event</li>
<li><strong><code>Handle</code> methods</strong> process messages and cascade new commands via return values</li>
<li><strong><code>TimeoutMessage</code></strong> schedules delayed compensation without external schedulers</li>
<li><strong><code>MarkCompleted()</code></strong> cleans up the saga state when the workflow is done</li>
<li><strong><code>NotFound</code> handlers</strong> gracefully handle messages for sagas that no longer exist</li>
</ul>
<p>The Saga pattern shines when you have multi-step processes with potential failures.
Instead of hoping everything goes right, you design for the cases where it doesn't.</p>
<p>What I really like about Wolverine's approach is how little code you need.
You skip the state machine DSL and explicit correlation config entirely.</p>
<p>If you want to go deeper on orchestrating distributed workflows and building real-world sagas,
check out <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a>.</p>
<p>Hope this was useful. See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_189.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Implementing the Inbox Pattern for Reliable Message Consumption]]></title>
            <link>https://milanjovanovic.tech/blog/implementing-the-inbox-pattern-for-reliable-message-consumption</link>
            <guid isPermaLink="false">implementing-the-inbox-pattern-for-reliable-message-consumption</guid>
            <pubDate>Sat, 04 Apr 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[The Outbox pattern guarantees reliable publishing. But what about the consumer side? The Inbox pattern ensures each incoming message is processed exactly once, even when the broker retries or delivers duplicates. Here's how to implement it in .NET with MassTransit and PostgreSQL.]]></description>
            <content:encoded><![CDATA[<p>The <a href="implementing-the-outbox-pattern"><strong>Outbox pattern</strong></a> gets a lot of attention, and rightly so.
But what about the consumer side?</p>
<p>Your publisher reliably sends a message.
The broker delivers it.
Your consumer processes it.
Then something goes wrong. A timeout, a crash, a network blip.
The broker <strong>redelivers the same message</strong>.
Your consumer runs the same logic twice.
This is a problem.</p>
<p>The <strong>Inbox pattern</strong> is the counterpart to the Outbox.
The Outbox ensures reliable <em>publishing</em>. The Inbox ensures reliable <em>consumption</em>.
Each incoming message is processed <strong>exactly once</strong>, even when the broker retries.</p>
<p>Here's how to implement it.</p>
<h2>Why You Need an Inbox</h2>
<p>Most message brokers provide <strong>at-least-once delivery</strong>.
The broker guarantees every message will be delivered,
but it <strong>doesn't</strong> guarantee each message arrives only once.</p>
<p>Here's a common failure path:</p>
<ol>
<li>The broker delivers a message to your consumer</li>
<li>Your consumer processes it successfully</li>
<li>Before the ACK reaches the broker, the connection drops</li>
<li>The broker assumes the message was lost and redelivers it</li>
<li>Your consumer processes the same message <strong>twice</strong></li>
</ol>
<p>You could make each handler <a href="the-idempotent-consumer-pattern-in-dotnet-and-why-you-need-it"><strong>idempotent</strong></a>.
That works, but it means every consumer needs to check a deduplication table before doing any work.
The Inbox centralizes this into a single mechanism at the infrastructure level.
I will talk more about the trade-offs between the Inbox and the Idempotent Consumer at the end.</p>
<p>The idea:</p>
<ol>
<li>A message arrives from the broker</li>
<li>Instead of processing it immediately, <strong>write it to an inbox table</strong></li>
<li>If the message already exists (duplicate), the write is silently ignored</li>
<li>A <a href="running-background-tasks-in-asp-net-core"><strong>background process</strong></a> reads unprocessed messages and handles them</li>
</ol>
<p>This decouples reception from processing.
The consumer becomes a thin persistence layer that can't produce duplicates.</p>
<p><img src="/blogs/mnw_188/inbox_pattern_sequence_diagram.png" alt="Inbox pattern sequence diagram showing message flow from broker to consumer to database and processor."></p>
<h2>Inbox Database Schema</h2>
<p>The <code>inbox_messages</code> table stores every incoming message:</p>
<pre><code class="language-sql">CREATE TABLE IF NOT EXISTS inbox_messages (
    id UUID PRIMARY KEY,
    type VARCHAR(255) NOT NULL,
    content JSONB NOT NULL,
    received_on_utc TIMESTAMP WITH TIME ZONE NOT NULL,
    processed_on_utc TIMESTAMP WITH TIME ZONE NULL,
    error TEXT NULL
);

CREATE INDEX IF NOT EXISTS idx_inbox_messages_unprocessed
ON public.inbox_messages (received_on_utc, processed_on_utc)
INCLUDE (id, type, content)
WHERE processed_on_utc IS NULL;
</code></pre>
<p>The structure mirrors the <a href="implementing-the-outbox-pattern"><strong>Outbox pattern's</strong></a> <code>outbox_messages</code> table.
The <code>id</code> enables idempotent inserts via <code>ON CONFLICT DO NOTHING</code>.
The filtered index keeps the index small since processed messages drop out automatically.</p>
<p>Messages between services use a shared <code>IntegrationEvent</code> base record:</p>
<pre><code class="language-csharp">public abstract record IntegrationEvent(Guid MessageId);

public sealed record OrderCreatedIntegrationEvent(Guid OrderId)
    : IntegrationEvent(Guid.CreateVersion7());
</code></pre>
<h2>Inbox Consumer</h2>
<p>The consumer is a <a href="using-masstransit-with-rabbitmq-and-azure-service-bus"><strong>MassTransit</strong></a> <code>IConsumer&lt;T&gt;</code>.
Instead of processing the message, it <strong>writes it to the inbox table</strong> and returns.
That's it.</p>
<p>We can make this generic so it works for any integration event:</p>
<pre><code class="language-csharp">internal sealed class InboxConsumer&lt;T&gt;(NpgsqlDataSource dataSource)
    : IConsumer&lt;T&gt; where T : IntegrationEvent
{
    public async Task Consume(ConsumeContext&lt;T&gt; context)
    {
        await using var connection = await dataSource.OpenConnectionAsync(
            context.CancellationToken);

        const string sql =
            @&quot;&quot;&quot;
            INSERT INTO public.inbox_messages (id, type, content, received_on_utc)
            VALUES (@Id, @Type, @Content::jsonb, @ReceivedOnUtc)
            ON CONFLICT (id) DO NOTHING;
            &quot;&quot;&quot;;

        await connection.ExecuteAsync(sql, new
        {
            Id = context.Message.MessageId,
            Type = typeof(T).FullName,
            Content = JsonSerializer.Serialize(context.Message),
            ReceivedOnUtc = DateTime.UtcNow
        });
    }
}
</code></pre>
<p><code>ON CONFLICT (id) DO NOTHING</code> is doing the heavy lifting.
If the broker delivers the same message twice, the second insert is silently ignored.
Crash after insert but before ACK? The next delivery is safely deduplicated.</p>
<h2>Inbox Processor</h2>
<p>The processor runs in a <a href="running-background-tasks-in-asp-net-core"><strong>background service</strong></a>
or a <a href="scheduling-background-jobs-with-quartz-net"><strong>scheduled job</strong></a>,
fetching unprocessed messages in batches and dispatching them for handling.</p>
<pre><code class="language-csharp">internal sealed class InboxProcessor(
    NpgsqlDataSource dataSource,
    IEventDispatcher eventDispatcher,
    ILogger&lt;InboxProcessor&gt; logger)
{
    private const int BatchSize = 1000;

    public async Task&lt;int&gt; Execute(CancellationToken cancellationToken = default)
    {
        await using var connection =
            await dataSource.OpenConnectionAsync(cancellationToken);
        await using var transaction =
            await connection.BeginTransactionAsync(cancellationToken);

        var messages = (await connection.QueryAsync&lt;InboxMessage&gt;(
            @&quot;&quot;&quot;
            SELECT id AS Id, type AS Type, content AS Content
            FROM inbox_messages
            WHERE processed_on_utc IS NULL
            ORDER BY received_on_utc
            LIMIT @BatchSize
            FOR UPDATE SKIP LOCKED
            &quot;&quot;&quot;,
            new { BatchSize },
            transaction: transaction)).AsList();

        var processedAt = DateTime.UtcNow;
        var results = new List&lt;(Guid Id, DateTime ProcessedAt, string? Error)&gt;(
            messages.Count);

        foreach (var message in messages)
        {
            try
            {
                var messageType = Type.GetType(message.Type)!;
                var deserialized = JsonSerializer.Deserialize(
                    message.Content, messageType)!;

                await eventDispatcher.DispatchAsync(deserialized, cancellationToken);

                results.Add((message.Id, processedAt, null));
            }
            catch (Exception ex)
            {
                logger.LogError(ex, &quot;Failed to process inbox message {Id}&quot;, message.Id);
                results.Add((message.Id, processedAt, ex.ToString()));
            }
        }

        if (results.Count &gt; 0)
        {
            await connection.ExecuteAsync(
                @&quot;&quot;&quot;
                UPDATE inbox_messages
                SET processed_on_utc = v.processed_on_utc,
                    error = v.error
                FROM UNNEST(@Ids, @ProcessedAts, @Errors)
                    AS v(id, processed_on_utc, error)
                WHERE inbox_messages.id = v.id
                &quot;&quot;&quot;,
                new
                {
                    Ids = results.Select(r =&gt; r.Id).ToArray(),
                    ProcessedAts = results.Select(r =&gt; r.ProcessedAt).ToArray(),
                    Errors = results.Select(r =&gt; r.Error).ToArray()
                },
                transaction: transaction);
        }

        await transaction.CommitAsync(cancellationToken);

        return messages.Count;
    }
}
</code></pre>
<ul>
<li><strong><code>FOR UPDATE SKIP LOCKED</code></strong> lets multiple processor instances run concurrently
without contention. I covered this in <a href="scaling-the-outbox-pattern"><strong>scaling the Outbox pattern</strong></a>.</li>
<li><strong>Batch update with <code>UNNEST</code></strong> writes all results in a single round-trip
using the same <a href="optimizing-bulk-database-updates-in-dotnet"><strong>bulk update approach</strong></a>.</li>
<li><strong>Error capture</strong>: failed messages get marked with the exception so they don't block the queue.</li>
</ul>
<p>If the process crashes mid-batch, the transaction rolls back
and messages get picked up on the next run.</p>
<h2>Things to Watch Out For</h2>
<p><strong>Table growth.</strong>
The inbox table grows indefinitely.
Delete processed messages after a retention period,
or partition by time range and drop old partitions.
You can also archive them to another table if you need to keep a history.</p>
<p><strong>Poison messages.</strong>
If a message consistently fails, it gets marked with an error each time.
Consider a max retry count. After N failures, dead-letter it and alert.</p>
<p><strong>Ordering.</strong>
<code>ORDER BY received_on_utc</code> gives you rough arrival-time ordering.
But with <code>SKIP LOCKED</code> and multiple processors, strict ordering is <strong>not</strong> guaranteed.
If you need <a href="solving-message-ordering-from-first-principles"><strong>per-aggregate ordering</strong></a>,
you'll need additional coordination.</p>
<p><strong>Monitoring.</strong>
Track the lag between <code>received_on_utc</code> and <code>processed_on_utc</code>.
If this gap grows, increase the batch size, decrease the polling interval,
or scale out more processor instances.</p>
<h2>Inbox vs. Idempotent Consumer</h2>
<p>Both the Inbox and the <a href="idempotent-consumer-handling-duplicate-messages"><strong>Idempotent Consumer</strong></a> prevent duplicate processing.
The difference is <em>when</em> processing happens and <em>who controls</em> retries.</p>
<p>The <a href="the-idempotent-consumer-pattern-in-dotnet-and-why-you-need-it"><strong>Idempotent Consumer</strong></a> processes messages inline.
It checks a deduplication table, does the work, and records the dedup entry in the same transaction.
If processing fails, the transaction rolls back, no dedup record is written,
and the <strong>broker</strong> redelivers the message on its own schedule.
You don't control retry timing or backoff.</p>
<p>The Inbox separates reception from processing.
The consumer writes the message and ACKs immediately. The broker is done.
If the processor fails, it records the error and moves on.
Retries are your responsibility: reset <code>processed_on_utc</code> to <code>NULL</code> for messages under a retry threshold,
or run a separate loop that picks up failed messages after a delay.</p>
<p>Use the <strong>Idempotent Consumer</strong> when your side effects are transactional
and broker-managed retries are good enough.
Use the <strong>Inbox</strong> when you need batching, custom retry policies,
or horizontal scaling via <code>FOR UPDATE SKIP LOCKED</code>.</p>
<h2>Summary</h2>
<p>The Inbox pattern is the consumer-side counterpart to the <a href="implementing-the-outbox-pattern"><strong>Outbox pattern</strong></a>.</p>
<ul>
<li><strong><code>ON CONFLICT DO NOTHING</code></strong> makes consumer inserts idempotent</li>
<li><strong>Separation of reception and processing</strong> gives you independent retry control</li>
<li><strong><code>FOR UPDATE SKIP LOCKED</code></strong> enables horizontal scaling of the processor</li>
<li><strong>Batch updates with <code>UNNEST</code></strong> minimize database round-trips</li>
</ul>
<p>If you want to see how I build <a href="event-driven-architecture-in-dotnet-with-rabbitmq"><strong>event-driven systems</strong></a> with these patterns,
check out <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a>.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_188.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Getting Started With PgVector in .NET for Simple Vector Search]]></title>
            <link>https://milanjovanovic.tech/blog/getting-started-with-pgvector-in-dotnet-for-simple-vector-search</link>
            <guid isPermaLink="false">getting-started-with-pgvector-in-dotnet-for-simple-vector-search</guid>
            <pubDate>Sat, 28 Mar 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[Vector search doesn't require a dedicated vector database. PostgreSQL with pgvector gives you similarity search right next to your relational data. Here's how to set it up in .NET with Aspire, Dapper, and Ollama.]]></description>
            <content:encoded><![CDATA[<p>Not every AI feature needs a dedicated vector database.</p>
<p>Dedicated vector databases like <a href="https://www.pinecone.io/">Pinecone</a>, <a href="https://qdrant.tech/">Qdrant</a>, and <a href="https://weaviate.io/">Weaviate</a> get all the attention.
But if your data already lives in PostgreSQL, you don't need another moving part.</p>
<p><a href="https://github.com/pgvector/pgvector">pgvector</a> is a PostgreSQL extension that adds vector storage and similarity search directly to your existing database.
You enable the extension, create a vector column, and start querying.</p>
<p>In this week's issue, I'll walk you through:</p>
<ul>
<li>What <a href="what-is-vector-search-a-concise-guide"><strong>vector search</strong></a> is and when you need it</li>
<li>Provisioning pgvector with .NET Aspire and Ollama</li>
<li>Generating embeddings with <a href="working-with-llms-in-dotnet-using-microsoft-extensions-ai"><strong>MEAI</strong></a> and storing with Dapper</li>
<li>Querying by semantic similarity using cosine distance</li>
</ul>
<p>Let's dive in.</p>
<h2>What Is Vector Search?</h2>
<p>Traditional database queries work with exact matches.
You search for <code>&quot;authentication&quot;</code> and get rows containing that exact word.
But what about rows that mention <code>&quot;login&quot;</code>, <code>&quot;sign-in&quot;</code>, or <code>&quot;identity verification&quot;</code>?
Those are semantically similar, but a <code>LIKE</code> query won't find them.</p>
<p><a href="what-is-vector-search-a-concise-guide"><strong>Vector search</strong></a> solves this by comparing <strong>meaning</strong> instead of text.</p>
<p>You convert text into an array of numbers (called an <strong>embedding</strong>) using a machine learning model.
Semantically similar text produces similar arrays.
Then, instead of matching keywords, you find the vectors that are <strong>closest</strong> to your query vector.</p>
<p>When would you use this?</p>
<ul>
<li><strong>Semantic search</strong> - Find results by meaning, not just keywords</li>
<li><strong>RAG (Retrieval-Augmented Generation)</strong> - Feed relevant context to an LLM</li>
<li><strong>Recommendations</strong> - &quot;Users who liked X also liked Y&quot;</li>
<li><strong>Deduplication</strong> - Find near-duplicate content</li>
</ul>
<p>The key insight is that you don't need a specialized database for this.
If you're already on PostgreSQL, pgvector gives you all of this as an extension.</p>
<h2>Provisioning Infrastructure With .NET Aspire</h2>
<p>We'll use <a href="dotnet-aspire-a-game-changer-for-cloud-native-development"><strong>.NET Aspire</strong></a>
to provision a pgvector-enabled PostgreSQL container and an <a href="how-to-extract-structured-data-from-images-using-ollama-in-dotnet"><strong>Ollama</strong></a>
instance with the <a href="https://ollama.com/library/qwen3-embedding">qwen3-embedding</a> embedding model.</p>
<pre><code class="language-csharp">var builder = DistributedApplication.CreateBuilder(args);

var ollama = builder.AddOllama(&quot;ollama&quot;)
    .WithLifetime(ContainerLifetime.Persistent)
    .WithDataVolume()
    .WithGPUSupport();

var embeddingModel = ollama.AddModel(&quot;qwen3-embedding:0.6b&quot;);

var postgres = builder.AddPostgres(&quot;postgres&quot;, port: 6432)
    .WithLifetime(ContainerLifetime.Persistent)
    .WithDataVolume()
    .WithImage(&quot;pgvector/pgvector&quot;, &quot;pg17&quot;)
    .AddDatabase(&quot;articles&quot;);

builder.AddProject&lt;Projects.PgVector_Articles&gt;(&quot;pgvector-articles&quot;)
    .WithReference(embeddingModel)
    .WithReference(postgres)
    .WaitFor(embeddingModel)
    .WaitFor(postgres);

builder.Build().Run();
</code></pre>
<p>The <code>pgvector/pgvector:pg17</code> image is PostgreSQL 17 with the pgvector extension pre-installed.
<code>WithLifetime(ContainerLifetime.Persistent)</code> keeps the containers running across app restarts so you don't lose data during development.
<code>WaitFor</code> ensures the database and model are ready before the API starts.</p>
<p>If you're not using Aspire, you can run the same <code>pgvector/pgvector:pg17</code> image via <code>docker compose</code> and point to it with a regular connection string.</p>
<h2>Configuring the API Project</h2>
<p>The API project needs a few packages:</p>
<pre><code class="language-bash">dotnet add package Aspire.Npgsql
dotnet add package Pgvector.Dapper
dotnet add package CommunityToolkit.Aspire.OllamaSharp
</code></pre>
<p><code>Pgvector.Dapper</code> provides the Dapper type handler for the <code>Vector</code> type.
Other than <a href="https://github.com/pgvector/pgvector-dotnet/tree/master/src/Pgvector.Dapper">Pgvector.Dapper</a>, there are also libraries for <a href="https://github.com/pgvector/pgvector-dotnet/tree/master/src/Pgvector">Npgsql</a> and <a href="https://github.com/pgvector/pgvector-dotnet/tree/master/src/Pgvector.EntityFrameworkCore">EF Core</a> if you prefer those instead.</p>
<p>Register the services in <code>Program.cs</code>:</p>
<pre><code class="language-csharp">builder.AddOllamaApiClient(&quot;ollama-qwen3-embedding&quot;)
    .AddEmbeddingGenerator();

builder.AddNpgsqlDataSource(&quot;articles&quot;, configureDataSourceBuilder: b =&gt;
{
    b.UseVector();
});

SqlMapper.AddTypeHandler(new VectorTypeHandler());
</code></pre>
<p><code>AddEmbeddingGenerator()</code> registers an <code>IEmbeddingGenerator&lt;string, Embedding&lt;float&gt;&gt;</code> using the <a href="working-with-llms-in-dotnet-using-microsoft-extensions-ai"><code>Microsoft.Extensions.AI</code></a> abstraction.
<code>UseVector()</code> enables pgvector type mapping on the Npgsql data source.
The <code>VectorTypeHandler</code> lets Dapper serialize and deserialize <code>Vector</code> parameters.</p>
<h2>Initializing the Database</h2>
<p>Before storing vectors, we need to enable the pgvector extension and create a table.</p>
<pre><code class="language-csharp">app.MapPost(&quot;/init&quot;, async (NpgsqlDataSource dataSource) =&gt;
{
    await using var conn = await dataSource.OpenConnectionAsync();

    await using var enableExt = new NpgsqlCommand(
        &quot;CREATE EXTENSION IF NOT EXISTS vector&quot;, conn);
    await enableExt.ExecuteNonQueryAsync();

    conn.ReloadTypes();

    await conn.ExecuteAsync(
        &quot;&quot;&quot;
        CREATE TABLE IF NOT EXISTS articles (
            id SERIAL PRIMARY KEY,
            url TEXT NOT NULL,
            title TEXT NOT NULL,
            embedding vector(1024) NOT NULL
        )
        &quot;&quot;&quot;);

    await conn.ExecuteAsync(
        &quot;&quot;&quot;
        CREATE INDEX IF NOT EXISTS articles_embedding_idx
        ON articles USING hnsw (embedding vector_cosine_ops)
        &quot;&quot;&quot;);

    return Results.Ok(&quot;Database initialized.&quot;);
});
</code></pre>
<p>A few things to note:</p>
<ul>
<li><code>CREATE EXTENSION IF NOT EXISTS vector</code> enables pgvector in the database</li>
<li><code>embedding vector(1024)</code> defines a vector column with 1024 dimensions, matching the Ollama embedding model's output (<code>qwen3-embedding:0.6b</code>)</li>
<li><code>conn.ReloadTypes()</code> refreshes Npgsql's type cache so it recognizes the new <code>vector</code> type</li>
<li>The <strong>HNSW index</strong> with <code>vector_cosine_ops</code> enables fast approximate nearest-neighbor search using cosine distance.</li>
</ul>
<p><a href="https://en.wikipedia.org/wiki/Hierarchical_navigable_small_world">HNSW</a> (Hierarchical Navigable Small World) is a graph-based algorithm that builds a multi-layer structure for efficient similarity lookups.</p>
<p>Without the index, pgvector does a sequential scan over every row.
That's fine for hundreds of rows, but HNSW keeps queries fast as the dataset grows.</p>
<h2>Generating and Storing Embeddings</h2>
<p>Now we generate embeddings for our content and store them alongside the data.</p>
<pre><code class="language-csharp">app.MapPost(&quot;/embeddings/generate&quot;, async (
    BlogService blogService,
    IEmbeddingGenerator&lt;string, Embedding&lt;float&gt;&gt; embeddingGenerator,
    NpgsqlDataSource dataSource,
    ILogger&lt;Program&gt; logger) =&gt;
{
    await using var conn = await dataSource.OpenConnectionAsync();
    conn.ReloadTypes();

    int count = 0;

    foreach (var articleUrl in File.ReadAllLines(&quot;sitemap_urls.txt&quot;))
    {
        var (title, content) = await blogService.GetTitleAndContentAsync(articleUrl);

        var embedding = await embeddingGenerator.GenerateAsync(content);

        await conn.ExecuteAsync(
            &quot;INSERT INTO articles (url, title, embedding) VALUES (@url, @title, @embedding)&quot;,
            new
            {
                url = articleUrl,
                title,
                embedding = new Vector(embedding.Vector.ToArray())
            });

        count++;
        logger.LogInformation(&quot;Processed ({Count}): {Url}&quot;, count, articleUrl);
    }

    return Results.Ok(new { processed = count });
});
</code></pre>
<p><code>embeddingGenerator.GenerateAsync(content)</code> sends the text to the Ollama model and returns a vector.
We wrap it in a <code>Pgvector.Vector</code> and Dapper handles the rest.</p>
<p>The <code>IEmbeddingGenerator</code> is provider-agnostic.
If you want to swap Ollama for OpenAI or Azure OpenAI later, only the registration in <code>Program.cs</code> changes.
Your endpoint code stays the same.</p>
<h2>Similarity Search With Cosine Distance</h2>
<p>This is where it gets interesting.
To search, we embed the query text and find the closest vectors in the database.</p>
<pre><code class="language-csharp">app.MapGet(&quot;/search&quot;, async (
    string query,
    IEmbeddingGenerator&lt;string, Embedding&lt;float&gt;&gt; embeddingGenerator,
    NpgsqlDataSource dataSource,
    int limit = 5) =&gt;
{
    var searchEmbedding = await embeddingGenerator.GenerateAsync(query);

    await using var con = await dataSource.OpenConnectionAsync();
    con.ReloadTypes();

    var embedding = new Vector(searchEmbedding.Vector.ToArray());

    var results = await con.QueryAsync&lt;SearchResult&gt;(
        @&quot;&quot;&quot;
        SELECT title, url, embedding &lt;=&gt; @embedding as distance
        FROM articles
        ORDER BY embedding &lt;=&gt; @embedding
        LIMIT @limit
        &quot;&quot;&quot;,
        new { embedding, limit });

    return Results.Ok(new { query, results });
});

record SearchResult(string Title, string Url, double Distance);
</code></pre>
<p>The <code>&lt;=&gt;</code> operator is pgvector's <strong>cosine distance</strong> function, where lower values mean more similar.
We order by distance ascending and take the top N matches.</p>
<p>The critical part: the query text must be embedded with the <strong>same model</strong> that produced the stored embeddings.
Different models produce vectors in different embedding spaces, and comparing them would be meaningless.</p>
<p>There are also shared embedding spaces models that can embed text and images into compatible vectors, but that's a more advanced topic.
One example is the <a href="https://blog.voyageai.com/2026/01/15/voyage-4/">Voyage 4 model family</a>.</p>
<p>A query like <code>&quot;how to secure an API&quot;</code> will surface articles about authentication, JWT validation, and authorization, even if they don't contain those exact words.</p>
<p>pgvector supports three distance operators:</p>
<ul>
<li><code>&lt;-&gt;</code> - L2 (Euclidean) distance, uses <code>vector_l2_ops</code></li>
<li><code>&lt;=&gt;</code> - Cosine distance, uses <code>vector_cosine_ops</code></li>
<li><code>&lt;#&gt;</code> - Inner product (negative), uses <code>vector_ip_ops</code></li>
</ul>
<p>Cosine distance is the most common choice for text embeddings.</p>
<h2>Summary</h2>
<p>You don't need a dedicated vector database to add semantic search to your application.
If you're already running PostgreSQL, pgvector gives you vector storage and similarity search without adding new infrastructure.</p>
<p>Here's what we covered:</p>
<ul>
<li><strong>pgvector</strong> is a PostgreSQL extension - enable it and you get a native <code>vector</code> column type</li>
<li><strong>.NET Aspire</strong> provisions pgvector-enabled PostgreSQL and Ollama with minimal configuration</li>
<li><strong>Embeddings</strong> turn text into vectors using models like <code>qwen3-embedding</code> via <code>IEmbeddingGenerator</code></li>
<li><strong>Similarity search</strong> uses the cosine distance operator (<code>&lt;=&gt;</code>) to find the closest matches</li>
<li><strong>HNSW indexes</strong> make vector queries fast at scale</li>
</ul>
<p>Your vectors live right next to your relational data, so you can join, filter, and paginate just like any other query without syncing between databases or managing extra infrastructure.</p>
<p>If you want to explore more advanced scenarios like <a href="building-semantic-search-with-amazon-s3-vectors-and-semantic-kernel"><strong>building semantic search with S3 Vectors and Semantic Kernel</strong></a>, I've covered that in a previous article.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_187.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Scaling SignalR With a Redis Backplane]]></title>
            <link>https://milanjovanovic.tech/blog/scaling-signalr-with-redis-backplane</link>
            <guid isPermaLink="false">scaling-signalr-with-redis-backplane</guid>
            <pubDate>Sat, 21 Mar 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[SignalR connections are server-local. Scale out to multiple instances and messages stop reaching the right clients. Here's how the Redis backplane fixes that - and what you still need to get right.]]></description>
            <content:encoded><![CDATA[<p>I ran into this one the hard way.</p>
<p>I built a <a href="adding-real-time-functionality-to-dotnet-applications-with-signalr"><strong>real-time notification feature with SignalR</strong></a>, tested it locally, everything worked great.
Then I scaled to two instances behind a load balancer, and notifications started disappearing for some users.</p>
<p>The code was fine.
The problem was that <strong>SignalR connections are bound to the server process that accepted them</strong>.
Each instance only knows about its own connections.
So when an API request lands on Server 1 but the user is connected to Server 2, the notification just... doesn't get delivered.</p>
<p>This is the SignalR scale-out problem, and it bites almost everyone who goes from one instance to more.</p>
<h2>Why SignalR Breaks When You Scale Out</h2>
<p>With a single instance, everything just works.</p>
<p><img src="/blogs/mnw_186/signalr_single_server.png" alt="SignalR single server deployment showing all clients connected to the same server."></p>
<p>The server holds the full map of who's connected, so sending a message to a user, a group, or all clients works because that map is complete.</p>
<p>But scale out to two or more instances, and that map fractures.</p>
<p><img src="/blogs/mnw_186/signalr_multi_server.png" alt="SignalR multi-server deployment showing clients connected to different servers."></p>
<p>Server 1 has no idea Client 3 or Client 4 even exist.
So when an order status change happens on Server 1 and needs to reach Client 3, Server 1 checks its connection map, finds nothing, and the message is quietly dropped.</p>
<h2>The Backplane Pattern</h2>
<p>The fix is a <strong>backplane</strong> - a shared messaging layer that sits between all your server instances.</p>
<p>Every server publishes outgoing messages to a central channel, and every server subscribes to that same channel.
When a message comes in, each server checks if any of its local connections should receive it.</p>
<p><img src="/blogs/mnw_186/signalr_backplane.png" alt="SignalR backplane deployment showing clients connected to different servers."></p>
<p>When Server 1 wants to notify Client 3:</p>
<ol>
<li>Server 1 publishes the message to the backplane</li>
<li>All servers receive the message</li>
<li>Server 2 recognizes Client 3 as one of its connections and delivers the notification</li>
</ol>
<p>From your code's perspective, it looks like every server can see every connection.
Redis works really well for this because its <a href="simple-messaging-in-dotnet-with-redis-pubsub"><strong>Pub/Sub</strong></a> delivers messages to all subscribers in near real-time.
And if you're already using Redis for <a href="caching-in-aspnetcore-improving-application-performance"><strong>distributed caching</strong></a>, you don't even need to spin up anything new.</p>
<h2>Setting It Up</h2>
<p>Let me walk through how I set this up in an order notification system.
Clients connect to a SignalR hub, authenticate via JWT, and get real-time status updates when an order changes.</p>
<h3>Install the NuGet Package</h3>
<pre><code class="language-bash">dotnet add package Microsoft.AspNetCore.SignalR.StackExchangeRedis
</code></pre>
<h3>Register the Backplane</h3>
<p>Chain <code>.AddStackExchangeRedis()</code> onto your <code>AddSignalR()</code> call:</p>
<pre><code class="language-csharp">builder.Services.AddSignalR()
    .AddStackExchangeRedis(builder.Configuration.GetConnectionString(&quot;cache&quot;)!);
</code></pre>
<p>If you're running with <a href="dotnet-aspire-a-game-changer-for-cloud-native-development"><strong>.NET Aspire</strong></a>,
you will have the Redis connection string registered via environment variables, so you can just pull it from configuration.
Reference the same named connection:</p>
<pre><code class="language-csharp">builder.AddRedisDistributedCache(&quot;cache&quot;);

builder.Services.AddSignalR()
    .AddStackExchangeRedis(builder.Configuration.GetConnectionString(&quot;cache&quot;)!);
</code></pre>
<p>If you have multiple SignalR apps sharing the same Redis instance, you'll want to add a channel prefix.
Otherwise, messages from one app will reach subscribers in every app on that Redis server.</p>
<pre><code class="language-csharp">builder.Services.AddSignalR()
    .AddStackExchangeRedis(connectionString, options =&gt;
    {
        options.Configuration.ChannelPrefix = RedisChannel.Literal(&quot;OrderNotifications&quot;);
    });
</code></pre>
<p>The nice thing is that your <code>IHubContext&lt;&gt;</code> call site doesn't change at all.
<code>Clients.User(...)</code> works the same whether you have one instance or ten - the backplane handles routing behind the scenes.</p>
<p>When I tested this with two replicas in <a href="dotnet-aspire-a-game-changer-for-cloud-native-development"><strong>.NET Aspire</strong></a>, I tagged each notification with the sending instance's ID.
A client on Replica 1 received a notification stamped with Replica 2's ID, which confirmed the message was crossing instances through Redis.</p>
<h2>The Sticky Sessions Requirement</h2>
<p>This is something you should know before you set up the backplane: <strong>you still need sticky sessions</strong>.</p>
<p>The Redis backplane solves message <em>routing</em>, but it does <strong>not</strong> remove the need for sticky sessions.</p>
<p>SignalR's connection negotiation is a two-step process:</p>
<ol>
<li>The client sends a <code>POST</code> to <code>/hub/negotiate</code> to obtain a connection token</li>
<li>The client uses that token to establish the WebSocket connection</li>
</ol>
<p>Both requests must land on the <strong>same server</strong>.
If your load balancer routes the negotiation to Server 1 but the WebSocket upgrade to Server 2, the connection fails.</p>
<p><img src="/blogs/mnw_186/sticky_sessions.png" alt="SignalR connection negotiation showing the need for sticky sessions."></p>
<p><strong>Make sure sticky sessions are enabled in your load balancer.</strong>
Most load balancers support this via IP hash or cookie affinity - check the docs for whichever you're using.</p>
<h2>What Happens When Redis Goes Down?</h2>
<p>One thing worth knowing: <strong>SignalR does not buffer messages when Redis is unavailable</strong>.</p>
<p>If Redis stops responding, any messages sent during the outage are simply lost.
SignalR may throw exceptions, but your existing WebSocket connections stay open - clients don't get disconnected.
Once Redis comes back, SignalR reconnects automatically.</p>
<p>For most real-time scenarios like order updates or live dashboards, this is fine.
The next state change triggers a fresh notification anyway, or the user can just reload.
If you're dealing with something more critical (financial data, operational alerts), you'll want a reconciliation strategy on reconnect or a durable queue running alongside.</p>
<h2>Redis Backplane vs. Azure SignalR Service</h2>
<p>If you're on Azure, the managed <a href="https://learn.microsoft.com/en-us/azure/azure-signalr/signalr-overview">Azure SignalR Service</a> is worth considering.
It proxies all client connections through the service, so sticky sessions aren't required and your servers only hold a small number of constant connections to the service.</p>
<p>The Redis backplane is the better fit when you're self-hosted, latency-sensitive, or already running Redis.
For everything else, Azure SignalR Service is the cleaner option.</p>
<h2>Summary</h2>
<p>Honestly, the Redis backplane is almost <em>too</em> simple to set up.
One method call on <code>AddSignalR()</code> and your app goes from silently dropping messages to routing them across every instance.
You don't need to make changes to your hub code, client code, or application logic.</p>
<p>Just remember two things:</p>
<ul>
<li><strong>You still need sticky sessions</strong></li>
<li><strong>Messages aren't buffered</strong> if Redis goes down temporarily</li>
</ul>
<p>Get those two right and SignalR scales out just as smoothly as the rest of your stack.</p>
<p>If you want to go deeper on building real-time features and APIs in .NET,
check out my <a href="/pragmatic-rest-apis"><strong>Pragmatic REST APIs</strong></a> course.</p>
<p>Hope this was useful. See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_186.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Optimizing Bulk Database Updates in .NET: From Naive to Lightning-Fast]]></title>
            <link>https://milanjovanovic.tech/blog/optimizing-bulk-database-updates-in-dotnet</link>
            <guid isPermaLink="false">optimizing-bulk-database-updates-in-dotnet</guid>
            <pubDate>Sat, 14 Mar 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[Seven approaches to bulk-updating rows in PostgreSQL from .NET using Dapper and EF Core, from naive per-row updates to binary COPY. Each step cuts down on round-trips or removes dynamic SQL entirely.]]></description>
            <content:encoded><![CDATA[<p>Every outbox processor, job queue, and batch pipeline hits the same problem at some point:
mark a set of rows as done in bulk.</p>
<p>I went through this recently while optimizing an outbox processor.
I measured seven different approaches against 1,000, 10,000, and 25,000 rows in PostgreSQL.
At 10,000 rows the slowest approach took 2,414ms and the fastest took 41ms.
The bottleneck was almost never the SQL.
It was how many times the code was talking to the database.</p>
<p>The scenario: mark a batch of orders as processed by setting a status and a <strong>unique</strong> <code>processed_at</code> timestamp per row.
That uniqueness constraint is what makes it tricky.
You can't use a simple <code>UPDATE ... WHERE id IN (...)</code> with a single value because every row gets a different timestamp.</p>
<p>Let's go through each approach.</p>
<h2>The Scenario</h2>
<p>A table of orders, each needing two updates: a <code>status</code> change to <code>&quot;Processed&quot;</code> and a unique <code>processed_at</code> timestamp.</p>
<pre><code class="language-sql">CREATE TABLE orders (
    id            UUID         NOT NULL PRIMARY KEY,
    customer_name TEXT         NOT NULL,
    status        TEXT         NOT NULL DEFAULT 'Pending',
    processed_at  TIMESTAMPTZ
);
</code></pre>
<p>The update payload is a simple record pairing each order with its timestamp:</p>
<pre><code class="language-csharp">record OrderUpdate(Guid Id, DateTime ProcessedAt);
</code></pre>
<p>With 10,000 of these, here's how each approach plays out.</p>
<h2>Approach 1: Naive Dapper, One UPDATE Per Row</h2>
<p>The first thing you'd try: loop through the list and fire one <code>UPDATE</code> per row.</p>
<pre><code class="language-csharp">await using var connection = new NpgsqlConnection(connectionString);
await connection.OpenAsync();
await using var transaction = await connection.BeginTransactionAsync();

foreach (var update in updates)
{
    await connection.ExecuteAsync(
        &quot;&quot;&quot;
        UPDATE orders
        SET processed_at = @ProcessedAt,
            status       = 'Processed'
        WHERE id = @Id
        &quot;&quot;&quot;,
        new { update.Id, update.ProcessedAt },
        transaction: transaction);
}

await transaction.CommitAsync();
</code></pre>
<p>10,000 rows means 10,000 round-trips to the database.
Each <code>ExecuteAsync</code> call sends the SQL, waits for PostgreSQL to respond, then moves on to the next one.
At 10,000 rows this took <strong>2,414ms</strong> in my benchmark. At 25,000 rows it was over 6 seconds.
Over a real network it would be worse.
The database is not slow. The constant back-and-forth is.</p>
<h2>Approach 2: EF Core SaveChanges, Batched Round-Trips</h2>
<p>EF Core batches the generated SQL statements to cut down on round-trips.
With <code>MaxBatchSize</code> set high enough, you can push all 10,000 updates in far fewer network calls.
The default batch size is 42, in case you were wondering.
You can read more about this in the <a href="https://learn.microsoft.com/en-us/ef/core/performance/efficient-updating">EF Core efficient updating docs</a>.</p>
<pre><code class="language-csharp">var options = new DbContextOptionsBuilder&lt;AppDbContext&gt;()
    .UseNpgsql(connectionString, o =&gt; o.MinBatchSize(5000).MaxBatchSize(10000))
    .Options;

await using var db = new AppDbContext(options);

var ids = updates.Select(u =&gt; u.Id).ToHashSet();
var orders = await db.Orders
    .Where(o =&gt; ids.Contains(o.Id))
    .ToListAsync();

var updateMap = updates.ToDictionary(u =&gt; u.Id);

foreach (var order in orders)
{
    order.Status = &quot;Processed&quot;;
    order.ProcessedAt = updateMap[order.Id].ProcessedAt;
}

await db.SaveChangesAsync();
</code></pre>
<p>It's a meaningful improvement over the loop.
At 10,000 rows it came in at <strong>1,030ms</strong>, roughly half the time of Approach 1.
But there's hidden cost in two places: the upfront <code>SELECT</code> to load all 10,000 entities into the change tracker,
and the fact that EF Core still generates 10,000 individual <code>UPDATE</code> statements.
They're packed into fewer round-trips, but the SQL is all still there.</p>
<h2>Approach 3: Dapper with a VALUES Table, One Statement, One Round-Trip</h2>
<p>Instead of sending N separate statements, you can send a single <code>UPDATE</code> that supplies all the new values inline
using a derived <code>VALUES</code> table:</p>
<pre><code class="language-sql">UPDATE orders
SET processed_at = v.processed_at,
    status       = 'Processed'
FROM (VALUES
    (@Id0, @ProcessedAt0),
    (@Id1, @ProcessedAt1),
    ...
) AS v(id, processed_at)
WHERE orders.id = v.id::uuid
</code></pre>
<p>Here's the C# code to build and execute that:</p>
<pre><code class="language-csharp">await using var connection = new NpgsqlConnection(connectionString);
await connection.OpenAsync();
await using var transaction = await connection.BeginTransactionAsync();

const string updateTemplate =
    @&quot;&quot;&quot;
    UPDATE orders
    SET processed_at = v.processed_at,
        status       = 'Processed'
    FROM (VALUES
        {0}
    ) AS v(id, processed_at)
    WHERE orders.id = v.id::uuid
    &quot;&quot;&quot;;

var paramNames = string.Join(
    &quot;,\n    &quot;,
    updates.Select((_, i) =&gt; $&quot;(@Id{i}, @ProcessedAt{i})&quot;));

var sql = string.Format(updateTemplate, paramNames);

var parameters = new DynamicParameters();
for (int i = 0; i &lt; updates.Count; i++)
{
    parameters.Add($&quot;Id{i}&quot;, updates[i].Id.ToString());
    parameters.Add($&quot;ProcessedAt{i}&quot;, updates[i].ProcessedAt);
}

await connection.ExecuteAsync(sql, parameters, transaction: transaction);

await transaction.CommitAsync();
</code></pre>
<p>PostgreSQL receives one statement, builds one execution plan, and updates all rows in a single pass.
One round-trip total.
This dropped from 2,414ms down to <strong>89ms</strong> at 10,000 rows.</p>
<p>There is a trade-off: the SQL string grows with the batch size.
At 10,000 rows you end up with 10,000 parameter pairs in the query text.
PostgreSQL allows a <strong>maximum of 65,535 parameters</strong>, so you have headroom,
but it is something to be aware of at very large batch sizes.
Approaches 6 and 7 avoid this entirely.</p>
<h2>Approach 4: EF Core ExecuteSqlRaw, Same SQL Inside EF Core</h2>
<p>The SQL here is exactly the same as Approach 3.
The reason to use this instead is if you're already working inside an EF Core <code>DbContext</code> and want your bulk update to share the same transaction as other EF Core operations.</p>
<pre><code class="language-csharp">await using var db = new AppDbContext(options);
await using var transaction = await db.Database.BeginTransactionAsync();

var paramEntries = new List&lt;NpgsqlParameter&gt;();
var valueClauses = new List&lt;string&gt;();

for (int i = 0; i &lt; updates.Count; i++)
{
    valueClauses.Add($&quot;(@Id{i}::uuid, @ProcessedAt{i})&quot;);
    paramEntries.Add(new NpgsqlParameter($&quot;Id{i}&quot;, updates[i].Id.ToString()));
    paramEntries.Add(new NpgsqlParameter($&quot;ProcessedAt{i}&quot;, updates[i].ProcessedAt));
}

var sql = string.Format(updateTemplate, string.Join(&quot;,\n    &quot;, valueClauses));

await db.Database.ExecuteSqlRawAsync(sql, paramEntries);

await transaction.CommitAsync();
</code></pre>
<p>The performance is the same as Approach 3 since it's the same SQL hitting the database.
In my benchmark it came in at <strong>166ms</strong> at 10,000 rows, a bit slower than Approach 3's 89ms.
The small gap is likely overhead from the EF Core transaction wrapper rather than the SQL itself.
What you get is the ability to mix change-tracked operations with raw SQL inside the same EF Core transaction.
Dapper and EF Core are not mutually exclusive.</p>
<p>One thing to watch out for: <code>ExecuteSqlRawAsync</code> doesn't accept Dapper's <code>DynamicParameters</code>.
You have to use <a href="https://www.npgsql.org/doc/api/Npgsql.NpgsqlParameter.html"><code>NpgsqlParameter</code></a> objects directly.
If you want a full overview of EF Core's raw SQL options,
I covered them in <a href="ef-core-raw-sql-queries"><strong>this article</strong></a>.</p>
<h2>Approach 5: Dapper CTE (WITH ... AS VALUES)</h2>
<p>This is a variation on Approach 3 that wraps the same <code>VALUES</code> data in a named CTE instead of an inline derived table:</p>
<pre><code class="language-sql">WITH updates(id, processed_at) AS (
    VALUES
        (@Id0::uuid, @ProcessedAt0),
        (@Id1::uuid, @ProcessedAt1),
        ...
)
UPDATE orders
SET processed_at = updates.processed_at,
    status       = 'Processed'
FROM updates
WHERE orders.id = updates.id
</code></pre>
<p>And here's the C# code to execute it:</p>
<pre><code class="language-csharp">await using var connection = new NpgsqlConnection(connectionString);
await connection.OpenAsync();
await using var transaction = await connection.BeginTransactionAsync();

var valueClauses = string.Join(
    &quot;,\n        &quot;,
    updates.Select((_, i) =&gt; $&quot;(@Id{i}::uuid, @ProcessedAt{i})&quot;));

var sql =
    @$&quot;&quot;&quot;
    WITH updates(id, processed_at) AS (
        VALUES
            {valueClauses}
    )
    UPDATE orders
    SET processed_at = updates.processed_at,
        status       = 'Processed'
    FROM updates
    WHERE orders.id = updates.id
    &quot;&quot;&quot;;

var parameters = new DynamicParameters();
for (int i = 0; i &lt; updates.Count; i++)
{
    parameters.Add($&quot;Id{i}&quot;, updates[i].Id.ToString());
    parameters.Add($&quot;ProcessedAt{i}&quot;, updates[i].ProcessedAt);
}

await connection.ExecuteAsync(sql, parameters, transaction: transaction);

await transaction.CommitAsync();
</code></pre>
<p>Still a single statement and a single round-trip.
PostgreSQL materializes the CTE once and joins against it for the update.
In the benchmark it came in at <strong>103ms</strong> at 10,000 rows, slightly behind the plain <code>VALUES</code> approach at 89ms.
The performance difference is small enough that this is really a style choice.
Some teams prefer the CTE form when the update logic is more involved and they want to name the data source explicitly.</p>
<h2>Approach 6: Dapper with UNNEST (PostgreSQL)</h2>
<p>If you're on PostgreSQL, there's a cleaner option: <a href="https://www.postgresql.org/docs/current/functions-array.html"><code>UNNEST</code></a>.
Instead of building <code>@Id0</code> through <code>@Id9999</code> dynamically, you pass two arrays as parameters and let PostgreSQL expand them:</p>
<pre><code class="language-csharp">await using var connection = new NpgsqlConnection(connectionString);
await connection.OpenAsync();
await using var transaction = await connection.BeginTransactionAsync();

var ids = updates.Select(u =&gt; u.Id).ToArray();
var processedAts = updates.Select(u =&gt; u.ProcessedAt).ToArray();

await connection.ExecuteAsync(
    @&quot;&quot;&quot;
    UPDATE orders
    SET processed_at = v.processed_at,
        status       = 'Processed'
    FROM UNNEST(@Ids, @ProcessedAts) AS v(id, processed_at)
    WHERE orders.id = v.id
    &quot;&quot;&quot;,
    new { Ids = ids, ProcessedAts = processedAts },
    transaction: transaction);

await transaction.CommitAsync();
</code></pre>
<p>This is my preferred approach when working with PostgreSQL:</p>
<ul>
<li><strong>No dynamic SQL.</strong> The query text is always the same. No <code>string.Format</code>, no growing parameter lists.</li>
<li><strong>Two parameters total.</strong> <code>@Ids</code> and <code>@ProcessedAts</code>.
Npgsql maps <code>Guid[]</code> and <code>DateTime[]</code> directly to PostgreSQL array types.</li>
<li><strong>Stable query plans.</strong> The SQL never changes,
so PostgreSQL can cache and reuse the execution plan regardless of how many rows you're updating.</li>
<li><strong>Scales cleanly.</strong> The query text stays constant for any batch size.
Only the array data grows, and it's sent as compact binary.</li>
</ul>
<p>The catch: <code>UNNEST</code> is PostgreSQL-specific.
If you're targeting multiple databases, stick with the <code>VALUES</code> approach from Approach 3, 4, or 5.
Or you can research the equivalent array expansion functions in your other database of choice.</p>
<h2>Approach 7: Temp Table + Binary COPY</h2>
<p>All the previous approaches pass data as SQL parameters.
At extreme batch sizes that starts to become a constraint, both from the 65,535 parameter limit and from the overhead of building large parameter lists.</p>
<p>A completely different path: skip parameters altogether.
Create a temporary staging table, bulk-load the data using <a href="https://www.npgsql.org/doc/copy.html">Npgsql's binary COPY</a>, then run a single <code>UPDATE ... FROM</code>.</p>
<pre><code class="language-csharp">await using var connection = new NpgsqlConnection(connectionString);
await connection.OpenAsync();
await using var transaction = await connection.BeginTransactionAsync();

await connection.ExecuteAsync(
    @&quot;&quot;&quot;
    CREATE TEMP TABLE temp_updates (
        id           UUID        NOT NULL,
        processed_at TIMESTAMPTZ NOT NULL
    ) ON COMMIT DROP
    &quot;&quot;&quot;,
    transaction: transaction);

await using (var writer = await connection.BeginBinaryImportAsync(
    &quot;COPY temp_updates (id, processed_at) FROM STDIN (FORMAT BINARY)&quot;))
{
    foreach (var u in updates)
    {
        await writer.StartRowAsync();
        await writer.WriteAsync(u.Id, NpgsqlTypes.NpgsqlDbType.Uuid);
        await writer.WriteAsync(u.ProcessedAt, NpgsqlTypes.NpgsqlDbType.TimestampTz);
    }
    await writer.CompleteAsync();
}

await connection.ExecuteAsync(
    @&quot;&quot;&quot;
    UPDATE orders
    SET processed_at = t.processed_at,
        status       = 'Processed'
    FROM temp_updates t
    WHERE orders.id = t.id
    &quot;&quot;&quot;,
    transaction: transaction);

await transaction.CommitAsync();
</code></pre>
<p>Binary COPY is Npgsql's most efficient data loading path.
It bypasses the SQL parameter system and streams rows directly in PostgreSQL's binary wire format.
The <code>ON COMMIT DROP</code> means the temp table is cleaned up automatically when the transaction ends.</p>
<p>The trade-off is complexity: you create a table, stream data into it, then fire the update.
It is technically two operations (COPY + UPDATE), but both are efficient.
At 10,000 rows it came in at <strong>41ms</strong>, matching UNNEST.
At larger batch sizes (say, 100k+ rows), binary COPY would likely pull ahead further since the data payload grows but the query text stays constant and there are no parameter limits to worry about.</p>
<h2>Benchmark Results</h2>
<p>Here are the numbers across all three batch sizes:</p>
<pre><code>| Approach                                   | 1,000 rows | 10,000 rows | 25,000 rows |
| ------------------------------------------ | ---------- | ----------- | ----------- |
| Approach 1: Naive Dapper                   | 317ms      | 2,414ms     | 6,283ms     |
| Approach 2: EF Core SaveChanges            | 575ms      | 1,030ms     | 1,767ms     |
| Approach 3: Dapper + VALUES table          | 19ms       | 89ms        | 233ms       |
| Approach 4: EF Core ExecuteSqlRaw + VALUES | 58ms       | 166ms       | 282ms       |
| Approach 5: Dapper + CTE                   | 13ms       | 103ms       | 251ms       |
| Approach 6: Dapper + UNNEST                | 12ms       | 41ms        | 92ms        |
| Approach 7: Temp table + binary COPY       | 11ms       | 41ms        | 93ms        |
</code></pre>
<p>A few things stand out.
EF Core <code>SaveChanges</code> is actually faster than naive Dapper at 10,000+ rows because its batching cuts down round-trips significantly.
But both are far behind the single-statement approaches.
The biggest jump in the table is from Approach 2 to Approach 3.
Approach 5 (CTE) is essentially the same performance as Approach 3 (VALUES), confirming it is a style choice rather than a performance one.
Approaches 6 and 7 are the fastest at every scale, and the gap over CTE and VALUES widens as batch size grows.</p>
<h2>Summary</h2>
<p>The takeaway from this exercise is simple: round-trips are expensive and individual SQL statements add up fast.</p>
<p>Reducing 10,000 database calls to 1 is where all the gains come from.
Everything else is secondary.</p>
<p>A few things worth keeping in mind:</p>
<ul>
<li><strong>EF Core <code>SaveChanges</code> is not a bulk update tool.</strong>
It reduces round-trips through batching, but it still generates N individual <code>UPDATE</code> statements
and requires a <code>SELECT</code> to load the change tracker.
For bulk mutations, raw SQL is a better fit.
The same logic applies to inserts - I covered that in
<a href="fast-sql-bulk-inserts-with-csharp-and-ef-core"><strong>Fast SQL Bulk Inserts with C# and EF Core</strong></a>.</li>
<li><strong>For PostgreSQL, <code>UNNEST</code> and binary COPY are the best options at scale.</strong>
Both use a fixed query text that doesn't grow with batch size and have no parameter count limits.
<code>VALUES</code> and CTE are solid choices for smaller batches or when you need to stay closer to portable SQL.</li>
<li><strong>Dapper and EF Core work well together.</strong>
You don't have to choose one or the other.
Query with EF Core, bulk-update with raw SQL, share the same transaction.</li>
</ul>
<p>If you want to go deeper on SQL performance more broadly,
I recently wrote about <a href="debunking-the-filter-early-join-later-sql-performance-myth"><strong>a common myth around filter and join ordering in SQL queries</strong></a>
that trips up a lot of developers.</p>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_185.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[5 Architecture Tests You Should Add to Your .NET Projects]]></title>
            <link>https://milanjovanovic.tech/blog/5-architecture-tests-you-should-add-to-your-dotnet-projects</link>
            <guid isPermaLink="false">5-architecture-tests-you-should-add-to-your-dotnet-projects</guid>
            <pubDate>Sat, 07 Mar 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn about five essential architecture tests that can help ensure the quality and maintainability of your .NET projects.]]></description>
            <content:encoded><![CDATA[<p>Every project starts with good intentions.
You agree on layer boundaries, naming conventions, dependency direction.
The diagram goes on Confluence.</p>
<p>Six months later, someone moves a domain service into the <code>Infrastructure</code> project because it needs database access.
Or a handler gets named <code>ProcessPaymentService</code> because the dev didn't know the convention.
Or a class that should be <code>internal</code> is <code>public</code> because that's the default.
Nobody catches it in code review.</p>
<p><a href="shift-left-with-architecture-testing-in-dotnet"><strong>Architecture tests</strong></a> stop this from happening.
They turn your architectural rules into automated tests that run in CI.
If someone violates a rule, the build fails.</p>
<p>Here are 5 types of architecture tests I add to every .NET project.</p>
<h2>ArchUnitNET and the Test Setup</h2>
<p><a href="https://github.com/TNG/ArchUnitNET">ArchUnitNET</a> is the .NET port of the Java <a href="https://www.archunit.org/">ArchUnit</a> library.
It lets you write <a href="enforcing-software-architecture-with-architecture-tests"><strong>architecture rules</strong></a>
using a fluent API and run them as regular xUnit tests.</p>
<pre><code class="language-bash"># there are other test frameworks supported, but I use xUnit
dotnet add package TngTech.ArchUnitNET.xUnit
</code></pre>
<p>You need a base class that loads all the assemblies you want to test.
Each layer gets an &quot;anchor type&quot; to grab a reference to the assembly at compile time:</p>
<pre><code class="language-csharp">public abstract class BaseTest
{
    protected static readonly Assembly DomainAssembly = typeof(User).Assembly;
    protected static readonly Assembly ApplicationAssembly = typeof(ICommand).Assembly;
    protected static readonly Assembly InfrastructureAssembly = typeof(ApplicationDbContext).Assembly;
    protected static readonly Assembly PresentationAssembly = typeof(Program).Assembly;

    protected static readonly Architecture Architecture = new ArchLoader()
        .LoadAssemblies(
            DomainAssembly,
            ApplicationAssembly,
            InfrastructureAssembly,
            PresentationAssembly)
        .Build();
}
</code></pre>
<p>The <code>ArchLoader</code> scans these assemblies and builds an in-memory model of all types and their dependencies.
Every test class inherits from <code>BaseTest</code>.</p>
<h2>1. Layer Dependency Tests</h2>
<p>The most important rule in <a href="/pragmatic-clean-architecture"><strong>Clean Architecture</strong></a> is the <strong>dependency rule</strong>.
Inner layers must not reference outer layers.
All dependencies point inward.</p>
<p>In most Clean Architecture setups, the project references already prevent the obvious violations.
You can't add a reference from <code>Application</code> to <code>Infrastructure</code>
because <code>Infrastructure</code> already references <code>Application</code>, and the compiler won't allow circular dependencies.</p>
<p>So why bother with these tests?</p>
<p>Because project references aren't the only way dependencies leak in.
A NuGet package used in Infrastructure might expose types that bleed into Application through transitive references.
Someone could reorganize the solution and change the project reference graph.
Or you might move to a <a href="what-is-a-modular-monolith"><strong>modular monolith</strong></a>
where multiple layers share an assembly, and the compiler can't help you at all.</p>
<p>These tests are a safety net.
And they double as living documentation of the intended architecture.</p>
<pre><code class="language-csharp">using static ArchUnitNET.Fluent.ArchRuleDefinition;

public class LayerTests : BaseTest
{
    private static readonly IObjectProvider&lt;IType&gt; DomainLayer =
        Types().That().ResideInAssembly(DomainAssembly).As(&quot;Domain layer&quot;);

    private static readonly IObjectProvider&lt;IType&gt; ApplicationLayer =
        Types().That().ResideInAssembly(ApplicationAssembly).As(&quot;Application layer&quot;);

    private static readonly IObjectProvider&lt;IType&gt; InfrastructureLayer =
        Types().That().ResideInAssembly(InfrastructureAssembly).As(&quot;Infrastructure Layer&quot;);

    private static readonly IObjectProvider&lt;IType&gt; PresentationLayer =
        Types().That().ResideInAssembly(PresentationAssembly).As(&quot;Presentation Layer&quot;);

    [Fact]
    public void DomainLayer_ShouldNotDependOn_ApplicationLayer()
    {
        Types().That().Are(DomainLayer).Should()
            .NotDependOnAny(ApplicationLayer)
            .Check(Architecture);
    }

    [Fact]
    public void DomainLayer_ShouldNotDependOn_InfrastructureLayer()
    {
        Types().That().Are(DomainLayer).Should()
            .NotDependOnAny(InfrastructureLayer)
            .Check(Architecture);
    }

    [Fact]
    public void DomainLayer_ShouldNotDependOn_PresentationLayer()
    {
        Types().That().Are(DomainLayer).Should()
            .NotDependOnAny(PresentationLayer)
            .Check(Architecture);
    }

    [Fact]
    public void ApplicationLayer_ShouldNotDependOn_InfrastructureLayer()
    {
        Types().That().Are(ApplicationLayer).Should()
            .NotDependOnAny(InfrastructureLayer)
            .Check(Architecture);
    }

    [Fact]
    public void ApplicationLayer_ShouldNotDependOn_PresentationLayer()
    {
        Types().That().Are(ApplicationLayer).Should()
            .NotDependOnAny(PresentationLayer)
            .Check(Architecture);
    }

    [Fact]
    public void InfrastructureLayer_ShouldNotDependOn_PresentationLayer()
    {
        Types().That().Are(InfrastructureLayer).Should()
            .NotDependOnAny(PresentationLayer)
            .Check(Architecture);
    }
}
</code></pre>
<p>Six tests, one for each illegal dependency direction.</p>
<p>The fluent API reads like English:
&quot;Types that are in the Domain layer should not depend on any types in the Application layer&quot;.
When a violation happens, the test tells you exactly which type depends on which.</p>
<p>This is the first architecture test I add to any project.
If you only add one type from this list, make it this one.</p>
<p>You can also extend this further.
For example, I sometimes add tests that specific namespaces within a layer can't reference each other
(like <code>Application.Orders</code> shouldn't depend on <code>Application.Users</code>).
This can be great for enforcing a <a href="vertical-slice-architecture"><strong>vertical slice architecture</strong></a>
where each feature is self-contained, or inside a <a href="/modular-monolith-architecture"><strong>modular monolith</strong></a>
where modules shouldn't depend on each other.</p>
<h2>2. Naming Convention Tests</h2>
<p>This one might seem minor, but it adds up fast.</p>
<p>When you have 50 command handlers and 3 of them are named <code>CreateOrderService</code> or <code>ProcessPaymentUseCase</code>,
searching the codebase becomes unreliable. You search for <code>CommandHandler</code> and miss three handlers.</p>
<p>ArchUnitNET lets you enforce naming rules by selecting classes based on the interfaces they implement:</p>
<pre><code class="language-csharp">using static ArchUnitNET.Fluent.ArchRuleDefinition;

public class NamingConventionTests : BaseTest
{
    [Fact]
    public void CommandHandlers_ShouldHave_NameEndingWith_CommandHandler()
    {
        Classes().That()
            .ImplementInterface(typeof(ICommandHandler&lt;&gt;))
            .Or()
            .ImplementInterface(typeof(ICommandHandler&lt;,&gt;))
            .And().DoNotResideInNamespace(&quot;Application.Abstractions.Behaviors&quot;)
            .Should().HaveNameEndingWith(&quot;CommandHandler&quot;)
            .Check(Architecture);
    }

    [Fact]
    public void QueryHandlers_ShouldHave_NameEndingWith_QueryHandler()
    {
        Classes().That()
            .ImplementInterface(typeof(IQueryHandler&lt;,&gt;))
            .And().DoNotResideInNamespace(&quot;Application.Abstractions.Behaviors&quot;)
            .Should().HaveNameEndingWith(&quot;QueryHandler&quot;)
            .Check(Architecture);
    }

    [Fact]
    public void Validators_ShouldHave_NameEndingWith_Validator()
    {
        Classes().That()
            .HaveNameEndingWith(&quot;Validator&quot;)
            .Should().ResideInAssembly(ApplicationAssembly)
            .Check(Architecture);
    }
}
</code></pre>
<p>One gotcha here: decorators like <code>ValidationBehavior</code> implement handler interfaces too.
They're decorators in the pipeline, not domain-specific handlers.
The <code>DoNotResideInNamespace(&quot;Application.Abstractions.Behaviors&quot;)</code> filter excludes them.
I learned this the hard way when every behavior started failing the naming check.</p>
<p>The validator test works in the opposite direction.
It says &quot;classes ending with <code>Validator</code> should live in the Application assembly&quot;.
I've seen validators accidentally placed in the Infrastructure project.
This catches that.</p>
<h2>3. Colocation Tests</h2>
<p>This is the test I wish I had earlier in my career.</p>
<p>When you use <a href="cqrs-pattern-the-way-it-should-have-been-from-the-start"><strong>CQRS</strong></a>,
you end up with pairs: a command (or query) and its handler.
I keep them in the same namespace so everything for a use case lives together.
<code>Application.TodoItems.Create</code> would contain both <code>CreateTodoItemCommand</code> and <code>CreateTodoItemCommandHandler</code>.</p>
<p>But nothing stops someone from putting the handler in a completely different namespace:</p>
<pre><code class="language-csharp">public class ColocationTests : BaseTest
{
    [Theory]
    [MemberData(nameof(GetHandlerAndCommandPairs))]
    public void Handlers_ShouldResideInSameNamespace_AsTheirCommandOrQuery(
        Type handlerType,
        Type commandOrQueryType)
    {
        handlerType.Namespace.ShouldBe(
            commandOrQueryType.Namespace,
            $&quot;{handlerType.Name} should be in the same namespace as {commandOrQueryType.Name}&quot;);
    }

    public static TheoryData&lt;Type, Type&gt; GetHandlerAndCommandPairs()
    {
        Type[] handlerInterfaces =
        [
            typeof(ICommandHandler&lt;&gt;),
            typeof(ICommandHandler&lt;,&gt;),
            typeof(IQueryHandler&lt;,&gt;)
        ];

        var pairs = new TheoryData&lt;Type, Type&gt;();

        IEnumerable&lt;Type&gt; handlers = ApplicationAssembly
            .GetTypes()
            .Where(t =&gt; t is { IsClass: true, IsAbstract: false, IsGenericTypeDefinition: false })
            .Where(t =&gt; t.DeclaringType is null);

        foreach (Type handler in handlers)
        {
            foreach (Type iface in handler.GetInterfaces())
            {
                if (!iface.IsGenericType)
                {
                    continue;
                }

                Type genericDef = iface.GetGenericTypeDefinition();

                if (!handlerInterfaces.Contains(genericDef))
                {
                    continue;
                }

                Type commandOrQueryType = iface.GetGenericArguments()[0];
                pairs.Add(handler, commandOrQueryType);
            }
        }

        return pairs;
    }
}
</code></pre>
<p>This test doesn't use ArchUnitNET.
It uses <strong>raw reflection</strong> combined with xUnit's <code>[Theory]</code> and <code>[MemberData]</code>.
ArchUnitNET can't express a rule like
&quot;this class should be in the same namespace as the generic type argument of its interface&quot;.
So we drop down to reflection.</p>
<p>The <code>GetHandlerAndCommandPairs</code> method scans the Application assembly,
finds all classes implementing a handler interface,
extracts the command/query type from the generic argument,
and returns pairs for the test to assert on.</p>
<p>This enforces a <a href="vertical-slice-architecture-structuring-vertical-slices"><strong>vertical slice</strong></a> style of organizing code.
When a new developer joins the team, they can find everything for a use case in one folder.</p>
<p>You can expand this to cover request and response types, validators, or anything else that should be colocated.</p>
<h2>4. Visibility Tests</h2>
<p>Command and query handlers are implementation details.
They get resolved through DI, not referenced directly.</p>
<p>But most developers make them <code>public</code> by default.
It's just muscle memory.
The problem is that a <code>public</code> handler can be referenced directly from another layer,
bypassing the abstractions you set up.</p>
<pre><code class="language-csharp">using static ArchUnitNET.Fluent.ArchRuleDefinition;

public class VisibilityTests : BaseTest
{
    [Fact]
    public void CommandHandlers_ShouldBeInternal()
    {
        Classes().That()
            .ImplementInterface(typeof(ICommandHandler&lt;&gt;))
            .Or()
            .ImplementInterface(typeof(ICommandHandler&lt;,&gt;))
            .Should().BeInternal()
            .Check(Architecture);
    }

    [Fact]
    public void QueryHandlers_ShouldBeInternal()
    {
        Classes().That()
            .ImplementInterface(typeof(IQueryHandler&lt;,&gt;))
            .Should().BeInternal()
            .Check(Architecture);
    }
}
</code></pre>
<p>If you're worried about DI not finding <code>internal</code> classes, don't be.
Assembly scanning discovers them just fine.</p>
<p>You could extend this to other types too.
I've thought about enforcing that EF Core configurations are <code>internal</code> as well,
since there's no reason for <code>OrderConfiguration</code> to be visible outside of Infrastructure.</p>
<h2>5. Dependency Guard Tests</h2>
<p>Layer tests guard against references to your own assemblies.
But infrastructure libraries can leak in through transitive NuGet references.</p>
<p>Your Domain layer shouldn't know about Entity Framework. Your Application layer shouldn't know about Npgsql.
The compiler won't stop this if the package is transitively available.</p>
<pre><code class="language-csharp">using static ArchUnitNET.Fluent.ArchRuleDefinition;

public class DependencyGuardTests : BaseTest
{
    [Fact]
    public void DomainLayer_ShouldNotDependOn_EntityFramework()
    {
        Types().That().ResideInAssembly(DomainAssembly).Should()
            .NotDependOnAnyTypesThat()
            .ResideInNamespace(&quot;Microsoft.EntityFrameworkCore&quot;)
            .Check(Architecture);
    }

    [Fact]
    public void ApplicationLayer_ShouldNotDependOn_EntityFramework()
    {
        Types().That().ResideInAssembly(ApplicationAssembly).Should()
            .NotDependOnAnyTypesThat()
            .ResideInNamespace(&quot;Microsoft.EntityFrameworkCore&quot;)
            .Check(Architecture);
    }
}
</code></pre>
<p>Add whatever libraries make sense for your project.</p>
<h2>Summary</h2>
<p>Architectural rules that only exist in documentation will be violated.
It's not a question of <em>if</em>, it's <em>when</em>.</p>
<p>All of these tests run in milliseconds and don't require any infrastructure to run.
They sit right next to your unit tests and run on every build.</p>
<p>Architecture tests are a safety net that catches violations before they reach production.</p>
<p>Start with layer dependency tests.
They take five minutes to set up and catch the most damaging violations.
Then add the rest as your codebase grows.</p>
<p>If you want to see how I structure Clean Architecture projects with these guardrails,
check out <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture</strong></a>.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_184.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How to Implement Two-Factor Authentication in ASP.NET Core]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-implement-two-factor-authentication-in-aspnetcore</link>
            <guid isPermaLink="false">how-to-implement-two-factor-authentication-in-aspnetcore</guid>
            <pubDate>Sat, 28 Feb 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[Passwords alone are not enough. Learn how to implement Two-Factor Authentication in .NET using TOTP, QR codes, and the Otp.NET library, with a secure setup flow and encrypted secrets at rest.]]></description>
            <content:encoded><![CDATA[<p>I got hit with a <a href="https://www.linkedin.com/feed/update/urn:li:activity:7432473773032808448/">security incident</a> recently.
Someone accessed an account of mine that had a strong, unique password.
But no second factor.</p>
<p>It was a wake-up call.
Passwords alone are not enough.
They get phished, leaked in breaches, or brute-forced.
A second factor changes the equation entirely.</p>
<p><strong>Two-Factor Authentication (2FA)</strong> adds an extra verification step beyond the password.
Even if an attacker steals the password, they still can't get in without the second factor.
It's one of the most effective security measures you can implement, and it's not that hard to build.</p>
<p>In this article, I'll walk you through implementing 2FA in .NET using <strong>Time-based One-Time Passwords (TOTP)</strong> with QR codes and authenticator apps like Google Authenticator.</p>
<p>We'll cover:</p>
<ul>
<li>How TOTP works under the hood</li>
<li>Generating QR codes for authenticator app setup</li>
<li>The correct setup flow to avoid putting users in a bad state</li>
<li>Validating one-time codes</li>
<li>Encrypting user secrets at rest</li>
</ul>
<p>Let's dive in.</p>
<h2>How TOTP Works</h2>
<p><a href="https://www.rfc-editor.org/rfc/rfc6238">TOTP (Time-based One-Time Password)</a> is the algorithm behind apps like Google Authenticator, Microsoft Authenticator, and Authy.</p>
<p>The idea is simple: a <strong>shared secret</strong> is established between the server and the user's authenticator app.
Both sides use that secret combined with the current time to generate a <strong>6-digit code</strong> that changes every 30 seconds.</p>
<p>Here's the flow:</p>
<ol>
<li>The server generates a <strong>unique secret key</strong> for the user</li>
<li>The user scans a <strong>QR code</strong> containing that secret into their authenticator app</li>
<li>Both the server and the app now independently generate the same time-based codes</li>
<li>At login, the user enters the current code from their app, and the server verifies it</li>
</ol>
<p><img src="/blogs/mnw_183/totp_flow.png" alt="TOTP flow showing how the server and authenticator app independently generate the same time-based code from a shared secret."></p>
<p>Because both sides compute the code independently, there's <strong>no network call</strong> to validate.
The server just checks: &quot;given this secret and the current time, does the code match?&quot;</p>
<p>This makes TOTP fast, offline-capable, and resistant to replay attacks (each code is only valid for a short window).</p>
<h2>Generating the Secret Key</h2>
<p>Every user needs their own unique secret key.
This key is the foundation of the entire 2FA system, so it must be cryptographically random.</p>
<p>We'll use the <a href="https://github.com/kspearrin/Otp.NET">Otp.NET</a> library for TOTP operations:</p>
<pre><code class="language-bash">dotnet add package Otp.NET
</code></pre>
<p>Generate a secret key for a user:</p>
<pre><code class="language-csharp">using OtpNet;

byte[] secretKey = KeyGeneration.GenerateRandomKey(); // 20 bytes by default (SHA-1)
string base32Secret = Base32Encoding.ToString(secretKey);
</code></pre>
<p><code>KeyGeneration.GenerateRandomKey()</code> produces a cryptographically secure random key.
We encode it as <strong>Base32</strong> because that's what the <a href="https://github.com/google/google-authenticator/wiki/Key-Uri-Format">otpauth URI scheme</a> expects.</p>
<p><strong>This secret must be stored securely.</strong>
It's the equivalent of a password.
If an attacker gets the secret, they can generate valid codes.
I'll cover encrypting it at rest later in this article.</p>
<h2>Creating the QR Code</h2>
<p>To set up 2FA, the user needs to scan a QR code with their authenticator app.
The QR code encodes an <code>otpauth://</code> URI that contains the secret key and metadata.</p>
<p>Install the <a href="https://github.com/codebude/QRCoder">QRCoder</a> library:</p>
<pre><code class="language-bash">dotnet add package QRCoder
</code></pre>
<p>Here's how to generate the QR code:</p>
<pre><code class="language-csharp">using QRCoder;

const string issuer = &quot;MyApp&quot;;
const string user = &quot;user@example.com&quot;;

string escapedIssuer = Uri.EscapeDataString(issuer);
string escapedUser = Uri.EscapeDataString(user);

string otpUri =
    $&quot;otpauth://totp/{escapedIssuer}:{escapedUser}&quot; +
    $&quot;?secret={base32Secret}&quot; +
    $&quot;&amp;issuer={escapedIssuer}&quot; +
    $&quot;&amp;digits=6&quot; +
    $&quot;&amp;period=30&quot;;

using var qrGenerator = new QRCodeGenerator();
using var qrCodeData = qrGenerator.CreateQrCode(otpUri, QRCodeGenerator.ECCLevel.Q);
using var qrCode = new PngByteQRCode(qrCodeData);
byte[] qrCodeImage = qrCode.GetGraphic(10);
</code></pre>
<p>Let's unpack the <code>otpauth://</code> URI parameters:</p>
<ul>
<li><strong><code>secret</code></strong> - The Base32-encoded shared secret</li>
<li><strong><code>issuer</code></strong> - Your application name (shown in the authenticator app)</li>
<li><strong><code>digits</code></strong> - Number of digits in the code (standard is 6)</li>
<li><strong><code>period</code></strong> - How often the code rotates in seconds (standard is 30)</li>
</ul>
<p>The <code>ECCLevel.Q</code> gives us a good balance between error correction and QR code size.
It means the QR code can still be scanned even if about 25% of it is damaged or obscured.</p>
<p>Here's what the generated QR code looks like:</p>
<div className="centered">
  ![A generated QR code encoding the otpauth URI for authenticator app setup.](/blogs/mnw_183/qr_code_example.png)
</div>
<p>And once the user scans it, the entry appears in their authenticator app:</p>
<div className="centered">
  ![Google Authenticator showing a TOTP entry for MyApp with a 6-digit code and a 30-second countdown timer.](/blogs/mnw_183/google_authenticator_entry.png)
</div>
<p>You can also display the <code>base32Secret</code> string alongside the QR code.
Some users prefer to type it in manually.</p>
<h2>The Setup Flow</h2>
<p>Getting the setup flow right is critical.
If you enable 2FA the moment the user requests it, before they've even scanned the QR code, you've locked them out.</p>
<p>Here's the correct flow:</p>
<ol>
<li><strong>User requests 2FA setup</strong> - Generate a secret key and store it as <em>pending</em> (not yet active)</li>
<li><strong>Show the QR code</strong> - The user scans it with their authenticator app</li>
<li><strong>User enters the first code</strong> - This proves they successfully set up their authenticator app</li>
<li><strong>Server validates the code</strong> - If it matches, activate 2FA for the user</li>
<li><strong>Generate recovery codes</strong> - Give the user backup codes in case they lose their device</li>
</ol>
<p>The key insight is step 3.
<strong>Never enable 2FA until the user has confirmed they can generate valid codes.</strong>
Otherwise, you'll end up with users who have 2FA &quot;enabled&quot; but no way to generate codes.</p>
<p><img src="/blogs/mnw_183/setup_flow.png" alt="Sequence diagram showing the 2FA setup flow: request setup, scan QR code, confirm first code, activate."></p>
<p>Here's what the API endpoints look like.
All 2FA endpoints must be <strong>protected</strong>, the user has to be authenticated first.
The best practice is to use <code>.RequireAuthorization()</code> on each endpoint:</p>
<pre><code class="language-csharp">app.MapPost(&quot;2fa/setup&quot;, async (HttpContext context, UserService userService) =&gt;
{
    var userId = context.User.GetUserId();

    byte[] secretKey = KeyGeneration.GenerateRandomKey();
    string base32Secret = Base32Encoding.ToString(secretKey);

    // Store the pending secret (encrypted) - NOT yet active
    await userService.StorePendingTwoFactorSecret(userId, base32Secret);

    string otpUri =
        $&quot;otpauth://totp/{Uri.EscapeDataString(&quot;MyApp&quot;)}:{Uri.EscapeDataString(userId)}&quot; +
        $&quot;?secret={base32Secret}&quot; +
        $&quot;&amp;issuer={Uri.EscapeDataString(&quot;MyApp&quot;)}&quot; +
        $&quot;&amp;digits=6&amp;period=30&quot;;

    using var qrGenerator = new QRCodeGenerator();
    using var qrCodeData = qrGenerator.CreateQrCode(otpUri, QRCodeGenerator.ECCLevel.Q);
    using var qrCode = new PngByteQRCode(qrCodeData);
    byte[] qrCodeImage = qrCode.GetGraphic(10);

    return Results.File(qrCodeImage, &quot;image/png&quot;);
})
.RequireAuthorization();
</code></pre>
<p>And the confirmation endpoint:</p>
<pre><code class="language-csharp">app.MapPost(&quot;2fa/confirm&quot;, async (
    ConfirmTwoFactorRequest request,
    HttpContext context,
    UserService userService) =&gt;
{
    var userId = context.User.GetUserId();

    string? pendingSecret = await userService.GetPendingTwoFactorSecret(userId);
    if (pendingSecret is null)
    {
        return Results.BadRequest(&quot;No pending 2FA setup found.&quot;);
    }

    byte[] secretKey = Base32Encoding.ToBytes(pendingSecret);
    var totp = new Totp(secretKey);

    bool isValid = totp.VerifyTotp(
        request.Code,
        out _,
        VerificationWindow.RfcSpecifiedNetworkDelay);

    if (!isValid)
    {
        return Results.BadRequest(&quot;Invalid code. Please try again.&quot;);
    }

    // Code is valid - activate 2FA
    await userService.ActivateTwoFactor(userId, pendingSecret);

    // Generate recovery codes
    var recoveryCodes = await userService.GenerateRecoveryCodes(userId);

    return Results.Ok(new { recoveryCodes });
})
.RequireAuthorization();

internal record ConfirmTwoFactorRequest(string Code);
</code></pre>
<p>This two-step approach guarantees you never activate 2FA for a user who can't actually use it.
If the user abandons the setup halfway through, the pending secret gets cleaned up and nothing breaks.</p>
<h2>The Login Flow With 2FA</h2>
<p>Here's a critical point that's easy to get wrong: <strong>don't issue a full access token until the user passes the 2FA check.</strong></p>
<p>If a user has 2FA enabled and you issue a JWT after they enter their password, you've already given them full access.
The 2FA step becomes meaningless.</p>
<p>The correct approach is a <strong>two-step login</strong>:</p>
<ol>
<li>The user submits their username and password</li>
<li>If credentials are valid and 2FA is enabled, return a <strong>limited-scope token</strong> (or session) that only allows calling the <code>2fa/validate</code> endpoint</li>
<li>The user submits their TOTP code</li>
<li>If the code is valid, issue the <strong>full access token</strong></li>
</ol>
<pre><code class="language-csharp">app.MapPost(&quot;auth/login&quot;, async (LoginRequest request, UserService userService) =&gt;
{
    var user = await userService.ValidateCredentials(request.Email, request.Password);
    if (user is null)
    {
        return Results.Unauthorized();
    }

    if (user.TwoFactorEnabled)
    {
        // Issue a short-lived, limited token that only permits 2FA validation
        var limitedToken = TokenService.GenerateLimitedToken(user.Id, purpose: &quot;2fa&quot;);

        return Results.Ok(new { requiresTwoFactor = true, token = limitedToken });
    }

    // No 2FA - issue full access token
    var accessToken = TokenService.GenerateAccessToken(user);

    return Results.Ok(new { accessToken });
});
</code></pre>
<p>The limited token should have a short expiration (2-3 minutes) and a claim or scope that restricts it to the <code>2fa/validate</code> endpoint only.
Your authorization policy on the validation endpoint can check for this specific claim.</p>
<p>This way, a stolen password alone never results in a full access token.</p>
<h2>Validating TOTP Codes</h2>
<p>Once 2FA is active, you need to validate codes during login.
Here's the validation logic:</p>
<pre><code class="language-csharp">app.MapPost(&quot;2fa/validate&quot;, async (
    ValidateOtpRequest request,
    HttpContext context,
    UserService userService) =&gt;
{
    var userId = context.User.GetUserId();

    string? secret = await userService.GetTwoFactorSecret(userId);
    if (secret is null)
    {
        return Results.BadRequest(&quot;2FA is not enabled.&quot;);
    }

    byte[] secretKey = Base32Encoding.ToBytes(secret);
    var totp = new Totp(secretKey);

    bool isValid = totp.VerifyTotp(
        request.Code,
        out long timeStepMatched,
        VerificationWindow.RfcSpecifiedNetworkDelay);

    return Results.Ok(new { isValid });
})
.RequireAuthorization();

internal record ValidateOtpRequest(string Code);
</code></pre>
<p>The <code>VerificationWindow.RfcSpecifiedNetworkDelay</code> parameter is important.
It allows a small window of tolerance around the current time step.
This accounts for clock drift between the server and the user's device.</p>
<p>Without a verification window, a code that was valid 2 seconds ago might be rejected because the server crossed into the next 30-second period.
The RFC-specified window typically allows one time step before and after the current one.</p>
<h3>Preventing Code Reuse</h3>
<p>One subtle but important point: <strong>a TOTP code should only be accepted once</strong>.</p>
<p>If an attacker intercepts a valid code (e.g., through shoulder surfing), they shouldn't be able to reuse it.
The <code>timeStepMatched</code> output parameter tells you which time step the code belongs to.
You can store the last used time step and reject any code from the same or earlier step:</p>
<pre><code class="language-csharp">bool isValid = totp.VerifyTotp(
    request.Code,
    out long timeStepMatched,
    VerificationWindow.RfcSpecifiedNetworkDelay);

if (isValid)
{
    long? lastUsedTimeStep = await userService.GetLastUsedTimeStep(userId);

    if (lastUsedTimeStep.HasValue &amp;&amp; timeStepMatched &lt;= lastUsedTimeStep.Value)
    {
        return Results.BadRequest(&quot;Code already used.&quot;);
    }

    await userService.UpdateLastUsedTimeStep(userId, timeStepMatched);
}
</code></pre>
<p>This prevents replay attacks within the verification window.</p>
<h3>Rate Limiting</h3>
<p>The validation endpoint is a brute-force target.
A 6-digit code has only 1,000,000 possible combinations.
Without rate limiting, an attacker could try all of them in minutes.</p>
<p>At a minimum, you should:</p>
<ul>
<li><strong>Limit attempts per user</strong> - Lock the account or add a delay after 3-5 failed attempts</li>
<li><strong>Use exponential backoff</strong> - Double the wait time after each failure</li>
<li><strong>Log failed attempts</strong> - Unusual patterns (many failures from one IP) are a red flag</li>
</ul>
<p><a href="http://ASP.NET">ASP.NET</a> Core has a built-in <a href="how-to-use-rate-limiting-in-aspnet-core"><strong>rate limiting middleware</strong></a> that makes this straightforward to add.</p>
<h2>Encrypting Secrets at Rest</h2>
<p>The TOTP secret key is the most sensitive piece of data in your 2FA system.
If someone dumps your database, they shouldn't be able to generate valid codes for your users.</p>
<p><strong>Never store TOTP secrets in plain text.</strong></p>
<p>Encrypt them before writing to the database and decrypt only when you need to verify a code.
I covered this in detail in my article on <a href="implementing-aes-encryption-with-csharp"><strong>implementing AES encryption with C#</strong></a>.</p>
<p>Here's the general approach:</p>
<pre><code class="language-csharp">public class UserService
{
    private readonly IEncryptionService _encryptionService;

    public async Task StorePendingTwoFactorSecret(string userId, string secret)
    {
        string encryptedSecret = _encryptionService.Encrypt(secret);

        // Store encryptedSecret in the database
        await _dbContext.Users
            .Where(u =&gt; u.Id == userId)
            .ExecuteUpdateAsync(u =&gt; u
                .SetProperty(x =&gt; x.PendingTwoFactorSecret, encryptedSecret));
    }

    public async Task&lt;string?&gt; GetTwoFactorSecret(string userId)
    {
        var user = await _dbContext.Users.FindAsync(userId);
        if (user?.TwoFactorSecret is null) return null;

        return _encryptionService.Decrypt(user.TwoFactorSecret);
    }
}
</code></pre>
<p>The encryption key itself should live in a <strong>key management service</strong> like
<a href="https://learn.microsoft.com/en-us/azure/key-vault/">Azure Key Vault</a>,
<a href="https://aws.amazon.com/kms/">AWS KMS</a>, or
<a href="https://www.vaultproject.io/">HashiCorp Vault</a>.
Never store it in your <code>appsettings.json</code> or source code.</p>
<h2>Recovery Codes</h2>
<p>What happens when a user loses their phone?</p>
<p>Without a recovery mechanism, they're permanently locked out of their account.
<strong>Recovery codes</strong> solve this.
They're one-time-use codes generated when the user enables 2FA.</p>
<pre><code class="language-csharp">public async Task&lt;List&lt;string&gt;&gt; GenerateRecoveryCodes(string userId, int count = 8)
{
    var codes = new List&lt;string&gt;();

    for (int i = 0; i &lt; count; i++)
    {
        // Generate a cryptographically random code
        var bytes = RandomNumberGenerator.GetBytes(5);
        var code = Convert.ToHexString(bytes).ToLower();
        codes.Add(code);
    }

    // Hash the codes before storing (same as passwords - one-way)
    var hashedCodes = codes
        .Select(c =&gt; BCrypt.Net.BCrypt.HashPassword(c))
        .ToList();

    await _dbContext.RecoveryCodes
        .Where(rc =&gt; rc.UserId == userId)
        .ExecuteDeleteAsync();

    _dbContext.RecoveryCodes.AddRange(
        hashedCodes.Select(h =&gt; new RecoveryCode
        {
            UserId = userId,
            CodeHash = h,
            IsUsed = false
        }));

    await _dbContext.SaveChangesAsync();

    // Return plain text codes to show the user ONCE
    return codes;
}
</code></pre>
<p>A few important details:</p>
<ul>
<li><strong>Hash the recovery codes</strong> before storing them. They're single-use passwords.
Use <code>bcrypt</code> (e.g. <a href="https://github.com/BcryptNet/bcrypt.net">Bcrypt.Net</a>) or similar.</li>
<li><strong>Show them only once.</strong> After the user dismisses the dialog, the plain text codes are gone.</li>
<li><strong>Mark codes as used.</strong> Each recovery code works exactly once.</li>
<li><strong>Generate enough codes.</strong> Eight to ten is standard. The user can regenerate them if they run low.</li>
</ul>
<p>When validating a recovery code, check each stored hash until you find a match:</p>
<pre><code class="language-csharp">public async Task&lt;bool&gt; ValidateRecoveryCode(string userId, string code)
{
    var storedCodes = await _dbContext.RecoveryCodes
        .Where(rc =&gt; rc.UserId == userId &amp;&amp; !rc.IsUsed)
        .ToListAsync();

    var matchingCode = storedCodes
        .FirstOrDefault(rc =&gt; BCrypt.Net.BCrypt.Verify(code, rc.CodeHash));

    if (matchingCode is null) return false;

    matchingCode.IsUsed = true;
    await _dbContext.SaveChangesAsync();

    return true;
}
</code></pre>
<h2>Putting It All Together</h2>
<p>Here's a minimal but complete setup showing the full 2FA flow.
I'm using a route group with <code>.RequireAuthorization()</code> so every endpoint underneath is protected:</p>
<pre><code class="language-csharp">using OtpNet;
using QRCoder;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddAuthentication().AddJwtBearer();
builder.Services.AddAuthorization();

var app = builder.Build();

app.UseAuthentication();
app.UseAuthorization();

var twoFactorGroup = app.MapGroup(&quot;2fa&quot;).RequireAuthorization();

twoFactorGroup.MapPost(&quot;setup&quot;, async (HttpContext context, UserService userService) =&gt;
{
    var userId = context.User.GetUserId();

    byte[] secretKey = KeyGeneration.GenerateRandomKey();
    string base32Secret = Base32Encoding.ToString(secretKey);

    await userService.StorePendingTwoFactorSecret(userId, base32Secret);

    string otpUri =
        $&quot;otpauth://totp/{Uri.EscapeDataString(&quot;MyApp&quot;)}:{Uri.EscapeDataString(userId)}&quot; +
        $&quot;?secret={base32Secret}&quot; +
        $&quot;&amp;issuer={Uri.EscapeDataString(&quot;MyApp&quot;)}&quot; +
        $&quot;&amp;digits=6&amp;period=30&quot;;

    using var qrGenerator = new QRCodeGenerator();
    using var qrCodeData = qrGenerator.CreateQrCode(otpUri, QRCodeGenerator.ECCLevel.Q);
    using var qrCode = new PngByteQRCode(qrCodeData);
    byte[] qrCodeImage = qrCode.GetGraphic(10);

    return Results.File(qrCodeImage, &quot;image/png&quot;);
});

twoFactorGroup.MapPost(&quot;confirm&quot;, async (
    ConfirmTwoFactorRequest request,
    HttpContext context,
    UserService userService) =&gt;
{
    var userId = context.User.GetUserId();

    string? pendingSecret = await userService.GetPendingTwoFactorSecret(userId);
    if (pendingSecret is null)
    {
        return Results.BadRequest(&quot;No pending 2FA setup found.&quot;);
    }

    byte[] secretKey = Base32Encoding.ToBytes(pendingSecret);
    var totp = new Totp(secretKey);

    bool isValid = totp.VerifyTotp(
        request.Code,
        out _,
        VerificationWindow.RfcSpecifiedNetworkDelay);

    if (!isValid)
    {
        return Results.BadRequest(&quot;Invalid code. Please try again.&quot;);
    }

    await userService.ActivateTwoFactor(userId, pendingSecret);
    var recoveryCodes = await userService.GenerateRecoveryCodes(userId);

    return Results.Ok(new { recoveryCodes });
});

twoFactorGroup.MapPost(&quot;validate&quot;, async (
    ValidateOtpRequest request,
    HttpContext context,
    UserService userService) =&gt;
{
    var userId = context.User.GetUserId();

    string? secret = await userService.GetTwoFactorSecret(userId);
    if (secret is null)
    {
        return Results.BadRequest(&quot;2FA is not enabled.&quot;);
    }

    byte[] secretKey = Base32Encoding.ToBytes(secret);
    var totp = new Totp(secretKey);

    bool isValid = totp.VerifyTotp(
        request.Code,
        out long timeStepMatched,
        VerificationWindow.RfcSpecifiedNetworkDelay);

    if (isValid)
    {
        long? lastUsedTimeStep = await userService.GetLastUsedTimeStep(userId);
        if (lastUsedTimeStep.HasValue &amp;&amp; timeStepMatched &lt;= lastUsedTimeStep.Value)
        {
            return Results.BadRequest(&quot;Code already used.&quot;);
        }

        await userService.UpdateLastUsedTimeStep(userId, timeStepMatched);
    }

    return Results.Ok(new { isValid });
});

app.Run();

internal record ConfirmTwoFactorRequest(string Code);
internal record ValidateOtpRequest(string Code);
</code></pre>
<p>If you don't need full control, <a href="integrate-keycloak-with-aspnetcore-using-oauth-2"><strong>Keycloak</strong></a> and
<a href="https://learn.microsoft.com/en-us/aspnet/core/security/authentication/identity"><strong>ASP.NET Core Identity</strong></a>
both support TOTP-based 2FA out of the box.
But building it yourself is worth it when you need a custom flow or
want to understand what's happening under the hood.</p>
<h2>Summary</h2>
<p>2FA is one of the highest-impact security features you can add to an application.
TOTP with authenticator apps is a solid choice because it's offline-capable, widely supported, and doesn't depend on SMS (which is vulnerable to SIM swapping).</p>
<p>The important parts to get right:</p>
<ul>
<li><strong>Use a proper setup flow.</strong> Generate the secret, show the QR code, and only activate 2FA after the user confirms their first code.</li>
<li><strong>Encrypt secrets at rest.</strong> The TOTP secret is as sensitive as a password. Encrypt it with <a href="implementing-aes-encryption-with-csharp"><strong>AES</strong></a> and store keys in a key vault.</li>
<li><strong>Prevent code reuse.</strong> Track the last used time step to block replay attacks.</li>
<li><strong>Provide recovery codes.</strong> Users lose phones. Hash the codes before storing them, just like passwords.</li>
</ul>
<p>If you're building APIs that handle sensitive operations, adding 2FA significantly raises the bar for attackers.
It's not bulletproof, but it stops the vast majority of credential-based attacks.</p>
<p>If you're looking for a deep dive into building secure APIs with authentication and encryption,
check out my <a href="/pragmatic-rest-apis"><strong>Pragmatic REST APIs</strong></a> course.</p>
<p>Hope this was useful. See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_183.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Where Vertical Slices Fit Inside the Modular Monolith Architecture]]></title>
            <link>https://milanjovanovic.tech/blog/where-vertical-slices-fit-inside-the-modular-monolith-architecture</link>
            <guid isPermaLink="false">where-vertical-slices-fit-inside-the-modular-monolith-architecture</guid>
            <pubDate>Sat, 21 Feb 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[Modular Monolith tells you how to split the system into modules. But it says nothing about how to organize code inside each module. Vertical Slice Architecture fills that gap, and the combination is incredibly powerful.]]></description>
            <content:encoded><![CDATA[<p>Most teams get the macro architecture right.
They build a <a href="what-is-a-modular-monolith"><strong>modular monolith</strong></a> with clear module boundaries, <a href="internal-vs-public-apis-in-modular-monoliths"><strong>public APIs</strong></a>, and proper <a href="modular-monolith-data-isolation"><strong>data isolation</strong></a>.</p>
<p>But then they stop thinking about architecture.
Every module gets the same internal structure, usually some form of layered architecture.</p>
<p>The thing is, <a href="/pragmatic-clean-architecture"><strong>Clean Architecture</strong></a> and <a href="vertical-slice-architecture"><strong>Vertical Slice Architecture</strong></a> aren't as far apart as people think.
Both focus on use cases and maximizing cohesion.
Clean Architecture just adds a rule for the direction of dependencies, which often leads to more abstractions and ceremony.
My <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture</strong></a> approach takes a middle ground, and it's quite similar to VSA in nature.</p>
<p>The real question isn't which one is &quot;better&quot;.
It's where each one shines inside your modular monolith.
And the beautiful part is: you can mix and match.</p>
<h2>Two Levels of Architecture</h2>
<p>There are two architectural decisions you need to make when building a modular monolith:</p>
<ol>
<li><strong>Macro architecture</strong> - How do you decompose the system into modules?
This covers module boundaries, <a href="modular-monolith-communication-patterns"><strong>communication patterns</strong></a>,
<a href="modular-monolith-data-isolation"><strong>data isolation</strong></a>, public API design, and how modules are deployed.</li>
<li><strong>Micro architecture</strong> - How do you organize code <em>inside</em> each module?
This covers folder structure, the direction of dependencies, how you implement use cases,
where validation lives, and how you access the database.</li>
</ol>
<p>Most articles about modular monoliths focus entirely on the macro level.
And for good reason.
Getting <strong>module boundaries</strong> wrong is <strong>expensive to fix</strong>.</p>
<p><img src="/blogs/mnw_182/modular_monolith.png" alt="Modular monolith."></p>
<p>But the micro level is equally important.
It determines how easy it is to add features, navigate code, and onboard new developers within a module.
It's the architecture your team interacts with every day.</p>
<p>The key insight is this:</p>
<blockquote>
<p>The macro architecture constrains how modules interact.
The micro architecture is a local decision each module can make independently.</p>
</blockquote>
<p>Your <code>Ticketing</code> module doesn't have to follow the same internal structure as your <code>Notifications</code> module.
The modular boundary gives you this freedom.</p>
<h3>Vertical Slices Are Not Modules</h3>
<p>I've seen people conflate vertical slices with modules.
On the macro level, a module can look like a &quot;vertical slice&quot; of the business domain.
But that analogy breaks down at the application level.</p>
<p>A module is a bounded context.
It owns its data, exposes a public API, and encapsulates a business capability.
A vertical slice is a feature implementation pattern.
It groups the request, handler, validation, and data access for a single use case.</p>
<p>Modules and vertical slices operate at different levels.
Modules define the boundaries of the system.
Vertical slices organize the code within those boundaries.</p>
<p><img src="/blogs/mnw_182/vertical_slices.png" alt="Vertical slices."></p>
<h2>Vertical Slices Inside a Module</h2>
<p><a href="vertical-slice-architecture-structuring-vertical-slices"><strong>Vertical Slice Architecture</strong></a>
organizes code by feature instead of by technical layer.
Each feature is a self-contained unit: request, handler, validation, data access, all in one place.</p>
<p>Inside a modular monolith module, this is a natural fit.
The module boundary already enforces separation from the rest of the system.
You don't need layers to protect you.
The module's <a href="internal-vs-public-apis-in-modular-monoliths"><strong>public API</strong></a> does that.</p>
<p>Here's what a <code>Ticketing</code> module looks like with vertical slices using one file per feature:</p>
<pre><code>📁 Modules/
|__ 📁 Ticketing
    |__ 📁 Features
        |__ 📁 AddItemToCart
            |__ #️⃣ AddItemToCart.cs
        |__ 📁 SubmitOrder
            |__ #️⃣ SubmitOrder.cs
        |__ 📁 GetOrder
            |__ #️⃣ GetOrder.cs
        |__ 📁 CancelOrder
            |__ #️⃣ CancelOrder.cs
        |__ 📁 RefundPayment
            |__ #️⃣ RefundPayment.cs
    |__ 📁 Data
        |__ #️⃣ TicketingDbContext.cs
    |__ 📁 Entities
        |__ #️⃣ Order.cs
        |__ #️⃣ Ticket.cs
    |__ #️⃣ ITicketingModule.cs
    |__ #️⃣ TicketingModule.cs
</code></pre>
<p>You can also use separate files per component if you prefer more granularity:</p>
<pre><code>📁 Modules/
|__ 📁 Ticketing
    |__ 📁 Features
        |__ 📁 SubmitOrder
            |__ #️⃣ SubmitOrderRequest.cs
            |__ #️⃣ SubmitOrderResponse.cs
            |__ #️⃣ SubmitOrderHandler.cs
            |__ #️⃣ SubmitOrderValidator.cs
            |__ #️⃣ SubmitOrderEndpoint.cs
        |__ 📁 GetOrder
            |__ #️⃣ GetOrderRequest.cs
            |__ #️⃣ GetOrderResponse.cs
            |__ #️⃣ GetOrderHandler.cs
            |__ #️⃣ GetOrderEndpoint.cs
    |__ 📁 Data
    |__ 📁 Entities
    |__ #️⃣ ITicketingModule.cs
    |__ #️⃣ TicketingModule.cs
</code></pre>
<p>Both approaches keep everything for a feature in one folder.
Compare this to having <code>Application</code>, <code>Domain</code>, and <code>Infrastructure</code>
folders with dozens of files scattered across layers.
The vertical slice version is flat, scannable, and easy to navigate.</p>
<p>Here's a concrete example using a static class to group the feature components:</p>
<pre><code class="language-csharp">public static class SubmitOrder
{
    public record Request(string CartId);
    public record Response(string OrderId, decimal Total);

    public class Validator : AbstractValidator&lt;Request&gt;
    {
        public Validator()
        {
            RuleFor(x =&gt; x.CartId).NotEmpty();
        }
    }

    public class Endpoint : IEndpoint
    {
        public void MapEndpoint(IEndpointRouteBuilder app)
        {
            app.MapPost(&quot;orders&quot;, Handler).WithTags(&quot;Ticketing&quot;);
        }

        public static async Task&lt;IResult&gt; Handler(
            Request request,
            IValidator&lt;Request&gt; validator,
            TicketingDbContext context)
        {
            var result = validator.Validate(request);
            if (!result.IsValid)
            {
                return Results.BadRequest(result.Errors);
            }

            var cart = await context.Carts
                .Include(c =&gt; c.Items)
                .FirstOrDefaultAsync(c =&gt; c.Id == request.CartId);

            if (cart is null)
            {
                return Results.NotFound();
            }

            var order = Order.Create(cart);

            context.Orders.Add(order);
            await context.SaveChangesAsync();

            return Results.Ok(
                new Response(order.Id, order.Total));
        }
    }
}
</code></pre>
<p>Adding a new feature means adding a new folder.
You're not touching shared code across layers.
You're not worrying about side effects.</p>
<h2>Choosing the Internal Architecture</h2>
<p>A common misconception is that VSA is only for simple modules and Clean Architecture is for complex ones.
That's not how it works.</p>
<p>Vertical slices work great with rich domain models.
You can have domain entities, value objects, and domain events inside a vertical slice.
The slice organizes the entry point and orchestration. The domain model handles the business rules.
As the slice grows in complexity, you <a href="refactoring-from-an-anemic-domain-model-to-a-rich-domain-model"><strong>push logic into the domain</strong></a>
just like you would in Clean Architecture.</p>
<p>Similarly, <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture</strong></a> works well for simpler modules too.
The structure is lightweight when the domain is simple.</p>
<p>So the decision isn't about complexity.
It's about what your team is comfortable with and what gives you the most clarity.</p>
<p>Here are a few things I consider:</p>
<ul>
<li><strong>Team familiarity.</strong> If your team already thinks in layers and dependency direction,
Clean Architecture will feel natural. If they prefer organizing around features, VSA reduces friction.</li>
<li><strong>Shared domain logic.</strong> When many use cases share the same domain entities,
having a dedicated <code>Domain</code> layer can make that sharing explicit.
With VSA, you'd extract shared logic into a separate folder, which also works.</li>
<li><strong>Independence of features.</strong> When features are mostly independent and rarely share behavior,
vertical slices keep things simple. Each feature is self-contained, and the mental model is straightforward.</li>
</ul>
<p>The modular boundary protects the rest of the system regardless of what you choose inside the module.
So pick what makes your team most productive and be willing to change as the module evolves.</p>
<h2>Takeaway</h2>
<p><a href="what-is-a-modular-monolith"><strong>Modular Monolith</strong></a> answers the macro question:
how to decompose the system into modules with clear boundaries.
<a href="vertical-slice-architecture"><strong>Vertical Slice Architecture</strong></a> answers the micro question:
how to organize code by feature inside those modules.</p>
<p>They operate at different levels.
Modular Monolith gives you high cohesion within each module and helps you manage coupling between modules.
Vertical Slice Architecture gives you high cohesion within each feature inside a module.</p>
<p>You don't have to pick one internal architecture for the entire system.
Each module can choose what works best for its context.
Some modules will use <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture</strong></a>.
Others will use vertical slices.
A well-defined module boundary makes this safe.</p>
<p>If you want to see how I build modular monoliths with this approach,
check out <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a>.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_182.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How to Extract Structured Data From Images Using Ollama in .NET]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-extract-structured-data-from-images-using-ollama-in-dotnet</link>
            <guid isPermaLink="false">how-to-extract-structured-data-from-images-using-ollama-in-dotnet</guid>
            <pubDate>Sat, 14 Feb 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[Vision models can do more than describe images. I used Ollama and Microsoft.Extensions.AI to extract structured receipt data into strongly typed C# objects, all running locally.]]></description>
            <content:encoded><![CDATA[<p>I wanted to see how well a <strong>vision model</strong> (LLM) could parse grocery receipts into structured data.</p>
<p>And I don't just mean describe what it sees.
I want to be able to actually extract line items, quantities, and prices into clean JSON.
Running entirely on my machine with <a href="https://ollama.com/"><strong>Ollama</strong></a> and a <strong>llama3.2-vision</strong> model.</p>
<p>It started as a quick experiment. I ended up spending a whole evening on it.</p>
<p>In this week's issue, I'll walk you through:</p>
<ul>
<li>Setting up Ollama with a vision model in .NET</li>
<li>Sending images to the model and getting structured output</li>
<li>Iterating on the system prompt to improve accuracy</li>
<li>Deserializing LLM responses into strongly typed C# objects</li>
<li>Testing whether the results are actually consistent</li>
</ul>
<p>Let's dive in.</p>
<h2>Setting Up Ollama With <a href="http://Microsoft.Extensions.AI">Microsoft.Extensions.AI</a></h2>
<p><a href="https://ollama.com/">Ollama</a> lets you run large language models locally.
You pull a model the same way you'd pull a Docker image, and it runs on your hardware.</p>
<pre><code class="language-bash">ollama pull llama3.2-vision:latest

# Then run the model locally
ollama run llama3.2-vision:latest
</code></pre>
<p>On the .NET side, <a href="https://learn.microsoft.com/en-us/dotnet/ai/ai-extensions"><strong>Microsoft.Extensions.AI</strong></a>
provides a unified <code>IChatClient</code> interface for talking to any LLM provider.
Combined with <a href="https://github.com/awaescher/OllamaSharp">OllamaSharp</a>, the setup is minimal:</p>
<pre><code class="language-csharp">var builder = Host.CreateApplicationBuilder();

builder.Services.AddChatClient(
    new OllamaApiClient(
        new Uri(&quot;http://localhost:11434&quot;),
        &quot;llama3.2-vision:latest&quot;));

var app = builder.Build();

var chatClient = app.Services.GetRequiredService&lt;IChatClient&gt;();
</code></pre>
<p>That gives us an <code>IChatClient</code> backed by a local vision model.
You don't have to manage any API keys or cloud dependencies.</p>
<p>The nice thing about <code>IChatClient</code> is that it's <strong>provider-agnostic</strong>.
If you want to swap Ollama for OpenAI or Azure later, your application code doesn't change.</p>
<h2>Sending an Image to the Model</h2>
<p>The simplest thing to try: send a receipt image and ask what's in it.</p>
<pre><code class="language-csharp">var message = new ChatMessage(
    ChatRole.User, &quot;What's in this image?&quot;);

message.Contents.Add(
    new DataContent(
        File.ReadAllBytes(&quot;receipts/receipt_1.png&quot;),
        &quot;image/png&quot;));

var response = await chatClient.GetResponseAsync([message]);

Console.WriteLine(response.Text);
</code></pre>
<p>You read the image bytes, wrap them in a <code>DataContent</code> with the appropriate MIME type, and attach it to a <code>ChatMessage</code>.</p>
<p>Here's the receipt I used for testing.</p>
<div className="centered">
  ![A grocery receipt with line items, quantities, and prices.](/blogs/mnw_181/receipt_1.png)
</div>
<p>And here's the raw text response from the model:</p>
<pre><code class="language-text">This image appears to be a receipt or invoice in a foreign language, likely Russian or another
Slavic language. The document is in black and white and features a large QR code at the bottom.
The text is in a blocky, old-style font and includes several columns of numbers and words.
The top of the page has a header with some information in a foreign language, followed by a
series of columns with various details such as date, time, and product information. The document
also includes some calculations and a total at the bottom. The overall design and layout suggest
that this is a receipt or invoice from a store or restaurant, but the specific details and
language make it difficult to understand without further context or translation.
</code></pre>
<p>The model correctly identified it as a receipt and listed the items on it.
That's impressive for a first try with zero fine-tuning.
But it's not very useful if we want to extract structured data.</p>
<p>The good thing is we can refine our prompt and ask for a more specific output format.</p>
<h2>Asking for JSON Output</h2>
<p>A text description isn't very useful programmatically. So I asked for structured JSON instead:</p>
<pre><code class="language-csharp">var message = new ChatMessage(ChatRole.User,
    @&quot;&quot;&quot;
    Extract all line items from this receipt.
    Respond in JSON format with this structure:
    {
        &quot;items&quot;: [
            {
                &quot;name&quot;: &quot;item name&quot;,
                &quot;quantity&quot;: 1.500,
                &quot;unitPrice&quot;: 0.00,
                &quot;totalPrice&quot;: 0.00
            }
        ],
        &quot;subtotal&quot;: 0.00
    }
    &quot;&quot;&quot;);
message.Contents.Add(
    new DataContent(
        File.ReadAllBytes(&quot;receipts/receipt_1.png&quot;),
        &quot;image/png&quot;));

var response = await chatClient.GetResponseAsync([message]);

Console.WriteLine(response.Text);
</code></pre>
<p>Here's the new response from the model:</p>
<pre><code class="language-text">Here is the receipt in JSON format with all line items extracted:

{
    &quot;items&quot;: [
        {
            &quot;name&quot;: &quot;limun /kg (A)&quot;,
            &quot;quantity&quot;: 280.00,
            &quot;unitPrice&quot;: 1.105,
            &quot;totalPrice&quot;: 309.40
        },
        {
            &quot;name&quot;: &quot;salata /kom (A)&quot;,
            &quot;quantity&quot;: 70.00,
            &quot;unitPrice&quot;: 3.00,
            &quot;totalPrice&quot;: 210.00
        },
        {
            &quot;name&quot;: &quot;susam 100g trpeza /kom (A)&quot;,
            &quot;quantity&quot;: 90.00,
            &quot;unitPrice&quot;: 1.00,
            &quot;totalPrice&quot;: 90.00
        }
    ],
    &quot;subtotal&quot;: 609.40
}
</code></pre>
<p>This worked surprisingly well on the first try.
The model returned a JSON blob with item names, quantities, and prices.
Some quantities were slightly off but the structure was correct.</p>
<p>For zero fine-tuning, I was impressed.</p>
<h2>Iterating on the System Prompt</h2>
<p>Here's what I found: when the model misreads a digit or rounds a number, you don't fix a bug in C#.
You refine the system prompt.</p>
<p>What is a system prompt?
It's the initial instruction you give the model to set the context for the conversation.
Notice the <code>ChatRole.System</code> in the code below.
Conversely, we use <code>ChatRole.User</code> for the message where we ask the question.</p>
<p>After a few rounds of this, my system prompt ended up reading like a specification document:</p>
<pre><code class="language-csharp">var systemMessage = new ChatMessage(ChatRole.System,
    @&quot;&quot;&quot;
    You are a receipt parsing assistant. Extract all line items from the receipt image.
    For each line item, extract the name, quantity, unit price, and total price.
    Quantity can be a decimal number (e.g. weight in kg like 0.550 or 1.105).
    Extract the subtotal which is the final total amount shown on the receipt.
    IMPORTANT: Read every digit exactly as printed on the receipt.
    Pay very close attention to each decimal digit - do NOT round or approximate.
    For example, if the receipt shows 1.105, report exactly 1.105, not 1.1 or 1.2.
    Verify that quantity * unitPrice = totalPrice for each line item.
    Don't invent items that aren't on the receipt.

    DECIMAL FORMAT: Receipts may use different number formats depending on locale.
    - Some use period as decimal separator: 7,499.00
    - Some use comma as decimal separator: 7.499,00
    First, detect which format the receipt uses by examining the numbers on it.
    Then, always output numbers in the JSON using a period as the decimal separator.
    For example: 7499.00, not 7.499,00 or 7,499.00.
    &quot;&quot;&quot;);
</code></pre>
<p>Every instruction in that prompt exists because the model got something wrong.</p>
<p>&quot;Read every digit exactly as printed&quot;: it was rounding <code>1.105</code> to <code>1.1</code>.</p>
<p>&quot;Don't invent items&quot;: it hallucinated a line item that wasn't on the receipt.</p>
<p>The entire decimal format section: my receipts use commas as decimal separators (European locale),
and the model kept confusing thousands separators with decimal points.</p>
<p>Each prompt iteration is like a debugging session with words instead of code.
It's not the most fun part, but it's necessary to get accurate results.</p>
<p>And to think we used to write code to tell computers what to do.
Now we write prompts to tell models how to think.
I digress...</p>
<h2>Strongly Typed Responses</h2>
<p>This is where <a href="working-with-llms-in-dotnet-using-microsoft-extensions-ai"><code>Microsoft.Extensions.AI</code></a> gets interesting.</p>
<p>Instead of parsing raw JSON strings yourself, you can call <code>GetResponseAsync&lt;T&gt;</code> and get back a <strong>strongly typed</strong> object:</p>
<pre><code class="language-csharp">var response = await chatClient.GetResponseAsync&lt;Receipt&gt;(
    [systemMessage, message],
    new ChatOptions { Temperature = 0 });

if (response.Result is { } receipt)
{
    Console.WriteLine(
        $&quot;\nExtracted {receipt.Items.Count} line items:&quot;);

    foreach (var item in receipt.Items)
    {
        Console.WriteLine(
            $&quot;  {item.Name} - &quot; +
            $&quot;Qty: {item.Quantity} x {item.UnitPrice:C}&quot; +
            $&quot; = {item.TotalPrice:C}&quot;);
    }

    Console.WriteLine($&quot;  Subtotal: {receipt.Subtotal:C}&quot;);
}
</code></pre>
<p>The <code>Receipt</code> and <code>LineItem</code> classes are plain C# objects:</p>
<pre><code class="language-csharp">public class Receipt
{
    public List&lt;LineItem&gt; Items { get; set; } = [];
    public decimal Subtotal { get; set; }
}

public class LineItem
{
    public string Name { get; set; } = string.Empty;
    public decimal Quantity { get; set; }
    public decimal UnitPrice { get; set; }
    public decimal TotalPrice { get; set; }
}
</code></pre>
<p>The library generates the JSON schema, sends it to the model, and deserializes the response.
You get back a <code>Receipt</code> object directly.</p>
<p>I also set <code>Temperature = 0</code> to make the output as deterministic as possible.
For data extraction, you want accuracy.
This isn't foolproof, but it helps.</p>
<p>Here's the object we get back from the model in Visual Studio:</p>
<p><img src="/blogs/mnw_181/chat_client_typed_response_in_vs.png" alt="Visual Studio screenshot showing the Receipt object with line items and subtotal extracted from the model response."></p>
<h2>Testing Consistency</h2>
<p>One thing I wanted to verify: if I send the same receipt with the same prompt five times, do I get the same result?</p>
<pre><code class="language-csharp">const int runs = 5;
Console.WriteLine($&quot;\n--- Consistency test ({runs} runs) ---&quot;);

var results = new List&lt;Receipt&gt;();
for (int i = 0; i &lt; runs; i++)
{
    Console.WriteLine($&quot;\nRun {i + 1}...&quot;);
    var testResponse = await chatClient.GetResponseAsync&lt;Receipt&gt;(
        [systemMessage, message],
        new ChatOptions { Temperature = 0 });

    if (testResponse.Result is { } r)
    {
        results.Add(r);
        Console.WriteLine(
            $&quot;  Items: {r.Items.Count}, &quot; +
            $&quot;Subtotal: {r.Subtotal:C}&quot;);

        foreach (var item in r.Items)
        {
            Console.WriteLine(
                $&quot;    {item.Name} - &quot; +
                $&quot;Qty: {item.Quantity} x {item.UnitPrice:C}&quot; +
                $&quot; = {item.TotalPrice:C}&quot;);
        }
    }
}
</code></pre>
<p>Then I compare every run against the baseline:</p>
<pre><code class="language-csharp">var baseline = results[0];
for (int i = 1; i &lt; results.Count; i++)
{
    bool match = baseline.Subtotal == results[i].Subtotal
        &amp;&amp; baseline.Items.Count == results[i].Items.Count
        &amp;&amp; baseline.Items.Zip(results[i].Items).All(pair =&gt;
            pair.First.Name == pair.Second.Name
            &amp;&amp; pair.First.Quantity == pair.Second.Quantity
            &amp;&amp; pair.First.UnitPrice == pair.Second.UnitPrice
            &amp;&amp; pair.First.TotalPrice == pair.Second.TotalPrice);

    Console.WriteLine(
        $&quot;  Run 1 vs Run {i + 1}: &quot; +
        $&quot;{(match ? &quot;MATCH&quot; : &quot;DIFFERENT&quot;)}&quot;);
}
</code></pre>
<p>Temperature 0 helps, but vision models aren't perfectly deterministic.
Most runs matched. Some didn't.
The differences were usually small - a misread digit, a slightly different item name.</p>
<p>This is worth keeping in mind when working with LLMs.
They're probabilistic systems.
Even with temperature 0, the same input can produce slightly different output.
If you need guaranteed accuracy, you'll want validation layers on top of this.</p>
<h2>Where I Want to Take This</h2>
<p>The receipt scanner is a starting point.
Once you have structured data from receipt images, you can build on top of it.</p>
<p>I've been thinking about extending this into a <strong>personal finance tracker</strong>.
Scan receipts, store the data, and use the same LLM to categorize purchases.
Groceries, household, electronics - let the model figure it out.</p>
<p>From there, you could generate <strong>weekly and monthly spending summaries</strong>.
How much did I spend on groceries this month?
How does that compare to last month?</p>
<p>You could also do <strong>multi-receipt aggregation</strong> for business trips.
Scan a stack of receipts and generate an expense report.</p>
<p>Or <strong>price tracking</strong> over time - detect when items at your usual store go up in price.</p>
<p>We could defintiely build <a href="building-semantic-search-with-amazon-s3-vectors-and-semantic-kernel"><strong>semantic search</strong></a>
on top of this too.
Search through your past receipts for specific items or price ranges.
This works by embedding the structured data and using <a href="what-is-vector-search-a-concise-guide"><strong>vector search</strong></a>
to find relevant entries.</p>
<p>The vision model handles the hard part: turning unstructured images into structured data.
Everything after that is regular application development.</p>
<p>I might build some of this out. We'll see.</p>
<h2>Summary</h2>
<p>Running a vision model locally with Ollama is straightforward to set up.
<code>Microsoft.Extensions.AI</code> and <code>OllamaSharp</code> make the .NET integration clean.
You get a provider-agnostic <code>IChatClient</code> with support for strongly typed responses.
You could also run this with <a href="building-generative-ai-applications-with-github-models-and-dotnet-aspire"><strong>Aspire and GitHub models</strong></a>
if you want to keep everything in the Microsoft ecosystem.</p>
<p>The system prompt is where most of the work happens.
Every line in mine was the result of the model getting something wrong and me adding a corrective instruction.</p>
<p>If you want to try this yourself:</p>
<ol>
<li>Install <a href="https://ollama.com/">Ollama</a></li>
<li>Pull the vision model: <code>ollama pull llama3.2-vision:latest</code></li>
<li>Create a .NET console app and add the <code>OllamaSharp</code> and <code>Microsoft.Extensions.AI</code> NuGet packages</li>
<li>Point it at a receipt and see what comes back</li>
</ol>
<p>Hope this was useful. See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_181.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Integrate Keycloak with ASP.NET Core Using OAuth 2.0]]></title>
            <link>https://milanjovanovic.tech/blog/integrate-keycloak-with-aspnetcore-using-oauth-2</link>
            <guid isPermaLink="false">integrate-keycloak-with-aspnetcore-using-oauth-2</guid>
            <pubDate>Sat, 07 Feb 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to integrate Keycloak with your .NET 10 API using Docker, Swagger UI with OAuth 2.0 Authorization Code flow, and JWT validation.]]></description>
            <content:encoded><![CDATA[<p><strong>Authentication</strong> is one of those things that's easy to get wrong and <strong>expensive to fix later</strong>.
Rolling your own auth system means dealing with password hashing, token management, session handling,
and a never-ending stream of security patches.</p>
<p>I was never a fan of this...</p>
<p>What if you could outsource all of that to a battle-tested <strong>identity provider</strong>?</p>
<p><a href="https://www.keycloak.org/">Keycloak</a> is an open-source identity and access management solution.
It handles user authentication, authorization, and identity brokering (social logins, enterprise SSO) out of the box.
You get a polished admin console, built-in support for <strong>OAuth 2.0</strong> and <strong>OpenID Connect</strong>, and it runs anywhere Docker does.</p>
<p>We'll spin up Keycloak as a container, create a realm with a public client,
and wire up Swagger UI to authenticate using the <a href="https://oauth.net/2/">OAuth 2.0</a> <strong>Authorization Code flow</strong>.
Then we'll add <a href="https://www.rfc-editor.org/rfc/rfc7519.html">JWT</a> validation to our .NET backend and
trace the entire authentication flow using the <a href="standalone-aspire-dashboard-setup-for-distributed-dotnet-applications"><strong>Aspire Dashboard</strong></a>.</p>
<h2>Running Keycloak as a Container</h2>
<p>The fastest way to spin up <strong>Keycloak</strong> is with <a href="https://en.wikipedia.org/wiki/Docker_(software)">Docker</a>.
We'll run it in development mode, which disables HTTPS and uses an embedded H2 database.
This is perfect for local development but <strong>not suitable for production</strong> (more on that later).</p>
<p>Here's a minimal <code>docker-compose.yml</code>:</p>
<pre><code class="language-yaml">services:
  keycloak:
    image: quay.io/keycloak/keycloak:26.5.2
    container_name: keycloak
    environment:
      - KC_BOOTSTRAP_ADMIN_USERNAME=admin
      - KC_BOOTSTRAP_ADMIN_PASSWORD=admin
    ports:
      - '8080:8080'
    command: start-dev
</code></pre>
<p>Start it with:</p>
<pre><code class="language-bash">docker compose up -d
</code></pre>
<p>Once Keycloak is running, navigate to <code>http://localhost:8080</code> and log in with <code>admin</code> / <code>admin</code>.</p>
<p><img src="/blogs/mnw_180/keycloak_admin_login.png" alt="The Keycloak admin login screen with username and password fields."></p>
<p>You should see the Keycloak admin console.</p>
<p><img src="/blogs/mnw_180/keycloak_admin_console.png" alt="The Keycloak admin console showing the master realm dashboard."></p>
<h2>Setting Up a Realm and Client</h2>
<p>Keycloak organizes everything into <strong>realms</strong>.
A realm is a space where you manage users, roles, and applications.
The <code>master</code> realm is reserved for Keycloak administration, so we'll create a new one for our application.</p>
<h3>Creating a Realm</h3>
<ol>
<li>Click the <strong>Manage Realms</strong> button in the top-left corner</li>
<li>Click <strong>Create realm</strong></li>
<li>Enter a name (e.g., <code>keycloak-demo</code>) and click <strong>Create</strong></li>
</ol>
<p><img src="/blogs/mnw_180/keycloak_create_realm_dialog.png" alt="The Keycloak create realm dialog with "></p>
<h3>Creating a Public Client</h3>
<p>Now we need to register our application.
In OAuth 2.0 terms, this is a <strong>client</strong>.
Since Swagger UI runs in the browser, we'll create a <strong>public client</strong> (no client secret).</p>
<ol>
<li>Go to <strong>Clients</strong> → <strong>Create client</strong></li>
<li>Set <strong>Client ID</strong> to <code>demo-api</code></li>
<li>Leave <strong>Client type</strong> as <code>OpenID Connect</code></li>
<li>Click <strong>Next</strong></li>
</ol>
<p><img src="/blogs/mnw_180/keycloak_create_client_step1.png" alt="The first step of creating a client in Keycloak, showing the Client ID field."></p>
<ol start="5">
<li>Enable <strong>Client authentication</strong>: Off (public client)</li>
<li>Check <strong>Standard flow</strong> (Authorization Code)</li>
<li>Choose <strong>PKCE Method</strong>: S256 (SHA-256)</li>
<li>Click <strong>Next</strong></li>
</ol>
<p><img src="/blogs/mnw_180/keycloak_create_client_step2.png" alt="The second step of creating a client showing authentication settings."></p>
<ol start="9">
<li>Configure the redirect URIs:
<ul>
<li><strong>Valid redirect URIs</strong>: <code>https://localhost:5001/*</code> (your API's Swagger URL)</li>
<li><strong>Web origins</strong>: <code>https://localhost:5001</code></li>
</ul>
</li>
<li>Click <strong>Save</strong></li>
</ol>
<p><img src="/blogs/mnw_180/keycloak_create_client_step3.png" alt="The third step showing redirect URI configuration."></p>
<h3>Creating a Test User</h3>
<p>We need a user to authenticate with.</p>
<ol>
<li>Go to <strong>Users</strong> → <strong>Add user</strong></li>
<li>Fill in the details (username, email, etc.)</li>
<li>Leave <strong>Email Verified</strong> checked to avoid email confirmation</li>
<li>Click <strong>Create</strong></li>
<li>Go to the <strong>Credentials</strong> tab</li>
<li>Click <strong>Set password</strong> and create a password (disable &quot;Temporary&quot;)</li>
</ol>
<p><img src="/blogs/mnw_180/keycloak_create_user.png" alt="The Keycloak user creation form."></p>
<p>You're now ready to authenticate users against Keycloak!</p>
<h2>The Authorization Code Flow</h2>
<p>Before we dive into code, let's understand what happens when a user authenticates.
The <a href="https://www.rfc-editor.org/rfc/rfc6749#section-4.1"><strong>Authorization Code flow</strong></a> is the recommended OAuth 2.0 flow for browser-based applications.</p>
<p>There's an important security enhancement called <strong>PKCE</strong> (<a href="https://www.rfc-editor.org/rfc/rfc7636">Proof Key for Code Exchange</a>)
that prevents authorization code interception attacks.
It works by having the client generate a random secret (the code verifier) and deriving a hash (the code challenge) sent in the initial authorization request.
When exchanging the authorization code for tokens, the client must present the original code verifier.</p>
<p>Here's the sequence:</p>
<p><img src="/blogs/mnw_180/authorization_code_flow.png" alt="A sequence diagram showing the OAuth 2.0 Authorization Code flow between Browser, API, and Keycloak."></p>
<ol>
<li><strong>User clicks &quot;Authorize&quot;</strong> in Swagger UI</li>
<li><strong>Browser redirects</strong> to Keycloak's authorization endpoint</li>
<li><strong>User logs in</strong> at Keycloak</li>
<li><strong>Keycloak redirects back</strong> with an authorization code</li>
<li><strong>Swagger UI exchanges</strong> the code for tokens (access token, refresh token, ID token)</li>
<li><strong>Swagger UI attaches</strong> the access token to API requests</li>
<li><strong>API validates</strong> the token signature and claims</li>
</ol>
<p>The beauty of this flow is that credentials never touch your application.
The user authenticates directly with Keycloak, and your API only sees signed tokens.</p>
<h2>Configuring Swagger UI with OAuth 2.0</h2>
<p>Now let's set up our .NET API to use Swagger UI as our OAuth 2.0 test client.</p>
<p>First, install the required packages:</p>
<pre><code class="language-bash">dotnet add package Swashbuckle.AspNetCore
</code></pre>
<p>Configure Swagger in your <code>Program.cs</code>:</p>
<pre><code class="language-csharp">var keycloakAuthority = builder.Configuration[&quot;Keycloak:Authority&quot;]!;
var keycloakClientId = builder.Configuration[&quot;Keycloak:ClientId&quot;]!;

builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen(options =&gt;
{
    options.SwaggerDoc(&quot;v1&quot;, new OpenApiInfo
    {
        Title = &quot;Demo API&quot;,
        Version = &quot;v1&quot;
    });

    // Define the OAuth 2.0 security scheme
    options.AddSecurityDefinition(nameof(SecuritySchemeType.OAuth2), new OpenApiSecurityScheme
    {
        Type = SecuritySchemeType.OAuth2,
        Flows = new OpenApiOAuthFlows
        {
            AuthorizationCode = new OpenApiOAuthFlow
            {
                AuthorizationUrl = new Uri($&quot;{keycloakAuthority}/protocol/openid-connect/auth&quot;),
                TokenUrl = new Uri($&quot;{keycloakAuthority}/protocol/openid-connect/token&quot;),
                Scopes = new Dictionary&lt;string, string&gt;
                {
                    { &quot;openid&quot;, &quot;OpenID Connect scope&quot; },
                    { &quot;profile&quot;, &quot;User profile&quot; }
                }
            }
        }
    });

    // Apply security to all operations
    options.AddSecurityRequirement(doc =&gt; new OpenApiSecurityRequirement
    {
        {
            new OpenApiSecuritySchemeReference(nameof(SecuritySchemeType.OAuth2), doc),
            []
        }
    });
});
</code></pre>
<p>And configure the Swagger UI middleware:</p>
<pre><code class="language-csharp">if (app.Environment.IsDevelopment())
{
    app.UseSwagger();
    app.UseSwaggerUI(options =&gt;
    {
        options.OAuthClientId(keycloakClientId); // Default Client ID
        options.OAuthUsePkce(); // Proof Key for Code Exchange (security enhancement)
    });
}
</code></pre>
<p>Your <code>appsettings.Development.json</code>:</p>
<pre><code class="language-json">{
  &quot;Keycloak&quot;: {
    &quot;Authority&quot;: &quot;http://localhost:8080/realms/keycloak-demo&quot;,
    &quot;ClientId&quot;: &quot;demo-api&quot;,
    &quot;Audience&quot;: &quot;account&quot;,
    &quot;Issuer&quot;: &quot;http://localhost:8080/realms/keycloak-demo&quot;,
    // Here we use the Docker service name for Keycloak
    &quot;MetadataAddress&quot;: &quot;http://keycloak:8080/realms/keycloak-demo/.well-known/openid-configuration&quot;
  }
}
</code></pre>
<p>Now when you open Swagger UI, you'll see an <strong>Authorize</strong> button.
Clicking it opens the OAuth flow, redirecting you to Keycloak to log in.</p>
<p><img src="/blogs/mnw_180/swagger_authorize_form.png" alt="Swagger UI showing the Authorize form for OAuth 2.0."></p>
<h2>Adding JWT Validation</h2>
<p>At this point, Swagger UI can obtain tokens, but our API isn't validating them yet.
Let's add JWT Bearer authentication.</p>
<p>Install the authentication package:</p>
<pre><code class="language-bash">dotnet add package Microsoft.AspNetCore.Authentication.JwtBearer
</code></pre>
<p>Configure authentication in <code>Program.cs</code>:</p>
<pre><code class="language-csharp">builder.Services.AddAuthentication(JwtBearerDefaults.AuthenticationScheme)
    .AddJwtBearer(options =&gt;
    {
        options.MetadataAddress = builder.Configuration[&quot;Keycloak:MetadataAddress&quot;]!;
        options.Audience = builder.Configuration[&quot;Keycloak:Audience&quot;];

        options.TokenValidationParameters = new TokenValidationParameters
        {
            ValidIssuer = builder.Configuration[&quot;Keycloak:Issuer&quot;]
        };

        // Required for HTTP in development (Keycloak uses HTTP by default in dev mode)
        options.RequireHttpsMetadata = !builder.Environment.IsDevelopment();
    });

builder.Services.AddAuthorization();
</code></pre>
<p>The default <code>TokenValidationParameters</code> will validate the token signature, expiration, issuer, and audience.</p>
<p>Add the middleware:</p>
<pre><code class="language-csharp">app.UseAuthentication();
app.UseAuthorization();
</code></pre>
<p>Create a protected endpoint:</p>
<pre><code class="language-csharp">app.MapGet(&quot;users/me&quot;, (ClaimsPrincipal user) =&gt;
{
    return Results.Ok(new
    {
        UserId = user.FindFirstValue(ClaimTypes.NameIdentifier),
        Email = user.FindFirstValue(ClaimTypes.Email),
        Name = user.FindFirstValue(&quot;preferred_username&quot;),
        Claims = user.Claims.Select(c =&gt; new { c.Type, c.Value })
    });
})
.RequireAuthorization();
</code></pre>
<h2>How JWT Validation Works</h2>
<p>When a request hits your protected endpoint, here's what happens under the hood:</p>
<p><img src="/blogs/mnw_180/jwt_validation_flow.png" alt="A sequence diagram showing how JWT validation works in ASP.NET Core."></p>
<ol>
<li><strong>Middleware extracts</strong> the <code>Authorization: Bearer &lt;token&gt;</code> header</li>
<li><strong>JWT Handler fetches</strong> Keycloak's public keys from the JWKS endpoint (cached)</li>
<li><strong>Signature validation</strong> proves the token wasn't tampered with</li>
<li><strong>Claims are extracted</strong> and the <code>ClaimsPrincipal</code> is populated</li>
<li><strong>Authorization middleware</strong> checks if the user meets the endpoint requirements</li>
<li><strong>Endpoint executes</strong> with access to <code>HttpContext.User</code></li>
</ol>
<p>The key insight here is that your API <strong>never contacts Keycloak</strong> to validate individual tokens.
It fetches the signing keys once and validates tokens locally.
This is what makes JWT-based authentication so fast.</p>
<h2>Observing the Flow with Aspire Dashboard</h2>
<p>If you're using <a href="https://aspire.dev"><strong>Aspire</strong></a>, you can observe the entire authentication flow in the distributed traces.</p>
<p>Here's what a successful authentication looks like:</p>
<p><img src="/blogs/mnw_180/aspire_auth_trace.png" alt="Aspire Dashboard showing a distributed trace of the authentication flow."></p>
<p>You can see:</p>
<ol>
<li>The initial request to <code>users/me</code> (with the Bearer token)</li>
<li>The outbound call to Keycloak's <code>.well-known/openid-configuration</code> endpoint</li>
<li>The outbound call to Keycloak's JWKS endpoint (fetching signing keys)</li>
<li>The response back to the client</li>
</ol>
<p>On subsequent requests, you won't see the JWKS call because the keys are cached.</p>
<p>This is why JWT validation adds virtually no latency after the initial key fetch.</p>
<h2>Production Considerations</h2>
<p>What we've built is great for development.
For production, you'll want to address a few things:</p>
<p><strong>1. HTTPS Everywhere</strong></p>
<p>Keycloak should run behind HTTPS.
Set <code>KC_HOSTNAME</code> and configure TLS certificates.</p>
<p><strong>2. Persistent Storage</strong></p>
<p>Replace the embedded H2 database with PostgreSQL or MySQL:</p>
<pre><code class="language-yaml">environment:
  - KC_DB=postgres
  - KC_DB_URL=jdbc:postgresql://postgres:5432/keycloak
  - KC_DB_USERNAME=keycloak
  - KC_DB_PASSWORD=secret
</code></pre>
<p><strong>3. Require HTTPS Metadata</strong></p>
<p>Remove <code>options.RequireHttpsMetadata = false</code> in production.</p>
<h2>Summary</h2>
<p>In about 10 minutes, we've set up:</p>
<ul>
<li>A <a href="containerize-your-dotnet-applications-without-a-dockerfile"><strong>containerized</strong></a> Keycloak instance</li>
<li>A realm with a public OAuth 2.0 client</li>
<li>Swagger UI acting as an OAuth client with Authorization Code + PKCE</li>
<li>JWT validation in <a href="http://ASP.NET">ASP.NET</a> Core</li>
<li>Observability with <a href="introduction-to-distributed-tracing-with-opentelemetry-in-dotnet"><strong>OpenTelemetry</strong></a> into the authentication flow</li>
</ul>
<p>What I really like about Keycloak is how easy it is to extend.
Want Google login? Configure it in Keycloak.
Need enterprise SSO? Add a SAML provider.
Your API code stays exactly the same because it just validates tokens.</p>
<p>If you want to see how I integrate Keycloak in a real-world system with role-based access control,
check out <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture</strong></a> and
<a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a>.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_180.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Containerize Your .NET Applications Without a Dockerfile]]></title>
            <link>https://milanjovanovic.tech/blog/containerize-your-dotnet-applications-without-a-dockerfile</link>
            <guid isPermaLink="false">containerize-your-dotnet-applications-without-a-dockerfile</guid>
            <pubDate>Sat, 31 Jan 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to containerize .NET applications without writing a single line of Dockerfile. The .NET SDK has built-in support for publishing directly to container images.]]></description>
            <content:encoded><![CDATA[<p>Containers have become the standard for deploying modern applications.
But if you've ever written a <a href="https://docs.docker.com/reference/dockerfile/">Dockerfile</a>, you know it can be tedious.
You need to understand multi-stage builds, pick the right base images, configure the right ports, and remember to copy files in the correct order.</p>
<p>What if I told you that <strong>you don't need a Dockerfile at all</strong>?</p>
<p>Since .NET 7, the SDK has built-in support for publishing your application directly to a container image.
You can do this with a single <code>dotnet publish</code> command.</p>
<p>In this week's newsletter, we'll explore:</p>
<ul>
<li>Why Dockerfile-less publishing matters</li>
<li>How to enable container publishing in your project</li>
<li>Customizing the container image</li>
<li>Publishing to container registries</li>
<li><strong>How I'm using this to deploy to a VPS</strong></li>
</ul>
<h2>The Traditional Approach: Writing a Dockerfile</h2>
<p>Before we look at the SDK approach, let's see what we're replacing.</p>
<p>A typical multi-stage Dockerfile for a .NET application looks like this:</p>
<pre><code class="language-bash">FROM mcr.microsoft.com/dotnet/aspnet:10.0 AS base
WORKDIR /app
EXPOSE 8080
EXPOSE 8081

FROM mcr.microsoft.com/dotnet/sdk:10.0 AS build
ARG BUILD_CONFIGURATION=Release
WORKDIR /src
COPY [&quot;src/MyApi/MyApi.csproj&quot;, &quot;src/MyApi/&quot;]
RUN dotnet restore &quot;src/MyApi/MyApi.csproj&quot;

COPY . .
WORKDIR &quot;/src/src/MyApi&quot;
RUN dotnet build &quot;MyApi.csproj&quot; -c $BUILD_CONFIGURATION  -o /app/build

FROM build AS publish
ARG BUILD_CONFIGURATION=Release
RUN dotnet publish &quot;MyApi.csproj&quot; -c $BUILD_CONFIGURATION -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT [&quot;dotnet&quot;, &quot;MyApi.dll&quot;]
</code></pre>
<p>This works, but there's a learning curve and maintenance overhead:</p>
<ul>
<li><strong>Maintenance burden</strong>: You need to update base image tags manually</li>
<li><strong>Layer caching</strong>: Getting the COPY order wrong kills your build cache</li>
<li><strong>Duplication</strong>: Every project needs a similar Dockerfile</li>
<li><strong>Context switching</strong>: You're writing Docker DSL, not .NET code</li>
</ul>
<p>The .NET SDK approach eliminates all of this.</p>
<h2>Enabling Container Publishing</h2>
<p>If you're running on .NET 10, you don't need to do anything special to enable container publishing.
This will work for <a href="http://ASP.NET">ASP.NET</a> Core apps, worker services, and console apps.</p>
<p>You can publish directly to a container image:</p>
<pre><code class="language-bash">dotnet publish --os linux --arch x64 /t:PublishContainer
</code></pre>
<p>That's it. The .NET SDK will:</p>
<ol>
<li>Build your application</li>
<li>Select the appropriate base image</li>
<li>Create a container image with your published output</li>
<li>Load it into your local OCI-compliant daemon</li>
</ol>
<p>The most popular option is Docker, but it also works with Podman.</p>
<p><img src="/blogs/mnw_179/dotnet_publish_container.png" alt="An image showing the output of the dotnet publish command creating a container image."></p>
<h2>Customizing the Container Image</h2>
<p>The SDK provides sensible defaults, but you'll often want to customize the image.
For a more comprehensive list of options, see the <a href="https://learn.microsoft.com/en-us/dotnet/core/containers/publish-configuration">official docs</a>.</p>
<p>I'll cover the most common customizations here.</p>
<h3>Setting the Image Name and Tag</h3>
<p>The <code>ContainerRepository</code> property sets the image name (repository).
The <code>ContainerImageTags</code> property sets one or more tags (separated by semicolons).
If you want a single tag, you can use <code>ContainerImageTag</code> instead.</p>
<pre><code class="language-xml">&lt;PropertyGroup&gt;
  &lt;ContainerRepository&gt;ghcr.io/USERNAME/REPOSITORY&lt;/ContainerRepository&gt;
  &lt;ContainerImageTags&gt;1.0.0;latest&lt;/ContainerImageTags&gt;
&lt;/PropertyGroup&gt;
</code></pre>
<p>From .NET 8 and onwards, when a tag isn't provided the default is <code>latest</code>.</p>
<h3>Choosing a Different Base Image</h3>
<p>By default, the SDK uses the following base images:</p>
<ul>
<li><code>mcr.microsoft.com/dotnet/runtime-deps</code> for self-contained apps</li>
<li><code>mcr.microsoft.com/dotnet/aspnet</code> image for <a href="http://ASP.NET">ASP.NET</a> Core apps</li>
<li><code>mcr.microsoft.com/dotnet/runtime</code> for other cases</li>
</ul>
<p>You can switch to a smaller or different image:</p>
<pre><code class="language-xml">&lt;PropertyGroup&gt;
  &lt;!-- Use the Alpine-based image for smaller size --&gt;
  &lt;ContainerBaseImage&gt;mcr.microsoft.com/dotnet/aspnet:10.0-alpine&lt;/ContainerBaseImage&gt;
&lt;/PropertyGroup&gt;
</code></pre>
<p>You could also do this by setting <code>ContainerFamily</code> to <code>alpine</code>, and letting the rest be inferred.</p>
<p>Here's the size difference between the default and Alpine images for an <a href="http://ASP.NET">ASP.NET</a> Core app:</p>
<p><img src="/blogs/mnw_179/container_image_size.png" alt="An image showing the size difference between the default and Alpine base images for ASP.NET Core applications."></p>
<pre><code class="language-text">| Base Image                                  | Size (MB) |
| ------------------------------------------- | --------- |
| mcr.microsoft.com/dotnet/aspnet:10.0        | 231.73    |
| mcr.microsoft.com/dotnet/aspnet:10.0-alpine | 122.65    |
</code></pre>
<p>You can see a significant size reduction by switching to <code>alpine</code>.</p>
<h3>Configuring Ports</h3>
<p>For web applications, the default exposed ports are <code>8080</code> and <code>8081</code> for HTTP and HTTPS.
These are inferred from <a href="http://ASP.NET">ASP.NET</a> Core environment variables (<code>ASPNETCORE_URLS</code>, <code>ASPNETCORE_HTTP_PORT</code>, <code>ASPNETCORE_HTTPS_PORT</code>).
The <code>Type</code> attribute can be <code>tcp</code> or <code>udp</code>.</p>
<pre><code class="language-xml">&lt;ItemGroup&gt;
  &lt;ContainerPort Include=&quot;8080&quot; Type=&quot;tcp&quot; /&gt;
  &lt;ContainerPort Include=&quot;8081&quot; Type=&quot;tcp&quot; /&gt;
&lt;/ItemGroup&gt;
</code></pre>
<h2>Publishing to a Container Registry</h2>
<p>Publishing locally is useful for development, but you'll want to push to a registry for deployment.
You can specify the target registry during publishing.</p>
<p>Here's an example publishing to GitHub Container Registry:</p>
<pre><code class="language-bash">dotnet publish --os linux --arch x64  /t:PublishContainer /p:ContainerRegistry=ghcr.io
</code></pre>
<p><strong>Authentication</strong>: The SDK uses your local Docker credentials.
Make sure you've logged in with <code>docker login</code> before publishing to a remote registry.</p>
<p>However, <strong>I don't use the above approach</strong>.
I prefer using docker CLI for the publishing step, as it gives me more control over authentication and tagging.</p>
<h2>CI/CD Integration</h2>
<p>Here's what I'm doing in my <a href="how-to-build-ci-cd-pipeline-with-github-actions-and-dotnet"><strong>GitHub Actions workflow</strong></a> to build and push my .NET app container.
I left out the boring bits of seting up the .NET environment and checking out code.</p>
<p>This will build the container image, tag it, and push it to <a href="https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry">GitHub Container Registry</a>:</p>
<pre><code class="language-yaml">- name: Publish
  run: dotnet publish &quot;${{ env.WORKING_DIRECTORY }}&quot; --configuration ${{ env.CONFIGURATION }} --os linux -t:PublishContainer
# Tag the build for later steps
- name: Log in to ghcr.io
  run: echo &quot;${{ env.DOCKER_PASSWORD }}&quot; | docker login ghcr.io -u &quot;${{ env.DOCKER_USERNAME }}&quot; --password-stdin
- name: Tag Docker image
  run:
    docker tag ${{ env.IMAGE_NAME }}:${{ github.sha }} ghcr.io/${{ env.DOCKER_USERNAME }}/${{ env.IMAGE_NAME }}:${{ github.sha }} |
    docker tag ${{ env.IMAGE_NAME }}:latest ghcr.io/${{ env.DOCKER_USERNAME }}/${{ env.IMAGE_NAME }}:latest
- name: Push Docker image
  run:
    docker push ghcr.io/${{ env.DOCKER_USERNAME }}/${{ env.IMAGE_NAME }}:${{ github.sha }} |
    docker push ghcr.io/${{ env.DOCKER_USERNAME }}/${{ env.IMAGE_NAME }}:latest
</code></pre>
<p>Once my images are up in the registry, I can deploy them to my VPS.</p>
<p>I'm using <a href="https://dokploy.com/">Dokploy</a> (a simple but powerful deployment tool for Docker apps) to pull the latest image and restart my service.</p>
<pre><code class="language-yaml">deploy:
  runs-on: ubuntu-latest
  needs: build-and-publish
  steps:
    - name: Trigger deployment
      run: |
        curl -X POST ${{ env.DEPLOYMENT_TRIGGER_URL }} \
          -H 'accept: application/json' \
          -H 'Content-Type: application/json' \
          -H 'x-api-key: ${{ env.DEPLOYMENT_TRIGGER_API_KEY }}' \
          -d '{
            &quot;applicationId&quot;: &quot;${{ env.DEPLOYMENT_TRIGGER_APP_ID }}&quot;
          }'
</code></pre>
<p>This kicks off a deployment on my VPS, pulling the latest image and restarting the container.</p>
<p><img src="/blogs/mnw_179/dokploy_deployment.png" alt="An image showing the output of the dokploy deployment command restarting the container."></p>
<p>By the way, I'm running my VPS on <a href="https://www.hetzner.com/cloud">Hetzner Cloud</a> - highly recommended if you're looking for affordable and reliable VPS hosting.</p>
<h2>When You Still Need a Dockerfile</h2>
<p>The SDK container support is powerful, but it doesn't cover every scenario.</p>
<p>You'll still need a Dockerfile when:</p>
<ul>
<li><strong>Installing system dependencies</strong>: If your app needs native libraries (like <code>libgdiplus</code> for image processing)</li>
<li><strong>Complex multi-stage builds</strong>: When you need to run custom build steps</li>
<li><strong>Non-.NET components</strong>: If your container needs additional services or tools</li>
</ul>
<p>For most web APIs and background services, the SDK approach is sufficient.</p>
<h2>Summary</h2>
<p>The .NET SDK's built-in container support removes the friction of containerization.</p>
<p>You get:</p>
<ul>
<li><strong>No Dockerfile to maintain</strong> - one less file to worry about</li>
<li><strong>Automatic base image selection</strong> - always uses the right image for your framework version</li>
<li><strong>MSBuild integration</strong> - configure everything in your <code>.csproj</code></li>
<li><strong>CI/CD friendly</strong> - works anywhere <code>dotnet</code> runs</li>
</ul>
<p>The days of copy-pasting Dockerfiles between projects are over.</p>
<p>Just enable the feature, customize what you need, and publish.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_179.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[A Practical Demo of Zero-Downtime Migrations Using Password Hashing]]></title>
            <link>https://milanjovanovic.tech/blog/a-practical-demo-of-zero-downtime-migrations-using-password-hashing</link>
            <guid isPermaLink="false">a-practical-demo-of-zero-downtime-migrations-using-password-hashing</guid>
            <pubDate>Sat, 24 Jan 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[A practical demo of the zero-downtime migration pattern: run old + new password hash formats side-by-side, migrate users on login, and remove legacy safely.]]></description>
            <content:encoded><![CDATA[<p>Security requirements evolve.
What was considered &quot;secure enough&quot; five years ago might not pass a security audit today.</p>
<p>You need to upgrade to a modern algorithm like <a href="https://en.wikipedia.org/wiki/Argon2">Argon2</a> or <a href="https://en.wikipedia.org/wiki/Bcrypt">Bcrypt</a>.
But here is the problem: hashing is a <strong>one-way operation</strong>.
You cannot reverse-engineer the existing hashes to &quot;upgrade&quot; them.</p>
<p>If you simply swap your <code>IPasswordHasher</code> implementation, you break the application.
Every single existing user who tries to log in will fail authentication because your new hasher doesn't understand the old format.</p>
<p>In this article, I want to demo a <strong>zero-downtime migration</strong> concept in practice.</p>
<p>Real systems have more constraints (and you should not build auth from scratch).
But this is a clean example of a <strong>pattern</strong> you can reuse for <a href="efcore-migrations-a-detailed-guide"><strong>database migrations</strong></a>:</p>
<ul>
<li>Move from old format to new format</li>
<li>Keep existing behavior working</li>
<li>Gradually migrate data</li>
<li>Delete legacy only when you are done</li>
</ul>
<p>Let's dive in.</p>
<h2>The Naive Approach and Why It Fails</h2>
<p>Let's imagine you have a simple authentication system.
You want to replace your legacy <code>PBKDF2</code> hasher with a standard <code>Argon2</code> implementation.</p>
<p>You might think, &quot;I'll just register the new implementation in the dependency injection container.&quot;</p>
<pre><code class="language-csharp">// Switching from LegacyHasher to ModernHasher
builder.Services.AddSingleton&lt;IPasswordHasher, ModernHasher&gt;();
</code></pre>
<p>Here is the failure scenario:</p>
<ol>
<li><strong>New Users:</strong> They register and log in perfectly.
Their passwords are hashed with Argon2 from day one.</li>
<li><strong>Existing Users:</strong> A user enters their correct password.
The system fetches the <em>old</em> <a href="https://en.wikipedia.org/wiki/PBKDF2">PBKDF2</a> hash from the database.</li>
<li><strong>The Crash:</strong> The <code>ModernHasher</code> tries to verify the PBKDF2 hash.
It fails immediately, returning <code>401 Unauthorized</code>.</li>
</ol>
<p>You have inadvertently locked out your entire user base.
We need a way to support <em>both</em> algorithms simultaneously without making the login code a mess.</p>
<h2>The Solution: Migration on Login</h2>
<p>The strategy is simple: we don't migrate the database in a batch job.
We migrate users lazily when they prove their identity.</p>
<p>The flow looks like this:</p>
<ol>
<li><strong>Attempt 1:</strong> Try to verify the password using the <strong>New</strong> algorithm.</li>
<li><strong>Attempt 2 (Fallback):</strong> If that fails, check if the <strong>Legacy</strong> algorithm can verify it.</li>
<li><strong>The Migration:</strong> If the <em>Legacy</em> verification succeeds:
<ul>
<li>Log the user in (Success).</li>
<li><strong>Immediately re-hash</strong> their password using the <strong>New</strong> algorithm.</li>
<li>Update the database record.</li>
</ul>
</li>
</ol>
<p>Future logins for this user will now succeed via the standard flow.</p>
<p><img src="/blogs/mnw_178/migration_on_login_flow.png" alt="A sequence diagram showing the migration on login flow, with attempts to verify using new and legacy hashers."></p>
<h2>Implementation with .NET Keyed Services</h2>
<p>In .NET 8, Microsoft introduced <strong>Keyed Services</strong>, which are perfect for this scenario.
They allow us to register multiple implementations of the same interface and retrieve them by name.</p>
<h3>1. Registering the Services</h3>
<p>We register both hashers in our <code>Program.cs</code>, assigning them unique keys:</p>
<pre><code class="language-csharp">// Register the implementations with specific keys
builder.Services.AddKeyedSingleton&lt;IPasswordHasher, Pbdkf2PasswordHasher&gt;(&quot;legacy&quot;);
builder.Services.AddKeyedSingleton&lt;IPasswordHasher, Argon2PasswordHasher&gt;(&quot;modern&quot;);

// (Optional) Register the modern one as the default for other services
builder.Services.AddSingleton&lt;IPasswordHasher, Argon2PasswordHasher&gt;();
</code></pre>
<h3>2. The Login Command Handler</h3>
<p>Now we implement the migration logic.
We inject both hashers using the <code>[FromKeyedServices]</code> attribute.</p>
<pre><code class="language-csharp">public class LoginCommandHandler(
    IUserRepository userRepository,
    [FromKeyedServices(&quot;modern&quot;)] IPasswordHasher newHasher,
    [FromKeyedServices(&quot;legacy&quot;)] IPasswordHasher legacyHasher)
{
    public async Task&lt;AuthenticationResult&gt; Handle(LoginCommand command)
    {
        var user = await userRepository.GetByEmailAsync(command.Email);
        if (user is null)
        {
            return AuthenticationResult.Fail();
        }

        // 1. Try the new algorithm first (Happy Path)
        if (newHasher.Verify(user.PasswordHash, command.Password))
        {
            return AuthenticationResult.Success(user);
        }

        // 2. Fallback: Check if it's a legacy hash
        if (legacyHasher.Verify(user.PasswordHash, command.Password))
        {
            // 3. MIGRATION STEP: Re-hash and save
            var newHash = newHasher.Hash(command.Password);

            user.UpdatePasswordHash(newHash);
            await userRepository.SaveChangesAsync();

            return AuthenticationResult.Success(user);
        }

        return AuthenticationResult.Fail();
    }
}

</code></pre>
<p>This code ensures that active users are automatically upgraded.
After a few months, the vast majority of your user base will be on the new algorithm.</p>
<h2>Real-World Improvements</h2>
<p>While the implementation above works, here are two improvements to make it production-ready.</p>
<h3>1. Algorithm Prefixes</h3>
<p>Relying on &quot;trial and error&quot; verification works, but it's cleaner to know exactly which algorithm was used to create a hash.</p>
<p>Standard algorithms often include a prefix (e.g., Bcrypt starts with <code>$2a$</code> or <code>$2b$</code>).
You can use this to route the request efficiently:</p>
<pre><code class="language-csharp">public bool IsLegacyHash(string hash)
{
    // This assumes we're storing a prefix for PBKDF2 hashes. Something to consider.
    return hash.StartsWith(&quot;pbkdf2$&quot;);
}
</code></pre>
<p>Another benefit this unlocks is being able to query the database for users still on the legacy format.</p>
<h3>2. Feature Flags</h3>
<p>Performing a database write during a login request adds latency.
If you have high traffic, you might want to control this roll-out.</p>
<p>By wrapping the migration logic behind a <a href="feature-flags-in-dotnet-and-how-i-use-them-for-ab-testing"><strong>Feature Flag</strong></a>, you can disable the &quot;write&quot; step if your database comes under load,
while still allowing users to log in via the read-only fallback.</p>
<pre><code class="language-csharp">if (await featureManager.IsEnabledAsync(FeatureFlags.MigratePasswords) &amp;&amp;
    legacyHasher.Verify(user.PasswordHash, command.Password))
{
    // Perform migration...
}
</code></pre>
<h2>Finishing the Migration</h2>
<p>After you run this for a while (usually a few months), most active accounts will be upgraded.
You can then run a cleanup script to identify any remaining legacy hashes and force those users to reset their passwords on the next login attempt.</p>
<p>At that point, you can remove:</p>
<ul>
<li>The legacy hasher registration</li>
<li>The legacy verification code path</li>
<li>The feature flag</li>
</ul>
<p>And the migration is complete.</p>
<h2>Summary</h2>
<p>A &quot;simple&quot; hashing upgrade is really a <strong>data migration</strong>.
This article is about the migration pattern.
Not about reinventing auth.</p>
<p>The <strong>zero-downtime pattern</strong> looks like this:</p>
<ol>
<li>New format for new writes</li>
<li>Support both formats for reads</li>
<li>Migrate old data gradually (migrate-on-login is a great trick)</li>
<li>Put it behind a feature flag</li>
<li>Delete legacy when you are done</li>
</ol>
<p>By allowing the old and new formats to coexist for a period of time, you achieve a seamless transition.
Once your monitoring shows that 99% of active users have migrated,
you can identify the users on the legacy format and force a password reset on their next attempt.</p>
<p>If you want to see a practical demo of this, <a href="https://youtu.be/7YUV4O9eMjQ"><strong>check out this video I made</strong></a>.</p>
<p>Hope this was helpful!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_178.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Solving the Distributed Cache Invalidation Problem with Redis and HybridCache]]></title>
            <link>https://milanjovanovic.tech/blog/solving-the-distributed-cache-invalidation-problem-with-redis-and-hybridcache</link>
            <guid isPermaLink="false">solving-the-distributed-cache-invalidation-problem-with-redis-and-hybridcache</guid>
            <pubDate>Sat, 17 Jan 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to solve the distributed cache invalidation problem in .NET 9 by implementing a Redis Pub/Sub backplane to synchronize HybridCache instances across multiple nodes.]]></description>
            <content:encoded><![CDATA[<p>Distributed systems are great for scalability, but they introduce a whole new class of problems.
One of the hardest problems to solve is <strong>cache invalidation</strong>.</p>
<p>In .NET 9, <a href="hybrid-cache-in-aspnetcore-new-caching-library"><strong>Microsoft introduced <code>HybridCache</code></strong></a> to simplify caching.
It's a fantastic library that combines the speed of in-memory caching (L1) with the durability of distributed caching (L2) like Redis.
It also handles &quot;cache stampede&quot; protection out of the box.</p>
<p>However, there is a catch.</p>
<p>When you run multiple instances of your application, <code>HybridCache</code> doesn't automatically synchronize the local L1 cache across all nodes.
If you update data on <strong>Node A</strong>, <strong>Node B</strong> will continue serving stale data from its in-memory cache until the entry expires.</p>
<p>While HybridCache is a massive step forward, the lack of a built-in backplane for invalidation is a known limitation.
In fact, there is an active discussion on the <a href="https://github.com/dotnet/extensions/issues/5517">dotnet/extensions GitHub repository</a> tracking this exact feature request.
Until that ships, we have to roll our own solution.</p>
<p>In this week's newsletter, we'll explore:</p>
<ul>
<li>The distributed caching dilemma</li>
<li>Why <code>HybridCache</code> doesn't solve this alone</li>
<li>Using Redis Pub/Sub as a backplane</li>
<li>Implementing real-time cache invalidation</li>
</ul>
<p>Let's dive in.</p>
<h2>The Distributed Caching Dilemma</h2>
<p>Let's imagine a typical production scenario.
You have an API running on multiple servers (or pods) behind a load balancer.</p>
<p>To improve performance, you introduce <a href="caching-in-aspnetcore-improving-application-performance"><strong>caching</strong></a>.
You want the speed of local memory, so you use <code>HybridCache</code>.</p>
<p>Here is the failure scenario:</p>
<ol>
<li><strong>User A</strong> updates their profile on <strong>Server 1</strong>.</li>
<li><strong>Server 1</strong> updates the database and clears its local cache.</li>
<li><strong>User A</strong> (or User B) hits <strong>Server 2</strong>.</li>
<li><strong>Server 2</strong> still holds the <em>old</em> profile data in its local <code>HybridCache</code>.</li>
<li>The user sees outdated information, since the local cache hasn't been invalidated.</li>
</ol>
<p><img src="/blogs/mnw_177/hybridcache_out_of_sync_scenario.png" alt="A sequence diagram showing two servers with HybridCache out of sync after a user update."></p>
<p><strong>Why Not Just Shorten the Cache Duration?</strong></p>
<p>A common &quot;hack&quot; to solve this is to simply reduce the L1 cache duration (TTL).
For example, setting the local cache to expire every 10 seconds.</p>
<p>While this reduces the window of inconsistency, it doesn't solve the problem.
It just masks it.</p>
<p>This approach introduces two new issues:</p>
<ul>
<li><strong>Increased Latency</strong>: You are now forcing your application to reach out to the distributed L2 cache (Redis) or the database much more frequently.</li>
<li><strong>Lost Efficiency</strong>: The main benefit of L1 caching is avoiding network requests entirely.
If you expire data too fast, you lose the performance gain for the majority of your traffic.</li>
</ul>
<p>For things like user permissions, feature flags, or pricing, &quot;mostly correct&quot; is often not good enough. You need immediate consistency.</p>
<h2>The Solution: Redis Pub/Sub Backplane</h2>
<p>To solve this, we need a <strong>backplane</strong>.
It's a communication channel that connects all our application nodes.</p>
<p>When a cache entry is removed or updated on one node, we publish a message to the backplane.
All other nodes subscribe to this channel and, upon receiving the message, remove the corresponding key from their local cache.</p>
<p>Redis is already a popular choice for the L2 cache, so it makes perfect sense to use its <a href="simple-messaging-in-dotnet-with-redis-pubsub"><strong>Pub/Sub feature</strong></a>
for this signaling mechanism.</p>
<p>It works like this:</p>
<ol>
<li><strong>Publisher:</strong> The node that modifies data publishes a <code>cache-invalidation</code> message with the cache key.</li>
<li><strong>Subscriber:</strong> All nodes listen to this channel.</li>
<li><strong>Action:</strong> When a message arrives, they call <code>HybridCache.RemoveAsync(key)</code>.</li>
</ol>
<p><img src="/blogs/mnw_177/cache_invalidation_message_fanout.png" alt="A sequence diagram showing cache invalidation messages being published and received by multiple servers."></p>
<h2>Implementing the Solution</h2>
<p>We will need the <code>StackExchange.Redis</code> library to handle the messaging.</p>
<p>Let's start by defining a simple service to handle the publishing.
This service will be responsible for notifying the rest of the system that a key has changed.</p>
<pre><code class="language-csharp">public interface ICacheInvalidator
{
    Task InvalidateAsync(string key, CancellationToken cancellationToken = default);
}

public class RedisCacheInvalidator(
    IConnectionMultiplexer connectionMultiplexer,
    ILogger&lt;RedisCacheInvalidator&gt; logger)
    : ICacheInvalidator
{
    private const RedisChannel Channel = RedisChannel.Literal(&quot;cache-invalidation&quot;);

    public async Task InvalidateAsync(string key, CancellationToken cancellationToken = default)
    {
        var subscriber = connectionMultiplexer.GetSubscriber();

        await subscriber.PublishAsync(Channel, new RedisValue(key));

        logger.LogInformation(&quot;Published invalidation for key: {Key}&quot;, key);
    }
}
</code></pre>
<p>Now, whenever you update an entity in your Command Handler or Service, you just call <code>ICacheInvalidator.InvalidateAsync</code>.</p>
<pre><code class="language-csharp">public class UpdateUserProfileHandler(
    AppDbContext dbContext,
    ICacheInvalidator cacheInvalidator,
    ILogger&lt;UpdateUserProfileHandler&gt; logger)
{
    public async Task Handle(int userId, string newName, CancellationToken ct)
    {
        // 1. Update the database
        var user = await dbContext.Users.FindAsync([userId], ct);
        if (user is null)
        {
             return;
        }

        user.Name = newName;
        await dbContext.SaveChangesAsync(ct);

        // 2. Invalidate the cache (Distributed)
        var cacheKey = $&quot;user:{userId}&quot;;
        await cacheInvalidator.InvalidateAsync(cacheKey, ct);

        logger.LogInformation(&quot;Updated user and invalidated cache for {UserId}&quot;, userId);
    }
}
</code></pre>
<h3>The Background Listener</h3>
<p>Next, we need a <a href="running-background-tasks-in-asp-net-core"><strong>background service</strong></a> that runs on every node.
It will subscribe to the Redis channel and evict keys from the local <code>HybridCache</code>.</p>
<p><strong>A quick note on self-publishing</strong>:
Because Redis Pub/Sub broadcasts to everyone subscribed, the node that published the invalidation will also receive the message.
In this implementation, we simply remove the key again.
It's redundant but harmless, and it keeps the code simple.</p>
<p>Note that we are injecting <code>HybridCache</code> directly into our background service.
An alternative is using <code>IMemoryCache</code>, since that is the L1 cache inside <code>HybridCache</code>.</p>
<pre><code class="language-csharp">public class CacheInvalidationService(
    IConnectionMultiplexer connectionMultiplexer,
    HybridCache hybridCache,
    ILogger&lt;CacheInvalidationService&gt; logger)
    : BackgroundService
{
    private const RedisChannel Channel = RedisChannel.Literal(&quot;cache-invalidation&quot;);

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        var subscriber = connectionMultiplexer.GetSubscriber();

        await subscriber.SubscribeAsync(Channel, (channel, value) =&gt;
        {
            string key = value.ToString();

            logger.LogInformation(&quot;Invalidating local cache for: {Key}&quot;, key);

            // This removes the item from the local L1 cache
            var task = hybridCache.RemoveAsync(key, stoppingToken);

            if (!task.IsCompleted)
            {
                task.GetAwaiter().GetResult();
            }
        });
    }
}

</code></pre>
<h3>Wiring It All Together</h3>
<p>Finally, we need to register these services in our DI container.</p>
<pre><code class="language-csharp">builder.Services.AddSingleton&lt;IConnectionMultiplexer&gt;(sp =&gt;
    ConnectionMultiplexer.Connect(&quot;&lt;REDIS_CONNECTION_STRING&gt;&quot;));

// Register HybridCache (defaults generally work fine for L1)
builder.Services.AddHybridCache();

// Register our invalidation services
builder.Services.AddSingleton&lt;ICacheInvalidator, RedisCacheInvalidator&gt;();
builder.Services.AddHostedService&lt;CacheInvalidationService&gt;();

</code></pre>
<p>Now, when <strong>Node A</strong> calls <code>InvalidateAsync(&quot;user:123&quot;)</code>, Redis pushes that message to <strong>Node B</strong>, <strong>Node C</strong>, and so on.
They all trigger <code>hybridCache.RemoveAsync(&quot;user:123&quot;)</code>, ensuring the next request fetches fresh data from the source (or the shared L2).</p>
<h2>A Better Way: FusionCache</h2>
<p>If building your own backplane feels like reinventing the wheel, you should look at <a href="https://github.com/ZiggyCreatures/FusionCache"><strong>FusionCache</strong></a>.</p>
<p>FusionCache is a mature, battle-tested library that has solved this exact problem for years.
It has a built-in backplane feature that automatically handles the Pub/Sub messaging for you.</p>
<p>Even better, FusionCache recently added an implementation of the HybridCache abstract class.
This means you can swap it in without changing much of your existing code.</p>
<pre><code class="language-csharp">// Using FusionCache's implementation of HybridCache
builder.Services.AddFusionCache()
    .WithBackplane(
        new RedisBackplane(new RedisBackplaneOptions { Configuration = &quot;&lt;REDIS_CONNECTION_STRING&gt;&quot; }))
    .AsHybridCache();
</code></pre>
<h2>Summary</h2>
<p><code>HybridCache</code> is a powerful addition to the .NET ecosystem, effectively merging the benefits of <code>IMemoryCache</code> and <code>IDistributedCache</code>.
However, for multi-node setups requiring high consistency, you still need a mechanism to synchronize the local caches.</p>
<p>Redis Pub/Sub offers a lightweight, effective solution to this problem.</p>
<p>By implementing a simple &quot;bus&quot; for invalidation messages, you get the best of both worlds:
the extreme performance of local caching and the data consistency of a distributed system.</p>
<p>Good luck out there, and see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_177.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Solving Message Ordering from First Principles]]></title>
            <link>https://milanjovanovic.tech/blog/solving-message-ordering-from-first-principles</link>
            <guid isPermaLink="false">solving-message-ordering-from-first-principles</guid>
            <pubDate>Sat, 10 Jan 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[Per-aggregate ordering is what we really want. But queues with competing consumers make it surprisingly easy to break. This article shows how the “fix” naturally evolves into a saga.]]></description>
            <content:encoded><![CDATA[<p>Most systems don't need <em>global</em> message ordering.</p>
<p>They need something simpler and more useful: <strong>events must be handled in order per aggregate</strong>.</p>
<p>Per <code>OrderId</code>, per <code>InvoiceId</code>, per <code>CustomerId</code>, or whatever your aggregate boundary is.
You can make this as broad or as narrow as you need.</p>
<p>This starts as an eventing problem, but if you follow the requirements to their logical conclusion, you'll end up with a workflow.
And that workflow has a name: <strong>a saga</strong>.</p>
<h2>Domain Events Feel Like the Clean Solution</h2>
<p><a href="how-to-use-domain-events-to-build-loosely-coupled-systems"><strong>Domain events</strong></a> are attractive because they come from first principles:</p>
<ul>
<li>An aggregate changes state</li>
<li>It emits events describing what happened</li>
<li>Handlers react and do useful work</li>
</ul>
<p>You also get a nice mental model:</p>
<blockquote>
<p>State change → Event → Reaction</p>
</blockquote>
<p>A typical example:</p>
<ul>
<li><code>OrderPlaced</code></li>
<li><code>PaymentCaptured</code></li>
<li><code>OrderShipped</code></li>
</ul>
<p>But there's a catch.</p>
<p>Domain events are <strong>brittle</strong> when you try to use them for integration.</p>
<p><img src="/blogs/mnw_176/domain_events_flow.png" alt="A sequence diagram showing an Aggregate committing a change, emitting Domain Events, which are then processed by multiple Event Handlers."></p>
<p>If you publish directly from your transaction, you're coupling business correctness to an unreliable side effect:</p>
<ul>
<li>The transaction succeeds but publishing fails</li>
<li>Publishing succeeds but the transaction rolls back</li>
<li>Consumers process duplicates</li>
<li>Retries cause reordering</li>
</ul>
<p>So we keep the model, but harden the delivery.</p>
<h2>The Outbox Makes Publishing Reliable (but not ordered)</h2>
<p>With an <a href="implementing-the-outbox-pattern"><strong>Outbox</strong></a>, we store outgoing events in the same transaction as the aggregate update.</p>
<p>Then a background publisher reads the Outbox and pushes events to a queue.</p>
<p>This fixes the reliability problem:</p>
<ul>
<li>If the transaction commits, the event is persisted</li>
<li>If the publisher crashes, it can resume later</li>
<li>We can retry safely</li>
</ul>
<p><img src="/blogs/mnw_176/outbox_flow.png" alt="A diagram showing an Aggregate committing a change along with an Outbox entry, which is then published to a Queue by a background process."></p>
<p>Now we've made event publishing reliable.</p>
<p>But we haven't made event handling ordered.</p>
<h2>Competing consumers Are Great, Until Order Matters</h2>
<p>The moment events hit a queue, we usually scale with the simplest lever: <a href="event-driven-architecture-in-dotnet-with-rabbitmq"><strong>competing consumers</strong></a>.</p>
<p>Multiple instances consume from the same queue to increase throughput.</p>
<p>That works… until ordering matters.</p>
<p>Two events for the same <code>OrderId</code> can be processed at the same time:</p>
<ul>
<li>Consumer A receives <code>PaymentCaptured</code></li>
<li>Consumer B receives <code>OrderPlaced</code></li>
<li>Side effects run out of order</li>
</ul>
<p>Even if the events were published in order, retries and redelivery can scramble processing order.</p>
<p>And now you have a subtle bug that only appears under load.</p>
<p><img src="/blogs/mnw_176/competing_consumers.png" alt="A sequence diagram showing multiple consumers processing messages from a Queue concurrently, leading to out-of-order handling of events for the same aggregate ID."></p>
<p>That's the key realization: queues scale work.
They don't preserve your invariants.</p>
<h2>What We Really Want is Per-Aggregate Ordering</h2>
<p>You don't need one ordered line for <em>everything</em>.</p>
<p>You need many independent ordered lines, one per aggregate.</p>
<p>That usually holds true because:</p>
<ul>
<li>Aggregates already define consistency boundaries</li>
<li>Events are naturally produced in order (v1, v2, v3…)</li>
<li>The &quot;correct&quot; order is the aggregate's own timeline</li>
</ul>
<p>If we could guarantee that <strong>only one handler processes events for a given aggregate at a time</strong>, most of the problem disappears.</p>
<p>The most direct solution is also the simplest: <strong>use a single consumer for the whole stream.</strong></p>
<p>That enforces ordering, assuming events are published in order.</p>
<p><img src="/blogs/mnw_176/single_consumer.png" alt="A sequence diagram showing a single consumer processing messages from a Queue, leading to ordered handling of events for the same aggregate ID."></p>
<p>But it has an obvious drawback.</p>
<h2>A Single Consumer Solves Ordering But Limits Scale</h2>
<p>One consumer means:</p>
<ul>
<li>Throughput ceiling (one worker)</li>
<li>Latency spikes under load</li>
<li>Scaling becomes vertical, not horizontal</li>
</ul>
<p>Even if your events are lightweight, you're artificially bottlenecking the system.</p>
<p>So we want:</p>
<ul>
<li><strong>Per-aggregate ordering</strong></li>
<li><strong>Horizontal scaling</strong></li>
<li><strong>Reliability (Outbox still stays)</strong></li>
</ul>
<p>This is where teams often &quot;invent&quot; the next step.</p>
<h2>Publish the Next Message From the Handler</h2>
<p>If competing consumers break ordering, one natural idea is:</p>
<blockquote>
<p>Don't let the queue decide what's next, we decide.</p>
</blockquote>
<p>Instead of dumping all events into the queue and letting consumers race, we move to a chained approach:</p>
<ol>
<li>Handle one message for an aggregate</li>
<li>When done, publish the <strong>next</strong> message to be handled</li>
</ol>
<p>Now, the system processes a single message at a time per aggregate.</p>
<p><img src="/blogs/mnw_176/choreographed_saga.png" alt="A diagram showing a sequence of events being processed one at a time per aggregate ID, with each event handler publishing the next event upon completion."></p>
<p>This is the key moment:</p>
<p>You've stopped building &quot;event handlers&quot;.</p>
<p>You've started building a <strong>workflow</strong>.</p>
<p>And that workflow is… a saga.</p>
<h2>Congratulations, You Built a Choreographed Saga</h2>
<p>A <a href="orchestration-vs-choreography"><strong>choreographed saga</strong></a> is a workflow where:</p>
<ul>
<li>Each step reacts to an event</li>
<li>Performs work</li>
<li>Emits the next event to trigger the next step</li>
</ul>
<p>There isn't a single central coordinator.</p>
<p>Instead, we have a chain of events: &quot;when X happens, do Y, then publish Z&quot;.</p>
<p>This pattern naturally fits your new requirement:</p>
<ul>
<li>Per-aggregate ordering is preserved (the chain is sequential)</li>
<li>You can scale across aggregates (many chains in flight)</li>
<li>Each step is isolated and retryable</li>
</ul>
<p>It also forces a useful discipline:</p>
<ul>
<li>&quot;What's the next step?&quot; becomes explicit</li>
<li>Boundaries between steps become clearer</li>
<li>You can observe the workflow as a sequence</li>
</ul>
<p>But choreography has a limitation: <strong>control is distributed</strong>, so tracking progress and handling exceptions can get messy.</p>
<p>So we take the final step.</p>
<h2>If You Want Control, Introduce a State Machine Saga</h2>
<p>When the workflow becomes important, you often want:</p>
<ul>
<li>A single place that knows the current state</li>
<li>Visibility into progress (&quot;where are we stuck?&quot;)</li>
<li>Explicit timeouts and retries</li>
<li>Compensating actions when something fails</li>
</ul>
<p>That's when you move from choreography to <strong>orchestration</strong> via a <a href="implementing-the-saga-pattern-with-masstransit"><strong>state machine saga</strong></a>:</p>
<ul>
<li>The saga holds the workflow state</li>
<li>Events drive transitions</li>
<li>The saga decides what message to publish next</li>
<li>You gain control and observability</li>
</ul>
<p><img src="/blogs/mnw_176/state_machine.png" alt="A state machine diagram showing states and transitions for a saga managing ordered message processing per aggregate ID."></p>
<p>This doesn't replace the Outbox, by the way.</p>
<p>You still want reliable publishing.</p>
<p>You've just made the workflow explicit.</p>
<h2>Broker Support Helps with Ordering, not Correctness</h2>
<p>It's worth calling out that you don't always have to build this yourself.</p>
<p>Many popular message brokers provide technical primitives for <strong>ordered processing per key</strong> (your aggregate ID):</p>
<ul>
<li><a href="https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/using-messagegroupid-property.html">Amazon SQS FIFO message groups</a> (per key)</li>
<li><a href="https://learn.microsoft.com/en-us/azure/service-bus-messaging/message-sessions">Azure Service Bus sessions</a> (per key)</li>
<li><a href="https://developer.confluent.io/courses/apache-kafka/partitions/">Kafka Partitions</a> in a log (key → partition → ordered stream)</li>
<li><a href="https://www.rabbitmq.com/docs/consumers?#single-active-consumer">RabbitMQ &quot;single active consumer&quot;</a> style semantics (per queue)</li>
</ul>
<p>These features can eliminate the most common failure mode of competing consumers: <strong>concurrent handling of messages for the same aggregate</strong>.</p>
<p>But even with perfect per-aggregate ordering, you still need patterns around it to keep the system correct:</p>
<ul>
<li><strong>Outbox</strong> to publish reliably (ordering is useless if events are lost)</li>
<li><a href="idempotent-consumer-handling-duplicate-messages"><strong>Idempotent consumers</strong></a> / <strong>Inbox</strong> because retries and duplicates still happen</li>
<li><strong>Consistency boundaries</strong> (what's safe to do inside the transaction vs outside)</li>
<li><strong>Timeouts</strong> + <strong>compensation</strong> when the &quot;ordered sequence&quot; is actually a business workflow that can partially fail</li>
</ul>
<p>So broker-level ordering is a great foundation.
It reduces accidental complexity.
It just doesn't remove the need to model long-running work explicitly when the business demands it.</p>
<h2>Takeaway</h2>
<p>If you follow the problem from first principles:</p>
<ul>
<li>Aggregates define the boundary where ordering matters</li>
<li>The Outbox makes event publishing reliable</li>
<li>Competing consumers break per-aggregate order</li>
<li>A single consumer restores order but caps throughput</li>
<li>Publishing &quot;the next message&quot; creates sequential progress per aggregate</li>
<li>That sequential progress is a saga (choreographed first, state machine when you need control)</li>
</ul>
<p>So you didn't reinvent something by accident.</p>
<p>You discovered that &quot;ordered handling per aggregate at scale&quot; is not a queue feature.</p>
<p>It's a workflow. And sagas are how we model workflows in distributed systems.</p>
<p>Once you see it that way, you stop fighting queues for ordering guarantees.</p>
<p>You design the workflow the business actually needs.</p>
<p>Hope this was helpful!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_176.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[The Urge to Build Something]]></title>
            <link>https://milanjovanovic.tech/blog/the-urge-to-build-something</link>
            <guid isPermaLink="false">the-urge-to-build-something</guid>
            <pubDate>Sat, 03 Jan 2026 00:00:00 GMT</pubDate>
            <description><![CDATA[Looking at my contribution graph now, I can see the whole journey mapped out in squares of green. The startup dreams. The pivot to teaching. The thousands of commits that represent years of building, learning, and growing. Every cluster tells a story. What will your graph look like a year from now?]]></description>
            <content:encoded><![CDATA[<p>There is a specific kind of restlessness that only developers understand.
It's a quiet, persistent itch.
A mental &quot;background process&quot; that runs while you're eating, showering, or trying to sleep.
I call it <strong>the urge to build</strong>.</p>
<p>Looking back at <a href="https://github.com/m-jovanovic">my GitHub contribution graphs</a> from the last few years, I see more than just green squares.
I see a timeline of my own evolution.</p>
<p>I was <strong>chasing that dream</strong> many developers have: build a small product, ship it, get users, maybe turn it into a business.
Most of those ideas didn't become a successful SaaS.</p>
<p>But that wasn't the point.</p>
<p>The point was that I was building constantly.
Learning constantly.
Shipping constantly.</p>
<p>And I loved it.</p>
<h2>The Indie Hacker Dream</h2>
<p>Back in 2020, my contribution graph shows the classic &quot;burst&quot; pattern.
I was chasing the indiehacker dream.
You know the one: build a SaaS, get to ramen profitability, escape the 9-to-5, tweet about your MRR milestones.
I consumed every bootstrapper podcast, every &quot;I made $10K/month&quot; blog post, every ProductHunt launch breakdown.</p>
<p>I was working on a project called <a href="https://github.com/m-jovanovic/expensely-server">Expensely</a>.
I spent late nights pouring myself into this &quot;genius&quot; idea: a budgeting app.</p>
<p>Nothing revolutionary.
The world didn't need another expense tracker.
But I needed to build it.</p>
<p>Looking back now, it was far from genius.
But at the time, I was convinced that this was <em>the one</em>.</p>
<p><img src="/blogs/mnw_175/github_contributions_2020.png" alt="A GitHub contribution graph from 2020 showing a burst of activity in the end of the year."></p>
<p>In 2021, that momentum stayed steady for five months before life (or perhaps reality) intervened.
Those green squares represent the <strong>fun periods</strong>.
They represent the thrill of architecting an application, the satisfaction of a passing test suite,
and the hope that you're building something that might change your life.</p>
<p><img src="/blogs/mnw_175/github_contributions_2021.png" alt="A GitHub contribution graph from 2021 showing steady activity for the first five months and then a tapering off."></p>
<p>Expensely never became a business.
Most side projects don't.
But here's what I've come to understand: the outcome wasn't the point.</p>
<p>Even though those projects didn't become the next big SaaS, they were the forge where my skills were sharpened.
You don't &quot;waste&quot; time building something that fails.
You only waste time when you don't build at all.
And I mean this both metaphorically and literally.
Humans are meant to create.
When we stop creating, we stagnate.</p>
<h2>What Building Actually Gives You</h2>
<p>When you build something from scratch - when you own every decision from the database schema to the button colors - you learn differently.
There's no senior developer to ask.
No established patterns to follow.
Every problem is yours to solve.</p>
<p>You can use <a href="https://fsharpforfunandprofit.com/rop/">Railway-Oriented Programming</a> if you want to, because who is going to stop you?</p>
<p>Jokes aside, <strong>functional programming</strong> is <strong>very useful</strong>. You should learn it.</p>
<p><img src="/blogs/mnw_175/rop_sample_code.png" alt="A screenshot of sample code implementing Railway-Oriented Programming in C#."></p>
<p>Through my side projects, I wrestled with authentication, background jobs, payment integrations, deployment pipelines.
I made mistakes and spent late nights fixing them.
I built features nobody asked for and skipped features everyone needed.</p>
<p>There is <strong>one thing side projects can't teach you</strong> (unless you get users): dealing with the <strong>consequences of your decisions</strong>.
When you build for yourself, the only person affected by your mistakes is you.
When you build for users, every bug, every outage, every poor design choice has real consequences.</p>
<p>By mid 2021, the project had wound down.
I was settling into a comfortable routine at my day job, working as a senior engineer at a big corporation.
There was no time left for side projects.</p>
<h2>The Unexpected Pivot</h2>
<p>By 2022, something changed.
My contribution graph exploded to over 1,600 commits.
But the focus shifted.
I wasn't building a product anymore.
I was building something more meaningful.
The repositories shifted to my tech blog and <a href="https://www.youtube.com/@MilanJovanovicTech">YouTube projects</a>.</p>
<p>I had <strong>discovered</strong> content creation.</p>
<p>The urge to build hadn't disappeared.
I found a new outlet.
Instead of building products for users, I was building educational content for developers.
Instead of SaaS metrics, I was tracking video views and newsletter subscribers.
To me it was something new, something exciting.</p>
<p><img src="/blogs/mnw_175/github_contributions_2022.png" alt="A GitHub contribution graph from 2022 showing steady activity throughout the year."></p>
<p>I discovered that while I loved building software, I loved <strong>explaining</strong> it even more.
I transitioned from an aspiring founder to a software engineering educator.
At first it was just me writing and recording <strong>things I wish someone had explained to me earlier</strong>.</p>
<p>Then something interesting happened.</p>
<p>People started reading.
Watching.
Replying.
Asking questions.
More people showed up.
And then even more.</p>
<p><img src="/blogs/mnw_175/youtube_milanjovanovictech.png" alt="A screenshot of the Milan Jovanovic Tech YouTube channel with over 140,000 subscribers."></p>
<p>Today, I get to help thousands of developers around the world improve their careers.
It is, without a doubt, the <strong>most rewarding work I've ever done</strong>.</p>
<div className="centered">
  ![A screenshot of testimonials from students of the Pragmatic Clean Architecture course.](/blogs/mnw_175/pca_testimonials.png)
</div>
<figure className="figure-center">
  ![A screenshot of testimonials from students of the Pragmatic Clean Architecture course.](/blogs/mnw_175/pca_testimonials.png)
  <figcaption>
    Student testimonials from [**Pragmatic Clean
    Architecture**](/pragmatic-clean-architecture).
  </figcaption>
</figure>
<p>This is genuinely fulfilling work.
When someone messages me saying my content helped them land a job
or finally understand a concept they'd struggled with for years, that feeling is hard to describe.
I'm doing something meaningful.</p>
<p>And yet.</p>
<h2>The Lingering Itch</h2>
<p>The urge to build something still remains.</p>
<p>It's not dissatisfaction.
It's not that content creation isn't &quot;real&quot; building, it absolutely is.
It's a different kind of building, certainly.
But this feeling I have is something more fundamental.
A part of my brain that wants to work on a fresh project.
That wants to solve a problem nobody has asked me to solve.
That wants to take an idea from nothing to something.</p>
<p>That's why I have a <a href="https://www.hetzner.com/">Hetzner</a> server sitting idle right now.
Never know when I might need it.
Right?
Also, I'm getting a great deal for €3.29/month.
Can't argue with that.</p>
<p><img src="/blogs/mnw_175/hetzner_server.png" alt="A screenshot of a Hetzner server management dashboard showing a server."></p>
<p>I think this urge never fully goes away for people like us.
We can channel it, redirect it, find new outlets for it.
But it's always there, lingering.
And maybe that's okay.
Maybe that restlessness is what makes us who we are.</p>
<h2>Why You Should Build Too</h2>
<p>If you're reading this with your own idea rattling around in your head - some app concept, some tool you wish existed,
some problem you think you could solve - I want to tell you something.</p>
<p><strong>Build it.</strong></p>
<p>Not because it will make you rich.
Statistically, it won't.
Not because it will become the next big thing.
It probably won't be that either.</p>
<p><strong>Build it because the person who finishes that project will not be the same person who started it.</strong>
You will learn things no tutorial can teach.
You will develop judgment that only comes from making real decisions with real consequences.
You will have stories, opinions, and experiences that set you apart.</p>
<p>The outcome is almost beside the point.
The transformation is the product.</p>
<p>Maybe you'll end up with a successful SaaS.
Maybe you'll end up with a failed project and a mass of hard-won knowledge.
Maybe, like me, you'll end up somewhere you never expected - doing work you couldn't have imagined when you wrote that first line of code.</p>
<p>You won't know until you build.</p>
<p>So open your terminal.
Type <code>git init</code>.
And start.</p>
<p>Let this be a year of building.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_175.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How to Build a High-Performance Cache Without External Libraries]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-build-a-high-performance-cache-without-external-libraries</link>
            <guid isPermaLink="false">how-to-build-a-high-performance-cache-without-external-libraries</guid>
            <pubDate>Sat, 27 Dec 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to build a high-performance cache from scratch in .NET, moving from a simple ConcurrentDictionary to an optimized keyed-locking system. This deep dive explores how to master concurrency patterns like double-checked locking to protect your APIs and improve application scalability.]]></description>
            <content:encoded><![CDATA[<p>A couple of days ago, I was looking at a piece of code that's doing too much work.
I'm sure you'll be able to draw a parallel to something in your own applications.
Maybe it's a database call that should be faster, or an external API that's starting to bill you by the thousands.</p>
<p>My first instinct is: <strong>&quot;I'll just cache it&quot;</strong>.</p>
<p>In .NET, that usually means reaching for <code>IMemoryCache</code> or plugging in a distributed cache like Redis.</p>
<p>But have you ever stopped to wonder what's actually happening inside those libraries?<br>
Why do we need all that complexity just to store a value in memory?</p>
<p>So I spent the afternoon trying to build a <a href="caching-in-aspnetcore-improving-application-performance"><strong>high-performance cache</strong></a> from scratch.</p>
<p>I don't recommend DIY-ing your own caching library for production use.
But I learn best by doing something myself.
Understanding these patterns (concurrency, race conditions, and <a href="introduction-to-locking-and-concurrency-control-in-dotnet-6"><strong>locking</strong></a>)
is what separates a &quot;coder&quot; from an engineer.</p>
<h2>The Starting Point</h2>
<p>I was working on a simple currency conversion handler.
We're calling a third-party API to get exchange rates.
The API returns the current exchange rate for a given currency code (like EUR, GBP, JPY) against USD.</p>
<p>This is the initial implementation:</p>
<pre><code class="language-csharp">public static class CurrencyConversion
{
    public static async Task&lt;IResult&gt; Handle(
		string currencyCode,
		decimal amount,
		CurrencyApiClient currencyClient)
	{
		// Validate currency code format (3 uppercase letters)
		if (string.IsNullOrWhiteSpace(currencyCode) ||
			currencyCode.Length != 3 ||
			!currencyCode.All(char.IsLetter))
		{
			return Results.BadRequest(
				new { error = &quot;Currency code must be a 3-letter uppercase code (e.g., EUR, GBP)&quot; });
		}

		// Validate amount (must be positive)
		if (amount &lt; 0)
		{
			return Results.BadRequest(new { error = &quot;Amount must be a positive number&quot; });
		}

		var rate = await currencyClient.GetExchangeRateAsync(currencyCode);

		if (rate == null)
		{
			return Results.NotFound(
				new { error = $&quot;Exchange rate for {currencyCode} not found or API error occurred&quot; });
		}

		var convertedAmount = amount * rate.Value;

		return Results.Ok(new ExchangeRateResponse(
			Currency: currencyCode,
			BaseCurrency: &quot;USD&quot;,
			Rate: rate.Value,
			Amount: amount,
			ConvertedAmount: convertedAmount
        ));
    }
}
</code></pre>
<p>This works fine in your local dev environment.
But in production, if 100 people hit this at the same time, you're making 100 identical network calls.
Your API provider will hate you (and you may even get rate limited), and your latency will spike.</p>
<p>Now let's build a cache to fix this without using any external libraries.
Remember, we're doing this for learning purposes only.</p>
<h2>Level 1: Adding a <code>ConcurrentDictionary</code></h2>
<p>Your first thought is probably to store the rates in a <code>ConcurrentDictionary</code>. It’s thread-safe, so it feels like the right tool.</p>
<pre><code class="language-csharp">private static readonly ConcurrentDictionary&lt;string, decimal&gt; Cache = new();

// In the Handler:
if (Cache.TryGetValue(currencyCode, out var cachedRate))
{
	return cachedRate;
}

var rate = await currencyClient.GetExchangeRateAsync(currencyCode);

Cache.TryAdd(currencyCode, rate.Value);
</code></pre>
<p>This definitely helps with performance under load.
Multiple threads can read and write to the dictionary without crashing.
But <code>ConcurrentDictionary</code> protects the <em>dictionary structure</em>, not your <em>logic</em>.</p>
<p>If 100 users request &quot;EUR&quot; at the exact same time, <code>TryGetValue</code> will return false for all of them.
They will all proceed to call the API.
This is a classic <a href="solving-race-conditions-with-ef-core-optimistic-locking"><strong>race condition</strong></a>.
You've protected your memory, but you haven't protected the external API.</p>
<p>There's also another problem with this approach: the rates never expire.</p>
<h2>Level 2: Adding Cache Expiration</h2>
<p>Currency rates don't stay the same forever.
We need a way to expire them.
Since <code>ConcurrentDictionary</code> doesn't have a &quot;Time to Live&quot; (TTL), we have to wrap our data.</p>
<pre><code class="language-csharp">// Store both the rate and the time it was created
private record CacheEntry(decimal Rate, DateTime CreatedAt);

// Our cache now stores CacheEntry objects
private static readonly ConcurrentDictionary&lt;string, CacheEntry&gt; Cache = new();
private static readonly TimeSpan CacheDuration = TimeSpan.FromMinutes(5);

// Check: Is it there? And is it still &quot;fresh&quot;?
if (Cache.TryGetValue(currencyCode, out var entry) &amp;&amp;
    (DateTime.UtcNow - entry.CreatedAt) &lt; CacheDuration)
{
	return entry.Rate;
}
</code></pre>
<p>Now we have expiration.
But we've actually created a new problem: <a href="https://en.wikipedia.org/wiki/Thundering_herd_problem">The Thundering Herd</a> (a.k.a <a href="https://en.wikipedia.org/wiki/Cache_stampede">Cache Stampede</a>).</p>
<p>Every 5 minutes, when the cache expires, all incoming traffic will see &quot;stale&quot; data and try to refresh it at once.</p>
<p>So we need to fix that next.</p>
<h2>Level 3: Solving the &quot;Cache Stampede&quot;</h2>
<p>To fix the stampede, we need to ensure that only <em>one</em> person can fetch the update while everyone else waits.</p>
<p>How do we do that in C#?</p>
<p>We use a <code>SemaphoreSlim</code> and a pattern called <a href="https://en.wikipedia.org/wiki/Double-checked_locking">Double-Checked Locking</a>.
We check the cache once (the &quot;fast path&quot;), then we lock, and then we check <em>again</em> to see if someone else filled it while we were waiting for the lock.</p>
<pre><code class="language-csharp">// Basically a mutex but async-friendly
private static readonly SemaphoreSlim Lock = new(1, 1);

public static async Task&lt;decimal&gt; GetRateAsync(string code, CurrencyApiClient client)
{
	// Fast path: No locking needed
	if (Cache.TryGetValue(code, out var entry) &amp;&amp; IsFresh(entry))
	{
		return entry.Rate;
	}

    var acquired = await Lock.WaitAsync(TimeSpan.FromSeconds(10)); // Avoid deadlocks
    if (!acquired)
    {
        throw new Exception(&quot;Could not acquire lock to fetch exchange rate.&quot;);
    }
	try
	{
		// Double-check: Did someone else finish the API call while we waited?
		if (Cache.TryGetValue(code, out entry) &amp;&amp; IsFresh(entry))
		{
			return entry.Rate;
		}

		var rate = await client.GetExchangeRateAsync(code);
		var newEntry = new CacheEntry(rate.Value, DateTime.UtcNow);

		// Atomically update the cache
		// This is safe because we're inside the lock
		Cache.AddOrUpdate(code, newEntry, (_, _) =&gt; newEntry);
		return rate.Value;
	}
	finally
	{
		// Always release the lock
		Lock.Release();
	}
}
</code></pre>
<p>This is an improvement.
But something still feels off.</p>
<p>Can you spot the <em>problem</em> with this code?</p>
<p>Our lock behaves like a global lock.
This means that if one thread is fetching &quot;EUR&quot;, all other threads (even those requesting &quot;JPY&quot;) are blocked until the &quot;EUR&quot; fetch completes.
This problem is called <strong>lock contention</strong>.</p>
<p>Let's fix that next.</p>
<h2>Level 4: Scaling with Keyed Locking</h2>
<p>The &quot;pro&quot; move here is <strong>Keyed Locking</strong>.
We create a lock for every specific currency.
Since the number of currencies is finite, this isn't too memory-intensive.</p>
<p>We need an additional <code>ConcurrentDictionary</code> to hold our semaphores, per currency code.</p>
<pre><code class="language-csharp">private static readonly ConcurrentDictionary&lt;string, SemaphoreSlim&gt; Locks = new();

// In the Handler:
var semaphore = Locks.GetOrAdd(currencyCode, _ =&gt; new SemaphoreSlim(1, 1));
if (!Cache.TryGetValue(currencyCode, out var cachedRate) &amp;&amp;
	DateTime.UtcNow - cachedRate?.CreatedAt &lt; CacheDuration)
{
    var acquired = await semaphore.WaitAsync(TimeSpan.FromSeconds(10));
    if (!acquired)
    {
        throw new Exception(&quot;Could not acquire lock to fetch exchange rate.&quot;);
    }
	try
	{
		// Fetch and update logic...
	}
	finally { semaphore.Release(); }
}
</code></pre>
<p>The only thing that changes is how we acquire the lock.
Now, if one thread is fetching &quot;EUR&quot;, other threads requesting &quot;JPY&quot; can proceed without waiting.
This is the most scalable version of our cache.</p>
<p><strong>But...</strong> It only works in memory.
So it's not suitable for distributed systems or multiple server instances.
There are also a few more edge cases to consider, but you can explore those as an exercise.</p>
<h2>The Final Code</h2>
<p>Here's the final version of our caching logic:</p>
<pre><code class="language-csharp">public static class CurrencyConversion
{
    private record CacheEntry(decimal Rate, DateTime CreatedAt);
    private static readonly ConcurrentDictionary&lt;string, CacheEntry&gt; Cache = new();
    private static readonly TimeSpan CacheDuration = TimeSpan.FromMinutes(5);

    private static readonly ConcurrentDictionary&lt;string, SemaphoreSlim&gt; Locks = new();

    public static async Task&lt;IResult&gt; Handle(
        string currencyCode,
        decimal amount,
        CurrencyApiClient currencyClient)
    {
        // Validate currency code format (3 uppercase letters)
        if (string.IsNullOrWhiteSpace(currencyCode) ||
            currencyCode.Length != 3 ||
            !currencyCode.All(char.IsLetter))
        {
            return Results.BadRequest(
                new { error = &quot;Currency code must be a 3-letter uppercase code (e.g., EUR, GBP)&quot; });
        }

        // Validate amount (must be positive)
        if (amount &lt; 0)
        {
            return Results.BadRequest(new { error = &quot;Amount must be a positive number&quot; });
        }

        decimal? rate;
        var semaphore = Locks.GetOrAdd(currencyCode, _ =&gt; new SemaphoreSlim(1, 1));
        if (!Cache.TryGetValue(currencyCode, out var cachedRate) &amp;&amp;
            DateTime.UtcNow - cachedRate?.CreatedAt &lt; CacheDuration)
        {
            var acquired = await semaphore.WaitAsync(TimeSpan.FromSeconds(10));
            if (!acquired)
            {
                throw new Exception(&quot;Could not acquire lock to fetch exchange rate.&quot;);
            }

            try
            {
                // Double-check locking pattern: check again inside the lock
                if (!Cache.TryGetValue(currencyCode, out cachedRate) &amp;&amp;
                    DateTime.UtcNow - cachedRate?.CreatedAt &lt; CacheDuration)
                {
                    rate = await currencyClient.GetExchangeRateAsync(currencyCode);

                    if (rate == null)
                    {
                        return Results.NotFound(
                            new { error = $&quot;Exchange rate for {currencyCode} not found or API error occurred&quot; });
                    }

                    Cache.AddOrUpdate(currencyCode,
                        _ =&gt; new CacheEntry(rate.Value, DateTime.UtcNow),
                        (_, _) =&gt; new CacheEntry(rate.Value, DateTime.UtcNow));
                }
                else
                {
                    rate = cachedRate!.Rate;
                }
            }
            finally
            {
                semaphore.Release();
            }
        }
        else
        {
            rate = cachedRate!.Rate;
        }

        var convertedAmount = amount * rate.Value;

        return Results.Ok(new ExchangeRateResponse(
            Currency: currencyCode,
            BaseCurrency: &quot;USD&quot;,
            Rate: rate.Value,
            Amount: amount,
            ConvertedAmount: convertedAmount
        ));
    }
}
</code></pre>
<p>The next step would be to extract the core caching logic into its own reusable class.
That way, you can use it in other parts of your application.</p>
<h2>Takeaway</h2>
<p>Why go through all this trouble?</p>
<p>It's easy to look at a simple <code>ConcurrentDictionary</code> and think you're done.
But as we've seen, the gap between &quot;it works&quot; and &quot;it scales&quot; is filled with edge cases that can bring a production system to its knees.</p>
<p>When you use a library, it handles these edge cases for you.
But building it yourself teaches you about the &quot;three pillars&quot; of high-performance code:</p>
<ol>
<li><strong>Thread safety</strong></li>
<li><strong>Lock contention</strong></li>
<li><strong>Resource protection</strong></li>
</ol>
<p>Sometimes the most &quot;boring&quot; parts of our infrastructure, like a cache, are actually the most architecturally interesting.</p>
<p>There's also that 1% of the time when you need a custom solution that no library can provide.
So it's worth knowing the fundamentals of how these things work.</p>
<p>Modern libraries like <a href="hybrid-cache-in-aspnetcore-new-caching-library"><strong>HybridCache</strong></a> or
<a href="https://github.com/ZiggyCreatures/FusionCache">FusionCache</a> handle this for you,
but understanding these patterns ensures you know exactly why your application behaves the way it does under load.</p>
<p>And since this is the last issue of the year, I wish you a fantastic New Year! 🎉</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_174.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Server-Sent Events in ASP.NET Core and .NET 10]]></title>
            <link>https://milanjovanovic.tech/blog/server-sent-events-in-aspnetcore-and-dotnet-10</link>
            <guid isPermaLink="false">server-sent-events-in-aspnetcore-and-dotnet-10</guid>
            <pubDate>Sat, 20 Dec 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[ASP.NET Core 10 introduces native Server-Sent Events as a lightweight, HTTP-native alternative to SignalR for simple one-way real-time updates like dashboards and notifications.]]></description>
            <content:encoded><![CDATA[<p><strong>Real-time updates</strong> are no longer a &quot;nice-to-have&quot; feature.
Most modern UI applications expect live data streams of some kind from the server.
For years, the go-to answer in the .NET ecosystem has been <a href="adding-real-time-functionality-to-dotnet-applications-with-signalr"><strong>SignalR</strong></a>.
While SignalR is incredibly powerful, it's <em>nice to have</em> other <strong>options</strong> for simpler use cases.</p>
<p>With the release of <a href="http://ASP.NET">ASP.NET</a> Core 10, we finally have a native, high-level API for <strong>Server-Sent Events</strong> (SSE).
It bridges the gap between basic HTTP polling and full-duplex WebSockets via SignalR.</p>
<h2>Why SSE Instead of SignalR?</h2>
<p><a href="https://dotnet.microsoft.com/en-us/apps/aspnet/signalr">SignalR</a> is a powerhouse that handles
<a href="https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API">WebSockets</a>, Long Polling, and
<a href="https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events">SSE</a> automatically, providing a full-duplex (<strong>two-way</strong>) communication channel.
However, it comes with a footprint: a specific protocol (Hubs), a required client-side library, and a need for &quot;sticky sessions&quot; or a backplane (like Redis) for scaling.</p>
<p>SSE is different because:</p>
<ul>
<li><strong>Unidirectional</strong>: It's designed <strong>specifically</strong> for streaming data <strong>from the server to the client</strong>.</li>
<li><strong>Native HTTP</strong>: It's just a standard HTTP request with a <code>text/event-stream</code> content type. No custom protocols.</li>
<li><strong>Automatic Reconnection</strong>: Browsers natively handle reconnections via the <strong>EventSource</strong> API.</li>
<li><strong>Lightweight</strong>: No heavy client libraries or complex handshake logic.</li>
</ul>
<h2>The Simplest Server-Sent Events Endpoint</h2>
<p>The beauty of the .NET 10 SSE API is its simplicity.
You can use the new <code>Results.ServerSentEvents</code> to return a stream of events from any <code>IAsyncEnumerable&lt;T&gt;</code>.
Because <code>IAsyncEnumerable</code> represents a stream of data that can arrive over time,
the server knows to keep the HTTP connection open rather than closing it after the first &quot;chunk&quot; of data.</p>
<p>Here's a minimal example of an SSE endpoint that streams order placements in real-time:</p>
<pre><code class="language-csharp">app.MapGet(&quot;orders/realtime&quot;, (
	ChannelReader&lt;OrderPlacement&gt; channelReader,
	CancellationToken cancellationToken) =&gt;
{
	// 1. ReadAllAsync returns an IAsyncEnumerable
	// 2. Results.ServerSentEvents tells the browser: &quot;Keep this connection open&quot;
	// 3. New data is pushed to the client as soon as it enters the channel
	return Results.ServerSentEvents(
        channelReader.ReadAllAsync(cancellationToken),
        eventType: &quot;orders&quot;);
});
</code></pre>
<p>When a client hits this endpoint:</p>
<ol>
<li>The server sends a <code>Content-Type: text/event-stream</code> header.</li>
<li>The connection stays active and idle while waiting for data.</li>
<li>As soon as your application pushes an order into the <code>Channel</code>, the <code>IAsyncEnumerable</code> yields that item,
and .NET immediately flushes it down the open HTTP pipe to the browser.</li>
</ol>
<p>It's an incredibly efficient way to handle &quot;push&quot; notifications without the overhead of a stateful protocol.</p>
<p>I'm using a <a href="lightweight-in-memory-message-bus-using-dotnet-channels"><strong><code>Channel</code></strong></a> here as a means to an end.
In a real application, you might have a <a href="scheduling-background-jobs-with-quartz-net"><strong>background service</strong></a>
that listens to a message queue (like RabbitMQ or Azure Service Bus)
or a database change feed, and pushes new events into the channel for connected clients to consume.</p>
<h2>Handling Missed Events</h2>
<p>The simple endpoint we just built is great, but it has one weakness: it's missing resilience.</p>
<p>One of the biggest challenges with real-time streams is connection drops.
By the time the browser automatically reconnects, several events might have already been sent and lost.
To solve this, SSE has a built-in mechanism: the <code>Last-Event-ID</code> <strong>header</strong>.
When a browser reconnects, it sends this ID back to the server.</p>
<p>In .NET 10, we can use the <a href="https://learn.microsoft.com/en-us/dotnet/api/system.net.serversentevents.sseitem-1?view=net-10.0"><code>SseItem&lt;T&gt;</code></a>
type to wrap our data with metadata like IDs and retry intervals.</p>
<p>By combining a simple in-memory <strong>OrderEventBuffer</strong> with the <strong>Last-Event-ID</strong> provided by the browser, we can &quot;replay&quot; missed messages upon reconnection:</p>
<pre><code class="language-csharp">app.MapGet(&quot;orders/realtime/with-replays&quot;, (
	ChannelReader&lt;OrderPlacement&gt; channelReader,
	OrderEventBuffer eventBuffer,
	[FromHeader(Name = &quot;Last-Event-ID&quot;)] string? lastEventId,
	CancellationToken cancellationToken) =&gt;
{
	async IAsyncEnumerable&lt;SseItem&lt;OrderPlacement&gt;&gt; StreamEvents()
	{
		// 1. Replay missed events from the buffer
		if (!string.IsNullOrWhiteSpace(lastEventId))
		{
			var missedEvents = eventBuffer.GetEventsAfter(lastEventId);
			foreach (var missedEvent in missedEvents)
			{
				yield return missedEvent;
			}
		}

		// 2. Stream new events as they arrive in the Channel
		await foreach (var order in channelReader.ReadAllAsync(cancellationToken))
		{
			var sseItem = eventBuffer.Add(order); // Buffer assigns a unique ID
			yield return sseItem;
		}
	}

	return TypedResults.ServerSentEvents(StreamEvents(), &quot;orders&quot;);
});
</code></pre>
<h2>Filtering Server-Sent Events by User</h2>
<p>Server-Sent Events is built on top of standard HTTP.
Because it is a standard <code>GET</code> request, your existing infrastructure &quot;just works&quot;:</p>
<ul>
<li><strong>Security</strong>: You can pass a standard JWT in the <code>Authorization</code> header.</li>
<li><a href="getting-the-current-user-in-clean-architecture"><strong>User Context</strong></a>: You can access <code>HttpContext.User</code> to extract a User ID and filter the stream.
You only send a user the data that belongs to them.</li>
</ul>
<p>Here's an example of an SSE endpoint that streams only the orders for the authenticated user:</p>
<pre><code class="language-csharp">app.MapGet(&quot;orders/realtime&quot;, (
	ChannelReader&lt;OrderPlacement&gt; channelReader,
	IUserContext userContext, // Injected context containing user metadata
	CancellationToken cancellationToken) =&gt;
{
	// The UserId is extracted from the JWT access token by the IUserContext
	var currentUserId = userContext.UserId;

	async IAsyncEnumerable&lt;OrderPlacement&gt; GetUserOrders()
	{
		await foreach (var order in channelReader.ReadAllAsync(cancellationToken))
		{
			// We only yield data that belongs to the authenticated user
			if (order.CustomerId == currentUserId)
			{
				yield return order;
			}
		}
	}

	return Results.ServerSentEvents(GetUserOrders(), &quot;orders&quot;);
})
.RequireAuthorization(); // Standard ASP.NET Core Authorization
</code></pre>
<p>Note that when you write a message to a <code>Channel</code> it's broadcast to <strong>all</strong> connected clients.
This isn't ideal for per-user streams.
You'll probably want to use something more robust for production.</p>
<h2>Consuming Server-Sent Events in JavaScript</h2>
<p>On the client side, you don't need to install a single npm package.
The browser's native <code>EventSource</code> API handles the heavy lifting, including the &quot;reconnect and send Last-Event-ID&quot; logic we discussed above.</p>
<pre><code class="language-javascript">const eventSource = new EventSource('/orders/realtime/with-replays');

// Listen for the specific 'orders' event type we defined in C#
eventSource.addEventListener('orders', (event) =&gt; {
  const payload = JSON.parse(event.data);
  console.log(`New Order ${event.lastEventId}:`, payload.data);
});

// Do something when the connection opens
eventSource.onopen = () =&gt; {
  console.log('Connection opened');
};

// Handle generic messages (if any)
eventSource.onmessage = (event) =&gt; {
  console.log('Received message:', event);
};

// Handle errors and reconnections
eventSource.onerror = () =&gt; {
  if (eventSource.readyState === EventSource.CONNECTING) {
    console.log('Reconnecting...');
  }
};
</code></pre>
<h2>Summary</h2>
<p>SSE in .NET 10 is the perfect middle ground for simple, one-way updates like dashboards, notification bells, and progress bars.
It's lightweight, HTTP-native, and easy to secure using your existing middleware.</p>
<p>However, <strong>SignalR</strong> remains the robust, battle-tested choice for complex bi-directional communication or massive scale requiring a backplane.</p>
<p>The goal isn't to replace SignalR, but to give you a simpler tool for simpler jobs.
Choose the lightest tool that solves your problem.</p>
<p>That's all for today. Hope this was helpful.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_173.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[The New .slnx Solution Format (migration guide)]]></title>
            <link>https://milanjovanovic.tech/blog/the-new-slnx-solution-format-migration-guide</link>
            <guid isPermaLink="false">the-new-slnx-solution-format-migration-guide</guid>
            <pubDate>Sat, 13 Dec 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[See what changes in .slnx, how to convert your existing .sln, and what to watch out for in CI.]]></description>
            <content:encoded><![CDATA[<p><strong>Solution files</strong> have always been <em>that one file</em> nobody wants to touch during a <strong>merge conflict</strong>.
I still remember the pain of resolving conflicts in large monorepo solutions with hundreds of projects.
Can I just say this was not fun?</p>
<p>Microsoft is (finally!) <a href="https://devblogs.microsoft.com/visualstudio/new-simpler-solution-file-format/">addressing that</a> with <strong><code>.slnx</code></strong>: an <strong>XML-based</strong>, simpler solution format designed to be easier to read, edit, and merge.</p>
<p>Here is your practical guide to the future of .NET solutions.</p>
<h2>The problem with <code>.sln</code></h2>
<p>Classic <code>.sln</code> files are verbose: GUID-heavy project entries + configuration blocks that explode as your solution grows.
It's also a frequent source of merge conflicts.</p>
<p>To appreciate the new format, we must look at the old one.</p>
<p>Here's a typical <code>.sln</code> file from a moderately sized .NET solution:</p>
<pre><code class="language-txt">Microsoft Visual Studio Solution File, Format Version 12.00
# Visual Studio Version 17
VisualStudioVersion = 17.7.34031.279
MinimumVisualStudioVersion = 10.0.40219.1
Project(&quot;{2150E333-8FDC-42A3-9474-1A3956D46DE8}&quot;) = &quot;Solution Items&quot;, &quot;Solution Items&quot;, &quot;{8FC526EA-218B-4615-8410-4E1850611F38}&quot;
	ProjectSection(SolutionItems) = preProject
		.editorconfig = .editorconfig
		Directory.Build.props = Directory.Build.props
		Directory.Packages.props = Directory.Packages.props
	EndProjectSection
EndProject
Project(&quot;{2150E333-8FDC-42A3-9474-1A3956D46DE8}&quot;) = &quot;src&quot;, &quot;src&quot;, &quot;{64A28C1B-09AF-426E-8721-D002BE554B48}&quot;
EndProject
Project(&quot;{9A19103F-16F7-4668-BE54-9A1E7A4F7556}&quot;) = &quot;SharedKernel&quot;, &quot;src\SharedKernel\SharedKernel.csproj&quot;, &quot;{166778A2-518F-47F0-BBC7-DB49C76A963C}&quot;
EndProject
Project(&quot;{9A19103F-16F7-4668-BE54-9A1E7A4F7556}&quot;) = &quot;Domain&quot;, &quot;src\Domain\Domain.csproj&quot;, &quot;{6448ADE8-34BC-4F2F-A68C-5B2D6BF4FB0B}&quot;
EndProject
Project(&quot;{9A19103F-16F7-4668-BE54-9A1E7A4F7556}&quot;) = &quot;Application&quot;, &quot;src\Application\Application.csproj&quot;, &quot;{0F576D4A-156D-4626-A4D5-83DD0F6FAFE7}&quot;
EndProject
Project(&quot;{9A19103F-16F7-4668-BE54-9A1E7A4F7556}&quot;) = &quot;Infrastructure&quot;, &quot;src\Infrastructure\Infrastructure.csproj&quot;, &quot;{C699FD09-4D82-4C4B-8744-4FD3B0D60EFC}&quot;
EndProject
Project(&quot;{9A19103F-16F7-4668-BE54-9A1E7A4F7556}&quot;) = &quot;Web.Api&quot;, &quot;src\Web.Api\Web.Api.csproj&quot;, &quot;{86506D03-3746-41E7-A645-97D3633981DB}&quot;
EndProject
Project(&quot;{2150E333-8FDC-42A3-9474-1A3956D46DE8}&quot;) = &quot;tests&quot;, &quot;tests&quot;, &quot;{1EB88D85-BE1E-46DE-99A2-2F02363060AF}&quot;
EndProject
Project(&quot;{9A19103F-16F7-4668-BE54-9A1E7A4F7556}&quot;) = &quot;ArchitectureTests&quot;, &quot;tests\ArchitectureTests\ArchitectureTests.csproj&quot;, &quot;{8D8E2A8A-D3FE-4230-BEF7-C427D6BD87DA}&quot;
EndProject
Project(&quot;{E53339B2-1760-4266-BCC7-CA923CBCF16C}&quot;) = &quot;docker-compose&quot;, &quot;docker-compose.dcproj&quot;, &quot;{34BB3069-D5D0-4046-ACAD-A2025ED7678F}&quot;
EndProject
Global
	GlobalSection(SolutionConfigurationPlatforms) = preSolution
		Debug|Any CPU = Debug|Any CPU
		Release|Any CPU = Release|Any CPU
	EndGlobalSection
	GlobalSection(ProjectConfigurationPlatforms) = postSolution
		{166778A2-518F-47F0-BBC7-DB49C76A963C}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
		{166778A2-518F-47F0-BBC7-DB49C76A963C}.Debug|Any CPU.Build.0 = Debug|Any CPU
		{166778A2-518F-47F0-BBC7-DB49C76A963C}.Release|Any CPU.ActiveCfg = Release|Any CPU
		{166778A2-518F-47F0-BBC7-DB49C76A963C}.Release|Any CPU.Build.0 = Release|Any CPU
		{6448ADE8-34BC-4F2F-A68C-5B2D6BF4FB0B}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
		{6448ADE8-34BC-4F2F-A68C-5B2D6BF4FB0B}.Debug|Any CPU.Build.0 = Debug|Any CPU
		{6448ADE8-34BC-4F2F-A68C-5B2D6BF4FB0B}.Release|Any CPU.ActiveCfg = Release|Any CPU
		{6448ADE8-34BC-4F2F-A68C-5B2D6BF4FB0B}.Release|Any CPU.Build.0 = Release|Any CPU
		{0F576D4A-156D-4626-A4D5-83DD0F6FAFE7}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
		{0F576D4A-156D-4626-A4D5-83DD0F6FAFE7}.Debug|Any CPU.Build.0 = Debug|Any CPU
		{0F576D4A-156D-4626-A4D5-83DD0F6FAFE7}.Release|Any CPU.ActiveCfg = Release|Any CPU
		{0F576D4A-156D-4626-A4D5-83DD0F6FAFE7}.Release|Any CPU.Build.0 = Release|Any CPU
		{C699FD09-4D82-4C4B-8744-4FD3B0D60EFC}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
		{C699FD09-4D82-4C4B-8744-4FD3B0D60EFC}.Debug|Any CPU.Build.0 = Debug|Any CPU
		{C699FD09-4D82-4C4B-8744-4FD3B0D60EFC}.Release|Any CPU.ActiveCfg = Release|Any CPU
		{C699FD09-4D82-4C4B-8744-4FD3B0D60EFC}.Release|Any CPU.Build.0 = Release|Any CPU
		{86506D03-3746-41E7-A645-97D3633981DB}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
		{86506D03-3746-41E7-A645-97D3633981DB}.Debug|Any CPU.Build.0 = Debug|Any CPU
		{86506D03-3746-41E7-A645-97D3633981DB}.Release|Any CPU.ActiveCfg = Release|Any CPU
		{86506D03-3746-41E7-A645-97D3633981DB}.Release|Any CPU.Build.0 = Release|Any CPU
		{8D8E2A8A-D3FE-4230-BEF7-C427D6BD87DA}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
		{8D8E2A8A-D3FE-4230-BEF7-C427D6BD87DA}.Debug|Any CPU.Build.0 = Debug|Any CPU
		{8D8E2A8A-D3FE-4230-BEF7-C427D6BD87DA}.Release|Any CPU.ActiveCfg = Release|Any CPU
		{8D8E2A8A-D3FE-4230-BEF7-C427D6BD87DA}.Release|Any CPU.Build.0 = Release|Any CPU
		{34BB3069-D5D0-4046-ACAD-A2025ED7678F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
		{34BB3069-D5D0-4046-ACAD-A2025ED7678F}.Debug|Any CPU.Build.0 = Debug|Any CPU
		{34BB3069-D5D0-4046-ACAD-A2025ED7678F}.Release|Any CPU.ActiveCfg = Release|Any CPU
		{34BB3069-D5D0-4046-ACAD-A2025ED7678F}.Release|Any CPU.Build.0 = Release|Any CPU
	EndGlobalSection
	GlobalSection(SolutionProperties) = preSolution
		HideSolutionNode = FALSE
	EndGlobalSection
	GlobalSection(NestedProjects) = preSolution
		{166778A2-518F-47F0-BBC7-DB49C76A963C} = {64A28C1B-09AF-426E-8721-D002BE554B48}
		{6448ADE8-34BC-4F2F-A68C-5B2D6BF4FB0B} = {64A28C1B-09AF-426E-8721-D002BE554B48}
		{0F576D4A-156D-4626-A4D5-83DD0F6FAFE7} = {64A28C1B-09AF-426E-8721-D002BE554B48}
		{C699FD09-4D82-4C4B-8744-4FD3B0D60EFC} = {64A28C1B-09AF-426E-8721-D002BE554B48}
		{86506D03-3746-41E7-A645-97D3633981DB} = {64A28C1B-09AF-426E-8721-D002BE554B48}
		{8D8E2A8A-D3FE-4230-BEF7-C427D6BD87DA} = {1EB88D85-BE1E-46DE-99A2-2F02363060AF}
	EndGlobalSection
	GlobalSection(ExtensibilityGlobals) = postSolution
		SolutionGuid = {B948A3CC-9872-4612-ABD2-BB3D49671542}
	EndGlobalSection
EndGlobal
</code></pre>
<p>Good luck trying to make sense of that during a merge conflict!</p>
<h2>What <code>.slnx</code> looks like</h2>
<p>A minimal <code>.slnx</code> is basically a list of projects in XML.</p>
<p>Here's the same solution as above, but in <code>.slnx</code> format.
We even have solution items, folders, and a docker-compose project.</p>
<pre><code class="language-xml">&lt;Solution&gt;
  &lt;Folder Name=&quot;/Solution Items/&quot;&gt;
    &lt;File Path=&quot;.editorconfig&quot; /&gt;
    &lt;File Path=&quot;Directory.Build.props&quot; /&gt;
    &lt;File Path=&quot;Directory.Packages.props&quot; /&gt;
  &lt;/Folder&gt;
  &lt;Folder Name=&quot;/src/&quot;&gt;
    &lt;Project Path=&quot;src/Application/Application.csproj&quot; /&gt;
    &lt;Project Path=&quot;src/Domain/Domain.csproj&quot; /&gt;
    &lt;Project Path=&quot;src/Infrastructure/Infrastructure.csproj&quot; /&gt;
    &lt;Project Path=&quot;src/SharedKernel/SharedKernel.csproj&quot; /&gt;
    &lt;Project Path=&quot;src/Web.Api/Web.Api.csproj&quot; /&gt;
  &lt;/Folder&gt;
  &lt;Folder Name=&quot;/tests/&quot;&gt;
    &lt;Project Path=&quot;tests/ArchitectureTests/ArchitectureTests.csproj&quot; /&gt;
  &lt;/Folder&gt;
  &lt;Project Path=&quot;docker-compose.dcproj&quot;&gt;
    &lt;Build /&gt;
  &lt;/Project&gt;
&lt;/Solution&gt;

</code></pre>
<p>It looks remarkably similar to a <code>.csproj</code> file.</p>
<h2>How to Migrate Today</h2>
<p>The <code>.slnx</code> format is available in recent versions of Visual Studio 2022 (v17.13+) and the .NET 9 SDK. Here is how you can switch.</p>
<p><strong>Option 1: The Command Line</strong></p>
<p>If you have the .NET 9 SDK installed (specifically 9.0.200 or later), you can migrate instantly via the CLI.</p>
<ol>
<li>
<p>Open your terminal in the solution folder.</p>
</li>
<li>
<p>Run the migration command:</p>
<pre><code class="language-bash">dotnet sln migrate
</code></pre>
</li>
<li>
<p>This creates a new <code>.slnx</code> file alongside your old <code>.sln</code>.</p>
</li>
</ol>
<p>At this point I would recommend deleting the old <code>.sln</code> file to avoid confusion.
There's no point in keeping both in the same repo.</p>
<p><strong>Option 2: Visual Studio &quot;Save As&quot;</strong></p>
<p>If you prefer the GUI, you can do this directly inside Visual Studio 2022 (or 2026).</p>
<ol>
<li>
<p>Select the Solution in the Solution Explorer.</p>
</li>
<li>
<p>Go to <code>File</code> &gt; <code>Save Solution As...</code>.</p>
<p><img src="/blogs/mnw_172/visual_studio_save_as.png" alt="The file menu in Visual Studio with the Save As option highlighted."></p>
</li>
<li>
<p>Change the &quot;Save as type&quot; dropdown to Xml Solution File (*.slnx).</p>
<p><img src="/blogs/mnw_172/save_as_xml_solution_file.png" alt="The Save As dialog in Visual Studio with the Xml Solution File option selected."></p>
</li>
</ol>
<h2>Why You Should Care</h2>
<ul>
<li><strong>Fewer Merge Conflicts</strong>: The #1 benefit.
Because the file is simple XML without random GUIDs changing, git merges become trivial.</li>
<li><strong>Human Readable</strong>: You can open this in Notepad, understand it, and edit it without breaking your entire build.</li>
<li><strong>Consistency</strong>: It finally aligns the solution format with the project format (<code>.csproj</code>), which moved to simplified XML years ago.</li>
<li><strong>Performance</strong>: Smaller file sizes and simpler parsing mean slightly faster load times for massive solutions.</li>
</ul>
<h2>Is it Ready?</h2>
<p>As of late 2025/early 2026, <code>.slnx</code> is technically a <strong>Preview</strong> feature.</p>
<p><strong>Safe to use</strong>?
Yes, the format is stable.</p>
<p><strong>Tooling support</strong>?
Visual Studio 2022, Visual Studio 2026 and Rider support it well.
So does the .NET CLI.
Some older CI/CD pipelines or 3rd party tools might not recognize the extension yet.</p>
<p><strong>My recommendation</strong>: Try it on a side project or a branch first.
If your CI pipeline passes, you are ready to modernize.</p>
<p><strong>I'm using this format</strong> in all my new projects and have migrated several existing ones without issues.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_172.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[DbContext is Not Thread-Safe: Parallelizing EF Core Queries the Right Way]]></title>
            <link>https://milanjovanovic.tech/blog/dbcontext-is-not-thread-safe-parallelizing-ef-core-queries-the-right-way</link>
            <guid isPermaLink="false">dbcontext-is-not-thread-safe-parallelizing-ef-core-queries-the-right-way</guid>
            <pubDate>Sat, 06 Dec 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to safely parallelize EF Core queries to improve performance by using IDbContextFactory to create isolated contexts, avoiding the thread-safety exceptions caused by sharing a single DbContext instance.]]></description>
            <content:encoded><![CDATA[<p>We have all built <em>that</em> endpoint.</p>
<p>You know the one: the &quot;Executive Dashboard&quot; or the &quot;User Summary&quot; screen.
It's the endpoint that needs to fetch three or four completely unrelated sets of data to paint a complete picture for the user.
It needs the last 50 orders, the current system health logs, the user's profile settings, and maybe a notification count.</p>
<p>So, you write the code the standard way:</p>
<pre><code class="language-csharp">var orders = await GetRecentOrdersAsync(userId);
var logs = await GetSystemLogsAsync();
var stats = await GetUserStatsAsync(userId);

return new DashboardDto(orders, logs, stats);
</code></pre>
<p>This works.
It's clean.
It's readable.
But there is a problem.</p>
<p>If <code>GetRecentOrdersAsync</code> takes 300ms, <code>GetSystemLogsAsync</code> takes 400ms, and <code>GetUserStatsAsync</code> takes 300ms,
your users are staring at a loading spinner for 1 full second (300 + 400 + 300).</p>
<p>In a distributed system, latency kills user experience.
Since these data sets are unrelated, we should be able to run them in parallel.
If we did, the total time would only be the duration of the slowest query (400ms).
That is a 60% performance improvement just by changing how we execute the code.</p>
<p>But if you try the naive approach with Entity Framework Core, your application will crash.</p>
<h2>The False Promise of Task.WhenAll</h2>
<p>The most common mistake developers make when trying to optimize this is wrapping their existing repository calls in tasks and waiting for them all at once.</p>
<p>It looks something like this:</p>
<pre><code class="language-csharp">// ❌ DO NOT DO THIS
public async Task&lt;DashboardData&gt; GetDashboardData(int userId)
{
    // These methods all use the same injected _dbContext
    var ordersTask = _repository.GetOrdersAsync(userId);
    var logsTask = _repository.GetLogsAsync();
    var statsTask = _repository.GetStatsAsync(userId);

    await Task.WhenAll(ordersTask, logsTask, statsTask); // BOOM 💥

    return new DashboardData(ordersTask.Result, logsTask.Result, statsTask.Result);
}
</code></pre>
<p>If you run this, you will immediately hit <a href="https://learn.microsoft.com/en-us/ef/core/dbcontext-configuration/#avoiding-dbcontext-threading-issues">this dreaded exception</a>:</p>
<blockquote>
<p>A second operation started on this context before a previous operation completed.
This is usually caused by different threads using the same instance of DbContext, however instance members are not guaranteed to be thread safe.</p>
</blockquote>
<p><strong>Why does this happen?</strong></p>
<p>The <code>DbContext</code> in EF Core is <strong>not thread-safe</strong>.
It is a stateful object designed to manage a single unit of work.
It maintains a &quot;Change Tracker&quot; to keep track of the entities you've loaded, and it wraps a single underlying database connection.</p>
<p>Database protocols (like the TCP stream for PostgreSQL or SQL Server) are generally synchronous at the connection level.
You cannot push two different SQL queries down the same wire at the exact same millisecond.
When you use <code>Task.WhenAll</code>, multiple threads try to grab that single connection simultaneously,
and EF Core steps in to throw the exception to prevent data corruption.</p>
<p>So, we have a dilemma: We want the speed of parallelism, but the <code>DbContext</code> forces us into sequential execution.</p>
<h2>The Solution</h2>
<p>Since .NET 5, EF Core has provided a first-class solution for this exact scenario: <code>IDbContextFactory&lt;T&gt;</code>.</p>
<p>Instead of injecting a scoped instance of your context (which lives for the entire HTTP request),
you inject a factory that allows you to create lightweight, independent instances of <code>DbContext</code> on demand.</p>
<p><strong><em>Note</em></strong>: While using the factory is the cleanest approach for Dependency Injection,
you can also manually instantiate the context (<code>using var context = new AppDbContext(options)</code>) if you have access to the <code>DbContextOptions</code>.</p>
<p>First, we need to register the factory in our <code>Program.cs</code>.</p>
<pre><code class="language-csharp">// This registers IDbContextFactory&lt;AppDbContext&gt; as a Singleton (by default)
// It also registers AppDbContext as Scoped for ease of use elsewhere
builder.Services.AddDbContextFactory&lt;AppDbContext&gt;(options =&gt;
{
    options.UseNpgsql(builder.Configuration.GetConnectionString(&quot;db&quot;));
});
</code></pre>
<p>Now, let's refactor our slow dashboard endpoint.
Instead of injecting <code>AppDbContext</code>, we inject <code>IDbContextFactory&lt;AppDbContext&gt;</code>.</p>
<p>Inside our method, we spin up a dedicated task for each query.
Inside each task, we create a brand new context, execute the query, and then immediately dispose of it.</p>
<pre><code class="language-csharp">using Microsoft.EntityFrameworkCore;

public class DashboardService(IDbContextFactory&lt;AppDbContext&gt; contextFactory)
{
    public async Task&lt;DashboardDto&gt; GetDashboardAsync(int userId)
    {
        // 1. Start the tasks (The queries start executing immediately upon invocation)
        var ordersTask = GetOrdersAsync(userId);
        var logsTask = GetSystemLogsAsync();
        var statsTask = GetUserStatsAsync(userId);

        // 2. Wait for all to complete
        await Task.WhenAll(ordersTask, logsTask, statsTask);

        // 3. Return results (using 'await Task.WhenAll' here unwraps the result cleanly)
        return new DashboardDto(
            await ordersTask,
            await logsTask,
            await statsTask
        );
    }

    private async Task&lt;List&lt;Order&gt;&gt; GetOrdersAsync(int userId)
    {
        // Create a fresh context for this specific operation
        await using var context = await contextFactory.CreateDbContextAsync();

        return await context.Orders
            .AsNoTracking()
            .Where(o =&gt; o.UserId == userId)
            .OrderByDescending(o =&gt; o.CreatedAt)
            .ThenByDescending(o =&gt; o.Amount)
            .Take(50)
            .ToListAsync();
    }

    private async Task&lt;List&lt;SystemLog&gt;&gt; GetSystemLogsAsync()
    {
        await using var context = await contextFactory.CreateDbContextAsync();

        return await context.SystemLogs
            .AsNoTracking()
            .OrderByDescending(l =&gt; l.Timestamp)
            .Take(50)
            .ToListAsync();
    }

    private async Task&lt;UserStats?&gt; GetUserStatsAsync(int userId)
    {
        await using var context = await contextFactory.CreateDbContextAsync();

        return await context.Users
            .Where(u =&gt; u.Id == userId)
            .Select(u =&gt; new UserStats { OrderCount = u.Orders.Count })
            .FirstOrDefaultAsync();
    }
}
</code></pre>
<p><strong>Key Concepts</strong></p>
<ol>
<li><strong>Isolation</strong>: Each task gets its own DbContext.
This means they get their own database connection.
There is no contention.</li>
<li><strong>Disposal</strong>: Notice the await using.
This is critical.
As soon as the query is done, we want to dispose of that context and return the connection to the pool.</li>
</ol>
<h2>The Benchmark</h2>
<p>To prove this works, I built a small .NET 10 app using Aspire and PostgreSQL..
Since I'm running this locally, the absolute times are very low.
If I used a remote database, the times would be higher, but the speedup ratio would be similar.</p>
<p><strong>Sequential Execution</strong>: ~36ms</p>
<p><img src="/blogs/mnw_171/sequential_ef_queries_trace.png" alt="A distributed trace showing sequential EF Core queries."></p>
<p>The waterfall is painfully obvious here.
Each operation waits for the previous one to finish.</p>
<p><strong>Parallel Execution</strong>: ~13ms</p>
<p><img src="/blogs/mnw_171/parallel_ef_queries_trace.png" alt="A distributed trace showing parallel EF Core queries."></p>
<p>By using the parallel approach, the timeline compresses.
All three database spans start at the same time and complete together.</p>
<h2>Trade-offs &amp; Conclusion</h2>
<p><code>IDbContextFactory</code> bridges the gap between EF Core's unit-of-work design and the reality of modern, parallel requirements.
It allows you to break out of the &quot;one request, one thread&quot; box without sacrificing safety.</p>
<p>However, use this pattern sparingly:</p>
<ul>
<li><strong>Connection pool starvation</strong>: A single HTTP request now occupies 3 database connections simultaneously instead of 1.
If you have high concurrency, you can easily exhaust your connection pool.</li>
<li><strong>Context overhead</strong>: If your queries are extremely fast (e.g., simple lookups by ID),
the overhead of creating multiple contexts and tasks might make the parallel version slower than the sequential one.</li>
</ul>
<p>Next time you are staring at a slow dashboard, don't reach for raw SQL immediately.
Check your awaits.
If they are lined up single-file, it might be time to introduce some parallelization.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_171.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Vertical Slice Architecture: Where Does the Shared Logic Live?]]></title>
            <link>https://milanjovanovic.tech/blog/vertical-slice-architecture-where-does-the-shared-logic-live</link>
            <guid isPermaLink="false">vertical-slice-architecture-where-does-the-shared-logic-live</guid>
            <pubDate>Sat, 29 Nov 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Deciding where shared logic lives is the most critical moment in Vertical Slice Architecture adoption, as choosing incorrectly reintroduces the coupling the architecture aims to eliminate. This article proposes a three-tier approach to sharing code that balances technical infrastructure, domain concepts, and local feature logic.]]></description>
            <content:encoded><![CDATA[<p><a href="vertical-slice-architecture"><strong>Vertical Slice Architecture</strong></a> (VSA) seems like a breath of fresh air when you first encounter it.
You stop jumping between seven layers to add a single field.
You delete the dozens of projects in your solution.
You feel liberated.</p>
<p>But when you start implementing more complex features, the cracks begin to show.</p>
<p>You build a <code>CreateOrder</code> slice.
Then <code>UpdateOrder</code>.
Then <code>GetOrder</code>.
Suddenly, you notice the repetition.
The address validation logic is in three places.
The pricing algorithm is needed by both Cart and Checkout.</p>
<p>You feel the urge to create a <code>Common</code> project or <code>SharedServices</code> folder.
This is the most critical moment in your VSA adoption.</p>
<p>Choose wrong, and you'll reintroduce the coupling you were trying to escape.
Choose right, and you maintain the independence that makes VSA worthwhile.</p>
<p>Here's how I approach <strong>shared code</strong> in <strong>Vertical Slice Architecture</strong>.</p>
<h2>The Guardrails vs. The Open Road</h2>
<p>To understand why this is hard, we need to look at what we left behind.</p>
<p><a href="clean-architecture-the-missing-chapter"><strong>Clean Architecture</strong></a> provides strict guardrails.
It tells you exactly where code lives: Entities go in Domain, interfaces go in Application, implementations go in Infrastructure.
It's safe.
It prevents mistakes, but it also prevents shortcuts when they're appropriate.</p>
<p><a href="vertical-slice-architecture-structuring-vertical-slices"><strong>Vertical Slice Architecture</strong></a> removes the guardrails.
It says, &quot;Organize code by feature, not technical concern&quot;.
This gives you speed and flexibility, but it shifts the burden of discipline onto <em>you</em>.</p>
<p>So what can you do about it?</p>
<h2>The Trap: The &quot;Common&quot; Junk Drawer</h2>
<p>The path of least resistance is to create a project (or folder) named <code>Shared</code>, <code>Common</code>, or <code>Utils</code>.</p>
<p>This is almost always a mistake.</p>
<p>Imagine a <code>Common.Services</code> project with an <code>OrderCalculationService</code> class.
It has a method for cart totals (used by Cart), another for historical revenue (used by Reporting),
and a helper for invoice formatting (used by Invoices).
Three unrelated concerns.
Three different change frequencies.
One class coupling them all together.</p>
<p>A <code>Common</code> project inevitably becomes a junk drawer for anything you can't be bothered to name properly.
It creates a tangled web of dependencies where unrelated features are coupled together because they happen to use the same helper method.</p>
<p>You've reintroduced the very coupling you tried to escape.</p>
<h2>The Decision Framework</h2>
<p>When I hit a potential sharing situation, I ask three questions:</p>
<p><strong>1. Is this infrastructural or domain?</strong></p>
<p>Infrastructure (database contexts, logging, HTTP clients) almost always gets shared. Domain concepts need more scrutiny.</p>
<p><strong>2. How stable is this concept?</strong></p>
<p>If it changes once a year, share it. If it changes with every feature request, keep it local.</p>
<p><strong>3. Am I past the &quot;Rule of Three&quot;?</strong></p>
<p>Duplicating the same code once is fine.
However, creating three duplicates should raise an eyebrow.
Don't abstract until you hit three.</p>
<p>We solve this by refactoring our code.
Let's look at some examples.</p>
<h2>The Three Tiers of Sharing</h2>
<p>Instead of binary &quot;Shared vs. Not Shared,&quot; think in three tiers.</p>
<h3>Tier 1: Technical Infrastructure (Share Freely)</h3>
<p>Pure plumbing that affects all slices equally: logging adapters, database connection factories, auth middleware,
the <a href="functional-error-handling-in-dotnet-with-the-result-pattern"><strong>Result pattern</strong></a>, validation pipelines.</p>
<p>Centralize this in a <code>Shared.Kernel</code> or <code>Infrastructure</code> project.
Note that this can also be a folder within your solution.
It rarely changes due to business requirements.</p>
<pre><code class="language-csharp">// ✅ Good Sharing: Technical Kernel
public readonly record struct Result
{
    public bool IsSuccess { get; }
    public string Error { get; }

    private Result(bool isSuccess, string error)
    {
        IsSuccess = isSuccess;
        Error = error;
    }

    public static Result Success() =&gt; new(true, string.Empty);
    public static Result Failure(string error) =&gt; new(false, error);
}
</code></pre>
<h3>Tier 2: Domain Concepts (Share and Push Logic Down)</h3>
<p>This is one of the best places to share logic.
Instead of scattering business rules across slices, push them into entities and value objects.</p>
<p>Here's an example:</p>
<pre><code class="language-csharp">// ✅ Good Sharing: Entity with Business Logic
public class Order
{
    public Guid Id { get; private set; }
    public OrderStatus Status { get; private set; }
    public List&lt;OrderLine&gt; Lines { get; private set; }

    public bool CanBeCancelled() =&gt; Status == OrderStatus.Pending;

    public Result Cancel()
    {
        if (!CanBeCancelled())
        {
            return Result.Failure(&quot;Only pending orders can be cancelled.&quot;);
        }

        Status = OrderStatus.Cancelled;
        return Result.Success();
    }
}
</code></pre>
<p>Now <code>CancelOrder</code>, <code>GetOrder</code>, and <code>UpdateOrder</code> all use the same business rules.
The logic lives in one place.</p>
<div className="note-panel">
  This implies an important concept: **different vertical slices can share the
  same domain model.**
</div>
<h3>Tier 3: Feature-Specific Logic (Keep It Local)</h3>
<p>Logic shared between related slices, like <code>CreateOrder</code> and <code>UpdateOrder</code>, doesn't need to go global.
Create a <code>Shared</code> folder (there's an exception to every rule) within the feature:</p>
<pre><code>📂 Features
└──📂 Orders
    ├──📂 CreateOrder
    ├──📂 UpdateOrder
    ├──📂 GetOrder
    └──📂 Shared
        ├──📄 OrderValidator.cs
        └──📄 OrderPricingService.cs
</code></pre>
<p>This also has a hiddene benefit.
If you delete the Orders feature, the shared logic goes with it.
No zombie code left behind.</p>
<p>Let's explore some advanced scenarios most people overlook.</p>
<h2>Cross-Feature Sharing</h2>
<p>What about <strong>sharing code between unrelated features</strong> in Vertical Slice Architecture?</p>
<p>The <code>CreateOrder</code> slice needs to check if a customer exists.
<code>GenerateInvoice</code> needs to calculate tax.
Orders and Customers both need to format notification messages.</p>
<p>This doesn't fit neatly into a feature's <code>Shared</code> folder.
So where does it go?</p>
<p><strong>First, ask: do you actually need to share?</strong></p>
<p>Most cross-feature &quot;sharing&quot; is just data access in disguise.</p>
<p>If <code>CreateOrder</code> needs customer data, it queries the database directly.
It doesn't call into the Customers feature.
Each slice owns its data access.
The <code>Customer</code> entity is shared (it lives in <code>Domain</code>), but there's no shared service between them.</p>
<p><strong>When you genuinely need shared logic</strong>, ask what it <em>is</em>:</p>
<ul>
<li><strong>Domain logic</strong> (business rules, calculations) → <code>Domain/Services</code></li>
<li><strong>Infrastructure</strong> (external APIs, formatting) → <code>Infrastructure/Services</code></li>
</ul>
<pre><code class="language-csharp">// Domain/Services/TaxCalculator.cs
public class TaxCalculator
{
    public decimal CalculateTax(Address address, decimal subtotal)
    {
        var rate = GetTaxRate(address.State, address.Country);
        return subtotal * rate;
    }
}
</code></pre>
<p>Both <code>CreateOrder</code> and <code>GenerateInvoice</code> can use it without coupling to each other.</p>
<p>Before creating any cross-feature service, ask: could this logic live on a domain entity instead?
Most &quot;shared business logic&quot; is actually data access, domain logic that belongs on an entity, or premature abstraction.</p>
<p>If you need to trigger a side effect in another feature, I recommend using <a href="event-driven-architecture-in-dotnet-with-rabbitmq"><strong>messaging and events</strong></a>.
Alternatively, the feature you want to call into can explore a facade (<a href="internal-vs-public-apis-in-modular-monoliths"><strong>public API</strong></a>) for that operation.</p>
<h2>When Duplication Is the Right Call</h2>
<p>Sometimes &quot;shared&quot; code isn't actually shared.
It just looks that way.</p>
<pre><code class="language-csharp">// Features/Orders/GetOrder
public record GetOrderResponse(Guid Id, decimal Total, string Status);

// Features/Orders/CreateOrder
public record CreateOrderResponse(Guid Id, decimal Total, string Status);
</code></pre>
<p>They're identical.
The temptation to create a <code>SharedOrderDto</code> is overwhelming.
Resist it.</p>
<p>Next week, <code>GetOrder</code> needs a tracking URL.
But <code>CreateOrder</code> happens before shipping, so there's no URL yet.
If you'd shared the DTO, you'd now have a nullable property that's confusingly empty half the time.</p>
<p><strong>Duplication is cheaper than the wrong abstraction.</strong></p>
<h2>The Practical Structure</h2>
<p>Here's what a mature Vertical Slice Architecture project looks like:</p>
<pre><code>📂 src
└──📂 Features
│   ├──📂 Orders
│   │   ├──📂 CreateOrder
│   │   ├──📂 UpdateOrder
│   │   └──📂 Shared          # Order-specific sharing
│   ├──📂 Customers
│   │   ├──📂 GetCustomer
│   │   └──📂 Shared          # Customer-specific sharing
│   └──📂 Invoices
│       └──📂 GenerateInvoice
└──📂 Domain
│   ├──📂 Entities
│   ├──📂 ValueObjects
│   └──📂 Services            # Cross-feature domain logic
└──📂 Infrastructure
│   ├──📂 Persistence
│   └──📂 Services
└──📂 Shared
    └──📂 Behaviors
</code></pre>
<ul>
<li><strong>Features</strong> — Self-contained slices. Each owns its request/response models.</li>
<li><strong>Features/[Name]/Shared</strong> — Local sharing between related slices.</li>
<li><strong>Domain</strong> — Entities, value objects, and domain services. Shared business logic lives here.</li>
<li><strong>Infrastructure</strong> — Technical concerns.</li>
<li><strong>Shared</strong> — Cross-cutting behaviors only.</li>
</ul>
<h2>The Rules</h2>
<p>After building several systems this way, here's what I've landed on:</p>
<ol>
<li>
<p><strong>Features own their request/response models.</strong> No exceptions.</p>
</li>
<li>
<p><strong>Push business logic into the domain.</strong> Entities and value objects are the best place to share business rules.</p>
</li>
<li>
<p><strong>Keep feature-family sharing local.</strong> If only Order slices need it, keep it in <code>Features/Orders/Shared</code> (feel free to find a better name than <code>Shared</code>).</p>
</li>
<li>
<p><strong>Infrastructure is shared by default</strong>. Database contexts, HTTP clients, logging. These are technical concerns.</p>
</li>
<li>
<p><strong>Apply the Rule of Three.</strong> Don't extract until you have three real usages with identical, stable logic.</p>
</li>
</ol>
<h2>Takeaway</h2>
<p>Vertical Slice Architecture asks: &quot;What feature does <em>this</em> belong to?&quot;</p>
<p>The shared code question is really asking: &quot;What do I do when the answer is <em>multiple features</em>?&quot;</p>
<p>Acknowledge that some concepts genuinely span features.
Give them a home based on their <em>nature</em> (domain, infrastructure, or cross-cutting behavior).
Resist the urge to share everything just because you could.</p>
<p>The goal isn't zero duplication.
It's code that's easy to change when requirements change.</p>
<p>And requirements always change.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_170.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[The False Comfort of the "Happy Path": Decoupling Your Services]]></title>
            <link>https://milanjovanovic.tech/blog/the-false-comfort-of-the-happy-path-decoupling-your-services</link>
            <guid isPermaLink="false">the-false-comfort-of-the-happy-path-decoupling-your-services</guid>
            <pubDate>Sat, 22 Nov 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Coupling isn't just about code structure; it's about failure boundaries. Discover how to ensure your critical business logic survives when external dependencies like email or analytics go down.]]></description>
            <content:encoded><![CDATA[<p><strong>Let's be honest: We've all written this code.</strong></p>
<p>It's Monday morning, you have a deadline, and you need to implement a user registration feature.
It's simple enough: save the user, send a welcome email, and track the signup in your analytics dashboard.</p>
<p>You write this:</p>
<pre><code class="language-csharp">public class UserService(
        IUserRepository userRepository,
        IEmailService emailService,
        IAnalyticsService analyticsService)
{
    public async Task RegisterUser(string email, string password)
    {
        var user = new User(email, password);
        await userRepository.SaveAsync(user);

        // 1. Directly coupled to email service (external API)
        await emailService.SendWelcomeEmail(user.Email);

        // 2. Directly coupled to analytics (this could be an external API)
        await analyticsService.TrackUserRegistration(user.Id);

        // What if we need to add more features?
        // This method will keep growing...
    }
}
</code></pre>
<p>It looks clean. It's readable. It works on your machine.</p>
<p>But this method is a <strong>ticking time bomb</strong>.</p>
<p>It assumes the &quot;Happy Path&quot; is the <em>only</em> path.
It assumes the network is reliable, the email provider is up, and the analytics API is fast.
In production, none of these are guaranteed.</p>
<p>Thinking further, I'm sure you can imagine similar code in your own projects.
It might not be this exact scenario, but the pattern is common: a single method that orchestrates multiple side effects in a linear fashion.</p>
<p>Let's break down why this code is dangerous and how we can refactor it into a robust, event-driven architecture.</p>
<h2>The Hidden Dangers of the &quot;God Method&quot;</h2>
<p>There are three major issues hiding in those ten lines of code.</p>
<h3>1. Temporal Coupling (Latency)</h3>
<p>When a user clicks &quot;Register,&quot; they have to wait for:</p>
<ol>
<li>The Database <strong>+</strong></li>
<li>The SMTP Server <strong>+</strong></li>
<li>The Analytics API</li>
</ol>
<p>If your analytics provider is having a bad day and takes 3 seconds to respond, <strong>your user waits 3 seconds</strong>.
You are punishing your user for the slowness of a background system they don't even care about.</p>
<h3>2. The Partial Failure State</h3>
<p>This is the <strong>most critical risk</strong>. Imagine this scenario:</p>
<ol>
<li><code>SaveAsync(user)</code> succeeds. The user is in the DB.</li>
<li><code>SendWelcomeEmail</code> succeeds. The user gets an email.</li>
<li><code>TrackUserRegistration</code> throws a <code>503 Service Unavailable</code>.</li>
</ol>
<p><strong>What happens now?</strong>
If you wrap this in a transaction and rollback, you have deleted the user from the DB... <strong>but you already sent them a welcome email.</strong>
The user tries to log in, but they don't exist.
Now what?</p>
<p>If you <em>don't</em> rollback, you have a user in your system that is missing from your analytics.
You have data inconsistency.</p>
<h3>3. Violation of Single Responsibility (SRP)</h3>
<p>You might argue that because we are using interfaces (<code>IEmailService</code>), we are decoupled.
That is true for <em>implementation details</em>, but false for <em>orchestration</em>.</p>
<p>The <code>UserService</code> currently has two reasons to change:</p>
<ol>
<li><strong>Core Domain Logic:</strong> &quot;We now require a username in addition to email.&quot;</li>
<li><strong>Notification Policy:</strong> &quot;Marketing wants to send an SMS in addition to the Email.&quot;</li>
</ol>
<p>The <code>UserService</code> should strictly be responsible for the <strong>state change</strong> (creating the user).
It should not be responsible for <strong>orchestrating the side effects</strong> of that change.</p>
<h2>Level 1: Logical Decoupling with Domain Events</h2>
<p>The first step to fixing this is to invert the control.
Instead of the <code>UserService</code> <em>commanding</em> other services to do things, it should simply <em>announce</em> that something happened.</p>
<p>We can use <a href="how-to-use-domain-events-to-build-loosely-coupled-systems"><strong>Domain Events</strong></a> to achieve this.</p>
<p>Here is the refactored <code>UserService</code>:</p>
<pre><code class="language-csharp">public class UserService(
        IUserRepository userRepository,
        IDomainEventDispatcher dispatcher,
        IUnitOfWork unitOfWork)
{
    public async Task RegisterUser(string email, string password)
    {
        // 1. Create the User Entity
        var user = new User(email, password);

        // 2. Capture the side effect as an event object
        var userRegisteredEvent = new UserRegisteredEvent(user.Id, user.Email);

        // 3. Add the entity to the repository
        await userRepository.AddAsync(user);

        // 4. Dispatch the event (Assuming in-process dispatching here for simplicity)
        // Note: Handlers for Email and Analytics are now completely separate classes.
        await dispatcher.Dispatch(userRegisteredEvent);

        await unitOfWork.SaveChangesAsync();
    }
}
</code></pre>
<p>The <code>UserService</code> is now stable.
Adding a &quot;Loyalty Points&quot; feature later doesn't require touching this method.
You just add a new handler for the <code>UserRegisteredEvent</code>.</p>
<p><strong>However, we haven't solved the reliability problem yet.</strong>
If the process crashes immediately after <code>Dispatch</code> but before <code>SaveChangesAsync</code> completes, we might send an email for a user that failed to save.
Or, if we save first and dispatch later, we might save the user but lose the event if the server crashes.</p>
<h2>Level 2: Reliability with the Outbox Pattern</h2>
<p>To fix this, we need <strong>Atomicity</strong>.
Atomicity means that a set of operations either all succeed or all fail together.</p>
<p>We need to guarantee that if the <code>User</code> is saved, the <code>UserRegisteredEvent</code> is also saved.</p>
<p>Enter the <a href="implementing-the-outbox-pattern"><strong>Outbox Pattern</strong></a>.</p>
<p>Instead of publishing the event immediately to a message bus, we save the event to an <code>OutboxMessages</code> table in the <strong>same database transaction</strong> as the user.</p>
<p>Here is the complete implementation logic:</p>
<pre><code class="language-csharp">public async Task RegisterUser(string email, string password)
{
    // 1. Create the Domain Event
    var user = new User(email, password);
    var domainEvent = new UserRegisteredEvent(user.Id, user.Email);

    // 2. Open a Transaction
    using var transaction = dbContext.Database.BeginTransaction();

    try
    {
        // 3. Save the User to the Users Table
        dbContext.Users.Add(user);

        // 4. Serialize the Event and Save to Outbox Table
        var outboxMessage = new OutboxMessage
        {
            Id = Guid.NewGuid(),
            Type = nameof(UserRegisteredEvent),
            Content = JsonSerializer.Serialize(domainEvent),
            OccurredOn = DateTime.UtcNow,
            ProcessedOn = null // Null means it hasn't been handled yet
        };

        dbContext.OutboxMessages.Add(outboxMessage);

        // 5. Commit BOTH changes atomically
        await dbContext.SaveChangesAsync();
        await transaction.CommitAsync();
    }
    catch
    {
        await transaction.RollbackAsync();
        throw;
    }
}
</code></pre>
<p>Now, a <a href="scheduling-background-jobs-with-quartz-net"><strong>background worker</strong></a> (running in a separate process) polls the <code>OutboxMessages</code> table.
It picks up the message and publishes it to your message bus (RabbitMQ, Azure Service Bus, etc.).</p>
<p>If the email service is down, the background worker just retries later.
<strong>We have achieved At-Least-Once delivery.</strong></p>
<h2>Level 3: Distributed Consistency with Sagas</h2>
<p>The Outbox pattern is perfect for side effects (fire-and-forget actions like emails).
But what if the subsequent action is <strong>mandatory</strong>?</p>
<p><strong>Scenario:</strong> When a user registers, we <em>must</em> create a crypto-wallet for them in the <code>WalletService</code>.
If the wallet creation fails (e.g., due to regulations), we cannot allow the user to exist in our system.</p>
<p>We can't just &quot;retry later&quot; if the <code>WalletService</code> says &quot;Fraud Detected.&quot;
We need to <strong>undo</strong> the user creation.</p>
<p>This is a distributed transaction, and we handle it with the <a href="implementing-the-saga-pattern-with-masstransit"><strong>Saga Pattern</strong></a>.
A Saga coordinates a series of steps.
If one fails, it executes <strong>Compensating Transactions</strong> to undo the previous work.</p>
<p>Here is how the failure scenario looks when using a <a href="orchestration-vs-choreography"><strong>Choreography-based Saga</strong></a>:</p>
<p><img src="/blogs/mnw_169/saga_sequence_diagram.png" alt="A Saga Sequence Diagram showing UserService creating a user, WalletService attempting to create a wallet, failing, and UserService deleting the user as a compensation action."></p>
<p>Here's the step-by-step breakdown of the flow:</p>
<ol>
<li><strong>UserService:</strong> Creates User → Publishes <code>UserCreated</code></li>
<li><strong>WalletService:</strong> Listens to <code>UserCreated</code> → Tries to create wallet
<ul>
<li><em>Failure:</em> Wallet creation fails</li>
<li><em>Action:</em> Publishes <code>WalletCreationFailed</code></li>
</ul>
</li>
<li><strong>UserService:</strong> Listens to <code>WalletCreationFailed</code> → <strong>Deletes/Deactivates the User</strong></li>
</ol>
<p>This ensures <strong>Eventual Consistency</strong>.
The system might be inconsistent for a few seconds (the user exists without a wallet),
but it will eventually settle into a valid state (the user is removed).</p>
<h2>Summary: A Heuristic for Decision Making</h2>
<p>You don't need Sagas for everything.
Over-engineering is just as bad as tight coupling.
Use this simple rule of thumb:</p>
<ol>
<li><strong>Is it a simple notification?</strong> (Email, Analytics, Cache Invalidation)
<ul>
<li><strong>Use Domain Events + Outbox.</strong> It's okay if it happens 5 seconds later.</li>
</ul>
</li>
<li><strong>Is it a critical business dependency?</strong> (Payments, Inventory, Account Status)
<ul>
<li><strong>Use a Saga.</strong> If step B fails, step A must be reverted.</li>
</ul>
</li>
</ol>
<p>Coupling isn't just about code structure.
It's about understanding and managing <strong>failure boundaries</strong>.
If your Analytics Service goes down, it shouldn't prevent a user from registering.
Build your systems to survive the unhappy path.</p>
<p>Hope this was helpful.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_169.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Exploring C# File-based Apps in .NET 10]]></title>
            <link>https://milanjovanovic.tech/blog/exploring-csharp-file-based-apps-in-dotnet-10</link>
            <guid isPermaLink="false">exploring-csharp-file-based-apps-in-dotnet-10</guid>
            <pubDate>Sat, 15 Nov 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[With .NET 10, Microsoft introduced file-based apps, which give you the ability to write C# code in a single .cs file and run it directly, without any project or solution files. This feature finally makes C# approachable for quick scripts and utilities]]></description>
            <content:encoded><![CDATA[<p>C# has always been a bit ceremony-heavy, and we all know it.
Even the simplest &quot;Hello World&quot; program traditionally needed a solution file, a project file,
and enough boilerplate to make you wonder if you should've just used a scripting language instead.</p>
<p>Well, Microsoft finally heard us.
With .NET 10, they've introduced <a href="run-csharp-scripts-with-dotnet-run-app-no-project-files-needed"><strong>file-based apps</strong></a>.
And honestly, it's about time.</p>
<h2>What are File-based Apps?</h2>
<p>The idea is simple: write your C# code in a single <code>.cs</code> file and run it directly. That's it.</p>
<p>No need to set up a whole project structure for that quick utility script you need to parse some CSV files or test an API endpoint.</p>
<p>You get to keep everything that makes C# great: the <strong>type safety</strong>, the <strong>performance</strong>, the <strong>rich standard library</strong>.
But now you can use it for those throwaway scripts where setting up a full project would've been overkill.
And yes, you can still <strong>reference NuGet packages</strong>, <strong>reference other C# projects</strong>, target specific SDKs,
and configure project properties, all from within that single file using special directives that start with <code>#:</code>.</p>
<p>The feature builds on <strong>top-level statements</strong> (remember those from C# 9?) and takes them to their logical conclusion.
If we're already letting people skip the class and <code>Main</code> method ceremony, why not let them skip the project ceremony too?</p>
<h2>Getting Started with File-based Apps</h2>
<p>Let's say you want to quickly check what day of the week a specific date falls on.
Create a file called <code>date-checker.cs</code>:</p>
<pre><code class="language-csharp">var targetDate = new DateTime(2025, 12, 31);
Console.WriteLine($&quot;New Year's 2025 falls on a {targetDate.DayOfWeek}&quot;);
Console.WriteLine($&quot;That's {(targetDate - DateTime.Today).Days} days from now&quot;);
</code></pre>
<p>Run it with:</p>
<pre><code class="language-bash">dotnet run date-checker.cs
</code></pre>
<p>The first time you run this, the CLI does some behind-the-scenes magic.
It creates a virtual project, compiles your code, and caches everything.
Subsequent runs are nearly instant because it's smart enough to know when nothing has changed.</p>
<h2>Real-world Example: Quick Data Processing</h2>
<p>Here's where things get interesting.
Say you need to quickly process some JSON data and generate a report.</p>
<p>Let's do something practical with <code>System.Text.Json</code> and <code>CsvHelper</code>:</p>
<pre><code class="language-csharp">#:package CsvHelper@33.1.0
using System.Text.Json;
using CsvHelper;
using System.Globalization;

var json = await File.ReadAllTextAsync(&quot;sales_data.json&quot;);
var sales = JsonSerializer.Deserialize&lt;List&lt;SaleRecord&gt;&gt;(json);

var topProducts = sales
    .GroupBy(s =&gt; s.Product)
    .Select(g =&gt; new {
        Product = g.Key,
        TotalRevenue = g.Sum(s =&gt; s.Amount),
        UnitsSold = g.Count()
    })
    .OrderByDescending(p =&gt; p.TotalRevenue)
    .Take(10);

using var writer = new StreamWriter(&quot;top_products.csv&quot;);
using var csv = new CsvWriter(writer, CultureInfo.InvariantCulture);
csv.WriteRecords(topProducts);

Console.WriteLine(&quot;Report generated! Check top_products.csv&quot;);

record SaleRecord(string Product, decimal Amount, DateTime Date);
</code></pre>
<p>Notice how we're mixing package references, async operations, LINQ, and record types, all in a single file that reads like a cohesive script.
This is the kind of thing you'd typically reach for Python for, but now you can stay in C# land.</p>
<h2>Building Something More Ambitious with Aspire</h2>
<p>You can actually create an <strong>Aspire AppHost</strong> in a single file, which is pretty wild when you think about it:</p>
<pre><code class="language-csharp">#:sdk Aspire.AppHost.Sdk@13.0.0
#:package Aspire.Hosting.AppHost@13.0.0

var builder = DistributedApplication.CreateBuilder(args);

var cache = builder.AddRedis(&quot;cache&quot;)
    .WithDataVolume();

var postgres = builder.AddPostgres(&quot;postgres&quot;)
    .WithDataVolume()
    .AddDatabase(&quot;tododb&quot;);

var todoApi = builder.AddProject&lt;Projects.TodoApi&gt;(&quot;api&quot;)
    .WithReference(cache)
    .WithReference(postgres);

builder.AddNpmApp(&quot;frontend&quot;, &quot;../TodoApp&quot;)
    .WithReference(todoApi)
    .WithReference(&quot;api&quot;)
    .WithHttpEndpoint(env: &quot;PORT&quot;)
    .WithExternalHttpEndpoints();

builder.Build().Run();
</code></pre>
<p>With the <code>#:sdk Aspire.AppHost.Sdk@13.0.0</code> directive, your single file becomes a full orchestrator for a distributed application.
You're defining infrastructure, wiring up dependencies, and setting up a complete development environment, all without creating a project file.
It's particularly useful when you're prototyping architectures or need to quickly spin up a test environment.</p>
<h2>Migrating to a Full Project</h2>
<p>Eventually, some scripts outgrow their single-file roots.
Maybe you need to split things into multiple files, or maybe you want proper IDE support for debugging.
The transition is painless:</p>
<pre><code class="language-bash">dotnet project convert MyUtility.cs
</code></pre>
<p>This generates a proper project structure while preserving all your package references and SDK choices.
Your code moves to <code>Program.cs</code>, and you get a <code>.csproj</code> that reflects all those <code>#:</code> directives you were using.</p>
<h2>Current Limitations</h2>
<p>Right now, it's strictly single-file.
If you need <strong>multiple files</strong>, you'll have to <a href="https://github.com/dotnet/sdk/issues/48174">wait for .NET 11</a> or convert to a full project.
Of course, you can still reference other projects or packages, but the main script has to be in one file.</p>
<p>The caching mechanism can occasionally get confused if you're rapidly iterating on package versions.
And while the IDE support is getting better, it's not quite at the same level as full projects yet, especially for IntelliSense with dynamically referenced packages.</p>
<p>But for what it's designed to do (make C# approachable for scripting scenarios), it works remarkably well.
You can use it for <strong>build scripts</strong>, <strong>data migration one-offs</strong>, quick API tests, or even teaching C# without overwhelming beginners with project structure.</p>
<h2>Where This Leaves Us</h2>
<p><strong>File-based apps</strong> feel like C# finally acknowledging that not every piece of code needs to be an enterprise application.
Sometimes you just need to parse a log file, sometimes you want to quickly test an algorithm,
and sometimes you're teaching someone programming and don't want to explain what a solution file is on day one.</p>
<p>The feature doesn't revolutionize C# development, but it <strong>fills a gap</strong> that's been annoying developers for years.
If you've been keeping a folder of scripts for quick tasks because C# felt too heavy, maybe it's time to give your favorite language another shot.</p>
<p>After all, the best language for a quick script is the one you already know, and now C# makes that argument a lot more compelling.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_168.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[The Idempotent Consumer Pattern in .NET (And Why You Need It)]]></title>
            <link>https://milanjovanovic.tech/blog/the-idempotent-consumer-pattern-in-dotnet-and-why-you-need-it</link>
            <guid isPermaLink="false">the-idempotent-consumer-pattern-in-dotnet-and-why-you-need-it</guid>
            <pubDate>Sat, 08 Nov 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Distributed systems don't fail cleanly: they retry, duplicate, and occasionally fail. This article shows how to design resilient message handlers in .NET using broker-level idempotency and the Idempotent Consumer pattern, so your system stays consistent no matter how many times a message is delivered.]]></description>
            <content:encoded><![CDATA[<p>Distributed systems are unreliable by nature.</p>
<p>I always recommend reading about the <a href="https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing">Fallacies of Distributed Computing</a> to understand the common pitfalls.</p>
<p>One of the key challenges is ensuring that messages are processed <strong>exactly once</strong>.
Theoretically, that's impossible to guarantee in most systems.
I won't dive into the <a href="https://en.wikipedia.org/wiki/CAP_theorem">CAP theorem</a> or the
<a href="https://en.wikipedia.org/wiki/Two_Generals%27_Problem">Two Generals Problem</a> here, but suffice to say:</p>
<ul>
<li>Messages can arrive out of order</li>
<li>Messages can be duplicated</li>
<li>Deliveries can be delayed</li>
</ul>
<p>If you design your system assuming every message will be processed exactly once, you're setting yourself up for subtle data corruption.</p>
<p>But we can design our system to apply side effects <strong>exactly once</strong> using the <a href="idempotent-consumer-handling-duplicate-messages"><strong>Idempotent Consumer</strong></a> pattern.</p>
<p>Let's unpack what can go wrong, how brokers help with idempotency, and how you can build an idempotent consumer in .NET.</p>
<h2>What Can Go Wrong When Publishing</h2>
<p>Let's say your service publishes an event when a new note is created:</p>
<pre><code class="language-csharp">await publisher.PublishAsync(new NoteCreated(note.Id, note.Title, note.Content));
</code></pre>
<p>We don't have to worry about the specific implementation of <code>publisher</code> or the message broker here.
It could be RabbitMQ, SQS, Azure Service Bus, etc.</p>
<p>Now imagine:</p>
<ul>
<li>The publisher sends the message to the broker</li>
<li>The broker stores it and sends an ACK</li>
<li><strong>Network glitch</strong>: the ACK never reaches the producer</li>
<li>Producer times out and <strong>retries</strong> the publish</li>
<li>The broker now has two <code>NoteCreated</code> events</li>
</ul>
<p>From the producer's perspective, it &quot;fixed&quot; a timeout.
But from the consumer's perspective, it received two events for the same note creation.</p>
<p><img src="/blogs/mnw_167/distributed_messaging_with_error.png" alt="Distributed messaging with network error causing duplicate messages."></p>
<p>And that's just one failure path.
You can also get duplicates from:</p>
<ul>
<li>Broker redeliveries</li>
<li>Consumer failures + retries</li>
</ul>
<p>So even if you do everything &quot;right&quot; on the publisher, the <strong>consumer still has to be defensive</strong>.</p>
<h2>Publisher-Side Idempotency (Let the Broker Handle It)</h2>
<p>Many message brokers already support idempotent publishing via message deduplication if you include a unique message ID.
<a href="messaging-made-easy-with-azure-service-bus"><strong>Azure Service Bus</strong></a>, for instance, can detect duplicates and ignore re-publishes for the same message ID within a configured window.
<a href="complete-guide-to-amazon-sqs-and-amazon-sns-with-masstransit"><strong>Amazon SQS</strong></a> and other brokers also offer similar guarantees.</p>
<p>You don't need to reinvent this logic in your application.
The key is to assign each message a stable identifier that uniquely represents the logical event you're sending.</p>
<p>For example, when publishing a NoteCreated event:</p>
<pre><code class="language-csharp">var message = new NoteCreated(note.Id, note.Title, note.Content)
{
    MessageId = Guid.NewGuid() // or you can use note.Id
};

await publisher.PublishAsync(message);

</code></pre>
<p>If the network drops after you send the message, your app might retry.
But when the broker sees the same <code>MessageId</code>, it knows this is a duplicate and safely discards it.
You get deduplication without any custom tracking tables or extra state in your service.</p>
<p>This broker-level idempotency solves a large class of <strong>producer-side issues</strong>: network retries, transient failures, and duplicated publishes.</p>
<p>What it doesn't handle are <strong>consumer retries</strong>, which happen when messages are redelivered or your service crashes mid-processing.</p>
<p>That's where the idempotent consumer pattern comes in.</p>
<h2>Implementing an Idempotent Consumer in .NET</h2>
<p>Here's an example of an idempotent consumer for a <code>NoteCreated</code> event:.</p>
<pre><code class="language-csharp">internal sealed class NoteCreatedConsumer(
    TagsDbContext dbContext,
    HybridCache hybridCache,
    ILogger&lt;Program&gt; logger) : IConsumer&lt;NoteCreated&gt;
{
    public async Task ConsumeAsync(ConsumeContext&lt;NoteCreated&gt; context)
    {
        // 1. Check if we've already processed this message for this consumer
        if (await dbContext.MessageConsumers.AnyAsync(c =&gt;
                c.MessageId == context.MessageId &amp;&amp;
                c.ConsumerName == nameof(NoteCreatedConsumer)))
        {
            return;
        }

        var request = new AnalyzeNoteRequest(
            context.Message.NoteId,
            context.Message.Title,
            context.Message.Content);

        try
        {
            using var transaction = await dbContext.Database.BeginTransactionAsync();

            // 2. Deterministic processing: derive tags from note content
            var tags = AnalyzeContentForTags(request.Title, request.Content);

            // 3. Persist tags
            var tagEntities = tags.Select(ProjectToTagEntity(request.NoteId)).ToList();
            dbContext.Tags.AddRange(tagEntities);

            // 4. Record that this message was processed
            dbContext.MessageConsumers.Add(new MessageConsumer
            {
                MessageId = context.MessageId,
                ConsumerName = nameof(NoteCreatedConsumer),
                ConsumedAtUtc = DateTime.UtcNow
            });

            await dbContext.SaveChangesAsync();
            await transaction.CommitAsync();

            // 5. Update cache
            await CacheNoteTags(request, tags);
        }
        catch (Exception ex)
        {
            logger.LogError(ex, &quot;Error analyzing note {NoteId}&quot;, request.NoteId);
            throw;
        }
    }
}
</code></pre>
<p>This is a typical idempotent consumer with a few important details.</p>
<p><strong>1. The Idempotency Key</strong></p>
<pre><code class="language-csharp">if (await dbContext.MessageConsumers.AnyAsync(c =&gt;
        c.MessageId == context.MessageId &amp;&amp;
        c.ConsumerName == nameof(NoteCreatedConsumer)))
{
    return;
}
</code></pre>
<p>You use:</p>
<ul>
<li><code>MessageId</code> from the transport (<code>context.MessageId</code>)</li>
<li><code>ConsumerName</code> (so multiple consumers can safely process the same message)</li>
</ul>
<p>If a duplicate message arrives, you short-circuit and do nothing.</p>
<p>What's also important here is having a <strong>unique constraint</strong> on <code>(MessageId, ConsumerName)</code> in the <code>MessageConsumers</code> table to prevent
<a href="solving-race-conditions-with-ef-core-optimistic-locking"><strong>race conditions</strong></a>.
So even if you have concurrent processing of the same message, only one will succeed in inserting the record.</p>
<p><strong>2. Atomic Side Effects + Idempotency Record</strong></p>
<p>The processing and storing the message consumer record happen <strong>in the same transaction</strong>:</p>
<pre><code class="language-csharp">using var transaction = await dbContext.Database.BeginTransactionAsync();

// write tags
dbContext.Tags.AddRange(tagEntities);

// write message-consumer record
dbContext.MessageConsumers.Add(new MessageConsumer { ... });

await dbContext.SaveChangesAsync();
await transaction.CommitAsync();
</code></pre>
<p>Why this matters:</p>
<ul>
<li>If processing fails, there is no entry in <code>MessageConsumers</code>, so the message can be retried</li>
<li>If processing succeeds, both the tags and the <code>MessageConsumer</code> row are committed together</li>
<li>You never end up in a state where the work is done but the message is not marked as processed, or vice versa</li>
</ul>
<p>This is the core of idempotency:</p>
<blockquote>
<p>Do this work exactly once per message ID, even under retries.</p>
</blockquote>
<p><strong>3. Handling At-Least-Once Delivery</strong></p>
<p>Most realistic setups are <strong>at-least-once</strong>:</p>
<ul>
<li>Consumer processes message</li>
<li>ACK fails / times out</li>
<li>Broker redelivers</li>
<li>Your code runs again</li>
</ul>
<p>With this pattern, the second run hits the <code>MessageConsumers</code> table and returns early.</p>
<p>No duplicate side effects.</p>
<p>This works, except for one caveat...</p>
<h2>Deterministic vs Non-Deterministic Handlers</h2>
<p>What happens when your handler calls something <em>outside</em> the database?
An email API, a payment gateway, or a background job queue?</p>
<p>These are all common side effects that need to be idempotent too.</p>
<p>Those calls sit outside your transaction boundary.
Your database might commit successfully, but if the network hiccups before the external service responds, you can't tell if the action happened or not.
On retry, your consumer might send another email or charge the credit card twice.</p>
<p>You've now crossed into the messy territory of non-deterministic handlers: operations that can't be repeated safely.</p>
<p>There are two main strategies to deal with this.</p>
<p><strong>1. Use an Idempotency Key in the External Call</strong></p>
<p>If the external service supports it, pass a stable identifier, like the message's <code>MessageId</code> with every request.
Many APIs, including payment processors and email platforms, let you specify an idempotency key header.
The service ensures that identical requests with the same key only execute once.</p>
<p>For example:</p>
<pre><code class="language-csharp">await emailService.SendAsync(new SendEmailRequest
{
    To = user.Email,
    Subject = &quot;Welcome!&quot;,
    Body = &quot;Thanks for signing up.&quot;,
    IdempotencyKey = context.MessageId
});

</code></pre>
<p>Even if the request is retried, the provider will recognize the key and skip the duplicate.
This is the simplest and most reliable approach, if your external dependency supports it.</p>
<p><strong>2. Store the Intent Locally</strong></p>
<p>If the external service doesn't support idempotency keys, you can simulate it.
Store a <strong>record of the intended action</strong> in your database before calling the external system.
For example, create a <code>PendingEmails</code> table that records which messages should be sent, keyed by message ID or user ID.</p>
<p>A background process can later read these pending records and perform the action once.
This makes the process deterministic, but at the cost of more complexity, extra tables, and background workers.
It's often overengineering unless the side effect is critical or irreversible, like payments or account provisioning.</p>
<p>The trade-off comes down to confidence.
If repeating the action has real consequences, introduce idempotency explicitly.
If not, retrying the operation might be acceptable.</p>
<h2>When Idempotent Consumer Isn't Needed</h2>
<p>Not every consumer needs the overhead of idempotency checks.
If your operation is already naturally idempotent, you can often skip the extra table and transaction logic.</p>
<p>Updating a projection, setting a status flag, or refreshing a cache are all examples of deterministic actions that can safely run multiple times.
For instance, &quot;set user's status to Active&quot; or &quot;rebuild the read model&quot; are operations that overwrite state rather than append to it.</p>
<p>Some handlers also use precondition checks to avoid duplication.
If the handler updates an entity, it can first check whether that entity is already in the desired state and return early.
That simple guard clause can be enough.</p>
<p>Don't blindly apply the <strong>Idempotent Consumer</strong> pattern everywhere.
Apply it where it protects you from real harm, where duplicate processing causes financial or data inconsistencies.</p>
<p>For everything else, <strong>simpler is better</strong>.</p>
<h2>Takeaway</h2>
<p>Distributed systems are unpredictable.
Retries, duplicates, and partial failures are part of normal operation.
You can't avoid them, but you can design your system so they don't impact you as much.</p>
<p>Use your broker's built-in <strong>message deduplication</strong> to prevent duplicates from the producer side.
For the consumer side, apply the <strong>Idempotent Consumer</strong> pattern to ensure side effects happen once, even under retries.
Keep the record of processed messages and the actual side effect in the same transaction.</p>
<p>Not every message handler needs this.
If your consumer is naturally idempotent or can short-circuit with a simple precondition, skip the extra complexity.
But for anything that modifies persistent state or calls external systems, idempotency isn't optional, it's the only way to keep your system consistent.</p>
<p>Build your consumers to tolerate retries.
And your distributed system will be that much more reliable.
The interesting part is that once you understand this principle, you start seeing it everywhere in real-world systems.</p>
<p>If you want to dive deeper into messaging patterns and learn how this is implemented in a production-grade system,
check out my <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a> course.
We'll build a full-featured application with distributed messaging, CQRS, and DDD patterns from scratch.</p>
<p>Hope this was helpful.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_167.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[What's New in EF Core 10: LeftJoin and RightJoin Operators in LINQ]]></title>
            <link>https://milanjovanovic.tech/blog/whats-new-in-ef-core-10-leftjoin-and-rightjoin-operators-in-linq</link>
            <guid isPermaLink="false">whats-new-in-ef-core-10-leftjoin-and-rightjoin-operators-in-linq</guid>
            <pubDate>Sat, 01 Nov 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[.NET 10 finally adds proper LeftJoin and RightJoin methods to LINQ, replacing the complex GroupJoin + DefaultIfEmpty pattern with clean, readable code that does exactly what it says.]]></description>
            <content:encoded><![CDATA[<p>If you've ever worked with databases, you know about <code>LEFT JOIN</code> (and conversely <code>RIGHT JOIN</code>).
It's one of those things we use often, if not all the time.
But in Entity Framework Core, doing a left join has always been... well, a bit of a pain.</p>
<p>I like joins that read like what they do.
Unfortunately, until now, LINQ didn't have a straightforward way to express left/right joins.
You had to jump through hoops with <code>GroupJoin</code> and <code>DefaultIfEmpty</code>, which made the code harder to read and maintain.</p>
<p>But <strong>.NET 10</strong> finally fixes this with the brand new <code>LeftJoin</code> and <code>RightJoin</code> methods.</p>
<h2>What's a LEFT JOIN (in plain words)?</h2>
<p>A <strong>LEFT JOIN</strong> returns <strong>all rows from the left side</strong> and the <strong>matching rows from the right side</strong>.
If there's <strong>no match</strong>, the right side is <strong>null</strong>.
Why we use it: to keep &quot;owners&quot; even when they have <strong>no related rows</strong> (e.g., show all products even if some don't have a review).</p>
<figure className="figure-center bordered">
  ![Animated left join.](/blogs/mnw_166/left_join.gif)
  <figcaption>
    Source: [Data
    School](https://dataschool.com/how-to-teach-people-sql/left-right-join-animated/)
  </figcaption>
</figure>
<h2>The Old Way (<code>GroupJoin</code> + <code>DefaultIfEmpty</code>)</h2>
<p>Before .NET 10, a left join in LINQ needed a <strong>group join</strong> (<code>GroupJoin</code>), then <code>DefaultIfEmpty</code> to keep left rows with no match.
It worked, but the intent was buried in noise.</p>
<p>There are two ways you could write it: <strong>query syntax</strong> and <strong>method syntax</strong>.</p>
<h3>Query syntax</h3>
<pre><code class="language-csharp">var query =
    from product in dbContext.Products
    join review in dbContext.Reviews on product.Id equals review.ProductId into reviewGroup
    from subReview in reviewGroup.DefaultIfEmpty()
    orderby product.Id, subReview.Id
    select new
    {
        ProductId = product.Id,
        product.Name,
        product.Price,
        ReviewId = (int?)subReview.Id ?? 0,
        Rating = (int?)subReview.Rating ?? 0,
        Comment = subReview.Comment ?? &quot;N/A&quot;
    };
</code></pre>
<p>Here's the SQL generated by EF Core for the above query:</p>
<pre><code class="language-sql">SELECT
    p.&quot;Id&quot; AS &quot;ProductId&quot;,
    p.&quot;Name&quot;,
    p.&quot;Price&quot;,
    COALESCE(r.&quot;Id&quot;, 0) AS &quot;ReviewId&quot;,
    COALESCE(r.&quot;Rating&quot;, 0) AS &quot;Rating&quot;,
    COALESCE(r.&quot;Comment&quot;, 'N/A') AS &quot;Comment&quot;
FROM &quot;Products&quot; AS p
LEFT JOIN &quot;Reviews&quot; AS r ON p.&quot;Id&quot; = r.&quot;ProductId&quot;
ORDER BY p.&quot;Id&quot;, COALESCE(r.&quot;Id&quot;, 0)
</code></pre>
<h3>Method syntax</h3>
<pre><code class="language-csharp">var query = dbContext.Products
    .GroupJoin(
        dbContext.Reviews,
        product =&gt; product.Id,
        review =&gt; review.ProductId,
        (product, reviewList) =&gt; new { product, subgroup = reviewList })
    .SelectMany(
        joinedSet =&gt; joinedSet.subgroup.DefaultIfEmpty(),
        (joinedSet, review) =&gt; new
        {
            ProductId = joinedSet.product.Id,
            joinedSet.product.Name,
            joinedSet.product.Price,
            ReviewId = (int?)review!.Id ?? 0,
            Rating = (int?)review!.Rating ?? 0,
            Comment = review!.Comment ?? &quot;N/A&quot;
        })
    .OrderBy(result =&gt; result.ProductId)
    .ThenBy(result =&gt; result.ReviewId);
</code></pre>
<p>Why this works: <code>GroupJoin</code> matches rows, <code>DefaultIfEmpty</code> inserts a single <strong>default</strong> (null) when no matches exist, so the left row still appears.
We then <strong>flatten</strong> with <code>SelectMany</code>.</p>
<p>I think we can all agree that this is way too verbose for something as common as a left join.</p>
<h2>The New Way in EF 10: <code>LeftJoin</code></h2>
<p>Now we can write what we mean.
<code>LeftJoin</code> is <strong>first-class LINQ</strong> and EF Core translates it to a <strong>LEFT JOIN</strong> in SQL.</p>
<pre><code class="language-csharp">var query = dbContext.Products
    .LeftJoin(
        dbContext.Reviews,
        product =&gt; product.Id,
        review =&gt; review.ProductId,
        (product, review) =&gt; new
        {
            ProductId = product.Id,
            product.Name,
            product.Price,
            ReviewId = (int?)review.Id ?? 0,
            Rating = (int?)review.Rating ?? 0,
            Comment = review.Comment ?? &quot;N/A&quot;
        })
    .OrderBy(x =&gt; x.ProductId)
    .ThenBy(x =&gt; x.ReviewId)
</code></pre>
<p>The generated SQL is identical to the previous example.</p>
<p>Why this is better:</p>
<ul>
<li><strong>Intent is clear</strong>: you see <code>LeftJoin</code>, you know what to expect.</li>
<li><strong>Less code</strong>, fewer moving parts (no <code>GroupJoin</code>, no <code>DefaultIfEmpty</code>, no <code>SelectMany</code>).</li>
<li><strong>Same result</strong>: all products kept, reviews may be null.</li>
</ul>
<div className="note-panel">
  **Note**: At the time of writing this article, C# **query syntax** (`from …
  select …`) doesn't have a `left join` or `right join` keyword yet. You should
  use the method syntax shown above.
</div>
<h2>Also New: <code>RightJoin</code></h2>
<p><code>RightJoin</code> keeps <strong>all rows from the right side</strong> and only matching rows from the left.
EF Core translates it to <strong>RIGHT JOIN</strong>.
It's handy when the &quot;must keep&quot; side is the <strong>second</strong> sequence.</p>
<p>Conceptually:</p>
<pre><code class="language-csharp">var query = dbContext.Reviews
    .RightJoin(
        dbContext.Products,
        review =&gt; review.ProductId,
        product =&gt; product.Id,
        (review, product) =&gt; new
        {
            ProductId = product.Id,
            product.Name,
            product.Price,
            ReviewId = (int?)review.Id ?? 0,
            Rating = (int?)review.Rating ?? 0,
            Comment = review.Comment ?? &quot;N/A&quot;
        });
</code></pre>
<p>Why use <code>RightJoin</code>: when your reporting starts from <strong>Reviews</strong> (keep all), and bring in matching <strong>Products</strong> if they exist.</p>
<p>Here's the generated SQL:</p>
<pre><code class="language-sql">SELECT
    p.&quot;Id&quot; AS &quot;ProductId&quot;,
    p.&quot;Name&quot;,
    p.&quot;Price&quot;,
    COALESCE(r.&quot;Id&quot;, 0) AS &quot;ReviewId&quot;,
    COALESCE(r.&quot;Rating&quot;, 0) AS &quot;Rating&quot;,
    COALESCE(r.&quot;Comment&quot;, 'N/A') AS &quot;Comment&quot;
FROM &quot;Reviews&quot; AS r
RIGHT JOIN &quot;Products&quot; AS p ON r.&quot;ProductId&quot; = p.&quot;Id&quot;
</code></pre>
<h2>Wrapping Up</h2>
<p>Think about how often you need left joins.
Showing all users with optional profile settings.
All products with optional reviews.
All orders with optional shipping info.
It's everywhere!</p>
<p>Before, developers would sometimes skip the proper left join and do two separate queries instead.
Or worse, they'd use an inner join and miss data.
Now there's no excuse - it's just as easy as any other LINQ method.</p>
<p>A few quick tips when writing LINQ queries:</p>
<ul>
<li>In the projection, guard nullable side: <code>review.Comment ?? &quot;N/A&quot;</code></li>
<li>Keep projections <strong>small</strong> to avoid pulling more columns than needed</li>
<li>Add indexes on the <strong>join keys</strong> for better query plans</li>
</ul>
<p>That's it.
With <code>LeftJoin</code> and <code>RightJoin</code>, the code finally matches the mental model.</p>
<p>See you next Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_166.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[The Interview Question That Changed How I Think About System Design]]></title>
            <link>https://milanjovanovic.tech/blog/the-interview-question-that-changed-how-i-think-about-system-design</link>
            <guid isPermaLink="false">the-interview-question-that-changed-how-i-think-about-system-design</guid>
            <pubDate>Sat, 25 Oct 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Discover how a simple interview question about report generation reveals the difference between optimizing code and designing scalable systems, and why the best solution isn't making it faster, but making it asynchronous.]]></description>
            <content:encoded><![CDATA[<p>About six or seven years ago, I went through an interview for a mid-level .NET role.</p>
<p>One of the questions has stayed with me ever since:</p>
<blockquote>
<p>A user clicks a button on the UI to generate an Excel or PDF report.
The report generation takes around five minutes (time can be arbitrary).
The user has to wait for it to finish.
How would you optimize this flow?</p>
</blockquote>
<p>At the time, I focused on what I knew best: <strong>performance</strong>.
I started thinking about how to make the report generation faster.
Maybe I could optimize the SQL queries, reduce data transformations, or <a href="caching-in-aspnetcore-improving-application-performance"><strong>cache</strong></a> parts of the result.
If I could get the process down from five minutes to one, that felt like a big win.</p>
<p>But even if I made it five times faster, the user still had to wait.
If the browser crashed, they lost everything.
If the network dropped, the process stopped.
If they closed the tab, all progress was gone.</p>
<p>It wasn't really a performance issue at all, it was a <strong>design issue</strong>.</p>
<h2>What I Missed Back Then</h2>
<p>Looking back, I realize I was <strong>stuck</strong> in the mindset of &quot;make the code faster&quot;.
Not that there's anything wrong with that, <strong>performance optimization is a valuable skill</strong>.
What I didn't see immediately was the <strong>bigger problem</strong>.
The app was doing all this work <strong>synchronously</strong>, holding the user hostage until it finished.
I did eventually figure it out, with a few nudges from the interviewer.</p>
<p><img src="/blogs/mnw_165/sync_reporting_flow.png" alt=""></p>
<p>The better question wasn't <em>&quot;How can I make this faster?&quot;</em></p>
<p>It was <em>&quot;Why is the user waiting in the first place?&quot;</em></p>
<p>If something takes minutes (or hours, days) to complete, it shouldn't block the user.
It should happen in the <strong>background</strong>, out of the <strong>main request flow</strong>, while the user moves on with their work.</p>
<p>Still, don't forget to optimize the code itself.
Database queries, data processing, and file generation all matter.
Maybe there's a missing index, an inefficient loop, or a better library for creating Excel files.
But those optimizations are just part of the solution, not the whole picture.</p>
<h2>How I'd Solve It Today</h2>
<p>Today, I'd still start with the same UI button.
The user clicks &quot;Generate Report,&quot; but instead of waiting, the <strong>backend accepts the request</strong>,
saves it somewhere (maybe as a job record in a database), and <strong>returns right away</strong>.
This is the essence of building <a href="building-async-apis-in-aspnetcore-the-right-way"><strong>asynchronous APIs</strong></a>.
The job is then picked up by a background worker.</p>
<p>The worker can be a <a href="running-background-tasks-in-asp-net-core"><strong>hosted service</strong></a>,
a <a href="scheduling-background-jobs-with-quartz-in-dotnet-advanced-concepts"><strong>Quartz job</strong></a>,
or even an <a href="building-fast-serverless-apis-with-minimal-apis-on-aws-lambda"><strong>AWS Lambda Function</strong></a> triggered by a queue message.
It handles the heavy lifting: pulling the data, building the file, and uploading it to storage like S3 or Azure Blob.</p>
<p>Once the report is ready, the worker updates the job status to &quot;completed&quot; and notifies the user.
That could be an email with a download link or a real-time SignalR message that shows up in the app.
The link points to the stored report, served securely from the backend.</p>
<p><img src="/blogs/mnw_165/async_reporting_flow.png" alt=""></p>
<p>Now, the user isn't waiting on a long-running HTTP request.
The server isn't holding open connections for minutes.
<strong>If something fails, it can be retried automatically</strong>.
You also have the option to <strong>track progress</strong> or <strong>cancel</strong> the job if needed.
And if a hundred users request reports at once, the system can scale without locking up.</p>
<p>The experience feels faster, even if the actual report generation time hasn't changed.
Because in the end, users don't care about performance metrics, they care about responsiveness.</p>
<h2>Why I Still Use This Question</h2>
<p>A few years later, I started using this exact question when interviewing other developers.
Not to trick anyone, but because it reveals how people think.</p>
<p>Some candidates go straight to optimizing code and queries, just like I did back then.
This is a good sign that they know their way around performance tuning.
I can proceed with further technical questions around <strong>algorithms</strong>, <strong>data structures</strong>, or <strong>database optimization</strong>.</p>
<p>Others pause for a moment and start thinking about user experience, <strong>background processing</strong>, and <strong>fault tolerance</strong>.
That's when the real conversation begins: queues, retries, notifications, secure file sharing, etc.
There are so many ways you can spin off this one scenario into a broader system design discussion.</p>
<p>There's <strong>no single right answer</strong>.
But there's a big difference between someone who focuses only on <em>code</em> and someone who can design a <em>scalable system</em>.</p>
<h2>The Lesson</h2>
<p>When I first heard this question, I thought about making the code faster.
Now I think about making the experience better.</p>
<p>Optimizing a query or loop can help, but it doesn't fix waiting, failures, or scalability.
If many users start the same report at once, a synchronous design breaks down fast.
An asynchronous flow keeps the system responsive and resilient, no matter the load.</p>
<p>That shift from optimizing functions to designing scalable systems is the difference between a good developer and a great one.</p>
<p>If you want to go deeper into building systems that scale, my <a href="/pragmatic-clean-architecture"><strong>Clean Architecture course</strong></a> walks you through exactly that.
You'll learn how to structure applications, separate concerns, and design systems that grow without breaking.</p>
<p>Hope this was helpful.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_165.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[6 Steps for Setting Up a New .NET Project the Right Way]]></title>
            <link>https://milanjovanovic.tech/blog/6-steps-for-setting-up-a-new-dotnet-project-the-right-way</link>
            <guid isPermaLink="false">6-steps-for-setting-up-a-new-dotnet-project-the-right-way</guid>
            <pubDate>Sat, 18 Oct 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to properly set up a new .NET project with EditorConfig for code consistency, Directory.Build.props for centralized configuration, central package management, static code analysis, Docker Compose or .NET Aspire for local orchestration, and GitHub Actions for CI/CD.]]></description>
            <content:encoded><![CDATA[<p>Starting a new .NET project is always exciting.
But it's also easy to skip the groundwork that makes a project scalable and maintainable.</p>
<p>Before you write your first line of business logic, there are a few key setup steps that make your life (and your teammates) much easier later on.</p>
<p>Here's how I usually set up a new .NET project in <strong>six simple steps</strong>.</p>
<h2>1. Enforce a Consistent Code Style</h2>
<p>The first thing I add is an <code>.editorconfig</code> file.</p>
<p>This file ensures everyone on your team uses the same formatting and naming conventions, reducing inconsistent indents or random naming rules.</p>
<p>You can create one directly in Visual Studio:</p>
<div class="centered">
  ![](/blogs/mnw_164/add_editorconfig.png)
</div>
<p>The default configuration is a great start.
But you can customize it further to fit your team's preferences.</p>
<p>Place it at the <strong>solution root</strong> so all projects follow the same rules.
You can still override specific settings in subfolders if needed by placing an <code>.editorconfig</code> file there.</p>
<p>Here are two sample <code>.editorconfig</code> files you can use:</p>
<ul>
<li><a href="https://github.com/dotnet/runtime/blob/main/.editorconfig">From the .NET runtime repo</a></li>
<li><a href="https://gist.github.com/m-jovanovic/417b7d0a641d7dd7d1972550fba298db">Created by me for general .NET projects</a></li>
</ul>
<h2>2. Centralize Build Configuration</h2>
<p>Next, I add a <code>Directory.Build.props</code> file to the solution root.
This file lets you define build settings that apply to every project in the solution.</p>
<p>Here's an example:</p>
<pre><code class="language-xml">&lt;Project&gt;
  &lt;PropertyGroup&gt;
    &lt;TargetFramework&gt;net10.0&lt;/TargetFramework&gt;
    &lt;ImplicitUsings&gt;enable&lt;/ImplicitUsings&gt;
    &lt;Nullable&gt;enable&lt;/Nullable&gt;
    &lt;TreatWarningsAsErrors&gt;true&lt;/TreatWarningsAsErrors&gt;
  &lt;/PropertyGroup&gt;
&lt;/Project&gt;
</code></pre>
<p>This keeps your <code>.csproj</code> files clean and consistent, since there's no need to repeat these properties in every project.</p>
<p>If you later want to enable static analyzers or tweak build options, you can do it once here.</p>
<p>What's cool about this is your <code>.csproj</code> files become basically empty, with only NuGet package references most of the time.</p>
<h2>3. Centralize Package Management</h2>
<p>As your solution grows, managing NuGet package versions across multiple projects gets painful.</p>
<p>That's where <a href="central-package-management-in-net-simplify-nuget-dependencies"><strong>central package management</strong></a> helps.</p>
<p>Create a file named <code>Directory.Packages.props</code> at the root:</p>
<pre><code class="language-xml">&lt;Project&gt;
  &lt;PropertyGroup&gt;
    &lt;ManagePackageVersionsCentrally&gt;true&lt;/ManagePackageVersionsCentrally&gt;
  &lt;/PropertyGroup&gt;

  &lt;ItemGroup&gt;
    &lt;PackageVersion Include=&quot;Microsoft.AspNetCore.OpenApi&quot; Version=&quot;10.0.0&quot; /&gt;
    &lt;PackageVersion Include=&quot;SonarAnalyzer.CSharp&quot; Version=&quot;10.15.0.120848&quot; /&gt;
  &lt;/ItemGroup&gt;
&lt;/Project&gt;
</code></pre>
<p>Now, when you reference a NuGet package in your project, you don't specify the version.
You can only use the package name like this:</p>
<pre><code class="language-xml">&lt;PackageReference Include=&quot;Microsoft.AspNetCore.OpenApi&quot; /&gt;
</code></pre>
<p>All versioning is handled centrally.
This makes dependency upgrades trivial and avoids version drift between projects.</p>
<p>You can still override versions in individual projects if needed.</p>
<h2>4. Add Static Code Analysis</h2>
<p><a href="improving-code-quality-in-csharp-with-static-code-analysis"><strong>Static code analysis</strong></a> helps catch potential bugs and maintain code quality.
.NET has a set of built-in analyzers, but I like to add <strong>SonarAnalyzer.CSharp</strong> for more comprehensive checks.</p>
<p>Let's install <strong>SonarAnalyzer.CSharp</strong> to catch potential code issues early:</p>
<pre><code class="language-powershell">Install-Package SonarAnalyzer.CSharp
</code></pre>
<p>Add it as a global package reference inside your <code>Directory.Build.props</code>:</p>
<pre><code class="language-xml">&lt;ItemGroup&gt;
  &lt;PackageReference Include=&quot;SonarAnalyzer.CSharp&quot; /&gt;
&lt;/ItemGroup&gt;
</code></pre>
<p>Combine this with:</p>
<pre><code class="language-xml">&lt;Project&gt;
  &lt;PropertyGroup&gt;
    &lt;TreatWarningsAsErrors&gt;true&lt;/TreatWarningsAsErrors&gt;
    &lt;AnalysisLevel&gt;latest&lt;/AnalysisLevel&gt;
    &lt;AnalysisMode&gt;All&lt;/AnalysisMode&gt;
    &lt;CodeAnalysisTreatWarningsAsErrors&gt;true&lt;/CodeAnalysisTreatWarningsAsErrors&gt;
    &lt;EnforceCodeStyleInBuild&gt;true&lt;/EnforceCodeStyleInBuild&gt;
  &lt;/PropertyGroup&gt;
&lt;/Project&gt;
</code></pre>
<p>…and your build will fail on serious code quality issues.
This can be a great safety net.</p>
<p>But it can also be noisy at first.
If some rules don't fit your context, you can adjust or suppress them in <code>.editorconfig</code> by setting the rule severity to <code>none</code>.</p>
<h2>5. Set Up Local Orchestration</h2>
<p>For a consistent local environment across your team, you'll want container orchestration.</p>
<p>You have two main options:</p>
<p><strong>Option 1: Docker Compose</strong></p>
<p>Add <strong>Docker Compose support</strong> in Visual Studio.
It will scaffold a <code>docker-compose.yml</code> file where you can define services like:</p>
<pre><code class="language-yaml">services:
  webapi:
    build: .
  postgres:
    image: postgres:18
    environment:
      POSTGRES_PASSWORD: password
</code></pre>
<p>This lets every developer spin up the same stack locally with one command.</p>
<p><strong>Option 2: .NET Aspire</strong></p>
<p><a href="dotnet-aspire-a-game-changer-for-cloud-native-development"><strong>.NET Aspire</strong></a> takes orchestration a step further.
It provides <a href="how-dotnet-aspire-simplifies-service-discovery"><strong>service discovery</strong></a>,
<a href="introduction-to-distributed-tracing-with-opentelemetry-in-dotnet"><strong>telemetry</strong></a>, and streamlined configuration, all integrated with your .NET projects.
It's become a <strong>personal favorite of mine</strong>.</p>
<p>You can add a .NET project and a Postgres resource with a few lines of code:</p>
<pre><code class="language-csharp">var postgres = builder.AddPostgres(&quot;demo-db&quot;);

builder.AddProject&lt;WebApi&gt;(&quot;webapi&quot;)
       .WithReference(postgres)
       .WaitFor(postgres);

builder.Build().Run();
</code></pre>
<p>Aspire still uses Docker under the hood but provides a richer developer experience.</p>
<p>Whether you pick Docker Compose or Aspire, the goal is the same: a repeatable, reliable local setup that works the same on every machine.</p>
<h2>6. Automate Builds with CI</h2>
<p>Finally, I set up a simple <a href="how-to-build-ci-cd-pipeline-with-github-actions-and-dotnet"><strong>GitHub Actions</strong></a> workflow to validate each commit.</p>
<p>Example <code>.github/workflows/build.yml</code>:</p>
<pre><code class="language-yaml">name: Build

on:
  push:
    # Filter to only run on main branch
    branches: [main]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-dotnet@v5
        with:
          dotnet-version: 10.0.x
      - run: dotnet restore
      - run: dotnet build --no-restore --configuration Release
      - run: dotnet test --no-build --configuration Release
</code></pre>
<p>This ensures your project always builds and passes tests, and it catches issues before they reach production.
If the CI build fails, you know something's wrong right away.</p>
<p>When it comes to testing, I highly recommend exploring:</p>
<ul>
<li><a href="shift-left-with-architecture-testing-in-dotnet"><strong>Architecture testing</strong></a> to enforce architectural rules in your codebase</li>
<li><a href="testcontainers-integration-testing-using-docker-in-dotnet"><strong>Integration testing with Testcontainers</strong></a>
to spin up real dependencies in Docker during tests (you can run this locally and in CI)</li>
</ul>
<p>This will give you confidence that your code works as expected in an (as close as possible) production-like environment.</p>
<h2>Wrapping Up</h2>
<p>That's a wrap.
Your <strong>new .NET project</strong> is now set up with:</p>
<ul>
<li>consistent code style</li>
<li>centralized build and package management</li>
<li>code quality enforcement</li>
<li>reproducible local orchestration</li>
<li>continuous integration</li>
</ul>
<p>These small setup steps save countless hours down the road and keep your codebase clean, predictable, and ready to scale.</p>
<p>Once your project setup is solid, the next step is designing scalable boundaries.
In my <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a> course, I show how to grow a .NET application without turning it into a tangled mess,
through clear module boundaries, messaging, and domain isolation.</p>
<p>If you're looking for a practical walkthrough of these steps, check out <a href="https://youtu.be/QRgtcbxJlo0"><strong>this video</strong></a>.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_164.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Building Fast Serverless APIs With Minimal APIs on AWS Lambda]]></title>
            <link>https://milanjovanovic.tech/blog/building-fast-serverless-apis-with-minimal-apis-on-aws-lambda</link>
            <guid isPermaLink="false">building-fast-serverless-apis-with-minimal-apis-on-aws-lambda</guid>
            <pubDate>Sat, 11 Oct 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to deploy ASP.NET Core Minimal APIs to AWS Lambda with just one library and a single line of code. We'll explore the setup process, measure real-world performance including cold start times, and discuss when serverless makes sense for your APIs.]]></description>
            <content:encoded><![CDATA[<p>Have you ever wondered if you could host a tiny .NET API without running servers 24/7?
You can!</p>
<p>AWS Lambda lets you run code on-demand, and with the <a href="https://www.nuget.org/packages/Amazon.Lambda.AspNetCoreServer.Hosting">Amazon.Lambda.AspNetCoreServer.Hosting</a> library,
you can plug an <a href="http://ASP.NET">ASP.NET</a> Core <a href="https://learn.microsoft.com/en-us/aspnet/core/fundamentals/minimal-apis">Minimal APIs</a> straight into <a href="https://aws.amazon.com/lambda/">AWS Lambda</a>.</p>
<p>In this article we'll set up a minimal API, deploy it, and discuss how it performs.</p>
<p>Don't worry if you're new to serverless, I'll keep everything simple enough to follow along.</p>
<h2>Getting Your Minimal APIs Lambda-Ready</h2>
<p>Let's start with the basics. You need just three things to turn your Minimal API into a Lambda function.</p>
<p>First, create your API:</p>
<pre><code class="language-bash">dotnet new webapi -n MyLambdaApi
cd MyLambdaApi
</code></pre>
<p>Second, add Amazon's hosting package:</p>
<pre><code class="language-powershell">Install-Package Amazon.Lambda.AspNetCoreServer.Hosting
</code></pre>
<p>Third, add one line to your <code>Program.cs</code>:</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);

// This line does all the Lambda magic
builder.Services.AddAWSLambdaHosting(LambdaEventSource.HttpApi);

var app = builder.Build();

app.MapGet(&quot;/&quot;, () =&gt; &quot;Hello from Lambda!&quot;);

app.Run();
</code></pre>
<p>That's it.
Your API now runs both locally (for testing) and in Lambda (for production).
When you run locally, it uses <a href="https://learn.microsoft.com/en-us/aspnet/core/fundamentals/servers/kestrel">Kestrel</a> like normal.
When you deploy the app to AWS, Lambda takes over.</p>
<h2>Ship It to AWS</h2>
<p>You'll need the <a href="https://github.com/aws/aws-extensions-for-dotnet-cli">Lambda tools</a> installed:</p>
<pre><code class="language-bash">dotnet tool install -g Amazon.Lambda.Tools
</code></pre>
<p>Then deploy with one command:</p>
<pre><code class="language-bash">dotnet lambda deploy-function
</code></pre>
<p>The tool asks you a few questions (like the function name, which IAM role to use).
Pick the defaults if you're just testing.
In a minute or two, your API is live with a URL like: <code>https://[abc123xyz].lambda-url.[region-name].on.aws/</code>.</p>
<p>From the AWS Management Console, you can find your function with its URL and all the other details.</p>
<div class="bordered">
  ![](/blogs/mnw_163/aws_console_lambda_function.png)
</div>
<h2>Measuring Cold Starts</h2>
<p>Here's where things become interesting (and problematic).
Lambda functions &quot;go to sleep&quot; when nobody uses them.
Waking them up takes time, this is the famous <strong>&quot;cold start&quot; problem</strong>.</p>
<p>I ran some simple tests with a basic Minimal API:</p>
<ul>
<li>First request (cold): 2,153 ms</li>
<li>Second request (warm): 154 ms</li>
<li>Third request (warm): 143 ms</li>
<li>After 10 minutes idle (cold again): 2,074 ms</li>
</ul>
<p>As you can see, the first request is slow due to the cold start.
Subsequent requests are much faster, around 150 ms.
After 10 minutes of inactivity, the function goes cold again, and the next request takes over 2 seconds.
There's an optimization feature called <a href="https://docs.aws.amazon.com/lambda/latest/dg/snapstart.html">SnapStart</a> that can help reduce cold start times,
but it has its own tradeoffs and isn't always suitable for every application.</p>
<p>Another thing to note is that I'm in Europe, and my Lambda function is in the US East (N. Virginia) region.
This adds network latency, so your results may vary based on your location and the region you choose.</p>
<p>Even when warm, the latency is higher than a typical server-hosted API.
This may be acceptable for low-traffic or non-critical endpoints, but it's something to consider.</p>
<p>Let's move beyond a &quot;Hello World&quot; example to something more realistic.</p>
<h2>CRUD Operations Benchmark</h2>
<p>To see how a simple API performs in the real world, I built a tiny CRUD app that creates a record, fetches it, updates it and then deletes it.
First I ran each operation once to see the raw latency.
Then I used a load-testing tool with 100 virtual users (VUs) to measure average latency under a bit more stress.</p>
<pre><code class="language-csharp">// POST /products - Create new product
app.MapPost(&quot;/products&quot;, async (CreateProductRequest request, NpgsqlDataSource dataSource) =&gt;
{
    const string sql =
        &quot;&quot;&quot;
        INSERT INTO Products (Name, Description, Price, CreatedAt)
        VALUES (@Name, @Description, @Price, @CreatedAt)
        RETURNING Id, Name, Description, Price, CreatedAt
        &quot;&quot;&quot;;

    await using var connection = await dataSource.OpenConnectionAsync();
    var product = await connection.QueryFirstAsync&lt;Product&gt;(sql, new
    {
        request.Name,
        request.Description,
        request.Price,
        CreatedAt = DateTime.UtcNow
    });

    return Results.Created(
        $&quot;/products/{product.Id}&quot;,
        new ProductResponse(
            product.Id,
            product.Name,
            product.Description,
            product.Price,
            product.CreatedAt));
});
// Other endpoints omitted for brevity:
// - GET /products/{id} - Get product by ID
// - PUT /products/{id} - Update product
// - DELETE /products/{id} - Delete product
</code></pre>
<p>The application uses .NET 8, <a href="https://www.npgsql.org/">Npgsql</a> and <a href="https://www.learndapper.com/">Dapper</a>
to interact with a PostgreSQL database running on <a href="https://aws.amazon.com/rds/">Amazon RDS</a>.
You can find the source code for this example (and the previous one) in <a href="https://github.com/m-jovanovic/minimal-apis-on-lambda">this repository</a>.</p>
<p>Here are the results:</p>
<p><strong>Single-call latency</strong></p>
<pre><code class="language-text">| Operation  | Latency (ms) | Notes                           |
| ---------- | ------------ | ------------------------------- |
| Create     |     537      | Cold start plus object creation |
| Read       |     134      | Simple GET of the new record    |
| Update     |     140      | Changing one property           |
| Delete     |     167      | Removing the record             |
</code></pre>
<p>The create call took half a second because it included a cold start and initialization overhead.
Once the function was warmed up, the other operations completed in under two hundred milliseconds.</p>
<p><strong>Load test with 100 virtual users</strong></p>
<p>During the load test, I simulated 100 clients hitting the API at once.
AWS automatically scaled the Lambda function to handle the traffic, and average latencies dropped because the functions were already warm.
Here are the averages:</p>
<ul>
<li><strong>CREATE avg</strong>: 129 ms</li>
<li><strong>READ avg</strong>: 132 ms</li>
<li><strong>UPDATE avg</strong>: 152 ms</li>
<li><strong>DELETE avg</strong>: 144 ms</li>
</ul>
<p>These numbers show that once your function is up and running, Lambda can respond quite quickly even when many users are making requests.
Of course, actual times will vary depending on what your API does and how it stores data.</p>
<p>Remember that each of these operations involves network calls to the database, which adds latency.
Overall, I don't find these numbers bad for a serverless setup.</p>
<h2>Summary</h2>
<p>Hosting minimal APIs in AWS Lambda is <strong>surprisingly straightforward</strong>.
With one library and a single method call, your <a href="http://ASP.NET">ASP.NET</a> Core code can run &quot;without servers&quot;.
You pay only for the compute you use.
Also, the AWS Lambda free tier includes one million free requests per month, which is great for testing and light usage.</p>
<p>However, there are tradeoffs.
Because of cold starts and the overhead of starting the .NET runtime, latency isn't as low as hosting on a dedicated server.
If your API is mission-critical or requires sub-100 ms responses at all times, you may need to look at provisioned concurrency, containers, or a traditional host.</p>
<p>Lambda shines for small, <a href="event-driven-architecture-in-dotnet-with-rabbitmq"><strong>event-driven</strong></a> or intermittent workloads that don't justify a full-time server.
But it's less suitable for latency-sensitive or heavy, long-running applications.
For occasional or low-traffic API endpoints, though, <strong>Lambda</strong> offers a <strong>cost-effective</strong> and <strong>simple</strong> option.</p>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_163.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Using Stored Procedures and Functions With EF Core and PostgreSQL]]></title>
            <link>https://milanjovanovic.tech/blog/using-stored-procedures-and-functions-with-ef-core-and-postgresql</link>
            <guid isPermaLink="false">using-stored-procedures-and-functions-with-ef-core-and-postgresql</guid>
            <pubDate>Sat, 04 Oct 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to use PostgreSQL stored procedures and functions with EF Core to handle complex queries, atomic operations with locking, and database-specific features while keeping the type safety and convenience of EF Core.]]></description>
            <content:encoded><![CDATA[<p>You're building a .NET application with <a href="https://learn.microsoft.com/en-us/ef/core/">EF Core</a>.
Most of your queries work fine with <a href="why-i-write-tall-linq-queries"><strong>LINQ</strong></a>, but now you're hitting scenarios where you need something more.</p>
<p>Maybe you have a complex report that joins five tables with aggregations and window functions.
Your LINQ query generates SQL that's slower than you'd like, and you know you could write better SQL by hand.</p>
<p>Or maybe you need to update inventory with proper locking to prevent race conditions.
You could manage <a href="working-with-transactions-in-ef-core"><strong>transactions</strong></a> and explicit locks in C#, but it feels like you're fighting the framework.</p>
<p>Here's what usually happens: you search for &quot;EF Core stored procedures&quot; and find conflicting advice.
Some articles say avoid raw SQL at all costs.
Others suggest abandoning EF entirely and writing <a href="http://ADO.NET">ADO.NET</a>.
Neither feels right.</p>
<p>Actually, <strong>EF Core works great with database functions and procedures</strong>.
You get the database's power for what it does best, and EF's convenience for everything else.
Let me show you how this actually works.</p>
<p>I'll use <a href="https://www.postgresql.org/">PostgreSQL</a> for examples, but the same principles apply to SQL Server and other relational databases.</p>
<h2>When Should You Even Use Raw SQL?</h2>
<p>Let's be honest: most of the time, <strong>LINQ is fine</strong>.
EF Core translates your C# into decent SQL, and you get type safety and refactoring support.</p>
<p>But there are times when raw SQL makes more sense:</p>
<p><strong>You need performance you can't get from LINQ</strong>.
Complex aggregations with multiple joins, <a href="https://www.postgresql.org/docs/current/tutorial-window.html">window functions</a>, or reporting queries often run faster when written directly in SQL.
You can test and tune the query in your database tool before bringing it into your code.</p>
<p><strong>You're using database-specific features</strong>.
PostgreSQL has powerful capabilities like <a href="https://www.postgresql.org/docs/current/textsearch.html">full-text search</a>, JSON operators,
and <a href="https://www.postgresql.org/docs/current/queries-with.html">common table expressions (CTEs)</a> that don't always have clean LINQ equivalents.
Sometimes the straightest path is just writing the SQL.</p>
<p><strong>You have existing database logic</strong>.
If your database already has stored procedures and functions (maybe from a <a href="what-rewriting-a-40-year-old-project-taught-me-about-software-development"><strong>legacy system</strong></a>),
calling them directly beats rewriting everything in C#.</p>
<p><strong>You need atomic operations with proper locking</strong>.
A stored procedure that coordinates multiple updates with <code>FOR UPDATE</code> locks (<a href="scaling-the-outbox-pattern"><strong>here's a good use case</strong></a>)
is simpler and safer than trying to manage that from application code.</p>
<p><strong>You want to reduce round trips</strong>.
One function call that aggregates data from five tables is more efficient than five separate LINQ queries.</p>
<p>Now let's see how to actually do this.</p>
<h2>Example 1: Simple Scalar Function</h2>
<p>Here's a straightforward function that tells you how many tickets are left:</p>
<pre><code class="language-sql">CREATE OR REPLACE FUNCTION ticketing.tickets_left(p_ticket_type_id uuid)
RETURNS numeric
LANGUAGE sql
AS $$
  SELECT tt.available_quantity
  FROM ticketing.ticket_types tt
  WHERE tt.id = p_ticket_type_id
$$;
</code></pre>
<p>Nothing fancy, just a query wrapped in a function.</p>
<p>Calling it from EF Core is straightforward:</p>
<pre><code class="language-csharp">app.MapGet(&quot;ticket-types/{ticketTypeId}/available-quantity&quot;,
async (Guid ticketTypeId, EventManagementContext dbContext) =&gt;
{
    var result = await dbContext.Database.SqlQuery&lt;int&gt;(
            $&quot;&quot;&quot;
             SELECT ticketing.tickets_left({ticketTypeId}) AS &quot;Value&quot;
             &quot;&quot;&quot;)
        .FirstAsync();

    return Results.Ok(result);
});
</code></pre>
<p>Notice the <code>AS &quot;Value&quot;</code> alias.
When EF Core maps to a primitive type, it expects a property named <code>Value</code>.
The quotes preserve the exact casing (PostgreSQL lowercases unquoted identifiers by default).</p>
<p>The interpolated string syntax (<code>$&quot;{ticketTypeId}&quot;</code>) might look dangerous, but EF Core converts this into a parameterized query automatically.
You're not building SQL strings, you're using C# interpolation as a convenient syntax for parameters.</p>
<h2>Example 2: Table-Valued Function</h2>
<p>Functions can return entire result sets, which is where they really shine:</p>
<pre><code class="language-sql">CREATE OR REPLACE FUNCTION ticketing.customer_order_summary(p_customer_id uuid)
RETURNS TABLE (
    order_id uuid,
    created_at_utc timestamptz,
    total_price numeric,
    currency text,
    item_count numeric
)
LANGUAGE sql
AS $$
SELECT
    o.id,
    o.created_at_utc,
    o.total_price,
    o.currency,
    COALESCE(SUM(oi.quantity), 0) AS item_count
FROM ticketing.orders o
LEFT JOIN ticketing.order_items oi ON oi.order_id = o.id
WHERE o.customer_id = p_customer_id
GROUP BY o.id, o.created_at_utc, o.total_price, o.currency
ORDER BY o.created_at_utc DESC
$$;
</code></pre>
<p>This function joins orders with their items, aggregates quantities, and returns multiple rows.
You could write this in LINQ, but the SQL is clearer and you can test it directly in your database tool.</p>
<p>To use it from C#, create a DTO that matches the function's output:</p>
<pre><code class="language-csharp">public class OrderSummaryDto
{
    public Guid OrderId { get; set; }
    public DateTime CreatedAtUtc { get; set; }
    public decimal TotalPrice { get; set; }
    public string Currency { get; set; }
    public int ItemCount { get; set; }
}
</code></pre>
<p>Then query the function like any other table:</p>
<pre><code class="language-csharp">app.MapGet(&quot;customers/{customerId}/order-summary&quot;,
async (Guid customerId, EventManagementContext dbContext) =&gt;
{
    var orders = await dbContext.Database
        .SqlQuery&lt;OrderSummaryDto&gt;(
            $&quot;&quot;&quot;
             SELECT
                order_id AS OrderId,
                created_at_utc AS CreatedAtUtc,
                total_price AS TotalPrice,
                currency AS Currency,
                item_count AS ItemCount
             FROM ticketing.customer_order_summary({customerId})
             &quot;&quot;&quot;)
        .ToListAsync();

    return Results.Ok(orders);
});
</code></pre>
<p>The key is mapping column names to your DTO properties using aliases.
EF Core handles the rest automatically.</p>
<p>This is a simple case without joins, but you can use this pattern in more complex queries too.
However, you will have to project into DTOs manually since EF Core can't translate joins in raw SQL into entity graphs.
Usually, you'll return a flat structure from functions anyway, and then map to richer models in C# if needed.</p>
<h2>Understanding PostgreSQL Functions vs Procedures</h2>
<p>PostgreSQL distinguishes between functions and procedures in important ways:</p>
<p><strong>Functions</strong> are designed to <strong>return values</strong>.
They can return scalar values, tables, or even complex JSON objects.
You call them with <code>SELECT</code> and can use them in queries like any other expression.
Functions run within a transaction and can be used in <code>WHERE</code> clauses, joins, and other query contexts.</p>
<p><strong>Procedures</strong> are designed for <strong>side effects</strong>.
They don't return values directly but can modify data and have <code>OUT</code> parameters.
You call them with <code>CALL</code> and they're ideal for complex operations that need to manage transactions explicitly or perform multiple related updates.</p>
<p>Think of it this way: use functions when you need data, use procedures when you need to change something.</p>
<p>This distinction matters because it affects how you design your database logic and how you call these routines from C#.</p>
<p>Let's see an example of a procedure.</p>
<h2>Example 3: Stored Procedure with Validation</h2>
<p>Here's where procedures really prove their worth.
Let's say you need to adjust ticket inventory, but you want to prevent <a href="solving-race-conditions-with-ef-core-optimistic-locking"><strong>race conditions</strong></a> and validate the operation:</p>
<pre><code class="language-sql">CREATE OR REPLACE PROCEDURE ticketing.adjust_available_quantity(
    p_ticket_type_id uuid,
    p_delta numeric,
    p_reason text DEFAULT 'manual-adjust'
)
LANGUAGE plpgsql
AS $$
DECLARE
    v_qty numeric;
    v_avail numeric;
    v_new_avail numeric;
BEGIN
    SELECT quantity, available_quantity
    INTO v_qty, v_avail
    FROM ticketing.ticket_types
    WHERE id = p_ticket_type_id
    FOR UPDATE;

    IF NOT FOUND THEN
        RAISE EXCEPTION 'ticket_type % not found', p_ticket_type_id;
    END IF;

    v_new_avail := v_avail + p_delta;

    IF v_new_avail &lt; 0 THEN
        RAISE EXCEPTION 'Cannot reduce below zero';
    END IF;

    IF v_new_avail &gt; v_qty THEN
        RAISE EXCEPTION 'Cannot exceed quantity';
    END IF;

    UPDATE ticketing.ticket_types
    SET available_quantity = v_new_avail
    WHERE id = p_ticket_type_id;
END;
$$;
</code></pre>
<p>This procedure does several important things:</p>
<ul>
<li><strong>Locks the row</strong> with <code>FOR UPDATE</code> so no other transaction can modify it until we're done</li>
<li><strong>Validates business rules</strong> before making changes</li>
<li><strong>Provides clear error messages</strong> when something goes wrong</li>
<li><strong>Keeps everything atomic</strong> in a single database round trip</li>
</ul>
<p>You could do all this in C# with manual transaction management and explicit locking, but it's more complex and error-prone.
Let the database handle what it's good at.</p>
<p>Here's how you call it from EF Core:</p>
<pre><code class="language-csharp">app.MapPut(&quot;ticket-types/{ticketTypeId}/available-quantity&quot;, async (
    Guid ticketTypeId,
    int quantity,
    EventManagementContext dbContext) =&gt;
{
    try
    {
        await dbContext.Database.ExecuteSqlAsync(
            $&quot;&quot;&quot;
             CALL ticketing.adjust_available_quantity({ticketTypeId},{quantity})
             &quot;&quot;&quot;);

        return Results.Ok(result);
    }
    catch (Exception e)
    {
        return Results.BadRequest(e.Message);
    }
});
</code></pre>
<p>The procedure doesn't return a value, but if it raises an exception (with <code>RAISE EXCEPTION</code>), PostgreSQL will propagate that to your C# code.
You can catch it and return a proper error response.</p>
<h2>About SQL Injection (Don't Panic)</h2>
<p>You might be looking at those interpolated strings and thinking &quot;wait, isn't this <a href="ef-core-raw-sql-queries"><strong>SQL injection waiting to happen</strong></a>?&quot;</p>
<p><strong>It's not</strong>.</p>
<p>When you write:</p>
<pre><code class="language-csharp">$&quot;SELECT * FROM users WHERE id = {userId}&quot;
</code></pre>
<p>EF Core doesn't concatenate strings.
It converts this into:</p>
<pre><code class="language-sql">SELECT * FROM users WHERE id = @p0
</code></pre>
<p><strong>The actual value is sent as a parameter</strong>, completely separate from the SQL text.
This works for all the examples in this article.</p>
<p>The interpolation syntax is just a convenient way to write parameterized queries.</p>
<p>The reason why is we're not actually passing in a <code>string</code> to the <code>SqlQuery</code> method,
but a <a href="https://learn.microsoft.com/en-us/dotnet/api/system.formattablestring"><code>FormattableString</code></a>.
This is a special type that captures the format and arguments separately, allowing EF Core to handle parameters.</p>
<p>Everything in this article works with <a href="https://learn.microsoft.com/en-us/sql/">SQL Server</a>,
<a href="https://dev.mysql.com/">MySQL</a>,
<a href="https://www.sqlite.org/index.html">SQLite</a>, and other databases EF Core supports.
The differences are mostly syntax.</p>
<h2>A Quick Word on Views</h2>
<p>Database views are like functions without parameters.
They're saved queries you can reference by name.</p>
<p>You can query them using <code>SqlQuery&lt;T&gt;</code> just like functions:</p>
<pre><code class="language-csharp">var results = await dbContext.Database
    .SqlQuery&lt;ActiveCustomerDto&gt;(
        $&quot;SELECT * FROM ticketing.active_customers&quot;)
    .ToListAsync();
</code></pre>
<p>Or you can map them to entity types in your <code>DbContext</code> for full LINQ support.</p>
<p>Views are great for frequently-used queries that don't need parameters.
Functions give you the flexibility of parameterization.</p>
<h2>Wrapping Up</h2>
<p>We've covered how to use PostgreSQL functions and procedures with EF Core, from simple scalar functions to complex procedures with validation and locking.</p>
<p>You learned when to use functions (when you need data back) versus procedures (when you need to modify data).
You saw how EF Core's <code>SqlQuery&lt;T&gt;</code> and <code>ExecuteSqlAsync</code> give you type safety while letting you write the SQL you need.
And you learned when raw SQL makes sense: complex aggregations, database-specific features, atomic operations with locking, and reducing round trips.</p>
<p><strong>EF Core</strong> doesn't force you to choose between LINQ and raw SQL.
You can use both.</p>
<p>Use functions when you need to return data, procedures when you need to modify data with complex logic,
and raw SQL queries when LINQ doesn't capture your requirements efficiently.
The combination of EF Core's convenience and the database's power gives you the flexibility to choose the right tool for each scenario.</p>
<p>That's all for today. Hope this was helpful.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_162.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Building Secure APIs with Role-Based Access Control in ASP.NET Core]]></title>
            <link>https://milanjovanovic.tech/blog/building-secure-apis-with-role-based-access-control-in-aspnetcore</link>
            <guid isPermaLink="false">building-secure-apis-with-role-based-access-control-in-aspnetcore</guid>
            <pubDate>Sat, 27 Sep 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to implement Role-Based Access Control (RBAC) in ASP.NET Core with custom authorization handlers, permission-based policies, and clean extension methods for both Minimal APIs and MVC controllers.]]></description>
            <content:encoded><![CDATA[<p>Authentication tells you <strong>who</strong> the user is.
Authorization tells you <strong>what</strong> they can do.</p>
<p>Most .NET developers start with simple role-based checks: &quot;Is this user an Admin?&quot;
But as your application grows, you quickly realize that roles alone aren't enough.
You need <strong>granular permissions</strong> that can be combined and assigned flexibly.</p>
<p>That's where <strong>Role-Based Access Control (RBAC)</strong> shines.
Instead of hardcoding role checks everywhere, you define specific permissions and let roles carry those permissions.
A user might be a <code>Manager</code> role, but what matters is whether they have the <code>users:delete</code> permission.</p>
<p>Let me show you how to build a flexible, <strong>permission-based authorization</strong> system in <a href="http://ASP.NET">ASP.NET</a> Core.</p>
<h2>Understanding RBAC Components</h2>
<p><a href="https://auth0.com/docs/manage-users/access-control/rbac">RBAC</a> has three key components that work together:</p>
<p><strong>Users</strong> → assigned to → <strong>Roles</strong> → which contain → <strong>Permissions</strong></p>
<p>Here's how it flows:</p>
<ul>
<li><strong>Users</strong>: Individual people using your system</li>
<li><strong>Roles</strong>: Groups of related permissions (Admin, Manager, Editor)</li>
<li><strong>Permissions</strong>: Specific actions users can perform (users:read, orders:create, reports:delete)</li>
</ul>
<p><img src="/blogs/mnw_161/rbac.png" alt=""></p>
<p>The beauty is in the flexibility.
You can assign multiple roles to a user, and roles can be modified without touching user assignments.
Need to give all Managers the ability to export reports?
Just add the <code>reports:export</code> permission to the <code>Manager</code> role.</p>
<p>This is much more maintainable than checking if someone is specifically an <code>Admin</code> or <code>Super Manager</code> in your code.</p>
<p>It also adds an extra extension point: you can implement custom permissions for some users without creating new roles.</p>
<h2>Building a Custom Authorization Handler</h2>
<p><a href="https://learn.microsoft.com/en-us/aspnet/core/security/authorization/introduction">ASP.NET Core's authorization system</a> is built around <strong>policies</strong> and <strong>requirements</strong>.
Let's create a custom handler that checks permissions stored in the user's claims:</p>
<pre><code class="language-csharp">public class PermissionAuthorizationRequirement(params string[] allowedPermissions)
    : AuthorizationHandler&lt;PermissionAuthorizationRequirement&gt;, IAuthorizationRequirement
{
    public string[] AllowedPermissions { get; } = allowedPermissions;

    protected override Task HandleRequirementAsync(
        AuthorizationHandlerContext context,
        PermissionAuthorizationRequirement requirement)
    {
        foreach (var permission in requirement.AllowedPermissions)
        {
            bool found = context.User.FindFirst(c =&gt;
                c.Type == CustomClaimTypes.Permission &amp;&amp;
                c.Value == permission) is not null;

            if (found)
            {
                context.Succeed(requirement);
                break;
            }
        }
        return Task.CompletedTask;
    }
}
</code></pre>
<p>Here's what's happening under the hood:</p>
<p>The class combines both the <strong>requirement</strong> (what permissions are needed) and the <strong>handler</strong> (how to check them).
This keeps related logic together and reduces boilerplate.</p>
<p>The handler looks through the user's claims for any claim with type <code>Permission</code> that matches one of the required permissions.
It's an <strong>OR operation</strong> - the user only needs <strong>one</strong> of the specified permissions to proceed.</p>
<p>If a matching permission is found, we call <code>context.Succeed(requirement)</code> and break out early.
No need to check the remaining permissions.</p>
<p>Alternatively, you could implement an <strong>AND operation</strong> if your use case requires all permissions to be present.</p>
<p>You'll need to define your custom claim type:</p>
<pre><code class="language-csharp">public static class CustomClaimTypes
{
    public const string Permission = &quot;permission&quot;;
}
</code></pre>
<p>And then you'll use this when issuing JWT tokens or setting up user claims.</p>
<pre><code class="language-csharp">var permissions = await (
        from role in dbContext.Roles
        join permission in dbContext.RolePermissions on role.Id equals permission.RoleId
        where roles.Contains(role.Name)
        select permission.Name)
    .Distinct()
    .ToArrayAsync();

List&lt;Claim&gt; claims =
[
    new(JwtRegisteredClaimNames.Sub, user.Id),
    new(JwtRegisteredClaimNames.Email, user.Email!),
    ..roles.Select(r =&gt; new Claim(ClaimTypes.Role, r)),
    ..permissions.Select(p =&gt; new Claim(CustomClaimTypes.Permission, p))
];

var tokenDescriptor = new SecurityTokenDescriptor
{
    Subject = new ClaimsIdentity(claims),
    Expires = DateTime.UtcNow.AddMinutes(configuration.GetValue&lt;int&gt;(&quot;Jwt:ExpirationInMinutes&quot;)),
    SigningCredentials = credentials,
    Issuer = configuration[&quot;Jwt:Issuer&quot;],
    Audience = configuration[&quot;Jwt:Audience&quot;]
};

var tokenHandler = new JsonWebTokenHandler();

string accessToken = tokenHandler.CreateToken(tokenDescriptor);
</code></pre>
<h2>Creating Clean APIs with Extension Methods</h2>
<p>Raw authorization policies work, but they're verbose. Let's create extension methods that make the developer experience much cleaner:</p>
<pre><code class="language-csharp">public static class PermissionExtensions
{
    public static void RequirePermission(
        this AuthorizationPolicyBuilder builder,
        params string[] allowedPermissions)
    {
        builder.AddRequirements(new PermissionAuthorizationRequirement(allowedPermissions));
    }
}
</code></pre>
<p>Now you can use this with <strong>Minimal APIs</strong>:</p>
<pre><code class="language-csharp">public static class Permissions
{
    public const string UsersRead = &quot;users:read&quot;;
    public const string UsersUpdate = &quot;users:update&quot;;
    public const string UsersDelete = &quot;users:delete&quot;;
}

app.MapGet(&quot;me&quot;, (ApplicationDbContext dbContext) =&gt;
{
    var user = await dbContext.Users
        .AsNoTracking()
        .Where(u =&gt; u.Id == int.Parse(User.FindFirstValue(JwtRegisteredClaimNames.Sub)!))
        .Select(u =&gt; new UserDto
        {
            u.Id,
            u.Email,
            u.FirstName,
            u.LastName
        })
        .SingleOrDefaultAsync();

    return Results.Ok(user);
})
.RequireAuthorization(policy =&gt; policy.RequirePermission(Permissions.UsersRead));
</code></pre>
<p>For <strong>MVC Controllers</strong>, create an attribute:</p>
<pre><code class="language-csharp">[AttributeUsage(AttributeTargets.Method | AttributeTargets.Class)]
public class RequirePermissionAttribute(params string[] permissions) : AuthorizeAttribute
{
    public RequirePermissionAttribute(params string[] permissions)
        : base(policy: string.Join(&quot;,&quot;, permissions))
    {
    }
}
</code></pre>
<p>Then register the policy in your DI container:</p>
<pre><code class="language-csharp">builder.Services.AddAuthorizationBuilder()
    .AddPolicy(&quot;users:read&quot;, policy =&gt; policy.RequirePermission(Permissions.UsersRead))
    .AddPolicy(&quot;users:update&quot;, policy =&gt; policy.RequirePermission(Permissions.UsersUpdate));
</code></pre>
<p>Usage becomes clean:</p>
<pre><code class="language-csharp">[RequirePermission(Permissions.UsersUpdate)]
public async Task&lt;IActionResult&gt; UpdateUser(int id, UpdateUserRequest request)
{
    // Your logic here
}
</code></pre>
<h2>Extension Points for Production</h2>
<p>The basic implementation works great, but we could improve it further.
Here are two key extension points:</p>
<h3>Type-Safe Permissions with Enums</h3>
<p>Instead of magic strings, use enums for compile-time safety:</p>
<pre><code class="language-csharp">public enum Permission
{
    UsersRead,
    UsersUpdate,
    UsersDelete,
    OrdersCreate,
    ReportsExport
}
</code></pre>
<p>You'll have to convert these to strings when issuing claims and checking permissions.
And also convert from a string to an enum, when reading from claims and validating the permissions.</p>
<h3>Server-Side Permission Resolution</h3>
<p>Rather than storing all permissions in JWT tokens (which can get large), fetch them server-side using <code>IClaimsTransformation</code>:</p>
<pre><code class="language-csharp">public class PermissionClaimsTransformation(IPermissionService permissionService)
    : IClaimsTransformation
{
    public async Task&lt;ClaimsPrincipal&gt; TransformAsync(ClaimsPrincipal principal)
    {
        if (principal.Identity?.IsAuthenticated != true)
        {
            return principal;
        }

        var userId = principal.FindFirst(ClaimTypes.NameIdentifier)?.Value;
        if (userId == null)
        {
            return principal;
        }

        // Fetch permissions from database, then cache
        // IMPORTANT: Cache these results to avoid DB hits on every request
        var permissions = await permissionService.GetUserPermissionsAsync(userId);

        var claimsIdentity = (ClaimsIdentity)principal.Identity;
        foreach (var permission in permissions)
        {
            claimsIdentity.AddClaim(new Claim(CustomClaimTypes.Permission, permission));
        }

        return principal;
    }
}
</code></pre>
<p>Register it in your DI container:</p>
<pre><code class="language-csharp">builder.Services.AddScoped&lt;IClaimsTransformation, PermissionClaimsTransformation&gt;();
</code></pre>
<p>This approach keeps your JWTs lightweight while still providing fast authorization checks through claims.</p>
<p>You can learn more about claims transformation in my <a href="master-claims-transformation-for-flexible-aspnetcore-authorization"><strong>previous article</strong></a>.</p>
<h2>Takeaway</h2>
<p><strong>RBAC</strong> transforms authorization from a maintenance headache into a flexible, scalable system.</p>
<p><strong>Start with permissions</strong>: Define what actions users can perform, not what roles they have.</p>
<p><strong>Custom authorization handlers</strong> give you complete control over how permissions are validated.</p>
<p><strong>Extension methods</strong> make the developer experience clean and consistent across your API.</p>
<p>For production systems, consider <strong>type-safe enums</strong> and <strong>server-side permission resolution</strong> to keep your tokens lean and your code maintainable.</p>
<p>The result? Authorization logic that's easy to understand, test, and modify as your application evolves.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_161.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Distributed Locking in .NET: Coordinating Work Across Multiple Instances]]></title>
            <link>https://milanjovanovic.tech/blog/distributed-locking-in-dotnet-coordinating-work-across-multiple-instances</link>
            <guid isPermaLink="false">distributed-locking-in-dotnet-coordinating-work-across-multiple-instances</guid>
            <pubDate>Sat, 20 Sep 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to coordinate work across multiple application instances using distributed locking in .NET, preventing race conditions and ensuring data consistency in scaled-out systems. Explore implementation approaches from PostgreSQL advisory locks to the DistributedLock library for production-ready solutions.]]></description>
            <content:encoded><![CDATA[<p>When you build applications that run across multiple servers or processes, you eventually run into the problem of concurrent access.
Multiple workers try to update the same resource at the same time, and you end up with race conditions, duplicated work, or corrupted data.</p>
<p>.NET provides excellent <a href="introduction-to-locking-and-concurrency-control-in-dotnet-6"><strong>concurrency control primitives</strong></a> for single-process scenarios,
like <code>lock</code>, <code>SemaphoreSlim</code>, and <code>Mutex</code>.
But when your application is scaled out across multiple instances, these primitives don't work anymore.</p>
<p>That's where <strong>distributed locking</strong> comes in.</p>
<p>Distributed locking provides a solution by ensuring <strong>only one node</strong> (application instance) can access a critical section at a time,
preventing race conditions and maintaining data consistency <strong>across your distributed system</strong>.</p>
<h2>Why and When You Need Distributed Locking</h2>
<p>In a single-process app, you can just use <code>lock</code> or the new <a href="https://learn.microsoft.com/en-us/dotnet/api/system.threading.lock">Lock class</a> in .NET 10.
But once you scale out, that's not enough, because each process has its own memory space.</p>
<p>A few common cases where distributed locks are valuable:</p>
<ul>
<li><strong>Background jobs</strong>: ensuring only one worker processes a particular job or resource at a time.</li>
<li><strong>Leader election</strong>: choosing a single process to perform periodic work (like applying async database projections).</li>
<li><strong>Avoiding double execution</strong>: ensuring scheduled tasks don't run multiple times when deployed to multiple instances.</li>
<li><strong>Coordinating shared resources</strong>: e.g., only one service instance performing a migration or cleanup at a time.</li>
<li><strong>Cache stampede prevention</strong>: ensuring only one instance refreshes the cache when a given cache key expires.</li>
</ul>
<p>The key value: consistency and safety across distributed environments.
Without this, you risk duplicate operations, corrupted state, or unnecessary load.</p>
<p>Now you know why distributed locking is important.</p>
<p>Let's look at some implementation options.</p>
<h2>DIY Distributed Locking with PostgreSQL Advisory Locks</h2>
<p>Let's start simple.
PostgreSQL has a feature called <a href="https://www.postgresql.org/docs/current/explicit-locking.html#ADVISORY-LOCKS">advisory locks</a> that's perfect for distributed locking.
Unlike table locks, these don't interfere with your data - they're purely for coordination.</p>
<p>Here's an example:</p>
<pre><code class="language-csharp">public class NightlyReportService(NpgsqlDataSource dataSource)
{
    public async Task ProcessNightlyReport()
    {
        await using var connection = dataSource.OpenConnection();

        var key = HashKey(&quot;nightly-report&quot;);

        var acquired = await connection.ExecuteScalarAsync&lt;bool&gt;(
            &quot;SELECT pg_try_advisory_lock(@key)&quot;,
            new { key });

        if (!acquired)
        {
            throw new ConflictException(&quot;Another instance is already processing the nightly report&quot;);
        }

        try
        {
            await DoWork();
        }
        finally
        {
            await connection.ExecuteAsync(
                &quot;SELECT pg_advisory_unlock(@key)&quot;,
                new { key });
        }
    }

    private static long HashKey(string key) =&gt;
        BitConverter.ToInt64(SHA256.HashData(Encoding.UTF8.GetBytes(key)), 0);

    private static Task DoWork() =&gt; Task.Delay(5000); // Your actual work here
}
</code></pre>
<p>Here's what's happening under the hood.</p>
<p>First, we convert our lock name into a number.
PostgreSQL <strong>advisory locks need numeric keys</strong>, so we hash <code>nightly-report</code> into a 64-bit integer.
Every node (application instance) must generate the same number for the same string, or this won't work.</p>
<p>Next, <code>pg_try_advisory_lock()</code> attempts to grab an exclusive lock on that number.
It returns <code>true</code> if successful, <code>false</code> if another connection already holds it.
This call doesn't block - it tells you immediately whether you got the lock.</p>
<p>If we get the lock, we do our work.
If not, we return a conflict response and let the other instance handle it.</p>
<p>The <code>finally</code> block ensures we always release the lock, even if something goes wrong.
PostgreSQL also <strong>automatically releases advisory locks when connections close</strong>, which is a nice safety net.</p>
<p>SQL Server has a similar feature with <a href="https://learn.microsoft.com/en-us/sql/relational-databases/system-stored-procedures/sp-getapplock-transact-sql">sp_getapplock</a>.</p>
<h2>Exploring the DistributedLock Library</h2>
<p>While the DIY approach works, production applications need more sophisticated features.
The <a href="https://github.com/madelson/DistributedLock">DistributedLock</a> library handles the edge cases and
provides multiple backend options (Postgres, Redis, SqlServer, etc.).
You know I'm a fan of not reinventing the wheel, so this is a great choice.</p>
<p>Install the package:</p>
<pre><code class="language-powershell">Install-Package DistributedLock
</code></pre>
<p>I'll use the approach with <code>IDistributedLockProvider</code> which works nicely with DI.
You can acquire a lock without having to know anything about the underlying infrastructure.
All you have to do is register a lock provider implementation in your DI container.</p>
<p>For example, using Postgres:</p>
<pre><code class="language-csharp">// Register the distributed lock provider
builder.Services.AddSingleton&lt;IDistributedLockProvider&gt;(
    (_) =&gt;
    {
        return new PostgresDistributedSynchronizationProvider(
            builder.Configuration.GetConnectionString(&quot;distributed-locking&quot;)!);
    });
</code></pre>
<p>Or if you want to use Redis with the <a href="https://redis.io/docs/latest/develop/clients/patterns/distributed-locks/">Redlock algorithm</a>:</p>
<pre><code class="language-csharp">// Requires StackExchange.Redis
builder.Services.AddSingleton&lt;IConnectionMultiplexer&gt;(
    (_) =&gt;
    {
        return ConnectionMultiplexer.Connect(
            builder.Configuration.GetConnectionString(&quot;redis&quot;)!);
    });

// Register the distributed lock provider
builder.Services.AddSingleton&lt;IDistributedLockProvider&gt;(
    (sp) =&gt;
    {
        var connectionMultiplexer = sp.GetRequiredService&lt;IConnectionMultiplexer&gt;();

        return new RedisDistributedSynchronizationProvider(connectionMultiplexer.GetDatabase());
    });
</code></pre>
<p>The usage is straightforward:</p>
<pre><code class="language-csharp">// You can also pass in a timeout, where the provider will keep retrying to acquire the lock
// until the timeout is reached.
IDistributedSynchronizationHandle? distributedLock = distributedLockProvider
    .TryAcquireLock(&quot;nightly-report&quot;);

// If we didn't get the lock, the object will be null
if (distributedLock is null)
{
    return Results.Conflict();
}

// It's important to wrap the lock in a using statement to ensure it's released properly
using (distributedLock)
{
    await DoWork();
}
</code></pre>
<p>The library handles all the tricky parts: timeouts, retries, and ensuring locks are released even in failure scenarios.</p>
<p>It also supports many backends (SQL Server, Azure, ZooKeeper, etc.), making it a solid choice for production workloads.</p>
<h2>Wrapping Up</h2>
<p><strong>Distributed locking</strong> isn't something you need every day.
But when you do, it saves you from subtle, painful bugs that only appear under load or in production.</p>
<p><strong>Start simple</strong>: if you're already using Postgres, <strong>advisory locks</strong> are a powerful tool.</p>
<p>For a cleaner developer experience, reach for the <strong>DistributedLock library</strong>.</p>
<p>Choose the backend that fits your infrastructure (Postgres, Redis, SQL Server, etc.).</p>
<p>The right lock at the right time ensures your system stays consistent, reliable, and resilient, even across multiple processes and servers.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_160.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Vertical Slice Architecture Is Easier Than You Think]]></title>
            <link>https://milanjovanovic.tech/blog/vertical-slice-architecture-is-easier-than-you-think</link>
            <guid isPermaLink="false">vertical-slice-architecture-is-easier-than-you-think</guid>
            <pubDate>Sat, 13 Sep 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how Vertical Slice Architecture organizes .NET code by business features instead of technical layers, keeping related functionality together and making your codebase easier to navigate and maintain.]]></description>
            <content:encoded><![CDATA[<p>Let's say you need to add an &quot;export user data&quot; <strong>feature</strong> to your .NET application.
Users click a button, your system generates their data export, uploads it to cloud storage, and emails them a secure download link.</p>
<p>In your current <strong>layered architecture</strong> with a <strong>technical folder structure</strong>, you'll probably touch six different folders:
<code>Controllers</code>, <code>Services</code>, <code>Models</code>, <code>DTOs</code>, <code>Repositories</code>, and <code>Validators</code>.
You'll scroll up and down your solution explorer, lose your train of thought,
and wonder why adding one feature requires editing files scattered across your entire codebase.</p>
<p>If this sounds familiar, you're not alone.
Most .NET developers start with the &quot;standard&quot; layered architecture, organizing code by <strong>technical concerns</strong> rather than <strong>business features</strong>.</p>
<p>But there's a better way: <a href="vertical-slice-architecture"><strong>Vertical Slice Architecture</strong></a>.</p>
<h2>What is Vertical Slice Architecture?</h2>
<p>Instead of organizing your code by technical layers (<code>Controllers</code>, <code>Services</code>, <code>Repositories</code>),
<strong>Vertical Slice Architecture</strong> organizes it by <strong>business features</strong>.
Each feature becomes a <strong>self-contained</strong> &quot;slice&quot; that includes <strong>everything needed for that specific functionality</strong>.</p>
<p>Think of it this way: traditional <a href="clean-architecture-folder-structure"><strong>layered architecture</strong></a> is like organizing a library by book size or color,
while <a href="vertical-slice-architecture-structuring-vertical-slices"><strong>vertical slices</strong></a> are like organizing by subject.
When you want to learn about history, you don't want to hunt through the entire library, you want all the history books in one place.</p>
<div className="centered">
  ![](/blogs/mnw_159/vertical_slice_architecture.png)
</div>
<p>Let's look at a practical example.</p>
<h2>The Traditional Approach vs. Vertical Slices</h2>
<p>Let's look at our data export example.
Here's how a typical .NET project would structure this feature:</p>
<p><strong>Traditional Layered Structure:</strong></p>
<pre><code>📁 Controllers/
└── UsersController.cs (export endpoint)
📁 Services/
├── IDataExportService.cs
├── DataExportService.cs
├── ICloudStorageService.cs
├── CloudStorageService.cs
├── IEmailService.cs
└── EmailService.cs
📁 Models/
├── ExportDataRequest.cs
└── ExportDataResponse.cs
📁 Repositories/
├── IUserRepository.cs
└── UserRepository.cs
</code></pre>
<p>Now here's the same functionality organized as vertical slices:</p>
<p><strong>Vertical Slice Structure:</strong></p>
<pre><code>📁 Features/
└──📁 Users/
   └──📁 ExportData/
      ├── ExportUserData.cs
      └── ExportUserDataEndpoint.cs
      📁 Create/
      └── CreateUser.cs
      📁 GetById/
      └── GetUserById.cs
</code></pre>
<p>The <code>ExportData</code> folder <strong>contains everything related</strong> to exporting user data: the request, response, business logic, and API endpoint.</p>
<p>Notice I'm still injecting <code>ICloudStorageClient</code> and <code>IEmailSender</code> rather than putting that logic directly in the handler.
These are genuine <strong>cross-cutting concerns</strong> that <strong>multiple features will use</strong>.
The key is distinguishing between 'shared because it should be' vs 'shared because this pattern told me to'.</p>
<h2>Show Me the Code</h2>
<p>I organize by domain first (<code>Users</code>), then by feature (<code>ExportData</code>).
Some teams prefer <code>Features/ExportUserData</code> directly, but I find the domain grouping helps when you have many features.
Related features stay visually grouped.</p>
<p>Here's what our data export <strong>feature slice</strong> looks like using a request, handler, and minimal APIs:</p>
<p><strong>Features/Users/ExportData/ExportUserData.cs</strong></p>
<pre><code class="language-csharp">public static class ExportUserData
{
    public record Request(Guid UserId) : IRequest&lt;Response&gt;;

    public record Response(string DownloadUrl, DateTime ExpiresAt);

    public class Handler(
        AppDbContext dbContext,
        ICloudStorageClient storageClient,
        IEmailSender emailSender)
        : IRequestHandler&lt;Request, Response&gt;
    {
        public async Task&lt;Response&gt; Handle(Request request, CancellationToken ct = default)
        {
            // Get user data
            var user = await dbContext.Users
                .Include(u =&gt; u.Orders)
                .Include(u =&gt; u.Preferences)
                .FirstOrDefaultAsync(u =&gt; u.Id == request.UserId, ct);

            if (user == null)
            {
                throw new NotFoundException($&quot;User {request.UserId} not found&quot;);
            }

            // Generate export data
            var exportData = new
            {
                user.Email,
                user.Name,
                user.CreatedAt,
                Orders = user.Orders.Select(o =&gt; new { o.Id, o.Total, o.Date }),
                Preferences = user.Preferences
            };

            // Upload to cloud storage
            var fileName = $&quot;user-data-{user.Id}-{DateTime.UtcNow:yyyyMMdd}.json&quot;;
            var expiresAtUtc = DateTime.UtcNow.AddDays(7);

            var downloadUrl = await storageClient.UploadAsJsonAsync(
                fileName,
                exportData,
                expiresAtUtc,
                ct);

            // Send email notification
            await emailSender.SendDataExportEmailAsync(user.Email, downloadUrl, ct);

            return new Response(downloadUrl, expiresAtUtc);
        }
    }

    // Simple validation using FluentValidation
    public sealed class Validator : AbstractValidator&lt;Request&gt;
    {
        public Validator()
        {
            RuleFor(r =&gt; r.UserId).NotEmpty();
        }
    }
}
</code></pre>
<p>Everything related to exporting user data is in one place: the database query, validation, business logic, cloud storage integration, and email notification.</p>
<p>The minimal API endpoint is straightforward:</p>
<pre><code class="language-csharp">public static class ExportUserDataEndpoint
{
    public static void Map(IEndpointRouteBuilder app)
    {
        app.MapPost(&quot;/users/{userId}/export&quot;, async (
            Guid userId,
            IRequestHandler&lt;ExportUserData.Request, ExportUserData.Response&gt; handler) =&gt;
        {
            var response = await handler.Handle(new ExportUserData.Request(userId));
            return Results.Ok(response);
        });
    }
}
</code></pre>
<p>We could even define the endpoint inside the <code>ExportUserData.cs</code> file if we wanted to keep everything together.
This is more a matter of preference and team conventions.
Either approach works well, from my experience.</p>
<h2>One File vs. Multiple Files: Your Choice</h2>
<p>You might have noticed something: I put everything in a single file.
This is a design choice with trade-offs.</p>
<p><strong>Single File Approach (ExportUserData.cs):</strong></p>
<pre><code class="language-csharp">public static class ExportUserData
{
    public record Request(Guid UserId) : IRequest&lt;Response&gt;;
    public record Response(string DownloadUrl, DateTime ExpiresAt);
    public class Handler : IRequestHandler&lt;Request, Response&gt; { /* ... */ }
    public class Validator : AbstractValidator&lt;Request&gt; { /* ... */ }
}
</code></pre>
<p><strong>Multiple Files Approach:</strong></p>
<pre><code>📁 ExportData/
├── ExportUserDataCommand.cs
├── ExportUserDataResponse.cs
├── ExportUserDataHandler.cs
├── ExportUserDataValidator.cs
└── ExportUserDataEndpoint.cs
</code></pre>
<p><strong>Single file is great when</strong>: the feature is straightforward, you want <strong>maximum locality</strong>, and the file doesn't exceed a few hundred lines of code.</p>
<p>Lines of code isn't a strict rule, but if a file grows beyond 300-400 lines, consider splitting it up for readability.
Again, this is a matter of team preference and not a hard rule I go by.
It's important to trust your instincts and what feels right for your team.</p>
<p><strong>Multiple files work better when</strong>: you have complex validation logic, multiple response types,
or when the handler grows large enough that you want to focus on one concern at a time.</p>
<p>You can even mix both approaches within the same project.</p>
<p>Both approaches keep related code together.
And this is what matters most in Vertical Slice Architecture.</p>
<h2>Why This Actually Works (And How to Start)</h2>
<p>The benefits of vertical slices become obvious once you try it.
Your brain doesn't have to remember which files are related to which features.
Everything lives together.</p>
<p>Need to modify the data export feature?
Everything's in the <code>ExportData</code> folder.
No hunting across <code>Controllers</code>, <code>Services</code>, and <code>Repositories</code> layers.
Each slice can evolve independently, so simple CRUD operations stay simple while complex features like data export can use sophisticated approaches.</p>
<p>You don't need to rewrite your entire application overnight.
Start with new features using vertical slices.
As you touch existing code, gradually move related pieces into feature folders.</p>
<p>Good architecture is about making your codebase easier to understand and modify.
When all the code for a feature lives together, you spend less mental energy navigating your solution and more time solving actual problems.</p>
<p>Here are a few resources if you want to learn more:</p>
<ul>
<li><a href="clean-architecture-the-missing-chapter"><strong>Clean Architecture: The Missing Chapter</strong></a></li>
<li><a href="what-is-a-modular-monolith"><strong>Modular Monolith Architecture</strong></a></li>
<li><a href="screaming-architecture"><strong>Screaming Architecture</strong></a></li>
</ul>
<p>All of these concepts tie together to help you build maintainable, scalable .NET applications.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_159.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Building Semantic Search with Amazon S3 Vectors and Semantic Kernel]]></title>
            <link>https://milanjovanovic.tech/blog/building-semantic-search-with-amazon-s3-vectors-and-semantic-kernel</link>
            <guid isPermaLink="false">building-semantic-search-with-amazon-s3-vectors-and-semantic-kernel</guid>
            <pubDate>Sat, 06 Sep 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to add semantic search to your website using Amazon S3 Vectors - a new vector database service that's 90% cheaper than alternatives. Complete implementation guide using Semantic Kernel and .NET, from generating embeddings to querying vector indexes.]]></description>
            <content:encoded><![CDATA[<p>I implemented <a href="how-i-implemented-full-text-search-on-my-website"><strong>full-text search</strong></a> on my static website using Lunr.js.
It works great for exact matches or phrases, but it doesn't understand meaning.
Someone searches for <a href="search?q=modular%20monolith">&quot;modular monolith&quot;</a> and finds posts that contain these phrases.
But when they search for &quot;database performance issues,&quot; they might miss my articles about query optimization and index tuning, even though that's exactly what they need.</p>
<p><a href="https://aws.amazon.com/blogs/aws/introducing-amazon-s3-vectors-first-cloud-storage-with-native-vector-support-at-scale/"><strong>Amazon S3 is now also a vector database</strong></a>,
and it's 90% cheaper (according to the announcement) than the alternatives.</p>
<p>For those of us running static sites or simple web apps, it means we can finally add semantic search without the operational overhead of
running a vector database like <a href="https://www.pinecone.io/">Pinecone</a>, <a href="https://weaviate.io/">Weaviate</a>, or <a href="https://qdrant.tech/">Qdrant</a>.</p>
<p>Instead, we can just use S3 buckets that understand vectors.</p>
<p>I'm adding it alongside my existing full-text search implementation, and the whole thing took an afternoon to build.</p>
<h2>How Semantic Search Works</h2>
<p>The entire <a href="https://cloud.google.com/discover/what-is-semantic-search">semantic search</a> flow is straightforward:</p>
<p><img src="/blogs/mnw_158/semantic_search_flow.png" alt="Semantic search flow"></p>
<p>You have to start with some data.
In my case, this data comes from the articles I've written over the years.</p>
<p>Then we need an embedding model.
Amazon Bedrock offers many to choose from.
You use an embedding model to convert text into vectors.
Vectors are numerical representations of your data that capture semantic meaning.</p>
<p>Finally, we store these vectors in an S3 Vector Bucket, a new type of S3 bucket designed specifically for vector storage and search.
When someone searches, we convert their query into a vector and find the closest matches in the bucket.</p>
<p>If you want to understand the fundamentals, I wrote an article explaining what <a href="what-is-vector-search-a-concise-guide"><strong>vector search</strong></a> is.</p>
<h2>Generating Embeddings with Semantic Kernel</h2>
<p>Microsoft's <a href="https://learn.microsoft.com/en-us/semantic-kernel/overview/">Semantic Kernel</a> makes working with Bedrock surprisingly clean.</p>
<p>We'll need to install a few NuGet packages:</p>
<pre><code class="language-powershell"># Semantic kernel packages
Install-Package Microsoft.SemanticKernel
Install-Package Microsoft.SemanticKernel.Connectors.Amazon

# AWS SDK for Bedrock
Install-Package AWSSDK.BedrockRuntime
</code></pre>
<p>We have to configure the embedding generator in our application.
Here we can specify which model to use for generating embeddings.
I'll use the Amazon Titan embedding model (<code>amazon.titan-embed-text-v2:0</code>) which produces 1024-dimensional vectors.</p>
<pre><code class="language-csharp">builder.Services.AddBedrockEmbeddingGenerator(&quot;amazon.titan-embed-text-v2:0&quot;);

builder.Services.AddTransient(sp =&gt;
{
    return new Kernel(sp);
});
</code></pre>
<p>And then we can use the <code>IEmbeddingGenerator</code> abstraction to generate embeddings:</p>
<pre><code class="language-csharp">var kernel = app.Services.GetRequiredService&lt;Kernel&gt;();

var embeddingGenerator = kernel.Services
    .GetRequiredService&lt;IEmbeddingGenerator&lt;string, Embedding&lt;float&gt;&gt;&gt;();

var articleContent = await blogService.GetBlogContentAsync(articleUrl);

Embedding&lt;float&gt; embedding = await embeddingGenerator.GenerateAsync(articleContent);

embeddings.Add((articleUrl, embedding.Vector.ToArray()));
</code></pre>
<p>The number of dimensions varies depending on the embedding model you choose.
You can find more information about that in the documentation for your specific model.</p>
<h2>Creating Your S3 Vector Bucket</h2>
<p>S3 Vectors uses an entirely new bucket type, not a feature you enable on existing buckets.
They're a fundamentally different storage system optimized for vector operations.
Creating one feels familiar if you've used S3 before:</p>
<div className="bordered">
  ![Creating S3 Vectors bucket](/blogs/mnw_158/s3_vectors_bucket.png)
</div>
<p>Pick a unique name for your vector bucket:</p>
<div className="bordered">
  ![Creating S3 Vectors bucket](/blogs/mnw_158/s3_vectors_bucket_create.png)
</div>
<p>And finally, you can create a <strong>vector index</strong> for your bucket.
You get to choose the number of dimensions in each vector.
This is dictated by the embedding model you use.
All vectors within a vector index should use the same embedding model.
Otherwise, you won't get the correct results when searching.
You also have to choose the <a href="https://en.wikipedia.org/wiki/Metric_space">distance metric</a> (e.g., cosine similarity, Euclidean distance) for your vector index.
I went with <a href="https://en.wikipedia.org/wiki/Cosine_similarity">cosine similarity</a>, which is a common choice for text embeddings.
The additional settings let you configure non-filterable metadata.
By default, all metadata is filterable.</p>
<div className="bordered">
  ![Creating S3 Vectors bucket](/blogs/mnw_158/s3_vectors_index_create.png)
</div>
<p>The UI is not polished at all, and you can't do much else.
I expect this will improve over time, but for now, everything else is done via the SDK or CLI.
There are certain limitations to be aware of, you can check out the docs <a href="https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-vectors-limitations.html">here</a>.</p>
<h2>Storing Vectors with Metadata</h2>
<p>Now that we have our vector index, we can start storing vector embeddings with metadata.
Metadata is interesting because it enables filtered searches.
Want to search only recent posts? Posts in a specific category?
The metadata makes it possible.</p>
<p>Here's how you can store vectors:</p>
<pre><code class="language-csharp">public async Task IndexBlogPost(BlogPost post)
{
    List&lt;float&gt; embedding = await GenerateEmbedding(post.Content);

    await s3VectorsClient.PutVectorsAsync(new PutVectorsRequest
    {
        VectorBucketName = &quot;mjtech-articles-semantic-search&quot;,
        IndexName = &quot;mjtech-article-content&quot;,
        Vectors = new List&lt;Vector&gt;
        {
            new PutInputVector
            {
                Key = post.Slug,
                Data = new VectorData
                {
                    Float32 = embedding
                },
                Metadata = new Document(new Dictionary&lt;string, Document&gt;
                {
                    [&quot;title&quot;] = post.Title,
                    [&quot;date&quot;] = post.PublishedDate.ToString(&quot;yyyy-MM-dd&quot;),
                    [&quot;category&quot;] = post.Category,
                    [&quot;url&quot;] = $&quot;/posts/{post.Slug}&quot;
                })
            }
        }
    });
}
</code></pre>
<p>But if you're indexing your entire blog archive, doing it one post at a time is costly.
You can batch together multiple posts and index them in a single API call.
The SDK makes it easy to do this by accepting a list of vectors.</p>
<h2>Querying Vector Indexes</h2>
<p>When someone searches your site, you convert their query into a vector and find the closest matches.</p>
<p>Here's how you can implement semantic search:</p>
<pre><code class="language-csharp">public async Task&lt;List&lt;SearchResult&gt;&gt; SemanticSearch(
    string query,
    int topK = 10)
{
    // Convert search query to vector (use same model as vectors!)
    List&lt;float&gt; queryEmbedding = await GenerateEmbedding(query);

    var request = new QueryVectorsRequest
    {
        VectorBucketName = &quot;mjtech-articles-semantic-search&quot;,
        IndexName = &quot;mjtech-article-content&quot;,
        QueryVector = new VectorData
        {
            Float32 = queryEmbedding
        },
        TopK = topK,
        ReturnMetadata = true,
        ReturnDistance = true
    };

    QueryVectorsResponse response = await s3VectorsClient.QueryVectorsAsync(request);

    return response.Vectors.Select(v =&gt; new SearchResult
    {
        Distance = v.Distance,
        Title = v.Metadata.AsDictionary()[&quot;title&quot;].ToString(),
        Url = v.Metadata.AsDictionary()[&quot;url&quot;].ToString(),
        Category = v.Metadata.AsDictionary()[&quot;category&quot;].ToString()
    }).ToList();
}
</code></pre>
<p>I omitted metadata filtering here, but you can explore the <a href="https://docs.aws.amazon.com/AmazonS3/latest/userguide/s3-vectors-metadata-filtering.html">documentation</a>
for more details.</p>
<h2>Next Steps</h2>
<p>Would I migrate from an existing vector database?
Probably not if everything's working.
The operational overhead would have to justify the switch.</p>
<p>But if you're adding semantic search for the first time, or your vector database bills are getting uncomfortable, S3 Vectors is a viable choice.
The setup takes an afternoon, the ongoing maintenance is zero, and your users get search that actually understands what they're looking for.</p>
<p>Don't forget that S3 Vectors is still in preview, so we can expect some changes before general availability.</p>
<p>Here's what I'm planning to do next:</p>
<ol>
<li>Automatically update the vector index when I publish a new post. I can do this in my CI/CD pipeline, where I can detect new posts and trigger a re-indexing.</li>
<li>Expose a search endpoint that uses the new semantic search capabilities. Combine the results with the full-text search results.</li>
<li>Make sure everything is performant and cost-effective. Acceptable latency is under 500ms.</li>
<li>Share details about the implementation and any challenges faced. Especially around the costs of using an embedding model and vector storage.</li>
</ol>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_158.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Standalone Aspire Dashboard Setup for Distributed .NET Applications]]></title>
            <link>https://milanjovanovic.tech/blog/standalone-aspire-dashboard-setup-for-distributed-dotnet-applications</link>
            <guid isPermaLink="false">standalone-aspire-dashboard-setup-for-distributed-dotnet-applications</guid>
            <pubDate>Sat, 30 Aug 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to run the Aspire Dashboard as a standalone container for instant traces, logs, and metrics in your .NET applications.]]></description>
            <content:encoded><![CDATA[<p>You've built a distributed .NET application.
Multiple services, databases, message queues.
Now something's slow, and you need to figure out why.</p>
<p><strong>The Aspire Dashboard runs perfectly as a standalone container</strong>,
giving you <a href="introduction-to-distributed-tracing-with-opentelemetry-in-dotnet"><strong>distributed tracing</strong></a>,
<a href="5-serilog-best-practices-for-better-structured-logging"><strong>structured logs</strong></a>,
and real-time metrics without the full orchestration framework.</p>
<p>While <a href="dotnet-aspire-a-game-changer-for-cloud-native-development"><strong>Aspire's orchestration</strong></a> is incredibly powerful for managing distributed applications,
sometimes you just need the observability piece.
Maybe you're already using <a href="using-dotnet-aspire-with-the-docker-publisher"><strong>Docker Compose</strong></a> or <a href="https://doineedkubernetes.com/">Kubernetes</a>.
Maybe you're debugging an existing system.
The standalone dashboard gives you valuable telemetry visualization with minimal setup.</p>
<p>Let's get it running in under 5 minutes.</p>
<h2>Why Run the Aspire Dashboard Standalone?</h2>
<p>Most teams already have their deployment story figured out.
Docker Compose, Kubernetes, or some platform-specific orchestration.
You don't want to rewrite everything just to get observability.</p>
<p>The standalone <strong>Aspire Dashboard</strong> hits a sweet spot <strong>for development</strong>:</p>
<ul>
<li><strong>Drop-in observability</strong> - Just add a container to your existing setup</li>
<li><strong>Full OpenTelemetry support</strong> - Works with any OTLP-compatible application</li>
<li><strong>Developer-friendly</strong> - Designed for local development and debugging</li>
<li><strong>Immediate value</strong> - See traces, logs, and metrics within minutes</li>
</ul>
<p>One caveat: it's <strong>in-memory only</strong>.
Perfect for development and debugging, not for production.
For production, you'll want something like <a href="introduction-to-distributed-tracing-with-opentelemetry-in-dotnet"><strong>Jaeger</strong></a>,
<a href="https://prometheus.io/">Prometheus</a>, or a commercial APM solution.</p>
<p>But for understanding what your code is doing right now?
It's exactly what you need.</p>
<h2>Step 1: Add the Dashboard Container</h2>
<p>Drop this into your <code>docker-compose.yml</code>:</p>
<pre><code class="language-yaml">aspire-dashboard:
  container_name: aspire-dashboard
  image: mcr.microsoft.com/dotnet/aspire-dashboard:13.0
  ports:
    - 18888:18888
</code></pre>
<p>That's it. The dashboard is running.
Navigate to <code>http://localhost:18888</code> and... you'll need a token.</p>
<p><img src="/blogs/mnw_157/aspire_dashboard_login.png" alt="Aspire Dashboard login screen"></p>
<p><strong>Check the container logs</strong> for the login link.
The dashboard generates a unique authentication token on startup:</p>
<p><img src="/blogs/mnw_157/aspire_dashboard_login_link.png" alt="Aspire Dashboard login link"></p>
<p>Click that link, and you're in.
Empty for now, but not for long.</p>
<h2>Step 2: Wire Up Your .NET Services</h2>
<p>Your services need to know where to send their telemetry.
Add these environment variables to your API containers:</p>
<pre><code class="language-yaml">users.api:
  image: ${DOCKER_REGISTRY-}usersapi
  build:
    context: .
    dockerfile: Users.Api/Dockerfile
  ports:
    - 5100:5100
    - 5101:5101
  environment:
    - OTEL_EXPORTER_OTLP_ENDPOINT=http://aspire-dashboard:18889
    - OTEL_EXPORTER_OTLP_PROTOCOL=grpc
  depends_on:
    - users.database
</code></pre>
<p>Notice port <code>18889</code>?
That's the OTLP ingestion endpoint.
The dashboard listens on <code>18888</code> for the UI, <code>18889</code> for telemetry data.</p>
<h2>Step 3: Configure OpenTelemetry in Your Code</h2>
<p>Install the necessary <a href="https://www.nuget.org/packages?q=OpenTelemetry">OpenTelemetry packages</a>:</p>
<pre><code class="language-xml">&lt;PackageReference Include=&quot;Npgsql.OpenTelemetry&quot; Version=&quot;9.0.3&quot; /&gt;
&lt;PackageReference Include=&quot;OpenTelemetry.Exporter.OpenTelemetryProtocol&quot; Version=&quot;1.12.0&quot; /&gt;
&lt;PackageReference Include=&quot;OpenTelemetry.Extensions.Hosting&quot; Version=&quot;1.12.0&quot; /&gt;
&lt;PackageReference Include=&quot;OpenTelemetry.Instrumentation.AspNetCore&quot; Version=&quot;1.12.0&quot; /&gt;
&lt;PackageReference Include=&quot;OpenTelemetry.Instrumentation.Http&quot; Version=&quot;1.12.0&quot; /&gt;
</code></pre>
<p>Then configure OpenTelemetry in your <code>Program.cs</code>:</p>
<pre><code class="language-csharp">builder.Services.AddOpenTelemetry()
    .ConfigureResource(resource =&gt; resource.AddService(builder.Environment.ApplicationName))
    .WithTracing(tracing =&gt; tracing
        .AddHttpClientInstrumentation()
        .AddAspNetCoreInstrumentation()
        .AddNpgsql())
    .WithMetrics(metrics =&gt; metrics
        .AddHttpClientInstrumentation()
        .AddAspNetCoreInstrumentation());

builder.Logging.AddOpenTelemetry(options =&gt;
{
    options.IncludeScopes = true;
    options.IncludeFormattedMessage = true;
});

builder.Services.AddOpenTelemetry().UseOtlpExporter();
</code></pre>
<p>This configuration:</p>
<ul>
<li><strong>Traces</strong> HTTP calls, <a href="http://ASP.NET">ASP.NET</a> Core requests, and database queries</li>
<li><strong>Collects metrics</strong> on request duration, response codes, and throughput</li>
<li><strong>Structured logging</strong> with full context and formatted messages</li>
<li><strong>Exports everything</strong> to the Aspire Dashboard via OTLP</li>
</ul>
<p>The <code>UseOtlpExporter()</code> method automatically picks up the <code>OTEL_EXPORTER_OTLP_ENDPOINT</code> environment variable you configured earlier.</p>
<h2>What You Get</h2>
<p>Start your application and make a few requests.
The dashboard immediately lights up with data.</p>
<h3>Structured Logs</h3>
<p>Every log entry includes full context: trace IDs, request paths, user identities.
Click any log to see the complete structured data.</p>
<p><img src="/blogs/mnw_157/structured_logs.png" alt="Aspire Dashboard structured logs"></p>
<h3>Distributed Traces</h3>
<p>See the complete request flow across all your services.
Which database query is slow? Which HTTP call is failing?
The trace view shows you exactly where time is spent.</p>
<p><img src="/blogs/mnw_157/distributed_traces.png" alt="Aspire Dashboard distributed traces"></p>
<p>You can click into a trace to see the individual spans and any metadata associated with them.</p>
<p><img src="/blogs/mnw_157/distributed_trace_details.png" alt="Aspire Dashboard distributed trace details"></p>
<h3>Real-Time Metrics</h3>
<p>Response times, error rates, throughput, all updating live.
Perfect for load testing or understanding traffic patterns.</p>
<p><img src="/blogs/mnw_157/metrics.png" alt="Aspire Dashboard metrics"></p>
<h2>Summary</h2>
<p>The standalone <strong>Aspire Dashboard</strong> is perfect for local development and debugging.
Spin up your stack, make requests, and instantly see what's happening across all your services.
Find bottlenecks in the trace view, correlate logs with requests, watch metrics update in real-time.</p>
<p>Remember: this is for development only since data is in-memory and disappears on restart.
That last part might be fixed soon, according to the <a href="https://youtu.be/zvBu0OOCVos"><strong>Aspire roadmap</strong></a>.
For production, you'll want proper solutions like Jaeger for tracing, Prometheus for metrics, or a commercial APM like Application Insights.</p>
<p>But for that immediate &quot;what is my code actually doing?&quot; question during development?
You've got professional observability in under 5 minutes.</p>
<p>Just add the container, configure OpenTelemetry, and start debugging like a pro.</p>
<p>That's all for today.</p>
<p>See you next Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_157.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[The Real Cost of Abstractions in .NET]]></title>
            <link>https://milanjovanovic.tech/blog/the-real-cost-of-abstractions-in-dotnet</link>
            <guid isPermaLink="false">the-real-cost-of-abstractions-in-dotnet</guid>
            <pubDate>Sat, 23 Aug 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Not all abstractions are created equal. Some isolate real volatility and protect your system from change. Others quietly pile up complexity and hide performance problems. Learn when to embrace abstractions versus when they become technical debt that slows down your team.]]></description>
            <content:encoded><![CDATA[<p>As developers, we love abstractions.
Repositories, services, mappers, wrappers.
They make our code look &quot;clean,&quot; they promise testability, and they give us the sense that we're building something flexible.</p>
<p><strong>Every abstraction is a loan.
You pay interest the moment you write it.</strong></p>
<p>Some abstractions earn their keep by isolating real volatility and protecting your system from change.
Others quietly pile up complexity, slow down onboarding, and hide performance problems behind layers of indirection.</p>
<p>Let's explore when abstractions pay dividends and when they become technical debt.</p>
<h2>When Abstractions Pay Off</h2>
<p>The best abstractions <strong>isolate true volatility</strong>, the parts of your system that you genuinely expect to change.</p>
<h3>Example: Payment Processing</h3>
<p>Your core business logic shouldn't depend directly on Stripe's SDK.
If you ever switch to Adyen or Braintree, you don't want that decision rippling through every corner of your codebase.
Here, an abstraction makes perfect sense:</p>
<pre><code class="language-csharp">public interface IPaymentProcessor
{
    Task ProcessAsync(Order order, CancellationToken ct);
}

public class StripePaymentProcessor : IPaymentProcessor
{
    public async Task ProcessAsync(Order order, CancellationToken ct)
    {
        // Stripe-specific implementation
        // Handle webhooks, error codes, etc.
    }
}

public class AdyenPaymentProcessor : IPaymentProcessor
{
    public async Task ProcessAsync(Order order, CancellationToken ct)
    {
        // Adyen-specific implementation
        // Different API, same business outcome
    }
}
</code></pre>
<p>Now your business logic can stay focused on the domain:</p>
<pre><code class="language-csharp">public class CheckoutService(IPaymentProcessor processor)
{
    public Task CheckoutAsync(Order order, CancellationToken cancellationToken) =&gt;
        processor.ProcessAsync(order, cancellationToken);
}
</code></pre>
<p>This abstraction isolates a genuinely unstable dependency (the payment provider) while keeping checkout logic independent.
When Stripe changes their API or you switch providers, only one class needs to change.</p>
<p><strong>That's a good abstraction</strong>.
It buys you optionality where you actually need it.</p>
<h2>When Abstractions Become Technical Debt</h2>
<p>Problems arise when we abstract things that aren't actually volatile.
We end up wrapping stable libraries or creating layers that don't add real value.
The &quot;clean&quot; layer you added today becomes tomorrow's maintenance burden.</p>
<h3>The Repository That Lost Its Way</h3>
<p>Most teams start with something reasonable:</p>
<pre><code class="language-csharp">public interface IUserRepository
{
    Task&lt;IEnumerable&lt;User&gt;&gt; GetAllAsync();
}
</code></pre>
<p>But as requirements evolve, so does the interface:</p>
<pre><code class="language-csharp">public interface IUserRepository
{
    Task&lt;IEnumerable&lt;User&gt;&gt; GetAllAsync();
    Task&lt;User?&gt; GetByEmailAsync(string email);
    Task&lt;IEnumerable&lt;User&gt;&gt; GetActiveUsersAsync();
    Task&lt;IEnumerable&lt;User&gt;&gt; GetUsersByRoleAsync(string role);
    Task&lt;IEnumerable&lt;User&gt;&gt; SearchAsync(string keyword, int page, int pageSize);
    Task&lt;IEnumerable&lt;User&gt;&gt; GetUsersWithRecentActivityAsync(DateTime since);
    // ...and it keeps growing
}
</code></pre>
<p>Suddenly, the repository is leaking <strong>query logic into its interface</strong>.
Every new way of fetching users means another method, and your &quot;abstraction&quot; becomes a grab bag of every possible query.</p>
<p>Meanwhile, Entity Framework already gives you all of this through LINQ: strongly typed queries that map directly to SQL.
Instead of leveraging that power, you've introduced an indirection layer that hides query performance characteristics and often performs worse.
The repository pattern made sense when ORMs were immature.
Today, it's often just ceremony.</p>
<p>I've been guilty of this myself.
But part of maturing as a developer is recognizing when patterns become anti-patterns.
Repositories make sense when they encapsulate complex query logic or provide a unified API over multiple data sources.
But you should strive to keep them focused on domain logic.
As soon as they explode into a myriad of methods for every possible query, it's a sign that the abstraction has failed.</p>
<h2>Service Wrappers: The Good and The Ugly</h2>
<p>Not all service wrappers are problematic. Context matters.</p>
<p><strong>✅ Good Example: External API Integration</strong></p>
<p>When integrating with external APIs, a wrapper provides genuine value by centralizing concerns:</p>
<pre><code class="language-csharp">public interface IGitHubClient
{
    Task&lt;UserDto?&gt; GetUserAsync(string username);
    Task&lt;IReadOnlyList&lt;RepoDto&gt;&gt; GetRepositoriesAsync(string username);
}

public class GitHubClient(HttpClient httpClient) : IGitHubClient
{
    public Task&lt;UserDto?&gt; GetUserAsync(string username) =&gt;
        httpClient.GetFromJsonAsync&lt;UserDto&gt;($&quot;/users/{username}&quot;);

    public Task&lt;IReadOnlyList&lt;RepoDto&gt;&gt; GetRepositoriesAsync(string username) =&gt;
        httpClient.GetFromJsonAsync&lt;IReadOnlyList&lt;RepoDto&gt;&gt;($&quot;/users/{username}/repos&quot;);
}
</code></pre>
<p>This wrapper isolates GitHub's API details.
When authentication changes or endpoints evolve, you update one place.
Your business logic never needs to know about HTTP headers, base URLs, or JSON serialization.</p>
<p><strong>❌ Bad Example: Pass-Through Services</strong></p>
<p>The trouble starts when we wrap our own stable services without adding business value:</p>
<pre><code class="language-csharp">public class UserService(IUserRepository userRepository)
{
    // Just forwarding calls with no added value
    public Task&lt;User?&gt; GetByIdAsync(Guid id) =&gt; userRepository.GetByIdAsync(id);
    public Task&lt;IEnumerable&lt;User&gt;&gt; GetAllAsync() =&gt; userRepository.GetAllAsync();
    public Task SaveAsync(User user) =&gt; userRepository.SaveAsync(user);
}
</code></pre>
<p>This <code>UserService</code> is pure indirection.
All it does is forward calls to the <code>IUserRepository</code>.
It doesn't enforce business rules, add validation, implement caching, or provide any real functionality.
It's a layer that exists only because &quot;services are good architecture.&quot;</p>
<p>As these anemic wrappers multiply, your codebase becomes a maze.
Developers waste time navigating layers instead of focusing on where business logic actually lives.</p>
<h2>Making Better Decisions</h2>
<p>Here's how to think about when abstractions are worth the investment:</p>
<h3>Abstract Policies, Not Mechanics</h3>
<ul>
<li><strong>Policies</strong> are decisions that might change: which payment provider to use, how to handle caching, retry strategies for external calls</li>
<li><strong>Mechanics</strong> are stable implementation details: EF Core's LINQ syntax, <code>HttpClient</code> configuration, JSON serialization</li>
</ul>
<p>Abstract policies because they give you flexibility.
Don't abstract mechanics, they're already stable APIs that rarely change in breaking ways.</p>
<h3>Wait for the Second Implementation</h3>
<p>If you only have one implementation, resist the interface urge.
A single implementation doesn't justify abstraction, it's premature generalization that adds complexity without benefit.</p>
<p>Consider this evolution:</p>
<pre><code class="language-csharp">// Step 1: Start concrete
public class EmailNotifier
{
    public async Task SendAsync(string to, string subject, string body)
    {
        // SMTP implementation
    }
}

// Step 2: Need SMS? Now abstract
public interface INotifier
{
    Task SendAsync(string to, string subject, string body);
}

public class EmailNotifier : INotifier { /* ... */ }
public class SmsNotifier : INotifier { /* ... */ }
</code></pre>
<p>The abstraction emerges naturally when you actually need it.
The interface reveals itself through real requirements, not imaginary ones.</p>
<h3>Keep Implementations Inside, Abstractions at Boundaries</h3>
<p>Inside your application, prefer <strong>concrete types</strong>.
Use Entity Framework directly, configure <code>HttpClient</code> as typed clients, work with domain entities.
Only introduce abstractions where your system meets the outside world: external APIs, third-party SDKs, infrastructure services.</p>
<p>That's where change is most likely, and where abstractions earn their keep.</p>
<h2>Refactoring Out Bad Abstractions</h2>
<p>Regularly review your abstractions with this question: If I removed this abstraction, would the code become simpler or more complex?</p>
<p>If removing an interface or service layer would make the code clearer and more direct,
that abstraction is probably costing more than it's worth.
Don't be afraid to delete unnecessary layers.
Simpler code is often better code.</p>
<p>When you identify problematic abstractions, here's how to safely remove them:</p>
<ol>
<li><strong>Identify the real consumers</strong>. Who actually needs the abstraction?</li>
<li><strong>Inline the interface</strong>. Replace abstract calls with concrete implementations.</li>
<li><strong>Delete the wrapper</strong>. Remove the unnecessary indirection.</li>
<li><strong>Simplify the calling code</strong>. Take advantage of the concrete API's features.</li>
</ol>
<p>For example, replacing a repository with direct EF Core usage:</p>
<pre><code class="language-csharp">// Before: Hidden behind repository
var users = await _userRepo.GetActiveUsersWithRecentOrders();

// After: Direct, optimized query
var users = await _context.Users
    .Where(u =&gt; u.IsActive)
    .Where(u =&gt; u.Orders.Any(o =&gt; o.CreatedAt &gt; DateTime.Now.AddDays(-30)))
    .Include(u =&gt; u.Orders.Take(5))
    .ToListAsync();
</code></pre>
<p>The concrete version is more explicit about what data it fetches and how, making performance characteristics visible and optimization possible.
If you need the same query in multiple places, you could move it into an extension method to make it shareable.</p>
<h2>The Bottom Line</h2>
<p>Abstractions are powerful tools for managing complexity and change, but they're not free.
Each one adds indirection, cognitive overhead, and maintenance burden.</p>
<p>The <a href="/pragmatic-clean-architecture"><strong>cleanest architecture</strong></a> isn't the one with the most layers.
It's the one where each layer has a clear, justified purpose.</p>
<p>Before adding your next abstraction, ask yourself:</p>
<ul>
<li>Am I abstracting a policy or just a mechanic?</li>
<li>Do I have two implementations, or am I speculating about future needs?</li>
<li>Will this make my system more adaptable, or just harder to follow?</li>
<li>If I removed this layer, would the code become simpler?</li>
</ul>
<p>Remember: abstractions are loans that accrue interest over time.
Make sure you're borrowing for the right reasons, not just out of habit.</p>
<p>The goal is to use abstractions intentionally, where they solve real problems and protect against genuine volatility.
Build abstractions that earn their keep.
Delete the ones that don't.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_156.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Building Generative AI Applications With GitHub Models and .NET Aspire]]></title>
            <link>https://milanjovanovic.tech/blog/building-generative-ai-applications-with-github-models-and-dotnet-aspire</link>
            <guid isPermaLink="false">building-generative-ai-applications-with-github-models-and-dotnet-aspire</guid>
            <pubDate>Sat, 16 Aug 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Discover how to integrate AI into your .NET applications in under an hour using GitHub Models and .NET Aspire 9.4. I'll walk you through building a simple AI-powered blog analyzer that automatically categorizes your content.]]></description>
            <content:encoded><![CDATA[<p>I wanted to see what the simplest practical AI app I could build was, and this is what I came up with.</p>
<p>Every week, I publish blog posts covering different topics - architecture patterns, cloud services, programming techniques, business insights.
Sometimes I write about DevOps, other times about security.
After years of writing, I realized I had no systematic way to categorize my content.
Sure, I could manually tag each post, but where's the fun in that?</p>
<p>So I built a simple AI-powered blog analyzer.
It fetches any blog post, extracts the content, and uses AI to automatically categorize it.
The entire thing took less than an hour to build thanks to <strong>.NET Aspire 9.4</strong> and it's new <strong>GitHub Models</strong> integration.</p>
<p>What surprised me wasn't just how easy it was to build, but how the integration completely removes the typical AI service complexity.
No juggling API keys in configuration files, no manual HTTP client setup, no wrestling with different SDK patterns for different AI providers.
You declare an AI model in your AppHost just like you would a database, and Aspire handles the rest.</p>
<p>Here's what I learned building this simple app, and how you can use the same patterns to add AI to your applications.</p>
<h2>What are GitHub Models?</h2>
<p><a href="https://docs.github.com/en/github-models">GitHub Models</a> is a service that provides access to AI models from OpenAI, Microsoft, Meta, and others through a single API.
You get free tier access for prototyping (with rate limits), an interactive playground for testing prompts, and pay-per-use billing when you're ready for production.</p>
<p>The models range from cost-effective options like <strong>GPT-4o-mini</strong> to more advanced models.
Each model has different strengths - some excel at reasoning, others at code generation or creative writing.</p>
<p>When you combine GitHub Models with .NET Aspire's orchestration, you get:</p>
<ul>
<li>Automatic API key management via <a href="https://learn.microsoft.com/en-us/dotnet/aspire/fundamentals/external-parameters">external parameters</a></li>
<li><a href="how-dotnet-aspire-simplifies-service-discovery"><strong>Service discovery</strong></a> between your components</li>
<li>Built-in <a href="health-checks-in-asp-net-core"><strong>health checks</strong></a> and <a href="introduction-to-distributed-tracing-with-opentelemetry-in-dotnet"><strong>telemetry</strong></a></li>
<li>Consistent configuration patterns</li>
</ul>
<h2>Setting Up the Integration</h2>
<p>The GitHub Models integration splits into two parts: configuring models in your <code>AppHost</code> and consuming them in your services.</p>
<h3>AppHost Configuration</h3>
<p>In your AppHost project, you define AI models as resources alongside your other services:</p>
<pre><code class="language-csharp">var builder = DistributedApplication.CreateBuilder(args);

var blogService = builder.AddExternalService(&quot;dotnet-blog&quot;, &quot;https://www.milanjovanovic.tech/blog&quot;);
var aiModel = builder.AddGitHubModel(&quot;ai-model&quot;, &quot;openai/gpt-4o-mini&quot;);

builder.AddProject&lt;Projects.GitHub_Models_Demo&gt;(&quot;github-models-demo&quot;)
    .WithReference(blogService)
    .WithReference(aiModel);
</code></pre>
<p>Notice how the AI model sits alongside the external blog service.
Both are resources that your main application depends on.
Aspire handles the connection details - you just declare what you need.</p>
<p>Make sure to install the <a href="https://www.nuget.org/packages/Aspire.Hosting.GitHub.Models">Aspire.Hosting.GitHub.Models</a> NuGet package to enable this integration.</p>
<p>To call the GitHub Models inference API you need a personal access token with the <code>models:read</code> permission.
When you call <code>AddGitHubModel</code>, Aspire automatically creates a parameter named <code>{resourceName}-gh-apikey</code> (for example, <code>ai-model-gh-apikey</code>)</p>
<p>You can populate the parameter through user secrets for local development:</p>
<pre><code class="language-json">{
  &quot;Parameters&quot;: {
    &quot;ai-model-gh-apikey&quot;: &quot;github_pat_YOUR_PERSONAL_ACCESS_TOKEN&quot;
  }
}
</code></pre>
<p>If you don't provide this value from configuration, Aspire will prompt you to enter it when you run the application.
You'll have an option to store the access token securely in user secrets.</p>
<h3>Client Setup</h3>
<p>In your consuming service, add the Azure AI client (which works with GitHub Models):</p>
<pre><code class="language-csharp">builder
    .AddAzureChatCompletionsClient(&quot;ai-model&quot;)
    .AddChatClient();
</code></pre>
<p>That's it.
No manual HTTP client configuration, no hardcoded endpoints.
The <code>ai-model</code> name matches what you defined in the AppHost, and Aspire wires everything together.</p>
<p>You'll need to install the <a href="https://www.nuget.org/packages/Aspire.Azure.AI.Inference">Aspire.Azure.AI.Inference</a> NuGet package to enable this integration.
It also exposes an integration with <a href="working-with-llms-in-dotnet-using-microsoft-extensions-ai"><strong>MEAI</strong></a> using the <code>AddChatClient</code> method.
This simplifies the process of interacting with LLMs in your applications.</p>
<h2>Building the Blog Analyzer</h2>
<p>Now let's build something useful.
We'll create a service that fetches blog posts and uses AI to categorize them.</p>
<h3>Fetching Blog Content</h3>
<p>First, we need to extract readable content from blog posts:</p>
<pre><code class="language-csharp">public async Task&lt;string&gt; GetBlogContentAsync(string slug)
{
    var response = await httpClient.GetAsync(slug);
    response.EnsureSuccessStatusCode();
    var htmlContent = await response.Content.ReadAsStringAsync();

    return ExtractArticleContent(htmlContent);
}
</code></pre>
<p>The <code>ExtractArticleContent</code> method (not shown) uses <a href="https://www.nuget.org/packages/htmlagilitypack/">HtmlAgilityPack</a> to pull out the main article text,
stripping away navigation, ads, and other page elements.</p>
<h3>AI-Powered Categorization</h3>
<p>Here's where it gets interesting.
We'll use the AI model to analyze the content and assign a category:</p>
<pre><code class="language-csharp">public async Task&lt;string&gt; SummarizeBlogAsync(string blogContent)
{
    var prompt =
        @&quot;&quot;&quot;
        You are a blog content assistant. Summarize the following blog post
        as one of the following categories: Technology, Business, Programming,
        Architecture, DevOps, Cloud, Security, General.
        Only those eight values are allowed. Be as concise as possible.
        I want a 1-word response with one of these options: Technology, Business,
        Programming, Architecture, DevOps, Cloud, Security, General.

        The blog content is: {blogContent}
        &quot;&quot;&quot;;

    var response = await chatClient.GetResponseAsync(prompt);

    if (!response.Messages.Any())
    {
        return &quot;General&quot;;
    }

    var category = response.Messages.First().Text switch
    {
        var s when s.Contains(&quot;Technology&quot;, StringComparison.OrdinalIgnoreCase) =&gt; &quot;Technology&quot;,
        var s when s.Contains(&quot;Business&quot;, StringComparison.OrdinalIgnoreCase) =&gt; &quot;Business&quot;,
        var s when s.Contains(&quot;Programming&quot;, StringComparison.OrdinalIgnoreCase) =&gt; &quot;Programming&quot;,
        var s when s.Contains(&quot;Architecture&quot;, StringComparison.OrdinalIgnoreCase) =&gt; &quot;Architecture&quot;,
        var s when s.Contains(&quot;DevOps&quot;, StringComparison.OrdinalIgnoreCase) =&gt; &quot;DevOps&quot;,
        var s when s.Contains(&quot;Cloud&quot;, StringComparison.OrdinalIgnoreCase) =&gt; &quot;Cloud&quot;,
        var s when s.Contains(&quot;Security&quot;, StringComparison.OrdinalIgnoreCase) =&gt; &quot;Security&quot;,
        var s when s.Contains(&quot;General&quot;, StringComparison.OrdinalIgnoreCase) =&gt; &quot;General&quot;,
        _ =&gt; &quot;General&quot;
    };

    return category;
}
</code></pre>
<p>A few things make this work reliably:</p>
<ul>
<li>The prompt is explicit about allowed categories</li>
<li>We request a single-word response to avoid parsing complex output</li>
<li>The switch expression handles variations in the AI's response</li>
<li>There's always a fallback to &quot;General&quot;</li>
</ul>
<h3>Exposing the API</h3>
<p>Finally, we wrap everything in a simple API endpoint:</p>
<pre><code class="language-csharp">app.MapPost(&quot;/summarize-blog&quot;, async (
    string slug,
    BlogService blogService,
    BlogSummarizer blogSummarizer) =&gt;
{
    var content = await blogService.GetBlogContentAsync(slug);
    var category = await blogSummarizer.SummarizeBlogAsync(content);

    return Results.Ok(new
    {
        slug,
        category,
        content = $&quot;{content.Substring(0, 50)}...&quot;
    });
});
</code></pre>
<p>Call this endpoint with a blog post URL slug, and you get back the category and a preview of the content.
The dependency injection handles service resolution, and Aspire manages the AI model connection behind the scenes.</p>
<p>Here's an example response:</p>
<pre><code class="language-json">{
  &quot;slug&quot;: &quot;screaming-architecture&quot;, // https://www.milanjovanovic.tech/blog/screaming-architecture
  &quot;category&quot;: &quot;Architecture&quot;,
  &quot;content&quot;: &quot;If you were to glance at the folder structure of y...&quot;
}
</code></pre>
<p>Here's the distributed trace in Aspire showing the request and response flow.
You can see the request to the blog service and the AI model.</p>
<p><img src="/blogs/mnw_155/distributed_trace.png" alt="Distributed trace in Aspire showing the request and response flow using GitHub Models"></p>
<h2>Wrapping Up</h2>
<p>The GitHub Models integration with .NET Aspire removes much of the complexity from adding AI to your applications.
You get:</p>
<ul>
<li>Simple configuration through the AppHost pattern</li>
<li>Automatic service discovery and connection management</li>
<li>Access to multiple AI models without vendor lock-in</li>
<li>The same observability and deployment benefits as other Aspire resources</li>
</ul>
<p>Whether you're <a href="working-with-llms-in-dotnet-using-microsoft-extensions-ai"><strong>adding AI to an existing system</strong></a> or building something new, this integration provides the foundation you need.
Start with simple categorization or summarization, then expand as you learn what works for your use case.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_155.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[The 5 Most Common REST API Design Mistakes (and How to Avoid Them)]]></title>
            <link>https://milanjovanovic.tech/blog/the-5-most-common-rest-api-design-mistakes-and-how-to-avoid-them</link>
            <guid isPermaLink="false">the-5-most-common-rest-api-design-mistakes-and-how-to-avoid-them</guid>
            <pubDate>Sat, 09 Aug 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Five REST API design mistakes I see all the time, with practical fixes. Use consistent resource naming, evolve contracts instead of bumping versions, add envelopes, pagination and filtering, return Problem Details, and bake in auth, authz, and rate limits from day one.]]></description>
            <content:encoded><![CDATA[<p>Bad APIs create friction for developers, increase maintenance costs, and make change risky.
Good API design doesn't mean following every &quot;best practice&quot; blindly.
It means choosing the right trade-offs for your context and sticking to them.</p>
<p>Here are 5 common mistakes I see repeatedly, why they cause problems, and how to avoid them with pragmatic, battle-tested solutions.</p>
<h2>1. Inconsistent Naming and Structure</h2>
<p>Naming is the first thing consumers see.
Inconsistency here leads to constant documentation lookups, broken expectations, and more bugs.</p>
<p>Look, we've all been there.
You successfully call <code>/users</code> and <code>/products</code>, so naturally you try <code>/orders</code>.
But nope, this API uses <code>/order-list</code> for some reason.
Now you're back to the docs, breaking your flow, wondering why anyone would do this.
Multiply this friction by dozens of endpoints, and you've built an API that makes developers want to flip tables.</p>
<p>I get it, deep URL hierarchies like <code>/users/{id}/habits/{habitId}/entries/{entryId}/comments/{commentId}</code> feel satisfying.
They mirror your beautiful domain model!
But here's the thing: you've just <strong>hardcoded your entire data structure</strong> into your URLs.
When business requirements change (and they will), you can't reorganize without breaking clients.</p>
<p>Plus, what happens when someone only has a comment ID?
They need to somehow figure out the user, habit, and entry IDs just to fetch one comment.
That's ridiculous.</p>
<p>Keep it simple with <strong>plural nouns</strong>: <code>/users</code>, <code>/habits</code>, <code>/entries</code>.
No more guessing if it's <code>user</code> or <code>users</code>.</p>
<p><strong>Only nest when something truly belongs to something else.</strong>
User settings belong to and die with the user, so <code>/users/{id}/settings</code> makes sense.
Comments can exist independently, so <code>/users/{id}/posts/{postId}/comments</code> can be simplified.</p>
<p>Instead, flatten with filters:</p>
<pre><code class="language-http"># Instead of deep nesting
GET /users/{userId}/habits/{habitId}/entries

# Use filters for flexibility
GET /entries?userId={userId}&amp;habitId={habitId}
GET /entries?habitId={habitId}  # Now you can get entries without knowing the user
</code></pre>
<p>And please, for the love of all that is holy, wrap your arrays:</p>
<pre><code class="language-json">{
  &quot;data&quot;: [
    {
      &quot;id&quot;: &quot;e_8YH&quot;,
      &quot;habitId&quot;: &quot;code-review&quot;,
      &quot;at&quot;: &quot;2025-08-08T09:17:34Z&quot;,
      &quot;value&quot;: 5,
      &quot;unit&quot;: &quot;reviews&quot;,
      &quot;tags&quot;: [&quot;team&quot;]
    },
    {
      &quot;id&quot;: &quot;e_8Z2&quot;,
      &quot;habitId&quot;: &quot;deep-work&quot;,
      &quot;at&quot;: &quot;2025-08-07T07:00:00Z&quot;,
      &quot;value&quot;: 2,
      &quot;unit&quot;: &quot;pomodoros&quot;,
      &quot;note&quot;: &quot;EF filters optimized&quot;
    }
  ],
  &quot;total&quot;: 42,
  &quot;hasMore&quot;: true,
  &quot;nextCursor&quot;: &quot;cursor_01J9KaBcd&quot;
}
</code></pre>
<p>I know it feels like pointless boilerplate now, but trust me, when you need to add pagination info six months from now,
you'll thank yourself for not having to break every client that expects a raw array.</p>
<p>Yeah, the <a href="https://en.wikipedia.org/wiki/REST">REST</a> purists will complain that filters aren't &quot;RESTful enough.&quot;
Let them.
Your API will be flexible, maintainable, and actually pleasant to use.
I'll take that over conceptual purity any day.</p>
<h2>2. Poor Versioning Strategy</h2>
<p>Everyone defaults to <a href="api-versioning-in-aspnetcore"><strong>versioning</strong></a> (<code>/v1/users</code>) thinking they're being smart about future changes.
Spoiler: they're not. They're creating a maintenance nightmare.</p>
<p>Here's what actually happens when you have v1, v2, and v3 running:</p>
<ul>
<li>Every bug needs to be fixed three times</li>
<li>Every security patch needs three deployments</li>
<li>Your docs become a choose-your-own-adventure novel</li>
<li>Support has no idea which version that angry customer is using</li>
<li>You spend weekends maintaining code you wrote two years ago</li>
</ul>
<p>But the worst part?
<strong>Versioning makes you lazy</strong>.
Instead of thinking &quot;how can I evolve this without breaking clients?&quot; you just think &quot;eh, I'll bump the version.&quot;
Now your clients have to rewrite their entire integration because you wanted to rename a field.</p>
<p>Watch this disaster unfold:</p>
<pre><code>v1: GET /users returns {id, name, email}
v2: GET /users returns {id, fullName, emailAddress}  // &quot;looks cleaner!&quot;
v3: GET /users returns {id, firstName, lastName, email}  // &quot;we need split names!&quot;
</code></pre>
<p>Congrats, you're now maintaining three different response formats for the same damn data.
That v1 client?
They'll never get new features unless they rewrite everything.
Found a critical bug? Hope you enjoy patching it three times!</p>
<p>Here's the <strong>radical idea</strong>: <strong>don't version at all</strong>.
I'm serious.</p>
<p>Add fields, don't replace them:</p>
<pre><code class="language-json">// What you ship first
{ &quot;id&quot;: 1, &quot;name&quot;: &quot;John Doe&quot; }

// What you ship later (keeping the old field)
{
  &quot;id&quot;: 1,
  &quot;name&quot;: &quot;John Doe&quot;,  // Still there! Mark it deprecated in docs
  &quot;firstName&quot;: &quot;John&quot;,
  &quot;lastName&quot;: &quot;Doe&quot;
}
</code></pre>
<p>Need optional features?
Use query parameters:</p>
<pre><code class="language-http">GET /users/{id}?include=habits,entries
GET /users/{id}?format=detailed
</code></pre>
<p>If you absolutely must make a breaking change (and really think about this), create a new resource:</p>
<pre><code class="language-http"># Old faithful, unchanged
GET /users/{id}

# New hotness
GET /userProfiles/{id}
</code></pre>
<p>When breaking changes are truly unavoidable, at least be a decent human about it.
Give people 6-12 months notice.
Run both versions in parallel.
Write a migration guide that doesn't suck.
And for crying out loud, monitor who's still using the old stuff so you can reach out before pulling the plug.</p>
<p>Yes, this means you need to actually think about your API design upfront.
You can't just YOLO field names and fix them later.
But that constraint will make you design better APIs, and future-you will buy present-you a beer.</p>
<p>If you want to learn more about this, I recommend reading <a href="https://medium.com/good-api/api-change-management-2fe5bba32e9b">API Change Management</a>.</p>
<h2>3. Ignoring Pagination, Filtering, and Searching</h2>
<p>That <code>GET /entries</code> endpoint works great with your 10 test records.
Then you launch, get actual users, and suddenly you're returning 100,000 entries in a single response.
Your API times out, your mobile users on crappy connections hate you, and your cloud bill makes you cry.</p>
<p>&quot;We'll add pagination later,&quot; you said.
Well, now it's later, and adding pagination means breaking every client that expects an array.
Nice job.</p>
<p>Without filtering, your clients are downloading thousands of records to find the five they actually need.
It's like making someone download all of Wikipedia to read one article.
Your servers are melting, serializing data nobody wants.
Your users are burning through their data plans.
Everyone loses.</p>
<p><strong>Filtering</strong> is for when you know exactly what you want:</p>
<pre><code class="language-http">GET /entries?habitId=123&amp;date=2025-08-01&amp;status=completed
</code></pre>
<p><strong>Searching</strong> is for when you kinda know what you want:</p>
<pre><code class="language-http">GET /entries/search?q=morning+run+park
</code></pre>
<p>Don't try to be clever and combine them.
Filtering uses your database indexes efficiently.
Searching needs full-text magic.
Mix them and you'll end up with something that does neither well.</p>
<p>For pagination, you've got two choices, and they both kinda suck in different ways.</p>
<p><strong>Offset/limit</strong> is what everyone starts with:</p>
<pre><code class="language-http">GET /entries?offset=100&amp;limit=50
             # or you can call them skip and take
             # or you can call them page and pageSize
             # use whatever you like best
</code></pre>
<p>It's dead simple and lets users jump to page 5, but here's the fun part: add or delete an item while someone's paginating, and they'll either skip entries or see duplicates.
Plus, asking for offset=10000 makes your database cry as it counts through all those rows.</p>
<p><a href="understanding-cursor-pagination-and-why-its-so-fast-deep-dive"><strong>Cursor-based pagination</strong></a> is the &quot;proper&quot; solution:</p>
<pre><code class="language-http">GET /entries?limit=50&amp;cursor=eyJpZCI6MTIzfQ==
</code></pre>
<p>Rock solid, no skipped items, consistent performance.
But you can't jump to arbitrary pages and cursors can become invalid if underlying data changes significantly.</p>
<p>Yeah, implementing all this is a pain.
You need cursor encoding, parameter validation, query optimization.
But trying to add it after launch?
Wouldn't recommend.
Just build it right the first time.</p>
<h2>4. Unclear or Inconsistent Error Handling</h2>
<p><code>{&quot;error&quot;: &quot;An error occurred&quot;}</code> — if you return this, I hate you.</p>
<p>Seriously, when your API spits out these useless errors, here's what happens: I try random stuff hoping something works.
I add defensive code everywhere because I don't trust you.
I flood your support channel asking what's wrong.
Then I complain about your API on social media (any publicity is good publicity, eh?).</p>
<p>A good error tells me three things: <strong>what broke</strong>, <strong>why it broke</strong>, and <strong>how to fix it</strong>.
Is that so hard?</p>
<p>Stop inventing your own janky error format.
Use <a href="problem-details-for-aspnetcore-apis"><strong>Problem Details</strong></a> (RFC 9457) like a civilized developer:</p>
<pre><code class="language-json">{
  &quot;type&quot;: &quot;https://api.example.com/errors/validation-failed&quot;,
  &quot;title&quot;: &quot;Validation Failed&quot;,
  &quot;status&quot;: 400,
  &quot;detail&quot;: &quot;The request body contains invalid fields&quot;,
  &quot;instance&quot;: &quot;/habits/123&quot;,
  &quot;errors&quot;: [
    {
      &quot;field&quot;: &quot;name&quot;,
      &quot;reason&quot;: &quot;Must be between 1 and 100 characters&quot;,
      &quot;value&quot;: &quot;&quot;
    },
    {
      &quot;field&quot;: &quot;frequency&quot;,
      &quot;reason&quot;: &quot;Must be one of: daily, weekly, monthly&quot;,
      &quot;value&quot;: &quot;sometimes&quot;
    }
  ]
}
</code></pre>
<p>See how that actually helps me fix the problem? Revolutionary, I know.</p>
<p>And please use the right status codes. It's not that hard:</p>
<ul>
<li><code>400 Bad Request</code>: You sent garbage</li>
<li><code>401 Unauthorized</code>: Who are you?</li>
<li><code>403 Forbidden</code>: I know who you are, but no</li>
<li><code>404 Not Found</code>: That thing doesn't exist</li>
<li><code>409 Conflict</code>: That conflicts with something</li>
<li><code>422 Unprocessable Entity</code>: I understand what you want, but it's wrong<br>
(<em>I don't use this personally, and prefer returning 400 for validation failures</em>)</li>
<li><code>429 Too Many Requests</code>: Slow down, cowboy</li>
<li><code>500 Internal Server Error</code>: We screwed up</li>
<li><code>503 Service Unavailable</code>: We're drowning, try again later</li>
</ul>
<p>Now, don't go leaking your entire stack trace in production like an amateur.
Give friendly errors to users, detailed errors in dev/staging, and log the gory details server-side where you can actually use them.</p>
<h2>5. Ignoring Security Until It's Too Late</h2>
<p>&quot;We'll add auth in phase 2&quot; — famous last words before your API becomes a data buffet for hackers.
Ask the <a href="https://www.teaforwomen.com/cyberincident">Tea app</a> how that worked out for them.</p>
<p>Here's what happens when you try to bolt on security later: Every client breaks when you add authentication.
That data you've been leaking? It's probably been scraped already.
Your compliance audit? Failed.
That one security incident? Your users will bring it up for years.</p>
<p><strong>Authentication</strong> (who are you?) and <strong>Authorization</strong> (what can you do?) are different things.
I've seen so many APIs that check if you're logged in but never check if you should actually access that data.
Don't be that person.</p>
<p><a href="advanced-rate-limiting-use-cases-in-dotnet"><strong>Rate limiting</strong></a> isn't just about stopping abuse, it's about fairness.
Start simple: 1000 requests per hour per API key.
When they hit the limit, return <code>429</code> with headers showing when they can try again.
Then get fancy: different limits for different endpoints, higher limits for paying customers,
lower limits for that one client who keeps doing weird stuff.</p>
<p><strong>HTTPS everywhere</strong>.
Yes, even for your internal &quot;no one will ever find this&quot; API.
It's 2025, not 2005.
<a href="https://letsencrypt.org/">Let's Encrypt</a> is free.
You have no excuse.</p>
<p>Look, security makes things slower and more complex.
Auth checks on every request, encryption overhead, state management for rate limiting, it all adds up.
But you know what's worse?
Explaining to your users why their data is being sold on the dark web.
Build security in from the start, or prepare for a world of pain.</p>
<h2>Final Thoughts</h2>
<p>Good API design isn't about perfection, it's about making intentional, informed decisions.
Every choice is a tradeoff.
Consistency might limit flexibility.
Security will impact performance.
Stability means slower innovation.</p>
<p>Here's what actually matters: <strong>know your tradeoffs and own them</strong>.
Document why you made these choices (future you will thank you).
Stay consistent even when it's tempting not to.
Design for evolution, not some imaginary perfect future.
And listen to your users, but don't turn your API into a frankenstein monster trying to please everyone.</p>
<p><strong>Your API is a promise to other developers</strong>.
Every time you break that promise (with a breaking change, an inconsistent pattern, or a useless error message) you lose their trust.
And trust me, developers hold grudges.</p>
<p>Build the API you'd want to use.
Your developers will thank you, your support team will thank you, and honestly, you'll thank yourself six months from now when you have to maintain this thing.</p>
<p>Want to dive deeper? Check out my <a href="/pragmatic-rest-apis"><strong>Pragmatic REST APIs</strong></a> course where I cover all of this (and more).</p>
<p>P.S. Any sharp language here is just me being snarky for emphasis.
I'm critiquing patterns, not people.
Don't take the strong wording too seriously.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_154.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How to Keep Your Data Boundaries Intact in a Modular Monolith]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-keep-your-data-boundaries-intact-in-a-modular-monolith</link>
            <guid isPermaLink="false">how-to-keep-your-data-boundaries-intact-in-a-modular-monolith</guid>
            <pubDate>Sat, 02 Aug 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Want real boundaries in your modular monolith? This article shows how to enforce them at the database level using Postgres schemas, roles, and EF Core — so your modules can't step on each other's data, even by accident.]]></description>
            <content:encoded><![CDATA[<p><a href="what-is-a-modular-monolith"><strong>Modular monoliths</strong></a> promise the productivity of a monolith and the clear boundaries of microservices.
They work because each module is self-contained: its domain model, behavior and data live behind a boundary.
But one of the hardest places to maintain those boundaries is in the database.</p>
<p>Nothing stops a developer from running a rogue <code>JOIN</code> across tables or bypassing a public API.</p>
<p>In previous articles, I described <a href="modular-monolith-data-isolation"><strong>four levels of data isolation</strong></a> (table, schema, database and alternative persistence)
and argued that <a href="internal-vs-public-apis-in-modular-monoliths"><strong>modules should expose explicit APIs</strong></a> to access their data.</p>
<p>This article goes deeper on the database side.
You'll learn how to carve out logical and physical boundaries using PostgreSQL schemas and roles and EF Core,
why those choices matter, and how to handle cross-cutting queries without breaking encapsulation.</p>
<h2>Why enforce database boundaries?</h2>
<p>In a modular monolith each module <strong>owns its data</strong>.
If Module A reaches into Module B's tables, you lose this constraint and your modules become tightly coupled.
Instead, B exposes a public API and hides its persistence logic from consumers.</p>
<p>Beyond clean code, enforcing boundaries at the database level protects you against mistakes
and makes it easier to extract a module into its own service later.</p>
<p>Database schemas act like folders: they let you organise objects and share a database among many users,
but a user can access any schema only if they have privileges.
This means we can deliberately lock each module to its own schema.</p>
<p><strong>The strategy</strong></p>
<ul>
<li>Create a schema per module and a dedicated database role.</li>
<li>Grant that role privileges only on its schema and set its default search path.</li>
<li>Use EF with one <code>DbContext</code> per module, setting a default schema and connection string per module.</li>
<li>For cross-cutting queries, publish a read-only database view that acts like a public API.</li>
<li><em>Optional</em>: Use row-level security policies to restrict access within a table.</li>
</ul>
<p>These practices give us <strong>enforceable boundaries</strong> while keeping the <strong>operational overhead low</strong>.</p>
<h2>Schemas, roles and search paths</h2>
<p>PostgreSQL lets you create multiple schemas within a single database.
A module can define its own schema and a role that owns it.
The role only has usage and table-level privileges on that schema.
For example, for an orders module:</p>
<pre><code class="language-sql">-- create a user for the module and its schema
CREATE ROLE orders_role LOGIN PASSWORD 'orders_secret';
CREATE SCHEMA orders AUTHORIZATION orders_role;
GRANT USAGE ON SCHEMA orders TO orders_role;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA orders TO orders_role;
ALTER ROLE orders_role SET search_path = orders;
</code></pre>
<p>The <code>ALTER ROLE</code> command sets the role's default search path so unqualified names resolve to the module's schema.
If you don't want to rely on the search path, always qualify table names (<code>orders.table_name</code>).</p>
<p>Row-level security (RLS) allows you to filter rows based on a policy expression.
You enable RLS on a table and define a policy referencing <code>current_user</code> or columns:</p>
<p>RLS is powerful for multi-tenant scenarios or sensitive data, but it adds complexity.
Start with schemas and roles and add RLS only when necessary.</p>
<h2>Configuring EF schemas and multiple DbContexts</h2>
<p>I assume readers are comfortable with EF Core, so we'll skip the basics and focus on what matters for modular monoliths:</p>
<ul>
<li>Use <a href="using-multiple-ef-core-dbcontext-in-single-application"><strong>one DbContext per module</strong></a>.
Each context contains only the entities of its module, and you call <code>modelBuilder.HasDefaultSchema(&quot;orders&quot;)</code> in <code>OnModelCreating</code>
to map entities to the correct schema.
Setting a default schema also affects sequences and migrations.</li>
<li>Provide a connection string per module using the module's role.
Even if modules share the same database, separate credentials ensure that a misconfigured context cannot access another schema.</li>
<li>Configure the migrations history table in each context using <code>MigrationsHistoryTable(&quot;__EFMigrationsHistory&quot;, &quot;orders&quot;)</code>
so that EF's migration metadata stays within the module's schema.</li>
</ul>
<p>These settings ensure EF Core queries and migrations respect the boundaries established by the database.</p>
<h2>Step-by-step: Enforcing module boundaries</h2>
<p>Suppose we have two modules (<strong>Orders</strong> and <strong>Shipping</strong>) and we want to enforce boundaries between them.
Here's what we need to do:</p>
<ol>
<li>
<p><strong>Create schemas and roles</strong>.
Use SQL to create orders and shipping schemas and their corresponding roles (<code>orders_role</code>, <code>shipping_role</code>).
Grant each role privileges only on its schema.</p>
<pre><code class="language-sql">-- Orders schema and role
CREATE ROLE orders_role LOGIN PASSWORD 'orders_secret';
CREATE SCHEMA orders AUTHORIZATION orders_role;
GRANT USAGE ON SCHEMA orders TO orders_role;
GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA orders TO orders_role;
ALTER ROLE orders_role SET search_path = orders;

 -- Shipping schema and role
 CREATE ROLE shipping_role LOGIN PASSWORD 'shipping_secret';
 CREATE SCHEMA shipping AUTHORIZATION shipping_role;
 GRANT USAGE ON SCHEMA shipping TO shipping_role;
 GRANT SELECT, INSERT, UPDATE, DELETE ON ALL TABLES IN SCHEMA shipping TO shipping_role;
 ALTER ROLE shipping_role SET search_path = shipping;
</code></pre>
</li>
<li>
<p><strong>Add connection strings</strong>. In configuration, define separate connection strings per module:</p>
<pre><code class="language-json">{
  &quot;ConnectionStrings&quot;: {
    &quot;Orders&quot;: &quot;Host=localhost;Database=appdb;Username=orders_role;Password=orders_secret&quot;,
    &quot;Shipping&quot;: &quot;Host=localhost;Database=appdb;Username=shipping_role;Password=shipping_secret&quot;
  }
}
</code></pre>
</li>
<li>
<p><strong>Define DbContexts</strong>. Each module defines its own context and sets the default schema. For the Orders module:</p>
<pre><code class="language-csharp">public class OrdersDbContext : DbContext
{
    public DbSet&lt;Order&gt; Orders { get; set; } = default!;
    public DbSet&lt;OrderLine&gt; OrderLines { get; set; } = default!;

    public OrdersDbContext(DbContextOptions&lt;OrdersDbContext&gt; options) : base(options) { }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        // set default schema for all entities in this context
        modelBuilder.HasDefaultSchema(&quot;orders&quot;);
        // optional: configure tables explicitly
        modelBuilder.Entity&lt;Order&gt;().ToTable(&quot;orders&quot;);
        modelBuilder.Entity&lt;OrderLine&gt;().ToTable(&quot;order_lines&quot;);
        base.OnModelCreating(modelBuilder);
    }
}
</code></pre>
<p>Register each context with its connection string and specify the migrations history table:</p>
<pre><code class="language-csharp">builder.Services.AddDbContext&lt;OrdersDbContext&gt;(options =&gt;
    options.UseNpgsql(builder.Configuration.GetConnectionString(&quot;Orders&quot;),
        o =&gt; o.MigrationsHistoryTable(&quot;__EFMigrationsHistory&quot;, &quot;orders&quot;)));

builder.Services.AddDbContext&lt;ShippingDbContext&gt;(options =&gt;
    options.UseNpgsql(builder.Configuration.GetConnectionString(&quot;Shipping&quot;),
        o =&gt; o.MigrationsHistoryTable(&quot;__EFMigrationsHistory&quot;, &quot;shipping&quot;)));
</code></pre>
</li>
<li>
<p><strong>Maintain migrations separately</strong>.
Because each module has its own context and schema, you maintain migrations separately.
When generating a migration, specify the context:</p>
<pre><code class="language-bash">dotnet ef migrations add InitialOrders --context OrdersDbContext --output-dir Data/Migrations/Orders
</code></pre>
<p>Repeat this for the Shipping context.
EF Core will generate migration classes that create tables within the specified schema (because of <code>HasDefaultSchema</code>).
Remember to apply the migrations in the correct order when deploying. You can automate this by having a migration runner iterate through the contexts.</p>
</li>
</ol>
<p>With schemas, roles and multiple contexts in place, the data boundary becomes enforceable at the database level:</p>
<ul>
<li>The <strong>Orders</strong> module's <code>DbContext</code> knows only about the orders schema and uses credentials that have no privileges on the <strong>Shipping</strong> schema.</li>
<li>The <strong>Shipping</strong> module cannot query <code>orders.orders</code> directly because its role lacks the necessary privileges.</li>
<li>Cross-module communication must go through the module's public API (or an asynchronous event).
This explicit coupling makes dependencies obvious and maintainable.</li>
</ul>
<h2>Cross-cutting queries</h2>
<p>Even in a modular system you occasionally need a screen that spans multiple modules,
such as an <strong>Order History</strong> page that shows order and shipping data.
Resist the temptation to <code>JOIN</code> across schemas.
Two approaches can help:</p>
<ul>
<li>
<p><strong>Dedicated read model</strong>.
One module owns a view model and subscribes to events from others.
This pattern works well when modules might be extracted later.</p>
</li>
<li>
<p><strong>Database views with privileges</strong>.
Since our modules share a database, we can create a read-only view in the public schema that joins the relevant tables.
We grant <code>SELECT</code> on the view to a special role or module.
This view acts like a controlled public API.
Consumers query the view but cannot access the underlying tables directly.
The trade-off is similar to calling a synchronous API: if you later split the database, the view will have to be replaced with a service call.</p>
</li>
</ul>
<p>For example:</p>
<pre><code class="language-sql">CREATE VIEW public.order_summary AS
SELECT o.id, o.total, s.status
FROM orders.orders o
JOIN shipping.shipments s ON s.order_id = o.id;

-- grant read access to a reporting role
GRANT SELECT ON public.order_summary TO reporting_role;
</code></pre>
<p>This approach lets you build dashboards or admin screens without breaking boundaries.</p>
<h2>Conclusion</h2>
<p>By giving each module its own schema, role and DbContext, and by controlling cross-module access via views or APIs,
you <strong>ensure that boundaries in your modular monolith are enforceable</strong> rather than aspirational.
This discipline makes it easier to evolve your system and paves the way for a future microservice extraction.</p>
<p>If you found this article valuable, check out my previous posts on modular monoliths:</p>
<ul>
<li><a href="what-is-a-modular-monolith">What Is a Modular Monolith?</a></li>
<li><a href="modular-monolith-data-isolation">Modular Monolith Data Isolation</a></li>
<li><a href="internal-vs-public-apis-in-modular-monoliths">Internal vs Public APIs in Modular Monoliths</a></li>
</ul>
<p>If you want a structured, hands-on approach to building modular systems, from defining boundaries to extracting services,
check out my <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a> course.
Join more than 2,100+ students who have mastered modular monoliths with it.
The course walks through these patterns in depth and shows how to apply them in a real codebase.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_153.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Named Query Filters in EF 10 (multiple query filters per entity)]]></title>
            <link>https://milanjovanovic.tech/blog/named-query-filters-in-ef-10-multiple-query-filters-per-entity</link>
            <guid isPermaLink="false">named-query-filters-in-ef-10-multiple-query-filters-per-entity</guid>
            <pubDate>Sat, 26 Jul 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[EF 10 introduces named query filters, allowing multiple filters on a single entity and letting you selectively disable specific filters without turning off all query filters. This article explains how the feature works, shows practical examples for combining soft deletion and multi-tenancy, highlights best practices like using constants for filter names, and explores where such fine-grained control is particularly useful.]]></description>
            <content:encoded><![CDATA[<p>Entity Framework Core's global query filters have long been a convenient way to apply common conditions to all queries on an entity.
They're especially handy in scenarios like <a href="implementing-soft-delete-with-ef-core"><strong>soft deletion</strong></a>
and <a href="multi-tenant-applications-with-ef-core"><strong>multi-tenancy</strong></a>,
where you want the same <code>WHERE</code> clause added automatically to every query.</p>
<p>Previous versions of EF Core, however, suffered from <strong>one big limitation</strong>: each entity type could only have one filter defined.
If you needed to combine multiple conditions (for example, soft-delete and tenant isolation)
you either had to write explicit <code>&amp;&amp;</code> expressions or manually disable and reapply filters in specific queries.</p>
<p>With EF 10, that changes.
The new <strong>named query filters</strong> feature lets you attach multiple filters to a single entity and reference them by name.
You can then disable individual filters as needed, rather than turning off all filters at once.</p>
<p>Let's explore this new capability, why it matters, and some practical ways to use it.</p>
<h2>What Are Query Filters?</h2>
<p>If you've used EF Core for a while, you may already be familiar with
<a href="https://learn.microsoft.com/en-us/ef/core/querying/filters">global query filters</a>.
A query filter is a condition that EF automatically applies to all queries for a particular entity type.
Under the hood, EF adds a <code>WHERE</code> clause whenever that entity is queried. Typical uses include:</p>
<ul>
<li><strong>Soft deletion</strong>: filtering out rows where IsDeleted is true so that deleted records don't show up in queries by default</li>
<li><strong>Multi-tenancy</strong>: filtering by a TenantId so that each tenant only sees its own data</li>
</ul>
<p>For example, a soft-delete filter might be configured like this:</p>
<pre><code class="language-csharp">modelBuilder.Entity&lt;Order&gt;()
    .HasQueryFilter(order =&gt; !order.IsDeleted);
</code></pre>
<p>With the filter in place, every query on <code>Orders</code> automatically excludes soft-deleted records.
To include deleted data (say, for an admin report), you can call <code>IgnoreQueryFilters()</code> on the query.
The downside is that all filters on that entity are disabled,
which opens the door to accidentally leaking data you don't intend to show.</p>
<h2>Using Multiple Query Filters</h2>
<p>Until now, EF permitted only one query filter per entity.
If you called <code>HasQueryFilter</code> twice on the same entity, the second call overwrote the first.
To combine filters you had to write a single expression with <code>&amp;&amp;</code>:</p>
<pre><code class="language-csharp">modelBuilder.Entity&lt;Order&gt;()
    .HasQueryFilter(order =&gt; !order.IsDeleted &amp;&amp; order.TenantId == tenantId);
</code></pre>
<p>This works but makes it impossible to selectively disable one condition.
<code>IgnoreQueryFilters()</code> disables both, forcing you to manually re-apply whichever filter you still need.
EF 10 introduces a better alternative: <strong>named query filters</strong>.</p>
<p>To attach multiple filters to an entity, call <code>HasQueryFilter</code> with a name for each filter:</p>
<pre><code class="language-csharp">modelBuilder.Entity&lt;Order&gt;()
    .HasQueryFilter(&quot;SoftDeletionFilter&quot;, order =&gt; !order.IsDeleted)
    .HasQueryFilter(&quot;TenantFilter&quot;, order =&gt; order.TenantId == tenantId);
</code></pre>
<p>Under the hood, EF creates separate filters identified by the names you provide.
You can now turn off just the soft-delete filter while keeping the tenant filter in place:</p>
<pre><code class="language-csharp">// Returns all orders (including soft‑deleted) for the current tenant
var allOrders = await context.Orders.IgnoreQueryFilters([&quot;SoftDeletionFilter&quot;]).ToListAsync();
</code></pre>
<p>If you omit the parameter array, <code>IgnoreQueryFilters()</code> disables all filters for the entity.</p>
<h2>Tip: Using Constants for Filter Names</h2>
<p>Named filters use string keys.
Hard-coding those names throughout your codebase makes it easy to introduce typos and brittle magic strings.
To avoid this, define constants or enums for your filter names and reuse them wherever needed.
For example:</p>
<pre><code class="language-csharp">public static class OrderFilters
{
    public const string SoftDelete = nameof(SoftDelete);
    public const string Tenant = nameof(Tenant);
}

modelBuilder.Entity&lt;Order&gt;()
    .HasQueryFilter(OrderFilters.SoftDelete, order =&gt; !order.IsDeleted)
    .HasQueryFilter(OrderFilters.Tenant, order =&gt; order.TenantId == tenantId);

// Later in your query
var allOrders = await context.Orders.IgnoreQueryFilters([OrderFilters.SoftDelete]).ToListAsync();
</code></pre>
<p>Having the filter names defined in a single place reduces duplication and improves maintainability.
Another best practice is to wrap the ignore call in an extension method or repository
so that consumers don't directly interact with filter names at all. For example:</p>
<pre><code class="language-csharp">public static IQueryable&lt;Order&gt; IncludeSoftDeleted(this IQueryable&lt;Order&gt; query)
    =&gt; query.IgnoreQueryFilters([OrderFilters.SoftDelete]);
</code></pre>
<p>This makes your intent explicit and centralizes the filter logic in one place.</p>
<h2>Wrapping Up</h2>
<p>The introduction of <strong>named query filters</strong> in EF 10 removes one of the longstanding limitations
of EF's <a href="how-to-use-global-query-filters-in-ef-core"><strong>global query filters</strong></a> feature.
You can now:</p>
<ul>
<li>Attach multiple filters to a single entity and manage them individually</li>
<li>Selectively disable specific filters in a LINQ query using <code>IgnoreQueryFilters([&quot;FilterName&quot;])</code></li>
<li>Simplify common patterns like <a href="implementing-soft-delete-with-ef-core"><strong>soft deletion</strong></a> plus
<a href="multi-tenant-applications-with-ef-core"><strong>multi-tenancy</strong></a> without resorting to complicated conditional logic</li>
</ul>
<p>Named query filters can become a powerful tool to keep your queries clean and your domain logic encapsulated.</p>
<p>Whether you're building SaaS applications that isolate tenant data or
ensuring that deleted records stay hidden until you explicitly need them,
EF 10's named query filters offer the flexibility you've been waiting for.</p>
<p>Give them a try in the preview and start thinking about how they can simplify your codebase.</p>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_152.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[PDF Reporting in .NET With HTML Templates and PuppeteerSharp (and it's free)]]></title>
            <link>https://milanjovanovic.tech/blog/pdf-reporting-in-dotnet-with-html-templates-and-puppeteersharp</link>
            <guid isPermaLink="false">pdf-reporting-in-dotnet-with-html-templates-and-puppeteersharp</guid>
            <pubDate>Sat, 19 Jul 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Generate PDF reports in .NET using HTML templates and a headless browser. We'll explore Handlebars.NET and PuppeteerSharp, compare alternatives, and analyze performance tradeoffs.]]></description>
            <content:encoded><![CDATA[<p>Sooner or later, every .NET developer needs to generate PDF reports.
And generating polished PDF reports in .NET doesn't have to be painful.
Let me explain how.</p>
<p>My go-to method?</p>
<p><strong>HTML to PDF conversion</strong>.
It's:</p>
<ul>
<li>Simple to implement</li>
<li>Very flexible</li>
<li>Ideal for stylized reports</li>
</ul>
<p>But many popular libraries require a commercial license.
This post walks you through a completely free approach using:</p>
<ul>
<li><strong><a href="http://Handlebars.NET">Handlebars.NET</a></strong> for templating</li>
<li><strong>PuppeteerSharp</strong> for headless rendering</li>
</ul>
<p>This gives you full control over layout, styling, and content.
It's perfect for invoices, dashboards, and exports.</p>
<p>We'll start from scratch and build up to a complex, dynamic invoice report with full styling, images, and even headers and footers.</p>
<h2>Why HTML + Headless Browser?</h2>
<p><strong>Pros:</strong></p>
<ul>
<li>Rich styling with CSS</li>
<li>Easy to preview/debug in browser</li>
<li>Supports charts/images via JS/CSS</li>
<li>Full control over layout (media queries, page breaks, etc.)</li>
</ul>
<p><strong>Cons:</strong></p>
<ul>
<li>Requires bundling a browser (e.g. Chromium)</li>
<li>Slower than native PDF libraries</li>
<li>Slightly more setup complexity</li>
</ul>
<h2>Setting Up the Project</h2>
<p>We'll start by installing the NuGet packages we need for Handlebars and PuppeteerSharp:</p>
<pre><code class="language-powershell">Install-Package Handlebars.Net
Install-Package PuppeteerSharp
</code></pre>
<p>Next, we'll create our first template.
It's a simple HTML document with some Handlebars placeholders.
You'll notice them with the <code>{{variable}}</code> syntax.
These placeholders will be replaced with actual data when rendering the template.</p>
<pre><code class="language-html">&lt;!-- Templates/InvoiceTemplate.html --&gt;
&lt;!-- The file extension doesn't really matter. --&gt;
&lt;html lang=&quot;en&quot;&gt;
  &lt;head&gt;
    &lt;style&gt;
      body {
        font-family: Arial;
      }
    &lt;/style&gt;
  &lt;/head&gt;
  &lt;body&gt;
    &lt;h1&gt;Invoice #{{Number}}&lt;/h1&gt;

    &lt;p&gt;Date: {{formatDate IssuedDate}}&lt;/p&gt;

    &lt;h2&gt;From:&lt;/h2&gt;
    &lt;p&gt;{{SellerAddress.CompanyName}}&lt;/p&gt;
    &lt;p&gt;{{SellerAddress.Email}}&lt;/p&gt;

    &lt;h2&gt;To:&lt;/h2&gt;
    &lt;p&gt;{{CustomerAddress.CompanyName}}&lt;/p&gt;
    &lt;p&gt;{{CustomerAddress.Email}}&lt;/p&gt;

    &lt;h2&gt;Items:&lt;/h2&gt;
    &lt;table&gt;
      &lt;tr&gt;
        &lt;th&gt;Name&lt;/th&gt;
        &lt;th&gt;Price&lt;/th&gt;
      &lt;/tr&gt;
      {{#each LineItems}}
      &lt;tr&gt;
        &lt;td&gt;{{Name}}&lt;/td&gt;
        &lt;td&gt;{{formatCurrency Price}}&lt;/td&gt;
      &lt;/tr&gt;
      {{/each}}
    &lt;/table&gt;

    &lt;p&gt;&lt;strong&gt;Total: {{formatCurrency Total}}&lt;/strong&gt;&lt;/p&gt;
  &lt;/body&gt;
&lt;/html&gt;
</code></pre>
<p>The function calls, like <code>{{formatDate IssuedDate}}</code>, are custom helpers we can define in Handlebars.
You register them like this:</p>
<pre><code class="language-csharp">Handlebars.RegisterHelper(&quot;formatDate&quot;, (context, arguments) =&gt;
{
    if (arguments[0] is DateOnly date)
    {
        return date.ToString(&quot;dd/MM/yyyy&quot;);
    }
    return arguments[0]?.ToString() ?? &quot;&quot;;
});
</code></pre>
<p>This allows us to format dates, currencies, or any other data type as needed.
You register these helpers before compiling the template, once per application start is enough.</p>
<h2>Rendering the Template and PDF</h2>
<p>How do we render this template and convert it to PDF?
We'll use <a href="https://github.com/Handlebars-Net/Handlebars.Net">Handlebars.NET</a> to compile the template with data, then PuppeteerSharp to render it to PDF.</p>
<p>First, we read the template file and compile it with <code>Handlebars</code>:</p>
<pre><code class="language-csharp">var template = File.ReadAllText(&quot;Templates/InvoiceTemplate.html&quot;);
var data = new {
    customer = &quot;Milan Jovanović&quot;,
    items = new[] {
        new { description = &quot;Software License&quot;, price = 99 },
        new { description = &quot;Support Plan&quot;, price = 49 }
    }
};

var compiledTemplate = Handlebars.Compile(template);

string html = compiledTemplate(data);
</code></pre>
<p>This gives us the final HTML with all placeholders replaced by actual data.</p>
<p><strong>Note</strong>: You don't necessarily need to use Handlebars.
You can use any templating engine that suits your needs,
like <a href="flexible-pdf-reporting-in-net-using-razor-views"><strong>Razor</strong></a> or <a href="https://github.com/scriban/scriban">Scriban</a>.</p>
<p>Next, we need to convert this HTML to PDF using PuppeteerSharp.
We'll launch a headless browser, set the content to our HTML, and then generate the PDF.
Here's how we do it:</p>
<pre><code class="language-csharp">// Ensure PuppeteerSharp has the browser binaries
var browserFetcher = new BrowserFetcher();
await browserFetcher.DownloadAsync(BrowserFetcher.DefaultChromiumRevision);

// Launch the browser and create a new page
using var browser = await Puppeteer.LaunchAsync(new LaunchOptions { Headless = true });
using var page = await browser.NewPageAsync();

// Set the content of the page to our compiled HTML
await page.SetContentAsync(html);

// Optional: wait for fonts to load if using custom fonts
await page.EvaluateExpressionHandleAsync(&quot;document.fonts.ready&quot;);

byte[] pdf = await page.PdfDataAsync(new PdfOptions {
    Format = PaperFormat.A4,
    PrintBackground = true,
    MarginOptions = new MarginOptions
    {
        Top = &quot;50px&quot;,
        Right = &quot;20px&quot;,
        Bottom = &quot;50px&quot;,
        Left = &quot;20px&quot;
    }
});
</code></pre>
<p>This gives us a byte array containing the PDF data.
You can then save it to a file or return it from an API endpoint.</p>
<p>Here's a simple example of returning it from a Minimal API endpoint:</p>
<pre><code class="language-csharp">return Results.File(pdf, &quot;application/pdf&quot;, &quot;invoice.pdf&quot;);
</code></pre>
<p>Here's what the document looks like when rendered:</p>
<figure className="figure-center">
  <div className="bordered">
    ![Simple PDF Template example rendered as PDF](/blogs/mnw_151/simple_template.png)
  </div>
  <figcaption>
    This is a simple PDF template example with dynamic templated content.
  </figcaption>
</figure>
<h2>Enhancements: Images, Header/Footer, Styling</h2>
<p>What are some improvements we can make to this basic setup?</p>
<p>For example, you can add images to your template using the <code>&lt;img&gt;</code> tag.
A simple approach is to use a base64-encoded image directly in the HTML.
Note that we're passing the image data using the <code>LogoBase64</code> variable.</p>
<pre><code class="language-html">&lt;img
  src=&quot;data:image/png;base64,{{LogoBase64}}&quot;
  alt=&quot;Logo&quot;
  style=&quot;height:50px; max-width:200px; object-fit:contain;&quot;
/&gt;
</code></pre>
<p>We can also render dynamic headers and footers using PuppeteerSharp's built-in support.
You can define these in the <code>PdfOptions</code> object when generating the PDF.
Here's an example:</p>
<pre><code class="language-csharp">var pdfOptions = new PdfOptions
{
    HeaderTemplate =
        @&quot;&quot;&quot;
        &lt;div style='font-size: 14px; text-align: center; padding: 10px;'&gt;
            &lt;span style='margin-right: 20px;'&gt;&lt;span class='title'&gt;&lt;/span&gt;&lt;/span&gt;
            &lt;span&gt;&lt;span class='date'&gt;&lt;/span&gt;&lt;/span&gt;
        &lt;/div&gt;
        &quot;&quot;&quot;,
    FooterTemplate =
        @&quot;&quot;&quot;
        &lt;div style='font-size: 14px; text-align: center; padding: 10px;'&gt;
            &lt;span style='margin-right: 20px;'&gt;Generated on &lt;span class='date'&gt;&lt;/span&gt;&lt;/span&gt;
            &lt;span&gt;Page &lt;span class='pageNumber'&gt;&lt;/span&gt; of &lt;span class='totalPages'&gt;&lt;/span&gt;&lt;/span&gt;
        &lt;/div&gt;
        &quot;&quot;&quot;,
    DisplayHeaderFooter = true
};
</code></pre>
<p>PuppeteerSharp uses CSS classes like <code>title</code>, <code>date</code>, <code>pageNumber</code>, and <code>totalPages</code> to inject dynamic values.
This could be different for some other libraries, so check the documentation.</p>
<p>Lastly, I want to mention that you can use CSS for advanced styling.
This can be inline in the HTML or in a separate CSS file.
You can also reference external stylesheets if needed, using the <code>&lt;link&gt;</code> tag.</p>
<p>Here's a complete example of a more complex template with images, headers, and footers:</p>
<pre><code class="language-html">&lt;html lang=&quot;en&quot;&gt;
  &lt;head&gt;
    &lt;meta charset=&quot;UTF-8&quot; /&gt;
    &lt;meta name=&quot;viewport&quot; content=&quot;width=device-width, initial-scale=1.0&quot; /&gt;
    &lt;title&gt;Invoice #{{Number}}&lt;/title&gt;
    &lt;style&gt;
      /* Omitted for brevity */
    &lt;/style&gt;
  &lt;/head&gt;
  &lt;body&gt;
    &lt;div class=&quot;invoice-container&quot;&gt;
      &lt;!-- Header with Logo --&gt;
      &lt;div
        style=&quot;display: flex; justify-content: space-between; align-items: flex-start;&quot;
      &gt;
        &lt;div&gt;
          &lt;h1 class=&quot;invoice-title&quot;&gt;Invoice #{{Number}}&lt;/h1&gt;
          &lt;div class=&quot;invoice-dates&quot;&gt;
            &lt;p&gt;&lt;strong&gt;Issued:&lt;/strong&gt; {{formatDate IssuedDate}}&lt;/p&gt;
            &lt;p&gt;&lt;strong&gt;Due:&lt;/strong&gt; {{formatDate DueDate}}&lt;/p&gt;
          &lt;/div&gt;
        &lt;/div&gt;
        &lt;div&gt;
          {{#if LogoBase64}}
          &lt;img
            src=&quot;data:image/png;base64,{{LogoBase64}}&quot;
            alt=&quot;Logo&quot;
            style=&quot;height:50px; max-width:200px; object-fit:contain;&quot;
          /&gt;
          {{/if}}
        &lt;/div&gt;
      &lt;/div&gt;
      &lt;hr
        style=&quot;margin: 20px 0; border: none; border-top: 1px solid #e9ecef;&quot;
      /&gt;

      &lt;!-- Addresses - Side by Side --&gt;
      &lt;div class=&quot;addresses&quot;&gt;
        &lt;!-- Seller Address --&gt;
        &lt;div class=&quot;address-box&quot;&gt;
          &lt;h3 class=&quot;address-title&quot;&gt;From:&lt;/h3&gt;
          &lt;div class=&quot;address-content&quot;&gt;
            &lt;p class=&quot;company-name&quot;&gt;{{SellerAddress.CompanyName}}&lt;/p&gt;
            &lt;p&gt;{{SellerAddress.Street}}&lt;/p&gt;
            &lt;p&gt;{{SellerAddress.City}}, {{SellerAddress.State}}&lt;/p&gt;
            &lt;p class=&quot;email&quot;&gt;{{SellerAddress.Email}}&lt;/p&gt;
          &lt;/div&gt;
        &lt;/div&gt;

        &lt;!-- Customer Address --&gt;
        &lt;div class=&quot;address-box&quot;&gt;
          &lt;h3 class=&quot;address-title&quot;&gt;Bill To:&lt;/h3&gt;
          &lt;div class=&quot;address-content&quot;&gt;
            &lt;p class=&quot;company-name&quot;&gt;{{CustomerAddress.CompanyName}}&lt;/p&gt;
            &lt;p&gt;{{CustomerAddress.Street}}&lt;/p&gt;
            &lt;p&gt;{{CustomerAddress.City}}, {{CustomerAddress.State}}&lt;/p&gt;
            &lt;p class=&quot;email&quot;&gt;{{CustomerAddress.Email}}&lt;/p&gt;
          &lt;/div&gt;
        &lt;/div&gt;
      &lt;/div&gt;

      &lt;!-- Items Table --&gt;
      &lt;div class=&quot;items-section&quot;&gt;
        &lt;h2 class=&quot;items-title&quot;&gt;Items&lt;/h2&gt;
        &lt;table class=&quot;items-table&quot;&gt;
          &lt;thead&gt;
            &lt;tr&gt;
              &lt;th&gt;#&lt;/th&gt;
              &lt;th&gt;Description&lt;/th&gt;
              &lt;th&gt;Price&lt;/th&gt;
              &lt;th&gt;Qty&lt;/th&gt;
              &lt;th&gt;Total&lt;/th&gt;
            &lt;/tr&gt;
          &lt;/thead&gt;
          &lt;tbody&gt;
            {{#each LineItems}}
            &lt;tr&gt;
              &lt;td&gt;{{@index}}&lt;/td&gt;
              &lt;td&gt;{{Name}}&lt;/td&gt;
              &lt;td&gt;{{formatCurrency Price}}&lt;/td&gt;
              &lt;td&gt;{{formatNumber Quantity}}&lt;/td&gt;
              &lt;td&gt;{{formatCurrency (multiply Price Quantity)}}&lt;/td&gt;
            &lt;/tr&gt;
            {{/each}}
          &lt;/tbody&gt;
        &lt;/table&gt;
      &lt;/div&gt;

      &lt;!-- Totals --&gt;
      &lt;div class=&quot;totals&quot;&gt;
        &lt;div class=&quot;totals-container&quot;&gt;
          &lt;div class=&quot;totals-row subtotal&quot;&gt;
            &lt;span&gt;Subtotal:&lt;/span&gt;
            &lt;span&gt;{{formatCurrency Subtotal}}&lt;/span&gt;
          &lt;/div&gt;
          &lt;div class=&quot;totals-row&quot;&gt;
            &lt;span&gt;Tax:&lt;/span&gt;
            &lt;span&gt;{{formatCurrency 0}}&lt;/span&gt;
          &lt;/div&gt;
          &lt;div class=&quot;totals-row total&quot;&gt;
            &lt;span&gt;Total:&lt;/span&gt;
            &lt;span&gt;{{formatCurrency Total}}&lt;/span&gt;
          &lt;/div&gt;
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
  &lt;/body&gt;
&lt;/html&gt;
</code></pre>
<p>And the rendered PDF looks like this:</p>
<figure className="figure-center">
  <div className="bordered">
    ![Complex PDF Template example rendered as PDF with images, headers, and footers](/blogs/mnw_151/complex_template.png)
  </div>
  <figcaption>
    This is a complex PDF template example with CSS stylization, tables, images,
    headers, and footers.
  </figcaption>
</figure>
<h2>Downloading Binaries at Application Start</h2>
<p>Here's a small tip: PuppeteerSharp requires the Chromium browser binaries to be downloaded at runtime.
You can do this by calling <code>BrowserFetcher.DownloadAsync()</code> before launching the browser.
This ensures the required browser version is available when you run your application.</p>
<p>A simple way to do this is to add it to your application startup code or a background service:</p>
<pre><code class="language-csharp">public class BrowserSetupService : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        var browserFetcher = new BrowserFetcher();
        await browserFetcher.DownloadAsync();
    }
}
</code></pre>
<p>Then register this service in your <code>Program.cs</code>:</p>
<pre><code class="language-csharp">builder.Services.AddHostedService&lt;BrowserSetupService&gt;();
</code></pre>
<h2>Performance Considerations</h2>
<p>Let's talk about performance.
How fast is this approach and can you use it at scale?</p>
<p>The first thing to consider is the use a headless browser like Chromium.
This can be slower than native PDF libraries, especially for large documents or high concurrency.
It also adds <strong>overhead at runtime</strong> since the browser binaries need to be <strong>downloaded</strong> and <strong>launched</strong>.</p>
<p>You should definitely consider moving this out of your main application.
You can use a background service or a separate microservice to handle PDF generation.
Event a cloud function can be a good fit if you need to scale.</p>
<p>But as with anything performance-related, it depends on your specific use case.
So measure and profile your application to see if this approach meets your needs.</p>
<p>Here are some benchmarks I ran on a sample invoice template.
It's nothing scientific, but it gives you an idea of the performance.
I didn't test this with concurrent requests, but rather just the time it takes to generate a single PDF.</p>
<p><strong>Cold Start</strong>: ~12s spent downloading + launching Chromium</p>
<p><strong>Warm Run</strong>: ~580ms</p>
<ul>
<li>Template + HTML generation: ~13ms</li>
<li>Browser reuse + rendering: ~550ms</li>
</ul>
<h2>Summary</h2>
<p>HTML + PuppeteerSharp is one of the most pragmatic approaches for PDF reporting in .NET.</p>
<p>It lets you:</p>
<ul>
<li>Design pixel-perfect layouts using familiar web technologies</li>
<li>Inject dynamic data cleanly with <a href="http://Handlebars.NET">Handlebars.NET</a></li>
<li>Output high-quality PDFs with full styling, tables, and images</li>
</ul>
<p>And all of this without relying on commercial libraries.</p>
<p>I've also written about <a href="how-to-easily-create-pdf-documents-in-aspnetcore"><strong>PDF generation</strong></a> in the past,
with libraries like <a href="https://www.questpdf.com/">QuestPdf</a> or <a href="https://ironpdf.com/">IronPdf</a>.
You can take a look at those if you want to compare approaches.</p>
<p>The cold start can be expensive, but once warmed up, rendering is fast and reliable.
You get total layout control, CSS styling, and even dynamic headers and footers with page numbers and timestamps.</p>
<p>If you're building internal dashboards, invoice generators, or export endpoints, this approach delivers excellent value.</p>
<p>If you need pixel-perfect PDF reports in .NET and want full design control,
combining <a href="http://Handlebars.NET">Handlebars.NET</a> with PuppeteerSharp is a powerful approach.</p>
<p>You'll trade some performance and setup cost for flexibility, but for most internal tools, dashboards, or customer-facing reports, it's worth it.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_151.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Global Error Handling in ASP.NET Core: From Middleware to Modern Handlers]]></title>
            <link>https://milanjovanovic.tech/blog/global-error-handling-in-aspnetcore-from-middleware-to-modern-handlers</link>
            <guid isPermaLink="false">global-error-handling-in-aspnetcore-from-middleware-to-modern-handlers</guid>
            <pubDate>Sat, 12 Jul 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn How to handle errors globally in ASP.NET Core using middleware, IProblemDetailsService, and the new IExceptionHandler in .NET 8. This week's issue walks you through pragmatic approaches to error handling, with real-world tips for validation, chaining, and observability.]]></description>
            <content:encoded><![CDATA[<p>Let's talk about something we all deal with but often put off until the last minute - error handling in our <a href="http://ASP.NET">ASP.NET</a> Core apps.</p>
<p>When something breaks in production, the last thing you want is a cryptic 500 error with zero context.
Proper error handling isn't just about logging exceptions.
It's about making sure your app fails gracefully and gives useful info to the caller (and you).</p>
<p>In this article, I'll walk through the main options for global error handling in <a href="http://ASP.NET">ASP.NET</a> Core.</p>
<p>We'll look at how I used to do it, what <a href="http://ASP.NET">ASP.NET</a> Core 9 offers now, and where each approach makes sense.</p>
<h2>Middleware-Based Error Handling</h2>
<p>The classic way to catch unhandled exceptions is with custom <a href="3-ways-to-create-middleware-in-asp-net-core"><strong>middleware</strong></a>.
This is where most of us start, and honestly, it still works great for most scenarios.</p>
<pre><code class="language-csharp">internal sealed class GlobalExceptionHandlerMiddleware(
    RequestDelegate next,
    ILogger&lt;GlobalExceptionHandlerMiddleware&gt; logger)
{
    public async Task InvokeAsync(HttpContext context)
    {
        try
        {
            await next(context);
        }
        catch (Exception ex)
        {
            logger.LogError(ex, &quot;Unhandled exception occurred&quot;);

            // Make sure to set the status code before writing to the response body
            context.Response.StatusCode = ex switch
            {
                ApplicationException =&gt; StatusCodes.Status400BadRequest,
                _ =&gt; StatusCodes.Status500InternalServerError
            };

            await context.Response.WriteAsJsonAsync(
                new ProblemDetails
                {
                    Type = ex.GetType().Name,
                    Title = &quot;An error occured&quot;,
                    Detail = ex.Message
                });
        }
    }
}
</code></pre>
<p>Don't forget to add the middleware to the request pipeline:</p>
<pre><code class="language-csharp">app.UseMiddleware&lt;GlobalExceptionHandlerMiddleware&gt;();
</code></pre>
<p>This approach is solid and works everywhere in your pipeline.
The beauty is its simplicity: wrap everything in a try-catch, log the error, and return a consistent response.</p>
<p>But once you start adding specific rules for different exception types (e.g. <code>ValidationException</code>, <code>NotFoundException</code>), this becomes a mess.
You end up with long <code>if</code> / <code>else</code> chains or more abstractions to handle each exception type.</p>
<p>Plus, you're manually crafting JSON responses, which means you're probably not following
<a href="https://www.rfc-editor.org/rfc/rfc9457">RFC 9457 (Problem Details)</a> standards.</p>
<h2>Enter IProblemDetailsService</h2>
<p>Microsoft recognized this pain point and gave us <code>IProblemDetailsService</code> to standardize error responses.
Instead of manually serializing our own error objects, we can use the built-in Problem Details format.</p>
<pre><code class="language-csharp">internal sealed class GlobalExceptionHandlerMiddleware(
    RequestDelegate next,
    IProblemDetailsService problemDetailsService,
    ILogger&lt;GlobalExceptionHandlerMiddleware&gt; logger)
{
    public async Task InvokeAsync(HttpContext context)
    {
        try
        {
            await next(context);
        }
        catch (Exception ex)
        {
            logger.LogError(ex, &quot;Unhandled exception occurred&quot;);

            // Make sure to set the status code before writing to the response body
            context.Response.StatusCode = ex switch
            {
                ApplicationException =&gt; StatusCodes.Status400BadRequest,
                _ =&gt; StatusCodes.Status500InternalServerError
            };

            await problemDetailsService.TryWriteAsync(new ProblemDetailsContext
            {
                HttpContext = context,
                Exception = ex,
                ProblemDetails = new ProblemDetails
                {
                    Type = ex.GetType().Name,
                    Title = &quot;An error occured&quot;,
                    Detail = ex.Message
                }
            });
        }
    }
}
</code></pre>
<p>This is much cleaner.
We're now using a standard format that API consumers expect, and we're not manually fiddling with JSON serialization.
But we're still stuck with that growing switch statement problem.
<a href="problem-details-for-aspnetcore-apis"><strong>You can learn more about using Problem Details in .NET here</strong></a>.</p>
<h2>The Modern Way: IExceptionHandler</h2>
<p><a href="http://ASP.NET">ASP.NET</a> Core 8 introduced <code>IExceptionHandler</code>, and it's a game-changer.
Instead of one massive middleware handling everything, we can create focused handlers for specific exception types.</p>
<p>Here's how it works:</p>
<pre><code class="language-csharp">internal sealed class GlobalExceptionHandler(
    IProblemDetailsService problemDetailsService,
    ILogger&lt;GlobalExceptionHandler&gt; logger) : IExceptionHandler
{
    public async ValueTask&lt;bool&gt; TryHandleAsync(
        HttpContext httpContext,
        Exception exception,
        CancellationToken cancellationToken)
    {
        logger.LogError(exception, &quot;Unhandled exception occurred&quot;);

        httpContext.Response.StatusCode = exception switch
        {
            ApplicationException =&gt; StatusCodes.Status400BadRequest,
            _ =&gt; StatusCodes.Status500InternalServerError
        };

        return await problemDetailsService.TryWriteAsync(new ProblemDetailsContext
        {
            HttpContext = httpContext,
            Exception = exception,
            ProblemDetails = new ProblemDetails
            {
                Type = exception.GetType().Name,
                Title = &quot;An error occured&quot;,
                Detail = exception.Message
            }
        });
    }
}
</code></pre>
<p>The key here is the return value.
If your handler can deal with the exception, return <code>true</code>.
If not, return <code>false</code> and let the next handler try.</p>
<p>Don't forget to register it with DI and the request pipeline:</p>
<pre><code class="language-csharp">builder.Services.AddExceptionHandler&lt;GlobalExceptionHandler&gt;();
builder.Services.AddProblemDetails();

// And in your pipeline
app.UseExceptionHandler();
</code></pre>
<p>This approach is so much cleaner.
Each handler has one job, and the code is easy to test and maintain.</p>
<h2>Chaining Exception Handlers</h2>
<p>You can chain multiple <a href="global-error-handling-in-aspnetcore-8"><strong>exception handlers</strong></a> together, and they'll run in the order you register them.
<a href="http://ASP.NET">ASP.NET</a> Core will use the first one that returns <code>true</code> from <code>TryHandleAsync</code>.</p>
<p>Example: One for validation errors, one global fallback.</p>
<pre><code class="language-csharp">builder.Services.AddExceptionHandler&lt;ValidationExceptionHandler&gt;();
builder.Services.AddExceptionHandler&lt;GlobalExceptionHandler&gt;();
</code></pre>
<p>Let's say you're using <a href="https://fluentvalidation.net/">FluentValidation</a> (and you should be).
Here's a complete setup:</p>
<pre><code class="language-csharp">internal sealed class ValidationExceptionHandler(
    IProblemDetailsService problemDetailsService,
    ILogger&lt;ValidationExceptionHandler&gt; logger) : IExceptionHandler
{
    public async ValueTask&lt;bool&gt; TryHandleAsync(
        HttpContext httpContext,
        Exception exception,
        CancellationToken cancellationToken)
    {
        if (exception is not ValidationException validationException)
        {
            return false;
        }

        logger.LogError(exception, &quot;Unhandled exception occurred&quot;);

        httpContext.Response.StatusCode = StatusCodes.Status400BadRequest;
        var context = new ProblemDetailsContext
        {
            HttpContext = httpContext,
            Exception = exception,
            ProblemDetails = new ProblemDetails
            {
                Detail = &quot;One or more validation errors occurred&quot;,
                Status = StatusCodes.Status400BadRequest
            }
        };

        var errors = validationException.Errors
            .GroupBy(e =&gt; e.PropertyName)
            .ToDictionary(
                g =&gt; g.Key.ToLowerInvariant(),
                g =&gt; g.Select(e =&gt; e.ErrorMessage).ToArray()
            );
        context.ProblemDetails.Extensions.Add(&quot;errors&quot;, errors);

        return await problemDetailsService.TryWriteAsync(context);
    }
}
</code></pre>
<p>And in your app, just throw like this:</p>
<pre><code class="language-csharp">// In your controller or service - IValidator&lt;CreateUserRequest&gt;
public async Task&lt;IActionResult&gt; CreateUser(CreateUserRequest request)
{
    await _validator.ValidateAndThrowAsync(request);

    // Your business logic here
}
</code></pre>
<p>The execution order is important.
The framework will try each handler in the order you registered them.
So put your most specific handlers first, and your catch-all handler last.</p>
<h2>Summary</h2>
<p>We've come a long way from the days of manually crafting error responses in middleware.
The evolution looks like this:</p>
<ul>
<li><a href="3-ways-to-create-middleware-in-asp-net-core"><strong>Middleware</strong></a>: Simple, works everywhere, but gets complex fast</li>
<li><a href="problem-details-for-aspnetcore-apis"><strong>IProblemDetailsService</strong></a>: Standardizes response format, still manageable</li>
<li><a href="global-error-handling-in-aspnetcore-8"><strong>IExceptionHandler</strong></a>: Modern, testable, and scales beautifully</li>
</ul>
<p>For new projects, I'd go straight to <code>IExceptionHandler</code>.
It's cleaner, more maintainable, and gives you the flexibility to handle different exception types exactly how you want.</p>
<p>The key takeaway?
Don't let error handling be an afterthought.
Set it up early, make it consistent, and your users (and your future self) will thank you when things inevitably go wrong.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_150.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Using .NET Aspire With the Docker Publisher]]></title>
            <link>https://milanjovanovic.tech/blog/using-dotnet-aspire-with-the-docker-publisher</link>
            <guid isPermaLink="false">using-dotnet-aspire-with-the-docker-publisher</guid>
            <pubDate>Sat, 05 Jul 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[A practical walkthrough of using .NET Aspire's Docker publisher to generate Docker Compose files from C# code. Learn how to set up an API with Postgres and Redis, publish to Docker Compose, and deploy to a VPS with minimal configuration.]]></description>
            <content:encoded><![CDATA[<p><a href="dotnet-aspire-a-game-changer-for-cloud-native-development"><strong>.NET Aspire</strong></a> is one of the most exciting additions to the .NET ecosystem in years.</p>
<p>It brings a fresh, modern approach to building cloud-native apps, with a focus on developer productivity,
great defaults, and tight integration across your entire stack.</p>
<p>One of the most asked-for features is the ability to publish your app to Docker Compose.
This is now available in the latest preview, and I'm excited to share how it works.</p>
<p>In this post, I'll walk you through how I used Aspire's <strong>Docker publisher</strong> to spin up a demo app that includes an API, a Postgres database, and a Redis cache.
Everything runs using Docker Compose, and Aspire generates the whole thing from C# code.</p>
<p>I'll show you how to set it up, explain what it's doing behind the scenes,
and give you a glimpse into how easy it is to take that setup and run it on a VPS or cloud server.</p>
<h2>The Demo App</h2>
<p>The app is intentionally simple, just enough to demonstrate how Aspire wires things together.</p>
<ul>
<li>API project: a minimal .NET Web API</li>
<li>Postgres: used for data storage</li>
<li>Redis: used for caching</li>
</ul>
<p>In a traditional setup, you'd manually connect these services, manage configuration files, handle environment variables,
and write a <code>docker-compose.yml</code> from scratch.</p>
<p>With Aspire, you declare everything in C# inside the AppHost project.
Here's a quick look at the setup:</p>
<pre><code class="language-csharp">var builder = DistributedApplication.CreateBuilder(args);

// Enables Docker publisher
builder.AddDockerComposeEnvironment(&quot;aspire-docker-demo&quot;);

var postgres = builder.AddPostgres(&quot;database&quot;)
    .WithDataVolume();

var database = postgres.AddDatabase(&quot;demo-db&quot;);

var redis = builder.AddRedis(&quot;cache&quot;);

var webApi = builder.AddProject&lt;Projects.Web_Api&gt;(&quot;web-api&quot;)
    .WithReference(database).WaitFor(postgres)
    .WithReference(redis).WaitFor(redis);

builder.Build().Run();
</code></pre>
<p>This gives you a development environment where Aspire runs Postgres and Redis in containers, and connects your API to them.</p>
<p>The <code>AddDockerComposeEnvironment</code> method enables the Docker publisher.
It's available in the <code>Aspire.Hosting.Docker</code> NuGet package, which is currently in preview.</p>
<pre><code class="language-powershell">Install-Package Aspire.Hosting.Docker -Version 9.3.1-preview.1.25305.6
</code></pre>
<h2>Installing the Aspire CLI</h2>
<p>To publish your app to Docker Compose, install the <strong>Aspire CLI</strong>:</p>
<pre><code class="language-bash">dotnet tool install --global aspire.cli --prerelease
</code></pre>
<p>Then you can run the <code>publish</code> command:</p>
<pre><code class="language-bash">aspire publish -o docker-compose-artifacts
</code></pre>
<p>This command will scan your solution for the Aspire project and generate a Docker Compose file and an <code>.env</code> file
based on the services you've defined.</p>
<div className="centered">
  ![Aspire publish command](/blogs/mnw_149/aspire_publish.png)
</div>
<h2>The Docker Compose File</h2>
<p>Let's examine what Aspire created for us:</p>
<pre><code class="language-yml">services:
  database:
    image: 'docker.io/library/postgres:17.4'
    environment:
      POSTGRES_HOST_AUTH_METHOD: 'scram-sha-256'
      POSTGRES_INITDB_ARGS: '--auth-host=scram-sha-256 --auth-local=scram-sha-256'
      POSTGRES_USER: 'postgres'
      POSTGRES_PASSWORD: '${DATABASE_PASSWORD}'
    ports:
      - '8000:5432'
    volumes:
      - type: 'volume'
        target: '/var/lib/postgresql/data'
        source: 'aspire.apphost-1f0ed76b33-database-data'
        read_only: false
    networks:
      - 'aspire'
  redis:
    image: 'docker.io/library/redis:7.4'
    command:
      - '-c'
      - 'redis-server --requirepass $$REDIS_PASSWORD'
    entrypoint:
      - '/bin/sh'
    environment:
      REDIS_PASSWORD: '${REDIS_PASSWORD}'
    ports:
      - '8001:6379'
    networks:
      - 'aspire'
  web-api:
    image: '${WEB_API_IMAGE}'
    environment:
      OTEL_DOTNET_EXPERIMENTAL_OTLP_EMIT_EXCEPTION_LOG_ATTRIBUTES: 'true'
      OTEL_DOTNET_EXPERIMENTAL_OTLP_EMIT_EVENT_LOG_ATTRIBUTES: 'true'
      OTEL_DOTNET_EXPERIMENTAL_OTLP_RETRY: 'in_memory'
      ASPNETCORE_FORWARDEDHEADERS_ENABLED: 'true'
      HTTP_PORTS: '8002'
      ConnectionStrings__demo-db: 'Host=database;Port=5432;Username=postgres;Password=${DATABASE_PASSWORD};Database=demo-db'
      ConnectionStrings__redis: 'redis:6379,password=${REDIS_PASSWORD}'
    ports:
      - '8003:8002'
      - '8005:8004'
    depends_on:
      database:
        condition: 'service_started'
      redis:
        condition: 'service_started'
    networks:
      - 'aspire'
networks:
  aspire:
    driver: 'bridge'
volumes:
  aspire.apphost-1f0ed76b33-database-data:
    driver: 'local'
</code></pre>
<p>This <code>docker-compose</code> file is generated from the C# code we wrote earlier.
It defines the services, their images, environment variables, ports, and dependencies.</p>
<p>The <code>.env</code> file contains some of the configuration we need:</p>
<pre><code class="language-txt"># Parameter database-password
DATABASE_PASSWORD=&lt;YOUR_STRONG_PASSWORD&gt;

# Container image name for web-api
# Change this to the imaage name in the container registry
WEB_API_IMAGE=web-api:latest

# Parameter redis-password
REDIS_PASSWORD=&lt;YOUR_STRONG_PASSWORD&gt;
</code></pre>
<p>It contains the passwords for the database and Redis, as well as the image name for the API.
You'll need to replace the placeholders with actual values.
For the API image, you can build and tag the image yourself, or use a pre-built one from a registry.</p>
<h2>Publishing and Running on a VPS</h2>
<p>Aspire doesn't deploy the app for you, but it gives you everything you need.</p>
<p>Once the Compose file is ready, deployment to a VPS is straightforward:</p>
<ol>
<li>Copy the artifacts to your server using (<code>scp</code> or <code>git</code>).</li>
<li>SSH into the VPS.</li>
<li>Run <code>docker compose up -d</code> inside the artifact directory.</li>
</ol>
<p>Make sure Docker and Docker Compose are installed on the server.</p>
<p>You can expose ports with a reverse proxy like Nginx or Caddy, and secure it with HTTPS using Let's Encrypt.</p>
<p>I'll cover this in more detail in a future post.</p>
<h2>Wrapping Up</h2>
<p>.NET Aspire with Docker Compose provides a smooth developer experience and a simple way to deploy full-stack apps.</p>
<p>You define everything in C#, test it locally, and publish it with one command.
No need to write Compose files or manage infrastructure manually.
This opens up new possibilities for deploying and managing cloud-native apps.
You're not locked into a specific cloud provider, and you can easily move between environments.</p>
<p>Can I just say how much I love this?
I'm really excited about the future of cloud-native development with .NET and Aspire.</p>
<p>If you liked this article and want to learn how I structure larger apps, check out <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a>.
It walks you through building clean, scalable .NET applications with strong internal boundaries and maintainability in mind.</p>
<p>Thanks for reading and stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_149.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Testcontainers Best Practices for .NET Integration Testing]]></title>
            <link>https://milanjovanovic.tech/blog/testcontainers-best-practices-dotnet-integration-testing</link>
            <guid isPermaLink="false">testcontainers-best-practices-dotnet-integration-testing</guid>
            <pubDate>Sat, 28 Jun 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Integration tests shouldn't rely on external infrastructure—but they also shouldn't mock everything away. In this post, we look at how to use Testcontainers in .NET to spin up real Postgres and Redis instances in your tests, how to manage container lifecycle using IAsyncLifetime, and how to structure your xUnit fixtures for speed and reliability]]></description>
            <content:encoded><![CDATA[<p>Integration tests with Testcontainers are powerful, but they can quickly become a maintenance nightmare if you don't follow the right patterns.</p>
<p>I've seen teams struggle with flaky tests, slow test suites, and configuration headaches that could have been avoided with better practices from the start.</p>
<p>Today, I'll show you the patterns that make <a href="https://testcontainers.com/">Testcontainers</a> tests reliable, fast, and easy to maintain.</p>
<h2>How Testcontainers Changes Integration Testing</h2>
<p>Traditional <a href="testcontainers-integration-testing-using-docker-in-dotnet"><strong>integration tests</strong></a> often rely on shared test databases or in-memory alternatives that don't match production behavior.
You either deal with test pollution between runs or sacrifice realism for speed.</p>
<p>Testcontainers solves this by spinning up real <a href="https://www.docker.com/">Docker</a> containers for your dependencies.
Your tests run against actual PostgreSQL, Redis, or any other service you use in production.
When tests complete, containers are destroyed, giving you a clean slate every time.</p>
<p>The magic happens through Docker's API.
Testcontainers manages the entire lifecycle: pulling images, starting containers, waiting for readiness, and cleanup.
Your test code just needs to know how to connect.</p>
<h2>Prerequisites</h2>
<p>First, make sure you have the necessary packages:</p>
<pre><code class="language-powershell">Install-Package Microsoft.AspNetCore.Mvc.Testing
Install-Package Testcontainers.PostgreSql
Install-Package Testcontainers.Redis
</code></pre>
<p>If you want to learn more about the basic setup, check out my article on
<a href="testcontainers-integration-testing-using-docker-in-dotnet"><strong>integrating testing with Testcontainers</strong></a>.</p>
<h2>Creating Test Containers</h2>
<p>Here's how to set up your containers with proper configuration:</p>
<pre><code class="language-csharp">PostgreSqlContainer _postgresContainer = new PostgreSqlBuilder()
    .WithImage(&quot;postgres:17&quot;)
    .WithDatabase(&quot;devhabit&quot;)
    .WithUsername(&quot;postgres&quot;)
    .WithPassword(&quot;postgres&quot;)
    .Build();

RedisContainer _redisContainer = new RedisBuilder()
    .WithImage(&quot;redis:latest&quot;)
    .Build();
</code></pre>
<p>To start and stop containers cleanly across your test suite, implement <code>IAsyncLifetime</code> in your <code>WebApplicationFactory</code>:</p>
<pre><code class="language-csharp">public sealed class IntegrationTestWebAppFactory : WebApplicationFactory&lt;Program&gt;, IAsyncLifetime
{
    public async Task InitializeAsync()
    {
        await _postgresContainer.StartAsync();
        await _redisContainer.StartAsync();
        // Start other dependencies here
    }

    public async Task DisposeAsync()
    {
        await _postgresContainer.StopAsync();
        await _redisContainer.StopAsync();
    }
}
</code></pre>
<p>This ensures containers are ready before tests run and cleaned up afterward.
This means no leftover Docker state or race conditions.</p>
<p><strong>A tip</strong>: pin your image versions (like <code>postgres:17</code>) to avoid surprises from upstream changes</p>
<p>I learned this the hard way when a minor version update caused my tests to fail unexpectedly.</p>
<h2>Pass Configuration to Your App</h2>
<p>The biggest mistake I see is hardcoding connection strings.
Testcontainers assigns dynamic ports.
Don't hardcode anything.</p>
<p>Instead, inject values via <code>WebApplicationFactory.ConfigureWebHost</code>:</p>
<pre><code class="language-csharp">protected override void ConfigureWebHost(IWebHostBuilder builder)
{
    builder.UseSetting(&quot;ConnectionStrings:Database&quot;, _postgresContainer.GetConnectionString());
    builder.UseSetting(&quot;ConnectionStrings:Redis&quot;, _redisContainer.GetConnectionString());
}
</code></pre>
<p>The key is to use the <code>UseSetting</code> method to pass connection strings dynamically.
It also avoids any race conditions or conflicts with other tests that might run in parallel.</p>
<p>This ensures your tests always connect to the right ports, regardless of what Docker assigns.</p>
<p>There's no need to remove services from the service collection or manually configure them (contrary to what you might find online).
Just set the connection strings, and your application will use them automatically.</p>
<h2>Share Expensive Setup with xUnit Collection Fixtures</h2>
<p>What's a test fixture?
A <strong>fixture</strong> is a shared context for your tests, allowing you to set up expensive resources like databases or message brokers once and reuse them across multiple tests.</p>
<p>This is where most teams get tripped up.
The choice between class and collection fixtures affects both test performance and isolation.</p>
<p><strong>Class Fixture</strong> - One container per test class:</p>
<p>Use class fixtures when tests modify global state or when debugging test interactions becomes difficult.</p>
<pre><code class="language-csharp">public class AddItemToCartTests : IClassFixture&lt;DevHabitWebAppFactory&gt;
{
    private readonly DevHabitWebAppFactory _factory;

    public AddItemToCartTests(DevHabitWebAppFactory factory)
    {
        _factory = factory;
    }

    [Fact]
    public async Task Should_ReturnFailure_WhenNotEnoughQuantity() { ... }
}
</code></pre>
<p><strong>Collection Fixture</strong> - One container shared across multiple test classes:</p>
<p>Use collection fixtures when your tests don't modify shared state or when you can reliably clean up between tests.</p>
<pre><code class="language-csharp">[CollectionDefinition(nameof(IntegrationTestCollection))]
public sealed class IntegrationTestCollection : ICollectionFixture&lt;DevHabitWebAppFactory&gt;
{
}
</code></pre>
<p>Then apply it to your test classes:</p>
<pre><code class="language-csharp">[Collection(nameof(IntegrationTestCollection))]
public class AddItemToCartTests : IntegrationTestFixture
{
    public AddItemToCartTests(DevHabitWebAppFactory factory) : base(factory) { }

    [Fact]
    public async Task Should_ReturnFailure_WhenNotEnoughQuantity()
    {
        Guid customerId = await Sender.CreateCustomerAsync(Guid.NewGuid());
        var command = new AddItemToCartCommand(customerId, ticketTypeId, Quantity + 1);

        Result result = await Sender.Send(command);

        result.Error.Should().Be(TicketTypeErrors.NotEnoughQuantity(Quantity));
    }
}
</code></pre>
<p>When to use which:</p>
<ul>
<li>Class fixtures when you need full isolation between test classes (slower but safer)</li>
<li>Collection fixtures when test classes don't interfere with each other (faster but requires discipline)</li>
</ul>
<p>With collection fixtures, you have to take care of cleaning up any state that might persist between tests.
This could include resetting databases, clearing caches, or removing test data.
If you don't do this, you risk tests affecting each other, leading to flaky results.</p>
<h2>Utility Methods for Auth and Cleanup</h2>
<p>Your fixture can expose helpers to simplify test writing:</p>
<pre><code class="language-csharp">public async Task&lt;HttpClient&gt; CreateAuthenticatedClientAsync() { ... }

protected async Task CleanupDatabaseAsync() { ... }
</code></pre>
<p>These methods can handle authentication setup and database cleanup, so you don't have to repeat boilerplate code in every test.
This lets your test code focus on assertions, not setup.</p>
<h2>Writing Maintainable Integration Tests</h2>
<p>With the infrastructure properly configured, your actual tests should focus on business logic:</p>
<pre><code class="language-csharp">[Fact]
public async Task Should_ReturnFailure_WhenNotEnoughQuantity()
{
    //Arrange
    Guid customerId = await Sender.CreateCustomerAsync(Guid.NewGuid());
    var eventId = Guid.NewGuid();
    var ticketTypeId = Guid.NewGuid();

    await Sender.CreateEventWithTicketTypeAsync(eventId, ticketTypeId, Quantity);

    var command = new AddItemToCartCommand(customerId, ticketTypeId, Quantity + 1);

    //Act
    Result result = await Sender.Send(command);

    //Assert
    result.Error.Should().Be(TicketTypeErrors.NotEnoughQuantity(Quantity));
}
</code></pre>
<p>Notice how the tests focus on business rules rather than infrastructure concerns.
The container complexity is hidden behind well-designed base classes and helper methods.
You're not mocking Postgres or Redis, you're testing real behavior.</p>
<h2>Conclusion</h2>
<p>Testcontainers transforms integration testing by giving you the confidence that comes from testing against real dependencies.
No more wondering if your in-memory database behavior matches production, or dealing with shared test environments that break when someone else runs their tests.</p>
<p>Start simple: pick one integration test that currently uses mocks or in-memory databases, and convert it to use Testcontainers.
You'll immediately notice the difference in confidence when that test passes.
Then gradually expand to cover your critical business flows.</p>
<p>If you want to learn how to structure your applications for testability from day one,
my <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture</strong></a> course covers integration testing with Testcontainers alongside domain modeling,
API design, and the architectural decisions that make applications maintainable over time.</p>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_148.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Monitoring .NET Applications with OpenTelemetry and Grafana]]></title>
            <link>https://milanjovanovic.tech/blog/monitoring-dotnet-applications-with-opentelemetry-and-grafana</link>
            <guid isPermaLink="false">monitoring-dotnet-applications-with-opentelemetry-and-grafana</guid>
            <pubDate>Sat, 21 Jun 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Instrumenting your .NET apps with OpenTelemetry is easy. But what about actually seeing those metrics and traces in action? Here's how to stream observability data from your .NET services into Grafana — in just a few lines of code.]]></description>
            <content:encoded><![CDATA[<p>Your .NET application is running in production, but you're flying blind.</p>
<p>When something breaks, you're stuck digging through logs, guessing at performance bottlenecks,
and trying to piece together what actually happened across your distributed system.</p>
<p>That ends today.</p>
<p><a href="https://grafana.com/">Grafana</a> is a complete observability platform that unifies metrics, logs, and traces in one place.</p>
<p>With Grafana, you get:</p>
<ul>
<li><strong>Unified dashboards</strong> that combine metrics, logs, and traces</li>
<li><strong>Advanced alerting</strong> that actually works when things go wrong</li>
<li><strong>Deep trace analysis</strong> to understand request flows across services</li>
<li><strong>Log correlation</strong> that connects your traces to the exact log entries that matter</li>
</ul>
<p><a href="https://grafana.com/products/cloud/">Grafana Cloud</a> makes this even easier.
No infrastructure to manage, automatic scaling, and built-in integrations with <a href="https://opentelemetry.io/">OpenTelemetry</a>.
There's a generous free tier that allows you to get started without any upfront costs.</p>
<p>When you combine Grafana with OpenTelemetry, you get vendor-neutral observability that actually delivers insights instead of just pretty charts.</p>
<h2>Setting Up OpenTelemetry in .NET</h2>
<p>First, install the core OpenTelemetry packages:</p>
<pre><code class="language-powershell">Install-Package OpenTelemetry.Extensions.Hosting
Install-Package OpenTelemetry.Exporter.OpenTelemetryProtocol
Install-Package OpenTelemetry.Instrumentation.AspNetCore
Install-Package OpenTelemetry.Instrumentation.Http
</code></pre>
<p>You can also add instrumentation for other libraries as needed:</p>
<pre><code class="language-powershell">Install-Package OpenTelemetry.Instrumentation.EntityFrameworkCore
Install-Package OpenTelemetry.Instrumentation.StackExchangeRedis
Install-Package Npgsql.OpenTelemetry
</code></pre>
<p>Configure OpenTelemetry in your <code>Program.cs</code>:</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);

builder.Services
    .AddOpenTelemetry()
    .ConfigureResource(resource =&gt; resource.AddService(serviceName))
    .WithTracing(tracing =&gt;
    {
        tracing
            .AddAspNetCoreInstrumentation()
            .AddHttpClientInstrumentation()
            .AddEntityFrameworkCoreInstrumentation()
            .AddRedisInstrumentation()
            .AddNpgsql();

        tracing.AddOtlpExporter();
    });

builder.Logging.AddOpenTelemetry(logging =&gt;
{
    logging.IncludeScopes = true;
    logging.IncludeFormattedMessage = true;

    logging.AddOtlpExporter();
});

var app = builder.Build();

// Your app configuration...

app.Run();
</code></pre>
<p>This configuration:</p>
<ul>
<li>Sets up <a href="introduction-to-distributed-tracing-with-opentelemetry-in-dotnet"><strong>tracing with OpenTelemetry</strong></a></li>
<li>Sets up automatic instrumentation for <a href="http://ASP.NET">ASP.NET</a> Core and HTTP requests</li>
<li>Adds Entity Framework Core and Redis instrumentation</li>
<li>Configures PostgreSQL instrumentation if you're using Npgsql</li>
<li>Configures OTLP export for traces and logs (we can also add metrics later)</li>
</ul>
<h2>Configuring OTLP Export to Grafana Cloud</h2>
<p>Get your Grafana Cloud credentials:</p>
<ol>
<li>
<p>Log into <a href="https://grafana.com/auth/sign-up/create-user">Grafana Cloud</a></p>
</li>
<li>
<p>Go to <strong>My Account</strong> → <strong>Stack Details</strong></p>
</li>
</ol>
<p><img src="/blogs/mnw_147/grafana_setup.png" alt="Grafana cloud stack details"></p>
<ol start="3">
<li>Find your <strong>OTLP endpoint</strong> (looks like <code>https://otlp-gateway-prod-eu-west-2.grafana.net/otlp</code>)</li>
</ol>
<p><img src="/blogs/mnw_147/grafana_setup_endpoint.png" alt="Grafana cloud endpoint"></p>
<ol start="4">
<li>Generate an <strong>API token</strong> with permissions</li>
</ol>
<p><img src="/blogs/mnw_147/grafana_setup_token.png" alt="Grafana cloud token"></p>
<p>You should also see the environment variables you can use to configure OpenTelemetry:</p>
<p><img src="/blogs/mnw_147/grafana_setup_env_vars.png" alt="Grafana cloud environment variables"></p>
<p>Configure the OTLP exporter in your <code>appsettings.json</code>:</p>
<pre><code class="language-json">{
  &quot;OTEL_EXPORTER_OTLP_ENDPOINT&quot;: &quot;https://otlp-gateway-prod-eu-west-2.grafana.net/otlp&quot;,
  &quot;OTEL_EXPORTER_OTLP_PROTOCOL&quot;: &quot;http/protobuf&quot;,
  &quot;OTEL_EXPORTER_OTLP_HEADERS&quot;: &quot;Authorization=Basic &lt;your-base64-encoded-token&gt;&quot;
}
</code></pre>
<p>You can also set these as environment variables in your hosting environment.</p>
<h2>Viewing and Analyzing Data in Grafana</h2>
<p>Start your application and generate some traffic.
Now head to your Grafana Cloud instance.</p>
<p><strong>Traces</strong></p>
<p>If everything is set up correctly, you should see traces from your application in the <strong>Traces</strong> section.</p>
<p>Here's an example of a trace view in Grafana Cloud.
It's a <code>POST users/register</code> request that contains multiple spans:</p>
<p><img src="/blogs/mnw_147/grafana_trace_1.png" alt="Grafana cloud trace example"></p>
<p>Here's another example of a trace that includes messages sent to a message broker (like RabbitMQ or Kafka).</p>
<p><img src="/blogs/mnw_147/grafana_trace_2.png" alt="Grafana cloud trace example"></p>
<p><strong>Logs</strong></p>
<p>You can also view logs in Grafana Cloud.</p>
<p>Here's an example of a log view that shows multiple log entries for our application.</p>
<p><img src="/blogs/mnw_147/grafana_logs_1.png" alt="Grafana cloud logs example"></p>
<p>You can drill down into individual log entries, filter by severity, and search for specific terms.</p>
<p><img src="/blogs/mnw_147/grafana_logs_2.png" alt="Grafana cloud logs example"></p>
<p>OpenTelemetry automatically correlates your logs with traces.
In the trace detail view, click <strong>Logs</strong> to see all log entries that occurred during that request.</p>
<p>Your logs will include trace and span IDs.</p>
<h2>Conclusion</h2>
<p>You now have full observability for your .NET application.</p>
<p>When something goes wrong in production, you won't be guessing anymore.
You'll see exactly which requests were slow, what errors occurred, and how they propagated through your system.</p>
<p>Grafana gives you the dashboards and alerting to catch problems before users do.
OpenTelemetry gives you the detailed traces to understand exactly what happened.</p>
<p>This setup scales from a single service to hundreds of microservices without changing your instrumentation code.</p>
<p><strong>Ready to take your observability further?</strong></p>
<p>This article showed you the basics, but modern applications need advanced patterns like distributed tracing across event-driven architectures,
custom instrumentation strategies, and observability-driven development practices.</p>
<p>I cover all of this in-depth in my <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a> course,
where you'll learn how to implement OpenTelemetry across complex, event-driven systems that actually scale in production.</p>
<p>That's all for today.</p>
<p>See you next Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_147.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Run C# Scripts With dotnet run app.cs (No Project Files Needed)]]></title>
            <link>https://milanjovanovic.tech/blog/run-csharp-scripts-with-dotnet-run-app-no-project-files-needed</link>
            <guid isPermaLink="false">run-csharp-scripts-with-dotnet-run-app-no-project-files-needed</guid>
            <pubDate>Sat, 14 Jun 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[With .NET 10, you can now run C# files directly. No project files, no Main method, just code. In this issue, I'll show you how to use the new `dotnet run app.cs` feature in practice: from quick one-liners to a real-world SQL seeding script.]]></description>
            <content:encoded><![CDATA[<p>.NET 10 just got a whole lot more lightweight.</p>
<p>You can now run a C# file directly with:</p>
<pre><code class="language-bash">dotnet run app.cs
</code></pre>
<p>That's it.
No <code>.csproj</code>.
No <code>Program.cs</code>.
No solution files.
Just a single C# file.</p>
<p>This new feature, introduced in <a href="https://devblogs.microsoft.com/dotnet/announcing-dotnet-run-app/">.NET 10 Preview 4</a>,
is a big step toward making C# more script-friendly, especially for quick utilities, dev tooling, and CLI-based workflows.</p>
<h2>Why This Matters</h2>
<p>For years, C# has been perceived as heavyweight for small scripts.
Compare that to Python, Bash, or even JavaScript, where you can just write a file and run it.</p>
<p>That barrier is now gone.</p>
<p>You can now:</p>
<ul>
<li>Write one-off scripts in <code>.cs</code> files</li>
<li>Use top-level statements</li>
<li>Reference NuGet packages inline</li>
<li>Share minimal reproducible examples without scaffolding a project</li>
</ul>
<p>And it runs on <strong>any OS</strong> with the .NET SDK installed.</p>
<h2>Minimal Example</h2>
<p>Here's a simple script that prints today's date:</p>
<pre><code class="language-csharp">Console.WriteLine($&quot;Today is {DateTime.Now:dddd, MMM dd yyyy}&quot;);
</code></pre>
<p>Run it:</p>
<pre><code class="language-bash">dotnet run app.cs
</code></pre>
<p>Output:</p>
<pre><code>Today is Saturday, Jun 14 2025
</code></pre>
<p>That's it.
No boilerplate, no boring <code>Main()</code> method.
Just top-level programs and C# code.</p>
<h2>Referencing NuGet Packages</h2>
<p>Let's say you want to make an HTTP request using <code>Flurl.Http</code>.</p>
<p>You can do this inline:</p>
<pre><code class="language-csharp">#:package Flurl.Http@4.0.2

using Flurl.Http;

var response = await &quot;https://api.github.com&quot;
    .WithHeader(&quot;Accept&quot;, &quot;application/vnd.github.v3+json&quot;)
    .WithHeader(&quot;User-Agent&quot;, &quot;dotnet-script&quot;)
    .GetAsync();

Console.WriteLine($&quot;Status code: {response.StatusCode}&quot;);

Console.WriteLine(await response.GetJsonAsync&lt;object&gt;());
</code></pre>
<p>To run it:</p>
<pre><code class="language-bash">dotnet run fetch.cs
</code></pre>
<p>Behind the scenes, the compiler downloads and restores NuGet dependencies automatically.</p>
<h2>Real-World Use Case: Seeding SQL Data</h2>
<p>Here's a script I recently used to seed some test data into my Postgres database.</p>
<pre><code class="language-csharp">#:package Dapper@2.1.66
#:package Npgsql@9.0.3

using Dapper;
using Npgsql;

const string connectionString = &quot;Host=localhost;Port=5432;Username=postgres;Password=postgres&quot;;

using var connection = new NpgsqlConnection(connectionString);
await connection.OpenAsync();

using var transaction = connection.BeginTransaction();

Console.WriteLine(&quot;Creating tables...&quot;);

await connection.ExecuteAsync(@&quot;
    CREATE TABLE IF NOT EXISTS users (
        id SERIAL PRIMARY KEY,
        name TEXT NOT NULL
    );
&quot;);

Console.WriteLine(&quot;Inserting users...&quot;);

for (int i = 1; i &lt;= 10_000; i++)
{
    await connection.ExecuteAsync(
        &quot;INSERT INTO users (name) VALUES (@Name);&quot;,
        new { Name = $&quot;User {i}&quot; });

    if (i % 1000 == 0)
    {
        Console.WriteLine($&quot;Inserted {i} users...&quot;);
    }
}

transaction.Commit();

Console.WriteLine(&quot;Done!&quot;);
</code></pre>
<p>Why did I write this as a script?
I didn't want to clutter my app with throwaway seed logic.
I just needed a quick way to populate my database with test data.
This script does exactly that, and I can run it with:</p>
<pre><code class="language-bash">dotnet run seed.cs
</code></pre>
<h2>File-Level Directives: The Magic Behind It</h2>
<p>The real power comes from file-level directives.
These let you configure your app without leaving the .cs file:</p>
<p><strong>Package References</strong></p>
<pre><code class="language-csharp">#:package Dapper@2.1.66
#:package Npgsql@9.0.3
</code></pre>
<p><strong>SDK Selection</strong></p>
<pre><code class="language-csharp">#:sdk Microsoft.NET.Sdk.Web
</code></pre>
<p>This tells .NET to treat your file as a web application, enabling <a href="http://ASP.NET">ASP.NET</a> Core features:</p>
<pre><code class="language-csharp">#:sdk Microsoft.NET.Sdk.Web
#:package Microsoft.AspNetCore.OpenApi@9.*

var builder = WebApplication.CreateBuilder();

builder.Services.AddOpenApi();

var app = builder.Build();

app.MapOpenApi();

app.MapGet(&quot;/&quot;, () =&gt; &quot;Hello from a file-based API!&quot;);
app.MapGet(&quot;/users/{id}&quot;, (int id) =&gt; new { Id = id, Name = $&quot;User {id}&quot; });

app.Run();
</code></pre>
<p>You now have a running web API.
No project file.
No <code>Startup.cs</code>.
Just C# that does what you want.</p>
<p><strong>MSBuild Properties</strong></p>
<p>You can also set MSBuild properties directly in the file:</p>
<pre><code class="language-csharp">#:property LangVersion preview
#:property Nullable enable
</code></pre>
<h2>When Your Script Grows Up</h2>
<p>The brilliant part? When your file-based app gets complex enough to need project structure, converting is seamless:</p>
<pre><code class="language-bash">dotnet project convert api.cs
</code></pre>
<p>This creates:</p>
<ul>
<li>A new folder named after your file</li>
<li>A proper <code>.csproj</code> file with all your directives converted to MSBuild properties</li>
<li>Your code moved to <code>api.cs</code> (or <code>Program.cs</code> if you prefer)</li>
<li>Everything ready for full project development</li>
</ul>
<p>Given our API example above, the generated <code>.csproj</code> looks like:</p>
<pre><code class="language-xml">&lt;Project Sdk=&quot;Microsoft.NET.Sdk.Web&quot;&gt;
  &lt;PropertyGroup&gt;
    &lt;TargetFramework&gt;net10.0&lt;/TargetFramework&gt;
    &lt;ImplicitUsings&gt;enable&lt;/ImplicitUsings&gt;
    &lt;Nullable&gt;enable&lt;/Nullable&gt;
  &lt;/PropertyGroup&gt;

  &lt;ItemGroup&gt;
    &lt;PackageReference Include=&quot;Microsoft.AspNetCore.OpenApi&quot; Version=&quot;9.*&quot; /&gt;
  &lt;/ItemGroup&gt;
&lt;/Project&gt;
</code></pre>
<p>Your file-based app evolves naturally into a project-based app.
No need to rewrite or restructure everything.
This makes it easy to start small and grow as needed, without losing the simplicity of the initial script.</p>
<h2>Takeaway</h2>
<p>The bottom line is this: C# just became significantly more approachable.
The barrier to entry dropped from &quot;learn project files and MSBuild&quot; to &quot;write C# and run it.&quot;</p>
<p>For experienced developers, this is a productivity boost for scripting and prototyping.
For newcomers, this removes the biggest stumbling block to getting started with C#.</p>
<p>The best part?
Microsoft didn't create a separate scripting language or runtime.
They made regular C# easier to use.
Your file-based apps are real .NET applications that can grow into full projects when needed.</p>
<p>The ceremony is dead. Long live practical C#.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_146.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Debunking the "Filter Early, JOIN Later" SQL Performance Myth]]></title>
            <link>https://milanjovanovic.tech/blog/debunking-the-filter-early-join-later-sql-performance-myth</link>
            <guid isPermaLink="false">debunking-the-filter-early-join-later-sql-performance-myth</guid>
            <pubDate>Sat, 07 Jun 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[That viral SQL performance tip about filtering before joining? It is complete nonsense. Here is why query optimizers make it irrelevant.]]></description>
            <content:encoded><![CDATA[<p>I came across a Medium article with 700+ claps promoting this &quot;SQL performance trick&quot;:</p>
<p><strong>&quot;Filter Early, JOIN Later&quot;</strong></p>
<figure className="figure-center">
  <div className="bordered">
    ![SQL performance tip that doesn](/blogs/mnw_145/sql_performance_tip.png)
  </div>
  <figcaption>Source: SQL Tricks that Cut My Query Time by 80%</figcaption>
</figure>
<p>The claim goes like this: instead of joining tables first and then filtering, you should filter in a subquery first, then join.</p>
<p>The supposed benefit?</p>
<blockquote>
<p>The database filtered the smaller table first, then did the JOIN — saving time and memory.</p>
</blockquote>
<p>Here is the thing - this advice is <strong>completely wrong</strong> for modern databases.</p>
<p>Let me show you why with actual data.</p>
<h2>The Supposed &quot;Optimization&quot;</h2>
<p>The article shows two queries. Here is the &quot;bad&quot; version:</p>
<pre><code class="language-sql">SELECT *
FROM users u
JOIN orders o ON u.id = o.user_id
WHERE o.total &gt; 500;
</code></pre>
<p>And the &quot;optimized&quot; version:</p>
<pre><code class="language-sql">SELECT *
FROM (
  SELECT * FROM orders WHERE total &gt; 500
) o
JOIN users u ON u.id = o.user_id;
</code></pre>
<p>The claim is that the second query is faster because it &quot;filters first, then joins.&quot;</p>
<p>Sounds logical, right?
<strong>Wrong.</strong></p>
<h2>Testing with Real Data</h2>
<p>I tested both queries on a PostgreSQL database with:</p>
<ul>
<li>10,000 users</li>
<li>5,000,000 orders (500 per user)</li>
<li>Filtering for orders &gt; $500</li>
</ul>
<p>Let me run <code>EXPLAIN ANALYZE</code> on both queries to see what actually happens.</p>
<h2>The Results</h2>
<p>Here are the execution plans for both queries:</p>
<p><strong>&quot;Bad&quot; Query Execution Plan:</strong></p>
<pre><code>Hash Join  (cost=280.00..96321.92 rows=2480444 width=27) (actual time=1.014..641.202 rows=2499245 loops=1)
  Hash Cond: (o.user_id = u.id)
  -&gt;  Seq Scan on orders o  (cost=0.00..89528.00 rows=2480444 width=14) (actual time=0.006..368.857 rows=2499245 loops=1)
        Filter: (total &gt; '500'::numeric)
        Rows Removed by Filter: 2500755
  -&gt;  Hash  (cost=155.00..155.00 rows=10000 width=13) (actual time=0.998..0.999 rows=10000 loops=1)
        Buckets: 16384  Batches: 1  Memory Usage: 577kB
        -&gt;  Seq Scan on users u  (cost=0.00..155.00 rows=10000 width=13) (actual time=0.002..0.341 rows=10000 loops=1)
Planning Time: 0.121 ms
Execution Time: 685.818 ms
</code></pre>
<p><strong>&quot;Optimized&quot; Query Execution Plan:</strong></p>
<pre><code>Hash Join  (cost=280.00..96321.92 rows=2480444 width=27) (actual time=1.019..640.613 rows=2499245 loops=1)
  Hash Cond: (orders.user_id = u.id)
  -&gt;  Seq Scan on orders  (cost=0.00..89528.00 rows=2480444 width=14) (actual time=0.005..368.260 rows=2499245 loops=1)
        Filter: (total &gt; '500'::numeric)
        Rows Removed by Filter: 2500755
  -&gt;  Hash  (cost=155.00..155.00 rows=10000 width=13) (actual time=1.004..1.005 rows=10000 loops=1)
        Buckets: 16384  Batches: 1  Memory Usage: 577kB
        -&gt;  Seq Scan on users u  (cost=0.00..155.00 rows=10000 width=13) (actual time=0.003..0.348 rows=10000 loops=1)
Planning Time: 0.118 ms
Execution Time: 685.275 ms
</code></pre>
<p><strong>The execution plans are identical.</strong></p>
<p>Both queries took ~685ms.
The &quot;optimization&quot; did absolutely nothing.</p>
<p>Here's the simplified execution plan, where I removed some details:</p>
<pre><code>Hash Join
  Hash Cond: (o.user_id = u.id)
  -&gt;  Seq Scan on orders o
        Filter: (total &gt; '500'::numeric)
        Rows Removed by Filter
  -&gt;  Hash
        -&gt;  Seq Scan on users u
</code></pre>
<p>The core operations are:</p>
<ol>
<li>Sequential Scan on <code>orders</code> table with filter applied</li>
<li>Sequential Scan on <code>users</code> table</li>
<li>Hash operation to build hash table from <code>users</code></li>
<li>Hash Join using the hash condition on <code>user_id</code></li>
</ol>
<h2>Query Optimizers Are Smarter Than You</h2>
<p>Modern databases use <strong>cost-based optimizers</strong>.
Here is what happens when you run a query:</p>
<ol>
<li><strong>Parser</strong> turns your SQL into an abstract syntax tree</li>
<li><strong>Optimizer</strong> rewrites your query into the most efficient form</li>
<li><strong>Executor</strong> runs the optimized plan</li>
</ol>
<p>The optimizer looks at your query and says:
&quot;I don't care how you wrote this. I will figure out the best way to execute it.&quot;</p>
<p>Both of our queries get rewritten to the same optimal plan:</p>
<ul>
<li>Filter the orders table first (because that eliminates rows early)</li>
<li>Build a hash table from users (the smaller table)</li>
<li>Hash join the filtered orders with users</li>
</ul>
<p><strong>The optimizer already does the &quot;optimization&quot; automatically.</strong></p>
<p>Your manual subquery does not make it faster - it just makes your SQL harder to read.</p>
<h2>How Cost-Based Optimization Works</h2>
<p>The query optimizer has statistics about your tables:</p>
<ul>
<li>Row counts</li>
<li>Data distribution</li>
<li>Index availability</li>
<li>Column selectivity</li>
</ul>
<p>It uses these stats to estimate the cost of different execution strategies:</p>
<ul>
<li>Which table to scan first</li>
<li>Which join algorithm to use (hash, nested loop, merge)</li>
<li>When to apply filters</li>
<li>Which indexes to use</li>
</ul>
<p>Then it picks the cheapest plan.
Your well-intentioned manual &quot;optimization&quot; gets ignored because the optimizer knows better.</p>
<h2>Summary</h2>
<p>The &quot;Filter Early, JOIN Later&quot; advice is a relic from ancient database systems that did not have sophisticated optimizers.</p>
<p>Modern databases like PostgreSQL, MySQL, and SQL Server already do predicate pushdown and join reordering automatically.
Your manual &quot;optimizations&quot; are pointless and make code harder to maintain.</p>
<p>Write clear, readable SQL.
Let the optimizer do its job.</p>
<p><strong>The real lesson?</strong>
Stop believing every performance tip you read online.
Use <code>EXPLAIN ANALYZE</code> to understand what your database is actually doing.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_145.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[YARP vs Nginx - A Quick Performance Comparison]]></title>
            <link>https://milanjovanovic.tech/blog/yarp-vs-nginx-a-quick-performance-comparison</link>
            <guid isPermaLink="false">yarp-vs-nginx-a-quick-performance-comparison</guid>
            <pubDate>Sat, 31 May 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[In this article, we will compare the performance of YARP and Nginx, two popular reverse proxy solutions.]]></description>
            <content:encoded><![CDATA[<p>When you're building .NET applications, choosing the right <strong>reverse proxy</strong> can make a huge difference.
Two popular options keep coming up: Microsoft's <a href="implementing-an-api-gateway-for-microservices-with-yarp"><strong>YARP</strong></a> (Yet Another Reverse Proxy)
and the tried-and-true <a href="https://nginx.org/"><strong>Nginx</strong></a>.</p>
<p>Here's the thing - everyone talks about which one is &quot;better,&quot; but rarely do you see actual numbers.
So I decided to put both through the same tests and see what happens.</p>
<p>I'll test both proxies using the exact same API, same hardware, and same load testing approach.
No bias, just data.</p>
<h2>The Test API</h2>
<p>I kept the test API super simple on purpose.
This way, we're measuring proxy performance, not how fast the backend can process complex requests:</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

app.MapGet(&quot;/hello&quot;, () =&gt;
{
    return Results.Ok(&quot;Hello world!&quot;);
});

app.Run();
</code></pre>
<p>This basic endpoint means we're testing the proxy itself, not waiting for complex business logic to run.</p>
<h2>YARP Configuration</h2>
<p>YARP is pretty nice to work with if you're already in the .NET world. The setup is straightforward:</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);

builder.Services
    .AddReverseProxy()
    .LoadFromConfig(builder.Configuration.GetSection(&quot;ReverseProxy&quot;));

var app = builder.Build();
app.MapReverseProxy();
app.Run();
</code></pre>
<p>It's equally simple to configure <a href="horizontally-scaling-aspnetcore-apis-with-yarp-load-balancing"><strong>load balancing</strong></a>
or <a href="implementing-api-gateway-authentication-with-yarp"><strong>authentication</strong></a>.</p>
<p>The routing setup is clean and uses a <code>**catch-all</code> pattern to forward everything:</p>
<pre><code class="language-json">{
  &quot;ReverseProxy&quot;: {
    &quot;Routes&quot;: {
      &quot;default&quot;: {
        &quot;ClusterId&quot;: &quot;hello&quot;,
        &quot;Match&quot;: { &quot;Path&quot;: &quot;{**catch-all}&quot; }
      }
    },
    &quot;Clusters&quot;: {
      &quot;hello&quot;: {
        &quot;Destinations&quot;: {
          &quot;destination1&quot;: {
            &quot;Address&quot;: &quot;http://hello.api:8080&quot;
          }
        }
      }
    }
  }
}
</code></pre>
<h2>Nginx Setup</h2>
<p>For Nginx, I went with Docker to keep things simple.
The configuration does the same job as YARP:</p>
<pre><code class="language-yml">nginx.proxy:
  image: nginx:alpine
  ports:
    - '3001:80'
  volumes:
    - ./nginx-proxy.conf:/etc/nginx/nginx.conf:ro
  depends_on:
    - hello.api
</code></pre>
<p>The Nginx config does exactly what YARP does - just with different syntax:</p>
<pre><code class="language-nginx">events {}

http {
    upstream backend {
        server hello.api:8080;
    }

    server {
        listen 80;
        location / {
            proxy_pass http://backend;
        }
    }
}
</code></pre>
<h2>Full Docker Compose Setup</h2>
<p>Here's the complete <code>docker-compose.yml</code> that ties everything together:</p>
<pre><code class="language-yml">services:
  hello.api:
    image: ${DOCKER_REGISTRY-}helloapi
    build:
      context: .
      dockerfile: Hello.Api/Dockerfile

  yarp.proxy:
    image: ${DOCKER_REGISTRY-}yarpproxy
    build:
      context: .
      dockerfile: Yarp.Proxy/Dockerfile
    ports:
      - 3000:8080
    depends_on:
      - hello.api

  nginx.proxy:
    image: nginx:alpine
    ports:
      - '3001:80'
    volumes:
      - ./nginx-proxy.conf:/etc/nginx/nginx.conf:ro
    depends_on:
      - hello.api
</code></pre>
<h2>Load Testing with k6</h2>
<p>I used k6 to hit both proxies with the same load patterns.
I repeated the test with different numbers of virtual users (VUs) to see how each proxy handles increasing traffic.
This keeps things fair - same test, same conditions:</p>
<p><strong>YARP Test Script</strong>:</p>
<pre><code class="language-js">import http from 'k6/http';
import { check } from 'k6';

export let options = {
  scenarios: {
    yarp: {
      executor: 'per-vu-iterations',
      vus: 200, // 10, 50, 100, 200
      iterations: 1000,
      exec: 'testYarp',
      startTime: '0s'
    }
  }
};

export function testYarp() {
  let res = http.get('http://localhost:3000/hello');
  check(res, {
    'YARP: status 200': (r) =&gt; r.status === 200
  });
}
</code></pre>
<p><strong>Nginx Test Script</strong>:</p>
<pre><code class="language-js">import http from 'k6/http';
import { check } from 'k6';

export let options = {
  scenarios: {
    nginx: {
      executor: 'per-vu-iterations',
      vus: 200, // 10, 50, 100, 200
      iterations: 1000,
      exec: 'testNginx',
      startTime: '0s'
    }
  }
};

export function testNginx() {
  let res = http.get('http://localhost:3001/hello');
  check(res, {
    'NGINX: status 200': (r) =&gt; r.status === 200
  });
}
</code></pre>
<h2>Performance Results</h2>
<p>Here's where things get interesting.
The numbers show a clear pattern:</p>
<pre><code>| VUs  | YARP RPS | NGINX RPS | YARP p90 Latency (ms) | NGINX p90 Latency (ms) | YARP p95 Latency (ms) | NGINX p95 Latency (ms) |
|------|----------|-----------|-----------------------|------------------------|-----------------------|------------------------|
| 10   | 12692    | 9756      | 1.04                  | 1.10                   | 1.06                  | 1.52                   |
| 50   | 27080    | 10614     | 2.70                  | 5.23                   | 3.18                  | 5.68                   |
| 100  | 32432    | 10324     | 4.66                  | 10.61                  | 5.43                  | 10.96                  |
| 200  | 36662    | 10169     | 7.77                  | 21.23                  | 8.81                  | 21.92                  |
</code></pre>
<p><strong>Request per second (RPS)</strong> is how many requests each proxy handled per second.</p>
<p><img src="/blogs/mnw_144/rps_comparison.png" alt="YARP vs Nginx RPS comparison"></p>
<p><strong>p90 latency</strong> is the time it took for 90% of requests to complete.</p>
<p><img src="/blogs/mnw_144/p90_comparison.png" alt="YARP vs Nginx p90 latency comparison"></p>
<p><strong>p95 latency</strong> is the time it took for 95% of requests to complete.</p>
<p><img src="/blogs/mnw_144/p95_comparison.png" alt="YARP vs Nginx p95 latency comparison"></p>
<h3>Throughput Analysis</h3>
<p>YARP really shines here.
It handles way more requests - almost 3.6x more at 200 users.
What's cool is how it scales up as you add more load.
Nginx stays pretty much flat around 10k requests per second, but YARP keeps climbing from 12k all the way to 36k.</p>
<h3>Latency Comparison</h3>
<p>The latency story is even more impressive for YARP.
At 200 users, YARP keeps response times under 8ms while Nginx hits 21ms.
That's a big difference when you're trying to keep your app fast.</p>
<h2>Hold Up - That's Not Fair</h2>
<p>Looking at these results, we're missing something important: <strong>this comparison isn't fair to Nginx</strong>.</p>
<p>The default Nginx configuration I used is fine for basic setups, but it's not optimized for high-throughput scenarios.
Nginx uses conservative defaults that work everywhere but don't push performance limits.</p>
<p>So let me fix the Nginx configuration and re-run the tests.</p>
<p>Here's the updated Nginx config with some tweaks to improve performance:</p>
<pre><code class="language-nginx">worker_processes auto;

events {
    worker_connections 65536;
    multi_accept on;
    use epoll;
}

http {
    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 30;
    keepalive_requests 1000;
    types_hash_max_size 4096;

    upstream backend {
        server hello.api:8080;
        keepalive 512;
    }

    server {
        listen 80;

        location / {
            proxy_pass http://backend;
            proxy_http_version 1.1;
            proxy_set_header Connection &quot;&quot;;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    }
}
</code></pre>
<h2>Performance Results - After Tuning</h2>
<p>Here are the results after re-running the tests:</p>
<pre><code>| VUs  | YARP RPS | NGINX RPS | YARP p90 Latency (ms) | NGINX p90 Latency (ms) | YARP p95 Latency (ms) | NGINX p95 Latency (ms) |
|------|----------|-----------|-----------------------|------------------------|-----------------------|------------------------|
| 10   | 12692    | 17572     | 1.04                  | 0.58                   | 1.06                  | 0.74                   |
| 50   | 27080    | 36687     | 2.70                  | 1.81                   | 3.18                  | 2.09                   |
| 100  | 32432    | 43289     | 4.66                  | 3.18                   | 5.43                  | 3.88                   |
| 200  | 36662    | 46850     | 7.77                  | 6.34                   | 8.81                  | 7.72                   |
</code></pre>
<p><img src="/blogs/mnw_144/rps_comparison_tuned.png" alt="YARP vs Nginx RPS comparison"></p>
<p><img src="/blogs/mnw_144/p90_comparison_tuned.png" alt="YARP vs Nginx p90 latency comparison"></p>
<p><img src="/blogs/mnw_144/p95_comparison_tuned.png" alt="YARP vs Nginx p95 latency comparison"></p>
<h3>Throughput Analysis</h3>
<p>Now this is more interesting.
Nginx actually edges out YARP in raw throughput - hitting 46k requests per second vs YARP's 36k at 200 users.
Both proxies scale well as load increases, but Nginx shows why it's been the go-to choice for high-traffic sites.</p>
<h3>Latency Comparison</h3>
<p>The latency story is pretty close.
At lower loads, Nginx actually has better response times.
At 200 users, both proxies keep response times reasonable - YARP at 7.77ms and Nginx at 6.34ms for p90 latency.
The difference isn't huge either way.</p>
<h2>Key Takeaways</h2>
<p><strong>Configuration matters more than you think</strong>.
The initial results showed YARP crushing Nginx, but that was with Nginx's conservative defaults.
Once properly tuned, Nginx shows why it's been powering the internet for years.</p>
<p><strong>Nginx wins on raw performance</strong>.
With proper configuration, Nginx handles more requests and keeps latency slightly lower.
That extra throughput matters when you're dealing with serious traffic.</p>
<p><strong>YARP offers better integration</strong>.
Even though Nginx edges out performance, YARP feels natural in .NET projects.
Same configuration style, same patterns, same tooling.
Sometimes that developer experience is worth more than a few extra requests per second.</p>
<p><strong>Always tune your tools</strong>.
This whole exercise shows why benchmarks with default configs can be misleading.
If you're choosing between these two, make sure you're comparing optimized configurations, not defaults.</p>
<p>The choice isn't as clear-cut as I initially thought.
Nginx wins on pure performance, but YARP wins on .NET integration.
Pick based on what matters more for your specific situation.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_144.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Building a Custom Domain Events Dispatcher in .NET]]></title>
            <link>https://milanjovanovic.tech/blog/building-a-custom-domain-events-dispatcher-in-dotnet</link>
            <guid isPermaLink="false">building-a-custom-domain-events-dispatcher-in-dotnet</guid>
            <pubDate>Sat, 24 May 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to build a lightweight, in-process domain events dispatcher in .NET without external dependencies. We'll explore the trade-offs between immediate consistency and coupling while implementing a strongly-typed solution from scratch.]]></description>
            <content:encoded><![CDATA[<p>Domain events are a powerful way to decouple parts of your system.
Instead of tightly coupling your logic, you can publish events and have other parts of your code subscribe to those events.
This pattern is especially valuable in <a href="https://en.wikipedia.org/wiki/Domain-driven_design">Domain-Driven Design</a> (DDD)
where business logic should remain focused and cohesive.</p>
<p>In this article, we'll walk through how to implement a lightweight, custom domain event dispatcher in .NET.
The core dispatching logic should not depend on third-party libraries.</p>
<p>We'll cover:</p>
<ul>
<li>Why you might want to use publish-subscribe in your application</li>
<li>How to define basic domain event abstractions</li>
<li>How to implement and register handlers</li>
<li>How to build a domain events dispatcher</li>
<li>Trade-offs and when to consider other options</li>
</ul>
<p>Let's get started.</p>
<h2>Why Domain Events Matter</h2>
<p>Before diving into implementation, let's understand the problem domain events solve.
Consider this tightly coupled code:</p>
<pre><code class="language-csharp">public class UserService
{
    public async Task RegisterUser(string email, string password)
    {
        var user = new User(email, password);
        await _userRepository.SaveAsync(user);

        // Directly coupled to email service
        await _emailService.SendWelcomeEmail(user.Email);

        // Directly coupled to analytics
        await _analyticsService.TrackUserRegistration(user.Id);

        // What if we need to add more features?
        // This method will keep growing...
    }
}
</code></pre>
<p>With domain events, we can decouple this:</p>
<pre><code class="language-csharp">public class UserService
{
    public async Task RegisterUser(string email, string password)
    {
        var user = new User(email, password);
        await _userRepository.SaveAsync(user);

        // Publish event - let other parts of the system react
        await _domainEventsDispatcher.DispatchAsync(
            [new UserRegisteredDomainEvent(user.Id, user.Email)]);
    }
}
</code></pre>
<p>Now the <code>UserService</code> focuses solely on user registration, while other concerns are handled through event handlers.</p>
<h2>Basic Abstractions</h2>
<p>Let's start by defining two simple interfaces that form the foundation of our event system:</p>
<pre><code class="language-csharp">// Marker interface for all domain events.
public interface IDomainEvent
{
    // We could add common properties here like:
    // DateTime OccurredAt { get; }
    // Guid EventId { get; }
}

// Generic interface for handling domain events.
public interface IDomainEventHandler&lt;in T&gt; where T : IDomainEvent
{
    Task Handle(T domainEvent, CancellationToken cancellationToken = default);
}
</code></pre>
<p>This design gives us type safety through generic constraints while keeping publishers and handlers completely decoupled.
You can add new events or handlers without touching existing code, and everything remains easily testable in isolation.</p>
<h2>Implementing Sample Handlers</h2>
<p>Let's add some sample handlers that demonstrate how different parts of your system can react to the same event:</p>
<pre><code class="language-csharp">// Handles sending welcome emails when users register
internal sealed class SendWelcomeEmailHandler(IEmailService emailService)
    : IDomainEventHandler&lt;UserRegisteredDomainEvent&gt;
{
    public async Task Handle(
        UserRegisteredDomainEvent domainEvent,
        CancellationToken cancellationToken = default)
    {
        // Send welcome email
        var welcomeEmail = new WelcomeEmail(domainEvent.Email, domainEvent.UserId);

        await emailService.SendAsync(welcomeEmail, cancellationToken);
    }
}

// Handles analytics tracking for new user registrations
internal sealed class TrackUserRegistrationHandler(IAnalyticsService analyticsService)
    : IDomainEventHandler&lt;UserRegisteredDomainEvent&gt;
{
    public async Task Handle(
        UserRegisteredDomainEvent domainEvent,
        CancellationToken cancellationToken = default)
    {
        // Track registration in analytics
        await analyticsService.TrackEvent(
            &quot;user_registered&quot;,
            new
            {
                user_id = domainEvent.UserId,
                registration_date = domainEvent.RegisteredAt
            },
            cancellationToken);
    }
}
</code></pre>
<p>To make this work, we need to register our handlers with the DI container.</p>
<p>Here's how to do it manually:</p>
<pre><code class="language-csharp">// In your Program.cs or Startup.cs
services.AddScoped&lt;IDomainEventHandler&lt;UserRegisteredDomainEvent&gt;, SendWelcomeEmailHandler&gt;();
services.AddScoped&lt;IDomainEventHandler&lt;UserRegisteredDomainEvent&gt;, TrackUserRegistrationHandler&gt;();
</code></pre>
<p>Or you can automate this registration using <a href="improving-aspnetcore-dependency-injection-with-scrutor"><strong>assembly scanning with Scrutor</strong></a>:</p>
<pre><code class="language-csharp">services.Scan(scan =&gt; scan.FromAssembliesOf(typeof(DependencyInjection))
    .AddClasses(classes =&gt; classes.AssignableTo(typeof(IDomainEventHandler&lt;&gt;)), publicOnly: false)
    .AsImplementedInterfaces()
    .WithScopedLifetime());
</code></pre>
<p>The important thing is that multiple handlers can react to the same event.</p>
<h2>The Dispatcher (Strongly Typed)</h2>
<p>Now we need something to orchestrate calling the handlers.
The dispatcher will take the domain events and call the appropriate handlers for each event.</p>
<pre><code class="language-csharp">public interface IDomainEventsDispatcher
{
    Task DispatchAsync(
        IEnumerable&lt;IDomainEvent&gt; domainEvents,
        CancellationToken cancellationToken = default);
}

internal sealed class DomainEventsDispatcher(IServiceProvider serviceProvider)
    : IDomainEventsDispatcher
{
    private static readonly ConcurrentDictionary&lt;Type, Type&gt; HandlerTypeDictionary = new();
    private static readonly ConcurrentDictionary&lt;Type, Type&gt; WrapperTypeDictionary = new();

    public async Task DispatchAsync(
        IEnumerable&lt;IDomainEvent&gt; domainEvents,
        CancellationToken cancellationToken = default)
    {
        foreach (IDomainEvent domainEvent in domainEvents)
        {
            using IServiceScope scope = serviceProvider.CreateScope();

            Type domainEventType = domainEvent.GetType();

            Type handlerType = HandlerTypeDictionary.GetOrAdd(
                domainEventType,
                et =&gt; typeof(IDomainEventHandler&lt;&gt;).MakeGenericType(et));

            IEnumerable&lt;object?&gt; handlers = scope.ServiceProvider.GetServices(handlerType);

            foreach (object? handler in handlers)
            {
                if (handler is null) continue;

                var handlerWrapper = HandlerWrapper.Create(handler, domainEventType);

                await handlerWrapper.Handle(domainEvent, cancellationToken);
            }
        }
    }

    // Abstract base class for strongly-typed handler wrappers
    private abstract class HandlerWrapper
    {
        public abstract Task Handle(IDomainEvent domainEvent, CancellationToken cancellationToken);

        public static HandlerWrapper Create(object handler, Type domainEventType)
        {
            Type wrapperType = WrapperTypeDictionary.GetOrAdd(
                domainEventType,
                et =&gt; typeof(HandlerWrapper&lt;&gt;).MakeGenericType(et));

            return (HandlerWrapper)Activator.CreateInstance(wrapperType, handler)!;
        }
    }

    // Generic wrapper that provides strong typing for handler invocation
    private sealed class HandlerWrapper&lt;T&gt;(object handler) : HandlerWrapper where T : IDomainEvent
    {
        private readonly IDomainEventHandler&lt;T&gt; _handler = (IDomainEventHandler&lt;T&gt;)handler;

        public override async Task Handle(
            IDomainEvent domainEvent,
            CancellationToken cancellationToken)
        {
            await _handler.Handle((T)domainEvent, cancellationToken);
        }
    }
}
</code></pre>
<p>The dispatcher uses a wrapper to eliminate reflection during handler execution while maintaining type safety.
When we encounter a <code>UserRegisteredDomainEvent</code>, we create a <code>HandlerWrapper&lt;UserRegisteredDomainEvent&gt;</code>
that holds a strongly-typed reference to <code>IDomainEventHandler&lt;UserRegisteredDomainEvent&gt;</code>.
The wrapper casts the generic <code>IDomainEvent</code> to the specific event type at runtime, but the actual handler invocation uses compile-time types.</p>
<p>This gives us the performance benefits of avoiding reflection in the hot path (handler execution) while only using reflection once during wrapper creation.
The trade-off is additional complexity, but the performance gain is significant if you're dispatching many events.</p>
<p>Don't forget to register the dispatcher with DI:</p>
<pre><code class="language-csharp">services.AddTransient&lt;IDomainEventsDispatcher, DomainEventsDispatcher&gt;();
</code></pre>
<h2>Usage Example</h2>
<p>Here's how to use the domain events dispatcher in your application:</p>
<pre><code class="language-csharp">public class UserController(
    IUserService userService,
    IDomainEventsDispatcher domainEventsDispatcher) : ControllerBase
{
    [HttpPost(&quot;register&quot;)]
    public async Task&lt;IActionResult&gt; Register([FromBody] RegisterUserRequest request)
    {
        try
        {
            // Create the user
            var user = await userService.CreateUserAsync(request.Email, request.Password);

            // Publish the domain event
            var userRegisteredEvent = new UserRegisteredDomainEvent(user.Id, user.Email);

            await domainEventsDispatcher.DispatchAsync([userRegisteredEvent]);

            return Ok(new { UserId = user.Id, Message = &quot;User registered successfully&quot; });
        }
        catch (Exception ex)
        {
            return BadRequest(new { Error = ex.Message });
        }
    }
}
</code></pre>
<p>You could also <a href="how-to-use-domain-events-to-build-loosely-coupled-systems"><strong>integrate domain events directly into your domain entities</strong></a>.</p>
<h2>Limitations and Tradeoffs</h2>
<p>This implementation runs entirely in-process, which has important implications.
All handlers execute synchronously within the same request context, but each gets its own DI scope.
This means:</p>
<ul>
<li>
<p><strong>Immediate feedback</strong>:
If any handler fails, the exception bubbles up to the caller immediately.
No silent failures or eventual consistency surprises.</p>
</li>
<li>
<p><strong>Caller control</strong>:
The code that dispatches events decides how to handle failures — rollback transactions, retry operations, or continue despite errors.
The dispatcher doesn't make these decisions for you.</p>
</li>
<li>
<p><strong>Reliability concerns</strong>:
If the process crashes after some handlers succeed but before others complete, there's no automatic recovery.
Events aren't persisted or retried.</p>
</li>
</ul>
<p>For critical side effects that can't be lost, consider the <a href="implementing-the-outbox-pattern"><strong>Outbox pattern</strong></a>.
Instead of dispatching events immediately, store them alongside your business data in the same transaction.
A background service can later retry failed events, ensuring nothing gets lost.
This decouples reliability from performance — your main operation completes quickly while events are processed reliably in the background.</p>
<h2>Wrapping Up</h2>
<p>Domain events are a powerful pattern for decoupling business logic, and you don't need a heavyweight framework to use them effectively.
The implementation we've built here provides a solid foundation that you can extend as your needs grow.</p>
<p>The beauty of rolling your own solution is that you understand every piece, making debugging and customization straightforward.
This pattern fits excellently in <a href="/pragmatic-domain-driven-design"><strong>Domain-Driven Design</strong></a> and
<a href="/pragmatic-clean-architecture"><strong>Clean Architecture</strong></a> systems where decoupling business logic is crucial.</p>
<p>For systems requiring bulletproof reliability or cross-service communication, invest in proper message infrastructure.
But for many applications, this simple approach hits the sweet spot between coupling and complexity.</p>
<p>The key insight is understanding your trade-offs upfront rather than discovering them in production.
Start simple, measure what matters, and evolve based on real requirements.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_143.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[CQRS Pattern the Way It Should've Been From the Start]]></title>
            <link>https://milanjovanovic.tech/blog/cqrs-pattern-the-way-it-should-have-been-from-the-start</link>
            <guid isPermaLink="false">cqrs-pattern-the-way-it-should-have-been-from-the-start</guid>
            <pubDate>Sat, 17 May 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to implement CQRS in .NET without relying on MediatR. This guide walks you through a lightweight setup using simple interfaces, decorators, and DI — no frameworks required.]]></description>
            <content:encoded><![CDATA[<p>MediatR is going commercial.</p>
<p><a href="https://www.jimmybogard.com/automapper-and-mediatr-going-commercial/">Jimmy Bogard recently announced</a>
that MediatR will adopt a commercial license model for companies above a certain size.</p>
<p>For many teams, this is a trigger to re-evaluate their usage and possibly look for alternatives.</p>
<p>And it's not a bad time to do so.
MediatR became almost synonymous with CQRS in .NET, despite the fact that <a href="stop-conflating-cqrs-and-mediatr"><strong>CQRS and MediatR are not the same thing</strong></a>.
Most projects use it as a thin dispatching layer for commands and queries — a use case that can be covered with a few straightforward abstractions.</p>
<p>By removing MediatR, you gain:</p>
<ul>
<li>Full control over your CQRS infrastructure</li>
<li>Predictable, explicit handler dispatching</li>
<li>Simpler debugging and onboarding</li>
<li>Cleaner DI setup and better testability</li>
</ul>
<p>In this article, I'll walk you through building a minimal CQRS setup with just a few interfaces and support for decorators.
No hidden DI magic.
Just clean, predictable code.</p>
<p>We'll cover:</p>
<ul>
<li>Defining <code>ICommand</code>, <code>IQuery</code>, and handler contracts</li>
<li>Adding support for decorators (logging, validation, etc.)</li>
<li>Registering everything with DI</li>
<li>A full working example in a real-world scenario</li>
</ul>
<p>Let's get started.</p>
<h2>Commands, Queries, and Handlers</h2>
<p>Let's start by defining the basic contracts for commands and queries.</p>
<pre><code class="language-csharp">// ICommand.cs
public interface ICommand;
public interface ICommand&lt;TResponse&gt;;

// IQuery.cs
public interface IQuery&lt;TResponse&gt;;
</code></pre>
<p>These interfaces exist purely as markers.
They allow us to structure application logic around intention — write operations go through <code>ICommand</code>, read operations through <code>IQuery</code>.</p>
<p>The handler interfaces follow the same model:</p>
<pre><code class="language-csharp">// ICommandHandler.cs
public interface ICommandHandler&lt;in TCommand&gt;
    where TCommand : ICommand
{
    Task&lt;Result&gt; Handle(TCommand command, CancellationToken cancellationToken);
}

public interface ICommandHandler&lt;in TCommand, TResponse&gt;
    where TCommand : ICommand&lt;TResponse&gt;
{
    Task&lt;Result&lt;TResponse&gt;&gt; Handle(TCommand command, CancellationToken cancellationToken);
}
</code></pre>
<pre><code class="language-csharp">// IQueryHandler.cs
public interface IQueryHandler&lt;in TQuery, TResponse&gt;
    where TQuery : IQuery&lt;TResponse&gt;
{
    Task&lt;Result&lt;TResponse&gt;&gt; Handle(TQuery query, CancellationToken cancellationToken);
}
</code></pre>
<p>These are nearly identical to MediatR's <code>IRequest</code> and <code>IRequestHandler</code> APIs, making migration trivial if you're moving off of MediatR.</p>
<p>You'll notice we're using a <code>Result</code> wrapper for all return types.
This is optional, but it promotes explicit success/failure handling and encourages consistency across the application boundary.
You can learn more about it in my <a href="functional-error-handling-in-dotnet-with-the-result-pattern"><strong>previous article</strong></a>.</p>
<p>These interfaces form a lightweight CQRS infrastructure, focused purely on intent and separation of concerns.
No mediator, no runtime indirection — just clear contracts for handling reads and writes.</p>
<h2>Practical Example: Command Handler</h2>
<p>To see these abstractions in action, let's implement a command that marks a todo item as completed.</p>
<pre><code class="language-csharp">// CompleteTodoCommand.cs
public sealed record CompleteTodoCommand(Guid TodoItemId) : ICommand;

// CompleteTodoCommandHandler.cs
internal sealed class CompleteTodoCommandHandler(
    IApplicationDbContext context,
    IDateTimeProvider dateTimeProvider,
    IUserContext userContext)
    : ICommandHandler&lt;CompleteTodoCommand&gt;
{
    public async Task&lt;Result&gt; Handle(CompleteTodoCommand command, CancellationToken cancellationToken)
    {
        TodoItem? todoItem = await context.TodoItems
            .SingleOrDefaultAsync(
                t =&gt; t.Id == command.TodoItemId &amp;&amp; t.UserId == userContext.UserId,
                cancellationToken);

        if (todoItem is null)
        {
            return Result.Failure(TodoItemErrors.NotFound(command.TodoItemId));
        }

        if (todoItem.IsCompleted)
        {
            return Result.Failure(TodoItemErrors.AlreadyCompleted(command.TodoItemId));
        }

        todoItem.IsCompleted = true;
        todoItem.CompletedAt = dateTimeProvider.UtcNow;

        todoItem.Raise(new TodoItemCompletedDomainEvent(todoItem.Id));

        await context.SaveChangesAsync(cancellationToken);

        return Result.Success();
    }
}
</code></pre>
<p>A few important things to note:</p>
<ul>
<li>The command is an immutable value object (just data, no behavior).</li>
<li>The handler encapsulates all business logic: validation, state change, raising domain events, and persistence.</li>
<li>There's no mediator, no <code>ISender</code>, no hidden dispatching. The handler is invoked directly via our custom abstractions.</li>
</ul>
<p>This makes intent explicit, avoids magic, and keeps the dependencies minimal.</p>
<p>We'll look at how to add decorators next, so we can introduce things like logging, validation, or transactions without modifying the handler itself.</p>
<h2>Decorators</h2>
<p>To support cross-cutting concerns like logging, validation, and transactions, we apply the <strong>decorator pattern</strong> around our handlers.
Technically, this is closer to the <strong>proxy pattern</strong>, since we're injecting behavior before/after delegating to the real handler.
But in the context of cross-cutting concerns, most people refer to this as a decorator — which is fine for our purposes.</p>
<p>Let's look at two examples: one for logging, one for validation.</p>
<pre><code class="language-csharp">using Serilog.Context;

internal sealed class LoggingCommandHandler&lt;TCommand, TResponse&gt;(
    ICommandHandler&lt;TCommand, TResponse&gt; innerHandler,
    ILogger&lt;CommandHandler&lt;TCommand, TResponse&gt;&gt; logger)
    : ICommandHandler&lt;TCommand, TResponse&gt;
    where TCommand : ICommand&lt;TResponse&gt;
{
    public async Task&lt;Result&lt;TResponse&gt;&gt; Handle(TCommand command, CancellationToken cancellationToken)
    {
        string commandName = typeof(TCommand).Name;

        logger.LogInformation(&quot;Processing command {Command}&quot;, commandName);

        Result&lt;TResponse&gt; result = await innerHandler.Handle(command, cancellationToken);

        if (result.IsSuccess)
        {
            logger.LogInformation(&quot;Completed command {Command}&quot;, commandName);
        }
        else
        {
            using (LogContext.PushProperty(&quot;Error&quot;, result.Error, true))
            {
                logger.LogError(&quot;Completed command {Command} with error&quot;, commandName);
            }
        }

        return result;
    }
}
</code></pre>
<p>This class wraps any <code>ICommandHandler&lt;TCommand, TResponse&gt;</code>, injecting the decorated handler as <code>innerHandler</code>.
It adds structured logging around the command execution without touching the core business logic.</p>
<p>Now a <a href="cqrs-validation-with-mediatr-pipeline-and-fluentvalidation"><strong>validation example with FluentValidation</strong></a>:</p>
<pre><code class="language-csharp">using FluentValidation;
using FluentValidation.Results;

internal sealed class ValidationCommandHandler&lt;TCommand, TResponse&gt;(
    ICommandHandler&lt;TCommand, TResponse&gt; innerHandler,
    IEnumerable&lt;IValidator&lt;TCommand&gt;&gt; validators)
    : ICommandHandler&lt;TCommand, TResponse&gt;
    where TCommand : ICommand&lt;TResponse&gt;
{
    public async Task&lt;Result&lt;TResponse&gt;&gt; Handle(TCommand command, CancellationToken cancellationToken)
    {
        // Validate the command using all registered validators
        ValidationFailure[] validationFailures = await ValidateAsync(command, validators);

        if (validationFailures.Length == 0)
        {
            return await innerHandler.Handle(command, cancellationToken);
        }

        // If validation fails, return a failure result with the errors
        return Result.Failure&lt;TResponse&gt;(CreateValidationError(validationFailures));
    }

    private static async Task&lt;ValidationFailure[]&gt; ValidateAsync&lt;TCommand&gt;(
        TCommand command,
        IEnumerable&lt;IValidator&lt;TCommand&gt;&gt; validators)
    {
        if (!validators.Any())
        {
            return [];
        }

        var context = new ValidationContext&lt;TCommand&gt;(command);

        ValidationResult[] validationResults = await Task.WhenAll(
            validators.Select(validator =&gt; validator.ValidateAsync(context)));

        ValidationFailure[] validationFailures = validationResults
            .Where(validationResult =&gt; !validationResult.IsValid)
            .SelectMany(validationResult =&gt; validationResult.Errors)
            .ToArray();

        return validationFailures;
    }

    private static ValidationError CreateValidationError(ValidationFailure[] validationFailures) =&gt;
        new(validationFailures.Select(f =&gt; Error.Problem(f.ErrorCode, f.ErrorMessage)).ToArray());
}
</code></pre>
<p>Each decorator handles a single concern and can be layered transparently around the core handler.</p>
<p><strong>Important:</strong> Since we're working with generic interfaces (<code>ICommandHandler&lt;,&gt;</code>, <code>IQueryHandler&lt;,&gt;</code>),
each decorator must explicitly target the same generic contract.
That means you'll need separate decorator classes for each handler abstraction you're using (e.g. command with result, command without result, query with result).</p>
<p>In the next section, we'll wire this up using <a href="https://github.com/khellang/Scrutor">Scrutor</a>.
It's a simple assembly scanning library that helps us register and decorate handlers cleanly.
Yes, it uses reflection, but only during startup — and it's fully transparent and predictable.</p>
<h2>DI Setup</h2>
<p>With our handlers and decorators in place, we can register everything using Scrutor.</p>
<pre><code class="language-csharp">services.Scan(scan =&gt; scan.FromAssembliesOf(typeof(DependencyInjection))
    .AddClasses(classes =&gt; classes.AssignableTo(typeof(IQueryHandler&lt;,&gt;)), publicOnly: false)
        .AsImplementedInterfaces()
        .WithScopedLifetime()
    .AddClasses(classes =&gt; classes.AssignableTo(typeof(ICommandHandler&lt;&gt;)), publicOnly: false)
        .AsImplementedInterfaces()
        .WithScopedLifetime()
    .AddClasses(classes =&gt; classes.AssignableTo(typeof(ICommandHandler&lt;,&gt;)), publicOnly: false)
        .AsImplementedInterfaces()
        .WithScopedLifetime());
</code></pre>
<p>This scans the application assembly and registers all command and query handlers (including internal types) as their respective interfaces.</p>
<p>Next, we apply decorators for validation and logging:</p>
<pre><code class="language-csharp">services.Decorate(typeof(ICommandHandler&lt;,&gt;), typeof(ValidationDecorator.CommandHandler&lt;,&gt;));
services.Decorate(typeof(ICommandHandler&lt;&gt;), typeof(ValidationDecorator.CommandBaseHandler&lt;&gt;));

services.Decorate(typeof(IQueryHandler&lt;,&gt;), typeof(LoggingDecorator.QueryHandler&lt;,&gt;));
services.Decorate(typeof(ICommandHandler&lt;,&gt;), typeof(LoggingDecorator.CommandHandler&lt;,&gt;));
services.Decorate(typeof(ICommandHandler&lt;&gt;), typeof(LoggingDecorator.CommandBaseHandler&lt;&gt;));
</code></pre>
<p>Each <code>Decorate</code> call wraps the previous registration.
<strong>Order matters</strong>, but it might not be intuitive at first glance.</p>
<p>The last decorator applied will be the outermost one at runtime.
So in this example:</p>
<ul>
<li>The <strong>base handler</strong> is first decorated by <strong>validation</strong></li>
<li>That composite is then decorated again by <strong>logging</strong></li>
</ul>
<p>Which means the <strong>logging decorator runs first</strong>, followed by <strong>validation</strong>, and then the core handler.</p>
<p>This order allows logging to capture the full command lifecycle, including any early exits from validation failures.</p>
<p>With this setup, you now have a fully functional and extensible CQRS pipeline:</p>
<ul>
<li>Custom handler interfaces</li>
<li>Clean decorator chain</li>
<li>Assembly-scanned DI setup</li>
</ul>
<h2>Usage from Minimal API</h2>
<p>Once everything is wired up, using a command handler from a Minimal API endpoint is straightforward:</p>
<pre><code class="language-csharp">internal sealed class Complete : IEndpoint
{
    public void MapEndpoint(IEndpointRouteBuilder app)
    {
        app.MapPut(&quot;todos/{id:guid}/complete&quot;, async (
            Guid id,
            ICommandHandler&lt;CompleteTodoCommand&gt; handler,
            CancellationToken cancellationToken) =&gt;
        {
            var command = new CompleteTodoCommand(id);

            Result result = await handler.Handle(command, cancellationToken);

            return result.Match(Results.NoContent, CustomResults.Problem);
        })
        .WithTags(Tags.Todos)
        .RequireAuthorization();
    }
}
</code></pre>
<p>We're injecting the appropriate <code>ICommandHandler&lt;CompleteTodoCommand&gt;</code> directly into the endpoint.
No need for <code>ISender</code>, no mediator layer, no runtime lookup.</p>
<p>This keeps the endpoint clean and focused on its primary responsibility: handling HTTP requests.</p>
<p>Everything is resolved explicitly by the container.
This makes the code easier to test, reason about, and trace while maintaining all the benefits of CQRS and separation of concerns.</p>
<h2>Conclusion</h2>
<p>CQRS doesn't require a complex framework.</p>
<p>With a few small interfaces, some decorator classes, and a clean DI setup, you can build a simple and flexible pipeline for handling commands and queries.
It's easy to understand, easy to test, and easy to extend.</p>
<p>If you want to see this pattern applied in a complete solution,
my <a href="/templates/clean-architecture"><strong>free Clean Architecture template</strong></a> includes everything covered in this article (fully wired up).</p>
<p>Use it as a reference or as a starting point for your next project.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_142.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[From Anemic Models to Behavior-Driven Models: A Practical DDD Refactor in C#]]></title>
            <link>https://milanjovanovic.tech/blog/from-anemic-models-to-behavior-driven-models-a-practical-ddd-refactor-in-csharp</link>
            <guid isPermaLink="false">from-anemic-models-to-behavior-driven-models-a-practical-ddd-refactor-in-csharp</guid>
            <pubDate>Sat, 10 May 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[A practical guide to transforming anemic domain models into behavior-rich aggregates in C# through incremental refactoring, enhancing code maintainability and business rule clarity.]]></description>
            <content:encoded><![CDATA[<p>If you've ever worked with a legacy C# codebase, you know the pain of an anemic domain model.
You have probably opened an <code>OrderService</code> (<em>all similarities to production code are merely a coincidence</em>) and thought <em>&quot;this file does everything.&quot;</em>
Pricing logic, discount rules, stock checks, database writes — <strong>all jam-packed into one class</strong>.
It works — until it doesn't.
New features turn into <strong>regression roulette</strong>, and test coverage plummets because the domain is buried under infrastructure.</p>
<p>This is the classic symptom of an anemic domain model, where entities are nothing but data holders, and all logic lives elsewhere.
It makes the system harder to reason about, and every change becomes a guessing game.
But what if we could push behavior back into the domain, one rule at a time?</p>
<p>In this article, we'll:</p>
<ol>
<li><strong>Inspect</strong> a typical anemic implementation.</li>
<li><strong>Identify</strong> hidden business rules that make it brittle.</li>
<li><strong>Refactor</strong> toward a behavior-rich aggregate one refactor at a time.</li>
<li><strong>Highlight</strong> the concrete payoffs so you can justify the change to teammates.</li>
</ol>
<p>Everything fits in a 6-minute read, but the pattern scales to any legacy system.</p>
<h2>Starting Point: God-like Service Class</h2>
<p>Below is an (unfortunately common) <code>OrderService</code>.
Besides calculating totals it also:</p>
<ul>
<li>applies a <strong>5 % VIP discount</strong>,</li>
<li>throws if any product is <strong>out of stock</strong>, and</li>
<li>rejects orders that would <strong>exceed the customer's credit limit</strong>.</li>
</ul>
<pre><code class="language-csharp">// OrderService.cs
public void PlaceOrder(Guid customerId, IEnumerable&lt;OrderItemDto&gt; items)
{
    var customer = _db.Customers.Find(customerId);
    if (customer is null)
    {
        throw new ArgumentException(&quot;Customer not found&quot;);
    }

    var order = new Order { CustomerId = customerId };

    foreach (var dto in items)
    {
        var inventory = _inventoryService.GetStock(dto.ProductId);
        if (inventory &lt; dto.Quantity)
        {
            throw new InvalidOperationException(&quot;Item out of stock&quot;);
        }

        var price = _pricingService.GetPrice(dto.ProductId);
        var lineTotal = price * dto.Quantity;
        if (customer.IsVip)
        {
            lineTotal *= 0.95m; // 5% discount for VIPs
        }

        order.Items.Add(new OrderItem
        {
            ProductId = dto.ProductId,
            Quantity = dto.Quantity,
            UnitPrice = price,
            LineTotal = lineTotal
        });
    }

    order.Total = order.Items.Sum(i =&gt; i.LineTotal);

    if (customer.CreditUsed + order.Total &gt; customer.CreditLimit)
    {
        throw new InvalidOperationException(&quot;Credit limit exceeded&quot;);
    }

    _db.Orders.Add(order);
    _db.SaveChanges();
}
</code></pre>
<h3>What's Wrong Here?</h3>
<ul>
<li><strong>Scattered rules:</strong> Discount application, stock validation, and credit-limit checks are buried inside the service.</li>
<li><strong>Tight coupling:</strong> <code>OrderService</code> must know about pricing, inventory, and EF Core just to place an order.</li>
<li><strong>Painful testing:</strong> Each unit test needs fakes for DB access, pricing, inventory, and VIP vs. non-VIP flows.</li>
</ul>
<div className='blockquote'>
<p><strong>Goal:</strong> Embed these rules <strong>inside the domain</strong> so the application layer only deals with orchestration.</p>
</div>
<h2>Guiding Principles Before We Touch Code</h2>
<ol>
<li><strong>Protect invariants close to the data.</strong>
Stock, discounts, and credit checks belong where the data lives — inside the <code>Order</code> aggregate.</li>
<li><strong>Expose intent, hide mechanics.</strong>
The application layer should read like a story: <em>&quot;place order&quot;</em>, not <em>&quot;calculate totals, check credit, write to DB&quot;</em>.</li>
<li><strong>Refactor in slices.</strong>
Each move is safe and compilable; no big-bang rewrites.</li>
<li><strong>Balance purity with pragmatism.</strong>
Move rules only when the payoff (clarity, safety, testability) beats the extra lines of code.</li>
</ol>
<h2>Step-by-Step Refactor</h2>
<p>The goal here isn't to chase purity or academic DDD.
It's to incrementally improve cohesion and make room for the domain to express itself.</p>
<p>At every step, we ask: Is this behavior something the domain should own?
If yes, we pull it inward.</p>
<h3>Embed Creation &amp; Validation Logic</h3>
<p>The first move is to make the aggregate responsible for building itself.
A static <code>Create</code> method gives us a single entry point where all invariants can fail fast.</p>
<p>While pushing stock validation into <code>Order</code> improves testability,
it does couple the order flow with inventory availability.
In some domains, you'd instead model this as a domain event and validate asynchronously.</p>
<pre><code class="language-csharp">// Order.cs (Factory Method)
public static Order Create(
    Customer customer,
    IEnumerable&lt;(Guid productId, int quantity)&gt; lines,
    IPricingService pricingService,
    IInventoryService inventoryService)
{
    var order = new Order(customer.Id);

    foreach (var (productId, quantity) in lines)
    {
        if (inventoryService.GetStock(productId) &lt; quantity)
        {
            throw new InvalidOperationException(&quot;Item out of stock&quot;);
        }

        var unitPrice = pricingService.GetPrice(productId);
        order.AddItem(productId, quantity, unitPrice, customer.IsVip);
    }

    order.EnsureCreditWithinLimit(customer);

    return order;
}
</code></pre>
<p><strong>Why?</strong>
Creation now <strong>fails fast</strong> if any invariant is broken.
The service no longer micromanages stock or discounts.</p>
<p>Notice how we're now following the &quot;Tell, Don't Ask&quot; principle.
Rather than the service checking conditions and then manipulating the Order,
we're telling the Order to create itself with the necessary validations built in.
This is a fundamental shift toward <strong>encapsulation</strong>.</p>
<p><strong>💡 On Injecting Services into Domain Methods</strong></p>
<p>Passing services like <code>IPricingService</code> or <code>IInventoryService</code> into a domain method such as <code>Order.Create</code> might seem unconventional at first glance.
But it's a deliberate design choice: it keeps the orchestration inside the domain model, where the business logic naturally belongs,
instead of bloating the application service with procedural workflows.</p>
<p>This approach maintains the entity's autonomy while still aligning with dependency injection principles — dependencies are passed explicitly, not resolved from within.
It's a powerful technique, but one that should be used selectively — only when the operation clearly fits within the domain's responsibility
and benefits from direct access to external services.</p>
<h3>Guard the Aggregate's Internal State</h3>
<pre><code class="language-csharp">// Order.cs (excerpt)
private readonly List&lt;OrderItem&gt; _items = new();
public IReadOnlyCollection&lt;OrderItem&gt; Items =&gt; _items.AsReadOnly(); // C# 12 -&gt; [.._items]

private void AddItem(Guid productId, int quantity, decimal unitPrice, bool isVip)
{
    if (quantity &lt;= 0)
    {
        throw new ArgumentException(&quot;Quantity must be positive&quot;);
    }

    var finalPrice = isVip ? unitPrice * 0.95m : unitPrice;
    _items.Add(new OrderItem(productId, quantity, finalPrice));

    RecalculateTotal();
}

private void EnsureCreditWithinLimit(Customer customer)
{
    if (customer.CreditUsed + Total &gt; customer.CreditLimit)
    {
        throw new InvalidOperationException(&quot;Credit limit exceeded&quot;);
    }
}
</code></pre>
<p><strong>Why bother?</strong></p>
<ul>
<li><strong>Encapsulation</strong>: Consumers can't mutate <code>_items</code> directly, ensuring invariants hold.</li>
<li><strong>Self-protection</strong>: The domain model protects its own consistency rather than relying on service-level checks.</li>
<li><strong>True OOP</strong>: Objects now combine data and behavior, as object-oriented programming intended.</li>
<li><strong>Simpler services</strong>: Application services can focus on coordination rather than business rules.</li>
</ul>
<h3>Shrink the Application Layer to Pure Orchestration</h3>
<pre><code class="language-csharp">public void PlaceOrder(Guid customerId, IEnumerable&lt;OrderLineDto&gt; lines)
{
    var customer = _db.Customers.Find(customerId);
    if (customer is null)
    {
        throw new ArgumentException(&quot;Customer not found&quot;);
    }
    var input = lines.Select(l =&gt; (l.ProductId, l.Quantity));

    var order = Order.Create(customer, input, _pricingService, _inventoryService);

    _db.Orders.Add(order);
    _db.SaveChanges();
}
</code></pre>
<p>The <code>PlaceOrder</code> method drops from <strong>44 lines</strong> to <strong>14</strong>, with <strong>zero business logic</strong>.</p>
<h2>What We Gained</h2>
<p><strong>Before the refactor</strong></p>
<ul>
<li>Service owned pricing, stock, discount, and credit checks.</li>
<li>Unit tests required heavy EF Core and service fakes.</li>
<li>Adding a new rule meant touching multiple files.</li>
</ul>
<p><strong>After the refactor</strong></p>
<ul>
<li>Aggregate owns all business rules; service only orchestrates.</li>
<li>Pure domain tests — no database container required.</li>
<li>Most changes are isolated to the <code>Order</code> aggregate.</li>
</ul>
<h2>Wrapping Up</h2>
<p>The real value in refactoring anemic models isn't technical — it's strategic.</p>
<p>By moving business logic closer to the data, you:</p>
<ul>
<li>Reduce the blast radius of changes</li>
<li>Make business rules explicit and testable</li>
<li>Open the door for tactical patterns like validation, events, and invariants</li>
</ul>
<p>But you don't need a big rewrite.
Start with one rule.
Refactor it.
Then the next.</p>
<p>That's how legacy systems evolve into maintainable architectures.</p>
<p>If you enjoyed this breakdown and want a hands-on, real-world guide to untangling messy services,
check out my course <a href="/pragmatic-domain-driven-design"><strong>Pragmatic Domain-Driven Design</strong></a>.
It's packed with before-and-after examples like this one.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_141.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Event-Driven Architecture in .NET with RabbitMQ]]></title>
            <link>https://milanjovanovic.tech/blog/event-driven-architecture-in-dotnet-with-rabbitmq</link>
            <guid isPermaLink="false">event-driven-architecture-in-dotnet-with-rabbitmq</guid>
            <pubDate>Sat, 03 May 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to implement flexible and resilient event-driven systems in .NET using RabbitMQ as a message broker with practical code examples for producers and consumers.]]></description>
            <content:encoded><![CDATA[<p><a href="https://en.wikipedia.org/wiki/Event-driven_architecture">Event-driven architecture</a> (EDA) can make applications more flexible and reliable.
Instead of one part of the system calling another directly, we let events flow through a message broker.
In this quick guide, I'll show you how to set up a simple event-driven system in .NET using <a href="https://www.rabbitmq.com/">RabbitMQ</a>.</p>
<p>We'll build a small example with a producer that sends events and a consumer that receives them.
For testing, I'll run RabbitMQ in a Docker container (with the Management UI enabled so we can see what's happening).
We'll use the official <a href="https://www.nuget.org/packages/rabbitmq.client/">RabbitMQ.Client</a> NuGet package in a .NET console app.</p>
<div className='blockquote'>
<p>Note: If you don't have RabbitMQ installed, you can run it quickly with Docker. For example:</p>
<pre><code class="language-bash">docker run -it --rm --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:4-management
</code></pre>
<p>This starts a RabbitMQ broker on localhost (AMQP port <code>5672</code>) and a management website at <code>http://localhost:15672</code>.</p>
</div>
<h2>RabbitMQ Basics</h2>
<p>Before coding, let's cover the basic components in RabbitMQ:</p>
<ul>
<li><strong>Producer</strong>: an application that sends messages (events) to RabbitMQ.</li>
<li><strong>Consumer</strong>: an application that receives messages from a queue.</li>
<li><strong>Queue</strong>: a mailbox inside RabbitMQ that stores messages.
Consumers read from queues.
Many producers can send to the same queue.</li>
<li><strong>Exchange</strong>: a routing mechanism that receives messages from producers and directs them to queues.
Producers actually send to an exchange instead of directly to a queue.
This decouples producers from specific queues - the exchange can decide where messages go, based on rules.</li>
</ul>
<p>In RabbitMQ, you can have multiple producers and multiple consumers.
Producers never send directly to a queue by name; instead, they send to an exchange.
The exchange decides which queues (if any) should get each message based on routing rules.</p>
<p>For now, we'll use a simple setup where the exchange will deliver all messages to one queue.</p>
<h2>Producer - Sending Events</h2>
<p>Let's start with the producer.
In our .NET console app, we'll use RabbitMQ.Client to connect to the RabbitMQ broker and send a message.</p>
<p>For instance, an <code>OrderPlaced</code> event could trigger downstream services - inventory, email notifications, etc. -
without the ordering system needing to call them directly.</p>
<pre><code class="language-csharp">var factory = new ConnectionFactory() { HostName = &quot;localhost&quot; };
using var connection = await factory.CreateConnectionAsync();
using var channel = await connection.CreateChannelAsync();

// Ensure the queue exists (create it if not already there)
await channel.QueueDeclareAsync(
    queue: &quot;orders&quot;,
    durable: true, // save to disk so the queue isn’t lost on broker restart
    exclusive: false, // can be used by other connections
    autoDelete: false, // don’t delete when the last consumer disconnects
    arguments: null);

// Create a message
var orderPlaced = new OrderPlaced
{
     OrderId = Guid.NewGuid(),
     Total = 99.99,
     CreatedAt = DateTime.UtcNow
};
var message = JsonSerializer.Serialize(orderPlaced);
var body = Encoding.UTF8.GetBytes(message);

// Publish the message
await channel.BasicPublishAsync(
    exchange: string.Empty, // default exchange
    routingKey: &quot;orders&quot;,
    mandatory: true, // fail if the message can’t be routed
    basicProperties: new BasicProperties { Persistent = true }, // message will be saved to disk
    body: body);

Console.WriteLine($&quot;Sent: {message}&quot;);
</code></pre>
<p>This code connects to RabbitMQ on <code>localhost</code>, declares a queue named <code>orders</code> (creates it if it doesn't exist already),
and publishes an <code>OrderPlaced</code> message to that queue.
We use an empty string for the exchange parameter, which tells RabbitMQ to use the default exchange.
The default exchange routes the message directly to the <code>orders</code> queue.</p>
<p>What's happening here:</p>
<ul>
<li>We declare a <strong>durable queue</strong>, so it survives RabbitMQ restarts</li>
<li>We mark the message as <strong>persistent</strong>, which tells RabbitMQ to write it to disk</li>
<li>We serialize an object into JSON and send it as a UTF-8 encoded byte array</li>
</ul>
<p>Now let's look at the consumer side.</p>
<h2>Consumer - Receiving Events</h2>
<p>Next, let's set up a consumer to receive messages from the queue.
The consumer will also connect to RabbitMQ and subscribe to the same queue.</p>
<p>To test this out, start the consumer application first (it will wait for messages), then run the producer application to send an event.</p>
<pre><code class="language-csharp">var factory = new ConnectionFactory() { HostName = &quot;localhost&quot; };
using var connection = await factory.CreateConnectionAsync();
using var channel = await connection.CreateChannelAsync();

// Declare (or check) the queue to consume from
await channel.QueueDeclareAsync(
    queue: &quot;orders&quot;,
    durable: true, // must match the producer's queue settings
    exclusive: false, // can be used by other connections
    autoDelete: false, // don’t delete when the last consumer disconnects
    arguments: null);

// Define a consumer and start listening
var consumer = new AsyncEventingBasicConsumer(channel);
consumer.ReceivedAsync += async (sender, eventArgs) =&gt;
{
    byte[] body = eventArgs.Body.ToArray();
    string message = Encoding.UTF8.GetString(body);
    var orderPlaced = JsonSerializer.Deserialize&lt;OrderPlaced&gt;(message);

    Console.WriteLine($&quot;Received: OrderPlaced - {orderPlaced.OrderId}&quot;);

    // Acknowledge the message
    await ((AsyncEventingBasicConsumer)sender)
        .Channel.BasicAckAsync(eventArgs.DeliveryTag, multiple: false);
};
await channel.BasicConsumeAsync(&quot;orders&quot;, autoAck: false, consumer);

Console.WriteLine(&quot;Waiting for messages...&quot;);
</code></pre>
<p>The consumer code declares the same <code>orders</code> queue and sets up an event handler for incoming messages.
We call <code>BasicConsumeAsync</code> to start listening on the queue.
RabbitMQ will push any new messages to our consumer's event handler.
Whenever a message arrives, the <code>consumer.ReceivedAsync</code> event fires, and we print out the message.</p>
<p>What's important here:</p>
<ul>
<li><code>autoAck: false</code> ensures we only acknowledge messages we actually process</li>
<li>If processing fails, we could use <code>BasicNack</code> to requeue or route to a dead-letter queue</li>
<li>Deserializing into a strongly typed object makes it easy to reason about the event</li>
</ul>
<p>So far we've had one consumer. But what if we run multiple consumers on the same queue?</p>
<h2>Competing Consumers - Scaling Out</h2>
<p>What if you have multiple consumers for the same queue?
RabbitMQ allows <strong>competing consumers</strong> on a queue.</p>
<p>If two or more consumers listen on the same queue, each message from that queue will be delivered to <strong>only one</strong> of them:</p>
<ul>
<li>RabbitMQ will distribute messages among the consumers (roughly in round-robin order)</li>
<li>This is great for scaling: you can run multiple instances of a worker to process messages in parallel</li>
</ul>
<p>In other words, consumers <em>compete</em> for messages on that queue.
This pattern helps spread the workload, but note that each individual message is still processed by a single consumer.</p>
<h2>Fanout Exchange: Broadcast to Multiple Consumers</h2>
<p>Competing consumers share the work by dividing messages, but sometimes you want every service to get the event.
That's where a <strong>fanout exchange</strong> comes in.</p>
<p>In RabbitMQ, a fanout exchange is used for broadcasting events to multiple consumers.
Instead of all consumers sharing one queue, each consumer has its own queue.
When the producer sends a message to a fanout exchange, the exchange copies and routes the message to all bound queues.
This way, every consumer receives a copy via its own queue.</p>
<p>To set this up in code, we declare a fanout exchange and bind queues to it.</p>
<p><strong>Producer</strong>:</p>
<pre><code class="language-csharp">// Producer setup for fanout
await channel.ExchangeDeclareAsync(
    exchange: &quot;orders&quot;,
    durable: true, // durable exchange
    autoDelete: false, // don’t delete when the last consumer disconnects
    type: ExchangeType.Fanout);

// Publish a message to the fanout exchange (routingKey is ignored for fanout)
var orderPlaced = new OrderPlaced
{
     OrderId = Guid.NewGuid(),
     Total = 99.99,
     CreatedAt = DateTime.UtcNow
};
var message = JsonSerializer.Serialize(orderPlaced);
var body = Encoding.UTF8.GetBytes(message);

await channel.BasicPublishAsync(
    exchange: &quot;orders&quot;,
    routingKey: string.Empty,
    mandatory: true,
    basicProperties: new BasicProperties { Persistent = true },
    body: body);
</code></pre>
<p><strong>Consumer</strong>:</p>
<pre><code class="language-csharp">// Consumer setup for fanout
await channel.ExchangeDeclareAsync(
    exchange: &quot;orders&quot;,
    durable: true,
    autoDelete: false,
    type: ExchangeType.Fanout);

// Create a queue for this consumer and bind it
await channel.QueueDeclareAsync(
    queue: &quot;orders-consumer-1&quot;,
    durable: true,
    exclusive: false,
    autoDelete: false,
    arguments: null);

await channel.QueueBindAsync(&quot;orders-consumer-1&quot;, &quot;orders&quot;, routingKey: string.Empty);

// Then consume messages from queueName as usual...
</code></pre>
<p>In the producer, we call <code>ExchangeDeclareAsync</code> to make sure an <code>orders</code> exchange exists (of type fanout).
We then <code>BasicPublishAsync</code> to that exchange.
For a fanout exchange, the <code>routingKey</code> can be an empty string because it's ignored (fanout sends to all queues regardless of any routing key).
On the consumer side, we declare the same exchange and then create a new <code>orders-consumer-1</code> queue.
We bind that queue to the <code>orders</code> exchange.
Now any message sent to the exchange will be delivered to this queue, and we can consume it.</p>
<p>If you run multiple consumer programs (each with its own queue bound to <code>orders</code> exchange),
each one will get every message (unlike the competing consumers scenario).
You can also peek into RabbitMQ's Management UI to see the exchange and queues in action.</p>
<div className="centered">
  ![RabbitMQ Management UI](/blogs/mnw_140/rabbitmq_ui.png)
</div>
<h2>Next Steps</h2>
<p>You can expand this basic setup with more advanced RabbitMQ features.
For example, you might use a <strong>direct exchange</strong> or <strong>topic exchange</strong> to route events to specific services,
set up acknowledgment and retry policies for robustness,
or implement <strong>dead-letter queues</strong> for error handling.
The core idea throughout is the same: decouple senders and receivers with a message broker, making your system more flexible and resilient.</p>
<p>If you want to explore event-driven architecture further, including patterns like the ones we touched on (and beyond),
check out my <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a> course.
It covers these concepts in depth with practical examples, so you can apply EDA in real-world projects.</p>
<p>Good luck out there, and see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_140.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Refactoring Overgrown Bounded Contexts in Modular Monoliths]]></title>
            <link>https://milanjovanovic.tech/blog/refactoring-overgrown-bounded-contexts-in-modular-monoliths</link>
            <guid isPermaLink="false">refactoring-overgrown-bounded-contexts-in-modular-monoliths</guid>
            <pubDate>Sat, 26 Apr 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to untangle bloated services and split them into clean, modular bounded contexts in a .NET modular monolith - one practical refactor at a time.]]></description>
            <content:encoded><![CDATA[<p>When you're building a <a href="what-is-a-modular-monolith"><strong>modular monolith</strong></a>, it's easy to let bounded contexts grow too large over time.
What started as a clean domain boundary slowly turns into a dumping ground for unrelated logic.
Before you know it, you have a massive context responsible for users, payments, notifications, and reporting - all tangled together.</p>
<p>This article is about tackling that mess.
We'll walk through how to identify an overgrown bounded context, and refactor it step-by-step into smaller, well-defined contexts.
You'll see practical techniques in action, with real .NET code and without theoretical fluff.</p>
<h2>Identifying an Overgrown Context</h2>
<p>You know you have a problem when:</p>
<ul>
<li>You're afraid to touch code because everything is interconnected</li>
<li>The same entity is used for 4 unrelated use cases</li>
<li>You see classes with 1000+ lines or services that do too much</li>
<li>Business logic from different subdomains bleeds into each other</li>
</ul>
<p>Here's a classic example.</p>
<p>We start with a <code>BillingContext</code> that now handles everything from notifications to reporting:</p>
<pre><code class="language-csharp">public class BillingService
{
    public void ChargeCustomer(int customerId, decimal amount) { ... }
    public void SendInvoice(int invoiceId) { ... }
    public void NotifyCustomer(int customerId, string message) { ... }
    public void GenerateMonthlyReport() { ... }
    public void DeactivateUserAccount(int userId) { ... }
}
</code></pre>
<p>This service has no clear boundaries.
It mixes <strong>Billing</strong>, <strong>Notifications</strong>, <strong>Reporting</strong>, and <strong>User Management</strong> into a single, bloated class.
Changing one feature could easily break another.</p>
<h2>Step 1: Identify Logical Subdomains</h2>
<p>We start by breaking this apart logically.
Think like a product owner.</p>
<p>Just ask: &quot;What domains are we really working with?&quot;</p>
<p>Group the methods:</p>
<ul>
<li><strong>Billing</strong>: <code>ChargeCustomer</code>, <code>SendInvoice</code></li>
<li><strong>Notifications</strong>: <code>NotifyCustomer</code></li>
<li><strong>Reporting</strong>: <code>GenerateMonthlyReport</code></li>
<li><strong>User Management</strong>: <code>DeactivateUserAccount</code></li>
</ul>
<p>Code within a <a href="monolith-to-microservices-how-a-modular-monolith-helps"><strong>bounded context</strong></a> should model a coherent domain.
When multiple domains are jammed into the same context, your architecture becomes misleading.</p>
<p>You can validate these groupings by checking:</p>
<ul>
<li>Which parts of the system change together?</li>
<li>Do teams use different vocabulary for each area?</li>
<li>Would you give each domain to a different team?</li>
</ul>
<p>If yes, it's a sign you're dealing with distinct contexts.</p>
<h2>Step 2: Extract One Context at a Time</h2>
<p>Don't try to do it all at once.
Start with something low-risk.</p>
<p>Let's begin by extracting <strong>Notifications</strong>.</p>
<p>Why <strong>Notifications</strong>?
Because it's a pure side-effect.
It doesn't impact business state, so it's easier to decouple safely.</p>
<p>Create a new module and move the logic there:</p>
<pre><code class="language-csharp">// New module: Notifications
public class NotificationService
{
    public void Send(int customerId, string message) { ... }
}
</code></pre>
<p>Then simplify the original <code>BillingService</code>:</p>
<pre><code class="language-csharp">public class BillingService
{
    private readonly NotificationService _notificationService;

    public BillingService(NotificationService notificationService)
    {
        _notificationService = notificationService;
    }

    public void ChargeCustomer(int customerId, decimal amount)
    {
        // Charge logic...
        _notificationService.Send(customerId, $&quot;You were charged ${amount}&quot;);
    }
}
</code></pre>
<p>This works. But now <strong>Billing</strong> <em>depends on</em> <strong>Notifications</strong>.
That's a coupling we want to avoid long-term.</p>
<p>Why?
Because a failure in <strong>Notifications</strong> could block a billing operation.
It also means <strong>Billing</strong> can't evolve independently.</p>
<p>Let's decouple with <a href="how-to-use-domain-events-to-build-loosely-coupled-systems"><strong>domain events</strong></a>:</p>
<pre><code class="language-csharp">public class CustomerChargedEvent
{
    public int CustomerId { get; init; }
    public decimal Amount { get; init; }
}

// Module: Billing
public class BillingService
{
    private readonly IDomainEventDispatcher _dispatcher;

    public BillingService(IDomainEventDispatcher dispatcher)
    {
        _dispatcher = dispatcher;
    }

    public void ChargeCustomer(int customerId, decimal amount)
    {
        // Charge logic...
        _dispatcher.Dispatch(new CustomerChargedEvent
        {
            CustomerId = customerId,
            Amount = amount
        });
    }
}

// Module: Notifications
public class CustomerChargedEventnHandler : IDomainEventHandler&lt;CustomerChargedEvent&gt;
{
    public Task Handle(CustomerChargedEvent @event)
    {
        // Send notification
    }
}
</code></pre>
<p>Now <strong>Billing</strong> doesn't even <em>know</em> about <strong>Notifications</strong>.
That's real modularity.
You can replace, remove, or enhance the <strong>Notifications</strong> module without touching <strong>Billing</strong>.</p>
<h2>Step 3: Migrate Data (If Needed)</h2>
<p>Most monoliths start with a single database.
That's fine.
But real modularity comes when each module controls its own <a href="modular-monolith-data-isolation"><strong>schema</strong></a>.</p>
<p>Why?
Because the database structure reflects ownership.
If everything touches the same tables, it's hard to enforce boundaries.</p>
<p>You don't have to do it all at once.
Start with:</p>
<ul>
<li>Creating a <a href="using-multiple-ef-core-dbcontext-in-single-application"><strong>separate <code>DbContext</code> per module</strong></a></li>
<li>Gradually migrate the tables to their own schemas</li>
<li>Read-only projections or database views for cross-context reads</li>
</ul>
<pre><code class="language-csharp">// Module: Billing
public class BillingDbContext : DbContext
{
    public DbSet&lt;Invoice&gt; Invoices { get; set; }
}

// Module: Notifications
public class NotificationsDbContext : DbContext
{
    public DbSet&lt;NotificationLog&gt; Logs { get; set; }
}
</code></pre>
<p>This separation enables independent schema evolution.
It also makes <a href="testing-modular-monoliths-system-integration-testing"><strong>testing</strong></a> faster and safer.</p>
<p>When migrating, use a transitional phase where both contexts read from the same underlying data.
Only switch write paths when confidence is high.</p>
<h2>Step 4: Repeat for Other Areas</h2>
<p>Apply the same playbook.
Target a clean split per subdomain.</p>
<p>Next up: <strong>Reporting</strong> and <strong>User Management</strong>.</p>
<p>Before:</p>
<pre><code class="language-csharp">billingService.GenerateMonthlyReport();
billingService.DeactivateUserAccount(userId);
</code></pre>
<p>After:</p>
<pre><code class="language-csharp">reportingService.GenerateMonthlyReport();
userService.DeactivateUser(userId);
</code></pre>
<p>Or via events:</p>
<pre><code class="language-csharp">_dispatcher.Dispatch(new MonthEndedEvent());
_dispatcher.Dispatch(new UserInactiveEvent(userId));
</code></pre>
<p>The goal here isn't just technical cleanliness - it's clarity.
Anyone looking at your solution should know what each module is responsible for.</p>
<p>And remember: boundaries should be enforced by code, not just by folder structure.
Different projects, separate EF models, and <a href="internal-vs-public-apis-in-modular-monoliths"><strong>explicit interfaces</strong></a> help enforce the split.
<a href="enforcing-software-architecture-with-architecture-tests"><strong>Architecture tests</strong></a> can also help ensure that modules don't break their boundaries.</p>
<h2>Takeaway</h2>
<p>Once you've finished the refactor, you'll have:</p>
<ul>
<li><strong>Smaller services</strong> focused on one job</li>
<li><strong>Decoupled modules</strong> that evolve independently</li>
<li><strong>Better tests</strong> and easier debugging</li>
<li><strong>Bounded contexts</strong> that actually match the domain</li>
</ul>
<p>This is more than structure, it's design that supports change.
You get loose coupling, testability, and clearer mental models.</p>
<p>You don't need microservices to get modularity.
You need to treat your monolith like a set of cooperating, isolated parts.</p>
<p>Start with one module.
Ship the change.
Repeat.</p>
<p>Want to go deeper into modular monolith design?
My full video course, <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a>,
walks you through building a real-world system from scratch - with clear boundaries, isolated modules, and practical patterns that scale.
Join 1,800+ students and start building better systems today.</p>
<p>That's all for today.</p>
<p>See you next Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_139.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Understanding Microservices: Core Concepts and Benefits]]></title>
            <link>https://milanjovanovic.tech/blog/understanding-microservices-core-concepts-and-benefits</link>
            <guid isPermaLink="false">understanding-microservices-core-concepts-and-benefits</guid>
            <pubDate>Sat, 19 Apr 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[What are microservices, and why might they be the right architectural choice for your organization? Microservices offer independently deployable, domain-focused components that provide flexibility when applied correctly to solve the right organizational problems.]]></description>
            <content:encoded><![CDATA[<p>I've been revisiting Sam Newman's excellent book <a href="https://www.oreilly.com/library/view/monolith-to-microservices/9781492047834/">&quot;Monolith to Microservices&quot;</a> recently,
and it's reminded me just how transformative this architectural approach can be <strong>when applied correctly</strong>.</p>
<p>As someone who's implemented microservices in various organizations, I wanted to share some valuable insights I've gained through both study and practical experience.</p>
<p>What exactly are microservices, and why might they be the right architectural choice for your organization?</p>
<p>Let's dive into the core concepts and benefits of microservices architecture.</p>
<h2>What Are Microservices?</h2>
<p>Microservices are <strong>independently deployable</strong> services modeled around a <strong>business domain</strong>.
Business domain is key here, but more on that later.
They communicate with each other via networks and offer many options for solving complex architectural problems.</p>
<p>Think of microservices as small, focused teams rather than a large department.
Each team has a specific responsibility, operates somewhat independently, and communicates clearly with other teams when needed.
Instead of one massive codebase that handles everything, you have multiple smaller codebases, each focusing on a specific business capability.</p>
<p><img src="/blogs/mnw_138/microservices_architecture.png" alt="Microservices architecture showing an event management system."></p>
<p>As Sam Newman defines them in &quot;Monolith to Microservices&quot;:</p>
<blockquote>
<p>Microservices are independently deployable services modeled around a business domain.
They communicate with each other via networks, and as an architecture choice offer many options for solving the problems you may face.</p>
</blockquote>
<p>Microservices give you a strategy to design a modular system and decompose it into bounded contexts.
However, the problem I often see is that developers use microservices to enforce code boundaries.
This is a mistake.
We'll fix that in a moment.</p>
<h2>Key Characteristics of Microservices</h2>
<p>Let's explore some of the key characteristics that define microservices:</p>
<h3>Independent Deployability</h3>
<p>You can make changes to a microservice and deploy it to production without having to deploy anything else.
This isn't just a theoretical ability - it's a discipline you practice for most of your releases.</p>
<p>The value here is significant: <strong>smaller deployments</strong> carry <strong>less risk</strong>, enable <strong>faster release cycles</strong>, and allow teams to test their changes in isolation.</p>
<p>When a critical bug appears in one service, you can fix and deploy just that service rather than orchestrating a full system release.
This is especially valuable in large systems where coordinating releases can be a logistical nightmare.</p>
<h3>Business Domain Focus</h3>
<p>Microservices are organized around <a href="screaming-architecture"><strong>business capabilities</strong></a> rather than technical layers.
Instead of having separate frontend, backend, and database teams (and the coordination that requires),
you might have teams dedicated to &quot;Event Management,&quot; &quot;Customer Accounts,&quot; or &quot;Attendance.&quot;</p>
<p>This alignment makes it easier to implement business functionality changes since all the related code - from UI to data storage - is grouped together.
When a business requirement changes, you can often change just one service rather than coordinating changes across multiple layers.</p>
<p><img src="/blogs/mnw_138/event_storming.png" alt="Event storming whiteboard with events grouped into logical boundaries."></p>
<h3>Data Ownership</h3>
<p>Microservices <strong>encapsulate data storage</strong> and retrieval, exposing data only via <a href="internal-vs-public-apis-in-modular-monoliths"><strong>well-defined interfaces</strong></a>.
Databases are hidden inside the service boundary rather than shared between services.</p>
<p>This stands in stark contrast to traditional approaches where multiple applications share a common database, often leading to tight coupling and risky schema changes.</p>
<p>When a service owns its data exclusively, it can:</p>
<ul>
<li>Evolve its internal data model without breaking other services</li>
<li>Implement the most appropriate storage technology for its needs</li>
<li>Provide a stable API for other services to access its data.</li>
</ul>
<p>It's not uncommon for multiple services to share the same database, but you're giving up some of the benefits of microservices by doing so.
In practice, one database per service is the most common approach.</p>
<h3>Network Communication</h3>
<p>Services communicate with each other via networks, making microservices a form of distributed system.
This network-based communication could use <a href="/pragmatic-rest-apis"><strong>REST APIs</strong></a>, message queues, gRPC, GraphQL, or other protocols depending on the specific needs.</p>
<div className="centered">
  ![A simple diagram with two boxes representing microservices that communicate over a network.](/blogs/mnw_138/network_communication.png)
</div>
<p>This explicit communication over networks allows services to be deployed independently and even run on different infrastructure.
But it also means dealing with network latency, potential failures, and serialization concerns.</p>
<p>Teams building microservices need to carefully design their <a href="modular-monolith-communication-patterns"><strong>inter-service communication patterns</strong></a>
to balance performance, reliability, and flexibility.</p>
<h2>The Origins of Microservices</h2>
<p>The term &quot;microservices&quot; has an interesting origin story.
In 2011, a software consultant named James Lewis became interested in what he called &quot;micro-apps&quot; - small services optimized to be easily replaceable.</p>
<p>The distinguishing feature was how small in scope these services were.
Some could be written or rewritten in just a few days.
As discussions evolved, the term &quot;microservices&quot; was adopted since these weren't self-contained applications but rather services working together.</p>
<p>It's worth noting that while &quot;micro&quot; is in the name, the size of a microservice isn't its defining characteristic.
Rather, it's about having services with well-defined boundaries that can be developed, deployed, and scaled independently.</p>
<h2>Key Benefits of Microservices</h2>
<p>What are the benefits of adopting a microservices architecture?</p>
<p>Why should you consider it for your organization?</p>
<p>Let's explore some of the key advantages:</p>
<h3>Flexibility and Adaptability</h3>
<p><strong>Microservices give you options</strong>.
They provide flexibility in how you can scale, evolve, and maintain your system over time.</p>
<ul>
<li>When business requirements change, you can modify just the affected services rather than risking changes to the entire system.</li>
<li>New capabilities can be introduced as new services without disrupting existing functionality.</li>
<li>As your understanding of the domain grows, service boundaries can evolve to better reflect that understanding.</li>
</ul>
<p>This ability to evolve incrementally is particularly valuable in rapidly changing business environments where time-to-market is critical.</p>
<h3>Technology Diversity</h3>
<p>With microservices, you can mix and match technology stacks.
Each service can use the programming language, database, or framework best suited for its specific requirements.
This practice is known as polyglot programming and <a href="modular-monolith-data-isolation"><strong>polyglot persistence</strong></a>.</p>
<p>For example, a recommendation engine might use Python with specialized machine learning libraries.
On the other hand, a transaction processing service might use .NET for its strong typing and performance characteristics.</p>
<p>A reporting service might use a columnar database optimized for analytics,
while a user profile service could use a document database that better fits its data model.</p>
<div className="centered">
  ![Event storming whiteboard with events grouped into logical boundaries.](/blogs/mnw_138/polyglot_persistence.png)
</div>
<p>This flexibility allows teams to choose the right tool for each job rather than compromising on a one-size-fits-all approach.</p>
<h3>Parallel Development</h3>
<p>Multiple teams can work on different services simultaneously without stepping on each other's toes.
This parallelization can significantly accelerate development velocity in larger organizations.</p>
<p>Each team can maintain its own release schedule, make technology decisions, and optimize for their specific service's needs without coordinating with every other team.
I've seen firsthand how this autonomy can reduce dependencies between teams, minimizing bottlenecks and wait times.</p>
<p>Organizations commonly structure their teams around services or groups of related services.
You might know this concept as <a href="https://en.wikipedia.org/wiki/Conway%27s_law">Conway's Law</a>:</p>
<blockquote>
<p>Organizations, who design systems, are constrained to produce designs which are copies of the communication structures of these organizations.</p>
</blockquote>
<h3>Targeted Scaling</h3>
<p>You can scale just the services that need it, rather than scaling the entire system.
This provides more efficient resource utilization and can reduce operational costs.</p>
<p>For example, if your product catalog needs to handle high traffic during a sale, you can scale just the catalog service without scaling your payment processing service.</p>
<p>This granular scaling becomes especially valuable as systems grow and different components have different performance characteristics.
Some services might be CPU-intensive while others are memory-intensive, and with microservices, you can optimize the infrastructure for each service's specific needs.</p>
<p>This approach can lead to significant cost savings compared to <a href="scaling-monoliths-a-practical-guide-for-growing-systems"><strong>scaling a monolith</strong></a>,
where all components must scale together regardless of their individual requirements.</p>
<h3>Organizational Alignment</h3>
<p>Microservices can help align your technical architecture with your organizational structure.
Teams can own specific services that correspond to their business domain expertise, promoting clearer ownership and accountability.</p>
<div className="centered">
  ![UML diagram showing the main bounded contexts in the system and their parts.](/blogs/mnw_138/evently_domain.png)
</div>
<p>This alignment reduces handoffs and coordination costs between teams, as each team has clear boundaries of responsibility.
It supports Conway's Law in a positive way.
Instead of having your communication structure accidentally create your architecture, you deliberately design both your teams and your services around business capabilities.</p>
<p>This approach can lead to more stable team structures and software boundaries over time, as business domains tend to evolve more slowly than technical implementations.</p>
<h2>Challenges to Consider With Microservices</h2>
<p>While microservices offer numerous benefits, they're not without challenges:</p>
<h3>Distributed System Complexity</h3>
<p>Network communication introduces latency, reliability challenges, and makes debugging more difficult.
Services must handle network failures gracefully, implement retries with backoff strategies,
and deal with the reality that a request might succeed but the response might get lost.</p>
<p><a href="introduction-to-distributed-tracing-with-opentelemetry-in-dotnet"><strong>Distributed tracing</strong></a> becomes essential to understand how requests flow through the system.</p>
<p><img src="/blogs/mnw_138/distributed_trace.png" alt="Distributed trace."></p>
<p>You'll need to develop strategies for handling partial system failures.
Concepts like circuit breakers and bulkheads become part of your everyday vocabulary.</p>
<h3>Operational Overhead</h3>
<p>Managing many services requires robust <a href="streamlining-dotnet-9-deployment-with-github-actions-and-azure"><strong>deployment pipelines</strong></a>, monitoring, and debugging tools.
You'll need to invest in automation for deployment, <a href="health-checks-in-asp-net-core"><strong>health checking</strong></a>, scaling,
and perhaps <a href="how-dotnet-aspire-simplifies-service-discovery"><strong>service discovery</strong></a>.
Each service needs monitoring, logging, and alerting.</p>
<p>This overhead can be substantial.
Organizations successfully running microservices typically have a strong DevOps culture and tooling to manage this complexity.</p>
<h3>Data Consistency</h3>
<p>Maintaining consistency across service boundaries becomes more challenging without the safety of <a href="working-with-transactions-in-ef-core"><strong>database transactions</strong></a>.</p>
<p>Implementing business processes that span multiple services often requires eventual consistency models and compensation mechanisms to handle failures.
You'll need to design your services with <a href="implementing-idempotent-rest-apis-in-aspnetcore"><strong>idempotency</strong></a> in mind and may need to implement patterns like the
<a href="implementing-the-saga-pattern-with-masstransit"><strong>Saga pattern</strong></a> to manage distributed transactions.</p>
<p>These approaches add complexity but can actually lead to more resilient systems when implemented correctly.</p>
<h3>Service Coordination</h3>
<p>Orchestrating workflows that span multiple services requires careful design.
Simple processes in a monolith can become complex choreographies in a microservice architecture.</p>
<p>You'll need to decide whether to use <a href="orchestration-vs-choreography"><strong>orchestration</strong></a> (where a central service directs the process)
or <a href="orchestration-vs-choreography"><strong>choreography</strong></a> (where services react to events without central coordination).
These patterns have different trade-offs in terms of coupling, resilience, and observability.</p>
<p>Designing these cross-service workflows often reveals subdomain boundaries you might have missed in initial modeling.</p>
<h2>Key Takeaway</h2>
<p>From my experience, the most important thing to remember is that microservices ultimately <strong>buy you options</strong>.
They provide flexibility but come with costs.</p>
<p>Are these costs worth the options you want to exercise?</p>
<p>For organizations with a large engineering team working on a complex system that needs to evolve quickly, microservices may be worth the overhead.
For smaller teams or systems with more stable requirements, a <a href="what-is-a-modular-monolith"><strong>well-designed monolith</strong></a> might be more appropriate.
Forget about resume-driven development for a moment.
It's about what solves your specific problems with <strong>acceptable trade-offs</strong>.</p>
<p>In my work, I've found that the most successful microservice adoptions start small, often by breaking off just one or two services from a monolith.
Then, gradually expanding as the organization builds the necessary skills and infrastructure.
This evolutionary approach reduces risk and allows teams to learn as they go.</p>
<p>I always like to reflect on these questions:</p>
<ol>
<li>What parts of the current architecture would benefit most from independent deployability?</li>
<li>What challenges might the organization face when adopting microservices?</li>
<li>How well does the current system align with business domains?</li>
</ol>
<p>If you want to dive deeper into building microservices - but starting from a monolith, check out <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a>.
There's an entire chapter dedicated to developing microservices, including advanced techniques such as API gateways, using message queues, and system integration testing.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_138.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[What is Vector Search? A Concise Guide]]></title>
            <link>https://milanjovanovic.tech/blog/what-is-vector-search-a-concise-guide</link>
            <guid isPermaLink="false">what-is-vector-search-a-concise-guide</guid>
            <pubDate>Sat, 12 Apr 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Vector search finds information based on meaning rather than exact keywords, delivering more intuitive results by converting content into numerical vectors that capture semantic relationships.]]></description>
            <content:encoded><![CDATA[<p><strong>Vector search</strong> is changing how we find information.
Unlike old search methods that look for exact words, vector search finds content based on meaning.
This makes search results more helpful and human-like.</p>
<p>When you search for &quot;quick healthy breakfast ideas,&quot; vector search can find articles about &quot;nutritious morning meals&quot; even if they don't use your exact words.
This happens because vector search understands what you mean, not just what you type.</p>
<p>In this week's newsletter, we'll break down how vector search works and why it's important.</p>
<h2>Understanding Vector Embeddings</h2>
<p>At the heart of vector search are <strong>vector embeddings</strong>.
These are lists of numbers that represent data.
But how do we get from words or images to numbers?</p>
<p>Here's how it works:</p>
<ol>
<li>We feed text, images, or other data into a <a href="working-with-llms-in-dotnet-using-microsoft-extensions-ai"><strong>large language model (LLM)</strong></a></li>
<li>The LLM turns each piece of data into a list of numbers (a vector)</li>
<li>These numbers capture the meaning of the data</li>
<li>Similar things get similar number patterns</li>
</ol>
<div className="centered">
  ![Vector space visualization.](/blogs/mnw_137/vector_space.png)
</div>
<p>Think of each number in the vector as describing one aspect of the data.
With hundreds or thousands of these numbers, vectors can capture complex meanings and relationships.</p>
<p>For example, in vector form, the words &quot;lion&quot; and &quot;bobcat&quot; would have similar number patterns because they refer to similar animals.
Meanwhile, &quot;cat&quot; would have a different pattern, though still somewhat similar since it's also a feline.</p>
<h2>How Vector Search Works</h2>
<p>When you search using vector search:</p>
<ol>
<li>Your search question gets turned into a vector</li>
<li>The system compares this vector to all the vectors in its database</li>
<li>It finds the vectors that are most similar to your search vector</li>
<li>It returns the data connected to those similar vectors</li>
</ol>
<p>To find similar vectors, the system measures how close they are to each other.
Think of each vector as a point in space - the closer two points are, the more similar their meanings.</p>
<p><img src="/blogs/mnw_137/vector_search.png" alt="Vector search visualization."></p>
<h3>Vector Databases</h3>
<p>To make vector search work well with lots of data, we need special storage systems called <strong>vector databases</strong>.
These databases are built to store and quickly search through millions or billions of vectors.</p>
<p>Vector databases do more than just store vectors - they organize them in smart ways that make searching faster.
They use special methods called &quot;Approximate Nearest Neighbor&quot; (ANN) algorithms that can find similar vectors without checking every single one in the database.</p>
<figure className="figure-center">
  <div className="bordered">
    ![Vector databases landscape.](/blogs/mnw_137/vector_databases_landscape.png)
  </div>
  <figcaption>
    Source: [What are Vector Databases? A Beginner's
    Guide](https://www.infracloud.io/blogs/vector-databases-beginners-guide/)
  </figcaption>
</figure>
<p>These databases also store the original content along with its vector, so when you get search results,
you see the actual text, images, or other data you were looking for.
Popular vector databases include Weaviate, Pinecone, and Qdrant.
But you can also turn PostgreSQL into a vector database using the pgvector extension.</p>
<p>The combination of vector embeddings and vector databases makes search extremely fast, even with millions of items to search through.
This speed makes vector search practical for real-world applications where users expect instant results.</p>
<h2>Key Differences from Traditional Search</h2>
<p>Traditional keyword search works like this:</p>
<ul>
<li>You type &quot;red shoes&quot;</li>
<li>The system finds pages with the words &quot;red&quot; and &quot;shoes&quot;</li>
<li>Results that mention these words more often rank higher</li>
</ul>
<p>This approach has problems:</p>
<ul>
<li>It misses related terms (&quot;scarlet footwear&quot;)</li>
<li>It doesn't understand context</li>
<li>It can't handle questions well</li>
</ul>
<p>An improvement to keyword search is <a href="how-i-implemented-full-text-search-on-my-website"><strong>full-text search</strong></a>, which looks at the whole text of documents.
However, this still has some shortcomings that vector search solves.</p>
<p>Vector search fixes these issues by focusing on meaning rather than exact words.
It can:</p>
<ul>
<li>Find content with related concepts</li>
<li>Understand the context of your search</li>
<li>Return helpful results even for complex questions</li>
</ul>
<p>This is why vector search powers many modern AI applications.
It helps chatbots find relevant information and makes recommendation systems more accurate.</p>
<h2>Summary</h2>
<p>Vector search represents a major step forward in how computers understand and retrieve information.
By converting data into number patterns that capture meaning, vector search can find connections that keyword search would miss.</p>
<p>This technology is behind many of the smart search features we now take for granted -
from finding similar products in online stores to helping AI assistants answer our questions.
As AI continues to advance, vector search will play an increasingly important role in helping us navigate the growing sea of digital information.</p>
<p>While not perfect, vector search bridges the gap between how computers store data and how humans think about meaning -
making our digital tools more helpful and intuitive to use.</p>
<p>In a future newsletter, we'll dive into the practical side of vector search.
You'll learn how to implement your own vector search system using popular tools and libraries, with step-by-step code examples and best practices.</p>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_137.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[MediatR and MassTransit Going Commercial: What This Means For You]]></title>
            <link>https://milanjovanovic.tech/blog/mediatr-and-masstransit-going-commercial-what-this-means-for-you</link>
            <guid isPermaLink="false">mediatr-and-masstransit-going-commercial-what-this-means-for-you</guid>
            <pubDate>Sat, 05 Apr 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Popular .NET libraries MediatR, AutoMapper, and MassTransit are transitioning to commercial licenses after over a decade of free open-source availability. This article examines the reasons behind these changes, what they mean for your projects, and the practical options available to developers moving forward.]]></description>
            <content:encoded><![CDATA[<p>Big changes are happening in the .NET ecosystem.
Three powerhouse libraries - MediatR, AutoMapper, and MassTransit - are moving to commercial licenses.
Not so long ago, Fluent Assertions also announced its plans to move to a commercial license.</p>
<p>As someone who's built countless systems with these tools over the past decade, I have thoughts.
And some strong opinions.</p>
<h2>The Libraries We Love (And Sometimes Hate)</h2>
<p>If you're a .NET developer, you likely use at least one of these:</p>
<p><a href="https://github.com/AutoMapper/AutoMapper"><strong>AutoMapper</strong></a> (794.7M downloads) transforms objects from one type to another.
It removes mountains of tedious mapping code that nobody enjoys writing.
One line replaces twenty.
<strong>I personally despise AutoMapper and mapping libraries in general</strong>, but I can't deny their popularity.</p>
<p><a href="https://github.com/jbogard/MediatR"><strong>MediatR</strong></a> (286.6M downloads) implements the <a href="cqrs-pattern-with-mediatr"><strong>mediator pattern</strong></a>.
It decouples requests from the objects handling them, promoting separation of concerns.
There's also the pipeline behavior feature, which allows you to add cross-cutting concerns.
I'm a huge fan and use it regularly in my projects.</p>
<p><a href="https://github.com/MassTransit/MassTransit"><strong>MassTransit</strong></a> (130.0M downloads) makes distributed messaging simple.
It wraps message brokers like <a href="using-masstransit-with-rabbitmq-and-azure-service-bus"><strong>RabbitMQ and Azure Service Bus</strong></a> with an elegant API.
Building event-driven systems becomes approachable.
This is another tool I love and often recommend.</p>
<p>These libraries aren't just popular - they're transformative.
They've shaped how we build .NET applications.</p>
<h2>The Maintainer's Reality</h2>
<p>Both announcements tell a similar story.</p>
<figure className="figure-center">
  <div className="bordered">
    ![AutoMapper and MediatR Going Commercial.](/blogs/mnw_136/mediatr_automapepr_announcement.png)
  </div>
  <figcaption>
    Source: [AutoMapper and MediatR Going
    Commercial](https://www.jimmybogard.com/automapper-and-mediatr-going-commercial/)
  </figcaption>
</figure>
<p>Jimmy Bogard (AutoMapper, MediatR) writes:</p>
<blockquote>
<p>You can see exactly where my contributions cratered and flat-lined. And that's just commits—issues, PRs, discussions, all my time dried up.</p>
</blockquote>
<p>His OSS work was previously sponsored by his former employer.
When he went independent, that support vanished.
His focus shifted to his consulting business.</p>
<figure className="figure-center">
  <div className="bordered">
    ![Announcing MassTransit v9.](/blogs/mnw_136/masstransit_announcement.png)
  </div>
  <figcaption>
    Source: [Announcing MassTransit
    v9](https://masstransit.io/introduction/v9-announcement)
  </figcaption>
</figure>
<p>Similarly, MassTransit has grown from &quot;a single assembly that supported MSMQ&quot; in 2007 to over thirty NuGet packages.
Its success created demands that are impossible to meet through volunteer work alone:</p>
<ul>
<li>Full-time development resources</li>
<li>Enterprise-grade support</li>
<li>Long-term sustainability</li>
</ul>
<p>Both maintainers face the same dilemma: how do you support widely used libraries when nobody pays you to do it?</p>
<h2>The Commercial Transition</h2>
<p>Here's what's happening:</p>
<p><strong>AutoMapper and MediatR</strong>: Jimmy hasn't shared specific timing or pricing yet.
He states, &quot;Short term, nothing will change.&quot;</p>
<p><strong>MassTransit</strong>: Moving from v8 (open source) to v9 (commercial) with this timeline:</p>
<ul>
<li>Q3 2025: v9 prerelease for early adopters</li>
<li>Q1 2026: v9 official release under commercial license</li>
<li>Through 2026: v8 security patches continue</li>
</ul>
<p>MassTransit's pricing targets:</p>
<ul>
<li>Small/medium businesses: $400/month or $4000/year</li>
<li>Large enterprises: $1200/month or $12000/year</li>
<li>Support for ISVs and consultants who build client applications</li>
</ul>
<p>None of this is set in stone.
The pricing aspect should be final by the time the commercial version is released.</p>
<h2>Why I Respect This Decision</h2>
<p>Both maintainers waited over a decade before making this move.
They've contributed immense value to our community for free.</p>
<p>Their announcements show careful consideration.
They're not abandoning users:</p>
<ul>
<li>Existing versions remain open source</li>
<li>Security patches will continue</li>
<li>Commercial licenses support sustainable development</li>
</ul>
<p>I honestly hope none of the above changes in the future.
The work they do is valuable.</p>
<p>Writing these libraries from scratch would cost your team far more than their license fees.</p>
<h2>Your Options Now (With My Take)</h2>
<p>If your project uses these libraries, you have choices:</p>
<ol>
<li>
<p><strong>Purchase the commercial license</strong><br>
This supports continued development and gets you new features and official support.</p>
</li>
<li>
<p><strong>Stay on the current open source version</strong><br>
MassTransit v8 and current MediatR/AutoMapper will remain available.
Security patches will continue through 2026 for MassTransit.
For MassTransit specifically, I'd consider staying on v8 for the long term if possible.</p>
</li>
<li>
<p><strong>Switch to alternatives</strong><br>
For AutoMapper: consider <a href="https://github.com/MapsterMapper/Mapster">Mapster</a> or <strong>manual mapping</strong> (my recommendation).</p>
<p>For MediatR: explore <a href="https://github.com/FastEndpoints/FastEndpoints">FastEndpoints</a> or <a href="stop-conflating-cqrs-and-mediatr"><strong>build a simple mediator yourself</strong></a>.</p>
<p>For MassTransit: look at raw client libraries like <a href="https://www.nuget.org/packages/rabbitmq.client/">RabbitMQ.Client</a> and
<a href="https://www.nuget.org/packages/Azure.Messaging.ServiceBus">Azure.Messaging.ServiceBus</a>,
and another option to consider is <a href="implementing-the-saga-pattern-with-rebus-and-rabbitmq"><strong>Rebus</strong></a>.</p>
</li>
<li>
<p><strong>Write equivalent functionality yourself</strong><br>
MediatR isn't too complex to build on your own.
I recommend giving it a try as an excellent coding exercise - it's probably the simplest way to move away from MediatR.</p>
<p>For AutoMapper, many teams have deep integrations with business logic in custom mappers.
This makes extracting and replacing it difficult.
Expect significant tech debt if you don't address this.</p>
<p>MassTransit, on the other hand, does so many things (and does them well) that migrating away would be challenging.
<a href="implementing-the-saga-pattern-with-masstransit"><strong>Saga support</strong></a> or the <a href="request-response-messaging-pattern-with-masstransit"><strong>request-response messaging</strong></a>
features are hard to replicate.
The only real alternative is diving into raw client libraries for your chosen message transport.</p>
</li>
</ol>
<p>Each option involves tradeoffs. The right choice depends on your project needs and budget.</p>
<h2>A Shift to Fundamentals</h2>
<p>These changes have made me reflect on something important: we should never lose sight of fundamentals.</p>
<p>We've been pampered and spoiled by these awesome libraries.
It's easy to lose sight of the actual problems they're solving and how they work under the hood.
People know how to use MediatR, but they don't understand the mechanisms behind it.</p>
<p>The same goes for MassTransit.
It abstracts away so many complexities of working with message brokers that it's possible to use it without knowing how RabbitMQ or Azure Service Bus actually works.</p>
<p>Remember this: using a library that abstracts something away doesn't excuse you from understanding the patterns and tools you're using.
The <strong>fundamentals are still there</strong> and always have been.
This might be the perfect opportunity to deepen your knowledge of what's happening beneath these abstractions.</p>
<h2>A Reality Check</h2>
<p>Open source isn't free. Someone pays - either with time or money.</p>
<p>I'm honestly tired of seeing developers complain about these changes.
Who are they to demand software for free?
The entitlement is astounding.
These maintainers have provided immense value for over a decade without asking for anything in return.</p>
<p>We've enjoyed years of exceptional tooling without directly funding it.
Now we face a reckoning.</p>
<p>As businesses reap massive productivity gains from these libraries, it's reasonable to ask: shouldn't some of that value flow back to the creators?</p>
<p>I hope these projects thrive under their new models.
They've earned support after years of thankless work.</p>
<p>Both my <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture</strong></a> and
<a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a> courses currently use MediatR and MassTransit extensively.
I plan to keep them on MediatR v12 and MassTransit v8 in the short term.
However, I'll also be updating them to show migration paths away from these libraries.</p>
<p>What's your take?
Will you stick with the open versions, pay for licenses, or explore alternatives?</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_136.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How .NET Aspire Simplifies Service Discovery]]></title>
            <link>https://milanjovanovic.tech/blog/how-dotnet-aspire-simplifies-service-discovery</link>
            <guid isPermaLink="false">how-dotnet-aspire-simplifies-service-discovery</guid>
            <pubDate>Sat, 29 Mar 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[.NET Aspire revolutionizes distributed application development by simplifying service discovery through configuration-based approaches that eliminate the complexity of service-to-service communication.]]></description>
            <content:encoded><![CDATA[<p>Unless you've been living under a rock, you know that <a href="https://learn.microsoft.com/en-us/dotnet/aspire/get-started/aspire-overview">.NET Aspire</a>
is changing (for the better) how we build distributed applications in .NET.
A simple way to think about Aspire: it makes all the difficult things in software development easy.</p>
<p>.NET Aspire is a cloud-native application stack that simplifies the development and deployment of distributed applications.
One of the key challenges when building multi-service applications is building reliable <a href="modular-monolith-communication-patterns"><strong>communication between services</strong></a>.</p>
<p>In this week's newsletter, I want to focus on one aspect of .NET Aspire - <a href="service-discovery-in-microservices-with-net-and-consul"><strong>service discovery</strong></a>.
Service discovery lets our services figure out how to locate other services they want to integrate with.
.NET Aspire tackles this challenge with a simple, configuration-based approach that reduces complexity and boilerplate code.</p>
<h2>Understanding Service Discovery</h2>
<p><a href="https://learn.microsoft.com/en-us/dotnet/core/extensions/service-discovery"><strong>Service discovery</strong></a>
is the process by which services in a distributed application locate and communicate with each other.
As applications scale and evolve, keeping track of service endpoints becomes increasingly challenging.
Services might run on different ports during development or be deployed to different environments in production,
making hard-coded service URLs impractical.</p>
<p>Traditional approaches to service discovery often introduce complexity:</p>
<ul>
<li>Manual configuration of service endpoints that must be updated as environments change</li>
<li>Complex intermediary systems that require additional maintenance</li>
<li>Custom code to handle service resolution and connection management</li>
</ul>
<p>While many service discovery implementations rely on centralized registries,
.NET Aspire takes a different approach by leveraging application configuration to connect services.
This design choice simplifies the development experience while maintaining flexibility for various deployment scenarios.
Aspire automatically takes care of wiring up the correct service URLs and injecting them into your application settings.</p>
<p><img src="/blogs/mnw_135/aspire_service_discovery.png" alt="Diagram explaining how the .NET Aspire service discovery mechanism works."></p>
<h2>Practical Example of Service Discovery</h2>
<p>To understand how .NET Aspire handles service discovery, let's look at a practical example of an application with multiple services:</p>
<pre><code class="language-csharp">var builder = DistributedApplication.CreateBuilder(args);

// Add services to the app
var apiService = builder.AddProject&lt;Projects.WeatherApi&gt;(&quot;weather-api&quot;);
var webFrontend = builder.AddProject&lt;Projects.WebFrontend&gt;(&quot;web-frontend&quot;)
    .WithReference(apiService);
    
builder.Build().Run();
</code></pre>
<p>In this App Host definition, we're creating two services: a weather API and a web frontend.
The <code>.WithReference()</code> method establishes a connection between these services, which enables service discovery.
This simple declaration tells <a href="dotnet-aspire-a-game-changer-for-cloud-native-development"><strong>.NET Aspire</strong></a>
that the web frontend depends on the weather API and needs to communicate with it.
Note that the <code>web-frontend</code> has to be a server-side application (like <a href="https://learn.microsoft.com/en-us/aspnet/core/blazor/hosting-models">Blazor Server</a>)
for Aspire to be able to inject the service URL.</p>
<p>With this configuration in place, the web frontend can now reach the API using the service name <code>weather-api</code>
without additional service discovery code:</p>
<pre><code class="language-csharp">// Configures the default Aspire services, including service discovery
builder.AddServiceDefaults();

// In Program.cs of the web-frontend project
builder.Services.AddHttpClient(&quot;weather-api&quot;, (_, client) =&gt; {
    // The service name &quot;weather-api&quot; automatically resolves to the correct address
    client.BaseAddress = new Uri(&quot;http://weather-api&quot;);
});
</code></pre>
<p>This works because .NET Aspire manages the mapping between service names and their actual endpoints.
When the application runs, service discovery ensures that requests to <code>http://weather-api</code> are routed to the appropriate destination.</p>
<h2>Service Discovery Under the Hood</h2>
<p>The previous example contains a bit of &quot;magic&quot; that might not be immediately clear.
The <code>AddServiceDefaults()</code> method configures the default services for the application, including service discovery.</p>
<p>If we were to configure everything manually, it would look something like this:</p>
<pre><code class="language-csharp">builder.Services.AddServiceDiscovery();

builder.Services.AddHttpClient(&quot;weather-api&quot;, (_, client) =&gt; {
    client.BaseAddress = new Uri(&quot;http://weather-api&quot;);
})
.AddServiceDiscovery();
</code></pre>
<p>The first <code>AddServiceDiscovery()</code> method registers the necessary services to enable service discovery in the application.
The second <code>AddServiceDiscovery()</code> method on the HTTP client configures it to use service discovery for resolving the base address.
This means that when the HTTP client makes requests to <code>http://weather-api</code>,
it will automatically resolve the correct endpoint based on the service discovery configuration.</p>
<p>We can also configure service discovery globally for all HTTP clients in the application:</p>
<pre><code class="language-csharp">builder.Services.ConfigureHttpClientDefaults(static http =&gt;
{
    // Turn on service discovery by default
    http.AddServiceDiscovery();
});
</code></pre>
<p>This configuration ensures that all HTTP clients in the application will use service discovery by default,
eliminating the need to configure it for each client individually.</p>
<p>What Aspire does at runtime for all of this to work is inject a set of configuration values.
The configuration value names are derived from the service names we defined in the App Host, plus the respective scheme (http or https).
Here's an example of what the <code>weather-api</code> configuration might look like:</p>
<pre><code class="language-json">{
  &quot;Services&quot;: {
    &quot;weather-api&quot;: {
      &quot;http&quot;: [
        &quot;localhost:8080&quot;
      ]
    }
  }
}
</code></pre>
<p>We can also configure service discovery to work with HTTPS by adding the <code>https</code> scheme to the service name:</p>
<pre><code class="language-csharp">// Specify the https scheme explicitly in the service name
builder.Services.AddHttpClient(&quot;weather-api&quot;, (_, client) =&gt; {
    client.BaseAddress = new Uri(&quot;https://weather-api&quot;);
});

// Alternatively, we can use the https+http scheme and let Aspire handle the conversion
builder.Services.AddHttpClient(&quot;weather-api-2&quot;, (_, client) =&gt; {
    client.BaseAddress = new Uri(&quot;https+http://weather-api&quot;);
});
</code></pre>
<p>A significant advantage of Aspire's service discovery is its consistent behavior across environments:</p>
<ul>
<li>During development, services might run on localhost with different ports</li>
<li>In testing environments, services could be containerized</li>
<li>In production, services might be deployed to Kubernetes or other platforms</li>
</ul>
<p>Your code remains unchanged across these scenarios because the service name abstraction shields you from the underlying networking details.</p>
<p>Note that you don't have to use .NET Aspire to benefit from <a href="https://learn.microsoft.com/en-us/dotnet/core/extensions/service-discovery">service discovery</a>.
It's available as a standalone library (<code>Microsoft.Extensions.ServiceDiscovery</code>) and you can use it in any .NET application.</p>
<h2>Service Discovery with YARP as a Proxy</h2>
<p>A powerful application of .NET Aspire's service discovery capabilities is in API gateway scenarios using
<a href="implementing-an-api-gateway-for-microservices-with-yarp"><strong>YARP</strong></a> (Yet Another Reverse Proxy).
Let's explore how to implement this pattern:</p>
<pre><code class="language-csharp">// In the App Host
var apiService = builder.AddProject&lt;Projects.WeatherApi&gt;(&quot;weather-api&quot;);
var userService = builder.AddProject&lt;Projects.UserApi&gt;(&quot;user-api&quot;);
var proxyService = builder.AddProject&lt;Projects.ApiGateway&gt;(&quot;api-gateway&quot;)
    .WithReference(apiService)
    .WithReference(userService);
</code></pre>
<p>We'll need to add the YARP NuGet package to the API gateway project:</p>
<pre><code class="language-powershell">Install-Package Yarp.ReverseProxy # Adds the YARP package
Install-Package Microsoft.Extensions.ServiceDiscovery.Yarp # Adds the YARP service discovery package
</code></pre>
<p>In the gateway project, we can configure YARP to use service discovery:</p>
<pre><code class="language-csharp">// In Program.cs of the api-gateway project
var builder = WebApplication.CreateBuilder(args);

// Cofigures the service discovery services
builder.Services.AddServiceDiscovery();

// Add YARP services
builder.Services.AddReverseProxy()
    .LoadFromConfig(builder.Configuration.GetSection(&quot;ReverseProxy&quot;))
    // Configures a destination resolver that can use service discovery
    .AddServiceDiscoveryDestinationResolver();

var app = builder.Build();

// Configure the HTTP request pipeline
app.MapReverseProxy();

app.Run();
</code></pre>
<p>The YARP configuration in <code>appsettings.json</code> leverages the service names for endpoint resolution:</p>
<pre><code class="language-json">{
  &quot;ReverseProxy&quot;: {
    &quot;Routes&quot;: {
      &quot;weather-route&quot;: {
        &quot;ClusterId&quot;: &quot;weather-cluster&quot;,
        &quot;Match&quot;: {
          &quot;Path&quot;: &quot;/weather/{**catch-all}&quot;
        },
        &quot;Transforms&quot;: [
          { &quot;PathRemovePrefix&quot;: &quot;/weather&quot; }
        ]
      },
      &quot;user-route&quot;: {
        &quot;ClusterId&quot;: &quot;user-cluster&quot;,
        &quot;Match&quot;: {
          &quot;Path&quot;: &quot;/users/{**catch-all}&quot;
        },
        &quot;Transforms&quot;: [
          { &quot;PathRemovePrefix&quot;: &quot;/users&quot; }
        ]
      }
    },
    &quot;Clusters&quot;: {
      &quot;weather-cluster&quot;: {
        &quot;Destinations&quot;: {
          &quot;destination1&quot;: {
            &quot;Address&quot;: &quot;http://weather-api&quot;
          }
        }
      },
      &quot;user-cluster&quot;: {
        &quot;Destinations&quot;: {
          &quot;destination1&quot;: {
            &quot;Address&quot;: &quot;http://user-api&quot;
          }
        }
      }
    }
  }
}
</code></pre>
<p>This configuration creates an API gateway that:</p>
<ul>
<li>Routes requests with path <code>/weather/*</code> to the weather API service</li>
<li>Routes requests with path <code>/users/*</code> to the user API service</li>
<li>Uses service names (<code>weather-api</code> and <code>user-api</code>) that service discovery resolves at runtime</li>
</ul>
<p>The beauty of this approach is that the gateway doesn't need to know the actual endpoints of the backend services.
It simply uses the service names, and .NET Aspire handles providing the configuration at runtime.
This makes the gateway configuration more portable and easier to maintain as the application evolves.</p>
<h2>Conclusion</h2>
<p>.NET Aspire transforms service discovery from a complex infrastructure challenge into a straightforward configuration concern.
By using a configuration-based approach rather than a centralized registry,
it simplifies the development experience while maintaining the flexibility needed for various deployment scenarios.</p>
<p>The key advantages of Aspire's service discovery include:</p>
<ul>
<li>Declarative service relationships in the App Host</li>
<li>Simple service name resolution that works across environments</li>
<li>Seamless integration with the .NET ecosystem and dependency injection</li>
<li>Powerful applications in patterns like API gateways with YARP</li>
</ul>
<p>As the .NET Aspire stack continues to evolve, its approach to service discovery represents
one of the ways it's making cloud-native development more accessible to .NET developers.
By reducing the complexity of service-to-service communication,
Aspire enables teams to focus on building features rather than wrestling with infrastructure concerns.</p>
<p>If you want to explore more robust service discovery solutions for large-scale distributed systems,
check out my previous article on <a href="service-discovery-in-microservices-with-net-and-consul"><strong>implementing service discovery with Consul</strong></a>.
It provides a complementary approach for scenarios that might require a more traditional service registry.</p>
<p>For those who found the YARP integration particularly interesting, my <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a>
course dives deeper into building scalable applications with YARP as an API gateway.
You'll learn how to leverage these patterns to create maintainable,
evolvable systems regardless of whether you're using microservices or a monolith.</p>
<p>That's all for today.</p>
<p>See you next Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_135.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Options Pattern Validation in ASP.NET Core With FluentValidation]]></title>
            <link>https://milanjovanovic.tech/blog/options-pattern-validation-in-aspnetcore-with-fluentvalidation</link>
            <guid isPermaLink="false">options-pattern-validation-in-aspnetcore-with-fluentvalidation</guid>
            <pubDate>Sat, 22 Mar 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Elevate your ASP.NET Core configuration with FluentValidation integration that catches configuration errors at startup, preventing silent failures and runtime exceptions with more expressive validation rules than Data Annotations.]]></description>
            <content:encoded><![CDATA[<p>If you've worked with the <a href="how-to-use-the-options-pattern-in-asp-net-core-7"><strong>Options Pattern</strong></a> in <a href="http://ASP.NET">ASP.NET</a> Core,
you're likely familiar with the built-in validation using <a href="https://learn.microsoft.com/en-us/aspnet/core/mvc/models/validation?view=aspnetcore-9.0#validation-attributes">Data Annotations</a>.
While functional, Data Annotations can be limiting for complex validation scenarios.</p>
<p>The <strong>Options Pattern</strong> lets you use classes to obtain strongly typed configuration objects at runtime.</p>
<p>The problem? You can't be certain that the configuration is valid until you try to use it.</p>
<p>So why not validate it at application startup?</p>
<p>In this article, we'll explore how to integrate the more powerful <a href="https://docs.fluentvalidation.net/en/latest/">FluentValidation</a>
library with <a href="http://ASP.NET">ASP.NET</a> Core's <a href="https://learn.microsoft.com/en-us/aspnet/core/fundamentals/configuration/options">Options Pattern</a>,
to build a robust validation solution that executes at application startup.</p>
<h2>Why FluentValidation Over Data Annotations?</h2>
<p>Data Annotations work well for simple validations, but FluentValidation offers several advantages:</p>
<ul>
<li>More expressive and flexible validation rules</li>
<li>Better support for complex conditional validations</li>
<li>Cleaner separation of concerns (validation logic separate from model)</li>
<li>Easier testing of validation rules</li>
<li>Better support for custom validation logic</li>
<li>Allows for injecting dependencies into validators</li>
</ul>
<h2>Understanding the Options Pattern Lifecycle</h2>
<p>Before diving deep into validation, it's important to understand the lifecycle of options in <a href="http://ASP.NET">ASP.NET</a> Core:</p>
<ul>
<li>Options are registered with the DI container</li>
<li>Configuration values are bound to options classes</li>
<li>Validation occurs (if configured)</li>
<li>Options are resolved when requested via <code>IOptions&lt;T&gt;</code>, <code>IOptionsSnapshot&lt;T&gt;</code>, or <code>IOptionsMonitor&lt;T&gt;</code></li>
</ul>
<p>The <code>ValidateOnStart()</code> method forces validation to occur during application startup rather than when options are first resolved.</p>
<h2>Common Configuration Failures Without Validation</h2>
<p>Without validation, configuration issues can manifest in several ways:</p>
<ul>
<li><strong>Silent failures</strong>: An incorrectly configured option may result in default values being used without warning</li>
<li><strong>Runtime exceptions</strong>: Configuration issues may only surface when the application tries to use invalid values</li>
<li><strong>Cascading failures</strong>: One misconfigured component can cause failures in dependent systems</li>
</ul>
<p>By validating at startup, you create a fast feedback loop that prevents these issues.</p>
<h2>Setting Up the Foundation</h2>
<p>First, let's add the FluentValidation package to our project:</p>
<pre><code class="language-powershell">Install-Package FluentValidation # base package
Install-Package FluentValidation.DependencyInjectionExtensions # for DI integration
</code></pre>
<p>For our example, we'll use a <code>GitHubSettings</code> class that requires validation:</p>
<pre><code class="language-csharp">public class GitHubSettings
{
    public const string ConfigurationSection = &quot;GitHubSettings&quot;;

    public string BaseUrl { get;init; }
    public string AccessToken { get; init; }
    public string RepositoryName { get; init; }
}
</code></pre>
<h2>Creating a FluentValidation Validator</h2>
<p>Next, we'll create a validator for our settings class:</p>
<pre><code class="language-csharp">public class GitHubSettingsValidator : AbstractValidator&lt;GitHubSettings&gt;
{
    public GitHubSettingsValidator()
    {
        RuleFor(x =&gt; x.BaseUrl).NotEmpty();

        RuleFor(x =&gt; x.BaseUrl)
            .Must(baseUrl =&gt; Uri.TryCreate(BaseUrl, UriKind.Absolute, out _))
            .When(x =&gt; !string.IsNullOrWhiteSpace(x.baseUrl))
            .WithMessage($&quot;{nameof(GitHubSettings.BaseUrl)} must be a valid URL&quot;);

        RuleFor(x =&gt; x.AccessToken)
            .NotEmpty();

        RuleFor(x =&gt; x.RepositoryName)
            .NotEmpty();
    }
}
</code></pre>
<h2>Building the FluentValidation Integration</h2>
<p>To integrate FluentValidation with the Options Pattern, we need to create a custom <code>IValidateOptions&lt;T&gt;</code> implementation:</p>
<pre><code class="language-csharp">using FluentValidation;
using Microsoft.Extensions.Options;

public class FluentValidateOptions&lt;TOptions&gt;
    : IValidateOptions&lt;TOptions&gt;
    where TOptions : class
{
    private readonly IServiceProvider _serviceProvider;
    private readonly string? _name;

    public FluentValidateOptions(IServiceProvider serviceProvider, string? name)
    {
        _serviceProvider = serviceProvider;
        _name = name;
    }

    public ValidateOptionsResult Validate(string? name, TOptions options)
    {
        if (_name is not null &amp;&amp; _name != name)
        {
            return ValidateOptionsResult.Skip;
        }

        ArgumentNullException.ThrowIfNull(options);

        using var scope = _serviceProvider.CreateScope();

        var validator = scope.ServiceProvider.GetRequiredService&lt;IValidator&lt;TOptions&gt;&gt;();

        var result = validator.Validate(options);
        if (result.IsValid)
        {
            return ValidateOptionsResult.Success;
        }

        var type = options.GetType().Name;
        var errors = new List&lt;string&gt;();

        foreach (var failure in result.Errors)
        {
            errors.Add($&quot;Validation failed for {type}.{failure.PropertyName} &quot; +
                       $&quot;with the error: {failure.ErrorMessage}&quot;);
        }

        return ValidateOptionsResult.Fail(errors);
    }
}
</code></pre>
<p>A few important notes about this implementation:</p>
<ol>
<li>We create a scoped service provider to properly resolve the validator (since validators are typically registered as scoped services)</li>
<li>We handle named options through the <code>_name</code> property</li>
<li>We build informative error messages that include the property name and error message</li>
</ol>
<h2>How the FluentValidation Integration Works</h2>
<p>When adding our custom FluentValidation integration, it's helpful to understand how it connects to <a href="http://ASP.NET">ASP.NET</a> Core's options system:</p>
<ol>
<li>The <code>IValidateOptions&lt;T&gt;</code> interface is the hook that <a href="http://ASP.NET">ASP.NET</a> Core provides for options validation</li>
<li>Our <code>FluentValidateOptions&lt;T&gt;</code> class implements this interface to bridge to FluentValidation</li>
<li>When <code>ValidateOnStart()</code> is called, <a href="http://ASP.NET">ASP.NET</a> Core resolves all <code>IValidateOptions&lt;T&gt;</code> implementations and runs them</li>
<li>If validation fails, an <code>OptionsValidationException</code> is thrown, preventing the application from starting</li>
</ol>
<h2>Creating Extension Methods for Easy Integration</h2>
<p>Now, let's create a few extension methods to make our validation easier to use:</p>
<pre><code class="language-csharp">public static class OptionsBuilderExtensions
{
    public static OptionsBuilder&lt;TOptions&gt; ValidateFluentValidation&lt;TOptions&gt;(
        this OptionsBuilder&lt;TOptions&gt; builder)
        where TOptions : class
    {
        builder.Services.AddSingleton&lt;IValidateOptions&lt;TOptions&gt;&gt;(
            serviceProvider =&gt; new FluentValidateOptions&lt;TOptions&gt;(
                serviceProvider,
                builder.Name));

        return builder;
    }
}
</code></pre>
<p>This extension method allows us to call <code>.ValidateFluentValidation()</code> when configuring options, similar to the built-in <code>.ValidateDataAnnotations()</code> method.</p>
<p>For even more convenience, we can create another extension method to simplify the entire configuration process:</p>
<pre><code class="language-csharp">public static class ServiceCollectionExtensions
{
    public static IServiceCollection AddOptionsWithFluentValidation&lt;TOptions&gt;(
        this IServiceCollection services,
        string configurationSection)
        where TOptions : class
    {
        services.AddOptions&lt;TOptions&gt;()
            .BindConfiguration(configurationSection)
            .ValidateFluentValidation() // Configure FluentValidation validation
            .ValidateOnStart(); // Validate options on application start

        return services;
    }
}
</code></pre>
<h2>Registering and Using the Validation</h2>
<p>There are a few ways to use our FluentValidation integration:</p>
<h3>Option 1: Standard Registration with Manual Validator Registration</h3>
<pre><code class="language-csharp">// Register the validator
builder.Services.AddScoped&lt;IValidator&lt;GitHubSettings&gt;, GitHubSettingsValidator&gt;();

// Configure options with validation
builder.Services.AddOptions&lt;GitHubSettings&gt;()
    .BindConfiguration(GitHubSettings.ConfigurationSection)
    .ValidateFluentValidation() // Configure FluentValidation validation
    .ValidateOnStart();
</code></pre>
<h3>Option 2: Using the Convenience Extension Method</h3>
<pre><code class="language-csharp">// Register the validator
builder.Services.AddScoped&lt;IValidator&lt;GitHubSettings&gt;, GitHubSettingsValidator&gt;();

// Use the convenience extension
builder.Services.AddOptionsWithFluentValidation&lt;GitHubSettings&gt;(GitHubSettings.ConfigurationSection);
</code></pre>
<h3>Option 3: Automatic Validator Registration</h3>
<p>If you have many validators and want to register them all at once, you can use FluentValidation's assembly scanning:</p>
<pre><code class="language-csharp">// Register all validators from assembly
builder.Services.AddValidatorsFromAssembly(typeof(Program).Assembly);

// Use the convenience extension
builder.Services.AddOptionsWithFluentValidation&lt;GitHubSettings&gt;(GitHubSettings.ConfigurationSection);
</code></pre>
<h2>What Happens at Runtime?</h2>
<p>With <code>.ValidateOnStart()</code>, the application will throw an exception during startup if any validation rules fail.
For example, if your <code>appsettings.json</code> is missing the required <code>AccessToken</code>, you'll see something like:</p>
<pre><code>Microsoft.Extensions.Options.OptionsValidationException:
    Validation failed for GitHubSettings.AccessToken with the error: 'Access Token' must not be empty.
</code></pre>
<p>This prevents your application from even starting with invalid configuration, ensuring issues are caught as early as possible.</p>
<h2>Working with Different Configuration Sources</h2>
<p><a href="http://ASP.NET">ASP.NET</a> Core's configuration system supports multiple sources.
When using the Options Pattern with FluentValidation, remember that validation works regardless of the source:</p>
<ul>
<li>Environment variables</li>
<li>Azure Key Vault</li>
<li>User secrets</li>
<li>JSON files</li>
<li>In-memory configuration</li>
</ul>
<p>This is particularly useful for containerized applications where configuration comes from environment variables or mounted secrets.</p>
<h2>Testing Your Validators</h2>
<p>One benefit of using FluentValidation is that validators are easy to test:</p>
<pre><code class="language-csharp">// Uses helper methods from FluentValidation.TestHelper
[Fact]
public void GitHubSettings_WithMissingAccessToken_ShouldHaveValidationError()
{
    // Arrange
    var validator = new GitHubSettingsValidator();
    var settings = new GitHubSettings { RepositoryName = &quot;test-repo&quot; };

    // Act
    TestValidationResult&lt;GitHubSettings&gt;? result = await validator.TestValidate(settings);

    // Assert
    result.ShouldHaveValidationErrorFor(x =&gt; x.BaseUrl);
    result.ShouldHaveValidationErrorFor(x =&gt; x.AccessToken);
}
</code></pre>
<h2>Summary</h2>
<p>By combining FluentValidation with the Options Pattern and <code>ValidateOnStart()</code>,
we create a powerful <a href="cqrs-validation-with-mediatr-pipeline-and-fluentvalidation"><strong>validation system</strong></a> that ensures our application has correct configuration at startup.</p>
<p>This approach:</p>
<ol>
<li>Provides more expressive validation rules than Data Annotations</li>
<li>Separates validation logic from configuration models</li>
<li>Catches configuration errors at application startup</li>
<li>Supports complex validation scenarios</li>
<li>Is easily testable</li>
</ol>
<p>This pattern is particularly valuable in <a href="monolith-to-microservices-how-a-modular-monolith-helps"><strong>microservice architectures</strong></a> or containerized applications
where configuration errors should be detected immediately rather than at runtime.</p>
<p>Remember to register your validators appropriately and use <code>.ValidateOnStart()</code> to ensure validation happens during application startup.</p>
<p>That's all for today.</p>
<p>See you next Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_134.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Streamlining .NET 9 Deployment With GitHub Actions and Azure]]></title>
            <link>https://milanjovanovic.tech/blog/streamlining-dotnet-9-deployment-with-github-actions-and-azure</link>
            <guid isPermaLink="false">streamlining-dotnet-9-deployment-with-github-actions-and-azure</guid>
            <pubDate>Sat, 15 Mar 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Build a robust, automated CI/CD pipeline for .NET 9 applications using GitHub Actions and Azure App Service to transform deployments from stressful events into reliable, repeatable processes.]]></description>
            <content:encoded><![CDATA[<p>I remember the days of deploying .NET applications by hand: publishing locally, copying files to servers, running scripts,
and crossing my fingers that nothing would break.
It was stressful, time-consuming, and honestly, a bit scary.</p>
<p>But those days are over.</p>
<p>After implementing <a href="https://en.wikipedia.org/wiki/CI/CD">CI/CD</a> pipelines for dozens of projects,
I've seen firsthand how automation transforms the deployment process from a dreaded chore into a reliable, even boring, part of development.</p>
<p>And boring deployments are good deployments.</p>
<p>In this article, I'll walk you through setting up a robust CI/CD pipeline for .NET 9 applications using
<a href="https://github.com/features/actions">GitHub Actions</a> and <a href="https://azure.microsoft.com/en-us/products/app-service">Azure App Service</a>.
I'll cover:</p>
<ul>
<li>What CI/CD is and why it matters for .NET developers</li>
<li>A complete workflow that builds, tests, and deploys your application</li>
<li>How to extend your pipeline with database migrations, code coverage, and more</li>
<li>Practical tips I've learned from real-world deployments</li>
</ul>
<p>Whether you're tired of manual deployments or looking to improve your existing automation,
this guide will help you build a robust CI/CD pipeline that you can easily extend to fit your needs.</p>
<h2>What is CI/CD and Why Should You Care?</h2>
<p>CI/CD stands for <strong>Continuous Integration</strong> and <strong>Continuous Delivery/Deployment</strong>.</p>
<p>In simple terms:</p>
<ul>
<li><strong>Continuous Integration (CI)</strong> means frequently merging code changes and running automated tests to catch issues early</li>
<li><strong>Continuous Delivery (CD)</strong> means getting those changes to production-ready environments quickly and safely</li>
<li><strong>Continuous Deployment (CD)</strong> is an extension of Continuous Delivery where every change that passes automated tests is deployed to production automatically</li>
</ul>
<p>The main benefits I've seen:</p>
<ol>
<li><strong>Faster feedback</strong>: Find bugs within minutes instead of days</li>
<li><strong>More stable releases</strong>: Small, incremental changes are easier to fix</li>
<li><strong>Time savings</strong>: Let automation handle repetitive tasks while you focus on writing code</li>
<li><strong>Consistent deployment</strong>: No more &quot;it works on my machine&quot; problems</li>
</ol>
<h2>My GitHub Actions Workflow for .NET 9</h2>
<p>Here's the workflow I use to deploy a simple time service API to Azure App Service:</p>
<pre><code class="language-yaml"># Name of the workflow as it appears in GitHub Actions UI
name: Time Service CI

# Define when this workflow will run
on:
  workflow_dispatch: # Allow manual triggering from GitHub UI
  push:
    branches:
      - main # Run automatically when code is pushed to main branch

# Environment variables used throughout the workflow
env:
  AZURE_WEBAPP_NAME: time-service
  AZURE_WEBAPP_PACKAGE_PATH: './Time.Api/publish'
  DOTNET_VERSION: '9.x'
  SOLUTION_PATH: 'Time.Api.sln'
  API_PROJECT_PATH: 'Time.Api'
  PUBLISH_DIR: './publish'

# Define the separate jobs that make up this workflow
jobs:
  # First job: build and test the application
  build-and-test:
    name: Build and Test
    runs-on: ubuntu-latest # Use Ubuntu runner for this job

    steps:
      - uses: actions/checkout@v4

      - name: Setup .NET
        uses: actions/setup-dotnet@v4
        with:
          dotnet-version: ${{ env.DOTNET_VERSION }}

      - name: Restore
        run: dotnet restore ${{ env.SOLUTION_PATH }}

      - name: Build
      run: dotnet build ${{ env.SOLUTION_PATH }}
        --configuration Release
        --no-restore

    - name: Test
      run: dotnet test ${{ env.SOLUTION_PATH }}
        --configuration Release
        --no-restore
        --no-build
        --verbosity normal


    - name: Publish
      run: dotnet publish ${{ env.API_PROJECT_PATH }}
        --configuration Release
        --no-restore
        --no-build
        --property:PublishDir=${{ env.PUBLISH_DIR }}

    # Store the published output as an artifact for later jobs
    - name: Publish Artifacts
      uses: actions/upload-artifact@v4
      with:
        name: webapp  # Name of the artifact
        path: ${{ env.AZURE_WEBAPP_PACKAGE_PATH }}

  # Second job: deploy the application to Azure
  deploy:
    name: Deploy to Azure
    runs-on: ubuntu-latest
    needs: [build-and-test] # This job depends on the build-and-test job

    steps:
      # Retrieve the artifacts from the build job
      - name: Download artifact from build job
        uses: actions/download-artifact@v4
        with:
          name: webapp
          path: ${{ env.AZURE_WEBAPP_PACKAGE_PATH }}

      # Deploy to Azure App Service using publish profile credentials
      - name: Deploy
        uses: azure/webapps-deploy@v2
        with:
          app-name: ${{ env.AZURE_WEBAPP_NAME }}
          # Authentication credentials stored as a secret
          publish-profile: ${{ secrets.AZURE_WEBAPP_PUBLISH_PROFILE }}
          package: '${{ env.AZURE_WEBAPP_PACKAGE_PATH }}'
</code></pre>
<p>This workflow does two main things: it builds and tests the code and then deploys it to Azure.</p>
<p>The first job checks out our repository, sets up .NET 9, and runs through a standard build process:
restore packages, build the solution, run tests, and publish the application.
The detailed comments in the YAML explain each step.
Once everything passes, it packages the application as an artifact for the next job.</p>
<p>The second job takes that artifact and deploys it to Azure App Service using a publish profile.
I store the publish profile as a <a href="https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions">GitHub secret</a> for security.
The <code>needs: [build-and-test]</code> line ensures deployment only happens if all tests pass, which protects our production environment from broken code.</p>
<figure className="figure-center">
  <div className="bordered">
    ![An example of what a workflow run looks like from the GitHub UI.](/blogs/mnw_133/ci_cd_pipeline.png)
  </div>
  <figcaption>
    Here's an example of what a workflow run looks like from the GitHub UI.
  </figcaption>
</figure>
<h2>Extending Your CI/CD Pipeline</h2>
<p>While the basic workflow gets your application deployed, most real-world projects need more sophisticated pipelines.
As your project grows, so should your CI/CD process.
Extensions to your pipeline could help catch issues earlier, ensure quality standards, and provide better visibility into your development process.</p>
<p>Here are some valuable additions to consider:</p>
<h3>1. Running Database Migrations</h3>
<p>Database schema changes can be tricky to coordinate with code deployments.
There are several approaches to handling this:</p>
<p><strong>Using EF Core Migration Bundles</strong>:</p>
<pre><code class="language-yaml">- name: Create migration bundle
  run: dotnet ef migrations bundle --project ${{ env.DATA_PROJECT }} --output ${{ env.MIGRATIONS_BUNDLE }}

- name: Apply migrations
  run: ${{ env.MIGRATIONS_BUNDLE }}
</code></pre>
<p><a href="efcore-migrations-a-detailed-guide"><strong>Migration bundles</strong></a> (introduced in EF Core 6.0) package your migrations into a standalone executable,
making them easier to run in deployment pipelines.</p>
<p><strong>Adding Manual Review for Migrations</strong>:</p>
<pre><code class="language-yaml">deploy-database:
  name: Deploy Database Changes
  environment: production
  runs-on: ubuntu-latest
  needs: [build-and-test]
</code></pre>
<p>This approach adds an environment with protection rules, requiring a DBA to review and approve migration scripts before they run.
This is safer for production databases with valuable data.</p>
<p><strong>Pros</strong>:</p>
<ul>
<li>No manual migration steps</li>
<li>Schema and code changes deploy together</li>
<li>Database changes are versioned with code</li>
</ul>
<p><strong>Cons</strong>:</p>
<ul>
<li>Failed migrations can be hard to roll back</li>
<li>Might need extra handling for production data</li>
<li>Requires secure database credentials in CI</li>
</ul>
<p>To minimize risks, I test migrations in a staging environment first and always back up production databases before deployment.</p>
<h3>2. Code Coverage Reports</h3>
<p>I like knowing how much of my code is covered by tests.
Here's an example of how to generate and publish code coverage reports to <a href="https://about.codecov.io/">Codecov</a>:</p>
<pre><code class="language-yaml">- name: Generate coverage report
  run: dotnet test ${{ env.SOLUTION_PATH }} /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura

- name: Publish coverage report
  uses: codecov/codecov-action@v5
  with:
    files: ./**/coverage.cobertura.xml
    fail_ci_if_error: true
    token: ${{ secrets.CODECOV_TOKEN }}
</code></pre>
<p>Adding minimum coverage requirements prevents drops in test coverage and encourages the team to maintain quality standards.
You can also configure it to fail builds when coverage falls below a threshold.</p>
<h3>3. Multi-Environment Deployment</h3>
<p>For larger projects, deploying to multiple environments with approval gates provides better control:</p>
<pre><code class="language-yaml">deploy-staging:
  name: Deploy to Staging
  environment: staging
  runs-on: ubuntu-latest
  needs: [build-and-test]
  steps:
    # Deployment steps...

deploy-production:
  name: Deploy to Production
  environment: production
  runs-on: ubuntu-latest
  needs: [deploy-staging]
  steps:
    # Deployment steps...
</code></pre>
<p>Adding protection rules to your production environment creates checkpoints where team members can verify changes before they reach users.</p>
<p>Here's an example of some GitHub Environment protection rules:</p>
<ul>
<li><strong>Required reviewers</strong>: Specify team members who must approve deployments</li>
<li><strong>Wait timers</strong>: Add a delay before deployments to give time for review</li>
<li><strong>Deployment branches</strong>: Restrict which branches can deploy to production</li>
</ul>
<p>These guardrails are especially important for critical environments where downtime can be costly.</p>
<h2>Final Thoughts</h2>
<p>A good CI/CD pipeline evolves with your project.
Start simple, focus on automating the most painful manual tasks first, then gradually add more features as needed.</p>
<p>The initial setup takes time, but the long-term benefits are huge.
My team now deploys multiple times per day instead of once every few weeks, with fewer bugs reaching production.</p>
<p>If you want to learn more about building robust APIs that complement your CI/CD process, check out my <a href="/pragmatic-rest-apis"><strong>Pragmatic REST APIs</strong></a> course.
It covers designing, implementing, and deploying production-ready APIs with best practices that work perfectly with the deployment pipeline we've discussed here.</p>
<p>What's your CI/CD setup like?
I'd love to hear how you've customized your workflows for .NET applications.</p>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_133.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Better Request Tracing with User Context in ASP.NET Core]]></title>
            <link>https://milanjovanovic.tech/blog/better-request-tracing-with-user-context-in-asp-net-core</link>
            <guid isPermaLink="false">better-request-tracing-with-user-context-in-asp-net-core</guid>
            <pubDate>Sat, 08 Mar 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Adding user context to request tracing in ASP.NET Core helps track issues and understand user behavior. This article shows how to implement middleware that enriches logs with user IDs for better troubleshooting and performance monitoring.]]></description>
            <content:encoded><![CDATA[<p>When building web applications, knowing what's happening behind the scenes is crucial.
In <a href="http://ASP.NET">ASP.NET</a> Core, we can make our lives easier by adding user context to our request tracing.
This helps us track issues, understand user behavior, and improve our applications.</p>
<p>Let me show you how to enhance your request tracing by adding user context in <a href="http://ASP.NET">ASP.NET</a> Core applications.</p>
<h2>Why Add User Context to Request Tracing?</h2>
<p>When something goes wrong in your application, having the user's ID in your logs makes it much easier to figure out what happened.
Instead of searching through thousands of log entries, you can filter by user ID and see only the relevant logs.</p>
<p>Adding user context also helps with:</p>
<ul>
<li>Tracking user journeys through your application</li>
<li>Identifying patterns in user behavior</li>
<li>Troubleshooting issues for specific users</li>
<li>Monitoring performance for different user segments</li>
</ul>
<h2>Implementing User Context Enrichment</h2>
<p>The core of our solution is a middleware component that extracts the user ID from the current user's claims and adds it to the current <strong>activity</strong> and <strong>logging scope</strong>.</p>
<p>A <strong>logging scope</strong> in <a href="http://ASP.NET">ASP.NET</a> Core lets you attach additional data to all log messages created within that scope.
<a href="structured-logging-in-asp-net-core-with-serilog"><strong>Structured logging</strong></a> frameworks like <a href="5-serilog-best-practices-for-better-structured-logging"><strong>Serilog</strong></a>
and the built-in logger support this feature.
For example, if you add a user ID to a logging scope, every log message within that scope will include that user ID,
even if the log message itself doesn't mention the user.
This makes it easy to correlate logs for a specific user across different parts of your application.</p>
<p>The <code>Activity</code> class is part of the .NET diagnostics infrastructure.
It represents a unit of work or operation and is designed for distributed tracing across service boundaries.
When you add a tag to <code>Activity.Current</code>, that information becomes part of the trace and can be used to filter and analyze requests in your monitoring systems.</p>
<p>Here's the code:</p>
<pre><code class="language-csharp">using System.Diagnostics;
using System.Security.Claims;

namespace MyApp.Middleware;

public sealed class UserContextEnrichmentMiddleware(
    RequestDelegate next,
    ILogger&lt;UserContextEnrichmentMiddleware&gt; logger)
{
    public async Task InvokeAsync(HttpContext context)
    {
        string? userId = context.User?.FindFirstValue(ClaimTypes.NameIdentifier);
        if (userId is not null)
        {
            Activity.Current?.SetTag(&quot;user.id&quot;, userId);

            var data = new Dictionary&lt;string, object&gt;
            {
                [&quot;UserId&quot;] = userId
            };

            using (logger.BeginScope(data))
            {
                await next(context);
            }
        }
        else
        {
            await next(context);
        }
    }
}
</code></pre>
<p>Let's break down what this middleware does:</p>
<ol>
<li>It extracts the user ID from the authenticated user's claims using <code>context.User?.FindFirstValue(ClaimTypes.NameIdentifier)</code></li>
<li>If a user ID is found, it adds the ID as a tag to the current activity with <code>Activity.Current?.SetTag(&quot;user.id&quot;, userId)</code></li>
<li>It creates a logging scope using <code>ILogger.BeginScope</code> with the user ID, which means all log entries within this scope will include the user ID</li>
<li>It calls the next middleware in the pipeline</li>
<li>If no user ID is found (for anonymous requests), it simply calls the next middleware</li>
</ol>
<h2>Logging Scopes and OpenTelemetry</h2>
<p>If you're using <a href="introduction-to-distributed-tracing-with-opentelemetry-in-dotnet"><strong>OpenTelemetry</strong></a> to export logs,
you have to configure the provider to include the log scopes on the generated log records.</p>
<p>Here's how:</p>
<pre><code class="language-csharp">builder.Logging.AddOpenTelemetry(options =&gt;
{
    options.IncludeScopes = true;
    options.IncludeFormattedMessage = true;
});
</code></pre>
<h2>Adding the Middleware to Your Application</h2>
<p>To use this <a href="3-ways-to-create-middleware-in-asp-net-core"><strong>middleware</strong></a> in your <a href="http://ASP.NET">ASP.NET</a> Core application, add it to your pipeline in the <code>Program.cs</code> file:</p>
<pre><code class="language-csharp">app.UseAuthentication();
app.UseAuthorization();

// Add the user context enrichment middleware after authentication
app.UseMiddleware&lt;UserContextEnrichmentMiddleware&gt;();

app.MapControllers();

app.Run();
</code></pre>
<p>Make sure to place it after the authentication and authorization middleware so that the user identity is available.</p>
<h2>Handling PII Concerns</h2>
<p>When adding user IDs to your logs and traces, you need to be careful about <a href="https://en.wikipedia.org/wiki/Personal_data">Personally Identifiable Information</a> (PII).
Here are some important points to consider:</p>
<ul>
<li>User IDs should be opaque identifiers (like GUIDs) that don't reveal personal information</li>
<li>Avoid logging email addresses, names, or other personal data</li>
<li>Make sure your logging configuration doesn't send PII to systems where it shouldn't go</li>
</ul>
<p>If you need to comply with data protection regulations, consider implementing log retention policies and the ability to purge user data from logs when needed.</p>
<h2>Expanding Context Enrichment</h2>
<p>User IDs are just the beginning.
We can add more context to make our logs and traces even more useful:</p>
<h3>Feature Flags</h3>
<p><a href="feature-flags-in-dotnet-and-how-i-use-them-for-ab-testing"><strong>Feature flags</strong></a> help us roll out new features gradually or enable them for specific users.
Adding feature flag information to our context gives us valuable insights:</p>
<pre><code class="language-csharp">// Inside the middleware
if (featureFlagService.IsEnabled(&quot;NewFeature&quot;, userId))
{
    Activity.Current?.SetTag(&quot;features.newfeature&quot;, &quot;enabled&quot;);
    // Add to logging scope as well
}
</code></pre>
<h3>Tenant Information for Multi-tenant Applications</h3>
<p>If your application serves <a href="multi-tenant-applications-with-ef-core"><strong>multiple tenants</strong></a>, adding tenant IDs is extremely helpful:</p>
<pre><code class="language-csharp">string? tenantId = context.User?.FindFirstValue(&quot;TenantId&quot;);
if (tenantId is not null)
{
    Activity.Current?.SetTag(&quot;tenant.id&quot;, tenantId);
    // Add to logging scope
}
</code></pre>
<h2>Takeaway</h2>
<p>Adding user context to your request tracing in <a href="http://ASP.NET">ASP.NET</a> Core is a simple but powerful technique.
By implementing the middleware we've explored, you'll see several important benefits:</p>
<ol>
<li>Faster troubleshooting - when users report issues, you can quickly find relevant logs</li>
<li>Better understanding of usage patterns - see which features are being used and by whom</li>
<li>Improved performance monitoring - identify slow requests for specific user segments</li>
<li>More effective A/B testing - track metrics for users with different feature flags</li>
</ol>
<p>Understanding logging scopes and the <code>Activity</code> class helps you get the most out of this technique.
Logging scopes ensure your log entries contain consistent contextual information,
while activities enable distributed tracing across service boundaries.
Remember to be careful with PII and make sure your logging practices comply with relevant regulations.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_132.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Introduction to Dapr for .NET Developers]]></title>
            <link>https://milanjovanovic.tech/blog/introduction-to-dapr-for-dotnet-developers</link>
            <guid isPermaLink="false">introduction-to-dapr-for-dotnet-developers</guid>
            <pubDate>Sat, 01 Mar 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Explore how Dapr helps .NET developers build better microservices with standardized building blocks, practical code examples, and seamless integration with .NET Aspire to simplify your distributed systems journey.]]></description>
            <content:encoded><![CDATA[<p>Building distributed systems has never been more important—or more challenging.
As .NET developers, we're constantly juggling <a href="service-discovery-in-microservices-with-net-and-consul">service discovery</a>,
state management,
<a href="simple-messaging-in-dotnet-with-redis-pubsub">messaging</a>,
<a href="building-resilient-cloud-applications-with-dotnet">resilience patterns</a>,
and various infrastructure SDKs.
Our business logic gets buried under mountains of plumbing code, and we become tightly coupled to specific technologies.</p>
<p>What if there was a better way?</p>
<p>Enter <a href="https://dapr.io/">Dapr</a> (Distributed Application Runtime), an open-source project that handles the complex infrastructure challenges so you can focus on what matters most:
your application's business logic.</p>
<p>In this article, we'll explore how Dapr transforms microservice development for .NET developers by:</p>
<ul>
<li>Abstracting away infrastructure-specific code behind consistent APIs</li>
<li>Providing standardized building blocks for common distributed system patterns</li>
<li>Enabling you to use the same code from local development to production</li>
<li>Integrating seamlessly with .NET and <a href="http://ASP.NET">ASP.NET</a> Core applications</li>
</ul>
<p>Whether you're building your first microservice or evolving a complex system, Dapr offers a simpler path forward.
Let's explore how it works.</p>
<h2>What is Dapr?</h2>
<p><a href="https://dapr.io/">Dapr</a> is a portable, event-driven runtime that simplifies building microservices.
As a graduated project within the <a href="https://www.cncf.io/">Cloud Native Computing Foundation</a> (CNCF), Dapr has proven its value in production environments.</p>
<p>At its core, Dapr provides standardized building blocks that abstract away the complexity of common microservice patterns.
Rather than wrestling with infrastructure-specific code, you can focus on your business logic while Dapr handles the rest.</p>
<p>Before Dapr, building a microservice architecture in .NET might require direct integration with multiple infrastructure components:</p>
<pre><code class="language-csharp">// Pre-Dapr approach - direct infrastructure dependencies
builder.Services.AddStackExchangeRedisCache(options =&gt; { options.Configuration = &quot;redis:6379&quot;; });

builder.Services.AddSingleton&lt;IMessageBroker&gt;(provider =&gt; new KafkaMessageBroker(&quot;kafka:9092&quot;));

builder.Services.AddSingleton&lt;ISecretManager&gt;(provider =&gt;
    new AzureKeyVaultClient(new Uri(&quot;https://myvault.vault.azure.net&quot;)));
</code></pre>
<p>This approach tightly couples your application to specific technologies.</p>
<p>With Dapr, you gain flexibility through standardized APIs:</p>
<pre><code class="language-csharp">// Dapr approach - simple, consistent APIs
builder.Services.AddDaprClient();

// Later in code:
// State management (could be Redis, Cosmos DB, etc.)
await daprClient.SaveStateAsync(&quot;statestore&quot;, &quot;customer-123&quot;, customerData);

// Pub/sub (could be Kafka, RabbitMQ, etc.)
await daprClient.PublishEventAsync(&quot;pubsub&quot;, &quot;orders&quot;, orderData);

// Secrets (could be Azure Key Vault, HashiCorp Vault, etc.)
var secret = await daprClient.GetSecretAsync(&quot;secretstore&quot;, &quot;api-keys&quot;);
</code></pre>
<p>The underlying providers can be swapped without code changes - just by updating Dapr component configuration files.</p>
<h2>The Sidecar Pattern: How Dapr Works</h2>
<p>Dapr uses the <a href="https://learn.microsoft.com/en-us/azure/architecture/patterns/sidecar">sidecar</a> architectural pattern,
where it runs as a separate process alongside your application:</p>
<figure className="figure-center">
  ![Dapr sidecar diagram with the building blocks and services.](/blogs/mnw_131/dapr_sidecar.png)
  <figcaption>Source: [Dapr](https://dapr.io/)</figcaption>
</figure>
<p>Your application communicates with the Dapr sidecar through HTTP or gRPC, and Dapr handles communication with infrastructure services.
This separation brings several benefits:</p>
<ul>
<li><strong>Language Agnostic</strong>: Dapr works with any programming language, including all .NET variants</li>
<li><strong>Cross-cutting Concerns</strong>: Security, observability, and resiliency are handled by Dapr, not your code</li>
<li><strong>Infrastructure Abstraction</strong>: Your application remains decoupled from specific technologies</li>
<li><strong>Simplified Development</strong>: Clean, maintainable code focused on business logic</li>
<li><strong>Production Ready</strong>: Built-in features that improve reliability in production environments</li>
</ul>
<h2>Building Blocks: Dapr's Core Capabilities</h2>
<p>Dapr offers several <a href="https://docs.dapr.io/concepts/building-blocks-concept/">building blocks</a> that solve common microservice challenges.
Each provides a standardized API that abstracts away infrastructure complexity:</p>
<ol>
<li><strong>Service Invocation</strong>: Enables reliable service-to-service communication with automatic service discovery, load balancing, and retries.</li>
<li><strong>State Management</strong>: Provides a unified way to store and retrieve state with features like concurrency control and transactions.</li>
<li><strong>Pub/Sub</strong>: Implements asynchronous messaging between services, allowing for loosely-coupled, event-driven architectures.</li>
<li><strong>Workflows</strong>: Enables you to define long running, persistent processes that span multiple microservices.</li>
<li><strong>Bindings</strong>: Connects your applications to external systems, either for triggering your app from external events or invoking external services.</li>
<li><strong>Actors</strong>: Implements the virtual actor pattern, making it easy to build stateful microservices with encapsulated state and behavior.</li>
<li><strong>Secrets</strong>: Offers secure access to sensitive configuration like connection strings and API keys from various secret stores.</li>
<li><strong>Configuration</strong>: Centralizes application settings with support for dynamic updates across multiple services.</li>
<li><strong>Distributed Lock</strong>: Provides mutually exclusive access to shared resources in a distributed environment.</li>
<li><strong>Cryptography</strong>: Offers encryption and decryption operations while handling key management.</li>
<li><strong>Jobs</strong>: Allows you to schedule and orchestrate jobs (e.g., schedule batch processing jobs to run every business day)</li>
<li><strong>Conversation</strong>: Lets you supply prompts to converse with different large language models (LLMs). Includes prompt caching and PII obfuscation.</li>
</ol>
<p>Here's an overview of Dapr's building blocks and the most popular services they interact with:</p>
<figure className="figure-center">
  ![Dapr components diagram with the base building blocks and services.](/blogs/mnw_131/dapr_components.png)
  <figcaption>Source: [Dapr](https://dapr.io/)</figcaption>
</figure>
<p>Let's explore the most commonly used building blocks in depth.</p>
<h2>Service Invocation</h2>
<p>The <a href="https://docs.dapr.io/developing-applications/building-blocks/service-invocation/service-invocation-overview/">service invocation</a>
building block enables reliable service-to-service communication with automatic mTLS encryption, retries, and observability.</p>
<p>This solves several challenging microservice problems:</p>
<ul>
<li><strong>Service Discovery</strong>: Finding where services are located</li>
<li><strong>Resilient Communication</strong>: Handling transient failures gracefully</li>
<li><strong>Load Balancing</strong>: Distributing requests across multiple instances</li>
<li><strong>Observability</strong>: Tracking requests across service boundaries</li>
<li><strong>Security</strong>: Encrypting traffic between services</li>
</ul>
<figure className="figure-center">
  ![Diagram showing how the service invocation flow looks like with Dapr.](/blogs/mnw_131/dapr_service_invocation.png)
  <figcaption>Source: [Dapr](https://dapr.io/)</figcaption>
</figure>
<p>Here's a simple example of invoking a service using Dapr's .NET SDK:</p>
<pre><code class="language-csharp">// Client application making a request
using Dapr.Client;

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddDaprClient();

var app = builder.Build();

app.MapGet(&quot;/checkout/{itemId}&quot;, async (int itemId, DaprClient daprClient) =&gt;
{
    // Create order data
    var orderData = new OrderData(itemId, DateTime.UtcNow);

    // Invoke the order-processing service
    var result = await daprClient.InvokeMethodAsync&lt;OrderData, string&gt;(
        &quot;order-processor&quot;,
        &quot;process-order&quot;,
        orderData);

    return Results.Ok(new { Message = $&quot;Order {itemId} processed: {result}&quot; });
});

await app.RunAsync();

public record OrderData(int ItemId, DateTime OrderedAt);
</code></pre>
<p>And the corresponding service handling the request:</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

app.MapPost(&quot;/process-order&quot;, (OrderData order) =&gt;
{
    Console.WriteLine($&quot;Processing order {order.ItemId} placed at {order.OrderedAt}&quot;);
    return $&quot;Order {order.ItemId} confirmation: #{Guid.NewGuid().ToString()[..8]}&quot;;
});

await app.RunAsync();

public record OrderData(int ItemId, DateTime OrderedAt);
</code></pre>
<p>What's happening here:</p>
<ul>
<li>The <code>checkout</code> service calls <code>InvokeMethodAsync</code> using the <code>DaprClient</code> to send a request to the <code>order</code> service</li>
<li>The Dapr sidecar for the checkout service receives this request</li>
<li>The Dapr sidecar looks up the location of the order service</li>
<li>The request is forwarded to the Dapr sidecar of the order service</li>
<li>The order service's Dapr sidecar forwards the request to the order service</li>
<li>The response follows the reverse path</li>
</ul>
<p>This process provides automatic service discovery, encryption, retries, and distributed tracing without any additional code.</p>
<h2>Publish &amp; Subscribe</h2>
<p>The <a href="https://docs.dapr.io/developing-applications/building-blocks/pubsub/pubsub-overview/">publish and subscribe</a>
building block provides asynchronous messaging between services with at-least-once delivery guarantees.
This pattern is essential for building resilient, loosely-coupled microservices that can:</p>
<ul>
<li>Process operations asynchronously without blocking the user</li>
<li>Continue functioning when downstream services are unavailable</li>
<li>Scale independently based on workload</li>
</ul>
<figure className="figure-center">
  ![Diagram showing how the publish subscribe flow looks like with Dapr.](/blogs/mnw_131/dapr_publish_subscribe.png)
  <figcaption>Source: [Dapr](https://dapr.io/)</figcaption>
</figure>
<p>Pub/Sub in Dapr follows this flow:</p>
<ul>
<li>A publisher service sends a message to a topic via the Dapr sidecar</li>
<li>The publisher Dapr sidecar converts the message to the <a href="https://cloudevents.io/">CloudEvent</a> format and forwards it to the configured message broker</li>
<li>Subscriber services receive the message through their Dapr sidecars</li>
<li>The subscriber application processes the message</li>
</ul>
<p>Dapr uses component configuration files to define the pub/sub message broker. Here's a typical Redis pub/sub component:</p>
<pre><code class="language-yaml">apiVersion: dapr.io/v1alpha1
kind: Component
metadata:
  name: order-events
spec:
  type: pubsub.redis
  version: v1
  metadata:
    - name: redisHost
      value: localhost:6379
    - name: redisPassword
      value: ''
</code></pre>
<p>The key parts of this configuration are:</p>
<ul>
<li><code>metadata.name</code>: The component name (<code>order-events</code>) that your application will reference when publishing/subscribing</li>
<li><code>spec.type</code>: The type of component (<code>pubsub.redis</code> in this case)</li>
<li><code>spec.metadata</code>: Configuration specific to the component type</li>
</ul>
<p>This file should be placed in a <code>components</code> directory where Dapr can discover it.
When running locally, this is typically <code>./components/</code> relative to your application.</p>
<p>What if you want to switch from Redis to RabbitMQ?
You'd replace the <code>spec.type</code> with <code>pubsub.rabbitmq</code> and update the <code>metadata</code> section accordingly.
This change doesn't require any code modifications in your application.
Isn't this flexibility amazing?</p>
<p>Here's how to publish events using Dapr:</p>
<pre><code class="language-csharp">// Publisher service
using Dapr.Client;

var builder = WebApplication.CreateBuilder(args);
builder.Services.AddDaprClient();

var app = builder.Build();

app.MapPost(&quot;/create-order&quot;, async (OrderRequest request, DaprClient daprClient) =&gt;
{
    var orderEvent = new OrderCreatedEvent(
        request.OrderId,
        request.CustomerId,
        request.Items,
        DateTime.UtcNow
    );

    // Publish event to &quot;orders&quot; topic
    await daprClient.PublishEventAsync(&quot;order-events&quot;, &quot;orders&quot;, orderEvent);

    return Results.Accepted();
});

await app.RunAsync();

public record OrderRequest(Guid OrderId, string CustomerId, List&lt;string&gt; Items);
public record OrderCreatedEvent(Guid OrderId, string CustomerId, List&lt;string&gt; Items, DateTime CreatedAt);
</code></pre>
<p>And here's how a subscriber would handle these events:</p>
<pre><code class="language-csharp">// Subscriber service
using Dapr;
using Microsoft.AspNetCore.OutputCaching;

var builder = WebApplication.CreateBuilder(args);

// Add Dapr event handling
builder.Services.AddDapr();
builder.Services.AddControllers();

var app = builder.Build();

// Required for Dapr pub/sub
app.UseCloudEvents();
app.MapSubscribeHandler();

// Subscribe to &quot;orders&quot; topic
app.MapPost(&quot;/events/orders&quot;, [Topic(&quot;order-events&quot;, &quot;orders&quot;)] async (OrderCreatedEvent orderEvent) =&gt;
{
    Console.WriteLine($&quot;Processing order {orderEvent.OrderId} for customer {orderEvent.CustomerId}&quot;);
    await ProcessOrderAsync(orderEvent);
    return Results.Ok();
});

await app.RunAsync();

async Task ProcessOrderAsync(OrderCreatedEvent orderEvent)
{
    // Process the order
    await Task.Delay(100); // Simulate work
}

public record OrderCreatedEvent(Guid OrderId, string CustomerId, List&lt;string&gt; Items, DateTime CreatedAt);
</code></pre>
<p>The key components:</p>
<ul>
<li>The publisher uses Dapr to send events to a topic</li>
<li>Dapr handles the interaction with the message broker (Kafka, Redis, etc.)</li>
<li>The subscriber decorates endpoints with <code>[Topic]</code> attributes</li>
<li>Dapr delivers the messages to the appropriate subscribers</li>
</ul>
<p>Note that the <code>name</code> defined in the component file (<code>order-events</code> in our example) must match the first parameter used in
<code>PublishEventAsync(&quot;order-events&quot;, ...)</code> and <code>[Topic(&quot;order-events&quot;, ...)]</code>.
If these names don't match exactly, messages won't flow correctly between services.</p>
<h2>Dapr and .NET Aspire: Better Together</h2>
<p><a href="https://www.diagrid.io/blog/net-aspire-dapr-what-are-they-and-how-they-complement-each-other">Dapr works seamlessly with .NET Aspire</a>,
Microsoft's new cloud-ready stack for building distributed applications.
While Aspire focuses on .NET-specific application orchestration, Dapr provides language-agnostic building blocks.</p>
<p>Here's how to integrate Dapr with a .NET Aspire application:</p>
<pre><code class="language-csharp">using CommunityToolkit.Aspire.Hosting.Dapr;

// Program.cs in the Aspire AppHost project
var builder = DistributedApplication.CreateBuilder(args);

// Add Aspire service and configure Dapr
var orderService = builder.AddProject&lt;Projects.OrderService&gt;(&quot;orderservice&quot;)
    .WithDaprSidecar(new DaprSidecarOptions
    {
        AppId = &quot;order-api&quot;,
        Config = &quot;./dapr/config.yaml&quot;,
        ResourcesPaths = [&quot;./dapr/components&quot;]
    });

// Add another service that can communicate with the order service via Dapr
var checkoutService = builder.AddProject&lt;Projects.CheckoutService&gt;(&quot;checkoutservice&quot;)
    .WithDaprSidecar(new DaprSidecarOptions
    {
        AppId = &quot;checkout-api&quot;,
        Config = &quot;./dapr/config.yaml&quot;,
        ResourcesPaths = [&quot;./dapr/components&quot;]
    })
    // Reference the order service by its Dapr app ID
    .WithReference(orderService);

builder.Build().Run();
</code></pre>
<p>Note that I'm using the <code>CommunityToolkit.Aspire.Hosting.Dapr</code> package, which is the official Dapr integration for .NET Aspire.
The <code>Aspire.Hosting.Dapr</code> library is now deprecated.</p>
<p>Here's an example of how a message flow might look in the Aspire dashboard:</p>
<p><img src="/blogs/mnw_131/dapr_aspire_distributed_trace.png" alt="A distributed trace from the Aspire dashboard showing a message flow from the checkout service to the order service."></p>
<h2>Learning with Dapr University</h2>
<p>If you're looking for a structured way to learn Dapr, I highly recommend checking out <a href="https://diagrid.ws/41oIYRX"><strong>Dapr University</strong></a>.
You can run the hands-on lessons completely for free.</p>
<p>As someone who started with limited Dapr experience, I found the &quot;Dapr 101&quot; course particularly valuable.
It provides hands-on exercises for State Management, Service Invocation, and Pub/Sub—exactly what you need to get started quickly.</p>
<p><img src="/blogs/mnw_131/dapr_university.png" alt="Dapr university learning platform."></p>
<h2>Conclusion</h2>
<p>Dapr simplifies microservice development for .NET developers by providing standardized building blocks that handle infrastructure complexity.
With its sidecar pattern, Dapr lets you focus on business logic while it manages cross-cutting concerns.
As you build distributed applications, consider how Dapr can help you:</p>
<ul>
<li>Accelerate development with ready-made patterns</li>
<li>Build more resilient systems with fewer lines of code</li>
<li>Avoid vendor lock-in through abstraction (building blocks)</li>
<li>Improve production reliability with built-in best practices</li>
</ul>
<p>Ready to dive deeper?
Check out <a href="https://diagrid.ws/41oIYRX"><strong>Dapr University</strong></a> for comprehensive courses and hands-on learning.</p>
<p>That's all for today. Hope this was helpful.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_131.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Building a Better MediatR Publisher With Channels (and why you shouldn't)]]></title>
            <link>https://milanjovanovic.tech/blog/building-a-better-mediatr-publisher-with-channels-and-why-you-shouldnt</link>
            <guid isPermaLink="false">building-a-better-mediatr-publisher-with-channels-and-why-you-shouldnt</guid>
            <pubDate>Sat, 22 Feb 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Discover why MediatR's notification publishers block your application, and explore a Channel-based solution before reaching for a message queue.]]></description>
            <content:encoded><![CDATA[<p>I've been meaning to write this article for a while now.
This problem has been bugging me, and I finally found the time to address it.</p>
<p>What problem is that?</p>
<p>Well, it's about MediatR's <a href="how-to-publish-mediatr-notifications-in-parallel">notification publishing</a> mechanism.</p>
<p>MediatR supports simple in-process publish/subscribe capabilities.
This lets you broadcast notifications to multiple handlers without coupling them directly to the publisher.</p>
<p>While MediatR's notification system appears asynchronous at first glance, <strong>it's not</strong>.</p>
<p>By asynchronous, I mean that the publishing thread should not wait for all handlers to complete.
Instead, it should return immediately after queuing the notification for processing.</p>
<p>In this article, we'll understand MediatR's notification publishing mechanics.
We'll use distributed tracing to examine its execution model, and explore alternatives for true asynchronous processing.</p>
<h2>The Notification Publisher</h2>
<p><a href="https://github.com/jbogard/MediatR">MediatR</a> provides two built-in implementations of its <code>INotificationPublisher</code> interface.
They each have distinct characteristics but share one crucial trait: they block the publishing thread until the handlers complete.</p>
<p>Here's the <code>INotificationPublisher</code> interface:</p>
<pre><code class="language-csharp">public interface INotificationPublisher
{
    Task Publish(
        IEnumerable&lt;NotificationHandlerExecutor&gt; handlerExecutors,
        INotification notification,
        CancellationToken cancellationToken);
}
</code></pre>
<p>This interface provides the contract for executing notification handlers, but the execution strategy is left to the implementing classes.</p>
<p>By default, MediatR uses the <code>ForeachAwaitPublisher</code>:</p>
<pre><code class="language-csharp">public class ForeachAwaitPublisher : INotificationPublisher
{
    public async Task Publish(
        IEnumerable&lt;NotificationHandlerExecutor&gt; handlerExecutors,
        INotification notification,
        CancellationToken cancellationToken)
    {
        foreach (var handler in handlerExecutors)
        {
            await handler.HandlerCallback(notification, cancellationToken).ConfigureAwait(false);
        }
    }
}
</code></pre>
<p>This implementation processes handlers sequentially, ensuring a predictable order of execution.</p>
<p>The alternative <code>TaskWhenAllPublisher</code> offers concurrent execution:</p>
<pre><code class="language-csharp">public class TaskWhenAllPublisher : INotificationPublisher
{
    public Task Publish(
        IEnumerable&lt;NotificationHandlerExecutor&gt; handlerExecutors,
        INotification notification,
        CancellationToken cancellationToken)
    {
        var tasks = handlerExecutors
            .Select(handler =&gt; handler.HandlerCallback(notification, cancellationToken))
            .ToArray();

        return Task.WhenAll(tasks);
    }
}
</code></pre>
<p>While this publisher executes handlers concurrently, it's crucial to understand that &quot;concurrent&quot; doesn't mean <a href="building-async-apis-in-aspnetcore-the-right-way">&quot;background processing&quot;</a>.
The publishing thread still waits for all handlers to complete before continuing.</p>
<h2>Proving the Point with OpenTelemetry</h2>
<p>To demonstrate the blocking nature of both publishers, let's set up a simple example with <a href="https://opentelemetry.io/">OpenTelemetry</a> tracing:</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);

// This will register all handlers in the same assembly as the Program class
// We're also configuring the notifcation publisher
builder.Services.AddMediatR(cfg =&gt;
{
    cfg.RegisterServicesFromAssemblyContaining&lt;Program&gt;();

    cfg.NotificationPublisherType = typeof(ForeachAwaitPublisher);
    // or we could say 👇
    // cfg.NotificationPublisherType = typeof(TaskWhenAllPublisher);
});

builder.Services
    .AddOpenTelemetry()
    .ConfigureResource(r =&gt; r.AddService(DiagnosticConfig.Source.Name))
    .WithTracing(tracing =&gt;
        tracing
            .AddAspNetCoreInstrumentation()
            .AddSource(DiagnosticConfig.Source.Name))
    .UseOtlpExporter();

var app = builder.Build();

// Dummy endpoint to trigger the notification
app.MapPost(&quot;orders&quot;, async (IMediator mediator) =&gt;
{
    using var activity = DiagnosticConfig.Source.StartActivity(&quot;CreateOrder&quot;);

    var orderId = Guid.NewGuid();
    // Just publish the notification, we don't care about doing &quot;real&quot; work here
    await mediator.Publish(new OrderCreatedNotification
    {
        OrderId = orderId,
        ParentId = activity?.Id // Propagating the parent activity ID
    });

    return Results.Ok(orderId);
});

app.Run();

// The simple notification class
public class OrderCreatedNotification : INotification
{
    public Guid OrderId { get; set; }
    public string? ParentId { get; set; }
}

// A slow handler to simulate blocking behavior
public class SlowOrderCreatedHandler(ILogger&lt;SlowOrderCreatedHandler&gt; logger)
    : INotificationHandler&lt;OrderCreatedNotification&gt;
{
    public async Task Handle(OrderCreatedNotification notification, CancellationToken token)
    {
        using var activity = DiagnosticConfig.Source.StartActivity(
            &quot;SlowOrderCreatedHandler.Handle&quot;,
            ActivityKind.Internal,
            notification.ParentId);

        await Task.Delay(2000, token); // Simulate work

        logger.LogInformation(
            &quot;Slow handler completed for order {OrderId}&quot;,
            notification.OrderId);
    }
}

// Defines the OpenTelemetry ActivitySource
internal static class DiagnosticConfig
{
    internal static readonly ActivitySource Source = new(&quot;Order.Service&quot;);
}
</code></pre>
<p>When we examine the resulting traces, we'll see that the handler execution spans are contained within the HTTP request span,
indicating that the request thread is blocked until all handlers complete.</p>
<p>Now, let's see how these publishers behave in practice.
I'll add a few more handlers to the mix to make the example more interesting.</p>
<h3>ForeachAwaitPublisher Traces</h3>
<p>You can see the sequential execution of handlers in the trace visualization.
The request span encompasses all handler execution, demonstrating the blocking nature of the <code>ForeachAwaitPublisher</code>.</p>
<p><img src="/blogs/mnw_130/foreachawait_publisher.png" alt="Distributed trace demonstrating notification handling."></p>
<h3>TaskWhenAllPublisher Traces</h3>
<p>Similarly, the <code>TaskWhenAllPublisher</code> shows concurrent handler execution within the request span.
We do get a slight improvement in handler execution time, but the request thread still waits for all handlers to complete before returning.</p>
<p><img src="/blogs/mnw_130/taskwhenall_publisher.png" alt="Distributed trace demonstrating notification handling."></p>
<h2>Building an Async Notification Publisher with Channels</h2>
<p>How can we make MediatR's notification publishing truly asynchronous?</p>
<p>We'll implement a custom <code>INotificationPublisher</code> that leverages <code>System.Threading.Channels</code> for true asynchronous processing.
This implementation will queue notifications for background processing, allowing the publishing thread to return immediately.</p>
<p>Here's the <code>ChannelPublisher</code>:</p>
<pre><code class="language-csharp">// The publisher just queues the notification for processing
public class ChannelPublisher(NotificationsQueue queue) : INotificationPublisher
{
    public async Task Publish(
        IEnumerable&lt;NotificationHandlerExecutor&gt; handlerExecutors,
        INotification notification,
        CancellationToken cancellationToken)
    {
        // Write the message to the channel, and return immediately
        await queue.Writer.WriteAsync(
            new NotificationEntry(handlerExecutors.ToArray(), notification),
            cancellationToken);
    }
}

// It's the Channel that handles the actual message passing
// We can control the capacity and backpressure handling here
public class NotificationsQueue(int capacity = 100)
{
    private readonly Channel&lt;NotificationEntry&gt; _queue =
        Channel.CreateBounded&lt;NotificationEntry&gt;(new BoundedChannelOptions(capacity)
        {
            FullMode = BoundedChannelFullMode.Wait // Backpressure handling
        });

    public ChannelReader&lt;NotificationEntry&gt; Reader =&gt; _queue.Reader;
    public ChannelWriter&lt;NotificationEntry&gt; Writer =&gt; _queue.Writer;
}

// A simple data structure to hold the notification and handlers
public record NotificationEntry(NotificationHandlerExecutor[] Handlers, INotification Notification);

// Program.cs
builder.Services.AddSingleton&lt;NotificationsQueue&gt;();
</code></pre>
<p>But this is just part of the solution.
We need a background service to process the queued notifications:</p>
<pre><code class="language-csharp">// We'll use the NotificationsQueue to read and process notifications
public class ChannelPublisherWorker(NotificationsQueue queue) : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        // Read notifications from the queue and process them
        await foreach (NotificationEntry entry in queue.Reader.ReadAllAsync(stoppingToken))
        {
            // Parallel.ForEachAsync for style points
            await Parallel.ForEachAsync(entry.Handlers, stoppingToken, async (executor, token) =&gt;
            {
                // We're finally executing the handler
                await executor.HandlerCallback(entry.Notification, token);
            });
        }
    }
}

// Program.cs
builser.Services.AddHostedService&lt;ChannelPublisherWorker&gt;();
</code></pre>
<p>This implementation offers several advantages:</p>
<ul>
<li>True background processing - the publisher returns immediately after queueing the notification</li>
<li>Backpressure handling through bounded channel capacity</li>
<li>Independent handler execution</li>
</ul>
<p>To use this publisher, register it with MediatR by setting the <code>NotificationPublisherType</code> to be <code>ChannelPublisher</code>:</p>
<pre><code class="language-csharp">services.AddMediatR(cfg =&gt;
{
    cfg.NotificationPublisherType = typeof(ChannelPublisher);
});
</code></pre>
<p>Let's see how this implementation performs in practice.</p>
<h2>Comparing Approaches With OpenTelemetry</h2>
<p>When we examine the traces with our <code>ChannelPublisher</code>, we'll see a significant difference:</p>
<ol>
<li>The HTTP request span completes quickly after queueing the notification</li>
<li>Handler execution spans appear as separate traces</li>
<li>Overall system responsiveness improves</li>
</ol>
<p><img src="/blogs/mnw_130/channel_publisher.png" alt="Distributed trace demonstrating notification handling."></p>
<p>This visualization clearly demonstrates the non-blocking nature of our implementation.</p>
<p>But is it worth it?</p>
<p>Here's what you should consider first before adopting this approach:</p>
<ul>
<li>The <code>ChannelPublisher</code> introduces additional complexity compared to the built-in publishers</li>
<li>Error handling is your responsibility (e.g., retrying failed handlers, <a href="https://en.wikipedia.org/wiki/Dead_letter_queue">dead-letter queue</a>)</li>
<li>And did I mention <a href="idempotent-consumer-handling-duplicate-messages">idempotent consumers</a>? Yeah... you need those too</li>
<li><a href="lightweight-in-memory-message-bus-using-dotnet-channels">Channels</a> aren't durable - messages are lost if the application crashes</li>
</ul>
<p>Before you know it, you might find yourself reinventing the wheel with a custom message queueing system.</p>
<p>Instead, consider using a real message broker like <a href="https://www.rabbitmq.com/">RabbitMQ</a>.
Combine it with a library like <a href="https://masstransit.io">MassTransit</a> or <a href="https://particular.net/nservicebus">NServiceBus</a> for a robust, scalable, and reliable messaging solution.</p>
<h2>Takeaway</h2>
<p>MediatR's notification system is great for simple in-process pub/sub scenarios.
However, the built-in publishers can become a bottleneck in high-throughput applications due to their blocking nature.</p>
<p>The <code>ChannelPublisher</code> implementation we explored offers true asynchronous processing.
However, it also comes with extra complexity around message handling and delivery guarantees.
Managing message persistence, error handling, retries, and idempotency quickly becomes challenging.</p>
<p>If your application requires these features,
you'll be better off adopting a mature solution like <a href="using-masstransit-with-rabbitmq-and-azure-service-bus">RabbitMQ</a>
or <a href="complete-guide-to-amazon-sqs-and-amazon-sns-with-masstransit">Amazon SQS</a>.</p>
<p>That's all for today. Hope this was helpful.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_130.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Understanding Cursor Pagination and Why It's So Fast (Deep Dive)]]></title>
            <link>https://milanjovanovic.tech/blog/understanding-cursor-pagination-and-why-its-so-fast-deep-dive</link>
            <guid isPermaLink="false">understanding-cursor-pagination-and-why-its-so-fast-deep-dive</guid>
            <pubDate>Sat, 15 Feb 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[While offset pagination is widely used, cursor-based pagination offers significant performance advantages - my tests show a 17x speedup when paginating through a million-record dataset in PostgreSQL. Let's take a deep dive into cursor pagination, compare it with offset pagination, and examine the SQL execution plans.]]></description>
            <content:encoded><![CDATA[<p>Pagination is crucial for efficiently handling large datasets.
While offset pagination is widely used and gets the job done, cursor-based pagination offers some interesting advantages for certain scenarios.</p>
<p>It's particularly valuable for real-time feeds, infinite scroll interfaces, and APIs where performance at scale matters -
like social media timelines, activity logs, or event streams where users frequently page through large datasets.</p>
<p>Let's explore both approaches using a simple <code>UserNotes</code> table and see how they perform with a million records.</p>
<p>We'll look at the implementation details, compare query performance, and discuss where each approach makes the most sense.</p>
<p>I've included real execution plans from <a href="https://www.postgresql.org">PostgreSQL</a> to demonstrate the significant performance differences between these approaches.</p>
<h2>Database Schema</h2>
<p>I created a simple table to demonstrate pagination techniques.
The table is seeded with <code>1,000,000</code> records for testing purposes, which should be enough to show the performance difference between offset and cursor pagination.</p>
<p>We'll use the following SQL schema for the examples:</p>
<pre><code class="language-sql">CREATE TABLE user_notes (
    id uuid NOT NULL,
    user_id uuid NOT NULL,
    note character varying(500),
    date date NOT NULL,
    CONSTRAINT pk_user_notes PRIMARY KEY (id)
);
</code></pre>
<p>And here's the C# class representing the <code>UserNote</code> entity:</p>
<pre><code class="language-csharp">public class UserNote
{
    public Guid Id { get; set; }
    public Guid UserId { get; set; }
    public string? Note { get; set; }
    public DateOnly Date { get; set; }
}
</code></pre>
<p>I will use PostgreSQL as the database, but the concepts also apply to other databases.</p>
<h2>Offset Pagination: The Traditional Approach</h2>
<p><a href="https://learn.microsoft.com/en-us/ef/core/querying/pagination#offset-pagination">Offset pagination</a> uses <code>Skip</code> and <code>Take</code> operations.
We <em>skip</em> a certain number of rows and <em>take</em> a fixed number of rows.
These usually translate to <code>OFFSET</code> and <code>LIMIT</code> in SQL queries.</p>
<p>Here's an example of offset pagination in <a href="http://ASP.NET">ASP.NET</a> Core:</p>
<pre><code class="language-csharp">app.MapGet(&quot;/offset&quot;, async (
    AppDbContext dbContext,
    int page = 1,
    int pageSize = 10,
    CancellationToken cancellationToken = default) =&gt;
{
    if (page &lt; 1) return Results.BadRequest(&quot;Page must be greater than 0&quot;);
    if (pageSize &lt; 1) return Results.BadRequest(&quot;Page size must be greater than 0&quot;);
    if (pageSize &gt; 100) return Results.BadRequest(&quot;Page size must be less than or equal to 100&quot;);

    var query = dbContext.UserNotes
        .OrderByDescending(x =&gt; x.Date)
        .ThenByDescending(x =&gt; x.Id);

    // Offset pagination typically counts the total number of items
    var totalCount = await query.CountAsync(cancellationToken);
    var totalPages = (int)Math.Ceiling(totalCount / (double)pageSize);

    // Skip and take the required number of items
    var items = await query
        .Skip((page - 1) * pageSize)
        .Take(pageSize)
        .ToListAsync(cancellationToken);

    return Results.Ok(new
    {
        Items = items,
        Page = page,
        PageSize = pageSize,
        TotalCount = totalCount,
        TotalPages = totalPages,
        HasNextPage = page &lt; totalPages,
        HasPreviousPage = page &gt; 1
    });
});
</code></pre>
<p>Note that I'm sorting the results by <code>Date</code> and <code>Id</code> in descending order.
This ensures consistent results when paginating.</p>
<p>Here's the generated SQL for offset pagination:</p>
<pre><code class="language-sql">-- This query is sent first
SELECT count(*)::int FROM user_notes AS u;

-- Followed by the actual data query
SELECT u.id, u.date, u.note, u.user_id
FROM user_notes AS u
ORDER BY u.date DESC, u.id DESC
LIMIT @pageSize OFFSET @offset;
</code></pre>
<h3>Limitations of Offset Pagination:</h3>
<ol>
<li>Performance degrades as offset increases because the database must scan and discard all rows before the offset</li>
<li>Risk of missing or duplicating items when data changes between pages</li>
<li>Inconsistent results with concurrent updates</li>
</ol>
<h2>Cursor-Based Pagination: A Faster Approach</h2>
<p><a href="https://learn.microsoft.com/en-us/ef/core/querying/pagination#keyset-pagination">Cursor pagination</a> uses a reference point (cursor) to fetch the next set of results.
This reference point is typically a <strong>unique identifier</strong> or a combination of fields that define the sort order.</p>
<p>I'll use the <code>Date</code> and <code>Id</code> fields to create a cursor for our <code>UserNotes</code> table.
The cursor is a composite of these two fields, allowing us to paginate efficiently.</p>
<p>Here's an example of cursor pagination in <a href="http://ASP.NET">ASP.NET</a> Core:</p>
<pre><code class="language-csharp">app.MapGet(&quot;/cursor&quot;, async (
    AppDbContext dbContext,
    DateOnly? date = null,
    Guid? lastId = null,
    int limit = 10,
    CancellationToken cancellationToken = default) =&gt;
{
    if (limit &lt; 1) return Results.BadRequest(&quot;Limit must be greater than 0&quot;);
    if (limit &gt; 100) return Results.BadRequest(&quot;Limit must be less than or equal to 100&quot;);

    var query = dbContext.UserNotes.AsQueryable();

    if (date != null &amp;&amp; lastId != null)
    {
        // Use the cursor to fetch the next set of results
        // If we were sorting in ASC order, we'd use &gt; instead of &lt;
        query = query.Where(x =&gt; x.Date &lt; date || (x.Date == date &amp;&amp; x.Id &lt;= lastId));
    }

    // Fetch the items and determine if there are more
    var items = await query
        .OrderByDescending(x =&gt; x.Date)
        .ThenByDescending(x =&gt; x.Id)
        .Take(limit + 1)
        .ToListAsync(cancellationToken);

    // Extract the cursor and ID for the next page
    bool hasMore = items.Count &gt; limit;
    DateOnly? nextDate = hasMore ? items[^1].Date : null;
    Guid? nextLastId = hasMore ? items[^1].Id : null;

    // Remove the extra item before returning results
    if (hasMore)
    {
        items.RemoveAt(items.Count - 1);
    }

    return Results.Ok(new
    {
        Items = items,
        NextDate = nextDate,
        NextLastId = nextLastId,
        HasMore = hasMore
    });
});
</code></pre>
<p>The sort order is the same as in the offset pagination example.
However, the sort order is critical for consistent results with cursor pagination.
Because the <code>Date</code> isn't a unique value in our table, we use the <code>Id</code> field to handle ties.
This ensures that we don't miss or duplicate items when paginating.</p>
<p>Here's the generated SQL for cursor pagination:</p>
<pre><code class="language-sql">SELECT u.id, u.date, u.note, u.user_id
FROM user_notes AS u
WHERE u.date &lt; @date OR (u.date = @date AND u.id &lt;= @lastId)
ORDER BY u.date DESC, u.id DESC
LIMIT @limit;
</code></pre>
<p>Note that there's no <code>OFFSET</code> in the query.
We're directly seeking the rows based on the cursor, which is more efficient than offset pagination.</p>
<p>The <code>COUNT</code> query is omitted in cursor pagination because we're not counting the total number of items.
This can be a limitation if you need to display the total number of pages upfront.
However, the performance benefits of cursor pagination often outweigh this limitation.</p>
<h3>Limitations of Cursor Pagination:</h3>
<ol>
<li>If users need to change sort fields dynamically, cursor pagination becomes significantly more complicated since the cursor must incorporate all sort conditions</li>
<li>Users can't jump to a specific page number - they must traverse sequentially through the pages</li>
<li>More complex to implement correctly compared to offset pagination, especially when handling ties and ensuring stable ordering</li>
</ol>
<h2>Examining the SQL Execution Plans</h2>
<p>I wanted to compare the execution plans for offset and cursor pagination.
I used the <code>EXPLAIN ANALYZE</code> command in PostgreSQL to see the <a href="https://www.postgresql.org/docs/current/using-explain.html">query plans</a>.</p>
<p>Here's the offset pagination query:</p>
<pre><code class="language-sql">SELECT u.id, u.date, u.note, u.user_id
FROM user_notes AS u
ORDER BY u.date DESC, u.id DESC
LIMIT 1000 OFFSET 900000;
</code></pre>
<p>I'm intentionally skipping <code>900,000</code> rows to exaggerate the performance impact.
After that, we fetch the next <code>1,000</code> rows.</p>
<p>Here's the query plan for offset pagination:</p>
<pre><code class="language-sql">EXPLAIN ANALYZE SELECT u.id, u.date, u.note, u.user_id
FROM user_notes AS u
ORDER BY u.date DESC, u.id DESC
LIMIT 1000 OFFSET 900000;

---
Limit  (cost=165541.59..165541.71 rows=1 width=52) (actual time=695.026..701.406 rows=1000 loops=1)
  -&gt;  Gather Merge  (cost=68312.50..165541.59 rows=833334 width=52) (actual time=342.475..684.567 rows=901000 loops=1)
        Workers Planned: 2
        Workers Launched: 2
        -&gt;  Sort  (cost=67312.48..68354.15 rows=416667 width=52) (actual time=327.846..450.295 rows=300841 loops=3)
              Sort Key: date DESC, id DESC
              Sort Method: external merge  Disk: 20440kB
              Worker 0:  Sort Method: external merge  Disk: 18832kB
              Worker 1:  Sort Method: external merge  Disk: 18912kB
              -&gt;  Parallel Seq Scan on user_notes u  (cost=0.00..14174.67 rows=416667 width=52) (actual time=1.035..22.876 rows=333333 loops=3)
Planning Time: 0.050 ms
JIT:
  Functions: 8
  Options: Inlining false, Optimization false, Expressions true, Deforming true
  Timing: Generation 0.243 ms (Deform 0.111 ms), Inlining 0.000 ms, Optimization 0.270 ms, Emission 4.085 ms, Total 4.598 ms
Execution Time: 704.217 ms
</code></pre>
<p>The total execution time is <code>704.217 ms</code> for offset pagination.</p>
<p>Here's the query returning the same set of rows using cursor pagination.
I had to hardcode the <code>@date</code> and <code>@lastId</code> values for this comparison:</p>
<pre><code class="language-sql">SELECT u.id, u.date, u.note, u.user_id
FROM user_notes AS u
WHERE u.date &lt; @date OR (u.date = @date AND u.id &lt;= @lastId)
ORDER BY u.date DESC, u.id DESC
LIMIT 1000;
</code></pre>
<p>Finally, here's the query plan for cursor pagination:</p>
<pre><code class="language-sql">EXPLAIN ANALYZE SELECT u.id, u.date, u.note, u.user_id
FROM user_notes AS u
WHERE u.date &lt; @date OR (u.date = @date AND u.id &lt;= @lastId)
ORDER BY u.date DESC, u.id DESC
LIMIT 1000;

---
Limit  (cost=20605.63..20722.31 rows=1000 width=52) (actual time=37.993..40.958 rows=1000 loops=1)
  -&gt;  Gather Merge  (cost=20605.63..30419.62 rows=84114 width=52) (actual time=37.992..40.921 rows=1000 loops=1)
        Workers Planned: 2
        Workers Launched: 2
        -&gt;  Sort  (cost=19605.61..19710.75 rows=42057 width=52) (actual time=24.611..24.630 rows=811 loops=3)
              Sort Key: date DESC, id DESC
              Sort Method: top-N heapsort  Memory: 240kB
              Worker 0:  Sort Method: top-N heapsort  Memory: 239kB
              Worker 1:  Sort Method: top-N heapsort  Memory: 238kB
              -&gt;  Parallel Seq Scan on user_notes u  (cost=0.00..17299.67 rows=42057 width=52) (actual time=0.009..21.462 rows=33333 loops=3)
                    Filter: ((date &lt; @date::date) OR ((date = @date::date) AND (id &lt;= @lastId::uuid)))
                    Rows Removed by Filter: 300000
Planning Time: 0.063 ms
Execution Time: 40.993 ms
</code></pre>
<p>The total execution time for cursor pagination is <code>40.993 ms</code>.</p>
<p>A whopping <code>17x</code> performance improvement with cursor pagination compared to offset pagination!</p>
<p>The performance with cursor pagination is consistent regardless of the page depth.
This is because we're directly seeking the rows based on the cursor, which is more efficient than offset pagination.
It's a huge advantage over offset pagination, especially for large datasets.</p>
<h2>Adding Indexes for Cursor Pagination</h2>
<p>I also tested the impact of indexes on <a href="https://use-the-index-luke.com/blog/2013-07/pagination-done-the-postgresql-way">cursor pagination</a>.
I created a composite index on the <code>Date</code> and <code>Id</code> fields to speed up the queries.
Or so I thought...</p>
<p>Here's the SQL command to create the composite index:</p>
<pre><code class="language-sql">CREATE INDEX idx_user_notes_date_id ON user_notes (date DESC, id DESC);
</code></pre>
<p>The index is created in descending order to match the sort order in our queries.</p>
<p>Let's see the query plan for cursor pagination with the composite index:</p>
<pre><code class="language-sql">EXPLAIN ANALYZE SELECT u.id, u.date, u.note, u.user_id
FROM user_notes AS u
WHERE u.date &lt; @date OR (u.date = @date AND u.id &lt;= @lastId)
ORDER BY u.date DESC, u.id DESC
LIMIT 1000;

---
Limit  (cost=0.42..816.55 rows=1000 width=52) (actual time=298.534..298.924 rows=1000 loops=1)
  -&gt;  Index Scan using idx_user_notes_date_id on user_notes u  (cost=0.42..82376.42 rows=100936 width=52) (actual time=298.532..298.888 rows=1000 loops=1)
        Filter: ((date &lt; @date::date) OR ((date = @date::date) AND (id &lt;= @lastId::uuid)))
        Rows Removed by Filter: 900000
Planning Time: 0.068 ms
Execution Time: 298.955 ms
</code></pre>
<p>We have an <code>Index Scan</code> using the composite index.
However, the execution time is <code>298.955 ms</code>, which is slower than the previous query without the index.</p>
<p>This might be because the dataset is too small to benefit from the index.
I have only <code>1,000,000</code> records in the table, which might not be enough to see the performance improvement with the index.</p>
<p>But wait, there's more to it!</p>
<p>What if we were to use a tuple comparison in SQL?</p>
<pre><code class="language-sql">EXPLAIN ANALYZE SELECT u.id, u.date, u.note, u.user_id
FROM user_notes AS u
WHERE (u.date, u.id) &lt;= (@date, @lastId)
ORDER BY u.date DESC, u.id DESC
LIMIT 1000;

---
Limit  (cost=0.42..432.81 rows=1000 width=52) (actual time=0.020..0.641 rows=1000 loops=1)
  -&gt;  Index Scan using idx_user_notes_date_id on user_notes u  (cost=0.42..43817.85 rows=101339 width=52) (actual time=0.019..0.606 rows=1000 loops=1)
        Index Cond: (ROW(date, id) &lt;= ROW(@date::date, @lastId::uuid))
Planning Time: 0.060 ms
Execution Time: 0.668 ms
</code></pre>
<p>Finally, the index is working.
The execution time is <code>0.668 ms</code>, which is significantly faster than the previous queries.</p>
<p>The query optimizer cannot determine whether the composite index can be used for row-level comparison.
However, the index is effectively used with a tuple comparison.</p>
<p>How do you translate this to EF Core?</p>
<p>The Postgres provider has <code>EF.Functions.LessThanOrEqual</code>, which accepts a <code>ValueTuple</code> as an argument.
We can use it to produce a <code>(u.date, u.id) &lt;= (@date, @lastId)</code> comparison in the query.
And this will utilize the composite index.</p>
<pre><code class="language-csharp">query = query.Where(x =&gt; EF.Functions.LessThanOrEqual(
    ValueTuple.Create(x.Date, x.Id),
    ValueTuple.Create(date, lastId)));
</code></pre>
<h2>Encoding the Cursor</h2>
<p>Here's a small utility class for encoding and decoding the cursor.
We'll use this to encode the cursor in the URL and decode it when fetching the next set of results.</p>
<p>The clients will receive the cursor as a Base64-encoded string.
They don't need to know the internal structure of the cursor.</p>
<pre><code class="language-csharp">using Microsoft.AspNetCore.Authentication; // For Base64UrlTextEncoder

public sealed record Cursor(DateOnly Date, Guid LastId)
{
    public static string Encode(DateOnly date, string lastId)
    {
        var cursor = new Cursor(date, lastId);
        string json = JsonSerializer.Serialize(cursor);
        return Base64UrlTextEncoder.Encode(Encoding.UTF8.GetBytes(json));
    }

    public static Cursor? Decode(string? cursor)
    {
        if (string.IsNullOrWhiteSpace(cursor))
        {
            return null;
        }

        try
        {
            string json = Encoding.UTF8.GetString(Base64UrlTextEncoder.Decode(cursor));
            return JsonSerializer.Deserialize&lt;Cursor&gt;(json);
        }
        catch
        {
            return null;
        }
    }
}
</code></pre>
<p>Here's an example of encoding and decoding the cursor:</p>
<pre><code class="language-csharp">string encodedCursor = Cursor.Encode(
  new DateOnly(2025, 2, 15),
  Guid.Parse(&quot;019500f9-8b41-74cf-ab12-25a48d4d4ab4&quot;));
// Result:
// eyJEYXRlIjoiMjAyNS0wMi0xNSIsIkxhc3RJZCI6IjAxOTUwMGY5LThiNDEtNzRjZi1hYjEyLTI1YTQ4ZDRkNGFiNCJ9

Cursor decodedCursor = Cursor.Decode(encodedCursor);
// Result:
// {
//     &quot;Date&quot;: &quot;2025-02-15&quot;,
//     &quot;LastId&quot;: &quot;019500f9-8b41-74cf-ab12-25a48d4d4ab4&quot;
// }
</code></pre>
<h2>Summary</h2>
<p>While offset pagination is simpler to implement, it suffers from significant performance degradation at scale.
My tests showed a 17x slowdown compared to cursor pagination when accessing deeper pages.</p>
<p>Cursor pagination maintains consistent performance regardless of page depth and works particularly well for real-time feeds and infinite scroll interfaces.</p>
<p>However, cursor pagination comes with tradeoffs.
It requires careful implementation, especially around cursor encoding and handling sort orders.
It also doesn't provide total page counts, making it unsuitable for interfaces that need to support paged navigation.</p>
<p>The choice between these approaches ultimately depends on your use case:</p>
<ul>
<li>Choose cursor pagination for performance-critical APIs, real-time feeds, infinite scroll, or any scenario where users frequently access deep pages</li>
<li>Stick with offset pagination for admin interfaces, small datasets, or when you need upfront page counts</li>
</ul>
<p>Another thing to consider: which page will your users typically land on?
If most users start at the first page and rarely visit other pages, offset pagination might be sufficient.
This will be the case for many applications.</p>
<p>Remember to use tuple comparisons and appropriate indexes to get the best performance from cursor pagination.</p>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_129.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Stop Conflating CQRS and MediatR]]></title>
            <link>https://milanjovanovic.tech/blog/stop-conflating-cqrs-and-mediatr</link>
            <guid isPermaLink="false">stop-conflating-cqrs-and-mediatr</guid>
            <pubDate>Sat, 08 Feb 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[The .NET ecosystem has gradually fused CQRS and MediatR together, creating a reflexive assumption that they're inseparable, but this mental shortcut has led teams down a path of unnecessary complexity. This article dispels common misconceptions by explaining how CQRS and MediatR are distinct tools solving different problems, and demonstrates how to implement CQRS effectively with or without MediatR.]]></description>
            <content:encoded><![CDATA[<p>&quot;We need to implement CQRS? Great, let me install MediatR.&quot;</p>
<p>If you've heard this in your development team - or perhaps said it yourself - you're not alone.
The .NET ecosystem has gradually fused these two concepts together, creating an almost reflexive response: CQRS equals MediatR.</p>
<p>This mental shortcut has led countless teams down a path of unnecessary complexity.
Others have avoided CQRS entirely, fearing the overhead of yet another messaging framework.</p>
<p>In this article, we'll dispel some common misconceptions and highlight the benefits of each pattern.</p>
<h2>Understanding CQRS in Its Pure Form</h2>
<p><a href="https://learn.microsoft.com/en-us/azure/architecture/patterns/cqrs">CQRS</a> is a pattern that separates read and write operations in your application.
The pattern suggests that the models used for reading data should be different from those used for writing data.</p>
<p>That's it.</p>
<p>No specific implementation details, no prescribed libraries, just a simple architectural principle.</p>
<p>The pattern emerged from the understanding that in many applications, especially those with complex domains,
the requirements for reading and writing data are fundamentally different.
Read operations often need to combine data from multiple sources or present it in specific formats for UI consumption.
Write operations need to enforce business rules, maintain consistency, and manage domain state.</p>
<p>This separation provides several benefits:</p>
<ul>
<li>Optimized read and write models for their specific purposes</li>
<li>Simplified maintenance as read and write concerns evolve independently</li>
<li>Enhanced scalability options for read and write operations</li>
<li>Clearer boundary between domain logic and presentation needs</li>
</ul>
<h2>MediatR: A Different Tool for Different Problems</h2>
<p><a href="https://github.com/jbogard/MediatR">MediatR</a> is an implementation of the mediator pattern.
Its primary purpose is to reduce direct dependencies between components by providing a central point of communication.
Instead of knowing about each other, the mediator connects the components.</p>
<p>The library provides several features:</p>
<ul>
<li>In-process messaging between components</li>
<li>Behavior pipelines for <a href="balancing-cross-cutting-concerns-in-clean-architecture">cross-cutting concerns</a></li>
<li><a href="how-to-publish-mediatr-notifications-in-parallel">Notification handling</a> (publish/subscribe)</li>
</ul>
<p>The indirection MediatR introduces is its most criticized aspect.
It can make code harder to follow, especially for newcomers to the codebase.
However, you can easily solve this problem by defining the requests in the same file as the handler.</p>
<h2>Why They Often Appear Together</h2>
<p>The frequent pairing of CQRS and MediatR isn't without reason.
MediatR's <a href="vertical-slice-architecture">request/response model</a> aligns well with CQRS's command/query separation.
Commands and queries can be implemented as MediatR requests, with handlers containing the actual implementation logic.</p>
<p>Here's an example command using MediatR:</p>
<pre><code class="language-csharp">public record CreateHabit(string Name, string? Description, int Priority) : IRequest&lt;HabitDto&gt;;

public sealed class CreateHabitHandler(ApplicationDbContext dbContext, IValidator&lt;CreateHabit&gt; validator)
    : IRequestHandler&lt;CreateHabit, HabitDto&gt;
{
    public async Task&lt;HabitDto&gt; Handle(CreateHabit request, CancellationToken cancellationToken)
    {
        await validator.ValidateAndThrowAsync(request);

        Habit habit = request.ToEntity();

        dbContext.Habits.Add(habit);

        await dbContext.SaveChangesAsync(cancellationToken);

        return habit.ToDto();
    }
}
</code></pre>
<p><a href="cqrs-pattern-with-mediatr">CQRS with MediatR</a> offers several advantages:</p>
<ul>
<li>Consistent handling of both commands and queries</li>
<li>Pipeline behaviors for logging, validation, and error handling</li>
<li>Clear separation of concerns through handler classes</li>
<li>Simplified testing through handler isolation</li>
</ul>
<p>However, this convenience comes at the cost of additional abstraction and complexity.
We have to define the request/response classes and handlers, write code for sending the requests, and so on.
This can be overkill for simple applications.</p>
<p>The question isn't whether this trade-off is universally good or bad but whether it's appropriate for your specific context.</p>
<h2>CQRS Without MediatR</h2>
<p>CQRS can be implemented just as easily without MediatR.
Here's a simple example of what it might look like.</p>
<p>You can define commands and queries as simple interfaces:</p>
<pre><code class="language-csharp">public interface ICommandHandler&lt;in TCommand, TResult&gt;
{
    Task&lt;TResult&gt; Handle(TCommand command, CancellationToken cancellationToken = default);
}

// Same thing for IQueryHandler
</code></pre>
<p>Then, you can implement your handlers and register them with dependency injection:</p>
<pre><code class="language-csharp">public record CreateOrderCommand(string CustomerId, List&lt;OrderItem&gt; Items)
    : ICommand&lt;CreateOrderResult&gt;;

public class CreateOrderCommandHandler : ICommandHandler&lt;CreateOrderCommand, CreateOrderResult&gt;
{
    public async Task&lt;CreateOrderResult&gt; Handle(
        CreateOrderCommand command,
        CancellationToken cancellationToken = default)
    {
        // implementation
    }
}

// DI registration...
builder.Services
    .AddScoped&lt;ICommandHandler&lt;CreateOrderCommand, CreateOrderResult&gt;, CreateOrderCommandHandler&gt;();
</code></pre>
<p>Finally, you can use the handler in your controller:</p>
<pre><code class="language-csharp">[ApiController]
[Route(&quot;orders&quot;)]
public class OrdersController : ControllerBase
{
    [HttpPost]
    public async Task&lt;ActionResult&lt;CreateOrderResult&gt;&gt; CreateOrder(
        CreateOrderCommand command,
        ICommandHandler&lt;CreateOrderCommand, CreateOrderResult&gt; handler)
    {
        var result = await handler.Handle(command);

        return Ok(result);
    }
}
</code></pre>
<p>What's the difference between this and the MediatR approach?</p>
<p>This approach provides the same separation of concerns but without the indirection.
It's direct, explicit, and often sufficient for many applications.</p>
<p>However, it lacks some of the conveniences that MediatR offers, such as pipeline behaviors and automatically registering handlers.
You also need to inject the specific handlers into your controllers, which can be cumbersome for larger applications.</p>
<h2>Takeaway</h2>
<p>CQRS and MediatR are distinct tools that solve different problems.
While they can work well together, treating them as inseparable does a disservice to both.
CQRS separates read and write concerns, while MediatR decouples components through a mediator.</p>
<p>The key is understanding what each pattern offers and making informed decisions based on your specific context.
Sometimes, you'll want both, sometimes just one, and sometimes neither.
That's the essence of thoughtful architecture: choosing the right tools for your specific needs.</p>
<p>If you want to learn more about implementing CQRS effectively as part of a clean,
maintainable architecture, check out <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture</strong></a>.
You'll learn how to apply these patterns in real-world scenarios, avoiding common pitfalls and over-engineering while building scalable applications.</p>
<p>That's all for today. Hope this was helpful.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_128.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Overriding Default HTTP Resilience Handlers in .NET]]></title>
            <link>https://milanjovanovic.tech/blog/overriding-default-http-resilience-handlers-in-dotnet</link>
            <guid isPermaLink="false">overriding-default-http-resilience-handlers-in-dotnet</guid>
            <pubDate>Sat, 01 Feb 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[While .NET 8's standard resilience handlers provide excellent defaults for HTTP clients, they lack built-in support for overriding these handlers for specific endpoints that require different resilience strategies. This article demonstrates how to implement a custom solution for overriding default handlers and discusses upcoming improvements planned by the .NET team.]]></description>
            <content:encoded><![CDATA[<p>Introducing <a href="building-resilient-cloud-applications-with-dotnet"><strong>.NET 8 resilience packages</strong></a>
built on top of <a href="https://github.com/App-vNext/Polly">Polly</a>
has made it much easier to build robust HTTP clients.
These packages provide standard resilience handlers that you can easily attach to <code>HttpClient</code> instances.
They implement common patterns like retry, circuit breaker, and timeout policies.</p>
<p>However, there is a significant limitation: once you configure the standard resilience handlers globally for all clients,
there is no built-in way to override them for specific cases.
This can be problematic when different endpoints require different resilience strategies.</p>
<p>In today's issue, I'll show you how to fix this and what the .NET team is doing about it.</p>
<h2>Standard Resilience Configuration</h2>
<p>Let's say you've configured default resilience handlers in your application startup.
<code>ConfigureHttpClientDefaults</code> is a convenient way to add standard resilience handlers to all <code>HttpClient</code> instances:</p>
<pre><code class="language-csharp">builder.Services
    .AddHttpClient()
    .ConfigureHttpClientDefaults(http =&gt; http.AddStandardResilienceHandler());
</code></pre>
<p>The .NET team runs many large-scale services in production, and they've found a standard set of resilience strategies that work
well for most scenarios.</p>
<p>The standard resilience handler combines five strategies to create a resilience pipeline:</p>
<ul>
<li>Rate limiter</li>
<li>Total request timeout</li>
<li>Retry</li>
<li>Circuit breaker</li>
<li>Attempt timeout</li>
</ul>
<p>You can customize the standard resilience pipeline by configuring the <code>HttpStandardResilienceOptions</code>.</p>
<p>Here's an example of how to configure it:</p>
<pre><code class="language-csharp">builder.Services.ConfigureHttpClientDefaults(http =&gt; http.AddStandardResilienceHandler(options =&gt;
{
    // Default is 2 seconds.
    options.Retry.Delay = TimeSpan.FromSeconds(1);

    // Default is 30 seconds.
    options.TotalRequestTimeout.Timeout = TimeSpan.FromSeconds(20);

    // Default is 0.1.
    options.CircuitBreaker.FailureRatio = 0.2;
}));
</code></pre>
<p>Okay, so we have our standard resilience pipeline set up.
Now all your <code>HttpClient</code> instances will use these resilience policies.</p>
<p>But what if you need different retry logic for a specific API endpoint or need to turn off circuit breaking for specific calls?</p>
<h2>The Problem</h2>
<p>Let's say you have a named <code>HttpClient</code> for calling the GitHub API,
and you want to configure specific resilience strategies for it:</p>
<pre><code class="language-csharp">builder.Services
    .AddHttpClient(&quot;github&quot;)
    .ConfigureHttpClient(client =&gt;
    {
        client.BaseAddress = new Uri(&quot;https://api.github.com&quot;);
    })
    .AddResilienceHandler(&quot;custom&quot;, pipeline =&gt;
    {
        pipeline.AddTimeout(TimeSpan.FromSeconds(10));

        pipeline.AddRetry(new HttpRetryStrategyOptions
        {
            MaxRetryAttempts = 3,
            BackoffType = DelayBackoffType.Exponential,
            UseJitter = true,
            Delay = TimeSpan.FromMilliseconds(500)
        });

        pipeline.AddTimeout(TimeSpan.FromSeconds(1));
    });
</code></pre>
<p>The <code>custom</code> policy won't be applied because we have a global resilience pipeline that overrides it.</p>
<p>This is a big oversight in the current implementation of the .NET resilience packages.</p>
<h2>The Solution</h2>
<p>The solution is to create an extension method that clears all handlers from the resilience pipeline.
This allows you to remove the default handlers and add your custom ones.</p>
<p>Here's how to implement it:</p>
<pre><code class="language-csharp">public static class ResilienceHttpClientBuilderExtensions
{
    public static IHttpClientBuilder RemoveAllResilienceHandlers(this IHttpClientBuilder builder)
    {
        builder.ConfigureAdditionalHttpMessageHandlers(static (handlers, _) =&gt;
        {
            for (int i = handlers.Count - 1; i &gt;= 0; i--)
            {
                if (handlers[i] is ResilienceHandler)
                {
                    handlers.RemoveAt(i);
                }
            }
        });
        return builder;
    }
}
</code></pre>
<p>Now you can use this extension method to implement custom resilience strategies:</p>
<pre><code class="language-csharp">builder.Services
    .AddHttpClient(&quot;github&quot;)
    .ConfigureHttpClient(client =&gt;
    {
        client.BaseAddress = new Uri(&quot;https://api.github.com&quot;);
    })
    .RemoveAllResilienceHandlers()
    .AddResilienceHandler(&quot;custom&quot;, pipeline =&gt;
    {
        // Configure the custom resilience pipeline...
    });

// Or use another standard resilience pipeline...
builder.Services
    .AddHttpClient(&quot;github-hedged&quot;)
    .RemoveAllResilienceHandlers()
    .AddStandardHedgingHandler();
</code></pre>
<h2>Future Improvements</h2>
<p>The .NET team is aware of this limitation, and better support for overriding default resilience handlers is planned for an upcoming release.
The <a href="https://github.com/dotnet/extensions/pull/5801">pull request for this API</a> is merged and should be available in a future release.</p>
<p>Until then, this workaround using <code>RemoveAllResilienceHandlers</code> is a drop-in replacement for the missing feature.</p>
<h2>Conclusion</h2>
<p>The ability to override default resilience handlers is much needed when building robust distributed systems.
While .NET's standard resilience handlers provide excellent defaults, real-world applications often require fine-tuned resilience strategies for different services.
The extension method presented here bridges this gap, allowing you to maintain both global defaults and specialized configurations where needed.</p>
<p>Want to dive deeper into building resilient cloud applications?
Check out my article about <a href="building-resilient-cloud-applications-with-dotnet"><strong>building resilient cloud applications with .NET</strong></a>.</p>
<p>Good luck out there, and see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_127.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Implementing AES Encryption With C#]]></title>
            <link>https://milanjovanovic.tech/blog/implementing-aes-encryption-with-csharp</link>
            <guid isPermaLink="false">implementing-aes-encryption-with-csharp</guid>
            <pubDate>Sat, 25 Jan 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to implement secure AES encryption in C# to protect sensitive application data like API keys and passwords, with practical code examples covering encryption, decryption, and key management best practices.]]></description>
            <content:encoded><![CDATA[<p>A single exposed API key or database password can compromise your entire infrastructure.
Yet many developers still store sensitive data with basic encoding or weak encryption.</p>
<p>Properly implemented encryption is your last line of defense.
When other security measures fail, it ensures stolen data remains unreadable.
This is especially crucial for API keys, database credentials, and user secrets that grant direct access to your systems.</p>
<p>In today's issue, we will cover implementing AES encryption in .NET with practical code examples and essential security considerations.</p>
<h2>Symmetric vs Asymmetric Encryption</h2>
<p><a href="https://en.wikipedia.org/wiki/Symmetric-key_algorithm">Symmetric encryption</a> (like AES) uses the same key for encryption and decryption.
It's fast and ideal for storing data that only your application needs to read.
The main challenge is securely storing the encryption key.</p>
<p><a href="https://en.wikipedia.org/wiki/Public-key_cryptography">Asymmetric encryption</a> (like RSA) uses different keys for encryption and decryption.
It's slower but allows secure communication between parties who don't share secrets.
Common uses include SSL/TLS and digital signatures.</p>
<p>For storing API keys and application secrets, symmetric encryption with AES is the appropriate choice.</p>
<figure>
  ![AES encryption algorithm.](/blogs/mnw_126/aes_encryption.png)
  <figcaption>AES encryption and decryption process block diagram.</figcaption>
</figure>
<h2>AES Encryption Implementation</h2>
<p>Let's examine a secure <a href="https://en.wikipedia.org/wiki/Advanced_Encryption_Standard">AES (Advanced Encryption Standard)</a> encryption implementation in C#.
This implementation uses AES-256, which provides the strongest security currently available in the AES standard.</p>
<pre><code class="language-csharp">public class Encryptor
{
    private const int KeySize = 256;
    private const int BlockSize = 128;

    public static EncryptionResult Encrypt(string plainText)
    {
        // Generate a random key and IV
        using var aes = Aes.Create();
        aes.KeySize = KeySize;
        aes.BlockSize = BlockSize;

        // Generate a random key and IV for each encryption operation
        aes.GenerateKey();
        aes.GenerateIV();

        byte[] encryptedData;

        // Create encryptor and encrypt the data
        using (var encryptor = aes.CreateEncryptor())
        using (var msEncrypt = new MemoryStream())
        {
            using (var csEncrypt = new CryptoStream(msEncrypt, encryptor, CryptoStreamMode.Write))
            using (var swEncrypt = new StreamWriter(csEncrypt))
            {
                swEncrypt.Write(plainText);
            }

            encryptedData = msEncrypt.ToArray();
        }

        // Package everything together, storing IV with the encrypted data
        var result = EncryptionResult.CreateEncryptedData(
            encryptedData,
            aes.IV,
            Convert.ToBase64String(aes.Key)
        );

        return result;
    }
}

public class EncryptionResult
{
    // The IV is prepended to the encrypted data
    public string EncryptedData { get; set; }
    public string Key { get; set; }

    public static EncryptionResult CreateEncryptedData(byte[] data, byte[] iv, string key)
    {
        // Combine IV and encrypted data
        var combined = new byte[iv.Length + data.Length];
        Array.Copy(iv, 0, combined, 0, iv.Length);
        Array.Copy(data, 0, combined, iv.Length, data.Length);

        return new EncryptionResult
        {
            EncryptedData = Convert.ToBase64String(combined),
            Key = key
        };
    }

    public (byte[] iv, byte[] encryptedData) GetIVAndEncryptedData()
    {
        var combined = Convert.FromBase64String(EncryptedData);

        // Extract IV and data
        var iv = new byte[16]; // AES block size is 16 bytes (128 / 8)
        var encryptedData = new byte[combined.Length - 16];

        Array.Copy(combined, 0, iv, 0, 16);
        Array.Copy(combined, 16, encryptedData, 0, encryptedData.Length);

        return (iv, encryptedData);
    }
}
</code></pre>
<p>Let's break down what's happening in this implementation:</p>
<ul>
<li>Every encryption operation generates a new random key and IV (Initialization Vector).
This is crucial - reusing either of these compromises security.
The IV prevents identical plaintext from producing identical ciphertext.</li>
<li>We use <code>CryptoStream</code> for efficient encryption of potentially large data.
The stream pattern ensures we don't load everything into memory at once.</li>
<li>The <code>EncryptionResult</code> class provides a way to package the encrypted data with its key and IV.
In production, the key should be stored separately in a key management service.</li>
</ul>
<h2>AES Decryption Implementation</h2>
<p>Here's the corresponding decryption implementation:</p>
<pre><code class="language-csharp">public class Decryptor
{
    private const int KeySize = 256;
    private const int BlockSize = 128;

    public static string Decrypt(EncryptionResult encryptionResult)
    {
        var key = Convert.FromBase64String(encryptionResult.Key);
        var (iv, encryptedData) = encryptionResult.GetIVAndEncryptedData();

        using var aes = Aes.Create();
        aes.KeySize = KeySize;
        aes.BlockSize = BlockSize;
        aes.Key = key;
        aes.IV = iv;

        // Create decryptor and decrypt the data
        using var decryptor = aes.CreateDecryptor();
        using var msDecrypt = new MemoryStream(encryptedData);
        using var csDecrypt = new CryptoStream(msDecrypt, decryptor, CryptoStreamMode.Read);
        using var srDecrypt = new StreamReader(csDecrypt);

        try
        {
            return srDecrypt.ReadToEnd();
        }
        catch (CryptographicException ex)
        {
            // Log the error securely - avoid exposing details
            throw new CryptographicException(&quot;Decryption failed&quot;, ex);
        }
    }
}
</code></pre>
<p>The decryption process reverses the encryption steps.
Note the error handling - we catch cryptographic exceptions but avoid exposing details that could help an attacker.
In production, you should log these errors securely for debugging while keeping security in mind.</p>
<h2>Usage Example</h2>
<p>Here's an example of encrypting and decrypting sensitive data using the implementations above:</p>
<pre><code class="language-csharp">// Encrypt sensitive data
var apiKey = &quot;your-sensitive-api-key&quot;;
var encryptionResult = Encryptor.Encrypt(apiKey);

// Output example: DCGT9kEwPglBonWWPa7PQPbr2I+6rskJ0lSFybbicvZ+wKMTU7cbJD2s3QSF2Yu6

// Store encrypted data in database
// IV is stored with the encrypted data
SaveToDatabase(encryptionResult.EncryptedData);

// Store key in key vault
await keyVault.StoreKeyAsync(&quot;apikey_1&quot;, encryptionResult.Key);

// Later, decrypt when needed
// IV is retrieved from the encrypted data
var encryptedData = LoadFromDatabase();
var key = await keyVault.GetKeyAsync(&quot;apikey_1&quot;);

var result = new EncryptionResult
{
    EncryptedData = encryptedData,
    Key = key,
    IV = iv
};

var decrypted = Decryptor.Decrypt(result);
</code></pre>
<h2>Takeaway</h2>
<p>AES encryption provides strong security for sensitive application data when implemented correctly.</p>
<p>Proper key management is very important.
Use a dedicated key storage service in production.
Popular options include <a href="https://learn.microsoft.com/en-us/azure/key-vault/">Azure Key Vault</a>,
<a href="https://aws.amazon.com/kms/">AWS Key Management Service</a>, and
<a href="https://www.vaultproject.io/">HashiCorp Vault</a>.</p>
<p>In my <a href="/pragmatic-rest-apis"><strong>Pragmatic REST APIs</strong></a> course, I cover secure data storage and encryption in more detail.
These are critical aspects of building secure and robust APIs and integrating with third-party APIs.
Check it out if you're interested in learning more.</p>
<p>Remember that encryption is just one part of a comprehensive security strategy.
Keep your encryption keys separate from encrypted data and rotate them regularly.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_126.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Scaling Monoliths: A Practical Guide for Growing Systems]]></title>
            <link>https://milanjovanovic.tech/blog/scaling-monoliths-a-practical-guide-for-growing-systems</link>
            <guid isPermaLink="false">scaling-monoliths-a-practical-guide-for-growing-systems</guid>
            <pubDate>Sat, 18 Jan 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[A well-designed monolith can scale remarkably well, despite industry trends pushing toward microservices. From database sharding to message queues, learn practical strategies to grow your monolithic system effectively.]]></description>
            <content:encoded><![CDATA[<p>Monoliths get a bad rap in our industry.
We're told they're legacy, that they don't scale, and that we need microservices to succeed.
After spending many years scaling systems from startups to enterprises, I can tell you this isn't true.
A well-designed monolith is often the right architecture choice, and it can scale remarkably well with the right approach.</p>
<p>In my experience building and scaling monolithic systems, I've found that the key to success isn't following trends.
It's understanding your scaling needs and applying the right solutions at the right time.
In this article, I'll share what I've learned about scaling monoliths effectively and when to use each approach.</p>
<h2>Understanding Scale</h2>
<p>A monolith puts all your code in one deployable unit.
This brings significant advantages: faster development cycles, simpler debugging, and straightforward deployments.
But as your system grows, you'll face scaling challenges.</p>
<div className="centered">
  ![Simple monolith system.](/blogs/mnw_125/monolith_system.png)
</div>
<p>Your database queries slow down as data volume grows. API endpoints that worked
fine with hundreds of users start timing out with thousands. Build times creep
up as your codebase expands. These are natural growing pains that every
successful system faces.</p>
<h2>Vertical Scaling</h2>
<p>Vertical scaling means giving your application more resources on a single machine.
It's the simplest scaling strategy and often the most effective first step.
Before diving into complex distributed systems, consider whether upgrading your existing infrastructure could solve your performance problems.</p>
<div className="centered">
  ![Example of vertically scaling a monolith.](/blogs/mnw_125/vertical_scaling.png)
</div>
<p>Vertical scaling works particularly well when there are clear resource bottlenecks.
If your CPU is consistently above 80% utilization, adding more cores will help.
If your database is I/O bound, upgrading to faster storage can dramatically improve performance.
Modern cloud platforms make this especially straightforward.
You can often upgrade with a few clicks and minimal downtime.</p>
<p>The benefits of vertical scaling extend beyond just performance.
It maintains your system's simplicity.
You don't need to redesign your architecture, implement new deployment patterns, or manage distributed system complexity.
Your monitoring, debugging, and operational procedures all stay the same.</p>
<p>However, vertical scaling does have limits.
You'll eventually hit a ceiling on what a single machine can handle.
Cloud providers have maximum instance sizes, and costs typically increase exponentially with larger instances.
More importantly, vertical scaling doesn't provide redundancy - you're still running on a single machine that represents a potential single point of failure.</p>
<p>Knowing when to move beyond vertical scaling is crucial.
Watch for these indicators:</p>
<ul>
<li>Costs are growing faster than your user base</li>
<li>You need better redundancy and fault tolerance</li>
<li>Your deployment downtime is impacting business operations</li>
<li>Your largest available instance size is approaching 70% utilization</li>
</ul>
<h2>Horizontal Scaling</h2>
<p><a href="horizontally-scaling-aspnetcore-apis-with-yarp-load-balancing"><strong>Horizontal scaling</strong></a> runs multiple instances of your application behind a load balancer.
It's the next step when vertical scaling reaches its limits.
It offers improved fault tolerance and nearly linear scaling capabilities.</p>
<div className="centered">
  ![Example of horizontally scaling a monolith.](/blogs/mnw_125/horizontal_scaling.png)
</div>
<p>The key to successful horizontal scaling lies in application design.
Your application must be stateless - each request should contain all the information needed to process it.</p>
<p>This means:</p>
<ul>
<li>Authentication should use tokens (like JWTs) rather than server-side sessions</li>
<li>Cached data should live in a distributed cache</li>
</ul>
<p>Load balancers are crucial in this architecture.
Whether you call it an <a href="implementing-an-api-gateway-for-microservices-with-yarp"><strong>API Gateway</strong></a>, Reverse Proxy, or Load Balancer, its job is to distribute traffic across your application instances.
Popular choices include:</p>
<ul>
<li><a href="https://nginx.org/en/">nginx</a>: Powerful, open-source, great for custom configurations</li>
<li><a href="https://microsoft.github.io/reverse-proxy/">YARP</a>: Microsoft's .NET reverse proxy, great for .NET applications</li>
<li>Cloud:
<a href="https://docs.aws.amazon.com/elasticloadbalancing/latest/application/introduction.html">AWS ALB</a>,
<a href="https://learn.microsoft.com/en-us/azure/application-gateway/overview">Azure Application Gateway</a>,
<a href="https://cloud.google.com/load-balancing?hl=en">Google Cloud Load Balancing</a></li>
</ul>
<p>Horizontal scaling provides several key benefits:</p>
<ul>
<li>Better fault tolerance through redundancy</li>
<li>Ability to handle more concurrent users</li>
<li>Rolling deployments with zero downtime</li>
<li>Cost-effective scaling (scale down when traffic is low)</li>
</ul>
<p>The main challenge in horizontal scaling isn't technical—it's architectural.
Your application needs to be designed for horizontal scaling from the start.
Converting a stateful application to a stateless one often requires significant refactoring.</p>
<h2>Database Scaling</h2>
<p>Database scaling is where most monoliths first hit real limitations.
Let's explore each scaling strategy in detail.</p>
<h3>Read Replicas</h3>
<p>Read replicas are often your first step in database scaling but come with significant trade-offs.
Read replicas maintain a copy of your primary database that serves read-only traffic.
When you run a query against a replica, you're not competing with writes on your primary database.</p>
<p>Each replica maintains an up-to-date copy of your data through replication.
Changes flow one-way: from primary to replicas.
This means any data written to your primary will eventually show up in your replicas.
That &quot;eventually&quot; is important - you're trading consistency for better read performance.</p>
<p>Most cloud providers make read replicas easy to set up.
AWS RDS and Azure SQL all support read replicas with minimal configuration.
They handle replication, monitoring, and failover for you.</p>
<p><img src="/blogs/mnw_125/database_read_replicas.png" alt="Example of database read replication for scaling a monolith."></p>
<p>When implementing read replicas, consider:</p>
<ul>
<li>Replication lag affects data freshness</li>
<li>Write volume impacts replication speed</li>
<li>Geographic location affects latency</li>
<li>Each replica adds to your costs</li>
</ul>
<h3>Materialized Views</h3>
<p>Sometimes read replicas aren't enough.
Perhaps you need to reshape your data for specific use cases, or you're running complex analytical queries that are slow even on a replica.
This is where materialized views come in.</p>
<p>A materialized view is a pre-computed dataset stored as a table.
Unlike regular views that compute their results on each query, materialized views store their results.
This makes them much faster to query but introduces a new challenge: keeping them up to date.</p>
<p>Materialized views excel at:</p>
<ul>
<li>Complex analytical queries</li>
<li>Data that updates on a schedule</li>
<li>Aggregations and summaries</li>
<li>Denormalized data for specific views</li>
</ul>
<p>The key trade-off is freshness versus performance.
You need to decide how often to refresh your materialized views.
Too often, and you're putting load on your database.
Too rarely, and your data gets stale.</p>
<h3>Database Sharding</h3>
<p>Sharding becomes necessary when your database grows beyond what a single instance can handle.
Sharding splits your data across multiple database instances, with each shard containing a distinct subset of your data.
The key to successful sharding lies in choosing the right sharding strategy for your use case.</p>
<p><strong>Range-based sharding</strong> splits data based on ranges of a key value - for example, customers A-M go to Shard 1, N-Z to Shard 2.
This approach works well with data that has a natural range distribution, like dates or alphabetical order,
but can lead to hotspots if certain ranges see more activity than others.</p>
<p><strong>Hash-based sharding</strong> applies a hash function to your sharding key to determine which shard holds the data.
The choice of hashing function is crucial.
It must distribute data evenly across your shards to prevent any single shard from becoming a bottleneck.
While this approach provides better data distribution, it makes range-based queries more complex since related data might live on different shards.</p>
<p><strong>Tenant-based sharding</strong> gives each tenant their own database.
This approach provides natural isolation and makes tenant-specific operations straightforward.
While it makes cross-tenant queries more complex, it's often the cleanest solution for multi-tenant systems where data isolation is important.</p>
<p><img src="/blogs/mnw_125/database_sharding.png" alt="Example of database sharding for scaling a monolith."></p>
<h2>Caching</h2>
<p>Caching is one of the most effective ways to improve your system's performance.
A well-implemented caching strategy can dramatically reduce database load and improve response times by storing frequently accessed data in memory.</p>
<p>Modern caching happens at multiple levels.
Browser caching reduces unnecessary network requests.
CDN caching brings your content closer to users.
<a href="caching-in-aspnetcore-improving-application-performance"><strong>Application-level caching</strong></a> with tools like Redis stores frequently accessed data in memory.
Database query caching reduces expensive computations.</p>
<p>The key to effective caching is understanding your data access patterns.
Frequently read, rarely changed data benefits most from caching.
Tools like Redis and Memcached excel at storing such data in memory, providing sub-millisecond access times.
Cloud providers offer managed caching services like Azure Cache for Redis,
handling the operational complexity of maintaining a distributed cache.</p>
<h2>Message Queues</h2>
<p>Message queues are a powerful tool for scaling your monolith.
They let you defer time-consuming operations and distribute work across multiple processors.
This keeps your API responsive while handling heavy tasks in the background.</p>
<p>Message queues transform your system's behavior under load.
Instead of processing everything synchronously, you can queue work for later.
This pattern works especially well for operations like:</p>
<ul>
<li>Processing uploaded files</li>
<li>Sending emails and notifications</li>
<li>Generating reports</li>
<li>Updating search indexes</li>
<li>Running batch operations</li>
</ul>
<p>The real power of message queues lies in their ability to handle traffic spikes.
When your system gets hit with a surge of requests, queues act as a buffer.
They let you accept work at peak rates but process it at a sustainable pace.</p>
<h2>Summary</h2>
<p>Scaling a monolith isn't about choosing between vertical scaling, horizontal scaling, caching, or any single approach.
It's about using the right tool at the right time.
Start with the simplest solution that solves your immediate problem, then add complexity only when needed.</p>
<p>A practical scaling journey often looks like this:</p>
<ol>
<li>Optimize your code and database queries</li>
<li>Add caching where it matters most</li>
<li>Scale vertically until it's no longer cost-effective</li>
<li>Move to horizontal scaling for better redundancy</li>
<li>Implement message queues for background work</li>
<li>Consider database sharding when data size demands it</li>
</ol>
<p>A well-designed monolith can handle significant load with just a subset of these techniques.
The key is to understand your system's actual bottlenecks and address them specifically.</p>
<p>If you're interested in building maintainable, scalable monoliths, my <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a> course dives deeper into these concepts.
You'll learn how to structure your code for long-term maintainability and scale.</p>
<p>Remember: don't let perfect be the enemy of good.
Start with the simplest solution that could work, measure everything, and scale what's necessary.
A well-designed monolith can take you further than you might think.</p>
<p>That's all for today. Hope this was helpful.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_125.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Working with LLMs in .NET using Microsoft.Extensions.AI]]></title>
            <link>https://milanjovanovic.tech/blog/working-with-llms-in-dotnet-using-microsoft-extensions-ai</link>
            <guid isPermaLink="false">working-with-llms-in-dotnet-using-microsoft-extensions-ai</guid>
            <pubDate>Sat, 11 Jan 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Microsoft.Extensions.AI provides a unified interface for integrating LLMs into .NET applications, allowing developers to switch between providers like Ollama, Azure, or OpenAI without changing application code. Through practical examples of chat completion, article summarization, and smart categorization, this article demonstrates how to leverage the library's features while running LLMs locally using Ollama.]]></description>
            <content:encoded><![CDATA[<p>I've been experimenting with different approaches to integrating LLMs into .NET apps,
and I want to share what I've learned about using <code>Microsoft.Extensions.AI</code>.</p>
<p>Large Language Models (LLMs) have revolutionized how we approach AI-powered applications.
While many developers are familiar with cloud-based solutions like OpenAI's GPT models,
running LLMs locally has become increasingly accessible thanks to projects like <a href="https://ollama.com/">Ollama</a>.</p>
<p>In this article, we'll explore how to use LLMs in .NET applications using <code>Microsoft.Extensions.AI</code>,
a powerful abstraction that extends the <a href="https://github.com/microsoft/semantic-kernel">Semantic Kernel</a> SDK.</p>
<h2>Understanding the Building Blocks</h2>
<h3>Large Language Models (LLMs)</h3>
<p>LLMs are deep learning models trained on vast amounts of data, capable of understanding and generating human-like text.
These models can perform various tasks such as text completion, summarization, classification, and engaging in conversation.
While traditionally accessed through cloud APIs, recent advances have made it possible to run them locally on standard hardware.</p>
<figure>
  ![Timeline of large language models.](/blogs/mnw_124/large_language_models.png)
  <figcaption>
    Source: [Weights &
    Biases](https://wandb.ai/vincenttu/blog_posts/reports/A-Survey-of-Large-Language-Models--VmlldzozOTY2MDM1)
  </figcaption>
</figure>
<h3>Ollama</h3>
<p>Ollama is an open-source project that simplifies running LLMs locally.
It provides a Docker container that can run various open-source models like Llama,
making it easy to experiment with AI without depending on cloud services.
Ollama handles model management and optimization and provides a simple API for interactions.</p>
<h3><a href="http://Microsoft.Extensions.AI">Microsoft.Extensions.AI</a></h3>
<p><a href="https://www.nuget.org/packages/Microsoft.Extensions.AI">Microsoft.Extensions.AI</a> is a library that provides a unified interface for working with LLMs in .NET applications.
Built on top of Microsoft's Semantic Kernel, it abstracts away the complexity of different LLM implementations,
allowing developers to switch between providers (like Ollama, Azure, or OpenAI) without changing application code.</p>
<h2>Getting Started</h2>
<p>Before diving into the examples, here's what you need to run LLMs locally:</p>
<ol>
<li>Docker running on your machine</li>
<li>Ollama container running with the <code>llama3</code> model:</li>
</ol>
<pre><code class="language-bash"># Pull the Ollama container
docker run --gpus all -d -v ollama_data:/root/.ollama -p 11434:11434 --name ollama ollama/ollama

# Pull the llama3 model
docker exec -it ollama ollama pull llama3
</code></pre>
<ol start="3">
<li>A few NuGet packages (I built this using a .NET 9 console application):</li>
</ol>
<pre><code class="language-powershell">Install-Package Microsoft.Extensions.AI # The base AI library
Install-Package Microsoft.Extensions.AI.Ollama # Ollama provider implementation
Install-Package Microsoft.Extensions.Hosting # For building the DI container
</code></pre>
<h2>Simple Chat Completion</h2>
<p>Let's start with a basic example of chat completion.
Here's the minimal setup:</p>
<pre><code class="language-csharp">var builder = Host.CreateApplicationBuilder();

builder.Services.AddChatClient(new OllamaChatClient(new Uri(&quot;http://localhost:11434&quot;), &quot;llama3&quot;));

var app = builder.Build();

var chatClient = app.Services.GetRequiredService&lt;IChatClient&gt;();

var response = await chatClient.GetResponseAsync(&quot;What is .NET? Reply in 50 words max.&quot;);

Console.WriteLine(response.Message.Text);
</code></pre>
<p>Nothing fancy here - we're just setting up dependency injection and asking a simple question.
If you're used to using raw API calls, you'll notice how clean this feels.</p>
<p>The <code>AddChatClient</code> extension method registers the chat client with the DI container.
This allows you to inject <code>IChatClient</code> into your services and interact with LLMs using a simple API.
The implementation uses the <code>OllamaChatClient</code> to communicate with the Ollama container running locally.</p>
<h2>Implementing Chat with History</h2>
<p>Building on the previous example, we can create an interactive chat that maintains conversation history.
This is useful for context-aware interactions and real-time chat applications.
All we need is a <code>List&lt;ChatMessage</code> to store the chat history:</p>
<pre><code class="language-csharp">var chatHistory = new List&lt;ChatMessage&gt;();

while (true)
{
   Console.WriteLine(&quot;Enter your prompt:&quot;);
   var userPrompt = Console.ReadLine();
   chatHistory.Add(new ChatMessage(ChatRole.User, userPrompt));

   Console.WriteLine(&quot;Response from AI:&quot;);
   var chatResponse = &quot;&quot;;
   await foreach (var item in chatClient.GetStreamingResponseAsync(chatHistory))
   {
       // We're streaming the response, so we get each message as it arrives
       Console.Write(item.Text);
       chatResponse += item.Text;
   }
   chatHistory.Add(new ChatMessage(ChatRole.Assistant, chatResponse));
   Console.WriteLine();
}
</code></pre>
<p>The cool part here is the streaming response - you get that nice, gradual text appearance like in ChatGPT.
We're also maintaining chat history, which lets the model understand context from previous messages, making conversations feel more natural.</p>
<h2>Getting Practical: Article Summarization</h2>
<p>Let's try something more useful - automatically summarizing articles.
I've been using this to process blog posts:</p>
<pre><code class="language-csharp">var posts = Directory.GetFiles(&quot;posts&quot;).Take(5).ToArray();
foreach (var post in posts)
{
   string prompt = $$&quot;&quot;&quot;
         You will receive an input text and the desired output format.
         You need to analyze the text and produce the desired output format.
         You not allow to change code, text, or other references.

         # Desired response

         Only provide a RFC8259 compliant JSON response following this format without deviation.

         {
            &quot;title&quot;: &quot;Title pulled from the front matter section&quot;,
            &quot;summary&quot;: &quot;Summarize the article in no more than 100 words&quot;
         }

         # Article content:

         {{File.ReadAllText(post)}}
         &quot;&quot;&quot;;

   var response = await chatClient.GetResponseAsync(prompt);
   Console.WriteLine(response.Message.Text);
   Console.WriteLine(Environment.NewLine);
}
</code></pre>
<p>Pro tip: Being specific about the output format (like requesting <a href="https://datatracker.ietf.org/doc/html/rfc8259">RFC8259</a> compliant JSON) helps get consistent results.
I learned this the hard way after dealing with occasionally malformed responses!</p>
<h2>Taking It Further: Smart Categorization</h2>
<p>Here's where it gets really interesting - we can get strongly typed responses directly from our LLM:</p>
<pre><code class="language-csharp">class PostCategory
{
    public string Title { get; set; } = string.Empty;
    public string[] Tags { get; set; } = [];
}

var posts = Directory.GetFiles(&quot;posts&quot;).Take(5).ToArray();
foreach (var post in posts)
{
    string prompt = $$&quot;&quot;&quot;
          You will receive an input text and the desired output format.
          You need to analyze the text and produce the desired output format.
          You not allow to change code, text, or other references.

          # Desired response

          Only provide a RFC8259 compliant JSON response following this format without deviation.

          {
             &quot;title&quot;: &quot;Title pulled from the front matter section&quot;,
             &quot;tags&quot;: &quot;Array of tags based on analyzing the article content. Tags should be lowercase.&quot;
          }

          # Article content:

          {{File.ReadAllText(post)}}
          &quot;&quot;&quot;;

    var response = await chatClient.GetResponseAsync&lt;PostCategory&gt;(prompt);

    Console.WriteLine(
      $&quot;{response.Result.Title}. Tags: {string.Join(&quot;,&quot;,response.Result.Tags)}&quot;);
}
</code></pre>
<p>The strongly typed approach provides compile-time safety and better IDE support, making it easier to maintain and refactor code that interacts with LLM responses.</p>
<h2>Flexibility with Different LLM Providers</h2>
<p>One of the key advantages of <code>Microsoft.Extensions.AI</code> is support for different providers.
While our examples use Ollama, you can easily switch to other providers:</p>
<pre><code class="language-csharp">// Using Azure OpenAI
builder.Services.AddChatClient(new AzureOpenAIClient(
        new Uri(&quot;AZURE_OPENAI_ENDPOINT&quot;),
        new DefaultAzureCredential())
            .AsChatClient());

// Using OpenAI
builder.Services.AddChatClient(new OpenAIClient(&quot;OPENAI_API_KEY&quot;).AsChatClient());
</code></pre>
<p>This flexibility allows you to:</p>
<ul>
<li>Start development with local models</li>
<li>Move to production with cloud providers</li>
<li>Switch between providers without changing application code</li>
<li>Mix different providers for different use cases (categorization, image recognition, etc.)</li>
</ul>
<h2>Takeaway</h2>
<p><code>Microsoft.Extensions.AI</code> makes it very simple to integrate LLMs into .NET applications.
Whether you're building a chat interface, processing documents, or adding AI-powered features to your application,
the library provides a clean, consistent API that works across different LLM providers.</p>
<p>I've only scratched the surface here.
Since integrating this into my projects, I've found countless uses:</p>
<ul>
<li>Automated content moderation for user submissions</li>
<li>Automated support ticket categorization</li>
<li>Content summarization for newsletters</li>
</ul>
<p>I'm also planning a small side project that will use LLMs to process images from a camera feed.
The idea is to detect anything unusual and trigger alerts in real-time.</p>
<p>What are you planning to build with this?
I'd love to hear about your projects and experiences.
The AI space is moving fast, but with tools like <code>Microsoft.Extensions.AI</code>, we can focus on building features rather than wrestling with infrastructure.</p>
<p>Good luck out there, and see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_124.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Unit Testing Clean Architecture Use Cases]]></title>
            <link>https://milanjovanovic.tech/blog/unit-testing-clean-architecture-use-cases</link>
            <guid isPermaLink="false">unit-testing-clean-architecture-use-cases</guid>
            <pubDate>Sat, 04 Jan 2025 00:00:00 GMT</pubDate>
            <description><![CDATA[Drawing from years of experience, I share my battle-tested approach to unit testing Clean Architecture use cases in .NET, focusing on the critical balance between code coverage and test quality. Through practical examples and real-world scenarios, I'll demonstrate how to write effective tests that not only catch bugs but also provide confidence in your code deployments.]]></description>
            <content:encoded><![CDATA[<p>Writing tests is a crucial part of my daily work.
Over the years, I've learned that good tests can make or break a project.</p>
<p>One project I worked on remains the best example of this.
It was a large, complex system with many moving parts.
We had a requirement that code coverage must be above 90%.</p>
<p>Code coverage doesn't directly translate to good tests, but it's a good starting point.
It's up to you to write quality tests that cover the most critical parts of your system.</p>
<p>Today, I want to share my approach to testing Clean Architecture use cases in .NET.</p>
<h2>Why Testing Matters</h2>
<p>I've seen many projects fail because of poor testing practices.
The codebase grows, changes become risky, and developers lose confidence in their deployments.
This is especially true for Clean Architecture projects, where we need to ensure our use cases work correctly.</p>
<p>Testing isn't just about catching bugs.
It's about having confidence in your code.
When I make changes, I want to know immediately if I've broken something.
Good tests give me that confidence.</p>
<h2>Understanding Different Testing Approaches</h2>
<p>Before diving into the specific examples, let's talk about testing types.
In my experience, there are three main types of tests you'll write:</p>
<ol>
<li>
<p><strong>Unit tests</strong> focus on testing individual components in isolation.
They're fast, reliable, and help you catch issues early.
I write these tests first, and they make up the majority of my test suite.</p>
</li>
<li>
<p><a href="testcontainers-integration-testing-using-docker-in-dotnet"><strong>Integration tests</strong></a> verify that different components work together correctly.
They're slower but essential for testing database operations or external services.</p>
</li>
<li>
<p><a href="testing-modular-monoliths-system-integration-testing"><strong>End-to-end tests</strong></a> check the entire system flow.
They're the slowest but provide confidence that everything works together.</p>
</li>
</ol>
<p>For this article, we'll focus on unit tests. They're the foundation of a solid test suite and the most common type you'll write.</p>
<h2>Breaking Down Our Use Case</h2>
<p>Looking at our <code>ReserveBookingCommandHandler</code> class, we have a typical Clean Architecture use case.
It handles apartment booking reservations with several business rules:</p>
<ol>
<li>The apartment must exist</li>
<li>The booking dates must not overlap with existing bookings</li>
<li>A new booking should be created if all checks pass</li>
</ol>
<p>This is a perfect example for unit testing because it has clear inputs, outputs, and dependencies we can mock.</p>
<pre><code class="language-csharp">internal sealed class ReserveBookingCommandHandler(
    IApartmentRepository apartmentRepository,
    IBookingRepository bookingRepository,
    IDateTimeProvider dateTimeProvider) : ICommandHandler&lt;ReserveBookingCommand, Guid&gt;
{
    public async Task&lt;Result&lt;Guid&gt;&gt; Handle(
        ReserveBookingCommand request,
        CancellationToken cancellationToken)
    {
        var apartment = await apartmentRepository.GetByIdAsync(request.ApartmentId, cancellationToken);

        if (apartment is null)
        {
            return Result.Failure&lt;Guid&gt;(ApartmentErrors.NotFound);
        }

        var duration = DateRange.Create(request.StartDate, request.EndDate);

        if (await bookingRepository.IsOverlappingAsync(apartment, duration, cancellationToken))
        {
            return Result.Failure&lt;Guid&gt;(BookingErrors.Overlap);
        }

        var booking = Booking.Create(
            apartment,
            duration,
            dateTimeProvider.UtcNow);

        bookingRepository.Add(booking);

        return booking.Id;
    }
}
</code></pre>
<h2>Setting Up Our Test Environment</h2>
<p>The test class setup shows the standard approach I use for all my handler tests. Let's break it down:</p>
<pre><code class="language-csharp">public class ReserveBookingCommandHandlerTests
{
    private readonly ReserveBookingCommandHandler _handler;
    private readonly IApartmentRepository _apartmentRepository;
    private readonly IBookingRepository _bookingRepository;
    private readonly IDateTimeProvider _dateTimeProvider;

    private static readonly Guid ApartmentId = Guid.NewGuid();
    private static readonly DateTime UtcNow = DateTime.UtcNow;

    public ReserveBookingCommandHandlerTests()
    {
        _apartmentRepository = Substitute.For&lt;IApartmentRepository&gt;();
        _bookingRepository = Substitute.For&lt;IBookingRepository&gt;();
        _dateTimeProvider = Substitute.For&lt;IDateTimeProvider&gt;();
        _dateTimeProvider.UtcNow.Returns(UtcNow);

        _handler = new ReserveBookingCommandHandler(
            _apartmentRepository,
            _bookingRepository,
            _dateTimeProvider);
    }
}
</code></pre>
<p>I'm using <a href="https://nsubstitute.github.io/">NSubstitute</a> to create mocks of our dependencies.
Each test starts with fresh mocks, preventing test interference.
The static fields provide consistent values across all tests.</p>
<p>Notice how I mock <code>IDateTimeProvider</code>.
This is crucial for testing time-dependent code.
Never use <code>DateTime.UtcNow</code> directly in your production code - it makes testing much harder.</p>
<h2>Testing the Not Found Scenario</h2>
<p>Our first test verifies the behavior when an apartment doesn't exist:</p>
<pre><code class="language-csharp">[Fact]
public async Task Handle_WhenApartmentDoesNotExist_ShouldReturnNotFoundError()
{
    // Arrange
    var command = new ReserveBookingCommand(
        ApartmentId,
        new DateOnly(2024, 1, 1),
        new DateOnly(2024, 1, 5));

    _apartmentRepository.GetByIdAsync(ApartmentId, Arg.Any&lt;CancellationToken&gt;())
        .Returns((Apartment?)null);

    // Act
    var result = await _handler.Handle(command, default);

    // Assert
    result.IsFailure.Should().BeTrue();
    result.Error.Should().Be(ApartmentErrors.NotFound);
}
</code></pre>
<p>This test follows the <strong>Arrange-Act-Assert</strong> pattern:</p>
<ol>
<li>Arrange: Set up the command and mock the repository to return <code>null</code></li>
<li>Act: Call the handler</li>
<li>Assert: Verify we get the correct error</li>
</ol>
<p>I use <a href="https://fluentassertions.com/">FluentAssertions</a> because it provides clear, readable assertions and better error messages than the standard Assert class.</p>
<h2>Handling Booking Conflicts</h2>
<p>The overlap test ensures we can't double-book apartments:</p>
<pre><code class="language-csharp">[Fact]
public async Task Handle_WhenBookingOverlaps_ShouldReturnOverlapError()
{
    // Arrange
    var command = new ReserveBookingCommand(
        ApartmentId,
        new DateOnly(2024, 1, 1),
        new DateOnly(2024, 1, 5));

    var apartment = new Apartment { Id = ApartmentId };
    _apartmentRepository.GetByIdAsync(ApartmentId, Arg.Any&lt;CancellationToken&gt;())
        .Returns(apartment);
    _bookingRepository.IsOverlappingAsync(apartment, Arg.Any&lt;DateRange&gt;(), Arg.Any&lt;CancellationToken&gt;())
        .Returns(true);

    // Act
    var result = await _handler.Handle(command, default);

    // Assert
    result.IsFailure.Should().BeTrue();
    result.Error.Should().Be(BookingErrors.Overlap);
}
</code></pre>
<p>Here, we verify the overlap check works correctly.
Notice how we:</p>
<ol>
<li>Mock the apartment repository to return a valid apartment</li>
<li>Mock the booking repository to indicate an overlap</li>
<li>Verify we get the overlap error</li>
</ol>
<h2>Testing Successful Bookings</h2>
<p>The happy path test ensures everything works when all conditions are met:</p>
<pre><code class="language-csharp">[Fact]
public async Task Handle_WhenValidRequest_ShouldCreateBooking()
{
    // Arrange
    var command = new ReserveBookingCommand(
        ApartmentId,
        new DateOnly(2024, 1, 1),
        new DateOnly(2024, 1, 5));

    var apartment = new Apartment { Id = ApartmentId };
    _apartmentRepository.GetByIdAsync(ApartmentId, Arg.Any&lt;CancellationToken&gt;())
        .Returns(apartment);
    _bookingRepository.IsOverlappingAsync(apartment, Arg.Any&lt;DateRange&gt;(), Arg.Any&lt;CancellationToken&gt;())
        .Returns(false);

    // Act
    var result = await _handler.Handle(command, default);

    // Assert
    result.IsSuccess.Should().BeTrue();
    await _bookingRepository.Received(1)
        .Add(Arg.Is&lt;Booking&gt;(b =&gt;
            b.Id == result.Value &amp;&amp;
            b.ApartmentId == ApartmentId));
}
</code></pre>
<p>This test is more complex because we need to:</p>
<ol>
<li>Set up multiple mocks</li>
<li>Verify the success result</li>
<li>Check that the booking was added with correct properties</li>
</ol>
<p>NSubstitute's <code>Received()</code> method lets us verify the <code>Add</code> method was called <strong>exactly once</strong> with the right booking.</p>
<h2>Verifying Exception Handling</h2>
<p>Testing exception scenarios is crucial for robust code:</p>
<pre><code class="language-csharp">[Fact]
public async Task Handle_WhenRepositoryThrowsOverlapException_ShouldPropagateException()
{
    // Arrange
    var command = new ReserveBookingCommand(
        ApartmentId,
        new DateOnly(2024, 1, 1),
        new DateOnly(2024, 1, 5));

    var apartment = new Apartment { Id = ApartmentId };
    _apartmentRepository.GetByIdAsync(ApartmentId, Arg.Any&lt;CancellationToken&gt;())
        .Returns(apartment);
    _bookingRepository.IsOverlappingAsync(apartment, Arg.Any&lt;DateRange&gt;(), Arg.Any&lt;CancellationToken&gt;())
        .Throws&lt;BookingOverlapException&gt;();

    // Act
    var act = () =&gt; _handler.Handle(command, default);

    // Assert
    await act.Should().ThrowAsync&lt;BookingOverlapException&gt;();
}
</code></pre>
<p>This test ensures exceptions propagate correctly. We:</p>
<ol>
<li>Set up the scenario</li>
<li>Make the repository throw an exception</li>
<li>Verify the exception bubbles up</li>
</ol>
<p>FluentAssertions makes testing async exceptions clean and readable.</p>
<h2>Understanding Test Coverage Limitations</h2>
<p>When we look back at our booking overlap test, there's an important distinction to make.
Our unit test verifies that our command handler behaves correctly when the booking repository reports an overlap.
However, it doesn't verify that the overlap detection logic itself works correctly.</p>
<p>Consider what we're actually testing:</p>
<pre><code class="language-csharp">_bookingRepository.IsOverlappingAsync(apartment, Arg.Any&lt;DateRange&gt;(), Arg.Any&lt;CancellationToken&gt;())
    .Returns(true);
</code></pre>
<p>We're simply telling our mock repository to return <code>true</code>.
This gives us confidence that our command handler correctly handles the overlap scenario,
but it does not tell us whether our actual overlap detection logic works correctly.</p>
<p>This is where integration tests become essential.
An integration test for this scenario would:</p>
<ul>
<li>Insert real bookings into a test database</li>
<li>Attempt to create overlapping bookings</li>
<li>Verify that the overlap detection works with real data</li>
</ul>
<p>The combination of unit and integration tests provides complete coverage:</p>
<ul>
<li>Unit tests verify the business logic flow</li>
<li>Integration tests verify the actual overlap detection logic</li>
</ul>
<p>This example highlights why we need different types of tests.
Unit tests are excellent for verifying behavior and logic flows,
but they can't verify the correctness of complex business rules that depend on real data interactions.</p>
<h2>Summary</h2>
<p>Unit testing Clean Architecture use cases requires careful thought about dependencies and behavior.
Here are the key points:</p>
<ul>
<li>Mock all external dependencies</li>
<li>Test both success and failure scenarios</li>
<li>Verify exception handling logic</li>
<li>Use descriptive test names</li>
<li>Follow the Arrange-Act-Assert pattern</li>
</ul>
<p>Good tests act as documentation.
They show how the code should behave and catch issues before they reach production.
Invest time in writing good tests - your future self will thank you.</p>
<p>Want to dive deeper into testing Clean Architecture applications?
I cover this and much more in my <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture</strong></a> course.
You'll learn how to write effective unit tests, integration tests, and end-to-end tests that give you real confidence in your system.</p>
<p>Writing quality tests is a skill that improves with practice and understanding.
Take the time to write them well.</p>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_123.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[What Rewriting a 40-Year-Old Project Taught Me About Software Development]]></title>
            <link>https://milanjovanovic.tech/blog/what-rewriting-a-40-year-old-project-taught-me-about-software-development</link>
            <guid isPermaLink="false">what-rewriting-a-40-year-old-project-taught-me-about-software-development</guid>
            <pubDate>Sat, 28 Dec 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[When tasked with modernizing a 40-year-old manufacturing system written in APL, I faced both technical complexity and organizational challenges that shaped my understanding of successful software development. This article shares key lessons learned during a four-year journey of rewriting critical legacy code while keeping a $10M business running smoothly.]]></description>
            <content:encoded><![CDATA[<p>&quot;Your task is to rewrite this system. It powers our entire operation. Oh, and it's written in APL.&quot;</p>
<p>That's how my journey with this legacy rewrite began.
For those unfamiliar with <a href="https://en.wikipedia.org/wiki/APL_(programming_language)">APL</a>,
it's a programming language from the 1960s known for its unique mathematical notation and array manipulation capabilities.
Finding developers who know APL today is about as easy as finding a floppy disk drive in a modern computer.</p>
<p>The system has grown over four decades.
It started as a simple inventory management tool and evolved into a comprehensive <a href="https://en.wikipedia.org/wiki/Enterprise_resource_planning">ERP</a> system.
More than 460+ database tables.
Countless business rules embedded in the code.
Complex integrations with every part of the business process.
The system is the backbone of a manufacturing operation, that generates over $10 million in annual revenue.</p>
<p>Our mission was clear but daunting: modernize this system using .NET, PostgreSQL, and React.</p>
<p>The catch?
The business needed to keep running during the transition.
No downtime.
No data loss.
No disruption to daily operations.</p>
<p>This wasn't just a technical challenge.
It was a lesson in managing complexity, understanding legacy business processes, and navigating organizational dynamics.</p>
<p>So here's that story and the lessons learned.</p>
<h2>Initial State: Understanding the Legacy</h2>
<p>The first challenge was understanding how this massive system actually worked.
The codebase had grown organically over four decades, maintained by a single development team.
They were now in their 60s and looking to retire.</p>
<p>Walking into the first codebase review was like opening a time capsule.
APL's concise syntax meant that complex business logic could be written in just a few lines.
Beautiful, if you could read it.
Terrifying, if you couldn't.
And most of us couldn't.</p>
<p>The original team was invaluable during the knowledge transfer.
They knew every quirk, every special case, every business rule that had been added over the decades.
But there's only so much you can learn from conversations.
Documentation was sparse.
What existed was outdated.
The real documentation was in the heads of the original developers.</p>
<p>We spent weeks mapping the system's functionality:</p>
<ul>
<li>The core manufacturing process was spread across 50+ tables with complex interdependencies</li>
<li>Inventory management touched nearly every part of the system</li>
<li>Custom reporting tools have been built over decades to meet specific business needs</li>
<li>Integration points with external components were handled through a maze of stored procedures</li>
</ul>
<p>Tables that started with basic schemas had grown to include hundreds of columns.
Some columns were no longer used but couldn't be removed because no one was sure if some obscure report still needed them.</p>
<p>What made this particularly challenging was the disconnect between the business processes and their technical implementation.
The business would describe a simple workflow, but the technical implementation would reveal layers of complexity added over years of edge cases and special requirements.</p>
<p>We needed a systematic approach to understanding this beast.
We started by mapping business processes and their corresponding technical implementations.
This helped us identify the core domains that would later influence our modular architecture.
More importantly, it helped us understand the true scope of what we were dealing with.</p>
<h2>The Product vs. Engineering Conflict</h2>
<p>Management wanted quick wins.
They pushed us to start with the simplest components.
This created tension between product management and the development team.</p>
<p>Product management's perspective was straightforward: show progress to the business.
They needed visible results to justify the investment in the rewrite.
The business was spending significant money, and they wanted to see returns quickly.</p>
<p>The development team saw a different reality.
We knew that starting with peripheral features meant building on shaky ground.
The core business logic would remain in the legacy system, making every integration point more complex.
This technical debt would compound over time.</p>
<p>As a technical lead, I strongly opposed this approach.
My argument was simple: the core manufacturing process was the heart of the system.
Every peripheral feature depended on it.
By postponing its migration, we created a tangled web of dependencies between old and new systems.
Each new feature we migrated would need complex synchronization with the legacy core.
We were building on quicksand.</p>
<p>I advocated for focusing on the core domain first.
Yes, it would take longer to show the first results.
But it would create a solid foundation for everything that followed.
The business would have to wait longer for visible progress, but the overall migration would be faster and more reliable.</p>
<p>Neither side was wrong in their objectives.
Product management had valid concerns about showing progress.
The development team had valid concerns about technical sustainability.
But this misalignment led to compromises that impacted the project timeline.
To this day, I believe we would have finished the migration sooner if we had started with the core business logic.</p>
<h2>Software Architecture: Building for the Future</h2>
<p>During the discovery phase, we identified distinct business domains within the system.
This led us to implement a <a href="what-is-a-modular-monolith"><strong>modular monolith architecture</strong></a>.
Each module would be self-contained but able to communicate with others through a shared event bus:</p>
<p><img src="/blogs/mnw_122/legacy_system_with_modular_monolith.png" alt="Modular monolith architecture with legacy system and a message broker for communication."></p>
<p>Key architectural decisions:</p>
<ol>
<li>
<p><a href="monolith-to-microservices-how-a-modular-monolith-helps"><strong>Modular monolith</strong></a>:
Each module represented a distinct business domain.
This provided a clear path to potential future microservices if needed.</p>
</li>
<li>
<p><a href="modular-monolith-communication-patterns"><strong>Asynchronous communication</strong></a>:
Modules communicated through events using RabbitMQ.
This reduced coupling and improved system resilience.</p>
</li>
<li>
<p><a href="modular-monolith-data-isolation"><strong>Shared database with boundaries</strong></a>:
While all modules used the same PostgreSQL database, each had its own set of tables and schemas.
This helped us maintain logical separation.</p>
</li>
<li>
<p><a href="dotnet-aspire-a-game-changer-for-cloud-native-development"><strong>Cloud-ready design</strong></a>:
The system was deployed to AWS using containerization.
A Jenkins pipeline enabled deployments to multiple environments in minutes.</p>
</li>
</ol>
<h2>The Data Sync Challenge</h2>
<p>The two-way data synchronization was more complex than initially anticipated.
Here's why we couldn't use existing <a href="https://en.wikipedia.org/wiki/Change_data_capture">change data capture</a> (CDC) solutions like <a href="https://debezium.io/">Debezium</a>:</p>
<ol>
<li>
<p><strong>Complex transformations</strong>:
Many legacy tables required data from multiple new tables.
This wasn't a simple one-to-one mapping that CDC tools excel at.</p>
</li>
<li>
<p><strong>Business logic in sync</strong>:
The sync process needed to apply business rules during transformation.
This went beyond what most replication tools provide.</p>
</li>
<li>
<p><strong>Bidirectional requirements</strong>:
We needed to sync both ways while preventing infinite loops.
The legacy system remained the source of truth for non-migrated components.</p>
</li>
</ol>
<p><img src="/blogs/mnw_122/data_sync.png" alt="Data sync flow between modern and legacy system."></p>
<p>We built a custom solution using RabbitMQ for message transport.
While this worked for us, the lesson remains: evaluate existing tools thoroughly before building custom solutions.
Even if you can't use them entirely, you might learn valuable patterns from their approaches.</p>
<h2>Key Technical Lessons</h2>
<ol>
<li>
<p><strong>Modular architecture pays off</strong>:
The modular monolith approach made the system easier to understand and maintain.
Each module had clear boundaries and responsibilities.</p>
</li>
<li>
<p><strong>Invest in deployment automation</strong>:
The CI/CD pipeline was crucial.
It allowed us to deploy confidently and frequently, reducing the risk of each change.</p>
</li>
<li>
<p><strong>Message-based integration</strong>:
Async communication between modules provided the flexibility needed for the gradual migration.</p>
</li>
<li>
<p><strong>Data sync complexity</strong>:
Don't underestimate the complexity of data synchronization in legacy migrations.
Whether using existing tools or building custom solutions, this will be a major challenge.</p>
</li>
</ol>
<h2>The Human Factor</h2>
<p>Technical challenges are only part of the story.
The success of legacy rewrites depends heavily on managing different stakeholders:</p>
<ol>
<li>Product Management needs to see progress</li>
<li>Development teams need time to do things right</li>
<li>The business needs to keep running</li>
<li>The legacy team needs to transfer knowledge</li>
</ol>
<p>Finding the right balance between these competing needs can be tricky.</p>
<p>We found several approaches that helped:</p>
<ul>
<li>Regular stakeholder meetings where each group could voice concerns</li>
<li>Transparent project tracking visible to all parties</li>
<li>Clear communication about technical decisions and their business impact</li>
<li>Celebration of both technical and business milestones</li>
<li>Documentation of both technical and institutional knowledge</li>
</ul>
<p>I can't stress enough how important it was to document the knowledge acquired over four decades of operating the legacy system.
When the original team retired, we had a comprehensive set of documents that explained every business rule and every edge case.</p>
<h2>Results That Matter</h2>
<p>Four years later, the system is thriving.
The cloud infrastructure provides reliability and scalability.
The <a href="/modular-monolith-architecture"><strong>modular monolith architecture</strong></a> makes it maintainable.
The automated deployment pipeline enables rapid updates.</p>
<p>But the journey taught us valuable lessons about balancing technical needs with business pressures.
Success in legacy rewrites requires more than just technical excellence.
It requires understanding the business domain, managing stakeholder expectations, and making pragmatic architectural decisions.</p>
<p>Software architecture matters, but so does the human factor. Plan for both.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_122.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Scheduling Background Jobs With Quartz in .NET (advanced concepts)]]></title>
            <link>https://milanjovanovic.tech/blog/scheduling-background-jobs-with-quartz-in-dotnet-advanced-concepts</link>
            <guid isPermaLink="false">scheduling-background-jobs-with-quartz-in-dotnet-advanced-concepts</guid>
            <pubDate>Sat, 21 Dec 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Quartz.NET is a powerful job scheduling library, but integrating it properly with ASP.NET Core requires careful consideration. Here's what I learned about setting up Quartz.NET with proper observability, persistence, and job scheduling patterns.]]></description>
            <content:encoded><![CDATA[<p>Most <a href="http://ASP.NET">ASP.NET</a> Core applications need to handle <a href="running-background-tasks-in-asp-net-core"><strong>background processing</strong></a> -
from sending reminder emails to running cleanup tasks.
While there are many ways to implement background jobs, <a href="https://www.quartz-scheduler.net/">Quartz.NET</a>
stands out with its robust scheduling capabilities, persistence options, and production-ready features.</p>
<p>In this article, we'll look at:</p>
<ul>
<li>Setting up <a href="http://Quartz.NET">Quartz.NET</a> with <a href="http://ASP.NET">ASP.NET</a> Core and proper observability</li>
<li>Implementing both on-demand and recurring jobs</li>
<li>Configuring persistent storage with PostgreSQL</li>
<li>Handling job data and monitoring execution</li>
</ul>
<p>Let's start with the basic setup and build our way up to a production-ready configuration.</p>
<h2>Setting Up Quartz With <a href="http://ASP.NET">ASP.NET</a> Core</h2>
<p>First, let's set up Quartz with proper instrumentation.</p>
<p>We'll need to install some NuGet packages:</p>
<pre><code class="language-powershell">Install-Package Quartz.Extensions.Hosting
Install-Package Quartz.Serialization.Json

# This might be in prerelease
Install-Package OpenTelemetry.Instrumentation.Quartz
</code></pre>
<p>Next, we'll configure the Quartz services and OpenTelemetry instrumentation and start the scheduler:</p>
<pre><code class="language-csharp">builder.Services.AddQuartz();

// Add Quartz.NET as a hosted service
builder.Services.AddQuartzHostedService(options =&gt;
{
    options.WaitForJobsToComplete = true;
});

builder.Services.AddOpenTelemetry()
    .WithTracing(tracing =&gt;
    {
        tracing
            .AddHttpClientInstrumentation()
            .AddAspNetCoreInstrumentation()
            .AddQuartzInstrumentation();
    })
    .UseOtlpExporter();
</code></pre>
<p>This is all we need at the start.</p>
<h2>Defining and Scheduling Jobs</h2>
<p>To define a <a href="scheduling-background-jobs-with-quartz-net"><strong>background job</strong></a>, you have to implement the <code>IJob</code> interface.
All job implementations run as scoped services, so you can inject dependencies as needed.
Quartz allows you to pass data to a job using the <code>JobDataMap</code> dictionary.
It's recommended to only use primitive types for job data to avoid serialization issues.</p>
<p>When executing the job, there are a few ways to fetch job data:</p>
<ul>
<li><code>JobDataMap</code> - a dictionary of key-value pairs
<ul>
<li><code>JobExecutionContext.JobDetail.JobDataMap</code> - job-specific data</li>
<li><code>JobExecutionContext.Trigger.TriggerDataMap</code> - trigger-specific data</li>
</ul>
</li>
<li><code>MergedJobDataMap</code> - combines job data with trigger data</li>
</ul>
<p>It's a best practice to use <code>MergedJobDataMap</code> to retrieve job data.</p>
<pre><code class="language-csharp">public class EmailReminderJob(ILogger&lt;EmailReminderJob&gt; logger, IEmailService emailService) : IJob
{
    public const string Name = nameof(EmailReminderJob);

    public async Task Execute(IJobExecutionContext context)
    {
        // Best practice: Prefer using MergedJobDataMap
        var data = context.MergedJobDataMap;

        // Get job data - note that this isn't strongly typed
        string? userId = data.GetString(&quot;userId&quot;);
        string? message = data.GetString(&quot;message&quot;);

        try
        {
            await emailService.SendReminderAsync(userId, message);

            logger.LogInformation(&quot;Sent reminder to user {UserId}: {Message}&quot;, userId, message);
        }
        catch (Exception ex)
        {
            logger.LogError(ex, &quot;Failed to send reminder to user {UserId}&quot;, userId);

            // Rethrow to let Quartz handle retry logic
            throw;
        }
    }
}
</code></pre>
<p>One thing to note: <code>JobDataMap</code> isn't strongly typed. This is a limitation we have to live with, but we can mitigate it by:</p>
<ol>
<li>Using constants for key names</li>
<li>Validating data early in the <code>Execute</code> method</li>
<li>Creating wrapper services for job scheduling</li>
</ol>
<p>Now, let's discuss scheduling jobs.</p>
<p>Here's how to schedule one-time reminders:</p>
<pre><code class="language-csharp">public record ScheduleReminderRequest(
    string UserId,
    string Message,
    DateTime ScheduleTime
);

// Schedule a one-time reminder
app.MapPost(&quot;/api/reminders/schedule&quot;, async (
    ISchedulerFactory schedulerFactory,
    ScheduleReminderRequest request) =&gt;
{
    var scheduler = await schedulerFactory.GetScheduler();

    var jobData = new JobDataMap
    {
        { &quot;userId&quot;, request.UserId },
        { &quot;message&quot;, request.Message }
    };

    var job = JobBuilder.Create&lt;EmailReminderJob&gt;()
        .WithIdentity($&quot;reminder-{Guid.NewGuid()}&quot;, &quot;email-reminders&quot;)
        .SetJobData(jobData)
        .Build();

    var trigger = TriggerBuilder.Create()
        .WithIdentity($&quot;trigger-{Guid.NewGuid()}&quot;, &quot;email-reminders&quot;)
        .StartAt(request.ScheduleTime)
        .Build();

    await scheduler.ScheduleJob(job, trigger);

    return Results.Ok(new { scheduled = true, scheduledTime = request.ScheduleTime });
})
.WithName(&quot;ScheduleReminder&quot;)
.WithOpenApi();

</code></pre>
<p>The endpoint schedules one-time email reminders using Quartz.
It creates a job with user data, sets up a trigger for the specified time, and schedules them together.
The <code>EmailReminderJob</code> receives a unique identity in the <code>email-reminders</code> group.</p>
<p>Here's a sample request you can use to test this out:</p>
<pre><code>POST /api/reminders/schedule
{
    &quot;userId&quot;: &quot;user123&quot;,
    &quot;message&quot;: &quot;Important meeting!&quot;,
    &quot;scheduleTime&quot;: &quot;2024-12-17T15:00:00&quot;
}
</code></pre>
<h2>Scheduling Recurring Jobs</h2>
<p>For recurring background jobs, you can use <a href="https://www.quartz-scheduler.net/documentation/quartz-3.x/tutorial/crontriggers.html">cron schedules</a>:</p>
<pre><code class="language-csharp">public record RecurringReminderRequest(
    string UserId,
    string Message,
    string CronExpression
);

// Schedule a recurring reminder
app.MapPost(&quot;/api/reminders/schedule/recurring&quot;, async (
    ISchedulerFactory schedulerFactory,
    RecurringReminderRequest request) =&gt;
{
    var scheduler = await schedulerFactory.GetScheduler();

    var jobData = new JobDataMap
    {
        { &quot;userId&quot;, request.UserId },
        { &quot;message&quot;, request.Message }
    };

    var job = JobBuilder.Create&lt;EmailReminderJob&gt;()
        .WithIdentity($&quot;recurring-{Guid.NewGuid()}&quot;, &quot;recurring-reminders&quot;)
        .SetJobData(jobData)
        .Build();

    var trigger = TriggerBuilder.Create()
        .WithIdentity($&quot;recurring-trigger-{Guid.NewGuid()}&quot;, &quot;recurring-reminders&quot;)
        .WithCronSchedule(request.CronExpression)
        .Build();

    await scheduler.ScheduleJob(job, trigger);

    return Results.Ok(new { scheduled = true, cronExpression = request.CronExpression });
})
.WithName(&quot;ScheduleRecurringReminder&quot;)
.WithOpenApi();
</code></pre>
<p>Cron triggers are more powerful than simple triggers.
They allow you to define complex schedules like &quot;every weekday at 10 AM&quot; or &quot;every 15 minutes&quot;.
Quartz supports cron expressions with seconds, minutes, hours, days, months, and years.</p>
<p>Here's a sample request if you want to test this:</p>
<pre><code>POST /api/reminders/schedule/recurring
{
    &quot;userId&quot;: &quot;user123&quot;,
    &quot;message&quot;: &quot;Daily standup&quot;,
    &quot;cronExpression&quot;: &quot;0 0 10 ? * MON-FRI&quot;
}
</code></pre>
<h2>Job Persistence Setup</h2>
<p>By default, Quartz uses in-memory storage, which means your jobs are lost when the application restarts.
For production environments, you'll want to use a persistent store.
Quartz supports several <a href="https://www.quartz-scheduler.net/documentation/quartz-3.x/tutorial/job-stores.html">database providers</a>,
including SQL Server, PostgreSQL, MySQL, and Oracle.</p>
<p>Let's look at how to set up persistent storage with proper schema isolation:</p>
<pre><code class="language-csharp">builder.Services.AddQuartz(options =&gt;
{
    options.AddJob&lt;EmailReminderJob&gt;(c =&gt; c
        .StoreDurably()
        .WithIdentity(EmailReminderJob.Name));

    options.UsePersistentStore(persistenceOptions =&gt;
    {
        persistenceOptions.UsePostgres(cfg =&gt;
        {
            cfg.ConnectionString = connectionString;
            cfg.TablePrefix = &quot;scheduler.qrtz_&quot;;
        },
        dataSourceName: &quot;reminders&quot;); // Database name

        persistenceOptions.UseNewtonsoftJsonSerializer();
        persistenceOptions.UseProperties = true;
    });
});
</code></pre>
<p>A few important things to note here:</p>
<ul>
<li>The <code>TablePrefix</code> setting helps organize Quartz tables in your database - in this case, placing them in a dedicated <code>scheduler</code> schema</li>
<li>You'll need to run the appropriate database scripts to create these tables</li>
<li>Each database provider has its own <a href="https://github.com/quartznet/quartznet/tree/main/database/tables">setup scripts</a> -
check the Quartz documentation for your chosen provider</li>
</ul>
<h3>Durable Jobs</h3>
<p>Notice how we're configuring the <code>EmailReminderJob</code> with <code>StoreDurably</code>?
This is a powerful pattern that lets you define your jobs once and reuse them with different triggers.
Here's how to schedule a stored job:</p>
<pre><code class="language-csharp">public async Task ScheduleReminder(string userId, string message, DateTime scheduledTime)
{
    var scheduler = await _schedulerFactory.GetScheduler();

    // Reference the stored job by its identity
    var jobKey = new JobKey(EmailReminderJob.Name);

    var trigger = TriggerBuilder.Create()
        .ForJob(jobKey)  // Reference the durable job
        .WithIdentity($&quot;trigger-{Guid.NewGuid()}&quot;)
        .UsingJobData(&quot;userId&quot;, userId)
        .UsingJobData(&quot;message&quot;, message)
        .StartAt(scheduledTime)
        .Build();

    await scheduler.ScheduleJob(trigger);  // Note: just passing the trigger
}
</code></pre>
<p>This approach has several benefits:</p>
<ul>
<li>Job definitions are centralized in your startup configuration</li>
<li>You can't accidentally schedule a job that hasn't been properly configured</li>
<li>Job configurations are consistent across all schedules</li>
</ul>
<h2>Summary</h2>
<p>Getting <strong>Quartz</strong> set up properly in .NET involves more than just adding the NuGet package.</p>
<p>Pay attention to:</p>
<ol>
<li>Proper job definition and data handling with <code>JobDataMap</code></li>
<li>Setting up both one-time and recurring job schedules</li>
<li>Configuring persistent storage with proper schema isolation</li>
<li>Using durable jobs to maintain consistent job definitions</li>
</ol>
<p>Each of these elements contributes to a reliable background processing system that can grow with your application's needs.
A good example of using background jobs is when you want to <a href="building-async-apis-in-aspnetcore-the-right-way"><strong>build asynchronous APIs</strong></a>.</p>
<p>Good luck out there, and I'll see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_121.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Internal vs. Public APIs in Modular Monoliths]]></title>
            <link>https://milanjovanovic.tech/blog/internal-vs-public-apis-in-modular-monoliths</link>
            <guid isPermaLink="false">internal-vs-public-apis-in-modular-monoliths</guid>
            <pubDate>Sat, 14 Dec 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Every article about modular monoliths tells you to use public APIs between modules, but they rarely explain why these APIs exist or how to design them properly. After building several large-scale modular monoliths, I've learned that public APIs are not just about clean code - they're about controlling chaos.]]></description>
            <content:encoded><![CDATA[<p>Every article about modular monoliths tells you to use public APIs between modules.
But they rarely tell you why these APIs exist or how to design them properly.</p>
<p>A modular monolith organizes an application into independent modules that have clear boundaries.
The module boundaries are logical and group related business capabilities together.</p>
<p>After building several large-scale modular monoliths, I've learned that public APIs are not just about clean code - they're about controlling chaos.
Let me show you what I mean.</p>
<h2>The Reality of Module Communication</h2>
<p>Here's what nobody tells you about public APIs in modular monoliths: they represent intentional coupling points.
Yes, you read that right.
Public APIs don't eliminate coupling - they make it explicit and controllable.</p>
<p>When Module A needs something from Module B, you have three options:</p>
<ol>
<li>Let Module A read directly from Module B's database</li>
<li>Let Module A access Module B's internal services</li>
<li>Create a public API that explicitly defines what Module A can do</li>
</ol>
<p><img src="/blogs/mnw_120/module_communication_options.png" alt="Module communication options: direct database access, calling internal services, calling public API."></p>
<p>The first two options lead to chaos.
I've seen entire systems become unmaintainable because every module was freely accessing the data and services of other modules.</p>
<p>The previous options are examples of synchronous communication between modules.
But you can also implement <a href="modular-monolith-communication-patterns"><strong>asynchonrous module communication</strong></a> using messaging.
We have to adjust the technical implementation.
However, modules still have a public API in message contracts.</p>
<h2>Why We Need Public APIs</h2>
<p>Public APIs serve three critical purposes:</p>
<ol>
<li><strong>Contract Definition</strong>: They explicitly state what other modules can and cannot do</li>
<li><strong>Dependency Control</strong>: They force you to think about module dependencies</li>
<li><strong>Change Management</strong>: They provide a stable interface while allowing internal changes</li>
</ol>
<p>Here's a practical example. Imagine you have an Orders module and a Shipping module.</p>
<p>This is what you want to avoid:</p>
<pre><code class="language-csharp">public class ShippingService
{
    private readonly OrdersDbContext _ordersDb; // Direct database access

    public async Task ShipOrder(string orderId)
    {
        // Directly reading from another module's database
        var order = await _ordersDb.Orders
            .Include(o =&gt; o.Lines)
            .FirstOrDefaultAsync(o =&gt; o.Id == orderId);

        // What happens if the Orders module changes its schema?
        // What if it moves to a different database?
    }
}
</code></pre>
<p>This is what you want to achieve instead:</p>
<pre><code class="language-csharp">public class ShippingService
{
    private readonly IOrdersModule _orders; // Public API access

    public async Task ShipOrder(string orderId)
    {
        // Using the public API
        var order = await _orders.GetOrderForShippingAsync(orderId);

        // The Orders module can change its internals
        // as long as it maintains this contract
    }
}
</code></pre>
<h2>Controlling What Gets Exposed</h2>
<p>The hardest part of designing public APIs is deciding what to expose.
Here's my rule of thumb:</p>
<ol>
<li>Start with nothing public</li>
<li>Expose only what other modules actually need</li>
<li>Design the API around use cases, not data</li>
</ol>
<p>Here's how this looks in practice:</p>
<pre><code class="language-csharp">public interface IOrdersModule
{
    // Don't expose generic CRUD operations
    // Task&lt;Order&gt; GetOrderAsync(string orderId); // Bad

    // Instead, expose specific use cases
    Task&lt;OrderShippingInfo&gt; GetOrderForShippingAsync(string orderId);
    Task&lt;OrderPaymentInfo&gt; GetOrderForPaymentAsync(string orderId);
    Task&lt;OrderSummary&gt; GetOrderForCustomerAsync(string orderId);
}
</code></pre>
<h2>Protecting Your Module's Data</h2>
<p>Public APIs aren't enough.
You also need to <a href="modular-monolith-data-isolation"><strong>protect your module's data</strong></a>.
Here's what I've found works:</p>
<ol>
<li><strong>Separate Schemas</strong>: Each module gets its own database schema.</li>
</ol>
<pre><code class="language-sql">CREATE SCHEMA Orders;
CREATE SCHEMA Shipping;

-- Orders module can only access its schema
CREATE USER OrdersUser WITH DEFAULT_SCHEMA = Orders;
GRANT SELECT, INSERT, UPDATE, DELETE ON SCHEMA::Orders TO OrdersUser;

-- Shipping module can only access its schema
CREATE USER ShippingUser WITH DEFAULT_SCHEMA = Shipping;
GRANT SELECT, INSERT, UPDATE, DELETE ON SCHEMA::Shipping TO ShippingUser;
</code></pre>
<p>We can also lock down the user's access to a given schema to only allow reading and writing data.</p>
<ol start="2">
<li><strong>Different Connection Strings</strong>: Each module gets its own database user with a respective connection string.</li>
</ol>
<pre><code class="language-csharp">builder.Services.AddDbContext&lt;OrdersDbContext&gt;(options =&gt;
    options.UseSqlServer(builder.Configuration.GetConnectionString(&quot;OrdersConnection&quot;)));

builder.Services.AddDbContext&lt;ShippingDbContext&gt;(options =&gt;
    options.UseSqlServer(builder.Configuration.GetConnectionString(&quot;ShippingConnection&quot;)));
</code></pre>
<p>If you want to learn more about this, check out this article about <a href="using-multiple-ef-core-dbcontext-in-single-application"><strong>using multiple EF Core DbContexts</strong></a>.</p>
<ol start="3">
<li><strong>Read Models</strong>: Create specific read models for other modules.</li>
</ol>
<pre><code class="language-csharp">internal class Order
{
    // Internal domain model with full complexity
}

public class OrderShippingInfo
{
    // Public DTO with only what shipping needs
    public string OrderId { get; init; }
    public Address ShippingAddress { get; init; }
    public List&lt;ShippingItem&gt; Items { get; init; }
}
</code></pre>
<h2>Dealing with Cross-Cutting Concerns</h2>
<p>Some features naturally span multiple modules.
For example, when a customer views their order history, you might need data from the Orders, Shipping, and Payments modules.</p>
<p>Don't try to force this through module APIs.
Instead:</p>
<ol>
<li>Create a separate query model</li>
<li>Use event-driven patterns to keep it updated</li>
<li>Own it in a dedicated module or one of the existing modules</li>
</ol>
<pre><code class="language-csharp">public class OrderHistoryModule
{
    public async Task&lt;CustomerOrderHistory&gt; GetOrderHistoryAsync(string customerId)
    {
        // Read from a dedicated read model that's kept
        // updated through events from other modules
        return await _orderHistoryRepository.GetCustomerHistoryAsync(customerId);
    }
}
</code></pre>
<h2>Summary</h2>
<p>Public APIs in modular monoliths are not about preventing coupling - they're about controlling it.
Every public API is a contract that says: &quot;Yes, these modules are coupled, and this is exactly how they depend on each other.&quot;</p>
<p>The goal isn't to eliminate dependencies between modules.
The goal is to make them explicit, controlled, and maintainable.</p>
<p>Get this right, and your modular monolith will be easier to maintain, test, and evolve.
Get it wrong, and you'll end up with a distributed big ball of mud.</p>
<p>Want to master building modular monoliths with clean APIs and event-driven patterns?
Check out my <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a> course, where I'll show you how to build maintainable systems
using practical examples from real projects.</p>
<p>That's all for today. Stay awesome, and I'll see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_120.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Central Package Management in .NET - Simplify NuGet Dependencies]]></title>
            <link>https://milanjovanovic.tech/blog/central-package-management-in-net-simplify-nuget-dependencies</link>
            <guid isPermaLink="false">central-package-management-in-net-simplify-nuget-dependencies</guid>
            <pubDate>Sat, 07 Dec 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Managing NuGet packages across multiple .NET projects used to be a nightmare of version mismatches and maintenance headaches, but Central Package Management (CPM) offers a powerful solution by letting you control all package versions from a single source of truth. Learn how CPM can simplify your dependency management, prevent version conflicts, and make your .NET development workflow smoother.]]></description>
            <content:encoded><![CDATA[<p>I remember the days when managing NuGet packages across multiple projects was a real pain.
You know what I mean - you open a large solution and find out every project uses a different version of the same package.
Not fun!</p>
<p>Let me show you how <strong>Central Package Management</strong> (CPM) in .NET can fix this problem once and for all.</p>
<h2>The Problem We Need to Solve</h2>
<p>I often work with solutions that have lots of projects.
It's not uncommon to have solutions with 30 or more projects.
Each one needs similar packages like Serilog or Polly.
Most test projects I create depend on xUnit.
Before CPM, keeping track of package versions was a mess:</p>
<ul>
<li>One project uses Serilog <code>4.1.0</code></li>
<li>Another uses Serilog <code>4.0.2</code></li>
<li>And somehow, a third one uses Serilog <code>3.1.1</code></li>
</ul>
<p>This causes real problems.
Different versions can behave differently, leading to weird bugs that are hard to track down.
I've wasted many hours fixing issues caused by version mismatches.</p>
<h2>How Central Package Management Helps</h2>
<p>Think of CPM as a control center for all your package versions.
Instead of setting versions in each project, you set them once in one place.
Then, you just reference a package you want to use without specifying the version.
It's that simple.</p>
<p>Here's what you need to use <a href="https://learn.microsoft.com/en-us/nuget/consume-packages/central-package-management">Central Package Management</a>:</p>
<ul>
<li>NuGet 6.2 or newer</li>
<li>.NET SDK 6.0.300 or newer</li>
<li>If you use Visual Studio, you need version 2022 17.2 or newer</li>
</ul>
<h2>Setting It Up</h2>
<p>Let me show you how to set up CPM.
It's easier than you might think.</p>
<ol>
<li>First, create a file called <code>Directory.Packages.props</code> in your solution's main folder:</li>
</ol>
<pre><code class="language-xml">&lt;Project&gt;
  &lt;PropertyGroup&gt;
    &lt;ManagePackageVersionsCentrally&gt;true&lt;/ManagePackageVersionsCentrally&gt;
  &lt;/PropertyGroup&gt;
  &lt;ItemGroup&gt;
    &lt;PackageVersion Include=&quot;Newtonsoft.Json&quot; Version=&quot;13.0.3&quot; /&gt;
    &lt;PackageVersion Include=&quot;Serilog&quot; Version=&quot;4.1.0&quot; /&gt;
    &lt;PackageVersion Include=&quot;Polly&quot; Version=&quot;8.5.0&quot; /&gt;
  &lt;/ItemGroup&gt;
&lt;/Project&gt;
</code></pre>
<p>Note the use of <code>PackageVersion</code> to define NuGet dependencies.</p>
<ol start="2">
<li>In your project files, you can list the packages using <code>PackageReference</code> without the version component:</li>
</ol>
<pre><code class="language-xml">&lt;ItemGroup&gt;
  &lt;PackageReference Include=&quot;Newtonsoft.Json&quot; /&gt;
  &lt;PackageReference Include=&quot;AutoMapper&quot; /&gt;
  &lt;PackageReference Include=&quot;Polly&quot; /&gt;
&lt;/ItemGroup&gt;
</code></pre>
<p>That's it!
Now all your projects will use the same package versions.</p>
<h2>Cool Things You Can Do</h2>
<h3>Need a Different Version for One Project?</h3>
<p>Sometimes you might need a specific project to use a different version.
No problem!
Just add this to your project file:</p>
<pre><code class="language-xml">&lt;PackageReference Include=&quot;Serilog&quot; VersionOverride=&quot;3.1.1&quot; /&gt;
</code></pre>
<p>The <code>VersionOverride</code> property lets you define the specific version you want to use.</p>
<h3>Want a Package in Every Project?</h3>
<p>If you have packages that every project needs, you can make them global.
Define a <code>GlobalPackageReference</code> in your props file:</p>
<pre><code class="language-xml">&lt;ItemGroup&gt;
  &lt;GlobalPackageReference Include=&quot;SonarAnalyzer.CSharp&quot; Version=&quot;10.3.0.106239&quot; /&gt;
&lt;/ItemGroup&gt;
</code></pre>
<p>Now every project gets this package automatically!</p>
<h2>Migrating Existing Projects to Central Package Management</h2>
<ol>
<li>Create the <code>Directory.Packages.props</code> file at the solution root</li>
<li>Move all package versions from your <code>.csproj</code> files</li>
<li>Remove version attributes from <code>PackageReference</code> elements</li>
<li>Build your solution and fix any version conflicts</li>
<li>Test thoroughly before committing</li>
</ol>
<p>Here's a Powershell script that will list all NuGet package versions in your solution:</p>
<pre><code class="language-powershell"># Scan all .csproj files and aggregate unique package versions
$packages = Get-ChildItem -Filter *.csproj -Recurse |
    Get-Content |
    Select-String -Pattern '&lt;PackageReference Include=&quot;([^&quot;]+)&quot; Version=&quot;([^&quot;]+)&quot;' -AllMatches |
    ForEach-Object { $_.Matches } |
    Group-Object { $_.Groups[1].Value } |
    ForEach-Object { @{
        Name = $_.Name
        Versions = $_.Group.ForEach({ $_.Groups[2].Value }) | Select-Object -Unique
    }} |
    Sort-Object { $_.Name }

# Display results
$packages | ForEach-Object {
    &quot;$($_.Name) versions:&quot;
    $_.Versions | ForEach-Object { &quot;  $_&quot; }
}
</code></pre>
<p>There's also a CLI tool called <a href="https://github.com/Webreaper/CentralisedPackageConverter">CentralisedPackageConverter</a>,
which you can use to automate the migration.
It will scan for all .NET project files within that folder tree, gather all the versioned references in the projects,
remove the versions from the project files, and write the entries to the <code>Directory.Packages.props</code> file.</p>
<pre><code class="language-bash"># Install the tool globally
dotnet tool install CentralisedPackageConverter --global

# Convert your solution to use Central Package Management
central-pkg-converter /PATH_TO_YOUR_SOLUTION_FOLDER
</code></pre>
<h2>When Should You Use CPM?</h2>
<p>I don't see a compelling reason for not using this by default.</p>
<p>I recommend using CPM when:</p>
<ul>
<li>You have many projects that share packages</li>
<li>You're tired of fixing version-related bugs</li>
<li>You want to make sure everyone uses the same versions</li>
</ul>
<p>I recently added CPM to a solution with 30 projects.</p>
<p>Here's what happened:</p>
<ul>
<li>Fewer merge conflicts</li>
<li>Caught version problems early</li>
<li>Made it easier for new team members</li>
</ul>
<p>This was especially helpful while migrating from .NET 8 to .NET 9.</p>
<p>You can combine CPM with <a href="improving-code-quality-in-csharp-with-static-code-analysis"><strong>build configuration and static code analysis</strong></a>.</p>
<h2>Wrapping Up</h2>
<p>My tips for success with <strong>Central Package Management</strong>:</p>
<ol>
<li>When you add CPM to an existing solution, do it in its own change/PR</li>
<li>If you override a version, add a comment explaining why</li>
<li>Check your package versions regularly for updates</li>
<li>Only make packages global if you really need them everywhere</li>
</ol>
<p>Since I started using Central Package Management, managing NuGet packages has become much easier.
It's like having a single source of truth for all your package versions.</p>
<p>Hope this was helpful.
See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_119.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Implementing the Saga Pattern With MassTransit]]></title>
            <link>https://milanjovanovic.tech/blog/implementing-the-saga-pattern-with-masstransit</link>
            <guid isPermaLink="false">implementing-the-saga-pattern-with-masstransit</guid>
            <pubDate>Sat, 30 Nov 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Long-running business processes often require multiple services working together, but traditional distributed transactions can be problematic at scale. Learn how to implement the Saga pattern using MassTransit to break complex workflows into manageable steps with proper error handling and state management.]]></description>
            <content:encoded><![CDATA[<p>Long-running business processes often involve multiple services working together.
Think about an e-commerce order: you need to process the payment, update inventory, and notify shipping.
Traditional distributed transactions using two-phase commit (2PC) seem like a solution, but they come with significant drawbacks.</p>
<p>The main issue?
Services can't make assumptions about how other services operate or how long they'll take.
What if the payment service needs manual approval?
What if the inventory check is delayed?
Holding database locks across multiple services for extended periods isn't practical and can lead to system-wide issues.</p>
<p>Let's look at how the Saga pattern solves these problems and implement it using MassTransit.</p>
<h2>Understanding the Saga Pattern</h2>
<p>A Saga is a sequence of related local transactions where each step has a defined action and a compensating action if something goes wrong.
Instead of one big atomic transaction, we break the process into manageable steps that can be coordinated.</p>
<p>Here's a simple order processing flow:</p>
<div className="centered">
  ![Sequence diagram showing an order processing flow with multiple services.](/blogs/mnw_118/order_flow.png)
</div>
<p>Each step is independent and can be compensated if needed.
If the inventory service reports items are out of stock, we can refund the payment.
This approach gives us flexibility and reliability without tight coupling.</p>
<h2>Implementing Sagas with MassTransit</h2>
<p><a href="using-masstransit-with-rabbitmq-and-azure-service-bus"><strong>MassTransit</strong></a> provides a state machine-based approach to implementing Sagas through its integration with Automatonymous.
Understanding how state machines work is crucial for implementing effective sagas.</p>
<h3>State Machine Fundamentals</h3>
<p>A <a href="https://masstransit.io/documentation/patterns/saga/state-machine">state machine</a> consists of several key components:</p>
<ol>
<li><strong>States</strong>: Represent the possible conditions of your saga instance</li>
<li><strong>Events</strong>: Messages that can trigger state transitions</li>
<li><strong>Behaviors</strong>: Actions that occur when events are received in specific states</li>
<li><strong>Instance</strong>: Contains the data and current state for a specific saga</li>
</ol>
<p>Here's the state machine diagram for our order processing saga:</p>
<div className="centered">
  ![State diagram showing an order saga state machine.](/blogs/mnw_118/order_saga_state_machine.png)
</div>
<p>Every state machine automatically includes <code>Initial</code> and <code>Final</code> states.
The <code>Initial</code> state is where new saga instances begin, and the <code>Final</code> state marks the end of a saga's lifecycle.</p>
<h3>Defining the Saga Instance</h3>
<p>The saga instance holds the data for a specific process:</p>
<pre><code class="language-csharp">public class OrderState : SagaStateMachineInstance
{
    public Guid CorrelationId { get; set; }
    public string CurrentState { get; set; }

    // Business data
    public decimal OrderTotal { get; set; }
    public string? PaymentIntentId { get; set; }
    public DateTime? OrderDate { get; set; }
    public string? CustomerEmail { get; set; }
}
</code></pre>
<p>The <code>CorrelationId</code> uniquely identifies the saga instance, while <code>CurrentState</code> tracks its current state.
Any additional properties store business data needed for the process.</p>
<h3>Defining Events</h3>
<p>Events are messages that can trigger state transitions. They must be correlated to a specific saga instance:</p>
<pre><code class="language-csharp">public record OrderSubmitted
{
    public Guid OrderId { get; init; }
    public decimal Total { get; init; }
    public string Email { get; init; }
}

public record PaymentProcessed
{
    public Guid OrderId { get; init; }
    public string PaymentIntentId { get; init; }
}

public record InventoryReserved
{
    public Guid OrderId { get; init; }
}

public record OrderFailed
{
    public Guid OrderId { get; init; }
    public string Reason { get; init; }
}
</code></pre>
<h3>Building the State Machine</h3>
<p>Let's implement the order processing flow as a state machine:</p>
<pre><code class="language-csharp">public class OrderStateMachine : MassTransitStateMachine&lt;OrderState&gt;
{
    public OrderStateMachine()
    {
        Event(() =&gt; OrderSubmitted, x =&gt; x.CorrelateById(m =&gt; m.Message.OrderId));
        Event(() =&gt; PaymentProcessed, x =&gt; x.CorrelateById(m =&gt; m.Message.OrderId));
        Event(() =&gt; InventoryReserved, x =&gt; x.CorrelateById(m =&gt; m.Message.OrderId));
        Event(() =&gt; OrderFailed, x =&gt; x.CorrelateById(m =&gt; m.Message.OrderId));

        InstanceState(x =&gt; x.CurrentState);

        Initially(
            When(OrderSubmitted)
                .Then(context =&gt;
                {
                    context.Saga.OrderTotal = context.Message.Total;
                    context.Saga.CustomerEmail = context.Message.Email;
                    context.Saga.OrderDate = DateTime.UtcNow;
                })
                .PublishAsync(context =&gt; context.Init&lt;ProcessPayment&gt;(new
                {
                    OrderId = context.Saga.CorrelationId,
                    Amount = context.Saga.OrderTotal
                }))
                .TransitionTo(ProcessingPayment)
        );

        During(ProcessingPayment,
            When(PaymentProcessed)
                .PublishAsync(context =&gt; context.Init&lt;ReserveInventory&gt;(new
                {
                    OrderId = context.Saga.CorrelationId
                }))
                .TransitionTo(ReservingInventory),
            When(OrderFailed)
                .TransitionTo(Failed)
                .Finalize()
        );

        During(ReservingInventory,
            When(InventoryReserved)
                .PublishAsync(context =&gt; context.Init&lt;OrderConfirmed&gt;(new
                {
                    OrderId = context.Saga.CorrelationId
                }))
                .TransitionTo(Completed)
                .Finalize(),
            When(OrderFailed)
                .PublishAsync(context =&gt; context.Init&lt;RefundPayment&gt;(new
                {
                    OrderId = context.Saga.CorrelationId,
                    Amount = context.Saga.OrderTotal
                }))
                .TransitionTo(Failed)
                .Finalize()
        );

        SetCompletedWhenFinalized();
    }

    public State ProcessingPayment { get; private set; }
    public State ReservingInventory { get; private set; }
    public State Completed { get; private set; }
    public State Failed { get; private set; }

    public Event&lt;OrderSubmitted&gt; OrderSubmitted { get; private set; }
    public Event&lt;PaymentProcessed&gt; PaymentProcessed { get; private set; }
    public Event&lt;InventoryReserved&gt; InventoryReserved { get; private set; }
    public Event&lt;OrderFailed&gt; OrderFailed { get; private set; }
}
</code></pre>
<p>The state machine defines the possible states and transitions.
Each step can trigger compensating actions if needed.
For example, if inventory reservation fails, we automatically trigger a payment refund.</p>
<h3>Implementing Message Consumers</h3>
<p>Services interact with the saga by consuming and publishing messages.
Here's an example of a payment processing consumer:</p>
<pre><code class="language-csharp">public class ProcessPaymentConsumer(
    IPaymentService paymentService,
    ILogger&lt;ProcessPaymentConsumer&gt; logger) : IConsumer&lt;ProcessPayment&gt;
{
    public async Task Consume(ConsumeContext&lt;ProcessPayment&gt; context)
    {
        try
        {
            var paymentResult = await paymentService.ProcessPaymentAsync(
                context.Message.OrderId,
                context.Message.Amount
            );

            if (paymentResult.Succeeded)
            {
                await context.Publish&lt;PaymentProcessed&gt;(new
                {
                    OrderId = context.Message.OrderId,
                    PaymentIntentId = paymentResult.PaymentIntentId
                });
            }
            else
            {
                await context.Publish&lt;OrderFailed&gt;(new
                {
                    OrderId = context.Message.OrderId,
                    Reason = paymentResult.FailureReason
                });
            }
        }
        catch (Exception ex)
        {
            logger.LogError(
                ex,
                &quot;Failed to process payment for order {OrderId}&quot;,
                context.Message.OrderId);

            await context.Publish&lt;OrderFailed&gt;(new
            {
                OrderId = context.Message.OrderId,
                Reason = &quot;Payment processing error&quot;
            });
        }
    }
}
</code></pre>
<p>Each consumer handles a specific part of the business process and communicates back to the saga through events.
This separation of concerns allows each service to focus on its specific responsibility while the saga coordinates the overall process.</p>
<h2>Configuring MassTransit with PostgreSQL Persistence</h2>
<p>To persist the saga state, we'll use PostgreSQL.
First, let's install the required packages:</p>
<pre><code class="language-powershell">Install-Package MassTransit.EntityFrameworkCore
Install-Package Npgsql.EntityFrameworkCore.PostgreSQL
</code></pre>
<p>Create a <code>DbContext</code> for saga persistence:</p>
<pre><code class="language-csharp">public class OrderSagaDbContext : SagaDbContext
{
    public OrderSagaDbContext(DbContextOptions options) : base(options)
    {
    }

    protected override IEnumerable&lt;ISagaClassMap&gt; Configurations
    {
        get
        {
            yield return new OrderStateMap();
        }
    }
}

public class OrderStateMap : SagaClassMap&lt;OrderState&gt;
{
    protected override void Configure(EntityTypeBuilder&lt;OrderState&gt; entity, ModelBuilder model)
    {
        entity.Property(x =&gt; x.CurrentState).HasMaxLength(64);
        entity.Property(x =&gt; x.CustomerEmail).HasMaxLength(256);
        entity.Property(x =&gt; x.PaymentIntentId).HasMaxLength(64);
    }
}
</code></pre>
<p>Configure MassTransit in your application:</p>
<pre><code class="language-csharp">builder.Services.AddDbContext&lt;OrderSagaDbContext&gt;(options =&gt;
    options.UseNpgsql(builder.Configuration.GetConnectionString(&quot;Postgres&quot;)));

builder.Services.AddMassTransit(x =&gt;
{
    x.AddSagaStateMachine&lt;OrderStateMachine, OrderState&gt;()
        .EntityFrameworkRepository(r =&gt;
        {
            r.ConcurrencyMode = ConcurrencyMode.Pessimistic;
            r.AddDbContext&lt;DbContext, OrderSagaDbContext&gt;();
            r.UsePostgres();
        });

    x.UsingRabbitMq((context, cfg) =&gt;
    {
        cfg.Host(builder.Configuration.GetConnectionString(&quot;RabbitMQ&quot;));
        cfg.ConfigureEndpoints(context);
    });
});
</code></pre>
<h2>Benefits of This Approach</h2>
<p>Using sagas with MassTransit provides several advantages:</p>
<ol>
<li><strong>Fault Tolerance</strong>: Each step can be retried independently.
And we can compensate failed operation, making our system more resilient.</li>
<li><strong>State Visibility</strong>: The saga's state machine provides clear insight into where each process stands,
making debugging and monitoring straightforward.</li>
<li><strong>Loose Coupling</strong>: Services communicate through messages, allowing them to evolve independently while maintaining process integrity.</li>
<li><strong>Maintainability</strong>: Changes can be made to individual steps without affecting others.</li>
</ol>
<p>The state machine approach also makes the business process explicit.
Each state and transition is clearly defined, making it easier to understand and maintain the workflow.</p>
<h2>Takeaway</h2>
<p>The Saga pattern with MassTransit provides a robust solution for managing distributed business processes.
Instead of dealing with distributed transactions, you get clear state management, automatic compensation for failures,
and the ability to handle long-running operations without blocking resources.</p>
<p>Want to dive deeper into event-driven architecture and distributed systems?
Check out my <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a> course,
where we explore sagas, event-driven patterns, and other essential techniques for building maintainable systems.</p>
<p>Good luck out there, and see you next week.</p>
<p><strong>P.S.</strong> You can find the code for the saga pattern implementation with MassTransit in <a href="https://github.com/m-jovanovic/saga-pattern-masstransit">this repository</a>.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_118.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Building Async APIs in ASP.NET Core - The Right Way]]></title>
            <link>https://milanjovanovic.tech/blog/building-async-apis-in-aspnetcore-the-right-way</link>
            <guid isPermaLink="false">building-async-apis-in-aspnetcore-the-right-way</guid>
            <pubDate>Sat, 23 Nov 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Not every API request needs to finish right away. Learn how to build better APIs by moving long-running tasks to the background. This guide shows practical examples using image processing in ASP.NET Core 9.]]></description>
            <content:encoded><![CDATA[<p>Most APIs follow a simple pattern.
The client sends a request.
The server does some work.
The server sends back a response.</p>
<p>This works well for fast operations like fetching data or simple updates.
But what about operations that take longer?</p>
<p>Think about processing large files, generating reports, or converting videos.
These operations can take minutes or even hours.</p>
<p>Making clients wait for these operations causes problems.</p>
<h2>Understanding Async APIs</h2>
<p>The key to handling long-running operations is to change how we think about API responses.
An async API splits work into two parts:</p>
<ul>
<li>Accept the request</li>
<li>Process it later</li>
</ul>
<p>First, we accept the request and return a tracking ID immediately.
This gives users a quick response.
Then, we process the actual work in the background, which won't block other requests.
Users can check the status of their request using the tracking ID whenever they want.</p>
<p>This is different from <code>async</code>/<code>await</code> in C#.
That's about handling many requests at once (concurrently).
This is about handling long-running tasks better.
We're not just making the code asynchronous - we're making the entire operation asynchronous from the user's perspective.</p>
<h2>The Problem with Sync APIs</h2>
<p>Let's see this in practice with image processing. A typical image upload API might look like this:</p>
<pre><code class="language-csharp">[HttpPost]
public async Task&lt;IActionResult&gt; UploadImage(IFormFile file)
{
    if (file is null)
    {
        return BadRequest();
    }

    // Save original image
    var originalPath = await SaveOriginalAsync(file);

    // Generate thumbnails
    var thumbnails = await GenerateThumbnailsAsync(originalPath);

    // Optimize all images
    await OptimizeImagesAsync(originalPath, thumbnails);

    return Ok(new { originalPath, thumbnails });
}
</code></pre>
<p>The client must wait while we save the file, generate thumbnails, and optimize images.
On a slow connection or with a large file, this request could time out.
The server is also stuck processing one image at a time.</p>
<p><img src="/blogs/mnw_117/sync_api_request.png" alt="Sequence diagram showing a synchronous API request."></p>
<h2>A Better Way: Async Processing</h2>
<p>Let's fix these problems. We'll split the work into two parts:</p>
<ol>
<li>Accept the upload and return quickly</li>
<li>Do the heavy work in the background</li>
</ol>
<p><img src="/blogs/mnw_117/async_api_request.png" alt="Sequence diagram showing an asynchronous API request."></p>
<h3>Uploading Images</h3>
<p>Here's the new upload endpoint:</p>
<pre><code class="language-csharp">[HttpPost]
public async Task&lt;IActionResult&gt; UploadImage(IFormFile? file)
{
    if (file is null)
    {
        return BadRequest(&quot;No file uploaded.&quot;);
    }

    if (!imageService.IsValidImage(file))
    {
        return BadRequest(&quot;Invalid image file.&quot;);
    }

    // Phase 1: Accept the work
    var id = Guid.NewGuid().ToString();
    var folderPath = Path.Combine(_uploadDirectory, &quot;images&quot;, id);
    var fileName = $&quot;{id}{Path.GetExtension(file.FileName)}&quot;;
    var originalPath = await imageService.SaveOriginalImageAsync(
        file,
        folderPath,
        fileName
    );

    // Queue Phase 2 for background processing
    var job = new ImageProcessingJob(id, originalPath, folderPath);
    await jobQueue.EnqueueAsync(job);

    // Return status URL immediately
    var statusUrl = GetStatusUrl(id);
    return Accepted(statusUrl, new { id, status = &quot;queued&quot; });
}
</code></pre>
<p>This new version only saves the original file during the HTTP request.
The heavy work moves to a background process.
The client immediately gets a status URL in the <code>Location</code> header instead of waiting.</p>
<h3>Checking Progress</h3>
<p>Clients can check their image's status using a separate endpoint:</p>
<pre><code class="language-csharp">[HttpGet(&quot;{id}/status&quot;)]
public IActionResult GetStatus(string id)
{
    if (!statusTracker.TryGetStatus(id, out var status))
    {
        return NotFound();
    }

    var response = new
    {
        id,
        status,
        links = new Dictionary&lt;string, string&gt;()
    };

    if (status == &quot;completed&quot;)
    {
        response.links = new Dictionary&lt;string, string&gt;
        {
            [&quot;original&quot;] = GetImageUrl(id),
            [&quot;thumbnail&quot;] = GetThumbnailUrl(id, width: 200),
            [&quot;preview&quot;] = GetThumbnailUrl(id, width: 800)
        };
    }

    return Ok(response);
}
</code></pre>
<h3>Processing Images in Background</h3>
<p>The real work happens in the background processor.
While the API handles new requests, a separate process works through the queued jobs.
This separation gives us flexibility in how we handle the processing.</p>
<p>For single-server deployments, we can use .NET's <a href="lightweight-in-memory-message-bus-using-dotnet-channels"><strong>Channel</strong></a> type to queue jobs in memory:</p>
<pre><code class="language-csharp">public class JobQueue
{
    private readonly Channel![Sequence diagram showing an asynchronous API request with server push for status updates.](/blogs/mnw_117/async_api_request_with_push.png)

[**SignalR and WebSockets**](adding-real-time-functionality-to-dotnet-applications-with-signalr) enable real-time communication between server and client.
When a job's status changes, the server immediately notifies interested clients.
This approach reduces network traffic and gives users instant feedback.

For longer-running jobs, email notifications make more sense.
Users don't need to keep their browsers open.
They can close the tab and come back when notified.
This works well for reports that take hours to generate or batch processes that run overnight.

Webhooks offer another option, especially for system-to-system communication.
When a job completes, your server can notify other systems.
This enables workflow automation and system integration without constant polling.

## Summary

Processing tasks asynchronously creates better experiences for everyone.
Users get immediate responses instead of watching spinning loading indicators.
They can start other tasks while waiting, and they'll know if something goes wrong.

The benefits extend beyond user experience.
Servers can handle more requests because they're not tied up with long-running tasks.
[**Background processors**](scheduling-background-jobs-with-quartz-net) can retry failed operations without affecting the main application.
You can even scale your processing separately from your web servers.

[**Error handling**](global-error-handling-in-aspnetcore-8) improves too.
When a long operation fails halfway through, you can save the progress and try again.
Users know exactly what's happening because they can check the status.
The system stays stable because one slow operation can't bring down your entire API.

That's all for today. Hope this was helpful.

---
</code></pre>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_117.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[HybridCache in ASP.NET Core - New Caching Library]]></title>
            <link>https://milanjovanovic.tech/blog/hybrid-cache-in-aspnetcore-new-caching-library</link>
            <guid isPermaLink="false">hybrid-cache-in-aspnetcore-new-caching-library</guid>
            <pubDate>Sat, 16 Nov 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[HybridCache in .NET 9 combines fast in-memory caching with distributed caching, solving common problems like cache stampede while adding features like tag-based invalidation. This guide shows you how to use HybridCache in your applications, from basic setup to real-world usage patterns with Entity Framework Core and minimal APIs.]]></description>
            <content:encoded><![CDATA[<p><a href="caching-in-aspnetcore-improving-application-performance"><strong>Caching</strong></a> is essential for building fast, scalable applications.
<a href="http://ASP.NET">ASP.NET</a> Core has traditionally offered two caching options: in-memory caching and distributed caching.
Each has its trade-offs.
In-memory caching using <code>IMemoryCache</code> is fast but limited to a single server.
Distributed caching with <code>IDistributedCache</code> works across multiple servers using a backplane.</p>
<p>.NET 9 introduces <code>HybridCache</code>, a new library that combines the best of both approaches.
It prevents common caching problems like cache stampede.
It also adds useful features like tag-based invalidation and better performance monitoring.</p>
<p>In this week's issue, I'll show you how to use <code>HybridCache</code> in your applications.</p>
<h2>What is HybridCache?</h2>
<p>The traditional caching options in <a href="http://ASP.NET">ASP.NET</a> Core have limitations.
In-memory caching is fast but limited to one server.
Distributed caching works across servers but is slower.</p>
<p><a href="https://learn.microsoft.com/en-us/aspnet/core/performance/caching/hybrid">HybridCache</a> combines both approaches and adds important features:</p>
<ul>
<li>Two-level caching (L1/L2)
<ul>
<li>L1: Fast in-memory cache</li>
<li>L2: Distributed cache (Redis, SQL Server, etc.)</li>
</ul>
</li>
<li>Protection against <a href="https://en.wikipedia.org/wiki/Cache_stampede">cache stampede</a> (when many requests hit an empty cache at once)</li>
<li>Tag-based cache invalidation</li>
<li>Configurable serialization</li>
<li>Metrics and monitoring</li>
</ul>
<p>The L1 cache runs in your application's memory.
The L2 cache can be Redis, SQL Server, or any other distributed cache.
You can use HybridCache with just the L1 cache if you don't need distributed caching.</p>
<h2>Installing HybridCache</h2>
<p>Install the <code>Microsoft.Extensions.Caching.Hybrid</code> NuGet package:</p>
<pre><code class="language-powershell">Install-Package Microsoft.Extensions.Caching.Hybrid
</code></pre>
<p>Add <code>HybridCache</code> to your services:</p>
<pre><code class="language-csharp">builder.Services.AddHybridCache(options =&gt;
{
    // Maximum size of cached items
    options.MaximumPayloadBytes = 1024 * 1024 * 10; // 10MB
    options.MaximumKeyLength = 512;

    // Default timeouts
    options.DefaultEntryOptions = new HybridCacheEntryOptions
    {
        Expiration = TimeSpan.FromMinutes(30),
        LocalCacheExpiration = TimeSpan.FromMinutes(30)
    };
});
</code></pre>
<p>For custom types, you can add your own serializer:</p>
<pre><code class="language-csharp">builder.Services.AddHybridCache()
    .AddSerializer&lt;CustomType, CustomSerializer&gt;();
</code></pre>
<h2>Using HybridCache</h2>
<p><code>HybridCache</code> provides several methods to work with cached data.
The most important ones are <code>GetOrCreateAsync</code>, <code>SetAsync</code>, and various remove methods.
Let's see how to use each one in real-world scenarios.</p>
<h3>Getting or Creating Cache Entries</h3>
<p>The <code>GetOrCreateAsync</code> method is your main tool for working with cached data.
It handles both cache hits and misses automatically.
If the data isn't in the cache, it calls your factory method to get the data, caches it, and returns it.</p>
<p>Here's an endpoint that gets product details:</p>
<pre><code class="language-csharp">app.MapGet(&quot;/products/{id}&quot;, async (
    int id,
    HybridCache cache,
    ProductDbContext db,
    CancellationToken ct) =&gt;
{
    var product = await cache.GetOrCreateAsync(
        $&quot;product-{id}&quot;,
        async token =&gt;
        {
            return await db.Products
                .Include(p =&gt; p.Category)
                .FirstOrDefaultAsync(p =&gt; p.Id == id, token);
        },
        cancellationToken: ct
    );

    return product is null ? Results.NotFound() : Results.Ok(product);
});
</code></pre>
<p>In this example:</p>
<ul>
<li>The cache key is unique per product</li>
<li>If the product is in the cache, it's returned immediately</li>
<li>If not, the factory method runs to get the data</li>
<li>Other concurrent requests for the same product wait for the first one to finish</li>
</ul>
<h3>Setting Cache Entries Directly</h3>
<p>Sometimes you need to update the cache directly, like after modifying data.
The <code>SetAsync</code> method handles this:</p>
<pre><code class="language-csharp">app.MapPut(&quot;/products/{id}&quot;, async (int id, Product product, HybridCache cache) =&gt;
{
    // First update the database
    await UpdateProductInDatabase(product);

    // Then update the cache with custom expiration
    var options = new HybridCacheEntryOptions
    {
        Expiration = TimeSpan.FromHours(1),
        LocalCacheExpiration = TimeSpan.FromMinutes(30)
    };

    await cache.SetAsync(
        $&quot;product-{id}&quot;,
        product,
        options
    );

    return Results.NoContent();
});
</code></pre>
<p>Key points about <code>SetAsync</code>:</p>
<ul>
<li>It updates both L1 and L2 cache</li>
<li>You can specify different timeouts for L1 and L2</li>
<li>It overwrites any existing value for the same key</li>
</ul>
<h3>Using Cache Tags</h3>
<p>Tags are powerful for managing groups of related cache entries.
You can invalidate multiple entries at once using tags:</p>
<pre><code class="language-csharp">app.MapGet(&quot;/categories/{id}/products&quot;, async (
    int id,
    HybridCache cache,
    ProductDbContext db,
    CancellationToken ct) =&gt;
{
    var tags = [$&quot;category-{id}&quot;, &quot;products&quot;];

    var products = await cache.GetOrCreateAsync(
        $&quot;products-by-category-{id}&quot;,
        async token =&gt;
        {
            return await db.Products
                .Where(p =&gt; p.CategoryId == id)
                .Include(p =&gt; p.Category)
                .ToListAsync(token);
        },
        tags: tags,
        cancellationToken: ct
    );

    return Results.Ok(products);
});

// Endpoint to invalidate all products in a category
app.MapPost(&quot;/categories/{id}/invalidate&quot;, async (
    int id,
    HybridCache cache,
    CancellationToken ct) =&gt;
{
    await cache.RemoveByTagAsync($&quot;category-{id}&quot;, ct);

    return Results.NoContent();
});
</code></pre>
<p>Tags are useful for:</p>
<ul>
<li>Invalidating all products in a category</li>
<li>Clearing all cached data for a specific user</li>
<li>Refreshing all related data when something changes</li>
</ul>
<h3>Removing Single Entries</h3>
<p>For direct cache invalidation of specific items, use <code>RemoveAsync</code>:</p>
<pre><code class="language-csharp">app.MapDelete(&quot;/products/{id}&quot;, async (int id, HybridCache cache) =&gt;
{
    // First delete from database
    await DeleteProductFromDatabase(id);

    // Then remove from cache
    await cache.RemoveAsync($&quot;product-{id}&quot;);

    return Results.NoContent();
});
</code></pre>
<p><code>RemoveAsync</code>:</p>
<ul>
<li>Removes the item from both L1 and L2 cache</li>
<li>Works immediately, no delay</li>
<li>Does nothing if the key doesn't exist</li>
<li>Is safe to call multiple times</li>
</ul>
<p>Remember that <code>HybridCache</code> handles all the complexity of distributed caching, serialization, and stampede protection for you.
You just need to focus on your cache keys and when to invalidate the cache.</p>
<h2>Adding Redis as L2 Cache</h2>
<p>To use <a href="https://redis.io/">Redis</a> as your distributed cache:</p>
<ol>
<li>Install the <code>Microsoft.Extensions.Caching.StackExchangeRedis</code> NuGet package:</li>
</ol>
<pre><code class="language-powershell">Install-Package Microsoft.Extensions.Caching.StackExchangeRedis
</code></pre>
<ol start="2">
<li>Configure Redis and <code>HybridCache</code>:</li>
</ol>
<pre><code class="language-csharp">// Add Redis
builder.Services.AddStackExchangeRedisCache(options =&gt;
{
    options.Configuration = &quot;your-redis-connection-string&quot;;
});

// Add HybridCache - it will automatically use Redis as L2
builder.Services.AddHybridCache();
</code></pre>
<p><code>HybridCache</code> will automatically detect and use Redis as the L2 cache.</p>
<h2>Summary</h2>
<p><code>HybridCache</code> simplifies caching in .NET applications.
It combines fast in-memory caching with distributed caching, prevents common problems like cache stampede,
and works well in both single-server and distributed systems.</p>
<p>Start with the default settings and basic usage patterns - the library is designed to be simple to use while solving complex caching problems.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_116.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Functional Programming in C#: The Practical Parts]]></title>
            <link>https://milanjovanovic.tech/blog/functional-programming-in-csharp-the-practical-parts</link>
            <guid isPermaLink="false">functional-programming-in-csharp-the-practical-parts</guid>
            <pubDate>Sat, 09 Nov 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Functional programming patterns can make your C# code safer and more maintainable, without getting lost in academic theory. Learn practical patterns you can use today to write better code.]]></description>
            <content:encoded><![CDATA[<p>Functional programming patterns can feel academic and abstract.
Terms like &quot;monads&quot; and &quot;functors&quot; scare many developers away.
But beneath the intimidating terminology are practical patterns that can make your code safer and more maintainable.</p>
<p>C# has embraced many functional programming features over the years.</p>
<ul>
<li>Records for immutability</li>
<li>LINQ for functional transformations</li>
<li>Lambda expressions for first-class functions</li>
</ul>
<p>These features aren't just syntax sugar - they help prevent bugs and make code easier to reason about.</p>
<p>Let's look at five practical patterns you can use in your C# projects today.</p>
<h2>Higher-Order Functions</h2>
<p>Higher-order functions can take other functions as parameters or return them as results.
They let you write code that's more flexible and composable because you can pass behavior around like data.</p>
<p>Common examples of higher-order functions are LINQ's <code>Where</code> and <code>Select</code>, which take functions to transform data.</p>
<p>Let's refactor this validation example with higher-order functions:</p>
<pre><code class="language-csharp">public class OrderValidator
{
    public bool ValidateOrder(Order order)
    {
        if (order.Items.Count == 0) return false;
        if (order.TotalAmount &lt;= 0) return false;
        if (order.ShippingAddress == null) return false;
        return true;
    }
}

// What if we need:
// - different validation rules for different countries?
// - to reuse some validations but not others?
// - to combine validations differently?
</code></pre>
<p>Here's how higher-order functions make this more flexible:</p>
<pre><code class="language-csharp">public static class OrderValidation
{
    public static Func&lt;Order, bool&gt; CreateValidator(string countryCode, decimal minimumOrderValue)
    {
        var baseValidations = CombineValidations(
            o =&gt; o.Items.Count &gt; 0,
            o =&gt; o.TotalAmount &gt;= minimumOrderValue,
            o =&gt; o.ShippingAddress != null
        );

        return countryCode switch
        {
            &quot;US&quot; =&gt; CombineValidations(
                baseValidations,
                order =&gt; IsValidUSAddress(order.ShippingAddress)),
            &quot;EU&quot; =&gt; CombineValidations(
                baseValidations,
                order =&gt; IsValidVATNumber(order.VatNumber)),
            _ =&gt; baseValidations
        };
    }

    private static Func&lt;Order, bool&gt; CombineValidations(params Func&lt;Order, bool&gt;[] validations) =&gt;
        order =&gt; validations.All(v =&gt; v(order));
}

// Usage
var usValidator = OrderValidation.CreateValidator(&quot;US&quot;, minimumOrderValue: 25.0m);
var euValidator = OrderValidation.CreateValidator(&quot;EU&quot;, minimumOrderValue: 30.0m);
</code></pre>
<p>The higher-order function approach makes validators composable, testable, and easy to extend.
Each validation rule is a simple function that we can compose.</p>
<h2>Errors as Values</h2>
<p>Error handling in C# often looks like this:</p>
<pre><code class="language-csharp">public class UserService
{
    public User CreateUser(string email, string password)
    {
        if (string.IsNullOrEmpty(email))
        {
            throw new ArgumentException(&quot;Email is required&quot;);
        }

        if (password.Length &lt; 8)
        {
            throw new ArgumentException(&quot;Password too short&quot;);
        }

        if (_userRepository.EmailExists(email))
        {
            throw new DuplicateEmailException(email);
        }

        // Create user...
    }
}
</code></pre>
<p>The problem?</p>
<ul>
<li><a href="https://youtu.be/E3dU9Y1CsnI">Exceptions are expensive</a></li>
<li>Callers often forget to handle exceptions</li>
<li>The method signature lies - it claims to return a User but might throw</li>
</ul>
<p>We can make errors explicit using the <a href="https://github.com/mcintyre321/OneOf">OneOf</a> library.
It provides discriminated unions for C#, using a custom type <code>OneOf&lt;T0, ... Tn&gt;</code>.</p>
<pre><code class="language-csharp">public class UserService
{
    public OneOf&lt;User, ValidationError, DuplicateEmailError&gt; CreateUser(string email, string password)
    {
        if (string.IsNullOrEmpty(email))
        {
            return new ValidationError(&quot;Email is required&quot;);
        }

        if (password.Length &lt; 8)
        {
            return new ValidationError(&quot;Password too short&quot;);
        }

        if (_userRepository.EmailExists(email))
        {
            return new DuplicateEmailError(email);
        }

        return new User(email, password);
    }
}
</code></pre>
<p>By making the errors explicit:</p>
<ul>
<li>The method signature tells the whole truth</li>
<li>Callers must handle all possible outcomes</li>
<li>No performance overhead from exceptions</li>
<li>The flow is easier to follow</li>
</ul>
<p>Here's how you use it:</p>
<pre><code class="language-csharp">var result = userService.CreateUser(email, password);

result.Switch(
    user =&gt; SendWelcomeEmail(user),
    validationError =&gt; HandleError(validationError),
    duplicateError =&gt; HandleError(duplicateError)
);
</code></pre>
<h2>Monadic Binding</h2>
<p>A <strong>monad</strong> is a container for values - like <code>List&lt;T&gt;</code>, <code>IEnumerable&lt;T&gt;</code>, or <code>Task&lt;T&gt;</code>.
What makes it special is that you can chain operations on the contained values without dealing with the container directly.
This chaining is called monadic binding.</p>
<p>You use monadic binding daily with LINQ, but you might not know it.
It's what allows us to chain operations that transform data.</p>
<p>Map (<code>Select</code>) transforms values:</p>
<pre><code class="language-csharp">// Simple transformations with Select (Map)
var numbers = new[] { 1, 2, 3, 4 };

var doubled = numbers.Select(x =&gt; x * 2);
</code></pre>
<p>Bind (<code>SelectMany</code>) transforms and flattens:</p>
<pre><code class="language-csharp">// Operations that return multiple values use SelectMany (Bind)
var folders = new[] { &quot;docs&quot;, &quot;photos&quot; };

var files = folders.SelectMany(folder =&gt; Directory.GetFiles(folder));
</code></pre>
<p>A popular example of applying monads in practice is the <a href="functional-error-handling-in-dotnet-with-the-result-pattern"><strong>Result pattern</strong></a>,
which provides a clean way to chain operations that might fail.</p>
<h2>Pure Functions</h2>
<p>Pure functions are predictable: they depend only on their inputs and don't change anything in the system.
No database calls, no API requests, no global state.
This constraint makes them easier to understand, test, and debug.</p>
<pre><code class="language-csharp">// Impure - relies on hidden state
public class PriceCalculator
{
    private decimal _taxRate;
    private List&lt;Discount&gt; _activeDiscounts;

    public decimal CalculatePrice(Order order)
    {
        var price = order.Items.Sum(i =&gt; i.Price);

        foreach (var discount in _activeDiscounts)
        {
            price -= discount.Calculate(price);
        }

        return price * (1 + _taxRate);
    }
}
</code></pre>
<p>Here's the same example as a pure function:</p>
<pre><code class="language-csharp">// Pure - everything is explicit
public static class PriceCalculator
{
    public static decimal CalculatePrice(
        Order order,
        decimal taxRate,
        IReadOnlyList&lt;Discount&gt; discounts)
    {
        var basePrice = order.Items.Sum(i =&gt; i.Price);

        var afterDiscounts = discounts.Aggregate(
            basePrice,
            (price, discount) =&gt; price - discount.Calculate(price));

        return afterDiscounts * (1 + taxRate);
    }
}
</code></pre>
<p>Pure functions are thread-safe, easy to test, and simple to reason about because all dependencies are explicit.</p>
<h2>Immutability</h2>
<p>Immutable objects can't be changed after creation.
Instead, they create new instances for every change.
This simple constraint eliminates entire categories of bugs: race conditions, accidental modifications, and inconsistent state.</p>
<p>Here's an example of a mutable type:</p>
<pre><code class="language-csharp">public class Order
{
    public List&lt;OrderItem&gt; Items { get; set; }
    public decimal Total { get; set; }
    public OrderStatus Status { get; set; }

    public void AddItem(OrderItem item)
    {
        Items.Add(item);
        Total += item.Price;
        // Bug: Thread safety issues
        // Bug: Can modify shipped orders
        // Bug: Total might not match Items
    }
}
</code></pre>
<p>Let's make this an immutable type:</p>
<pre><code class="language-csharp">public record Order
{
    public ImmutableList&lt;OrderItem&gt; Items { get; init; }
    public OrderStatus Status { get; init; }
    public decimal Total =&gt; Items.Sum(x =&gt; x.Price);

    public Order AddItem(OrderItem item)
    {
        if (Status != OrderStatus.Created)
        {
            throw new InvalidOperationException(&quot;Can't modify shipped orders&quot;);
        }

        return this with
        {
            Items = Items.Add(item)
        };
    }
}
</code></pre>
<p>The immutable version:</p>
<ul>
<li>Is thread-safe by default</li>
<li>Makes invalid states impossible</li>
<li>Keeps data and calculations consistent</li>
<li>Makes changes explicit and traceable</li>
</ul>
<h2>Takeaway</h2>
<p><a href="how-to-apply-functional-programming-in-csharp"><strong>Functional programming</strong></a> isn't just about writing &quot;cleaner&quot; code.
These patterns fundamentally change how you handle complexity:</p>
<ul>
<li><strong>Push errors to compile time</strong> - Catch problems before running the code</li>
<li><strong>Make invalid states impossible</strong> - Don't rely on documentation or conventions</li>
<li><strong>Make the happy path obvious</strong> - When everything is explicit, the flow is clear</li>
</ul>
<p>You can adopt these patterns gradually.
Start with one class, one module, one feature.
The goal isn't to write purely functional code.
The goal is to write code that's safer, more predictable, and easier to maintain.</p>
<p>Hope this was helpful.
See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_115.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Clean Architecture: The Missing Chapter]]></title>
            <link>https://milanjovanovic.tech/blog/clean-architecture-the-missing-chapter</link>
            <guid isPermaLink="false">clean-architecture-the-missing-chapter</guid>
            <pubDate>Sat, 02 Nov 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Clean Architecture's famous diagram is often misinterpreted as a project structure, leading developers to create artificial technical layers that scatter business logic across their codebase. Learn what the diagram really means and how to properly organize your code around business capabilities using components and clear boundaries.]]></description>
            <content:encoded><![CDATA[<p>I see the same mistake happen over and over again.</p>
<p>Developers discover Clean Architecture, get excited about its principles, and then... they turn the famous Clean Architecture diagram into a project structure.</p>
<p>But here's the thing: <strong>Clean Architecture is not about folders</strong>.
It's about dependencies.</p>
<p>Simon Brown wrote a &quot;missing chapter&quot; for Uncle Bob's Clean Architecture book that addresses exactly this issue.
Yet somehow, this crucial message got lost along the way.</p>
<p>Today, I'll show you what Uncle Bob's Clean Architecture diagram really means and how you should actually organize your code.
We'll look at practical examples that you can use in your projects right now.</p>
<p>Let's clear up this common misconception once and for all.</p>
<h2>The Problem With Traditional Layering</h2>
<p>Almost every .NET developer has built a solution that looks like this:</p>
<ul>
<li><code>MyApp.Web</code> for controllers and views</li>
<li><code>MyApp.Business</code> for services and business logic</li>
<li><code>MyApp.Data</code> for repositories and data access</li>
</ul>
<p>It's the default approach. It's what we see in tutorials. It's what we teach juniors.</p>
<p>And it's completely wrong.</p>
<h3>Why Layer-Based Organization Fails</h3>
<p>When you organize code by technical layers, you scatter related components across multiple projects.
A single feature, like managing policies, ends up spread across your entire codebase:</p>
<ul>
<li>Policies controller in the Web layer</li>
<li>Policy service in the Business layer</li>
<li>Policy repository in the Data layer</li>
</ul>
<p>Here's what you'll see when looking at the folder structure:</p>
<pre><code class="language-text">📁 MyApp.Web
|__ 📁 Controllers
    |__ #️⃣ PoliciesController.cs
📁 MyApp.Business
|__ 📁 Services
    |__ #️⃣ PolicyService.cs
📁 MyApp.Data
|__ 📁 Repositories
    |__ #️⃣ PolicyRepository.cs
</code></pre>
<p>Here's a visual representation of the layer-based architecture:</p>
<div className="centered">
  ![Feature Scattering in Layer-Based Architecture.](/blogs/mnw_114/layered_architecture.png)
</div>
<p>This fragmentation creates several problems:</p>
<ol>
<li>
<p><strong>Violates Common Closure Principle</strong> - Classes that change together should stay together.
When your &quot;Policies&quot; feature changes, you're touching three different projects.</p>
</li>
<li>
<p><strong>Hidden dependencies</strong> - Public interfaces everywhere make it possible to bypass layers.
Nothing stops a controller from directly accessing a repository.</p>
</li>
<li>
<p><strong>No business intent</strong> - Opening your solution tells you nothing about what the application does.
It only shows technical implementation details.</p>
</li>
<li>
<p><strong>Harder maintenance</strong> - Making changes requires jumping between multiple projects.</p>
</li>
</ol>
<p>The worst part? This approach doesn't even achieve what it promises.
Despite the separate projects, you often end up with a &quot;big ball of mud&quot; because public access modifiers allow any class to reference any other class.</p>
<h3>The Real Intent of Layers</h3>
<p>Clean Architecture's circles were never meant to represent projects or folders.
They represent different levels of policy, with dependencies pointing inward toward business rules.</p>
<p>You can achieve this without splitting your code into artificial technical layers.</p>
<p>Let me show you a better way.</p>
<h2>Better Approaches to Code Organization</h2>
<p>Instead of splitting your code by technical layers, you have two better options: <strong>package by feature</strong> or <strong>package by component</strong>.</p>
<p>Let's look at both.</p>
<h3>Package by Feature</h3>
<p>Organizing by feature is a solid option.
Each feature gets its own namespace and contains everything needed to implement that feature.</p>
<pre><code class="language-text">📁 MyApp.Policies
|__ 📁 RenewPolicy
    |__ #️⃣ RenewPolicyCommand.cs
    |__ #️⃣ RenewPolicyHandler.cs
    |__ #️⃣ PolicyValidator.cs
    |__ #️⃣ PolicyRepository.cs
|__ 📁 ViewPolicyHistory
    |__ #️⃣ PolicyHistoryQuery.cs
    |__ #️⃣ PolicyHistoryHandler.cs
    |__ #️⃣ PolicyHistoryViewModel.cs
</code></pre>
<p>Here's a diagram representing this structure:</p>
<div className="centered">
  ![Vertical slice architecture for package by feature organization.](/blogs/mnw_114/feature_folder_architecture.png)
</div>
<p>This approach:</p>
<ul>
<li>Makes features explicit</li>
<li>Keeps related code together</li>
<li>Simplifies navigation</li>
<li>Makes it easier to maintain and modify features</li>
</ul>
<p>If you want to learn more, check out my article about <a href="vertical-slice-architecture"><strong>vertical slice architecture</strong></a>.</p>
<h3>Package by Component</h3>
<p>A component is a cohesive group of related functionality with a well-defined interface.
Component-based organization is more coarse-grained than feature folders.
Think of it as a mini application that handles one specific business capability.</p>
<p>This is very similar to how I define modules in a <a href="what-is-a-modular-monolith"><strong>modular monolith</strong></a>.</p>
<p>Here's what a component-based organization looks like:</p>
<pre><code class="language-text">📁 MyApp.Web
|__ 📁 Controllers
    |__ #️⃣ PoliciesController.cs
📁 MyApp.Policies
|__ #️⃣ PoliciesComponent.cs     // Public interface
|__ #️⃣ PolicyService.cs         // Implementation detail
|__ #️⃣ PolicyRepository.cs      // Implementation detail
</code></pre>
<p>The key difference? Only <code>PoliciesComponent</code> is public.
Everything else is internal to the component.</p>
<div className="centered">
  ![Feature Scattering in Layer-Based Architecture.](/blogs/mnw_114/component_architecture.png)
</div>
<p>This means:</p>
<ul>
<li>No bypassing layers</li>
<li>Clear dependencies</li>
<li>Real encapsulation</li>
<li>Business intent visible in the structure</li>
</ul>
<h3>Which One Should You Choose?</h3>
<p>Choose <strong>Package by Feature</strong> when:</p>
<ul>
<li>You have many small, independent features</li>
<li>Your features don't share much code</li>
<li>You want maximum flexibility</li>
</ul>
<p>Choose <strong>Package by Component</strong> when:</p>
<ul>
<li>You have clear business capabilities</li>
<li>You want strong encapsulation</li>
<li>You might split into microservices later</li>
</ul>
<p>Both approaches achieve what Clean Architecture really wants: proper dependency management and business focus.</p>
<p>Here's a side-by-side comparison of these architectural approaches:</p>
<figure>
  <div className="centered">
    ![Comparison between layered, vertical slice and component architectural approaches.](/blogs/mnw_114/architecture_comparison.png)
  </div>
  <figcaption>
    Greyed-out types are internal to the defining assembly.
  </figcaption>
</figure>
<p>In the Missing Chapter of Clean Architecture, Simon Brown argues strongly for package by component.
The key insight is that components are the natural way to slice a system.
They represent complete business capabilities, not just technical features.</p>
<p>My recommendation?
Start with package by component.
Within the component, organize around features.</p>
<h2>Practical Examples</h2>
<p>Let's transform a typical layered application into a clean, component-based structure.
We'll use an insurance policy system as an example.</p>
<h3>The Traditional Way</h3>
<p>Here's how most developers structure their solution:</p>
<pre><code class="language-csharp">// MyApp.Data
public interface IPolicyRepository
{
    Task&lt;Policy&gt; GetByIdAsync(string policyNumber);
    Task SaveAsync(Policy policy);
}

// MyApp.Business
public class PolicyService : IPolicyService
{
    private readonly IPolicyRepository _repository;

    public PolicyService(IPolicyRepository repository)
    {
        _repository = repository;
    }

    public async Task RenewPolicyAsync(string policyNumber)
    {
        var policy = await _repository.GetByIdAsync(policyNumber);
        // Business logic here
        await _repository.SaveAsync(policy);
    }
}

// MyApp.Web
public class PoliciesController : ControllerBase
{
    private readonly IPolicyService _policyService;

    public PoliciesController(IPolicyService policyService)
    {
        _policyService = policyService;
    }

    [HttpPost(&quot;renew/{policyNumber}&quot;)]
    public async Task&lt;IActionResult&gt; RenewPolicy(string policyNumber)
    {
        await _policyService.RenewPolicyAsync(policyNumber);
        return Ok();
    }
}
</code></pre>
<p>The problem?
Everything is public.
Any class can bypass the service and go straight to the repository.</p>
<h3>The Clean Way</h3>
<p>Here's the same functionality organized as a proper component:</p>
<pre><code class="language-csharp">// The only public contract
public interface IPoliciesComponent
{
    Task RenewPolicyAsync(string policyNumber);
}

// Everything below is internal to the component
internal class PoliciesComponent : IPoliciesComponent
{
    private readonly IRenewPolicyHandler _renewPolicyHandler;

    // Public constructor for DI
    public PoliciesComponent(IRenewPolicyHandler renewPolicyHandler)
    {
        _renewPolicyHandler = renewPolicyHandler;
    }

    public async Task RenewPolicyAsync(string policyNumber)
    {
        await _renewPolicyHandler.HandleAsync(policyNumber);
    }
}

internal interface IRenewPolicyHandler
{
    Task HandleAsync(string policyNumber);
}

internal class RenewPolicyHandler : IRenewPolicyHandler
{
    private readonly IPolicyRepository _repository;

    internal RenewPolicyHandler(IPolicyRepository repository)
    {
        _repository = repository;
    }

    public async Task HandleAsync(string policyNumber)
    {
        var policy = await _repository.GetByIdAsync(policyNumber);
        // Business logic for policy renewal here
        await _repository.SaveAsync(policy);
    }
}

internal interface IPolicyRepository
{
    Task&lt;Policy&gt; GetByIdAsync(string policyNumber);
    Task SaveAsync(Policy policy);
}
</code></pre>
<p>The key improvements are:</p>
<ol>
<li>
<p><strong>Single public interface</strong> - Only <code>IPoliciesComponent</code> is public. Everything else is internal.</p>
</li>
<li>
<p><strong>Protected dependencies</strong> - No way to bypass the component and access the repository directly.</p>
</li>
<li>
<p><strong>Clear dependencies</strong> - All dependencies flow inward through the component.</p>
</li>
<li>
<p><strong>Proper encapsulation</strong> - Implementation details are truly hidden.</p>
</li>
</ol>
<p>This is how you would register the services with dependency injection:</p>
<pre><code class="language-csharp">services.AddScoped&lt;IPoliciesComponent, PoliciesComponent&gt;();
services.AddScoped&lt;IRenewPolicyHandler, RenewPolicyHandler&gt;();
services.AddScoped&lt;IPolicyRepository, SqlPolicyRepository&gt;();
</code></pre>
<p>This structure enforces Clean Architecture principles through compiler-checked boundaries, not just conventions.</p>
<p>The compiler won't let you bypass the component's public interface.
That's much stronger than hoping developers follow the rules.</p>
<h2>Best Practices and Limitations</h2>
<p>Let's discuss something that is often overlooked: the practical limitations of enforcing Clean Architecture in .NET.</p>
<h3>The Limits of Encapsulation</h3>
<p>The <code>internal</code> keyword in .NET provides protection within a single assembly.
Here's what that means in practice:</p>
<pre><code class="language-csharp">// In a single project:
public interface IPoliciesComponent { } // Public contract
internal class PoliciesComponent : IPoliciesComponent { }
internal class PolicyRepository { }

// Someone could still do this:
public class BadPoliciesComponent : IPoliciesComponent
{
    public BadPoliciesComponent()
    {
        // Nothing stops them from creating a bad implementation
    }
}
</code></pre>
<p>While <code>internal</code> helps, it doesn't prevent all architectural violations.</p>
<h3>The Trade-offs</h3>
<p>Some teams split their code into separate assemblies for stronger encapsulation:</p>
<pre><code class="language-plaintext">MyCompany.Policies.Core.dll
MyCompany.Policies.Infrastructure.dll
MyCompany.Policies.Api.dll
</code></pre>
<p>This comes with trade-offs:</p>
<ol>
<li><strong>More complex build process</strong> - Multiple projects need to be compiled and referenced.</li>
<li><strong>Harder navigation</strong> - Jumping between assemblies in the IDE is slower.</li>
<li><strong>Deployment complexity</strong> - More DLLs to manage and deploy.</li>
</ol>
<h3>A Pragmatic Approach</h3>
<p>Here's what I recommend:</p>
<ol>
<li>
<p><strong>Use a single assembly</strong></p>
<ul>
<li>Keep related code together</li>
<li>Use <code>internal</code> for implementation details</li>
<li>Make only the component interfaces public</li>
<li>Add <code>sealed</code> to prevent inheritance when possible</li>
</ul>
</li>
<li>
<p><strong>Enforce through architecture testing</strong></p>
<ul>
<li>Add architecture tests to verify dependencies</li>
<li>Automatically check for architectural violations</li>
<li>Fail the build if someone bypasses the rules</li>
</ul>
</li>
</ol>
<pre><code class="language-csharp">[Fact]
public void Controllers_Should_Only_Depend_On_Component_Interfaces()
{
    var allTypes = Types.InAssembly(Assembly.GetExecutingAssembly());

    TestResult? result = allTypes
        .That()
        .ResideInNamespace(&quot;MyApp.Controllers&quot;)
        .Should()
        .OnlyHaveDependenciesOn(
            allTypes
                .That()
                .HaveNameEndingWith(&quot;Component&quot;)
                .Or()
                .HaveNameStartingWith(&quot;IPolicy&quot;)
                .GetTypes()
                .Select(t =&gt; t.FullName!)
                .ToArray())
        .GetResult();

    result.IsSuccessful.Should().BeTrue();
}
</code></pre>
<p>Want to learn more about enforcing architecture through testing?
Check out my article on <a href="enforcing-software-architecture-with-architecture-tests"><strong>architecture testing</strong></a>.</p>
<p>Remember: Clean Architecture is about managing dependencies, not about achieving perfect encapsulation.
Use the tools the language gives you, but don't over-complicate things chasing an impossible ideal.</p>
<h2>Conclusion</h2>
<p>Clean Architecture isn't about projects, folders, or perfect encapsulation.</p>
<p>It's about:</p>
<ul>
<li>Organizing code around business capabilities</li>
<li>Managing dependencies effectively</li>
<li>Keeping related code together</li>
<li>Making boundaries explicit</li>
</ul>
<p>Start with a single project.
Use components.
Make interfaces public and implementations internal.
Add architecture tests if you need more control.</p>
<p>And remember: <strong>pragmatism beats purism</strong>.
Your architecture should help you ship features faster, not slow you down with artificial constraints.</p>
<p>Want to learn more?
Check out my <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture</strong></a> course,
where I'll show you how to build maintainable applications with proper boundaries, clear dependencies, and business-focused components.</p>
<p>That's all for today. Stay awesome, and I'll see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_114.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Implementing Idempotent REST APIs in ASP.NET Core]]></title>
            <link>https://milanjovanovic.tech/blog/implementing-idempotent-rest-apis-in-aspnetcore</link>
            <guid isPermaLink="false">implementing-idempotent-rest-apis-in-aspnetcore</guid>
            <pubDate>Sat, 26 Oct 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to implement idempotency in ASP.NET Core Web APIs to improve reliability and prevent duplicate operations in distributed systems.]]></description>
            <content:encoded><![CDATA[<p>Idempotency is a crucial concept for REST APIs that ensures the reliability and consistency of your system.
An idempotent operation can be repeated multiple times without changing the result beyond the initial API request.
This property is especially important in distributed systems, where network failures or timeouts can lead to repeated requests.</p>
<p>Implementing idempotency in your API brings several benefits:</p>
<ul>
<li>It prevents unintended duplicate operations</li>
<li>It improves reliability in distributed systems</li>
<li>It helps handle network issues and retries gracefully</li>
</ul>
<p>In this week's issue, we'll explore how to implement idempotency in <a href="http://ASP.NET">ASP.NET</a> Core APIs, ensuring your system remains robust and reliable.</p>
<h2>What is Idempotence?</h2>
<p>Idempotence, in the context of web APIs, means that making multiple identical requests should have the same effect as making a single request.
In other words, no matter how many times a client sends the same request, the server-side effect should only occur once.</p>
<p>The <a href="https://www.rfc-editor.org/rfc/rfc9110">RFC 9110</a> standard about HTTP Semantics offers a definition we could use.
Here's what it says about <strong>idempotent methods</strong>:</p>
<figure>
<blockquote>
<p>A request method is considered &quot;idempotent&quot; if the intended effect on the server of multiple identical requests with
that method is the same as the effect for a single such request.</p>
<p>Of the request methods defined by this specification,
PUT, DELETE, and safe request methods [(GET, HEAD, OPTIONS, and TRACE) - author's note] are idempotent.</p>
</blockquote>
  <figcaption>
    _&mdash; [RFC 9110 (HTTP Semantics), Section 9.2.2, Paragraph 1](https://www.rfc-editor.org/rfc/rfc9110#section-9.2.2-1)_
  </figcaption>
</figure>
<p>However, the following paragraph is quite interesting.
It clarifies that the server can implement &quot;other non-idempotent side effects&quot; that don't apply to the resource.</p>
<figure>
<blockquote>
<p>... the idempotent property only applies to what has been requested by the user;
a server is free to log each request separately, retain a revision control history, or implement other non-idempotent side effects for each idempotent request.</p>
</blockquote>
  <figcaption>
    _&mdash; [RFC 9110 (HTTP Semantics), Section 9.2.2, Paragraph 2](https://www.rfc-editor.org/rfc/rfc9110#section-9.2.2-2)_
  </figcaption>
</figure>
<p>The benefits of implementing idempotency extend beyond just adhering to HTTP method semantics.
It significantly improves the reliability of your API, especially in distributed systems where network issues can lead to retried requests.
By implementing idempotency, you prevent duplicate operations that could occur due to client retries.</p>
<h2>Which HTTP Methods are Idempotent?</h2>
<p>Several HTTP methods are inherently idempotent:</p>
<ul>
<li><code>GET</code>, <code>HEAD</code>: Retrieve data without modifying the server state.</li>
<li><code>PUT</code>: Update a resource, resulting in the same state regardless of repetition.</li>
<li><code>DELETE</code>: Remove a resource with the same outcome for multiple requests.</li>
<li><code>OPTIONS</code>: Retrieve communication options information.</li>
</ul>
<p><code>POST</code> is not inherently idempotent, as it typically creates resources or processes data.
Repeated <code>POST</code> requests could create multiple resources or trigger multiple actions.</p>
<p>However, we can implement idempotency for <code>POST</code> methods using custom logic.</p>
<p><strong>Note</strong>: While <code>POST</code> requests aren't naturally idempotent, we can design them to be.
For example, checking for existing resources before creation ensures that repeated <code>POST</code> requests don't result in duplicate actions or resources.</p>
<h2>Implementing Idempotency in <a href="http://ASP.NET">ASP.NET</a> Core</h2>
<p>To implement idempotency, we'll use a strategy involving idempotency keys:</p>
<ol>
<li>The client generates a unique key for each operation and sends it in a custom header.</li>
<li>The server checks if it has seen this key before:
<ul>
<li>For a new key, process the request and store the result.</li>
<li>For a known key, return the stored result without reprocessing.</li>
</ul>
</li>
</ol>
<p>This ensures that retried requests (e.g., due to network issues) are processed only once on the server.</p>
<p>We can implement idempotency for controllers by combining an <code>Attribute</code> and <code>IAsyncActionFilter</code>.
Now, we can specify the <code>IdempotentAttribute</code> to apply idempotency to a controller endpoint.</p>
<p><strong>Note</strong>: When a request fails (returns 4xx/5xx), we don't cache the response.
This allows clients to retry with the same idempotency key.
However, this means a failed request followed by a successful one with the same key will succeed - make sure this aligns with your business requirements.</p>
<pre><code class="language-csharp">[AttributeUsage(AttributeTargets.Method)]
internal sealed class IdempotentAttribute : Attribute, IAsyncActionFilter
{
    private const int DefaultCacheTimeInMinutes = 60;
    private readonly TimeSpan _cacheDuration;

    public IdempotentAttribute(int cacheTimeInMinutes = DefaultCacheTimeInMinutes)
    {
        _cacheDuration = TimeSpan.FromMinutes(cacheTimeInMinutes);
    }

    public async Task OnActionExecutionAsync(
        ActionExecutingContext context,
        ActionExecutionDelegate next)
    {
        // Parse the Idempotence-Key header from the request
        if (!context.HttpContext.Request.Headers.TryGetValue(
                &quot;Idempotence-Key&quot;,
                out StringValues idempotenceKeyValue) ||
            !Guid.TryParse(idempotenceKeyValue, out Guid idempotenceKey))
        {
            context.Result = new BadRequestObjectResult(&quot;Invalid or missing Idempotence-Key header&quot;);
            return;
        }

        IDistributedCache cache = context.HttpContext
            .RequestServices.GetRequiredService&lt;IDistributedCache&gt;();

        // Check if we already processed this request and return a cached response (if it exists)
        string cacheKey = $&quot;Idempotent_{idempotenceKey}&quot;;
        string? cachedResult = await cache.GetStringAsync(cacheKey);
        if (cachedResult is not null)
        {
            IdempotentResponse response = JsonSerializer.Deserialize&lt;IdempotentResponse&gt;(cachedResult)!;

            var result = new ObjectResult(response.Value) { StatusCode = response.StatusCode };
            context.Result = result;

            return;
        }

        // Execute the request and cache the response for the specified duration
        ActionExecutedContext executedContext = await next();

        if (executedContext.Result is ObjectResult { StatusCode: &gt;= 200 and &lt; 300 } objectResult)
        {
            int statusCode = objectResult.StatusCode ?? StatusCodes.Status200OK;
            IdempotentResponse response = new(statusCode, objectResult.Value);

            await cache.SetStringAsync(
                cacheKey,
                JsonSerializer.Serialize(response),
                new DistributedCacheEntryOptions { AbsoluteExpirationRelativeToNow = _cacheDuration }
            );
        }
    }
}

internal sealed class IdempotentResponse
{
    [JsonConstructor]
    public IdempotentResponse(int statusCode, object? value)
    {
        StatusCode = statusCode;
        Value = value;
    }

    public int StatusCode { get; }
    public object? Value { get; }
}
</code></pre>
<p><strong>Note</strong>: There's a small <a href="solving-race-conditions-with-ef-core-optimistic-locking"><strong>race condition</strong></a> window between checking and setting the cache.
For absolute consistency, we should consider using a distributed lock pattern, though this adds complexity and latency.</p>
<p>Now, we can apply this attribute to our controller actions:</p>
<pre><code class="language-csharp">[ApiController]
[Route(&quot;api/[controller]&quot;)]
public class OrdersController : ControllerBase
{
    [HttpPost]
    [Idempotent(cacheTimeInMinutes: 60)]
    public IActionResult CreateOrder([FromBody] CreateOrderRequest request)
    {
        // Process the order...

        return CreatedAtAction(nameof(GetOrder), new { id = orderDto.Id }, orderDto);
    }
}
</code></pre>
<p><strong>Idempotency with Minimal APIs</strong></p>
<p>To implement idempotency with Minimal APIs, we can use an <code>IEndpointFilter</code>.</p>
<pre><code class="language-csharp">internal sealed class IdempotencyFilter(int cacheTimeInMinutes = 60)
    : IEndpointFilter
{
    public async ValueTask&lt;object?&gt; InvokeAsync(
        EndpointFilterInvocationContext context,
        EndpointFilterDelegate next)
    {
        // Parse the Idempotence-Key header from the request
        if (TryGetIdempotenceKey(out Guid idempotenceKey))
        {
            return Results.BadRequest(&quot;Invalid or missing Idempotence-Key header&quot;);
        }

        IDistributedCache cache = context.HttpContext
            .RequestServices.GetRequiredService&lt;IDistributedCache&gt;();

        // Check if we already processed this request and return a cached response (if it exists)
        string cacheKey = $&quot;Idempotent_{idempotenceKey}&quot;;
        string? cachedResult = await cache.GetStringAsync(cacheKey);
        if (cachedResult is not null)
        {
            IdempotentResponse response = JsonSerializer.Deserialize&lt;IdempotentResponse&gt;(cachedResult)!;
            return new IdempotentResult(response.StatusCode, response.Value);
        }

        object? result = await next(context);

        // Execute the request and cache the response for the specified duration
        if (result is IStatusCodeHttpResult { StatusCode: &gt;= 200 and &lt; 300 } statusCodeResult
            and IValueHttpResult valueResult)
        {
            int statusCode = statusCodeResult.StatusCode ?? StatusCodes.Status200OK;
            IdempotentResponse response = new(statusCode, valueResult.Value);

            await cache.SetStringAsync(
                cacheKey,
                JsonSerializer.Serialize(response),
                new DistributedCacheEntryOptions
                {
                    AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(cacheTimeInMinutes)
                }
            );
        }

        return result;
    }
}

// We have to implement a custom result to write the status code
internal sealed class IdempotentResult : IResult
{
    private readonly int _statusCode;
    private readonly object? _value;

    public IdempotentResult(int statusCode, object? value)
    {
        _statusCode = statusCode;
        _value = value;
    }

    public Task ExecuteAsync(HttpContext httpContext)
    {
        httpContext.Response.StatusCode = _statusCode;

        return httpContext.Response.WriteAsJsonAsync(_value);
    }
}
</code></pre>
<p>Now, we can apply this endpoint filter to our Minimal API endpoint:</p>
<pre><code class="language-csharp">app.MapPost(&quot;/api/orders&quot;, CreateOrder)
    .RequireAuthorization()
    .WithOpenApi()
    .AddEndpointFilter&lt;IdempotencyFilter&gt;();
</code></pre>
<p>An alternative to the previous two implementations is implementing idempotency logic in a custom middleware.</p>
<h2>Best Practices and Considerations</h2>
<p>Here are the key things I always keep in mind when implementing idempotency.</p>
<p>Cache duration is tricky.
I aim to cover reasonable retry windows without holding onto stale data.
A reasonable cache time typically ranges from a few minutes to 24-48 hours, depending on your specific use case.</p>
<p>Concurrency can be a pain, especially in high-traffic APIs.
A thread-safe implementation using a distributed lock works great.
It keeps things in check when multiple requests hit at once.
But this should be a rare occurrence.</p>
<p>For distributed setups, Redis is my go-to.
It's perfect as a shared cache, keeping idempotency consistent across all your API instances.
Plus, it handles distributed locking.</p>
<p>What if a client reuses an idempotency key with a different request body?
I return an error in this case.
My approach is to hash the request body and store it with the idempotency key.
When a request comes in, I compare the request body hashes.
If they differ, I return an error.
This prevents misuse of idempotency keys and maintains the integrity of your API.</p>
<h2>Summary</h2>
<p>Implementing idempotency in REST APIs enhances service reliability and consistency.
It ensures identical requests yield the same result, preventing unintended duplicates and gracefully handling network issues.</p>
<p>While our implementation provides a foundation, I recommend adapting it to your needs.
Focus on critical operations in your APIs, especially those that modify the system state or trigger important business processes.</p>
<p>By embracing idempotency, you're building more robust and user-friendly APIs.</p>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_113.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Problem Details for ASP.NET Core APIs]]></title>
            <link>https://milanjovanovic.tech/blog/problem-details-for-aspnetcore-apis</link>
            <guid isPermaLink="false">problem-details-for-aspnetcore-apis</guid>
            <pubDate>Sat, 19 Oct 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Discover how to leverage Problem Details in ASP.NET Core to create consistent, informative API error responses that enhance developer experience and comply with RFC 9457. This comprehensive guide explores the latest features in .NET 8 and 9, demonstrating how to implement custom exception handlers, utilize IProblemDetailsService, and apply best practices for robust error handling in your APIs.]]></description>
            <content:encoded><![CDATA[<p>When developing HTTP APIs, providing consistent and informative error responses is crucial for a smooth developer experience.
<strong>Problem Details</strong> in <a href="http://ASP.NET">ASP.NET</a> Core offers a standardized solution to this challenge, ensuring your APIs communicate errors effectively and uniformly.</p>
<p>In this article, we'll explore the latest developments in <strong>Problem Details</strong>, including:</p>
<ul>
<li>The new <a href="https://www.rfc-editor.org/rfc/rfc9457">RFC 9457</a> that refines the Problem Details standard</li>
<li>Using the .NET 8 <code>IExceptionHandler</code> for global exception handling</li>
<li>Using the <code>IProblemDetailsService</code> for customizing Problem Details</li>
</ul>
<p>Let's dive into these features and see how they can improve your API's error handling.</p>
<h2>Understanding Problem Details</h2>
<p>Problem Details is a machine-readable format for specifying errors in HTTP API responses.
HTTP status codes don't always contain enough details about errors to be helpful.
The Problem Details specification defines a JSON (and XML) document format to describe problems.</p>
<p>Problem Details includes:</p>
<ul>
<li><code>type</code>: A URI reference that identifies the problem type</li>
<li><code>title</code>: A short, human-readable summary of the problem type</li>
<li><code>status</code>: The HTTP status code</li>
<li><code>detail</code>: A human-readable explanation specific to this occurrence of the problem</li>
<li><code>instance</code>: A URI reference that identifies the specific occurrence of the problem</li>
</ul>
<p><a href="https://www.rfc-editor.org/rfc/rfc9457">RFC 9457</a>, which replaces <a href="https://www.rfc-editor.org/rfc/rfc7807">RFC 7807</a>,
introduces improvements such as clarifying the use of the type field and providing guidelines for extending <strong>Problem Details</strong>.</p>
<p>Here's an example Problem Details response:</p>
<pre><code class="language-json">Content-Type: application/problem+json

{
  &quot;type&quot;: &quot;https://tools.ietf.org/html/rfc9110#section-15.5.5&quot;,
  &quot;title&quot;: &quot;Not Found&quot;,
  &quot;status&quot;: 404,
  &quot;detail&quot;: &quot;The habit with the specified identifier was not found&quot;,
  &quot;instance&quot;: &quot;PUT /api/habits/aadcad3f-8dc8-443d-be44-3d99893ba18a&quot;
}
</code></pre>
<h2>Implementing Problem Details</h2>
<p>Let's see how to implement Problem Details in <a href="http://ASP.NET">ASP.NET</a> Core.
We want to return a Problem Details response for unhandled exceptions.
By calling <code>AddProblemDetails</code>, we're configuring the application to use the Problem Details format for failed requests.
With <code>UseExceptionHandler</code>, we introduce an exception handling middleware to the request pipeline.
By adding <code>UseStatusCodePages</code>, we're introducing a middleware that will convert error responses with an empty body to a Problem Details response.</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);

// Adds services for using Problem Details format
builder.Services.AddProblemDetails();

var app = builder.Build();

// Converts unhandled exceptions into Problem Details responses
app.UseExceptionHandler();

// Returns the Problem Details response for (empty) non-successful responses
app.UseStatusCodePages();

app.Run();
</code></pre>
<p>When we encounter an unhandled exception, it will be translated to a Problem Details response:</p>
<pre><code class="language-json">Content-Type: application/problem+json

{
  &quot;type&quot;: &quot;https://tools.ietf.org/html/rfc9110#section-15.6.1&quot;,
  &quot;title&quot;: &quot;An error occurred while processing your request.&quot;,
  &quot;status&quot;: 500
}
</code></pre>
<p>Now, let's explore how we can customize this response.</p>
<h2>Global Error Handling</h2>
<p>We have a few options for <a href="global-error-handling-in-aspnetcore-8">implementing global error handling</a>.
The most popular approach is creating a custom exception handling middleware.
You wrap the API request in a <code>try-catch</code> statement and return a response based on any caught exception.</p>
<p>With .NET 8, we can use the <code>IExceptionHandler</code> that runs in the built-in exception handling middleware.
This handler allows you to tailor the Problem Details response for specific exceptions.
Returning <code>true</code> from the <code>TryHandleAsync</code> method short-circuits the pipeline and returns the API response.
If we return <code>false</code>, the next handler in the chain attempts to handle the exception.</p>
<p>We can map different exception types to appropriate HTTP status codes, providing more precise error information to API consumers.</p>
<p>Here's an example <code>CustomExceptionHandler</code> implementation:</p>
<pre><code class="language-csharp">internal sealed class CustomExceptionHandler : IExceptionHandler
{
    public async ValueTask&lt;bool&gt; TryHandleAsync(
        HttpContext httpContext,
        Exception exception,
        CancellationToken cancellationToken)
    {
        int status = exception switch
        {
            ArgumentException =&gt; StatusCodes.Status400BadRequest,
            _ =&gt; StatusCodes.Status500InternalServerError
        };
        httpContext.Response.StatusCode = status;

        var problemDetails = new ProblemDetails
        {
            Status = status,
            Title = &quot;An error occurred&quot;,
            Type = exception.GetType().Name,
            Detail = exception.Message
        };

        await httpContext.Response.WriteAsJsonAsync(problemDetails, cancellationToken);

        return true;
    }
}

// In Program.cs
builder.Services.AddExceptionHandler&lt;CustomExceptionHandler&gt;();
</code></pre>
<h2>Using The ProblemDetailsService</h2>
<p>Calling <code>AddProblemDetails</code> registers a default implementation of the <code>IProblemDetailsService</code>.
The <code>IProblemDetailsService</code> will set the response status code based on the <code>ProblemDetails.Status</code>.</p>
<p>Here's how we can use it in the <code>CustomExceptionHandler</code>:</p>
<pre><code class="language-csharp">public class CustomExceptionHandler(IProblemDetailsService problemDetailsService) : IExceptionHandler
{
    public async ValueTask&lt;bool&gt; TryHandleAsync(
        HttpContext httpContext,
        Exception exception,
        CancellationToken cancellationToken)
    {
        var problemDetails = new ProblemDetails
        {
            Status = exception switch
            {
                ArgumentException =&gt; StatusCodes.Status400BadRequest,
                _ =&gt; StatusCodes.Status500InternalServerError
            },
            Title = &quot;An error occurred&quot;,
            Type = exception.GetType().Name,
            Detail = exception.Message
        };

        return await problemDetailsService.TryWriteAsync(new ProblemDetailsContext
        {
            Exception = exception,
            HttpContext = httpContext,
            ProblemDetails = problemDetails
        });
    }
}
</code></pre>
<p>This approach seems very similar to the previous one, where we wrote to the response body.
However, using the <code>IProblemDetailsService</code> gives an easy way to customize all Problem Details responses.</p>
<p>We can return Problem Details in controllers using the <code>Problem</code> method, or <code>Results.Problem</code> in Minimal APIs.
These methods respect the configured Problem Details customizations (more on this in the next section).</p>
<pre><code class="language-csharp">IdentityUser identityUser = new() { UserName = registerUserDto.UserName, Email = registerUserDto.Email };
IdentityResult result = await userManager.CreateAsync(identityUser, registerUserDto.Password);

if (!result.Succeeded)
{
    // return Results.Problem - Minimal APIs
    return Problem(
        type: &quot;Bad Request&quot;,
        title: &quot;Identity failure&quot;,
        detail: result.Errors.First().Description,
        statusCode: StatusCodes.Status400BadRequest);
}
</code></pre>
<h2>Customizing Problem Details</h2>
<p>We can pass a delegate to the <code>AddProblemDetails</code> method to set the <code>CustomizeProblemDetails</code>.
You can use this to add extra information to all Problem Details responses.</p>
<p>This is an excellent place for solving cross-cutting concerns, like setting the <code>instance</code> value and adding diagnostics information.</p>
<pre><code class="language-csharp">builder.Services.AddProblemDetails(options =&gt;
{
    options.CustomizeProblemDetails = context =&gt;
    {
        context.ProblemDetails.Instance =
            $&quot;{context.HttpContext.Request.Method} {context.HttpContext.Request.Path}&quot;;

        context.ProblemDetails.Extensions.TryAdd(&quot;requestId&quot;, context.HttpContext.TraceIdentifier);

        Activity? activity = context.HttpContext.Features.Get&lt;IHttpActivityFeature&gt;()?.Activity;
        context.ProblemDetails.Extensions.TryAdd(&quot;traceId&quot;, activity?.Id);
    };
});
</code></pre>
<p>This customization adds the request path, a request ID, and a trace ID to every Problem Details response, enhancing debuggability and traceability of errors.</p>
<pre><code class="language-json">Content-Type: application/problem+json

{
  &quot;type&quot;: &quot;https://tools.ietf.org/html/rfc9110#section-15.5.5&quot;,
  &quot;title&quot;: &quot;Not Found&quot;,
  &quot;status&quot;: 404,
  &quot;instance&quot;: &quot;PUT /api/habits/aadcad3f-8dc8-443d-be44-3d99893ba18a&quot;,
  &quot;traceId&quot;: &quot;00-63d4af1807586b0d98901ae47944192d-9a8635facb90bf76-01&quot;,
  &quot;requestId&quot;: &quot;0HN7C8PRNMGIA:00000001&quot;
}
</code></pre>
<p>You can use the <code>traceId</code> to find the distributed traces and logs in a monitoring system like Seq.</p>
<div className="bordered">
  ![Seq user interface showing a distributed trace with the same trace identifier as the problem details response.](/blogs/mnw_112/seq_tracing.png)
</div>
<h2>Handling Specific Exceptions (Status Codes)</h2>
<p>.NET 9 introduces a simpler way to map exceptions to status codes.
Great news for fans of throwing exceptions.
You can use the <code>StatusCodeSelector</code> to define the mappings.
This makes it easier to maintain consistent error responses across your API.</p>
<pre><code class="language-csharp">app.UseExceptionHandler(new ExceptionHandlerOptions
{
    StatusCodeSelector = ex =&gt; ex switch
    {
        ArgumentException =&gt; StatusCodes.Status400BadRequest,
        NotFoundException =&gt; StatusCodes.Status404NotFound,
        _ =&gt; StatusCodes.Status500InternalServerError
    }
});
</code></pre>
<p>If you use this together with an <code>IExceptionHandler</code> that sets the <code>StatusCode</code>, then the <code>StatusCodeSelector</code> is ignored.</p>
<h2>Takeaway</h2>
<p>Implementing Problem Details in your <a href="http://ASP.NET">ASP.NET</a> Core APIs is more than just a best practice -
it's a standard for improving the developer experience of your API consumers.
By providing consistent, detailed, and well-structured error responses, you make it easier for clients to understand and handle error scenarios gracefully.</p>
<p>As you implement these practices in your own projects, you'll discover even more ways to tailor Problem Details to your specific needs.
I shared what worked well for my use cases.</p>
<p>{/* Update this when the course is out */}
Problem Details is just one of the best practices covered in my upcoming <a href="/rest-apis-in-aspnetcore"><strong>REST APIs course</strong></a>.
Check it out if you're looking for a comprehensive guide.</p>
<p>Good luck out there, and I'll see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_112.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Scaling the Outbox Pattern (2B+ messages per day)]]></title>
            <link>https://milanjovanovic.tech/blog/scaling-the-outbox-pattern</link>
            <guid isPermaLink="false">scaling-the-outbox-pattern</guid>
            <pubDate>Sat, 12 Oct 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Learn how to supercharge your Outbox pattern implementation, scaling to 30,500 messages per second. Through strategic optimizations in database queries, message publishing, and parallel processing, I'll show you how to handle over 2.8 billion messages daily while maintaining system reliability.]]></description>
            <content:encoded><![CDATA[<p>In last week's newsletter, I talked about <a href="implementing-the-outbox-pattern">implementing the Outbox pattern</a>.
It's a crucial tool for reliable distributed messaging.
But implementing it is just the first step.</p>
<p>The real challenge? Scaling it to handle massive message volumes.</p>
<p>Today, we're taking it up a notch.
We'll start with a basic Outbox processor and transform it into a high-performance engine capable of handling over 2 billion messages daily.</p>
<p>Let's dive in!</p>
<h2>Starting Point</h2>
<p>This is our starting point.
We have an <code>OutboxProcessor</code> that polls for unprocessed messages and publishes them to a queue.
The first few things we can tweak are the <strong>frequency</strong> and <strong>batch size</strong>.</p>
<pre><code class="language-csharp">internal sealed class OutboxProcessor(NpgsqlDataSource dataSource, IPublishEndpoint publishEndpoint)
{
    private const int BatchSize = 1000;

    public async Task&lt;int&gt; Execute(CancellationToken cancellationToken = default)
    {
        await using var connection = await dataSource.OpenConnectionAsync(cancellationToken);
        await using var transaction = await connection.BeginTransactionAsync(cancellationToken);

        var messages = await connection.QueryAsync&lt;OutboxMessage&gt;(
            @&quot;&quot;&quot;
            SELECT *
            FROM outbox_messages
            WHERE processed_on_utc IS NULL
            ORDER BY occurred_on_utc LIMIT @BatchSize
            &quot;&quot;&quot;,
            new { BatchSize },
            transaction: transaction);

        foreach (var message in messages)
        {
            try
            {
                var messageType = Messaging.Contracts.AssemblyReference.Assembly.GetType(message.Type);
                var deserializedMessage = JsonSerializer.Deserialize(message.Content, messageType);

                await publishEndpoint.Publish(deserializedMessage, messageType, cancellationToken);

                await connection.ExecuteAsync(
                    @&quot;&quot;&quot;
                    UPDATE outbox_messages
                    SET processed_on_utc = @ProcessedOnUtc
                    WHERE id = @Id
                    &quot;&quot;&quot;,
                    new { ProcessedOnUtc = DateTime.UtcNow, message.Id },
                    transaction: transaction);
            }
            catch (Exception ex)
            {
                await connection.ExecuteAsync(
                    @&quot;&quot;&quot;
                    UPDATE outbox_messages
                    SET processed_on_utc = @ProcessedOnUtc, error = @Error
                    WHERE id = @Id
                    &quot;&quot;&quot;,
                    new { ProcessedOnUtc = DateTime.UtcNow, Error = ex.ToString(), message.Id },
                    transaction: transaction);
            }
        }

        await transaction.CommitAsync(cancellationToken);

        return messages.Count;
    }
}
</code></pre>
<p>Let's assume that we run the <code>OutboxProcessor</code> continuously.
I increased that batch size to <code>1000</code>.</p>
<p>How many messages are we able to process?</p>
<p>I'll run the Outbox processing for 1 minute and count how many messages were processed.</p>
<p>The baseline implementation processed <strong>81,000</strong> messages in one minute or <strong>1,350 MPS</strong> (messages per second).</p>
<p>Not bad, but let's see how much we can improve this.</p>
<h2>Measuring Each Step</h2>
<p>You can't improve what you can't measure. Right?
So, I'll use a <code>Stopwatch</code> to measure the total execution time and the time each step takes.</p>
<p>Notice that I also split the publish and update steps.
It's so I can measure the time for publishing and updating separately.
This will be important later because I want to optimize each step separately.</p>
<p>With the baseline implementation, here are the execution times for each step:</p>
<ul>
<li>Query time: ~70ms</li>
<li>Publish time: ~320ms</li>
<li>Update time: ~300ms</li>
</ul>
<pre><code class="language-csharp">internal sealed class OutboxProcessor(
    NpgsqlDataSource dataSource,
    IPublishEndpoint publishEndpoint,
    ILogger&lt;OutboxProcessor&gt; logger)
{
    private const int BatchSize = 1000;

    public async Task&lt;int&gt; Execute(CancellationToken cancellationToken = default)
    {
        var totalStopwatch = Stopwatch.StartNew();
        var stepStopwatch = new Stopwatch();

        await using var connection = await dataSource.OpenConnectionAsync(cancellationToken);
        await using var transaction = await connection.BeginTransactionAsync(cancellationToken);

        stepStopwatch.Restart();
        var messages = (await connection.QueryAsync&lt;OutboxMessage&gt;(
            @&quot;&quot;&quot;
            SELECT *
            FROM outbox_messages
            WHERE processed_on_utc IS NULL
            ORDER BY occurred_on_utc LIMIT @BatchSize
            &quot;&quot;&quot;,
            new { BatchSize },
            transaction: transaction)).AsList();
        var queryTime = stepStopwatch.ElapsedMilliseconds;

        var updateQueue = new ConcurrentQueue&lt;OutboxUpdate&gt;();

        stepStopwatch.Restart();
        foreach (var message in messages)
        {
            try
            {
                var messageType = Messaging.Contracts.AssemblyReference.Assembly.GetType(message.Type);
                var deserializedMessage = JsonSerializer.Deserialize(message.Content, messageType);

                await publishEndpoint.Publish(deserializedMessage, messageType, cancellationToken);

                updateQueue.Enqueue(new OutboxUpdate
                {
                    Id = message.Id,
                    ProcessedOnUtc = DateTime.UtcNow
                });
            }
            catch (Exception ex)
            {
                updateQueue.Enqueue(new OutboxUpdate
                {
                    Id = message.Id,
                    ProcessedOnUtc = DateTime.UtcNow,
                    Error = ex.ToString()
                });
            }
        }
        var publishTime = stepStopwatch.ElapsedMilliseconds;

        stepStopwatch.Restart();
        foreach (var outboxUpdate in updateQueue)
        {
            await connection.ExecuteAsync(
                @&quot;&quot;&quot;
                UPDATE outbox_messages
                SET processed_on_utc = @ProcessedOnUtc, error = @Error
                WHERE id = @Id
                &quot;&quot;&quot;,
                outboxUpdate,
                transaction: transaction);
        }
        var updateTime = stepStopwatch.ElapsedMilliseconds;

        await transaction.CommitAsync(cancellationToken);

        totalStopwatch.Stop();
        var totalTime = totalStopwatch.ElapsedMilliseconds;

        OutboxLoggers.Processing(logger, totalTime, queryTime, publishTime, updateTime, messages.Count);

        return messages.Count;
    }

    private struct OutboxUpdate
    {
        public Guid Id { get; init; }
        public DateTime ProcessedOnUtc { get; init; }
        public string? Error { get; init; }
    }
}
</code></pre>
<p>Now, onto the fun part!</p>
<h2>Optimizing Read Queries</h2>
<p>The first thing I want to optimize is the query for fetching unprocessed messages.
Performing a <code>SELECT *</code> query will have an impact if we don't need all the columns (hint: we don't).</p>
<p>Here's the current SQL query:</p>
<pre><code class="language-sql">SELECT *
FROM outbox_messages
WHERE processed_on_utc IS NULL
ORDER BY occurred_on_utc LIMIT @BatchSize
</code></pre>
<p>We can modify the query to return only the columns we need.
This will save us some bandwidth but will not significantly improve performance.</p>
<pre><code class="language-sql">SELECT id AS Id, type AS Type, content as Content
FROM outbox_messages
WHERE processed_on_utc IS NULL
ORDER BY occurred_on_utc LIMIT @BatchSize
</code></pre>
<p>Let's examine the execution plan for this query.
You'll see it's performing a table scan.
I'm running this on PostgreSQL, and here's what I get from <code>EXPLAIN ANALYZE</code>:</p>
<pre><code>Limit  (cost=86169.40..86286.08 rows=1000 width=129) (actual time=122.744..124.234 rows=1000 loops=1)
  -&gt;  Gather Merge  (cost=86169.40..245080.50 rows=1362000 width=129) (actual time=122.743..124.198 rows=1000 loops=1)
        Workers Planned: 2
        Workers Launched: 2
        -&gt;  Sort  (cost=85169.38..86871.88 rows=681000 width=129) (actual time=121.478..121.492 rows=607 loops=3)
              Sort Key: occurred_on_utc
              Sort Method: top-N heapsort  Memory: 306kB
              Worker 0:  Sort Method: top-N heapsort  Memory: 306kB
              Worker 1:  Sort Method: top-N heapsort  Memory: 306kB
              -&gt;  Parallel Seq Scan on outbox_messages  (cost=0.00..47830.88 rows=681000 width=129) (actual time=0.016..67.481 rows=666667 loops=3)
                    Filter: (processed_on_utc IS NULL)
Planning Time: 0.051 ms
Execution Time: 124.298 ms
</code></pre>
<p>Now, I'll create an index that &quot;covers&quot; the query for fetching unprocessed messages.
A covered index contains all the columns needed to satisfy a query without accessing the table itself.</p>
<p>The index will be on the <code>occurred_on_utc</code> and <code>processed_on_utc</code> columns.
It will include the <code>id</code>, <code>type</code>, and <code>content</code> columns.
Lastly, we'll apply a filter to index unprocessed messages only.</p>
<pre><code class="language-sql">CREATE INDEX IF NOT EXISTS idx_outbox_messages_unprocessed
ON public.outbox_messages (occurred_on_utc, processed_on_utc)
INCLUDE (id, type, content)
WHERE processed_on_utc IS NULL
</code></pre>
<p>Let me explain the reasoning behind each decision:</p>
<ul>
<li>Indexing the <code>occurred_on_utc</code> will store the entries in the index in ascending order.
This matches the <code>ORDER BY occurred_on_utc</code> statement in the query.
This means the query can scan the index without sorting the results.
The results are already in the correct sort order.</li>
<li>Including the columns we select in the index allows us to return them from the index entry.
This avoids reading the values from the table rows.</li>
<li>Filtering for unprocessed messages in the index satisfies the <code>WHERE processed_on_utc IS NULL</code> statement.</li>
</ul>
<p><strong>Caveat</strong>: PostgreSQL has a maximum index row size of <strong>2712B</strong> (don't ask how I know).
The columns in the <code>INCLUDE</code> list are also part of the index row (B-tree tuple).
The <code>content</code> column contains the serialized JSON message, so it's the most likely culprit to make us exceed this limit.
There's no way around it, so my advice is to keep your messages as small as possible.
You could exclude this column from the <code>INCLUDE</code> list for a minor performance hit.</p>
<p>Here's the updated execution plan after creating this index:</p>
<pre><code>Limit  (cost=0.43..102.82 rows=1000 width=129) (actual time=0.016..0.160 rows=1000 loops=1)
  -&gt;  Index Only Scan using idx_outbox_messages_unprocessed on outbox_messages  (cost=0.43..204777.36 rows=2000000 width=129) (actual time=0.015..0.125 rows=1000 loops=1)
        Heap Fetches: 0
Planning Time: 0.059 ms
Execution Time: 0.189 ms
</code></pre>
<p>Because we have a covered index, the execution plan only contains an <code>Index Only Scan</code> and <code>Limit</code> operation.
There's no filtering or sorting that needs to happen, which is why we see a massive performance improvement.</p>
<p>What's the performance impact on the query time?</p>
<ul>
<li>Query time: 70ms → 1ms <strong>(-98.5%)</strong></li>
</ul>
<h2>Optimizing Message Publishing</h2>
<p>The next thing we can optimize is how we're publishing messages to the queue.
I'm using the <code>IPublishEndpoint</code> from MassTransit to publish to RabbitMQ.</p>
<p>To be more precise here, we're publishing to an exchange.
The exchange will then route the message to the appropriate queue.</p>
<p>But how can we optimize this?</p>
<p>A micro-optimization we can do is introduce a cache for the message types used in serialization.
Performing reflection constantly for every message type is expensive, so we'll do the reflection once, and store the result.</p>
<pre><code class="language-csharp">var messageType = Messaging.Contracts.AssemblyReference.Assembly.GetType(message.Type);
</code></pre>
<p>The cache can be a <code>ConcurrentDictionary</code>, and we'll use <code>GetOrAdd</code> to retrieve the cached types.</p>
<p>I'll extract this piece of code to the <code>GetOrAddMessageType</code> helper method:</p>
<pre><code class="language-csharp">private static readonly ConcurrentDictionary&lt;string, Type&gt; TypeCache = new();

private static Type GetOrAddMessageType(string typeName)
{
    return TypeCache.GetOrAdd(
        typeName,
        name =&gt; Messaging.Contracts.AssemblyReference.Assembly.GetType(name));
}
</code></pre>
<p>This is what our message publishing step looks like.
The biggest problem is we're waiting for the <code>Publish</code> to complete by awaiting it.
The <code>Publish</code> takes some time because it's waiting for confirmation from the message broker.
We're doing this in a loop, which makes it even less efficient.</p>
<pre><code class="language-csharp">var updateQueue = new ConcurrentQueue&lt;OutboxUpdate&gt;();

foreach (var message in messages)
{
    try
    {
        var messageType = Messaging.Contracts.AssemblyReference.Assembly.GetType(message.Type);
        var deserializedMessage = JsonSerializer.Deserialize(message.Content, messageType);

        // We're waiting for the message broker confirmation here.
        await publishEndpoint.Publish(deserializedMessage, messageType, cancellationToken);

        updateQueue.Enqueue(new OutboxUpdate
        {
            Id = message.Id,
            ProcessedOnUtc = DateTime.UtcNow
        });
    }
    catch (Exception ex)
    {
        updateQueue.Enqueue(new OutboxUpdate
        {
            Id = message.Id,
            ProcessedOnUtc = DateTime.UtcNow,
            Error = ex.ToString()
        });
    }
}
</code></pre>
<p>We can improve this by publishing the messages in a batch.
In fact, the <code>IPublishEndpoint</code> has a <code>PublishBatch</code> extension method.
If we peek inside, here's what we'll find:</p>
<pre><code class="language-csharp">// MassTransit implementation
public static Task PublishBatch(
    this IPublishEndpoint endpoint,
    IEnumerable&lt;object&gt; messages,
    CancellationToken cancellationToken = default)
{
    return Task.WhenAll(messages.Select(x =&gt; endpoint.Publish(x, cancellationToken)));
}
</code></pre>
<p>So we can transform the collection of messages into a list of publishing tasks that we can await using <code>Task.WhenAll</code>.</p>
<pre><code class="language-csharp">var updateQueue = new ConcurrentQueue&lt;OutboxUpdate&gt;();

var publishTasks = messages
    .Select(message =&gt; PublishMessage(message, updateQueue, publishEndpoint, cancellationToken))
    .ToList();

await Task.WhenAll(publishTasks);

// I extracted the message publishing into a separate method for readability.
private static async Task PublishMessage(
    OutboxMessage message,
    ConcurrentQueue&lt;OutboxUpdate&gt; updateQueue,
    IPublishEndpoint publishEndpoint,
    CancellationToken cancellationToken)
{
    try
    {
        var messageType = GetOrAddMessageType(message.Type);
        var deserializedMessage = JsonSerializer.Deserialize(message.Content, messageType);

        await publishEndpoint.Publish(deserializedMessage, messageType, cancellationToken);

        updateQueue.Enqueue(new OutboxUpdate
        {
            Id = message.Id,
            ProcessedOnUtc = DateTime.UtcNow
        });
    }
    catch (Exception ex)
    {
        updateQueue.Enqueue(new OutboxUpdate
        {
            Id = message.Id,
            ProcessedOnUtc = DateTime.UtcNow,
            Error = ex.ToString()
        });
    }
}
</code></pre>
<p>What's the improvement for the message publishing step?</p>
<ul>
<li>Publish time: 320ms → 289ms <strong>(-9.8%)</strong></li>
</ul>
<p>As you can see, it's not significantly faster.
But this is needed for us to benefit from other optimizations I have in store.</p>
<h2>Optimizing Update Queries</h2>
<p>The next step in our optimization journey is addressing the query updating the processed Outbox messages.</p>
<p>The current implementation is inefficient because we send one query to the database for each Outbox message.</p>
<pre><code class="language-csharp">foreach (var outboxUpdate in updateQueue)
{
    await connection.ExecuteAsync(
        @&quot;&quot;&quot;
        UPDATE outbox_messages
        SET processed_on_utc = @ProcessedOnUtc, error = @Error
        WHERE id = @Id
        &quot;&quot;&quot;,
        outboxUpdate,
        transaction: transaction);
}
</code></pre>
<p>If you didn't get the memo by now, batching is the name of the game.
We want a way to send one large <code>UPDATE</code> query to the database.</p>
<p>We have to construct the SQL for this batch query manually.
We'll use the <code>DynamicParameters</code> type from Dapper to provide all the parameters.</p>
<pre><code class="language-csharp">var updateSql =
    @&quot;&quot;&quot;
    UPDATE outbox_messages
    SET processed_on_utc = v.processed_on_utc,
        error = v.error
    FROM (VALUES
        {0}
    ) AS v(id, processed_on_utc, error)
    WHERE outbox_messages.id = v.id::uuid
    &quot;&quot;&quot;;

var updates = updateQueue.ToList();
var paramNames = string.Join(&quot;,&quot;, updates.Select((_, i) =&gt; $&quot;(@Id{i}, @ProcessedOn{i}, @Error{i})&quot;));

var formattedSql = string.Format(updateSql, paramNames);

var parameters = new DynamicParameters();

for (int i = 0; i &lt; updates.Count; i++)
{
    parameters.Add($&quot;Id{i}&quot;, updates[i].Id.ToString());
    parameters.Add($&quot;ProcessedOn{i}&quot;, updates[i].ProcessedOnUtc);
    parameters.Add($&quot;Error{i}&quot;, updates[i].Error);
}

await connection.ExecuteAsync(formattedSql, parameters, transaction: transaction);
</code></pre>
<p>This will produce a SQL query that looks something like this:</p>
<pre><code class="language-sql">UPDATE outbox_messages
SET processed_on_utc = v.processed_on_utc,
    error = v.error
FROM (VALUES
    (@Id0, @ProcessedOn0, @Error0),
    (@Id1, @ProcessedOn1, @Error1),
    (@Id2, @ProcessedOn2, @Error2),
    -- A few hundred rows in beteween
    (@Id999, @ProcessedOn999, @Error999)
) AS v(id, processed_on_utc, error)
WHERE outbox_messages.id = v.id::uuid
</code></pre>
<p>Instead of sending one update query per message, we can send one query to update all messages.</p>
<p>This will obviously give us a noticeable performance benefit:</p>
<ul>
<li>Update time: 300ms → 52ms <strong>(-82.6%)</strong></li>
</ul>
<h2>How Far Did We Get?</h2>
<p>Let's test out the performance improvement with the current optimizations.
The changes we made so far focus on improving the speed of the <code>OutboxProcessor</code>.</p>
<p>Here are the rough numbers I'm seeing for the individual steps:</p>
<ul>
<li>Query time: ~<strong>1ms</strong></li>
<li>Publish time: ~<strong>289ms</strong></li>
<li>Update time: ~<strong>52ms</strong></li>
</ul>
<p>I'll run the Outbox processing for 1 minute and count the number of processed messages.</p>
<p>The optimized implementation processed <strong>162,000</strong> messages in one minute or <strong>2,700 MPS</strong>.</p>
<p>For reference, this allows us to process more than 230 million messages per day.</p>
<p>But we're just getting started.</p>
<h2>Parallel Outbox Processing</h2>
<p>If we want to take this further, we have to scale out the <code>OutboxProcessor</code>.
The problem we could face here is processing the same message more than once.
So, we need to implement some form of locking on the current batch of messages.</p>
<p>PostgreSQL has a convenient <code>FOR UPDATE</code> statement that we can use here.
It will lock the selected rows for the duration of the current transaction.
However, we must add the <code>SKIP LOCKED</code> statement to allow other queries to skip the locked rows.
Otherwise, any other query will be blocked until the current transaction is completed.</p>
<p>Here's the updated query:</p>
<pre><code class="language-sql">SELECT id AS Id, type AS Type, content as Content
FROM outbox_messages
WHERE processed_on_utc IS NULL
ORDER BY occurred_on_utc LIMIT @BatchSize
FOR UPDATE SKIP LOCKED
</code></pre>
<p>To scale out the <code>OutboxProcessor</code>, we simply run multiple instances of the background job.</p>
<p>I'll simulate this using <code>Parallel.ForEachAsync</code>, where I can control the <code>MaxDegreeOfParallelism</code>.</p>
<pre><code class="language-csharp">var parallelOptions = new ParallelOptions
{
    MaxDegreeOfParallelism = _maxParallelism,
    CancellationToken = cancellationToken
};

await Parallel.ForEachAsync(
    Enumerable.Range(0, _maxParallelism),
    parallelOptions,
    async (_, token) =&gt;
    {
        await ProcessOutboxMessages(token);
    });
</code></pre>
<p>We can process <strong>179,000</strong> messages in one minute or <strong>2,983 MPS</strong> with five (5) workers.</p>
<p>I thought this was supposed to be <em>much</em> faster. What gives?</p>
<p>Without parallel processing, we were able to get ~2,700 MPS.</p>
<p>A new <strong>bottleneck</strong> appears: publishing the messages in batches.</p>
<p>The publish time went from ~289ms to ~1,540ms.</p>
<p>Interestingly, if you multiply the base publish time (for one worker) by the number of workers, you roughly get to the new publish time.</p>
<p>We're wasting a lot of time waiting for the acknowledgment from the message broker.</p>
<p>How can we fix this?</p>
<h2>Batching Message Publishing</h2>
<div className="note-panel">
  **Note**: `ConfigureBatchPublish` was deprecated with `MassTransit.RabbitMQ`
  v8.3.2. This version uses the new `RabbitMQ.Client` v7, which was rewritten to
  use the TPL and async/await. You will see a similar performance improvement
  just from upgrading to this version.
</div>
<p>RabbitMQ supports publishing messages in batches.
We can enable this feature when configuring MassTransit by calling the <code>ConfigureBatchPublish</code> method.
MassTransit will buffer messages before sending them to RabbitMQ, to increase throughput.</p>
<pre><code class="language-csharp">builder.Services.AddMassTransit(x =&gt;
{
    x.UsingRabbitMq((context, cfg) =&gt;
    {
        cfg.Host(builder.Configuration.GetConnectionString(&quot;Queue&quot;), hostCfg =&gt;
        {
            hostCfg.ConfigureBatchPublish(batch =&gt;
            {
                batch.Enabled = true;
            });
        });

        cfg.ConfigureEndpoints(context);
    });
});
</code></pre>
<p>With only this small change, let's rerun our test with five workers.</p>
<p>This time around, we're able to process <strong>1,956,000</strong> messages in one minute.</p>
<p>Which gives us a blazing ~<strong>32,500 MPS</strong>.</p>
<p>This is more than 2.8 billion processed messages per day.</p>
<p>I could call it a day here, but there's one more thing I want to show you.</p>
<h2>Turning Off Publisher Confirmation (Dangerous)</h2>
<p>One more thing you <em>can</em> do (<strong>which I don't recommend</strong>) is turn off publisher confirmation.
This means that calling <code>Publish</code> won't wait until the message is confirmed by the broker (ack'd).
It could lead to <strong>reliability issues</strong> and potentially <strong>losing messages</strong>.</p>
<p>That being said, I did manage to get ~37,000 MPS with publisher confirmation turned off.</p>
<pre><code class="language-csharp">cfg.Host(builder.Configuration.GetConnectionString(&quot;Queue&quot;), hostCfg =&gt;
{
    hostCfg.PublisherConfirmation = false; // Dangerous. I don't recommend it.
    hostCfg.ConfigureBatchPublish(batch =&gt;
    {
        batch.Enabled = true;
    });
});
</code></pre>
<h2>Key Considerations for Scaling</h2>
<p>While we've achieved impressive throughput, consider these factors when implementing these techniques in a real-world system:</p>
<ol>
<li>
<p><strong>Consumer Capacity</strong>:
Can your consumers keep up?
Boosting producer throughput without matching consumer capacity can create backlogs.
Consider the entire pipeline when scaling.</p>
</li>
<li>
<p><strong>Delivery Guarantees</strong>:
Our optimizations maintain at-least-once delivery.
Design consumers to be idempotent to handle occasional duplicate messages.</p>
</li>
<li>
<p><strong>Message Ordering</strong>:
Parallel processing with <code>FOR UPDATE SKIP LOCKED</code> may cause out-of-order messages.
For strict ordering, consider the <strong>Inbox pattern</strong> on the consumer side to buffer messages.
An Inbox allows us to process messages in the correct order, even if they arrive out of sequence.</p>
</li>
<li>
<p><strong>Reliability vs. Performance Trade-offs</strong>:
Turning off publisher confirmation increases speed but risks message loss.
Weigh performance against reliability based on your specific needs.</p>
</li>
</ol>
<p>By addressing these factors, you'll create a high-performance Outbox processor that integrates smoothly with your system architecture.</p>
<h2>Summary</h2>
<p>We've come a long way from our initial Outbox processor.
Here's what we accomplished:</p>
<ol>
<li>Optimized database queries with smart indexing</li>
<li>Improved message publishing with batching</li>
<li>Streamlined database updates with batching</li>
<li>Scaled out Outbox processing with parallel workers</li>
<li>Leveraged RabbitMQ's batch publishing feature</li>
</ol>
<p>The result?
We boosted processing from 1,350 messages per second to an impressive <strong>32,500 MPS</strong>.
That's over 2.8 billion messages per day!</p>
<p>Scaling isn't just about raw speed - it's about identifying and addressing bottlenecks at each step.
By measuring, optimizing, and rethinking our approach, we achieved massive performance gains.</p>
<p>That's all for today. Hope this was helpful.</p>
<p><strong>P.S.</strong> You can find the <a href="https://github.com/m-jovanovic/outbox-scaling">source code</a> here.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_111.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Implementing the Outbox Pattern]]></title>
            <link>https://milanjovanovic.tech/blog/implementing-the-outbox-pattern</link>
            <guid isPermaLink="false">implementing-the-outbox-pattern</guid>
            <pubDate>Sat, 05 Oct 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Discover how the Outbox pattern solves the dual-write problem in distributed systems, ensuring data consistency between your database and external components. This article provides practical strategies for implementing and scaling the Outbox pattern in .NET applications.]]></description>
            <content:encoded><![CDATA[<p>In distributed systems, we often face the challenge of keeping our database and external systems in sync.
Imagine saving an order to a database and then publishing a message to a message broker.
If either operation fails, your system ends up in an inconsistent state.</p>
<p>The <a href="outbox-pattern-for-reliable-microservices-messaging"><strong>Outbox pattern</strong></a> solves this problem by treating message publication as part of your database transaction.
Instead of publishing messages directly, we save them to an Outbox table in our database, ensuring atomic operations.
A separate process then reliably publishes these messages.</p>
<p>In this newsletter, we'll dive into implementing this pattern in .NET, covering everything from setup to scaling.</p>
<h2>Why Do We Need the Outbox Pattern?</h2>
<p>The transactional Outbox pattern fixes a common problem in distributed systems.
This problem happens when you need to do two things at once: save data and communicate with an external component.</p>
<p>Consider scenarios like sending order confirmation emails, notifying other systems about new client registrations,
or updating inventory levels after an order is placed.
Each of these involves a local data change coupled with an external communication or update.</p>
<p>For example, imagine a microservice that needs to:</p>
<ul>
<li>Save a new order in its database</li>
<li>Tell other systems about this new order</li>
</ul>
<p>If one of these steps fails, your system could end up in an inconsistent state.
Maybe the order is saved, but no one else knows about it.
Or everyone thinks there's a new order, but it's not actually in the database.</p>
<p>Here's a <code>CreateOrderCommandHandler</code> without the Outbox pattern:</p>
<pre><code class="language-csharp">public class CreateOrderCommandHandler(
    IOrderRepository orderRepository,
    IProductInventoryChecker inventoryChecker,
    IUnitOfWork unitOfWork,
    IEventBus eventBus) : IRequestHandler&lt;CreateOrderCommand, OrderDto&gt;
{
    public async Task&lt;OrderDto&gt; Handle(CreateOrderCommand request, CancellationToken cancellationToken)
    {
        var order = new Order(request.CustomerId, request.ProductId, request.Quantity, inventoryChecker);

        await orderRepository.AddAsync(order);

        await unitOfWork.CommitAsync(cancellationToken);

        // The database transaction is completed at this point.

        await eventBus.Send(new OrderCreatedIntegrationEvent(order.Id));

        return new OrderDto { Id = order.Id, Total = order.Total };
    }
}
</code></pre>
<p>This code has a potential consistency problem.
After the database transaction is committed, two things could go wrong:</p>
<ol>
<li>
<p>The application might crash right after the transaction is committed but before the event is sent.
The order would be created in the database, but other systems wouldn't know about it.</p>
</li>
<li>
<p>The event bus might be down or unreachable when we try to send the event.
This would result in the order being created without notifying other systems.</p>
</li>
</ol>
<p>The transactional Outbox pattern helps solve this problem by ensuring that the database update and event publication are treated as a single atomic operation.</p>
<p><img src="/blogs/mnw_110/outbox_pattern.png" alt="Flow diagram explaining how the Outbox Pattern works."></p>
<p>The sequence diagram illustrates how the Outbox pattern solves our consistency challenge.
Instead of trying to save data and send a message as separate steps, we save both the order and an Outbox message in a <strong>single database transaction</strong>.
This is an all-or-nothing operation - we can't end up in an inconsistent state.</p>
<p>A separate Outbox processor handles the actual message sending.
It continuously checks for unsent messages in the Outbox table and publishes them to the message queue.
The processor marks messages as sent after successful publishing, preventing duplicates.</p>
<p>An important thing to realize here is that the Outbox pattern gives us <a href="https://www.cloudcomputingpatterns.org/at_least_once_delivery/">at-least-once delivery</a>.
The Outbox message will be sent at least once, but it could also be sent multiple times in case of retries.
This means we have to make our message consumers idempotent.</p>
<h2>Implementing the Outbox Pattern</h2>
<p>First, let's create our Outbox table where we will store messages:</p>
<pre><code class="language-sql">CREATE TABLE outbox_messages (
    id UUID PRIMARY KEY,
    type VARCHAR(255) NOT NULL,
    content JSONB NOT NULL,
    occurred_on_utc TIMESTAMP WITH TIME ZONE NOT NULL,
    processed_on_utc TIMESTAMP WITH TIME ZONE NULL,
    error TEXT NULL
);

-- We can consider adding this index since we will be querying for unprocessed messages often
-- and it will contain the rows in the correct sort order for our query.
CREATE INDEX IF NOT EXISTS idx_outbox_messages_unprocessed
ON outbox_messages (occurred_on_utc, processed_on_utc)
INCLUDE (id, type, content)
WHERE processed_on_utc IS NULL;
</code></pre>
<p>I'll use PostgreSQL as the database for this example.
Notice the <code>jsonb</code> type for the <code>content</code> column.
It allows for indexing and querying of the JSON data if needed in the future.</p>
<p>Now, let's create a class to represent our Outbox entry:</p>
<pre><code class="language-csharp">public sealed class OutboxMessage
{
    public Guid Id { get; init; }
    public string Type { get; init; }
    public string Content { get; init; }
    public DateTime OccurredOnUtc { get; init; }
    public DateTime? ProcessedOnUtc { get; init; }
    public string? Error { get; init; }
}
</code></pre>
<p>Here's how we can add a message to the Outbox:</p>
<pre><code class="language-csharp">public async Task AddToOutbox&lt;T&gt;(T message, NpgsqlDataSource dataSource)
{
    var outboxMessage = new OutboxMessage
    {
        Id = Guid.NewGuid(),
        OccurredOnUtc = DateTime.UtcNow,
        Type = typeof(T).FullName, // We'll need this for deserialization
        Content = JsonSerializer.Serialize(message)
    };

    await using var connection = await dataSource.OpenConnectionAsync();
    await connection.ExecuteAsync(
        @&quot;&quot;&quot;
        INSERT INTO outbox_messages (id, occurred_on_utc, type, content)
        VALUES (@Id, @OccurredOnUtc, @Type, @Content::jsonb)
        &quot;&quot;&quot;,
        outboxMessage);
}
</code></pre>
<p>Here's the <code>CreateOrderCommandHandler</code> using the Outbox pattern:</p>
<pre><code class="language-csharp">public class CreateOrderCommandHandler(
    IOrderRepository orderRepository,
    IProductInventoryChecker inventoryChecker,
    IUnitOfWork unitOfWork,
    NpgsqlDataSource dataSource) : IRequestHandler&lt;CreateOrderCommand, OrderDto&gt;
{
    public async Task&lt;OrderDto&gt; Handle(CreateOrderCommand request, CancellationToken cancellationToken)
    {
        var order = new Order(request.CustomerId, request.ProductId, request.Quantity, inventoryChecker);

        await orderRepository.AddAsync(order);

        // We can add the Outbox message before committing the transaction,
        // so both operations are part of the same transaction.
        await AddToOutbox(new OrderCreatedIntegrationEvent(order.Id), dataSource);

        await unitOfWork.CommitAsync(cancellationToken);

        return new OrderDto { Id = order.Id, Total = order.Total };
    }
}
</code></pre>
<p>An elegant approach to implementing this is using <a href="how-to-use-domain-events-to-build-loosely-coupled-systems"><strong>domain events</strong></a> to represent notifications.
When something significant happens in the domain, we will raise a domain event.
Before completing the transaction, we can pick up all events and store them as Outbox messages.
You could do this from the unit of work or with an <a href="how-to-use-ef-core-interceptors"><strong>EF Core interceptor</strong></a>.</p>
<h2>Processing the Outbox</h2>
<p>The Outbox processor is the next component we'll need.
This could be a <em>physically</em> separate process or a background worker in the same process.</p>
<p>I'll use Quartz to <a href="scheduling-background-jobs-with-quartz-net"><strong>schedule background jobs</strong></a> for Outbox processing.
It's a robust library with excellent support for scheduling recurring jobs.</p>
<p>Now, let's implement the <code>OutboxProcessorJob</code>:</p>
<pre><code class="language-csharp">[DisallowConcurrentExecution]
public class OutboxProcessorJob(
    NpgsqlDataSource dataSource,
    IPublishEndpoint publishEndpoint,
    Assembly integrationEventsAssembly) : IJob
{
    public async Task Execute(IJobExecutionContext context)
    {
        await using var connection = await dataSource.OpenConnectionAsync();
        await using var transaction = await connection.BeginTransactionAsync();

        // You can make the limit a parameter, to control the batch size.
        // We can also select just the id, type, and content columns.
        var messages = await connection.QueryAsync&lt;OutboxMessage&gt;(
            @&quot;&quot;&quot;
            SELECT id AS Id, type AS Type, content AS Content
            FROM outbox_messages
            WHERE processed_on_utc IS NULL
            ORDER BY occurred_on_utc LIMIT 100
            &quot;&quot;&quot;,
            transaction: transaction);

        foreach (var message in messages)
        {
            try
            {
                var messageType = integrationEventsAssembly.GetType(message.Type);
                var deserializedMessage = JsonSerializer.Deserialize(message.Content, messageType);

                // We should introduce retries here to improve reliability.
                await publishEndpoint.Publish(deserializedMessage);

                await connection.ExecuteAsync(
                    @&quot;&quot;&quot;
                    UPDATE outbox_messages
                    SET processed_on_utc = @ProcessedOnUtc
                    WHERE id = @Id
                    &quot;&quot;&quot;,
                    new { ProcessedOnUtc = DateTime.UtcNow, message.Id },
                    transaction: transaction);
            }
            catch (Exception ex)
            {
                // We can also introduce error logging here.

                await connection.ExecuteAsync(
                    @&quot;&quot;&quot;
                    UPDATE outbox_messages
                    SET processed_on_utc = @ProcessedOnUtc, error = @Error
                    WHERE id = @Id
                    &quot;&quot;&quot;,
                    new { ProcessedOnUtc = DateTime.UtcNow, Error = ex.ToString(), message.Id },
                    transaction: transaction);
            }
        }

        await transaction.CommitAsync();
    }
}
</code></pre>
<p>This approach uses polling to periodically fetch unprocessed messages from the database.
Polling can increase the load on the database, as we'll need to query for unprocessed messages frequently.</p>
<p>An alternative way to process Outbox messages is by using <a href="https://microservices.io/patterns/data/transaction-log-tailing.html">Transaction log tailing</a>.
We can implement this using <a href="https://www.npgsql.org/doc/replication.html">Postgres logical replication</a>.
The database will stream changes from the Write-Ahead Log (WAL) to our application, and we'll process these messages and publish them to the message broker.
You can use this to implement a push-based Outbox processor.</p>
<h2>Considerations and Tradeoffs</h2>
<p>The Outbox pattern, while effective, introduces additional complexity and database writes.
In high-throughput systems, it's crucial to monitor its performance to ensure it doesn't become a bottleneck.</p>
<p>I recommend implementing retry mechanisms in the Outbox processor to <a href="building-resilient-cloud-applications-with-dotnet"><strong>improve reliability</strong></a>.
Consider using exponential backoff for transient failures and a circuit breaker for persistent issues to prevent system overload during outages.</p>
<p>It's essential that you implement <a href="idempotent-consumer-handling-duplicate-messages"><strong>idempotent message consumers</strong></a>.
Network issues or processor restarts can lead to multiple deliveries of the same message, so your consumers must handle repeated processing safely.</p>
<p>Over time, the Outbox table can grow significantly, potentially impacting database performance.
It's important to implement an archiving strategy early on.
Consider moving processed messages to cold storage or deleting them after a set period.</p>
<h2>Scaling Outbox Processing</h2>
<p>As your system grows, you may find that a single Outbox processor can't keep up with the volume of messages.
This can lead to increased latency between when an event occurs and when it's processed by consumers.</p>
<p>One straightforward approach is to increase the frequency of the Outbox processor job.
You should consider running it every few seconds.
This can significantly reduce the delay in message processing.</p>
<p>Another effective strategy is to increase the batch size when fetching unprocessed messages.
By processing more messages in each run, you can improve throughput.
However, be cautious not to make the batches so large that they cause long-running transactions.</p>
<p>For high-volume systems, processing the Outbox in parallel can be very effective.
Implement a locking mechanism to claim batches of messages, allowing multiple processors to work simultaneously without conflict.
You can use <code>SELECT ... FOR UPDATE SKIP LOCKED</code> to claim a batch of messages.
This approach can dramatically increase your processing capacity.</p>
<p>Here's an in-depth article on <a href="scaling-the-outbox-pattern"><strong>scaling the outbox pattern</strong></a>
that covers these strategies in more detail.
I'll show you how we can reach more than 30,000 messages per second (2B+ messages per day).</p>
<h2>Wrapping Up</h2>
<p>The Outbox pattern is a powerful tool for maintaining data consistency in distributed systems.
By decoupling database operations from message publishing, the Outbox pattern ensures that your system remains reliable even in the face of failures.</p>
<p>Remember to keep your consumers idempotent, implement proper scaling strategies, and manage your Outbox table growth.</p>
<p>While it adds some complexity, the benefits of guaranteed message delivery make it a valuable pattern in many scenarios.</p>
<p>If you're looking to implement the Outbox pattern in a robust, production-ready way, you can check out <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture</strong></a>.
It includes an entire section on implementing the Outbox pattern, along with other essential patterns for building maintainable and scalable .NET applications.</p>
<p>That's all for today. Stay awesome, and I'll see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_110.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Breaking It Down: How to Migrate Your Modular Monolith to Microservices]]></title>
            <link>https://milanjovanovic.tech/blog/breaking-it-down-how-to-migrate-your-modular-monolith-to-microservices</link>
            <guid isPermaLink="false">breaking-it-down-how-to-migrate-your-modular-monolith-to-microservices</guid>
            <pubDate>Sat, 28 Sep 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Transitioning from a modular monolith to microservices can significantly enhance your system's scalability and your team's productivity, but it's a journey that requires careful planning and execution. In this practical guide, I'll walk you through the key steps of this migration process, from preparing your monolith and selecting the right module to extract, to implementing effective inter-service communication and data migration strategies, all based on real-world experience with .NET applications.]]></description>
            <content:encoded><![CDATA[<p>As your application grows, you might find yourself considering a move from a modular monolith to microservices.
This transition isn't just a technical shift.
It's a strategic move that can reshape how your entire system operates.
But let's be clear: it's not a magic solution and comes with its own challenges.</p>
<p>In this article, I'll share my experience of migrating from a modular monolith to microservices.</p>
<p>We'll explore why you might consider microservices, how to prepare for the migration and the critical steps in the migration process -
from choosing your first module to implementing inter-service communication and managing data migration.</p>
<p>Let's dive in!</p>
<h2>Why Consider Microservices?</h2>
<p>A <a href="what-is-a-modular-monolith"><strong>modular monolith</strong></a> serves as an excellent starting point for many applications.
It's simpler to develop, easier to understand, and faster to deploy than a distributed system.
But as your application grows, you might start facing some problems.</p>
<p>Here are the key challenges I've encountered with modular monoliths:</p>
<ul>
<li>High-load modules can become bottlenecks, affecting the entire system's performance</li>
<li>As the monolith grows, deployments become riskier and more time-consuming</li>
<li>Large teams working on a single codebase often step on each other's toes</li>
<li>You're locked into a single technology stack for the entire application</li>
</ul>
<p>Microservices can address these issues, but it's not a decision to be taken lightly.</p>
<p><img src="/blogs/mnw_109/modular_monolith_vs_microservices.png" alt="Modular monolith to microservices comparison."></p>
<p>Here's how microservices can help:</p>
<ul>
<li><strong>Scalability</strong>: You can scale individual services based on their specific load, optimizing resource use.</li>
<li><strong>Independent deployability</strong>: Updates to one service don't require redeploying the entire application, reducing risk and downtime.</li>
<li><strong>Team autonomy</strong>: Separate teams can own different services, leading to faster development cycles.</li>
</ul>
<p>The good news? If you've built a well-designed modular monolith, you're already halfway there.
The clear boundaries between modules in your monolith can serve as a blueprint for your microservices architecture.</p>
<p>Remember, microservices come with their own complexities in areas like data consistency and inter-service communication.
But for the right use cases, they can provide the flexibility and scalability needed for growing applications.</p>
<h2>Preparing for Migration</h2>
<p>The success of your migration largely depends on how well you prepare.
Your best starting point is a well-structured modular monolith.</p>
<p>Here are key areas to focus on:</p>
<ol>
<li>
<p><strong>Review module boundaries</strong>:
I can't stress this enough: clear module boundaries are crucial.
Each module should have a distinct responsibility and minimal dependencies on others.
If boundaries are blurry, refactor before migrating.</p>
</li>
<li>
<p><strong>Ensure proper data encapsulation</strong>:
Modules should not directly access each other's data stores.
I make sure each module owns and manages its data exclusively.
No shared tables, no direct database access across modules.
This clean separation makes it much easier to extract modules into microservices later.</p>
</li>
<li>
<p><strong>Implement clean public APIs</strong>:
Define clear contracts between modules to facilitate future separation.
Modules should communicate through well-defined APIs, not by accessing each other's internals.</p>
</li>
</ol>
<p>Here's an example of the above:</p>
<pre><code class="language-csharp">// Good: Clear module boundary
namespace BookingsModule;

public class CreateBooking
{
    private readonly IBookingRepository _bookingRepository;
    private readonly IPaymentGateway _paymentGateway; // Public API

    public CreateBooking(IBookingRepository bookingRepository, IPaymentGateway paymentGateway)
    {
        _bookingRepository = bookingRepository;
        _paymentGateway = paymentGateway;
    }

    public async Task&lt;BookingResult&gt; CreateBookingAsync(BookingRequest request)
    {
        var booking = await _bookingRepository.CreateAsync(Booking.FromRequest(request));

        // Accessing the other module's data through an abstraction
        var paymentResult = await _paymentGateway.ProcessPaymentAsync(booking.Id, booking.TotalAmount);

        return new BookingResult(booking, paymentResult);
    }
}
</code></pre>
<p>By focusing on these areas, you're not just preparing for migration - you're improving your modular monolith.
Even if you decide not to migrate, these steps will make your system more maintainable and scalable.</p>
<h2>Choosing and Extracting the First Module</h2>
<p>Selecting the right module to extract first can set the tone for your entire migration.
In my experience, it's crucial to start with a module that's self-contained and has clear boundaries.</p>
<p>When I'm evaluating modules, I look for these characteristics:</p>
<ul>
<li>Low coupling with other modules</li>
<li>High cohesion within the module</li>
<li>A distinct business function</li>
<li>Potential performance or scalability gains from separation</li>
</ul>
<p>For instance, in an e-commerce system, I might choose the product catalog module.
It typically has a clear purpose, doesn't heavily depend on other modules, and could benefit from independent scaling.</p>
<p>Once you've chosen your module, here's the extraction process I follow:</p>
<ol>
<li>Create a new project for the microservice.</li>
<li>Move the module's code to the new project.
This often reveals hidden dependencies, which is valuable information.</li>
<li>Update the dependencies, ensuring the microservice is self-contained.
This might involve copying some shared code or refactoring to remove unnecessary dependencies.</li>
<li>Set up a separate database for the microservice.
This enforces data independence.</li>
<li>Implement a data migration strategy (more on this later)</li>
</ol>
<p>Here's what this might look like in practice:</p>
<p><img src="/blogs/mnw_109/modular_monolith_extraction.png" alt="Modular monolith extraction process into microservices."></p>
<p>Remember, the goal isn't perfection on the first try.
I often iterate on this process, gradually refining the separation between the new microservice and the monolith.</p>
<p>The first extraction is a learning experience.
It'll show you how to approach subsequent modules and help you refine the overall migration strategy.</p>
<h2>Implementing Inter-Service Communication</h2>
<p>Once you've extracted a module into a microservice, you have to update the <a href="modular-monolith-communication-patterns"><strong>module's communication</strong></a>
with the rest of the system.
This typically involves transitioning from direct method calls to network-based communication.</p>
<p>Here's how I approach this transition:</p>
<ol>
<li>Replace direct method calls with HTTP API calls.
I often use libraries like <a href="refit-in-dotnet-building-robust-api-clients-in-csharp"><strong>Refit</strong></a> to simplify API interactions.
Here's a before and after example:</li>
</ol>
<pre><code class="language-csharp">// Before: Direct method call
BookingDto booking = await _bookingService.GetAsync(bookingId);

// After: HTTP API call
var response = await _httpClient.GetAsync($&quot;http://bookings-service/api/bookings/{bookingId}&quot;);

var booking = await response.Content.ReadFromJsonAsync&lt;BookingDto&gt;();
</code></pre>
<ol start="2">
<li>
<p>For asynchronous communication, consider implementing a messaging system.
While the implementation details vary, tools like <a href="using-masstransit-with-rabbitmq-and-azure-service-bus"><strong>RabbitMQ</strong></a>,
<a href="complete-guide-to-amazon-sqs-and-amazon-sns-with-masstransit"><strong>Amazon SQS and SNS</strong></a>,
or <a href="messaging-made-easy-with-azure-service-bus"><strong>Azure Service Bus</strong></a>
can significantly improve system resilience and decoupling.</p>
</li>
<li>
<p>Network communication introduces new failure modes, so I always implement <a href="building-resilient-cloud-applications-with-dotnet"><strong>resilience patterns</strong></a>.
Here's how I use Polly with resilience pipelines:</p>
</li>
</ol>
<pre><code class="language-csharp">using Polly;

var pipeline = new ResiliencePipelineBuilder&lt;HttpResponseMessage&gt;()
    .AddRetry(new RetryStrategyOptions
    {
        ShouldHandle = new PredicateBuilder().Handle&lt;ConflictException&gt;(),
        Delay = TimeSpan.FromSeconds(1),
        MaxRetryAttempts = 2,
        BackoffType = DelayBackoffType.Exponential,
        UseJitter = true
    })
    .AddTimeout(new TimeoutStrategyOptions
    {
        Timeout = TimeSpan.FromSeconds(10)
    })
    .Build();

var response = await pipeline.ExecuteAsync(
    async ct =&gt; await _httpClient.GetAsync($&quot;http://bookings-service/api/bookings/{bookingId}&quot;, ct),
    cancellationToken);
</code></pre>
<p>Transitioning to HTTP-based communication always brings new problems to solve.
Serialization, error handling, and <a href="api-versioning-in-aspnetcore"><strong>API versioning</strong></a> become crucial.
I've learned to plan for these aspects from the start to avoid headaches down the line.</p>
<h2>Implementing an API Gateway</h2>
<p>As you extract more services, managing the communication between clients and your microservices can become complex.
An API Gateway can help manage this complexity by providing a single entry point for all clients.</p>
<p>YARP (Yet Another Reverse Proxy) is an excellent tool for <a href="implementing-an-api-gateway-for-microservices-with-yarp"><strong>implementing an API Gateway</strong></a> in .NET.
Here's how I typically set it up:</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);

builder.Services.AddReverseProxy()
    .LoadFromConfig(builder.Configuration.GetSection(&quot;ReverseProxy&quot;));

var app = builder.Build();

app.MapReverseProxy();

app.Run();
</code></pre>
<p>Then, I configure the routing in <code>appsettings.json</code>:</p>
<pre><code class="language-json">{
  &quot;ReverseProxy&quot;: {
    &quot;Routes&quot;: {
      &quot;bookings-route&quot;: {
        &quot;ClusterId&quot;: &quot;bookings-cluster&quot;,
        &quot;Match&quot;: {
          &quot;Path&quot;: &quot;/api/bookings/{**catch-all}&quot;
        }
      },
      &quot;payments-route&quot;: {
        &quot;ClusterId&quot;: &quot;payments-cluster&quot;,
        &quot;Match&quot;: {
          &quot;Path&quot;: &quot;/api/payments/{**catch-all}&quot;
        }
      }
    },
    &quot;Clusters&quot;: {
      &quot;bookings-cluster&quot;: {
        &quot;Destinations&quot;: {
          &quot;destination1&quot;: {
            &quot;Address&quot;: &quot;http://bookings-service/&quot;
          }
        }
      },
      &quot;payments-cluster&quot;: {
        &quot;Destinations&quot;: {
          &quot;destination1&quot;: {
            &quot;Address&quot;: &quot;http://payments-service/&quot;
          }
        }
      }
    }
  }
}
</code></pre>
<p>This setup routes requests to the appropriate microservice based on the path.
For example, any request to <code>/api/bookings/*</code> will be routed to the <code>bookings-service</code>.</p>
<p><img src="/blogs/mnw_109/modular_monolith_extraction_api_gateway.png" alt="Modular monolith extraction process into microservices with an API gateway introduced."></p>
<p>In my projects, I've found that an API Gateway often becomes a critical point for implementing cross-cutting concerns.
As your architecture evolves, consider adding features like:</p>
<ul>
<li><a href="advanced-rate-limiting-use-cases-in-dotnet"><strong>Rate limiting</strong></a> to protect your services from overload</li>
<li><a href="caching-in-aspnetcore-improving-application-performance"><strong>Caching</strong></a> to improve response times for frequently requested data</li>
<li>Request/response transformation to adapt your internal APIs for external consumption</li>
</ul>
<p>Start simple, but be prepared to evolve your gateway as you learn more about your system's needs and usage patterns.</p>
<h2>Data Migration Strategy</h2>
<p>Data migration is often the trickiest part of moving from a modular monolith to microservices. In my experience, there are two main approaches:</p>
<ol>
<li>
<p><strong>The &quot;One and Done&quot; Approach</strong></p>
<p>For simpler systems or those that can handle some downtime, I use this method:</p>
<ul>
<li>Create a new database for the microservice</li>
<li>Copy the relevant schema and data from the monolith</li>
<li>Switch the application to use the new database</li>
</ul>
<p>It's quick and simple but requires a brief downtime.</p>
</li>
<li>
<p><strong>The Synchronization Approach</strong></p>
<p>For complex systems needing minimal disruption, I use this method:</p>
<ul>
<li>Copy the schema and initial data to the new database</li>
<li>Set up a sync mechanism (like <a href="https://en.wikipedia.org/wiki/Change_data_capture">change data capture</a>) to keep the new database updated</li>
<li>Gradually shift traffic to the new microservice</li>
<li>Eventually, remove the old schema from the monolith</li>
</ul>
<p>This approach is more complex but allows for a smoother transition.</p>
</li>
</ol>
<p>In both cases, I always ensure I have a solid backup and rollback plan.
The key is to choose the method that best fits your system's needs and tolerance for complexity.</p>
<p>Remember, if your modular monolith already uses <a href="modular-monolith-data-isolation"><strong>separate database schemas for each module</strong></a>, you're starting with an advantage.
This logical isolation makes the migration process significantly easier.</p>
<h2>Summary</h2>
<p>Migrating from a modular monolith to microservices is a journey I've taken several times, and it's always both challenging and rewarding.
The process we've covered here - from preparing your monolith and choosing the right module to extract
to implementing proper communication patterns - forms the foundation of a successful migration.</p>
<p>Don't rush the process.
Careful planning and incremental changes are key.
Each step teaches you something valuable about your system.</p>
<p>While we've covered the fundamentals, there's always more to learn. I recommend exploring these advanced topics:</p>
<ul>
<li><a href="service-discovery-in-microservices-with-net-and-consul"><strong>Service discovery</strong></a> and API gateway patterns</li>
<li><a href="introduction-to-distributed-tracing-with-opentelemetry-in-dotnet"><strong>Monitoring and observability</strong></a> in a microservices environment</li>
<li>Advanced <a href="how-to-build-ci-cd-pipeline-with-github-actions-and-dotnet"><strong>deployment strategies</strong></a>, like blue-green deployment</li>
</ul>
<p>If you're ready to master this process and gain hands-on experience, I've put together a comprehensive
<a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a> course.
There's an entire chapter dedicated to extracting modules and moving to microservices.
You'll gain practical skills to confidently navigate your microservices transformation.</p>
<p>Remember, the goal isn't to blindly adopt microservices but to evolve your architecture to best serve your business needs.
Sometimes, a well-structured modular monolith is the right solution.
Other times, a full microservices architecture is the way to go.
The key is understanding the trade-offs and making informed decisions.</p>
<p>Good luck out there, and I'll see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_109.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How I Implemented Full-Text Search On My Website]]></title>
            <link>https://milanjovanovic.tech/blog/how-i-implemented-full-text-search-on-my-website</link>
            <guid isPermaLink="false">how-i-implemented-full-text-search-on-my-website</guid>
            <pubDate>Sat, 21 Sep 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[This article walks you through how I used Lunr.js to add powerful search capabilities to a Next.js static site, dealing with challenges like client-side processing and optimizing for performance along the way.]]></description>
            <content:encoded><![CDATA[<p>So, I've got this blog I've been running for a couple of years now.
It's a Next.js and TypeScript static site, all optimized for SEO.
Sounds pretty standard, right?
Well, it was, until I ran into a problem.</p>
<p>As my content grew, finding specific articles became a real pain.
It wasn't just me - readers were struggling too.
What's the point of writing all this stuff if no one can find it when they need it?</p>
<p>That's when I decided to look into full-text search.
Now, implementing full-text search on a static site isn't exactly straightforward.
It's not like you can just slap a database on there and call it a day.
You've got to get creative.</p>
<p>In this article, I'll walk you through how I turned my site's search from useless to actually functional.</p>
<h2>The Old Search Approach</h2>
<p>Here's how the search function worked before:</p>
<ol>
<li>I would generate a JSON document at build time containing each blog post's metadata.
This document contains the data needed to perform the search and render the results.</li>
<li>When searching the articles, I would fetch the JSON document on the client and perform the search.
Since this document changes only once a week, I can cache it on the client to improve performance.</li>
</ol>
<p>Here's the code that would generate the search data JSON document:</p>
<pre><code class="language-ts">export const generateSearchData = () =&gt; {
  const posts = getAllPosts();

  const searchData = posts.map((post) =&gt; ({
    slug: post.meta.slug,
    title: post.meta.title,
    excerpt: post.meta.excerpt,
    coverImage: post.meta.coverImage,
    date: post.meta.date,
    searchableContent:
      `${post.meta.title} ${post.meta.excerpt} ${post.content}`.toLowerCase()
  }));

  fs.mkdirSync('./search', { recursive: true });
  fs.writeFileSync(
    path.join(process.cwd(), 'search', 'search-data.json'),
    JSON.stringify(searchData)
  );
};
</code></pre>
<p>And then the search function is pretty simple (but not very smart):</p>
<pre><code class="language-typescript">export const search(keyword) = () =&gt; {
  const res = await fetch('/search/search-data.json');

  const searchData = await res.json();

  return searchData.filter((p) =&gt;
    p.searchableContent.includes(keyword.toLowerCase())
  );
}
</code></pre>
<p>This approach has a few issues:</p>
<ul>
<li>The search data JSON document is large (861KB) and will continue to grow</li>
<li>It's slow on large datasets and won't scale well</li>
<li>It uses an exact match to find results</li>
<li>It doesn't rank results by relevance</li>
</ul>
<p>I wasn't happy with this, so here's how I implemented a much better solution.</p>
<h2>Introducing Full-Text Search</h2>
<p>To fix these problems, I turned to <strong>full-text search</strong> using <a href="https://lunrjs.com/">Lunr.js</a>.
But what exactly is full-text search?</p>
<p>Full-text search is a technique that allows fast and efficient searching of large volumes of text by creating an index of all words in a document collection
and returning ranked results based on relevance to the search query.</p>
<p>Full-text search works by:</p>
<ol>
<li><strong>Indexing</strong>: Breaking down text into individual words (tokens)</li>
<li><strong>Stemming</strong>: Reducing words to their base form (e.g., &quot;running&quot; to &quot;run&quot;)</li>
<li><strong>Ranking</strong>: Scoring results based on relevance</li>
</ol>
<p>Another common operation is stop word removal.
Common words that don't add much meaning (like &quot;the&quot;, &quot;and&quot;, &quot;is&quot;) are often removed to save space and improve relevance.</p>
<p>At the heart of full-text search is an <a href="https://en.wikipedia.org/wiki/Inverted_index">inverted index</a>.
It's a data structure that maps each unique term to the documents containing it.
It's &quot;inverted&quot; because it goes from terms to documents, rather than documents to terms.</p>
<p>When searching, results are often ranked using <a href="https://en.wikipedia.org/wiki/Tf-idf">TF-IDF</a> (Term Frequency-Inverse Document Frequency).
This gives higher scores to terms that are frequent in a document but rare across all documents.</p>
<p>That's the rundown on full-text search. Let's see how to implement it.</p>
<h2>Implementing Full-Text Search</h2>
<p>I chose Lunr.js because it's lightweight and works great for static websites. No server required.</p>
<p>I updated the <code>generateSearchData</code> function to create and store a full-text search index.
This function runs once at build time, so it's not expensive.</p>
<pre><code class="language-typescript">export const generateSearchData = () =&gt; {
  const posts = getAllPosts();

  // Generate search data as before.

  const index = lunr(function () {
    this.ref('slug');
    this.field('title', { boost: 10 });
    this.field('content', { boost: 5 });

    searchData.forEach((doc) =&gt; {
      this.add(doc);
    });
  });

  // Store search data as before.

  // And store the search index.
  fs.writeFileSync(
    path.join(process.cwd(), 'search', 'search-index.json'),
    JSON.stringify(index)
  );
};
</code></pre>
<p>I'm <strong>boosting</strong> the title with a factor of 10 and the content with a factor of 5.
If the search term matches the title, it will have a higher relevance score.</p>
<p>The downside (for now) is that the index file is big, 2.5MB.
If I were to download this on the client every time, the costs would quickly add up.
I have 100,000 unique visitors per month, which would equate to <code>250GB</code> of network egress.</p>
<p>Now, I have to update the search function to use the full-text index:</p>
<pre><code class="language-typescript">export const search(keyword) = () =&gt; {
  const res = await fetch('/search/search-data.json');
  const searchData = await res.json();

  const res = await fetch('/search/search-index.json');
  const searchIndex = lunr.Index.load(indexJson);

  return searchIndex
    .search(keyword)
    .map((result) =&gt; {
      const post = searchData?.find((post) =&gt; post.slug === result.ref);
      return post ? { ...post, score: result.score } : null;
    })
    .filter((result) =&gt; result !== null)
    .sort((a, b) =&gt; b.score - a.score);
}
</code></pre>
<p>What's happening here:</p>
<ul>
<li>We're loading the search index from a JSON document using <code>lunr.Index.load</code></li>
<li>The index has a <code>search</code> function allowing us to perform full-text search</li>
<li>We're expanding the result to also include the relevance score</li>
</ul>
<p>This allows me to sort the search results based on relevance instead of chronologically.</p>
<p>Here's what the old search implementation returns when searching for &quot;resilience&quot;:</p>
<div className="bordered centered">
  ![The web page showing search results for the keyword ](/blogs/mnw_108/old_search.png)
</div>
<p>The most relevant article is at the fourth spot, sorted chronologically.
You can see how this isn't a good user experience.</p>
<p>But with full-text search, when you search for a keyword like &quot;resilience&quot;, you get nicer results:</p>
<div className="bordered centered">
  ![The web page showing full-text search results for the keyword ](/blogs/mnw_108/full_text_search.png)
</div>
<p>I also added a relevance score next to each article because it looks cool.</p>
<h2>Optimizing The Search</h2>
<p>Now, I have a powerful search, but the index was huge (a whopping 2.5MB) and will continue to grow.
The search data file is also 861KB.</p>
<p>Enter <a href="https://en.wikipedia.org/wiki/Brotli">Brotli</a> compression:</p>
<ul>
<li>I compressed the search index and data using Brotli on the server side</li>
<li>In the browser, I decompress the files before using them to perform the search</li>
</ul>
<pre><code class="language-typescript">import { compress } from 'brotli';

export const generateSearchData = () =&gt; {
  // Create the search data and index

  fs.writeFileSync(
    path.join(process.cwd(), 'search', 'search-index.br'),
    compress(Buffer.from(JSON.stringify(index)))
  );
  fs.writeFileSync(
    path.join(process.cwd(), 'search', 'search-data.br'),
    compress(Buffer.from(JSON.stringify(searchData)))
  );
};
</code></pre>
<p>The results:</p>
<ul>
<li>2.5MB → 193KB <strong>(-92%)</strong></li>
<li>861KB → 180KB <strong>(-79%)</strong></li>
</ul>
<p>If you want to learn more about compression algorithms, check out this <a href="response-compression-in-aspnetcore"><strong>article about response compression</strong></a>.</p>
<p>After fetching the compressed search data and full-text index on the client, I cache them for subsequent searches.</p>
<p>The searches are lightning-fast now, and the results are more relevant.</p>
<h2>Server-Side Options for Full-Text Search</h2>
<p>While Lunr.js works great for my static site, larger apps with more data need a robust full-text search solution.
So, here are some some server-side options you could explore.</p>
<p><a href="https://lucene.apache.org/">Lucene</a> is the foundation of many search engines.
It's written in Java but has ports to other languages.
Lucene provides robust full-text indexing and search capabilities.
It is also highly efficient and customizable, making it a popular choice for developers who need fine-grained control over their search functionality.</p>
<p><a href="https://solr.apache.org/">Apache Solr</a> builds on top of Lucene, offering additional features like distributed indexing, replication, and load-balanced querying.
Solr includes powerful capabilities such as faceting and highlighting, which can greatly enhance the search experience.</p>
<p>If you're already using <a href="https://www.postgresql.org/">PostgreSQL</a>, the built-in <a href="https://www.postgresql.org/docs/current/textsearch.html">full-text search</a> is worth considering.
PostgreSQL uses its own text search engine, which supports multiple languages and custom dictionaries.
While not as feature-rich as dedicated search engines, it can be a convenient option for applications that want to keep their stack simple.</p>
<p>I've been tinkering with PostgreSQL <a href="https://www.npgsql.org/efcore/mapping/full-text-search.html">full-text search using EF Core</a>.
Here's what a full-text search query looks like:</p>
<pre><code class="language-csharp">var blogs = context
    .BlogPosts
    .Where(b =&gt;
        EF.Functions.ToTsVector(&quot;english&quot;, b.Title + &quot; &quot; + b.Excerpt + &quot; &quot; + b.Content)
            .Matches(EF.Functions.PhraseToTsQuery(&quot;english&quot;, searchTerm)))
    .Select(b =&gt; new
    {
        b.Slug,
        b.Title,
        b.Excerpt,
        b.Date,
        Rank = EF.Functions.ToTsVector(&quot;english&quot;, b.Title + &quot; &quot; + b.Excerpt + &quot; &quot; + b.Content)
            .Rank(EF.Functions.PhraseToTsQuery(&quot;english&quot;, searchTerm))
    })
    .OrderByDescending(b =&gt; b.Rank)
    .ToList();
</code></pre>
<p>But I'll have to write a dedicated article to cover this, so let's wrap it up.</p>
<h2>Wrapping Up</h2>
<p>The old search functionality was, frankly, embarrassing.</p>
<p>It was slow, dumb as a rock, and about as useful as a chocolate teapot.</p>
<p>Now? It's actually not half bad.</p>
<ul>
<li>Searches are lightning-fast</li>
<li>Results are more relevant</li>
<li>The user experience is much smoother</li>
</ul>
<p>I'm not gonna lie, there was a moment of frustration when I thought about giving up.
But I'm glad I stuck with it.
There's something satisfying about building it yourself, you know?</p>
<p>It might not be pretty, but it works.
And sometimes, that's all that matters.</p>
<p>The Brotli compression bit was a pleasant surprise.
I thought I'd end up with a massive index file that would make mobile users cry.
But nope, it all works smoothly.
Go figure.</p>
<p>Have you tried implementing search on your site? How'd it go?</p>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_108.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[.NET Aspire: A Game-Changer for Cloud-Native Development?]]></title>
            <link>https://milanjovanovic.tech/blog/dotnet-aspire-a-game-changer-for-cloud-native-development</link>
            <guid isPermaLink="false">dotnet-aspire-a-game-changer-for-cloud-native-development</guid>
            <pubDate>Sat, 14 Sep 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[.NET Aspire promises to revolutionize cloud-native development for .NET applications, but does it live up to the hype? In this hands-on review, I'll share my experience migrating a real project to Aspire, exploring its strengths, limitations, and potential impact on how we build distributed systems.]]></description>
            <content:encoded><![CDATA[<p>I've been tinkering with .NET Aspire lately, and I've got some thoughts to share.
If you're curious about this new cloud-native development tool from Microsoft, stick around.
I'll break down what's great, what's not, and how you can start using it.</p>
<blockquote>
<p>.NET Aspire is an opinionated, cloud-ready stack for building observable, production-ready, distributed applications.</p>
</blockquote>
<p><a href="https://learn.microsoft.com/en-us/dotnet/aspire/get-started/aspire-overview"><strong>.NET Aspire</strong></a> is Microsoft's latest offering for cloud-native application development.
It aims to simplify the process of building, deploying, and managing distributed applications.</p>
<p>Distributed applications often consist of small applications that consume external services like databases, message brokers, and caching.
.NET Aspire gives you a set of tools to make building distributed applications easier.</p>
<h2>.NET Aspire Orchestration</h2>
<p>How are you setting up a local development environment?
I often use <strong>Docker Compose</strong> to configure my applications and run external services.
It's a simple setup, but you need to manage environment variables and connection strings.
If you're not familiar with Docker, it can prove to be quite tricky sometimes.</p>
<p>Here's a <code>docker-compose.yml</code> file from a recent project:</p>
<pre><code class="language-yml">services:
  contentplatform-api:
    image: ${DOCKER_REGISTRY-}contentplatform-api
    container_name: ContentPlatform.Api
    build:
      context: .
      dockerfile: ContentPlatform.Api/Dockerfile
    ports:
      - 5000:8080
      - 5001:8081

  contentplatform-reporting-api:
    image: ${DOCKER_REGISTRY-}contentplatform-reporting-api
    container_name: ContentPlatform.Reporting.Api
    build:
      context: .
      dockerfile: ContentPlatform.Reporting.Api/Dockerfile
    ports:
      - 6000:8080
      - 6001:8081

  contentplatform-presentation:
    image: contentplatform-ui:latest
    container_name: ContentPlatform.Presentation
    environment:
      - ASPNETCORE_ENVIRONMENT=Development
    ports:
      - 3000:80

  contentplatform-db:
    image: postgres:latest
    container_name: ContentPlatform.Db
    environment:
      - POSTGRES_DB=contentplatform
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
    volumes:
      - ./.containers/db:/var/lib/postgresql/data
    ports:
      - 5432:5432

  contentplatform-mq:
    image: rabbitmq:management
    container_name: ContentPlatform.RabbitMq
    hostname: contentplatform-mq
    volumes:
      - ./.containers/queue/data/:/var/lib/rabbitmq
      - ./.containers/queue/log/:/var/log/rabbitmq
    environment:
      RABBITMQ_DEFAULT_USER: guest
      RABBITMQ_DEFAULT_PASS: guest
</code></pre>
<p>This sets up two APIs, a client application, PostgreSQL, and RabbitMQ.
I also have to configure the connection strings manually to connect to these services.</p>
<p>So, I decided to migrate this application to .NET Aspire and documented the process.</p>
<p>You can right-click an existing project in Visual Studio and select <code>Add &gt; .NET Aspire Orchestrator Support...</code>.</p>
<figure className="figure-center">
  ![Context menu with ](/blogs/mnw_107/aspire_orchestration.png)
  <figcaption>
    Source:
    [Microsoft](https://learn.microsoft.com/en-us/dotnet/aspire/get-started/add-aspire-existing-app)
  </figcaption>
</figure>
<p>This will add an <code>AppHost</code> and <code>ServiceDefaults</code> project to your solution.
You will then repeat this for the remaining projects in your solution to enlist them all in Aspire orchestration.</p>
<p>The <code>AppHost</code> project is responsible for orchestration.
You can define your entire application stack in a single, readable file.
Running the <code>AppHost</code> project from Visual Studio will start the required applications and services.</p>
<p>Here's the setup for my application using Aspire:</p>
<pre><code class="language-csharp">IDistributedApplicationBuilder builder = DistributedApplication.CreateBuilder(args);

var postgres = builder.AddPostgres(&quot;contentplatform-db&quot;)
    .WithPgAdmin();

var rabbitMq = builder.AddRabbitMQ(&quot;contentplatform-mq&quot;)
    .WithManagementPlugin();

builder.AddProject&lt;Projects.ContentPlatform_Api&gt;(&quot;contentplatform-api&quot;)
    .WithReference(postgres)
    .WithReference(rabbitMq);

builder.AddProject&lt;Projects.ContentPlatform_Reporting_Api&gt;(&quot;contentplatform-reporting-api&quot;)
    .WithReference(postgres)
    .WithReference(rabbitMq);

builder.AddProject&lt;Projects.ContentPlatform_Presentation&gt;(&quot;contentplatform-presentation&quot;);

builder.Build().Run();
</code></pre>
<p>The Aspire version is much more concise and readable.
Adding new services or changing configurations is straightforward.
You also get built-in observability.
Aspire includes tools for logging, metrics, and distributed tracing out of the box, making it easier to monitor and debug your applications.</p>
<p>When you run the application, you can see your applications and services on the Aspire dashboard:</p>
<p><img src="/blogs/mnw_107/content_platfrom_resources.png" alt="Aspire dashboard resource views showing the application services and containers running."></p>
<h3>Orchestration - The Bad Parts</h3>
<p>There are a few things I don't like with the current Aspire setup.</p>
<p>The <code>AppHost</code> project needs to reference all other projects to enlist them in orchestration.
If your services are all in one solution, this might be fine.
But what about large microservices systems?</p>
<p>We can go around this limitation by building a Docker image for an external service.
There's an <code>AddContainer</code> method that allows us to configure container resources.
However, we won't be able to debug these services.</p>
<p>The <code>ServiceDefaults</code> projects needs to be visible to all other applications.
Again, this works perfectly fine if everything is in one solution.
We can also distribute this project as a NuGet package for complex systems.</p>
<h2>.NET Aspire Integrations</h2>
<p>If you're wondering how I configured PostgreSQL and RabbitMQ in the previous example, this is made available using Aspire Integrations.
These are NuGet packages that allow you to integrate with popular services, such as Redis or PostgreSQL.
Aspire integrations take care of many cloud-native concerns for you, like adding health checks and telemetry.</p>
<p>You can right-click on the <code>AppHost</code> project and select <code>Add &gt; .NET Aspire package...</code> to see the list of available integrations:</p>
<p><img src="/blogs/mnw_107/aspire_integrations.png" alt="NuGet browser view showing a list of .NET Aspire integrations."></p>
<p>If we want to add Redis to our project, we can install the <code>Aspire.Hosting.Redis</code> package.
Then, we would configure the Redis integration in the <code>AppHost</code> project:</p>
<pre><code class="language-csharp">var builder = DistributedApplication.CreateBuilder(args);

// Other service omitted for brevity

var redis = builder.AddRedis(&quot;contentplatform-cache&quot;);

builder.AddProject&lt;Projects.ContentPlatform_Api&gt;(&quot;contentplatform-api&quot;)
    .WithReference(postgres)
    .WithReference(rabbitMq)
    .WithReference(redis);

builder.Build().Run();
</code></pre>
<p>You can find a list of supported <a href="https://learn.microsoft.com/en-us/dotnet/aspire/fundamentals/integrations-overview">Aspire integrations</a> in the documentation.</p>
<p>When you configure a resource with a known connection string format, Aspire automatically injects an environment variable.
The connection string name will have the same name as the respective resource.</p>
<ul>
<li><code>WithReference(postgres)</code> produces <code>ConnectionStrings__contentplatform-db=&quot;&lt;VALUE&gt;&quot;</code></li>
<li><code>WithReference(rabbitMq)</code> produces <code>ConnectionStrings__contentplatform-mq=&quot;&lt;VALUE&gt;&quot;</code></li>
<li><code>WithReference(redis)</code> produces <code>ConnectionStrings__contentplatform-cache=&quot;&lt;VALUE&gt;&quot;</code></li>
</ul>
<p>This lets you use logical connection string names when configuring your services:</p>
<pre><code class="language-csharp">builder.Services.AddDbContext&lt;ApplicationDbContext&gt;(o =&gt;
    o.UseNpgsql(builder.Configuration.GetConnectionString(&quot;contentplatform-db&quot;)));
</code></pre>
<h2>Service Defaults and OpenTelemetry</h2>
<p>One of Aspire's killer features is its built-in observability stack.
It integrates <a href="introduction-to-distributed-tracing-with-opentelemetry-in-dotnet"><strong>OpenTelemetry</strong></a>, providing distributed tracing, metrics, and logging out of the box.</p>
<p>When you enlist a project in .NET Aspire orchestration, there are some updates made to the <code>Program</code> file automatically:</p>
<ul>
<li><code>AddServiceDefaults</code> is called to configure OpenTelemetry, health checks, and service discovery</li>
<li><code>MapDefaultEndpoints</code> is called to expose the health check endpoint</li>
</ul>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);

builder.AddServiceDefaults();

// Other code omitted for brevity

var app = builder.Build();

app.MapDefaultEndpoints();

// Other code omitted for brevity

app.Run();
</code></pre>
<p>You can customize <code>AddServiceDefaults</code> according to your requirements.
For example, if you're using MassTransit, you can add the respective tracing configuration for this library.</p>
<p>Here's the distributed traces view on the Aspire dashboard.
You can see a <code>POST</code> request hitting the <code>contentplatform-api</code> service, publishing an <code>ArticleCreatedEvent</code>, and consuming that message in the <code>contentplatform-reporting-api</code> service.</p>
<p><img src="/blogs/mnw_107/content_platfrom_traces.png" alt="Aspire dashboard traces views showing one distributed trace."></p>
<p>For local development, the .NET Aspire dashboard provides a UI for viewing telemetry data.
In a production environment, you can configure the OpenTelemetry server to receive telemetry data using the <code>OTEL_EXPORTER_OTLP_ENDPOINT</code> environment variable.</p>
<h2>Deploying .NET Aspire Applications</h2>
<p>.NET Aspire simplifies the deployment process for distributed applications, especially when targeting Azure.
To deploy an Aspire application, you first generate a <strong>manifest file</strong> using the <code>dotnet run</code> command with specific parameters.
This manifest is a JSON file that describes all the resources defined in your Aspire project, including services, databases, and other dependencies.</p>
<figure>
  ![.NET Aspire manifest JSON file example.](/blogs/mnw_107/aspire_manifest.png)
  <figcaption>
    Source:
    [Microsoft](https://learn.microsoft.com/en-us/dotnet/aspire/deployment/manifest-format)
  </figcaption>
</figure>
<p>Deployment tools can use the manifest to set up the necessary infrastructure in your target environment.
Aspire generates the required configuration for Azure Container Apps or Kubernetes for Azure deployments.
It handles tasks like setting up networking, scaling services, and configuring monitoring automatically.</p>
<p>Here's a simple example of generating a manifest:</p>
<pre><code>dotnet run --project ContentPlatform.AppHost\ContentPlatform.AppHost.csproj `
    -- --publisher manifest --output-path ../aspire-manifest.json
</code></pre>
<p>This command creates a JSON manifest file that deployment tools can use to set up your application in the cloud or on-premises infrastructure.</p>
<p>You can learn more about <a href="https://learn.microsoft.com/en-us/dotnet/aspire/deployment/overview">Aspire deployment</a> in the documentation.</p>
<h2>Summary</h2>
<p>I've used .NET Aspire a lot lately, and I'm genuinely impressed.</p>
<p>Aspire makes building complex systems much easier.
I can set up a distributed system with just a few lines of C# code.
This is much simpler than using Docker Compose.
The built-in observability and monitoring tools are also great.</p>
<p>While .NET Aspire is now production-ready, the ecosystem around it is still growing.
Developers, particularly those new to cloud-native concepts, might face a learning curve.</p>
<p>Should you adopt Aspire in your .NET projects?</p>
<p>If you're building distributed applications, especially for Azure, I'd say give it a try.
However, you might want to evaluate carefully if you work on simpler applications or use non-Azure cloud services.</p>
<p>That's all for today.</p>
<p>See you next week.</p>
<p><strong>P.S.</strong> You can find the source code for this example in <a href="https://github.com/m-jovanovic/aspire-orchestration"><strong>this repository</strong></a>.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_107.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Refit in .NET: Building Robust API Clients in C#]]></title>
            <link>https://milanjovanovic.tech/blog/refit-in-dotnet-building-robust-api-clients-in-csharp</link>
            <guid isPermaLink="false">refit-in-dotnet-building-robust-api-clients-in-csharp</guid>
            <pubDate>Sat, 07 Sep 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Discover how Refit simplifies API consumption in .NET applications, turning your HTTP API into a seamless, strongly-typed interface. This comprehensive guide covers everything from basic setup to advanced features, helping you build robust, maintainable API clients in C# with minimal boilerplate code.]]></description>
            <content:encoded><![CDATA[<p>As a .NET developer, I've spent countless hours working with external APIs.
It's a crucial part of modern software development, but let's be honest - it can be a real pain sometimes.</p>
<p>We've all been there, wrestling with <code>HttpClient</code>, writing repetitive code, and hoping we didn't miss a parameter or header somewhere.</p>
<p>That's why I want to introduce you to <strong>Refit</strong>, a library that's been a game-changer for me.</p>
<p>Imagine turning your API into a live interface - sounds too good to be true, right?
But that's exactly what Refit does.
It handles all the HTTP heavy lifting, letting you focus on what matters: your application logic.</p>
<p>In this article, I'll explain how Refit can transform the way you work with APIs in your .NET projects.</p>
<h2>What is Refit?</h2>
<p><a href="https://github.com/reactiveui/refit">Refit</a> is a type-safe REST library for .NET.
It allows you to define your API as an interface, which Refit then implements for you.
This approach reduces boilerplate code and makes your API calls more readable and maintainable.</p>
<p>You describe your API endpoints using method signatures and attributes, and Refit takes care of the rest.</p>
<p>Let me break down why I find Refit so powerful:</p>
<ul>
<li><strong>Automatic serialization and deserialization</strong>:
You won't have to convert your objects to JSON and back.
Refit handles all of that for you.</li>
<li><strong>Strongly-typed API definitions</strong>:
Refit helps you catch errors early.
If you mistype a parameter or use the wrong data type, you'll know at compile time, not when your app crashes in production.</li>
<li><strong>Support for various HTTP methods</strong>:
GET, POST, PUT, PATCH, DELETE - Refit has you covered.</li>
<li><strong>Request/response manipulations</strong>:
You can add custom headers or handle specific content types in a straightforward way.</li>
</ul>
<p>But what I appreciate most about Refit is how it promotes clean, readable code.
Your API calls become self-documenting.
Anyone reading your code can quickly understand what each method does without diving into implementation details.</p>
<h2>Setting Up and Using Refit in Your Project</h2>
<p>Let's set up Refit and see it in action using the <a href="https://jsonplaceholder.typicode.com/">JSONPlaceholder API</a>.
We'll implement a full CRUD interface and demonstrate its usage in a <a href="automatically-register-minimal-apis-in-aspnetcore"><strong>Minimal API application</strong></a>.</p>
<p>First, install the required NuGet packages:</p>
<pre><code class="language-powershell">Install-Package Refit
Install-Package Refit.HttpClientFactory
</code></pre>
<p>Now, let's create our Refit interface:</p>
<pre><code class="language-csharp">using Refit;

public interface IBlogApi
{
    [Get(&quot;/posts/{id}&quot;)]
    Task&lt;Post&gt; GetPostAsync(int id);

    [Get(&quot;/posts&quot;)]
    Task&lt;List&lt;Post&gt;&gt; GetPostsAsync();

    [Post(&quot;/posts&quot;)]
    Task&lt;Post&gt; CreatePostAsync([Body] Post post);

    [Put(&quot;/posts/{id}&quot;)]
    Task&lt;Post&gt; UpdatePostAsync(int id, [Body] Post post);

    [Delete(&quot;/posts/{id}&quot;)]
    Task DeletePostAsync(int id);
}

public class Post
{
    public int Id { get; set; }
    public string Title { get; set; }
    public string Body { get; set; }
    public int UserId { get; set; }
}
</code></pre>
<p>We define our <code>IBlogApi</code> interface with methods for all CRUD operations: GET (single and list), POST, PUT, and DELETE.
The <code>Post</code> class represents the structure of our blog posts.</p>
<p>Then you have to register Refit in your dependency injection container:</p>
<pre><code class="language-csharp">using Refit;

builder.Services
    .AddRefitClient&lt;IBlogApi&gt;()
    .ConfigureHttpClient(c =&gt; c.BaseAddress = new Uri(&quot;https://jsonplaceholder.typicode.com&quot;));
</code></pre>
<p>Finally, we can use <code>IBlogApi</code> in our Minimal API endpoints:</p>
<pre><code class="language-csharp">app.MapGet(&quot;/posts/{id}&quot;, async (int id, IBlogApi api) =&gt;
    await api.GetPostAsync(id));

app.MapGet(&quot;/posts&quot;, async (IBlogApi api) =&gt;
    await api.GetPostsAsync());

app.MapPost(&quot;/posts&quot;, async ([FromBody] Post post, IBlogApi api) =&gt;
    await api.CreatePostAsync(post));

app.MapPut(&quot;/posts/{id}&quot;, async (int id, [FromBody] Post post, IBlogApi api) =&gt;
    await api.UpdatePostAsync(id, post));

app.MapDelete(&quot;/posts/{id}&quot;, async (int id, IBlogApi api) =&gt;
    await api.DeletePostAsync(id));
</code></pre>
<p>What I love about this setup is its simplicity.
We've created a fully functional API that communicates with an external service, all in just a few lines of code.
No manual HTTP requests, no raw JSON handling - Refit takes care of all that for us.</p>
<h2>Query Parameters and Route Binding</h2>
<p>When working with APIs, you often need to send data as part of the URL, either in the route or as query parameters.
Refit makes this process simple and type-safe.</p>
<p>Let's extend our <code>IBlogApi</code> interface with some more complex scenarios:</p>
<pre><code class="language-csharp">public interface IBlogApi
{
    // Other methods omitted for brevity

    [Get(&quot;/posts&quot;)]
    Task&lt;List&lt;Post&gt;&gt; GetPostsAsync([Query] PostQueryParameters parameters);

    [Get(&quot;/users/{userId}/posts&quot;)]
    Task&lt;List&lt;Post&gt;&gt; GetUserPostsAsync(int userId);
}

public class PostQueryParameters
{
    public int? UserId { get; set; }
    public string? Title { get; set; }
}
</code></pre>
<p>Let's break this down:</p>
<ul>
<li><code>GetPostsAsync</code> uses an object to represent query parameters.
This approach is excellent for endpoints with many optional parameters.
Refit will automatically convert this object into a query string.</li>
<li><code>GetUserPostsAsync</code> demonstrates passing in route parameters (<code>userId</code>) directly.</li>
</ul>
<p>Using an object for query parameters makes your code type-safe and <a href="5-awesome-csharp-refactoring-tips"><strong>refactoring-friendly</strong></a>.
If you need to add a new query parameter, you just add a property to <code>PostQueryParameters</code>.
Your existing code won't break, and your IDE can help you discover the new options.</p>
<h2>Dynamic Headers and Authentication</h2>
<p>Another common requirement when integrating with APIs is including custom headers or authentication tokens with your requests.
Refit provides several ways to handle this, from simple static headers to dynamic, request-specific authentication.</p>
<p>Let's explore some scenarios:</p>
<pre><code class="language-csharp">public interface IBlogApi
{
    [Headers(&quot;User-Agent: MyAwesomeApp/1.0&quot;)]
    [Get(&quot;/posts&quot;)]
    Task&lt;List&lt;Post&gt;&gt; GetPostsAsync();

    [Get(&quot;/secure-posts&quot;)]
    Task&lt;List&lt;Post&gt;&gt; GetSecurePostsAsync([Header(&quot;Authorization&quot;)] string bearerToken);

    [Get(&quot;/user-posts&quot;)]
    Task&lt;List&lt;Post&gt;&gt; GetUserPostsAsync([Authorize(scheme: &quot;Bearer&quot;)] string token);
}
</code></pre>
<ul>
<li>You can add a static header to all requests by using the <code>Headers</code> attribute</li>
<li>With the <code>Header</code> attribute, you can pass a header value dynamically as a parameter</li>
<li>The <code>Authorize</code> attribute is a convenient way to add Bearer token authentication</li>
</ul>
<p>But what if you need to add the same dynamic header to all requests?</p>
<p>That's where <code>DelegatingHandler</code> comes in handy.</p>
<p>You can learn more about <a href="extending-httpclient-with-delegating-handlers-in-aspnetcore"><strong>using delegating handlers in this article</strong></a>.</p>
<p>I've found delegating handlers especially helpful in providing API keys, which are typically static.</p>
<h2>JSON Serialization Options</h2>
<p>Refit gives you flexibility when choosing and configuring your JSON serializer.
By default, Refit uses <code>System.Text.Json</code>, is the built-in JSON serializer in modern .NET versions.</p>
<p>However, you can easily switch to <code>Newtonsoft.Json</code> if you need its features.</p>
<p>Here's how you can configure Refit to use it.</p>
<p>First, install the Newtonsoft.Json support package:</p>
<pre><code class="language-powershell">Install-Package Refit.Newtonsoft.Json
</code></pre>
<p>Then, configure Refit to use Newtonsoft.Json:</p>
<pre><code class="language-csharp">using Newtonsoft.Json;
using Newtonsoft.Json.Serialization;
using Refit;

builder.Services.AddRefitClient&lt;IBlogApi&gt;(new RefitSettings
{
    ContentSerializer = new NewtonsoftJsonContentSerializer(new JsonSerializerSettings
    {
        ContractResolver = new CamelCasePropertyNamesContractResolver(),
        NullValueHandling = NullValueHandling.Ignore
    })
})
.ConfigureHttpClient(c =&gt; c.BaseAddress = new Uri(&quot;https://jsonplaceholder.typicode.com&quot;));
</code></pre>
<p>This setup uses camel case for property names and ignores null values when serializing.</p>
<p><code>System.Text.Json</code> is faster and uses less memory, making it a great default choice.
However, <code>Newtonsoft.Json</code> offers more features and might be necessary for compatibility with older systems or specific serialization needs.</p>
<h2>Handling HTTP Responses</h2>
<p>While Refit's default behavior of automatically deserializing responses into your defined types is convenient, there are times when you need more control over the HTTP response.</p>
<p>Refit provides two options for these scenarios:
<a href="https://learn.microsoft.com/en-us/dotnet/api/system.net.http.httpresponsemessage"><code>HttpResponseMessage</code></a> and <code>ApiResponse&lt;T&gt;</code>.</p>
<p>Let's update the <code>IBlogApi</code> to use these types:</p>
<pre><code class="language-csharp">public interface IBlogApi
{
    [Get(&quot;/posts/{id}&quot;)]
    Task&lt;HttpResponseMessage&gt; GetPostRawAsync(int id);

    [Get(&quot;/posts/{id}&quot;)]
    Task&lt;ApiResponse&lt;Post&gt;&gt; GetPostWithMetadataAsync(int id);

    [Post(&quot;/posts&quot;)]
    Task&lt;ApiResponse&lt;Post&gt;&gt; CreatePostAsync([Body] Post post);
}
</code></pre>
<p>Now, let's break down how to use these, starting with <code>HttpResponseMessage</code>:</p>
<pre><code class="language-csharp">HttpResponseMessage response = await blogApi.GetPostRawAsync(1);

if (response.IsSuccessStatusCode)
{
    var content = await response.Content.ReadAsStringAsync();
    var post = JsonSerializer.Deserialize&lt;Post&gt;(content);
    Console.WriteLine($&quot;Retrieved post: {post.Title}&quot;);
}
else
{
    Console.WriteLine($&quot;Error: {response.StatusCode}&quot;);
}
</code></pre>
<p>This approach gives you full control over the HTTP response. You can access status codes, headers, and the raw content.
But you will have to deal with deserialization manually.</p>
<p><code>ApiResponse&lt;T&gt;</code> is a Refit-specific type that wraps the deserialized content and response metadata.
It's a great middle ground when you need the typed response and access to headers or status codes.</p>
<p>Here's a more complex example using <code>ApiResponse&lt;T&gt;</code> for creating a post:</p>
<pre><code class="language-csharp">var newPost = new Post { Title = &quot;New Post&quot;, Body = &quot;Content&quot;, UserId = 1 };

ApiResponse&lt;Post&gt; createResponse = await blogApi.CreatePostAsync(newPost);

if (createResponse.IsSuccessStatusCode)
{
    var createdPost = createResponse.Content;
    var locationHeader = createResponse.Headers.Location;
    Console.WriteLine($&quot;Created post with ID: {createdPost.Id}&quot;);
    Console.WriteLine($&quot;Location: {locationHeader}&quot;);
}
else
{
    Console.WriteLine($&quot;Error: {createResponse.Error.Content}&quot;);
    Console.WriteLine($&quot;Status: {createResponse.StatusCode}&quot;);
}
</code></pre>
<p>This approach allows you to access the created resource, check specific headers like <code>Location</code>, and handle errors gracefully.</p>
<h2>Takeaway</h2>
<p>Refit transforms the way we interact with APIs in .NET applications.
Converting your API into a strongly typed interface simplifies your code, enhances type safety, and improves maintainability.</p>
<p>The key Refit benefits we've explored include:</p>
<ul>
<li>Simplified API calls with automatic serialization and deserialization</li>
<li>Flexible parameter handling for complex queries</li>
<li>Easy management of headers and authentication</li>
<li>Options for JSON serialization to fit your project's needs</li>
<li>Granular control over HTTP responses when required</li>
</ul>
<p>In my experience, Refit shines in projects of all sizes.
I've used it in small applications and large-scale microservices architectures.
It eliminates boilerplate code, reduces the risk of errors, and allows you to focus on your application's core logic rather than the intricacies of HTTP communication.</p>
<p>Remember, while Refit makes API interactions more straightforward, it's not a substitute for understanding the underlying principles of RESTful communication and HTTP.</p>
<p>That's all for today. Stay awesome, and I'll see you next week.</p>
<p><strong>P.S.</strong> You can find the source code for this example in <a href="https://github.com/m-jovanovic/refit-client-example"><strong>this repository</strong></a>.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_106.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Introduction to Event Sourcing for .NET Developers]]></title>
            <link>https://milanjovanovic.tech/blog/introduction-to-event-sourcing-for-net-developers</link>
            <guid isPermaLink="false">introduction-to-event-sourcing-for-net-developers</guid>
            <pubDate>Sat, 31 Aug 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Discover event sourcing in .NET through a beginner's eyes. Explore core concepts, benefits, and real-world challenges.]]></description>
            <content:encoded><![CDATA[<p>I've been coding in .NET for years, but I never built an event sourced system.
Event sourcing has always intrigued me, though.
The idea of capturing every change and having a complete history of your data - it's fascinating.</p>
<p>So, I decided to dive in.
Not as an expert but as a curious developer.</p>
<p>In this newsletter, I'm sharing my journey into event sourcing.</p>
<ul>
<li>What is it really?</li>
<li>Why does it matter?</li>
<li>And how might it change the way we think about our .NET apps?</li>
</ul>
<p>We'll look at the core concepts of event sourcing, potential benefits, and even some practical examples.</p>
<h2>What is Event Sourcing?</h2>
<blockquote>
<p>Event Sourcing is an architectural design pattern where changes that occur in a domain are immutably stored as events in an append-only log.</p>
</blockquote>
<p><em>— <a href="https://www.eventstore.com/event-sourcing">Event Store</a></em></p>
<p>When I first encountered event sourcing, it seemed complex.
But stripped down, it's a surprisingly simple idea: store changes, not just the current state.</p>
<p>Think of a bank account or wallet.
Normally, we'd just save the balance.
With event sourcing, we record every deposit and withdrawal.
The balance is then calculated from these events.</p>
<p>This diagram illustrates the difference:</p>
<p><img src="/blogs/mnw_105/event_sourcing.png" alt="Event sourcing comparison to traditional data storage."></p>
<p>This shift from storing state to storing events is the essence of event sourcing.
It's like keeping a detailed diary of your application's data rather than just a snapshot.
It's not just about where you are but how you got there.
For me, this was a lightbulb moment.</p>
<h2>Why Use Event Sourcing?</h2>
<p>As I dug deeper into event sourcing, I kept asking myself: &quot;Why would I use this instead of traditional data storage?&quot;</p>
<p>Here's what I've discovered:</p>
<ul>
<li><strong>Full Audit Trail</strong>: Every change is recorded.
This is huge for businesses dealing with sensitive data or financial transactions.
Imagine being able to trace every step of an order's journey or every modification to a user's account.</li>
<li><strong>Debugging Time Machine</strong>: With event sourcing, you can reconstruct the state of your application at any point in time.
As a developer, this feels like a superpower.
Tracking down bugs becomes less about guesswork and more about replay.</li>
<li><strong>Business Insights</strong>: All those stored events?
They're a goldmine of data.
You can analyze patterns, user behavior, or system performance in ways that might be impossible with just current-state data.</li>
<li><strong>Flexibility</strong>: Need to add a new feature that requires historical data?
With event sourcing, it's already there.
This flexibility could have saved me from many &quot;I wish we had kept that information&quot; moments.</li>
</ul>
<p>Real-world use cases for event sourcing started to make sense:</p>
<ul>
<li>E-commerce platforms leveraging it for order tracking and inventory management.</li>
<li>Financial systems use it for accurate transaction histories.</li>
<li>IoT applications use it to analyze sensor data over time.</li>
</ul>
<p>While it's not a silver bullet (what is in programming?), I'm beginning to see why so many developers are excited about event sourcing.
It's not just about storing data; it's about intent and behavior.</p>
<h2>Core Concepts And Practical Examples</h2>
<p>As I started to implement event sourcing, understanding the core concepts became much easier with a concrete example.
Let's walk through a simple bank account scenario to see how event sourcing works.</p>
<h3>Events</h3>
<p>Events are immutable records of something that happened.
In our bank account example, we might have events like <code>AccountOpened</code>, <code>MoneyDeposited</code>, and <code>MoneyWithdrawn</code>.</p>
<p>Here's how we might define these in C#:</p>
<pre><code class="language-csharp">public record AccountOpened(Guid AccountId, DateTime OpenedAt);
public record MoneyDeposited(decimal Amount, DateTime DepositedAt);
public record MoneyWithdrawn(decimal Amount, DateTime WithdrawnAt);
</code></pre>
<p><a href="records-anonymous-types-non-destructive-mutation"><strong>Records</strong></a> are a perfect fit for events, as they are immutable by design.</p>
<h3>State</h3>
<p>In event sourcing, the current state is calculated by applying all events in order.</p>
<p>Here's how our <code>Account</code> class looks:</p>
<pre><code class="language-csharp">public class Account
{
    public Guid Id { get; private set; }
    public decimal Balance { get; private set; }

    private List&lt;object&gt; _events = new List&lt;object&gt;();

    public Account(Guid id)
    {
        ApplyEvent(new AccountCreated(id));
    }

    public void Deposit(decimal amount)
    {
        ApplyEvent(new MoneyDeposited(amount));
    }

    public void Withdraw(decimal amount)
    {
        if (Balance &gt;= amount)
        {
            ApplyEvent(new MoneyWithdrawn(amount));
        }
        else
        {
            throw new InvalidOperationException(&quot;Insufficient funds&quot;);
        }
    }

    private void ApplyEvent(object @event)
    {
        _events.Add(@event);

        switch (@event)
        {
            case AccountCreated e:
                Id = e.AccountId;
                Balance = 0;
                break;
            case MoneyDeposited e:
                Balance += e.Amount;
                break;
            case MoneyWithdrawn e:
                Balance -= e.Amount;
                break;
        }
    }
}
</code></pre>
<p>Notice how the <code>Account</code> class maintains its state.
Each method (<code>Deposit</code>, <code>Withdraw</code>) doesn't directly modify the balance.
Instead, it creates and applies an event.
The <code>ApplyEvent</code> method then updates the state based on these events.</p>
<h3>Event Store</h3>
<p>In our simple example, we're using a list (<code>_events</code>) to store events.
In a real system, we would persist these events in a database.
The key principle remains: events are appended, never modified.</p>
<p>For production systems, there are specialized event sourcing databases like <a href="https://www.eventstore.com/">EventStoreDB</a>.</p>
<p>There's also <a href="fast-document-database-in-net-with-marten"><strong>Marten</strong></a>, a .NET library that adds document database and event sourcing capabilities to PostgreSQL.</p>
<h2>Putting It All Together</h2>
<p>Here's how we might use our event sourced <code>Account</code>:</p>
<ul>
<li>An action (like depositing money) triggers the creation of an event.</li>
<li>The event is stored in the event store (in our simple example, it's just added to the <code>_events</code> list).</li>
<li>The event is applied to update the current state of the <code>Account</code>.</li>
<li>We can rebuild the state by replaying all events in order when needed.</li>
</ul>
<pre><code class="language-csharp">var account = new Account(Guid.NewGuid());
account.Deposit(100);
account.Withdraw(30);
account.Deposit(50);

Console.WriteLine($&quot;Final balance: {account.Balance}&quot;); // Output: Final balance: 120
</code></pre>
<p>We'd store these events in a database in a real event sourcing system.
This allows us to replay the events on demand to produce the current state.</p>
<h2>Challenges and Considerations</h2>
<p>Since I started researching event sourcing, I've seen its potential and its hurdles.</p>
<p>Event sourcing itself is a simple idea.
However, the underlying complexity of this approach concerns me.
There's a significant learning curve from event sourcing basics to <em>applying event sourcing in production</em>.</p>
<p>It's not just about storing data differently.
It's a fundamental shift in how you model and think about your domain.
This complexity extends to the infrastructure level.</p>
<p>Performance is another consideration that's often overlooked.
While appending events is typically fast, reconstructing the current state from a long history of events can be slow.
Real-world systems often need to implement caching strategies or snapshots to mitigate this.
Event sourcing is also eventually consistent on the read side.</p>
<p>One of the trickiest aspects I've encountered is event schema evolution (event versioning).
As your system grows and changes, so will your events.
Managing these changes without breaking existing event streams is a challenge that requires careful planning and design.
I'm still researching best practices.</p>
<h2>In Summary</h2>
<p>Event sourcing has a steep learning curve, even for an experienced developer.
It requires a fundamental shift in how you think about data and system design.</p>
<p>If you want to give it a try, start small.
Implement a simple event-sourced system in a side project.
It's the best way to grapple with the concepts hands-on.
As you do, you might find that Domain-Driven Design (DDD) principles align well with event sourcing.</p>
<p>Remember, the goal isn't to use event sourcing everywhere but to understand where it can add value.</p>
<p>If you're ready to explore this topic further, check out <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a>.
There's an entire chapter on Event-Driven Architecture, which directly complements what you've learned about event sourcing here.</p>
<p>In a future newsletter, we'll explore a more real-world application of event sourcing.</p>
<p>Good luck out there, and I'll see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_105.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Screaming Architecture]]></title>
            <link>https://milanjovanovic.tech/blog/screaming-architecture</link>
            <guid isPermaLink="false">screaming-architecture</guid>
            <pubDate>Sat, 24 Aug 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[If you were to glance at the folder structure of your system, could you tell what the system is about? Your architecture should communicate what problems it solves. This approach is called sreaming architecture.]]></description>
            <content:encoded><![CDATA[<p>If you were to glance at the folder structure of your system, could you tell what the system is about?
And here's a more interesting question.
Could a new developer on your team easily understand what the system does based on the folder structure?</p>
<p>Your architecture should communicate what problems it solves.
Organizing your system around use cases leads to a structure aligned with the business domain.
This approach is called <strong>screaming architecture</strong>.</p>
<p><a href="https://blog.cleancoder.com/uncle-bob/2011/09/30/Screaming-Architecture.html">Screaming architecture</a> is a term coined by Robert Martin (Uncle Bob).
He argues that a software system's structure should communicate what the system is about.
He draws a parallel between looking at a blueprint for a building, where you can tell the purpose of the building based on the blueprint.</p>
<p>In this article, I want to show some practical examples and discuss the benefits of screaming architecture.</p>
<h2>A Use Case Driven Approach</h2>
<p>A use case represents a specific interaction or task that a user wants to achieve within your system.
It encapsulates the business logic required to fulfill that task.
A use case is a high-level description of a user's goal.
For example, &quot;reserving an apartment&quot; or &quot;purchasing a ticket&quot;.
It focuses on the <em>what</em> of the system's behavior, not the <em>how</em>.</p>
<p>When you look at the folder structure and source code files of your system:</p>
<ul>
<li>Do they scream: Apartment Booking System or Ticketing System?</li>
<li>Or do they scream <a href="http://ASP.NET">ASP.NET</a> Core?</li>
</ul>
<p>Here's an example of a folder structure organized around technical concerns:</p>
<pre><code class="language-powershell">📁 Api/
|__ 📁 Controllers
|__ 📁 Entities
|__ 📁 Exceptions
|__ 📁 Repositories
|__ 📁 Services
    |__ #️⃣ ApartmentService.cs
    |__ #️⃣ BookingService.cs
    |__ ...
|__ 📁 Models
</code></pre>
<p>Somewhere inside these folders, we'll find concrete classes that contain the system's behavior.
You'll notice that the cohesion with this folder structure is low.</p>
<p>How does screaming architecture help?</p>
<p>A use case driven approach will place the system's use cases as the top-level concept.
I also like to group related use cases into a top-level feature folder.
Inside a use case folder, we may find technical concepts required to implement it.</p>
<p><a href="vertical-slice-architecture"><strong>Vertical slice architecture</strong></a> also approaches this from a similar perspective.</p>
<pre><code class="language-powershell">📁 Api/
|__ 📁 Apartments
    |__ 📁 ReserveApartment
    |__ ...
|__ 📁 Bookings
    |__ 📁 CancelBooking
    |__ ...
|__ 📁 Payments
|__ 📁 Reviews
|__ 📁 Disputes
|__ 📁 Invoicing
</code></pre>
<p>The use case driven folder structure helps us better understand user needs and aligns development efforts with business goals.</p>
<h2>Screaming Architecture Benefits</h2>
<p>The benefits of organizing our system around use cases are:</p>
<ul>
<li>Improved cohesion since related use cases are close together</li>
<li>High coupling for a single use case and its related use cases</li>
<li>Low coupling between unrelated use cases</li>
<li>Easier navigation through the solution</li>
</ul>
<h2>Bounded Contexts and Vertical Slices</h2>
<p>We have many techniques for discovering the high-level modules within our system.
For example, we could use <a href="https://www.eventstorming.com/">event storming</a> to explore the system's use cases.
Domain exploration happens before we write a single line of code.</p>
<p>The next step is decomposing the larger problem domain into smaller sub-domains and later bounded contexts.
This gives us loosely coupled high-level modules that we can translate into code.</p>
<p><img src="/blogs/mnw_104/bounded_contexts.png" alt="Bounded contexts."></p>
<p>The overarching idea here is thinking about cohesion around functionalities.
We want to organize our system so that the cohesion between the components is high.
Bounded contexts, vertical slices, and screaming architecture are complementary concepts.</p>
<p>Here's a screaming architecture example for this system.
Let's say the <code>Ticketing</code> module uses <a href="clean-architecture-folder-structure"><strong>Clean Architecture</strong></a> internally.
But we can still organize the system around feature folders and use cases.
An alternative approach could be organizing around <a href="vertical-slice-architecture-structuring-vertical-slices"><strong>vertical slices</strong></a>, resulting in a less nested folder structure.</p>
<pre><code class="language-powershell">📁 Modules/
|__ 📁 Attendance
    |__ ...
|__ 📁 Events
    |__ ...
|__ 📁 Ticketing
    |__ 📁 Application
        |__ 📁 Carts
            |__ 📁 AddItemToCart
            |__ 📁 ClearCart
            |__ 📁 GetCart
            |__ 📁 RemoveItemFromCart
        |__ 📁 Orders
            |__ 📁 SubmitOrder
            |__ 📁 CancelOrder
            |__ 📁 GetOrder
        |__ 📁 Payments
            |__ 📁 RefundPayment
        |__ ...
    |__ 📁 Domain
        |__ 📁 Customers
        |__ 📁 Orders
        |__ 📁 Payments
        |__ 📁 Tickets
        |__ ...
    |__ 📁 infrastructure
        |__ 📁 Authentication
        |__ 📁 Customers
        |__ 📁 Database
        |__ 📁 Orders
        |__ 📁 Payments
        |__ 📁 Tickets
        |__ ...
|__ 📁 Users
    |__ ...
</code></pre>
<p>The example above is a small part of the system I built inside of <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a>.</p>
<h2>Takeaway</h2>
<p><strong>Screaming Architecture</strong> isn't just a catchy phrase, it's an approach that can profoundly impact how you build software.
By organizing your system around use cases, you align your codebase with the core business domain.
Your system exists to solve the business domain problems.</p>
<p>Remember, the goal is to create a system that communicates its purpose through its structure.
Embrace a use case-driven approach, break down complex domains into bounded contexts.
Build a system that truly &quot;screams&quot; about the problems it solves.</p>
<p>If you want to explore these powerful ideas further, check out <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture</strong></a>.
I share my entire framework for building robust applications from the ground up and organizing the system around use cases.</p>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_104.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Complete Guide to Amazon SQS and Amazon SNS With MassTransit]]></title>
            <link>https://milanjovanovic.tech/blog/complete-guide-to-amazon-sqs-and-amazon-sns-with-masstransit</link>
            <guid isPermaLink="false">complete-guide-to-amazon-sqs-and-amazon-sns-with-masstransit</guid>
            <pubDate>Sat, 17 Aug 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[In this article, we'll explore how to use Amazon SQS and SNS for asynchronous messaging in .NET applications. We'll also see how MassTransit simplifies the process, enabling you to build robust message-driven systems.]]></description>
            <content:encoded><![CDATA[<p>Have you ever wondered how large-scale systems handle traffic spikes or maintain performance even when parts of the system are temporarily down?
The answer lies in asynchronous messaging.</p>
<p>Asynchronous messaging is, at its core, about decoupling.
Our components can operate independently and communicate through a message queue or topic.
If one service (component) is temporarily unavailable, the others can continue working.
This improves our system's scalability, resilience, and fault tolerance.</p>
<p>In this article, we'll explore how to use Amazon SQS and SNS for asynchronous messaging in .NET applications.</p>
<p>We'll also see how MassTransit simplifies the process, enabling you to build robust message-driven systems.</p>
<p>Let's dive in.</p>
<h2>What is Amazon SQS?</h2>
<p><a href="https://aws.amazon.com/sqs/">Amazon Simple Queue Service</a> (SQS) is a fully managed message queueing service.
It facilitates the decoupling and scaling of microservices and distributed systems.</p>
<p>SQS acts as a reliable middleman for asynchronous communication.
It enables different components of your architecture to exchange messages without needing to be online or directly connected at the same time.
Messages are stored in queues and consumed on demand.</p>
<p>SQS offers two distinct queue types depending on your requirements:</p>
<ul>
<li><strong>Standard Queues</strong>: Ideal for high-throughput scenarios.
Standard Queues provide at-least-once delivery and best-effort ordering.</li>
<li><strong>FIFO Queues</strong>: Recommended when maintaining message order is required.
FIFO Queues guarantee exactly-once processing and preserve the sequence in which messages are sent.</li>
</ul>
<p>Let's say we have two services - <code>Stock</code> and <code>Reporting</code>.
When a user creates a purchase order in the <code>Stock</code> service, we want to notify the <code>Reporting</code> service.</p>
<p>SQS allows us to create decoupled communication between these services.
The <code>Stock</code> service sends a message to an SQS queue, and the <code>Reporting</code> service can poll from the queue to consume messages.</p>
<p><img src="/blogs/mnw_103/amazon_sqs.png" alt="Amazon SQS."></p>
<p>It's interesting to highlight that SQS uses a polling mechanism for message consumers.
When a consumer polls for new messages, SQS starts a <a href="https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-visibility-timeout.html">visibility timeout</a>.
SQS doesn't automatically delete the messages.
Instead, messages are hidden from other consumers until the timeout expires.</p>
<p>When the consumer successfully processes a message, it's removed from the queue.
But if the visibility timeout expires, the message becomes visible and can be delivered again.
Other consumers can receive this message when polling from SQS.
This is why SQS offers at-least-once delivery (for standard queues).
You will have to implement <a href="idempotent-consumer-handling-duplicate-messages"><strong>idempotency in the consumer</strong></a>.</p>
<h2>Amazon SQS and Competing Consumers</h2>
<p>Let's introduce another service into our system - the <code>Risk Management</code> service.
When multiple consumers (services) poll an SQS queue, each wants to retrieve and process messages as they become available.
However, once a message is successfully received and processed by one consumer, it's removed from the queue.</p>
<p>Why is this a problem?</p>
<p>Other consumers who might have been polling will miss out on that specific message.
This is known as <a href="https://learn.microsoft.com/en-us/azure/architecture/patterns/competing-consumers">competing consumers</a>.</p>
<p>Let's consider the example of <code>Reporting</code> and <code>Risk Management</code> services polling the same queue.
If a new message arrives, only one of these services will &quot;win&quot; the race and retrieve it for processing.
The other service won't find that message even if it polls moments later.</p>
<p><img src="/blogs/mnw_103/amazon_sqs_competing_consumers.png" alt="Amazon SQS and competing consumers."></p>
<p>So, how can we solve this?</p>
<p>We could introduce a dedicated queue for each service.
However, the producer now needs to publish to multiple queues.
This creates a possibility for some (not so) interesting partial failures.
What happens if we successfully publish to one queue but fail to publish to the other?</p>
<p>You can see how this becomes difficult to scale while maintaining reliability.</p>
<p>Luckily, there's a solution.</p>
<h2>Amazon SNS to The Rescue</h2>
<p><a href="https://aws.amazon.com/sns/">Amazon Simple Notification Service</a> (SNS) is a fully managed pub/sub messaging service.
It allows publishers to send messages to multiple subscribers (topics) simultaneously.</p>
<p>SNS operates on the principle of publishers and subscribers.
Publishers send messages to an SNS topic, while subscribers express interest in specific topics and receive messages published to those topics.
This decoupled architecture allows you to add or remove subscribers without impacting the publisher or other subscribers.</p>
<p>SNS seamlessly integrates with SQS, allowing you to create a powerful combination where SNS handles the fan-out of messages,
and SQS queues ensure that each message is processed exclusively by a single consumer (service).</p>
<p>Instead of sending a message to the queue, the <code>Stock</code> service now publishes to an SNS topic.
Both the <code>Reporting</code> and <code>Risk Management</code> services create their own SQS queues and subscribe these queues to the SNS topic.
When a new message is published to the SNS topic, SNS delivers it to both SQS queues.
Each queue receives its own copy of the message.</p>
<p>If we want to introduce a new service, we'll create a new SQS queue and subscribe it to the topic.</p>
<p><img src="/blogs/mnw_103/amazon_sns.png" alt="Amazon SNS."></p>
<h2>MassTransit Integration With SQS and SNS</h2>
<p>How can we use SNS and SQS from a .NET application?</p>
<p>You could use the official AWS SDKs.
The benefit is you'll have more control over messaging.
However, you will need to write more code to receive and handle messages successfully.</p>
<p>So, I want to suggest a different approach.</p>
<p>MassTransit is one of the most popular messaging libraries in .NET.
It provides a set of messaging abstractions on top of the supported message transports.</p>
<p>I wrote an article about <a href="using-masstransit-with-rabbitmq-and-azure-service-bus"><strong>using MassTransit with RabbitMQ and Azure Service Bus</strong></a>.</p>
<p>But we'll focus on using MassTransit with SQS and SNS.</p>
<p>Let's start by installing the NuGet package we'll need:</p>
<pre><code class="language-powershell">Install-Package MassTransit.AmazonSQS
</code></pre>
<p>Next, we'll need to configure MassTransit with our .NET applications.</p>
<p>Here's the MassTransit configuration for the <code>Stock</code> service:</p>
<pre><code class="language-csharp">builder.Services.AddMassTransit(configure =&gt;
{
    configure.AddConsumer&lt;PurchaseOrderSentConsumer&gt;().Endpoint(e =&gt; e.InstanceId = &quot;stocks&quot;);

    configure.UsingAmazonSqs((context, cfg) =&gt;
    {
        cfg.Host(&quot;eu-central-1&quot;, h =&gt;
        {
            h.AccessKey(builder.Configuration[&quot;AmazonSqs:AccessKey&quot;]!);
            h.SecretKey(builder.Configuration[&quot;AmazonSqs:SecretKey&quot;]!);

            h.Scope(&quot;stocks-platform&quot;, scopeTopics: true);
        });

        cfg.ConfigureEndpoints(context, new KebabCaseEndpointNameFormatter(&quot;stocks-platform-&quot;, false));
    });
});
</code></pre>
<p>Here are a few things I want to highlight here:</p>
<ul>
<li><code>UsingAmazonSqs</code> - Configures SQS (and SNS) as the message transport.</li>
<li><code>Scope</code> - Adds a prefix to SNS topic names. This helps distinguish topics from other applications and environments.</li>
<li><code>Endpoint</code> - Sets the <code>InstanceId</code>, which is appended to the endpoint (queue) name.</li>
<li><code>ConfigureEndpoints</code> - Allows us to specify a prefix for endpoint (queue) names. The idea is the same as the topic name prefix.</li>
</ul>
<p>You'll also need to configure an <a href="https://masstransit.io/documentation/configuration/transports/amazon-sqs#example-iam-policy">IAM policy</a>
that gives MassTransit the required permissions for AWS resources.</p>
<p>And with this setup in place, you can use MassTransit to publish messages.
Here's an endpoint that accepts a purchase order and publishes a <code>PurchaseOrderSent</code> message.</p>
<pre><code class="language-csharp">app.MapPost(&quot;purchase-orders&quot;, async (PurchaseOrderRequest request, IPublishEndpoint publishEndpoint) =&gt;
{
    var purchaseOrder = new PurchaseOrder
    {
        Id = Guid.NewGuid(),
        Ticker = request.Ticker,
        LimitPrice = request.LimitPrice,
        Quantity = request.Quantity
    };

    OrdersDb.Add(purchaseOrder);

    await publishEndpoint.Publish(new PurchaseOrderSent(purchaseOrder.Id));

    return Results.Ok(purchaseOrder);
});
</code></pre>
<p>We will process the purchase order in the <code>PurchaseOrderSentConsumer</code>.
If it's successfully processed (filled), we will publish an <code>OrderFilled</code> message.</p>
<p>The <code>Risk Management</code> service can subscribe to this message using MassTransit.
The configuration is almost identical, with the only difference being the endpoint's <code>InstanceId</code>.</p>
<pre><code class="language-csharp">builder.Services.AddMassTransit(configure =&gt;
{
    configure.AddConsumer&lt;OrderFilledConsumer&gt;().Endpoint(e =&gt; e.InstanceId = &quot;risk-management&quot;);

    configure.UsingAmazonSqs((context, cfg) =&gt;
    {
        cfg.Host(&quot;eu-central-1&quot;, h =&gt;
        {
            h.AccessKey(builder.Configuration[&quot;AmazonSqs:AccessKey&quot;]!);
            h.SecretKey(builder.Configuration[&quot;AmazonSqs:SecretKey&quot;]!);

            h.Scope(&quot;stocks-platform&quot;, true);
        });

        cfg.ConfigureEndpoints(context, new KebabCaseEndpointNameFormatter(&quot;stocks-platform-&quot;, false));
    });
});
</code></pre>
<h2>Broker Topology in AWS</h2>
<p>MassTransit will automatically create the required queues, topics, and subscriptions in AWS.
If needed, you can further configure the SQS and SNS resources.</p>
<p>Of course, you can also create the required infrastructure in AWS and tell MassTransit to use it.</p>
<p>But let's keep it simple and allow MassTransit to provision the AWS resources.</p>
<p>By default, MassTransit creates standard SQS queues.
Here's what we get in Amazon SQS.</p>
<div className="bordered">
  ![Amazon SQS queues created by MassTransit.](/blogs/mnw_103/amazon_sqs_queues.png)
</div>
<p>And here's what we get in Amazon SNS:</p>
<div className="bordered">
  ![Amazon SNS topics created by MassTransit.](/blogs/mnw_103/amazon_sns_topics.png)
</div>
<p>MassTransit automatically configures the required subscriptions between the topic and queues.</p>
<p>And we are ready to start publishing and consuming messages.</p>
<h2>In Summary</h2>
<p>Asynchronous communication and decoupling are pivotal in achieving scalability, resilience, and fault tolerance.
Amazon SQS and SNS provide the building blocks of message-driven architectures in the AWS cloud.</p>
<p>We explored the core concepts of SQS and SNS, understanding how they enable reliable message delivery and fan-out capabilities.</p>
<p>MassTransit provides an excellent abstraction layer over SQS and SNS, simplifying development.
We can focus on solving the business problems and delivering value to our users.</p>
<p>The combination of Amazon SQS, SNS, and MassTransit gives us robust tools for building modern, event-driven applications.</p>
<p>Thanks for reading, and I'll see you next week!</p>
<p><strong>P.S.</strong> You can find the source code for this article in <a href="https://github.com/m-jovanovic/aws-tutorials"><strong>this repository</strong></a>, under the <code>Amazon SQS and SNS</code> folder.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_103.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[5 EF Core Features You Need To Know]]></title>
            <link>https://milanjovanovic.tech/blog/5-ef-core-features-you-need-to-know</link>
            <guid isPermaLink="false">5-ef-core-features-you-need-to-know</guid>
            <pubDate>Sat, 10 Aug 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[EF Core is powerful, and knowing a few key features can save you lots of time and frustration. I've cherry-picked five essential features that you really need to know.]]></description>
            <content:encoded><![CDATA[<p>Okay, let's be honest.
We all have a million things on our plates, and diving deep into every nook and cranny of EF Core might not be
at the top of your priority list.</p>
<p>But here's the deal: EF Core is powerful, and knowing a few key features can save you lots of time and frustration.</p>
<p>So, I won't bombard you with every single EF Core feature under the sun.</p>
<p>Instead, I've cherry-picked five essential ones that you really need to know.</p>
<p>We'll go through:</p>
<ul>
<li><strong>Query Splitting</strong> - your database's new best friend</li>
<li><strong>Bulk Updates and Deletes</strong> - efficiency on steroids</li>
<li><strong>Raw SQL Queries</strong> - when you need to go rogue</li>
<li><strong>Query Filters</strong> - keeping things nice and tidy</li>
<li><strong>Eager Loading</strong> - because lazy isn't so great</li>
</ul>
<p>Let's get started!</p>
<h2>Query Splitting</h2>
<p><a href="how-to-improve-performance-with-ef-core-query-splitting"><strong>Query splitting</strong></a> is one of those EF Core features that you rarely need.
Until one day, you do.
Query splitting is helpful in scenarios where you're eager loading multiple collections.
It helps us avoid the <a href="https://learn.microsoft.com/en-us/ef/core/performance/efficient-querying#avoid-cartesian-explosion-when-loading-related-entities">cartesian explosion</a> problem.</p>
<p>Let's say we want to retrieve a department with all its teams and employees.
We might write a query like this:</p>
<pre><code class="language-csharp">Department department =
    context.Departments
        .Include(d =&gt; d.Teams)
        .Include(d =&gt; d.Employees)
        .Where(d =&gt; d.Id == departmentId)
        .First();
</code></pre>
<p>This translates to a single SQL query with two JOINs.
However, since these <code>JOIN</code> statements are on the same level, the database will return a <em>cross product</em>.
Each row from <code>Teams</code> will be joined with each row <code>Employees</code>.
In that case, the database returns many rows, significantly impacting performance.</p>
<p>Here's how we can avoid these performance issues with query splitting:</p>
<pre><code class="language-csharp">Department department =
    context.Departments
        .Include(d =&gt; d.Teams)
        .Include(d =&gt; d.Employees)
        .Where(d =&gt; d.Id == departmentId)
        .AsSplitQuery()
        .First();
</code></pre>
<p>With <code>AsSplitQuery</code>, EF Core will execute an additional SQL query for each collection navigation.</p>
<p>However, be cautious not to overuse query splitting.
I use split queries when I've <em>measured</em> that they consistently perform better.</p>
<p>Split queries have more round trips to the database, which might be slower if database latency is high.
There is also no consistency guarantee across multiple SQL queries.</p>
<h2>Bulk Updates and Deletes</h2>
<p>EF Core 7 added two new APIs for performing <a href="how-to-use-the-new-bulk-update-feature-in-ef-core-7"><strong>bulk updates and deletes</strong></a>,
<code>ExecuteUpdate</code> and <code>ExecuteDelete</code>.
They allow you to efficiently update a large number of rows in one round trip to the database.</p>
<p>Here's a practical example.</p>
<p>The company has decided to give a 5% raise to all employees in the &quot;Sales&quot; department.
Without bulk updates, we might iterate through each employee and update their salary individually:</p>
<pre><code class="language-csharp">var salesEmployees = context.Employees
    .Where(e =&gt; e.Department == &quot;Sales&quot;)
    .ToList();

foreach (var employee in salesEmployees)
{
    employee.Salary *= 1.05m;
}

context.SaveChanges();
</code></pre>
<p>This approach involves multiple database roundtrips, which can be inefficient, especially for large datasets.</p>
<p>We can achieve the same in one roundtrip using <code>ExecuteUpdate</code>:</p>
<pre><code class="language-csharp">context.Employees
    .Where(e =&gt; e.Department == &quot;Sales&quot;)
    .ExecuteUpdate(s =&gt; s.SetProperty(e =&gt; e.Salary, e =&gt; e.Salary * 1.05m));
</code></pre>
<p>This executes a single SQL <code>UPDATE</code> statement, directly modifying the salaries in the database without loading entities into memory, giving us improved performance.</p>
<p>Here's another example.
Let's say an e-commerce platform wants to delete all shopping carts older than one year.</p>
<p>Here's how we could do this with <code>ExecuteDelete</code>:</p>
<pre><code class="language-csharp">context.Carts
    .Where(o =&gt; o.CreatedOn &lt; DateTime.Now.AddYears(-1))
    .ExecuteDelete();
</code></pre>
<p>This results in a single SQL <code>DELETE</code> statement, directly removing the old shopping carts from the database.</p>
<p>However, bulk updates bypass the EF change tracker.
This could be problematic, and I wrote about the <a href="what-you-need-to-know-about-ef-core-bulk-updates"><strong>caveats of bulk updates in this article</strong></a>.</p>
<h2>Raw SQL Queries</h2>
<p>EF Core 8 added a new feature that allows us to query unmapped types with raw SQL.</p>
<p>Suppose we want to retrieve data from a database view, stored procedure, or a table that doesn't directly correspond to any of our entity classes.</p>
<p>For example, we want to retrieve a sales summary for each product.
With EF Core 8, we can define a simple <code>ProductSummary</code> class representing the structure of the result set and query it directly:</p>
<pre><code class="language-csharp">public class ProductSummary
{
    public int ProductId { get; set; }
    public string ProductName { get; set; }
    public decimal TotalSales { get; set; }
}

var productSummaries = await context.Database
    .SqlQuery&lt;ProductSummary&gt;(
        @$&quot;&quot;&quot;
        SELECT p.ProductId, p.ProductName, SUM(oi.Quantity * oi.UnitPrice) AS TotalSales
        FROM Products p
        JOIN OrderItems oi ON p.ProductId = oi.ProductId
        WHERE p.CategoryId = {categoryId}
        GROUP BY p.ProductId, p.ProductName
        &quot;&quot;&quot;)
    .ToListAsync();
</code></pre>
<p>The <code>SqlQuery</code> method returns an <code>IQueryable</code>, which allows you to compose raw SQL queries with LINQ.
This combines the power of raw SQL with the expressiveness of LINQ.</p>
<p>Remember to use parameterized queries to prevent SQL injection vulnerabilities.
The <code>SqlQuery</code> method accepts a <code>FormattableString</code>, which means you can safely use an interpolated string.
Each argument is converted to a SQL parameter.</p>
<p>You can learn more about <a href="ef-core-raw-sql-queries"><strong>raw SQL queries in this article</strong></a>.</p>
<h2>Query Filters</h2>
<p><a href="how-to-use-global-query-filters-in-ef-core"><strong>Query filters</strong></a> are like reusable <code>WHERE</code> clauses you can apply to your entities.
These filters are automatically added to LINQ queries whenever you retrieve entities of the corresponding type.
This saves you from repeatedly writing the same filtering logic in multiple places within your application.</p>
<p>Query Filters are commonly used for scenarios like:</p>
<ul>
<li><a href="implementing-soft-delete-with-ef-core"><strong>Soft Deletes</strong></a>: Filter out records marked as deleted.</li>
<li><a href="multi-tenant-applications-with-ef-core"><strong>Multi-tenancy</strong></a>: Filter data based on the current tenant.</li>
<li>Row-level security: Restrict access to certain records based on user roles or permissions.</li>
</ul>
<p>In a multi-tenant application, you often need to filter data based on the current tenant.
Query filters allow us to handle this requirement easily:</p>
<pre><code class="language-csharp">public class Product
{
    public int Id { get; set; }
    public string Name { get; set; }
    // Associate products with tenants
    public int TenantId { get; set; }
}

protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    // The current TenantId is set based on the current request/context
    modelBuilder.Entity&lt;Product&gt;().HasQueryFilter(p =&gt; p.TenantId == _currentTenantId);
}

// Now, queries automatically filter based on the tenant:
var productsForCurrentTenant = context.Products.ToList();
</code></pre>
<p>Configuring multiple query filters on the same entity will only apply the last one.
You can combine multiple query filters using <code>&amp;&amp;</code> (AND) and <code>||</code> (OR) operators.</p>
<p>You can use <code>IgnoreQueryFilters</code> to bypass the filters in specific queries when needed.</p>
<h2>Eager Loading</h2>
<p>Eager Loading is a feature in EF Core that allows you to load related entities along with your main entity in a single database query.
By fetching all necessary data in a single query, you can improve application performance.
This is especially true when dealing with complex object graphs or when lazy loading would result in many small, inefficient queries.</p>
<p>Here's an example <code>VerifyEmail</code> use case.
We want to load an <code>EmailVerificationToken</code> and eagerly load a <code>User</code> with the <code>Include</code> method because we want to modify both entities at the same time.</p>
<pre><code class="language-csharp">internal sealed class VerifyEmail(AppDbContext context)
{
    public async Task&lt;bool&gt; Handle(Guid tokenId)
    {
        EmailVerificationToken? token = await context.EmailVerificationTokens
            .Include(e =&gt; e.User)
            .FirstOrDefaultAsync(e =&gt; e.Id == tokenId);

        if (token is null || token.ExpiresOnUtc &lt; DateTime.UtcNow || token.User.EmailVerified)
        {
            return false;
        }

        token.User.EmailVerified = true;

        context.EmailVerificationTokens.Remove(token);

        await context.SaveChangesAsync();

        return true;
    }
}
</code></pre>
<p>EF Core will generate a single SQL query that joins the <code>EmailVerificationToken</code> and <code>User</code> tables, retrieving all the necessary data in one go.</p>
<p>Eager loading (and query splitting, which we mentioned earlier) isn't a silver bullet.
Consider using projections if you only need specific properties from related entities to avoid fetching unnecessary data.</p>
<h2>Summary</h2>
<p>So, there you have it!
Five EF Core features that, frankly, you can't afford <em>not</em> to know.
Remember, mastering EF Core takes time, but these features provide a solid foundation to build upon.</p>
<p>Another piece of advice is to deeply understand how your database works.
Mastering SQL also allows you to get the most value from EF Core.</p>
<p>While we focused on five key features, there are many other EF Core features worth exploring:</p>
<ul>
<li><a href="solving-race-conditions-with-ef-core-optimistic-locking"><strong>Optimistic concurrency control</strong></a></li>
<li><a href="efcore-migrations-a-detailed-guide"><strong>Database migrations</strong></a></li>
<li><a href="unleash-ef-core-performance-with-compiled-queries"><strong>Compiled queries</strong></a></li>
<li><a href="working-with-transactions-in-ef-core"><strong>Transactions</strong></a></li>
<li><a href="how-to-use-ef-core-interceptors"><strong>Interceptors</strong></a></li>
</ul>
<p>EF Core is continuously evolving, so keep an eye on the latest updates and releases to stay ahead.</p>
<p>Good luck out there, and see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_102.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Improving Code Quality in C# With Static Code Analysis]]></title>
            <link>https://milanjovanovic.tech/blog/improving-code-quality-in-csharp-with-static-code-analysis</link>
            <guid isPermaLink="false">improving-code-quality-in-csharp-with-static-code-analysis</guid>
            <pubDate>Sat, 03 Aug 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Static code analysis helps you build secure, maintainable, and high-quality C# code. In this week's newsletter, we'll explore how to integrate it into your .NET projects.]]></description>
            <content:encoded><![CDATA[<p>Writing good code is important for any software project.
It's also something I deeply care about.
However, it can be hard to spot problems by just reading through everything.</p>
<p>Luckily, there's a tool that can help: <strong>static code analysis</strong>.</p>
<p>It's like having an extra pair of eyes automatically checking your code.
Static code analysis helps you build secure, maintainable, and high-quality C# code.</p>
<p>Here's what we are going to cover in this week's newsletter:</p>
<ul>
<li>Static code analysis</li>
<li>Static analysis in .NET</li>
<li>Finding security risks</li>
</ul>
<p>Let's see how static code analysis can help us improve our code quality.</p>
<h2>What is Static Code Analysis?</h2>
<p>Static code analysis is a way to examine your code without actually running it.
It reports any issues related to security, performance, coding style, or best practices.</p>
<p>With static code analysis, you can <a href="https://en.wikipedia.org/wiki/Shift-left_testing">&quot;shift left&quot;</a>.
This allows you to find and fix issues early in the development process when they're less expensive to solve.</p>
<p>By writing high-quality code, you'll be able to build systems that are more reliable, scalable, and easier to maintain over time.
Investing in code quality will pay dividends in the later stages of any project.</p>
<p>You can integrate static code analysis into your <a href="how-to-build-ci-cd-pipeline-with-github-actions-and-dotnet"><strong>CI pipeline</strong></a> for a quick feedback loop.
We can also pair this with <a href="shift-left-with-architecture-testing-in-dotnet"><strong>architecture testing</strong></a> to enforce additional coding standards.</p>
<h2>Static Code Analysis in .NET</h2>
<p>.NET has built-in Roslyn analyzers that inspect your C# code for code style and quality issues.
Code analysis is enabled by default if your project targets .NET 5 or later.</p>
<p>The best way I found to configure static code analysis is using <code>Directory.Build.props</code>.
It's an XML file where you can configure common project properties.
You can place the <code>Directory.Build.props</code> file in the root folder so it will apply to all projects.</p>
<p>You can configure the <code>TargetFramework</code>, <code>ImplicitUsings</code>, <code>Nullable</code> (nullable reference types), etc.
But what we care about is configuring static code analysis.</p>
<p>Here are some properties we can configure:</p>
<ul>
<li><code>TreatWarningsAsErrors</code> - Treat all warnings as errors.</li>
<li><code>CodeAnalysisTreatWarningsAsErrors</code> - Treat code quality (CAxxxx) warnings as errors.</li>
<li><code>EnforceCodeStyleInBuild</code> - Enables code-style analysis (&quot;IDExxxx&quot;) rules.</li>
<li><code>AnalysisLevel</code> - Specifies which analyzers to enable. The default value is <code>latest</code>.</li>
<li><code>AnalysisMode</code> - Configures the predefined code analysis configuration.</li>
</ul>
<p>We can also install additional NuGet packages to our projects.
<code>SonarAnalyzer.CSharp</code> contains additional code analyzers to help us write clean, safe, and reliable code.
This library comes from the same company that built <a href="https://www.sonarsource.com/products/sonarqube/">SonarQube</a>.</p>
<pre><code class="language-xml">&lt;Project&gt;
  &lt;PropertyGroup&gt;
    &lt;TargetFramework&gt;net8.0&lt;/TargetFramework&gt;
    &lt;ImplicitUsings&gt;enable&lt;/ImplicitUsings&gt;
    &lt;Nullable&gt;enable&lt;/Nullable&gt;

    &lt;!-- Configure code analysis. --&gt;
    &lt;AnalysisLevel&gt;latest&lt;/AnalysisLevel&gt;
    &lt;AnalysisMode&gt;All&lt;/AnalysisMode&gt;
    &lt;TreatWarningsAsErrors&gt;true&lt;/TreatWarningsAsErrors&gt;
    &lt;CodeAnalysisTreatWarningsAsErrors&gt;true&lt;/CodeAnalysisTreatWarningsAsErrors&gt;
    &lt;EnforceCodeStyleInBuild&gt;true&lt;/EnforceCodeStyleInBuild&gt;
  &lt;/PropertyGroup&gt;

  &lt;ItemGroup Condition=&quot;'$(MSBuildProjectExtension)' != '.dcproj'&quot;&gt;
    &lt;PackageReference Include=&quot;SonarAnalyzer.CSharp&quot; Version=&quot;*&quot;&gt;
      &lt;PrivateAssets&gt;all&lt;/PrivateAssets&gt;
      &lt;IncludeAssets&gt;
        runtime; build; native; contentfiles; analyzers; buildtransitive
      &lt;/IncludeAssets&gt;
    &lt;/PackageReference&gt;
  &lt;/ItemGroup&gt;
&lt;/Project&gt;
</code></pre>
<p>The built-in .NET analyzers and the ones from <code>SonarAnalyzer.CSharp</code> can be very helpful.
But they can also make a lot of noise with too many build warnings.</p>
<p>When you encounter code analysis rules that you don't consider helpful, you can turn them off.
You can configure individual code analysis rules in the <code>.editorconfig</code> file.</p>
<pre><code># S125: Sections of code should not be commented out
dotnet_diagnostic.S125.severity = none

# S1075: URIs should not be hardcoded
dotnet_diagnostic.S1075.severity = none

# S2094: Classes should not be empty
dotnet_diagnostic.S2094.severity = none

# S3267: Loops should be simplified with &quot;LINQ&quot; expressions
dotnet_diagnostic.S3267.severity = none
</code></pre>
<h2>Finding (and Fixing) Security Risks</h2>
<p>Static code analysis can help you detect potential security vulnerabilities in your code.
Here's an example of a <code>PasswordHasher</code> using only <code>10,000</code> iterations to generate a password hash.
The <code>S5344</code> rule, from <code>SonarAnalyzer.CSharp</code>, detects this issue and warns us.
The recommended minimal number of iterations is <code>100,000</code>.</p>
<p>You can navigate to the explanation for <a href="https://rules.sonarsource.com/csharp/RSPEC-5344/">S5344</a> to learn more:</p>
<blockquote>
<p>Weakly hashed password storage poses a significant security risk to software applications.</p>
</blockquote>
<p>With <code>TreatWarningsAsErrors</code> turned on, your build will fail until you solve this issue.
This reduces the chance of introducing security risks in production.</p>
<div className="bordered">
  ![Example of static code analysis warning.](/blogs/mnw_101/static_code_analysis.png)
</div>
<h2>Conclusion</h2>
<p>Static code analysis is a powerful tool I include in all my C# projects.
It helps me catch problems early, leads to more reliable and secure code, and saves time and effort.
While the initial setup and fine-tuning of rules might take some time, the long-term benefits are undeniable.</p>
<p>Remember, static code analysis is a tool that complements your existing development practices.</p>
<p>You can create a robust development process that consistently delivers high-quality software by combining
static code analysis with other techniques like code reviews, unit testing, and continuous integration.</p>
<p>Embrace static code analysis.
Your future self (and your team) will thank you.</p>
<p>That's all for today.</p>
<p>See you next week.</p>
<p><strong>P.S.</strong> Here's a sample <a href="https://gist.github.com/m-jovanovic/417b7d0a641d7dd7d1972550fba298db">.editorconfig</a> file you can add to your projects and customize to fit your needs.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_101.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Simple Messaging in .NET With Redis Pub/Sub]]></title>
            <link>https://milanjovanovic.tech/blog/simple-messaging-in-dotnet-with-redis-pubsub</link>
            <guid isPermaLink="false">simple-messaging-in-dotnet-with-redis-pubsub</guid>
            <pubDate>Sat, 27 Jul 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Redis is a popular choice for caching data, but its capabilities go far beyond that. One of its lesser-known features is Pub/Sub support. Redis channels offer an interesting approach for implementing real-time messaging in your .NET applications.]]></description>
            <content:encoded><![CDATA[<p>Redis is a popular choice for <a href="caching-in-aspnetcore-improving-application-performance"><strong>caching data</strong></a>, but its capabilities go far beyond that.
One of its lesser-known features is Pub/Sub support.
Redis channels offer an interesting approach for implementing real-time messaging in your .NET applications.
However, as you'll soon see, channels also have some drawbacks.</p>
<p>In this week's newsletter, we'll explore:</p>
<ul>
<li>Basics of Redis channels</li>
<li>Practical use cases for channels</li>
<li>Implementing a Pub/Sub example in .NET</li>
<li>Cache invalidation in distributed systems</li>
</ul>
<p>Let's dive in.</p>
<h2>Redis Channels</h2>
<p>Redis channels are named communication channels that implement the <a href="https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern">Publish/Subscribe messaging paradigm</a>.
Each channel is identified by a unique name (e.g., <code>notifications</code>, <code>updates</code>).
Channels facilitate message delivery from publishers to subscribers.</p>
<p>Publishers use the <code>PUBLISH</code> command to send messages to a specific channel.
Subscribers use the <code>SUBSCRIBE</code> command to register interest in receiving messages from a channel.</p>
<p><img src="/blogs/mnw_100/redis_channel.png" alt="Redis channel with publisher and three subscribers."></p>
<p>Redis channels follow a topic-based publish-subscribe model.
Multiple publishers can send messages to a channel, and multiple subscribers can receive messages from that channel.</p>
<p>However, it's crucial to note that Redis channels do not store messages.
If there are no subscribers for a channel when a message is published, that message is immediately discarded.</p>
<p>Redis channels have an <strong>at-most-once delivery</strong> semantics.</p>
<h2>Practical Use Cases</h2>
<p>Given that Redis channels operate with <strong>at-most-once delivery</strong> (messages might be lost if there are no subscribers),
they are well-suited for scenarios where occasional message loss is acceptable and real-time or near-real-time communication is desired.</p>
<p>Here are a few possible use cases:</p>
<ul>
<li><strong>Social media feeds</strong>: Broadcasting new posts or updates to users.</li>
<li><strong>Live score updates</strong>: Sending live game scores or sports updates to subscribers.</li>
<li><strong>Chat applications</strong>: Delivering chat messages in real-time to active participants.</li>
<li><strong>Collaborative editing</strong>: Propagating changes in collaborative editing environments.</li>
<li><strong>Distributed cache updates</strong>: Invalidating cache entries across multiple servers when data changes. We'll cover this in detail later in the article.</li>
</ul>
<p>Redis channels aren't the best choice for critical data where message loss is unacceptable.
In such cases, you should consider a more reliable messaging system.</p>
<p>Let's see how we can use Redis channels in .NET.</p>
<h2>Pub/Sub With Redis Channels</h2>
<p>We will use the <code>StackExchange.Redis</code> library to send messages with Redis channels.</p>
<p>Let's start by installing it:</p>
<pre><code class="language-powershell">Install-Package StackExchange.Redis
</code></pre>
<p>You can run <a href="https://redis.io/">Redis</a> locally in a Docker container.
The default port is <code>6379</code>.</p>
<pre><code>docker run -it -p 6379:6379 redis
</code></pre>
<p>Here's a simple background service that'll act as our message <code>Producer</code>.</p>
<p>We're creating a <code>ConnectionMultiplexer</code> by connecting to our Redis instance.
This allows us to obtain an <code>ISubscriber</code> that we can use for pub/sub messaging.
The <code>ISubscriber</code> will enable us to publish a message to a channel by specifying the channel name.</p>
<pre><code class="language-csharp">public class Producer(ILogger&lt;Producer&gt; logger) : BackgroundService
{
    private static readonly string ConnectionString = &quot;localhost:6379&quot;;
    private static readonly ConnectionMultiplexer Connection =
        ConnectionMultiplexer.Connect(ConnectionString);

    private const string Channel = &quot;messages&quot;;

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        var subscriber = Connection.GetSubscriber();

        while (!stoppingToken.IsCancellationRequested)
        {
            var message = new Message(Guid.NewGuid(), DateTime.UtcNow);

            var json = JsonSerializer.Serialize(message);

            await subscriber.PublishAsync(Channel, json);

            logger.LogInformation(
                &quot;Sending message: {Channel} - {@Message}&quot;,
                message);

            await Task.Delay(5000, stoppingToken);
        }
    }
}
</code></pre>
<p>Let's also introduce a separate background service for consuming messages.</p>
<p>The <code>Consumer</code> connects to the same Redis instance and obtains an <code>ISubscriber</code>.
The <code>ISubscriber</code> exposes a <code>SubscribeAsync</code> method that we can use to subscribe to messages from a given channel.
This method accepts a callback delegate that we can use to handle the message.</p>
<pre><code class="language-csharp">public class Consumer(ILogger&lt;Consumer&gt; logger) : BackgroundService
{
    private static readonly string ConnectionString = &quot;localhost:6379&quot;;
    private static readonly ConnectionMultiplexer Connection =
        ConnectionMultiplexer.Connect(ConnectionString);

    private const string Channel = &quot;messages&quot;;

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        var subscriber = Connection.GetSubscriber();

        await subscriber.SubscribeAsync(Channel, (channel, message) =&gt;
        {
            var message = JsonSerializer.Deserialize&lt;Message&gt;(message);

            logger.LogInformation(
                &quot;Received message: {Channel} - {@Message}&quot;,
                channel,
                message);
        });
    }
}
</code></pre>
<p>Finally, here's what we get when we run both the <code>Producer</code> and <code>Consumer</code> services:</p>
<p><img src="/blogs/mnw_100/redis_pub_sub.gif" alt="Redis channels publish/subscribe demo."></p>
<h2>Cache Invalidation in Distributed Systems</h2>
<p>In a recent project, I tackled a common challenge in distributed systems: keeping the caches in sync.
We were using a two-level caching approach.
First, we had an in-memory cache on each web server for super-fast access.
Second, we had a shared Redis cache to avoid hitting our database too often.</p>
<p>The problem was that when data changed in the database, we needed a way to quickly tell all the web servers to clear their in-memory caches.
This is where Redis Pub/Sub came to the rescue.
We set up a Redis channel specifically for cache invalidation messages.</p>
<p>Each application would run a <code>CacheInvalidationBackgroundService</code> that subscribes to messages from the cache invalidation channel.</p>
<pre><code class="language-csharp">public class CacheInvalidationBackgroundService(
    IServiceProvider serviceProvider)
    : BackgroundService
{
    public const string Channel = &quot;cache-invalidation&quot;;

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        await subscriber.SubscribeAsync(Channel, (channel, key) =&gt;
        {
            var cache = serviceProvider.GetRequiredService&lt;IMemoryCache&gt;();

            cache.Remove(key);

            return Task.CompletedTask;
        });
    }
}
</code></pre>
<p>Whenever data changes in the database, we publish a message on this channel with the cache key of the updated data.
All the web servers are subscribed to this channel, so they instantly know to remove the old data from their in-memory caches.
Since the in-memory cache is wiped if the application isn't running, losing cache invalidation messages isn't a problem.
This keeps our caches consistent and ensures our users always see the most up-to-date information.</p>
<h2>In Summary</h2>
<p>Redis Pub/Sub is not a silver bullet for every messaging need, but its simplicity and speed make it a valuable tool.
Channels allow us to easily implement communication between loosely coupled components.</p>
<p>Redis channels have at-most-once delivery semantics, so they're best suited for cases where the occasional dropped message is acceptable.</p>
<p>I used it to solve the challenge of synchronizing caches across multiple servers.
This allowed our system to serve up-to-date data without sacrificing performance.</p>
<p><strong>P.S.</strong> When you're ready to dive deeper into creating message-driven systems, check out <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a>.
I have an entire module dedicated to building reliable distributed messaging and event-driven architecture.</p>
<p>Good luck out there, and see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_100.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Testing Modular Monoliths: System Integration Testing]]></title>
            <link>https://milanjovanovic.tech/blog/testing-modular-monoliths-system-integration-testing</link>
            <guid isPermaLink="false">testing-modular-monoliths-system-integration-testing</guid>
            <pubDate>Sat, 20 Jul 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[System integration testing is the perfect testing approach for modular monoliths. It's an approach to testing the interactions between various modules within a single system.]]></description>
            <content:encoded><![CDATA[<p>Modular monoliths strike a balance between the simplicity of monolithic architecture and the flexibility of microservices.
By breaking down applications into cohesive modules, modular monoliths enable easier development and maintenance.
However, they still have a single codebase and deployment unit.</p>
<p>A critical aspect of the success of a modular monolith is the interaction between its modules.</p>
<p><strong>System integration testing (SIT)</strong> is an approach to verifying the collaboration of these modules.
It involves testing the integration points and communication mechanisms between modules.</p>
<p>System integration testing allows us to validate the entire system's behavior.
This allows us to catch problems early before they become big headaches.</p>
<p>In this article, we'll learn how to test a modular monolith using system integration testing.</p>
<p>We'll look at real-world examples and discuss why this testing approach is useful.</p>
<h2>What Is System Integration Testing?</h2>
<p>System integration testing (SIT) is an approach to testing the interactions between various modules within a single system.
The system could contain many external services, which should also be included during testing.</p>
<p>System integration testing is the perfect testing approach for modular monoliths.
It allows you to mimic the real-world execution of your application (as you'll see later).</p>
<p>The key benefits of system integration testing are:</p>
<ul>
<li><strong>Detecting integration issues</strong>: Discover problems from module integrations that unit testing might miss.</li>
<li><strong>Validating business logic</strong>: Confirm that end-to-end business processes function correctly across modules.</li>
<li><strong>Ensuring data integrity</strong>: Verify that data flows accurately and consistently between modules.</li>
<li><strong>Improving system stability</strong>: Identify and resolve issues early, leading to a more reliable system.</li>
<li><strong>Building confidence</strong>: Ensure that the system is well-integrated and ready for deployment.</li>
</ul>
<h2>Modular Monolith System Example</h2>
<p>A modular monolith consists of multiple modules, each with a distinct responsibility.
Modules represent high-level components with well-defined boundaries.
The modules essentially group together related functionalities (use cases).
If you want to learn more about this software architecture, check out this <a href="what-is-a-modular-monolith"><strong>introduction to modular monoliths</strong></a>.</p>
<p>Let's consider a hypothetical ticketing system that consists of several modules:</p>
<ul>
<li><strong>Users Module</strong>: Handles user administration and authentication.</li>
<li><strong>Events Module</strong>: Manages events, scheduling, and ticket availability.</li>
<li><strong>Ticketing Module</strong>: Allows users to purchase tickets for various events.</li>
<li><strong>Attendance Module</strong>: Allows users to check into events using their tickets.</li>
</ul>
<figure>
  ![Modular monolith UML diagram.](/blogs/mnw_099/modular_monolith.png)
  <figcaption>
    Source: [Modular Monolith Architecture](/modular-monolith-architecture)
  </figcaption>
</figure>
<p>During system integration testing, we want to ensure that all these modules interact correctly and
fulfill the business requirements of the ticketing system.</p>
<h2>Testing Modular Monoliths: User Registration</h2>
<p>The modules in our ticketing system represent different <a href="monolith-to-microservices-how-a-modular-monolith-helps"><strong>bounded contexts</strong></a>.
The <strong>Users Module</strong> uses the term <code>User</code> as its core entity.
However, the <strong>Ticketing Module</strong> uses the term <code>Customer</code> because it's more aligned with its core responsibility of selling tickets.
Conceptually, both entities represent the same person within our system.</p>
<p>Here's the scenario we want to test:</p>
<ul>
<li>A user registers with our application through the <strong>Users Module</strong></li>
<li>The <strong>Users Module</strong> publishes an integration event to notify other modules</li>
<li>The <strong>Ticketing Module</strong> handles the integration event and creates a customer record</li>
</ul>
<p><img src="/blogs/mnw_099/user_registration.png" alt="User registration flow diagram."></p>
<p>If you haven't figured it out by now, our <a href="modular-monolith-communication-patterns"><strong>modules communicate asynchronously</strong></a> using messaging.</p>
<p>Why is this important for our tests?</p>
<p>There is a time delay from when something occurs in one module until the other modules are notified and handle the integration events.
Our system integration tests will have to take into account this delay.</p>
<p>Here's the test for this scenario:</p>
<pre><code class="language-csharp">public class RegisterUserTests : BaseIntegrationTest
{
    public RegisterUserTests(IntegrationTestWebAppFactory factory)
        : base(factory)
    {
    }

    [Fact]
    public async Task RegisterUser_Should_PropagateToTicketingModule()
    {
        // [Users Module] - Register user
        var command = new RegisterUserCommand(
            Faker.Internet.Email(),
            Faker.Internet.Password(6),
            Faker.Name.FirstName(),
            Faker.Name.LastName());

        Result&lt;Guid&gt; userResult = await Sender.Send(command);

        userResult.IsSuccess.Should().BeTrue();

        // [Ticketing Module] - Get customer
        Result&lt;CustomerResponse&gt; customerResult = await Poller.WaitAsync(
            TimeSpan.FromSeconds(15),
            async () =&gt;
            {
                var query = new GetCustomerQuery(userResult.Value);

                var customerResult = await Sender.Send(query);

                return customerResult;
            });

        // Assert
        customerResult.IsSuccess.Should().BeTrue();
        customerResult.Value.Should().NotBeNull();
    }
}
</code></pre>
<p>A lot is going on here, so let's unpack the steps:</p>
<ul>
<li>We're using the <code>WebApplicationFactory</code> to run an in-memory application instance</li>
<li>Any external dependencies run in a Docker container using <a href="testcontainers-integration-testing-using-docker-in-dotnet"><strong>Testcontainers</strong></a></li>
<li>We send the <code>RegisterUserCommand</code> to register a user in the <strong>Users Module</strong></li>
<li>Then, we have to poll the <strong>Ticketing Module</strong> by sending a <code>GetCustomerQuery</code></li>
<li>The <code>Poller</code> allows us to wait until the system eventually becomes consistent</li>
</ul>
<p>You can learn more about <a href="testcontainers-integration-testing-using-docker-in-dotnet"><strong>integration testing with Testcontainers in this article</strong></a>.</p>
<p>System integration tests take longer to execute than integration tests for one module.
This is a side effect of testing a bigger slice of the overall system in each test case.</p>
<p>The <code>Poller</code> class allows you to execute a delegate until you receive a successful result or a timeout occurs.
You can customize the timeout duration based on the scenario you are testing.</p>
<pre><code class="language-csharp">internal static class Poller
{
    private static readonly Error Timeout =
        Error.Failure(&quot;Poller.Timeout&quot;, &quot;The poller has time out&quot;);

    internal static async Task&lt;Result&lt;T&gt;&gt; WaitAsync&lt;T&gt;(
        TimeSpan timeout,
        Func&lt;Task&lt;Result&lt;T&gt;&gt;&gt; func)
    {
        using var timer = new PeriodicTimer(TimeSpan.FromSeconds(1));

        DateTime endTimeUtc = DateTime.UtcNow.Add(timeout);
        while (DateTime.UtcNow &lt; endTimeUtc &amp;&amp;
               await timer.WaitForNextTickAsync())
        {
            Result&lt;T&gt; result = await func();

            if (result.IsSuccess)
            {
                return result;
            }
        }

        return Result.Failure&lt;T&gt;(Timeout);
    }
}
</code></pre>
<h2>Testing Modular Monoliths: Adding Ticket to Cart</h2>
<p>Here's a more complex scenario testing adding a ticket to the customer's cart:</p>
<ul>
<li>A user registers with our application through the <strong>Users Module</strong></li>
<li>The <strong>Users Module</strong> publishes an integration event to notify other modules</li>
<li>The <strong>Ticketing Module</strong> handles the integration event and creates a customer record</li>
<li>The <strong>Ticketing Module</strong> creates a dummy event and adds a ticket to the customer's cart</li>
</ul>
<p>This scenario mimics a user registering with our system, finding a ticket they want to purchase, and adding it to their cart.
We can extend this scenario with more test cases, like ticket purchases.</p>
<pre><code class="language-csharp">public sealed class AddItemToCartTests : BaseIntegrationTest
{
    private const decimal Quantity = 10;

    public AddItemToCartTests(IntegrationTestWebAppFactory factory)
        : base(factory)
    {
    }

    [Fact]
    public async Task Customer_ShouldBeAbleTo_AddItemToCart()
    {
        // [Users Module] - Register user
        var command = new RegisterUserCommand(
            Faker.Internet.Email(),
            Faker.Internet.Password(6),
            Faker.Name.FirstName(),
            Faker.Name.LastName());

        Result&lt;Guid&gt; userResult = await Sender.Send(command);

        userResult.IsSuccess.Should().BeTrue();

        // [Ticketing Module] - Get customer
        Result&lt;CustomerResponse&gt; customerResult = await Poller.WaitAsync(
            TimeSpan.FromSeconds(15),
            async () =&gt;
            {
                var query = new GetCustomerQuery(userResult.Value);
                var customerResult = await Sender.Send(query);
                return customerResult;
            });

        customerResult.IsSuccess.Should().BeTrue();

        // [Ticketing Module] - Add item to cart
        CustomerResponse customer = customerResult.Value;
        var ticketTypeId = Guid.NewGuid();

        await Sender.CreateEventAsync(Guid.NewGuid(), ticketTypeId, Quantity);

        Result result = await Sender.Send(
            new AddItemToCartCommand(customer.Id, ticketTypeId, Quantity));

        // Assert
        result.IsSuccess.Should().BeTrue();
    }
}
</code></pre>
<p>What I like about system integration testing is that the test cases aren't complicated to write.
The tests execute the use cases of our system and verify the side effects.
Building your system around <a href="building-your-first-use-case-with-clean-architecture"><strong>use cases</strong></a> has a few advantages,
the main one being that you can focus on the core business logic.
But it also makes (integration) testing easier.
The use case runs by sending a command or a query.
We can execute the use cases in some logical order and check that we get the expected result.</p>
<h2>In Conclusion</h2>
<p>System integration testing is a crucial step in creating a successful modular monolith.
By testing how different modules work together, you can find and fix problems before they become bigger issues.
This makes your software more reliable and saves you time and resources in the long run.</p>
<p>If you want to dive deeper into building modular systems, check out <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a>.
There's an entire chapter dedicated to testing,
including advanced techniques like system integration testing, using Testcontainers for external services, and automated testing with CI/CD pipelines.</p>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_099.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Building Your First Use Case With Clean Architecture]]></title>
            <link>https://milanjovanovic.tech/blog/building-your-first-use-case-with-clean-architecture</link>
            <guid isPermaLink="false">building-your-first-use-case-with-clean-architecture</guid>
            <pubDate>Sat, 13 Jul 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[This is a question I often hear: how do I design my use case with Clean Architecture? In this article, we'll explore a practical example of how to apply Clean Architecture principles by building a user registration feature.]]></description>
            <content:encoded><![CDATA[<p>This is a question I often hear: how do I design my use case with Clean Architecture?</p>
<p>I understand the confusion.
Figuring out what to place in the Domain, Application, and Infrastructure layer can seem complicated.
If that's not enough, we also have to decide what makes up a use case and what should be abstracted away.</p>
<p>However, things become simpler if we adhere to the main rule in Clean Architecture — <strong>the Dependency Rule</strong>.
This rule states that source code dependencies can only point inwards.</p>
<p>In this newsletter, we'll explore a practical example of how to apply Clean Architecture principles by building a user registration feature.</p>
<h2>Clean Architecture</h2>
<p><a href="why-clean-architecture-is-great-for-complex-projects"><strong>Clean Architecture</strong></a> has emerged as a guiding principle for crafting maintainable, scalable, and testable applications.
At its core, Clean Architecture emphasizes the <strong>separation of concerns</strong> and the <strong>dependency rule</strong>.
The dependency rule dictates that dependencies should point inward toward higher-level modules.
By following this rule, you create a system where the core business logic of your application is decoupled from external dependencies.
This makes it more adaptable to changes and easier to test.</p>
<p><img src="/blogs/mnw_098/clean_architecture.png" alt="Clean Architecture diagram."></p>
<p>The Domain layer encapsulates enterprise-wide business rules.
It contains domain entities, where an entity is typically an object with methods.</p>
<p>The Application layer contains application-specific business rules and encapsulates all of the system's use cases.
A use case orchestrates the flow of data to and from the domain entities and calls the methods exposed by the entities
to achieve its goals.</p>
<p>The Infrastructure and Presentation layers deal with external concerns.
Here, you will implement any abstractions defined in the inner layers.</p>
<h2>Describing The Use Case</h2>
<p>What does it mean for a user to register with our application?
It means they reserve an email address (or username) to identify themselves and be able to interact with our system.
The user could provide other information, such as a first and last name, an address, and a phone number.</p>
<p>The first step in building any feature is clearly defining the desired result.</p>
<p>For user registration, this is what the required operations are:</p>
<ul>
<li>The user provides an email and password for registration</li>
<li>Verify that the email was not reserved previously by an existing account</li>
<li>Hash the password using some cryptographic hash function (e.g., SHA-256, SHA-512)</li>
<li>Store the user in the database and (optionally) return an access token to the client</li>
</ul>
<p>We could also consider any domain-specific rules or validations that we must enforce.
A good example is password strength, where we could implement minimum length and complexity requirements.</p>
<p>Now that we have our requirements let's see how to translate them into a use case.</p>
<h2>Implementing the Use Case</h2>
<p>With our requirements in place, we can now define the user registration use case.
In <a href="clean-architecture-and-the-benefits-of-structured-software-design"><strong>Clean Architecture</strong></a>, use cases live in the Application layer and orchestrate the interactions between domain entities and external dependencies.</p>
<p>Let's name our use case <code>RegisterUser</code>.
Its input will be a <code>RegistrationRequest</code> object containing the user's registration data,
and its output will be a <code>RegistrationResult</code> object indicating the outcome of the registration attempt.
Notice that we are using a feature-driven name for the use case.</p>
<p>What about any external dependencies?
If the use case needs to interact with an external system or infrastructure component, we abstract that behind an interface.
Remember, your application's core business logic should be decoupled from external dependencies.</p>
<p>The <code>RegisterUser</code> class will use dependency injection to get the necessary dependencies:</p>
<ul>
<li><code>IUserRepository</code>: An interface for accessing user data from the database.</li>
<li><code>IPasswordHasher</code>: An interface for hashing passwords securely.</li>
</ul>
<p>The <code>RegisterUser</code> use case will follow these steps:</p>
<ol>
<li>Validate input data</li>
<li>Check for existing <code>User</code></li>
<li>Hash the password</li>
<li>Create a new <code>User</code> entity</li>
<li>Save the <code>User</code> to the database</li>
<li>Return the result</li>
</ol>
<p>Finally, here's the code for our <code>RegisterUser</code> use case:</p>
<pre><code class="language-csharp">public class RegisterUser(
    IUserRepository userRepository,
    IPasswordHasher passwordHasher)
{
    public async Task&lt;RegistrationResult&gt; Handle(RegistrationRequest request)
    {
        // Validation omitted for brevity

        if (await userRepository.ExistsAsync(request.Email))
        {
            return RegistrationResult.EmailNotUnique;
        }

        var passwordHash = passwordHasher.Hash(request.Password);

        var user = User.Create(
            request.FirstName, request.LastName, request.Email, passwordHash);

        await userRepository.InsertAsync(user);

        return RegistrationResult.Success;
    }
}
</code></pre>
<p>A big benefit of this approach is that we can immediately write tests for the <code>RegisterUser</code> use case.
We can provide mocks for external dependencies in the tests.
We don't need the implementations to exist for this code to compile.
With mocks, we can test our business rules and validate our implementation.</p>
<p><strong>Action step</strong>: How would you extend the <code>RegisterUser</code> use case with more functionality?</p>
<p>Here are two examples:</p>
<ul>
<li>Adding an external identity provider</li>
<li>Implementing email verification</li>
</ul>
<h2>Where Clean Architecture Becomes Muddled</h2>
<p>By designing our application with Clean Architecture, we produce a system independent of external concerns.
We define abstractions in the Application layer and implement them in the Infrastructure layer.
So far, so good.</p>
<p>However, this doesn't mean you can disregard how you integrate with external dependencies.</p>
<p>In theory, we should be able to &quot;swap&quot; the implementation for any external concern and call it a day.
In practice, this couldn't be further from the truth.</p>
<p>Let me give you two practical examples using the user registration flow.</p>
<h3>Race Conditions</h3>
<p>The <code>RegisterUser</code> use case has a race condition.
Concurrent requests could pass the check for email uniqueness and proceed to register the user.</p>
<p>We could prevent this race condition by introducing a lock before checking for email uniqueness.
That way, only one request will pass the check and proceed to save the user in the database.</p>
<pre><code class="language-csharp">if (await userRepository.ExistsAsync(request.Email))
{
    return RegistrationResult.EmailNotUnique;
}
</code></pre>
<p>However, there is a much more elegant way to solve this.
We can introduce a unique index on the <code>Email</code> column in the database.
A unique index guarantees that only one transaction can write the unique value to the database.
The losing transaction will return an error.</p>
<p>We can handle this exception on the application side and return an appropriate error message to the user.
The <code>IUserRepository.InsertAsync</code> method implementation can encapsulate this logic.</p>
<h3>Changing Hash Functions</h3>
<p>Let's say we found a security flaw in the hash function used in the <code>IPasswordHasher</code> implementation.
So, we spend a few minutes switching to a more secure hash function.
The tests for the <code>RegisterUser</code> use case are all green, and everything seems fine.</p>
<p>The problem? All existing users can no longer log in to the system.</p>
<p>When an existing user tries to log in with their email and password, the new <code>IPasswordHasher.Hash</code> implementation
returns a different password hash from the one stored in the database.</p>
<p>The correct approach is to phase out the old password hash for existing users.
We can add a column in the database that says which hash function produced the hash.
We will verify the user's password using the correct hashing function during the login process.</p>
<p>If the user's password hash still uses the old hashing function, we will verify their password first.
Then, we can use the password (which we have in memory) to produce a hash using the new hash function.
We will store the hash in the database and update the hash function column to the new algorithm.</p>
<p>Slowly, we will phase out passwords using the old hash function.</p>
<h2>Conclusion</h2>
<p>I hope this was helpful in understanding how to apply Clean Architecture principles to a real-world scenario.
By focusing on the core business logic first (what it means for a user to register), we can define the requirements for our use case.
Translating these requirements into a series of steps within the use case is the easy part.</p>
<p>But Clean Architecture won't save you from bad engineering.
If you don't understand what you are abstracting away, it will become a problem in the long term.</p>
<p>If you want to go deeper, my flagship course, <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture</strong></a>,
takes the guesswork out of structuring your project the right way.
I share my entire framework for building robust applications from the ground up -
from building a rich domain model to creating use cases to getting your application ready for production.</p>
<p>And that's all for this week.</p>
<p>See you next Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_098.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Service Discovery in Microservices With .NET and Consul]]></title>
            <link>https://milanjovanovic.tech/blog/service-discovery-in-microservices-with-net-and-consul</link>
            <guid isPermaLink="false">service-discovery-in-microservices-with-net-and-consul</guid>
            <pubDate>Sat, 06 Jul 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Service discovery is a pattern that allows developers to use logical names to refer to external services, instead of physical IP addresses and ports. In this week's issue, we'll see how to implement service discovery in your .NET microservices with Consul.]]></description>
            <content:encoded><![CDATA[<p>Microservices have revolutionized how we build and scale applications.
By breaking down larger systems into smaller, independent services, we gain flexibility, agility, and the ability to adapt to changing requirements quickly.
However, microservices systems are also very dynamic.
Services can come and go, scale up or down, and even move around within your infrastructure.</p>
<p>This dynamic nature presents a significant challenge. How do your services find and communicate with each other reliably?</p>
<p>Hardcoding IP addresses and ports is a recipe for fragility.
If a service instance changes location or a new instance spins up, your entire system could grind to a halt.</p>
<p>Service discovery acts as a central directory for your microservices.
It provides a mechanism for services to register themselves and discover the locations of other services.</p>
<p>In this week's issue, we'll see how to implement service discovery in your .NET microservices with Consul.</p>
<h2>What is Service Discovery?</h2>
<p>Service discovery is a pattern that allows developers to use logical names to refer to external services instead of physical IP addresses and ports.
It provides a centralized location for services to register themselves.
Clients can query the service registry to find out the service's physical address.
This is a common pattern in large-scale distributed systems, such as Netflix and Amazon.</p>
<p>Here's what the service discovery flow looks like:</p>
<ol>
<li>The service will register itself with the service registry</li>
<li>The client must query the service registry to get the physical address</li>
<li>The client sends the request to the service using the resolved physical address</li>
</ol>
<p><img src="/blogs/mnw_097/service_discovery_flow.png" alt="Service discovery flow."></p>
<p>The same concept applies when we have multiple services we want to call.
Each service would register itself with the service registry.
The client uses a logical name to reference a service and resolves the physical address from the service registry.</p>
<p><img src="/blogs/mnw_097/service_discovery_microservices.png" alt="Service discovery with multiple microservices."></p>
<p>The most popular solutions for service discovery are Netflix <a href="https://github.com/Netflix/eureka">Eureka</a> and HashiCorp <a href="https://www.consul.io/">Consul</a>.</p>
<p>There is also a lightweight solution from Microsoft in the <code>Microsoft.Extensions.ServiceDiscovery</code> library.
It uses application settings to resolve the physical addresses for services, so some manual work is still required.
However, you can store service locations in <a href="https://azure.microsoft.com/en-us/products/app-configuration">Azure App Configuration</a> for a centralized service registry.
I will explore this service discovery library in some future articles.</p>
<p>But now I want to show you how to integrate Consul with .NET applications.</p>
<h2>Setting Up the Consul Server</h2>
<p>The simplest way to run the Consul server locally is using a Docker container.
You can create a container instance of the <code>hashicorp/consul</code> image.</p>
<p>Here's an example of configuring the Consul service as part of the <code>docker-compose</code> file:</p>
<pre><code class="language-yml">consul:
  image: hashicorp/consul:latest
  container_name: Consul
  ports:
    - '8500:8500'
</code></pre>
<p>If you navigate to <code>localhost:8500</code>, you will be greeted by the Consul Dashboard.</p>
<p><img src="/blogs/mnw_097/consul_dashboard.png" alt="Consul dashboard."></p>
<p>Now, let's see how to register our services with Consul.</p>
<h2>Service Registration in .NET With Consul</h2>
<p>We'll use the <a href="https://docs.steeltoe.io/api/v3/discovery/">Steeltoe Discovery</a> library to implement service discovery with Consul.
The Consul client implementation lets your applications register services with a Consul server and discover services registered by other applications.</p>
<p>Let's install the <code>Steeltoe.Discovery.Consul</code> library:</p>
<pre><code class="language-powershell">Install-Package Steeltoe.Discovery.Consul
</code></pre>
<p>We have to configure some services by calling <code>AddServiceDiscovery</code> and explicitly configuring the Consul service discovery client.
The alternative is calling <code>AddDiscoveryClient</code> which uses reflection at runtime to determine which service registry is available.</p>
<pre><code class="language-csharp">using Steeltoe.Discovery.Client;
using Steeltoe.Discovery.Consul;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddServiceDiscovery(o =&gt; o.UseConsul());

var app = builder.Build();

app.Run();
</code></pre>
<p>Finally, our service can register with Consul by configuring the logical service name through application settings.
When the application starts, the <code>reporting-service</code> logical name will be added to the Consul service registry.
Consul will store the respective physical address of this service.</p>
<pre><code class="language-json">{
  &quot;Consul&quot;: {
    &quot;Host&quot;: &quot;localhost&quot;,
    &quot;Port&quot;: 8500,
    &quot;Discovery&quot;: {
      &quot;ServiceName&quot;: &quot;reporting-service&quot;,
      &quot;Hostname&quot;: &quot;reporting-api&quot;,
      &quot;Port&quot;: 8080
    }
  }
}
</code></pre>
<p>When we start the application and open the Consul dashboard, we should be able to see the <code>reporting-service</code> and its respective physical address.</p>
<p><img src="/blogs/mnw_097/consul_dashboard_with_service.png" alt="Consul dashboard with registered service."></p>
<h2>Using Service Discovery</h2>
<p>We can use service discovery when making HTTP calls with an <code>HttpClient</code>.
Service discovery allows us to use a logical name for the service we want to call.
When sending a network request, the service discovery client will replace the logical name with a correct physical address.</p>
<p>In this example, we're configuring the base address of the <code>ReportingServiceClient</code> typed client to <code>http://reporting-service</code>
and adding service discovery by calling <code>AddServiceDiscovery</code>.</p>
<p>Load balancing is an optional step, and we can configure it by calling <code>AddRoundRobinLoadBalancer</code> or <code>AddRandomLoadBalancer</code>.
You can also configure a custom load balancing strategy by providing an <code>ILoadBalancer</code> implementation.</p>
<pre><code class="language-csharp">builder.Services
    .AddHttpClient&lt;ReportingServiceClient&gt;(client =&gt;
    {
        client.BaseAddress = new Uri(&quot;http://reporting-service&quot;);
    })
    .AddServiceDiscovery()
    .AddRoundRobinLoadBalancer();
</code></pre>
<p>We can use the <code>ReportingServiceClient</code> typed client like a regular <code>HttpClient</code> to make requests.
The service discovery client sends the request to the external service's IP address.</p>
<pre><code class="language-csharp">app.MapGet(&quot;articles/{id}/report&quot;,
    async (Guid id, ReportingServiceClient client) =&gt;
    {
        var response = await client
            .GetFromJsonAsync&lt;Response&gt;($&quot;api/reports/article/{id}&quot;);

        return response;
    });
</code></pre>
<h2>Takeaway</h2>
<p>Service discovery simplifies the management of microservices by automating service registration and discovery.
This eliminates the need for manual configuration updates, reducing the risk of errors.</p>
<p>Services can discover each other's locations on demand, ensuring that communication channels remain open even as the service landscape evolves.
By enabling services to discover alternative service instances in case of outages or failures, service discovery enhances the overall resilience of the microservices system.</p>
<p>Mastering service discovery gives you a powerful tool to build modern distributed applications.</p>
<p>You can grab the <a href="https://github.com/m-jovanovic/service-discovery-consul">source code for this example here</a>.</p>
<p>Thanks for reading, and I'll see you next week!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_097.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Flexible PDF Reporting in .NET Using Razor Views]]></title>
            <link>https://milanjovanovic.tech/blog/flexible-pdf-reporting-in-net-using-razor-views</link>
            <guid isPermaLink="false">flexible-pdf-reporting-in-net-using-razor-views</guid>
            <pubDate>Sat, 29 Jun 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[In this article, we'll explore the power of using Razor views for flexible PDF reporting in .NET. We'll see how to create report templates with Razor views, convert them to HTML, and then transform that HTML into beautifully formatted PDF documents.]]></description>
            <content:encoded><![CDATA[<p>I'll never forget when I was working on a project that required generating weekly sales reports for a client.
The initial solution involved a clunky process of exporting data, manipulating it in spreadsheets, and manually creating PDFs.
It was tedious, error-prone, and it sucked up way too much of my time.
I knew there had to be a better way.</p>
<p>That's when I discovered the power of combining Razor views with HTML-to-PDF conversion.
You have more control over formatting the document.
You can use modern CSS to style the HTML markup, which will be applied when exporting to a PDF document.
It's also simple to implement in <a href="http://ASP.NET">ASP.NET</a> Core.</p>
<p>Here's what we'll cover:</p>
<ul>
<li>Understanding Razor views</li>
<li>Converting Razor views to HTML</li>
<li>HTML to PDF conversion in .NET</li>
<li>Putting it all together with Minimal APIs</li>
</ul>
<p>Let's dive in!</p>
<h2>Razor Views</h2>
<p>Razor <a href="https://learn.microsoft.com/en-us/aspnet/core/mvc/views">views</a>
are an HTML template with embedded <a href="https://learn.microsoft.com/en-us/aspnet/core/mvc/views/razor">Razor</a> markup.
Razor allows you to write and execute .NET code inside a web page.
Views have a special <code>.cshtml</code> file extension.
They're commonly used in <a href="https://learn.microsoft.com/en-us/aspnet/core/mvc/overview">ASP.NET Core MVC</a>,
<a href="https://learn.microsoft.com/en-us/aspnet/core/razor-pages">Razor Pages</a>,
and <a href="https://learn.microsoft.com/en-us/aspnet/core/blazor">Blazor</a>.</p>
<p>However, you can define Razor views in a class library or <a href="http://ASP.NET">ASP.NET</a> Core Web API project.</p>
<p>You can use the <code>Model</code> object to pass in data to a Razor view.
Inside the <code>.cshtml</code> file, you'll specify the model type with the <code>@model</code> keyword.
In the example below, I'm specifying that the <code>Invoice</code> class is the model for this view.
You can access the model instance with the <code>Model</code> property in the view.</p>
<p>This is the <code>InvoiceReport.cshtml</code> view we'll use to generate a PDF invoice.</p>
<p>You can write CSS in the Razor view inline or reference a stylesheet.
I'm using the <a href="https://tailwindcss.com/">Tailwind CSS</a> utility framework, which uses inline CSS.
I usually delegate this to a front-end engineer on my team so they can stylize the report as needed.</p>
<pre><code class="language-csharp">@using System.Globalization
@using HtmlToPdf.Contracts

@model HtmlToPdf.Contracts.Invoice

@{
    IFormatProvider cultureInfo = CultureInfo.CreateSpecificCulture(&quot;en-US&quot;);
    var subtotal = Model.LineItems.Sum(li =&gt; li.Price * li.Quantity).ToString(&quot;C&quot;, cultureInfo);
    var total = Model.LineItems.Sum(li =&gt; li.Price * li.Quantity).ToString(&quot;C&quot;, cultureInfo);
}

&lt;script src=&quot;https://cdn.tailwindcss.com&quot;&gt;&lt;/script&gt;

&lt;div class=&quot;min-w-7xl flex flex-col bg-gray-200 space-y-4 p-10&quot;&gt;
    &lt;h1 class=&quot;text-2xl font-semibold&quot;&gt;Invoice #@Model.Number&lt;/h1&gt;

    &lt;p&gt;Issued date: @Model.IssuedDate.ToString(&quot;dd/MM/yyyy&quot;)&lt;/p&gt;
    &lt;p&gt;Due date: @Model.DueDate.ToString(&quot;dd/MM/yyyy&quot;)&lt;/p&gt;

    &lt;div class=&quot;flex justify-between space-x-4&quot;&gt;
        &lt;div class=&quot;bg-gray-100 rounded-lg flex flex-col space-y-1 p-4 w-1/2&quot;&gt;
            &lt;p class=&quot;font-medium&quot;&gt;Seller:&lt;/p&gt;
            &lt;p&gt;@Model.SellerAddress.CompanyName&lt;/p&gt;
            &lt;p&gt;@Model.SellerAddress.Street&lt;/p&gt;
            &lt;p&gt;@Model.SellerAddress.City&lt;/p&gt;
            &lt;p&gt;@Model.SellerAddress.State&lt;/p&gt;
            &lt;p&gt;@Model.SellerAddress.Email&lt;/p&gt;
        &lt;/div&gt;
        &lt;div class=&quot;bg-gray-100 rounded-lg flex flex-col space-y-1 p-4 w-1/2&quot;&gt;
            &lt;p class=&quot;font-medium&quot;&gt;Bill to:&lt;/p&gt;
            &lt;p&gt;@Model.CustomerAddress.CompanyName&lt;/p&gt;
            &lt;p&gt;@Model.CustomerAddress.Street&lt;/p&gt;
            &lt;p&gt;@Model.CustomerAddress.City&lt;/p&gt;
            &lt;p&gt;@Model.CustomerAddress.State&lt;/p&gt;
            &lt;p&gt;@Model.CustomerAddress.Email&lt;/p&gt;
        &lt;/div&gt;
    &lt;/div&gt;

    &lt;div class=&quot;flex flex-col bg-white rounded-lg p-4 space-y-2&quot;&gt;
        &lt;h2 class=&quot;text-xl font-medium&quot;&gt;Items:&lt;/h2&gt;
        &lt;div class=&quot;&quot;&gt;
            &lt;div class=&quot;flex space-x-4 font-medium&quot;&gt;
                &lt;p class=&quot;w-10&quot;&gt;#&lt;/p&gt;
                &lt;p class=&quot;w-52&quot;&gt;Name&lt;/p&gt;
                &lt;p class=&quot;w-20&quot;&gt;Price&lt;/p&gt;
                &lt;p class=&quot;w-20&quot;&gt;Quantity&lt;/p&gt;
            &lt;/div&gt;

            @foreach ((int index, LineItem item) in Model.LineItems.Select((li, i) =&gt; (i + 1, li)))
            {
                &lt;div class=&quot;flex space-x-4&quot;&gt;
                    &lt;p class=&quot;w-10&quot;&gt;@index&lt;/p&gt;
                    &lt;p class=&quot;w-52&quot;&gt;@item.Name&lt;/p&gt;
                    &lt;p class=&quot;w-20&quot;&gt;@item.Price.ToString(&quot;C&quot;, cultureInfo)&lt;/p&gt;
                    &lt;p class=&quot;w-20&quot;&gt;@item.Quantity.ToString(&quot;N2&quot;)&lt;/p&gt;
                &lt;/div&gt;
            }
        &lt;/div&gt;
    &lt;/div&gt;

    &lt;div class=&quot;flex flex-col items-end bg-gray-50 space-y-2 p-4 rounded-lg&quot;&gt;
        &lt;p&gt;Subtotal: @subtotal&lt;/p&gt;
        &lt;p&gt;Total: &lt;span class=&quot;font-semibold&quot;&gt;@total&lt;/span&gt;&lt;/p&gt;
    &lt;/div&gt;
&lt;/div&gt;
</code></pre>
<h2>Converting Razor Views to HTML</h2>
<p>The next thing we'll need is a way to convert the Razor view into HTML.
We can do this with the <a href="https://github.com/soundaranbu/Razor.Templating.Core"><code>Razor.Templating.Core</code></a> library.
It provides a simple API to render a <code>.cshtml</code> file into a <code>string</code>.</p>
<pre><code class="language-powershell">Install-Package Razor.Templating.Core
</code></pre>
<p>You can use the <code>RazorTemplateEngine</code> static class to call the <code>RenderAsync</code> method.
It accepts the path to the Razor view and the model instance that will be passed to the view.</p>
<p>Here's what that will look like:</p>
<pre><code class="language-csharp">Invoice invoice = invoiceFactory.Create();

string html = await RazorTemplateEngine.RenderAsync(
    &quot;Views/InvoiceReport.cshtml&quot;,
    invoice);
</code></pre>
<p>Alternatively, you can use the <code>IRazorTemplateEngine</code> instead of the static class.
In that case, you must call <code>AddRazorTemplating</code> to register the required services with DI.
This is also required if you want to use dependency injection inside the Razor views with <code>@inject</code>.
It's recommended that you call <code>AddRazorTemplating</code> after registering all dependencies.</p>
<pre><code class="language-csharp">services.AddRazorTemplating();
</code></pre>
<h2>HTML to PDF conversion</h2>
<p>Now that we've converted our Razor view into HTML, we can use it to generate a PDF report.
Many libraries offer this functionality.
The library I've used most often is <a href="https://ironpdf.com/">IronPDF</a>.
It's a paid library (and well worth it), but I know developers also want free options, so I'll list some alternatives at the end.</p>
<p>We can use IronPDF's <code>ChromePdfRenderer</code>, which uses an embedded Chrome browser.
The renderer exposes the <code>RenderHtmlAsPdf</code> method, which generates a <code>PdfDocument</code>.
Once you have the document, you can store it on the file system or export it as binary data.</p>
<pre><code class="language-csharp">var renderer = new ChromePdfRenderer();

using var pdfDocument = renderer.RenderHtmlAsPdf(html);

pdfDocument.SaveAs($&quot;invoice-{invoice.Number}.pdf&quot;);
</code></pre>
<p>If you're looking for free options, check out <a href="https://github.com/hardkoded/puppeteer-sharp">Puppeteer Sharp</a>.
It's a .NET port of the <a href="https://github.com/puppeteer/puppeteer">Puppeteer</a> library, which allows you to run a headless Chrome browser.</p>
<p>Another (conditionally) free option to consider is <a href="https://www.nuget.org/packages/NReco.PdfGenerator/">NReco.PdfGenerator</a>.
However, it's only free for single-server deployments.</p>
<h2>Putting It All Together</h2>
<p>Let's use everything we discussed to create a Minimal API endpoint to generate an invoice PDF report and return it as a file response.
Here's the code snippet:</p>
<pre><code class="language-csharp">app.MapGet(&quot;invoice-report&quot;, async (InvoiceFactory invoiceFactory) =&gt;
{
    Invoice invoice = invoiceFactory.Create();

    var html = await RazorTemplateEngine.RenderAsync(
        &quot;Views/InvoiceReport.cshtml&quot;,
        invoice);

    var renderer = new ChromePdfRenderer();

    using var pdfDocument = renderer.RenderHtmlAsPdf(html);

    return Results.File(
        pdfDocument.BinaryData,
        &quot;application/pdf&quot;,
        $&quot;invoice-{invoice.Number}.pdf&quot;);
});
</code></pre>
<p>This is what the generated PDF report looks like:</p>
<p><img src="/blogs/mnw_096/invoice.png" alt="Invoice report."></p>
<p>You can grab the source code for this sample <a href="https://github.com/m-jovanovic/razor-view-html-to-pdf">here</a>.
Feel free to try out a different library for HTML to PDF conversion.</p>
<h2>Summary</h2>
<p>In this article, we've explored the power of using Razor views for flexible PDF reporting in .NET.
We've seen how to create report templates with Razor views, convert them to HTML, and then transform that HTML into beautifully formatted PDF documents.</p>
<p>Whether you need to generate invoices, sales reports, or any other kind of structured document, this approach offers a simple and customizable solution.</p>
<p>Here's what you can explore next:</p>
<ul>
<li><a href="https://youtu.be/XYdcdVWsWos">PDF Reporting in .NET Using IronPDF and Razor Views</a></li>
<li><a href="how-to-easily-create-pdf-documents-in-aspnetcore">Creating PDF Reports With QuestPDF</a></li>
</ul>
<p>That's all for this week. Stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_096.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[What You Need To Know About EF Core Bulk Updates]]></title>
            <link>https://milanjovanovic.tech/blog/what-you-need-to-know-about-ef-core-bulk-updates</link>
            <guid isPermaLink="false">what-you-need-to-know-about-ef-core-bulk-updates</guid>
            <pubDate>Sat, 22 Jun 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[EF Core 7 introduced two powerful new methods, ExecuteUpdate and ExecuteDelete. However, there's an important caveat: these bulk operations bypass the EF Core Change Tracker.]]></description>
            <content:encoded><![CDATA[<p>When you're dealing with thousands or even millions of records, efficiency is king.
That's where <a href="how-to-use-the-new-bulk-update-feature-in-ef-core-7"><strong>EF Core bulk update</strong></a> capabilities come into play.</p>
<p>EF Core 7 introduced two powerful new methods, <code>ExecuteUpdate</code> and <code>ExecuteDelete</code>.
They're designed to simplify bulk updates in your database.
Both methods have their respective async overloads - <code>ExecuteUpdateAsync</code> and <code>ExecuteDeleteAsync</code>.
<a href="https://learn.microsoft.com/en-us/ef/core/saving/execute-insert-update-delete">EF bulk updates</a> offer significant performance advantages over traditional approaches.</p>
<p>However, there's an <strong>important caveat</strong>: these bulk operations bypass the EF Core <strong>Change Tracker</strong>.
This disconnect can lead to unexpected behavior if you're not aware of it.</p>
<p>In this week's issue, we'll dive into the details of bulk updates in EF Core.</p>
<h2>Understanding the EF Core ChangeTracker</h2>
<p>When you load entities from the database with EF Core, the <code>ChangeTracker</code> starts tracking them.
As you update properties, delete entities, or add new ones, the <code>ChangeTracker</code> records these changes.</p>
<pre><code class="language-csharp">using (var context = new AppDbContext())
{
    // Load a product
    var product = context.Products.FirstOrDefault(p =&gt; p.Id == 1);
    product.Price = 99.99; // Modify a property

    // At this point, the ChangeTracker knows that 'product' has been modified

    // Add a new product
    var newProduct = new Product { Name = &quot;New Gadget&quot;, Price = 129.99 };
    context.Products.Add(newProduct);

    // Delete a product
    context.Products.Remove(product);

    context.SaveChanges(); // Persist all changes to the database
}
</code></pre>
<p>When you call <code>SaveChanges</code>, EF Core uses the <code>ChangeTracker</code> to determine which SQL commands to execute.
This ensures that the database is perfectly synchronized with your modifications.
The <code>ChangeTracker</code> acts as a bridge between your in-memory object model and your database.</p>
<p>If you're already familiar with how EF Core works, this serves mostly as a reminder.</p>
<h2>Bulk Updates and the ChangeTracker Disconnect</h2>
<p>Now, let's focus on how <a href="how-to-use-the-new-bulk-update-feature-in-ef-core-7"><strong>bulk updates in EF Core</strong></a>
interact with the <code>ChangeTracker</code> - or rather, how they don't interact with it.
This design decision might seem counterintuitive, but there's a solid reason behind it: <strong>performance</strong>.</p>
<p>By directly executing SQL statements against the database, EF Core eliminates the overhead of tracking individual entity modifications.</p>
<pre><code class="language-csharp">using (var context = new AppDbContext())
{
    // Increase price of all electronics by 10%
    context.Products
        .Where(p =&gt; p.Category == &quot;Electronics&quot;)
        .ExecuteUpdate(
            s =&gt; s.SetProperty(p =&gt; p.Price, p =&gt; p.Price * 1.10));

    // In-memory Product instances with Category == &quot;Electronics&quot;
    // will STILL have their old price
}
</code></pre>
<p>In this example, we're increasing the price of all products in the <code>Electronics</code> category by 10%.
The <code>ExecuteUpdate</code> method efficiently translates the operation into a single SQL <code>UPDATE</code> statement.</p>
<pre><code class="language-sql">UPDATE [p]
SET [p].[Price] = [p].[Price] * 1.10
FROM [Products] as [p];
</code></pre>
<p>However, if you inspect the <code>Product</code> instances that EF Core has already loaded into memory, you'll find that their <code>Price</code> properties haven't changed.
This might seem surprising if you aren't aware of how bulk updates interact with the change tracker.</p>
<p>Everything we discussed up to this point also applies to the <code>ExecuteDelete</code> method.</p>
<p><a href="how-to-use-ef-core-interceptors"><strong>EF Core interceptors</strong></a> do not trigger for <code>ExecuteUpdate</code> and <code>ExecuteDelete</code> operations.
If you need to track or modify bulk update operations, you can create database triggers that fire whenever a relevant table is updated or deleted.
This allows you to log details and perform additional actions.</p>
<h2>The Problem: Maintaining Consistency</h2>
<p>If <code>ExecuteUpdate</code> completes successfully, the changes are directly committed to the database.
This is because bulk operations bypass the <code>ChangeTracker</code> and don't participate in the usual transaction managed by <code>SaveChanges</code>.</p>
<p>If <code>SaveChanges</code> subsequently fails due to an error (e.g., validation error, database constraint violation, connection issue),
you'll be in an inconsistent state.
The changes made by <code>ExecuteUpdate</code> are already persisted.
Any changes made &quot;in memory&quot; are lost.</p>
<p>The most reliable way to ensure consistency is to wrap both <code>ExecuteUpdate</code> and the operations that lead to <code>SaveChanges</code> in a transaction:</p>
<pre><code class="language-csharp">using (var context = new AppDbContext())
using (var transaction = context.Database.BeginTransaction())
{
    try
    {
        context.Products
            .Where(p =&gt; p.Category == &quot;Electronics&quot;)
            .ExecuteUpdate(
                s =&gt; s.SetProperty(p =&gt; p.Price, p =&gt; p.Price * 1.10));

        // ... other operations that modify entities

        context.SaveChanges();

        transaction.Commit();
    }
    catch (Exception ex)
    {
        // You could also let the transaction go out of scope.
        // This would automatically rollback any changes.
        transaction.Rollback();

        // Proceed to handle the exception...
    }
}
</code></pre>
<p>If <code>SaveChanges</code> fails, the transaction will be rolled back, reverting the changes made by both <code>ExecuteUpdate</code> and any other operations within the transaction.
This keeps your database in a consistent state.</p>
<h2>Summary</h2>
<p>EF Core bulk update features, <code>ExecuteUpdate</code> and <code>ExecuteDelete</code>, are invaluable tools for optimizing performance.
By bypassing the <code>ChangeTracker</code> and executing raw SQL directly, they deliver significant speed improvements compared to traditional methods.</p>
<p>However, it's crucial to be mindful of the potential pitfalls associated with this approach.
The disconnect between in-memory entities and the database state can lead to unexpected results if not handled correctly.</p>
<p>My rule of thumb is to create an explicit <a href="working-with-transactions-in-ef-core"><strong>database transaction</strong></a> when I want to make additional entity changes.
We can be confident that all the changes will persist in the database or none of them will.</p>
<p>I hope this was helpful, and I'll see you next week.</p>
<p><strong>P.S.</strong> Get the <a href="https://github.com/m-jovanovic/ef-bulk-updates">source code</a> and try out the examples from this issue.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_095.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[From Transaction Scripts to Domain Models: A Refactoring Journey]]></title>
            <link>https://milanjovanovic.tech/blog/from-transaction-scripts-to-domain-models-a-refactoring-journey</link>
            <guid isPermaLink="false">from-transaction-scripts-to-domain-models-a-refactoring-journey</guid>
            <pubDate>Sat, 15 Jun 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Transaction Scripts organizes business logic by procedures where each procedure handles a single request from the presentation. We will explore when you should consider introducing a Domain Model.]]></description>
            <content:encoded><![CDATA[<p>I once led the development of a fitness tracking app.
We started with Transaction Scripts to handle features like workout creation and exercise logging.
It was a simple and effective approach for the app's early stages.</p>
<p>However, as we added more complex features, our business logic became bloated.
New business rules intertwined with existing logic, making the code difficult to maintain.
Each change could introduce unintended consequences.</p>
<p>We solved our problem by introducing a Domain Model.
The domain model shifts the focus from procedures to domain objects.
Our code became more expressive, easier to reason about, and less prone to errors.</p>
<p>This experience taught me a valuable lesson, that I want to share in this newsletter.</p>
<h2>Transaction Script</h2>
<p>At its core, business applications operate through distinct interactions (transactions) with their users.
These transactions can range from simple data retrieval to complex operations involving multiple validations, calculations, and updates to the system's database.</p>
<p>The Transaction Script pattern provides a simple way to encapsulate the logic behind each transaction.
It organizes all the necessary steps, from data access to business rules, into a single, self-contained procedure.</p>
<blockquote>
<p>Organizes business logic by procedures where each procedure handles a single request from the presentation.</p>
</blockquote>
<p><em>— <a href="https://martinfowler.com/eaaCatalog/transactionScript.html">Transaction Script</a>, Patterns of Enterprise Application Architecture</em></p>
<p>Here's an example of adding exercises to a workout:</p>
<pre><code class="language-csharp">internal sealed class AddExercisesCommandHandler(
    IWorkoutRepository workoutRepository,
    IUnitOfWork unitOfWork)
    : ICommandHandler&lt;AddExercisesCommand&gt;
{
    public async Task&lt;Result&gt; Handle(
        AddExercisesCommand request,
        CancellationToken cancellationToken)
    {
        Workout? workout = await workoutRepository.GetByIdAsync(
            request.WorkoutId,
            cancellationToken);

        if (workout is null)
        {
            return Result.Failure(WorkoutErrors.NotFound(request.WorkoutId));
        }

        List&lt;Error&gt; errors = [];
        foreach (ExerciseRequest exerciseDto in request.Exercises)
        {
            if (exerciseDto.TargetType == TargetType.Distance &amp;&amp;
                exerciseDto.DistanceInMeters is null)
            {
                errors.Add(ExerciseErrors.MissingDistance);

                continue;
            }

            if (exerciseDto.TargetType == TargetType.Time &amp;&amp;
                exerciseDto.DurationInSeconds is null)
            {
                errors.Add(ExerciseErrors.MissingDuration);

                continue;
            }

            var exercise = new Exercise(
                Guid.NewGuid(),
                workout.Id,
                exerciseDto.ExerciseType,
                exerciseDto.TargetType,
                exerciseDto.DistanceInMeters,
                exerciseDto.DurationInSeconds);

            workouts.Exercises.Add(exercise);
        }

        if (errors.Count != 0)
        {
            return Result.Failure(new ValidationError(errors.ToArray()));
        }

        await unitOfWork.SaveChangesAsync(cancellationToken);

        return Result.Success();
    }
}
</code></pre>
<p>There isn't much logic here. We're just checking whether the workout exists and whether the exercises are valid.</p>
<p>What happens when we start to add more logic?</p>
<p>Let's add another business rule.
We must enforce a limit on the number of exercises allowed in a single workout (e.g., no more than 10 exercises).</p>
<pre><code class="language-csharp">internal sealed class AddExercisesCommandHandler(
    IWorkoutRepository workoutRepository,
    IUnitOfWork unitOfWork)
    : ICommandHandler&lt;AddExercisesCommand&gt;
{
    public async Task&lt;Result&gt; Handle(
        AddExercisesCommand request,
        CancellationToken cancellationToken)
    {
        Workout? workout = await workoutRepository.GetByIdAsync(
            request.WorkoutId,
            cancellationToken);

        if (workout is null)
        {
            return Result.Failure(WorkoutErrors.NotFound(request.WorkoutId));
        }

        List&lt;Error&gt; errors = [];
        foreach (ExerciseRequest exerciseDto in request.Exercises)
        {
            if (exerciseDto.TargetType == TargetType.Distance &amp;&amp;
                exerciseDto.DistanceInMeters is null)
            {
                errors.Add(ExerciseErrors.MissingDistance);

                continue;
            }

            if (exerciseDto.TargetType == TargetType.Time &amp;&amp;
                exerciseDto.DurationInSeconds is null)
            {
                errors.Add(ExerciseErrors.MissingDuration);

                continue;
            }

            var exercise = new Exercise(
                Guid.NewGuid(),
                workout.Id,
                exerciseDto.ExerciseType,
                exerciseDto.TargetType,
                exerciseDto.DistanceInMeters,
                exerciseDto.DurationInSeconds);

            workouts.Exercises.Add(exercise);

            if (workouts.Exercise.Count &gt; 10)
            {
                return Result.Failure(
                    WorkoutErrors.MaxExercisesReached(workout.Id));
            }
        }

        if (errors.Count != 0)
        {
            return Result.Failure(new ValidationError(errors.ToArray()));
        }

        await unitOfWork.SaveChangesAsync(cancellationToken);

        return Result.Success();
    }
}
</code></pre>
<p>We can continue adding more business logic to the transaction script.
For example, we can introduce exercise type restrictions or enforce a specific exercise order.
You can imagine how the complexity will keep increasing over time.</p>
<p>Another concern is code duplication between transaction scripts.
This could happen if we need similar logic in multiple transaction scripts.
You may be tempted to solve this by calling one transaction script from the other, but this will introduce a different set of problems.</p>
<p>So, how can we solve these problems?</p>
<h2>Refactoring to Domain Model</h2>
<p>What is a domain model?</p>
<blockquote>
<p>An object model of the domain that incorporates both behavior and data.</p>
</blockquote>
<p><em>— <a href="https://martinfowler.com/eaaCatalog/domainModel.html">Domain Model</a>, Patterns of Enterprise Application Architecture</em></p>
<p>A domain model lets you encapsulate domain logic (behavior) and state changes (data) inside an object.
In Domain-Driven Design terminology, we would call this an aggregate.</p>
<p>An aggregate in DDD is a cluster of objects treated as a single unit for data changes.
The aggregate represents a consistency boundary.
It helps maintain consistency by ensuring that certain invariants always hold true for the entire aggregate.
In our workout example, the <code>Workout</code> class can be treated as an aggregate root that encompasses all the exercises within it.</p>
<p>What does this have to do with a transaction script?</p>
<p>We can move the domain logic and state changes from the transaction script into the aggregate.
This is often called &quot;pushing logic down&quot; into the domain.</p>
<p>Here's what the domain model will look like when we extract the domain logic:</p>
<pre><code class="language-csharp">public sealed class Workout
{
    private readonly List&lt;Exercise&gt; _exercises = [];

    // Omitting the constructor and other propreties for brevity.

    public Result AddExercises(ExerciseModel[] exercises)
    {
        List&lt;Error&gt; errors = [];
        foreach (var exerciseModel in exercises)
        {
            if (exerciseModel.TargetType == TargetType.Distance &amp;&amp;
                exerciseModel.DistanceInMeters is null)
            {
                errors.Add(ExerciseErrors.MissingDistance);

                continue;
            }

            if (exerciseModel.TargetType == TargetType.Time &amp;&amp;
                exerciseModel.DurationInSeconds is null)
            {
                errors.Add(ExerciseErrors.MissingDuration);

                continue;
            }

            var exercise = new Exercise(
                Guid.NewGuid(),
                workout.Id,
                exerciseDto.ExerciseType,
                exerciseDto.TargetType,
                exerciseDto.DistanceInMeters,
                exerciseDto.DurationInSeconds);

            workouts.Exercises.Add(exercise);

            if (workouts.Exercise.Count &gt; 10)
            {
                return Result.Failure(
                    WorkoutErrors.MaxExercisesReached(workout.Id));
            }
        }

        if (errors.Count != 0)
        {
            return Result.Failure(new ValidationError(errors.ToArray()));
        }

        return Result.Success();
    }
}
</code></pre>
<p>With the domain logic inside of the domain model, we can easily share it between transaction scripts.
Testing the domain model is simpler than testing the transaction script.
With a transaction script, we must provide any dependencies (possibly as mocks) for testing.
However, we can test the domain model in isolation.</p>
<p>The updated transaction script becomes much more straightforward and focused on its primary task:</p>
<pre><code class="language-csharp">internal sealed class AddExercisesCommandHandler(
    IWorkoutRepository workoutRepository,
    IUnitOfWork unitOfWork)
    : ICommandHandler&lt;AddExercisesCommand&gt;
{
    public async Task&lt;Result&gt; Handle(
        AddExercisesCommand request,
        CancellationToken cancellationToken)
    {
        Workout? workout = await workoutRepository.GetByIdAsync(
            request.WorkoutId,
            cancellationToken);

        if (workout is null)
        {
            return Result.Failure(WorkoutErrors.NotFound(request.WorkoutId));
        }

        var exercises = request.Exercises.Select(e =&gt; e.ToModel()).ToArray();

        var result = workout.AddExercises(exercises);

        if (result.IsFailure)
        {
            return result;
        }

        await unitOfWork.SaveChangesAsync(cancellationToken);

        return Result.Success();
    }
}
</code></pre>
<h2>Takeaway</h2>
<p>Transaction Scripts are a practical starting point for simple applications.
They offer a straightforward approach to implementing use cases.
Transaction scripts are the recommended approach to start building <a href="vertical-slice-architecture-structuring-vertical-slices">vertical slices</a>.
However, transaction scripts can become difficult to maintain as the application grows.</p>
<p><a href="refactoring-from-an-anemic-domain-model-to-a-rich-domain-model">Refactoring toward a Domain Model</a> allows you to encapsulate business logic in domain objects.
This promotes code reusability and makes your application more adaptable to changes.
Pushing logic down also improves testability and maintainability.</p>
<p>Should you use a Transaction Script or a Domain Model?</p>
<p>Here's a pragmatic approach you should consider.
Start with a transaction script, but pay attention to growing complexity.
When you notice a transaction script has too many concerns, consider adding a domain model.
Remember, the domain model should encapsulate some of the complexity of the domain logic.</p>
<p>Thanks for reading, and I'll see you next week!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_094.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Caching in ASP.NET Core: Improving Application Performance]]></title>
            <link>https://milanjovanovic.tech/blog/caching-in-aspnetcore-improving-application-performance</link>
            <guid isPermaLink="false">caching-in-aspnetcore-improving-application-performance</guid>
            <pubDate>Sat, 08 Jun 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Caching is one of the simplest techniques to significantly improve your application's performance. In this newsletter, we will explore how to implement caching in ASP.NET Core applications.]]></description>
            <content:encoded><![CDATA[<p>Caching is one of the simplest techniques to significantly improve your application's performance.
It's the process of temporarily storing data in a faster access location.
You will typically cache the results of expensive operations or frequently accessed data.</p>
<p>Caching allows subsequent requests for the same data to be served from the cache instead of fetching the data from its source.</p>
<p><a href="http://ASP.NET">ASP.NET</a> Core offers several types of caches, such as <code>IMemoryCache</code>, <code>IDistributedCache</code>, and the upcoming <code>HybridCache</code> (.NET 9).</p>
<p>In this newsletter, we will explore how to implement <a href="https://learn.microsoft.com/en-us/aspnet/core/performance/caching/memory">caching in ASP.NET Core</a> applications.</p>
<h2>How Caching Improves Application Performance</h2>
<p>Caching improves your application's performance by reducing latency and server load while enhancing scalability and user experience.</p>
<ul>
<li><strong>Faster data retrieval</strong>: Cached data can be accessed much faster than retrieving it from the source (like a database or an API).
Caches are typically stored in memory (RAM).</li>
<li><strong>Fewer database queries</strong>: Caching frequently accessed data reduces the number of database queries.
This reduces the load on the database server.</li>
<li><strong>Lower CPU usage</strong>: Rendering web pages or processing API responses can consume significant CPU resources.
Caching the results reduces the need for repetitive CPU-intensive tasks.</li>
<li><strong>Handling increased traffic</strong>: By reducing the load on backend systems, caching allows your application to handle
more concurrent users and requests.</li>
<li><strong>Distributed caching</strong>: Distributed cache solutions like <a href="https://redis.io/">Redis</a> enable scaling the cache across multiple servers,
further improving performance and resilience.</li>
</ul>
<p>In a recent project I worked on, we used Redis to scale to more than 1,000,000 users.
We only had one SQL Server instance with a read-replica for reporting.
The power of caching, eh?</p>
<h2>Caching Abstractions in <a href="http://ASP.NET">ASP.NET</a> Core</h2>
<p><a href="http://ASP.NET">ASP.NET</a> Core provides two primary abstractions for working with caches:</p>
<ul>
<li><code>IMemoryCache</code>: Stores data in the memory of the web server.
Simple to use but not suitable for distributed scenarios.</li>
<li><code>IDistributedCache</code>: Offers a more robust solution for distributed applications.
It allows you to store cached data in a distributed cache like Redis.</li>
</ul>
<p>We have to register these services with DI to use them.
<code>AddDistributedMemoryCache</code> will configure the in-memory implementation of <code>IDistributedCache</code>, which isn't distributed.</p>
<pre><code class="language-csharp">builder.Services.AddMemoryCache();

builder.Services.AddDistributedMemoryCache();
</code></pre>
<p>Here's how you can use the <code>IMemoryCache</code>.
We will first check if the cached value is present and return it directly if it's there.
Otherwise, we must fetch the value from the database and cache it for subsequent requests.</p>
<pre><code class="language-csharp">app.MapGet(
    &quot;products/{id}&quot;,
    (int id, IMemoryCache cache, AppDbContext context) =&gt;
    {
        if (!cache.TryGetValue(id, out Product product))
        {
            product = context.Products.Find(id);

            var cacheEntryOptions = new MemoryCacheEntryOptions()
                .SetAbsoluteExpiration(TimeSpan.FromMinutes(10))
                .SetSlidingExpiration(TimeSpan.FromMinutes(2));

            cache.Set(id, product, cacheEntryOptions);
        }

        return Results.Ok(product);
    });
</code></pre>
<p>Cache expiration is another important topic to discuss.
We want to remove cache entries that aren't used and become stale.
You can pass in the <code>MemoryCacheEntryOptions</code>, allowing you to configure cache expiration.
For example, we can set the <code>AbsoluteExpiration</code> and <code>SlidingExpiration</code> values to control when the cache entry will expire.</p>
<h2>Cache-Aside Pattern</h2>
<p>The cache-aside pattern is the most common caching strategy. Here's how it works:</p>
<ol>
<li><strong>Check the cache</strong>: Look for the requested data in the cache.</li>
<li><strong>Fetch from source (if cache miss)</strong>: If the data isn't in the cache, fetch it from the source.</li>
<li><strong>Update the cache</strong>: Store the fetched data in the cache for subsequent requests.</li>
</ol>
<p><img src="/blogs/mnw_093/cache_aside.png" alt="Cache-aside pattern."></p>
<p>Here's how you can implement the cache-aside pattern as an extension method for <code>IDistributedCache</code>:</p>
<pre><code class="language-csharp">public static class DistributedCacheExtensions
{
    public static DistributedCacheEntryOptions DefaultExpiration =&gt; new()
    {
        AbsoluteExpirationRelativeToNow = TimeSpan.FromMinutes(2)
    };

    public static async Task&lt;T&gt; GetOrCreateAsync&lt;T&gt;(
        this IDistributedCache cache,
        string key,
        Func&lt;Task&lt;T&gt;&gt; factory,
        DistributedCacheEntryOptions? cacheOptions = null)
    {
        var cachedData = await cache.GetStringAsync(key);

        if (cachedData is not null)
        {
            return JsonSerializer.Deserialize&lt;T&gt;(cachedData);
        }

        var data = await factory();

        await cache.SetStringAsync(
            key,
            JsonSerializer.Serialize(data),
            cacheOptions ?? DefaultExpiration);

        return data;
    }
}
</code></pre>
<p>We're using <code>JsonSerializer</code> to manage serialization to and from a JSON string.
The <code>SetStringAsync</code> method also accepts a <code>DistributedCacheEntryOptions</code> argument to control cache expiration.</p>
<p>Here's how we would use this extension method:</p>
<pre><code class="language-csharp">app.MapGet(
    &quot;products/{id}&quot;,
    (int id, IDistributedCache cache, AppDbContext context) =&gt;
    {
        var product = cache.GetOrCreateAsync($&quot;products-{id}&quot;, async () =&gt;
        {
            var productFromDb = await context.Products.FindAsync(id);

            return productFromDb;
        });

        return Results.Ok(product);
    });
</code></pre>
<h2>Pros and Cons of In-Memory Caching</h2>
<p>Pros:</p>
<ul>
<li>Extremely fast</li>
<li>Simple to implement</li>
<li>No external dependencies</li>
</ul>
<p>Cons:</p>
<ul>
<li>Cache data is lost if the server restarts</li>
<li>Limited to the memory (RAM) of a single server</li>
<li>Cache data is not shared across multiple instances of your application</li>
</ul>
<h2>Distributed Caching With Redis</h2>
<p><a href="https://redis.io/">Redis</a> is a popular in-memory data store often used as a high-performance distributed cache.
To use Redis in your <a href="http://ASP.NET">ASP.NET</a> Core application, you can use the <code>StackExchange.Redis</code> library.</p>
<p>However, there's also the <code>Microsoft.Extensions.Caching.StackExchangeRedis</code> library,
allowing you to integrate Redis with <code>IDistributedCache</code>.</p>
<pre><code class="language-powershell">Install-Package Microsoft.Extensions.Caching.StackExchangeRedis
</code></pre>
<p>Here's how you can configure it with DI by providing a connection string to Redis:</p>
<pre><code class="language-csharp">string connectionString = builder.Configuration.GetConnectionString(&quot;Redis&quot;);

builder.Services.AddStackExchangeRedisCache(options =&gt;
{
    options.Configuration = connectionString;
});
</code></pre>
<p>An alternative approach is to register an <code>IConnectionMultiplexer</code> as a service.
Then, we will use it to provide a function for the <code>ConnectionMultiplexerFactory</code>.</p>
<pre><code class="language-csharp">string connectionString = builder.Configuration.GetConnectionString(&quot;Redis&quot;);

IConnectionMultiplexer connectionMultiplexer =
    ConnectionMultiplexer.Connect(connectionString);

builder.Services.AddSingleton(connectionMultiplexer);

builder.Services.AddStackExchangeRedisCache(options =&gt;
{
    options.ConnectionMultiplexerFactory =
        () =&gt; Task.FromResult(connectionMultiplexer);
});
</code></pre>
<p>Now, when you inject <code>IDistributedCache</code>, it will use Redis under the hood.</p>
<h2>Cache Stampede and HybridCache</h2>
<p>The in-memory cache implementations in <a href="http://ASP.NET">ASP.NET</a> Core are susceptible to race conditions, which can cause a cache stampede.
A <a href="https://en.wikipedia.org/wiki/Cache_stampede">cache stampede</a> happens when concurrent requests encounter a cache miss and try to fetch the data from the source.
This can overload your application and negate the benefits of caching.</p>
<p>Locking is one solution for the cache stampede problem.
.NET offers many options for <a href="introduction-to-locking-and-concurrency-control-in-dotnet-6">locking and concurrency control</a>.
The most commonly used locking primitives are the <code>lock</code> statement and the <code>Semaphore</code> (or <code>SemaphoreSlim</code>) class.</p>
<p>Here's how we could use <code>SemaphoreSlim</code> to introduce locking before fetching data:</p>
<pre><code class="language-csharp">public static class DistributedCacheExtensions
{
    private static readonly SemaphoreSlim Semaphore = new SemaphoreSlim(1, 1);

    // Arguments omitted for brevity
    public static async Task&lt;T&gt; GetOrCreateAsync&lt;T&gt;(...)
    {
        // Fetch data from cache, and return if present

        // Cache miss
        try
        {
            await Semaphore.WaitAsync();

            // Check if the data was added to the cache by another request

            // If not, proceed to fetch data and cache it
            var data = await factory();

            await cache.SetStringAsync(
                key,
                JsonSerializer.Serialize(data),
                cacheOptions ?? DefaultExpiration);
        }
        finally
        {
            Semaphore.Release();
        }

        return data;
    }
}
</code></pre>
<p>The previous implementation has a lock contention issue since all requests have to wait for the semaphore.
A much better solution would be locking based on the <code>key</code> value.</p>
<p>.NET 9 introduces a new caching abstraction called <a href="hybrid-cache-in-aspnetcore-new-caching-library"><code>HybridCache</code></a>, which aims to solve the shortcomings of <code>IDistributedCache</code>.
Learn more about this in the <a href="https://learn.microsoft.com/en-us/aspnet/core/performance/caching/hybrid">Hybrid cache documentation</a>.</p>
<h2>Summary</h2>
<p>Caching is a powerful technique for improving web application performance.
<a href="http://ASP.NET">ASP.NET</a> Core's caching abstractions make it easy to implement various caching strategies.</p>
<p>We can choose from <code>IMemoryCache</code> for in-memory cache and <code>IDistributedCache</code> for distributed caching.</p>
<p>Here are a few guidelines to wrap up this week's issue:</p>
<ul>
<li>Use <code>IMemoryCache</code> for simple, in-memory caching</li>
<li>Implement the cache aside pattern to minimize database hits</li>
<li>Consider Redis as a high-performance distributed cache implementation</li>
<li>Use <code>IDistributedCache</code> for sharing cached data across multiple applications</li>
</ul>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_093.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Vertical Slice Architecture: Structuring Vertical Slices]]></title>
            <link>https://milanjovanovic.tech/blog/vertical-slice-architecture-structuring-vertical-slices</link>
            <guid isPermaLink="false">vertical-slice-architecture-structuring-vertical-slices</guid>
            <pubDate>Sat, 01 Jun 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Are you tired of organizing your project across layers? Vertical Slice Architecture is a compelling alternative to traditional layered architectures.]]></description>
            <content:encoded><![CDATA[<p>Are you tired of organizing your project across layers?</p>
<p>Vertical Slice Architecture is a compelling alternative to traditional layered architectures.
<a href="vertical-slice-architecture">VSA</a> flips the script on how we structure code.</p>
<p>Instead of horizontal layers (Presentation, Application, Domain), VSA organizes code by feature.
Each feature encompasses everything it needs, from API endpoints to data access.</p>
<p>In this newsletter, we will explore how you can structure vertical slices in VSA.</p>
<h2>Understanding Vertical Slices</h2>
<p>At its core, a vertical slice represents a self-contained unit of functionality.
It's a slice through the entire application stack.
It encapsulates all the code and components necessary to fulfill a specific feature.</p>
<p>In traditional layered architectures, code is organized horizontally across the various layers.
One feature implementation can be scattered across many layers.
Changing a feature requires modifying the code in multiple layers.</p>
<p>VSA addresses this by grouping all the code for a feature into a single slice.</p>
<p>This shift in perspective brings several advantages:</p>
<ul>
<li><strong>Improved cohesion</strong>: Code related to a specific feature resides together, making it easier to understand, modify, and test.</li>
<li><strong>Reduced complexity</strong>: VSA simplifies your application's mental model by avoiding the need to navigate multiple layers.</li>
<li><strong>Focus on business logic</strong>: The structure naturally emphasizes the business use case over technical implementation details.</li>
<li><strong>Easier maintenance</strong>: Changes to a feature are localized within its slice, reducing the risk of unintended side effects.</li>
</ul>
<p><img src="/blogs/mnw_092/vertical_slices.png" alt="Vertical slices."></p>
<h2>Implementing Vertical Slice Architecture</h2>
<p>Here's an example vertical slice representing the <code>CreateProduct</code> feature.
We use a static class to represent the feature and group the related types.
Each feature can have a respective <code>Request</code> and <code>Response</code> class.
The use case with the business logic can be in a <a href="automatically-register-minimal-apis-in-aspnetcore">Minimal API endpoint</a>.</p>
<p>A vertical slice is likely either a command or a query.
This approach gives us <a href="cqrs-pattern-with-mediatr">CQRS</a> out of the box.</p>
<pre><code class="language-csharp">public static class CreateProduct
{
    public record Request(string Name, decimal Price);
    public record Response(int Id, string Name, decimal Price);

    public class Endpoint : IEndpoint
    {
        public void MapEndpoint(IEndpointRouteBuilder app)
        {
            app.MapPost(&quot;products&quot;, Handler).WithTags(&quot;Products&quot;);
        }

        public static IResult Handler(Request request, AppDbContext context)
        {
            var product = new Product
            {
                Name = request.Name,
                Price = request.Price
            };

            context.Products.Add(product);

            context.SaveChanges();

            return Results.Ok(
                new Response(product.Id, product.Name, product.Price));
        }
    }
}
</code></pre>
<p>I want to mention a few benefits of structuring your application like this.</p>
<p>The code for the entire <code>CreateProduct</code> feature is tightly grouped within a single file.
This makes it extremely easy to locate, understand, and modify everything related to this functionality.
We don't need to navigate multiple layers (like controllers, services, repositories, etc.).</p>
<p>Directly using <code>AppDbContext</code> within the endpoint might tightly couple the slice to your database technology.
Depending on your project's size and requirements, you could consider abstracting data access (using a repository pattern)
to make the slice more adaptable to changes in the persistence layer.</p>
<h2>Introducing Validation in Vertical Slices</h2>
<p>Vertical slices usually need to solve some <a href="balancing-cross-cutting-concerns-in-clean-architecture">cross-cutting concerns</a>, one of which is validation.
Validation is the gatekeeper, preventing invalid or malicious data from entering your system.
We can easily implement <a href="cqrs-validation-with-mediatr-pipeline-and-fluentvalidation">validation with the FluentValidation library</a>.</p>
<p>Within your slice, you'd define a <code>Validator</code> class that encapsulates the rules specific to your feature's request model.
It also supports dependency injection, so we can run complex validations here.</p>
<pre><code class="language-csharp">public class Validator : AbstractValidator&lt;Request&gt;
{
    public Validator()
    {
        RuleFor(x =&gt; x.Name).NotEmpty().MaximumLength(100);
        RuleFor(x =&gt; x.Price).GreaterThanOrEqualTo(0);
        // ... other rules
    }
}
</code></pre>
<p>This validator can then be injected into your endpoint using dependency injection, allowing you to perform validation before processing the request.</p>
<pre><code class="language-csharp">public static class CreateProduct
{
    public record Request(string Name, decimal Price);
    public record Response(int Id, string Name, decimal Price);

    public class Validator : AbstractValidator&lt;Request&gt; // { ... }

    public class Endpoint : IEndpoint
    {
        public void MapEndpoint(IEndpointRouteBuilder app)
        {
            app.MapPost(&quot;products&quot;, Handler).WithTags(&quot;Products&quot;);
        }

        public static IResult Handler(
            Request request,
            IValidator&lt;Request&gt; validator,
            AppDbContext context)
        {
            var validationResult = await validator.Validate(request);
            if (!validationResult.IsValid)
            {
                return Results.BadRequest(validationResult.Errors);
            }

            // ... (Create product and return response)
        }
    }
}
</code></pre>
<h2>Handling Complex Features and Shared Logic</h2>
<p>The previous examples were simple.
But what do we do with complex features and shared logic?</p>
<p>VSA excels at managing self-contained features.
However, real-world applications often involve complex interactions and shared logic.</p>
<p>Here are a few strategies you can consider to address this:</p>
<ul>
<li><strong>Decomposition</strong>: Break down complex features into smaller, more manageable vertical slices.
Each slice should represent a cohesive piece of the overall feature.</li>
<li><strong>Refactoring</strong>: When a vertical slice becomes difficult to maintain, you can apply some refactoring techniques.
The most common ones I use are <code>Extract method </code> and <code>Extract class</code>.</li>
<li><strong>Extract shared logic</strong>: Identify common logic that's used across multiple features.
Create a separate class (or extension method) to reference it from your vertical slices as needed.</li>
<li><strong>Push logic down</strong>: Write vertical slices using procedural code, like a <a href="https://martinfowler.com/eaaCatalog/transactionScript.html">Transaction Script</a>.
Then, you can identify parts of the business logic that naturally belong to the domain entities.</li>
</ul>
<p>You and your team will need to understand code smells and refactorings to make the most of VSA.</p>
<h2>Summary</h2>
<p>Vertical Slice Architecture is more than just a way to structure your code.
By focusing on features, VSA allows you to create cohesive and maintainable applications.
Vertical slices are self-contained, making unit and integration testing more straightforward.</p>
<p>VSA brings benefits in terms of code organization and development speed, making it a valuable tool in your toolbox.
Code is grouped by feature, making it easier to locate and understand.
The structure aligns with the way business users think about features.
Changes are localized, reducing the risk of regressions and enabling faster iterations.</p>
<p>Consider embracing <a href="vertical-slice-architecture">Vertical Slice Architecture</a> in your next project.
It's a big mindset shift from <a href="/pragmatic-clean-architecture">Clean Architecture</a>.
However, they both have their place and even share similar ideas.</p>
<p>That's all for this week. Stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_092.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Shift Left With Architecture Testing in .NET]]></title>
            <link>https://milanjovanovic.tech/blog/shift-left-with-architecture-testing-in-dotnet</link>
            <guid isPermaLink="false">shift-left-with-architecture-testing-in-dotnet</guid>
            <pubDate>Sat, 25 May 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[In this newsletter, we'll explore how architecture testing can safeguard our project's architecture. Architecture tests can help us shift left and detect architectural issues faster.]]></description>
            <content:encoded><![CDATA[<p>Picture this: You're part of a team building a shiny new .NET application.
You've carefully chosen your software architecture.
It could be microservices, a <a href="/modular-monolith-architecture"><strong>modular monolith</strong></a>, or something else entirely.
You've decided which database you will use and all the other tools you need.
Everyone's excited, the code is flowing, and features are getting shipped.</p>
<p>Fast forward a few months (or years), and things might look different.</p>
<p>The codebase has grown, and new features have been added.
Maybe your team has even changed, with new developers coming on board.
Adding new features becomes a pain, and bugs are popping up left and right.</p>
<p>And slowly but surely, the neat architecture you started with has turned into a <a href="https://deviq.com/antipatterns/big-ball-of-mud">big ball of mud</a>.
What went wrong? And more importantly, what can we do about it?</p>
<p>Today, I want to show you how architecture testing can prevent this problem.</p>
<h2>Technical Debt</h2>
<p>Technical debt is the consequence of prioritizing development speed over well-designed code.
It happens when teams cut corners to meet deadlines, make quick fixes, or don't understand the architecture clearly.</p>
<p>Each shortcut or hack adds to the pile, making the code harder to understand, change, and maintain.
But why do developers take these shortcuts in the first place?</p>
<p>Don't developers care about keeping the code clean?</p>
<p>Well, the truth is, most developers do care.
If you're reading this newsletter, odds are you also care.
But, developers are often under pressure to deliver features quickly.
Sometimes, the quickest way to do that is to take a shortcut.</p>
<p>Plus, not everyone has a deep understanding of software architecture, or they might disagree on what the &quot;right&quot; architecture is.
And let's be honest: some developers want to get their code working and move on to the next thing.</p>
<h2>Architecture Testing</h2>
<p>Luckily, there's a way to enforce software architecture on your project before things get out of hand.
It's called <a href="enforcing-software-architecture-with-architecture-tests"><strong>architecture testing</strong></a>.
These are automated tests that check whether your code follows the architectural rules you've set up.</p>
<p>With architecture testing, you can <a href="https://en.wikipedia.org/wiki/Shift-left_testing">&quot;shift left&quot;</a>.
This enables you to find and fix problems early in the development process when they're much easier and cheaper to deal with.</p>
<p>Think of it like a safety net for your software architecture and design rules.
If someone accidentally breaks a rule, the test will catch it and alert you.
Bonus points if you integrate architecture testing into your <a href="how-to-build-ci-cd-pipeline-with-github-actions-and-dotnet"><strong>CI pipeline</strong></a>.</p>
<p>There are a few libraries you can use for architecture testing.
I prefer working with the <a href="https://github.com/BenMorris/NetArchTest">NetArchTest</a> library, which I'll use for the examples.</p>
<p>You can check out this article to learn the <a href="enforcing-software-architecture-with-architecture-tests"><strong>fundamentals of architecture testing</strong></a>.</p>
<p>Let's see how to write some architecture tests.</p>
<h2>Architecture Testing: Modular Monolith</h2>
<p>You built an application using the <a href="what-is-a-modular-monolith"><strong>modular monolith architecture</strong></a>.
But how can you maintain the constraints between the modules?</p>
<ul>
<li>Modules aren't allowed to reference each other</li>
<li>Modules can only call the public API of other modules</li>
</ul>
<p>Here's an architecture test that enforces these module constraints.
The <code>Ticketing</code> module is not allowed to reference the other modules directly.
However, it can reference the public API of other modules (integration events in this example).
The entry point is the <code>Types</code> class, which exposes a fluent API to build the rules you want to enforce.
NetArchTest allows us to enforce the direction of dependencies between modules.</p>
<pre><code class="language-csharp">[Fact]
public void TicketingModule_ShouldNotHaveDependencyOn_AnyOtherModule()
{
    string[] otherModules = [
        UsersNamespace,
        EventsNamespace,
        AttendanceNamespace];

    string[] integrationEventsModules = [
        UsersIntegrationEventsNamespace,
        EventsIntegrationEventsNamespace,
        AttendanceIntegrationEventsNamespace];

    List&lt;Assembly&gt; ticketingAssemblies =
    [
        typeof(Order).Assembly,
        Modules.Ticketing.Application.AssemblyReference.Assembly,
        Modules.Ticketing.Presentation.AssemblyReference.Assembly,
        typeof(TicketingModule).Assembly
    ];

    Types.InAssemblies(ticketingAssemblies)
        .That()
        .DoNotHaveDependencyOnAny(integrationEventsModules)
        .Should()
        .NotHaveDependencyOnAny(otherModules)
        .GetResult()
        .ShouldBeSuccessful();
}
</code></pre>
<p>If you want to learn how to build robust and scalable systems using this architectural approach, check out <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a>.</p>
<h2>Architecture Testing: Clean Architecture</h2>
<p>We can also write architecture tests for <a href="why-clean-architecture-is-great-for-complex-projects"><strong>Clean Architecture</strong></a>.
The inner layers aren't allowed to reference the outer layers.
Instead, the inner layers define abstractions and the outer layers implement these abstractions.</p>
<p>For example, the <code>Domain</code> layer isn't allowed to reference the <code>Application</code> layer.
Here's an architecture test enforcing this rule:</p>
<pre><code class="language-csharp">[Fact]
public void DomainLayer_ShouldNotHaveDependencyOn_ApplicationLayer()
{
    Types.InAssembly(DomainAssembly)
        .Should()
        .NotHaveDependencyOn(ApplicationAssembly.GetName().Name)
        .GetResult()
        .ShouldBeSuccessful();
}
</code></pre>
<p>It's also simple introduce a rule that the <code>Application</code> layer isn't allowed to reference the <code>Infrastructure</code> layer.
The architecture test will fail whenever someone in the team breaks the dependency rule.</p>
<pre><code class="language-csharp">[Fact]
public void ApplicationLayer_ShouldNotHaveDependencyOn_InfrastructureLayer()
{
    Types.InAssembly(ApplicationAssembly)
        .Should()
        .NotHaveDependencyOn(InfrastructureAssembly.GetName().Name)
        .GetResult()
        .ShouldBeSuccessful();
}
</code></pre>
<p>We can introduce more architecture tests for the <code>Infrastructure</code> and <code>Presentation</code> layers, if needed.</p>
<p>Ready to learn more about building production-ready applications using this architectural approach?
You should check out <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture</strong></a>.</p>
<h2>Architecture Testing: Design Rules</h2>
<p>Architecture testing is also useful for enforcing design rules in your code.
If your team has coding standards everyone should follow, architecture testing can help you enforce them.</p>
<p>For example, we want to ensure that all domain events are sealed types.
You can use the <code>BeSealed</code> method to enforce a design rule that types implementing <code>IDomainEvent</code> or <code>DomainEvent</code> should be sealed.</p>
<pre><code class="language-csharp">[Fact]
public void DomainEvents_Should_BeSealed()
{
    Types.InAssembly(DomainAssembly)
        .That()
        .ImplementInterface(typeof(IDomainEvent))
        .Or()
        .Inherit(typeof(DomainEvent))
        .Should()
        .BeSealed()
        .GetResult()
        .ShouldBeSuccessful();
}
</code></pre>
<p>An interesting design rule could be requiring all domain entities not to have a public constructor.
Instead, you would create an <code>Entity</code> instance through a static factory method.
This approach improves the encapsulation of your <code>Entity</code>.</p>
<p>Here's an architecture test enforcing this design rule:</p>
<pre><code class="language-csharp">[Fact]
public void Entities_ShouldOnlyHave_PrivateConstructors()
{
    IEnumerable&lt;Type&gt; entityTypes = Types.InAssembly(DomainAssembly)
        .That()
        .Inherit(typeof(Entity))
        .GetTypes();

    var failingTypes = new List&lt;Type&gt;();
    foreach (Type entityType in entityTypes)
    {
        ConstructorInfo[] constructors = entityType
            .GetConstructors(BindingFlags.Public | BindingFlags.Instance);

        if (constructors.Any())
        {
            failingTypes.Add(entityType);
        }
    }

    failingTypes.Should().BeEmpty();
}
</code></pre>
<p>Another thing you can do with architecture tests is enforce naming conventions in your code.
Here's an example of requiring all command handlers to have a name ending with <code>CommandHandler</code>:</p>
<pre><code class="language-csharp">[Fact]
public void CommandHandler_ShouldHave_NameEndingWith_CommandHandler()
{
    Types.InAssembly(ApplicationAssembly)
        .That()
        .ImplementInterface(typeof(ICommandHandler&lt;&gt;))
        .Or()
        .ImplementInterface(typeof(ICommandHandler&lt;,&gt;))
        .Should()
        .HaveNameEndingWith(&quot;CommandHandler&quot;)
        .GetResult()
        .ShouldBeSuccessful();
}
</code></pre>
<h2>Summary</h2>
<p>Even the most well-planned software projects decay because of technical debt.
Most developers have good intentions.
However, time pressure, misunderstandings, and resistance to rules all contribute to this problem.</p>
<p><a href="enforcing-software-architecture-with-architecture-tests"><strong>Architecture testing</strong></a> acts as a safeguard.
It prevents your codebase from turning into a big ball of mud.
By catching architectural violations early on, you can shift left.
Short feedback loops avoid costly rework and improve developer productivity.
It also ensures the long-term health of your project.</p>
<p>A few key takeaways:</p>
<ul>
<li><strong>Technical debt is inevitable</strong>: It slows down development, introduces bugs, and frustrates developers.</li>
<li><strong>Architecture testing is your safety net</strong>: It helps you catch architectural violations before they become problematic.</li>
<li><strong>Start small and iterate</strong>: You don't have to test everything at once. Focus on the most critical rules first.</li>
<li><strong>Make it part of your workflow</strong>: Integrate architecture tests into your CI/CD pipeline so they run automatically.</li>
</ul>
<p><strong>Action point</strong>: Start by exploring popular .NET architecture testing libraries like <a href="https://github.com/TNG/ArchUnitNET">ArchUnitNET</a>
or <a href="https://github.com/BenMorris/NetArchTest">NetArchTest</a>.
Experiment with writing tests for common architectural rules and gradually integrate them into your development workflow.</p>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_091.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[EF Core Migrations: A Detailed Guide]]></title>
            <link>https://milanjovanovic.tech/blog/efcore-migrations-a-detailed-guide</link>
            <guid isPermaLink="false">efcore-migrations-a-detailed-guide</guid>
            <pubDate>Sat, 18 May 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[In this newsletter, we'll break down the essentials of EF Migrations. We'll explore creating migrations, SQL scripts, applying migrations, migration tooling, and more.]]></description>
            <content:encoded><![CDATA[<p>Managing database schemas as your applications grow can quickly become a headache.
Manual changes are error-prone and time-consuming.
This can easily lead to inconsistencies between development and production environments.
I've seen these issues firsthand on countless projects, and it's not pretty.
How can we do better?</p>
<p>Enter Entity Framework (EF) Migrations, a powerful tool that lets you version your database schemas.</p>
<p>Imagine this: Instead of writing SQL scripts, you define your changes in code.
Need to add a column?
Rename a table?
No problem - EF Migrations has you covered.
It tracks every modification to the data model.
You can review, test, and apply changes confidently, even across different environments.</p>
<p>In this newsletter, we'll break down the essentials of EF Migrations:</p>
<ul>
<li><strong>Creating Migrations</strong>: Defining and generating migrations that capture your schema changes.</li>
<li><strong>Migration SQL Scripts</strong>: Understanding the SQL generated by your migrations and how to use it.</li>
<li><strong>Applying Migrations</strong>: Different ways to apply migrations to your database.</li>
<li><strong>Migration Tools</strong>: Exploring additional tools and frameworks for managing database migration.</li>
<li><strong>EF Migration Best Practices:</strong>: I'll share my recommendations from using EF for years.</li>
</ul>
<p>We have many examples to cover, so let's dive in.</p>
<h2>Creating Migrations</h2>
<p>If you're completely new to EF migrations, I recommend checking out the <a href="https://learn.microsoft.com/en-us/ef/core/managing-schemas/migrations">EF migrations docs</a>
to grasp the fundamentals.
Moving forward, I'll assume you have some prior knowledge of EF Core.</p>
<p>We'll need an entity and a database context before we can create migrations with EF.</p>
<p>Let's define a simple <code>Product</code> entity:</p>
<pre><code class="language-csharp">public class Product
{
    public int Id { get; set; }

    public string Name { get; set; } = string.Empty;

    public string? Description { get; set; }

    public decimal Price { get; set; }
}
</code></pre>
<p>We will also need a <code>DbContext</code> implementation, so let's define the <code>AppDbContext</code> class.
In the <code>OnModelCreating</code> method, we're going to configure the <code>Product</code> entity.</p>
<pre><code class="language-csharp">public class AppDbContext : DbContext
{
    public AppDbContext(DbContextOptions&lt;AppDbContext&gt; options)
        : base(options)
    {
    }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity&lt;Product&gt;(builder =&gt;
        {
            builder.ToTable(&quot;Products&quot;, tableBuilder =&gt;
            {
                tableBuilder.HasCheckConstraint(
                    &quot;CK_Price_NotNegative&quot;,
                    sql: $&quot;{nameof(Product.Price)} &gt; 0&quot;);
            });

            builder.HasKey(p =&gt; p.Id);

            builder.Property(p =&gt; p.Name).HasMaxLength(100);

            builder.Property(p =&gt; p.Description).HasMaxLength(1000);

            builder.Property(p =&gt; p.Price).HasPrecision(18, 2);

            builder.HasIndex(p =&gt; p.Name).IsUnique();
        });
    }
}
</code></pre>
<p>Let's break down a few of the methods we're using:</p>
<ul>
<li><code>ToTable</code> - Configures the table name for the specific entity.
It also allows us to provide <code>TableBuilder</code> delegate.
We can use it to configure a check constraint using <code>HasCheckConstraint</code>.</li>
<li><code>HasKey</code> - Configures the table's primary key.
EF will also pick up the <code>Id</code> property by convention, so this step is optional.</li>
<li><code>HasProperty</code> - Represents the entry point for configuring individual properties of the entity.</li>
<li><code>HasIndex</code> - Defines an index on the specified property (or properties).
We can also declare that the index should be unique by calling <code>IsUnique</code>.</li>
</ul>
<p>We're now ready to create our first migration.
I'm going to use the PowerShell syntax:</p>
<pre><code>Add-Migration Create_Database
</code></pre>
<p>This will create the first database migration called <code>Create_Database</code>.
The migration will apply the configuration we defined in the <code>OnModelCreating</code> method.
It contains the <code>Up</code> and <code>Down</code> methods, allowing us to apply or revert changes to the database.</p>
<p>Note that some operations are destructive (like removing a column) and can't be easily reverted.
It's up to you to examine the generated migration and prevent any possible data loss.</p>
<pre><code class="language-csharp">using Microsoft.EntityFrameworkCore.Migrations;
using Npgsql.EntityFrameworkCore.PostgreSQL.Metadata;

public partial class Create_Database : Migration
{
    protected override void Up(MigrationBuilder migrationBuilder)
    {
        migrationBuilder.CreateTable(
            name: &quot;Products&quot;,
            columns: table =&gt; new
            {
                Id = table.Column&lt;int&gt;(type: &quot;integer&quot;, nullable: false)
                    .Annotation(&quot;Npgsql:ValueGenerationStrategy&quot;, NpgsqlValueGenerationStrategy.IdentityByDefaultColumn),
                Name = table.Column&lt;string&gt;(type: &quot;character varying(100)&quot;, maxLength: 100, nullable: false),
                Description = table.Column&lt;string&gt;(type: &quot;character varying(1000)&quot;, maxLength: 1000, nullable: true),
                Price = table.Column&lt;decimal&gt;(type: &quot;numeric(18,2)&quot;, precision: 18, scale: 2, nullable: false)
            },
            constraints: table =&gt;
            {
                table.PrimaryKey(&quot;PK_Products&quot;, x =&gt; x.Id);
                table.CheckConstraint(&quot;CK_Price_NotNegative&quot;, &quot;Price &gt; 0&quot;);
            });

        migrationBuilder.CreateIndex(
            name: &quot;IX_Products_Name&quot;,
            table: &quot;Products&quot;,
            column: &quot;Name&quot;,
            unique: true);
    }

    protected override void Down(MigrationBuilder migrationBuilder)
    {
        migrationBuilder.DropTable(
            name: &quot;Products&quot;);
    }
}
</code></pre>
<h2>Customizing Migrations</h2>
<p>We can also modify the migration files if we need to apply some custom changes.</p>
<p>A notable example is renaming a column.
Let's say we rename the <code>Description</code> property to <code>ShortDescription</code>.
In some EF versions, this would result in the following migration:</p>
<pre><code class="language-csharp">migrationBuilder.DropColumn(
    name: &quot;Description&quot;,
    table: &quot;Customers&quot;);

migrationBuilder.AddColumn&lt;string&gt;(
    name: &quot;ShortDescription&quot;,
    table: &quot;Products&quot;,
    nullable: true);
</code></pre>
<p>What's the problem here?
By calling <code>DropColumn</code> first, we will remove the column from the database and lose valuable data.</p>
<p>What we actually want to do is rename the existing column.
So, we can modify the migration file to use the <code>RenameColumn</code> method:</p>
<pre><code class="language-csharp">migrationBuilder.RenameColumn(
    name: &quot;Description&quot;,
    table: &quot;Products&quot;,
    newName: &quot;ShortDescription&quot;);
</code></pre>
<p>Another example is executing custom SQL commands from your migrations.
Custom SQL commands are helpful when we can't express something through the EF fluent API.
I've used it in the past to migrate data from one column to another or define a complex index.</p>
<pre><code class="language-csharp">public partial class Update_Products : Migration
{
    protected override void Up(MigrationBuilder migrationBuilder)
    {
        migrationBuilder.Sql(&quot;&lt;YOUR CUSTOM SQL HERE&gt;&quot;);
    }

    protected override void Down(MigrationBuilder migrationBuilder)
    {
        // You are also responsible for reverthing any changes.
    }
}
</code></pre>
<h2>Migration SQL Scripts</h2>
<p>You can use the <code>Script-Migration</code> command to generate SQL scripts from your migrations.
This is useful for reviewing changes before applying them to the database.
SQL scripts allow us to execute migrations in environments without direct access to the EF tooling.</p>
<p>Remember, you are responsible for preventing any data loss when executing EF migrations.
Review the migrations carefully before applying them to the database.</p>
<p>Here are a few ways you can execute the <code>Script-Migration</code> command:</p>
<pre><code>Script-Migration

Script-Migration &lt;FromMigration&gt;

Script-Migration &lt;FromMigration&gt; &lt;ToMigration&gt;
</code></pre>
<p>The <code>&lt;FromMigration&gt;</code> argument should be the name of the last migration applied to the database.
It's your responsibility to apply the script appropriately, and only to databases in the correct migration state.</p>
<p>Here's what the SQL script for the <code>Create_Database</code> migration looks like:</p>
<pre><code class="language-sql">CREATE TABLE IF NOT EXISTS &quot;__EFMigrationsHistory&quot; (
    &quot;MigrationId&quot; character varying(150) NOT NULL,
    &quot;ProductVersion&quot; character varying(32) NOT NULL,
    CONSTRAINT &quot;PK___EFMigrationsHistory&quot; PRIMARY KEY (&quot;MigrationId&quot;)
);

START TRANSACTION;

CREATE TABLE &quot;Products&quot; (
    &quot;Id&quot; integer GENERATED BY DEFAULT AS IDENTITY,
    &quot;Name&quot; character varying(100) NOT NULL,
    &quot;Description&quot; character varying(1000),
    &quot;Price&quot; numeric(18,2) NOT NULL,
    CONSTRAINT &quot;PK_Products&quot; PRIMARY KEY (&quot;Id&quot;),
    CONSTRAINT &quot;CK_Price_NotNegative&quot; CHECK (Price &gt; 0)
);

CREATE UNIQUE INDEX &quot;IX_Products_Name&quot; ON &quot;Products&quot; (&quot;Name&quot;);

INSERT INTO &quot;__EFMigrationsHistory&quot; (&quot;MigrationId&quot;, &quot;ProductVersion&quot;)
VALUES ('20240516095344_Create_Database', '8.0.5');

COMMIT;
</code></pre>
<p>You can also specify an <code>-Idempotent</code> argument to the <code>Script-Migration</code> command.
The <code>Script-Migration</code> command will generate SQL scripts that only apply migrations that haven't been applied already.
This is useful if you're not sure what the last migration applied to the database.</p>
<pre><code>Script-Migration -Idempotent
</code></pre>
<p>Here's what the idempotent SQL script looks like:</p>
<pre><code class="language-sql">CREATE TABLE IF NOT EXISTS &quot;__EFMigrationsHistory&quot; (
    &quot;MigrationId&quot; character varying(150) NOT NULL,
    &quot;ProductVersion&quot; character varying(32) NOT NULL,
    CONSTRAINT &quot;PK___EFMigrationsHistory&quot; PRIMARY KEY (&quot;MigrationId&quot;)
);

START TRANSACTION;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM &quot;__EFMigrationsHistory&quot; WHERE &quot;MigrationId&quot; = '20240516095344_Create_Database') THEN
    CREATE TABLE &quot;Products&quot; (
        &quot;Id&quot; integer GENERATED BY DEFAULT AS IDENTITY,
        &quot;Name&quot; character varying(100) NOT NULL,
        &quot;Description&quot; character varying(1000),
        &quot;Price&quot; numeric(18,2) NOT NULL,
        CONSTRAINT &quot;PK_Products&quot; PRIMARY KEY (&quot;Id&quot;),
        CONSTRAINT &quot;CK_Price_NotNegative&quot; CHECK (Price &gt; 0)
    );
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM &quot;__EFMigrationsHistory&quot; WHERE &quot;MigrationId&quot; = '20240516095344_Create_Database') THEN
    CREATE UNIQUE INDEX &quot;IX_Products_Name&quot; ON &quot;Products&quot; (&quot;Name&quot;);
    END IF;
END $EF$;

DO $EF$
BEGIN
    IF NOT EXISTS(SELECT 1 FROM &quot;__EFMigrationsHistory&quot; WHERE &quot;MigrationId&quot; = '20240516095344_Create_Database') THEN
    INSERT INTO &quot;__EFMigrationsHistory&quot; (&quot;MigrationId&quot;, &quot;ProductVersion&quot;)
    VALUES ('20240516095344_Create_Database', '8.0.5');
    END IF;
END $EF$;
COMMIT;
</code></pre>
<h2>Applying Migrations</h2>
<p>How do we apply EF migrations to the database?</p>
<p>We have a few options:</p>
<ul>
<li>SQL scripts</li>
<li>Command-line tools</li>
<li>Apply migrations through code</li>
<li>Migration bundles</li>
</ul>
<p>We discussed SQL scripts in the previous section, so I won't mention them again.</p>
<h3>Command-line Tools</h3>
<p>The most common approach to applying database migrations is using the CLI.
You can use either the <code>dotnet ef</code> tool or the PowerShell commands.
For example, you can execute the <code>Update-Database</code> command from PowerShell to apply any pending migrations.</p>
<pre><code>Update-Database -Migration &lt;ToMigration&gt; -Connection &lt;ConnectionString&gt;
</code></pre>
<p>Here are the documentation links if you want to learn more:</p>
<ul>
<li><a href="https://learn.microsoft.com/en-us/ef/core/cli/">EF Core CLI documentation</a></li>
<li><a href="https://learn.microsoft.com/en-us/ef/core/cli/powershell">EF Core PowerShell documentation</a></li>
</ul>
<h3>Applying Migrations through Code</h3>
<p>Here's a helper method for applying database migrations.
It uses an <code>IServiceScope</code> to resolve a <code>DbContext</code> instance and uses it to call the <code>Migrate</code> method.</p>
<pre><code class="language-csharp">public static void ApplyMigration&lt;TDbContext&gt;(IServiceScope scope)
    where TDbContext : DbContext
{
    using TDbContext context = scope.ServiceProvider
        .GetRequiredService&lt;TDbContext&gt;();

    context.Database.Migrate();
}
</code></pre>
<p>You can apply migrations when the application is starting.
I <strong>don't recommend</strong> using this approach for production environments.
Migration can fail, concurrency issues exist, and rolling back migrations is challenging.
However, this approach can be helpful in local development and when scaffolding databases for integration testing.</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);

var app = builder.Build();

if (app.Environment.IsDevelopment())
{
    using IServiceScope scope = app.ApplicationServices.CreateScope();

    ApplyMigration&lt;AppDbContext&gt;(scope);
}

app.Run();
</code></pre>
<h3>Migration Bundles</h3>
<p>Migration bundles are executable files that you can use to apply database migrations.
They're self-contained and can be executed from CI pipelines.</p>
<p>You can use the <code>Bundle-Migration</code> command to create a migration bundle:</p>
<pre><code>Bundle-Migration -Connection &lt;ConnectionString&gt;
</code></pre>
<p>This will create an <code>efbundle.exe</code> file that we can run to apply any pending database migrations.</p>
<p>I recommend reading the <a href="https://learn.microsoft.com/en-us/ef/core/managing-schemas/migrations/applying#bundles">migration bundles documentation</a> to learn more.</p>
<h2>Additional Database Migration Tools</h2>
<p>What can you do if you don't want to use EF Core migrations?</p>
<p>I wanted to mention some additional tools you can use for database schema versioning and running migrations:</p>
<ul>
<li><a href="https://github.com/fluentmigrator/fluentmigrator">FluentMigrator</a>: A migration framework for .NET with a fluent API for defining migrations.</li>
<li><a href="https://github.com/DbUp/DbUp">DbUp</a>: A lightweight library for applying SQL scripts to databases.</li>
<li><a href="https://erikbra.github.io/grate/">Grate</a>: An automated database deployment (change management) system that relies on SQL scripts.</li>
<li><a href="https://flywaydb.org/">Flyway</a>: An open-source database migration tool that simplifies the management and versioning of database schema changes.</li>
</ul>
<p>We won't do a deep dive on these tools, but I recommend you check out their documentation.</p>
<h2>EF Core Migrations Best Practices</h2>
<p>I want to wrap up this issue with a few tips from my experience of working with EF Core migrations over the years:</p>
<ul>
<li><strong>Use meaningful migration names</strong>: Don't name migrations with dates or generic descriptions.
Use clear, descriptive names that indicate the purpose of the migration.
Good examples: <code>AddProductsTable</code>, <code>RenameDescriptionToShortDescription</code>.
This makes it much easier to understand your migration history and find specific changes.</li>
<li><strong>Keep migrations small and focused</strong>: Avoid creating massive migrations containing multiple unrelated changes.
Smaller migrations are easier to review, test, and troubleshoot if something goes wrong.
Aim for one migration per feature or logical change.</li>
<li><strong>Test migrations thoroughly</strong>: Before applying migrations to production, test them in a development or staging environment.
Development and staging environments should mirror your production setup as closely as possible.
This will help catch any unexpected issues or data loss risks before they affect real users.</li>
<li><strong>Beware of destructive changes</strong>: Some operations, like dropping columns or tables, can lead to irreversible data loss.
Carefully consider the consequences before including such changes in migrations.
Provide a way to migrate data or create a backup plan.</li>
<li><strong>Avoid merge conflicts</strong>: Solving merge conflicts for EF migration snapshots can be a real headache.
Be mindful of this when working in a team that creates many database migrations.
It's recommended to always be up-to-date with the latest migration before creating a new one.
This should minimize the chance of creating merge conflicts.</li>
</ul>
<p>My preferred approach to applying migrations is using SQL scripts.
Depending on the project scope and complexity, we could do this manually or through a tool that automates the process.
This allows me to review the migration and identify any potential problems.</p>
<p>I hope this was helpful!</p>
<p>Thanks for reading, and I'll see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_090.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Building Resilient Cloud Applications With .NET]]></title>
            <link>https://milanjovanovic.tech/blog/building-resilient-cloud-applications-with-dotnet</link>
            <guid isPermaLink="false">building-resilient-cloud-applications-with-dotnet</guid>
            <pubDate>Sat, 11 May 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[By designing your applications with resilience in mind, you can create robust and reliable systems, even when the going gets tough. In this newsletter, we'll explore the tools and techniques we have in .NET to build resilient systems.]]></description>
            <content:encoded><![CDATA[<p>From my experience working with microservices systems, things don't always go as planned.
Network requests randomly fail, application servers become overloaded, and unexpected errors appear.
That's where resilience comes in.</p>
<p>Resilient applications can recover from transient failures and continue to function.
Resilience is achieved by designing applications that can handle failures gracefully and recover quickly.</p>
<p>By designing your applications with resilience in mind, you can create robust and reliable systems, even when the going gets tough.</p>
<p>In this newsletter, we'll explore the tools and techniques we have in .NET to build resilient systems.</p>
<h2>Resilience: Why You Should Care</h2>
<p>Sending HTTP requests is a common approach for remote communication between services.
However, HTTP requests are susceptible to failures from network or server issues.
These failures can disrupt service availability, especially as dependencies increase and the risk of cascading failures grows.</p>
<p>So, how can you improve the resilience of your applications and services?</p>
<p>Here are a few strategies you can consider to increase resilience:</p>
<ul>
<li><strong>Retries</strong>: Retry requests that fail due to transient errors.</li>
<li><strong>Timeouts</strong>: Cancel requests that exceed a specified time limit.</li>
<li><strong>Fallbacks</strong>: Define alternative actions or results for failed operations.</li>
<li><strong>Circuit Breakers</strong>: Temporarily suspend communication with unavailable services.</li>
</ul>
<p>You can use these strategies individually or in combination for optimal HTTP request resilience.</p>
<p>Let's see how we can introduce resilience in a .NET application.</p>
<h2>Resilience Pipelines</h2>
<p>With .NET 8, integrating resilience into your applications has become much simpler.
We can use <code>Microsoft.Extensions.Resilience</code> and <code>Microsoft.Extensions.Http.Resilience</code>, which are built on top of <a href="https://github.com/App-vNext/Polly">Polly</a>.
Polly is a .NET resilience and transient fault-handling library.
Polly allows us to define resilience strategies such as retry, circuit breaker, timeout, rate-limiting, fallback, and hedging.</p>
<p>Polly received a new API surface in its latest version (V8), which was implemented in collaboration with Microsoft.
You can learn more about the <a href="https://youtu.be/PqVQFUCTzUM"><strong>Polly V8 API in this video</strong></a>.</p>
<p>If you were previously using <code>Microsoft.Extensions.Http.Polly</code>, it is recommended that you switch to one of the previously mentioned packages.</p>
<p>Let's start by installing the required NuGet packages:</p>
<pre><code class="language-powershell">Install-Package Microsoft.Extensions.Resilience
Install-Package Microsoft.Extensions.Http.Resilience
</code></pre>
<p>To use resilience, you must first build a pipeline consisting of resilience <a href="https://www.pollydocs.org/strategies/">strategies</a>.
Each strategy that we configure as part of the pipeline will execute in order of configuration.
Order is important with resilience pipelines.
Keep that in mind.</p>
<p>We start by creating an instance of <code>ResiliencePipelineBuilder</code>, which allows us to configure resilience strategies.</p>
<pre><code class="language-csharp">ResiliencePipeline pipeline = new ResiliencePipelineBuilder()
    .AddRetry(new RetryStrategyOptions
    {
        ShouldHandle = new PredicateBuilder().Handle&lt;ConflictException&gt;(),
        Delay = TimeSpan.FromSeconds(1),
        MaxRetryAttempts = 2,
        BackoffType = DelayBackoffType.Exponential,
        UseJitter = true
    })
    .AddTimeout(new TimeoutStrategyOptions
    {
        Timeout = TimeSpan.FromSeconds(10)
    })
    .Build();

await pipeline.ExecuteAsync(
    async ct =&gt; await httpClient.GetAsync(&quot;https://modularmonolith.com&quot;, ct),
    cancellationToken);
</code></pre>
<p>Here's what we're adding to the resilience pipeline:</p>
<ul>
<li><code>AddRetry</code> - Configures a retry resilience strategy, which we can further configure by passing in a <code>RetryStrategyOptions</code> instance.
We can provide a predicate for the <code>ShouldHandle</code> property to define which exceptions the resilience strategy should handle.
The retry strategy also comes with some sensible <a href="https://www.pollydocs.org/strategies/retry.html#defaults">default values</a>.</li>
<li><code>AddTimeout</code> - Configures a timeout strategy that will throw a <code>TimeoutRejectedException</code> if the delegate does not complete before the timeout.
We can provide a custom timeout by passing in a <code>TimeoutStrategyOptions</code> instance.
The default timeout is 30 seconds.</li>
</ul>
<p>Finally, we can <code>Build</code> the resilience pipeline and get back a configured <code>ResiliencePipeline</code> instance that will apply the respective resilience strategies.
To use the <code>ResiliencePipeline</code>, we can call the <code>ExecuteAsync</code> method and pass in a delegate.</p>
<h2>Resilience Pipelines and Dependency Injection</h2>
<p>Configuring a resilience pipeline every time we want to use it is cumbersome.
.NET 8 introduces a new extension method for the <code>IServiceCollection</code> interface that allows us to register resilience pipelines with dependency injection.</p>
<p>Instead of manually configuring resilience every time, you ask for a pre-made pipeline by name.</p>
<p>We start by calling the <code>AddResiliencePipeline</code> method, which allows us to configure the resilience pipeline.
Each resilience pipeline needs to have a unique key.
We can use this key to resolve the respective resilience pipeline instance.</p>
<p>In this example, we're passing in a <code>string</code> key which allows us to configure the non-generic <code>ResiliencePipelineBuilder</code>.</p>
<pre><code class="language-csharp">services.AddResiliencePipeline(&quot;retry&quot;, builder =&gt;
{
    builder.AddRetry(new RetryStrategyOptions
    {
        Delay = TimeSpan.FromSeconds(1),
        MaxRetryAttempts = 2,
        BackoffType = DelayBackoffType.Exponential,
        UseJitter = true
    });
});
</code></pre>
<p>However, we can also specify generic arguments when calling <code>AddResiliencePipeline</code>.
This allows us to configure a typed resilience pipeline using <code>ResiliencePipelineBuilder&lt;TResult&gt;</code>.
Using this approach, we can access the <a href="https://www.pollydocs.org/strategies/hedging.html">hedging</a> and <a href="https://www.pollydocs.org/strategies/fallback.html">fallback</a> strategies.</p>
<p>In the following example, we're configuring a fallback strategy by calling <code>AddFallback</code>.
This allows us to provide a fallback value that we can return in case of a failure.
The fallback could be a static value or come from another HTTP request or the database.</p>
<pre><code class="language-csharp">services.AddResiliencePipeline&lt;string, GitHubUser?&gt;(&quot;gh-fallback&quot;, builder =&gt;
{
    builder.AddFallback(new FallbackStrategyOptions&lt;GitHubUser?&gt;
    {
        FallbackAction = _ =&gt;
            Outcome.FromResultAsValueTask&lt;GitHubUser?&gt;(GitHubUser.Empty)
    });
});
</code></pre>
<p>To use resilience pipelines configured with dependency injection, we can use the <code>ResiliencePipelineProvider</code>.
It exposes a <code>GetPipeline</code> method for obtaining the pipeline instance.
We have to provide the key used to register the resilience pipeline.</p>
<pre><code class="language-csharp">app.MapGet(&quot;users&quot;, async (
    HttpClient httpClient,
    ResiliencePipelineProvider&lt;string&gt; pipelineProvider) =&gt;
{
    ResiliencePipeline&lt;GitHubUser?&gt; pipeline =
        pipelineProvider.GetPipeline&lt;GitHubUser?&gt;(&quot;gh-fallback&quot;);

    var user = await pipeline.ExecuteAsync(async token =&gt;
        await httpClient.GetAsync(&quot;api/users&quot;, token),
        cancellationToken);
});
</code></pre>
<h2>Resilience Strategies and Polly</h2>
<p><a href="https://www.pollydocs.org/strategies/">Resilience strategies</a> are the core component of Polly.
They're designed to run custom callbacks while introducing an additional layer of resilience.
We can't run these strategies directly.
Instead, we execute them through a resilience pipeline.</p>
<p>Polly categorizes resilience strategies into <strong>reactive</strong> and <strong>proactive</strong>.
Reactive strategies handle specific exceptions or results.
Proactive strategies decide to cancel or reject the execution of callbacks using a rate limiter or a timeout resilience strategy.</p>
<p>Polly has the following built-in resilience strategies:</p>
<ul>
<li><strong>Retry</strong>: The classic &quot;try again&quot; approach.
Works great for temporary network glitches.
You can configure how many retries you have and even add some randomness (jitter) to avoid overloading the system if everyone retries at once.</li>
<li><strong>Circuit-breaker</strong>: Like an electrical circuit breaker, this prevents hammering a failing system.
If errors pile up, the circuit breaker &quot;trips&quot; temporarily to give the system time to recover.</li>
<li><strong>Fallback</strong>: Provides a safe, default response if your primary call fails.
It might be a cached result or a simple &quot;service unavailable&quot; message.</li>
<li><strong>Hedging</strong>: Makes multiple requests simultaneously, taking the first successful response.
It is helpful if your system has numerous ways of handling something.</li>
<li><strong>Timeout</strong>: Prevents requests from hanging forever by terminating them if the timeout is exceeded.</li>
<li><strong>Rate-limiter</strong>: Throttles outgoing requests to prevent overwhelming external services.</li>
</ul>
<h2>HTTP Request Resilience</h2>
<p>Sending HTTP calls to external services is how your application interacts with the outside world.
These could be third-party services like payment gateways and identity providers or other services your team owns and operates.</p>
<p>The <code>Microsoft.Extensions.Http.Resilience</code> library comes with ready-to-use resilience pipelines for sending HTTP requests.</p>
<p>We can add resilience to outgoing <a href="the-right-way-to-use-httpclient-in-dotnet"><strong>HttpClient requests</strong></a> using the <code>AddStandardResilienceHandler</code> method.</p>
<pre><code class="language-csharp">services.AddHttpClient&lt;GitHubService&gt;(static (httpClient) =&gt;
{
    httpClient.BaseAddress = new Uri(&quot;https://api.github.com/&quot;);
})
.AddStandardResilienceHandler();
</code></pre>
<p>This also means you can eliminate any <a href="extending-httpclient-with-delegating-handlers-in-aspnetcore"><strong>delegating handlers</strong></a> you previously used for resilience.</p>
<p>The standard resilience handler combines five Polly strategies to create a resilience pipeline suitable for most scenarios.
The standard pipeline contains the following strategies:</p>
<ul>
<li><strong>Rate limiter</strong>: Limits the maximum number of concurrent requests sent to the dependency.</li>
<li><strong>Total request timeout</strong>: Introduces a total timeout, including any retry attempts.</li>
<li><strong>Retry</strong>: Retries a request if it fails because of a timeout or a transient error.</li>
<li><strong>Circuit breaker</strong>: Prevents sending further requests if too many failures are detected.</li>
<li><strong>Attempt timeout</strong>: Introduces a timeout for an individual request.</li>
</ul>
<p>You can customize any aspect of the standard resilience pipeline by configuring the <code>HttpStandardResilienceOptions</code>.</p>
<h2>Takeaway</h2>
<p>Resilience isn't just a buzzword; it's a core principle for building reliable software systems.
We're fortunate to have powerful tools like <code>Microsoft.Extensions.Resilience</code> and Polly at our disposal.
We can use them to design systems that gracefully handle any transient failures.</p>
<p>Good <a href="introduction-to-distributed-tracing-with-opentelemetry-in-dotnet"><strong>monitoring and observability</strong></a>
are essential to understand how your resilience mechanisms work in production.
Remember, the goal isn't to eliminate failures but to gracefully handle them and keep your application functioning.</p>
<p>Ready to dive deeper into resilient architecture?
My advanced course on <a href="/modular-monolith-architecture"><strong>building modular monoliths</strong></a> will equip you with the skills to design and implement robust, scalable systems.
Check out <a href="/modular-monolith-architecture"><strong>Modular Monolith Architecture</strong></a>.</p>
<p><strong>Challenge</strong>: Take a look at your existing .NET projects.
Are there any critical areas where a little resilience could go a long way?
Pick one and try applying some of the techniques we've discussed here.</p>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_089.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Implementing API Gateway Authentication With YARP]]></title>
            <link>https://milanjovanovic.tech/blog/implementing-api-gateway-authentication-with-yarp</link>
            <guid isPermaLink="false">implementing-api-gateway-authentication-with-yarp</guid>
            <pubDate>Sat, 04 May 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[In this newsletter, we'll explore how you can implement API gateway authentication using YARP (Yet Another Reverse Proxy), a powerful and flexible reverse proxy library for .NET applications.]]></description>
            <content:encoded><![CDATA[<p>API gateways provide clients with a single point of entry.
This streamlines their interactions with your system and ensures the security of your microservices or distributed system.</p>
<p>One critical aspect of API gateways is authentication - ensuring only authorized users and applications can access your valuable data and resources.</p>
<p>In this newsletter, we'll explore how you can implement API gateway authentication using <a href="https://microsoft.github.io/reverse-proxy/index.html">YARP</a> (Yet Another Reverse Proxy),
a powerful and flexible reverse proxy library for .NET applications.</p>
<p>Here's what we will cover:</p>
<ul>
<li>The role of API gateways</li>
<li>Configuring authentication with YARP</li>
<li>Creating custom authorization policies</li>
</ul>
<p>Let's dive in.</p>
<h2>The Role of API Gateways</h2>
<p>An <a href="implementing-an-api-gateway-for-microservices-with-yarp"><strong>API gateway</strong></a> is the &quot;front door&quot; to your backend services and APIs.
It acts as an intermediary layer, handling client requests and routing them to the appropriate destinations.</p>
<p>The key benefits of API gateways are:</p>
<ul>
<li><strong>Centralized access</strong>: All incoming requests must first pass through the gateway. This simplifies management and monitoring.</li>
<li><strong>Service abstraction</strong>: Clients interact only with the gateway. We can hide the complexity of the backend architecture from clients.</li>
<li><strong>Performance enhancement</strong>: Implement techniques like caching and <a href="horizontally-scaling-aspnetcore-apis-with-yarp-load-balancing"><strong>load balancing</strong></a> to optimize API performance.</li>
<li><strong>Authentication and Authorization</strong>: API gateways verify user and application identities, enforcing whether a request is allowed or not.</li>
</ul>
<figure>
  ![API Gateway diagram.](/blogs/mnw_088/api_gateway.png)
  <figcaption>
    Source: [Modular Monolith Architecture](/modular-monolith-architecture)
  </figcaption>
</figure>
<h2>Configuring Authentication With YARP</h2>
<p>We can use the API gateway to authenticate and authorize requests before they are proxied to the destination servers.
This can reduce the load on the destination servers, and introduce a layer of security.
Implementing authentication on the API gateway ensures consistent policies are implemented across your applications.</p>
<p>If you're new to YARP, I recommend first reading about <a href="implementing-an-api-gateway-for-microservices-with-yarp"><strong>how to implement an API gateway with YARP</strong></a>.</p>
<p>By default, YARP won't authenticate or authorize requests unless enabled in the route or application configuration.</p>
<p>We can start by introducing <a href="https://docs.microsoft.com/aspnet/core/security/authentication/">authentication</a>
and <a href="https://learn.microsoft.com/en-us/aspnet/core/security/authorization/introduction">authorization</a> middleware:</p>
<pre><code class="language-csharp">app.UseAuthentication();
app.UseAuthorization();

app.MapReverseProxy();
</code></pre>
<p>This allows us to configure the authorization policy by providing the <code>AuthorizationPolicy</code> value in the route configuration.</p>
<p>There are two special values we can specify in a route's authorization parameter:</p>
<ul>
<li><code>default</code> - The route will require an authenticated user.</li>
<li><code>anonymous</code> - The route will not require authorization regardless of any other configuration.</li>
</ul>
<p>Here's how we can enforce that all incoming requests must be authenticated:</p>
<pre><code class="language-json">{
  // This is how we define reverse proxy routes.
  &quot;Routes&quot;: {
    &quot;api-route&quot;: {
      &quot;ClusterId&quot;: &quot;api-cluster&quot;,
      &quot;AuthorizationPolicy&quot;: &quot;default&quot;,
      &quot;Match&quot;: {
        &quot;Path&quot;: &quot;api/{**catch-all}&quot;
      }
    }
  }
}
</code></pre>
<p>We want to authorize any incoming request as soon as it hits the API gateway.
However, the destination server may still need to know who the user is (authentication) and what they can do (authorization).</p>
<p>YARP will pass any credentials to the proxied request.
By default, cookies, bearer tokens, and API keys will flow to the destination server.</p>
<h2>Creating Custom Authentication Policies</h2>
<p>YARP can utilize the powerful <a href="https://learn.microsoft.com/en-us/aspnet/core/security/authorization/policies">authorization policies</a> feature in <a href="http://ASP.NET">ASP.NET</a> Core.
We can specify a policy per route in the proxy configuration, and the rest is handled by existing <a href="http://ASP.NET">ASP.NET</a> Core authentication and authorization components.</p>
<pre><code class="language-json">{
  // This is how we define auth policies for reverse proxy routes.
  &quot;Routes&quot;: {
    &quot;api-route1&quot;: {
      &quot;ClusterId&quot;: &quot;api-cluster&quot;,
      &quot;AuthorizationPolicy&quot;: &quot;is-vip&quot;,
      &quot;Match&quot;: {
        &quot;Path&quot;: &quot;api/hello-vip&quot;
      }
    },
    &quot;api-route2&quot;: {
      &quot;ClusterId&quot;: &quot;api-cluster&quot;,
      &quot;AuthorizationPolicy&quot;: &quot;default&quot;,
      &quot;Match&quot;: {
        &quot;Path&quot;: &quot;api/{**catch-all}&quot;
      }
    }
  }
}
</code></pre>
<p>Here's how we can create a custom <code>is-vip</code> policy with two components.
It requires an authenticated user and <code>vip</code> claim with one of the defined allowed values to be present .
To use this policy, we can just specify it as the value for the <code>AuthorizationPolicy</code> in the route configuration.</p>
<pre><code class="language-csharp">services.AddAuthorization(options =&gt;
{
    options.AddPolicy(&quot;is-vip&quot;, policy =&gt;
        policy
            .RequireAuthenticatedUser()
            .RequireClaim(&quot;vip&quot;, allowedValues: true.ToString()));
});
</code></pre>
<h2>Summary</h2>
<p>API gateways provide a unified access point, streamlining client interactions and securing your backend services.
Authentication is an essential element of API gateway security, controlling who can access your resources.</p>
<p>YARP offers a versatile solution for building .NET API gateways.
By integrating with <a href="http://ASP.NET">ASP.NET</a> Core's authentication and authorization frameworks, YARP enables robust security mechanisms.</p>
<p>This flexibility really shines with support for custom authorization policies.
This allows you to define <a href="master-claims-transformation-for-flexible-aspnetcore-authorization"><strong>granular access control</strong></a>
based on user roles, claims, or other attributes.</p>
<p>Thanks for reading, and I'll see you next week.</p>
<p><strong>P.S.</strong> Here's the complete <a href="https://github.com/m-jovanovic/yarp-authentication">source code</a> for this article if you want to try it out.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_088.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Request-Response Messaging Pattern With MassTransit]]></title>
            <link>https://milanjovanovic.tech/blog/request-response-messaging-pattern-with-masstransit</link>
            <guid isPermaLink="false">request-response-messaging-pattern-with-masstransit</guid>
            <pubDate>Sat, 27 Apr 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[When building distributed systems with .NET, direct calls between services can create tight coupling. The request-response messaging pattern can allow distributed services to communicate in a loosely coupled way.]]></description>
            <content:encoded><![CDATA[<p>Building distributed applications might seem simple at first. It's just servers talking to each other. Right?</p>
<p>However, it opens a set of potential problems you must consider.
What if the network has a hiccup?
A service unexpectedly crashes?
You try to scale, and everything crumbles under the load?
This is where the way your distributed system communicates becomes critical.</p>
<p>Traditional synchronous communication, where services call each other directly, is inherently fragile.
It creates tight coupling, making your whole application vulnerable to single points of failure.</p>
<p>To combat this, we can turn to distributed messaging
(and introduce an entirely different set of problems, but that's a story for another issue).</p>
<p>One powerful tool for achieving this in the .NET world is MassTransit.</p>
<p>In this week's issue, we'll explore MassTransit's implementation of the request-response pattern.</p>
<h2>Request-Response Messaging Pattern Introduction</h2>
<p>Let's start by explaining how the request-response messaging pattern works.</p>
<p>The request-response pattern is just like making a traditional function call but over the network.
One service, the requester, sends a request message and waits for a corresponding response message.
This is a <a href="modular-monolith-communication-patterns"><strong>synchronous communication approach</strong></a> from the requester's side.</p>
<p>The good parts:</p>
<ul>
<li><strong>Loose Coupling</strong>: Services don't need direct knowledge of each other, only of the message contracts.
This makes changes and scaling easier.</li>
<li><strong>Location Transparency</strong>: The requester doesn't need to know <em>where</em> the responder is located, leading to improved flexibility.</li>
</ul>
<p>The bad parts:</p>
<ul>
<li><strong>Latency</strong>: The overhead of messaging adds some additional latency.</li>
<li><strong>Complexity</strong>: Introducing a messaging system and managing the additional infrastructure can increase project complexity.</li>
</ul>
<p><img src="/blogs/mnw_087/request_response.png" alt="Request response messaging pattern diagram."></p>
<h2>Request-Response Messaging With MassTransit</h2>
<p><a href="using-masstransit-with-rabbitmq-and-azure-service-bus"><strong>MassTransit</strong></a>
supports the <a href="https://masstransit.io/documentation/concepts/requests">request-response</a> messaging pattern out of the box.
We can use a <strong>request client</strong> to send requests and wait for a response.
The request client is asynchronous and supports the <code>await</code> keyword.
The request will also have a timeout of 30 seconds by default, to prevent waiting for the response for too long.</p>
<p>Let's imagine a scenario where you have an order processing system that needs to fetch an order's latest status.
We can fetch the status from an Order Management service.
With MassTransit, you'll create a request client to initiate the process.
This client will send a <code>GetOrderStatusRequest</code> message onto the bus.</p>
<pre><code class="language-csharp">public record GetOrderStatusRequest
{
    public string OrderId { get; init; }
}
</code></pre>
<p>On the Order Management side, a responder (or consumer) will be listening for <code>GetOrderStatusRequest</code> messages.
It receives the request, potentially queries a database to get the status,
and then sends a <code>GetOrderStatusResponse</code> message back onto the bus.
The original request client will be waiting for this response and can then process it accordingly.</p>
<pre><code class="language-csharp">public class GetOrderStatusRequestConsumer : IConsumer&lt;GetOrderStatusRequest&gt;
{
    public async Task Consume(ConsumeContext&lt;GetOrderStatusRequest&gt; context)
    {
        // Get the order status from a database.

        await context.ResponseAsync&lt;GetOrderStatusResponse&gt;(new
        {
            // Set the respective response properties.
        });
    }
}
</code></pre>
<h2>Getting User Permissions In a Modular Monolith</h2>
<p>Here's a real-world scenario where my team decided to implement this pattern.
We were building a <a href="/modular-monolith-architecture"><strong>modular monolith</strong></a>,
and one of the modules was responsible for managing user permissions.
The other modules could call out to the Users module to get the user's permissions.
And this works great while we are still inside a monolith system.</p>
<p>However, at one point we needed to extract one module into a separate service.
This meant that the communication with the Users module using simple method calls would no longer work.</p>
<p>Luckily, we were already using MassTransit and RabbitMQ for messaging inside the system.</p>
<p>So, we decided to use the MassTransit request-response feature to implement this.</p>
<p>The new service will inject an <code>IRequestClient&lt;GetUserPermissions&gt;</code>.
We can use it to send a <code>GetUserPermissions</code> message and await a response.</p>
<p>A very powerful feature of MassTransit is that you can await more than one response message.
In this example, we're waiting for a <code>PermissionsResponse</code> or an <code>Error</code> response.
This is great, because we also have a way to handle failures in the consumer.</p>
<pre><code class="language-csharp">internal sealed class PermissionService(
    IRequestClient&lt;GetUserPermissions&gt; client)
    : IPermissionService
{
    public async Task&lt;Result&lt;PermissionsResponse&gt;&gt; GetUserPermissionsAsync(
        string identityId)
    {
        var request = new GetUserPermissions(identityId);

        Response&lt;PermissionsResponse, Error&gt; response =
            await client.GetResponse&lt;PermissionsResponse, Error&gt;(request);

        if (response.Is(out Response&lt;Error&gt; errorResponse))
        {
            return Result.Failure&lt;PermissionsResponse&gt;(errorResponse.Message);
        }

        if (response.Is(out Response&lt;PermissionsResponse&gt; permissionResponse))
        {
            return permissionResponse.Message;
        }

        return Result.Failure&lt;PermissionsResponse&gt;(NotFound);
    }
}
</code></pre>
<p>In the Users module, we can easily implement the <code>GetUserPermissionsConsumer</code>.
It will respond with a <code>PermissionsResponse</code> if the permissions are found or an <code>Error</code> in case of a failure.</p>
<pre><code class="language-csharp">public sealed class GetUserPermissionsConsumer(
    IPermissionService permissionService)
    : IConsumer&lt;GetUserPermissions&gt;
{
    public async Task Consume(ConsumeContext&lt;GetUserPermissions&gt; context)
    {
        Result&lt;PermissionsResponse&gt; result =
            await permissionService.GetUserPermissionsAsync(
                context.Message.IdentityId);

        if (result.IsSuccess)
        {
            await context.RespondAsync(result.Value);
        }
        else
        {
            await context.RespondAsync(result.Error);
        }
    }
}
</code></pre>
<h2>Closing Thoughts</h2>
<p>By embracing messaging patterns with MassTransit, you're building on a much sturdier foundation.
Your .NET services are now less tightly coupled, giving you the flexibility to evolve them independently
and weather those inevitable network glitches or service outages.</p>
<p>The <a href="https://youtu.be/NjsoykEOkrk">request-response pattern</a> is a powerful tool in your messaging arsenal.
MassTransit makes it remarkably easy to implement, ensuring that requests and responses are delivered reliably.</p>
<p>We can use request-response to implement communication between modules in a <a href="/modular-monolith-architecture"><strong>modular monolith</strong></a>.
However, don't take this to the extreme, or your system might suffer from increased latency.</p>
<p>Start small, experiment, and see how the reliability and flexibility of messaging can transform your development experience.</p>
<p>That's all for this week. Stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_087.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Introduction to Distributed Tracing With OpenTelemetry in .NET]]></title>
            <link>https://milanjovanovic.tech/blog/introduction-to-distributed-tracing-with-opentelemetry-in-dotnet</link>
            <guid isPermaLink="false">introduction-to-distributed-tracing-with-opentelemetry-in-dotnet</guid>
            <pubDate>Sat, 20 Apr 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Distributed systems offer flexibility but introduce complexity, making troubleshooting a headache. Understanding how requests flow through your system is crucial for debugging and performance optimization. OpenTelemetry is an open-source observability framework that makes this possible.]]></description>
            <content:encoded><![CDATA[<p>If you're building or maintaining distributed .NET applications, understanding how they behave is key to ensuring reliability and performance.</p>
<p>Distributed systems offer flexibility but introduce complexity, making troubleshooting a headache.
Understanding how requests flow through your system is crucial for debugging and performance optimization.</p>
<p>OpenTelemetry is an open-source observability framework that makes this possible.</p>
<p>In this article, we'll dive into what OpenTelemetry is, how to use it in your .NET projects, and the powerful insights it provides.</p>
<h2>OpenTelemetry Introduction</h2>
<p><a href="https://opentelemetry.io/">OpenTelemetry</a> (OTel) is a vendor-neutral, open-source standard for instrumenting applications to generate telemetry data.
OpenTelemetry contains APIs, SDKs, tools, and integrations for creating and managing this telemetry data (traces, metrics, and logs).</p>
<p>Telemetry data includes:</p>
<ul>
<li><strong>Traces</strong>: Represent the flow of requests through distributed systems, showing timings and relationships between services.</li>
<li><strong>Metrics</strong>: Numerical measurements of system behavior over time (e.g., request counts, error rates, memory usage).</li>
<li><strong>Logs</strong>: Textual records of events with rich contextual information. Structured logs.</li>
</ul>
<figure>
  ![OpenTelemetry Reference Architecture.](/blogs/mnw_086/otel.png)
  <figcaption>
    Source: [https://opentelemetry.io/docs/](https://opentelemetry.io/docs/)
  </figcaption>
</figure>
<p>OpenTelemetry provides a unified way to collect this data, making it easier to understand the behavior and health of complex distributed applications.</p>
<p>We can export the telemetry data we are collecting to a service capable of processing it and providing us with an interface to analyze it.</p>
<p>We're going to configure OpenTelemetry to export traces directly to <a href="https://www.jaegertracing.io/">Jaeger</a>.</p>
<h2>Adding OpenTelemetry to .NET Applications</h2>
<p>OpenTelemetry provides libraries and SDKs to add code (instrumentation) into your .NET applications.
These instrumentations automatically capture the traces, metrics, and logs we are interested in.</p>
<p>We're going to install the following NuGet packages:</p>
<pre><code class="language-powershell"># Automatic tracing, metrics
Install-Package OpenTelemetry.Extensions.Hosting

# Telemetry data exporter
Install-Package OpenTelemetry.Exporter.OpenTelemetryProtocol

# Instrumentation packages
Install-Package OpenTelemetry.Instrumentation.Http
Install-Package OpenTelemetry.Instrumentation.AspNetCore
Install-Package OpenTelemetry.Instrumentation.EntityFrameworkCore
Install-Package OpenTelemetry.Instrumentation.StackExchangeRedis
Install-Package Npgsql.OpenTelemetry
</code></pre>
<p>Once we have these NuGet packages installed, it's time to configure some services.</p>
<pre><code class="language-csharp">services
    .AddOpenTelemetry()
    .ConfigureResource(resource =&gt; resource.AddService(serviceName))
    .WithTracing(tracing =&gt;
    {
        tracing
            .AddAspNetCoreInstrumentation()
            .AddHttpClientInstrumentation()
            .AddEntityFrameworkCoreInstrumentation()
            .AddRedisInstrumentation()
            .AddNpgsql();

        tracing.AddOtlpExporter();
    });
</code></pre>
<ul>
<li><code>AddAspNetCoreInstrumentation</code> - This enables <a href="http://ASP.NET">ASP.NET</a> Core instrumentation.</li>
<li><code>AddHttpClientInstrumentation</code> - This enables <code>HttpClient</code> instrumentation for outgoing requests.</li>
<li><code>AddEntityFrameworkCoreInstrumentation</code> - This enables EF Core instrumentation.</li>
<li><code>AddRedisInstrumentation</code> - This enables Redis instrumentation.</li>
<li><code>AddNpgsql</code> - This enables PostgreSQL instrumentation.</li>
</ul>
<p>With all of these instrumentations configured, our application will start collecting a lot of valuable traces at runtime.</p>
<p>We also need to configure an environment variable for the exporter added with <code>AddOtlpExporter</code> to work correctly.
We can set <code>OTEL_EXPORTER_OTLP_ENDPOINT</code> through application settings.
The address specified here will point to a local Jaeger instance.</p>
<pre><code>OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317
</code></pre>
<h2>Running Jaeger Locally</h2>
<p><a href="https://www.jaegertracing.io/">Jaeger</a> is an open source, distributed tracing platform.
Jaeger maps the flow of requests and data as they travel through a distributed system.
These requests could be calling out to multiple services, and Jaeger knows how to piece all of this information together.</p>
<p>Here's how to run Jaeger inside a Docker container:</p>
<pre><code>docker run -d -p 4317:4317 -p 16686:16686 jaegertracing/all-in-one:latest
</code></pre>
<p>We're using the <code>jaegertracing/all-in-one:latest</code> image, and exposing the <code>4317</code> to accept telemetry data.
The Jaeger user interface will be exposed on the <code>16686</code> port.</p>
<h2>Distributed Tracing</h2>
<p>After installing the OpenTelemetry libraries and configuring tracing in our applications, we can send some requests to generate telemetry data.
We can then access Jaeger to start analyzing our distributed traces.</p>
<p><strong>Registering a new user</strong></p>
<p>Here's an example of registering a new user with the system.
We're accessing the API gateway (<code>Evently.Gateway</code>) service, which proxies the request to the <code>Evently.Api</code> service.
And you can see that the <code>Evently.Api</code> service makes a few HTTP requests before persisting a new record in the database.</p>
<p><img src="/blogs/mnw_086/trace_1.png" alt="Distributed trace."></p>
<p><strong>Publishing a message with MassTransit</strong></p>
<p>Here's another distributed trace where we publish the <code>UserRegisteredIntegrationEvent</code> over a message bus.
You can see that it's being consumed by two different services that write some data to the database.</p>
<p><img src="/blogs/mnw_086/trace_2.png" alt="Distributed trace."></p>
<p><strong>Examining additional trace information</strong></p>
<p>Distributed traces can include some useful contextual information.
Here's an example trace representing a database command.
This comes from the PostgreSQL instrumentation, and we can see the SQL query that we are executing.</p>
<p><img src="/blogs/mnw_086/trace_3.png" alt="Distributed trace."></p>
<p><strong>Complex distributed traces</strong></p>
<p>Here's a more complex distributed trace, which includes:</p>
<ul>
<li>Three .NET applications</li>
<li>PostgreSQL database</li>
<li>Redis distributed cache</li>
</ul>
<p>We're sending a request to get the customer's cart.
The request will first hit the API gateway, which proxies it to the <code>Evently.Ticketing.Api</code> service that owns the data.
However, the <code>Evently.Ticketing.Api</code> service needs to reach out to the <code>Evently.Api</code> service to get the authorization information.
And all of this leads to the distributed trace you can see below.</p>
<p><img src="/blogs/mnw_086/trace_4.png" alt="Distributed trace."></p>
<h2>Summary</h2>
<p>Understanding modern applications, especially distributed ones, can be a real mind-bender.
OpenTelemetry is like having X-ray vision into your system.</p>
<p>While adding OpenTelemetry takes some upfront work, consider it an investment.
That investment pays off big time when problems pop up.
Instead of frantic guesswork, you have precise data to zero in on issues fast.</p>
<p>Is OpenTelemetry a magic bullet for all your problems? Nope.</p>
<p>But it's an excellent tool to add to your troubleshooting arsenal, especially as your .NET applications grow and get more complex.</p>
<p>If you're curious where the distributed traces come from, it's from the application we're building in my <a href="/modular-monolith-architecture"><strong>Modular Monolith course</strong></a>.</p>
<p>That's all for today. Stay awesome, and I'll see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_086.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[A Clever Way To Implement Pessimistic Locking in EF Core]]></title>
            <link>https://milanjovanovic.tech/blog/a-clever-way-to-implement-pessimistic-locking-in-ef-core</link>
            <guid isPermaLink="false">a-clever-way-to-implement-pessimistic-locking-in-ef-core</guid>
            <pubDate>Sat, 13 Apr 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Sometimes, especially in high-traffic scenarios, you absolutely need to ensure that only one process can modify a piece of data at a time. Entity Framework Core is a fantastic tool, but it doesn't have a direct mechanism for pessimistic locking. In this article, I'll show you how we can solve that problem with raw SQL queries.]]></description>
            <content:encoded><![CDATA[<p>Sometimes, especially in high-traffic scenarios, you absolutely need to ensure that only one process can modify a piece of data at a time.</p>
<p>Imagine you're building the ticket sales system for a wildly popular concert.
Customers are eagerly grabbing tickets, and the last few could sell out simultaneously.
If you're not careful, multiple customers might think they've secured the final seat, leading to overbooking and disappointment!</p>
<p>Entity Framework Core is a fantastic tool, but it doesn't have a direct mechanism for pessimistic locking.
<a href="solving-race-conditions-with-ef-core-optimistic-locking">Optimistic locking</a> (using versions) can work, but in high-contention scenarios, it can lead to a lot of retries.</p>
<p>So, how can we solve this problem with EF Core?</p>
<h2>The Scenario in More Detail</h2>
<p>Here's a simplified code snippet to illustrate our ticketing challenge:</p>
<pre><code class="language-csharp">public async Task Handle(CreateOrderCommand request)
{
    await using DbTransaction transaction = await unitOfWork
        .BeginTransactionAsync();

    Customer customer = await customerRepository.GetAsync(request.CustomerId);

    Order order = Order.Create(customer);
    Cart cart = await cartService.GetAsync(customer.Id);

    foreach (CartItem cartItem in cart.Items)
    {
        // Uh-oh... what if two requests hit this at the same time?
        TicketType ticketType = await ticketTypeRepository.GetAsync(
            cartItem.TicketTypeId);

        ticketType.UpdateQuantity(cartItem.Quantity);

        order.AddItem(ticketType, cartItem.Quantity, cartItem.Price);
    }

    orderRepository.Insert(order);

    await unitOfWork.SaveChangesAsync();

    await transaction.CommitAsync();

    await cartService.ClearAsync(customer.Id);
}
</code></pre>
<p>The example above is contrived, but it should be enough to explain the problem.
During checkout, we verify the <code>AvailableQuantity</code> for each ticket.</p>
<p>What will happen if we get concurrent requests trying to purchase the same ticket?</p>
<p>The worst-case scenario is we end up &quot;overselling&quot; the tickets.
Concurrent requests could see available tickets for sale and complete the checkout.</p>
<p>So, how do we solve this?</p>
<h2>Raw SQL to the Rescue!</h2>
<p>Since EF Core doesn't offer pessimistic locking directly, we'll dip into a bit of good old-fashioned SQL.
We will replace the <code>GetAsync</code> call to fetch the ticket with <code>GetWithLockAsync</code>.</p>
<p>Thankfully, EF Core makes this easy with <a href="ef-core-raw-sql-queries">raw SQL queries</a>:</p>
<pre><code class="language-csharp">public async Task&lt;TicketType&gt; GetWithLockAsync(Guid id)
{
    return await context
        .TicketTypes
        .FromSql(
            $@&quot;
            SELECT id, event_id, name, price, currency, quantity
            FROM ticketing.ticket_types
            WHERE id = {id}
            FOR UPDATE NOWAIT&quot;) // PostgreSQL: Lock or fail immediately
        .SingleAsync();
}
</code></pre>
<p>Understanding the magic:</p>
<ul>
<li><code>FOR UPDATE NOWAIT</code>: This is the heart of pessimistic locking in PostgreSQL.
It tells the database &quot;Grab this row, lock it for me, and if it's already locked, raise an error right now.&quot;</li>
<li><strong>Error Handling</strong>: We'd wrap our <code>GetWithLockAsync</code> call in a <code>try-catch</code> block to gracefully handle locking failures, either retrying or notifying the user.</li>
</ul>
<p>Since there isn't a built-in way in EF Core to add query hints, we have to write raw SQL queries.
We can use the PostgreSQL <code>SELECT FOR UPDATE</code> statement to acquire a row-level lock on the selected rows.
Any competing transactions will be blocked until the current transaction releases the lock.
This is a very simple way to implement pessimistic locking.</p>
<h2>Flavors of Locking and When to Use Them</h2>
<p>To prevent the operation from waiting for other transactions to release any locked rows, you can combine <code>FOR UPDATE</code> with:</p>
<ul>
<li><code>NO WAIT</code> - Reports an error if the row can't be locked instead of waiting.</li>
<li><code>SKIP LOCKED</code> - Skips any selected rows that cannot be locked.</li>
</ul>
<p>Skipping locked rows comes with a caveat - you will get inconsistent results from the database.
However, this can be useful to avoid lock contention when multiple consumers access a queue-like table.
Implementing the <a href="outbox-pattern-for-reliable-microservices-messaging">Outbox pattern</a> is a great example of this.</p>
<p><strong>SQL Server</strong>: You'd use the <code>WITH (UPDLOCK, READPAST)</code> query hint for a similar effect.</p>
<h2>Pessimistic Locking vs. Serializable Transactions</h2>
<p><strong>Serializable</strong> transactions offer the highest level of data consistency.
They guarantee that all transactions are executed as if they happened in a strict, sequential order, even if they occur simultaneously.
This eliminates the possibility of anomalies like dirty reads (seeing uncommitted data) or non-repeatable reads (data changing between reads).</p>
<p>Here's how it works:</p>
<ul>
<li>When a transaction starts under the Serializable isolation level, the database locks all the data the transaction might access.</li>
<li>These locks are held until the entire transaction is committed or rolled back.</li>
<li>Any other transaction attempting to access the locked data will be blocked until the first transaction releases its locks.</li>
</ul>
<p>While Serializable transactions provide the ultimate isolation, they come with a significant cost:</p>
<ul>
<li><strong>Performance Overhead</strong>: Locking a large chunk of data can severely impact performance,
especially in high-concurrency scenarios.</li>
<li><strong>Deadlocks</strong>: With so much locking happening, there's a higher risk of deadlocks.
These occur when two or more transactions are waiting for locks held by each other, creating a stalemate.</li>
</ul>
<p>Pessimistic locking with <code>SELECT FOR UPDATE</code> offers a more targeted approach to data isolation.
You explicitly lock the specific rows you need to modify.
Other transactions attempting to access the locked rows are blocked until the lock is released.</p>
<p>By locking only the necessary data, pessimistic locking avoids the performance overhead associated with locking everything.
Since you're locking fewer resources, the chances of deadlocks are lower.</p>
<h2>When to Use Each Approach</h2>
<p>The best approach depends on your specific needs:</p>
<ul>
<li><strong>Serializable Transactions</strong>: Ideal for scenarios involving highly sensitive data where even the slightest inconsistency is unacceptable.
Examples include financial transactions and medical record updates.</li>
<li><strong>Pessimistic Locking</strong>: A great choice for most use cases, especially in high-traffic applications.
It provides strong consistency while maintaining good performance and reducing deadlock risks.</li>
</ul>
<h2>Takeaway</h2>
<p>I hope this exploration of pessimistic locking has been helpful.
It's a powerful tool to have in your arsenal if you have scenarios where absolute data consistency is paramount.</p>
<p>Both <strong>Serializable transactions</strong> and pessimistic locking with <code>SELECT FOR UPDATE</code> are excellent options for ensuring data consistency.
Consider the level of isolation required, potential performance impact, and the likelihood of deadlocks when making your choice.</p>
<p>That's all for today. Stay awesome, and I'll see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_085.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Master Claims Transformation for Flexible ASP.NET Core Authorization]]></title>
            <link>https://milanjovanovic.tech/blog/master-claims-transformation-for-flexible-aspnetcore-authorization</link>
            <guid isPermaLink="false">master-claims-transformation-for-flexible-aspnetcore-authorization</guid>
            <pubDate>Sat, 06 Apr 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Claims-based authorization mechanisms are central to modern authorization in ASP.NET Core. However, the access tokens issued by your Identity Provider (IDP) might not always perfectly align with your application's internal authorization needs. The solution? Claims transformation.]]></description>
            <content:encoded><![CDATA[<p><a href="https://learn.microsoft.com/en-us/aspnet/core/security/authorization/claims">Claims-based authorization</a> mechanisms are central to modern authorization in <a href="http://ASP.NET">ASP.NET</a> Core.
However, the access tokens issued by your Identity Provider (IDP) might not always perfectly align with your application's internal authorization needs.</p>
<p>External IDPs like <a href="https://www.microsoft.com/en-us/security/business/identity-access/microsoft-entra-id">Microsoft Entra ID</a> (previously Azure AD) or
<a href="https://auth0.com">Auth0</a> might have their own schema for claims or might not directly issue all the claims your application needs for its authorization logic.</p>
<p>The solution? Claims transformation.</p>
<p>Claims transformation allows you to modify the claims before the application uses them for authorization.</p>
<p>In today's issue, we will:</p>
<ul>
<li>Explore the concept of claims transformation in <a href="http://ASP.NET">ASP.NET</a> Core</li>
<li>Explore the <code>IClaimsTransformation</code> interface with practical examples</li>
<li>Address considerations for security and RBAC (Role-Based Access Control)</li>
</ul>
<h2>How Does Claims Transformation Work?</h2>
<p>They say a picture is worth a thousand words.
In software engineering, we have something called <a href="https://en.wikipedia.org/wiki/Unified_Modeling_Language">UML</a> diagrams that we can use to paint a picture.</p>
<p>Here's a <a href="https://en.wikipedia.org/wiki/Sequence_diagram">sequence diagram</a> showing the claims transformation flow:</p>
<ol>
<li>The user authenticates with the Identity Provider</li>
<li>The user calls the backend API and provides an access token</li>
<li>The backend API performs claims transformation and authorization</li>
<li>If the user is correctly authorized, the backend API returns a response</li>
</ol>
<p><img src="/blogs/mnw_084/claims_transformation_sequence_diagram.png" alt="Claims transformation sequence diagram."></p>
<p>Let's see how to implement this in <a href="http://ASP.NET">ASP.NET</a> Core.</p>
<h2>Simple Claims Transformation</h2>
<p>Claims can be created from any user or identity data issued by a trusted identity provider.
A claim is a name-value pair that represents the subject's identity, not what the subject can do.</p>
<p>The core of <a href="https://learn.microsoft.com/en-us/aspnet/core/security/authentication/claims#extend-or-add-custom-claims-using-iclaimstransformation">claims transformation</a>
in <a href="http://ASP.NET">ASP.NET</a> Core is the <a href="https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.authentication.iclaimstransformation"><code>IClaimsTransformation</code></a> interface.</p>
<p>It exposes a single method to transform claims:</p>
<pre><code class="language-csharp">public interface IClaimsTransformation
{
    Task&lt;ClaimsPrincipal&gt; TransformAsync(ClaimsPrincipal principal);
}
</code></pre>
<p>Here's a simple example of using <code>IClaimsTransformation</code> to add a custom claim:</p>
<pre><code class="language-csharp">internal static class CustomClaims
{
    internal const string CardType = &quot;card_type&quot;;
}

internal sealed class CustomClaimsTransformation : IClaimsTransformation
{
    public Task&lt;ClaimsPrincipal&gt; TransformAsync(ClaimsPrincipal principal)
    {
        if (principal.HasClaim(claim =&gt; claim.Type == CustomClaims.CardType))
        {
            return Task.FromResult(principal);
        }

        ClaimsIdentity claimsIdentity = new ClaimsIdentity();

        claimsIdentity.AddClaim(new Claim(CustomClaims.CardType, &quot;platinum&quot;));

        principal.AddIdentity(claimsIdentity);

        return Task.FromResult(principal);
    }
}
</code></pre>
<p>The <code>CustomClaimsTransformation</code> class should be registered as a service:</p>
<pre><code class="language-csharp">builder.Services
    .AddTransient&lt;IClaimsTransformation, CustomClaimsTransformation&gt;();
</code></pre>
<p>Finally, you can define a custom authorization policy that uses this claim:</p>
<pre><code class="language-csharp">builder.Services.AddAuthorization(options =&gt;
{
    options.AddPolicy(
        &quot;HasPlatinumCard&quot;,
        builder =&gt; builder
            .RequireAuthenticatedUser()
            .RequireClaim(CustomClaims.CardType, &quot;platinum&quot;));
});
</code></pre>
<p>There are a few caveats with using <code>IClaimsTransformation</code> you should be aware of:</p>
<ul>
<li><strong>Might execute multiple times</strong>: The <code>TransformAsync</code> method might get called multiple times.
Claims transformation should be idempotent to avoid adding the same claim multiple times to the <code>ClaimsPrincipal</code>.</li>
<li><strong>Potential performance impact</strong>: Since it's executed on authentication requests, be mindful of your transformation logic's performance,
especially if it involves external calls (database, APIs). Consider caching where appropriate.</li>
</ul>
<h2>Implementing RBAC With Claims Transformation</h2>
<p><a href="https://auth0.com/docs/manage-users/access-control/rbac">Role-Based Access Control (RBAC)</a> is an authorization model where permissions are assigned to roles,
and users are granted roles.
Claims transformation helps implement RBAC smoothly.
By adding role claims and potentially permission claims, authorization logic throughout your application can be simplified.
Another benefit is that you can keep the access token smaller and free of any role or permission claims.</p>
<p>Let's consider a scenario where your application manages resources at a granular level,
but your identity provider only provides coarse-grained roles like <code>Registered</code> or <code>Member</code>.
You could use claims transformation to map the <code>Member</code> role to specific fine-grained permissions like <code>SubmitOrder</code> and <code>PurchaseTicket</code>.</p>
<p>Here's a more complex <code>CustomClaimsTransformation</code> implementation.
We send a database query using <code>GetUserPermissionsQuery</code> and get the <code>PermissionsResponse</code> back.
The <code>PermissionsResponse</code> contains the user's permissions, which are added as custom claims.</p>
<pre><code class="language-csharp">internal sealed class CustomClaimsTransformation(
    IServiceProvider serviceProvider)
    : IClaimsTransformation
{
    public async Task&lt;ClaimsPrincipal&gt; TransformAsync(
        ClaimsPrincipal principal)
    {
        if (principal.HasClaim(c =&gt; c.Type == CustomClaims.Sub ||
                                    c.Type == CustomClaims.Permission))
        {
            return principal;
        }

        using IServiceScope scope = serviceProvider.CreateScope();

        ISender sender = scope.ServiceProvider.GetRequiredService&lt;ISender&gt;();

        string identityId = principal.GetIdentityId();

        Result&lt;PermissionsResponse&gt; result = await sender.Send(
            new GetUserPermissionsQuery(identityId));

        if (result.IsFailure)
        {
            throw new ClaimsAuthorizationException(
                nameof(GetUserPermissionsQuery), result.Error);
        }

        var claimsIdentity = new ClaimsIdentity();

        claimsIdentity.AddClaim(
            new Claim(CustomClaims.Sub, result.Value.UserId.ToString()));

        foreach (string permission in result.Value.Permissions)
        {
            claimsIdentity.AddClaim(
                new Claim(CustomClaims.Permission, permission));
        }

        principal.AddIdentity(claimsIdentity);

        return principal;
    }
}
</code></pre>
<p>Now that the <code>ClaimsPrincipal</code> contains the permissions as custom claims, you can do some interesting things.
For example, you can implement a permission-based <code>AuthorizationHandler</code>:</p>
<pre><code class="language-csharp">internal sealed class PermissionAuthorizationHandler
    : AuthorizationHandler&lt;PermissionRequirement&gt;
{
    protected override Task HandleRequirementAsync(
        AuthorizationHandlerContext context,
        PermissionRequirement requirement)
    {
        HashSet&lt;string&gt; permissions = context.User.GetPermissions();

        if (permissions.Contains(requirement.Permission))
        {
            context.Succeed(requirement);
        }

        return Task.CompletedTask;
    }
}
</code></pre>
<h2>Takeaway</h2>
<p>Claims transformation is an elegant way to bridge the gap between claims provided by identity providers and the needs of your <a href="http://ASP.NET">ASP.NET</a> Core application.
The <code>IClaimsTransformation</code> interface enables you to customize the claims of the current <code>ClaimsPrincipal</code>.
Whether you need to add roles, map external groups to internal permissions, or extract additional information from a user profile,
claims transformation offers the flexibility to do so.</p>
<p>However, it's important to use claims transformation with a few key considerations in mind:</p>
<ul>
<li>Claims transformations are executed on each request.</li>
<li>The <code>IClaimsTransformation</code> should be idempotent. It should not add existing claims to the <code>ClaimsPrincipal</code> if executed multiple times.</li>
<li>Design your transformations efficiently, and consider caching the results if you're fetching external data to enrich your claims.</li>
</ul>
<p>If you want to see a complete implementation of RBAC in <a href="http://ASP.NET">ASP.NET</a> Core, check out this <a href="https://www.youtube.com/playlist?list=PLYpjLpq5ZDGtJOHUbv7KHuxtYLk1nJPw5">Authentication &amp; Authorization playlist</a>.</p>
<p>Hope this was helpful.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_084.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Horizontally Scaling ASP.NET Core APIs With YARP Load Balancing]]></title>
            <link>https://milanjovanovic.tech/blog/horizontally-scaling-aspnetcore-apis-with-yarp-load-balancing</link>
            <guid isPermaLink="false">horizontally-scaling-aspnetcore-apis-with-yarp-load-balancing</guid>
            <pubDate>Sat, 30 Mar 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[When a single server reaches its limits, performance degrades, leading to slow response times, errors, or complete downtime. We'll dive into load balancing, why it matters, and how YARP simplifies the process for .NET applications.]]></description>
            <content:encoded><![CDATA[<p>Modern web applications need to serve increasing numbers of users and handle surges in traffic.
When a single server reaches its limits, performance degrades, leading to slow response times, errors, or complete downtime.</p>
<p>Load balancing is a key technique to address these challenges and improve the scalability of your application.</p>
<p>In this article, we will explore:</p>
<ul>
<li>How to use <a href="implementing-an-api-gateway-for-microservices-with-yarp">YARP (Yet Another Reverse Proxy)</a> to implement load balancing</li>
<li>How to leverage horizontal scaling for performance gains</li>
<li>How to utilize K6 as a load testing tool</li>
</ul>
<p>We'll dive into load balancing, why it matters, and how YARP simplifies the process for .NET applications.</p>
<h2>Types of Software Scalability</h2>
<p>Before exploring YARP and load balancing further, let's cover the fundamentals of scaling.</p>
<p>There are two main approaches:</p>
<ul>
<li><strong>Vertical Scaling</strong>: Involves upgrading individual servers with more powerful hardware - more CPU cores, RAM, and faster storage.
However, this has a few limitations: costs escalate quickly, and you'll still hit a performance ceiling.</li>
<li><strong>Horizontal Scaling</strong>: Involves adding more servers to your infrastructure and distributing the load intelligently among them.
This approach offers greater scalability potential, as you can continue adding servers to handle more traffic.</li>
</ul>
<p>Horizontal scaling is where load balancing comes in, and YARP shines bright in this approach.</p>
<h2>Adding a Reverse Proxy</h2>
<p><a href="https://microsoft.github.io/reverse-proxy/index.html">YARP</a> is a high-performance reverse proxy library from Microsoft.
It's designed with modern microservice architectures in mind.
A reverse proxy sits in front of your backend servers, acting as a traffic director.</p>
<p>Setting up YARP is quite straightforward.
You'll install the YARP NuGet package, create a basic configuration to define your backend destinations, and then activate the YARP middleware.
YARP allows you to perform routing and transformation tasks on incoming requests before they reach your backend servers.</p>
<p>First, let's install the <code>Yarp.ReverseProxy</code> NuGet package:</p>
<pre><code class="language-powershell">Install-Package Yarp.ReverseProxy
</code></pre>
<p>Then, we're going to configure the required application services and introduce the YARP middleware to the request pipeline:</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);

builder.Services.AddReverseProxy()
    .LoadFromConfig(builder.Configuration.GetSection(&quot;ReverseProxy&quot;));

var app = builder.Build();

app.MapReverseProxy();

app.Run();
</code></pre>
<p>All that's left is adding the YARP configuration to our <code>appsettings.json</code> file.
YARP uses <code>Routes</code> to represent incoming requests to the reverse proxy and <code>Clusters</code> to define the downstream services.
The <code>{**catch-all}</code> pattern allows us to easily route all incoming requests.</p>
<pre><code class="language-json">{
  &quot;ReverseProxy&quot;: {
    &quot;Routes&quot;: {
      &quot;api-route&quot;: {
        &quot;ClusterId&quot;: &quot;api-cluster&quot;,
        &quot;Match&quot;: {
          &quot;Path&quot;: &quot;{**catch-all}&quot;
        },
        &quot;Transforms&quot;: [{ &quot;PathPattern&quot;: &quot;{**catch-all}&quot; }]
      }
    },
    &quot;Clusters&quot;: {
      &quot;api-cluster&quot;: {
        &quot;Destinations&quot;: {
          &quot;destination1&quot;: {
            &quot;Address&quot;: &quot;http://api:8080&quot;
          }
        }
      }
    }
  }
}
</code></pre>
<p>This configures YARP as a pass-through proxy, but let's update it to support horizontal scaling.</p>
<h2>Scaling Out With YARP Load Balancing</h2>
<p>The core of horizontal scaling with YARP lies in its various <a href="https://microsoft.github.io/reverse-proxy/articles/load-balancing.html">load balancing strategies</a>:</p>
<ul>
<li><code>PowerOfTwoChoices</code>: Selects two random destinations and selects the one with the least assigned requests.</li>
<li><code>FirstAlphabetical</code>: Selects the alphabetically first available destination server.</li>
<li><code>LeastRequests</code>: Sends requests to servers with the least assigned requests.</li>
<li><code>RoundRobin</code>: Distributes requests evenly across backend servers.</li>
<li><code>Random</code>: Randomly selects a backend server for each request.</li>
</ul>
<p>You configure these strategies within YARP's configuration file.
The load balancing strategy can be configured using the <code>LoadBalancingPolicy</code> property on the cluster.</p>
<p>Here's what the updated YARP configuration looks like with <code>RoundRobin</code> load balancing:</p>
<pre><code class="language-json">{
  &quot;ReverseProxy&quot;: {
    &quot;Routes&quot;: {
      &quot;api-route&quot;: {
        &quot;ClusterId&quot;: &quot;api-cluster&quot;,
        &quot;Match&quot;: {
          &quot;Path&quot;: &quot;{**catch-all}&quot;
        },
        &quot;Transforms&quot;: [{ &quot;PathPattern&quot;: &quot;{**catch-all}&quot; }]
      }
    },
    &quot;Clusters&quot;: {
      &quot;api-cluster&quot;: {
        &quot;LoadBalancingPolicy&quot;: &quot;RoundRobin&quot;,
        &quot;Destinations&quot;: {
          &quot;destination1&quot;: {
            &quot;Address&quot;: &quot;http://api-1:8080&quot;
          },
          &quot;destination2&quot;: {
            &quot;Address&quot;: &quot;http://api-2:8080&quot;
          },
          &quot;destination3&quot;: {
            &quot;Address&quot;: &quot;http://api-3:8080&quot;
          }
        }
      }
    }
  }
}
</code></pre>
<p>Here's a diagram of what our system could look like with a YARP load balancer and horizontally scaled application servers.</p>
<p>The incoming API requests will first hit YARP, distributing the traffic to the application servers based on the load balancing strategy.
In this example, there's one database serving multiple application instances.</p>
<p><img src="/blogs/mnw_083/horizontal_scaling.png" alt="Horizontal scaling with a YARP load balancer."></p>
<p>And now, let's do some performance testing.</p>
<h2>Performance Testing with K6</h2>
<p>To see the impact of our horizontal scaling efforts, we need to do some load testing.
<a href="https://k6.io/">K6</a> is a modern, developer-friendly load testing tool.
We'll write K6 scripts to simulate user traffic on our application and compare metrics like average response time and the number of successful requests per second.</p>
<p>The application we're going to scale horizontally has two API endpoints.
The <code>POST /users</code> endpoint creates a new user, saves the user to a <a href="https://www.postgresql.org/">PostgreSQL</a> database, and returns the user's identifier.
The <code>GET /users/id</code> endpoint returns a user with the given identifer if it exists.</p>
<p>Here's a k6 performance test that will:</p>
<ul>
<li>Ramp up to <strong>20 virtual users</strong></li>
<li>Send a <code>POST</code> request to the <code>/users</code> endpoint</li>
<li>Check that the response is <code>201 Created</code></li>
<li>Send a <code>GET</code> request to the <code>/users/{id}</code> endpoint</li>
<li>Check that the response is <code>200 OK</code></li>
</ul>
<p>Note that all API requests go through the YARP load balancer.</p>
<pre><code class="language-js">import { check } from 'k6';
import http from 'k6/http';

export const options = {
  stages: [
    { duration: '10s', target: 20 },
    { duration: '1m40s', target: 20 },
    { duration: '10s', target: 0 }
  ]
};

export default function () {
  const proxyUrl = 'http://localhost:3000';

  const response = http.post(`${proxyUrl}/users`);

  check(response, {
    'response code was 201': (res) =&gt; res.status == 201
  });

  const userResponse = http.get(`${proxyUrl}/users/${response.body}`);

  check(userResponse, {
    'response code was 200': (res) =&gt; res.status == 200
  });
}
</code></pre>
<p>To make the performance testing results more consistent, we can limit the available resources on the <a href="https://www.docker.com">Docker</a> containers to <code>1 CPU</code> and <code>0.5G</code> of RAM.</p>
<pre><code class="language-yml">services:
  api:
    image: ${DOCKER_REGISTRY-}loadbalancingapi
    cpus: 1
    mem_limit: '0.5G'
    ports:
      - 5000:8080
    networks:
      - proxybackend
</code></pre>
<p>Finally, here are the k6 performance testing results:</p>
<pre><code>|    API Instances   |     Request Duration     |    Requests Per Second    |
|------------------- |------------------------- |---------------------------:
|         1          |         9.68 ms          |          2260/s           |
|--------------------|------------------------- |---------------------------|
|         2          |         6.57 ms          |          2764/s           |
|--------------------|------------------------- |---------------------------|
|         3          |         5.62 ms          |          3227/s           |
|--------------------|------------------------- |---------------------------|
|         5          |         4.65 ms          |          3881/s           |
</code></pre>
<h2>Summary</h2>
<p>Horizontal scaling, coupled with effective load balancing, can significantly enhance the performance and scalability of your web applications.
The benefits of horizontal scaling become especially apparent in high-traffic scenarios where a single server can no longer cope with the demand.</p>
<p>YARP is a powerful and user-friendly reverse proxy server for .NET applications.
However, highly complex, large-scale distributed systems might benefit from specialized, standalone load-balancing solutions.
These dedicated solutions can offer more granular control and sophisticated features.</p>
<p>If you want to learn more, here's how to <a href="implementing-an-api-gateway-for-microservices-with-yarp">build an API Gateway using YARP</a>.</p>
<p>You can find the <a href="https://github.com/m-jovanovic/yarp-load-balancing">source code</a> for this example on GitHub.</p>
<p>That's all for today. I'll see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_083.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Fast SQL Bulk Inserts With C# and EF Core]]></title>
            <link>https://milanjovanovic.tech/blog/fast-sql-bulk-inserts-with-csharp-and-ef-core</link>
            <guid isPermaLink="false">fast-sql-bulk-inserts-with-csharp-and-ef-core</guid>
            <pubDate>Sat, 23 Mar 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Explore various methods for fast bulk inserts in SQL with C# and EF Core, highlighting techniques like Dapper, EF Core optimizations, EF Core Bulk Extensions, and SQL Bulk Copy.]]></description>
            <content:encoded><![CDATA[<p>Whether you're building a data analytics platform, migrating a legacy system, or onboarding a surge of new users,
there will likely come a time when you'll need to insert a massive amount of data into your database.</p>
<p>Inserting the records one by one feels like watching paint dry in slow motion.
Traditional methods won't cut it.</p>
<p>So, understanding fast bulk insert techniques with C# and EF Core becomes essential.</p>
<p>In today's issue, we'll explore several options for performing bulk inserts in C#:</p>
<ul>
<li>Dapper</li>
<li>EF Core</li>
<li>EF Core Bulk extensions</li>
<li>SQL Bulk Copy</li>
<li>Entity Framework Extensions</li>
</ul>
<p>The examples are based on a <code>User</code> class with a respective <code>Users</code> table in <strong>SQL Server</strong>.</p>
<pre><code class="language-csharp">public class User
{
    public int Id { get; set; }
    public string Email { get; set; }
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public string PhoneNumber { get; set; }
}
</code></pre>
<p>This isn't a complete list of bulk insert implementations.
There are a few options I didn't explore, like manully generating SQL statements
and using <a href="https://learn.microsoft.com/en-us/sql/relational-databases/tables/use-table-valued-parameters-database-engine?view=sql-server-ver16">Table-Valued parameters</a>.</p>
<h2>What Is EF Core Bulk Insert?</h2>
<p>When working with large datasets, standard EF Core patterns quickly become a bottleneck.
Using <code>Add</code> and <code>SaveChanges</code> for thousands of records means EF tracks every entity in memory
and generates individual SQL <code>INSERT</code> statements, each requiring its own round trip to the database.</p>
<p>EF Core bulk insert techniques solve this by taking one of two approaches:
either batching many <code>INSERT</code> statements into a single network round trip (the built-in <code>AddRange</code>),
or bypassing EF's change tracker entirely and streaming data using the database's native bulk copy protocol
(like SQL Server's <code>SqlBulkCopy</code>).
The result can reduce insert time from minutes to seconds for large datasets.</p>
<h2>EF Core Simple Approach</h2>
<p>Let's start with a simple example using EF Core.
We're creating an <code>ApplicationDbContext</code> instance, adding a <code>User</code> object, and calling <code>SaveChangesAsync</code>.
This will insert each record to the database one by one.
In other words, each record requires one round trip to the database.</p>
<pre><code class="language-csharp">using var context = new ApplicationDbContext();

foreach (var user in GetUsers())
{
    context.Users.Add(user);

    await context.SaveChangesAsync();
}
</code></pre>
<p>The results are as poor as you'd expect:</p>
<pre><code>EF Core - Add one and save, for 100 users: 20 ms
EF Core - Add one and save, for 1,000 users: 260 ms
EF Core - Add one and save, for 10,000 users: 8,860 ms
</code></pre>
<p>I omitted the results with <code>100,000</code> and <code>1,000,000</code> records because they took too long to execute.</p>
<p>We'll use this as a &quot;how not to do bulk inserts&quot; example.</p>
<h2>Dapper Simple Insert</h2>
<p><a href="https://github.com/DapperLib/Dapper">Dapper</a> is a simple SQL-to-object mapper for .NET.
It allows us to easily insert a collection of objects into the database.</p>
<p>I'm using Dapper's feature to unwrap a collection into a SQL <code>INSERT</code> statement.</p>
<pre><code class="language-csharp">using var connection = new SqlConnection(connectionString);
connection.Open();

const string sql =
    @&quot;
    INSERT INTO Users (Email, FirstName, LastName, PhoneNumber)
    VALUES (@Email, @FirstName, @LastName, @PhoneNumber);
    &quot;;

await connection.ExecuteAsync(sql, GetUsers());
</code></pre>
<p>The results are much better than the initial example:</p>
<pre><code>Dapper - Insert range, for 100 users: 10 ms
Dapper - Insert range, for 1,000 users: 113 ms
Dapper - Insert range, for 10,000 users: 1,028 ms
Dapper - Insert range, for 100,000 users: 10,916 ms
Dapper - Insert range, for 1,000,000 users: 109,065 ms
</code></pre>
<h2>EF Core Add and Save</h2>
<p>However, EF Core still didn't throw in the towel.
The first example was poorly implemented on purpose.
EF Core can batch multiple SQL statements together, so let's use that.</p>
<p>If we make a simple change, we can get significantly better performance.
First, we're adding all the objects to the <code>ApplicationDbContext</code>.
Then, we're going to call <code>SaveChangesAsync</code> only once.</p>
<p>EF will create a batched SQL statement - group many <code>INSERT</code> statements together - and send them to the database together.
This reduces the number of round trips to the database, giving us improved performance.</p>
<pre><code class="language-csharp">using var context = new ApplicationDbContext();

foreach (var user in GetUsers())
{
    context.Users.Add(user);
}

await context.SaveChangesAsync();
</code></pre>
<p>Here are the benchmark results of this implementation:</p>
<pre><code>EF Core - Add all and save, for 100 users: 2 ms
EF Core - Add all and save, for 1,000 users: 18 ms
EF Core - Add all and save, for 10,000 users: 203 ms
EF Core - Add all and save, for 100,000 users: 2,129 ms
EF Core - Add all and save, for 1,000,000 users: 21,557 ms
</code></pre>
<p>Remember, it took Dapper <strong>109 seconds</strong> to insert <code>1,000,000</code> records.
We can achieve the same with EF Core batched queries in <strong>~21 seconds</strong>.</p>
<h2>EF Core AddRange and Save</h2>
<p>This is an alternative to the previous example.
Instead of calling <code>Add</code> for all objects, we can call <code>AddRange</code> and pass in a collection.</p>
<p>I wanted to show this implementation because I prefer it over the previous one.</p>
<pre><code class="language-csharp">using var context = new ApplicationDbContext();

context.Users.AddRange(GetUsers());

await context.SaveChangesAsync();
</code></pre>
<p>The results are very similar to the previous example:</p>
<pre><code>EF Core - Add range and save, for 100 users: 2 ms
EF Core - Add range and save, for 1,000 users: 18 ms
EF Core - Add range and save, for 10,000 users: 204 ms
EF Core - Add range and save, for 100,000 users: 2,111 ms
EF Core - Add range and save, for 1,000,000 users: 21,605 ms
</code></pre>
<h2>EF Core Bulk Insert with EF Core Bulk Extensions</h2>
<p>There's an awesome library called <a href="https://github.com/borisdj/EFCore.BulkExtensions">EF Core Bulk Extensions</a> that we can use to squeeze out more performance.
You can do a lot more than bulk inserts with this library, so it's worth exploring.
This library is open source, and has a community license if you meet the free usage criteria.
Check the <a href="https://github.com/borisdj/EFCore.BulkExtensions?#license">licensing section</a> for more details.</p>
<p>For our use case, the <code>BulkInsertAsync</code> method is an excellent choice.
We can pass the collection of objects, and it will perform an SQL bulk insert.</p>
<pre><code class="language-csharp">using var context = new ApplicationDbContext();

await context.BulkInsertAsync(GetUsers());
</code></pre>
<p>The performance is equally amazing:</p>
<pre><code>EF Core - Bulk Extensions, for 100 users: 1.9 ms
EF Core - Bulk Extensions, for 1,000 users: 8 ms
EF Core - Bulk Extensions, for 10,000 users: 76 ms
EF Core - Bulk Extensions, for 100,000 users: 742 ms
EF Core - Bulk Extensions, for 1,000,000 users: 8,333 ms
</code></pre>
<p>For comparison, we needed <strong>~21 seconds</strong> to insert <code>1,000,000</code> records with EF Core batched queries.
We can do the same with the <a href="https://github.com/borisdj/EFCore.BulkExtensions">Bulk Extensions</a> library in just <strong>8 seconds</strong>.</p>
<h2>SQL Bulk Insert with SqlBulkCopy</h2>
<p>If we can't get the desired performance from EF Core, we can try using <a href="https://learn.microsoft.com/en-us/dotnet/api/system.data.sqlclient.sqlbulkcopy"><code>SqlBulkCopy</code></a>.
SQL Server supports <a href="https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/sql/bulk-copy-operations-in-sql-server">bulk copy operations</a> natively, so let's use this.</p>
<p>This implementation is slightly more complex than the EF Core examples.
We need to configure the <code>SqlBulkCopy</code> instance and create a <code>DataTable</code> containing the objects we want to insert.</p>
<pre><code class="language-csharp">using var bulkCopy = new SqlBulkCopy(ConnectionString);

bulkCopy.DestinationTableName = &quot;dbo.Users&quot;;

bulkCopy.ColumnMappings.Add(nameof(User.Email), &quot;Email&quot;);
bulkCopy.ColumnMappings.Add(nameof(User.FirstName), &quot;FirstName&quot;);
bulkCopy.ColumnMappings.Add(nameof(User.LastName), &quot;LastName&quot;);
bulkCopy.ColumnMappings.Add(nameof(User.PhoneNumber), &quot;PhoneNumber&quot;);

await bulkCopy.WriteToServerAsync(GetUsersDataTable());
</code></pre>
<p>However, the performance is blazing fast:</p>
<pre><code>SQL Bulk Copy, for 100 users: 1.7 ms
SQL Bulk Copy, for 1,000 users: 7 ms
SQL Bulk Copy, for 10,000 users: 68 ms
SQL Bulk Copy, for 100,000 users: 646 ms
SQL Bulk Copy, for 1,000,000 users: 7,339 ms
</code></pre>
<p>Here's how you can create a <code>DataTable</code> and populate it with a list of objects:</p>
<pre><code class="language-csharp">DataTable GetUsersDataTable()
{
    var dataTable = new DataTable();

    dataTable.Columns.Add(nameof(User.Email), typeof(string));
    dataTable.Columns.Add(nameof(User.FirstName), typeof(string));
    dataTable.Columns.Add(nameof(User.LastName), typeof(string));
    dataTable.Columns.Add(nameof(User.PhoneNumber), typeof(string));

    foreach (var user in GetUsers())
    {
        dataTable.Rows.Add(
            user.Email, user.FirstName, user.LastName, user.PhoneNumber);
    }

    return dataTable;
}
</code></pre>
<h2>Entity Framework Bulk Insert with EF Core Extensions</h2>
<p>Can we do better than <code>SqlBulkCopy</code>?</p>
<p>Maybe, at least my benchmark results suggest that we can.</p>
<p>There's another awesome library called <a href="https://entityframework-extensions.net/?utm_source=milanjovanovic&amp;utm_medium=newsletter">Entity Framework Extensions</a>.
It's much more than just a bulk insert library - so I highly recommend checking it out.
However, we'll use it for bulk inserts today.</p>
<p>For our use case, the <code>BulkInsertOptimizedAsync</code> method is an excellent choice.
We can pass the collection of objects, and it will perform an SQL bulk insert.
It'll also do some optimizations under the hood to improve performance.</p>
<pre><code class="language-csharp">using var context = new ApplicationDbContext();

await context.BulkInsertOptimizedAsync(GetUsers());
</code></pre>
<p>The performance is nothing short of amazing:</p>
<pre><code>EF Core - Entity Framework Extensions, for 100 users: 1.86 ms
EF Core - Entity Framework Extensions, for 1,000 users: 6.9 ms
EF Core - Entity Framework Extensions, for 10,000 users: 66 ms
EF Core - Entity Framework Extensions, for 100,000 users: 636 ms
EF Core - Entity Framework Extensions, for 1,000,000 users: 7,106 ms
</code></pre>
<h2>EF Core Bulk Insert vs SqlBulkCopy vs Dapper</h2>
<p>Not all bulk insert options are equal. Here's a comparison to help you choose the right approach:</p>
<pre><code>| Method                                                 | Relative Speed | EF Core Integration         | Database Support   | License          |
| ------------------------------------------------------ | -------------- | --------------------------- | ------------------ | ---------------- |
| EF Core `AddRange` + `SaveChanges`                     | Medium         | Full (change tracking)      | All EF providers   | Free             |
| Dapper `ExecuteAsync`                                  | Medium         | None                        | Any ADO.NET        | Free             |
| EFCore.BulkExtensions `BulkInsertAsync`                | Fast           | Partial (bypasses tracking) | Multiple providers | Free (community) |
| `SqlBulkCopy`                                          | Fastest        | None                        | SQL Server only    | Free             |
| Entity Framework Extensions `BulkInsertOptimizedAsync` | Fastest        | Partial (bypasses tracking) | Multiple providers | Commercial       |
</code></pre>
<p><strong>EF Core <code>AddRange</code></strong> is the simplest option and works out of the box, but it goes through the change tracker, making it slower for large datasets.</p>
<p><strong>Dapper</strong> is a good fit if you're already using it in your stack, but it doesn't integrate with your EF Core context.</p>
<p><strong>EFCore.BulkExtensions</strong> hits a sweet spot: near-<code>SqlBulkCopy</code> speed, works with your existing <code>DbContext</code>, and is open source.</p>
<p><strong><code>SqlBulkCopy</code></strong> is the raw speed champion for SQL Server but requires boilerplate <code>DataTable</code> setup and is SQL Server-only.</p>
<p><strong>Entity Framework Extensions</strong> matches <code>SqlBulkCopy</code> performance with EF Core integration, but requires a commercial license.</p>
<h2>When Should You Use Each EF Core Bulk Insert Method?</h2>
<p>Choose the right method based on your scenario:</p>
<pre><code>| Scenario                                                              | Recommended Method                 |
| --------------------------------------------------------------------- | ---------------------------------- |
| Small datasets (under 1,000 rows)                                     | EF Core `AddRange` + `SaveChanges` |
| Medium datasets with existing Dapper infrastructure                   | Dapper `ExecuteAsync`              |
| Large datasets, open-source required, EF Core codebase                | `EFCore.BulkExtensions`            |
| Maximum speed, SQL Server only                                        | `SqlBulkCopy`                      |
| Maximum speed with EF Core integration, commercial license acceptable | Entity Framework Extensions        |
| PostgreSQL, MySQL, or other non-SQL Server databases                  | `EFCore.BulkExtensions`            |
</code></pre>
<h2>Results</h2>
<p>Here are the results for all the bulk insert implementations:</p>
<pre><code>| Method             |   Size     |      Speed
|------------------- |----------- |----------------:
| EF_OneByOne        | 100        |      19.800 ms |
| EF_OneByOne        | 1000       |     259.870 ms |
| EF_OneByOne        | 10000      |   8,860.790 ms |
| EF_OneByOne        | 100000     |            N/A |
| EF_OneByOne        | 1000000    |            N/A |

| Dapper_Insert      | 100        |      10.650 ms |
| Dapper_Insert      | 1000       |     113.137 ms |
| Dapper_Insert      | 10000      |   1,027.979 ms |
| Dapper_Insert      | 100000     |  10,916.628 ms |
| Dapper_Insert      | 1000000    | 109,064.815 ms |

| EF_AddAll          | 100        |       2.064 ms |
| EF_AddAll          | 1000       |      17.906 ms |
| EF_AddAll          | 10000      |     202.975 ms |
| EF_AddAll          | 100000     |   2,129.370 ms |
| EF_AddAll          | 1000000    |  21,557.136 ms |

| EF_AddRange        | 100        |       2.035 ms |
| EF_AddRange        | 1000       |      17.857 ms |
| EF_AddRange        | 10000      |     204.029 ms |
| EF_AddRange        | 100000     |   2,111.106 ms |
| EF_AddRange        | 1000000    |  21,605.668 ms |

| BulkExtensions     | 100        |       1.922 ms |
| BulkExtensions     | 1000       |       7.943 ms |
| BulkExtensions     | 10000      |      76.406 ms |
| BulkExtensions     | 100000     |     742.325 ms |
| BulkExtensions     | 1000000    |   8,333.950 ms |

| BulkCopy           | 100        |       1.721 ms |
| BulkCopy           | 1000       |       7.380 ms |
| BulkCopy           | 10000      |      68.364 ms |
| BulkCopy           | 100000     |     646.219 ms |
| BulkCopy           | 1000000    |   7,339.298 ms |

| EF Extensions      | 100        |       1.860 ms |
| EF Extensions      | 1000       |       6.923 ms |
| EF Extensions      | 10000      |      68.106 ms |
| EF Extensions      | 100000     |     636.231 ms |
| EF Extensions      | 1000000    |   7,106.891 ms |
</code></pre>
<h2>Takeaway</h2>
<p><code>SqlBulkCopy</code> holds the crown for maximum raw speed and simplicity.
However, <a href="https://entityframework-extensions.net/?utm_source=milanjovanovic&amp;utm_medium=newsletter">Entity Framework Extensions</a>
deliver fantastic performance while maintaining the ease of use that EF Core is known for.</p>
<p>The best choice hinges on your project's specific demands:</p>
<ul>
<li>Performance is all that matters? <code>SqlBulkCopy</code> is your solution.</li>
<li>Need excellent speed and streamlined development? EF Core is a smart choice.</li>
<li>Want a balance between performance and ease of use? Consider using <a href="https://entityframework-extensions.net/?utm_source=milanjovanovic&amp;utm_medium=newsletter">Entity Framework Extensions</a>.</li>
</ul>
<p>I leave it up to you to decide which option is best for your use case.</p>
<p>Hope this was helpful.</p>
<p>See you next week.</p>
<h2>Frequently Asked Questions</h2>
<h3>What is the fastest way to bulk insert in EF Core?</h3>
<p>The fastest approach depends on your requirements.
<code>EFCore.BulkExtensions</code> and Entity Framework Extensions both offer native bulk insert methods that bypass EF's change tracking and generate optimized SQL.
For pure speed, <code>SqlBulkCopy</code> from <code>System.Data.SqlClient</code> is typically the fastest for SQL Server.</p>
<h3>Does EF Core support bulk insert natively?</h3>
<p>EF Core does not have a dedicated bulk insert API.
Using <code>AddRange</code> with <code>SaveChanges</code> batches SQL inserts but still goes through the change tracker.
For true bulk inserts, you need a library like <code>EFCore.BulkExtensions</code> or raw <code>SqlBulkCopy</code>.</p>
<h3>What is the difference between bulk insert and batch insert in EF Core?</h3>
<p>Batch insert groups multiple <code>INSERT</code> statements into a single round trip using EF Core's built-in batching.
Bulk insert bypasses EF entirely and streams data in a highly optimized binary format (like BCP for SQL Server), making it significantly faster for large datasets.</p>
<h3>Is SqlBulkCopy faster than EF Core BulkExtensions?</h3>
<p><code>SqlBulkCopy</code> is generally the fastest option because it uses SQL Server's native bulk copy protocol and skips all ORM overhead.
<code>EFCore.BulkExtensions</code> is slightly slower but integrates with your existing EF Core context and works across multiple database providers.</p>
<h3>Can you bulk insert with EF Core and keep change tracking?</h3>
<p>No. The primary benefit of bulk insert libraries is that they bypass EF Core's change tracker.
If you need change tracking, use the standard EF Core <code>Add</code> or <code>AddRange</code> approach, but expect significantly worse performance for large datasets.</p>
<h3>Is Dapper good for bulk insert operations?</h3>
<p>Dapper is decent for medium-sized bulk inserts.
It lets you pass a collection to <code>ExecuteAsync</code> and generates individual <code>INSERT</code> statements in one round trip.
It is faster than EF Core's naive approach but slower than <code>SqlBulkCopy</code> or <code>EFCore.BulkExtensions</code> for very large datasets.</p>
<h3>What is EFCore.BulkExtensions and when should you use it?</h3>
<p><code>EFCore.BulkExtensions</code> is an open-source library that extends EF Core with <code>BulkInsert</code>, <code>BulkUpdate</code>, <code>BulkDelete</code>, and <code>BulkMerge</code> methods.
Use it when you need fast bulk operations that still integrate cleanly with your EF Core data model.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_082.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Implementing Soft Delete With EF Core]]></title>
            <link>https://milanjovanovic.tech/blog/implementing-soft-delete-with-ef-core</link>
            <guid isPermaLink="false">implementing-soft-delete-with-ef-core</guid>
            <pubDate>Sat, 16 Mar 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[A soft delete is a data persistence strategy that prevents the permanent deletion of records from your database. Today, we'll dive into the details of how to implement soft deletes using EF Core.]]></description>
            <content:encoded><![CDATA[<p>To delete or not to delete, that is the question (pun intended).</p>
<p>The traditional way to remove information in a database is through a &quot;hard delete.&quot;
A hard delete permanently erases a record from the database table.
While this seems straightforward, it presents a significant risk: once that data is gone, it's gone for good.</p>
<p>Instead of physically removing a record, a soft delete marks it as deleted, usually by setting a flag like <code>IsDeleted</code> to <code>true</code>.
The record remains in the database, but it's effectively hidden from regular application queries.</p>
<p>Today, we'll dive into the details of how to implement soft deletes using EF Core.
We'll discuss global query filters, explore efficient ways to handle soft-deleted data, and weigh the trade-offs.</p>
<h2>What Is a Soft Delete?</h2>
<p>A soft delete is a data persistence strategy that prevents the permanent deletion of records from your database.
Instead of removing data from the database, a flag is set on the record, indicating it as &quot;deleted.&quot;</p>
<p>This approach allows the application to ignore these records during normal queries.
However, you can restore these records if necessary.
Soft delete is also practical if you want to keep foreign key constraints in place.
Soft delete is a &quot;non-destructive&quot; operation in contrast with hard delete, where data is completely removed from the database.</p>
<p>A hard delete uses the SQL <code>DELETE</code> statement:</p>
<pre><code class="language-sql">DELETE FROM bookings.Reviews
WHERE Id = @BookingId;
</code></pre>
<p>A soft delete, on the other hand, uses an <code>UPDATE</code> statement:</p>
<pre><code class="language-sql">UPDATE bookings.Reviews
SET IsDeleted = 1, DeletedOnUtc = @UtcNow
WHERE Id = @BookingId;
</code></pre>
<p>The data is still present in the database, and the operation can be undone.</p>
<p>But you need to remember to filter out soft-deleted data when querying the database:</p>
<pre><code class="language-sql">SELECT *
FROM bookings.Reviews
WHERE IsDeleted = 0;
</code></pre>
<p>Let's see how we can implement soft delete with <a href="https://learn.microsoft.com/en-us/ef/core/">EF Core</a>.</p>
<h2>Soft Deletes Using EF Core Interceptors</h2>
<p><a href="https://learn.microsoft.com/en-us/ef/core/logging-events-diagnostics/interceptors">EF Core interceptors</a>
provide a powerful mechanism for intercepting and modifying database operations.
For example, you can intercept the saving changes operation to implement soft delete functionality.</p>
<p>Let's create an <code>ISoftDeletable</code> marker interface to represent soft-deletable entities:</p>
<pre><code class="language-csharp">public interface ISoftDeletable
{
    bool IsDeleted { get; set; }

    DateTime? DeletedOnUtc { get; set; }
}
</code></pre>
<p>The entities that should support soft delete will implement this interface.
You will need to apply the respective database migration to create these columns.</p>
<p>The next component we need is a <code>SaveChangesInterceptor</code>, which allows us to hook into the <code>SavingChangesAsync</code> (or <code>SavingChanges</code>) method.
We can access the <code>ChangeTracker</code> and look for entries that implement <code>ISoftDeletable</code> and are flagged for deletion.
We can figure this out by checking if the entity state is <code>EntityState.Deleted</code>.</p>
<p>When we find the entities flagged for deletion, we loop through them and update their state to <code>EntityState.Modified</code>.
You should also set the respective values for the <code>IsDeleted</code> and <code>DeletedOnUtc</code> properties.
This will cause EF to generate an <code>UPDATE</code> operation instead of a <code>DELETE</code> operation.</p>
<pre><code class="language-csharp">public sealed class SoftDeleteInterceptor : SaveChangesInterceptor
{
    public override ValueTask&lt;InterceptionResult&lt;int&gt;&gt; SavingChangesAsync(
        DbContextEventData eventData,
        InterceptionResult&lt;int&gt; result,
        CancellationToken cancellationToken = default)
    {
        if (eventData.Context is null)
        {
            return base.SavingChangesAsync(
                eventData, result, cancellationToken);
        }

        IEnumerable&lt;EntityEntry&lt;ISoftDeletable&gt;&gt; entries =
            eventData
                .Context
                .ChangeTracker
                .Entries&lt;ISoftDeletable&gt;()
                .Where(e =&gt; e.State == EntityState.Deleted);

        foreach (EntityEntry&lt;ISoftDeletable&gt; softDeletable in entries)
        {
            softDeletable.State = EntityState.Modified;
            softDeletable.Entity.IsDeleted = true;
            softDeletable.Entity.DeletedOnUtc = DateTime.UtcNow;
        }

        return base.SavingChangesAsync(eventData, result, cancellationToken);
    }
}
</code></pre>
<p>This approach ensures that all delete operations across the application respect the soft delete policy.</p>
<p>You'll need to register the <code>SoftDeleteInterceptor</code> with dependency injection and configure it with the <code>ApplicationDbContext</code>.</p>
<pre><code class="language-csharp">services.AddSingleton&lt;SoftDeleteInterceptor&gt;();

services.AddDbContext&lt;ApplicationDbContext&gt;(
    (sp, options) =&gt; options
        .UseSqlServer(connectionString)
        .AddInterceptors(
            sp.GetRequiredService&lt;SoftDeleteInterceptor&gt;()));
</code></pre>
<p>If you want to learn more, here's an article with a few practical use cases for <a href="how-to-use-ef-core-interceptors">EF Core interceptors</a>.</p>
<h2>Automatically Filtering Soft-Deleted Data</h2>
<p>To ensure that soft-deleted records are automatically excluded from queries, we can use <a href="how-to-use-global-query-filters-in-ef-core">EF Core global query filters</a>.
We can apply query filters to entities using the <code>OnModelCreating</code> method to automatically exclude records marked as deleted.
This feature dramatically simplifies writing queries.</p>
<p>Here's how to configure the soft delete query filter:</p>
<pre><code class="language-csharp">public sealed class ApplicationDbContext(
    DbContextOptions&lt;UsersDbContext&gt; options) : DbContext(options)
{
    public DbSet&lt;Review&gt; Reviews { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity&lt;Review&gt;().HasQueryFilter(r =&gt; !r.IsDeleted);
    }
}
</code></pre>
<p>A limitation is you can't have more than one query filter configured per entity.</p>
<p>However, it's sometimes useful to explicitly include soft-deleted records.
You can achieve this using the <code>IgnoreQueryFilters</code> method.</p>
<pre><code class="language-csharp">dbContex.Reviews
    .IgnoreQueryFilters()
    .Where(r =&gt; r.ApartmentId == apartmentId)
    .ToList();
</code></pre>
<h2>Faster Queries Using Filtered Index</h2>
<p>To enhance query performance, especially in tables with a significant number of soft-deleted records, you can use <strong>filtered indexes</strong>.
A filtered index only includes records that meet the specified criteria.
This reduces the index size and improves query execution times for operations that exclude filtered records.
Most popular databases support filtered indexes.</p>
<p>Here's how you can configure a <a href="https://learn.microsoft.com/en-us/ef/core/modeling/indexes?tabs=data-annotations#index-filter">filtered index with EF Core</a>:</p>
<pre><code class="language-csharp">public sealed class ApplicationDbContext(
    DbContextOptions&lt;UsersDbContext&gt; options) : DbContext(options)
{
    public DbSet&lt;Review&gt; Reviews { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity&lt;Review&gt;().HasQueryFilter(r =&gt; !r.IsDeleted);

        modelBuilder.Entity&lt;Review&gt;()
            .HasIndex(r =&gt; r.IsDeleted)
            .HasFilter(&quot;IsDeleted = 0&quot;);
    }
}
</code></pre>
<p>The <code>HasFilter</code> method accepts the SQL filter for records that will be included in the index.</p>
<p>You can also create a filtered index using SQL:</p>
<pre><code class="language-sql">CREATE INDEX IX_Reviews_IsDeleted
ON bookings.Reviews (IsDeleted)
WHERE IsDeleted = 0;
</code></pre>
<p>You can learn more about filtered indexes from the documentation:</p>
<ul>
<li><a href="https://learn.microsoft.com/en-us/sql/relational-databases/indexes/create-filtered-indexes?view=sql-server-ver16">SQL Server filtered index</a></li>
<li><a href="https://www.postgresql.org/docs/16/indexes-partial.html">PostgreSQL partial index</a></li>
</ul>
<h2>Do You Really Need Soft Deletes?</h2>
<p>It's worthwhile to think through if you even need to soft delete records.</p>
<p>In enterprise systems, you're typically not thinking about &quot;deleting&quot; data.
There are business concepts that don't involve deleting data.
A few examples are canceling an order, refunding a payment, or voiding an invoice.
These &quot;destructive&quot; operations return the system to a previous state.
But from a business perspective, you aren't really deleting data.</p>
<p>Soft deletes are helpful if there is a risk of accidental deletion.
They allow you to easily restore soft-deleted records.</p>
<p>In any case, consider if soft deletes make sense from a business perspective.</p>
<h2>Takeaway</h2>
<p>Soft deletes offer a valuable safety net for data recovery and can enhance historical data tracking.
However, it's crucial to assess whether they truly align with your application's specific requirements.
Consider factors like the importance of deleted data recovery, any auditing needs, and your industry's regulations.
Creating a filtered index can improve query performance on tables with soft-deleted records.</p>
<p>If you decide that soft deletes are a good fit, EF Core provides the tools necessary for a streamlined implementation.</p>
<p>Thanks for reading, and I'll see you next week!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_081.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[What Is a Modular Monolith?]]></title>
            <link>https://milanjovanovic.tech/blog/what-is-a-modular-monolith</link>
            <guid isPermaLink="false">what-is-a-modular-monolith</guid>
            <pubDate>Sat, 09 Mar 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Modular monoliths blend the simplicity and robustness of traditional monolithic applications with the flexibility and scalability of microservices. Today, I'll introduce you to the modular monolith architecture and why you should know about it.]]></description>
            <content:encoded><![CDATA[<p>I've worked with many different software architectures over the years.</p>
<p>There's one that clearly stands out for its benefits: <strong>Modular Monolith architecture</strong>.</p>
<p>Modular monoliths blend the simplicity and robustness of traditional monolithic applications with the flexibility and scalability of microservices.
I'm tempted to say they bring together the best of both worlds.</p>
<p>The modular monolith architecture allows you to work in a unified codebase with clearly defined boundaries and independent modules.
You can have a high development velocity without the complexity of distributed systems.</p>
<p>Today, I'll introduce you to the modular monolith architecture and why you should know about it.</p>
<h2>What is a Modular Monolith?</h2>
<p>A <strong>modular monolith</strong> is an architectural pattern that structures the application into independent modules or components with well-defined boundaries.
The modules are split based on logical boundaries, grouping together related functionalities.
This approach significantly improves the cohesion of the system.</p>
<p>The modules are loosely coupled, which further promotes modularity and separation of concerns.
Modules communicate through a public API, and you can learn more about this in my article on <a href="modular-monolith-communication-patterns">modular monolith communication patterns</a>.</p>
<p>But what are the benefits of a modular design?</p>
<p><img src="/blogs/mnw_080/modular_monolith.png" alt="Modular monolith."></p>
<p>If we take the example of an apartment booking system illustrated above.
During the holiday season, the system is expecting a traffic spike.
The bookings and payments modules need to scale so they can be deployed independently.
At the end of the holiday season, they can be merged back into a single deployment.
Modular monoliths give you this kind of flexibility.</p>
<h2>Modular Architecture</h2>
<p>Modular monoliths introduce a few important technical challenges that we will need to solve.</p>
<p>To achieve a modular architecture, the modules:</p>
<ul>
<li>Must be independent and interchangeable</li>
<li>Must be able to provide the required functionality</li>
<li>Must have a well-defined interface exposed to other modules</li>
</ul>
<p>Is it possible for a module to be completely independent?
Not really.
That would mean it's not integrated with other modules.
We want loosely coupled modules and to keep the number of dependencies low.
We can use a few techniques to keep the modules independent, and having good <a href="modular-monolith-data-isolation">data isolation</a> is one example.</p>
<p>Another factor you need to consider is how strong the dependency is.
If two modules are very &quot;chatty&quot;, you might have incorrectly defined the boundaries.
You should consider merging these modules together.</p>
<p>Remember, a module is a grouping of related functionalities accessed via a <a href="modular-monolith-communication-patterns">well-defined interface</a>.</p>
<p>Having a modular architecture allows you to easily extract modules into separate services.</p>
<h2>Monolith First</h2>
<p>Microservices have become the most popular architectural pattern in recent years, and for good reason.
Microservices offer many benefits like clearly defined service boundaries, independent deployments, independent scalability, and much more.</p>
<p>However, most teams would be better off starting with a monolith application.</p>
<p>A monolith is an architectural pattern where all components are deployed as a single physical deployment unit.</p>
<p>Here's an interesting quote from Martin Fowler:</p>
<blockquote>
<p>You shouldn't start a new project with microservices, even if you're sure your application will be big enough to make it worthwhile.</p>
</blockquote>
<p><em>— <a href="https://martinfowler.com/bliki/MonolithFirst.html">Martin Fowler</a></em></p>
<p>And I wholeheartedly agree with this. Better yet, consider starting with a modular monolith.</p>
<p>Even Google is jumping on board the modular monolith trend in their recent research paper, <a href="https://dl.acm.org/doi/pdf/10.1145/3593856.3595909">Towards Modern Development of Cloud Applications</a>.</p>
<p>Here are the five main challenges Google identified with microservices:</p>
<ul>
<li><strong>Performance</strong> - The overhead of serializing data and sending it across the network has a noticeable impact on performance.</li>
<li><strong>Correctness</strong> - It's difficult to reason about the correctness of a distributed system when there are many interactions between components.</li>
<li><strong>Management</strong> - We have to manage multiple different applications, each with its release schedule.</li>
<li><strong>Frozen APIs</strong> - Once an API is established, it becomes hard to change without breaking any existing API consumers.</li>
<li><strong>Development speed</strong> - Making a change in one microservice may affect many other microservices, which requires carefully planning deployments.</li>
</ul>
<p>When you factor in the complexity of distributed systems, starting with a modular monolith becomes increasingly compelling.
I also recommend reading about the <a href="https://en.wikipedia.org/wiki/Fallacies_of_distributed_computing">fallacies of distributed computing</a> if you're unfamiliar with them.</p>
<p>Well-defined, in-process components (modules) can be an excellent stepping stone to out-of-process components (services).</p>
<h2>Benefits of a Modular Monolith</h2>
<p>Modular monoliths have many benefits. So, I want to highlight a few that I consider important:</p>
<ul>
<li><strong>Simplified deployment</strong> - Unlike microservices, which require complex deployment strategies, a modular monolith can be deployed as a single unit.</li>
<li><strong>Improved performance</strong> - Communication between modules occurs in-process. This means that there's no network latency or data serialization/deserialization overhead.</li>
<li><strong>Enhanced development velocity</strong> - There's a single codebase to manage, simplifying debugging and the overall development experience.</li>
<li><strong>Easier transaction management</strong> - Managing transactions in a distributed system is very challenging. Modular monoliths simplify this since modules can share the same database.</li>
<li><strong>Lower operational complexity</strong> - Modular monoliths reduce the operational overhead that comes with managing and deploying a distributed microservices system.</li>
<li><strong>Easier transition to Microservices</strong> - A well-structured modular monolith offers a clear path to a microservices architecture.
You can gradually <a href="monolith-to-microservices-how-a-modular-monolith-helps">extract modules into separate services</a> when the need arises.</li>
</ul>
<h2>Modular Monolith vs Microservices</h2>
<p>The biggest difference between modular monoliths and microservices is how they're deployed.
Microservices elevate the logical boundaries inside a modular monolith into physical boundaries.</p>
<p>Microservices give you a clear strategy for modularity and decomposing the bounded contexts.
But, you can also achieve this without building a distributed system.
The problem is people end up using microservices to enforce code boundaries.</p>
<p><img src="/blogs/mnw_080/modular_monolith_vs_microservices.png" alt="Modular monolith vs. microservices."></p>
<p>Instead, you can build a modular monolith to get most of the same benefits.
Modular monoliths give you high cohesion, low coupling, data encapsulation, focus on business functionalities, and more.</p>
<p>Microservices give you all that, plus independent deployments, independent scalability, and the ability to use different technology stacks per service.</p>
<blockquote>
<p>Choose microservices for the benefits, not because your monolithic codebase is a mess.</p>
</blockquote>
<p><em>— <a href="https://twitter.com/simonbrown">Simon Brown</a></em></p>
<h2>Next Steps</h2>
<p>Modular monoliths offer a compelling way to structure applications.
They balance the benefits of well-organized code, scalability potential, and a smooth path for transitioning to microservices if needed.
If you want to improve the maintainability and adaptability of your software, consider exploring modular monoliths.</p>
<p>Want to dive deeper into modular monoliths? Check out these resources:</p>
<ul>
<li><a href="modular-monolith-communication-patterns">Modular Monolith Communication Patterns</a></li>
<li><a href="modular-monolith-data-isolation">Modular Monolith Data Isolation</a></li>
<li><a href="monolith-to-microservices-how-a-modular-monolith-helps">Monolith to Microservices: How a Modular Monolith Helps</a></li>
<li><a href="https://youtu.be/Xo3rsiZYsJQ">Modular Monoliths: How To Build One &amp; Lessons Learned</a></li>
<li><a href="https://youtu.be/z3piPJ7x4WU">How to Structure a Modular Monolith Project in .NET</a></li>
<li><a href="https://youtu.be/5dilYMii9T4">Getting Started with Modular Monoliths in .NET</a></li>
<li><a href="/modular-monolith-architecture">Modular Monolith Architecture course</a></li>
</ul>
<p>That's all for today. Stay awesome, and I'll see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_080.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Lightweight In-Memory Message Bus Using .NET Channels]]></title>
            <link>https://milanjovanovic.tech/blog/lightweight-in-memory-message-bus-using-dotnet-channels</link>
            <guid isPermaLink="false">lightweight-in-memory-message-bus-using-dotnet-channels</guid>
            <pubDate>Sat, 02 Mar 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Suppose you're building a modular monolith, a type of software architecture where different components are organized into loosely coupled modules. Or you might need to process data asynchronously. You'll need a tool or service that allows you to implement this.]]></description>
            <content:encoded><![CDATA[<p>Suppose you're building a modular monolith, a type of software architecture where different components are organized into loosely coupled modules.
Or you might need to process data asynchronously.
You'll need a tool or service that allows you to implement this.</p>
<p>Messaging plays a crucial role in modern software architecture, enabling communication and coordination between loosely coupled components.</p>
<p>An in-memory message bus is particularly useful when high performance and low latency are critical requirements.</p>
<p>In today's issue, we will:</p>
<ul>
<li>Create the required messaging abstractions</li>
<li>Build an in-memory message bus using channels</li>
<li>Implement an integration event processor background job</li>
<li>Demonstrate how to publish and consume messages asynchronously</li>
</ul>
<p>Let's dive in.</p>
<h2>When To Use an In-Memory Message Bus</h2>
<p>I have to preface this by saying that an in-memory message bus is far from a silver bullet.
There are many caveats to using it, as you will soon learn.</p>
<p>But first, let's start with the pros of using an in-memory message bus:</p>
<ul>
<li>Because it works in memory, you have a very low-latency messaging system</li>
<li>You can implement asynchronous (non-blocking) communication between components</li>
</ul>
<p>However, there are a few drawbacks to this approach:</p>
<ul>
<li>Potential for losing messages if the application process goes down</li>
<li>It only works inside of a single process, so it's not useful in distributed systems</li>
</ul>
<p>A practical use case for an in-memory message bus is when building a modular monolith.
You can implement <a href="modular-monolith-communication-patterns">communication between modules</a> using integration events.
When you need to extract some modules into a separate service, you can replace the in-memory bus with a distributed one.</p>
<h2>Defining The Messaging Abstractions</h2>
<p>We will need a few abstractions to build our simple messaging system.
From the client's perspective, we really only need two things.
One abstraction is to publish messages, and another is to define a message handler.</p>
<p>The <code>IEventBus</code> interface exposes the <code>PublishAsync</code> method.
This is what we will use to publish messages.
There's also a generic constraint defined that only allows passing in an <code>IIntegrationEvent</code> instance.</p>
<pre><code class="language-csharp">public interface IEventBus
{
    Task PublishAsync&lt;T&gt;(
        T integrationEvent,
        CancellationToken cancellationToken = default)
        where T : class, IIntegrationEvent;
}
</code></pre>
<p>I want to be practical with the <code>IIntegrationEvent</code> abstraction, so I'll use <a href="https://github.com/jbogard/MediatR">MediatR</a> for the pub-sub support.
The <code>IIntegrationEvent</code> interface will inherit from <code>INotification</code>.
This allows us to easily define <code>IIntegrationEvent</code> handlers using <code>INotificationHandler&lt;T&gt;</code>.
Also, the <code>IIntegrationEvent</code> has an identifier, so we can track its execution.</p>
<p>The abstract <code>IntegrationEvent</code> serves as a base class for concrete implementations.</p>
<pre><code class="language-csharp">using MediatR;

public interface IIntegrationEvent : INotification
{
    Guid Id { get; init; }
}

public abstract record IntegrationEvent(Guid Id) : IIntegrationEvent;
</code></pre>
<h2>Simple In-Memory Queue Using Channels</h2>
<p>The <code>System.Threading.Channels</code> namespace provides data structures for asynchronously passing messages between producers and consumers.
Channels implement the <a href="https://en.wikipedia.org/wiki/Producer%E2%80%93consumer_problem">producer/consumer pattern</a>.
Producers asynchronously produce data, and consumers asynchronously consume that data.
It's an essential pattern for building loosely coupled systems.</p>
<p>One of the primary motivations behind the adoption of <a href="https://learn.microsoft.com/en-us/dotnet/core/extensions/channels">.NET Channels</a>
is their exceptional performance characteristics.
Unlike traditional message queues, Channels operate entirely in memory.
This has the disadvantage of the potential for message loss if the application crashes.</p>
<p>The <code>InMemoryMessageQueue</code> creates an unbounded channel using the <code>Channel.CreateUnbounded</code> bounded.
This means the channel can have any number of readers and writers.
It also exposes a <code>ChannelReader</code> and <code>ChannelWriter</code>, which allow consumers to publish and consume messages.</p>
<pre><code class="language-csharp">internal sealed class InMemoryMessageQueue
{
    private readonly Channel&lt;IIntegrationEvent&gt; _channel =
        Channel.CreateUnbounded&lt;IIntegrationEvent&gt;();

    public ChannelReader&lt;IIntegrationEvent&gt; Reader =&gt; _channel.Reader;

    public ChannelWriter&lt;IIntegrationEvent&gt; Writer =&gt; _channel.Writer;
}
</code></pre>
<p>You also need to register the <code>InMemoryMessageQueue</code> as a singleton with dependency injection:</p>
<pre><code class="language-csharp">builder.Services.AddSingleton&lt;InMemoryMessageQueue&gt;();
</code></pre>
<h2>Implementing The Event Bus</h2>
<p>The <code>IEventBus</code> implementation is now straightforward with the use of channels.
The <code>EventBus</code> class uses the <code>InMemoryMessageQueue</code> to access the <code>ChannelWriter</code> and write an event to the channel.</p>
<pre><code class="language-csharp">internal sealed class EventBus(InMemoryMessageQueue queue) : IEventBus
{
    public async Task PublishAsync&lt;T&gt;(
        T integrationEvent,
        CancellationToken cancellationToken = default)
        where T : class, IIntegrationEvent
    {
        await queue.Writer.WriteAsync(integrationEvent, cancellationToken);
    }
}
</code></pre>
<p>We will register the <code>EventBus</code> as a singleton service with dependency injection because it's stateless:</p>
<pre><code class="language-csharp">builder.Services.AddSingleton&lt;IEventBus, EventBus&gt;();
</code></pre>
<h2>Consuming Integration Events</h2>
<p>With the <code>EventBus</code> implementing the producer, we need a way to consume the published <code>IIntegrationEvent</code>.
We can implement a simple <a href="running-background-tasks-in-asp-net-core">background service</a> using the built-in <code>IHostedService</code> abstraction.</p>
<p>The <code>IntegrationEventProcessorJob</code> depends on the <code>InMemoryMessageQueue</code>, but this time for reading (consuming) messages.
We'll use the <code>ChannelReader.ReadAllAsync</code> method to get back an <code>IAsyncEnumerable</code>.
This allows us to consume all the messages in the <code>Channel</code> asynchronously.</p>
<p>The <code>IPublisher</code> from MediatR helps us connect the <code>IIntegrationEvent</code> with the respective handlers.
It's important to resolve it from a custom scope if you want to inject scoped services into the event handlers.</p>
<pre><code class="language-csharp">internal sealed class IntegrationEventProcessorJob(
    InMemoryMessageQueue queue,
    IServiceScopeFactory serviceScopeFactory,
    ILogger&lt;IntegrationEventProcessorJob&gt; logger)
    : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        await foreach (IIntegrationEvent integrationEvent in
            queue.Reader.ReadAllAsync(stoppingToken))
        {
            try
            {
                using IServiceScope scope = serviceScopeFactory.CreateScope();

                IPublisher publisher = scope.ServiceProvider
                    .GetRequiredService&lt;IPublisher&gt;();

                await publisher.Publish(integrationEvent, stoppingToken);
            }
            catch (Exception ex)
            {
                logger.LogError(
                    ex,
                    &quot;Something went wrong! {IntegrationEventId}&quot;,
                    integrationEvent.Id);
            }
        }
    }
}
</code></pre>
<p>Don't forget to register the hosted service:</p>
<pre><code class="language-csharp">builder.Services.AddHostedService&lt;IntegrationEventProcessorJob&gt;();
</code></pre>
<h2>Using The In-Memory Message Bus</h2>
<p>With all of the necessary abstractions in place, we can finally use the in-memory message bus.</p>
<p>The <code>IEventBus</code> service will write the message to the <code>Channel</code> and immediately return.
This allows you to publish messages in a non-blocking way, which can improve performance.</p>
<pre><code class="language-csharp">internal sealed class RegisterUserCommandHandler(
    IUserRepository userRepository,
    IEventBus eventBus)
    : ICommandHandler&lt;RegisterUserCommand&gt;
{
    public async Task&lt;User&gt; Handle(
        RegisterUserCommand command,
        CancellationToken cancellationToken)
    {
        // First, register the user.
        User user = CreateFromCommand(command);

        userRepository.Insert(user);

        // Now we can publish the event.
        await eventBus.PublishAsync(
            new UserRegisteredIntegrationEvent(user.Id),
            cancellationToken);

        return user;
    }
}
</code></pre>
<p>This solves the producer side, but we also need to create a consumer for the <code>UserRegisteredIntegrationEvent</code> message.
This part is greatly simplified because I'm using MediatR in this implementation.</p>
<p>We need to define an <code>INotificationHandler</code> implementation handling the integration event <code>UserRegisteredIntegrationEvent</code>.
This will be the <code>UserRegisteredIntegrationEventHandler</code>.</p>
<p>When the background job reads the <code>UserRegisteredIntegrationEvent</code> from the <code>Channel</code>, it will publish the message and execute the handler.</p>
<pre><code class="language-csharp">internal sealed class UserRegisteredIntegrationEventHandler
    : INotificationHandler&lt;UserRegisteredIntegrationEvent&gt;
{
    public async Task Handle(
        UserRegisteredIntegrationEvent event,
        CancellationToken cancellationToken)
    {
        // Asynchronously handle the event.
    }
}
</code></pre>
<h2>Improvement Points</h2>
<p>While our basic in-memory message bus is functional, there are several areas we can improve:</p>
<ul>
<li><strong>Resilience</strong> - We can introduce retries when we run into exceptions, which will improve the reliability of the message bus.</li>
<li><strong>Idempotency</strong> - Ask yourself if you want to handle the same message twice.
The <a href="idempotent-consumer-handling-duplicate-messages">idempotent consumer pattern</a> elegantly solves this problem.</li>
<li><strong>Dead Letter Queue</strong> - Sometimes, we won't be able to handle a message correctly.
It's a good idea to introduce a persistent storage for these messages.
This is called a <a href="https://aws.amazon.com/what-is/dead-letter-queue/">Dead Letter Queue</a>, and it allows for troubleshooting at a later time.</li>
</ul>
<p>We've covered the key aspects of building an in-memory message bus using .NET Channels.
You can extend this further by implementing the improvements for a more robust solution.</p>
<p>Remember that this implementation only works inside of one process.
Consider using a real message broker if you need a more reliable solution.</p>
<p>That's all for today. I'll see you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_079.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Automatically Register Minimal APIs in ASP.NET Core]]></title>
            <link>https://milanjovanovic.tech/blog/automatically-register-minimal-apis-in-aspnetcore</link>
            <guid isPermaLink="false">automatically-register-minimal-apis-in-aspnetcore</guid>
            <pubDate>Sat, 24 Feb 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[In ASP.NET Core applications using Minimal APIs, registering each API endpoint with app.MapGet, app.MapPost, etc. can introduce repetitive code. Today, I'll show you how to automatically register your Minimal APIs with a simple abstraction.]]></description>
            <content:encoded><![CDATA[<p>In <a href="http://ASP.NET">ASP.NET</a> Core applications using <a href="https://learn.microsoft.com/en-us/aspnet/core/fundamentals/minimal-apis/overview?view=aspnetcore-8.0">Minimal APIs</a>,
registering each API endpoint with <code>app.MapGet</code>, <code>app.MapPost</code>, etc., can introduce repetitive code.
As projects grow, this manual process becomes increasingly time-consuming and prone to maintenance headaches.</p>
<p>You can try grouping the Minimal API endpoints using extension methods so as not to clutter the <code>Program</code> file.
This approach scales well as the project grows.
However, it feels like reinventing controllers.</p>
<p>I like to view each Minimal API endpoint as a standalone component.</p>
<p>The vision I have in my mind aligns nicely with the concept of <a href="vertical-slice-architecture">vertical slices.</a></p>
<p>Today, I'll show you how to register your Minimal APIs automatically with a simple abstraction.</p>
<h2>The Endpoint Comes First</h2>
<p>Automatically registering Minimal APIs significantly reduces boilerplate, streamlining development.
It makes your codebase more concise and improves maintainability by establishing a centralized registration mechanism.</p>
<p>Let's create a simple <code>IEndpoint</code> abstraction to represent a single endpoint.</p>
<p>The <code>MapEndpoint</code> accepts an <code>IEndpointRouteBuilder</code>, which we can use to call <code>MapGet</code>, <code>MapPost</code>, etc.</p>
<pre><code class="language-csharp">public interface IEndpoint
{
    void MapEndpoint(IEndpointRouteBuilder app);
}
</code></pre>
<p>Each <code>IEndpoint</code> implementation should contain exactly one Minimal API endpoint definition.</p>
<p>Nothing prevents you from registering multiple endpoints in the <code>MapEndpoint</code> method.
But you (really) shouldn't.</p>
<p>Additionally, you could implement a code analyzer or <a href="enforcing-software-architecture-with-architecture-tests">architecture test</a> to enforce this rule.</p>
<pre><code class="language-csharp">public class GetFollowerStats : IEndpoint
{
    public void MapEndpoint(IEndpointRouteBuilder app)
    {
        app.MapGet(&quot;users/{userId}/followers/stats&quot;, async (
            Guid userId,
            ISender sender) =&gt;
        {
            var query = new GetFollowerStatsQuery(userId);

            Result&lt;FollowerStatsResponse&gt; result = await sender.Send(query);

            return result.Match(Results.Ok, CustomResults.Problem);
        })
        .WithTags(Tags.Users);
    }
}
</code></pre>
<h2>Sprinkle Some Reflection Magic</h2>
<p>Reflection allows us to dynamically examine code at runtime.
For Minimal API registration, we'll use reflection to scan our .NET assemblies and find classes that implement <code>IEndpoint</code>.
Then, we will configure them as services with dependency injection.</p>
<p>The <code>Assembly</code> parameter should be the assembly that contains the <code>IEndpoint</code> implementations.
If you want to have endpoints in multiple assemblies (projects), you can easily extend this method to accept a collection.</p>
<pre><code class="language-csharp">public static IServiceCollection AddEndpoints(
    this IServiceCollection services,
    Assembly assembly)
{
    ServiceDescriptor[] serviceDescriptors = assembly
        .DefinedTypes
        .Where(type =&gt; type is { IsAbstract: false, IsInterface: false } &amp;&amp;
                       type.IsAssignableTo(typeof(IEndpoint)))
        .Select(type =&gt; ServiceDescriptor.Transient(typeof(IEndpoint), type))
        .ToArray();

    services.TryAddEnumerable(serviceDescriptors);

    return services;
}
</code></pre>
<p>We only need to call this method once from the <code>Program</code> file:</p>
<pre><code class="language-csharp">builder.Services.AddEndpoints(typeof(Program).Assembly);
</code></pre>
<h2>Registering Minimal APIs</h2>
<p>The final step in our implementation is to register the endpoints automatically.
We can create an extension method on the <code>WebApplication</code>, which lets us resolve services using the <code>IServiceProvider</code>.</p>
<p>We're looking for all registrations of the <code>IEndpoint</code> service.
These will be the endpoint classes we can now register with the application by calling <code>MapEndpoint</code>.</p>
<p>I'm also adding an option to pass in a <code>RouteGroupBuilder</code> if you want to apply conventions to all endpoints.
A great example is adding a route prefix, authentication, or <a href="api-versioning-in-aspnetcore">API versioning.</a></p>
<pre><code class="language-csharp">public static IApplicationBuilder MapEndpoints(
    this WebApplication app,
    RouteGroupBuilder? routeGroupBuilder = null)
{
    IEnumerable&lt;IEndpoint&gt; endpoints = app.Services
        .GetRequiredService&lt;IEnumerable&lt;IEndpoint&gt;&gt;();

    IEndpointRouteBuilder builder =
        routeGroupBuilder is null ? app : routeGroupBuilder;

    foreach (IEndpoint endpoint in endpoints)
    {
        endpoint.MapEndpoint(builder);
    }

    return app;
}
</code></pre>
<h2>Putting It All Together</h2>
<p>Here's what the <code>Program</code> file could look like when we put it all together.</p>
<p>We're calling <code>AddEndpoints</code> to register the <code>IEndpoint</code> implementations.</p>
<p>Then, we're calling <code>MapEndpoints</code> to automatically register the Minimal APIs.</p>
<p>I'm also configuring a route prefix and API Versioning for each endpoint using a <code>RouteGroupBuilder</code>.</p>
<pre><code class="language-csharp">WebApplicationBuilder builder = WebApplication.CreateBuilder(args);

builder.Services.AddEndpointsApiExplorer();
builder.Services.AddSwaggerGen();

builder.Services.AddEndpoints(typeof(Program).Assembly);

WebApplication app = builder.Build();

ApiVersionSet apiVersionSet = app.NewApiVersionSet()
    .HasApiVersion(new ApiVersion(1))
    .ReportApiVersions()
    .Build();

RouteGroupBuilder versionedGroup = app
    .MapGroup(&quot;api/v{version:apiVersion}&quot;)
    .WithApiVersionSet(apiVersionSet);

app.MapEndpoints(versionedGroup);

app.Run();
</code></pre>
<h2>Takeaway</h2>
<p>Automatic Minimal API registration with techniques like reflection can significantly improve developer efficiency and project maintainability.</p>
<p>While highly beneficial, it's important to acknowledge the potential <strong>performance impact of reflection</strong> on application startup.</p>
<p>So, an improvement point could be using source generators for pre-compiled registration logic.</p>
<p>A few alternatives worth exploring:</p>
<ul>
<li><a href="how-to-structure-minimal-apis">Extension methods</a></li>
<li><a href="https://fast-endpoints.com/">FastEndpoints</a></li>
<li><a href="https://github.com/CarterCommunity/Carter">Carter</a></li>
</ul>
<p>Hope this was helpful.</p>
<p>See you next week.</p>
<p><strong>P.S.</strong> Here's the complete <a href="https://github.com/m-jovanovic/minimal-endpoints">source code</a> for this article.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_078.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Using Scoped Services From Singletons in ASP.NET Core]]></title>
            <link>https://milanjovanovic.tech/blog/using-scoped-services-from-singletons-in-aspnetcore</link>
            <guid isPermaLink="false">using-scoped-services-from-singletons-in-aspnetcore</guid>
            <pubDate>Sat, 17 Feb 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Did you ever need to inject a scoped service into a singleton service? I'll explain how you can solve this problem and safely use scoped services from within singletons in ASP.NET Core.]]></description>
            <content:encoded><![CDATA[<p>Did you ever need to inject a scoped service into a singleton service?</p>
<p>I often need to resolve a scoped service, like the EF Core <code>DbContext</code>, in a background service.</p>
<p>Another example is when you need to resolve a scoped service in <a href="http://ASP.NET">ASP.NET</a> Core middleware.</p>
<p>If you ever tried this, you were probably greeted with an exception similar to this one:</p>
<pre><code>System.InvalidOperationException: Cannot consume scoped service 'Scoped' from singleton 'Singleton'.
</code></pre>
<p>Today, I'll explain how you can solve this problem and safely use scoped services from within singletons in <a href="http://ASP.NET">ASP.NET</a> Core.</p>
<h2><a href="http://ASP.NET">ASP.NET</a> Core Service Lifetimes</h2>
<p><a href="http://ASP.NET">ASP.NET</a> Core has three <a href="https://learn.microsoft.com/en-us/dotnet/core/extensions/dependency-injection#service-lifetimes">service lifetimes</a>:</p>
<ul>
<li>Transient</li>
<li>Singleton</li>
<li>Scoped</li>
</ul>
<p><a href="https://learn.microsoft.com/en-us/dotnet/core/extensions/dependency-injection#transient">Transient services</a> are created each time they're requested from the service container.</p>
<p><a href="https://learn.microsoft.com/en-us/dotnet/core/extensions/dependency-injection#scoped">Scoped services</a> are created once within the scope's lifetime.
For <a href="http://ASP.NET">ASP.NET</a> Core applications, a new scope is created for each request.
This is how you can resolve scoped services within a given request.</p>
<p><a href="http://ASP.NET">ASP.NET</a> Core applications also have a root <code>IServiceProvider</code> used to resolve <a href="https://learn.microsoft.com/en-us/dotnet/core/extensions/dependency-injection#singleton">singleton services.</a></p>
<p>So, what can we do if resolving a scoped service from a singleton throws an exception?</p>
<h2>The Solution - <code>IServiceScopeFactory</code></h2>
<p>What if you want to resolve a scoped service inside a <a href="running-background-tasks-in-asp-net-core">background service</a>?</p>
<p>You can create a new scope (<code>IServiceScope</code>) with its own <code>IServiceProvider</code> instance.
The scoped <code>IServiceProvider</code> can be used to resolve scoped services.
When the scope is disposed, all disposable services created within that scope are also disposed.</p>
<p>Here's an example of using the <code>IServiceScopeFactory</code> to create a new <code>IServiceScope</code>.
We're using the scope to resolve the <code>ApplicationDbContext</code>, which is a scoped service.</p>
<p>The <code>BackgroundJob</code> is registered as a singleton when calling <code>AddHostedService&lt;BackgroundJob&gt;</code>.</p>
<pre><code class="language-csharp">public class BackgroundJob(IServiceScopeFactory serviceScopeFactory)
    : BackgroundService
{
    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        using IServiceScope scope = serviceScopeFactory.CreateScope();

        var dbContext = scope
            .ServiceProvider
            .GetRequiredService&lt;ApplicationDbContext&gt;();

        // Do some background processing with the EF database context.
        await DoWorkAsync(dbContext);
    }
}
</code></pre>
<h2>Scoped Services in Middleware</h2>
<p>What if you want to use a scoped service in <a href="3-ways-to-create-middleware-in-asp-net-core">ASP.NET Core middleware</a>?</p>
<p>Middleware is constructed once per application lifetime.</p>
<p>If you try injecting a scoped service, you'll get an exception:</p>
<pre><code>System.InvalidOperationException: Cannot resolve scoped service 'Scoped' from root provider.
</code></pre>
<p>There are two ways to get around this.</p>
<p>First, you could use the previous approach with creating a new scope using <code>IServiceScopeFactory</code>.
You'll be able to resolve scoped services.
But, they won't share the same lifetime as the other scoped service in the same request.
This could even be a problem depending on your requirements.</p>
<p>Is there a better way?</p>
<p>Middleware allows you to inject scoped services in the <code>InvokeAsync</code> method.
The injected services will use the current request's scope, so they'll have the same lifetime as any other scoped service.</p>
<pre><code class="language-csharp">public class ConventionalMiddleware(RequestDelegate next)
{
    public async Task InvokeAsync(
        HttpContext httpContext,
        IMyScopedService scoped)
    {
        scoped.DoSomething();

        await _next(httpContext);
    }
}
</code></pre>
<h2><code>IServiceScopeFactory</code> vs. <code>IServiceProvider</code></h2>
<p>You might see examples using the <code>IServiceProvider</code> to create a scope instead of the <code>IServiceScopeFactory</code>.</p>
<p>What's the difference between these two approaches?</p>
<p>The <a href="https://github.com/aspnet/DependencyInjection/blob/94b9cc9ace032f838e068702cc70ce57cc883bc7/src/DI.Abstractions/ServiceProviderServiceExtensions.cs#L125"><code>CreateScope</code> method from <code>IServiceProvider</code></a>
resolves an <code>IServiceScopeFactory</code> instance and calls <code>CreateScope()</code> on it:</p>
<pre><code class="language-csharp">public static IServiceScope CreateScope(this IServiceProvider provider)
{
    return provider.GetRequiredService&lt;IServiceScopeFactory&gt;().CreateScope();
}
</code></pre>
<p>So, if you want to use the <code>IServiceProvider</code> directly to create a scope, that's fine.</p>
<p>However, the <code>IServiceScopeFactory</code> is a more direct way to achieve the desired result.</p>
<h2>Summary</h2>
<p>Understanding the difference between Transient, Scoped, and Singleton lifetimes is crucial for managing dependencies in <a href="http://ASP.NET">ASP.NET</a> Core applications.</p>
<p>The <code>IServiceScopeFactory</code> provides a solution when you need to resolve scoped services from singletons.
It allows you to create a new scope, which you can use to resolve scoped services.</p>
<p>In middleware, we can inject scoped services into the <code>InvokeAsync</code> method.
This also ensures the services use the current request's scope and lifecycle.</p>
<p>Thanks for reading, and I'll see you next week!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_077.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Getting the Current User in Clean Architecture]]></title>
            <link>https://milanjovanovic.tech/blog/getting-the-current-user-in-clean-architecture</link>
            <guid isPermaLink="false">getting-the-current-user-in-clean-architecture</guid>
            <pubDate>Sat, 10 Feb 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[The applications you build serve your users (customers), to help them solve some problems. It's a common requirement that you will need to know who the current application user is.]]></description>
            <content:encoded><![CDATA[<p>The applications you build serve your users (customers) to help them solve some problems.
It's a common requirement that you will need to know who the current application user is.</p>
<p>How do you get the current user's information in a Clean Architecture use case?</p>
<p>Use cases live in the Application layer, where you can't introduce external concerns.
Otherwise, you will be breaking the dependency rule.</p>
<p>Let's say you want to know who the current user is to determine if they can access some resource.
This is your typical resource-based authorization check.
But you have to interact with the identity provider to get this information.
This breaks the <a href="clean-architecture-and-the-benefits-of-structured-software-design">dependency rule in Clean Architecture.</a></p>
<p>I've seen this problem confuse developers who are new to Clean Architecture.</p>
<p>In today's issue, I'll show you how to access the current user's information in a clean way.</p>
<h2>Start With an Abstraction</h2>
<p>The inner layers in Clean Architecture define abstractions for external concerns.
From the Application layer's perspective, authentication and user identity are external concerns.</p>
<p>The Infrastructure layer deals with external concerns, including authentication and identity management.
This is where you would implement the abstraction.</p>
<p>My preferred approach is creating an <code>IUserContext</code> abstraction.
The main information I need is the <code>UserId</code> of the current user.
But you can expand the <code>IUserContext</code> with any other data you think is necessary.</p>
<pre><code class="language-csharp">public interface IUserContext
{
    bool IsAuthenticated { get; }

    Guid UserId { get; }
}
</code></pre>
<p>Let's see how to implement the <code>IUserContext</code>.</p>
<h2>Implementing the UserContext</h2>
<p>The <code>UserContext</code> class is the <code>IUserContext</code> implementation in the Infrastructure layer.
We need to inject the <code>IHttpContextAccessor</code>, which allows us to access the <a href="https://learn.microsoft.com/en-us/dotnet/api/system.security.claims.claimsprincipal?view=net-8.0"><code>ClaimsPrincipal</code></a>
through the <code>User</code> property.
The <code>ClaimsPrincipal</code> gives you access to the current user's claims, containing the required information.</p>
<p>In this example, I'm throwing an exception if any of the properties evaluate to <code>null</code>.
You can decide if throwing an exception makes sense for you.</p>
<p>I also want to share an important remark here about <code>IHttpContextAccessor</code>.
We're using it to access the <code>HttpContext</code> instance — <strong>which only exists during an API request</strong>.
Outside an API request, the <code>HttpContext</code> will be null, and the <code>UserContext</code> will throw an exception when accessing its properties.</p>
<pre><code class="language-csharp">internal sealed class UserContext(IHttpContextAccessor httpContextAccessor)
    : IUserContext
{
    public Guid UserId =&gt;
        httpContextAccessor
            .HttpContext?
            .User
            .GetUserId() ??
        throw new ApplicationException(&quot;User context is unavailable&quot;);

    public bool IsAuthenticated =&gt;
        httpContextAccessor
            .HttpContext?
            .User
            .Identity?
            .IsAuthenticated ??
        throw new ApplicationException(&quot;User context is unavailable&quot;);
}
</code></pre>
<p>Here's the <code>GetUserId</code> extension method that's used in the <code>UserContext.UserId</code> property.
It's looking for a claim with the <code>ClaimTypes.NameIdentifier</code> name, and parsing that value into a <code>Guid</code>.
You can replace this with a different type to match the user identity in your system.</p>
<pre><code class="language-csharp">internal static class ClaimsPrincipalExtensions
{
    public static Guid GetUserId(this ClaimsPrincipal? principal)
    {
        string? userId = principal?.FindFirstValue(ClaimTypes.NameIdentifier);

        return Guid.TryParse(userId, out Guid parsedUserId) ?
            parsedUserId :
            throw new ApplicationException(&quot;User id is unavailable&quot;);
    }
}
</code></pre>
<h2>Using The Current User Information</h2>
<p>Now that you have the <code>IUserContext</code>, you can use it from the Application layer.</p>
<p>A common requirement is checking if the current user can access some resources.</p>
<p>Here's an example using the <code>GetInvoiceQueryHandler</code>, which queries the database for an invoice.
After projecting the result to an <code>InvoiceResponse</code> object, we check if the current user is the one to whom the invoice was issued.
You can also apply this check as part of the database query.
But performing it in memory lets you return a different response to the user when they aren't authorized.
For example, a <a href="https://www.rfc-editor.org/rfc/rfc7231#section-6.5.3">403 Forbidden</a> might be appropriate.</p>
<pre><code class="language-csharp">class GetInvoiceQueryHandler(IAppDbContext dbContext, IUserContext userContext)
    : IQueryHandler&lt;GetInvoiceQuery, InvoiceResponse&gt;
{
    public async Task&lt;Result&lt;InvoiceResponse&gt;&gt; Handle(
        GetInvoiceQuery request,
        CancellationToken cancellationToken)
    {
        InvoiceResponse? invoiceResponse = await dbContext
            .Invoices
            .ProjectTo&lt;InvoiceResponse&gt;()
            .FirstOrDefaultAsync(
                invoice =&gt; invoice.Id == request.InvoiceId,
                cancellationToken);

        if (invoiceResponse is null ||
            invoiceResponse.IssuedToUserId != userContext.UserId)
        {
            return Result.Failure&lt;InvoiceResponse&gt;(InvoiceErrors.NotFound);
        }

        return invoiceResponse;
    }
}
</code></pre>
<h2>Takeaway</h2>
<p>Incorporating user identity and authentication into <a href="why-clean-architecture-is-great-for-complex-projects">Clean Architecture</a>
doesn't have to compromise the integrity of your design.
The Application layer should remain decoupled from external concerns such as identity management.</p>
<p>We respect the <a href="clean-architecture-and-the-benefits-of-structured-software-design">Clean Architecture dependency rule</a> by
abstracting user-related information through the <code>IUserContext</code> interface and implementing it within the Infrastructure layer.</p>
<p>With this strategy, you can effectively manage user information, support authorization checks, and ensure your application remains robust and adaptable to future changes.</p>
<p>Remember, the key is in defining clear abstractions and respecting the architecture's boundaries.</p>
<p>Hope this was helpful.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_076.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How I Made My EF Core Query 3.42x Faster With Batching]]></title>
            <link>https://milanjovanovic.tech/blog/how-i-made-my-efcore-query-faster-with-batching</link>
            <guid isPermaLink="false">how-i-made-my-efcore-query-faster-with-batching</guid>
            <pubDate>Sat, 03 Feb 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[EF Core is a fantastic ORM if you're building .NET applications. Today, I'll show you a simple idea I used to get an almost 4x performance improvement.]]></description>
            <content:encoded><![CDATA[<p><a href="https://learn.microsoft.com/en-us/ef/core/">EF Core</a> is a fantastic ORM if you're building .NET applications.</p>
<p>But it's a tool like any other. And you can end up using it in a suboptimal way.</p>
<p>Today, I'll show you a simple idea I used to get an almost <strong>4x performance improvement</strong>.</p>
<p>I'm not saying you'll see the same result, but understanding the idea will make your queries faster.</p>
<h2>Why This Query is Suboptimal</h2>
<p>Here's the example I want to use to explain this powerful idea.
It's taken from a production app I was working on, but I simplified it for this example.</p>
<p>We're using an <code>InvoiceService</code> to get a collection of invoices for a given company.
The invoices could come from a third-party API or some other persistence store.
We're lacking detailed line item information, so we're querying the database to fill in the missing data.</p>
<p>The highlighted <a href="https://learn.microsoft.com/en-us/ef/core/querying/">LINQ query</a> below isn't bad by itself.
It returns all the line items in one database query (round trip).</p>
<p>But it's missing one important realization that can unlock further performance gains.</p>
<p>Because we're iterating over the invoices, we're <strong>querying the database many times</strong>.</p>
<pre><code class="language-csharp">app.MapGet(&quot;invoices/{companyId}&quot;, (
    long companyId,
    InvoiceService invoiceService,
    AppDbContext dbContext) =&gt;
{
    IEnumerable&lt;Invoice&gt; invoices = invoiceService.GetForCompanyId(
        companyId,
        take: 10);

    var invoiceDtos = new List&lt;InvoiceDto&gt;();
    foreach (var invoice in invoices)
    {
        var invoiceDto = new InvoiceDto
        {
            Id = invoice.Id,
            CompanyId = invoice.CompanyId,
            IssuedDate = invoice.IssuedDate,
            DueDate = invoice.DueDate,
            Number = invoice.Number
        };

        var lineItemDtos = await dbContext
            .LineItems
            .Where(li =&gt; invoice.LineItemIds.Contains(li.Id))
            .Select(li =&gt; new LineItemDto
            {
                Id = li.Id,
                Name = li.Name,
                Price = li.Price,
                Quantity = li.Quantity
            })
            .ToArrayAsync();

        invoiceDto.LineItems = lineItemDtos;

        invoiceDtos.Add(invoiceDto);
    }

    return invoiceDtos;
});
</code></pre>
<p>Once you figure this out, the solution comes down to applying a simple idea.</p>
<p>Instead of fetching the line items for each invoice, we can query all the line items ahead of time.</p>
<h2>Batching to the Rescue</h2>
<p>Here's the same query, but refactored to only query the line items once.
This means there's just a single round trip to the database.</p>
<p>There are three components to the final design:</p>
<ul>
<li>Querying all the <code>LineItems</code> in a single database round-trip</li>
<li>Creating a <code>LineItemDto</code> dictionary for fast lookup</li>
</ul>
<p>Once we have the dictionary, we can loop through the invoices and assign the line items.
Populating a line item becomes a dictionary lookup (cheap) instead of a database query (expensive).</p>
<p>Before deciding if this solution makes sense, you should consider a few more things.</p>
<p>How many records can you load from the database at once?</p>
<p>Each invoice contains ~20 line items on average, and we're only fetching ten invoices.
So, we're loading ~200 line items from the database.
Most applications can handle this load.
But things could be different if you're fetching thousands of rows.</p>
<pre><code class="language-csharp">app.MapGet(&quot;invoices/{companyId}&quot;, (
    long companyId,
    InvoiceService invoiceService,
    AppDbContext dbContext) =&gt;
{
    IEnumerable&lt;Invoice&gt; invoices = invoiceService.GetForCompanyId(
        companyId,
        take: 10);

    long[] lineItemIds = invoices
        .SelectMany(invoice =&gt; invoice.LineItemIds)
        .ToArray();

    var lineItemDtos = await dbContext
        .LineItems
        .Where(li =&gt; lineItemIds.Contains(li.Id))
        .Select(li =&gt; new LineItemDto
        {
            Id = li.Id,
            Name = li.Name,
            Price = li.Price,
            Quantity = li.Quantity
        })
        .ToListAsync();

    Dictionary&lt;long, LineItemDto&gt; lineItemsDictionary =
        lineItemDtos.ToDictionary(keySelector: li =&gt; li.Id);

    var invoiceDtos = new List&lt;InvoiceDto&gt;();
    foreach (var invoice in invoices)
    {
        var invoiceDto = new InvoiceDto
        {
            Id = invoice.Id,
            CompanyId = invoice.CompanyId,
            IssuedDate = invoice.IssuedDate,
            DueDate = invoice.DueDate,
            Number = invoice.Number,
            LineItems = invoice
                .LineItemIds
                .Select(li =&gt; lineItemsDictionary[li])
                .ToArray()
        };

        invoiceDtos.Add(invoiceDto);
    }

    return invoiceDtos;
})
</code></pre>
<h2>How Much Faster?</h2>
<p>It seems plausible that the batch variant would be faster. Right?</p>
<p>We have N queries (one per invoice) in the first version and a single query in the batched version.</p>
<p>Here are the benchmark results I got using <a href="https://github.com/dotnet/BenchmarkDotNet">BenchmarkDotNet</a>:</p>
<p><img src="/blogs/mnw_075/benchmark.png" alt=""></p>
<p>The foreach version takes <strong>1913.3 us</strong> (microseconds) on average.<br>
The batched version takes <strong>558.6 us</strong> on average.</p>
<p>That's <strong>3.42x faster</strong> with the batched version. This is with a local SQL database.</p>
<p>The batched version should be even faster if you're querying a remote database because of the impact of network round-trip time.
It quickly adds up when you have N queries (foreach version).</p>
<h2>Takeaway</h2>
<p>The power of this approach lies in its simplicity and efficiency.
By batching database queries, we significantly reduce the number of round trips to the database.
This is often one of the biggest performance bottlenecks.</p>
<p>But it's crucial to understand that this approach is not a one-size-fits-all solution.</p>
<p>EF Core offers many features and optimizations, but it's up to the developer to use them effectively.</p>
<p>Finally, always remember to measure and benchmark.
The improvements we saw in this case were quantified through benchmarks.
Without proper measurement, it's easy to make changes that inadvertently degrade performance.</p>
<p>Thanks for reading, and stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_075.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How to Build a URL Shortener With .NET]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-build-a-url-shortener-with-dotnet</link>
            <guid isPermaLink="false">how-to-build-a-url-shortener-with-dotnet</guid>
            <pubDate>Sat, 27 Jan 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[A URL shortener is a simple yet powerful tool that converts long URLs into more manageable, shorter versions. Today, I'll guide you through the design, implementation, and considerations for creating your URL shortener in .NET]]></description>
            <content:encoded><![CDATA[<p>A <strong>URL shortener</strong> is a simple yet powerful tool that converts long URLs into more manageable, shorter versions.
This is particularly useful for sharing links on platforms with character limits or improving user experience by reducing clutter.
Two popular URL shorteners are <a href="https://bitly.com/">Bitly</a> and <a href="https://tinyurl.com/app">TinyURL</a>.
Designing a URL shortener is an interesting challenge with fun problems to solve.</p>
<p>But how would you build a URL shortener in .NET?</p>
<p>URL shorteners have two core functionalities:</p>
<ul>
<li>Generating a unique code for a given URL</li>
<li>Redirecting users who access the short link to the original URL</li>
</ul>
<p>Today, I'll guide you through the design, implementation, and considerations for creating your URL shortener.</p>
<h2>URL Shortener System Design</h2>
<p>Here's the high-level system design for our URL shortener.
We want to expose two endpoints.
One to shorten a long URL and the other to redirect users based on a shortened URL.
The shortened URLs are stored in a <a href="https://www.postgresql.org/">PostgreSQL</a> database in this example.
We can introduce a distributed cache like <a href="https://redis.io/">Redis</a> to the system to improve read performance.</p>
<p><img src="/blogs/mnw_074/url_shortener.png" alt="URL shortener system design. It contains two API endpoints, a PostgreSQL database, and a Redis cache."></p>
<p>We first need to ensure a large number of short URLs.
We're going to assign a unique code to each long URL, and use it to generate the shortened URL.
The unique code length and set of characters determine how many short URLs the system can generate.
We will discuss this in more detail when we implement unique code generation.</p>
<p>We're going to use the random code generation strategy.
It's straightforward to implement and has an acceptably low rate of collisions.
The trade-off we're making is increased latency, but we will also explore other options.</p>
<h2>The Data Model</h2>
<p>Let's start by figuring out what we will store in the database.
Our data model is straightforward.
We have a <code>ShortenedUrl</code> class representing the URLs stored in our system:</p>
<pre><code class="language-csharp">public class ShortenedUrl
{
    public Guid Id { get; set; }

    public string LongUrl { get; set; } = string.Empty;

    public string ShortUrl { get; set; } = string.Empty;

    public string Code { get; set; } = string.Empty;

    public DateTime CreatedOnUtc { get; set; }
}
</code></pre>
<p>This class includes properties for the original URL (<code>LongUrl</code>), the shortened URL (<code>ShortUrl</code>), and a unique code (<code>Code</code>) that represents the shortened URL.
The <code>Id</code> and <code>CreatedOnUtc</code> fields are used for database and tracking purposes.
The users will send the unique <code>Code</code> to our system, which will try to find a matching <code>LongUrl</code> and redirect them.</p>
<p>In addition, we will also define an EF <code>ApplicationDbContext</code> class, which is responsible for configuring our entity and setting up our database context.
I'm doing two things here to improve performance:</p>
<ul>
<li>Configuring the <code>Code</code> maximum length with <code>HasMaxLength</code></li>
<li>Defining a unique index on the <code>Code</code> column</li>
</ul>
<p>A unique index shields us from concurrency conflicts, so we will never have duplicate <code>Code</code> values persisted in the database.
Setting the maximum length for this column saves storage space, and it's a requirement for indexing string columns in some databases.</p>
<p>Note that some databases treat strings in a case-insensitive way.
This severely reduces the number of available short URLs.
You want to configure the database to treat the unique code in a case-sensitive way.</p>
<pre><code class="language-csharp">public class ApplicationDbContext : DbContext
{
    public ApplicationDbContext(DbContextOptions options)
        : base(options)
    {
    }

    public DbSet&lt;ShortenedUrl&gt; ShortenedUrls { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.Entity&lt;ShortenedUrl&gt;(builder =&gt;
        {
            builder
                .Property(shortenedUrl =&gt; shortenedUrl.Code)
                .HasMaxLength(ShortLinkSettings.Length);

            builder
                .HasIndex(shortenedUrl =&gt; shortenedUrl.Code)
                .IsUnique();
        });
    }
}
</code></pre>
<h2>Unique Code Generation</h2>
<p>The most crucial part of our URL shortener is generating a unique code for each URL.
There are a few different algorithms you can choose to implement this.
We want an even distribution of unique codes across all possible values.
This helps to reduce potential collisions.</p>
<p>I will implement a random, unique code generator with a predefined alphabet.
It's simple to implement, and the chance of collision is relatively low.
Still, there are more performant solutions than this, but more on this later.</p>
<p>Let's define a <code>ShortLinkSettings</code> class that contains two constants.
One is for defining the length of the unqualified code we will generate.
The other constant is the alphabet we will use to generate the random code.</p>
<pre><code class="language-csharp">public static class ShortLinkSettings
{
    public const int Length = 7;
    public const string Alphabet =
        &quot;ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789&quot;;
}
</code></pre>
<p>The alphabet has <code>62</code> characters, which gives us <code>62^7</code> possible unique code combinations.</p>
<p>If you're wondering, this is <code>3,521,614,606,208</code> combinations.</p>
<p>Spelled out: three trillion, five hundred twenty-one billion, six hundred fourteen million, six hundred six thousand, two hundred eight.</p>
<p>Those are quite a few unique codes, which will be enough for our URL shortener.</p>
<p>Now, let's implement our <code>UrlShorteningService</code>, which handles generating unique codes.
This service generates a random string of the specified length using our predefined alphabet.
It checks against the database to ensure uniqueness.</p>
<pre><code class="language-csharp">public class UrlShorteningService(ApplicationDbContext dbContext)
{
    private readonly Random _random = new();

    public async Task&lt;string&gt; GenerateUniqueCode()
    {
        var codeChars = new char[ShortLinkSettings.Length];
        const int maxValue = ShortLinkSettings.Alphabet.Length;

        while (true)
        {
            for (var i = 0; i &lt; ShortLinkSettings.Length; i++)
            {
                var randomIndex = _random.Next(maxValue);

                codeChars[i] = ShortLinkSettings.Alphabet[randomIndex];
            }

            var code = new string(codeChars);

            if (!await dbContext.ShortenedUrls.AnyAsync(s =&gt; s.Code == code))
            {
                return code;
            }
        }
    }
}
</code></pre>
<p><strong>Downsides and Improvement Points</strong></p>
<p>The downside of this implementation is increased latency because we're checking each code we generate against the database.
An improvement point could be generating the unique codes in the database ahead of time.</p>
<p>Another improvement point could be using a fixed number of iterations instead of an infinite loop.
In case of multiple collisions in a row, the current implementation would continue until a unique value is found.
Consider throwing an exception instead after a few collisions in a row.</p>
<h2>URL Shortening</h2>
<p>Now that our core business logic is ready, we can expose an endpoint to shorten URLs.
We can use a simple Minimal API endpoint.</p>
<p>This endpoint accepts a URL, validates it, and then uses the <code>UrlShorteningService</code> to create a shortened URL, which is then saved to the database.
We return the full shortened URL to the client.</p>
<pre><code class="language-csharp">public record ShortenUrlRequest(string Url);

app.MapPost(&quot;shorten&quot;, async (
    ShortenUrlRequest request,
    UrlShorteningService urlShorteningService,
    ApplicationDbContext dbContext,
    HttpContext httpContext) =&gt;
{
    if (!Uri.TryCreate(request.Url, UriKind.Absolute, out _))
    {
        return Results.BadRequest(&quot;The specified URL is invalid.&quot;);
    }

    var code = await urlShorteningService.GenerateUniqueCode();

    var request = httpContext.Request;

    var shortenedUrl = new ShortenedUrl
    {
        Id = Guid.NewGuid(),
        LongUrl = request.Url,
        Code = code,
        ShortUrl = $&quot;{request.Scheme}://{request.Host}/{code}&quot;,
        CreatedOnUtc = DateTime.UtcNow
    };

    dbContext.ShortenedUrls.Add(shortenedUrl);

    await dbContext.SaveChangesAsync();

    return Results.Ok(shortenedUrl.ShortUrl);
});
</code></pre>
<p>There is a minor <a href="solving-race-conditions-with-ef-core-optimistic-locking">race condition</a> here, as we generate a unique code first and then insert it into the database.
A concurrent request could generate the same unique code and insert it into the database before we complete our transaction.
However, the chances of this happening are low, so I decided not to handle that case.</p>
<p>Remember that the unique index in the database is still guarding us against duplicate values.</p>
<h2>URL Redirection</h2>
<p>The second use case for a URL shortener is redirection when accessing a shortened URL.</p>
<p>We will expose another Minimal API endpoint for this feature.
The endpoint will accept a unique code, find the respective shortened URL, and redirect the user to the original long URL.
You can implement additional validation for the specified code before checking if there's a shortened URL in the database.</p>
<pre><code class="language-csharp">app.MapGet(&quot;{code}&quot;, async (string code, ApplicationDbContext dbContext) =&gt;
{
    var shortenedUrl = await dbContext
        .ShortenedUrls
        .SingleOrDefaultAsync(s =&gt; s.Code == code);

    if (shortenedUrl is null)
    {
        return Results.NotFound();
    }

    return Results.Redirect(shortenedUrl.LongUrl);
});
</code></pre>
<p>This endpoint looks up the code in the database and, if found, redirects the user to the original long URL.
The response will have a <a href="https://datatracker.ietf.org/doc/html/rfc7231#section-6.4.3">302 (Found) status code</a> per the HTTP standards.</p>
<h2>URL Shortener Improvement Points</h2>
<p>While our basic URL shortener is functional, there are several areas we can improve:</p>
<ul>
<li><strong>Caching</strong>: Implement caching to reduce database load for frequently accessed URLs.</li>
<li><strong>Horizontal Scaling</strong>: Design the system to scale horizontally to handle increased load.</li>
<li><strong>Data Sharding</strong>: Implement data sharding to distribute data across multiple databases.</li>
<li><strong>Analytics</strong>: Introduce analytics to track URL usage and expose reports to users.</li>
<li><strong>User Accounts</strong>: Allow users to create accounts to manage their URLs.</li>
</ul>
<p>We've covered the key components of building a URL shortener using .NET.
You can take this further and implement the improvements points for a more robust solution.</p>
<p>If you want to see me build this from scratch, here's a <a href="https://youtu.be/2UoA_PoEvuA">video tutorial on YouTube.</a></p>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_074.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Balancing Cross-Cutting Concerns in Clean Architecture]]></title>
            <link>https://milanjovanovic.tech/blog/balancing-cross-cutting-concerns-in-clean-architecture</link>
            <guid isPermaLink="false">balancing-cross-cutting-concerns-in-clean-architecture</guid>
            <pubDate>Sat, 20 Jan 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Cross-cutting concerns are software aspects that affect the entire application. These are your common application-level functionalities that span several layers and tiers. Cross-cutting concerns should be centralized in one location. This prevents code duplication and tight coupling between components. In today's newsletter, I'll show you how to integrate cross-cutting concerns in Clean Architecture.]]></description>
            <content:encoded><![CDATA[<p>Cross-cutting concerns are software aspects that affect the entire application.
These are your common application-level functionalities that span several layers and tiers.
Cross-cutting concerns should be centralized in one location.
This prevents code duplication and tight coupling between components.</p>
<p>A few examples of cross-cutting concerns are:</p>
<ul>
<li>Authentication &amp; Authorization</li>
<li>Logging and tracing</li>
<li>Exception handling</li>
<li>Validation</li>
<li>Caching</li>
</ul>
<p>In today's newsletter, I'll show you how to integrate cross-cutting concerns in Clean Architecture.</p>
<h2>Cross-Cutting Concerns in Clean Architecture</h2>
<p>In Clean Architecture, <a href="https://en.wikipedia.org/wiki/Cross-cutting_concern">cross-cutting concerns</a>
play an essential role in ensuring the maintainability and scalability of your system.
Ideally, these concerns should be handled separately from the core business logic.
This aligns with Clean Architecture's principles, emphasizing the decoupling of concerns and modularity.
Your core business rules remain uncluttered, and the architecture stays clean and adaptable.</p>
<p>Ideally, you want to implement cross-cutting concerns in the Infrastructure layer.
You can use <a href="http://ASP.NET">ASP.NET</a> Core middleware, <a href="decorator-pattern-in-asp-net-core">decorators</a>, or MediatR pipeline behaviors.
Whichever approach you decide to use, the guiding idea remains the same.</p>
<p><img src="/blogs/mnw_073/clean_architecture_cross_cutting_concerns.png" alt="Cross-cutting concerns in Clean Architecture"></p>
<p>Let's see how to implement logging, validation, and caching as cross-cutting concerns.</p>
<h2>Cross-Cutting Concern #1 - Logging</h2>
<p>Logging is a fundamental aspect of software development, allowing you to look into an application's behavior.
It's vital for debugging, monitoring application health, and tracking user activities and system anomalies.
In the context of Clean Architecture, logging must be implemented in a way that maintains the separation of concerns.</p>
<p>An elegant way to achieve this is with <a href="cqrs-pattern-with-mediatr">MediatR's</a> <code>IPipelineBehavior</code>.
By encapsulating the logging logic inside a pipeline behavior, we ensure that logging is treated as a distinct concern, separate from business logic.
This approach enables us to capture detailed information about requests flowing through the application.</p>
<p>Effective logging should be consistent, context-rich, and non-intrusive.
Using <a href="5-serilog-best-practices-for-better-structured-logging">Serilog's</a> structured logging
capabilities, we can create logs that are not only informative but also easily queryable.
This is essential for understanding the state of the application at any given moment.</p>
<p>When done correctly, <a href="structured-logging-in-asp-net-core-with-serilog">structured logging</a>
provides invaluable insights into your application without cluttering the core logic.
It's a balance of granularity and clarity, ensuring that your logs are a helpful tool rather than a source of noise.</p>
<pre><code class="language-csharp">using Serilog.Context;

internal sealed class RequestLoggingPipelineBehavior&lt;TRequest, TResponse&gt;(
    ILogger&lt;RequestLoggingPipelineBehavior&lt;TRequest, TResponse&gt;&gt; logger)
    : IPipelineBehavior&lt;TRequest, TResponse&gt;
    where TRequest : class
    where TResponse : Result
{
    public async Task&lt;TResponse&gt; Handle(
        TRequest request,
        RequestHandlerDelegate&lt;TResponse&gt; next,
        CancellationToken cancellationToken)
    {
        string requestName = typeof(TRequest).Name;

        logger.LogInformation(
            &quot;Processing request {RequestName}&quot;,
            requestName);

        TResponse result = await next();

        if (result.IsSuccess)
        {
            logger.LogInformation(
                &quot;Completed request {RequestName}&quot;,
                requestName);
        }
        else
        {
            using (LogContext.PushProperty(&quot;Error&quot;, result.Error, true))
            {
                logger.LogError(
                    &quot;Completed request {RequestName} with error&quot;,
                    requestName);
            }
        }

        return result;
    }
}
</code></pre>
<h2>Cross-Cutting Concern #2 - Validation</h2>
<p>Validation is a critical cross-cutting concern in software engineering.
It serves as the first line of defense against incorrect data entering your system.
Validation guards the application against inconsistent data states and potential security vulnerabilities.</p>
<p>In the example below, I'm creating a <a href="cqrs-validation-with-mediatr-pipeline-and-fluentvalidation">validation pipeline behavior</a>.
This setup allows for a clean separation of validation logic from business logic.
The pipeline behavior ensures that each request is validated before it reaches the core processing logic.</p>
<p>In approaching validation, it's crucial to distinguish between two types:</p>
<ul>
<li>Input validation</li>
<li>Business rule validation</li>
</ul>
<p>Input validation checks for the correctness and format of the data (like string length, number ranges, and date formats), ensuring it meets the basic criteria before processing.</p>
<p>On the other hand, business rule validation is more about ensuring that the data adheres to your domain's specific rules and logic.</p>
<p>Effective validation practices significantly contribute to the resilience and reliability of an application.
By enforcing validation rules, you can maintain a high data quality standard and ensure a better user experience.</p>
<pre><code class="language-csharp">internal sealed class ValidationPipelineBehavior&lt;TRequest, TResponse&gt;(
    IEnumerable&lt;IValidator&lt;TRequest&gt;&gt; validators)
    : IPipelineBehavior&lt;TRequest, TResponse&gt;
    where TRequest : class
{
    public async Task&lt;TResponse&gt; Handle(
        TRequest request,
        RequestHandlerDelegate&lt;TResponse&gt; next,
        CancellationToken cancellationToken)
    {
        ValidationFailure[] validationFailures = await ValidateAsync(request);

        if (validationFailures.Length != 0)
        {
            throw new ValidationException(validationFailures);
        }

        return await next();
    }

    private async Task&lt;ValidationFailure[]&gt; ValidateAsync(TRequest request)
    {
        if (!validators.Any())
        {
            return [];
        }

        var context = new ValidationContext&lt;TRequest&gt;(request);

        ValidationResult[] validationResults = await Task.WhenAll(
            validators.Select(validator =&gt; validator.ValidateAsync(context)));

        ValidationFailure[] validationFailures = validationResults
            .Where(validationResult =&gt; !validationResult.IsValid)
            .SelectMany(validationResult =&gt; validationResult.Errors)
            .ToArray();

        return validationFailures;
    }
}
</code></pre>
<h2>Cross-Cutting Concern #3: Caching</h2>
<p>Caching is an essential cross-cutting concern in software development.
It's primarily aimed at enhancing performance and scalability.
Caching involves temporarily storing data in a fast-access layer.
This reduces the need to fetch or calculate the same information repeatedly.</p>
<p>The caching pipeline behavior, which you see below, implements the <a href="https://learn.microsoft.com/en-us/azure/architecture/patterns/cache-aside">Cache Aside pattern.</a>
This pattern involves checking the cache before processing the request and updating the cache with new data as needed.
It's a popular caching strategy due to its simplicity and effectiveness.
Here's a <a href="https://youtu.be/LOEYZRE72wE">video tutorial</a> if you want to see how I implemented this.</p>
<p>When implementing caching, it's crucial to consider:</p>
<ul>
<li><strong>What to Cache:</strong> Identify data that is expensive to compute or retrieve and stable enough to be cached.</li>
<li><strong>Cache Invalidations:</strong> Determine when and how cached data should be invalidated.</li>
<li><strong>Cache Configuration:</strong> Configure cache settings like expiration and size appropriately.</li>
</ul>
<p>Effective caching improves response times and reduces the load on your system, making it a critical strategy for building scalable .NET applications.</p>
<pre><code class="language-csharp">internal sealed class QueryCachingPipelineBehavior&lt;TRequest, TResponse&gt;(
    ICacheService cacheService,
    ILogger&lt;QueryCachingPipelineBehavior&lt;TRequest, TResponse&gt;&gt; logger)
    : IPipelineBehavior&lt;TRequest, TResponse&gt;
    where TRequest : ICachedQuery
    where TResponse : Result
{
    public async Task&lt;TResponse&gt; Handle(
        TRequest request,
        RequestHandlerDelegate&lt;TResponse&gt; next,
        CancellationToken cancellationToken)
    {
        TResponse? cachedResult = await cacheService.GetAsync&lt;TResponse&gt;(
            request.CacheKey,
            cancellationToken);

        string requestName = typeof(TRequest).Name;
        if (cachedResult is not null)
        {
            logger.LogInformation(&quot;Cache hit for {RequestName}&quot;, requestName);

            return cachedResult;
        }

        logger.LogInformation(&quot;Cache miss for {RequestName}&quot;, requestName);

        TResponse result = await next();

        if (result.IsSuccess)
        {
            await cacheService.SetAsync(
                request.CacheKey,
                result,
                request.Expiration,
                cancellationToken);
        }

        return result;
    }
}
</code></pre>
<h2>What To Do Next</h2>
<p>Managing cross-cutting concerns such as logging, caching, validation, and exception handling is not just about technical implementation.
It's about aligning these aspects with the core principles of <a href="clean-architecture-and-the-benefits-of-structured-software-design">Clean Architecture.</a>
By adopting the decoupling techniques we discussed, you can ensure that your .NET projects are robust and maintainable.</p>
<p>Each step you take towards refining your handling of cross-cutting concerns is a step towards a better software architecture.
I encourage you to experiment with these strategies in your own .NET projects.
If you want a structured guide covering these aspects in-depth, take a look at <a href="/pragmatic-clean-architecture">Pragmatic Clean Architecture.</a></p>
<p>Remember, the beauty of software development lies in the continuous evolution and relentless pursuit of improvement.</p>
<p>Hope this was helpful.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_073.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Extending HttpClient With Delegating Handlers in ASP.NET Core]]></title>
            <link>https://milanjovanovic.tech/blog/extending-httpclient-with-delegating-handlers-in-aspnetcore</link>
            <guid isPermaLink="false">extending-httpclient-with-delegating-handlers-in-aspnetcore</guid>
            <pubDate>Sat, 13 Jan 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[Delegating handlers are like ASP.NET Core middleware. Except they work with the HttpClient. I'll show you how to work with delegating handlers]]></description>
            <content:encoded><![CDATA[<p><a href="https://learn.microsoft.com/en-us/dotnet/api/system.net.http.delegatinghandler?view=net-8.0">Delegating handlers</a> are like <a href="3-ways-to-create-middleware-in-asp-net-core">ASP.NET Core middleware</a>.
Except they work with the <a href="the-right-way-to-use-httpclient-in-dotnet"><code>HttpClient</code></a>.
The <a href="http://ASP.NET">ASP.NET</a> Core request pipeline allows you to introduce custom behavior with <a href="3-ways-to-create-middleware-in-asp-net-core">middleware.</a>
You can solve many cross-cutting concerns using middleware — logging, tracing, validation, authentication, authorization, etc.</p>
<p>But, an important aspect here is that middleware works with incoming HTTP requests to your API.
Delegating handlers work with outgoing requests.</p>
<p><a href="https://learn.microsoft.com/en-us/dotnet/api/system.net.http.httpclient?view=net-8.0"><code>HttpClient</code></a> is my preferred way of sending HTTP requests in <a href="http://ASP.NET">ASP.NET</a> Core.
It's straightforward to use and solves most of my use cases.
You can use delegating handlers to extend the <code>HttpClient</code> with behavior before or after sending an HTTP request.</p>
<p>Today, I want to show you how to use a <a href="https://learn.microsoft.com/en-us/dotnet/api/system.net.http.delegatinghandler?view=net-8.0"><code>DelegatingHandler</code></a> to introduce:</p>
<ul>
<li>Logging</li>
<li>Resiliency</li>
<li>Authentication</li>
</ul>
<h2>Configuring an HttpClient</h2>
<p>Here's a very simple application that:</p>
<ul>
<li>Configures the <code>GitHubService</code> class as a typed HTTP client</li>
<li>Sets the <code>HttpClient.BaseAddress</code> to point to the GitHub API</li>
<li>Exposes an endpoint that retrieves a GitHub user by their username</li>
</ul>
<p>We're going to extend the <code>GitHubService</code> behavior using delegating handlers.</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);

builder.Services.AddHttpClient&lt;GitHubService&gt;(httpClient =&gt;
{
    httpClient.BaseAddress = new Uri(&quot;https://api.github.com&quot;);
});

var app = builder.Build();

app.MapGet(&quot;api/users/{username}&quot;, async (
    string username,
    GitHubService gitHubService) =&gt;
{
    var content = await gitHubService.GetByUsernameAsync(username);

    return Results.Ok(content);
});

app.Run();
</code></pre>
<p>The <code>GitHubService</code> class is a <a href="the-right-way-to-use-httpclient-in-dotnet#replacing-named-clients-with-typed-clients">typed client</a> implementation.
Typed clients allow you to expose a strongly typed API and hide the <code>HttpClient</code>.
The runtime takes care of providing a configured <code>HttpClient</code> instance through dependency injection.
You also don't have to think about disposing of the <code>HttpClient</code>.
It's resolved from an underlying <code>IHttpClientFactory</code> that manages the <code>HttpClient</code> lifetime.</p>
<pre><code class="language-csharp">public class GitHubService(HttpClient client)
{
    public async Task&lt;GitHubUser?&gt; GetByUsernameAsync(string username)
    {
        var url = $&quot;users/{username}&quot;;

        return await client.GetFromJsonAsync&lt;GitHubUser&gt;(url);
    }
}
</code></pre>
<h2>Logging HTTP Requests Using Delegating Handlers</h2>
<p>Let's start with a simple example.
We will add logging before and after sending an HTTP request.
For this, we will to create a custom delegating handler - <code>LoggingDelegatingHandler</code>.</p>
<p>The custom delegating handler implements the <code>DelegatingHandler</code> base class.
Then, you can override the <code>SendAsync</code> method to introduce additional behavior.</p>
<pre><code class="language-csharp">public class LoggingDelegatingHandler(ILogger&lt;LoggingDelegatingHandler&gt; logger)
    : DelegatingHandler
{
    protected override async Task&lt;HttpResponseMessage&gt; SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        try
        {
            logger.LogInformation(&quot;Before HTTP request&quot;);

            var result = await base.SendAsync(request, cancellationToken);

            result.EnsureSuccessStatusCode();

            logger.LogInformation(&quot;After HTTP request&quot;);

            return result;
        }
        catch (Exception e)
        {
            logger.LogError(e, &quot;HTTP request failed&quot;);

            throw;
        }
    }
}
</code></pre>
<p>You also need to register the <code>LoggingDelegatingHandler</code> with dependency injection.
Delegating handlers must be registered as <strong>transient</strong> services.</p>
<p>The <code>AddHttpMessageHandler</code> method adds the <code>LoggingDelegatingHandler</code> as a delegating handler for the <code>GitHubService</code>.
Any HTTP request sent using the <code>GitHubService</code> will first go through the <code>LoggingDelegatingHandler</code>.</p>
<pre><code class="language-csharp">builder.Services.AddTransient&lt;LoggingDelegatingHandler&gt;();

builder.Services.AddHttpClient&lt;GitHubService&gt;(httpClient =&gt;
{
    httpClient.BaseAddress = new Uri(&quot;https://api.github.com&quot;);
})
.AddHttpMessageHandler&lt;LoggingDelegatingHandler&gt;();
</code></pre>
<p>Let's see what else we can do.</p>
<h2>Adding Resiliency With Delegating Handlers</h2>
<p>Building <a href="https://learn.microsoft.com/en-us/dotnet/core/resilience/?tabs=dotnet-cli">resilient</a> applications is an important requirement for cloud development.</p>
<p>The <code>RetryDelegatingHandler</code> class uses <a href="https://github.com/App-vNext/Polly">Polly</a> to create an <code>AsyncRetryPolicy</code>.
The retry policy wraps the HTTP request and retries it in case of a transient failure.</p>
<pre><code class="language-csharp">public class RetryDelegatingHandler : DelegatingHandler
{
    private readonly AsyncRetryPolicy&lt;HttpResponseMessage&gt; _retryPolicy =
        Policy&lt;HttpResponseMessage&gt;
            .Handle&lt;HttpRequestException&gt;()
            .RetryAsync(2);

    protected override async Task&lt;HttpResponseMessage&gt; SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        var policyResult = await _retryPolicy.ExecuteAndCaptureAsync(
            () =&gt; base.SendAsync(request, cancellationToken));

        if (policyResult.Outcome == OutcomeType.Failure)
        {
            throw new HttpRequestException(
                &quot;Something went wrong&quot;,
                policyResult.FinalException);
        }

        return policyResult.Result;
    }
}
</code></pre>
<p>You also need to register the <code>RetryDelegatingHandler</code> with dependency injection.
Also, remember to configure it as a message handler.
In this example, I'm chaining two delegating handlers together, and they will run one after another.</p>
<pre><code class="language-csharp">builder.Services.AddTransient&lt;RetryDelegatingHandler&gt;();

builder.Services.AddHttpClient&lt;GitHubService&gt;(httpClient =&gt;
{
    httpClient.BaseAddress = new Uri(&quot;https://api.github.com&quot;);
})
.AddHttpMessageHandler&lt;LoggingDelegatingHandler&gt;()
.AddHttpMessageHandler&lt;RetryDelegatingHandler&gt;();
</code></pre>
<h2>Solving Authentication With Delegating Handlers</h2>
<p>Authentication is a cross-cutting concern you will have to solve in any microservices application.
A common use case for delegating handlers is adding the <code>Authorization</code> header before sending an HTTP request.</p>
<p>For example, the GitHub API requires an access token to be present for authenticating incoming requests.
The <code>AuthenticationDelegatingHandler</code> class adds the <code>Authorization</code> header value from the <code>GitHubOptions</code>.
Another requirement is specifying the <code>User-Agent</code> header, which is set from the app configuration.</p>
<pre><code class="language-csharp">public class AuthenticationDelegatingHandler(IOptions&lt;GitHubOptions&gt; options)
    : DelegatingHandler
{
    protected override Task&lt;HttpResponseMessage&gt; SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        request.Headers.Add(&quot;Authorization&quot;, options.Value.AccessToken);
        request.Headers.Add(&quot;User-Agent&quot;, options.Value.UserAgent);

        return base.SendAsync(request, cancellationToken);
    }
}
</code></pre>
<p>Don't forget to configure the <code>AuthenticationDelegatingHandler</code> with the <code>GitHubService</code>:</p>
<pre><code class="language-csharp">builder.Services.AddTransient&lt;AuthenticationDelegatingHandler&gt;();

builder.Services.AddHttpClient&lt;GitHubService&gt;(httpClient =&gt;
{
    httpClient.BaseAddress = new Uri(&quot;https://api.github.com&quot;);
})
.AddHttpMessageHandler&lt;LoggingDelegatingHandler&gt;()
.AddHttpMessageHandler&lt;RetryDelegatingHandler&gt;()
.AddHttpMessageHandler&lt;AuthenticationDelegatingHandler&gt;();
</code></pre>
<p>Here's a more involved authentication example using the <code>KeyCloakAuthorizationDelegatingHandler</code>.
This is a delegating handler that acquires the access token from <a href="https://www.keycloak.org/">Keycloak</a>.
Keycloak is an open-source identity and access management service.</p>
<p>I used Keycloak as the identity provider in my <a href="/pragmatic-clean-architecture">Pragmatic Clean Architecture</a> course.</p>
<p>The delegating handler in this example uses an <a href="https://oauth.net/2/">OAuth 2.0</a> <a href="https://www.oauth.com/oauth2-servers/access-tokens/client-credentials/">client credentials</a> grant flow to obtain an access token.
This grant is used when applications request an access token to access their own resources, not on behalf of a user.</p>
<pre><code class="language-csharp">public class KeyCloakAuthorizationDelegatingHandler(
    IOptions&lt;KeycloakOptions&gt; keycloakOptions)
    : DelegatingHandler
{
    protected override async Task&lt;HttpResponseMessage&gt; SendAsync(
        HttpRequestMessage request,
        CancellationToken cancellationToken)
    {
        var authToken = await GetAccessTokenAsync();

        request.Headers.Authorization = new AuthenticationHeaderValue(
            JwtBearerDefaults.AuthenticationScheme,
            authToken.AccessToken);

        var httpResponseMessage = await base.SendAsync(
            request,
            cancellationToken);

        httpResponseMessage.EnsureSuccessStatusCode();

        return httpResponseMessage;
    }

    private async Task&lt;AuthToken&gt; GetAccessTokenAsync()
    {
        var params = new KeyValuePair&lt;string, string&gt;[]
        {
            new(&quot;client_id&quot;, _keycloakOptions.Value.AdminClientId),
            new(&quot;client_secret&quot;, _keycloakOptions.Value.AdminClientSecret),
            new(&quot;scope&quot;, &quot;openid email&quot;),
            new(&quot;grant_type&quot;, &quot;client_credentials&quot;)
        };

        var content = new FormUrlEncodedContent(params);

        var authRequest = new HttpRequestMessage(
            HttpMethod.Post,
            new Uri(_keycloakOptions.TokenUrl))
        {
            Content = content
        };

        var response = await base.SendAsync(authRequest, cancellationToken);

        response.EnsureSuccessStatusCode();

        return await response.Content.ReadFromJsonAsync&lt;AuthToken&gt;() ??
               throw new ApplicationException();
    }
}
</code></pre>
<h2>Takeaway</h2>
<p>Delegating handlers give you a powerful mechanism to extend the behavior when sending requests with an <code>HttpClient</code>.
You can use delegating handlers to solve cross-cutting concerns, similar to how you would use middleware.</p>
<p>Here are a few ideas on how you could use delegating handlers:</p>
<ul>
<li>Logging before and after sending HTTP requests</li>
<li>Introducing resilience policies (retry, fallback)</li>
<li>Validating the HTTP request content</li>
<li>Authenticating with an external API</li>
</ul>
<p>I'm sure you can come up with a few use cases yourself.</p>
<p>I made a video showing how to <a href="https://youtu.be/_u6v4D6qgDI">implement delegating handlers</a>, and you can <a href="https://youtu.be/_u6v4D6qgDI">watch it here.</a></p>
<p>Thanks for reading, and stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_072.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Using MassTransit with RabbitMQ and Azure Service Bus]]></title>
            <link>https://milanjovanovic.tech/blog/using-masstransit-with-rabbitmq-and-azure-service-bus</link>
            <guid isPermaLink="false">using-masstransit-with-rabbitmq-and-azure-service-bus</guid>
            <pubDate>Sat, 06 Jan 2024 00:00:00 GMT</pubDate>
            <description><![CDATA[MassTransit is an open-source distributed application framework for .NET. It provides a messaging abstraction on top of the supported message transports. In this week's issue, I'll show you how to use MassTransit.]]></description>
            <content:encoded><![CDATA[<p><a href="https://masstransit.io/">MassTransit</a> is an open-source distributed application framework for .NET.
It provides a messaging abstraction on top of the supported message transports.
MassTransit lets you focus on adding business value instead of worrying about messaging complexity.</p>
<p>MassTransit supports many message transport technologies.
Here are a few that are popular:</p>
<ul>
<li>RabbitMQ</li>
<li>Azure Service Bus</li>
<li>Amazon SQS</li>
<li>Kafka</li>
</ul>
<p>In today's newsletter, I'll show you how to install and configure MassTransit in .NET.
We'll connect MassTransit to a few message brokers - RabbitMQ and Azure Service Bus.
And we will also cover how to publish and consume messages with MassTransit.</p>
<h2>Why Use MassTransit?</h2>
<p>MassTransit solves many challenges of building distributed applications.
You (almost) don't have to think about the underlying message transport.
This allows you to focus on providing business value.</p>
<p>Here are a few things MassTransit does for you:</p>
<ul>
<li><strong>Message routing</strong> - Type-based publish/subscribe, automatic broker topology configuration</li>
<li><strong>Exception handling</strong> - Messages can be retried or moved to an error queue</li>
<li><strong>Dependency injection</strong> - Service collection configuration and scope service provider</li>
<li><strong>Request-Response</strong> - Handle requests with automatic response routing</li>
<li><strong>Observability</strong> - Native <a href="https://opentelemetry.io/">Open Telemetry (OTEL)</a> support</li>
<li><strong>Scheduling</strong> - Schedule message delivery using transport delay, <a href="http://Quartz.NET">Quartz.NET</a>, or Hangfire</li>
<li><strong>Sagas</strong> - Reliable, durable, event-driven workflow orchestration</li>
</ul>
<p>Let's see how to start using MassTransit.</p>
<h2>Installing and Configuring MassTransit with RabbitMQ</h2>
<p>You need to install the <code>MassTransit</code> library.
If you already have a message transport, you can install the respective transport library.
Let's add the <code>MassTransit.RabbitMQ</code> library to configure <a href="https://www.rabbitmq.com/">RabbitMQ</a> as the transport mechanism.</p>
<pre><code class="language-powershell">Install-Package MassTransit

Install-Package MassTransit.RabbitMQ
</code></pre>
<p>Then, you can configure the required services for <code>MassTransit</code>.
The <code>AddMassTransit</code> method accepts a delegate where you can configure many settings.
For example, you can set the messaging endpoints to use kebab case naming by calling <code>SetKebabCaseEndpointNameFormatter</code>.
This is also where you configure the transport mechanism.
Calling <code>UsingRabbitMq</code> allows you to connect RabbitMQ as the transport mechanism.</p>
<pre><code class="language-csharp">builder.Services.AddMassTransit(busConfigurator =&gt;
{
    busConfigurator.SetKebabCaseEndpointNameFormatter();

    busConfigurator.UsingRabbitMq((context, configurator) =&gt;
    {
        configurator.Host(&quot;localhost&quot;, &quot;/&quot;, h =&gt;
        {
            h.Username(&quot;guest&quot;);
            h.Password(&quot;guest&quot;);
        });

        configurator.ConfigureEndpoints(context);
    });
});
</code></pre>
<p>MassTransit takes care of setting up the required broker topology.
RabbitMQ supports exchanges and queues, so messages are sent or published to exchanges.
RabbitMQ routes those messages through exchanges to the appropriate queues.</p>
<p>You can start RabbitMQ locally inside a Docker container:</p>
<pre><code class="language-csharp">docker run -d --name rabbitmq -p 5672:5672
</code></pre>
<h2>Configuring MassTransit with Azure Service Bus</h2>
<p><a href="https://azure.microsoft.com/en-us/products/service-bus">Azure Service Bus</a>
is a cloud-based message broker with support for queues and topics.
MassTransit fully supports Azure Service Bus, including many advanced features and capabilities.
However, you must be on the Standard or Premium tier of the Microsoft Azure Service Bus service.</p>
<p>To configure MassTransit to work with Azure Service Bus, you need to install the required transport library:</p>
<pre><code class="language-powershell">Install-Package MassTransit.Azure.ServiceBus.Core
</code></pre>
<p>Then, you can connect to Azure Service Bus by calling <code>UsingAzureServiceBus</code> and providing the connection string.
Everything else remains the same.
MassTransit takes care of configuring the broker topology.
MassTransit sends messages to topics, and Azure Service Bus routes those messages to the appropriate queues.</p>
<pre><code class="language-csharp">builder.Services.AddMassTransit(busConfigurator =&gt;
{
    busConfigurator.SetKebabCaseEndpointNameFormatter();

    busConfigurator.UsingAzureServiceBus((context, configurator) =&gt;
    {
        configurator.Host(&quot;&lt;CONNECTION_STRING&gt;&quot;);

        configurator.ConfigureEndpoints(context);
    });
});
</code></pre>
<h2>Using the MassTransit In Memory Transport</h2>
<p>You can also configure MassTransit to use an in-memory transport.
It's useful for testing, because it doesn't require a message broker to be running.
Another advantage is that it's fast.</p>
<p>However, there's a big problem with the in-memory transport - <strong>it's not durable.</strong></p>
<p>If the message bus is stopped, all messages are lost.
Don't use the in-memory transport for production systems.</p>
<p>It will only work on a single machine.
So, it doesn't make sense for distributed applications.</p>
<pre><code class="language-csharp">builder.Services.AddMassTransit(busConfigurator =&gt;
{
    busConfigurator.SetKebabCaseEndpointNameFormatter();

    busConfigurator.UsingInMemory((context, configurator) =&gt;
    {
        configurator.ConfigureEndpoints(context);
    });
});
</code></pre>
<h2>Message Types</h2>
<p>MassTransit requires message types to be <a href="https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/keywords/reference-types">reference types.</a>
So, you can use a <code>class</code>, <code>record</code> or <code>interface</code> to define a message.</p>
<p>You'll often create two types of messages: commands and events.</p>
<p>A command is an instruction to perform some action.
Commands are intended to have exactly one consumer.
Commands are expressed using verbs first: <code>CreateArticle</code>, <code>PublishArticle</code>, <code>ShareArticle</code>.</p>
<p>An event represents that something of significance happened.
Events can have one or many consumers.
Events should have a name in the past tense: <code>ArticleCreated</code>, <code>ArticlePublished</code>, <code>ArticleShared</code>.</p>
<p>Here's an example <code>ArticleCreated</code> message containing information about an article:</p>
<pre><code class="language-csharp">public record ArticleCreated
{
    public Guid Id { get; init; }
    public string Title { get; init; }
    public string Content { get; init; }
    public DateTime CreatedOnUtc { get; init; }
}
</code></pre>
<p>Using <code>public set</code> or <code>public init</code> properties is recommended to avoid serialization problems with <code>System.Text.Json</code>.</p>
<h2>Publishing and Consuming Messages</h2>
<p>You can use the <code>IPublishEndpoint</code> service to publish messages with MassTransit.
The framework routes the message to the appropriate queue or topic based on the message type.</p>
<pre><code class="language-csharp">app.MapPost(&quot;article&quot;, async (
    CreateArticleRequest request,
    IPublishEndpoint publishEndpoint) =&gt;
{
    await publishEndpoint.Publish(new ArticleCreated
    {
        Id = Guid.NewGuid(),
        Title = request.Title,
        Content = request.Content,
        CreatedOnUtc = DateTime.UtcNow
    });

    return Results.Accepted();
});
</code></pre>
<p>To consume an <code>ArticleCreated</code> message, you need to implement the <code>IConsumer</code> interface.
The <code>IConsumer</code> has one <code>Consume</code> method where you place your business logic.
The consumer also gives you access to the <code>ConsumeContext</code>, which you can use to send further messages.</p>
<pre><code class="language-csharp">public class ArticleCreatedConsumer(ApplicationDbContext dbContext)
    : IConsumer&lt;ArticleCreatedEvent&gt;
{
    public async Task Consume(ConsumeContext&lt;ArticleCreatedEvent&gt; context)
    {
        ArticleCreated message = context.Message;

        var article = new Article
        {
            Id = message.Id,
            Title = message.Title,
            Content = message.Content,
            CreatedOnUtc = message.CreatedOnUtc
        };

        dbContext.Add(article);

        await dbContext.SaveChangesAsync();
    }
}
</code></pre>
<p>MassTransit doesn't automatically know that the <code>ArticleCreatedConsumer</code> exists.
You have to configure the consumer when calling <code>AddMassTransit</code>.
The <code>AddConsumer</code> method registers the consumer type with the bus.</p>
<pre><code class="language-csharp">builder.Services.AddMassTransit(busConfigurator =&gt;
{
    busConfigurator.SetKebabCaseEndpointNameFormatter();

    busConfigurator.AddConsumer&lt;ArticleCreatedConsumer&gt;();

    // ...
});
</code></pre>
<h2>Next Steps</h2>
<p><strong>MassTransit</strong> is an excellent messaging library I often use when building distributed applications.
The setup is straightforward, and there are only a few important abstractions.
You need to know abstraction for publishing messages (<code>IPublishEndpoint</code>) and consuming messages (<code>IConsumer</code>).
MassTransit takes care of doing the heavy lifting for you.</p>
<p>If you aren't already using it, I highly recommend adding MassTransit to your toolbox.</p>
<p>Here are some more practical <strong>MassTransit learning resources</strong>:</p>
<ul>
<li><a href="https://youtu.be/NjsoykEOkrk">Request-Response pattern with MassTransit</a></li>
<li><a href="https://youtu.be/CTKWFMZVIWA">RabbitMQ and MassTransit tutorial</a></li>
<li><a href="https://youtu.be/MzC0PgYocmk">Microservices with RabbitMQ</a></li>
<li><a href="implementing-the-saga-pattern-with-rebus-and-rabbitmq">Saga pattern with RabbitMQ</a></li>
<li><a href="orchestration-vs-choreography">Orchestration vs Choreography</a></li>
<li><a href="messaging-made-easy-with-azure-service-bus">Messaging with Azure Service Bus</a></li>
</ul>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_071.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[API Versioning in ASP.NET Core]]></title>
            <link>https://milanjovanovic.tech/blog/api-versioning-in-aspnetcore</link>
            <guid isPermaLink="false">api-versioning-in-aspnetcore</guid>
            <pubDate>Sat, 30 Dec 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[API versioning allows your API to evolve independently from the clients using it. I'll show you how to implement API versioning in ASP.NET Core.]]></description>
            <content:encoded><![CDATA[<p>In the past year, I built and maintained a large public API.
The API has dozens of integrations, serving mainly mobile applications.</p>
<p>When your API is serving so many clients, breaking changes are expensive.</p>
<p>So, everything I implemented on the public API had to be planned.</p>
<p>Adding a new field? Forget about it.</p>
<p>Renaming existing fields? Forget about it.</p>
<p>If I wanted to introduce breaking changes, I had to version the API.</p>
<p>Today, I'll show you how to implement API versioning in <a href="http://ASP.NET">ASP.NET</a> Core.</p>
<h2>What Is API Versioning?</h2>
<p>API versioning is the practice of assigning distinct identifiers to different iterations of an API,
so that clients can target a specific version and continue working even as the API evolves.</p>
<p>Without versioning, every change you make to your API is potentially a breaking change for clients already integrated with it.
With .NET API versioning, you can introduce a new <code>v2</code> endpoint with the updated behavior while <code>v1</code> continues to work exactly as before.
Clients upgrade on their own schedule, and you maintain full control over which old versions to deprecate and remove.</p>
<p>This is what makes API versioning essential for any public-facing or long-lived .NET API.
The same principles apply whether you're building with <a href="http://ASP.NET">ASP.NET</a> Core MVC controllers or Minimal APIs.</p>
<h2>Why API Versioning Matters in .NET APIs</h2>
<p>API versioning allows your API to evolve independently from the clients using it.</p>
<p>Introducing breaking changes to your API is a bad user experience.
API versioning gives you a mechanism to avoid exposing breaking changes to clients.
Instead of making a breaking change, you introduce a new API version.</p>
<p>What's the definition of a breaking change?</p>
<p>This isn't an exhaustive list, but a few examples of breaking changes are:</p>
<ul>
<li>Removing or renaming APIs or API parameters</li>
<li>Changing the behavior of existing APIs</li>
<li>Changing the API response contract</li>
<li>Changing the API error codes</li>
</ul>
<p>You can decide what a breaking change means for your API.
For example, adding a new field to the response doesn't have to be a breaking change.</p>
<p>Let's see how to implement API versioning.</p>
<h2>How to Implement API Versioning in .NET Core</h2>
<p>Let's start by installing three NuGet packages that we'll need to implement API versioning:</p>
<ul>
<li><code>Asp.Versioning.Http</code></li>
<li><code>Asp.Versioning.Mvc</code></li>
<li><code>Asp.Versioning.Mvc.ApiExplorer</code></li>
</ul>
<pre><code class="language-powershell">Install-Package Asp.Versioning.Http # This is needed for Minimal APIs
Install-Package Asp.Versioning.Mvc # This is needed for Controllers
Install-Package Asp.Versioning.Mvc.ApiExplorer
</code></pre>
<p>This allows us to call <code>AddApiVersioning</code> and provide a delegate to configure the <code>ApiVersioningOptions</code>.</p>
<pre><code class="language-csharp">builder.Services.AddApiVersioning(options =&gt;
{
    options.DefaultApiVersion = new ApiVersion(1);
    options.ReportApiVersions = true;
    options.AssumeDefaultVersionWhenUnspecified = true;
    options.ApiVersionReader = ApiVersionReader.Combine(
        new UrlSegmentApiVersionReader(),
        new HeaderApiVersionReader(&quot;X-Api-Version&quot;));
})
.AddMvc() // This is needed for controllers
.AddApiExplorer(options =&gt;
{
    options.GroupNameFormat = &quot;'v'V&quot;;
    options.SubstituteApiVersionInUrl = true;
});
</code></pre>
<p>Here's the explanation for the <code>ApiVersioningOptions</code> properties:</p>
<ul>
<li><code>DefaultApiVersion</code> - Sets the default API version. Typically, this will be <code>v1.0</code>.</li>
<li><code>ReportApiVersions</code> - Reports the supported API versions in the <code>api-supported-versions</code> response header.</li>
<li><code>AssumeDefaultVersionWhenUnspecified</code> - Uses the <code>DefaultApiVersion</code> when the client didn't provide an explicit version.</li>
<li><code>ApiVersionReader</code> - Configures how to read the API version specified by the client. The default value is <code>QueryStringApiVersionReader</code>.</li>
</ul>
<p>The <code>AddApiExplorer</code> method is helpful if you are using Swagger.
It will fix the endpoint routes and substitute the API version route parameter.</p>
<h2>Types of API Versioning in .NET</h2>
<p>There are several strategies for .NET API versioning, each with different tradeoffs.</p>
<h3>URL Versioning</h3>
<p>URL versioning embeds the version directly in the request path:</p>
<p><code>GET https://localhost:5001/api/v1/workouts</code></p>
<p>This is the most explicit approach. The version is visible at a glance, trivial to test in a browser or with curl,
and is the most widely-used strategy for public .NET APIs.
The <code>UrlSegmentApiVersionReader</code> in <code>Asp.Versioning.Http</code> handles this automatically.</p>
<h3>Header Versioning</h3>
<p>Header versioning passes the API version via a custom request header:</p>
<p><code>GET https://localhost:5001/api/workouts</code> with header <code>X-Api-Version: 1</code></p>
<p>This keeps your URLs clean, but the version is invisible unless clients inspect outgoing headers.
The <code>HeaderApiVersionReader</code> supports this strategy.</p>
<h3>Query String Versioning</h3>
<p>Query string versioning appends the version as a query parameter:</p>
<p><code>GET https://localhost:5001/api/workouts?api-version=1</code></p>
<p>This is the default behavior in <code>Asp.Versioning.Http</code> using <code>QueryStringApiVersionReader</code>.
It's easy to use during development and testing without any special tooling.</p>
<p>There are a few other ways to implement API versioning, such as using the <code>accept</code> or <code>content-type</code> headers, but they aren't used often.</p>
<p>The <code>Asp.Versioning.Http</code> library provides several <code>IApiVersionReader</code> implementations to support these strategies:</p>
<ul>
<li><code>UrlSegmentApiVersionReader</code></li>
<li><code>HeaderApiVersionReader</code></li>
<li><code>QueryStringApiVersionReader</code></li>
<li><code>MediaTypeApiVersionReader</code></li>
</ul>
<p><a href="https://github.com/Microsoft/api-guidelines/blob/master/Guidelines.md#12-versioning">Microsoft's API versioning guidelines</a>
suggest using URL or query string parameter versioning.</p>
<p>I use URL versioning almost exclusively in the applications I'm developing.</p>
<h2>Versioning Controllers</h2>
<p>To implement API versioning in <a href="http://ASP.NET">ASP.NET</a> controllers, you have to decorate the controller with the <code>ApiVersion</code> attribute.</p>
<p>The <code>ApiVersion</code> attribute allows you to specify which API versions that <code>WorkoutsController</code> supports.
In this case, the controller supports both <code>v1</code> and <code>v2</code>.
You use the <code>MapToApiVersion</code> attribute on the endpoints to specify the concrete API version.</p>
<p>The route parameter <code>v{v:apiVersion}</code> lets you specify the API version using <code>v1</code> or <code>v2</code> in the URL.</p>
<pre><code class="language-csharp">[ApiVersion(1)]
[ApiVersion(2)]
[ApiController]
[Route(&quot;api/v{v:apiVersion}/workouts&quot;)]
public class WorkoutsController : ControllerBase
{
    [MapToApiVersion(1)]
    [HttpGet(&quot;{workoutId}&quot;)]
    public IActionResult GetWorkoutV1(Guid workoutId)
    {
        return Ok(new GetWorkoutByIdQuery(workoutId).Handle());
    }

    [MapToApiVersion(2)]
    [HttpGet(&quot;{workoutId}&quot;)]
    public IActionResult GetWorkoutV2(Guid workoutId)
    {
        return Ok(new GetWorkoutByIdQuery(workoutId).Handle());
    }
}
</code></pre>
<h2>Deprecating API Versions</h2>
<p>If you want to deprecate an old API version, you can set the <code>Deprecated</code> property on the <code>ApiVersion</code> attribute.
The deprecated API versions will be reported using the <code>api-deprecated-versions</code> response header.</p>
<pre><code class="language-csharp">[ApiVersion(1, Deprecated = true)]
[ApiVersion(2)]
[ApiController]
[Route(&quot;api/v{v:apiVersion}/workouts&quot;)]
public class WorkoutsController : ControllerBase
{
}
</code></pre>
<h2>Versioning Minimal APIs</h2>
<p>Versioning Minimal APIs requires you to define an <code>ApiVersionSet</code>, which you'll pass to the endpoints.</p>
<ul>
<li><code>NewApiVersionSet</code> - Creates a new <code>ApiVersionSetBuilder</code> that you can use to configure the <code>ApiVersionSet</code>.</li>
<li><code>HasApiVersion</code> - Indicates that the <code>ApiVersionSet</code> supports the specified <code>ApiVersion</code>.</li>
<li><code>ReportApiVersions</code>- Indicates that all APIs in the <code>ApiVersionSet</code> will report their versions.</li>
</ul>
<p>After creating the <code>ApiVersionSet</code>, you must pass it to a Minimal API endpoint by calling <code>WithApiVersionSet</code>.
You can map to an explicit API version by calling <code>MapToApiVersion</code>.</p>
<pre><code class="language-csharp">ApiVersionSet apiVersionSet = app.NewApiVersionSet()
    .HasApiVersion(new ApiVersion(1))
    .HasApiVersion(new ApiVersion(2))
    .ReportApiVersions()
    .Build();

app.MapGet(&quot;api/v{version:apiVersion}/workouts/{workoutId}&quot;, async (
    Guid workoutId,
    ISender sender,
    CancellationToken ct) =&gt;
{
    var query = new GetWorkoutByIdQuery(workoutId);

    Result&lt;WorkoutResponse&gt; result = await sender.Send(query, ct);

    return result.Match(Results.Ok, CustomResults.Problem);
})
.WithApiVersionSet(apiVersionSet)
.MapToApiVersion(1);
</code></pre>
<p>Specifying the <code>ApiVersionSet</code> for each Minimal API endpoint can be cumbersome.
So you can define a route group and set the <code>ApiVersionSet</code> only once.
Route groups are also practical because they allow you to specify the route prefix.</p>
<pre><code class="language-csharp">ApiVersionSet apiVersionSet = app.NewApiVersionSet()
    .HasApiVersion(new ApiVersion(1))
    .ReportApiVersions()
    .Build();

RouteGroupBuilder group = app
    .MapGroup(&quot;api/v{version:apiVersion}&quot;)
    .WithApiVersionSet(apiVersionSet);

group.MapGet(&quot;workouts&quot;, ...);
group.MapGet(&quot;workouts/{workoutId}&quot;, ...);
</code></pre>
<h2>API Versioning Best Practices in .NET</h2>
<p>Here are key practices to keep in mind when implementing API versioning in .NET Core:</p>
<ul>
<li><strong>Version from day one.</strong> It's far easier to add API versioning before you have external clients. Even a single version (<code>v1</code>) gives you room to introduce <code>v2</code> later without painful migrations.</li>
<li><strong>Define what a breaking change is.</strong> Agree as a team on what constitutes a breaking change (removing fields, changing response contracts, modifying error codes), and document it. Adding a new optional field or a new endpoint is typically safe.</li>
<li><strong>Deprecate, don't delete.</strong> When retiring a version, mark it as deprecated first using <code>[ApiVersion(1, Deprecated = true)]</code>. Give clients time to migrate before removing it entirely.</li>
<li><strong>Report supported versions.</strong> Set <code>ReportApiVersions = true</code> so that clients can discover available versions via the <code>api-supported-versions</code> response header.</li>
<li><strong>Keep old versions stable.</strong> Once a version is published, treat it as immutable. New features go into a new version, not the old one.</li>
<li><strong>Prefer URL versioning for public APIs.</strong> It's the most discoverable, easiest to cache, and works across all HTTP clients without special configuration.</li>
</ul>
<h2>Takeaway</h2>
<p>API versioning is one of the best practices for designing modern APIs.
Consider implementing API versioning from the first release.
This makes it easier for clients to support future API versions.
And it gets your team used to managing breaking changes and versioning the API.</p>
<p>You can use the <code>Asp.Versioning.Http</code> library to add API versioning in <a href="http://ASP.NET">ASP.NET</a> Core.
Define the supported API versions, and start using them in your endpoints.</p>
<p>Remember to agree as a team what represents a breaking change.
This should be well documented in the team's API design guidelines.</p>
<p>My preferred way to implement API versioning is using URL versioning. It's simple and explicit.</p>
<p>And since this is the last issue for the year, I wish you a happy and prosperous new year.</p>
<p>Thanks for reading, and stay awesome!</p>
<h2>Frequently Asked Questions</h2>
<h3>What is API versioning?</h3>
<p>API versioning is a strategy that allows your API to evolve independently from the clients consuming it.
Instead of making breaking changes that force all clients to update at once, you introduce a new version of the API and let clients migrate on their own schedule.</p>
<h3>Is API versioning required in <a href="http://ASP.NET">ASP.NET</a> Core?</h3>
<p>API versioning is not required by default in <a href="http://ASP.NET">ASP.NET</a> Core, but it is strongly recommended for public or long-lived APIs.
The <code>Asp.Versioning.Mvc</code> NuGet package adds full versioning support.</p>
<h3>What is the best API versioning strategy in .NET?</h3>
<p>The most common strategy is URL segment versioning (e.g. <code>/api/v1/resource</code>) because it is explicit and easy to test in a browser.
Header versioning is cleaner but less discoverable.
The best choice depends on your client base.</p>
<h3>Does <a href="http://ASP.NET">ASP.NET</a> Core support API versioning out of the box?</h3>
<p>Not natively. You need to install the <code>Asp.Versioning.Http</code> and <code>Asp.Versioning.Mvc</code> packages, then call <code>AddApiVersioning()</code> in your service registration.</p>
<h3>How do you version Minimal APIs in .NET?</h3>
<p>With Minimal APIs, you assign endpoint groups to an API version group using <code>MapGroup</code> and tag them with the appropriate <code>ApiVersion</code> attribute.
The <code>Asp.Versioning.Http</code> package provides the necessary extension methods.</p>
<h3>Is adding a new response field a breaking change?</h3>
<p>Adding a new field to a response is generally not a breaking change for well-written API clients.
However, removing or renaming fields, changing types, or altering behavior are all breaking changes.</p>
<h3>Should I use URL or header API versioning in .NET?</h3>
<p>URL versioning is simpler to implement and easier for clients to consume.
Header versioning keeps URLs clean but requires clients to set a custom header.
Most public APIs use URL versioning.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_070.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Value Objects in .NET (DDD Fundamentals)]]></title>
            <link>https://milanjovanovic.tech/blog/value-objects-in-dotnet-ddd-fundamentals</link>
            <guid isPermaLink="false">value-objects-in-dotnet-ddd-fundamentals</guid>
            <pubDate>Sat, 23 Dec 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Value Objects are one of the building blocks of Domain-Driven Design. Today, I'll show you some best practices for implementing Value Objects.]]></description>
            <content:encoded><![CDATA[<p><strong>Value Objects</strong> are one of the building blocks of Domain-Driven Design.
DDD is a software development approach for solving problems in complex domains.</p>
<p>Value objects encapsulate a set of primitive values and related invariants.
A few examples of value objects are money and date range objects.
Money consists of an amount and currency.
A date range consists of start and end dates.</p>
<p>Today, I'll show you some best practices for implementing Value Objects.</p>
<h2>What are Value Objects?</h2>
<p>Let's start with the definition from the Domain-Driven Design book:</p>
<blockquote>
<p>An object that represents a descriptive aspect of the domain with no conceptual identity is called a Value Object.
Value Objects are instantiated to represent elements of the design that we care about only for what they are, not who or which they are.</p>
</blockquote>
<p><em>— <a href="http://www.amazon.com/Domain-Driven-Design-Tackling-Complexity-Software/dp/0321125215">Eric Evans</a></em></p>
<p>Value objects are different from entities - they don't have a concept of identity.
They encapsulate primitive types in the domain and solve <a href="https://refactoring.guru/smells/primitive-obsession">primitive obsession.</a></p>
<p>There are two main qualities of Value Objects:</p>
<ul>
<li>They are immutable</li>
<li>They have no identity</li>
</ul>
<p>Another quality of value objects is structural equality.
Two value objects are equal if their values are the same.
This quality is the least important in practice.
However, there are cases where you want only some values to determine equality.</p>
<h2>Implementing Value Objects</h2>
<p>The most important quality of value objects is immutability.
The values of a value object can't change once an object is created.
If you want to change an individual value, you need to replace the entire value object.</p>
<p>Here's a <code>Booking</code> entity with primitive values representing an address and the start and end dates of the booking.</p>
<pre><code class="language-csharp">public class Booking
{
    public string Street { get; init; }
    public string City { get; init; }
    public string State { get; init; }
    public string Country { get; init; }
    public string ZipCode { get; init; }

    public DateOnly StartDate { get; init; }
    public DateOnly EndDate { get; init; }
}
</code></pre>
<p>You can replace these primitive values with <code>Address</code> and <code>DateRange</code> value objects.</p>
<pre><code class="language-csharp">public class Booking
{
    public Address Address { get; init; }

    public DateRange Period { get; init; }
}
</code></pre>
<p>But how do you implement value objects?</p>
<h3>C# Records</h3>
<p>You can use C# <a href="records-anonymous-types-non-destructive-mutation">records</a> to represent value objects.
Records are immutable by design, and they have structural equality.
We want both of these qualities for our value objects.</p>
<p>For example, you can represent an <code>Address</code> value object using a <code>record</code> with a <a href="https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/classes-and-structs/instance-constructors#primary-constructors">primary constructor.</a>
The advantage of this approach is conciseness.</p>
<pre><code class="language-csharp">public record Address(
    string Street,
    string City,
    string State,
    string Country,
    string ZipCode);
</code></pre>
<p>However, you lose this advantage when defining a private constructor.
This will happen when you want to enforce invariants while creating the value object.
Another issue with using records is avoiding value object invariants using the <code>with</code> expression.</p>
<pre><code class="language-csharp">public record Address
{
    private Address(
        string street,
        string city,
        string state,
        string country,
        string zipCode)
    {
        Street = street;
        City = city;
        State = state;
        Country = country;
        ZipCode = zipCode;
    }

    public string Street { get; init; }
    public string City { get; init; }
    public string State { get; init; }
    public string Country { get; init; }
    public string ZipCode { get; init; }

    public static Result&lt;Address&gt; Create(
        string street,
        string city,
        string state,
        string country,
        string zipCode)
    {
        // Check if the address is valid

        return new Address(street, city, state, country, zipCode);
    }
}
</code></pre>
<h3>Base Class</h3>
<p>The alternative way to implement value objects is with a <code>ValueObject</code> base class.
The base class handles structural equality with the <code>GetAtomicValues</code> abstract method.
<code>ValueObject</code> implementations have to implement this method and define the equality components.</p>
<p>The advantage of using a <code>ValueObject</code> base class is that it's explicit.
It's clear which classes in your domain represent value objects.
Another advantage is being able to control the equality components.</p>
<p>Here's a <code>ValueObject</code> base class I use in my projects:</p>
<pre><code class="language-csharp">public abstract class ValueObject : IEquatable&lt;ValueObject&gt;
{
    public static bool operator ==(ValueObject? a, ValueObject? b)
    {
        if (a is null &amp;&amp; b is null)
        {
            return true;
        }

        if (a is null || b is null)
        {
            return false;
        }

        return a.Equals(b);
    }

    public static bool operator !=(ValueObject? a, ValueObject? b) =&gt;
        !(a == b);

    public virtual bool Equals(ValueObject? other) =&gt;
        other is not null &amp;&amp; ValuesAreEqual(other);

    public override bool Equals(object? obj) =&gt;
        obj is ValueObject valueObject &amp;&amp; ValuesAreEqual(valueObject);

    public override int GetHashCode() =&gt;
        GetAtomicValues().Aggregate(
            default(int),
            (hashcode, value) =&gt;
                HashCode.Combine(hashcode, value.GetHashCode()));

    protected abstract IEnumerable&lt;object&gt; GetAtomicValues();

    private bool ValuesAreEqual(ValueObject valueObject) =&gt;
        GetAtomicValues().SequenceEqual(valueObject.GetAtomicValues());
}
</code></pre>
<p>The <code>Address</code> value object implementation would look like this:</p>
<pre><code class="language-csharp">public sealed class Address : ValueObject
{
    public string Street { get; init; }
    public string City { get; init; }
    public string State { get; init; }
    public string Country { get; init; }
    public string ZipCode { get; init; }

    protected override IEnumerable&lt;object&gt; GetAtomicValues()
    {
        yield return Street;
        yield return City;
        yield return State;
        yield return Country;
        yield return ZipCode;
    }
}
</code></pre>
<h2>When To Use Value Objects?</h2>
<p>I use value objects to solve primitive obsession and encapsulate domain invariants.
Encapsulation is an important aspect of any domain model.
You shouldn't be able to create a value object in an invalid state.</p>
<p>Value objects also give you type safety.
Take a look at this method signature:</p>
<pre><code class="language-csharp">public interface IPricingService
{
    decimal Calculate(Apartment apartment, DateOnly start, DateOnly end);
}
</code></pre>
<p>Then, compare it to this method signature, where we added value objects.
You can see how the <code>IPricingService</code> with value objects is much more explicit.
You also get the benefit of type safety.
When compiling the code, value objects reduce the chance of errors creeping in.</p>
<pre><code class="language-csharp">public interface IPricingService
{
    PricingDetails Calculate(Apartment apartment, DateRange period);
}
</code></pre>
<p>Here are a few more things you should consider to decide if you need value objects:</p>
<ul>
<li><strong>Complexity of invariants</strong> - If enforcing complex invariants, consider using value objects</li>
<li><strong>Number of primitives</strong> - Value objects make sense when encapsulating many primitive values</li>
<li><strong>Number of duplications</strong> - If you need to enforce invariants only in a few places in the code, you can manage without value objects</li>
</ul>
<h2>Persisting Value Objects With EF Core</h2>
<p>Value objects are part of domain entities, and you need to save them in the database.</p>
<p>I'll show you how to use EF <a href="https://learn.microsoft.com/en-us/ef/core/modeling/owned-entities">Owned Types</a>
and <a href="https://devblogs.microsoft.com/dotnet/announcing-ef8-rc1/#complex-types-as-value-objects">Complex Types</a>
to persist value objects.</p>
<h3>Owned Types</h3>
<p><a href="https://learn.microsoft.com/en-us/ef/core/modeling/owned-entities">Owned Types</a> can be configured by calling the <code>OwnsOne</code> method when configuring the entity.
This tells EF to persist the <code>Address</code> and <code>Price</code> value objects to the same table as the <code>Apartment</code> entity.
The value objects are represented with additional columns in the <code>apartments</code> table.</p>
<pre><code class="language-csharp">public void Configure(EntityTypeBuilder&lt;Apartment&gt; builder)
{
    builder.ToTable(&quot;apartments&quot;);

    builder.OwnsOne(property =&gt; property.Address);

    builder.OwnsOne(property =&gt; property.Price, priceBuilder =&gt;
    {
        priceBuilder.Property(money =&gt; money.Currency)
            .HasConversion(
                currency =&gt; currency.Code,
                code =&gt; Currency.FromCode(code));
    });
}
</code></pre>
<p>A few more remarks about owned types:</p>
<ul>
<li>Owned types have a hidden key value</li>
<li>No support for optional (nullable) owned types</li>
<li>Owned collections are supported with <code>OwnsMany</code></li>
<li>Table splitting allows you to persist owned types separately</li>
</ul>
<h3>Complex Types</h3>
<p><a href="https://devblogs.microsoft.com/dotnet/announcing-ef8-rc1/#complex-types-as-value-objects">Complex Types</a> are a new EF feature available in .NET 8.
They aren't identified or tracked by a key value.
Complex types have to be part of an entity type.</p>
<p>Complex types are more appropriate for representing value objects with EF.</p>
<p>Here's how you can configure an <code>Address</code> value object as a complex type:</p>
<pre><code class="language-csharp">public void Configure(EntityTypeBuilder&lt;Apartment&gt; builder)
{
    builder.ToTable(&quot;apartments&quot;);

    builder.ComplexProperty(property =&gt; property.Address);
}
</code></pre>
<p>A few limitations for complex types:</p>
<ul>
<li>No support for collections</li>
<li>No support for nullable values</li>
</ul>
<h2>Takeaway</h2>
<p>Value objects help design a rich domain model.
You can use them to solve primitive obsession and encapsulate domain invariants.
Value objects can reduce errors by preventing the instantiation of invalid domain objects.</p>
<p>You can use a <code>record</code> or <code>ValueObject</code> base class to represent value objects.
This should depend on your specific requirements and the complexity of your domain.
I use records by default unless I need some qualities of a <code>ValueObject</code> base class.
For example, a base class is practical when you want to control equality components.</p>
<p>More learning material about value objects:</p>
<ul>
<li><a href="https://youtu.be/P5CRea21R2E">Solving primitive obsession with value objects</a></li>
<li><a href="https://youtu.be/LhCD5CUSP6g">Using EF 8 Complex types for value objects</a></li>
</ul>
<p>Hope this was helpful.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_069.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[5 Serilog Best Practices For Better Structured Logging]]></title>
            <link>https://milanjovanovic.tech/blog/5-serilog-best-practices-for-better-structured-logging</link>
            <guid isPermaLink="false">5-serilog-best-practices-for-better-structured-logging</guid>
            <pubDate>Sat, 16 Dec 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Serilog is a structured logging library for .NET. It's also my preferred logging library in the projects I'm developing. I want to share 5 practical tips for better structured logging with Serilog.]]></description>
            <content:encoded><![CDATA[<p><a href="https://serilog.net/">Serilog</a> is a <a href="structured-logging-in-asp-net-core-with-serilog">structured logging</a> library for .NET.</p>
<p>It's also my preferred logging library in the projects I'm developing.</p>
<p>Serilog supports many logging destinations called <a href="https://github.com/serilog/serilog/wiki/Provided-Sinks">Sinks.</a>
The log destinations range from console and file sinks to managed logging services such as <a href="https://github.com/serilog-contrib/serilog-sinks-applicationinsights">Application Insights.</a></p>
<p>Today, I want to share 5 practical tips for better structured logging with Serilog.</p>
<h2>Use The Configuration System</h2>
<p>There are two ways you can configure Serilog in <a href="http://ASP.NET">ASP.NET</a> Core:</p>
<ul>
<li>Fluent API</li>
<li>Configuration system</li>
</ul>
<p>The Fluent API allows you to write code and easily configure Serilog.
The downside is you are hardcoding your configuration.
Any configuration changes require deploying a new version.</p>
<p>I prefer using the <a href="http://ASP.NET">ASP.NET</a> configuration system to set up Serilog.
The benefit is you can change the logging configuration without redeploying your application.</p>
<p>You'll need to install the <code>Serilog.Settings.Configuration</code> library.</p>
<p>This allows you to configure Serilog using the configuration system:</p>
<pre><code class="language-csharp">builder.Host.UseSerilog((context, loggerConfig) =&gt;
    loggerConfig.ReadFrom.Configuration(context.Configuration));
</code></pre>
<p>Here's a Serilog configuration with <a href="https://github.com/serilog/serilog-sinks-console">Console</a> and <a href="https://github.com/datalust/serilog-sinks-seq">Seq</a> sinks.
We also configure a few <a href="https://github.com/serilog/serilog/wiki/Enrichment">Serilog enrichers</a> to enrich application logs with extra information.</p>
<pre><code class="language-json">{
  &quot;Serilog&quot;: {
    &quot;Using&quot;: [&quot;Serilog.Sinks.Console&quot;, &quot;Serilog.Sinks.Seq&quot;],
    &quot;MinimumLevel&quot;: {
      &quot;Default&quot;: &quot;Information&quot;,
      &quot;Override&quot;: {
        &quot;Microsoft&quot;: &quot;Information&quot;
      }
    },
    &quot;WriteTo&quot;: [
      { &quot;Name&quot;: &quot;Console&quot; },
      {
        &quot;Name&quot;: &quot;Seq&quot;,
        &quot;Args&quot;: { &quot;serverUrl&quot;: &quot;http://localhost:5341&quot; }
      }
    ],
    &quot;Enrich&quot;: [&quot;FromLogContext&quot;, &quot;WithMachineName&quot;, &quot;WithThreadId&quot;]
  }
}
</code></pre>
<h2>Use Serilog Request Logging</h2>
<p>You can install the <code>Serilog.AspNetCore</code> library to add Serilog logging for the <a href="http://ASP.NET">ASP.NET</a> Core request pipeline.
It adds <a href="http://ASP.NET">ASP.NET</a>'s internal operations to the same Serilog sinks as your application events.</p>
<p>All you need to do is call the <code>UseSerilogRequestLogging</code> method:</p>
<pre><code class="language-csharp">app.UseSerilogRequestLogging();
</code></pre>
<p>The <code>SourceContext</code> for these structured logs is <code>Serilog.AspNetCore.RequestLoggingMiddleware</code>.</p>
<p>Here's an example structured log produced by this middleware:</p>
<pre><code class="language-json">{
  &quot;@t&quot;: &quot;2023-12-16T00:00:00.0000000Z&quot;,
  &quot;@mt&quot;: &quot;HTTP {RequestMethod} {RequestPath} responded {StatusCode} in {Elapsed:0.0000} ms&quot;,
  &quot;@m&quot;: &quot;HTTP POST /api/users responded 409 in 24.7928 ms&quot;,
  &quot;@i&quot;: &quot;37aa1435&quot;,
  &quot;@r&quot;: [&quot;24.7928&quot;],
  &quot;@tr&quot;: &quot;61a449a8606fdb64e88d6c64b7b7354e&quot;,
  &quot;@sp&quot;: &quot;163ed90674cb12f6&quot;,
  &quot;ConnectionId&quot;: &quot;0HMVSP0L8FVEN&quot;,
  &quot;CorrelationId&quot;: &quot;0HMVSP0L8FVEN:0000000B&quot;,
  &quot;Elapsed&quot;: 24.792778,
  &quot;RequestId&quot;: &quot;0HMVSP0L8FVEN:0000000B&quot;,
  &quot;RequestMethod&quot;: &quot;POST&quot;,
  &quot;RequestPath&quot;: &quot;/api/users&quot;,
  &quot;SourceContext&quot;: &quot;Serilog.AspNetCore.RequestLoggingMiddleware&quot;,
  &quot;StatusCode&quot;: 409
}
</code></pre>
<h2>Enrich Your Logs With CorrelationId</h2>
<p>How can you track all the logs belonging to the same request?</p>
<p>You can add a <code>CorrelationId</code> property to your structured logs.</p>
<p>This also works across multiple applications.
You need to pass the <code>CorrelationId</code> using an HTTP header.
For example, you could use a custom <code>X-Correlation-Id</code> header.</p>
<p>In the <code>RequestContextLoggingMiddleware</code>, I'm adding the <code>CorrelationId</code> to the Serilog <a href="https://github.com/serilog/serilog/wiki/Enrichment#the-logcontext"><code>LogContext</code></a>.
This will make it available to all logs created during this application request.</p>
<pre><code class="language-csharp">public class RequestContextLoggingMiddleware
{
    private const string CorrelationIdHeaderName = &quot;X-Correlation-Id&quot;;
    private readonly RequestDelegate _next;

    public RequestContextLoggingMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public Task Invoke(HttpContext context)
    {
        string correlationId = GetCorrelationId(context);

        using (LogContext.PushProperty(&quot;CorrelationId&quot;, correlationId))
        {
            return _next.Invoke(context);
        }
    }

    private static string GetCorrelationId(HttpContext context)
    {
        context.Request.Headers.TryGetValue(
            CorrelationIdHeaderName, out StringValues correlationId);

        return correlationId.FirstOrDefault() ?? context.TraceIdentifier;
    }
}
</code></pre>
<p>I like to create an extension method for adding the <a href="3-ways-to-create-middleware-in-asp-net-core">middleware.</a>
The <code>UseRequestContextLogging</code> method will add the <code>RequestContextLoggingMiddleware</code> to the request pipeline.
Note that the order of registering middleware is important.
If you want the <code>CorrelationId</code> in all your logs, you want to place this middleware at the start.</p>
<pre><code class="language-csharp">public static IApplicationBuilder UseRequestContextLogging(
    this IApplicationBuilder app)
{
    app.UseMiddleware&lt;RequestContextLoggingMiddleware&gt;();

    return app;
}
</code></pre>
<h2>Log Important Application Events</h2>
<p>In general, I try to log important events in my application.
This includes current request information, errors, failures, unexpected values, branching points, etc.</p>
<p>I'm a proponent of using the <a href="functional-error-handling-in-dotnet-with-the-result-pattern">Result pattern</a> to express application failures.
So, having a custom middleware to log request processing results is important.</p>
<p>Some developers prefer using exceptions to achieve the same functionality.
I disagree with this.
Using exceptions for flow control is a bad practice.
But still, don't forget to add a <a href="global-error-handling-in-aspnetcore-8">global exception handler</a> for unhandled exceptions.</p>
<p>If you're using the <a href="cqrs-pattern-with-mediatr">CQRS pattern with MediatR</a>, you can easily add logging for all application requests.</p>
<p>In the <code>RequestLoggingPipelineBehavior</code> I'm pushing the <code>Error</code> property to the <code>LogContext</code>.
The error object is deconstructed into a JSON value in the structured log.
This lets me filter my logs based on the error details.</p>
<pre><code class="language-csharp">internal sealed class RequestLoggingPipelineBehavior&lt;TRequest, TResponse&gt;
    : IPipelineBehavior&lt;TRequest, TResponse&gt;
    where TRequest : class
    where TResponse : Result
{
    private readonly ILogger _logger;

    public RequestLoggingPipelineBehavior(ILogger logger)
    {
        _logger = logger;
    }

    public async Task&lt;TResponse&gt; Handle(
        TRequest request,
        RequestHandlerDelegate&lt;TResponse&gt; next,
        CancellationToken cancellationToken)
    {
        string requestName = typeof(TRequest).Name;

        _logger.LogInformation(
            &quot;Processing request {RequestName}&quot;, requestName);

        TResponse result = await next();

        if (result.IsSuccess)
        {
            _logger.LogInformation(
                &quot;Completed request {RequestName}&quot;, requestName);
        }
        else
        {
            using (LogContext.PushProperty(&quot;Error&quot;, result.Error, true))
            {
                _logger.LogError(
                    &quot;Completed request {RequestName} with error&quot;, requestName);
            }
        }

        return result;
    }
}
</code></pre>
<h2>Use Seq for Local Development</h2>
<p><a href="https://datalust.co/seq">Seq</a> is a self-hosted search, analysis, and alerting server built for structured log data.
It's free to use for local development.
It offers advanced search and filtering capabilities on the structured log data.</p>
<p>You can spin up a Seq instance in a <a href="https://www.docker.com/">Docker</a> container:</p>
<pre><code class="language-yaml">version: '3.4'

services:
  seq:
    image: datalust/seq:latest
    container_name: seq
    environment:
      - ACCEPT_EULA=Y
    ports:
      - 5341:5341
      - 8081:80
</code></pre>
<p>You can start filtering data when you configure Serilog to write application logs to the Seq instance.</p>
<p><img src="/blogs/mnw_068/seq.png" alt=""></p>
<h2>Summary</h2>
<p>Structured logs follow follow the same structure.
And since structured logs are machine-readable, you can search them for specific information.
Structured logs provide more context and details about application errors.
They make it easier to identify and fix problems.</p>
<p>You can use Serilog's powerful <code>LogContext</code> to enrich your logs with a <code>CorrelationId</code>.
This lets you easily track all logs related to a single application request.</p>
<p>When you have structured logging set up, you'll want to search and analyze your logs.
Seq is an excellent tool for this that you can use for local development.</p>
<p>If you want to get started with Seq, check out my <a href="https://youtu.be/mT8ZkXafuZk">beginner Seq tutorial.</a></p>
<p>Thanks for reading, and stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_068.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Modular Monolith Data Isolation]]></title>
            <link>https://milanjovanovic.tech/blog/modular-monolith-data-isolation</link>
            <guid isPermaLink="false">modular-monolith-data-isolation</guid>
            <pubDate>Sat, 09 Dec 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Modular monoliths are an architectural approach that's becoming very popular. They combine the benefits of modularity and monolithic design. Data isolation ensures that modules are independent and loosely coupled. Today, I will show you four data isolation approaches for modular monoliths]]></description>
            <content:encoded><![CDATA[<p>Modular monoliths are an architectural approach that's becoming very popular.
They combine the benefits of modularity and monolithic design.</p>
<p>Modular monoliths try to solve the shortcomings of monolithic and microservice architectures.</p>
<p>One problem I often see with monolithic architectures is tight coupling between components.</p>
<p>This leads to dependencies between different parts of the system.</p>
<p>Modular monoliths enforce better architectural practices with well-defined module boundaries and <a href="modular-monolith-communication-patterns"><strong>communication patterns.</strong></a></p>
<p>But one aspect you can't overlook is data isolation between modules.</p>
<p>Data isolation ensures that modules are independent and loosely coupled.</p>
<p>Today, I will show you four data isolation approaches for modular monoliths:</p>
<ul>
<li>Separate table</li>
<li>Separate schema</li>
<li>Separate database</li>
<li>Different persistence</li>
</ul>
<h2>Why Is Data Isolation Important?</h2>
<p>Let's first understand why data isolation is important in a modular monolith architecture.</p>
<p>A modular monolith has strict rules for data integrity:</p>
<ul>
<li>Each module can only access its own tables</li>
<li>No sharing of tables or objects between modules</li>
<li>Joins are only allowed between tables of the same module</li>
</ul>
<p>Modules inside a modular monolith should be self-contained.
Each module handles its own data.
Other modules can access that data using the module's public API.</p>
<p>What are the benefits of this design?</p>
<p>Keeping modules isolated from each other promotes modularity and loose coupling.
It makes it easier to introduce new changes to the system.
There are fewer unintended side effects when components are loosely coupled.</p>
<p>If you are using a relational database, you can still maintain referential integrity.
Removing the foreign keys when extracting tables is not a problem.</p>
<p><img src="/blogs/mnw_067/monolith_components.png" alt=""></p>
<h2>Level 1 - Separate Table</h2>
<p>The simplest solution is to have no isolation at the database level.
Tables for all modules live inside one database.
It's not easy to determine which tables belong to which module.</p>
<p>I'm only mentioning this approach for the sake of completeness.</p>
<p>But this approach works fine up to a particular application size.</p>
<p>However, the more tables you have, the harder it becomes to keep them isolated between modules.</p>
<p>You can improve this by adding logical isolation between tables.</p>
<p><img src="/blogs/mnw_067/separate_table.png" alt=""></p>
<h2>Level 2 - Separate Schema</h2>
<p>Grouping related tables in the database is a way to introduce logical isolation.
You can implement this using database schemas.
Each module has a unique schema containing the module's tables.</p>
<p>Now, it becomes easy to distinguish which module contains which tables.</p>
<p>An easy way to implement this is using <a href="using-multiple-ef-core-dbcontext-in-single-application"><strong>multiple EF Core database contexts.</strong></a></p>
<p>You can also introduce rules to prevent querying data from other modules.
For example, you could implement this using <a href="enforcing-software-architecture-with-architecture-tests"><strong>architecture tests.</strong></a></p>
<p>I always start with logical data isolation when building a modular monolith.</p>
<p>But what if this isn't enough?</p>
<p><img src="/blogs/mnw_067/separate_schema.png" alt=""></p>
<h2>Level 3 - Separate Database</h2>
<p>The next data isolation level is moving each module's data into separate databases.
This approach has more constraints than data isolation using schemas.</p>
<p>This is the way to go if you need strict data isolation rules between modules.
But, the downside is more operational complexity.
You have to manage infrastructure for multiple databases.</p>
<p>However, this is an excellent step toward <a href="monolith-to-microservices-how-a-modular-monolith-helps"><strong>extracting modules.</strong></a></p>
<p>First, you move the tables of the module you want to extract into a separate database.
This also forces you to solve any database coupling problems between your modules.
You're ready to extract the module once you move the tables into a separate database.</p>
<p>Can we take the module data isolation further?</p>
<p><img src="/blogs/mnw_067/separate_db.png" alt=""></p>
<h2>Level 4 - Different Persistence</h2>
<p>Who says you have to use the same database type for all modules?</p>
<p>I work with relational (SQL) databases most of the time.
Relational databases are amazing and solve a wide range of problems.
But sometimes, a document or graph database is a much better solution.</p>
<p>The idea here is similar: you're doing data isolation using separate databases.</p>
<p>However, you can introduce a different database type to solve specific problems.
For example, you can use a relational database for one module.
And a graph or column-store database for another module.
You also have to maintain different persistence models in your application.</p>
<p>This could be a worthwhile tradeoff for your use case.
But it takes careful planning.</p>
<p><img src="/blogs/mnw_067/separate_db_type.png" alt=""></p>
<h2>Summary</h2>
<p>Modular monoliths are excellent if you don't need microservices right away.
You develop your application as a monolith with distinct boundaries inside the system.
You still have the flexibility to <a href="monolith-to-microservices-how-a-modular-monolith-helps"><strong>extract modules and move to microservices.</strong></a>
But you have faster development speed with a modular monolith.</p>
<p>Modules have to comply with a few rules.
They can only access their own tables.
They can't share tables with other modules.
And they can't directly query tables of other modules.
These rules help to enforce data isolation between modules.</p>
<p>But you still have to implement data isolation at the database level.</p>
<p>There are four options you can choose from:</p>
<ul>
<li>Separate table</li>
<li>Separate schema</li>
<li>Separate database</li>
<li>Different persistence</li>
</ul>
<p>I always go for logical isolation using schemas.
It's easy to implement and helps me understand my boundaries better.
Depending on the requirements, I can introduce separate databases later.</p>
<p>Hope this was helpful.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_067.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Global Error Handling in ASP.NET Core 8]]></title>
            <link>https://milanjovanovic.tech/blog/global-error-handling-in-aspnetcore-8</link>
            <guid isPermaLink="false">global-error-handling-in-aspnetcore-8</guid>
            <pubDate>Sat, 02 Dec 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Exceptions are for exceptional situations. But they will inevitably happen in your applications, and you need to handle them. You can implement a global exception handling mechanism or handle only specific exceptions. ASP.NET Core gives you a few options on how to implement this. So which one should you choose? Today, I want to show you an old and new way to handle exceptions in ASP.NET Core 8.]]></description>
            <content:encoded><![CDATA[<p>Exceptions are for exceptional situations.
I even wrote about <a href="functional-error-handling-in-dotnet-with-the-result-pattern"><strong>avoiding exceptions entirely.</strong></a></p>
<p>But they will inevitably happen in your applications, and you need to handle them.</p>
<p>You can implement a global exception handling mechanism or handle only specific exceptions.</p>
<p><a href="http://ASP.NET">ASP.NET</a> Core gives you a few options on how to implement this. So which one should you choose?</p>
<p>Today, I want to show you an <em>old</em> and <em>new</em> way to handle exceptions in <a href="http://ASP.NET">ASP.NET</a> Core 8.</p>
<h2>Old Way: Exception Handling Midleware</h2>
<p>The standard to implement exception handling in <a href="http://ASP.NET">ASP.NET</a> Core is using middleware.
Middleware allows you to introduce logic before or after executing HTTP requests.
You can easily extend this to implement exception handling.
Add a <code>try-catch</code> statement in the middleware and return an error HTTP response.</p>
<p>There are <a href="3-ways-to-create-middleware-in-asp-net-core"><strong>3 ways to create middleware</strong></a> in <a href="http://ASP.NET">ASP.NET</a> Core:</p>
<ul>
<li>Using <a href="3-ways-to-create-middleware-in-asp-net-core#adding-middleware-with-request-delegates"><strong>request delegates</strong></a></li>
<li>By <a href="3-ways-to-create-middleware-in-asp-net-core#adding-middleware-by-convention"><strong>convention</strong></a></li>
<li><a href="3-ways-to-create-middleware-in-asp-net-core#adding-factory-based-middleware"><code>IMiddleware</code></a></li>
</ul>
<p>The convention-based approach requires you to define an <code>InvokeAsync</code> method.</p>
<p>Here's an <code>ExceptionHandlingMiddleware</code> defined by convention:</p>
<pre><code class="language-csharp">public class ExceptionHandlingMiddleware
{
    private readonly RequestDelegate _next;
    private readonly ILogger&lt;ExceptionHandlingMiddleware&gt; _logger;

    public ExceptionHandlingMiddleware(
        RequestDelegate next,
        ILogger&lt;ExceptionHandlingMiddleware&gt; logger)
    {
        _next = next;
        _logger = logger;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        try
        {
            await _next(context);
        }
        catch (Exception exception)
        {
            _logger.LogError(
                exception, &quot;Exception occurred: {Message}&quot;, exception.Message);

            var problemDetails = new ProblemDetails
            {
                Status = StatusCodes.Status500InternalServerError,
                Title = &quot;Server Error&quot;
            };

            context.Response.StatusCode =
                StatusCodes.Status500InternalServerError;

            await context.Response.WriteAsJsonAsync(problemDetails);
        }
    }
}
</code></pre>
<p>The <code>ExceptionHandlingMiddleware</code> will catch any unhandled exception and return a <a href="https://www.rfc-editor.org/rfc/rfc7807.html">Problem Details</a> response.
You can decide how much information you want to return to the caller.
In this example, I'm hiding the exception details.</p>
<p>You also need to add this middleware to the <a href="http://ASP.NET">ASP.NET</a> Core request pipeline:</p>
<pre><code class="language-csharp">app.UseMiddleware&lt;ExceptionHandlingMiddleware&gt;();
</code></pre>
<h2>New Way: IExceptionHandler</h2>
<p><a href="https://learn.microsoft.com/en-us/aspnet/core/introduction-to-aspnet-core?view=aspnetcore-8.0">ASP.NET Core 8</a>
introduces a new <a href="https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.diagnostics.iexceptionhandler?view=aspnetcore-8.0"><code>IExceptionHandler</code></a>
abstraction for managing exceptions.
The built-in exception handler middleware uses <code>IExceptionHandler</code> implementations to handle exceptions.</p>
<p>This interface has only one <code>TryHandleAsync</code> method.</p>
<p><code>TryHandleAsync</code> attempts to handle the specified exception within the <a href="http://ASP.NET">ASP.NET</a> Core pipeline.
If the exception can be handled, it should return <code>true</code>.
If the exception can't be handled, it should return <code>false</code>.
This allows you to implement custom exception-handling logic for different scenarios.</p>
<p>Here's a <code>GlobalExceptionHandler</code> implementation:</p>
<pre><code class="language-csharp">internal sealed class GlobalExceptionHandler : IExceptionHandler
{
    private readonly ILogger&lt;GlobalExceptionHandler&gt; _logger;

    public GlobalExceptionHandler(ILogger&lt;GlobalExceptionHandler&gt; logger)
    {
        _logger = logger;
    }

    public async ValueTask&lt;bool&gt; TryHandleAsync(
        HttpContext httpContext,
        Exception exception,
        CancellationToken cancellationToken)
    {
        _logger.LogError(
            exception, &quot;Exception occurred: {Message}&quot;, exception.Message);

        var problemDetails = new ProblemDetails
        {
            Status = StatusCodes.Status500InternalServerError,
            Title = &quot;Server error&quot;
        };

        httpContext.Response.StatusCode = problemDetails.Status.Value;

        await httpContext.Response
            .WriteAsJsonAsync(problemDetails, cancellationToken);

        return true;
    }
}
</code></pre>
<h2>Configuring IExceptionHandler Implementations</h2>
<p>You need two things to add an <code>IExceptionHandler</code> implementation to the <a href="http://ASP.NET">ASP.NET</a> Core request pipeline:</p>
<ol>
<li>Register the <code>IExceptionHandler</code> service with dependency injection</li>
<li>Register the <a href="https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.diagnostics.exceptionhandlermiddleware?view=aspnetcore-8.0"><code>ExceptionHandlerMiddleware</code></a>
with the request pipeline</li>
</ol>
<p>You call the <code>AddExceptionHandler</code> method to register the <code>GlobalExceptionHandler</code> as a service.
It's registered with a <a href="improving-aspnetcore-dependency-injection-with-scrutor"><strong>singleton lifetime</strong></a>.
So be careful about injecting services with a different lifetime.</p>
<p>I'm also calling <code>AddProblemDetails</code> to generate a <a href="https://www.rfc-editor.org/rfc/rfc7807.html">Problem Details</a> response for common exceptions.</p>
<pre><code class="language-csharp">builder.Services.AddExceptionHandler&lt;GlobalExceptionHandler&gt;();
builder.Services.AddProblemDetails();
</code></pre>
<p>You also need to call <code>UseExceptionHandler</code> to add the <code>ExceptionHandlerMiddleware</code> to the request pipeline:</p>
<pre><code class="language-csharp">app.UseExceptionHandler();
</code></pre>
<h2>Chaining Exception Handlers</h2>
<p>You can add multiple <code>IExceptionHandler</code> implementations, and they're called in the order they are registered.
A possible use case for this is using exceptions for flow control.</p>
<p>You can define custom exceptions like <code>BadRequestException</code> and <code>NotFoundException</code>.
They correspond with the HTTP status code you would return from the API.</p>
<p>Here's a <code>BadRequestExceptionHandler</code> implementation:</p>
<pre><code class="language-csharp">internal sealed class BadRequestExceptionHandler : IExceptionHandler
{
    private readonly ILogger&lt;BadRequestExceptionHandler&gt; _logger;

    public BadRequestExceptionHandler(ILogger&lt;BadRequestExceptionHandler&gt; logger)
    {
        _logger = logger;
    }

    public async ValueTask&lt;bool&gt; TryHandleAsync(
        HttpContext httpContext,
        Exception exception,
        CancellationToken cancellationToken)
    {
        if (exception is not BadRequestException badRequestException)
        {
            return false;
        }

        _logger.LogError(
            badRequestException,
            &quot;Exception occurred: {Message}&quot;,
            badRequestException.Message);

        var problemDetails = new ProblemDetails
        {
            Status = StatusCodes.Status400BadRequest,
            Title = &quot;Bad Request&quot;,
            Detail = badRequestException.Message
        };

        httpContext.Response.StatusCode = problemDetails.Status.Value;

        await httpContext.Response
            .WriteAsJsonAsync(problemDetails, cancellationToken);

        return true;
    }
}
</code></pre>
<p>And here's a <code>NotFoundExceptionHandler</code> implementation:</p>
<pre><code class="language-csharp">internal sealed class NotFoundExceptionHandler : IExceptionHandler
{
    private readonly ILogger&lt;NotFoundExceptionHandler&gt; _logger;

    public NotFoundExceptionHandler(ILogger&lt;NotFoundExceptionHandler&gt; logger)
    {
        _logger = logger;
    }

    public async ValueTask&lt;bool&gt; TryHandleAsync(
        HttpContext httpContext,
        Exception exception,
        CancellationToken cancellationToken)
    {
        if (exception is not NotFoundException notFoundException)
        {
            return false;
        }

        _logger.LogError(
            notFoundException,
            &quot;Exception occurred: {Message}&quot;,
            notFoundException.Message);

        var problemDetails = new ProblemDetails
        {
            Status = StatusCodes.Status404NotFound,
            Title = &quot;Not Found&quot;,
            Detail = notFoundException.Message
        };

        httpContext.Response.StatusCode = problemDetails.Status.Value;

        await httpContext.Response
            .WriteAsJsonAsync(problemDetails, cancellationToken);

        return true;
    }
}
</code></pre>
<p>You also need to register both exception handlers by calling <code>AddExceptionHandler</code>:</p>
<pre><code class="language-csharp">builder.Services.AddExceptionHandler&lt;BadRequestExceptionHandler&gt;();
builder.Services.AddExceptionHandler&lt;NotFoundExceptionHandler&gt;();
</code></pre>
<p>The <code>BadRequestExceptionHandler</code> will execute first and try to handle the exception.
If the exception isn't handled, <code>NotFoundExceptionHandler</code> will execute next and attempt to handle the exception.</p>
<h2>Takeaway</h2>
<p>Using middleware for exception handling is an excellent solution in <a href="http://ASP.NET">ASP.NET</a> Core.
However, it's great that we have new options using the <code>IExceptionHandler</code> interface.
I will use the new approach in <a href="http://ASP.NET">ASP.NET</a> Core 8 projects.</p>
<p>I'm very much against using exceptions for flow control.
Exceptions are a last resort when you can't continue normal application execution.
The <a href="functional-error-handling-in-dotnet-with-the-result-pattern"><strong>Result pattern</strong></a> is a better alternative.</p>
<p>Exceptions are also <a href="https://github.com/dotnet/aspnetcore/issues/46280#issuecomment-1527898867">extremely expensive</a>,
as David Fowler noted:</p>
<p><img src="/blogs/mnw_066/fowler_comment.png" alt=""></p>
<p>If you want to get rid of exceptions in your code, <a href="https://youtu.be/WCCkEe_Hy2Y"><strong>check out this video.</strong></a></p>
<p>Thanks for reading, and stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_066.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[5 Awesome C# Refactoring Tips]]></title>
            <link>https://milanjovanovic.tech/blog/5-awesome-csharp-refactoring-tips</link>
            <guid isPermaLink="false">5-awesome-csharp-refactoring-tips</guid>
            <pubDate>Sat, 25 Nov 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Refactoring is a technique for restructuring existing code without changing its behavior. You can think of refactoring as a series of small code transformations. One change (refactoring) does little. But a sequence of refactors produces a significant transformation. There's no better way to learn refactoring than practicing. So I prepared a refactoring exercise for you.]]></description>
            <content:encoded><![CDATA[<p>Refactoring is a technique for restructuring existing code without changing its behavior.
You can think of refactoring as a series of small code transformations.</p>
<p>One change (refactoring) does little.
But a sequence of refactors produces a significant transformation.</p>
<p>There's no better way to learn refactoring than practicing.</p>
<p>So I prepared a refactoring exercise for you.</p>
<p>Today I'm going to refactor some poorly written code.</p>
<p>And I'll show you 5 awesome refactoring techniques along the way:</p>
<ul>
<li>Extract method</li>
<li>Extract interface</li>
<li>Extract class</li>
<li>Functional code</li>
<li>Pushing logic down</li>
</ul>
<h2>Starting Point</h2>
<p>We will refactor the <code>CustomerService</code> below to try to improve the code.</p>
<p>I want to achieve three goals with this refactor.</p>
<p>Or rather, I want to improve three qualities of the <code>CustomerService</code>:</p>
<ul>
<li>Testability</li>
<li>Readability</li>
<li>Maintainability</li>
</ul>
<p>To improve these qualities, we need to figure out what's preventing us from attaining them.</p>
<p>So, let's first understand what the <code>CustomerService</code> is doing on a high level:</p>
<ul>
<li>Validation of the input arguments</li>
<li>Fetching the <code>Company</code> and creating a new <code>Customer</code></li>
<li>Calculating if the <code>Customer</code> has a credit limit and the amount</li>
<li>Saving the <code>Customer</code> to the database if they meet a specific condition</li>
</ul>
<p>You can see quite a few things are happening inside the <code>AddCustomer</code> method.</p>
<p>It's almost 100 lines of code, which reduces readability.</p>
<p>It's difficult to test because we can't control any external dependencies.</p>
<p>It's impossible to extend the behavior of the <code>CustomerService</code> without changing the code.</p>
<p>But we can fix all these problems.
Let me show you how.</p>
<pre><code class="language-csharp">public class CustomerService
{
    public bool AddCustomer(
        string firstName,
        string lastName,
        string email,
        DateTime dateOfBirth,
        int companyId)
    {
        if (string.IsNullOrEmpty(firstName) || string.IsNullOrEmpty(lastName))
        {
            return false;
        }

        if (!email.Contains('@') &amp;&amp; !email.Contains('.'))
        {
            return false;
        }

        var now = DateTime.Now;
        var age = now.Year - dateOfBirth.Year;
        if (dateOfBirth.Date &gt; now.AddYears(-age))
        {
            age -= 1;
        }

        if (age &lt; 21)
        {
            return false;
        }

        var companyRepository = new CompanyRepository();
        var company = companyRepository.GetById(companyId);

        var customer = new Customer
        {
            Company = company,
            DateOfBirth = dateOfBirth,
            EmailAddress = email,
            Firstname = firstName,
            Surname = lastName
        };

        if (company.Type == &quot;VeryImportantClient&quot;)
        {
            // Skip credit check
            customer.HasCreditLimit = false;
        }
        else if (company.Type == &quot;ImportantClient&quot;)
        {
            // Do credit check and double credit limit
            customer.HasCreditLimit = true;
            using var creditService = new CustomerCreditServiceClient();

            var creditLimit = creditService.GetCreditLimit(
                customer.Firstname,
                customer.Surname,
                customer.DateOfBirth);

            creditLimit *= 2;
            customer.CreditLimit = creditLimit;
        }
        else
        {
            // Do credit check
            customer.HasCreditLimit = true;
            using var creditService = new CustomerCreditServiceClient();

            var creditLimit = creditService.GetCreditLimit(
                customer.Firstname,
                customer.Surname,
                customer.DateOfBirth);

            customer.CreditLimit = creditLimit;
        }

        if (customer.HasCreditLimit &amp;&amp; customer.CreditLimit &lt; 500)
        {
            return false;
        }

        var customerRepository = new CustomerRepository();
        customerRepository.AddCustomer(customer);

        return true;
    }
}
</code></pre>
<h2>Refactoring the Validation</h2>
<p>The first part of the code validating input parameters is pretty concise.
It also follows the early return principle.</p>
<p>The validation consists of simple input validation and a calculation for the customer's age.</p>
<p>I would start with an <strong>extract method</strong> refactor to move the validation into one place.</p>
<pre><code class="language-csharp">if (string.IsNullOrEmpty(firstName) || string.IsNullOrEmpty(lastName))
{
    return false;
}

if (!email.Contains('@') &amp;&amp; !email.Contains('.'))
{
    return false;
}

var now = DateTime.Now;
var age = now.Year - dateOfBirth.Year;
if (dateOfBirth.Date &gt; now.AddYears(-age))
{
    age -= 1;
}

if (age &lt; 21)
{
    return false;
}
</code></pre>
<p>Calculating the age isn't part of the validation flow, so I'll extract that into the <code>CalculateAge</code> method.</p>
<pre><code class="language-csharp">int CalculateAge(DateTime dateOfBirth, DateTime now)
{
    var age = now.Year - dateOfBirth.Year;
    if (dateOfBirth.Date &gt; now.AddYears(-age))
    {
        age -= 1;
    }

    return age;
}
</code></pre>
<p>Then, I'll create the <code>IsValid</code> method to encapsulate all the validation rules.
Instead of writing many <code>if-else</code> statements, I can write a single <code>bool</code> expression.</p>
<p>I also introduced a <code>minimumAge</code> constant to improve readability.</p>
<p>You can see how the <code>CalculateAge</code> method helps simplify the validation check.</p>
<pre><code class="language-csharp">bool IsValid(
    string firstName,
    string lastName,
    string email,
    DateTime dateOfBirth)
{
    const int minimumAge = 21;

    return !string.IsNullOrEmpty(firstName) &amp;&amp;
           !string.IsNullOrEmpty(lastName) &amp;&amp;
           (email.Contains('@') || email.Contains('.')) &amp;&amp;
           CalculateAge(dateOfBirth, DateTime.Now) &gt;= minimumAge;
}
</code></pre>
<p>This simplifies the validation code in <code>AddCustomer</code> to:</p>
<pre><code class="language-csharp">if (!IsValid(firstName, lastName, email, dateOfBirth))
{
    return false;
}
</code></pre>
<h2>Refactoring Towards Dependency Injection</h2>
<p>The next problem I want to solve is introducing <a href="improving-aspnetcore-dependency-injection-with-scrutor">dependency injection</a> to the <code>CustomerService</code>.</p>
<p>Dependency injection allows us to achieve <strong>Inversion of Control (IoC).</strong>
We depend only on interfaces at compile time and on implementations at run time.</p>
<p>The dependency injection pattern has a few important benefits.</p>
<p>You don't have to know how to initialize or dispose of external dependencies.</p>
<p>It also improves testability since you now depend on interfaces.
Interfaces can be mocked to make unit testing easier.</p>
<p>So let's update the <code>CustomerService</code> not to initialize the dependencies directly:</p>
<pre><code class="language-csharp">var companyRepository = new CompanyRepository();

using var creditService = new CustomerCreditServiceClient();

var customerRepository = new CustomerRepository();
</code></pre>
<p>Instead, we will inject them as constructor arguments.
You can even use the <a href="https://learn.microsoft.com/en-us/dotnet/csharp/whats-new/tutorials/primary-constructors">C# 12 primary constructor</a> feature.</p>
<pre><code class="language-csharp">public class CustomerService(
    CompanyRepository companyRepository,
    CustomerRepository customerRepository,
    CustomerCreditServiceClient creditService)
{
    // ...
}
</code></pre>
<p>The next step would be introducing interfaces for these dependencies.
This comes down to an <strong>extract interface</strong> refactor.</p>
<h2>Refactoring the Credit Limit Calculation</h2>
<p>The credit limit calculation is the most complicated part of the code.
There are different business rules based on the company type.</p>
<p>I try to notice patterns in the code before refactoring.
So here are a few of my observations.</p>
<p>Multiple <code>if-else</code> statements based on the <code>Type</code> property make me wonder if I'll need to extend this in the future.
Adding a new rule would mean another <code>if-else</code> check.
The <a href="https://refactoring.guru/design-patterns/strategy">strategy pattern</a> could be an alternative, but a <code>switch</code> statement will also work fine.</p>
<p>Another thing that stands out is the <strong>code duplication</strong> in the last two blocks.
This usually means I can do an <strong>extract method</strong> refactoring to reduce code duplication.</p>
<pre><code class="language-csharp">if (company.Type == &quot;VeryImportantClient&quot;)
{
    // Skip credit check
    customer.HasCreditLimit = false;
}
else if (company.Type == &quot;ImportantClient&quot;)
{
    // Do credit check and double credit limit
    customer.HasCreditLimit = true;
    using var creditService = new CustomerCreditServiceClient();

    var creditLimit = creditService.GetCreditLimit(
        customer.Firstname,
        customer.Surname,
        customer.DateOfBirth);

    creditLimit *= 2;
    customer.CreditLimit = creditLimit;
}
else
{
    // Do credit check
    customer.HasCreditLimit = true;
    using var creditService = new CustomerCreditServiceClient();

    var creditLimit = creditService.GetCreditLimit(
        customer.Firstname,
        customer.Surname,
        customer.DateOfBirth);

    customer.CreditLimit = creditLimit;
}
</code></pre>
<p>The first thing I want to do is introduce an <code>enum</code> for the <code>CompanyType</code>.</p>
<p>This is a <a href="8-tips-to-write-clean-code">clean coding principle</a> I often use.
It improves the readability and extensibility of the code.</p>
<pre><code class="language-csharp">public enum CompanyType
{
    Regular = 0,
    ImportantClient = 1,
    VeryImportantClient = 2
}
</code></pre>
<p>The next thing that bothers me is that the credit limit calculation doesn't belong to the <code>CustomerService</code>.
It violates the <a href="https://en.wikipedia.org/wiki/Single-responsibility_principle">single responsibility principle</a>.</p>
<p>So I want to introduce a dedicated <code>CreditLimitCalculator</code> using an <strong>extract class</strong> refactoring.
I replaced the <code>if-else</code> statements with a <a href="https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/operators/switch-expression">switch expression</a> that I can easily extend in the future.</p>
<pre><code class="language-csharp">public class CreditLimitCalculator(
    CustomerCreditServiceClient customerCreditServiceClient)
{
    public (bool HasCreditLimit, decimal? CreditLimit) Calculate(
        Customer customer,
        Company company)
    {
        return company.Type switch
        {
            CompanyType.VeryImportantClient =&gt; (false, null),
            CompanyType.ImportantClient =&gt; (true, GetCreditLimit(customer) * 2),
            _ =&gt; (true, GetCreditLimit(customer))
        };
    }

    private decimal GetCreditLimit(Customer customer)
    {
        return customerCreditServiceClient.GetCreditLimit(
            customer.FirstName,
            customer.LastName,
            customer.DateOfBirth);
    }
}
</code></pre>
<h2>Reviewing the Refactoring (So Far)</h2>
<p>Let's pause momentarily and review the refactored version of the <code>CustomerService</code>.
I'm confident you will find it more readable and easier to understand.
We can easily test this class and verify that the behavior is correct.</p>
<p>I would usually stop the refactoring at this point, since I'm happy with the results.</p>
<p>But can we take this further?</p>
<pre><code class="language-csharp">public class CustomerService(
    CompanyRepository companyRepository,
    CustomerRepository customerRepository,
    CreditLimitCalculator creditLimitCalculator)
{
    public bool AddCustomer(
        string firstName,
        string lastName,
        string email,
        DateTime dateOfBirth,
        int companyId)
    {
        if (!IsValid(firstName, lastName, email, dateOfBirth))
        {
            return false;
        }

        var company = companyRepository.GetById(companyId);

        var customer = new Customer
        {
            Company = company,
            DateOfBirth = dateOfBirth,
            EmailAddress = email,
            FirstName = firstName,
            LastName = lastName
        };

        (customer.HasCreditLimit, customer.CreditLimit) =
            creditLimitCalculator.Calculate(customer, company);

        if (customer is { HasCreditLimit: true, CreditLimit: &lt; 500 })
        {
            return false;
        }

        customerRepository.AddCustomer(customer);

        return true;
    }
}
</code></pre>
<h2>Taking It Further - Pushing Logic Down</h2>
<p>This part is optional, but I want to show you how to simplify the <code>CustomerService</code> by pushing logic into the domain.</p>
<p>What if we moved the responsibility of creating a <code>Customer</code> into the class?</p>
<p>I often use the <strong>static factory</strong> pattern to implement this.
The caveat is I have to take a dependency on <code>CreditLimitCalculator</code>.
I'm trading off domain model purity to get business rules completeness.</p>
<p>I also added the <code>IsUnderCreditLimit</code> method to wrap the credit limit check.</p>
<pre><code class="language-csharp">public class Customer
{
    // Properties omited

    public static Customer Create(
        Company company,
        string firstName,
        string lastName,
        string email,
        DateTime dateOfBirth,
        CreditLimitCalculator creditLimitCalculator)
    {
        var customer = new Customer
        {
            Company = company,
            DateOfBirth = dateOfBirth,
            EmailAddress = email,
            FirstName = firstName,
            LastName = lastName
        };

        (customer.HasCreditLimit, customer.CreditLimit) =
            creditLimitCalculator.Calculate(customer, company);

        return customer;
    }

    public bool IsUnderCreditLimit() =&gt; HasCreditLimit &amp;&amp; CreditLimit &lt; 500;
}
</code></pre>
<p>This is what the <code>CustomerService</code> looks like now:</p>
<pre><code class="language-csharp">public class CustomerService(
    CompanyRepository companyRepository,
    CustomerRepository customerRepository,
    CreditLimitCalculator creditLimitCalculator)
{
    public bool AddCustomer(
        string firstName,
        string lastName,
        string email,
        DateTime dateOfBirth,
        int companyId)
    {
        if (!IsValid(firstName, lastName, email, dateOfBirth))
        {
            return false;
        }

        var company = companyRepository.GetById(companyId);

        var customer = Customer.Create(
            company,
            firstName,
            lastName,
            email,
            dateOfBirth,
            creditLimitCalculator);

        if (customer.IsUnderCreditLimit())
        {
            return false;
        }

        customerRepository.AddCustomer(customer);

        return true;
    }
}
</code></pre>
<p>What do you think about this implementation?</p>
<h2>Next Steps</h2>
<p>First of all, congrats on making it to the end.
This was a much longer newsletter issue than usual.
What do you think of this format?</p>
<p>Writing unit tests before starting the refactoring would be a great idea.
Unit tests will help detect any changes in behavior.</p>
<p>Remember, refactoring is transforming the existing code without changing the behavior.</p>
<p>Here are a few ideas on how you could further refactor the code:</p>
<ul>
<li><a href="https://refactoring.guru/design-patterns/strategy"><strong>Strategy pattern</strong></a> for the credit limit calculation</li>
<li><a href="functional-error-handling-in-dotnet-with-the-result-pattern"><strong>Result object</strong></a> to represent the method status</li>
</ul>
<p>If you want to try this refactoring exercise, you can find the complete <a href="https://github.com/m-jovanovic/refactoring-katas"><strong>source code here.</strong></a></p>
<p>Hope this was helpful.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_065.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How To Use EF Core Interceptors]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-use-ef-core-interceptors</link>
            <guid isPermaLink="false">how-to-use-ef-core-interceptors</guid>
            <pubDate>Sat, 18 Nov 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[EF Core is my favorite ORM for .NET applications. Yet, its many fantastic features sometimes go unnoticed. For example, query splitting, query filters, and interceptors.
EF interceptors are interesting because you can do powerful things with them. For example, you can hook into materialization, handle optimistic concurrency errors, or add query hints.
The most practical use case is adding behavior when saving changes to the database.]]></description>
            <content:encoded><![CDATA[<p>EF Core is my favorite ORM for .NET applications.
Yet, its many fantastic features sometimes go unnoticed.
For example, <a href="how-to-improve-performance-with-ef-core-query-splitting"><strong>query splitting</strong></a>,
<a href="how-to-use-global-query-filters-in-ef-core"><strong>query filters</strong></a>,
and interceptors.</p>
<p>EF interceptors are interesting because you can do powerful things with them.
For example, you can hook into materialization, handle optimistic concurrency errors, or add query hints.</p>
<p>The most practical use case is adding behavior when saving changes to the database.</p>
<p>Today I want to show you three unique use cases for EF Core interceptors:</p>
<ul>
<li>Audit logging</li>
<li>Publishing domain events</li>
<li>Persisting Outbox messages</li>
</ul>
<h2>What are EF Interceptors?</h2>
<p><a href="https://learn.microsoft.com/en-us/ef/core/logging-events-diagnostics/interceptors">EF Core interceptors</a>
allow you to intercept, change, or suppress EF Core operations.
Every interceptor implements the <code>IInterceptor</code> interface.
A few common derived interfaces include <code>IDbCommandInterceptor</code>, <code>IDbConnectionInterceptor</code>, and <code>IDbTransactionInterceptor</code>.</p>
<p>The most popular one is the <code>ISaveChangesInterceptor</code>. It allows you to add behavior before or after saving changes.</p>
<p>Interceptors are registered for each <code>DbContext</code> instance when configuring the context.</p>
<pre><code class="language-csharp">public interface IInterceptor
{
}
</code></pre>
<p>You don't have to implement these interfaces directly.
It's better to use concrete implementations and override the needed methods.</p>
<p>For example, I'll show you how to use the <code>SaveChangesInterceptor</code>.</p>
<h2>Audit Logging With EF Interceptors</h2>
<p>An audit log of entity changes is a valuable feature in some applications.
You write additional audit information every time an entity is created or modified.
The audit log could also contain the complete before/after values, depending on your requirements.</p>
<p>However, let's use a simple example to make it easy to understand.</p>
<p>I have an <code>IAuditable</code> interface with two properties representing when an entity was created or modified.</p>
<pre><code class="language-csharp">public interface IAuditable
{
    DateTime CreatedOnUtc { get; }

    DateTime? ModifiedOnUtc { get; }
}
</code></pre>
<p>Next, I'll implement an <code>UpdateAuditableInterceptor</code> interceptor to write the audit values.
It uses the <code>ChangeTracker</code> to find all <code>IAuditable</code> instances and sets the respective property value.</p>
<p>I want to highlight that I'm overriding the <code>SavingChangesAsync</code> method here.
<code>SavingChangesAsync</code> runs before the changes are saved in the database and any updates applied inside the <code>UpdateAuditableInterceptor</code>
are also part of the current database transaction.</p>
<p>This implementation can be easily extended to include the information about the current user.</p>
<pre><code class="language-csharp">internal sealed class UpdateAuditableInterceptor : SaveChangesInterceptor
{
    public override ValueTask&lt;InterceptionResult&lt;int&gt;&gt; SavingChangesAsync(
        DbContextEventData eventData,
        InterceptionResult&lt;int&gt; result,
        CancellationToken cancellationToken = default)
    {
        if (eventData.Context is not null)
        {
            UpdateAuditableEntities(eventData.Context);
        }

        return base.SavingChangesAsync(eventData, result, cancellationToken);
    }

    private static void UpdateAuditableEntities(DbContext context)
    {
        DateTime utcNow = DateTime.UtcNow;
        var entities = context.ChangeTracker.Entries&lt;IAuditable&gt;().ToList();

        foreach (EntityEntry&lt;IAuditable&gt; entry in entities)
        {
            if (entry.State == EntityState.Added)
            {
                SetCurrentPropertyValue(
                    entry, nameof(IAuditable.CreatedOnUtc), utcNow);
            }

            if (entry.State == EntityState.Modified)
            {
                SetCurrentPropertyValue(
                    entry, nameof(IAuditable.ModifiedOnUtc), utcNow);
            }
        }

        static void SetCurrentPropertyValue(
            EntityEntry entry,
            string propertyName,
            DateTime utcNow) =&gt;
            entry.Property(propertyName).CurrentValue = utcNow;
    }
}
</code></pre>
<h2>Publish Domain Events With EF Interceptors</h2>
<p>Another use case for EF interceptors is <a href="how-to-use-domain-events-to-build-loosely-coupled-systems"><strong>publishing domain events.</strong></a>
Domain events are a DDD tactical pattern to create loosely coupled systems.</p>
<p>Domain events allow you to express side effects explicitly and provide a better separation of concerns in the domain.</p>
<p>You can create an <code>IDomainEvent</code> interface, which derives from <code>MediatR.INotification</code>.
This allows you to use the <code>IPublisher</code> to publish domain events and handle them asynchronously.</p>
<pre><code class="language-csharp">using MediatR;

public interface IDomainEvent : INotification
{
}
</code></pre>
<p>Then, I'll create a <code>PublishDomainEventsInterceptor</code> that also inherits from <code>SaveChangesInterceptor</code>.
However, this time, we're using the <code>SavedChangesAsync</code> to publish the domain events <em>after</em> saving changes in the database.</p>
<p>This has two important implications:</p>
<ol>
<li>The entire workflow is now eventually consistent. Domain event handlers will save changes to the database after the original transaction.</li>
<li>If any domain event handlers fail, we risk failing the request even though the initial transaction was completed successfully.</li>
</ol>
<p>You can make this process more reliable by using an <a href="outbox-pattern-for-reliable-microservices-messaging"><strong>Outbox.</strong></a></p>
<pre><code class="language-csharp">internal sealed class PublishDomainEventsInterceptor : SaveChangesInterceptor
{
    private readonly IPublisher _publisher;

    public PublishDomainEventsInterceptor(IPublisher publisher)
    {
        _publisher = publisher;
    }

    public override async ValueTask&lt;int&gt; SavedChangesAsync(
        SaveChangesCompletedEventData eventData,
        int result,
        CancellationToken cancellationToken = default)
    {
        if (eventData.Context is not null)
        {
            await PublishDomainEventsAsync(eventData.Context);
        }

        return result;
    }

    private async Task PublishDomainEventsAsync(DbContext context)
    {
        var domainEvents = context
            .ChangeTracker
            .Entries&lt;Entity&gt;()
            .Select(entry =&gt; entry.Entity)
            .SelectMany(entity =&gt;
            {
                List&lt;IDomainEvent&gt; domainEvents = entity.DomainEvents;

                entity.ClearDomainEvents();

                return domainEvents;
            })
            .ToList();

        foreach (IDomainEvent domainEvent in domainEvents)
        {
            await _publisher.Publish(domainEvent);
        }
    }
}
</code></pre>
<h2>Store Outbox Messages With EF Interceptors</h2>
<p>Instead of <a href="how-to-use-domain-events-to-build-loosely-coupled-systems"><strong>publishing domain events</strong></a> as part of the EF transaction, you can convert them to Outbox messages.</p>
<p>Here's an <code>InsertOutboxMessagesInterceptor</code> that does precisely this.</p>
<p>It overrides the <code>SavingChangesAsync</code> method.
Which means it runs inside the current EF transaction before saving changes.</p>
<p>The <code>InsertOutboxMessagesInterceptor</code> converts any domain events into an <code>OutboxMessage</code> and adds it to the respective <code>DbSet&lt;OutboxMessage&gt;</code>.
This means they will be saved to the database with any existing changes inside the same transaction.</p>
<p>This is an atomic operation.</p>
<p>Either everything succeeds or everything fails.</p>
<p>There's no in-between state like in the <code>PublishDomainEventsInterceptor</code>.</p>
<p>You can then create a background worker that will process the Outbox messages.</p>
<p>And this is how you implement the <a href="outbox-pattern-for-reliable-microservices-messaging"><strong>Outbox pattern</strong></a> with EF Core.</p>
<pre><code class="language-csharp">using Newtonsoft.Json;

public sealed class InsertOutboxMessagesInterceptor : SaveChangesInterceptor
{
    private static readonly JsonSerializerSettings Serializer = new()
    {
        TypeNameHandling = TypeNameHandling.All
    };

    public override ValueTask&lt;InterceptionResult&lt;int&gt;&gt; SavingChangesAsync(
        DbContextEventData eventData,
        InterceptionResult&lt;int&gt; result,
        CancellationToken cancellationToken = default)
    {
        if (eventData.Context is not null)
        {
            InsertOutboxMessages(eventData.Context);
        }

        return base.SavingChangesAsync(eventData, result, cancellationToken);
    }

    private static void InsertOutboxMessages(DbContext context)
    {
        context
            .ChangeTracker
            .Entries&lt;Entity&gt;()
            .Select(entry =&gt; entry.Entity)
            .SelectMany(entity =&gt;
            {
                List&lt;IDomainEvent&gt; domainEvents = entity.DomainEvents;

                entity.ClearDomainEvents();

                return domainEvents;
            })
            .Select(domainEvent =&gt; new OutboxMessage
            {
                Id = domainEvent.Id,
                OccurredOnUtc = domainEvent.OccurredOnUtc,
                Type = domainEvent.GetType().Name,
                Content = JsonConvert.SerializeObject(domainEvent, Serializer)
            })
            .ToList();

        context.Set&lt;OutboxMessage&gt;().AddRange(outboxMessages);
    }
}
</code></pre>
<h2>Configuring EF Interceptors Using Dependency Injection</h2>
<p>EF interceptors should be lightweight and stateless.
You can add them to the <code>DbContext</code> by calling <code>AddInterceptors</code> and passing in the interceptor instances.</p>
<p>I like to configure the interceptors with Dependency Injection for two reasons:</p>
<ul>
<li>It allows me also to use DI in the interceptors (be mindful that they are singletons)</li>
<li>To simplify adding the interceptors to the <code>DbContext</code> using <code>AddDbContext</code></li>
</ul>
<p>Here's how you can configure the <code>UpdateAuditableInterceptor</code> and <code>InsertOutboxMessagesInterceptor</code> with the <code>ApplicationDbContext</code>:</p>
<pre><code class="language-csharp">services.AddSingleton&lt;UpdateAuditableInterceptor&gt;();
services.AddSingleton&lt;InsertOutboxMessagesInterceptor&gt;();

services.AddDbContext&lt;IApplicationDbContext, ApplicationDbContext&gt;(
    (sp, options) =&gt; options
        .UseSqlServer(connectionString)
        .AddInterceptors(
            sp.GetRequiredService&lt;UpdateAuditableInterceptor&gt;(),
            sp.GetRequiredService&lt;InsertOutboxMessagesInterceptor&gt;()));
</code></pre>
<h2>Closing Thoughts</h2>
<p>Interceptors allow you to do almost anything with an EF Core operation.
But with great power comes great responsibility.
You should be mindful that interceptors have an impact on performance.
Calls to external services or handling events will slow down the operation.</p>
<p>Remember, you don't necessarily have to use EF interceptors.
You can achieve the same behavior by overriding the <code>SaveChangesAsync</code> method on the <code>DbContext</code> and adding your custom logic there.</p>
<p>I showed you a few practical use cases for EF interceptors in this week's issue.</p>
<p>But, if you want to see more examples, I have a few videos about:</p>
<ul>
<li><a href="https://youtu.be/mAlO3OuoQvo"><strong>Auditing logging</strong></a></li>
<li><a href="https://youtu.be/AHzWJ_SMqLo"><strong>Publishing domain events</strong></a></li>
<li><a href="https://youtu.be/XALvnX7MPeo"><strong>Implementing the Outbox pattern</strong></a></li>
</ul>
<p>Thanks for reading, and stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_064.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How To Easily Create PDF Documents in ASP.NET Core]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-easily-create-pdf-documents-in-aspnetcore</link>
            <guid isPermaLink="false">how-to-easily-create-pdf-documents-in-aspnetcore</guid>
            <pubDate>Sat, 11 Nov 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Reporting is essential for business applications like e-commerce, shipping, fintech, etc. One of the most popular document formats for reporting purposes is PDF. Today I want to show you a few interesting ways to generate PDF files in .NET.]]></description>
            <content:encoded><![CDATA[<p>Reporting is essential for business applications like e-commerce, shipping, fintech, etc.</p>
<p>One of the most popular document formats for reporting purposes is <a href="https://en.wikipedia.org/wiki/PDF">PDF.</a></p>
<p>PDF stands for Portable Document Format.
It's a file format to present documents (including text formatting and images) independently of application software, hardware, and operating systems.</p>
<p>Some common problems .NET developers will face when working with PDF files:</p>
<ul>
<li>Creating dynamic PDF documents</li>
<li>Designing a consistent page layout</li>
<li>Customizing fonts on printed documents</li>
</ul>
<p>Today I want to show you a few interesting ways to generate PDF files in .NET.</p>
<h2>Creating PDF Files With QuestPDF</h2>
<p><a href="https://www.questpdf.com/">QuestPDF</a> is an open-source .NET library for generating PDF documents.
It exposes a fluent API you can use to compose together many simple elements to create complex documents.
Unlike other libraries, it does not rely on HTML-to-PDF conversion.</p>
<p>Let's install the QuestPDF NuGet package:</p>
<pre><code class="language-powershell">Install-Package QuestPDF
</code></pre>
<p>Here's how you can generate a simplified invoice with QuestPDF:</p>
<pre><code class="language-csharp">using QuestPDF.Fluent;
using QuestPDF.Helpers;
using QuestPDF.Infrastructure;

Document.Create(container =&gt;
{
    container.Page(page =&gt;
    {
        page.Margin(50);
        page.Size(PageSizes.A4);
        page.PageColor(Colors.White);
        page.DefaultTextStyle(x =&gt; x.FontSize(16));

        page.Header()
            .AlignCenter()
            .Text(&quot;Invoice #: 2023-77&quot;)
            .SemiBold().FontSize(24).FontColor(Colors.Grey.Darken4);

        page.Content()
            .Table(table =&gt;
            {
                table.ColumnsDefinition(columns =&gt;
                {
                    columns.ConstantColumn(20);
                    columns.RelativeColumn();
                    columns.RelativeColumn();
                });

                table.Header(header =&gt;
                {
                    header.Cell().Text(&quot;#&quot;);
                    header.Cell().Text(&quot;Product&quot;);
                    header.Cell().AlignRight().Text(&quot;Price&quot;);
                });

                foreach (var lineItem in lineItems)
                {
                    table.Cell().Text(lineItem.Index.ToString());
                    table.Cell().Text(lineItem.Name);
                    table.Cell().Text($&quot;${lineItem.Price}&quot;);
                }
            });
    });
})
.GeneratePdf(&quot;invoice.pdf&quot;);;
</code></pre>
<p>What I like about QuestPDF:</p>
<ul>
<li>Fluent API</li>
<li>Easy to use</li>
<li>Good <a href="https://www.questpdf.com/introduction.html">documentation</a></li>
</ul>
<p>What I don't like about QuestPDF:</p>
<ul>
<li>Having to write a lot of code to create documents</li>
<li>Limited scope of features</li>
<li>No HTML-to-PDF support</li>
</ul>
<p><strong>Licensing</strong></p>
<p>QuestPDF is free for small companies and development use.
There's also a commercial license for larger companies.
You can check out the licensing details <a href="https://www.questpdf.com/license/">here.</a></p>
<h2>HTML to PDF Conversion With IronPDF</h2>
<p>The more common approach for generating PDF files is using an HTML template.</p>
<p>My favorite library that supports this is <a href="https://ironpdf.com/">IronPDF.</a></p>
<p>IronPDF is a C# PDF library that allows for fast and efficient manipulation of PDF files.
It also has many valuable features, like <a href="https://ironpdf.com/how-to/pdfa/">exporting to PDF/A format</a>
and <a href="https://ironpdf.com/how-to/signing/">digitally signing PDF documents.</a></p>
<p>But what's the idea behind using an HTML template?</p>
<p>First of all, you have more control over formatting the document.
You can use CSS to style the HTML markup, which will be applied when exporting to a PDF document.</p>
<p>An interesting implementation approach is using <a href="https://learn.microsoft.com/en-us/aspnet/core/mvc/views/overview?view=aspnetcore-7.0">ASP.NET Core MVC views</a>
and the <a href="https://learn.microsoft.com/en-us/aspnet/core/mvc/views/razor?view=aspnetcore-7.0">Razor syntax.</a>
You can pass an object to the view at runtime to render dynamic HTML content.</p>
<p>I've used this approach with MVC views on a few projects with excellent results.</p>
<p>Let's start by installing the IronPDF NuGet package:</p>
<pre><code class="language-powershell">Install-Package IronPdf
</code></pre>
<p>I'm using a strongly typed Razor view to define my markup.
The <code>InoviceViewModel</code> class is the model, and it's used to create dynamic content.</p>
<pre><code class="language-tsx">@model ViewModels.InoviceViewModel

&lt;div&gt;Invoice number: @Model.InvoiceNumber&lt;/div&gt;
&lt;div&gt;Invoice date: @Model.InvoiceDate&lt;/div&gt;
&lt;br/&gt;
&lt;span&gt;Line items:&lt;/span&gt;
&lt;ul&gt;
    @foreach(var lineItem in Model.LineItems)
    {
        &lt;li&gt;@lineItem.Name | @lineItem.Price&lt;/li&gt;
    }
&lt;/ul&gt;
</code></pre>
<p>Now, you need to use the IronPDF <code>ChromePdfRenderer</code> to convert the HTML to a PDF document.</p>
<pre><code class="language-csharp">var html = ConvertRazorViewToHtml(invoice);

var renderer = new ChromePdfRenderer();

var pdf = renderer.RenderHtmlAsPdf(html);

pdf.SaveAs($&quot;invoice-{invoice.InvoiceNumber}.pdf&quot;);
</code></pre>
<p>It really is that simple.</p>
<p><strong>Licensing</strong></p>
<p>IronPDF is free for development use and has multiple pricing tiers for commercial use that you can check out <a href="https://ironpdf.com/licensing/">here.</a></p>
<h2>Merging Multiple PDF Files</h2>
<p>Another common requirement I've seen is merging multiple PDF files.
For example, you could implement a feature to merge the monthly receipts for the accounting department.</p>
<p>You can use the <code>PdfDocument.Merge</code> method to implement this.
It accepts a <code>PdfDocument</code> collection as the argument.
You'll first have to load the PDF documents into memory before merging them.</p>
<p>Here's an example:</p>
<pre><code class="language-csharp">var pdfs = new List&lt;PdfDocument&gt;();

pdfs.Add(PdfDocument.FromFile(&quot;google-invoice.pdf&quot;));
pdfs.Add(PdfDocument.FromFile(&quot;google-ads-invoice.pdf&quot;));
pdfs.Add(PdfDocument.FromFile(&quot;converkit-invoice.pdf&quot;));

PdfDocument mergedPdfDocument = PdfDocument.Merge(pdfs);

mergedPdfDocument.SaveAs(&quot;merged-invoices.pdf&quot;);
</code></pre>
<h2>Exporting PDF Files From an API</h2>
<p>It's pretty straightforward to return a PDF file from an API endpoint in <a href="http://ASP.NET">ASP.NET</a> Core.</p>
<p>Minimal APIs have the <code>Results.File</code> method accepting either a file path, stream, or byte array.
You also need to specify the content type and an optional file name.
The <a href="https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types">MIME type</a>
for PDF files is <code>application/pdf</code>.</p>
<p>Here's how you can return a PDF file from a byte array:</p>
<pre><code class="language-csharp">app.MapGet(&quot;newsletter/download&quot;, () =&gt;
{
    var renderer = new ChromePdfRenderer();

    var pdf = renderer.RenderHtmlAsPdf(&quot;&lt;h1&gt;The .NET Weekly&lt;/h1&gt;&quot;);

    return Results.File(pdf.BinaryData, &quot;application/pdf&quot;, &quot;newsletter.pdf&quot;);
});
</code></pre>
<h2>Takeaway</h2>
<p>Choosing which PDF library you will use in .NET is an important consideration to make.
And while pricing is a significant factor, the features you want to use will also dictate your choice.</p>
<p>QuestPDF is an excellent choice if you're looking for a (mostly) free option with rich features.
The library is constantly improved, and new features are being added.
However, it doesn't support HTML-to-PDF conversion and modifying existing documents.</p>
<p>IronPDF is the library I've used most often on commercial projects.
It has fantastic features for working with PDF files, with many customization options.
The HTML-to-PDF conversion works like a charm.</p>
<p>The hardest part is picking the right tool for the job.</p>
<p>So I hope this is helpful.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_063.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Vertical Slice Architecture]]></title>
            <link>https://milanjovanovic.tech/blog/vertical-slice-architecture</link>
            <guid isPermaLink="false">vertical-slice-architecture</guid>
            <pubDate>Sat, 04 Nov 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Layered architectures are the foundation of many software systems. However, layered architectures organize the system around technical layers. And the cohesion between layers is low.
What if you wanted to organize the system around features instead? This is where Vertical Slice Architecture comes in.]]></description>
            <content:encoded><![CDATA[<p>Layered architectures are the foundation of many software systems.
However, layered architectures organize the system around technical layers.
And the cohesion between layers is low.</p>
<p>What if you wanted to organize the system around features instead?</p>
<p>Minimize coupling between unrelated features and maximize coupling in a single feature.</p>
<p>Today I want to talk about <strong>Vertical Slice Architecture</strong>, which does precisely that.</p>
<h2>The Problem With Layered Architectures</h2>
<p>Layered architectures organize the software system into layers or tiers.
Each of the layers is typically one project in your solution.
Some of the popular implementations are N-tier architecture or Clean architecture.</p>
<p>Layered architectures focus on separating the concerns of the various components.
This makes it easier to understand and maintain the software system.
And there are many benefits of <a href="clean-architecture-and-the-benefits-of-structured-software-design"><strong>structured software design,</strong></a>
such as maintainability, flexibility, and loose coupling.</p>
<p><img src="/blogs/mnw_062/clean_architecture.png" alt=""></p>
<p>However, layered architectures also impose constraints or rigid rules on your system.
The direction of dependencies between layers is pre-determined.</p>
<p>For example, in Clean Architecture:</p>
<ul>
<li>Domain should have no dependencies</li>
<li>Application layer can reference the Domain</li>
<li>Infrastructure can reference both Application and Domain</li>
<li>Presentation can reference both Application and Domain</li>
</ul>
<p>You end up having high coupling inside a layer and low coupling between layers.
This doesn't mean layered architectures are bad.
But it does mean you will have many abstractions between individual layers.
And more abstractions mean increased complexity because there are more components to maintain.</p>
<h2>What is Vertical Slice Architecture?</h2>
<p>I first heard about <a href="https://www.jimmybogard.com/vertical-slice-architecture">Vertical Slice Architecture</a> from Jimmy Bogard.
He's also the creator of some popular open-source libraries like <a href="https://github.com/jbogard/MediatR">MediatR</a> and <a href="https://github.com/AutoMapper/AutoMapper">Automapper.</a></p>
<p>Vertical Slice Architecture was born from the pain of working with layered architectures.
They force you to make changes in many different layers to implement a feature.</p>
<p>Let's imagine what adding a new feature looks like in a layered architecture:</p>
<ul>
<li>Updating the domain model</li>
<li>Modifying validation logic</li>
<li>Creating a use case with MediatR</li>
<li>Exposing an API endpoint from a controller</li>
</ul>
<p>The cohesion is low because you are creating many files in different layers.</p>
<p>Vertical slices take a different approach:</p>
<blockquote>
<p>Minimize coupling between slices, and maximize coupling in a slice.</p>
</blockquote>
<p>Here's how you can visualize vertical slices:</p>
<p><img src="/blogs/mnw_062/vertical_slice_architecture.png" alt=""></p>
<p>All the files for a single use case are grouped inside one folder.
So, the cohesion for a single use case is very high.
This simplifies the development experience.
It's easy to find all the relevant components for each feature since they are close together.</p>
<h2>Implementing Vertical Slices</h2>
<p>If you're building an API, the system already breaks down into commands (POST/PUT/DELETE) and queries (GET).
By splitting the requests into commands and queries, you're getting the benefits of the <a href="cqrs-pattern-with-mediatr"><strong>CQRS pattern.</strong></a></p>
<p>Vertical slices narrowly focus on a single feature.
This allows you to treat each use case separately and tailor the implementation to the specific requirements.
One vertical slice can use EF Core to implement a GET request.
Another vertical slice can use Dapper with raw SQL queries.</p>
<p><img src="/blogs/mnw_062/vertical_slices.png" alt=""></p>
<p>Another benefit of implementing vertical slices like this is:</p>
<blockquote>
<p>New features only add code, you're not changing shared code and worrying about side effects.</p>
</blockquote>
<p>However, vertical slices have their own set of challenges.
Because you are implementing much of the business logic inside a single use case, you need to be able to spot code smells.
As the use case grows, it can end up doing too much.
You will have to refactor the code by <a href="refactoring-from-an-anemic-domain-model-to-a-rich-domain-model"><strong>pushing logic to the domain.</strong></a></p>
<h2>Solution Structure With REPR Pattern</h2>
<p>Layered architectures, such as Clean architecture, organize the solution across layers.
This results in a <a href="clean-architecture-folder-structure"><strong>folder structure grouped by technical concerns.</strong></a></p>
<p>Vertical slice architecture, on the other hand, organizes the code around features or use cases.</p>
<p>An interesting approach to structuring APIs around features is using the <a href="https://deviq.com/design-patterns/repr-design-pattern">REPR pattern.</a>
It stands for Request-EndPoint-Response.
This aligns perfectly with the idea of vertical slices.
You can achieve this with the MediatR library, for example.</p>
<p>The REPR pattern defines that web API endpoints should have three components:</p>
<ul>
<li>Request</li>
<li>Endpoint</li>
<li>Response</li>
</ul>
<p>Here's an example solution structure in .NET.
You'll notice the <code>Features</code> folder, which contains the vertical slices.
Each vertical slice implements one API request (or use case).</p>
<pre><code>🔗 RunTracker.API
|__ 📁 Database
|__ 📁 Entities
    |__ #️⃣ Activity.cs
    |__ #️⃣ Workout.cs
    |__ #️⃣ ...
|__ 📁 Features
    |__ 📁 Activities
        |__ 📁 GetActivity
            |__ #️⃣ ActivityResponse.cs
            |__ #️⃣ GetActivityEndpoint.cs
            |__ #️⃣ GetActivityQuery.cs
            |__ #️⃣ GetActivityQueryHandler.cs
        |__ 📁 CreateActivity
            |__ #️⃣ CreateActivity.cs
                |__ #️⃣ CreateActivity.Command.cs
                |__ #️⃣ CreateActivity.Endpoint.cs
                |__ #️⃣ CreateActivity.Handler.cs
                |__ #️⃣ CreateActivity.Validator.cs
    |__ 📁 Workouts
    |__ 📁 ...
|__ 📁 Middleware
|__ 📄 appsettings.json
|__ 📄 appsettings.Development.json
|__ #️⃣ Program.cs
</code></pre>
<p>A few more libraries for implementing the REPR pattern:</p>
<ul>
<li><a href="https://github.com/FastEndpoints/FastEndpoints">FastEndpoints</a></li>
<li><a href="https://github.com/ardalis/ApiEndpoints">ApiEndpoints</a></li>
</ul>
<h2>Next Steps</h2>
<p>Some of you may not like the idea of grouping all the files related to a feature in a single folder.</p>
<p>However, there's a lot of value in grouping by features in general.
You don't have to implement vertical slices.
But you can apply this concept to your domain by grouping files around aggregates, for example.
This is the approach I show in <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture.</strong></a></p>
<p>I made a video about <a href="https://youtu.be/msjnfdeDCmo"><strong>Vertical Slice Architecture,</strong></a>
showing how to implement the concepts discussed in today's issue. Check it out <a href="https://youtu.be/msjnfdeDCmo"><strong>here.</strong></a></p>
<p>Thanks for reading, and stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_062.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Functional Error Handling in .NET With the Result Pattern]]></title>
            <link>https://milanjovanovic.tech/blog/functional-error-handling-in-dotnet-with-the-result-pattern</link>
            <guid isPermaLink="false">functional-error-handling-in-dotnet-with-the-result-pattern</guid>
            <pubDate>Sat, 28 Oct 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[How should you handle errors in your code? This has been a topic of many discussions, and I want to share my opinion.
One school of thought suggests using exceptions for flow control. This is not a good approach because it makes the code harder to reason about. The caller must know the implementation details and which exceptions to handle.
Exceptions are for exceptional situations.
Today, I want to show you how to implement error handling using the Result pattern.
It's a functional approach to error handling, making your code more expressive.]]></description>
            <content:encoded><![CDATA[<p>How should you handle errors in your code?</p>
<p>This has been a topic of many discussions, and I want to share my opinion.</p>
<p>One school of thought suggests using exceptions for flow control.
This is not a good approach because it makes the code harder to reason about.
The caller must know the implementation details and which exceptions to handle.</p>
<p>Exceptions are for exceptional situations.</p>
<p>Today, I want to show you how to implement error handling using the <strong>Result pattern.</strong></p>
<p>It's a functional approach to error handling, making your code more expressive.</p>
<h2>Exceptions For Flow Control</h2>
<p>Using exceptions for flow control is an approach to implement the <strong>fail-fast</strong> principle.</p>
<p>As soon as you encounter an error in the code, you throw an exception —
effectively terminating the method, and making the caller responsible for handling the exception.</p>
<p>The problem is the caller must know which exceptions to handle.
And this isn't obvious from the method signature alone.</p>
<p>Another common use case is throwing exceptions for validation errors.</p>
<p>Here's an example in the <code>FollowerService</code>:</p>
<pre><code class="language-csharp">public sealed class FollowerService
{
    private readonly IFollowerRepository _followerRepository;

    public FollowerService(IFollowerRepository followerRepository)
    {
        _followerRepository = followerRepository;
    }

    public async Task StartFollowingAsync(
        User user,
        User followed,
        DateTime createdOnUtc,
        CancellationToken cancellationToken = default)
    {
        if (user.Id == followed.Id)
        {
            throw new DomainException(&quot;Can't follow yourself&quot;);
        }

        if (!followed.HasPublicProfile)
        {
            throw new DomainException(&quot;Can't follow non-public profile&quot;);
        }

        if (await _followerRepository.IsAlreadyFollowingAsync(
                user.Id,
                followed.Id,
                cancellationToken))
        {
            throw new DomainException(&quot;Already following&quot;);
        }

        var follower = Follower.Create(user.Id, followed.Id, createdOnUtc);

        _followerRepository.Insert(follower);
    }
}
</code></pre>
<h2>Use Exceptions for Exceptional Situations</h2>
<p>A rule of thumb I follow is to use exceptions for exceptional situations.
Since you already expect potential errors, why not make it explicit?</p>
<p>You can group all application errors into two groups:</p>
<ul>
<li>Errors you know how to handle</li>
<li>Errors you don't know how to handle</li>
</ul>
<p>Exceptions are an excellent solution for the errors you don't know how to handle.
And you should catch and handle them at the lowest level possible.</p>
<p>What about the errors you know how to handle?</p>
<p>You can handle them in a functional way with the <strong>Result pattern.</strong>
It's explicit and clearly expresses the intent that the method can fail.
The drawback is the caller has to manually check if the operation failed.</p>
<h2>Expressing Errors Using the Result Pattern</h2>
<p>The first thing you will need is an <code>Error</code> class to represent application errors.</p>
<ul>
<li><code>Code</code> - unique name for the error in the application</li>
<li><code>Description</code> - contains developer-friendly details about the error</li>
</ul>
<pre><code class="language-csharp">public sealed record Error(string Code, string Description)
{
    public static readonly Error None = new(string.Empty, string.Empty);
}
</code></pre>
<p>Then, you can implement the <code>Result</code> class using the <code>Error</code> to describe the failure.
This implementation is very bare-bones, and you could add many more features.
In most cases, you also need a generic <code>Result&lt;T&gt;</code> class, which will wrap a value inside.</p>
<p>Here's what the <code>Result</code> class looks like:</p>
<pre><code class="language-csharp">public class Result
{
    private Result(bool isSuccess, Error error)
    {
        if (isSuccess &amp;&amp; error != Error.None ||
            !isSuccess &amp;&amp; error == Error.None)
        {
            throw new ArgumentException(&quot;Invalid error&quot;, nameof(error));
        }

        IsSuccess = isSuccess;
        Error = error;
    }

    public bool IsSuccess { get; }

    public bool IsFailure =&gt; !IsSuccess;

    public Error Error { get; }

    public static Result Success() =&gt; new(true, Error.None);

    public static Result Failure(Error error) =&gt; new(false, error);
}
</code></pre>
<p>The only way to create a <code>Result</code> instance is by using static methods:</p>
<ul>
<li><code>Success</code> - creates a success result</li>
<li><code>Failure</code> - creates a failure result with the specified <code>Error</code></li>
</ul>
<p>If you want to avoid building your own <code>Result</code> class, take a look at the <a href="https://github.com/altmann/FluentResults">FluentResults</a> library.</p>
<h2>Applying the Result Pattern</h2>
<p>Now that we have the <code>Result</code> class let's see how to apply it in practice.</p>
<p>Here's a refactored version of the <code>FollowerService</code>.
Notice a few things:</p>
<ul>
<li>No more throwing exceptions</li>
<li>The <code>Result</code> return type is explicit</li>
<li>It's clear which errors the method returns</li>
</ul>
<p>Another benefit of error handling using the <strong>Result pattern</strong> is that it's easier to test.</p>
<pre><code class="language-csharp">public sealed class FollowerService
{
    private readonly IFollowerRepository _followerRepository;

    public FollowerService(IFollowerRepository followerRepository)
    {
        _followerRepository = followerRepository;
    }

    public async Task&lt;Result&gt; StartFollowingAsync(
        User user,
        User followed,
        DateTime utcNow,
        CancellationToken cancellationToken = default)
    {
        if (user.Id == followed.Id)
        {
            return Result.Failure(FollowerErrors.SameUser);
        }

        if (!followed.HasPublicProfile)
        {
            return Result.Failure(FollowerErrors.NonPublicProfile);
        }

        if (await _followerRepository.IsAlreadyFollowingAsync(
                user.Id,
                followed.Id,
                cancellationToken))
        {
            return Result.Failure(FollowerErrors.AlreadyFollowing);
        }

        var follower = Follower.Create(user.Id, followed.Id, utcNow);

        _followerRepository.Insert(follower);

        return Result.Success();
    }
}
</code></pre>
<h2>Documenting Application Errors</h2>
<p>You can use the <code>Error</code> class to document all possible errors in your application.</p>
<p>One approach is to create a static class called <code>Errors</code>.
It will have nested classes inside containing the specific errors.
The usage would look like <code>Errors.Followers.NonPublicProfile</code>.</p>
<p>However, the approach I like to use is to create a specific class containing the errors.</p>
<p>Here's the <code>FollowerErrors</code> class documenting the possible errors for the <code>Follower</code> entity:</p>
<pre><code class="language-csharp">public static class FollowerErrors
{
    public static readonly Error SameUser = new Error(
        &quot;Followers.SameUser&quot;, &quot;Can't follow yourself&quot;);

    public static readonly Error NonPublicProfile = new Error(
        &quot;Followers.NonPublicProfile&quot;, &quot;Can't follow non-public profiles&quot;);

    public static readonly Error AlreadyFollowing = new Error(
        &quot;Followers.AlreadyFollowing&quot;, &quot;Already following&quot;);
}
</code></pre>
<p>Instead of static fields, you can also use static methods returning an error.
You would call this method with a concrete argument to get an <code>Error</code> instance.</p>
<pre><code class="language-csharp">public static class FollowerErrors
{
    public static Error NotFound(Guid id) =&gt; new Error(
        &quot;Followers.NotFound&quot;, $&quot;The follower with Id '{id}' was not found&quot;);
}
</code></pre>
<h2>Converting Results Into API Responses</h2>
<p>The <code>Result</code> object will eventually reach the Minimal API (or controller) endpoint in <a href="http://ASP.NET">ASP.NET</a> Core.
Minimal APIs return an <code>IResult</code> response, and controllers return an <code>IActionResult</code> response.
Regardless, you must convert the <code>Result</code> instance into a valid API response.</p>
<p>The straightforward approach is checking the <code>Result</code> state and returning an HTTP response.
Here's an example where we check the <code>Result.IsFailure</code> flag:</p>
<pre><code class="language-csharp">app.MapPost(
    &quot;users/{userId}/follow/{followedId}&quot;,
    (Guid userId, Guid followedId, FollowerService followerService) =&gt;
    {
        var result = await followerService.StartFollowingAsync(
            userId,
            followedId,
            DateTime.UtcNow);

        if (result.IsFailure)
        {
            return Results.BadRequest(result.Error);
        }

        return Results.NoContent();
    });
</code></pre>
<p>However, this is an excellent opportunity for a more functional approach.
You can implement the <code>Match</code> extension method to provide a callback for each <code>Result</code> state.
The <code>Match</code> method will execute the respective callback and return the result.</p>
<p>Here's the implementation of <code>Match</code>:</p>
<pre><code class="language-csharp">public static class ResultExtensions
{
    public static T Match&lt;T&gt;(
        this Result result,
        Func&lt;T&gt; onSuccess,
        Func&lt;Error, T&gt; onFailure)
    {
        return result.IsSuccess ? onSuccess() : onFailure(result.Error);
    }
}
</code></pre>
<p>And this is how you would use the <code>Match</code> method in a Minimal API endpoint:</p>
<pre><code class="language-csharp">app.MapPost(
    &quot;users/{userId}/follow/{followedId}&quot;,
    (Guid userId, Guid followedId, FollowerService followerService) =&gt;
    {
        var result = await followerService.StartFollowingAsync(
            userId,
            followedId,
            DateTime.UtcNow);

        return result.Match(
            onSuccess: () =&gt; Results.NoContent(),
            onFailure: error =&gt; Results.BadRequest(error));
    });
</code></pre>
<p>Much more concise. Don't you think so?</p>
<h2>Summary</h2>
<p>If you take one thing with you from this week's issue, it should be this: exceptions are for exceptional situations.
Moreover, you should only use exceptions for errors you don't know how to handle.
In all other cases, expressing the error clearly with the <strong>Result pattern</strong> is more valuable.</p>
<p>Using the <code>Result</code> class allows you to:</p>
<ul>
<li>Express the intent that a method <em>could</em> fail</li>
<li>Encapsulate an application error inside</li>
<li>Provide a functional way to handle errors</li>
</ul>
<p>Additionally, you can document all application errors with the <code>Error</code> class.
This is helpful for developers to know which errors they need to handle.</p>
<p>You can even convert this to actual <em>documentation</em>.
For example, I wrote a simple program that scans the project for all <code>Error</code> fields.
It then converts this into a table format and uploads it to a Confluence page.</p>
<p>So I encourage you to try the <strong>Result pattern</strong> and see how it can improve your code.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_061.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[CQRS Pattern With MediatR]]></title>
            <link>https://milanjovanovic.tech/blog/cqrs-pattern-with-mediatr</link>
            <guid isPermaLink="false">cqrs-pattern-with-mediatr</guid>
            <pubDate>Sat, 21 Oct 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Today I want to show you how to use the CQRS pattern to build fast and scalable applications. The CQRS pattern separates the writes and reads in the application. This separation can be logical or physical and has many benefits. I'm also going to show you how to implement CQRS in your application using MediatR.]]></description>
            <content:encoded><![CDATA[<p>Today I want to show you how to use the <strong>CQRS</strong> pattern to build fast and scalable applications.</p>
<p>The CQRS pattern separates the writes and reads in the application.</p>
<p>This separation can be logical or physical and has many benefits:</p>
<ul>
<li>Complexity management</li>
<li>Improved performance</li>
<li>Scalability</li>
<li>Flexibility</li>
<li>Security</li>
</ul>
<p>I'm also going to show you how to implement CQRS in your application using MediatR.</p>
<p>But first, we have to understand what CQRS is.</p>
<h2>What Exactly is CQRS?</h2>
<p><a href="https://learn.microsoft.com/en-us/azure/architecture/patterns/cqrs">CQRS</a> stands for <strong>Command Query Responsibility Segregation</strong>.
The CQRS pattern uses separate models for reading and updating data.
The benefits of using CQRS are complexity management, improved performance, scalability, and security.</p>
<p>The standard approach for working with a database is using the same model to query and update data.
This is simple and works great for most CRUD operations.
However, in more complex applications, it becomes difficult to maintain.
On the write side, you could have complex business logic and validation in the model.
On the read side, you may need to perform many different queries.</p>
<p>Also, consider how we create the data model.
Applying SQL data modeling best practices will give you a normalized database.
This is generally fine, but it's optimized for writing.</p>
<p>Having separate models for commands and queries allows you to scale them independently.
The separation could be logical while using the same database.
You could split the subsystems for commands and queries into separate services.
And you can even have multiple databases optimized for writing or reading data.</p>
<h2>How Is It Different From CQS?</h2>
<p><a href="https://en.wikipedia.org/wiki/Command%E2%80%93query_separation">CQS</a> stands for <strong>Command Query Separation</strong>.
It's a term coined by Bertrand Meyer in his book <a href="https://en.wikipedia.org/wiki/Object-Oriented_Software_Construction">Object-Oriented Software Construction.</a></p>
<p>The basic premise of CQS is splitting an object's methods into <strong>Commands</strong> and <strong>Queries</strong>.</p>
<ul>
<li><strong>Commands</strong>: Change the state of a system but don't return a value</li>
<li><strong>Queries</strong>: Return a value and don't change the state of the system (no side effects)</li>
</ul>
<p>This doesn't mean a command can never return a value.
A typical example is popping a value from a stack.
It returns a value and changes the state of the system.
But the intent is what matters here.</p>
<p>CQS is a <em>principle.</em>
You can follow this principle if it makes sense, but be pragmatic.</p>
<p>CQRS is the evolution of CQS.
CQRS works on the architectural level.
At the same time, CQS works on the method (or class) level.</p>
<h2>Many Flavors of CQRS</h2>
<p>Here's a high-level overview of a CQRS system using multiple databases.
Commands update the write database.
Then, you need to synchronize the updates with the read database.
This introduces eventual consistency to CQRS systems.</p>
<p>Eventual consistency significantly increases the complexity of your application.
You must consider what happens if the synchronization process fails, and have a fault tolerance strategy.</p>
<p><img src="/blogs/mnw_060/cqrs.png" alt="Diagram of a system using CQRS with two databases."></p>
<p>There are many flavors of this approach:</p>
<ul>
<li>SQL database on the write side and NoSQL database (for example, <a href="https://ravendb.net/">RavenDB</a>) on the read side</li>
<li>Event sourcing on the write side and NoSQL database on the read side</li>
<li>Using Redis or some other distributed cache on the read side</li>
</ul>
<p>Separating the models for updating and reading data allows you to choose the best database for your requirements.</p>
<h2>Logical CQRS Architecture</h2>
<p>How do you apply the CQRS pattern to your system?
I prefer using <a href="https://github.com/jbogard/MediatR">MediatR.</a></p>
<p>MediatR implements the <a href="https://refactoring.guru/design-patterns/mediator">mediator pattern</a> to solve a simple problem - decoupling the in-process sending of messages from handling messages.</p>
<p>You can extend MediatR's <code>IRequest</code> interface with a custom <code>ICommand</code> and <code>IQuery</code> abstraction.
This allows you to define commands and queries in your system explicitly.</p>
<p>On the write side, I typically use <a href="https://learn.microsoft.com/en-us/ef/core/">EF Core</a> and a rich domain model to encapsulate business logic.
The command flow uses EF to load an entity into memory, execute the domain logic, and save the changes to the database.</p>
<p>On the read side, I want as little indirection as possible.
Using <a href="https://github.com/DapperLib/Dapper">Dapper</a> with raw SQL queries is an excellent choice.
You can also create views in the database and query them.
Alternatively, you could use EF Core to execute queries with projections.</p>
<p><img src="/blogs/mnw_060/cqrs_application.png" alt="Diagram of an application using CQRS on the architectural level."></p>
<h2>Implementing CQRS With MediatR</h2>
<p>Implementing CQRS with MediatR has two components:</p>
<ul>
<li>Defining your command or query class</li>
<li>Implementing the respective command or query handler</li>
</ul>
<p>I made an in-depth video explaining this process, and you can <a href="https://youtu.be/vdi-p9StmG0">watch it here.</a></p>
<p>You use the <code>ISender</code> interface to <code>Send</code> the command or query.
MediatR takes care of routing the command or query to the respective handler.</p>
<p>The request will pass through the <em>request pipeline</em>.
It's a wrapper around each request, and you can use it to solve cross-cutting concerns with <code>IPipelineBehavior</code>.
For example, you can implement <a href="cqrs-validation-with-mediatr-pipeline-and-fluentvalidation">validation for commands with FluentValidation.</a></p>
<pre><code class="language-csharp">[ApiController]
[Route(&quot;api/bookings&quot;)]
public class BookingsController : ControllerBase
{
    private readonly ISender _sender;

    public BookingsController(ISender sender)
    {
        _sender = sender;
    }

    [HttpPut(&quot;{id}/confirm&quot;)]
    public async Task&lt;IActionResult&gt; ConfirmBooking(
        Guid id,
        CancellationToken cancellationToken)
    {
        var command = new ConfirmBookingCommand(id);

        var result = await _sender.Send(command, cancellationToken);

        if (result.IsFailure)
        {
            return BadRequest(result.Error);
        }

        return NoContent();
    }
}

</code></pre>
<p>Here's an example of a command handler with repositories and a rich domain model:</p>
<pre><code class="language-csharp">internal sealed class ConfirmBookingCommandHandler
    : ICommandHandler&lt;ConfirmBookingCommand&gt;
{
    private readonly IDateTimeProvider _dateTimeProvider;
    private readonly IBookingRepository _bookingRepository;
    private readonly IUnitOfWork _unitOfWork;

    public ConfirmBookingCommandHandler(
        IDateTimeProvider dateTimeProvider,
        IBookingRepository bookingRepository,
        IUnitOfWork unitOfWork)
    {
        _dateTimeProvider = dateTimeProvider;
        _bookingRepository = bookingRepository;
        _unitOfWork = unitOfWork;
    }

    public async Task&lt;Result&gt; Handle(
        ConfirmBookingCommand request,
        CancellationToken cancellationToken)
    {
        var booking = await _bookingRepository.GetByIdAsync(
            request.BookingId,
            cancellationToken);

        if (booking is null)
        {
            return Result.Failure(BookingErrors.NotFound);
        }

        var result = booking.Confirm(_dateTimeProvider.UtcNow);

        if (result.IsFailure)
        {
            return result;
        }

        await _unitOfWork.SaveChangesAsync(cancellationToken);

        return Result.Success();
    }
}
</code></pre>
<p>Here's an example of a query handler that uses Dapper and raw SQL:</p>
<pre><code class="language-csharp">internal sealed class SearchApartmentsQueryHandler
    : IQueryHandler&lt;SearchApartmentsQuery, IReadOnlyList&lt;ApartmentResponse&gt;&gt;
{
    private static readonly int[] ActiveBookingStatuses =
    {
        (int)BookingStatus.Reserved,
        (int)BookingStatus.Confirmed,
        (int)BookingStatus.Completed
    };

    private readonly ISqlConnectionFactory _sqlConnectionFactory;

    public SearchApartmentsQueryHandler(
        ISqlConnectionFactory sqlConnectionFactory)
    {
        _sqlConnectionFactory = sqlConnectionFactory;
    }

    public async Task&lt;Result&lt;IReadOnlyList&lt;ApartmentResponse&gt;&gt;&gt; Handle(
        SearchApartmentsQuery request,
        CancellationToken cancellationToken)
    {
        if (request.StartDate &gt; request.EndDate)
        {
            return new List&lt;ApartmentResponse&gt;();
        }

        using var connection = _sqlConnectionFactory.CreateConnection();

        const string sql = &quot;&quot;&quot;
            SELECT
                a.id AS Id,
                a.name AS Name,
                a.description AS Description,
                a.price_amount AS Price,
                a.price_currency AS Currency,
                a.address_country AS Country,
                a.address_state AS State,
                a.address_zip_code AS ZipCode,
                a.address_city AS City,
                a.address_street AS Street
            FROM apartments AS a
            WHERE NOT EXISTS
            (
                SELECT 1
                FROM bookings AS b
                WHERE
                    b.apartment_id = a.id AND
                    b.duration_start &lt;= @EndDate AND
                    b.duration_end &gt;= @StartDate AND
                    b.status = ANY(@ActiveBookingStatuses)
            )
            &quot;&quot;&quot;;

        var apartments = await connection
            .QueryAsync&lt;ApartmentResponse, AddressResponse, ApartmentResponse&gt;(
                sql,
                (apartment, address) =&gt;
                {
                    apartment.Address = address;

                    return apartment;
                },
                new
                {
                    request.StartDate,
                    request.EndDate,
                    ActiveBookingStatuses
                },
                splitOn: &quot;Country&quot;);

        return apartments.ToList();
    }
}
</code></pre>
<h2>Closing Thoughts</h2>
<p>Separating commands and queries can improve performance and scalability in the long run.
You can optimize commands and queries differently based on your requirements.</p>
<p>Commands encapsulate complex business logic and validation.
Using EF Core and a rich domain model is an excellent solution.</p>
<p>Queries are all about performance, so you want to use what's fastest.
This could be raw SQL queries with Dapper, EF Core projections, or Redis.</p>
<p>If you want the system I use to build scalable applications with CQRS and MediatR, check out <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture.</strong></a></p>
<p>Stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_060.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Improving ASP.NET Core Dependency Injection With Scrutor]]></title>
            <link>https://milanjovanovic.tech/blog/improving-aspnetcore-dependency-injection-with-scrutor</link>
            <guid isPermaLink="false">improving-aspnetcore-dependency-injection-with-scrutor</guid>
            <pubDate>Sat, 14 Oct 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Dependency injection (DI) is one of the most exciting features of ASP.NET Core. It helps us build more testable and maintainable applications. However, ASP.NET Core's built-in DI system sometimes needs a little help to achieve more advanced scenarios.
So I want to introduce you to a powerful library for enhancing your ASP.NET Core DI - Scrutor.]]></description>
            <content:encoded><![CDATA[<p>Dependency injection (DI) is one of the most exciting features of <a href="http://ASP.NET">ASP.NET</a> Core.
It helps us build more testable and maintainable applications.
However, <a href="http://ASP.NET">ASP.NET</a> Core's built-in DI system sometimes needs a little help to achieve more advanced scenarios.</p>
<p>So I want to introduce you to a powerful library for enhancing your <a href="http://ASP.NET">ASP.NET</a> Core DI - <a href="https://www.nuget.org/packages/Scrutor">Scrutor.</a></p>
<p>If you're an <a href="http://ASP.NET">ASP.NET</a> Core developer, you're already familiar with Dependency Injection.
It's a fundamental part of building modular and maintainable applications.</p>
<p>Let's explore how Scrutor can simplify and enhance your DI setup.</p>
<h2>What is Dependency Injection?</h2>
<p><a href="https://learn.microsoft.com/en-us/aspnet/core/fundamentals/dependency-injection?view=aspnetcore-7.0">Dependency Injection</a>
is a software design pattern used in <a href="http://ASP.NET">ASP.NET</a> Core to achieve the <a href="https://learn.microsoft.com/en-us/dotnet/architecture/modern-web-apps-azure/architectural-principles#dependency-inversion">Inversion of Control (IOC)</a>
principle.
This promotes loose coupling and makes your code more testable, maintainable, and extensible.</p>
<p>DI allows you to inject dependencies into your classes rather than create them within the class.
The framework takes care of providing the required instances at runtime.
It also manages the disposal of these dependencies based on the service lifetime.</p>
<p>Here's an example of combining constructor and method injection in a controller:</p>
<pre><code class="language-csharp">[ApiController]
[Route(&quot;api/activities&quot;)]
public class ActivitiesController : ControllerBase
{
    private readonly ILogger&lt;ActivitiesController&gt; _logger;

    // Constructor injection
    public ActivitiesController(ILogger&lt;ActivitiesController&gt; logger)
    {
        _logger = logger;
    }

    [HttpGet]
    public async Task&lt;IActionResult&gt; Get(ISender sender) // Method injection
    {
        var activities = await sender.Send(new GetActivitiesQuery());

        return Ok(activities);
    }
}
</code></pre>
<h2>Service Lifetimes in <a href="http://ASP.NET">ASP.NET</a> Core</h2>
<p>Before we dive into Scrutor, let's briefly discuss <a href="https://learn.microsoft.com/en-us/dotnet/core/extensions/dependency-injection#service-lifetimes">service lifetimes in ASP.NET Core.</a>
When you register a service in the DI container, you specify its lifetime.
The service lifetime defines how long the DI container should maintain the service.</p>
<p><a href="http://ASP.NET">ASP.NET</a> Core provides three main lifetimes:</p>
<ul>
<li><strong>Singleton</strong>: A single instance of the service is created and reused throughout the application's lifetime.</li>
<li><strong>Scoped</strong>: A new instance is created for each scope (usually a web request). Services created in the same scope share the same instance.</li>
<li><strong>Transient</strong>: A new instance is created every time the service is requested.</li>
</ul>
<p>Understanding service lifetimes is crucial when designing your application's architecture.</p>
<h2>What is Scrutor?</h2>
<p>The <a href="https://github.com/khellang/Scrutor">Scrutor library</a> improves your dependency injection code by extending the existing features from <code>Microsoft.Extensions.DependencyInjection</code>.</p>
<p>These extensions add support for advanced assembly scanning and service decoration.</p>
<p>To get started using Scrutor, you need to install the NuGet package:</p>
<pre><code class="language-powershell">Install-Package Scrutor
</code></pre>
<h2>Assembly Scanning With Scrutor</h2>
<p>One of the most powerful features of Scrutor is its ability to perform assembly scanning.
Rather than manually registering each service, Scrutor allows you to scan your assemblies for types that should be registered with the DI container.
This can significantly reduce the boilerplate code required for service registration, making your code cleaner and more maintainable.</p>
<p>The entry point for assembly scanning is the <code>Scan</code> method, which accepts a delegate to define the DI setup.</p>
<p>Here's an example of scanning two assemblies and registering the classes inside as scoped services:</p>
<pre><code class="language-csharp">builder.Services.Scan(selector =&gt; selector
    .FromAssemblies(
        typeof(PersistenceAssembly).Assembly,
        typeof(InfrastructureAssembly).Assembly)
    .AddClasses(publicOnly: false)
    .UsingRegistrationStrategy(RegistrationStrategy.Skip)
    .AsMatchingInterface()
    .WithScopedLifetime());
</code></pre>
<p>Let's unpack what's happening here:</p>
<ul>
<li><code>FromAssemblies</code> - allows you to specify which assemblies to scan</li>
<li><code>AddClasses</code> - adds the classes from the selected assemblies</li>
<li><code>UsingRegistrationStrategy</code> - defines which <code>RegistrationStrategy</code> to use</li>
<li><code>AsMatchingInterface</code> - registers the types as matching interfaces (<code>ClassName</code> → <code>IClassName</code>)</li>
<li><code>WithScopedLifetime</code> - registers the types with a scoped service lifetime</li>
</ul>
<p>There are three values for <code>RegistrationStrategy</code> you can use:</p>
<ul>
<li><code>RegistrationStrategy.Skip</code> - skips registrations if service already exists</li>
<li><code>RegistrationStrategy.Append</code>- appends a new registration for existing services</li>
<li><code>RegistrationStrategy.Throw</code>- throws when trying to register an existing service</li>
</ul>
<p>You can also specify a filter to <code>AddClasses</code> to select specific types you want to configure.
Here's an example of registering repository implementations:</p>
<pre><code class="language-csharp">services.Scan(scan =&gt; scan
    .FromAssemblies(typeof(PersistenceAssembly).Assembly)
    .AddClasses(
        filter =&gt; filter.Where(x =&gt; x.Name.EndsWith(&quot;Repository&quot;)),
        publicOnly: false)
    .UsingRegistrationStrategy(RegistrationStrategy.Throw)
    .AsMatchingInterface()
    .WithScopedLifetime());
</code></pre>
<h2>Service Decoration With Scrutor</h2>
<p>Service decoration is another valuable feature offered by Scrutor.
It enables you to modify or extend services during registration without changing the original implementation.</p>
<p>This is incredibly useful when adding cross-cutting concerns or other modifications to services without altering their core functionality.
For example, you can implement a <a href="decorator-pattern-in-asp-net-core">caching decorator for repositories.</a></p>
<p>Here's how you can configure a decorator with Scrutor's <code>Decorate</code> method:</p>
<pre><code class="language-csharp">services.AddScoped&lt;IActivitiesRepository, ActivitiesRepository&gt;();

services.Decorate&lt;IActivitiesRepository, PermissionActivitiesRepository&gt;();
</code></pre>
<p>It will decorate the <code>ActivitiesRepository</code> service using the <code>PermissionActivitiesRepository</code>.
This also means that <code>PermissionActivitiesRepository</code> can inject an <code>IActivitiesRepository</code> instance, and at runtime, this is resolved as <code>ActivitiesRepository</code>.</p>
<p>Here's how you can implement the <code>PermissionActivitiesRepository</code>:</p>
<pre><code class="language-csharp">public class PermissionActivitiesRepository : IActivitiesRepository
{
    private readonly IActivitiesRepository _decorated;
    private readonly IPermissionChecker _permissionChecker;

    public PermissionActivitiesRepository(
        IActivitiesRepository decorated,
        IPermissionChecker permissionChecker)
    {
        _decorated = decorated;
        _permissionChecker = permissionChecker;
    }

    public List&lt;Activity&gt; Get()
    {
        if (!_permissionChecker.HasPermission(Permissions.FetchActivities))
        {
            return new();
        }

        return _decorated.Get();
    }
}
</code></pre>
<h2>Takeaway</h2>
<p>Scrutor can improve your <a href="http://ASP.NET">ASP.NET</a> Core DI by simplifying service registration through assembly scanning and enabling service decoration.
You can use Scrutor's capabilities to write cleaner, more maintainable, and flexible DI code while reducing the complexity of your startup configuration.</p>
<p>Assembly scanning can reduce the boilerplate code required for service registration.
It also allows you to create custom conventions for registering services.</p>
<p>Service decoration has been a real game-changer for me.
It's the simplest way to introduce cross-cutting concerns in your application.
For example, I used to add an idempotency check before handling events.</p>
<p>Hope this was valuable.</p>
<p>Stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_059.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Getting Started With NServiceBus in .NET]]></title>
            <link>https://milanjovanovic.tech/blog/getting-started-with-nservicebus-in-dotnet</link>
            <guid isPermaLink="false">getting-started-with-nservicebus-in-dotnet</guid>
            <pubDate>Sat, 07 Oct 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[NServiceBus is a feature-rich messaging framework supporting many different message transports. It's developed and maintained by Particular Software. And it simplifies the process of building complex distributed systems across various cloud-based queueing technologies.
The basic building blocks of NServiceBus are messages and endpoints. A message contains the required information to execute a business operation. Endpoints are logical entities that send and receive messages.]]></description>
            <content:encoded><![CDATA[<p>NServiceBus is a feature-rich messaging framework supporting many different message transports.
It's developed and maintained by <a href="https://particular.net/">Particular Software.</a>
And it simplifies the process of building complex distributed systems across various cloud-based queueing technologies.</p>
<p>The basic building blocks of NServiceBus are messages and endpoints.
A message contains the required information to execute a business operation.
Endpoints are logical entities that send and receive messages.</p>
<p>And now let's see how to get started with NServiceBus, from installation and setup to building your first NServiceBus endpoint.</p>
<p>In this week's newsletter, you will learn how to:</p>
<ul>
<li>Configure an endpoint to use Azure Service Bus</li>
<li>Send and publish messages using <code>IMessageSession</code></li>
<li>Handle messages with NServiceBus</li>
</ul>
<p>Let's dive in!</p>
<h2>What is NServiceBus?</h2>
<p><a href="https://go.particular.net/milanjovanovic">NServiceBus</a> is a messaging framework and platform that simplifies building reliable, scalable, and maintainable distributed systems.
It's designed to address the challenges that arise when building applications that are distributed across multiple servers.</p>
<p>One of NServiceBus's foundational principles is its embrace of a message-driven architecture.
In this model, components communicate by sending and receiving messages.
Messages are the fundamental units of communication, representing commands, events, or data that services exchange.</p>
<p>Why is this significant?</p>
<p>Message-driven architectures offer several advantages:</p>
<ul>
<li>Asynchronous communication</li>
<li>Loose coupling</li>
<li>Reliability</li>
</ul>
<p>NServiceBus supports the powerful publish/subscribe (pub/sub) messaging pattern.
This pattern allows services to publish events and subscribe to events of interest.
When a service publishes an event, all interested subscribers receive a copy of the event.
This is a key feature for building event-driven architectures, where services react to and process events in response to various actions or changes in the system.</p>
<h2>Configuring the NServiceBus Endpoint</h2>
<p>NServiceBus uses the concept of an <em>endpoint</em> to send and receive messages.
It's a logical component that communicates with other components.
You define your message handlers and sagas inside of an endpoint.</p>
<p>Let's start by installing the <code>NServiceBus</code> NuGet package:</p>
<pre><code class="language-powershell">Install-Package NServiceBus.Extensions.Hosting
</code></pre>
<p>Now you can configure an <em>endpoint</em> to use <a href="https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-messaging-overview">Azure Service Bus</a>
to send messages:</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder();

builder.Host.UseNServiceBus(context =&gt;
{
    var endpointConfiguration = new EndpointConfiguration(&quot;Training&quot;);

    var transport = endpointConfiguration
        .UseTransport&lt;AzureServiceBusTransport&gt;();

    var connectionString = builder.Configuration
        .GetConnectionString(&quot;AzureServiceBusConnectionString&quot;);
    transport.ConnectionString(connectionString);

    endpointConfiguration.EnableInstallers();

    return endpointConfiguration;
});

var app = builder.Build();

app.Run();
</code></pre>
<p>The call to <code>UseNServiceBus</code> tells the host to use NServiceBus.
Inside the callback, you can configure the endpoint that will start when the host runs.</p>
<p>One more important aspect is calling <code>EnableInstallers</code> to set up the Azure Service Bus topology.
This will tell NServiceBus to create the required queues, so you don't have to do it manually.</p>
<h2>Publishing Messages in NServiceBus</h2>
<p>The next building block you need in any messaging system is the messages.
Messages are C# classes or interfaces that contain meaningful data for the business process.</p>
<p>NServiceBus supports three types of messages:</p>
<ul>
<li><code>ICommand</code> - sends a request to perform an action</li>
<li><code>IEvent</code> - communicates that something significant occurred</li>
<li><code>IMessage</code> - for messages that aren't commands or events (typically for replies in <em>request-response</em>)</li>
</ul>
<p>Events can have more than one handler, while a command should have only one handler.</p>
<p>Let's create our first message contract:</p>
<pre><code class="language-csharp">using NServiceBus;

public class WorkoutCreated : IEvent
{
    public Guid Id [ get; set; ]
}
</code></pre>
<p>The <code>WorkoutCreated</code> message is an event that we will publish after creating a new <code>Workout</code>.</p>
<p>You can use the <code>IMessageSession</code> service to send messages from your controllers or Minimal API endpoints.</p>
<pre><code class="language-csharp">app.MapPost(&quot;api/workouts&quot;, async (
    Workout workout,
    AppDbContext context,
    IMessageSession messageSession) =&gt;
{
    context.Add(workout);

    await context.SaveChangesAsync();

    await messageSession.Publish(new  WorkoutCreated { Id = workout.Id });

    return Results.Ok(workout);
});
</code></pre>
<p>NServiceBus has some built-in validation when sending messages.
You have to specify an <code>ICommand</code> when calling the <code>Send</code> method, or you will get an exception.
Similarly, you have to specify an <code>IEvent</code> when calling the <code>Publish</code> method.</p>
<h2>Handling Messages With NServiceBus</h2>
<p>Once you send a message, you need a way to handle it and run some business logic.
To handle a message, you need to implement the <code>IHandleMessages</code> interface and specify which message you are handling.</p>
<p>Here's an implementation of the <code>WorkoutCreatedHandler</code>:</p>
<pre><code class="language-csharp">public class WorkoutCreatedHandler : IHandleMessages&lt;WorkoutCreated&gt;
{
    private readonly ILogger&lt;WorkoutCreated&gt; _logger;

    public WorkoutCreatedHandler(ILogger&lt;WorkoutCreated&gt; logger)
    {
        _logger = logger;
    }

    public async Task Handle(
        WorkoutCreated message,
        IMessageHandlerContext context)
    {
        logger.LogInformation(&quot;Processing workout - {Id}&quot;, message.Id);

        // Continue to process the message.
    }
}
</code></pre>
<p>Implementing <code>IHandleMessages&lt;WorkoutCreated&gt;</code> tells NServiceBus how to process the <code>WorkoutCreated</code> message when an endpoint receives it.
This interface defines only one method: <code>Handle</code>.</p>
<p>The <code>Handle</code> method has an <code>IMessageHandlerContext</code> parameter, which allows you to send more messages.
This can be helpful when implementing a choreographed saga.
Processing one message triggers the next step in the chain until the entire process is completed.</p>
<h2>In Summary</h2>
<p>In this week's issue, we discussed NServiceBus, a robust messaging framework for building distributed systems in .NET.
You learned how to configure NServiceBus with the Azure Service Bus transport.
We discussed the different message types in NServiceBus and how to publish and handle a message.</p>
<p>Building distributed systems is a complex endeavor, but NServiceBus simplifies many challenges.
By embracing a message-driven architecture and leveraging NServiceBus's features,
you'll be well-equipped to create resilient, scalable, and maintainable applications in the .NET ecosystem.</p>
<p>Further reading:</p>
<ul>
<li><a href="https://go.particular.net/milanjovanovic/getting-started-with-nservicebus">NServiceBus step-by-step tutorial</a></li>
<li><a href="https://go.particular.net/milanjovanovic/live-coding-your-first-nservicebus-system">Live coding an NServiceBus system</a></li>
<li><a href="https://go.particular.net/milanjovanovic/monitoring-demo">NServiceBus monitoring demo</a></li>
<li><a href="messaging-made-easy-with-azure-service-bus">Messaging with Azure Service Bus</a></li>
<li><a href="implementing-the-saga-pattern-with-rebus-and-rabbitmq">Implementing the Saga pattern</a></li>
<li><a href="orchestration-vs-choreography">Orchestration vs Choreography</a></li>
</ul>
<p>Hope this was helpful.</p>
<p>I'll see you next week!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_058.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[CQRS Validation with MediatR Pipeline and FluentValidation]]></title>
            <link>https://milanjovanovic.tech/blog/cqrs-validation-with-mediatr-pipeline-and-fluentvalidation</link>
            <guid isPermaLink="false">cqrs-validation-with-mediatr-pipeline-and-fluentvalidation</guid>
            <pubDate>Sat, 30 Sep 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Validation is an essential cross-cutting concern that you need to solve in your application. You want to ensure the request is valid before you consider processing it.
Another important question you need to answer is how you approach different types of validation. For example, I consider input and business validation differently, and each deserves a specific solution.
I want to show you an elegant solution for validation using MediatR and FluentValidation.
If you aren't using CQRS with MediatR, don't worry. Everything I explain about validation can easily be adapted to other paradigms.]]></description>
            <content:encoded><![CDATA[<p>Validation is an essential cross-cutting concern that you need to solve in your application.
You want to ensure the request is valid before you consider processing it.</p>
<p>Another important question you need to answer is how you approach different types of validation.
For example, I consider input and business validation differently, and each deserves a specific solution.</p>
<p>I want to show you an elegant solution for validation using <a href="https://github.com/jbogard/MediatR">MediatR</a>
and <a href="https://docs.fluentvalidation.net/en/latest/index.html">FluentValidation.</a></p>
<p>If you aren't using <a href="https://learn.microsoft.com/en-us/azure/architecture/patterns/cqrs">CQRS</a> with MediatR, don't worry.
Everything I explain about validation can easily be adapted to other paradigms.</p>
<p>Here's what I'm going to talk about in this week's newsletter:</p>
<ul>
<li>Standard validation approach</li>
<li>Input vs business validation</li>
<li>Separating validation logic</li>
<li>Generic <code>ValidationBehavior</code></li>
</ul>
<p>Let's dive in.</p>
<h2>The Standard Command Validation Approach</h2>
<p>The standard way of implementing validation is right before processing the command.
The validation is tightly coupled to the command handler, which could be problematic.</p>
<p>I find this approach difficult to maintain as the complexity of the validation increases.
Each change to the validation logic also touches the handler, and the handler itself can grow out of control.</p>
<p>It also makes it harder to differentiate between input and <em>business</em> validation.</p>
<p>Here's an example <code>ShipOrderCommandHandler</code> that checks if the <code>ShippingAddress.Country</code> is one of the supported countries:</p>
<pre><code class="language-csharp">internal sealed class ShipOrderCommandHandler
    : IRequestHandler&lt;ShipOrderCommand&gt;
{
    private readonly IOrderRepository _orderRepository;
    private readonly IShippingService _shippingService;
    private readonly ShipmentSettings _shipmentSettings;

    public async Task Handle(
        ShipOrderCommand command,
        CancellationToken cancellationToken)
    {
        if (!_shipmentSettings
                .SupportedCountries
                .Contains(command.ShippingAddress.Country))
        {
            throw new ArgumentException(nameof(ShipOrderCommand.Address));
        }

        var order = _orderRepository.Get(command.OrderId);

        _shippingService.ShipTo(
            command.ShippingAddress,
            command.ShippingMethod);
    }
}
</code></pre>
<p>What if we can separate command validation and command handling?</p>
<h2>Input Validation and Business Validation</h2>
<p>I mentioned input and <em>business</em> validation in the previous section.</p>
<p>Here's how I consider them to be different:</p>
<ul>
<li><strong>Input validation</strong> - We only validate that the command is <em>processable</em>.
These are simple validations, such as checking for <code>null</code> values, empty strings, etc.</li>
<li><strong>Business validation</strong> - We validate the command to satisfy the business rules.
This includes checking the system state for required preconditions before processing the command.</li>
</ul>
<p>Another way to compare them is cheap vs. expensive.
Input validation is usually cheap to execute and can be done in memory.
While business validation involves reading state and is slower.</p>
<p>So, input validation sits at the entry point of the use case before handling the request.
After it completes, we have a <em>valid</em> command.
And this is a rule I always follow - an invalid command should never reach the handler.</p>
<h2>Input Validation With FluentValidation</h2>
<p><a href="https://docs.fluentvalidation.net/en/latest/index.html">FluentValidation</a> is an excellent validation library for .NET,
which uses a fluent interface and lambda expressions for building strongly typed validation rules.</p>
<p>Here's the <code>ShipOrderCommand</code> that we want to validate:</p>
<pre><code class="language-csharp">public sealed record ShipOrderCommand : IRequest
{
    public Guid OrderId { get; set; }

    public string ShippingMethod { get; set; }

    public Address ShippingAddress { get; set; }
}
</code></pre>
<p>To implement a validator with <a href="https://github.com/FluentValidation/FluentValidation">FluentValidation,</a>
you create a class that inherits from the <code>AbstractValidator&lt;T&gt;</code> base class.
Then, you can add the validation rules from the constructor using <code>RuleFor</code>:</p>
<pre><code class="language-csharp">public sealed class ShipOrderCommandValidator
    : AbstractValidator&lt;ShipOrderCommand&gt;
{
    public ShipOrderCommandValidator(ShipmentSettings settings)
    {
        RuleFor(command =&gt; command.OrderId)
            .NotEmpty()
            .WithMessage(&quot;The order identifier can't be empty.&quot;);

        RuleFor(command =&gt; command.ShippingMethod)
            .NotEmpty()
            .WithMessage(&quot;The shipping method can't be empty.&quot;);

        RuleFor(command =&gt; command.ShippingAddress)
            .NotNull()
            .WithMessage(&quot;The shipping address can't be empty.&quot;);

        RuleFor(command =&gt; command.ShippingAddress.Country)
            .Must(country =&gt; settings.SupportedCountries.Contains(country))
            .WithMessage(&quot;The shipping country isn't supported.&quot;);
    }
}
</code></pre>
<p>The naming convention I like to use is the name of the command and append <em>Validator</em>.
You can also enforce this by writing <a href="enforcing-software-architecture-with-architecture-tests">architecture tests.</a></p>
<p>To automatically register all validators from an assembly, you need to call the <code>AddValidatorsFromAssembly</code> method:</p>
<pre><code class="language-csharp">services.AddValidatorsFromAssembly(ApplicationAssembly.Assembly);
</code></pre>
<h2>Running Validation From the Use Case</h2>
<p>To run the <code>ShipOrderCommandValidator</code>, you can use the <code>IValidator&lt;T&gt;</code> service and inject it from the constructor.</p>
<p>The validator exposes a few methods you can call, like <code>Validate</code>, <code>ValidateAsync</code>, or <code>ValidateAndThrow</code>.</p>
<p>The <code>Validate</code> method returns a <code>ValidationResult</code> object which contains two properties:</p>
<ul>
<li><code>IsValid</code> - a boolean flag saying whether the validation succeeded</li>
<li><code>Errors</code> - a collection of <code>ValidationFailure</code> objects containing any validation failures</li>
</ul>
<p>Alternatively, calling the <code>ValidateAndThrow</code> method throws a <code>ValidationException</code> if validation fails.</p>
<pre><code class="language-csharp">internal sealed class ShipOrderCommandHandler
    : IRequestHandler&lt;ShipOrderCommand&gt;
{
    private readonly IOrderRepository _orderRepository;
    private readonly IShippingService _shippingService;
    private readonly IValidator&lt;ShipOrderCommand&gt; _validator;

    public async Task Handle(
        ShipOrderCommand command,
        CancellationToken cancellationToken)
    {
        _validator.ValidateAndThrow(command);

        var order = _orderRepository.Get(command.OrderId);

        _shippingService.ShipTo(
            command.ShippingAddress,
            command.ShippingMethod);
    }
}
</code></pre>
<p>This approach forces you to define an explicit dependency on <code>IValidator</code> in every command handler.</p>
<p>What if we can implement this cross-cutting concern in a more generic way?</p>
<h2>MediatR Validation Pipeline</h2>
<p>Here's a complete implementation of a <code>ValidationBehavior</code> using FluentValidation and MediatR's <code>IPipelineBehavior</code>.</p>
<p>The <code>ValidationBehavior</code> acts as a middleware for the request pipeline and performs validation.
If the validation fails, it will throw a custom <code>ValidationException</code> with a collection of <code>ValidationError</code> objects.</p>
<p>I also want to highlight the use of <code>ValidateAsync</code>, which allows you to define asynchronous validation rules.
You must call the <code>ValidateAsync</code> method if you have asynchronous rules.
Otherwise, the validator will throw an exception.</p>
<pre><code class="language-csharp">public sealed class ValidationBehavior&lt;TRequest, TResponse&gt;
    : IPipelineBehavior&lt;TRequest, TResponse&gt;
    where TRequest : ICommandBase
{
    private readonly IEnumerable&lt;IValidator&lt;TRequest&gt;&gt; _validators;

    public ValidationBehavior(IEnumerable&lt;IValidator&lt;TRequest&gt;&gt; validators)
    {
        _validators = validators;
    }

    public async Task&lt;TResponse&gt; Handle(
        TRequest request,
        RequestHandlerDelegate&lt;TResponse&gt; next,
        CancellationToken cancellationToken)
    {
        var context = new ValidationContext&lt;TRequest&gt;(request);

        var validationFailures = await Task.WhenAll(
            _validators.Select(validator =&gt; validator.ValidateAsync(context)));

        var errors = validationFailures
            .Where(validationResult =&gt; !validationResult.IsValid)
            .SelectMany(validationResult =&gt; validationResult.Errors)
            .Select(validationFailure =&gt; new ValidationError(
                validationFailure.PropertyName,
                validationFailure.ErrorMessage))
            .ToList();

        if (errors.Any())
        {
            throw new Exceptions.ValidationException(errors);
        }

        var response = await next();

        return response;
    }
}
</code></pre>
<p>Don't forget to register the <code>ValidationBehavior</code> with MediatR by calling <code>AddOpenBehavior</code>:</p>
<pre><code class="language-csharp">services.AddMediatR(config =&gt;
{
    config.RegisterServicesFromAssemblyContaining&lt;ApplicationAssembly&gt;();

    config.AddOpenBehavior(typeof(ValidationBehavior&lt;,&gt;));
});
</code></pre>
<h2>Handling Validation Exceptions</h2>
<p>Here's a custom <code>ValidationExceptionHandlingMiddleware</code> middleware that only handles the custom <code>ValidationException</code>.
It converts the exception to a <code>ProblemDetails</code> response and includes any validation errors.</p>
<p>You can easily expand this to be a generic global exception handler.</p>
<pre><code class="language-csharp">public sealed class ValidationExceptionHandlingMiddleware
{
    private readonly RequestDelegate _next;

    public ValidationExceptionHandlingMiddleware(RequestDelegate next)
    {
        _next = next;
    }

    public async Task InvokeAsync(HttpContext context)
    {
        try
        {
            await _next(context);
        }
        catch (Exceptions.ValidationException exception)
        {
            var problemDetails = new ProblemDetails
            {
                Status = StatusCodes.Status400BadRequest,
                Type = &quot;ValidationFailure&quot;,
                Title = &quot;Validation error&quot;,
                Detail = &quot;One or more validation errors has occurred&quot;
            };

            if (exception.Errors is not null)
            {
                problemDetails.Extensions[&quot;errors&quot;] = exception.Errors;
            }

            context.Response.StatusCode = StatusCodes.Status400BadRequest;

            await context.Response.WriteAsJsonAsync(problemDetails);
        }
    }
}
</code></pre>
<p>You also need to include the middleware in the request pipeline by calling <code>UseMiddleware</code>:</p>
<pre><code class="language-csharp">app.UseMiddleware&lt;ExceptionHandlingMiddleware&gt;();
</code></pre>
<h2>Takeaway</h2>
<p>This implementation of <code>ValidationBehavior</code> is something I use in real projects, and it works incredibly well.
If I don't want to throw an exception, I can update the <code>ValidationBehavior</code> to return a result object instead.</p>
<p>How do you apply this if you're not using MediatR?</p>
<p>I'm using an <code>IPipelineBehavior</code>, which allows me to implement a <em>middleware</em> wrapping each request.</p>
<p>So, all you need is a way to implement middleware and place your validation inside.
And I like having options, so here are <a href="3-ways-to-create-middleware-in-asp-net-core">three ways to create middleware in ASP.NET Core.</a></p>
<p>Hope this was valuable.</p>
<p>Stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_057.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Monolith to Microservices: How a Modular Monolith Helps]]></title>
            <link>https://milanjovanovic.tech/blog/monolith-to-microservices-how-a-modular-monolith-helps</link>
            <guid isPermaLink="false">monolith-to-microservices-how-a-modular-monolith-helps</guid>
            <pubDate>Sat, 23 Sep 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[You start building a beautiful monolith system. Maybe a modular monolith. The system grows, and requirements are ever-changing. Slowly, cracks begin to appear in the system. This could be for organizational reasons and distributing the work across a team. Or it could be because of scaling issues and performance bottlenecks. You begin the process of evaluating the benefits and tradeoffs of possible solutions. At last, you come to a decision. It's time to migrate parts of the system to individual services. So, how do we approach this migration from monolith to microservices?]]></description>
            <content:encoded><![CDATA[<p>You start building a beautiful monolith system.
Maybe a <a href="what-is-a-modular-monolith"><strong>modular monolith</strong></a>.</p>
<p>The system grows over time, and requirements are ever-changing. Slowly, cracks begin to appear in the system.</p>
<p>This could be for organizational reasons and distributing the work across a team.
Or it could be because of scaling issues and performance bottlenecks.</p>
<p>You begin evaluating the possible solutions, and the benefits and tradeoffs of each one.
At last, you come to a decision.</p>
<p>It's time to migrate parts of the system to individual (micro)services.</p>
<p>So, how do we approach this migration from monolith to microservices?</p>
<p>That is the topic of this week's newsletter.</p>
<p>Let's dive in!</p>
<h2>Decoupling Using Bounded Contexts</h2>
<p>The first step in moving from a monolith to microservices is identifying the bounded contexts.
Because they represent cohesive parts of the domain that are candidates for extraction.</p>
<p>One solution is to identify <a href="https://martinfowler.com/bliki/BoundedContext.html">bounded contexts</a>
using the domain-driven design strategic modeling.</p>
<p>Bounded contexts define the explicit boundaries between modules and separate the responsibilities.
This is one of the biggest challenges when migrating to microservices.
<a href="https://learn.microsoft.com/en-us/azure/architecture/microservices/model/domain-analysis">Identifying good boundaries</a>
between modules ensures microservices are narrowly focused on one problem domain.</p>
<p>Defining boundaries is also easier in a monolith because you aren't working with a distributed system.
<a href="https://cloud.google.com/architecture/microservices-architecture-refactoring-monoliths">Refactoring bad boundaries</a>
is less risky, and you have more freedom to &quot;get it right&quot;.</p>
<p><img src="/blogs/mnw_056/bounded_contexts.png" alt="Bounded contexts."></p>
<p>And the size of the bounded context shouldn't worry you.
Instead, focus on <a href="https://go.particular.net/right-sized-services">service boundaries.</a></p>
<p>The next problem you need to solve is coupling.
Coupling is manifested in two ways:</p>
<ul>
<li>Database dependencies</li>
<li>Communication between modules</li>
</ul>
<p>You can solve these problems from the start by building a modular monolith.
But I'll also explain the guiding principles you can use to solve coupling.</p>
<h2>How a Modular Monolith Solves Coupling</h2>
<p>A <a href="what-is-a-modular-monolith">modular monolith</a> is a catchy name for a monolith system built from a few
bounded contexts (modules) and following a set of principles to control coupling.
Each module contains a cohesive set of functionalities and is isolated from other modules in the system.
The isolation refers to database dependencies and inter-module communication.</p>
<p><img src="/blogs/mnw_056/modular_monolith.png" alt="Modular monolith."></p>
<p>You can think of a module as a distinct application within the system.
A module has its own domain, entities, use cases, database tables.
The modules are deployed together as a single executable application.
But they are otherwise independent.</p>
<p>You can apply different architectural approaches to each module, like <a href="/pragmatic-clean-architecture"><strong>Clean Architecture</strong></a>.</p>
<p>So I mentioned that you need to reduce the coupling between modules.</p>
<p>Here are two principles to solve database coupling:</p>
<ul>
<li>Modules can't share tables in the database</li>
<li>Modules can't directly query the database tables of other modules</li>
</ul>
<p>Sharing database tables leads to a high degree of coupling, and this is exactly what you are trying to avoid.
You can isolate the data for each module on a logical level using schemas or physically with different databases.</p>
<p>A module should expose a public API that other modules can call.
This public API is the entry point into the module.
And this is the only way for modules to communicate.</p>
<p>Communication between modules can be <a href="modular-monolith-communication-patterns#synchronous-communication-with-method-calls"><strong>synchronous</strong></a>
using method calls, or <a href="modular-monolith-communication-patterns#asynchronous-communication-with-messaging"><strong>asynchronous</strong></a>
using a message bus.</p>
<p>My preferred approach is asynchronous communication using messaging.
It's loosely coupled and makes the transition to microservices easier.</p>
<h2>Adding a Message Broker To The System</h2>
<p>To implement asynchronous communication between modules, you can introduce a message broker.
But you don't need to introduce a full-blown message broker from the start.</p>
<p>You can implement messaging between modules using an abstraction like <a href="https://masstransit.io">MassTransit</a> while abstracting away the transport mechanism.</p>
<p>MassTransit has an in-memory transport that works well inside a single process.
It's very fast.
But it isn't durable, and you can lose messages if the bus is stopped.</p>
<p>You only need to configure a different transport mechanism when introducing a real message broker.
But you don't need to change your messaging code.</p>
<p><img src="/blogs/mnw_056/modular_monolith_queue.png" alt="Modular monolith with a queue."></p>
<p>What is the purpose of messaging inside a modular monolith?</p>
<p>Designing your system like this makes the modules loosely coupled and independent.
The price you pay in increased complexity at the start is justified as the project matures.</p>
<h2>Extracting Modules to Microservices</h2>
<p>We decided to move from a monolith system to microservices.
Since we built our system in a modular way, the migration comes down to extracting a module into a new process.</p>
<p>You should introduce a <a href="implementing-an-api-gateway-for-microservices-with-yarp"><strong>reverse proxy</strong></a> in front of your services to route incoming traffic.
This will hide the implementation details of the microservices system from client applications.</p>
<p>The new microservice needs to connect to the message bus, but we don't need to change anything in our code.
Using messaging for communication between modules simplifies the migration process.
This might remind you of <a href="https://go.particular.net/break-that-big-ball-of-mud">event-driven architecture</a>.</p>
<p>If you implement inter-module communication using method calls, you must replace that implementation with HTTP calls over the network.
Because you're now building a distributed system, and the previous implementation using method calls will not work.
You also need to consider authentication, fault tolerance...</p>
<p><img src="/blogs/mnw_056/extracting_modules.png" alt="Microservices with extracting modules."></p>
<p><a href="breaking-it-down-how-to-migrate-your-modular-monolith-to-microservices"><strong>Extracting modules</strong></a> from the monolith system leads to replacing all the functionalities of the old system with new microservices.
This process of migrating to microservices follows the <a href="https://learn.microsoft.com/en-us/azure/architecture/patterns/strangler-fig">strangler fig pattern.</a></p>
<h2>Closing Thoughts</h2>
<p>The biggest blocker for moving from a monolith to microservices is coupling.
Coupling is a change preventer.
So, this is the first thing you need to tackle.</p>
<p>You need to solve coupling at the database level and between components in the code.
Building the system in a modular way can prevent these problems from the start.</p>
<p>Which is why a Modular monolith is an excellent approach.</p>
<p>You can identify bounded contexts in the system and use them as the boundaries in the monolith.
And getting the boundaries right is easier in a monolith.</p>
<p><a href="breaking-it-down-how-to-migrate-your-modular-monolith-to-microservices"><strong>Migrating to microservices</strong></a> comes down to extracting the modules into individual services.</p>
<p>Of course, you still need to think about security and fault tolerance because you now have a distributed system.</p>
<p>Talking about architecture in abstract terms can be difficult to grasp, but it's important when discussing conceptual solutions.</p>
<p>I'll show you a practical Modular monolith implementation soon to complete the circle.</p>
<p>Until then, I hope this was valuable.</p>
<p>See you next week!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_056.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Feature Flags in .NET and How I Use Them for A/B Testing]]></title>
            <link>https://milanjovanovic.tech/blog/feature-flags-in-dotnet-and-how-i-use-them-for-ab-testing</link>
            <guid isPermaLink="false">feature-flags-in-dotnet-and-how-i-use-them-for-ab-testing</guid>
            <pubDate>Sat, 16 Sep 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[The ability to conditionally turn features on or off in your application without redeploying the code is a powerful tool.
It lets you quickly iterate on new features and frequently integrate your changes with the main branch.
You can use feature flags to achieve this.
Feature flags are a software development technique that allows you to wrap application features in a conditional statement. You can then toggle the feature on or off in runtime to control which features are enabled.]]></description>
            <content:encoded><![CDATA[<p>The ability to conditionally turn features on or off in your application without redeploying the code is a powerful tool.</p>
<p>It lets you quickly iterate on new features and frequently integrate your changes with the main branch.</p>
<p>You can use <strong>feature flags</strong> to achieve this.</p>
<p><strong>Feature flags</strong> are a software development technique that allows you to wrap application features in a conditional statement.
You can then toggle the feature on or off in runtime to control which features are enabled.</p>
<p>We have a lot to cover in this week's newsletter:</p>
<ul>
<li>Feature flag fundamentals in .NET</li>
<li>Feature filters and phased rollouts</li>
<li>Trunk-based development</li>
<li>A/B testing</li>
</ul>
<p>Let's dive in!</p>
<h2>Feature Flags In .NET</h2>
<p><a href="https://github.com/microsoft/FeatureManagement-Dotnet">Feature flags</a> provide a way for .NET and <a href="http://ASP.NET">ASP.NET</a> Core applications to turn features on or off dynamically.</p>
<p>To get started, you need to install the <code>Microsoft.FeatureManagement</code> library in your project:</p>
<pre><code class="language-powershell">Install-Package Microsoft.FeatureManagement
</code></pre>
<p>This library will allow you to develop and expose application functionality based on features.
It's useful when you have special requirements when a new feature should be enabled and under what conditions.</p>
<p>The next step is to register the required services with dependency injection by calling <code>AddFeatureManagement</code>:</p>
<pre><code class="language-csharp">builder.Services.AddFeatureManagement();
</code></pre>
<p>And you are ready to create your first feature flag.
Feature flags are built on top of the .NET configuration system.
Any .NET configuration provider can act as the backbone for exposing feature flags.</p>
<p>Let's create a feature flag called <code>ClipArticleContent</code> in our <code>appsettings.json</code> file:</p>
<pre><code class="language-json">&quot;FeatureManagement&quot;: {
  &quot;ClipArticleContent&quot;: false
}
</code></pre>
<p>By convention, feature flags have to be defined in the <code>FeatureManagement</code> configuration section.
But you can change this by providing a different configuration section when calling <code>AddFeatureManagement</code>.</p>
<p>Microsoft recommends exposing feature flags using enums and then consuming them with the <code>nameof</code> operator.
For example, you would write <code>nameof(FeatureFlags.ClipArticleContent)</code>.</p>
<p>However, I prefer defining feature flags as constants in a static class because it simplifies the usage.</p>
<pre><code class="language-csharp">// Using enums
public enum FeatureFlags
{
    ClipArticleContent = 1
}

// Using constants
public static class FeatureFlags
{
    public const string ClipArticleContent = &quot;ClipArticleContent&quot;;
}
</code></pre>
<p>To check the feature flag state, you can use the <code>IFeatureManager</code> service.
In this example, if the <code>ClipArticleContent</code> feature flag is turned on, we will return only the first thirty characters of the article's content.</p>
<pre><code class="language-csharp">app.MapGet(&quot;articles/{id}&quot;, async (
    Guid id,
    IGetArticle query,
    IFeatureManager featureManager) =&gt;
{
    var article = query.Execute(id);

    if (await featureManager.IsEnabledAsync(FeatureFlags.ClipArticleContent))
    {
        article.Content = article.Content.Substring(0, 50);
    }

    return Results.Ok(article);
});
</code></pre>
<p>You can also apply feature flags on a controller or endpoint level using the <code>FeatureGate</code> attribute:</p>
<pre><code class="language-csharp">[FeatureGate(FeatureFlags.EnableArticlesApi)]
public class ArticlesController : Controller
{
   // ...
}
</code></pre>
<p>This covers the fundamentals of using feature flags, and now, let's tackle more advanced topics.</p>
<h2>Feature Filters And Phased Rollouts</h2>
<p>The feature flags I showed you in the previous section were like a simple on-off switch.
Although practical, you might want more flexibility from your feature flags.</p>
<p>The <code>Microsoft.FeatureManagement</code> package comes with a few built-in feature filters that allow you to create dynamic rules for enabling feature flags.</p>
<p>The available feature filters are
<a href="https://learn.microsoft.com/en-us/dotnet/api/microsoft.featuremanagement.featurefilters.percentagefilter?view=azure-dotnet"><code>Microsoft.Percentage</code></a>,
<a href="https://learn.microsoft.com/en-us/dotnet/api/microsoft.featuremanagement.featurefilters.timewindowfilter?view=azure-dotnet"><code>Microsoft.TimeWindow</code></a>
and <a href="https://learn.microsoft.com/en-us/dotnet/api/microsoft.featuremanagement.featurefilters.targetingfilter?view=azure-dotnet"><code>Microsoft.Targeting</code></a>.</p>
<p>Here's an example of defining a <code>ShowArticlePreview</code> feature flag that uses a percentage filter:</p>
<pre><code class="language-json">&quot;FeatureFlags&quot;: {
  &quot;ClipArticleContent&quot;: false,
  &quot;ShowArticlePreview&quot;: {
    &quot;EnabledFor&quot;: [
      {
        &quot;Name&quot;: &quot;Percentage&quot;,
        &quot;Parameters&quot;: {
          &quot;Value&quot;: 50
        }
      }
    ]
  }
}
</code></pre>
<p>This means the feature flag will be randomly turned on 50% of the time.
The downside is the same user might see different behavior on subsequent requests.
A more realistic scenario is to have the feature flag state be cached for the duration of the user's session.</p>
<p>To use the <code>PercentageFilter</code>, you need to enable it by calling <code>AddFeatureFilter</code>:</p>
<pre><code class="language-csharp">builder.Services.AddFeatureManagement().AddFeatureFilter&lt;PercentageFilter&gt;();
</code></pre>
<p>Another interesting feature filter is the <code>TargetingFilter</code>, which allows you to target specific users.
Targeting is used in phased rollouts, where you want to introduce a new feature to your users gradually.
You start by enabling the feature for a small percentage of users and slowly increase the rollout percentage while monitoring how the system responds.</p>
<h2>Trunk-based Development and Feature Flags</h2>
<p><a href="https://trunkbaseddevelopment.com/">Trunk-based development</a> is a Git branching strategy where all developers work in short-lived branches or directly in the trunk, the main codebase.
The <em>&quot;trunk&quot;</em> is the main branch of your repository.
If you're using Git, it will be either the <code>main</code> or <code>master</code> branch.
Trunk-based development avoids the &quot;merge hell&quot; problem caused by long-lived branches.</p>
<p>So, how do feature flags fit into trunk-based development?</p>
<p>The only way to ensure the trunk is always releasable is to hide incomplete features behind feature flags.
You continue pushing changes to the trunk as you work on the feature while the feature flag remains turned off on the main branch.
When the feature is complete, you turn on the feature flag and release it to production.</p>
<p><img src="/blogs/mnw_055/trunk_based_development.png" alt="Trunk based development."></p>
<h2>How I Used Feature Flags for A/B Testing On My Website</h2>
<p>A/B testing (split testing) is an experiment where two or more variants of a page (or feature) are shown randomly to users.
Statistical analysis is performed in the background to determine which variation performs better for a given conversion goal.</p>
<p>Here's an example A/B test I performed on my website:</p>
<p><img src="/blogs/mnw_055/split_test.png" alt="Split test with two variants."></p>
<p>The hypothesis was that removing the image and focusing on the benefits would make more people want to subscribe.
I measure this using the conversion rate, which is the number of people visiting the page divided by the number of people subscribing.</p>
<p>I'm using a platform called <a href="https://posthog.com">Posthog</a> to run experiments, which automatically calculates the results.</p>
<p>You can see that the <em>test</em> variant has a significantly higher conversion rate, so it becomes the winner of the A/B test.</p>
<p><img src="/blogs/mnw_055/experiment_results.png" alt="Split test with two variants experiment results."></p>
<h2>Takeaway</h2>
<p>The ability to dynamically turn features on or off without deploying the code is like a superpower.
Feature flags give you this ability with very little work.</p>
<p>You can work with feature flags in .NET by installing the <code>Microsoft.FeatureManagement</code> library.
Feature flags build on top of the .NET configuration system, and you can check the feature flag state using the <code>IFeatureManager</code> service.</p>
<p>Another use case for feature flags is A/B testing.
I run weekly experiments on my website, testing changes that will improve my conversion rate.
Feature flags help me decide which version of the website to show to the user.
And then, I can measure results based on the user's actions.</p>
<p>I also made a video about <a href="https://youtu.be/QVEUgIC7Wpo"><strong>feature flagging in .NET,</strong></a> and you can watch it <a href="https://youtu.be/QVEUgIC7Wpo"><strong>here</strong></a> if you want to learn more.</p>
<p>Hope this was valuable.</p>
<p>Stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_055.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Solving Race Conditions With EF Core Optimistic Locking]]></title>
            <link>https://milanjovanovic.tech/blog/solving-race-conditions-with-ef-core-optimistic-locking</link>
            <guid isPermaLink="false">solving-race-conditions-with-ef-core-optimistic-locking</guid>
            <pubDate>Sat, 09 Sep 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[How often do you think about concurrency conflicts when writing code?
You write the code for a new feature, confirm that it works, and call it a day.
But one week later, you find out you introduced a nasty bug because you didn't think about concurrency.
The most common issue is race conditions with two competing threads executing the same function. If you don't consider this during development, you introduce the risk of leaving the system in a corrupted state.
In this week's newsletter, I'll challenge you to spot the race condition in a method for reserving a booking. The business requirement is you can't have two overlapping reservations for the same dates.
And then, I'll show you how to solve this race condition using EF Core optimistic concurrency.
Let's dive in!]]></description>
            <content:encoded><![CDATA[<p>How often do you think about concurrency conflicts when writing code?</p>
<p>You write the code for a new feature, confirm that it works, and call it a day.</p>
<p>But one week later, you find out you introduced a nasty bug because you didn't think about concurrency.</p>
<p>The most common issue is race conditions with two competing threads executing the same function.
If you don't consider this during development, you introduce the risk of leaving the system in a corrupted state.</p>
<p>In this week's newsletter, I'll challenge you to spot the race condition in a method for reserving a booking.
The business requirement is you can't have two overlapping reservations for the same dates.</p>
<p>And then, I'll show you how to solve this race condition using EF Core optimistic concurrency.</p>
<p>Let's dive in!</p>
<h2>What's Wrong With This Code?</h2>
<p>There's a race condition hiding somewhere in this code snippet.</p>
<p>Can you see it?</p>
<pre><code class="language-csharp">public Result&lt;Guid&gt; Handle(
    ReserveBooking command,
    AppDbContext dbContext)
{
    var user = dbContext.Users.GetById(command.UserId);
    var apartment = dbContext.Apartments.GetById(command.ApartmentId);
    var (startDate, endDate) = command;

    if (dbContext.Bookings.IsOverlapping(apartment, startDate, endDate))
    {
        return Result.Failure&lt;Guid&gt;(BookingErrors.Overlap);
    }

    var booking = Booking.Reserve(apartment, user, startDate, endDate);

    dbContext.Add(booking);

    dbContext.SaveChanges();

    return booking.Id;
}
</code></pre>
<p>The call to <code>IsOverlapping</code> is an optimistic check to see if there's an existing booking for the specified dates.</p>
<pre><code class="language-csharp">if (dbContext.Bookings.IsOverlapping(apartment, startDate, endDate)) { }
</code></pre>
<p>If it returns <code>true</code>, we're trying to double-book the apartment.
So we return a failure, and the method completes.</p>
<p>But if it returns <code>false</code>, we reserve a booking and call <code>SaveChanges</code> to persist the changes in the database.</p>
<p>And there lies the problem.</p>
<p>There's a chance for a concurrent request to pass the <code>IsOverlapping</code> check and attempt to reserve the booking.
Without any concurrency control, both requests will succeed, and we will end up with an inconsistent state in the database.</p>
<p>So how can we solve this?</p>
<h2>Optimistic Concurrency With EF Core</h2>
<p>The pessimistic concurrency approach acquires a lock for the data before modifying it.
It's slower and causes competing transactions to be blocked until the lock is released.
EF Core doesn't support this approach out of the box.</p>
<p>You can also solve this problem using optimistic concurrency with EF Core.
It doesn't take any locks, but any data modifications will fail to save if the data has changed since it was queried.</p>
<p>To implement optimistic concurrency in EF Core, you need to configure a property as a <em>concurrency token</em>.
It's loaded and tracked with the entity.
When you call <code>SaveChanges</code>, EF Core will compare the value of the concurrency token to the value in the database.</p>
<p>Let's assume we're using SQL Server, which has a native <a href="https://learn.microsoft.com/en-us/sql/t-sql/data-types/rowversion-transact-sql?view=sql-server-ver16"><code>rowversion</code></a> column.
The <code>rowversion</code> automatically changes when the row is updated, so it's a great option for a concurrency token.</p>
<p>To configure a <code>byte[]</code> property as a concurrency token you can decorate it with the <code>Timestamp</code> attribute.
It will be mapped to a <code>rowversion</code> column in SQL Server.</p>
<pre><code class="language-csharp">public class Apartment
{
    public Guid Id { get; set; }

    [Timestamp]
    public byte[] Version { get; set; }
}
</code></pre>
<p>I prefer a different approach because attributes pollute the entity.</p>
<p>You can do the same with the Fluent API.
I will even use a shadow property to hide the concurrency token from the entity class.</p>
<pre><code class="language-csharp">protected override void OnModelCreating(ModelBuilder modelBuilder)
{
    modelBuilder.Entity&lt;Apartment&gt;()
        .Property&lt;byte[]&gt;(&quot;Version&quot;)
        .IsRowVersion();
}
</code></pre>
<p>The exact configuration will differ based on the database you are using, so check the documentation.</p>
<h2>How Optimistic Concurrency Works In Practice</h2>
<p>So here's what changes when we configure the concurrency token.</p>
<p>When loading the <code>Apartment</code> entity, EF will also load the concurrency token.</p>
<pre><code class="language-sql">SELECT a.Id, a.Version
FROM Apartments a
WHERE a.Id = @p0
</code></pre>
<p>And when we call <code>SaveChanges</code>, the update statement will compare the concurrency token value with the one in the database:</p>
<pre><code class="language-sql">UPDATE Apartments a
SET a.LastBookedOnUtc = @p0
WHERE a.Id = @p1 AND a.Version = @p2;
</code></pre>
<p>If the <code>rowversion</code> in the database changes, the number of updated rows will be <code>0</code>.</p>
<p>EF Core expects to update <code>1</code> row, so it will throw a <code>DbUpdateConcurrencyException</code>, which you need to handle.</p>
<h2>Handling Concurrency Exceptions</h2>
<p>Now that you know how to use optimistic concurrency with EF Core, you can fix the previous code snippet.</p>
<p>If two concurrent requests pass the <code>IsOverlapping</code> check, only one can complete the <code>SaveChanges</code> call.
The other concurrent request will run into a <code>Version</code> mismatch in the database and throw a <code>DbUpdateConcurrencyException</code>.</p>
<p>In case of a concurrency conflict, we need to add a <code>try-catch</code> statement to catch the <code>DbUpdateConcurrencyException</code>.
How you handle the actual exception depends on your business requirements.
And sometimes, <a href="https://go.particular.net/milanjovanovic/raceconditions">race conditions</a> might not even exist.</p>
<pre><code class="language-csharp">public Result&lt;Guid&gt; Handle(
    ReserveBooking command,
    AppDbContext dbContext)
{
    var user = dbContext.Users.GetById(command.UserId);
    var apartment = dbContext.Apartments.GetById(command.ApartmentId);
    var (startDate, endDate) = command;

    if (dbContext.Bookings.IsOverlapping(apartment, startDate, endDate))
    {
        return Result.Failure&lt;Guid&gt;(BookingErrors.Overlap);
    }

    try
    {
        var booking = Booking.Reserve(apartment, user, startDate, endDate);

        dbContext.Add(booking);

        dbContext.SaveChanges();

        return booking.Id;
    }
    catch (DbUpdateConcurrencyException)
    {
        return Result.Failure&lt;Guid&gt;(BookingErrors.Overlap);
    }
}
</code></pre>
<p>If you're wondering how will this even work, here's the missing piece.
The <code>Booking.Reserve</code> method will update the <code>LastBookedOnUtc</code> property of the <code>Apartment</code> entity.</p>
<pre><code class="language-csharp">public static Booking Reserve(Apartment apartment, User user, DateTime startDate, DateTime endDate)
{
    apartment.LastBookedOnUtc = DateTime.UtcNow;

    return new Booking
    {
        Id = Guid.NewGuid(),
        Apartment = apartment,
        User = user,
        StartDate = startDate,
        EndDate = endDate
    };
}
</code></pre>
<p>When we call <code>SaveChanges</code>, this will cause an update to the <code>Apartment</code> entity, which will include the concurrency token check.
This ensures that if two requests try to update the same entity simultaneously, one will succeed while the other will fail with a <code>DbUpdateConcurrencyException</code>.</p>
<h2>When Should You Use Optimistic Concurrency?</h2>
<p>Optimistic concurrency considers the best scenario is also the most probable one.
It assumes conflicts between transactions will be infrequent and doesn't acquire locks on the data.
This means your system can scale better because there is no blocking slowing down performance.</p>
<p>However, you must still expect concurrency conflicts and implement custom logic to handle them.</p>
<p>Optimistic concurrency is a good choice if your application doesn't expect many conflicts.</p>
<p>Another reason to use optimistic concurrency is when you can't hold an open connection to the database for the length of the transaction.
This is required for pessimistic locking.</p>
<p>Hope this was helpful.</p>
<p>I'll see you next week!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_054.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Testcontainers - Integration Testing Using Docker In .NET]]></title>
            <link>https://milanjovanovic.tech/blog/testcontainers-integration-testing-using-docker-in-dotnet</link>
            <guid isPermaLink="false">testcontainers-integration-testing-using-docker-in-dotnet</guid>
            <pubDate>Sat, 02 Sep 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Modern software applications rarely work in isolation. On the contrary, a typical application will talk to several external systems like databases, messaging systems, cache providers, and many 3rd party services. And it's up to you to ensure everything functions correctly.
Hopefully, I don't have to convince you about the value of writing tests.
You should be writing tests. Period.
However, I do want to discuss the value of integration testing.
Unit tests are helpful to test business logic in isolation, without any external services. They are easy to write and provide almost instant feedback.
But you can't be fully confident in your application without integration tests.
So, in this week's newsletter, I'll show you how to use Docker for integration testing.
Here's what we will use to write integration tests:
- Testcontainers - Docker - xUnit
Let's dive in!]]></description>
            <content:encoded><![CDATA[<p>Modern software applications rarely work in isolation.
On the contrary, a typical application will talk to several external systems like databases, messaging systems, cache providers, and many 3rd party services.
And it's up to you to ensure everything functions correctly.</p>
<p>Hopefully, I don't have to convince you about the value of writing tests.</p>
<p>You should be writing tests.
Period.</p>
<p>However, I do want to discuss the <em>value</em> of <strong>integration testing</strong>.</p>
<p><strong>Unit tests</strong> are helpful to test business logic in isolation, without any external services.
They are easy to write and provide almost instant feedback.</p>
<p>But you can't be fully confident in your application without <strong>integration tests</strong>.</p>
<p>So, in this week's newsletter, I'll show you how to use <strong>Docker</strong> for integration testing.</p>
<p>Here's what we will use to write <strong>integration tests</strong>:</p>
<ul>
<li><strong>Testcontainers</strong></li>
<li>Docker</li>
<li><a href="creating-data-driven-tests-with-xunit"><strong>xUnit</strong></a></li>
</ul>
<p>Let's dive in!</p>
<h2>What is Testcontainers?</h2>
<p><a href="https://dotnet.testcontainers.org/">Testcontainers</a> is a library for writing tests with throwaway Docker containers.</p>
<p>Why should you use it?</p>
<p>Integration testing is considered &quot;difficult&quot; because you have to maintain testing infrastructure.
Before running tests, you need to make sure the database is up and running.
You also have to seed any data required for the tests.
If you have tests running in parallel on the same database, they could interfere with each other.</p>
<p>A possible solution could be using in-memory variations of the required services.
But this isn't much different from using mocks.
In-memory services might not have all the features of the production service.</p>
<p>Testcontainers solves this by using Docker to spin up real services for integration testing.</p>
<p>Here's an example of creating a <strong>SQL Server</strong> container:</p>
<pre><code class="language-csharp">MsSqlContainer dbContainer = new MsSqlBuilder()
    .WithImage(&quot;mcr.microsoft.com/mssql/server:2022-latest&quot;)
    .WithPassword(&quot;Strong_password_123!&quot;)
    .Build();
</code></pre>
<p>You can then use the <code>MsSqlContainer</code> instance to get a connection string for the database running inside the container.</p>
<p>Do you see how this is valuable for writing integration tests?</p>
<p>No more need for mocks or fake in-memory databases.
Instead, you can use the real deal.</p>
<p>I won't do a deep dive into this library here, so refer to the documentation for more information.</p>
<h2>Implementing a Custom WebApplicationFactory</h2>
<p><strong><a href="http://ASP.NET">ASP.NET</a> Core</strong> provides an in-memory test server that we can use to spin up an application instance for running tests.
The <code>Microsoft.AspNetCore.Mvc.Testing</code> package provides the <code>WebApplicationFactory</code> class that we will use as the base for our implementation.</p>
<p><code>WebApplicationFactory&lt;TEntryPoint&gt;</code> is used to create a <code>TestServer</code> for the integration tests.</p>
<p>The custom <code>IntegrationTestWebAppFactory</code> will do a few things:</p>
<ul>
<li>Create and configure a <code>MsSqlContainer</code> instance</li>
<li>Call <code>ConfigureTestServices</code> to set up EF Core with the container database</li>
<li>Start and stop the container instance with <code>IAsyncLifetime</code></li>
</ul>
<p><code>MsSqlContainer</code> has a <code>GetConnectionString</code> method to grab the connection string for the current container.
Note that this can change between tests, as each test class will create a separate container instance.
Test cases inside the same test class will use the same container instance.
So keep that in mind if you need to do a cleanup between tests.</p>
<p>Another thing to keep in mind is <strong>database migrations</strong>.
You will have to run them manually before every test to create the required database structure.</p>
<p>Starting the container instance is done asynchronously using <code>IAsyncLifetime</code>.
The container is started inside <code>StartAsync</code> before any of the tests run.
And it's stopped inside <code>StopAsync</code>.</p>
<p>Here's the complete code for <code>IntegrationTestWebAppFactory</code>:</p>
<pre><code class="language-csharp">public class IntegrationTestWebAppFactory
    : WebApplicationFactory&lt;Program&gt;,
      IAsyncLifetime
{
    private readonly MsSqlContainer _dbContainer = new MsSqlBuilder()
        .WithImage(&quot;mcr.microsoft.com/mssql/server:2022-latest&quot;)
        .WithPassword(&quot;Strong_password_123!&quot;)
        .Build();

    protected override void ConfigureWebHost(IWebHostBuilder builder)
    {
        builder.ConfigureTestServices(services =&gt;
        {
            var descriptorType =
                typeof(DbContextOptions&lt;ApplicationDbContext&gt;);

            var descriptor = services
                .SingleOrDefault(s =&gt; s.ServiceType == descriptorType);

            if (descriptor is not null)
            {
                services.Remove(descriptor);
            }

            services.AddDbContext&lt;ApplicationDbContext&gt;(options =&gt;
                options.UseSqlServer(_dbContainer.GetConnectionString()));
        });
    }

    public Task InitializeAsync()
    {
        return _dbContainer.StartAsync();
    }

    public new Task DisposeAsync()
    {
        return _dbContainer.StopAsync();
    }
}
</code></pre>
<h2>Creating The Base Test Class</h2>
<p>The base test class will implement a class fixture interface <code>IClassFixture</code>.
It indicates the class contains tests and provides shared object instances across the test cases inside.
This is a good place to instantiate any services that are required for most tests.</p>
<p>For example, I'm creating an <code>IServiceScope</code> for resolving scoped services inside the tests.</p>
<ul>
<li><code>ISender</code> for sending commands and queries</li>
<li><code>ApplicationDbContext</code> for database setup or verifying results</li>
</ul>
<pre><code class="language-csharp">public abstract class BaseIntegrationTest
    : IClassFixture&lt;IntegrationTestWebAppFactory&gt;,
      IDisposable
{
    private readonly IServiceScope _scope;
    protected readonly ISender Sender;
    protected readonly ApplicationDbContext DbContext;

    protected BaseIntegrationTest(IntegrationTestWebAppFactory factory)
    {
        _scope = factory.Services.CreateScope();

        Sender = _scope.ServiceProvider.GetRequiredService&lt;ISender&gt;();

        DbContext = _scope.ServiceProvider
            .GetRequiredService&lt;ApplicationDbContext&gt;();
    }

    public void Dispose()
    {
        _scope?.Dispose();
        DbContext?.Dispose();
    }
}
</code></pre>
<p>With all the infrastructure in place, we're finally ready to write the tests.</p>
<h2>Putting It All Together - Writing Integration Tests</h2>
<p>Here's a <code>ProductTests</code> class with an integration test inside.</p>
<p>I use the <em>Arrange-Act-Assert</em> pattern to structure tests:</p>
<ul>
<li><em>Arrange</em> - create the <code>CreateProduct.Command</code> instance</li>
<li><em>Act</em> - send the command using <code>ISender</code> and store the result</li>
<li><em>Assert</em> - use the result from the <em>Act</em> step to verify the database state</li>
</ul>
<p>The value of writing integration tests like this is that you can use the complete <strong>MediatR</strong> request pipeline.
If you have any <code>IPipelineBehavior</code> wrapping the request, it will also be executed.</p>
<p>The same applies if you write your business logic inside service classes.
Instead of resolving the <code>ISender</code>, you would resolve the specific services you want to test.</p>
<p>Most importantly, this test uses a real database instance running inside a <strong>Docker container</strong>.</p>
<pre><code class="language-csharp">public class ProductTests : BaseIntegrationTest
{
    public ProductTests(IntegrationTestWebAppFactory factory)
        : base(factory)
    {
    }

    [Fact]
    public async Task Create_ShouldCreateProduct()
    {
        // Arrange
        var command = new CreateProduct.Command
        {
            Name = &quot;AMD Ryzen 7 7700X&quot;,
            Category = &quot;CPU&quot;,
            Price = 223.99m
        };

        // Act
        var productId = await Sender.Send(command);

        // Assert
        var product = DbContext
            .Products
            .FirstOrDefault(p =&gt; p.Id == productId);

        Assert.NotNull(product);
    }
}
</code></pre>
<h2>Running Integration Tests In CI/CD Pipelines</h2>
<p>You can also run integration tests with <strong>Testcontainers</strong> inside <strong>CI/CD pipelines</strong>.
The only requirement is that it supports Docker.</p>
<p><strong>GitHub Actions</strong> does support Docker.
If you are hosting your project there, integration tests will work out of the box.</p>
<p>You can learn more about <a href="how-to-build-ci-cd-pipeline-with-github-actions-and-dotnet"><strong>building a CI/CD pipeline with GitHub Actions here.</strong></a></p>
<p>And if you want a plug-in solution, here's a GitHub Actions workflow you can use:</p>
<pre><code class="language-yaml">name: Run Tests 🚀

on:
  workflow_dispatch:
  push:
    branches:
      - main

jobs:
  run-tests:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Setup .NET
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: '7.0.x'

      - name: Restore
        run: dotnet restore ./Products.Api.sln

      - name: Build
        run: dotnet build ./Products.Api.sln --no-restore

      - name: Test
        run: dotnet test ./Products.Api.sln --no-build
</code></pre>
<h2>Takeaway</h2>
<p><strong>Testcontainers</strong> is an excellent solution for writing <strong>integration tests</strong> with Docker.
You can spin up and configure any <strong>Docker</strong> image and use it from your application.
This is far better than using mocks or in-memory variations because they lack many features.</p>
<p>If you have a CI/CD pipeline that supports Docker, Testcontainers will work out of the box.</p>
<p>Only a few integration tests will drastically improve your confidence in the system.</p>
<p>You can grab the <a href="https://github.com/m-jovanovic/testcontainers-sample"><strong>source code for this newsletter</strong></a> on my GitHub.<br>
It's completely free, so what are you waiting for?</p>
<p>And if you prefer video, here's a quick tutorial on <a href="https://youtu.be/tj5ZCtvgXKY"><strong>integration testing with Testcontainers.</strong></a></p>
<p>Hope this was valuable.</p>
<p>Stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_053.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Orchestration vs Choreography]]></title>
            <link>https://milanjovanovic.tech/blog/orchestration-vs-choreography</link>
            <guid isPermaLink="false">orchestration-vs-choreography</guid>
            <pubDate>Sat, 26 Aug 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[More than 63%+ of organizations said in a Dzone survey that they are adopting Microservices for some or all of their applications.
As more businesses adopt the use of Microservice architectures, we as developers have to become more skilled with Microservices communication.
Working with distributed systems is both fun and challenging at the same time. One of those challenges is designing effective communication between services.
More centralization or less centralization? More coupling or less coupling? More control or less control?
These are only a few questions you need to answer.
In this week's newsletter, we will:
- Break down Orchestration vs. Choreography - Understand key differences and tradeoffs between them - Create a framework for deciding which approach to use
Let's dive in!]]></description>
            <content:encoded><![CDATA[<p>More than 63%+ of organizations said in a <a href="https://dzone.com/articles/new-research-shows-63-percent-of-enterprises-are-a">Dzone survey</a>
that they are adopting <strong>Microservices</strong> for some or all of their applications.</p>
<p>As more businesses adopt the use of Microservice architectures, we as developers have to become more skilled with Microservices communication.</p>
<p>Working with <strong>distributed systems</strong> is both fun and challenging at the same time.
One of those challenges is designing <strong>effective communication</strong> between services.</p>
<p>More centralization or less centralization?
More coupling or less coupling?
More control or less control?</p>
<p>These are only a few questions you need to answer.</p>
<p>In this week's newsletter, we will:</p>
<ul>
<li>Break down <strong>Orchestration vs. Choreography</strong></li>
<li>Understand key differences and <strong>tradeoffs</strong> between them</li>
<li>Create a framework for deciding which approach to use</li>
</ul>
<p>Let's dive in!</p>
<h2>What Are Microservices?</h2>
<p><strong>Microservices</strong> are a <strong>software architecture</strong> style where an application is built from small, autonomous services.
Each microservice serves a distinct purpose and can be independently deployed.</p>
<p>You probably already know this, so this is a quick refresher.</p>
<p><img src="/blogs/mnw_052/microservices_hell.png" alt=""></p>
<p>Here are the key tenets of Microservices:</p>
<ul>
<li>Independent development</li>
<li>Deployment independence</li>
<li>Technological freedom</li>
<li>Scalability</li>
<li>Resilience</li>
</ul>
<p>A significant challenge is designing effective inter-service communication within this distributed environment.</p>
<p>Inside a Monolith system, communication happens through direct method calls.
This is a straightforward approach that works well when all components live within a single process.
However, this doesn't work with microservices.</p>
<h2>Orchestration - Command-driven Communication</h2>
<p><strong>Orchestration</strong> is a centralized approach to Microservices communication.
One of the services takes on the role of the <strong>orchestrator</strong> and coordinates the communication between services.</p>
<p>Orchestration uses <strong>command-driven</strong> communication.
The command communicates the intent of the action.
The sender wants something to happen, and the recipient doesn't need to know who sent the command.</p>
<p>An example of orchestration can be a <a href="implementing-the-saga-pattern-with-rebus-and-rabbitmq"><strong>Saga implemented with RabbitMQ.</strong></a></p>
<p><img src="/blogs/mnw_052/orchestration.png" alt=""></p>
<p>It has some nice <strong>benefits</strong>:</p>
<ul>
<li>Simple</li>
<li>Centralized</li>
<li>Ease of troubleshooting</li>
<li>Monitoring is straightforward</li>
</ul>
<p><strong>Orchestration</strong> is typically <strong>simpler</strong> to implement and maintain than choreography.
Because there is a central coordinator, you can manage and monitor service interactions.
This, in turn, improves troubleshooting, as you know where to look when something goes wrong.</p>
<p>The <strong>drawbacks</strong> of orchestration are:</p>
<ul>
<li>Tight coupling</li>
<li>Single point of failure</li>
<li>Difficulty adding, removing, or replacing microservices</li>
</ul>
<h2>Choreography - Event-driven Communication</h2>
<p><strong>Choreography</strong> is a <strong>decentralized</strong> communication approach.
Choreography uses <strong>event-driven</strong> communication - as opposed to orchestration, which uses commands.</p>
<p>An <strong>event</strong> is something that has happened in the past and is a fact.
The sender does not know who will handle the event or what will happen after processing it.</p>
<p>I talked about events in-depth in the newsletter about <a href="how-to-use-domain-events-to-build-loosely-coupled-systems"><strong>publishing domain events.</strong></a></p>
<p><img src="/blogs/mnw_052/choreography.png" alt=""></p>
<p>The most important <strong>benefits</strong> of choreography are:</p>
<ul>
<li>Loose coupling</li>
<li>Ease of maintenance</li>
<li>Decentralized control</li>
<li>Asynchronous communication</li>
</ul>
<p><strong>Choreography</strong> allows microservices to be <strong>loosely coupled</strong>, which means they can operate independently and asynchronously.
This makes the system more scalable and resilient.
A failure of one microservice won't necessarily affect microservices.</p>
<p>Of course, there are <strong>downsides</strong> to choreography:</p>
<ul>
<li>Complexity</li>
<li>Monitoring is difficult</li>
<li>Difficulty troubleshooting</li>
</ul>
<p>It's more complex to implement and maintain than orchestration.</p>
<p>Effective monitoring is the biggest challenge from my experience.</p>
<h2>The Crossroads: Which One Should You Choose?</h2>
<p>So, how do you know which one to pick for your system?</p>
<p>I always start with the requirements of the system I'm building.
And then, I look at the pros and cons of orchestration vs. choreography.
Here's a small framework to help you decide.</p>
<p><strong>Orchestration</strong> excels when:</p>
<ul>
<li>You need to wait for the completion of intermediate steps (such as credit card payment confirmation)</li>
<li>You need to make a conditional choice of subsequent steps</li>
<li>The process must be carried out atomically (entirely or not at all)</li>
<li>The process needs to be centralized in one place for monitoring</li>
</ul>
<p>This also means you will need a central database managed by the orchestrator to handle all workflow-related state.</p>
<p><strong>Choreography</strong> works best when:</p>
<ul>
<li>The process can rely on the input message without needing additional context</li>
<li>Steps clearly follow one another</li>
<li>Progress is made in one direction</li>
</ul>
<p>You can benefit from increased flexibility (such as modifying individual steps in isolation)
Unfortunately, choreography can make it difficult to trace, debug, or monitor the processes triggered by an event.
And the larger the event stream is, the more challenging it becomes.</p>
<p>So, think carefully before opting for orchestration or choreography.</p>
<p>Both approaches bring their advantages but also downsides.</p>
<h2>Takeaway</h2>
<p><strong>Orchestration</strong> defines a sequence of steps that each microservice must follow.
This is great for identifying and addressing complex service interdependencies.
Another benefit is that business logic can be managed and monitored in one place.</p>
<p><strong>Choreography</strong>, on the other hand, is a decentralized technique for microservices communication.
Each service can operate independently while still being part of the larger architecture.</p>
<p>To decide which approach to use, you should observe your system and identify what you stand to gain or lose.
Everything is a tradeoff.</p>
<p>There's also an <strong>alternative approach</strong> I want to mention.</p>
<p>You could go for a <strong>hybrid approach</strong> that integrates orchestration and choreography.</p>
<p>In a <strong>hybrid approach</strong>, you decide which communication technique to use for a specific workflow.
Some workflows can benefit from orchestration, and others can benefit more from choreography.</p>
<p>Hope this was helpful.</p>
<p>I'll see you next week!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_052.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Advanced Rate Limiting Use Cases In .NET]]></title>
            <link>https://milanjovanovic.tech/blog/advanced-rate-limiting-use-cases-in-dotnet</link>
            <guid isPermaLink="false">advanced-rate-limiting-use-cases-in-dotnet</guid>
            <pubDate>Sat, 19 Aug 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Rate limiting is about restricting the number of requests to your application. It's usually applied within a specific time window or based on some other criteria.
Rate limiting is practical for a few reasons: - Improves security - Guards against DDoS attacks - Prevents overloading of application servers - Reduces costs by preventing unnecessary resource consumption
But you need to know how to implement it correctly, or you could grind your system to a halt.
In this week's newsletter I'll teach you: - How to rate limit users by IP address - How to rate limit users by their identity - How to apply rate limiting on the reverse proxy
So let's dive in.]]></description>
            <content:encoded><![CDATA[<p><strong>Rate limiting</strong> is about restricting the number of requests to your application.
It's usually applied within a specific time window or based on other criteria.</p>
<p>It's helpful for a few reasons:</p>
<ul>
<li>Improves security</li>
<li>Guards against DDoS attacks</li>
<li>Prevents overloading of application servers</li>
<li>Reduces costs by preventing unnecessary resource consumption</li>
</ul>
<p><strong>.NET 7</strong> shipped with a <strong>built-in rate limiter</strong>, but you need to know how to implement it correctly.
Or you could grind your system to a halt - and we don't want that.</p>
<p>In this week's newsletter, I'll teach you:</p>
<ul>
<li>How to rate limit users by <strong>IP address</strong></li>
<li>How to rate limit users by their <strong>identity</strong></li>
<li>How to apply <strong>rate limiting</strong> on the <strong>reverse proxy</strong></li>
</ul>
<p>So let's dive in!</p>
<h2>Built-In Rate Limiting In .NET 7</h2>
<p>Starting with .NET 7, we have access to built-in <strong>rate limiting middleware</strong> in the <code>Microsoft.AspNetCore.RateLimiting</code> namespace.
The API is straightforward, and you can create a rate limit policy with a few lines of code.</p>
<p>We can use one of the four <strong>rate limiting algorithms</strong>:</p>
<ul>
<li>Fixed window</li>
<li>Sliding window</li>
<li>Token bucket</li>
<li>Concurrency</li>
</ul>
<p>Here's how to define a <strong>rate limit policy</strong> by calling the <code>AddTokenBucketLimiter</code> method:</p>
<pre><code class="language-csharp">builder.Services.AddRateLimiter(rateLimiterOptions =&gt;
{
    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;

    rateLimiterOptions.AddTokenBucketLimiter(&quot;token&quot;, options =&gt;
    {
        options.TokenLimit = 1000;
        options.ReplenishmentPeriod = TimeSpan.FromHours(1);
        options.TokensPerPeriod = 700;
        options.AutoReplenishment = true;
    });
});
</code></pre>
<p>Now you can reference the <code>token</code> rate limit policy on your endpoint or controller.</p>
<p>You also have to add the <code>RateLimitingMiddleware</code> to the request pipeline:</p>
<pre><code class="language-csharp">app.UseRateLimiter();
</code></pre>
<p>You can learn more about <a href="how-to-use-rate-limiting-in-aspnet-core"><strong>rate limiting in .NET 7 here,</strong></a> so I won't go deeper into the fundamentals.</p>
<h2>Rate Limiting Users By IP Address</h2>
<p>The approach I just showed you has a <strong>problem</strong> - the <strong>rate limit policy</strong> is global and <strong>applies to all users</strong>.</p>
<p>Most of the time, you don't want to do this.
Rate limiting should be granular and apply to <strong>individual users</strong>.</p>
<p>Luckily, you can achieve this by creating a <code>RateLimitPartition</code>.</p>
<p>The <code>RateLimitPartition</code> has two components:</p>
<ul>
<li>Partition key</li>
<li>Rate limiter policy</li>
</ul>
<p>Here's how to define a rate limiter with a fixed window policy, and the <strong>partition key</strong> is the user's <strong>IP address</strong>.</p>
<pre><code class="language-csharp">builder.Services.AddRateLimiter(options =&gt;
{
    options.AddPolicy(&quot;fixed-by-ip&quot;, httpContext =&gt;
        RateLimitPartition.GetFixedWindowLimiter(
            partitionKey: httpContext.Connection.RemoteIpAddress?.ToString(),
            factory: _ =&gt; new FixedWindowRateLimiterOptions
            {
                PermitLimit = 10,
                Window = TimeSpan.FromMinutes(1)
            }));
});
</code></pre>
<p>Rate limiting by <strong>IP address</strong> can be a good layer of security for <strong>unauthenticated users</strong>.
You don't know who is accessing your system and can't apply more granular rate limiting.
This can help protect your system from malicious users trying to perform a DDoS attack.</p>
<p>You can also <a href="https://learn.microsoft.com/en-us/aspnet/core/performance/rate-limit?view=aspnetcore-7.0#create-chained-limiters"><strong>create chained limiters</strong></a> using the <code>CreateChained</code> API.
It allows you to pass in multiple <code>PartitionedRateLimiter</code>, which are combined into one <code>PartitionedRateLimiter</code>.
The chained limiter runs all the input limiters in sequence (one by one).</p>
<p>If your application is running behind a <strong>reverse proxy</strong>, you need to make sure not to rate limit the proxy IP address.
Reverse proxies usually <strong>forward</strong> the original IP address with the <code>X-Forwarded-For</code> header.
So you can use it as the <strong>partition key</strong>:</p>
<pre><code class="language-csharp">builder.Services.AddRateLimiter(options =&gt;
{
    options.AddPolicy(&quot;fixed-by-ip&quot;, httpContext =&gt;
        RateLimitPartition.GetFixedWindowLimiter(
            httpContext.Request.Headers[&quot;X-Forwarded-For&quot;].ToString(),
            factory: _ =&gt; new FixedWindowRateLimiterOptions
            {
                PermitLimit = 10,
                Window = TimeSpan.FromMinutes(1)
            }));
});
</code></pre>
<h2>Rate Limiting Users By Identity</h2>
<p>If you require users to <strong>authenticate</strong> with your API, you can determine who the current is.
Then you can use the user's <strong>identity</strong> as the <strong>partition key</strong> for a <code>RateLimitPartition</code>.</p>
<p>Here's how you would create such a rate limit policy:</p>
<pre><code class="language-csharp">builder.Services.AddRateLimiter(options =&gt;
{
    options.AddPolicy(&quot;fixed-by-user&quot;, httpContext =&gt;
        RateLimitPartition.GetFixedWindowLimiter(
            partitionKey: httpContext.User.Identity?.Name?.ToString(),
            factory: _ =&gt; new FixedWindowRateLimiterOptions
            {
                PermitLimit = 10,
                Window = TimeSpan.FromMinutes(1)
            }));
});
</code></pre>
<p>I'm using the <code>User.Identity</code> value on the <code>HttpContext</code> to get the current user's <code>Name</code> claim.
This usually corresponds to the <code>sub</code> claim inside a JWT - which is the user identifier.</p>
<h2>Applying Rate Limting On The Reverse Proxy</h2>
<p>In a robust implementation, you want to <strong>rate limit</strong> on the <strong>reverse proxy</strong> level before the request hits your API.
And if you have a distributed system, this is a requirement.
Otherwise, your system wouldn't function correctly.</p>
<p>There are many reverse proxy implementations to choose from.</p>
<p><strong>YARP</strong> is a reverse proxy with excellent .NET integration.
Not surprising since it was written in C#.
You can learn more about <a href="implementing-an-api-gateway-for-microservices-with-yarp"><strong>building an API Gateway with YARP here.</strong></a></p>
<p>To implement rate limiting on the reverse proxy with <strong>YARP</strong> you need to:</p>
<ul>
<li>Define a rate limit policy (covered in previous examples)</li>
<li>Configure the <code>RateLimiterPolicy</code> for the route in YARP settings</li>
</ul>
<pre><code class="language-json">&quot;products-route&quot;: {
  &quot;ClusterId&quot;: &quot;products-cluster&quot;,
  &quot;RateLimiterPolicy&quot;: &quot;sixty-per-minute-fixed&quot;,
  &quot;Match&quot;: {
    &quot;Path&quot;: &quot;/products/{**catch-all}&quot;
  },
  &quot;Transforms&quot;: [
    { &quot;PathPattern&quot;: &quot;{**catch-all}&quot; }
  ]
}
</code></pre>
<p>The built-in rate limiter middleware uses an <strong>in-memory</strong> store to track the number of requests.
If you want to run your reverse proxy in a high-availability setup, you will need to use a <strong>distributed cache</strong>.
A nice option to look into is using a <a href="https://github.com/cristipufu/aspnetcore-redis-rate-limiting"><strong>Redis backplane for rate limiting.</strong></a></p>
<h2>Closing Thoughts</h2>
<p>With the <code>PartitionedRateLimiter</code> you can easily create granular rate limit policies.</p>
<p>The two common approaches are:</p>
<ul>
<li>Rate limiting by <strong>IP address</strong></li>
<li>Rate limiting by the <strong>user identifier</strong></li>
</ul>
<p>I was really excited to see the .NET team ship rate limiting.
But, the current implementation has its shortcomings.
The main issue is that it only works <strong>in memory</strong>.
For a <strong>distributed</strong> solution, you need to implement something yourself or use an external library.</p>
<p>You can use the <strong>YARP</strong> reverse proxy to build robust and scalable distributed systems.
And it only takes a few lines of code to add <strong>rate limiting</strong> on the reverse proxy level.
I'm using it extensively in my systems.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_051.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Mastering Dapper Relationship Mappings]]></title>
            <link>https://milanjovanovic.tech/blog/mastering-dapper-relationship-mappings</link>
            <guid isPermaLink="false">mastering-dapper-relationship-mappings</guid>
            <pubDate>Sat, 12 Aug 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Dapper is a lightweight object-relational mapper in .NET. It's popular because it's easy to use and fast at the same time.
Dapper extends the IDbConnection interface with methods for sending SQL queries to the database.
But, because of the nature of SQL, mapping the result into an object model can be tricky.
So in this week's newsletter, I'll teach you how to map: - Simple queries - One-to-one relationships - One-to-many relationships - Many-to-many relationships
Let's dive in.]]></description>
            <content:encoded><![CDATA[<p><strong>Dapper</strong> is a lightweight <strong>object-relational mapper</strong> in .NET.
It's popular because it's easy to use and fast at the same time.</p>
<p>Dapper extends the <code>IDbConnection</code> interface with methods for sending SQL queries to the database.</p>
<p>But, because of the nature of SQL, mapping the result into an object model can be tricky.</p>
<p>So in this week's newsletter, I'll teach you how to map:</p>
<ul>
<li>Simple queries</li>
<li>One-to-one relationships</li>
<li>One-to-many relationships</li>
<li>Many-to-many relationships</li>
</ul>
<p>Let's dive in!</p>
<h2>Simple Mapping</h2>
<p>Let's first see how to do a <strong>simple mapping</strong> using Dapper.</p>
<p>Writing a query with Dapper has three parts:</p>
<ul>
<li>Creating an <code>IDbConnection</code> instance</li>
<li>Writing the SQL query</li>
<li>Calling a method that Dapper exposes</li>
</ul>
<p>We will write a SQL query to load a set of <code>LineItem</code> objects for a specific <code>Order</code>.</p>
<pre><code class="language-csharp">public class LineItem
{
    public long LineItemId { get; init; }

    public long OrderId { get; init; }

    public decimal Price { get; init; }

    public string Currency { get; init; }

    public decimal Quantity { get; init; }
}
</code></pre>
<p>Here's the SQL query returning the result we need:</p>
<pre><code class="language-sql">SELECT Id AS LineItemId, OrderId, Price, Currency, Quantity
FROM LineItems
WHERE OrderId = @OrderId
</code></pre>
<p>I'm parameterizing the <code>Order</code> identifier using the <code>@OrderId</code> syntax.
This is a Dapper convention.
It's important that you use <strong>parameterized queries</strong> to <strong>avoid SQL injection attacks</strong>.</p>
<p>The mapping is straightforward in this case because we are only returning one type from the database.</p>
<p>We call the <code>QueryAsync</code> method and specify <code>LineItem</code> as the return type.
Make sure to pass in the arguments for this method, the SQL query, and the <code>OrderId</code> parameter.
I prefer creating anonymous objects for Dapper parameters.</p>
<pre><code class="language-csharp">using var connection = new SqlConnection();

var lineItems = await connection.QueryAsync&lt;LineItem&gt;(
    sql,
    new { OrderId = orderId });
</code></pre>
<p>That's everything you need for a simple mapping.</p>
<h2>Dapper One To One Relationship Mapping</h2>
<p>What if the object we want to return from the SQL query contains a <strong>nested object</strong>?</p>
<p>Here's an updated <code>LineItem</code> type that also contains a <code>Product</code> inside.</p>
<pre><code class="language-csharp">public class LineItem
{
    public long LineItemId { get; init; }

    public long OrderId { get; init; }

    public decimal Price { get; init; }

    public string Currency { get; init; }

    public decimal Quantity { get; init; }

    public Product Product { get; init; }
}

public class Product
{
    public long ProductId { get; init; }

    public string Name { get; init; }
}
</code></pre>
<p>Now you need to return two types in the same query.</p>
<p>Here's the updated SQL query with a join on the <code>Products</code> table:</p>
<pre><code class="language-sql">SELECT li.Id AS LineItemId, li.OrderId, li.Price, li.Currency, li.Quantity,
       p.Id AS ProductId, p.Name
FROM LineItems li
JOIN Products p ON p.Id = li.ProductId
WHERE li.OrderId = @OrderId
</code></pre>
<p>This query is more complicated because we need to use Dapper's <strong>multi-mapping</strong> feature.</p>
<p>In the <code>QueryAsync</code> method, we specify both <code>LineItem</code> and <code>Product</code> as return types and <code>LineItem</code> as the final return type for the method.</p>
<p>We must also tell Dapper how to map the <code>LineItem</code> and <code>Product</code> from the result set into a single <code>LineItem</code> object.</p>
<p>And we need to specify the <code>splitOn</code> argument, which tells Dapper where one object ends and the next begins.</p>
<pre><code class="language-csharp">using var connection = new SqlConnection();

var lineItems = await connection.QueryAsync&lt;LineItem, Product, LineItem&gt;(
    sql,
    (lineItem, product) =&gt;
    {
        lineItem.Product = product;

        return lineItem;
    },
    new { OrderId = orderId },
    splitOn: &quot;ProductId&quot;);
</code></pre>
<p>We write more code to make this work, but it should be easy to wrap your head around it.</p>
<h2>Dapper One To Many Relationship Mapping</h2>
<p>Another frequent situation is mapping a <strong>one-to-many relationship</strong> from SQL into an object model.</p>
<p>Because you are joining two tables, the result set will contain duplicate data on the &quot;one&quot; side of the relationship.</p>
<p>For this example, let's use an <code>Order</code> with a list of <code>LineItem</code> objects.</p>
<pre><code class="language-csharp">public class Order
{
    public long OrderId { get; init; }

    public List&lt;LineItem&gt; LineItems { get; init; } = new();
}

public class LineItem
{
    public long LineItemId { get; init; }

    public long OrderId { get; init; }

    public decimal Price { get; init; }

    public string Currency { get; init; }

    public decimal Quantity { get; init; }
}
</code></pre>
<p>Here's the SQL query returning the data we need from the database:</p>
<pre><code class="language-sql">SELECT o.Id AS OrderId,
       li.Id AS LineItemId, li.OrderId, li.Price, li.Currency, li.Quantity
FROM Orders o
JOIN LineItems li ON li.OrderId = o.Id
WHERE o.Id = @OrderId
</code></pre>
<p>We're going to get back duplicate <code>Order</code> data because of the <code>JOIN</code>.
But we only want to return one <code>Order</code> with all the line items.</p>
<p>The Dapper mapping function only gives us the <code>Order</code> and <code>LineItem</code> for the current row in the result set.</p>
<p>One way to solve this is to use a <code>Dictionary</code> to store the <code>Order</code> and reuse it inside the mapping.</p>
<ul>
<li>Store the <code>Order</code> in the dictionary if it's not there</li>
<li>If it is there, add the <code>LineItem</code> to the existing <code>Order</code> instance</li>
</ul>
<pre><code class="language-csharp">using var connection = new SqlConnection();

var ordersDictionary = new Dictionary&lt;long, Order&gt;();

await connection.QueryAsync&lt;Order, LineItem, Order&gt;(
    sql,
    (order, lineItem) =&gt;
    {
        if (ordersDictionary.TryGetValue(order.OrderId, out var existingOrder))
        {
            order = existingOrder;
        }
        else
        {
            ordersDictionary.Add(order.OrderId, order);
        }

        order.LineItems.Add(lineItem);

        return order;
    },
    new { OrderId = orderId },
    splitOn: &quot;LineItemId&quot;);

var mappedOrder = ordersDictionary[orderId];
</code></pre>
<p>A <strong>many-to-many relationship</strong> would use the same idea, except you'll need two dictionaries for each side of the relationship.</p>
<h2>In Summary</h2>
<p><strong>Dapper</strong> is a fantastic library for writing fast database queries using SQL.</p>
<p>Because of how SQL works, mapping into an object model is sometimes complicated.</p>
<p>There are four common scenarios:</p>
<ul>
<li>Simple mapping - a flat structure mapped directly from SQL to an object</li>
<li>One-to-one mapping - provide a mapping function to connect two objects</li>
<li>One-to-many mapping - manage a dictionary for the &quot;one&quot; side of the relationship</li>
<li>Many-to-many mapping - same as above, except you need a dictionary for both sides of the relationship</li>
</ul>
<p>Now you have a cheat sheet for mapping relationships with Dapper.</p>
<p>Hope this was helpful.</p>
<p>I'll see you next week!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_050.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Modular Monolith Communication Patterns]]></title>
            <link>https://milanjovanovic.tech/blog/modular-monolith-communication-patterns</link>
            <guid isPermaLink="false">modular-monolith-communication-patterns</guid>
            <pubDate>Sat, 05 Aug 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Modular monoliths are becoming more popular in the software engineering community.
The allure of Microservices is becoming less compelling. We also have seasoned veterans of our industry saying you should reconsider:
> You shouldn't start a new project with microservices, even if you're sure your application will be big enough to make it worthwhile.
— Martin Fowler
Modular monoliths give you the logical architecture of Microservices without the operational complexity. You can safely determine the boundaries between modules. And refactoring is straightforward and less risky. They can also be easily migrated into Microservices if you decide to do so.
I've built and maintained several Modular monolith systems in the past years.
In this week's newsletter, I want to focus on the communication patterns in the Modular monolith architecture.
But first, let me explain what is a Modular monolith.]]></description>
            <content:encoded><![CDATA[<p><strong>Modular monoliths</strong> are becoming more popular in the software engineering community.</p>
<p>The allure of <strong>Microservices</strong> is becoming less compelling.
We also have seasoned veterans of our industry saying you should reconsider:</p>
<blockquote>
<p>You shouldn't start a new project with microservices, even if you're sure your application will be big enough to make it worthwhile.</p>
</blockquote>
<p><em>— <a href="https://martinfowler.com/bliki/MonolithFirst.html">Martin Fowler</a></em></p>
<p>Modular monoliths give you the <strong>logical architecture</strong> of Microservices without the operational complexity.
You can safely determine the boundaries between modules.
And refactoring is straightforward and less risky.
They can also be easily migrated into Microservices if you decide to do so.</p>
<p>I've built and maintained several large <strong>Modular monolith</strong> systems in the past years.</p>
<p>In this week's newsletter, I want to focus on the <strong>communication patterns</strong> in the Modular monolith architecture.</p>
<p>But first, let me explain what is a <strong>Modular monolith</strong>.</p>
<h2>What Is a Modular Monolith?</h2>
<p>Here's one definition of what a <strong>Modular monolith</strong> is:</p>
<blockquote>
<p>A Modular Monolith is a software design approach in which a monolith is designed with an emphasis on interchangeable (and potentially reusable) modules.</p>
</blockquote>
<p>The problem with most monolith systems is that they become <strong>tightly coupled</strong> over time.
Components are deeply intertwined.
Making a change in one component impacts many others.
Introducing new features is difficult and error-prone.</p>
<p>Modular monoliths aim to solve these problems.</p>
<p><img src="/blogs/mnw_049/modular_monolith_diagram.png" alt=""></p>
<p>A Modular monolith consists of many <strong>loosely coupled</strong> modules.
Modules represent cohesive sets of functionalities.
Modules are also <strong>independent</strong> of each other.</p>
<p>Here are a few examples:</p>
<ul>
<li>Payments module</li>
<li>Shipping module</li>
<li>Booking module</li>
<li>Reviews module</li>
</ul>
<p>If this concept reminds you of Microservices, that's because it should.</p>
<p>Microservices represent self-contained services encapsulating a set of functionalities of the larger system, much like modules in a Modular monolith.</p>
<p>For a Modular monolith to be loosely coupled, you need to solve how modules will communicate.</p>
<p>Modules cannot reference each other directly except through their public APIs.</p>
<p>There are two widely used communication patterns.
Both have pros and cons and a set of tradeoffs that you need to understand.</p>
<h2>Synchronous Communication With Method Calls</h2>
<p>The first and easiest communication pattern is simple <strong>method calls</strong> between modules.
Method calls are <strong>synchronous</strong> and very fast because they're in memory.</p>
<p>Module A calls a method declared on the <strong>public API</strong> of Module B and waits until it receives a result.</p>
<p>Each module exposes a <strong>public API</strong>, which can be an <code>interface</code> in .NET.</p>
<p>The module will implement this interface internally and hide any implementation details.
You can use the <code>internal</code> keyword to make the implementation inaccessible outside the module.</p>
<p>Modules depend on the interfaces at compile-time.
At runtime, <strong>dependency injection</strong> will provide the respective implementation.</p>
<p><img src="/blogs/mnw_049/modular_monolith_sync_communication.png" alt=""></p>
<p>The benefits of this approach are:</p>
<ul>
<li>Speed of in-memory calls</li>
<li>Easy to implement</li>
<li>No indirection</li>
</ul>
<p>But, the drawback of this approach is <strong>strong coupling</strong>.</p>
<p><strong>Synchronous communication</strong> means that the modules will be tightly coupled.
If one of the modules is unavailable, it will affect any dependent modules.
You can introduce a retry mechanism, but this only goes so far.</p>
<h2>Asynchronous Communication With Messaging</h2>
<p>The second communication pattern is asynchronous <strong>messaging</strong> between modules.</p>
<p>Module A sends a message to the message broker in a fire-and-forget fashion.
Module B subscribes to relevant messages and handles them accordingly.</p>
<p>Modules don't need to know about each other, but they do need to know about the message contracts.</p>
<p><strong>Message contracts</strong> are the <strong>public API</strong> of a module in this scenario.</p>
<p><img src="/blogs/mnw_049/modular_monolith_async_communication.png" alt=""></p>
<p>The benefits of this approach are:</p>
<ul>
<li>High availability</li>
<li>Loose coupling</li>
</ul>
<p><strong>Asynchronous communication</strong> gives us loose coupling because modules communicate using messages.
Module B doesn't need to be available for Module A to send a message.</p>
<p>The obvious drawback of this approach is <strong>increased complexity</strong>.</p>
<p>We're introducing a <strong>message broker</strong> to the system.
This is another infrastructure component we have to manage.
It's also a single point of failure.
If the message broker fails, so does communication between the modules.</p>
<p>You can prevent message loss by storing messages in an <a href="outbox-pattern-for-reliable-microservices-messaging"><strong>Outbox</strong></a> before publishing them.
We can always send the messages again from the database in case of a message broker failure.</p>
<h2>Takeaway</h2>
<p><strong>Synchronous communication</strong> between modules is easy to implement, and it's performant. But it comes at the cost of tight coupling between modules.</p>
<p><strong>Asynchronous communication</strong> using a message broker is loosely coupled. But it's more complex to implement.</p>
<p>So which <strong>communication pattern</strong> should you be using?</p>
<p>It depends.</p>
<p>Asynchronous communication can help you build <strong>loosely coupled</strong> and <strong>independent</strong> modules.
Migrating a <strong>Modular monolith</strong> using messaging into a <strong>distributed system</strong> is much easier.</p>
<p>You extract a module into its own deployment unit.
And the communication between modules remains the same.
Because you are using messaging, you don't need to reimplement anything.</p>
<p><strong>Synchronous method calls</strong> are an excellent choice to increase development velocity and reduce operational complexity.</p>
<p>I'll let the software architect in you decide.</p>
<p>Thanks for reading.</p>
<p>And stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_049.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Why Clean Architecture Is Great For Complex Projects]]></title>
            <link>https://milanjovanovic.tech/blog/why-clean-architecture-is-great-for-complex-projects</link>
            <guid isPermaLink="false">why-clean-architecture-is-great-for-complex-projects</guid>
            <pubDate>Sat, 29 Jul 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[I've been using Clean Architecture for 6+ years on large scale applications serving thousands of customers and millions of requests. Today I want to talk about why it's a great approach for structuring your applications.
I'm aware that Clean Architecture isn't a silver bullet, so I will discuss what types of systems can benefit from this architecture.
Clean architecture isn't revolutionary.
But it's prescriptive about how you should structure the code.
It's an evolution of the layered architecture, focusing on the core domain and the direction of dependencies. All dependencies should point inwards, applying dependency inversion.
Here are some of the promises of Clean Architecture: - Maintainability - Testability - Loose coupling - Separation of concerns
It's independent of UI, databases, or external services - but you also need to be pragmatic (more on this later).
Let's dive in.]]></description>
            <content:encoded><![CDATA[<p>I've been using <strong>Clean Architecture</strong> for 6+ years on large-scale applications serving thousands of customers and millions of requests.
Today I want to talk about why it's a great approach for structuring your applications.</p>
<p>I'm aware that Clean Architecture isn't a silver bullet, so I will discuss what types of systems can benefit from this architecture.</p>
<p>Clean architecture isn't revolutionary.</p>
<p>But it's <strong>prescriptive</strong> about how you should structure the code.</p>
<p>It's an evolution of layered architecture, focusing on the core domain and the direction of dependencies.
All dependencies should point inwards, applying <strong>dependency inversion</strong>.</p>
<p>Here are some of the promises of Clean Architecture:</p>
<ul>
<li>Maintainability</li>
<li>Testability</li>
<li>Loose coupling</li>
<li>Separation of concerns</li>
</ul>
<p>It's <strong>independent</strong> of UI, databases, or external services - but you also need to be <strong>pragmatic</strong> (more on this later).</p>
<p>Let's dive in!</p>
<h2>What Is Clean Architecture?</h2>
<p>Clean Architecture was created by <a href="https://en.wikipedia.org/wiki/Robert_C._Martin"><strong>Robert C. Martin</strong></a>, aka Uncle Bob.</p>
<p>Clean Architecture is an approach to organizing a software system to <strong>separate the concerns</strong> of the various components.
Making the system easier to understand and maintain.</p>
<p>You can think about Clean Architecture as a domain-centric approach to organizing dependencies.</p>
<p>There are similar architectures that follow the same domain-centric idea.
You may know them as the Hexagonal and Onion architectures.
They are more or less interchangeable with each other.
And they all place the core domain at the center of the architecture.</p>
<p>This is my high-level interpretation of <strong>Clean Architecture</strong>:</p>
<p><img src="/blogs/mnw_048/clean_architecture.png" alt=""></p>
<p>There are four layers inside:</p>
<ul>
<li>Domain</li>
<li>Application</li>
<li>Infrastructure</li>
<li>Presentation</li>
</ul>
<p>Let's see what should live inside each layer.</p>
<h2>Clean Architecture Layers</h2>
<p>Here's a breakdown of what should live inside each <strong>layer</strong> of the <strong>Clean Architecture</strong>.</p>
<p><strong>Domain</strong></p>
<ul>
<li>Contains the core business rules &amp; logic</li>
<li>It should be independent of other layers in the system</li>
<li>Should be persistence ignorant - the persistence mechanism shouldn't influence your domain model</li>
<li><strong>Examples:</strong> Entities, Value Objects, Domain services, Domain events, Enums, Repository interfaces</li>
</ul>
<p><strong>Application</strong></p>
<ul>
<li>Contains the application use cases</li>
<li>Contains application-specific business rules</li>
<li>Orchestrates the domain entities to perform business operations</li>
<li>It should be independent of external concerns (but it doesn't have to be)</li>
<li><strong>Examples:</strong> Application services, Commands, Queries, External service interfaces, Exceptions</li>
</ul>
<p><strong>Infrastructure</strong></p>
<ul>
<li>Contains anything related to external concerns</li>
<li>Implements interfaces defined in the layers below</li>
<li><strong>Examples:</strong> PostgreSQL, Keycloak, AWS S3, RabbitMQ, Kafka, SendGrid</li>
</ul>
<p><strong>Presentation</strong></p>
<ul>
<li>Represents the entry point to the system</li>
<li>Accepts data from the outside and passes it to the use cases</li>
<li>Acts as the composition root for dependency injection</li>
<li><strong>Examples:</strong> <a href="http://ASP.NET">ASP.NET</a> Core, gRPC</li>
</ul>
<p><a href="clean-architecture-folder-structure"><strong>Here's an example</strong></a> of how to structure the Clean Architecture on a solution level.
You can also group related components together by feature.
It leads to better cohesion and is an excellent option if your project is more complex.</p>
<p>I also made a few videos covering the Clean Architecture project setup:</p>
<ul>
<li><a href="https://youtu.be/tLk4pZZtiDY"><strong>Clean Architecture walkthrough</strong></a> (100k+ views)</li>
<li><a href="https://youtu.be/fe4iuaoxGbA"><strong>Clean Architecture project setup from scratch</strong></a> (40k+ views)</li>
</ul>
<h2>Where Should You Use Clean Architecture?</h2>
<p><strong>Clean Architecture</strong> is very versatile and applies to various domains and systems.</p>
<p>But, you should play to its strengths and use it only when there's a tangible benefit.</p>
<p>I use the <strong>Clean Architecture</strong> when I want to:</p>
<ul>
<li>Apply Domain-Driven Design</li>
<li>Solve complex business logic</li>
<li>Build highly testable projects</li>
<li>Enforce design policies via the architecture</li>
</ul>
<p>If the above is true for your project, then Clean Architecture is an excellent option.</p>
<p>You should also consider the <a href="clean-architecture-and-the-benefits-of-structured-software-design"><strong>benefits of Clean Architecture.</strong></a></p>
<h2>The Case For Being Pragmatic</h2>
<p>I try to be <strong>pragmatic</strong> when using <strong>Clean Architecture</strong>.</p>
<p>Applying what I like and having the freedom of &quot;breaking&quot; Clean Architecture if it will simplify things.</p>
<p>Can this be called Clean Architecture, then?
No, not in the purest sense.</p>
<p>But, I still get <strong>most</strong> of the <strong>benefits</strong> of Clean Architecture.</p>
<p>Here's an example, when I'm applying CQRS in Clean Architecture to implement the use cases.</p>
<p>On the command (write) side, it's valuable to be independent of external concerns, so I will use repositories behind an interface.
I can control the repository contract, and unit testing is straightforward.</p>
<p>But, on the query (read) side, I want to return the response as fast as possible.
Creating an abstraction only adds indirection and reduces performance.</p>
<p>A better approach is to use the EF or Dapper in the handler and query the database.
It's simple, fast, and you can use all the features offered by the ORM.
You don't need a lot of complexity or abstractions on the query side.</p>
<p>I should call this approach <a href="/pragmatic-clean-architecture"><strong>Pragmatic Clean Architecture</strong>.</a></p>
<h2>Closing Thoughts</h2>
<p><strong>Clean Architecture</strong> gives you a <strong>standard</strong> for organizing your solution.</p>
<p>You don't have to reinvent the wheel every time at the start of the project.</p>
<p>But, the layered structure and architectural constraints can increase the <strong>complexity</strong> of smaller projects.<br>
So make sure your project is complex enough to apply Clean Architecture.</p>
<p>Another <strong>caveat</strong> of Clean Architecture is the danger of <strong>over-engineering</strong>.</p>
<p>Don't follow the principles religiously without considering the specific project requirements.<br>
The overhead of maintaining so many layers and abstractions may not be justified.</p>
<p>Be <strong>pragmatic</strong> and try to make the best decision possible.</p>
<p>Sometimes that means straying from the paradigm.</p>
<p>Hope this was helpful.</p>
<p>I'll see you next week!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_048.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How To Use Domain Events To Build Loosely Coupled Systems]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-use-domain-events-to-build-loosely-coupled-systems</link>
            <guid isPermaLink="false">how-to-use-domain-events-to-build-loosely-coupled-systems</guid>
            <pubDate>Sat, 22 Jul 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[In software engineering, "coupling" means how much different parts of a software system depend on each other. If they are tightly coupled, changes to one part can affect many others. But if they are loosely coupled, changes to one part won't cause big problems in the rest of the system.
Domain events are a Domain-Driven Design (DDD) tactical pattern that we can use to build loosely coupled systems.
You can raise a domain event from the domain, which represents a fact that has occurred. And other components in the system can subscribe subscribe to this event, and handle it accordingly.
Here's what you will learn in this week's newsletter: - What are domain events - How they're different from integration events - How to implement & raise domain events - How to publish domain events with EF Core - How to handle domain events with MediatR
We have a lot to cover so let's dive in.]]></description>
            <content:encoded><![CDATA[<p>In software engineering, &quot;coupling&quot; means how much different parts of a software system depend on each other.
If they are <strong>tightly coupled</strong>, changes to one part can affect many others.
But if they are <strong>loosely coupled</strong>, changes to one part won't cause big problems in the rest of the system.</p>
<p><strong>Domain events</strong> are a <strong>Domain-Driven Design (DDD)</strong> tactical pattern that we can use to build <strong>loosely coupled</strong> systems.</p>
<p>You can raise a <strong>domain event</strong> from the domain, which represents a fact that has occurred.
And other components in the system can subscribe to this event and handle it accordingly.</p>
<p>Here's what you will learn in this week's newsletter:</p>
<ul>
<li>What are <a href="#what-are-domain-events">domain events</a></li>
<li>How they're <a href="#domain-events-versus-integration-events">different from integration events</a></li>
<li>How to <a href="#implementing-domain-events">implement</a> &amp; <a href="#raising-domain-events">raise domain events</a></li>
<li>How to <a href="#how-to-publish-domain-events-with-ef-core">publish domain events</a> with EF Core</li>
<li>How to <a href="#how-to-handle-domain-events">handle domain events</a> with MediatR</li>
</ul>
<p>We have a lot to cover, so let's dive in!</p>
<h2>What Are Domain Events?</h2>
<p>An <strong>event</strong> is something that has happened in the past.</p>
<p>It is a fact.</p>
<p>Unchangeable.</p>
<p>A <strong>domain event</strong> is something that happened in the domain, and other parts of the domain should be aware of it.</p>
<p><strong>Domain events</strong> allow you to express side effects explicitly, and provide a better separation of concerns in the domain.
They're an ideal way to trigger side effects across multiple aggregates inside the domain.</p>
<p>It's your responsibility to ensure that publishing a <strong>domain event</strong> is transactional.
You'll see why this is easier said than done.</p>
<p><img src="/blogs/mnw_047/domain_events.png" alt=""></p>
<h2>Domain Events Versus Integration Events</h2>
<p>You may have heard of <strong>integration events</strong>, and you're now wondering what's the difference between them and <strong>domain events</strong>.</p>
<p>Semantically, they're the same thing: a representation of something that occurred in the past.</p>
<p>However, their <strong>intent is different</strong> and this is important to understand.</p>
<p>Domain events:</p>
<ul>
<li>Published and consumed within a single domain</li>
<li>Sent using an in-memory message bus</li>
<li>Can be processed synchronously or asynchronously</li>
</ul>
<p>Integration events:</p>
<ul>
<li>Consumed by other subsystems (microservices, Bounded Contexts)</li>
<li>Sent with a message broker over a queue</li>
<li>Processed completely asynchronously</li>
</ul>
<p>So if you're wondering what type of event you should publish, think about the intent and who should be handling the event.</p>
<p><strong>Domain events</strong> can also be used to <strong>generate integration events</strong>, which leave the domain boundary.</p>
<h2>Implementing Domain Events</h2>
<p>My preferred approach to implement <strong>domain events</strong> is creating an <code>IDomainEvent</code> abstraction and implementing MediatR <code>INotification</code>.</p>
<p>The benefit is you can use <strong>MediatR's publish-subscribe</strong> support to publish a notification to one or multiple handlers.</p>
<pre><code class="language-csharp">using MediatR;

public interface IDomainEvent : INotification
{
}
</code></pre>
<p>Now you can implement a concrete domain event.</p>
<p>Here are a few <strong>constraints</strong> to consider when <strong>designing domain events</strong>:</p>
<ul>
<li>Immutability - domain events are facts, and should be immutable</li>
<li>Fat vs Thin domain events - how much information do you need?</li>
<li>Use past tense for event naming</li>
</ul>
<pre><code class="language-csharp">public class CourseCompletedDomainEvent : IDomainEvent
{
    public Guid CourseId { get; init; }
}
</code></pre>
<h2>Raising Domain Events</h2>
<p>After you create your domain events, you want to raise them from the domain.</p>
<p>My approach is creating an <code>Entity</code> base class, because only entities are allowed to raise domain events.
You can further encapsulate raising domain events by making the <code>RaiseDomainEvent</code> method <code>protected</code>.</p>
<p>We're storing domain events in an internal collection, to prevent anyone else from accessing it.
The <code>GetDomainEvents</code> method is there to get a snapshot of the collection, and the <code>ClearDomainEvents</code> method to clear the internal collection.</p>
<pre><code class="language-csharp">public abstract class Entity : IEntity
{
    private readonly List&lt;IDomainEvent&gt; _domainEvents = new();

    public IReadOnlyList&lt;IDomainEvent&gt; GetDomainEvents()
    {
        return _domainEvents.ToList();
    }

    public void ClearDomainEvents()
    {
        _domainEvents.Clear();
    }

    protected void RaiseDomainEvent(IDomainEvent domainEvent)
    {
        _domainEvents.Add(domainEvent);
    }
}
</code></pre>
<p>Now you're entities can inherit from the <code>Entity</code> base class and raise domain events:</p>
<pre><code class="language-csharp">public class Course : Entity
{
    public Guid Id { get; private set; }

    public CourseStatus Status { get; private set; }

    public DateTime? CompletedOnUtc { get; private set; }

    public void Complete()
    {
        Status = CourseStatus.Completed;
        CompletedOnUtc = DateTime.UtcNow;

        RaiseDomainEvent(new CourseCompletedDomainEvent { CourseId = this.Id });
    }
}
</code></pre>
<p>And all that's left to do is <strong>publish</strong> the <strong>domain events</strong>.</p>
<h2>How To Publish Domain Events With EF Core</h2>
<p>An elegant solution for publishing domain events is using <strong>EF Core</strong>.</p>
<p>Since EF Core acts as a <strong>Unit of Work</strong>, you can use it to gather all <strong>domain events</strong> in the current transaction and publish them.</p>
<p>I don't like to complicate things, and simply override the <code>SaveChangesAsync</code> method to publish the domain events after persisting the changes in the database.
But you could also use an interceptor.</p>
<pre><code class="language-csharp">public class ApplicationDbContext : DbContext
{
    public override async Task&lt;int&gt; SaveChangesAsync(
        CancellationToken cancellationToken = default)
    {
        // When should you publish domain events?
        //
        // 1. BEFORE calling SaveChangesAsync
        //     - domain events are part of the same transaction
        //     - immediate consistency
        // 2. AFTER calling SaveChangesAsync
        //     - domain events are a separate transaction
        //     - eventual consistency
        //     - handlers can fail

        var result = await base.SaveChangesAsync(cancellationToken);

        await PublishDomainEventsAsync();

        return result;
    }
}
</code></pre>
<p>The most <strong>important decision</strong> you will have to make here is <strong>when to publish the domain events</strong>.</p>
<p>I think it makes the most sense to publish after calling <code>SaveChangesAsync</code>.
In other words, after saving changes to the database.</p>
<p>This comes with a few tradeoffs:</p>
<ul>
<li>Eventual consistency - because messages are processed after the original transactions</li>
<li>Database inconsistency risk - because handling domain events can fail</li>
</ul>
<p>Eventual consistency is something I can live with, so I choose to make this tradeoff.</p>
<p>However, introducing a risk of database inconsistency is a big concern.</p>
<p>You can solve this with the <a href="outbox-pattern-for-reliable-microservices-messaging"><strong>Outbox pattern,</strong></a>
where you persist your changes to the database and the domain events (as outbox messages) in a single transaction.
Now you have a guaranteed atomic transaction, and the domain events are processed asynchronously using a background job.</p>
<p>If you're wondering what's inside the <code>PublishDomainEventsAsync</code> method:</p>
<pre><code class="language-csharp">private async Task PublishDomainEventsAsync()
{
    var domainEvents = ChangeTracker
        .Entries&lt;Entity&gt;()
        .Select(entry =&gt; entry.Entity)
        .SelectMany(entity =&gt;
        {
            var domainEvents = entity.GetDomainEvents();

            entity.ClearDomainEvents();

            return domainEvents;
        })
        .ToList();

    foreach (var domainEvent in domainEvents)
    {
        await _publisher.Publish(domainEvent);
    }
}
</code></pre>
<h2>How To Handle Domain Events</h2>
<p>With all of the plumbing we created so far, we're ready to implement a handler for the domain events.
Luckily, this is the simplest step in the process.</p>
<p>All you have to do is define a class implementing <code>INotificationHandler&lt;T&gt;</code> and specify your domain event type as the generic argument.</p>
<p>Here's a handler for the <code>CourseCompletedDomainEvent</code>, which takes the domain event and publishes a <code>CourseCompletedIntegrationEvent</code> to notify other systems.</p>
<pre><code class="language-csharp">public class CourseCompletedDomainEventHandler
    : INotificationHandler&lt;CourseCompletedDomainEvent&gt;
{
    private readonly IBus _bus;

    public CourseCompletedDomainEventHandler(IBus bus)
    {
        _bus = bus;
    }

    public async Task Handle(
        CourseCompletedDomainEvent domainEvent,
        CancellationToken cancellationToken)
    {
        await _bus.Publish(
            new CourseCompletedIntegrationEvent(domainEvent.CourseId),
            cancellationToken);
    }
}
</code></pre>
<h2>In Summary</h2>
<p><strong>Domain events</strong> can help you build a loosely coupled system.
You can use them to separate the core domain logic from the side effects, which can be handled asynchronously.</p>
<p>There's no need to reinvent the wheel for implementing domain events, and you can use the <strong>EF Core</strong> and <strong>MediatR</strong> libraries to build this.</p>
<p>You will have to make the decision when you want to publish domain events.
Publishing before or after saving changes to the database both have their set of <strong>tradeoffs</strong>.</p>
<p>I prefer <strong>publishing</strong> domain events <strong>after saving changes</strong> to the database, and I use the <a href="outbox-pattern-for-reliable-microservices-messaging"><strong>Outbox pattern</strong></a>
to add transactional guarantees.
This approach introduces eventual consistency, but it's also more reliable.</p>
<p>Hope this was helpful.</p>
<p>See you next week!</p>
<p><strong>Today's action step:</strong>
Take a look at <a href="https://youtu.be/AHzWJ_SMqLo"><strong>this video,</strong></a> where I explain how to implement domain events to build a decoupled system that scales.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_047.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[8 Tips To Write Clean Code]]></title>
            <link>https://milanjovanovic.tech/blog/8-tips-to-write-clean-code</link>
            <guid isPermaLink="false">8-tips-to-write-clean-code</guid>
            <pubDate>Sat, 15 Jul 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Clean code is code that's easy to read, maintain, and understand.
I consider writing clean code a skill.
And it's a skill that you can learn and improve with deliberate practice.
My favorite approach to practice clean coding is doing refactoring exercises.
So I prepared one for you today, and we're going to improve one step at a time by applying clean code principles.
Let's get started.]]></description>
            <content:encoded><![CDATA[<p><strong>Clean code</strong> is code that's easy to read, maintain, and understand.</p>
<p>I consider writing <strong>clean code</strong> a skill.</p>
<p>And it's a <strong>skill</strong> that <strong>you can learn</strong> and improve with deliberate practice.</p>
<p>My favorite approach for practicing <strong>clean coding</strong> is doing <strong>refactoring exercises</strong>.</p>
<p>So I prepared one for you today, and we're going to improve it one step at a time by applying <strong>clean code principles</strong>.</p>
<p>Let's dive in!</p>
<h2>Starting Point</h2>
<p>I like starting with a problem when trying to learn new concepts.</p>
<p>And the more illustrative the problem, the better.</p>
<p>So we'll use some poorly written code as the starting point for our refactoring.</p>
<p>And in each step, I will highlight what the current issue is and how we will fix it.</p>
<p>Here's what I see when I look at the <code>Process</code> method:</p>
<ul>
<li>Deep nesting of code - 4 levels, to be precise</li>
<li>Precondition checks are applied one after the other</li>
<li>Throwing exceptions to represent a failure</li>
</ul>
<p>How can we turn this into <strong>clean code</strong>?</p>
<pre><code class="language-csharp">public void Process(Order? order)
{
    if (order != null)
    {
        if (order.IsVerified)
        {
            if (order.Items.Count &gt; 0)
            {
                if (order.Items.Count &gt; 15)
                {
                    throw new Exception(
                        &quot;The order &quot; + order.Id + &quot; has too many items&quot;);
                }

                if (order.Status != &quot;ReadyToProcess&quot;)
                {
                    throw new Exception(
                        &quot;The order &quot; + order.Id + &quot; isn't ready to process&quot;);
                }

                order.IsProcessed = true;
            }
        }
    }
}
</code></pre>
<h2>#1: Early Return Principle</h2>
<p>It should be painfully obvious by now that the initial version is deeply nested because of the <code>if</code> statements applying precondition checks.</p>
<p>We'll solve this using the <strong>early return principle</strong>, which states that we should return from a method as soon as the conditions for that have been met.</p>
<p>In the case of the <code>Process</code> method, this means moving from a deeply nested structure to a set of <strong>guard clauses</strong>.</p>
<pre><code class="language-csharp">public void Process(Order? order)
{
    if (order is null)
    {
        return;
    }

    if (!order.IsVerified)
    {
        return;
    }

    if (order.Items.Count == 0)
    {
        return;
    }

    if (order.Items.Count &gt; 15)
    {
        throw new Exception(
            &quot;The order &quot; + order.Id + &quot; has too many items&quot;);
    }

    if (order.Status != &quot;ReadyToProcess&quot;)
    {
        throw new Exception(
            &quot;The order &quot; + order.Id + &quot; isn't ready to process&quot;);
    }

    order.IsProcessed = true;
}
</code></pre>
<h2>#2: Merge If Statements To Improve Readability</h2>
<p>The <strong>early return principle</strong> makes the <code>Process</code> method more readable.</p>
<p>But there's no need to have one <strong>guard clause</strong> after another.</p>
<p>So we can merge all of them into one <code>if</code> statement.</p>
<p>The behavior of the <code>Process</code> method remains unchanged, but we remove a lot of excess code.</p>
<pre><code class="language-csharp">public void Process(Order? order)
{
    if (order is null ||
        !order.IsVerified ||
        order.Items.Count == 0)
    {
        return;
    }

    if (order.Items.Count &gt; 15)
    {
        throw new Exception(
            &quot;The order &quot; + order.Id + &quot; has too many items&quot;);
    }

    if (order.Status != &quot;ReadyToProcess&quot;)
    {
        throw new Exception(
            &quot;The order &quot; + order.Id + &quot; isn't ready to process&quot;);
    }

    order.IsProcessed = true;
}
</code></pre>
<h2>#3: Use LINQ For More Concise Code</h2>
<p>A quick improvement can be using <strong>LINQ</strong> to make the code more concise and expressive.</p>
<p>Instead of checking for <code>Items.Count == 0</code>, I prefer using the LINQ <code>Any</code> method.</p>
<p>You could argue that LINQ has worse performance, but I always optimize for readability.</p>
<p>There are far more expensive operations in an application than a simple method call.</p>
<pre><code class="language-csharp">public void Process(Order? order)
{
    if (order is null ||
        !order.IsVerified ||
        !order.Items.Any())
    {
        return;
    }

    if (order.Items.Count &gt; 15)
    {
        throw new Exception(
            &quot;The order &quot; + order.Id + &quot; has too many items&quot;);
    }

    if (order.Status != &quot;ReadyToProcess&quot;)
    {
        throw new Exception(
            &quot;The order &quot; + order.Id + &quot; isn't ready to process&quot;);
    }

    order.IsProcessed = true;
}
</code></pre>
<h2>#4: Replace Boolean Expression With Descriptive Method</h2>
<p>Merging multiple conditions into one <code>if</code> statement means writing less code, but it can <strong>decrease readability</strong> with <strong>complex conditions</strong>.</p>
<p>However, you can fix this and improve readability by using a variable or method with a <strong>descriptive name</strong>.</p>
<p>I prefer using methods, so I will introduce the <code>IsProcessable</code> method to represent the precondition check.</p>
<pre><code class="language-csharp">public void Process(Order? order)
{
    if (!IsProcessable(order))
    {
        return;
    }

    if (order.Items.Count &gt; 15)
    {
        throw new Exception(
            &quot;The order &quot; + order.Id + &quot; has too many items&quot;);
    }

    if (order.Status != &quot;ReadyToProcess&quot;)
    {
        throw new Exception(
            &quot;The order &quot; + order.Id + &quot; isn't ready to process&quot;);
    }

    order.IsProcessed = true;
}

static bool IsProcessable(Order? order)
{
    return order is not null &amp;&amp;
           order.IsVerified &amp;&amp;
           order.Items.Any();
}
</code></pre>
<h2>#5: Prefer Throwing Custom Exceptions</h2>
<p>Now let's talk about throwing exceptions.
I like to use exceptions for <em>&quot;exceptional&quot;</em> situations only, and I don't use them for flow control in my code.</p>
<p>Having said that, if you <em>do</em> want to use exceptions for flow control, it's better to use <strong>custom exceptions</strong>.</p>
<p>You can introduce valuable contextual information and better describe the reason for throwing the exception.</p>
<p>And if you want to handle these exceptions globally, you can create a base class to be able to catch specific exceptions.</p>
<pre><code class="language-csharp">public void Process(Order? order)
{
    if (!IsProcessable(order))
    {
        return;
    }

    if (order.Items.Count &gt; 15)
    {
        throw new TooManyLineItemsException(order.Id);
    }

    if (order.Status != &quot;ReadyToProcess&quot;)
    {
        throw new NotReadyForProcessingException(order.Id);
    }

    order.IsProcessed = true;
}

static bool IsProcessable(Order? order)
{
    return order is not null &amp;&amp;
           order.IsVerified &amp;&amp;
           order.Items.Any();
}
</code></pre>
<h2>#6: Fix Magic Numbers With Constants</h2>
<p>A common <strong>code smell</strong> I see is the use of <strong>magic numbers</strong>.</p>
<p>They are usually easy to spot because they're used to check if numeric some condition applies.</p>
<p>The problem with <strong>magic numbers</strong> is that they <strong>carry no meaning</strong>.</p>
<p>The code is harder to reason about, and more error-prone.</p>
<p>Fixing <strong>magic numbers</strong> should be straightforward, and one solution is introducing a constant.</p>
<pre><code class="language-csharp">const int MaxNumberOfLineItems = 15;

public void Process(Order? order)
{
    if (!IsProcessable(order))
    {
        return;
    }

    if (order.Items.Count &gt; MaxNumberOfLineItems)
    {
        throw new TooManyLineItemsException(order.Id);
    }

    if (order.Status != &quot;ReadyToProcess&quot;)
    {
        throw new NotReadyForProcessingException(order.Id);
    }

    order.IsProcessed = true;
}

static bool IsProcessable(Order? order)
{
    return order is not null &amp;&amp;
           order.IsVerified &amp;&amp;
           order.Items.Any();
}
</code></pre>
<h2>#7: Fix Magic Strings With Enums</h2>
<p>Similar to <strong>magic numbers</strong>, we have the <strong>magic strings</strong> <strong>code smell</strong>.</p>
<p>A typical use case for <strong>magic strings</strong> is to represent some sort of state.</p>
<p>You'll notice that we're comparing the <code>Order.Status</code> value to a <strong>magic string</strong> to check if the order is ready to process.</p>
<p>A few <strong>problems</strong> with <strong>magic strings</strong>:</p>
<ul>
<li>Easy to make mistakes (typo)</li>
<li>Lack of strong typing</li>
<li>Not refactoring proof</li>
</ul>
<p>Let's create an <code>OrderStatus</code> <code>enum</code> to represent the possible states:</p>
<pre><code class="language-csharp">enum OrderStatus
{
    Pending = 0,
    ReadyToProcess = 1,
    Processed = 2
}
</code></pre>
<p>And now we have to use the appropriate <code>OrderStatus</code> in the check:</p>
<pre><code class="language-csharp">const int MaxNumberOfLineItems = 15;

public void Process(Order? order)
{
    if (!IsProcessable(order))
    {
        return;
    }

    if (order.Items.Count &gt; MaxNumberOfLineItems)
    {
        throw new TooManyLineItemsException(order.Id);
    }

    if (order.Status != OrderStatus.ReadyToProcess)
    {
        throw new NotReadyForProcessingException(order.Id);
    }

    order.IsProcessed = true;
    order.Status = OrderStatus.Processed;
}

static bool IsProcessable(Order? order)
{
    return order is not null &amp;&amp;
           order.IsVerified &amp;&amp;
           order.Items.Any();
}
</code></pre>
<h2>#8: Use The Result Object Pattern</h2>
<p>I said I don't prefer using exceptions for flow control. But how can we fix this?</p>
<p>One solution is using the <strong>result object pattern</strong>.</p>
<p>You can use a generic <code>Result</code> class to represent all types of results or a specific one like <code>ProcessOrderResult</code>.</p>
<p>To make your result objects encapsulated, expose a set of factory methods to create the concrete result type.</p>
<pre><code class="language-csharp">public class ProcessOrderResult
{
    private ProcessOrderResult(
        ProcessOrderResultType type,
        long orderId,
        string message)
    {
        Type = type;
        OrderId = orderId;
        Message = message;
    }

    public ProcessOrderResultType Type { get; }

    public long OrderId { get; }

    public string? Message { get; }

    public static ProcessOrderResult NotProcessable() =&gt;
      new(ProcessOrderResultType.NotProcessable, default, &quot;Not processable&quot;);

    public static ProcessOrderResult TooManyLineItems(long oderId) =&gt;
      new(ProcessOrderResultType.TooManyLineItems, orderId, &quot;Too many items&quot;);

    public static ProcessOrderResult NotReadyForProcessing(long oderId) =&gt;
      new(ProcessOrderResultType.NotReadyForProcessing, oderId, &quot;Not ready&quot;);

    public static ProcessOrderResult Success(long oderId) =&gt;
      new(ProcessOrderResultType.Success, oderId, &quot;Success&quot;);
}
</code></pre>
<p>Using an <code>enum</code> like <code>ProcessOrderResultType</code> will make consuming the result object easier with switch expressions.
Here's the <code>enum</code> to represent the <code>ProcessOrderResult.Type</code>:</p>
<pre><code class="language-csharp">public enum ProcessOrderResultType
{
    NotProcessable = 0,
    TooManyLineItems = 1,
    NotReadyForProcessing = 2,
    Success = 3
}
</code></pre>
<p>And now the <code>Process</code> method becomes:</p>
<pre><code class="language-csharp">const int MaxNumberOfLineItems = 15;

public ProcessOrderResult Process(Order? order)
{
    if (!IsProcessable(order))
    {
        return ProcessOrderResult.NotProcessable();
    }

    if (order.Items.Count &gt; MaxNumberOfLineItems)
    {
        return ProcessOrderResult.TooManyLineItems(order);
    }

    if (order.Status != OrderStatus.ReadyToProcess)
    {
        return ProcessOrderResult.NotReadyForProcessing(order);
    }

    order.IsProcessed = true;
    order.Status = OrderStatus.Processed;

    return ProcessOrderResult.Success(order);
}

static bool IsProcessable(Order? order)
{
    return order is not null &amp;&amp;
           order.IsVerified &amp;&amp;
           order.Items.Any();
}
</code></pre>
<p>Here's how using an <code>enum</code> for the <code>ProcessOrderResult.Type</code> allows you to write a switch expression:</p>
<pre><code class="language-csharp">var result = Process(order);

result.Type switch
{
    ProcessOrderResultType.TooManyLineItems =&gt;
        Console.WriteLine($&quot;Too many line items: {result.OrderId}&quot;),

    ProcessOrderResultType.NotReadyForProcessing =&gt;
        Console.WriteLine($&quot;Not ready for processing {result.OrderId}&quot;),

    ProcessOrderResultType.Success =&gt;
        Console.WriteLine($&quot;Processed successfully {result.OrderId}&quot;),

    _ =&gt; Console.WriteLine(&quot;Failed to process: {OrderId}&quot;, result.OrderId),
};
</code></pre>
<h2>Takeaway</h2>
<p>That's it, 8 tips to write <strong>clean code</strong>:</p>
<ul>
<li><a href="#1-early-return-principle">Early return principle</a></li>
<li><a href="#2-merge-if-statements-to-improve-readability">Merge multiple if statements</a></li>
<li><a href="#3-use-linq-for-more-concise-code">Use LINQ for conciseness</a></li>
<li><a href="#4-replace-boolean-expression-with-descriptive-method">Replace boolean expression with method</a></li>
<li><a href="#5-prefer-throwing-custom-exceptions">Prefer throwing custom exceptions</a></li>
<li><a href="#6-fix-magic-numbers-with-constants">Replace magic numbers with constants</a></li>
<li><a href="#7-fix-magic-strings-with-enums">Replace magic string with enums</a></li>
<li><a href="#8-use-the-result-object-pattern">Use the result object pattern</a></li>
</ul>
<p>Writing <strong>clean code</strong> is a matter of deliberate practice and experience.</p>
<p>Most people will read about <strong>clean coding principles</strong>, but few will strive to apply them daily.</p>
<p>This is where you can set yourself apart.</p>
<p>I also made a video about these <strong>clean code</strong> tips, you can watch it <a href="https://youtu.be/McDvyFglkvU">here.</a></p>
<p>Hope this was helpful.</p>
<p>See you next week!</p>
<p><strong>Today's action step:</strong>
Take a look at your project and see if you're making some of the mistakes I highlighted here.
And then fix them using the clean code tips I shared with you.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_046.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Implementing an API Gateway For Microservices With YARP]]></title>
            <link>https://milanjovanovic.tech/blog/implementing-an-api-gateway-for-microservices-with-yarp</link>
            <guid isPermaLink="false">implementing-an-api-gateway-for-microservices-with-yarp</guid>
            <pubDate>Sat, 08 Jul 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Large Microservice-based systems can consist of tens or even hunders of individual services. A client application needs to have all of this information to be able to make requests to the relevant microservice directly.
However, this has numerous issues such as security concerns, complexity, and coupling.
We can solve this by introducing an API gateway that acts as a reverse proxy to accept API calls from the client application, forwarding this traffic to the appropriate service.
The API gateway also enforces security and ensures scalability and high availability.
In this week's newsletter, I'll show you how to implement an API gateway using the YARP reverse proxy.
Here's what we will cover: - Difference between API gateway and reverse proxy - Installing and configuring YARP - Creating an API gateway with YARP - Authentication and rate limiting on the API gateway
Let's get started.]]></description>
            <content:encoded><![CDATA[<p>Large <strong>Microservice-based</strong> systems can consist of tens or even hundreds of individual services.
A client application needs to have all of this information to be able to make requests to the relevant <strong>microservice</strong> directly.</p>
<p>However, this has numerous issues, such as security concerns, increased complexity, and coupling.</p>
<p>We can solve this by introducing an <strong>API gateway</strong> that acts as a <strong>reverse proxy</strong> to accept API calls from the client application and forward them to the appropriate service.</p>
<p>The <strong>API gateway</strong> also enforces security and ensures scalability and high availability.</p>
<p>In this week's newsletter, I'll show you how to implement an <strong>API gateway</strong> for your <strong>microservices system</strong> using the <strong>YARP reverse proxy</strong>.</p>
<p>Here's what we will cover:</p>
<ul>
<li>Difference between <strong>API gateway</strong> and <strong>reverse proxy</strong></li>
<li>Installing and configuring <strong>YARP</strong></li>
<li>Creating an <strong>API gateway</strong> with <strong>YARP</strong></li>
<li><strong>Authentication</strong> and <strong>rate limiting</strong> on the <strong>API gateway</strong></li>
</ul>
<p>Let's dive in.</p>
<h2>What's The Difference Between an API Gateway And a Reverse Proxy?</h2>
<p>A <strong>reverse proxy</strong> and an <strong>API gateway</strong> are similar concepts, but they serve different purposes.</p>
<p>A <strong>reverse proxy</strong> acts as an intermediary between clients and servers.
The clients can only call the backend servers through the <strong>reverse proxy</strong>, which forwards the request to the appropriate server.
It hides the implementation details of individual servers inside the internal network.</p>
<p>A <strong>reverse proxy</strong> is commonly used for:</p>
<ul>
<li>Load balancing</li>
<li>Caching</li>
<li>Security</li>
<li>SSL termination</li>
</ul>
<p><img src="/blogs/mnw_045/reverse_proxy.png" alt=""></p>
<p>An <strong>API gateway</strong> is a specific type of <strong>reverse proxy</strong> designed for managing APIs.
It acts as a single entry point for API consumers to the various backend services.</p>
<p>The key characteristics of an <strong>API gateway</strong> are:</p>
<ul>
<li>Request routing and composition</li>
<li>Request/response transformations</li>
<li>Authentication and authorization</li>
<li>Rate limiting</li>
<li>Monitoring</li>
</ul>
<p>Also, note that an <strong>API gateway</strong> can perform <strong>load balancing</strong> and other functionalities mentioned for reverse proxies.</p>
<p>Now let's see how to use a <strong>reverse proxy</strong> to implement an <strong>API gateway</strong>.</p>
<h2>Installing And Configuring YARP</h2>
<p><strong>YARP</strong> (Yet Another Reverse Proxy) is a library developed by Microsoft to address the needs of various teams needing to build a <strong>reverse proxy</strong>.
It's open source and built with .NET, so it integrates nicely with the existing ecosystem.</p>
<p>Let's install <code>Yarp.ReverseProxy</code> <strong>NuGet</strong> package to get started:</p>
<pre><code class="language-powershell">Install-Package Yarp.ReverseProxy
</code></pre>
<p>Next, we're going to call:</p>
<ul>
<li><code>AddReverseProxy</code> to add the required services for <strong>YARP</strong></li>
<li><code>LoadFromConfig</code> to load the <strong>reverse proxy</strong> configuration from application settings</li>
<li><code>MapReverseProxy</code> to introduce the <strong>reverse proxy</strong> middleware</li>
</ul>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);

builder.Services.AddReverseProxy()
    .LoadFromConfig(builder.Configuration.GetSection(&quot;ReverseProxy&quot;));

var app = builder.Build();

app.MapReverseProxy();

app.Run();
</code></pre>
<p>We need to tell the <strong>YARP</strong> reverse proxy how to route the incoming requests to the individual microservices.</p>
<p><strong>YARP</strong> uses the concept of <code>Routes</code> to represent request patterns for the proxy and <code>Clusters</code> to represent the services to forward those requests.</p>
<pre><code class="language-json">{
    &quot;ReverseProxy&quot;: {
        &quot;Routes&quot;: {
            ...
        },
        &quot;Clusters&quot;: {
            ...
        }
    }
}
</code></pre>
<p>Here's an example <strong>YARP configuration</strong> with a <code>{**catch-all}</code> pattern that will route all incoming requests to the one destination server.</p>
<pre><code class="language-json">{
  &quot;ReverseProxy&quot;: {
    &quot;Routes&quot;: {
      &quot;ROUTE_NAME&quot;: {
        &quot;ClusterId&quot;: &quot;CLUSTER_NAME&quot;,
        &quot;Match&quot;: {
          &quot;Path&quot;: &quot;{**catch-all}&quot;
        }
      }
    },
    &quot;Clusters&quot;: {
      &quot;CLUSTER_NAME&quot;: {
        &quot;Destinations&quot;: {
          &quot;destination1&quot;: {
            &quot;Address&quot;: &quot;https://www.milanjovanovic.tech/&quot;
          }
        }
      }
    }
  }
}
</code></pre>
<h2>Implementing an API Gateway With YARP</h2>
<p>We can use <strong>YARP</strong> to build an <strong>API gateway</strong> by providing the configuration for the services we want to route traffic to.</p>
<p>I created a sample <a href="https://github.com/m-jovanovic/yarp-api-gateway-sample">API gateway implementation with YARP</a> on GitHub, so you can give it a try.
The system has two services, the <code>Users.Api</code> and <code>Products.Api</code>, which are .NET 7 applications.</p>
<p>If a request comes in matching the <code>/users-service/{**catch-all}</code>, for example <code>/users-service/users</code>, it will be routed to the <code>users-cluster</code>.
The same logic applies for the <code>products-cluster</code>. We can apply more advanced transformations through the <code>Transforms</code> section.</p>
<pre><code class="language-json">{
  &quot;ReverseProxy&quot;: {
    &quot;Routes&quot;: {
      &quot;users-route&quot;: {
        &quot;ClusterId&quot;: &quot;users-cluster&quot;,
        &quot;Match&quot;: {
          &quot;Path&quot;: &quot;/users-service/{**catch-all}&quot;
        },
        &quot;Transforms&quot;: [{ &quot;PathPattern&quot;: &quot;{**catch-all}&quot; }]
      },
      &quot;products-route&quot;: {
        &quot;ClusterId&quot;: &quot;products-cluster&quot;,
        &quot;Match&quot;: {
          &quot;Path&quot;: &quot;/products-service/{**catch-all}&quot;
        },
        &quot;Transforms&quot;: [{ &quot;PathPattern&quot;: &quot;{**catch-all}&quot; }]
      }
    },
    &quot;Clusters&quot;: {
      &quot;users-cluster&quot;: {
        &quot;Destinations&quot;: {
          &quot;destination1&quot;: {
            &quot;Address&quot;: &quot;https://localhost:5201/&quot;
          }
        }
      },
      &quot;products-cluster&quot;: {
        &quot;Destinations&quot;: {
          &quot;destination1&quot;: {
            &quot;Address&quot;: &quot;https://localhost:5101/&quot;
          }
        }
      }
    }
  }
}
</code></pre>
<p>We now have a functioning <strong>API gateway</strong> built with <strong>YARP</strong>, routing requests to two individual services.</p>
<p>But what else can we do with <strong>YARP</strong>?</p>
<h2>Adding Authentication</h2>
<p>The <strong>API gateway</strong> can enforce <strong>authentication</strong> and <strong>authorization</strong> at the entry point to the system before letting authenticated requests proceed.</p>
<p>And <strong>YARP</strong> supports integrating with the existing authentication &amp; authorization middleware.</p>
<p>You first need to define an <strong>authorization policy</strong>:</p>
<pre><code class="language-csharp">builder.Services.AddAuthorization(options =&gt;
{
    options.AddPolicy(&quot;authenticated&quot;, policy =&gt;
        policy.RequireAuthenticatedUser());
});
</code></pre>
<p>And call <code>UseAuthentication</code> and <code>UseAuthorization</code> to add the respective middleware to the request pipeline.
It's important to add them before calling <code>MapReverseProxy</code>.</p>
<pre><code class="language-csharp">app.UseAuthentication();

app.UseAuthorization();

app.MapReverseProxy();
</code></pre>
<p>Now all you have to do is add the <code>AuthorizationPolicy</code> section to the reverse proxy configuration:</p>
<pre><code class="language-json">&quot;users-route&quot;: {
  &quot;ClusterId&quot;: &quot;users-cluster&quot;,
  &quot;AuthorizationPolicy&quot;: &quot;authenticated&quot;,
  &quot;Match&quot;: {
    &quot;Path&quot;: &quot;/users-service/{**catch-all}&quot;
  },
  &quot;Transforms&quot;: [
    { &quot;PathPattern&quot;: &quot;{**catch-all}&quot; }
  ]
}
</code></pre>
<p><strong>YARP</strong> will forward most credentials to the proxied services, such as cookies or bearer tokens because it might be important to identify the user in the individual microservices.</p>
<h2>Adding Rate Limiting</h2>
<p>You can also use an <strong>API gateway</strong> to introduce <strong>rate limiting</strong> to your system.
It's a technique to limit the number of requests to your API to <strong>improve security</strong> and reduce the load on the servers.</p>
<p>You can learn more about <a href="how-to-use-rate-limiting-in-aspnet-core">how to use rate limiting in .NET here.</a></p>
<p>As you can already guess, <strong>YARP</strong> supports the native <strong>rate limiting</strong> mechanism added in .NET 7.</p>
<p>All you need to do is define a <strong>rate limit policy</strong>:</p>
<pre><code class="language-csharp">builder.Services.AddRateLimiter(rateLimiterOptions =&gt;
{
    rateLimiterOptions.AddFixedWindowLimiter(&quot;fixed&quot;, options =&gt;
    {
        options.Window = TimeSpan.FromSeconds(10);
        options.PermitLimit = 5;
    });
});
</code></pre>
<p>Then you need to call <code>UseRateLimiter</code> to add the rate limiter middleware to the request pipeline.
It's important to do it before calling <code>MapReverseProxy</code>.</p>
<pre><code class="language-csharp">app.UseRateLimiter();

app.MapReverseProxy();
</code></pre>
<p>And then, you can apply rate limiting to the desired route using the <code>RateLimiterPolicy</code> section in the reverse proxy configuration:</p>
<pre><code class="language-json">&quot;products-route&quot;: {
  &quot;ClusterId&quot;: &quot;products-cluster&quot;,
  &quot;RateLimiterPolicy&quot;: &quot;fixed&quot;,
  &quot;Match&quot;: {
    &quot;Path&quot;: &quot;/products-service/{**catch-all}&quot;
  },
  &quot;Transforms&quot;: [
    { &quot;PathPattern&quot;: &quot;{**catch-all}&quot; }
  ]
}
</code></pre>
<h2>In Summary</h2>
<p>An <strong>API gateway</strong> is a critical component for a robust <strong>microservices system</strong> implementation.</p>
<p>And <strong>YARP</strong> is an excellent option if you want to build an <strong>API gateway</strong> with .NET.</p>
<p>I created a sample API gateway implementation with YARP, which you can find <a href="https://github.com/m-jovanovic/yarp-api-gateway-sample">here.</a>
The system consists of two APIs, and the API gateway is configured to route requests between them.
It also implements:</p>
<ul>
<li><a href="https://microsoft.github.io/reverse-proxy/articles/authn-authz.html">Authentication</a></li>
<li><a href="how-to-use-rate-limiting-in-aspnet-core">Rate limiting</a></li>
</ul>
<p>In this newsletter, we only scratched the surface of what's possible with <strong>YARP</strong>.</p>
<p>Here are some useful resources if you want to learn more:</p>
<ul>
<li><a href="https://microsoft.github.io/reverse-proxy/articles/index.html">YARP docs</a></li>
<li><a href="https://microsoft.github.io/reverse-proxy/articles/load-balancing.html">Load balancing</a></li>
<li><a href="https://microsoft.github.io/reverse-proxy/articles/session-affinity.html">Session affinity</a></li>
<li><a href="https://microsoft.github.io/reverse-proxy/articles/transforms.html">Request/response transformations</a></li>
</ul>
<p>That's all for today.</p>
<p>Hope it was helpful.</p>
<p><strong>Today's action step:</strong>
Download the <a href="https://github.com/m-jovanovic/yarp-api-gateway-sample">source code</a> for the sample application implementing an API gateway with YARP, and take it for a spin.
You can challenge yourself by creating multiple instances of a single service and configuring load balancing with the various load balancing algorithms.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_045.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Response Compression In ASP.NET Core]]></title>
            <link>https://milanjovanovic.tech/blog/response-compression-in-aspnetcore</link>
            <guid isPermaLink="false">response-compression-in-aspnetcore</guid>
            <pubDate>Sat, 01 Jul 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Reducing the size of your API responses can noticeably improve the performance of your application.
And since network bandwidth is a limited resource, you should at least consider the benefits of response compression.
Here's what you'll learn in this week's newsletter: - How to configure response compression in .NET - Server-based vs. application-based compression - Possible security risks and mitigation strategies - How to configure the available compression providers - How much network bandwidth you could be saving
Let's get started.]]></description>
            <content:encoded><![CDATA[<p>Reducing the size of your API responses can noticeably improve the performance of your application.</p>
<p>And since network bandwidth is a limited resource, you should at least consider the benefits of <strong>response compression</strong>.</p>
<p>Here's what you'll learn in this week's newsletter:</p>
<ul>
<li>How to configure <strong>response compression</strong> in .NET</li>
<li>Server-based vs. application-based compression</li>
<li>Possible <strong>security risks</strong> and <strong>mitigation strategies</strong></li>
<li>How to configure the available compression providers</li>
<li>How much network bandwidth you could be saving</li>
</ul>
<p>Let's get started!</p>
<h2>Configuring Response Compression</h2>
<p>Using <strong>response compression</strong> in an .NET applications is remarkably easy.</p>
<p>You only have to call these two methods:</p>
<ul>
<li><code>AddResponseCompression</code> - to configure the default services for response compression</li>
<li><code>UseResponseCompression</code> - to add the response compression middleware to the request pipeline</li>
</ul>
<p>The <code>UseResponseCompression</code> method should be called before any middleware that compresses responses.</p>
<p><strong>Response compression</strong> isn't turned on by default for HTTPS, so you have to enable it by setting <code>EnableForHttps</code> to <code>true</code>.</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);

builder.Services.AddResponseCompression(options =&gt;
{
    options.EnableForHttps = true;
});

var app = builder.Build();

app.UseResponseCompression();

app.MapGet(&quot;/&quot;, () =&gt; &quot;This response will be compressed 📦&quot;);

app.Run();
</code></pre>
<p>It really is that simple.</p>
<p>But...</p>
<h2>When Should You Use Response Compression?</h2>
<p>Ideally, you want to be using <strong>server-based response compression</strong> if your application server supports it.</p>
<p>Because the middleware performs <strong>response compression</strong> at the <strong>application level</strong>, it will typically have worse performance.</p>
<p>If you are hosting your application and you can't use server-based compression, then using the response compression middleware is justified.</p>
<p>One more concern should be <strong>security</strong>, because using <strong>response compression over HTTPS</strong> can expose you to <a href="https://en.wikipedia.org/wiki/CRIME">CRIME</a> and <a href="https://en.wikipedia.org/wiki/BREACH">BREACH</a> attacks</p>
<p>Here's what you can do to improve security:</p>
<ul>
<li>You can mitigate CRIME and BREACH attacks by introducing <strong>anti-forgery tokens</strong> in <a href="http://ASP.NET">ASP.NET</a> Core</li>
<li>Don't send application secrets as part of the request body</li>
<li>Implement a <a href="how-to-use-rate-limiting-in-aspnet-core"><strong>rate limiter</strong></a></li>
</ul>
<h2>Configuring Compression Providers</h2>
<p>There are two compression providers added by default when you call <code>AddResponseCompression</code>:</p>
<ul>
<li><code>BrotliCompressionProvider</code></li>
<li><code>GzipCompressionProvider</code></li>
</ul>
<p>You can further customize the available providers by adding custom compression providers if you want to.</p>
<p>Compression will default to <strong>Brotli</strong> compression when it's supported by the client.
Otherwise, it will default to <strong>Gzip</strong> if that is the supported compression format.</p>
<pre><code class="language-csharp">builder.Services.AddResponseCompression(options =&gt;
{
    options.EnableForHttps = true;
    options.Providers.Add&lt;BrotliCompressionProvider&gt;();
    options.Providers.Add&lt;GzipCompressionProvider&gt;();
    options.MimeTypes = ResponseCompressionDefaults.MimeTypes;
});
</code></pre>
<p>The interesting thing is you can configure the <code>CompressionLevel</code> for the <strong>Brotli</strong> and <strong>Gzip</strong> compression providers.</p>
<p>There are four possible values:</p>
<ul>
<li><code>Optimal</code> - tries to balance response size and compression speed</li>
<li><code>Fastest</code> - sacrifices optimal compression for improved speed</li>
<li><code>NoCompression</code> - self explanatory</li>
<li><code>SmallestSize</code> - sacrifices compression speed to make the output as small as possible</li>
</ul>
<p>The default value is <code>CompressionLevel.Fastest</code>.</p>
<pre><code class="language-csharp">builder.Services.Configure&lt;BrotliCompressionProviderOptions&gt;(options =&gt;
{
    options.Level = CompressionLevel.Optimal;
});

builder.Services.Configure&lt;GzipCompressionProviderOptions&gt;(options =&gt;
{
    options.Level = CompressionLevel.SmallestSize;
});
</code></pre>
<h2>How Much Can You Save?</h2>
<p>Let's find out how much network bandwidth we can save by using <strong>response compression</strong>.</p>
<p>Here's a minimal API endpoint returning a list of <code>Message</code> objects:</p>
<pre><code class="language-csharp">app.MapGet(&quot;/&quot;, () =&gt; Results.Ok(
    Enumerable
        .Range(1, 100)
        .Select(num =&gt; new Message
        {
            Id = num,
            Content = $&quot;This is the message #{num}&quot;
        })));
</code></pre>
<p>And here are the results with different providers and compression levels:</p>
<ul>
<li>No compression - 4.5kB</li>
<li><strong>Gzip</strong> + <code>CompressionLevel.Fastest</code> - 569B</li>
<li><strong>Gzip</strong> + <code>CompressionLevel.SmallestSize</code> - 539B</li>
<li><strong>Gzip</strong> + <code>CompressionLevel.Optimal</code> - 554B</li>
<li><strong>Brotli</strong> + <code>CompressionLevel.Fastest</code> - 400B</li>
<li><strong>Brotli</strong> + <code>CompressionLevel.SmallestSize</code> - 296B</li>
<li><strong>Brotli</strong> + <code>CompressionLevel.Optimal</code> - 319B</li>
</ul>
<p><strong>Brotli</strong> is the clear winner, which is why it's the <strong>default compression provider</strong>.</p>
<p>In the best case scenario, you can reduce the response size by ~93.5%.
Multiply this by the number of requests you're serving daily, and then you can begin to estimate the possible network savings.</p>
<p>One more thing I noticed is that using <code>CompressionLevel.SmallestSize</code> has a noticeable <strong>negative impact</strong> on response time.
I can't say this was surprising, so I suggest to simply keep using the default compression level.</p>
<h2>In Summary</h2>
<p><a href="https://learn.microsoft.com/en-us/aspnet/core/performance/response-compression?view=aspnetcore-7.0">Response compression</a>
is an interesting technique to improve API performance and reduce network costs.</p>
<p>Ideally, you'd want to be using <strong>server-based response compression</strong> if it's supported by your application server.
If that's not the case, <strong>application-based compression</strong> is available in .NET with the response compression middleware.</p>
<p>What's the cost of response compression?</p>
<p>It will increase the CPU load, and can expose some security risks over HTTPS, but there are ways to mitigate this.</p>
<p>In my experience, the <strong>default configuration values</strong> for the compression provider and compression level give excellent results.</p>
<p>That's all for this week.</p>
<p>See you next Saturday.</p>
<p><strong>Today's action step:</strong>
To see the value of response compression, I suggest enabling it in your application and examining the changes in response size.
You can try the different compression providers by varying the <code>Accept-Encoding</code> header, and also configure different compression levels in your application.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_044.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Adding Real-Time Functionality To .NET Applications With SignalR]]></title>
            <link>https://milanjovanovic.tech/blog/adding-real-time-functionality-to-dotnet-applications-with-signalr</link>
            <guid isPermaLink="false">adding-real-time-functionality-to-dotnet-applications-with-signalr</guid>
            <pubDate>Sat, 24 Jun 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Today's modern applications must deliver the latest information without refreshing the user interface.
If you need to introduce real-time functionality to your application in .NET, there's one library you will most likely reach for - SignalR.
SignalR allows you to push content from your server-side code to any connected clients as changes happen in real-time.
Here's what I'll teach you in this week's newsletter: - Creating your first SignalR Hub - Testing SignalR from Postman - Creating strongly typed hubs - Sending messages to a specific user
Let's see why SignalR is so powerful and how easy it is to build real-time applications.]]></description>
            <content:encoded><![CDATA[<p>Today's modern applications must deliver the latest information without refreshing the user interface.</p>
<p>If you need to introduce <strong>real-time</strong> functionality to your application in .NET, there's one library you will most likely reach for - <strong>SignalR</strong>.</p>
<p><strong>SignalR</strong> allows you to push content from your server-side code to any connected clients as changes happen in real-time.</p>
<p>Here's what I'll teach you in this week's newsletter:</p>
<ul>
<li>Creating your first <strong>SignalR</strong> <code>Hub</code></li>
<li>Testing <strong>SignalR</strong> from <strong>Postman</strong></li>
<li>Creating strongly typed hubs</li>
<li>Sending messages to a specific user</li>
</ul>
<p>Let's see why <strong>SignalR</strong> is so powerful and how easy it is to build <strong>real-time</strong> applications with it.</p>
<h2>Installing And Configuring SignalR</h2>
<p>To start using <strong>SignalR</strong> you'll need to:</p>
<ul>
<li>Install the NuGet package</li>
<li>Create the <code>Hub</code> class</li>
<li>Register the SignalR services</li>
<li>Map and expose the hub endpoint so clients can connect to it</li>
</ul>
<p>Let's start by installing the <code>Microsoft.AspNetCore.SignalR.Client</code> NuGet package:</p>
<pre><code class="language-powershell">Install-Package Microsoft.AspNetCore.SignalR.Client
</code></pre>
<p>Then you need a SignalR <code>Hub</code>, which is the central component in your application responsible for managing clients and sending messages.</p>
<p>Let's create a <code>NotificationsHub</code> by inheriting from the base <code>Hub</code> class:</p>
<pre><code class="language-csharp">public sealed class NotificationsHub : Hub
{
    public async Task SendNotification(string content)
    {
        await Clients.All.SendAsync(&quot;ReceiveNotification&quot;, content);
    }
}
</code></pre>
<p>The SignalR <code>Hub</code> exposes a few useful properties:</p>
<ul>
<li><code>Clients</code> - used to invoke methods on the clients connected to this hub</li>
<li><code>Groups</code> - an abstraction for adding and removing connections from groups</li>
<li><code>Context</code> - used for accessing information about the hub caller connection</li>
</ul>
<p>You can learn more about the <code>Hub</code> class <a href="https://learn.microsoft.com/en-us/dotnet/api/microsoft.aspnetcore.signalr.hub?view=aspnetcore-7.0">here</a>.</p>
<p>Lastly, you need to register the SignalR services by calling the <code>AddSignalR</code> method.
You also need to call the <code>MapHub&lt;T&gt;</code> method, where you specify the <code>NotificationsHub</code> class and the path clients will use to connect to the hub.</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);

builder.Services.AddSignalR();

var app = builder.Build();

app.MapHub&lt;NotificationsHub&gt;(&quot;notifications-hub&quot;);

app.Run();
</code></pre>
<p>Now let's see how we can test the <code>NotificationsHub</code>.</p>
<h2>Connecting To SignalR Hub From Postman</h2>
<p>To test SignalR, you need a client that will connect to the <code>Hub</code> instance.
You could create a simple application with Blazor or JavaScript, but I will show you a different approach.</p>
<p>We will use Postman's <strong>WebSocket Request</strong> to connect to the <code>NotificationsHub</code>.</p>
<p><img src="/blogs/mnw_043/postman_websocket_request.png" alt=""></p>
<p>Here's what we need to do:</p>
<ul>
<li>Connect to the <code>NotificationsHub</code></li>
<li>Set the communication protocol to JSON</li>
<li>Send messages to call the <code>NotificationsHub</code> methods</li>
</ul>
<p>All messages need to end with a null termination character, which is just the ASCII character <code>0x1E</code>.</p>
<p>Let's start off by sending this message to set the communication protocol to JSON:</p>
<pre><code class="language-json">{
  &quot;protocol&quot;: &quot;json&quot;,
  &quot;version&quot;: 1
}?
</code></pre>
<p>You'll receive this response from the hub.</p>
<p><img src="/blogs/mnw_043/postman_set_protocol_request.png" alt=""></p>
<p>We need a slightly different message format to call a message on the <code>Hub</code>.
The key is specifying the <code>arguments</code> and <code>target</code>, which is the actual hub method we want to call.</p>
<p>Let's say we want to call the <code>SendNotification</code> method on the <code>NotificationsHub</code>:</p>
<pre><code class="language-json">{
  &quot;arguments&quot;: [&quot;This is the notification message.&quot;],
  &quot;target&quot;: &quot;SendNotification&quot;,
  &quot;type&quot;: 1
}?
</code></pre>
<p>This will be the response we get back from the <code>NotificationsHub</code>:</p>
<p><img src="/blogs/mnw_043/postman_send_notification_request.png" alt=""></p>
<h2>Strongly Typed Hubs</h2>
<p>The base <code>Hub</code> class uses the <code>SendAsync</code> method to send messages to connected clients.
Unfortunately, we have to use strings to specify client-side methods to invoke, and it's easy to make a mistake.
There's also nothing enforcing which parameters are used.</p>
<p>SignalR supports <strong>strongly typed hubs</strong> that aim to solve this.</p>
<p>First, you need to define a client interface, so let's create a simple <code>INotificationsClient</code> abstraction:</p>
<pre><code class="language-csharp">public interface INotificationsClient
{
    Task ReceiveNotification(string content);
}
</code></pre>
<p>The arguments don't have to be primitive types and can also be objects. SignalR will take care of serialization on the client side.</p>
<p>After that, you need to update the <code>NotificationsHub</code> class to inherit from the <code>Hub&lt;T&gt;</code> class to make it strongly typed:</p>
<pre><code class="language-csharp">public sealed class NotificationsHub : Hub&lt;INotificationsClient&gt;
{
    public async Task SendNotification(string content)
    {
        await Clients.All.ReceiveNotification(content);
    }
}
</code></pre>
<p>You will lose access to the <code>SendAsync</code> method, and only the methods defined in your client interface will be available.</p>
<h2>Sending Server-Side Messages With <code>HubContext</code></h2>
<p>What good is a <code>NotificationsHub</code> if we can't send notifications from the backend to connected clients?<br>
Not much.</p>
<p>You can use the <code>IHubContext&lt;THub&gt;</code> interface access to the <code>Hub</code> instance in your backend code.</p>
<p>And you can use <code>IHubContext&lt;THub, TClient&gt;</code> for a strongly typed hub.</p>
<p>Here's a simple Minimal API endpoint that injects an <code>IHubContext&lt;NotificationsHub, INotificationsClient&gt;</code> for our strongly typed hub and uses it
to send a notification to all connected clients:</p>
<pre><code class="language-csharp">app.MapPost(&quot;notifications/all&quot;, async (
    string content,
    IHubContext&lt;NotificationsHub, INotificationsClient&gt; context) =&gt;
{
    await context.Clients.All.ReceiveNotification(content);

    return Results.NoContent();
});
</code></pre>
<h2>Sending Messages To a Specific User</h2>
<p>The real value of SignalR is being able to <strong>send messages</strong>, or notifications in this example, to a <strong>specific user</strong>.</p>
<p>I've seen some complicated implementations that manage a dictionary with a user identifier and a map of active connections.
Why would you do that when SignalR already supports this functionality?</p>
<p>You can call the <code>User</code> method and pass it the <code>userId</code> to scope the <code>ReceiveNotification</code> message to that specific user.</p>
<pre><code class="language-csharp">app.MapPost(&quot;notifications/user&quot;, async (
    string userId,
    string content,
    IHubContext&lt;NotificationsHub, INotificationsClient&gt; context) =&gt;
{
    await context.Clients.User(userId).ReceiveNotification(content);

    return Results.NoContent();
});
</code></pre>
<p>How does <strong>SignalR</strong> know which user to send the message to?</p>
<p>It uses the <code>DefaultUserIdProvider</code> internally to extract the user identifier from the claims.
To be specific, it's using the <code>ClaimTypes.NameIdentifier</code> claim.
This also implies that you should be authenticated when connecting to the <code>Hub</code>, for example, by passing a JWT.</p>
<pre><code class="language-csharp">public class DefaultUserIdProvider : IUserIdProvider
{
    public virtual string? GetUserId(HubConnectionContext connection)
    {
        return connection.User?.FindFirst(ClaimTypes.NameIdentifier)?.Value;
    }
}
</code></pre>
<p>By default, all the methods in a hub can be called by unauthenticated users.
So you need to decorate it with an <code>Authorize</code> attribute to only allow authenticated clients to access the hub.</p>
<pre><code class="language-csharp">[Authorize]
public sealed class NotificationsHub : Hub&lt;INotificationsClient&gt;
{
    public async Task SendNotification(string content)
    {
        await Clients.All.ReceiveNotification(content);
    }
}
</code></pre>
<h2>In Summary</h2>
<p>Adding <strong>real-time functionality</strong> to your application creates room for innovation and adds value to your users.</p>
<p>With <strong>SignalR</strong>, you can start building real-time apps in .NET in minutes.</p>
<p>You need to grasp one concept - the <code>Hub</code> class.
SignalR abstracts away the message transport mechanism, so you don't have to worry about it.</p>
<p>Make sure to send authenticated requests to <strong>SignalR</strong> hubs and turn on authentication on the <code>Hub</code>.
SignalR will internally track the users connecting to your hubs, allowing you to send them messages based on the user identifier.</p>
<p>That's all for today.</p>
<p>Thanks for reading, and have an awesome Saturday.</p>
<p><strong>Today's action step:</strong>
Look at your project and try to find an opportunity to add real-time functionality.
Commit 30 min. to build a simple proof of concept with SignalR and see if it can improve your project.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_043.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Refactoring From an Anemic Domain Model To a Rich Domain Model]]></title>
            <link>https://milanjovanovic.tech/blog/refactoring-from-an-anemic-domain-model-to-a-rich-domain-model</link>
            <guid isPermaLink="false">refactoring-from-an-anemic-domain-model-to-a-rich-domain-model</guid>
            <pubDate>Sat, 17 Jun 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Is the Anemic domain model an antipattern? It's a domain model without any behavior and only data properties.
Anemic domain models work great in simple applications, but they are difficult to maintain and evolve if you have rich business logic.
The important parts of your business logic and rules end up being scattered all over the application. It reduces cohesiveness and reusability, and makes adding new features more difficult.
Rich domain model attempts to solve this by encapsulating as much of the business logic as possible.
But how can you design a Rich domain model?
This is a never-ending process of moving business logic into the domain and refining your domain model.
Let's see how to refactor from an Anemic domain model to a Rich domain model.]]></description>
            <content:encoded><![CDATA[<p>Is the <strong>anemic domain model</strong> an <strong>antipattern</strong>?</p>
<p>It's a domain model without any behavior and only data properties.</p>
<p>Anemic domain models work great in simple applications, but they are difficult to maintain and evolve if you have rich business logic.</p>
<p>The important parts of your business logic and rules end up being scattered all over the application.
It reduces cohesiveness and reusability, and makes adding new features more difficult.</p>
<p><strong>Rich domain model</strong> attempts to solve this by encapsulating as much of the business logic as possible.</p>
<p>But how can you design a <strong>rich domain model</strong>?</p>
<p>This is a never-ending process of moving business logic into the domain and refining your domain model.</p>
<p>Let's see how to <strong>refactor</strong> from an <strong>anemic domain model</strong> to a <strong>rich domain model</strong>.</p>
<h2>Working With Anemic Domain Model</h2>
<p>To understand what working with an <strong>anemic domain model</strong> looks like, I'll use an example of handling a <code>SendInvitationCommand</code>.</p>
<p>I omitted the class and its dependencies so that we can focus on the <code>Handle</code> method.
It loads some entities from the database, performs validation, executes the business logic, and finally persists the changes in the database and sends an email.</p>
<p>It already implements some good practices like using repositories and returning result objects.</p>
<p>However, it's working with an <strong>anemic domain model</strong>.</p>
<p>A few things indicating this:</p>
<ul>
<li>Parameterless constructors</li>
<li>Public property setters</li>
<li>Exposed collections</li>
</ul>
<p>In other words - the classes representing domain entities contain only data properties and no behavior.</p>
<p>The <strong>problems</strong> of an <strong>anemic domain model</strong> are:</p>
<ul>
<li>Discoverability of operations</li>
<li>Potential code duplication</li>
<li>Lack of encapsulation</li>
</ul>
<p>We'll apply a few techniques to push logic down into the domain, and try to make the model more domain-driven.
I hope you'll be able to see the value and benefits this will bring.</p>
<pre><code class="language-csharp">public async Task&lt;Result&gt; Handle(SendInvitationCommand command)
{
    var member = await _memberRepository.GetByIdAsync(command.MemberId);

    var gathering = await _gatheringRepository.GetByIdAsync(command.GatheringId);

    if (member is null || gathering is null)
    {
        return Result.Failure(Error.NullValue);
    }

    if (gathering.Creator.Id == member.Id)
    {
        throw new Exception(&quot;Can't send invitation to the creator.&quot;);
    }

    if (gathering.ScheduledAtUtc &lt; DateTime.UtcNow)
    {
        throw new Exception(&quot;Can't send invitation for the past.&quot;);
    }

    var invitation = new Invitation
    {
        Id = Guid.NewGuid(),
        Member = member,
        Gathering = gathering,
        Status = InvitationStatus.Pending,
        CreatedOnUtc = DateTime.UtcNow
    };

    gathering.Invitations.Add(invitation);

    _invitationRepository.Add(invitation);

    await _unitOfWork.SaveChangesAsync();

    await _emailService.SendInvitationSentEmailAsync(member, gathering);

    return Result.Success();
}
</code></pre>
<h2>Moving Business Logic Into The Domain</h2>
<p>The goal is to move as much of the business logic as possible into the domain.</p>
<p>Let's start with the <code>Invitation</code> entity and defining a constructor for it.
I can simplify the design by setting the <code>Status</code> and <code>CreatedOnUtc</code> properties inside the constructor.
I'm also going to make it <code>internal</code> so that an <code>Invitation</code> instance can only be created within the domain.</p>
<pre><code class="language-csharp">public sealed class Invitation
{
    internal Invitation(Guid id, Gathering gathering, Member member)
    {
        Id = id;
        Member = member;
        Gathering = gathering;
        Status = InvitationStatus.Pending;
        CreatedOnUtc = DateTime.Now;
    }

    // Data properties omitted for brevity.
}
</code></pre>
<p>The reason I made the <code>Invitation</code> constructor <code>internal</code> is so that I can introduce a new method on the <code>Gathering</code> entity.
Let's call it <code>SendInvitation</code> and it will be responsible for instantiating a new <code>Invitation</code> instance and adding it to the internal collection.</p>
<p>Currently, the <code>Gathering.Invitations</code> collection is <code>public</code>, which means anyone can obtain a reference and modify the collection.</p>
<p>We don't want to allow this, so what we can do is encapsulate this collection behind a <code>private</code> field.
This moves the responsibility for managing the <code>_invitations</code> collection to the <code>Gathering</code> class.</p>
<p>Here's how the <code>Gathering</code> class looks like now:</p>
<pre><code class="language-csharp">public sealed class Gathering
{
    private readonly List&lt;Invitation&gt; _invitations;

    // Other members omitted for brevity.

    public void SendInvitation(Member member)
    {
        var invitation = new Invitation(Guid.NewGuid(), gathering, member);

        _invitations.Add(invitation);
    }
}
</code></pre>
<h2>Moving Validation Rules Into The Domain</h2>
<p>The next thing we can do is move the validation rules into the <code>SendInvitation</code> method, further enriching the domain model.</p>
<p>Unfortunately, this is still a bad practice because of throwing &quot;expected&quot; exceptions when a validation fails.
If you want to use exceptions to enforce your validation rules you should at least do it right, and use specific exceptions instead of generic ones.</p>
<p>But it would be even better to use a <strong>result object</strong> to express validation errors.</p>
<pre><code class="language-csharp">public sealed class Gathering
{
    // Other members omitted for brevity.

    public void SendInvitation(Member member)
    {
        if (gathering.Creator.Id == member.Id)
        {
            throw new Exception(&quot;Can't send invitation to the creator.&quot;);
        }

        if (gathering.ScheduledAtUtc &lt; DateTime.UtcNow)
        {
            throw new Exception(&quot;Can't send invitation for the past.&quot;);
        }

        var invitation = new Invitation(Guid.NewGuid(), gathering, member);

        _invitations.Add(invitation);
    }
}
</code></pre>
<p>Here's how using <strong>result objects</strong> would look like:</p>
<pre><code class="language-csharp">public sealed class Gathering
{
    // Other members omitted for brevity.

    public Result SendInvitation(Member member)
    {
        if (gathering.Creator.Id == member.Id)
        {
            return Result.Failure(DomainErrors.Gathering.InvitingCreator);
        }

        if (gathering.ScheduledAtUtc &lt; DateTime.UtcNow)
        {
            return Result.Failure(DomainErrors.Gathering.AlreadyPassed);
        }

        var invitation = new Invitation(Guid.NewGuid(), gathering, member);

        _invitations.Add(invitation);

        return Result.Success();
    }
}
</code></pre>
<p>The benefit of this approach is we can introduce constants for possible domain errors.
The catalog of domain errors will act as <strong>documentation</strong> for your domain, and make it more expressive.</p>
<p>Finally, here's how the <code>Handle</code> method looks like with all the changes so far:</p>
<pre><code class="language-csharp">public async Task&lt;Result&gt; Handle(SendInvitationCommand command)
{
    var member = await _memberRepository.GetByIdAsync(command.MemberId);

    var gathering = await _gatheringRepository.GetByIdAsync(command.GatheringId);

    if (member is null || gathering is null)
    {
        return Result.Failure(Error.NullValue);
    }

    var result = gathering.SendInvitation(member);

    if (result.IsFailure)
    {
        return Result.Failure(result.Errors);
    }

    await _unitOfWork.SaveChangesAsync();

    await _emailService.SendInvitationSentEmailAsync(member, gathering);

    return Result.Success();
}
</code></pre>
<p>If you take a closer look at the <code>Handle</code> method you'll notice it's doing two things:</p>
<ul>
<li>Persisting changes to the database</li>
<li>Sending an email</li>
</ul>
<p>This means it's <strong>not atomic</strong>.</p>
<p>There's a potential for the database transaction to complete, and the email sending to fail.
Also, sending the email will slow down the method which could affect performance.</p>
<p>How can make this method atomic?</p>
<p>By sending the email in the background. It's not important for our business logic, so this is safe to do.</p>
<h2>Expressing Side Effects With Domain Events</h2>
<p>You can use <strong>domain events</strong> to express that something occurred in your domain that might be interesting to other components in your system.</p>
<p>I often use <strong>domain events</strong> to trigger actions in the background, like sending a notification or email.</p>
<p>Let's introduce an <code>InvitationSentDomainEvent</code>:</p>
<pre><code class="language-csharp">public record InvitationSentDomainEvent(Invitation Invitation) : IDomainEvent;
</code></pre>
<p>We're going to raise this <strong>domain event</strong> inside the <code>SendInvitation</code> method:</p>
<pre><code class="language-csharp">public sealed class Gathering
{
    private readonly List&lt;Invitation&gt; _invitations;

    // Other members omitted for brevity.

    public Result SendInvitation(Member member)
    {
        if (gathering.Creator.Id == member.Id)
        {
            return Result.Failure(DomainErrors.Gathering.InvitingCreator);
        }

        if (gathering.ScheduledAtUtc &lt; DateTime.UtcNow)
        {
            return Result.Failure(DomainErrors.Gathering.AlreadyPassed);
        }

        var invitation = new Invitation(Guid.NewGuid(), gathering, member);

        _invitations.Add(invitation);

        Raise(new InvitationSentDomainEvent(invitation));

        return Result.Success();
    }
}
</code></pre>
<p>The goal is to remove the code responsible for sending the email from the <code>Handle</code> method:</p>
<pre><code class="language-csharp">public async Task&lt;Result&gt; Handle(SendInvitationCommand command)
{
    var member = await _memberRepository.GetByIdAsync(command.MemberId);

    var gathering = await _gatheringRepository.GetByIdAsync(command.GatheringId);

    if (member is null || gathering is null)
    {
        return Result.Failure(Error.NullValue);
    }

    var result = gathering.SendInvitation(member);

    if (result.IsFailure)
    {
        return Result.Failure(result.Errors);
    }

    await _unitOfWork.SaveChangesAsync();

    return Result.Success();
}
</code></pre>
<p>We only want to worry about executing the business logic and persisting any changes to the database.
Part of those changes will also be the <strong>domain event</strong>, which the system will publish in the background.</p>
<p>Of course, we need a respective <strong>handler</strong> for the <strong>domain event</strong>:</p>
<pre><code class="language-csharp">public sealed class InvitationSentDomainEventHandler
    : IDomainEventHandler&lt;InvitationSentDomainEvent&gt;
{
    private readonly IEmailService _emailService;

    public InvitationSentDomainEventHandler(IEmailService emailService)
    {
        _emailService = emailService;
    }

    public async Task Handle(InvitationSentDomainEvent domainEvent)
    {
        await _emailService.SendInvitationSentEmailAsync(
            domainEvent.Invitation.Member,
            domainEvent.Invitation.Gathering);
    }
}
</code></pre>
<p>We achieved two things:</p>
<ul>
<li>Handling the <code>SendInvitationCommand</code> is now atomic</li>
<li>Email is sent in the background, and can be safely retried in case of an error</li>
</ul>
<h2>Takeaway</h2>
<p>Designing a <strong>rich domain model</strong> is a gradual process, and you can slowly evolve the domain model over time.</p>
<p>The first step could be making your domain model more defensive:</p>
<ul>
<li>Hiding constructors with the <code>internal</code> keyword</li>
<li>Encapsulating collection access</li>
</ul>
<p>The benefit is your domain models will have a fine-grained public API (methods) which act as an entry point for executing the business logic.</p>
<p>It's easy to test behavior when it's encapsulated in a class without having to mock external dependencies.</p>
<p>You can raise <strong>domain events</strong> to notify the system that something of important occurred, and any interested components can subscribe to that domain event.
Domain events allow you to develop a <strong>decoupled</strong> system, where you focus on the core domain logic, and don't have to worry about the side effects.</p>
<p>However, this doesn't mean that every system needs a <strong>rich domain model</strong>.</p>
<p>You should be pragmatic and decide when the complexity is worth it.</p>
<p>That's all for this week.</p>
<p>See you next Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_042.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[The Right Way To Use HttpClient In .NET]]></title>
            <link>https://milanjovanovic.tech/blog/the-right-way-to-use-httpclient-in-dotnet</link>
            <guid isPermaLink="false">the-right-way-to-use-httpclient-in-dotnet</guid>
            <pubDate>Sat, 10 Jun 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[If you're building a .NET application, chances are high that you'll need to call an external API over HTTP.
The easy way to make HTTP requests in .NET is to use the HttpClient to send those requests. And it's a great abstraction to work with, especially with the methods supporting JSON payloads and responses.
Unfortunately, it's easy to misuse the HttpClient.
Port exhaustion and DNS behavior are some of the most common problems.
So here's what you need to know about work with HttpClient: - How not to use HttpClient - How to simplify configuration with IHttpClientFactory - How to configure typed clients - Why you should avoid typed clients in singleton services - When to use which option
Let's dive in.]]></description>
            <content:encoded><![CDATA[<p>If you're building a <strong>.NET</strong> application, chances are high that you'll need to call an <strong>external API</strong> over <strong>HTTP</strong>.</p>
<p>The easy way to make HTTP requests in .NET is to use the <code>HttpClient</code> to send those requests.
And it's a great abstraction to work with, especially with the methods supporting <strong>JSON</strong> payloads and responses.</p>
<p>Unfortunately, it's easy to misuse the <code>HttpClient</code>.</p>
<p><strong>Port exhaustion</strong> and <strong>DNS behavior</strong> are some of the most common problems.</p>
<p>So here's what you need to know about working with <code>HttpClient</code>:</p>
<ul>
<li>How not to use <code>HttpClient</code></li>
<li>How to simplify configuration with <code>IHttpClientFactory</code></li>
<li>How to configure <strong>typed clients</strong></li>
<li>Why you should avoid <strong>typed clients</strong> in singleton services</li>
<li>When to use which option</li>
</ul>
<p>Let's dive in!</p>
<h2>What Is HttpClient in .NET?</h2>
<p><code>HttpClient</code> is the primary class in .NET for sending HTTP requests and receiving HTTP responses from a URI.
It lives in the <code>System.Net.Http</code> namespace and is included in the .NET runtime (you don't need extra packages).</p>
<p>You can use it to call REST APIs, download files, or communicate with any HTTP-based service.
It supports all standard HTTP methods (GET, POST, PUT, DELETE, PATCH) and has built-in support for JSON
through extension methods like <code>GetFromJsonAsync</code> and <code>PostAsJsonAsync</code>.</p>
<p>While <code>HttpClient</code> is easy to get started with, it's also easy to misuse.
The most common mistakes are creating too many instances (causing port exhaustion) and holding on to instances for too long (causing stale DNS responses).
The rest of this article covers the right patterns for working with <code>HttpClient</code> in .NET.</p>
<h2>The Naive Way To Use HttpClient</h2>
<p>The simplest way to work with the <code>HttpClient</code> is to just create a new instance, set the required properties and use it to send requests.</p>
<p>What could possibly go wrong?</p>
<p><code>HttpClient</code> instances are meant to be <strong>long-lived</strong>, and reused throughout the lifetime of the application.</p>
<p>Each instance uses its own <strong>connection pool</strong> for isolation purposes, but also to prevent <strong>port exhaustion</strong>.
If a server is under high load, and your application is constantly creating new connections, it could lead to exhausting the available ports.
This will cause an exception at runtime, when trying to send a request.</p>
<p>So how can you avoid this?</p>
<pre><code class="language-csharp">public class GitHubService
{
    private readonly GitHubSettings _settings;

    public GitHubService(IOptions&lt;GitHubSettings&gt; settings)
    {
        _settings = settings.Value;
    }

    public async Task&lt;GitHubUser?&gt; GetUserAsync(string username)
    {
        using var client = new HttpClient();

        client.DefaultRequestHeaders.Add(&quot;Authorization&quot;, _settings.GitHubToken);
        client.DefaultRequestHeaders.Add(&quot;User-Agent&quot;, _settings.UserAgent);
        client.BaseAddress = new Uri(&quot;https://api.github.com&quot;);

        GitHubUser? user = await client
            .GetFromJsonAsync&lt;GitHubUser&gt;($&quot;users/{username}&quot;);

        return user;
    }
}
</code></pre>
<h2>The Smart Way To Create HttpClient Using IHttpClientFactory</h2>
<p>Instead of managing the <code>HttpClient</code> lifetime yourself, you can use an <code>IHttpClientFactory</code> to create the <code>HttpClient</code> instance.</p>
<p>Simply call the <code>CreateClient</code> method and use the returned <code>HttpClient</code> instance to send your HTTP requests.</p>
<p>Why is this a better approach?</p>
<p>The expensive part of the <code>HttpClient</code> is the actual message handler - <code>HttpMessageHandler</code>.
Each <code>HttpMessageHandler</code> has an internal HTTP <strong>connection pool</strong> that can be reused.</p>
<p>The <code>IHttpClientFactory</code> will <strong>cache</strong> the <code>HttpMessageHandler</code> and reuse it when creating a new <code>HttpClient</code> instance.</p>
<p>An important note here is that <code>HttpClient</code> instances created by <code>IHttpClientFactory</code> are meant to be <strong>short-lived</strong>.</p>
<pre><code class="language-csharp">public class GitHubService
{
    private readonly GitHubSettings _settings;
    private readonly IHttpClientFactory _factory;

    public GitHubService(
        IOptions&lt;GitHubSettings&gt; settings,
        IHttpClientFactory factory)
    {
        _settings = settings.Value;
        _factory = factory;
    }

    public async Task&lt;GitHubUser?&gt; GetUserAsync(string username)
    {
        using var client = _factory.CreateClient();

        client.DefaultRequestHeaders.Add(&quot;Authorization&quot;, _settings.GitHubToken);
        client.DefaultRequestHeaders.Add(&quot;User-Agent&quot;, _settings.UserAgent);
        client.BaseAddress = new Uri(&quot;https://api.github.com&quot;);

        GitHubUser? user = await client
            .GetFromJsonAsync&lt;GitHubUser&gt;($&quot;users/{username}&quot;);

        return user;
    }
}
</code></pre>
<h2>Reducing Code Duplication With Named Clients</h2>
<p>Using <code>IHttpClientFactory</code> will solve most of the issues of manually creating an <code>HttpClient</code>.
However, we still need to configure the default request parameters every time we obtain a new <code>HttpClient</code> from the <code>CreateClient</code> method.</p>
<p>You can configure a <strong>named client</strong> by calling the <code>AddHttpClient</code> method and passing in the desired name.
The <code>AddHttpClient</code> accepts a delegate that you can use to configure the default parameters on the <code>HttpClient</code> instance.</p>
<pre><code class="language-csharp">services.AddHttpClient(&quot;github&quot;, (serviceProvider, client) =&gt;
{
    var settings = serviceProvider
        .GetRequiredService&lt;IOptions&lt;GitHubSettings&gt;&gt;().Value;

    client.DefaultRequestHeaders.Add(&quot;Authorization&quot;, settings.GitHubToken);
    client.DefaultRequestHeaders.Add(&quot;User-Agent&quot;, settings.UserAgent);

    client.BaseAddress = new Uri(&quot;https://api.github.com&quot;);
});
</code></pre>
<p>The main difference is you now have to obtain the client by passing the name of the client to <code>CreateClient</code>.</p>
<p>But the use of the <code>HttpClient</code> looks a lot simpler:</p>
<pre><code class="language-csharp">public class GitHubService
{
    private readonly IHttpClientFactory _factory;

    public GitHubService(IHttpClientFactory factory)
    {
        _factory = factory;
    }

    public async Task&lt;GitHubUser?&gt; GetUserAsync(string username)
    {
        using var client = _factory.CreateClient(&quot;github&quot;);

        GitHubUser? user = await client
            .GetFromJsonAsync&lt;GitHubUser&gt;($&quot;users/{username}&quot;);

        return user;
    }
}
</code></pre>
<h2>Replacing Named Clients With Typed Clients</h2>
<p>The downside of using <strong>named clients</strong> is having to resolve an <code>HttpClient</code> by passing in a name every time.</p>
<p>There's a better way to achieve the same behavior by configuring a <strong>typed client</strong>.
You can do this by calling the <code>AddClient&lt;TClient&gt;</code> method and configuring the service that will consume the <code>HttpClient</code>.</p>
<p>Under the hood, this is still using a <strong>named client</strong>, where the name is the same as the type name.</p>
<p>And this will also register <code>GitHubService</code> with a <strong>transient lifetime</strong>.</p>
<pre><code class="language-csharp">services.AddHttpClient&lt;GitHubService&gt;((serviceProvider, client) =&gt;
{
    var settings = serviceProvider
        .GetRequiredService&lt;IOptions&lt;GitHubSettings&gt;&gt;().Value;

    client.DefaultRequestHeaders.Add(&quot;Authorization&quot;, settings.GitHubToken);
    client.DefaultRequestHeaders.Add(&quot;User-Agent&quot;, settings.UserAgent);

    client.BaseAddress = new Uri(&quot;https://api.github.com&quot;);
});
</code></pre>
<p>Inside of <code>GitHubService</code>, you inject and use the typed <code>HttpClient</code> instance which will have all of the configuration applied.</p>
<p>No more dealing with <code>IHttpClientFactory</code> and creating <code>HttpClient</code> instances manually.</p>
<pre><code class="language-csharp">public class GitHubService
{
    private readonly HttpClient client;

    public GitHubService(HttpClient client)
    {
        _client = client;
    }

    public async Task&lt;GitHubUser?&gt; GetUserAsync(string username)
    {
        GitHubUser? user = await client
            .GetFromJsonAsync&lt;GitHubUser&gt;($&quot;users/{username}&quot;);

        return user;
    }
}
</code></pre>
<h2>Why You Should Avoid Typed Clients In Singleton Services</h2>
<p>You could run into a <strong>problem</strong> if you inject a <strong>typed client</strong> into a <strong>singleton service</strong>.
Since the <strong>typed client</strong> is <strong>transient</strong>, injecting it in a <strong>singleton service</strong> will cause it to be cached for the lifetime of the <strong>singleton service</strong>.</p>
<p>This will prevent the <strong>typed client</strong> from reacting to DNS changes.</p>
<p>If you want to use a <strong>typed client</strong> in a <strong>singleton service</strong>, the recommened approach is using <code>SocketsHttpHandler</code> as the primary handler,
and configuring the <code>PooledConnectionLifetime</code>.</p>
<p>Since the <code>SocketsHttpHandler</code> will handle connection pooling, you can disable recycling at the <code>IHttpClientFactory</code> level by setting <code>HandlerLifetime</code> to <code>Timeout.InfiniteTimeSpan</code>.</p>
<pre><code class="language-csharp">services.AddHttpClient&lt;GitHubService&gt;((serviceProvider, client) =&gt;
{
    var settings = serviceProvider
        .GetRequiredService&lt;IOptions&lt;GitHubSettings&gt;&gt;().Value;

    client.DefaultRequestHeaders.Add(&quot;Authorization&quot;, settings.GitHubToken);
    client.DefaultRequestHeaders.Add(&quot;User-Agent&quot;, settings.UserAgent);

    client.BaseAddress = new Uri(&quot;https://api.github.com&quot;);
})
.ConfigurePrimaryHttpMessageHandler(() =&gt;
{
    return new SocketsHttpHandler()
    {
        PooledConnectionLifetime = TimeSpan.FromMinutes(15)
    };
})
.SetHandlerLifetime(Timeout.InfiniteTimeSpan);
</code></pre>
<h2>SocketsHttpHandler vs HttpClientHandler</h2>
<p>Under the hood, every <code>HttpClient</code> uses an <code>HttpMessageHandler</code> to manage TCP connections.
.NET has two primary handler implementations worth knowing.</p>
<p><strong><code>HttpClientHandler</code></strong> is the traditional handler, available since .NET Framework.
On modern .NET runtimes it internally delegates to <code>SocketsHttpHandler</code>.
It exposes familiar properties for cookies, automatic decompression, and proxy configuration.</p>
<p><strong><code>SocketsHttpHandler</code></strong> is the fully managed, cross-platform handler that debuted in .NET Core 2.1 and is now the default on all current .NET runtimes.
Unlike <code>HttpClientHandler</code>, it exposes <code>PooledConnectionLifetime</code> and <code>PooledConnectionIdleTimeout</code>,
giving you direct control over how long connections stay in the pool before being recycled.
This is exactly what makes <code>SocketsHttpHandler</code> valuable when using a typed <code>HttpClient</code> in a singleton service, as
you can ensure DNS changes are picked up without relying on <code>IHttpClientFactory</code> handler recycling.</p>
<p>For most applications using <code>IHttpClientFactory</code>, you don't need to choose explicitly.
Configure <code>SocketsHttpHandler</code> directly only when you need fine-grained control over connection-pool behavior.</p>
<h2>When Should You Use Which Option?</h2>
<p>I showed you a few possible options for working with <code>HttpClient</code>.</p>
<p>But which one should you use and when?</p>
<p>Microsoft was kind enough to provide us with a set of best practices and <a href="https://learn.microsoft.com/en-us/dotnet/fundamentals/networking/http/httpclient-guidelines#recommended-use">recommended use</a>
for <code>HttpClient</code>.</p>
<ul>
<li>Use a <code>static</code> or <strong>singleton</strong> <code>HttpClient</code> instance with a <code>PooledConnectionLifetime</code> configured, since this solves both port exhaustion and tracking DNS changes</li>
<li>Use <code>IHttpClientFactory</code> if you want to move the configuration to one place, but remember that clients are meant to be <strong>short-lived</strong></li>
<li>Use a <strong>typed client</strong> if you want the <code>IHttpClientFactory</code> configurability</li>
</ul>
<p>I prefer working with a <strong>typed client</strong>, and I'm mindful of the fact that it's configured as a <strong>transient service</strong>.</p>
<p>Thanks for reading, and have an awesome Saturday.</p>
<h2>Frequently Asked Questions</h2>
<h3>What is HttpClient in .NET used for?</h3>
<p><code>HttpClient</code> in .NET is used to send HTTP requests and receive HTTP responses from a URI.
It supports GET, POST, PUT, DELETE, and other HTTP methods, and can deserialize JSON responses directly into .NET objects.</p>
<h3>Should HttpClient be static or injected in .NET?</h3>
<p><code>HttpClient</code> should neither be created per-request (causes port exhaustion) nor used as a naive singleton (stale DNS).
The recommended approach is to use <code>IHttpClientFactory</code>, which manages lifetime and connection pooling correctly.</p>
<h3>What does AddHttpClient do in <a href="http://ASP.NET">ASP.NET</a> Core?</h3>
<p><code>AddHttpClient</code> registers <code>IHttpClientFactory</code> with the DI container and allows you to configure named or typed <code>HttpClient</code> instances with base addresses, headers, retry policies, and other settings.</p>
<h3>Is IHttpClientFactory required in .NET?</h3>
<p><code>IHttpClientFactory</code> is not strictly required, but it is the recommended way to create <code>HttpClient</code> instances in <a href="http://ASP.NET">ASP.NET</a> Core applications to avoid port exhaustion and DNS staleness issues.</p>
<h3>What is the difference between HttpClientHandler and SocketsHttpHandler?</h3>
<p><code>SocketsHttpHandler</code> is the default handler on .NET Core 2.1+ and provides better performance and connection pooling.
<code>HttpClientHandler</code> is the traditional handler that delegates to the OS-level handler.
<code>SocketsHttpHandler</code> gives you more control over connection lifetime via <code>PooledConnectionLifetime</code>.</p>
<h3>Why does HttpClient cause socket exhaustion?</h3>
<p>Creating a new <code>HttpClient</code> instance for each request creates a new socket connection that is not immediately released after disposal.
Sockets enter a <code>TIME_WAIT</code> state, and if you create many instances quickly, you exhaust the available port range.</p>
<h3>How do you configure a primary HTTP message handler in .NET?</h3>
<p>Use <code>ConfigurePrimaryHttpMessageHandler</code> when registering the client:</p>
<pre><code class="language-csharp">builder.Services
    .AddHttpClient&lt;MyService&gt;()
    .ConfigurePrimaryHttpMessageHandler(() =&gt; new SocketsHttpHandler
    {
        PooledConnectionLifetime = TimeSpan.FromMinutes(2)
    });
</code></pre>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_041.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Scheduling Background Jobs With Quartz.NET]]></title>
            <link>https://milanjovanovic.tech/blog/scheduling-background-jobs-with-quartz-net</link>
            <guid isPermaLink="false">scheduling-background-jobs-with-quartz-net</guid>
            <pubDate>Sat, 03 Jun 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[If you're building a scalable application, it's a common requirement to offload some work in your application to a background job.
A few examples of that are: - Publishing email notifications - Generating reports - Updating a cache - Image processing
How can you create a recurring background job in .NET?
Quartz.NET is a full-featured, open source job scheduling system that can be used from smallest apps to large scale enterprise systems.
There are three concepts you need to understand in Quartz.NET: - Job - the actual background task you want to run - Trigger - the trigger controlling when a job runs - Scheduler - it's responsible for coordinating jobs and triggers
Let's see how we can use Quartz.NET to create and schedule background jobs.]]></description>
            <content:encoded><![CDATA[<p>If you're building a scalable application, it's a common requirement to offload some work in your application to a <strong>background job</strong>.</p>
<p>Here are a few examples of that:</p>
<ul>
<li>Publishing email notifications</li>
<li>Generating reports</li>
<li>Updating a cache</li>
<li>Image processing</li>
</ul>
<p>How can you create a recurring <strong>background job</strong> in .NET?</p>
<p><a href="https://www.quartz-scheduler.net/"><strong>Quartz.NET</strong></a> is a full-featured, open source job scheduling system that can be used from smallest apps to large scale enterprise systems.</p>
<p>There are three concepts you need to understand in <strong><a href="http://Quartz.NET">Quartz.NET</a></strong>:</p>
<ul>
<li><strong>Job</strong> - the actual background task you want to run</li>
<li><strong>Trigger</strong> - the trigger controlling when a job runs</li>
<li><strong>Scheduler</strong> - responsible for coordinating jobs and triggers</li>
</ul>
<p>Let's see how we can use <strong><a href="http://Quartz.NET">Quartz.NET</a></strong> to create and schedule <strong>background jobs</strong>.</p>
<h2>Adding The <a href="http://Quartz.NET">Quartz.NET</a> Hosted Service</h2>
<p>The first thing we need to do is install the <strong><a href="http://Quartz.NET">Quartz.NET</a></strong> NuGet package.
There are a few to pick from, but we're going to install the <code>Quartz.Extensions.Hosting</code> library:</p>
<pre><code class="language-powershell">Install-Package Quartz.Extensions.Hosting
</code></pre>
<p>The reason we're using this library is because it integrates nicely with .NET using an <code>IHostedService</code> instance.</p>
<p>To get the <strong><a href="http://Quartz.NET">Quartz.NET</a></strong> hosted service up and running, we need two things:</p>
<ul>
<li>Add the required services with the DI container</li>
<li>Add the hosted service</li>
</ul>
<pre><code class="language-csharp">services.AddQuartz(configure =&gt;
{
    configure.UseMicrosoftDependencyInjectionJobFactory();
});

services.AddQuartzHostedService(options =&gt;
{
    options.WaitForJobsToComplete = true;
});
</code></pre>
<p><strong><a href="http://Quartz.NET">Quartz.NET</a></strong> will create jobs by fetching them from the DI container.
This also means you can use scoped services in your jobs, not just singleton or transient services.</p>
<p>Setting the <code>WaitForJobsToComplete</code> option to <code>true</code> will ensure that <strong><a href="http://Quartz.NET">Quartz.NET</a></strong> waits for the jobs to complete gracefully before exiting.</p>
<h2>Creating Background Jobs With <code>IJob</code></h2>
<p>To crate a background job with <strong><a href="http://Quartz.NET">Quartz.NET</a></strong> you need to implement the <code>IJob</code> interface.</p>
<p>It only exposes a single method - <code>Execute</code> - where you can place the code for your background job.</p>
<p>A few things worth noting here:</p>
<ul>
<li>We're using DI to inject the <code>ApplicationDbContext</code> and <code>IPublisher</code> services</li>
<li>The job is decorated with <code>DisallowConcurrentExecution</code> to prevent running the same job concurrently</li>
</ul>
<pre><code class="language-csharp">[DisallowConcurrentExecution]
public class ProcessOutboxMessagesJob : IJob
{
    private readonly ApplicationDbContext _dbContext;
    private readonly IPublisher _publisher;

    public ProcessOutboxMessagesJob(
        ApplicationDbContext dbContext,
        IPublisher publisher)
    {
        _dbContext = dbContext;
        _publisher = publisher;
    }

    public async Task Execute(IJobExecutionContext context)
    {
        List&lt;OutboxMessage&gt; messages = await _dbContext
            .Set&lt;OutboxMessage&gt;()
            .Where(m =&gt; m.ProcessedOnUtc == null)
            .Take(20)
            .ToListAsync(context.CancellationToken);

        foreach (OutboxMessage outboxMessage in messages)
        {
            IDomainEvent? domainEvent = JsonConvert
                .DeserializeObject&lt;IDomainEvent&gt;(
                    outboxMessage.Content,
                    new JsonSerializerSettings
                    {
                        TypeNameHandling = TypeNameHandling.All
                    });

            if (domainEvent is null)
            {
                continue;
            }

            await _publisher.Publish(domainEvent, context.CancellationToken);

            outboxMessage.ProcessedOnUtc = DateTime.UtcNow;

            await _dbContext.SaveChangesAsync();
        }
    }
}
</code></pre>
<p>Now that the <strong>background job</strong> is ready, we need to register it with the <strong>DI</strong> container and add a trigger that will run the job.</p>
<h2>Configuring the Job</h2>
<p>I mentioned at the start that there are three key concepts in <strong><a href="http://Quartz.NET">Quartz.NET</a></strong>:</p>
<ul>
<li>Job</li>
<li>Trigger</li>
<li>Scheduler</li>
</ul>
<p>We already implemented the <code>ProcessOutboxMessagesJob</code> background job in the previous section.</p>
<p>The <strong><a href="http://Quartz.NET">Quartz.NET</a></strong> library will take care of the scheduler.</p>
<p>And this leaves us with configuring the <strong>trigger</strong> for our <code>ProcessOutboxMessagesJob</code>.</p>
<pre><code class="language-csharp">services.AddQuartz(configure =&gt;
{
    var jobKey = new JobKey(nameof(ProcessOutboxMessagesJob));

    configure
        .AddJob&lt;ProcessOutboxMessagesJob&gt;(jobKey)
        .AddTrigger(
            trigger =&gt; trigger.ForJob(jobKey).WithSimpleSchedule(
                schedule =&gt; schedule.WithIntervalInSeconds(10).RepeatForever()));

    configure.UseMicrosoftDependencyInjectionJobFactory();
});
</code></pre>
<p>We need to uniquely identify our <strong>background job</strong> with a <code>JobKey</code>.
I like to keep it simple and use the job name.</p>
<p>Calling <code>AddJob</code> will register the <code>ProcessOutboxMessagesJob</code> with DI and also with Quartz.</p>
<p>After that we configure a trigger for this job by calling <code>AddTrigger</code>.
You need to associate the job with the trigger by calling <code>ForJob</code>, and then you configure the schedule for the background job.
In this example, I'm scheduling the job to run every ten seconds and repeat forever while the hosted service is running.</p>
<p><strong>Quartz</strong> also has support for configuring triggers using <a href="https://www.quartz-scheduler.net/documentation/quartz-3.x/tutorial/crontriggers.html">cron expressions.</a></p>
<h2>Job Persistence</h2>
<p>By default, Quartz configures all jobs using the <code>RAMJobStore</code> which is the most performant because it keeps all of its data in RAM.
However, this also means it's volatile and you can lose all scheduling information when your application stops or crashes.</p>
<p>It could be useful to have a persistent job store in some scenarios and there's a built in <code>AdoJobStore</code> which works with SQL databases.
You need to create a set of database tables for <a href="http://Quartz.NET">Quartz.NET</a> to use.</p>
<p>You can learn more about this in the <a href="https://www.quartz-scheduler.net/documentation/quartz-3.x/tutorial/job-stores.html">job stores documentation.</a></p>
<h2>Takeaway</h2>
<p><strong><a href="http://Quartz.NET">Quartz.NET</a></strong> makes running <strong>background jobs</strong> in .NET easy, and you can use all the power of DI in your <strong>background jobs</strong>.
It's also flexible for various scheduling requirements with configuration via code or using cron expressions.</p>
<p>There's some room for improvement to make scheduling jobs easier and reduce boilerplate:</p>
<ul>
<li>Add an extension method to simplify configuring jobs with a simple schedule</li>
<li>Add an extension method to simplify configuring jobs with a cron schedule from application settings</li>
</ul>
<p>If you want to see a tutorial on using <strong><a href="http://Quartz.NET">Quartz.NET</a></strong>, I made an in-depth video about <a href="https://youtu.be/XALvnX7MPeo"><strong>using Quartz for processing Outbox messages</strong></a>.</p>
<p>That's all for this week.</p>
<p>See you next Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_040.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How To Build a CI/CD Pipeline With GitHub Actions And .NET]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-build-ci-cd-pipeline-with-github-actions-and-dotnet</link>
            <guid isPermaLink="false">how-to-build-ci-cd-pipeline-with-github-actions-and-dotnet</guid>
            <pubDate>Sat, 27 May 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Do you want to streamline your software development process and accelerate your release cycles?
Imagine being able to automatically build, test, and deploy your .NET applications with every code change.
With CI/CD, you can significantly reduce manual effort and focus more on creating software, ensuring faster and more reliable releases.
And it's never been easier to get started with CI/CD.
GitHub Actions are completely free and simple to use.
So here's what we'll cover: - Introduction to CI/CD & GitHub Actions - Creating a build & test pipeline for .NET - Creating a deployment pipeline for Azure App Service
Let's dive in!]]></description>
            <content:encoded><![CDATA[<p>Do you want to streamline your software development process and accelerate your release cycles?</p>
<p>Imagine being able to automatically build, test, and deploy your .NET applications with every code change.</p>
<p>With <strong>CI/CD</strong>, you can significantly reduce manual effort and focus more on creating software, ensuring faster and more reliable releases.</p>
<p>And it's never been easier to get started with <strong>CI/CD</strong>.</p>
<p><strong>GitHub Actions</strong> are completely free and simple to use.</p>
<p>So here's what we'll cover:</p>
<ul>
<li>Introduction to <strong>CI/CD</strong> &amp; <strong>GitHub Actions</strong></li>
<li>Creating a <strong>build &amp; test pipeline</strong> for <strong>.NET</strong></li>
<li>Creating a <strong>deployment pipeline</strong> for <strong>Azure App Service</strong></li>
</ul>
<p>Let's dive in.</p>
<h2>What Is Continuous Integration And Delivery?</h2>
<p>I'll try to briefly explain what <strong>CI/CD</strong> is, before we take a look at <strong>GitHub Actions</strong>.</p>
<p><strong>CI/CD</strong> is a method to increase the frequency of delivering new features by adding automation to your software development workflow.</p>
<p><strong>Continuous Integration (&quot;CI&quot;)</strong> refers to the automation process of syncing new code to a repository.
Any new changes to the application code are immediately built, tested and merged.</p>
<p><strong>Continuous Delivery, or Deployment, (&quot;CD&quot;)</strong> refers to the process of automating the deployment part of the workflow.
When you make a change which gets merged to the repository, this step takes care of deploying those changes to the production environment (or any other environment).</p>
<h2>Continuous Integration With GitHub Actions</h2>
<p>If you're using <strong>GitHub</strong>, getting started with <strong>Continuous Integration</strong> has never been easier.</p>
<p>You can use <a href="https://github.com/features/actions"><strong>GitHub Actions</strong></a> to automate your build, test, and deployment <strong>pipeline</strong>.
You can create workflows that build and test every commit to your repository, or deploy to production when a new tag is created.</p>
<p>To create a <strong>GitHub Action</strong>, you write a <em>workflow</em> to be triggered when some <em>event</em> occurs in your repository.
An example event is a commit to the main branch, creation of a tag, or you can manually run the workflow.</p>
<p>Here's a <strong>GitHub Actions</strong> workflow to build and test a .NET project:</p>
<pre><code class="language-yaml">name: Build &amp; Test 🧪

on:
  push:
    branches:
      - main

env:
  DOTNET_VERSION: '7.0.x'

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Setup .NET 📦
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: ${{ env.DOTNET_VERSION }}

      - name: Install dependencies 📂
        run: dotnet restore WebApi

      - name: Build 🧱
        run: dotnet build WebApi --configuration Release --no-restore

      - name: Test 🧪
        run: dotnet test WebApi --configuration Release --no-build
</code></pre>
<p>Let's unwrap what is happening here:</p>
<ul>
<li>Defining an event to trigger the workflow</li>
<li>Setting up the <strong>.NET SDK</strong> with the version from <code>env.DOTNET_VERSION</code></li>
<li>Restoring, building and testing the project using the <code>dotnet</code> CLI tool</li>
</ul>
<p>You can add this to your <strong>GitHub</strong> repository today, and start getting instant feedback when you commit code to the repository.</p>
<p>When a workflow run fails due to a build error or a failed test, you'll get an email notification.</p>
<h2>Continuous Delivery To Azure With GitHub Actions</h2>
<p><strong>Continuous Integration</strong> is a great way to start out with <strong>CI/CD</strong>, but the real value lies in <strong>automating</strong> your <strong>deployment</strong> process.</p>
<p>Imagine this:</p>
<ul>
<li>You make a change to your codebase</li>
<li>The commit triggers the <strong>deployment pipeline</strong></li>
<li>A few minutes later your changes are live in production</li>
</ul>
<p>Usually it's a little more nuanced, because we need to think about configuration, database migrations, etc.
But try to see the big picture here.</p>
<p>If you're running your application in the cloud, for example on <strong>Azure</strong>, chances are there's an existing <strong>GitHub Action</strong> you can use.</p>
<p>Here's a <strong>deployment pipeline</strong> I use to publish my application to an <strong>Azure App Service</strong> instance:</p>
<pre><code class="language-yaml">name: Publish 🚀

on:
  push:
    branches:
      - main

env:
  AZURE_WEBAPP_NAME: web-api
  AZURE_WEBAPP_PACKAGE_PATH: './publish'
  DOTNET_VERSION: '7.0.x'

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v3

      - name: Setup .NET 📦
        uses: actions/setup-dotnet@v3
        with:
          dotnet-version: ${{ env.DOTNET_VERSION }}

      - name: Build and Publish 📂
        run: |
          dotnet restore WebApi
          dotnet build WebApi -c Release --no-restore
          dotnet publish WebApi -c Release --no-build
            --output '${{ env.AZURE_WEBAPP_PACKAGE_PATH }}'

      - name: Deploy to Azure 🌌
        uses: azure/webapps-deploy@v2
        with:
          app-name: ${{ env.AZURE_WEBAPP_NAME }}
          publish-profile: ${{ secrets.AZURE_PUBLISH_PROFILE }}
          package: '${{ env.AZURE_WEBAPP_PACKAGE_PATH }}'
</code></pre>
<p>This workflow is pretty similar to the previous one, with the differences being:</p>
<ul>
<li>Adding a publish step and configuring the output path</li>
<li>Using the <code>azure/webapps-deploy@v2</code> action to deploy to <strong>Azure</strong></li>
</ul>
<p>If you need to safely and securely expose secret values in your workflows, you can use <strong>GitHub secrets</strong>.
You can define the secrets on GitHub, and use them in actions without having to add them to source control.</p>
<p>In the deployment workflow I'm using <code>secrets.AZURE_PUBLISH_PROFILE</code> to access my publish profile for the App Service instance.</p>
<h2>In Summary</h2>
<p><strong>Continuous Integration and Delivery</strong> can transform your development process by increasing the speed at which you release changes.</p>
<p>Try adding up how much time you spend on deployments.
I'm pretty sure you'll be surprised by the time savings potential of automating them.</p>
<p>And the good part is you will typically set up your <strong>build and deployment pipelines</strong> once, and then continue benefiting from them for the lifetime of your project.</p>
<p>If you want a step by step guide, I made a video showing <a href="https://youtu.be/QP0pi7xe24s"><strong>how to implement a CI/CD pipeline from scratch.</strong></a>
And the <strong>source code</strong> is also public, so you can add the <strong>GitHub Actions</strong> workflow file to your project.</p>
<p>Thanks for reading.</p>
<p>Hope that was helpful!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_039.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Multi-Tenant Applications With EF Core]]></title>
            <link>https://milanjovanovic.tech/blog/multi-tenant-applications-with-ef-core</link>
            <guid isPermaLink="false">multi-tenant-applications-with-ef-core</guid>
            <pubDate>Sat, 20 May 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Most software applications today are built around the concept of multi-tenancy.
One application serves multiple customers, while keeping their data isolated.
You can approach multi-tenancy in two ways: - Single database & logical isolation of tenants - Multiple databases & physical isolation of tenants
Which option you decide to use will depend mostly on your requirements. Some industries like healthcare require a high degree of data isolation, and using a database per tenant is a must.
So how are we going to implement multi-tenancy support with EF Core?
We can use Query Filters to apply a tenant filter to all database queries.
Implement it once, and you can almost forget about it.]]></description>
            <content:encoded><![CDATA[<p>Most software applications today are built around the concept of <strong>multi-tenancy</strong>.</p>
<p>One application serves multiple customers, while keeping their data <strong>isolated</strong>.</p>
<p>You can approach <strong>multi-tenancy</strong> in two ways:</p>
<ul>
<li><strong>Single database</strong> and <strong>logical isolation</strong> of tenants</li>
<li><strong>Multiple databases</strong> and <strong>physical isolation</strong> of tenants</li>
</ul>
<p>Which option you decide to use will depend mostly on your requirements.
Some industries like healthcare require a high degree of <strong>data isolation</strong>, and using a <strong>database per tenant</strong> is a must.</p>
<p>So how are we going to <strong>implement multi-tenancy</strong> support with <strong>EF Core</strong>?</p>
<p>We can use <strong>Query Filters</strong> to apply a <strong>tenant filter</strong> to all database queries.</p>
<p>Implement it once and you can almost forget about it.</p>
<p>Let's see what are some of the problems we need to solve.</p>
<h2>How To Use EF Core Query Filters</h2>
<p>If you want an in-depth dive into <strong>Query filters</strong>, take a look at the newsletter issue where I talked about
<a href="how-to-use-global-query-filters-in-ef-core"><strong>using query filters with EF Core.</strong></a></p>
<p>Here's a quick refresher on <strong>Query filters</strong>:</p>
<ul>
<li>Configure the <strong>query filter</strong> by calling <code>HasQueryFilter</code> for your entity</li>
<li><strong>EF</strong> will apply it to all queries for that entity</li>
<li>You can turn it off with <code>IgnoreQueryFilters</code></li>
<li>Only <strong>one</strong> query filter <strong>per entity</strong> is allowed</li>
</ul>
<p>And here's a simple example:</p>
<pre><code class="language-csharp">modelBuilder
   .Entity&lt;Order&gt;()
   .HasQueryFilter(order =&gt; !order.IsDeleted);
</code></pre>
<p>All queries to the <code>Order</code> table will include an <code>IsDeleted = FALSE</code> condition.</p>
<h2>Single Database Multi-Tenancy With EF Core</h2>
<p>You will need two things to <strong>implement multi-tenancy</strong> on a <strong>single database</strong>:</p>
<ul>
<li>A way to know <strong>who</strong> the <strong>current tenant</strong> is</li>
<li>A way to <strong>filter</strong> the <strong>data</strong> for that <strong>tenant</strong> only</li>
</ul>
<p>The typical approach for <strong>multi-tenancy</strong> on a <strong>single database</strong> is having a <code>TenantId</code> column in your tables.
And then filtering on that column when querying the database.</p>
<p>You can use the <strong>Query filters</strong> feature in <strong>EF Core</strong> to apply a <strong>global filter</strong> for some entity.</p>
<p>Inside of the <code>OnModelCreating</code> method we configure the query filter on the <code>Order</code> entity:</p>
<pre><code class="language-csharp">public class OrdersDbContext : DbContext
{
    private readonly string _tenantId;

    public OrdersDbContext(
        DbContextOptions&lt;OrdersDbContext&gt; options,
        TenantProvider tenantProvider)
        : base(options)
    {
        _tenantId = tenantProvider.TenantId;
    }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder
            .Entity&lt;Order&gt;
            .HasQueryFilter(o =&gt; o.TenantId == _tenantId);
    }
}
</code></pre>
<p>We're using the <code>TenantProvider</code> class to get the <strong>current tenant</strong> value.</p>
<p>Here's what the <code>TenantProvider</code> implementation looks like:</p>
<pre><code class="language-csharp">public sealed class TenantProvider
{
    private const string TenantIdHeaderName = &quot;X-TenantId&quot;;

    private readonly IHttpContextAccessor _httpContextAccessor;

    public TenantProvider(IHttpContextAccessor httpContextAccessor)
    {
        _httpContextAccessor = httpContextAccessor;
    }

    public string TenantId =&gt; _httpContextAccessor
        .HttpContext
        .Request
        .Headers[TenantIdHeaderName];
}
</code></pre>
<p>The <code>TenantId</code> is coming from the HTTP request header in this example.</p>
<p>A few other options to get the <code>TenantId</code> are:</p>
<ul>
<li>Query string - <code>api/orders?tenantId=example-tenant-id</code></li>
<li>JWT Claim</li>
<li>API Key</li>
</ul>
<p>If you want a more <strong>secure implementation</strong> you should go with <strong>JWT Claims</strong> or <strong>API Keys</strong> to provide the <code>TenantId</code> value.</p>
<h2>Separate Databases Multi-Tenancy With EF Core</h2>
<p>What if we want to <strong>isolate</strong> each tenant to a <strong>separate database</strong>?</p>
<p>Here are the changes we need to make:</p>
<ul>
<li>Applying different <strong>connection string per tenant</strong></li>
<li>Resolving the connection string for each tenant <em>somehow</em></li>
</ul>
<p>You can't use <strong>Query filters</strong> here, since we are working with different databases.</p>
<p>So you will need to store the <strong>tenant information</strong> and <strong>connection strings</strong> somewhere.</p>
<p>A simple example would be store them in the <strong>application settings</strong>:</p>
<pre><code class="language-json">&quot;Tenants&quot;: {
    { &quot;Id&quot;: &quot;tenant-1&quot;, &quot;ConnectionString&quot;: &quot;Host=tenant1.db;Database=tenant1&quot; },
    { &quot;Id&quot;: &quot;tenant-2&quot;, &quot;ConnectionString&quot;: &quot;Host=tenant2.db;Database=tenant2&quot; }
}
</code></pre>
<p>You can then register an <code>IOptions</code> instance with a list of <code>Tenant</code> objects.</p>
<p>And we need to slightly modify the <code>TenantProvider</code> class to return a <strong>connection string</strong> for the current tenant:</p>
<pre><code class="language-csharp">public sealed class TenantProvider
{
    private const string TenantIdHeaderName = &quot;X-TenantId&quot;;

    private readonly IHttpContextAccessor _httpContextAccessor;
    private readonly TenantSettings _tenantSettings;

    public TenantProvider(
        IHttpContextAccessor httpContextAccessor,
        IOptions&lt;TenantSettings&gt; tenantsOptions)
    {
        _httpContextAccessor = httpContextAccessor;
        _tenants = tenantsOptions.Value;
    }

    public string TenantId =&gt; _httpContextAccessor
        .HttpContext
        .Request
        .Headers[TenantIdHeaderName];

    public string GetConnectionString()
    {
        return _tenantSettings.Tenants.Single(t =&gt; t.Id == TenantId);
    }
}
</code></pre>
<p>And the last part is registering your <code>DbContext</code> to <strong>dynamically resolve</strong> the <strong>connection string</strong> for the current <strong>tenant</strong>.</p>
<pre><code class="language-csharp">builder.Services.AddDbContext&lt;OrdersDbContext&gt;((sp, o) =&gt;
{
    var tenantProvider = sp.GetRequiredService&lt;TenantProvider&gt;();

    var connectionString = tenantProvider.GetConnectionString();

    o.UseSqlServer(connectionString);
});
</code></pre>
<p>On every request, we create a new <code>OrdersDbContext</code> and connect to the appropriate database for that tenant.</p>
<p>You should definitely consider storing the tenant <strong>connection strings</strong> in a <strong>secure</strong> place like <strong>Azure Key Vault</strong>.</p>
<h2>Closing Thoughts</h2>
<p>I hope you now have a better understanding of how to build a <strong>multi-tenant system</strong> with <strong>EF Core</strong>.</p>
<p>I showed you the bare bones implementation, which you can improve to make it more robust and secure.</p>
<p>Building <strong>multi-tenant systems</strong> isn't easy, but when you understand the basic principles it shouldn't be too difficult either.</p>
<p>That's all for this week.</p>
<p>See you next Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_038.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Visualize Your Software Architecture With The C4 Model]]></title>
            <link>https://milanjovanovic.tech/blog/visualize-your-software-architecture-with-the-c4-model</link>
            <guid isPermaLink="false">visualize-your-software-architecture-with-the-c4-model</guid>
            <pubDate>Sat, 13 May 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Software architecture diagrams are a great way to communicate how you are planning to build a software system or how an existing software system works.
However, the majority of software architecture diagrams I've seen are a total mess.
If only there was a standard way to visualize your software architecture...
Enter the C4 model, which stands for context, containers, components, and code.
The C4 model is a lightweight approach to describing your software architecture.
And it's essentially just two things, a predefined set of common abstractions and four diagram types: - Context diagram - high-level view of the system - Container diagram - shows the objects running inside a system - Component diagram - shows the building blocks that make each container - Code diagram - rarely used, typically UML or ER diagram
Let's see how we can use the C4 model to visualize our software architecture.]]></description>
            <content:encoded><![CDATA[<p><strong>Software architecture</strong> diagrams are a great way to communicate how you are planning to build a software system or how an existing software system works.</p>
<p>However, the majority of software architecture <strong>diagrams</strong> I've seen are a total mess.</p>
<p>If only there was a standard way to visualize your software architecture...</p>
<p>Enter the <strong>C4 model</strong>, which stands for <strong>context</strong>, <strong>containers</strong>, <strong>components</strong>, and <strong>code</strong>.</p>
<p>The <strong>C4 model</strong> is a lightweight approach to describing your <strong>software architecture</strong>.</p>
<p>And it's essentially just two things, a predefined set of common <strong>abstractions</strong> and four diagram types:</p>
<ul>
<li><strong>Context diagram</strong> - high-level view of the system</li>
<li><strong>Container diagram</strong> - shows the objects running inside a system</li>
<li><strong>Component diagram</strong> - shows the building blocks that make each container</li>
<li><strong>Code diagram</strong> - rarely used, typically UML or ER diagram</li>
</ul>
<p>Let's see how we can use the <strong>C4 model</strong> to visualize our <strong>software architecture</strong>.</p>
<h2>First You Need Abstractions</h2>
<p>Before we dive into the <strong>C4 diagrams</strong>, we first need to understand the ubiquitous language of the <strong>C4 model</strong>.</p>
<blockquote>
<p>A software system is made up of one or more containers (applications and data stores),
each of which contains one or more components, which in turn are implemented by one or more code elements (classes, interfaces, objects, functions, etc).
And people may use the software systems that we build.</p>
</blockquote>
<p>The <strong>C4 model</strong> defines a set of <strong>abstractions</strong> that you can use to describe your software systems:</p>
<ul>
<li>Person - the end user using your system</li>
<li>Software system - highest level of abstraction that delivers value to end users</li>
<li>Container - applications and data stores that make up a system</li>
<li>Component - building blocks/modules that make up the container</li>
</ul>
<p>With these high-level concepts in place, let's take a look at how to use them in our <strong>C4 diagrams</strong>.</p>
<h2>System Context Diagram</h2>
<p>The <strong>System Context diagram</strong> is a high-level view of your software system.</p>
<p>It shows your software system as the central part, and any external systems and users that your system interacts with.</p>
<p>It should be <strong>technology agnostic</strong>, and the focus on the people and software systems instead of low-level details.</p>
<p><img src="/blogs/mnw_037/system_context_diagram.jpg" alt=""></p>
<p>The intended audience for the <strong>System Context Diagram</strong> is <em>everybody</em>.</p>
<p>If you can show it to non-technical people and they are able to understand it, then you know you're on the right track.</p>
<h2>Container Diagram</h2>
<p>When you zoom into one software system, you get to the <strong>Container diagram</strong>.</p>
<p>Your software system is comprised of multiple running parts - <strong>containers</strong>.</p>
<p>A <strong>container</strong> can be a:</p>
<ul>
<li>Web application</li>
<li>Single-page application</li>
<li>Database</li>
<li>File system</li>
<li>Object store</li>
<li>Message broker</li>
</ul>
<p>You can look at a <strong>container</strong> as a <strong>deployment unit</strong> that executes code or stores data.</p>
<p>The <strong>Container diagram</strong> shows the high-level view of the software architecture and the <strong>major technology choices</strong>.</p>
<p><img src="/blogs/mnw_037/container_diagram.jpg" alt=""></p>
<p>The <strong>Container diagram</strong> is intended for technical people inside and outside of the software development team:</p>
<ul>
<li>Software architects</li>
<li>Developers</li>
<li>Operations/support staff</li>
</ul>
<h2>Component Diagram</h2>
<p>Next you can zoom into an individual <strong>container</strong> to decompose it into its building blocks.</p>
<p>The <strong>Component diagram</strong> show the individual <strong>components</strong> that make up a <strong>container</strong>:</p>
<ul>
<li>What each of the components are</li>
<li>The technology and implementation details</li>
</ul>
<p><img src="/blogs/mnw_037/component_diagram.jpg" alt=""></p>
<p>The <strong>Component diagram</strong> is intended for software architects and developers.</p>
<h2>Code Diagram</h2>
<p>Finally, you can zoom into each <strong>component</strong> to show how it is implemented with <strong>code</strong>, typically using a UML class diagram or an ER diagram.</p>
<p>This level is rarely used as it goes into too much technical detail for most use cases.</p>
<p>However, there are supplementary diagrams that can be useful to fill in missing information by showcasing:</p>
<ul>
<li>Sequence of events</li>
<li>Deployment information</li>
<li>How systems interact at a higher level</li>
</ul>
<p>It's only recommended for the most <strong>important or complex components</strong>.</p>
<p>Of course, the target audience are software architects and developers.</p>
<h2>Takeaway</h2>
<p>The <strong>C4 model</strong> is a tool for visual and verbal communication, enabling your team to talk in the same language and have a common understanding of the design.</p>
<p>It's valuable for many reasons, to name a few:</p>
<ul>
<li>Product and business people benefit from understanding user behavior at the Context level</li>
<li>Context and Container diagrams provide a simple view of how software systems work</li>
<li>New developers can quickly understand system flows</li>
</ul>
<p>Now that you understand what the <strong>C4 model</strong> is, there's one question left to answer.</p>
<p>Should you use the <strong>C4 model</strong> to express your <strong>software architecture</strong>?</p>
<p>I can't give you a definitive answer.</p>
<p>I used it with a lot of success.</p>
<p>But I think you should at least give it a try.</p>
<p>You can learn more about the <strong>C4 model</strong> <a href="https://c4model.com/">here.</a></p>
<p>Thank you for reading, and have an awesome Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_037.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Enforcing Software Architecture With Architecture Tests]]></title>
            <link>https://milanjovanovic.tech/blog/enforcing-software-architecture-with-architecture-tests</link>
            <guid isPermaLink="false">enforcing-software-architecture-with-architecture-tests</guid>
            <pubDate>Sat, 06 May 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Software architecture is a blueprint for how you should structure your system. You can follow this blueprint strictly, or you can give yourself varying levels of freedom.
But when deadlines are tight, and you start cutting corners, that beautiful software architecture you built crumbles like a house of cards.
How can you enforce your software architecture?
By writing architecture tests.
Architecture tests are automated tests that verify the structure and design of your code.
You can use them to enforce your software architecture and the direction of dependencies of your projects.
In this week's issue I'll explain how to: - Write architecture tests - Enforce architecture - Enforce design rules
Let's dive in.]]></description>
            <content:encoded><![CDATA[<p><strong>Software architecture</strong> is a blueprint for how you should structure your system.
You can follow this blueprint strictly, or you can give yourself varying levels of freedom.</p>
<p>But when deadlines are tight, and you start cutting corners, that beautiful software architecture you built crumbles like a house of cards.</p>
<p>How can you <strong>enforce</strong> your <strong>software architecture</strong>?</p>
<p>By writing <strong>architecture tests</strong>.</p>
<p><strong>Architecture tests</strong> are automated tests that verify the structure and design of your code.</p>
<p>You can use them to enforce your software architecture and the direction of dependencies of your projects.</p>
<p>In this week's issue I'll explain how to:</p>
<ul>
<li>Write architecture tests</li>
<li>Enforce architecture</li>
<li>Enforce design rules</li>
</ul>
<p>Let's dive in!</p>
<h2>Writing Architecture Tests</h2>
<p>You write <strong>architecture tests</strong> the same as any unit test in your application.
There's an excellent library for writing architecture tests that already implements the boilerplate code we need to start writing tests.</p>
<p>We're going to use the <code>NetArchTest.Rules</code> library for writing architecture tests.</p>
<p>First, you have to install the NuGet package:</p>
<pre><code class="language-powershell">Install-Package NetArchTest.Rules
</code></pre>
<p>And now you can use it to write rules in your test project.</p>
<p>The starting point for writing architecture tests is the static <code>Types</code> class, which you can use to load a set of types.</p>
<p>Once you have loaded your types you can further filter them to find a more specific set of types.</p>
<p>Some of the available filtering methods:</p>
<ul>
<li><code>ResideInNamespace</code></li>
<li><code>AreClasses</code></li>
<li><code>AreInterfaces</code></li>
<li><code>HaveNameStartingWith</code></li>
<li><code>HaveNameEndingWith</code></li>
</ul>
<p>Finally, when you are satisfied with your selection, you can write the rule you want to enforce by calling <code>Should</code> or <code>ShouldNot</code>
and applying the condition you want to check.</p>
<p>Here's an example checking that all classes in the domain assembly are sealed:</p>
<pre><code class="language-csharp">var result = Types
  .InAssembly(DomainAssembly)
  .That()
  .AreClasses()
  .Should()
  .BeSealed()
  .GetResult();

Assert.True(result.IsSuccessful);
</code></pre>
<h2>Enforcing Architecture Rules</h2>
<p><strong>Architecture tests</strong> are particularly useful to <strong>enforce software architecture rules</strong> in a layered architecture or <strong>Modular Monolith</strong>.</p>
<p>Let's take the example of the Clean architecture:</p>
<ul>
<li>Domain should not have any dependencies</li>
<li>Application should not depend on Infrastructure</li>
<li>Infrastructure should depend on Application and Domain</li>
</ul>
<p>Here's how you can write tests for enforcing architecture rules.</p>
<p><strong>Domain should not have any dependencies</strong></p>
<pre><code class="language-csharp">var result = Types
  .InAssembly(DomainAssembly)
  .ShouldNot()
  .HaveDependencyOnAny(&quot;Application&quot;, &quot;Infrastructure&quot;)
  .GetResult();

Assert.True(result.IsSuccessful);
</code></pre>
<p><strong>Application should not depend on Infrastructure</strong></p>
<pre><code class="language-csharp">var result = Types
  .InAssembly(AplicationAssembly)
  .Should()
  .NotHaveDependencyOn(&quot;Infrastructure&quot;)
  .GetResult();

Assert.True(result.IsSuccessful);
</code></pre>
<p><strong>Infrastructure should depend on Application and Domain</strong></p>
<p>How the <code>NetArchTest.Rules</code> library works is by scanning the imported namespaces of your types.</p>
<p>Because of this, writing negative conditions like in the previous two examples is straightforward.</p>
<p>But writing positive conditions has to be scoped to a more specific set of types.</p>
<p>For example, we can validate this dependency by checking that all repositories must have a dependency on the <code>Domain</code> namespace.</p>
<pre><code class="language-csharp">var result = Types
  .InAssembly(InfrastructureAssembly)
  .HaveNameEndingWith(&quot;Repository&quot;)
  .Should()
  .HaveDependencyOn(&quot;Domain&quot;)
  .GetResult();

Assert.True(result.IsSuccessful);
</code></pre>
<h2>Enforcing Design Rules</h2>
<p>Another valuable <strong>use case</strong> for <strong>architecture tests</strong> is <strong>enforcing design rules</strong> in your application.</p>
<p>Design rules are more specific than project references, and focus on the implementation details of your classes.</p>
<p>Here are some <strong>design rules</strong> that you can enforce:</p>
<ul>
<li>Services must be internal</li>
<li>Entities and Value objects must be sealed</li>
<li>Controllers can't depend on repositories directly</li>
<li>Command (or query) handlers must follow a naming convention</li>
</ul>
<p>The possibilities are endless, and it's up to you how many design rules you want to enforce.</p>
<p>Here's how you can write tests for enforcing design rules.</p>
<p><strong>Command handlers must end with <code>CommandHandler</code></strong></p>
<pre><code class="language-csharp">var result = Types
    .InAssembly(ApplicationAssembly)
    .That()
    .ImplementInterface(typeof(ICommandHandler))
    .Should()
    .HaveNameEndingWith(&quot;CommandHandler&quot;)
    .GetResult();

Assert.True(result.IsSuccessful);
</code></pre>
<p><strong>Controllers can't directly reference repositories</strong></p>
<pre><code class="language-csharp">var result = Types
    .InAssembly(ApiAssembly)
    .That()
    .HaveNameEndingWith(&quot;Controller&quot;)
    .ShouldNot()
    .HaveDependencyOn(&quot;Infrastructure.Repositories&quot;)
    .GetResult();

Assert.True(result.IsSuccessful);
</code></pre>
<h2>Takeaway</h2>
<p><strong>Architecture tests</strong> are an easy way to <strong>enforce software architecture</strong> and design rules with automated tests.</p>
<p>One of the best investments you can make as a software engineer is writing automated tests.
You write the tests once, and use them to verify your system forever.
Granted, you also have to maintain the tests over time as your system grows.</p>
<p>Manually enforcing software architecture with pair programming and constant PR reviews is:</p>
<ul>
<li>Error prone</li>
<li>Time consuming</li>
<li>Not cost effective</li>
</ul>
<p><strong>Architecture tests</strong> really shine here, since you can write them quickly and reduce the cost of enforcing your software architecture rules to zero.</p>
<p>Thanks for reading.</p>
<p>Hope that was helpful!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_036.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Health Checks In ASP.NET Core For Monitoring Your Applications]]></title>
            <link>https://milanjovanovic.tech/blog/health-checks-in-asp-net-core</link>
            <guid isPermaLink="false">health-checks-in-asp-net-core</guid>
            <pubDate>Sat, 29 Apr 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[We all want to build robust and reliable applications that can scale indefinitely and handle any number of requests.
But with distributed systems and microservices architectures growing in complexity, it's becoming increasingly harder to monitor the health of our applications.
It's vital that you have a system in place to receive quick feedback of your application health.
That's where health checks come in.
Health checks provide a way to monitor and verify the health of various components of an application including: - Databases - APIs - Caches - External services
Here's what I'll show you in this week's newsletter: - What are health checks - Adding a custom health check - Using existing health check libraries - Customizing the health checks response format
Let's see how to implement health checks in ASP.NET Core.]]></description>
            <content:encoded><![CDATA[<p>We all want to build <strong>robust</strong> and <strong>reliable</strong> applications that can scale indefinitely and handle any number of requests.</p>
<p>But with <strong>distributed systems</strong> and <strong>microservices architectures</strong> growing in complexity, it's becoming increasingly harder to <strong>monitor</strong> the <strong>health</strong> of our applications.</p>
<p>It's vital that you have a system in place to receive quick feedback of your application <strong>health.</strong></p>
<p>That's where <strong>health checks</strong> come in.</p>
<p><strong>Health checks</strong> provide a way to monitor and verify the health of various components of an application including:</p>
<ul>
<li>Databases</li>
<li>APIs</li>
<li>Caches</li>
<li>External services</li>
</ul>
<p>Here's what I'll show you in this week's newsletter:</p>
<ul>
<li><a href="#what-are-health-checks">What are health checks</a></li>
<li><a href="#adding-custom-health-checks">Adding a custom health check</a></li>
<li><a href="#using-existing-health-check-libraries">Using existing health check libraries</a></li>
<li><a href="#formatting-health-checks-response">Customizing the health checks response format</a></li>
</ul>
<p>Let's see how to implement <strong>health checks</strong> in <strong><a href="http://ASP.NET">ASP.NET</a> Core</strong>.</p>
<h2>What Are Health Checks?</h2>
<p><strong>Health checks</strong> are a proactive mechanism for monitoring and verifying the <strong>health</strong> and <strong>availability</strong> of an application in <strong><a href="http://ASP.NET">ASP.NET</a> Core.</strong></p>
<p><a href="http://ASP.NET">ASP.NET</a> Core has <strong>built-in support</strong> for implementing <strong>health checks.</strong></p>
<p>Here's the basic configuration, which registers the health check services and adds the <code>HealthCheckMiddleware</code> to respond at the specified URL.</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);

builder.Services.AddHealthChecks();

var app = builder.Build();

app.MapHealthChecks(&quot;/health&quot;);

app.Run();
</code></pre>
<p>The health check returns a <code>HealthStatus</code> value indicating the health of the service.</p>
<p>There are three distinct <code>HealthStatus</code> values:</p>
<ul>
<li><code>HealthStatus.Healthy</code></li>
<li><code>HealthStatus.Degraded</code></li>
<li><code>HealthStatus.Unhealthy</code></li>
</ul>
<p>You can use the <code>HealthStatus</code> to indicate the different states of your application.</p>
<p>For example, if the application is functioning slower than expected you can return <code>HealthStatus.Degraded</code>.</p>
<h2>Adding Custom Health Checks</h2>
<p>You can create <strong>custom health checks</strong> by implementing the <code>IHealthCheck</code> interface.</p>
<p>For example, you can implement a check to see if your <strong>SQL</strong> database is available.</p>
<p>It's important to use a query that can complete quickly in the database, like <code>SELECT 1</code>.</p>
<p>Here's a <strong>custom health check</strong> implementation example in the <code>SqlHealthCheck</code> class:</p>
<pre><code class="language-csharp">public class SqlHealthCheck : IHealthCheck
{
    private readonly string _connectionString;

    public SqlHealthCheck(IConfiguration configuration)
    {
        _connectionString = configuration.GetConnectionString(&quot;Database&quot;);
    }

    public async Task&lt;HealthCheckResult&gt; CheckHealthAsync(
        HealthCheckContext context,
        CancellationToken cancellationToken = default)
    {
        try
        {
            using var sqlConnection = new SqlConnection(_connectionString);

            await sqlConnection.OpenAsync(cancellationToken);

            using var command = sqlConnection.CreateCommand();
            command.CommandText = &quot;SELECT 1&quot;;

            await command.ExecuteScalarAsync(cancellationToken);

            return HealthCheckResult.Healthy();
        }
        catch(Exception ex)
        {
            return HealthCheckResult.Unhealthy(
                context.Registration.FailureStatus,
                exception: ex);
        }
    }
}
</code></pre>
<p>After you implement the <strong>custom health check</strong>, you need to register it.</p>
<p>The previous call to <code>AddHealthChecks</code> now becomes:</p>
<pre><code class="language-csharp">builder.Services.AddHealthChecks()
    .AddCheck&lt;SqlHealthCheck&gt;(&quot;custom-sql&quot;, HealthStatus.Unhealthy);
</code></pre>
<p>We're giving it a custom name and setting which status to use as the failure result in <code>HealthCheckContext.Registration.FailureStatus</code>.</p>
<p>But stop and think for a moment.</p>
<p>Do you want to implement a <strong>custom health check</strong> on your own for <strong>every external service</strong> that you have?</p>
<p>Of course not! There's a better solution.</p>
<h2>Using Existing Health Check Libraries</h2>
<p>Before you start implementing a custom <strong>health check</strong> for everything, you should first see if there's already an <strong>existing library.</strong></p>
<p>In the <a href="https://github.com/Xabaril/AspNetCore.Diagnostics.HealthChecks"><code>AspNetCore.Diagnostics.HealthChecks</code></a> repository you can find
a wide collection <strong>health check</strong> packages for frequently used services and libraries.</p>
<p>Here are just a few examples:</p>
<ul>
<li>SQL Server - <code>AspNetCore.HealthChecks.SqlServer</code></li>
<li>Postgres - <code>AspNetCore.HealthChecks.Npgsql</code></li>
<li>Redis - <code>AspNetCore.HealthChecks.Redis</code></li>
<li>RabbitMQ - <code>AspNetCore.HealthChecks.RabbitMQ</code></li>
<li>AWS S3 - <code>AspNetCore.HealthChecks.Aws.S3</code></li>
<li>SignalR - <code>AspNetCore.HealthChecks.SignalR</code></li>
</ul>
<p>Here's how to add health checks for <strong>PostgreSQL</strong> and <strong>RabbitMQ</strong>:</p>
<pre><code class="language-csharp">builder.Services.AddHealthChecks()
    .AddCheck&lt;SqlHealthCheck&gt;(&quot;custom-sql&quot;, HealthStatus.Unhealthy);
    .AddNpgSql(pgConnectionString)
    .AddRabbitMQ(rabbitConnectionString)
</code></pre>
<h2>Formatting Health Checks Response</h2>
<p>By default, the endpoint returning you <strong>health check</strong> status will return a string value representing a <code>HealthStatus</code>.</p>
<p>This isn't practical if you have <strong>multiple health checks</strong> configured, as you'd want to view the health status individually per service.</p>
<p>To make matters worse, if one of the services is failing the entire response will return <code>Unhealthy</code> and you don't know what's causing the issue.</p>
<p>You can solve this by providing a <code>ResponsWriter</code>, and there's an existing one in the <code>AspNetCore.HealthChecks.UI.Client</code> library.</p>
<p>Let's install the <strong>NuGet</strong> package:</p>
<pre><code class="language-powershell">Install-Package AspNetCore.HealthChecks.UI.Client
</code></pre>
<p>And you need to slightly update the call to <code>MapHealthChecks</code> to use the <code>ResponseWriter</code> coming from this library:</p>
<pre><code class="language-csharp">app.MapHealthChecks(
    &quot;/health&quot;,
    new HealthCheckOptions
    {
        ResponseWriter = UIResponseWriter.WriteHealthCheckUIResponse
    });
</code></pre>
<p>After making these changes, here's what the response from the health check endpoint looks like:</p>
<pre><code class="language-json">{
  &quot;status&quot;: &quot;Unhealthy&quot;,
  &quot;totalDuration&quot;: &quot;00:00:00.3285211&quot;,
  &quot;entries&quot;: {
    &quot;npgsql&quot;: {
      &quot;data&quot;: {},
      &quot;duration&quot;: &quot;00:00:00.1183517&quot;,
      &quot;status&quot;: &quot;Healthy&quot;,
      &quot;tags&quot;: []
    },
    &quot;rabbitmq&quot;: {
      &quot;data&quot;: {},
      &quot;duration&quot;: &quot;00:00:00.1189561&quot;,
      &quot;status&quot;: &quot;Healthy&quot;,
      &quot;tags&quot;: []
    },
    &quot;custom-sql&quot;: {
      &quot;data&quot;: {},
      &quot;description&quot;: &quot;Unable to connect to the database.&quot;,
      &quot;duration&quot;: &quot;00:00:00.2431813&quot;,
      &quot;exception&quot;: &quot;Unable to connect to the database.&quot;,
      &quot;status&quot;: &quot;Unhealthy&quot;,
      &quot;tags&quot;: []
    }
  }
}
</code></pre>
<h2>Takeaway</h2>
<p>Application monitoring is important to track availability, resource usage, and changes to performance in your application.</p>
<p>I've used <strong>health checks</strong> before to implement <strong>failover scenarios</strong> in a <strong>cloud deployment</strong>.
When one application instance stops responding with a healthy result, a new one is created to continue serving requests.</p>
<p>It's easy to monitor the health of your <a href="http://ASP.NET">ASP.NET</a> Core applications by <strong>exposing health checks</strong> for your services.</p>
<p>You can decide to implement <strong>custom health checks</strong>, but first consider if there are <strong>existing solutions</strong>.</p>
<p>Thank you for reading, and have an awesome Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_035.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Idempotent Consumer - Handling Duplicate Messages]]></title>
            <link>https://milanjovanovic.tech/blog/idempotent-consumer-handling-duplicate-messages</link>
            <guid isPermaLink="false">idempotent-consumer-handling-duplicate-messages</guid>
            <pubDate>Sat, 22 Apr 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[What happens when a message is retried in an event-driven system?
It happens more often than you think.
The worst case scenario is that the message is processed twice, and the side effects can also be applied more than once.
Do you want your bank account to be double charged?
I'll assume the answer is no, of course.
You can use the Idempotent Consumer pattern to solve this problem.
In this week's issue I'll show you: - How the Idempotent Consumer pattern works - How to implement an Idempotent Consumer - The tradeoffs you need to consider
Let's see why the Idempotent Consumer pattern is valuable.]]></description>
            <content:encoded><![CDATA[<p>What happens when a <strong>message is retried</strong> in an event-driven system?</p>
<p>It happens more often than you think.</p>
<p>The <strong>worst case scenario</strong> is that the <strong>message is processed twice</strong>, and the <strong>side effects</strong> can also be applied more than once.</p>
<p>Do you want your bank account to be double charged?</p>
<p>I'll assume the answer is no, of course.</p>
<p>You can use the <strong>Idempotent Consumer</strong> pattern to solve this problem.</p>
<p>In this week's issue I will show you:</p>
<ul>
<li>How the Idempotent Consumer pattern works</li>
<li>How to implement an Idempotent Consumer</li>
<li>The tradeoffs you need to consider</li>
</ul>
<p>Let's see why the <strong>Idempotent Consumer</strong> pattern is valuable.</p>
<h2>How The Idempotent Consumer Pattern Works</h2>
<p>What's the idea behind the <strong>Idempotent Consumer pattern</strong>?</p>
<blockquote>
<p>An idempotent operation is one that has no additional effect if it is called more than once with the same input parameters.</p>
</blockquote>
<p>We want to avoid handling the same message more than once.</p>
<p>This would require <strong>Exactly-once</strong> message delivery guarantees from our messaging system.
And this is a really hard problem to solve in distributed systems.</p>
<p>A looser delivery guarantee is <strong>At-least-once</strong>, where we are aware that retries can happen and we can receive the same message more than once.</p>
<p>The <strong>Idempotent Consumer</strong> pattern works well with <strong>At-least-once</strong> message delivery, and solves the <strong>problem of duplicate messages.</strong></p>
<p>Here's what the algorithm looks like from the moment we receive a message:</p>
<ol>
<li>Was the message already processed?</li>
<li>If yes, it's a duplicate and there's nothing to do</li>
<li>If not, we need to handle the message</li>
<li>We also need to store the message identifier</li>
</ol>
<p><img src="/blogs/mnw_034/idempotent_consumer_algorithm.png" alt=""></p>
<p>We need a <strong>unique identifier</strong> for every <strong>message</strong> we receive, and a table in the database to store processed messages.</p>
<p>However, it's interesting how we choose the implement the message handling and storing of the processed message identifier.</p>
<p>You can implement the idempotent consumer as a decorator around a regular message handler.</p>
<p>I'll show you two implementations:</p>
<ul>
<li>Lazy idempotent consumer</li>
<li>Eager idempotent consumer</li>
</ul>
<h2>Lazy Idempotent Consumer</h2>
<p>The <strong>lazy idempotent consumer</strong> matches the flow shown in the algorithm above.</p>
<p>Lazy refers to how we store the message identifer to mark it as processed.</p>
<p>In the happy path, we handle the message and store the message identifier.</p>
<p>If the message handler throws an exception, we never store the message identifier and the consumer can be executed again.</p>
<p>Here's what the implementation looks like:</p>
<pre><code class="language-csharp">public class IdempotentConsumer&lt;T&gt; : IHandleMessages&lt;T&gt;
    where T : IMessage
{
    private readonly IMessageRepository _messageRepository;
    private readonly IHandleMessages&lt;T&gt; _decorated;

    public IdempotentConsumer(
        IMessageRepository messageRepository,
        IHandleMessages&lt;T&gt; decorated)
    {
        _messageRepository = messageRepository;
        _decorated = decorated;
    }

    public async Task Handle(T message)
    {
        if (_messageRepository.IsProcessed(message.Id))
        {
            return;
        }

        await _decorated.Handle(message);

        _messageRepository.Store(message.Id);
    }
}
</code></pre>
<h2>Eager Idempotent Consumer</h2>
<p>The <strong>eager idempotent consumer</strong> is slightly different from the lazy implementation, but the end result is the same.</p>
<p>In this version, we eagerly store the message identifier in the database and then continue to handle the message.</p>
<p>If the handler throws an exception, we need to perform cleanup in the database and remove the eagerly stored message identifier.</p>
<p>Otherwise, we risk leaving the system in an inconsistent state since the message was never handled correctly.</p>
<p>Here's what the implementation looks like:</p>
<pre><code class="language-csharp">public class IdempotentConsumer&lt;T&gt; : IHandleMessages&lt;T&gt;
    where T : IMessage
{
    private readonly IMessageRepository _messageRepository;
    private readonly IHandleMessages&lt;T&gt; _decorated;

    public IdempotentConsumer(
        IMessageRepository messageRepository,
        IHandleMessages&lt;T&gt; decorated)
    {
        _messageRepository = messageRepository;
        _decorated = decorated;
    }

    public async Task Handle(T message)
    {
        try
        {
            if (_messageRepository.IsProcessed(message.Id))
            {
                return;
            }

            _messageRepository.Store(message.Id);

            await _decorated.Handle(message);
        }
        catch (Exception e)
        {
            _messageRepository.Remove(message.Id);

            throw;
        }
    }
}
</code></pre>
<h2>In Summary</h2>
<p>Idempotency is an interesting problem to solve in a software system.</p>
<p>Some operations are <strong>naturally idempotent</strong>, and we don't need the overhead of the <strong>Idempotent Consumer</strong> pattern.</p>
<p>However, for those operations that aren't naturally idempotent, the <strong>Idempotent Consumer</strong> is a great solution.</p>
<p>The high-level algorithm is simple, and you can take two approaches in the implementation:</p>
<ul>
<li>Lazy storing of message identifiers</li>
<li>Eager storing of message identifiers</li>
</ul>
<p>I prefer to use the <strong>lazy approach</strong>, and only <strong>store the message identifier</strong> in the database when the <strong>handler completes successfully.</strong></p>
<p>It's easier to reason about and there is one less call to the database.</p>
<p>Thanks for reading.</p>
<p>Hope that was helpful.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_034.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[EF Core Raw SQL Queries]]></title>
            <link>https://milanjovanovic.tech/blog/ef-core-raw-sql-queries</link>
            <guid isPermaLink="false">ef-core-raw-sql-queries</guid>
            <pubDate>Sat, 15 Apr 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[EF Core is getting many new and exciting features in the upcoming version.
EF7 introduced support for returning scalar types using SQL queries.
And now we're getting support for querying unmapped types with raw SQL queries in EF8.
This is exactly what Dapper offers out of the box, and it's good to see EF Core catching up.
In this week's newsleter, I'm going to cover how to use EF Core for: - Raw SQL queries - Composing SQL queries with LINQ - Executing data modification with SQL
Let's dive in.]]></description>
            <content:encoded><![CDATA[<p><strong>EF Core</strong> is getting many new and exciting features in the upcoming version.</p>
<p><strong>EF7</strong> introduced support for returning <strong>scalar types</strong> using <strong>SQL</strong> queries.</p>
<p>And now we're getting support for <strong>querying unmapped types</strong> with <strong>raw SQL queries</strong> in <strong>EF8.</strong></p>
<p>This is exactly what <strong>Dapper</strong> offers out of the box, and it's good to see <strong>EF Core</strong> catching up.</p>
<p>In this week's newsletter, I'm going to cover how to use <strong>EF Core</strong> for:</p>
<ul>
<li><a href="#ef-core-and-sql-queries">Raw SQL queries</a></li>
<li><a href="#composing-sql-queries-with-linq">Composing SQL queries with LINQ</a></li>
<li><a href="#sql-queries-for-data-modifications">Executing data modifications with SQL</a></li>
</ul>
<p>Let's dive in!</p>
<h2>EF Core And SQL Queries</h2>
<p><strong>EF7</strong> added support for <strong>raw SQL queries</strong> returning scalar types.
<strong>EF8</strong> is taking this a step further with raw SQL queries that can return any mappable type, without having to include it in the <strong>EF model</strong>.</p>
<p>You can query unmapped types with the <code>SqlQuery</code> and <code>SqlQueryRaw</code> methods.</p>
<p>The <code>SqlQuery</code> method uses string interpolation to parameterize the query, protecting against SQL injection attacks.</p>
<p>Here's an example query returning an <code>OrderSummary</code> list:</p>
<pre><code class="language-csharp">var startDate = new DateOnly(2023, 1, 1);

var ordersIn2023 = await dbContext
    .Database
    .SqlQuery&lt;OrderSummary&gt;(
        $&quot;SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn &gt;= {startDate}&quot;)
    .ToListAsync();
</code></pre>
<p>This will be the <strong>SQL</strong> sent to the database:</p>
<pre><code class="language-sql">SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn &gt;= @p0
</code></pre>
<p>The type used for the query result can have a parameterized constructor.
The property names don't need to match the column names in the database, but they do have to match the names of the values in the result set.</p>
<p>You can also execute raw SQL queries and return results with:</p>
<ul>
<li>Views</li>
<li>Functions</li>
<li>Stored procedures</li>
</ul>
<h2>Composing SQL Queries With LINQ</h2>
<p>An interesting thing about <code>SqlQuery</code> is that it returns <code>IQueryable</code>, which can be further composed with <strong>LINQ.</strong></p>
<p>You can add a <code>Where</code> statement after calling <code>SqlQuery</code>:</p>
<pre><code class="language-csharp">var startDate = new DateOnly(2023, 1, 1);

var ordersIn2023 = await dbContext
    .Database
    .SqlQuery&lt;OrderSummary&gt;(&quot;SELECT * FROM OrderSummaries AS o&quot;)
    .Where(o =&gt; o.CreatedOn &gt;= startDate)
    .ToListAsync();
</code></pre>
<p>However, the generated <strong>SQL</strong> isn't optimal:</p>
<pre><code class="language-sql">SELECT s.Id, s.CustomerId, s.TotalPrice, s.CreatedOn
FROM (
    SELECT * FROM OrderSummaries AS o
) AS s
WHERE s.CreatedOn &gt;= @p0
</code></pre>
<p>Another possibility is to combine an <code>OrderBy</code> statement with <code>Skip</code> and <code>Take</code>:</p>
<pre><code class="language-csharp">var startDate = new DateOnly(2023, 1, 1);

var ordersIn2023 = await dbContext
    .Database
    .SqlQuery&lt;OrderSummary&gt;(
        $&quot;SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn &gt;= {startDate}&quot;)
    .OrderBy(o =&gt; o.Id)
    .Skip(10)
    .Take(5)
    .ToListAsync();
</code></pre>
<p>This would be the generated <strong>SQL</strong> for the previous query:</p>
<pre><code class="language-sql">SELECT s.Id, s.CustomerId, s.TotalPrice, s.CreatedOn
FROM (
    SELECT * FROM OrderSummaries AS o WHERE o.CreatedOn &gt;= @p0
) AS s
ORDER BY s.Id
OFFSET @p1 ROWS FETCH NEXT @p2 ROWS ONLY
</code></pre>
<p>In case you're wondering, the performance is similar to <strong>LINQ</strong> queries using <code>Select</code> projections.</p>
<p>I ran some benchmarks, and didn't notice any significant performance improvement.</p>
<p>This feature will be very useful if you're more comfortable with writing <strong>SQL</strong> or you want to fetch data from views, functions, and stored procedures.</p>
<h2>SQL Queries For Data Modifications</h2>
<p>If you want to modify data in the database with <strong>SQL</strong>, you will typically write a query that doesn't return a result.</p>
<p>The SQL query can be an <code>UPDATE</code> or <code>DELETE</code> statement, or even a stored procedure call.</p>
<p>You can use the <code>ExecuteSql</code> method to execute this type of query with <strong>EF Core</strong>:</p>
<pre><code class="language-csharp">var startDate = new DateOnly(2023, 1, 1);

dbContext.Database.ExecuteSql(
    $&quot;UPDATE Orders SET Status = 5 WHERE CreatedOn &gt;= {startDate}&quot;);
</code></pre>
<p><code>ExecuteSql</code> also protects from SQL injection by parameterizing arguments, just like <code>SqlQuery</code>.</p>
<p>With <strong>EF7</strong> you can write the above query with <strong>LINQ</strong> and the <code>ExecuteUpdate</code> method.
There's also the <code>ExecuteDelete</code> method for deleting records.</p>
<h2>In Summary</h2>
<p><strong>EF7</strong> introduced support for raw SQL queries returning <strong>scalar</strong> values.</p>
<p><strong>EF8</strong> will add support for <strong>raw SQL queries</strong> returning <strong>unmapped types</strong> with <code>SqlQuery</code> and <code>SqlQueryRaw</code>.</p>
<p>I like the direction that <strong>EF</strong> is going, introducing more flexibility for querying the database.</p>
<p>The performance isn't as good as <strong>Dapper</strong>, unfortunately.
But it's close enough that network costs will play the bigger factor.</p>
<p>I will probably be using only <strong>EF</strong> moving forward since it covers more use cases.</p>
<p>Thank you for reading, and have an awesome Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_033.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How To Use Rate Limiting In ASP.NET Core]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-use-rate-limiting-in-aspnet-core</link>
            <guid isPermaLink="false">how-to-use-rate-limiting-in-aspnet-core</guid>
            <pubDate>Sat, 08 Apr 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Rate limiting is a technique to limit the number of requests to a server or an API.
A limit is introduced within a given time period to prevent server overload and protect against abuse.
In ASP.NET Core 7 we have a built-in rate limiter middleware, that's easy to integrate into your API.
We're going to cover four rate limiting algorithms: - Fixed window - Sliding window - Token bucket - Concurrency
Let's see how we can work with rate limiting.]]></description>
            <content:encoded><![CDATA[<p>Rate limiting is a technique to limit the number of requests to a server or an API.</p>
<p>A limit is introduced within a given time period to prevent server overload and protect against abuse.</p>
<p>In <a href="http://ASP.NET">ASP.NET</a> Core 7, we have a built-in rate limiter middleware that's easy to integrate into your API.</p>
<p>We're going to cover four rate limiter algorithms:</p>
<ul>
<li><a href="#fixed-window-limiter">Fixed window</a></li>
<li><a href="#sliding-window-limiter">Sliding window</a></li>
<li><a href="#token-bucket-limiter">Token bucket</a></li>
<li><a href="#concurrency-limiter">Concurrency</a></li>
</ul>
<p>Let's see how we can work with rate limiting.</p>
<h2>What Is Rate Limiting?</h2>
<p>Rate limiting is about restricting the number of requests to an API, usually within a specific time window or based on other criteria.</p>
<p>This is practical for a few reasons:</p>
<ul>
<li>Prevents overloading of servers or applications</li>
<li>Improves security and guards against DDoS attacks</li>
<li>Reduces costs by preventing unnecessary resource usage</li>
</ul>
<p>In a multi-tenant application, each unique user can have a limitation on the number of API requests.</p>
<h2>Configuring Rate Limiting</h2>
<p><a href="http://ASP.NET">ASP.NET</a> Core 7 introduced built-in rate limiting middleware in the <code>Microsoft.AspNetCore.RateLimiting</code> namespace.</p>
<p>To add rate limiting to your application, you first need to register the rate limiting services:</p>
<pre><code class="language-csharp">builder.Services.AddRateLimiter(options =&gt;
{
    options.RejectionStatusCode = StatusCodes.Status429TooManyRequests;

    // We'll talk about adding specific rate limiting policies later.
});
</code></pre>
<p>I suggest updating the <code>RejectionStatusCode</code> to <code>429 (Too Many Requests)</code> because it's more correct.
The default value is <code>503 (Service Unavailable)</code>.</p>
<p>And you also have to apply the <code>RateLimitingMiddleware</code>:</p>
<pre><code class="language-csharp">app.UseRateLimiter();
</code></pre>
<p>That's everything you'll need.</p>
<p>Let's see the rate limiting algorithms we can use.</p>
<h2>Fixed Window Limiter</h2>
<p>The <code>AddFixedWindowLimiter</code> method configures a fixed window limiter.</p>
<p>The <code>Window</code> value determines the time window.</p>
<p>When a time window expires, a new one starts, and the request limit is reset.</p>
<pre><code class="language-csharp">builder.Services.AddRateLimiter(rateLimiterOptions =&gt;
{
    rateLimiterOptions.AddFixedWindowLimiter(&quot;fixed&quot;, options =&gt;
    {
        options.PermitLimit = 10;
        options.Window = TimeSpan.FromSeconds(10);
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
    });
});
</code></pre>
<h2>Sliding Window Limiter</h2>
<p>The sliding window algorithm is similar to the fixed window, but it introduces segments in a window.</p>
<p>Here's how it works:</p>
<ul>
<li>Each time window is divided into multiple segments</li>
<li>The window slides one segment each segment interval</li>
<li>The segment interval is (window_time)/(segments_per_window)</li>
<li>When a segment expires, the requests taken in that segment are added to the current segment</li>
</ul>
<p>The <code>AddSlidingWindowLimiter</code> method configures a sliding window limiter.</p>
<pre><code class="language-csharp">builder.Services.AddRateLimiter(rateLimiterOptions =&gt;
{
    rateLimiterOptions.AddSlidingWindowLimiter(&quot;sliding&quot;, options =&gt;
    {
        options.PermitLimit = 10;
        options.Window = TimeSpan.FromSeconds(10);
        options.SegmentsPerWindow = 2;
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
    });
});
</code></pre>
<h2>Token Bucket Limiter</h2>
<p>The token bucket algorithm is similar to the sliding window, but instead of adding back the requests from the expired segment,
a fixed number of tokens are added after each replenishment period.</p>
<p>The total number of tokens can never exceed the token limit.</p>
<p>The <code>AddTokenBucketLimiter</code> method configures a token bucket limiter.</p>
<pre><code class="language-csharp">builder.Services.AddRateLimiter(rateLimiterOptions =&gt;
{
    rateLimiterOptions.AddTokenBucketLimiter(&quot;token&quot;, options =&gt;
    {
        options.TokenLimit = 100;
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
        options.ReplenishmentPeriod = TimeSpan.FromSeconds(10);
        options.TokensPerPeriod = 20;
        options.AutoReplenishment = true;
    });
});
</code></pre>
<p>When <code>AutoReplenishment</code> is <code>true</code>, an internal timer will execute every <code>ReplenishmentPeriod</code> and replenish the tokens.</p>
<h2>Concurrency Limiter</h2>
<p>The concurrency limiter is the most straightforward algorithm, and it just limits the number of concurrent requests.</p>
<p>The <code>AddConcurrencyLimiter</code> method configures a concurrency limiter.</p>
<pre><code class="language-csharp">builder.Services.AddRateLimiter(rateLimiterOptions =&gt;
{
    rateLimiterOptions.AddConcurrencyLimiter(&quot;concurrency&quot;, options =&gt;
    {
        options.PermitLimit = 10;
        options.QueueProcessingOrder = QueueProcessingOrder.OldestFirst;
        options.QueueLimit = 5;
    });
});
</code></pre>
<p>There's no time component involved in this case. The only parameter is the number of concurrent requests.</p>
<h2>Using Rate Limiting In Your API</h2>
<p>Now that we have configured our rate limiting policies, let's see how we can use them in our API.</p>
<p>There are slight differences between controllers and minimal API endpoints, so I'll cover them in separate examples.</p>
<p><strong>Controllers</strong></p>
<p>To add rate limiting on a controller we use the <code>EnableRateLimiting</code> and <code>DisableRateLimiting</code> attributes.</p>
<p><code>EnableRateLimiting</code> can be applied on the controller or on the individual endpoints.</p>
<pre><code class="language-csharp">[EnableRateLimiting(&quot;fixed&quot;)]
public class TransactionsController
{
    private readonly ISender _sender;

    public TransactionsController(ISender sender)
    {
        _sender = sender;
    }

    [EnableRateLimiting(&quot;sliding&quot;)]
    public async Task&lt;IActionResult&gt; GetTransactions()
    {
        return Ok(await _sender.Send(new GetTransactionsQuery()));
    }

    [DisableRateLimiting]
    public async Task&lt;IActionResult&gt; GetTransactionById(int id)
    {
        return Ok(await _sender.Send(new GetTransactionByIdQuery(id)));
    }
}
</code></pre>
<p>In the previous example:</p>
<ul>
<li>All endpoints in the <code>TransactionsController</code> will use a <strong>fixed window</strong> policy</li>
<li>The <code>GetTransactions</code> endpoint will use a <strong>sliding window</strong> policy</li>
<li>The <code>GetTransactionById</code> endpoint won't have any rate limiting applied</li>
</ul>
<p><strong>Minimal APIs</strong></p>
<p>In a Minimal API endpoint you can configure the rate limit policy by calling <code>RequireRateLimiting</code> and specifying the policy name.</p>
<p>We're using the <strong>token bucket</strong> policy in this example.</p>
<pre><code class="language-csharp">app.MapGet(&quot;/transactions&quot;, async (ISender sender) =&gt;
{
    return Results.Ok(await sender.Send(new GetTransactionsQuery()));
})
.RequireRateLimiting(&quot;token&quot;);
</code></pre>
<h2>Closing Thoughts</h2>
<p>It's great that we can quickly introduce <strong>rate limiting</strong> in <a href="http://ASP.NET">ASP.NET</a> Core.</p>
<p>You can choose from one of the existing rate limiter algorithms:</p>
<ul>
<li>Fixed window</li>
<li>Sliding window</li>
<li>Token bucket</li>
<li>Concurrency</li>
</ul>
<p>Here are some resources if you want to learn more about rate limiting:</p>
<ul>
<li><a href="https://learn.microsoft.com/en-us/azure/architecture/patterns/rate-limiting-pattern">Rate Limiting pattern</a></li>
<li><a href="https://devblogs.microsoft.com/dotnet/announcing-rate-limiting-for-dotnet/">Announcing Rate Limiting for .NET</a></li>
</ul>
<p>I'm excited to try out rate limiting in my projects.</p>
<p>That's all for today.</p>
<p>Have an awesome Saturday!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_032.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Implementing The Saga Pattern With Rebus And RabbitMQ]]></title>
            <link>https://milanjovanovic.tech/blog/implementing-the-saga-pattern-with-rebus-and-rabbitmq</link>
            <guid isPermaLink="false">implementing-the-saga-pattern-with-rebus-and-rabbitmq</guid>
            <pubDate>Sat, 01 Apr 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Designing long-lived processes in a distributed environment is an interesting engineering challenge.
And a well known pattern for solving this problem is a Saga.
A Saga is a sequence of local transactions, where each local transaction updates the Saga state and publishes a message triggering the next step in the Saga.
Sagas come in two forms:
Orchestrated Choreographed With an orchestrated Saga, there's a central component responsible for orchestrating the individual steps.
In a choreographed Saga, processes work independently but coordinate with each other using events.
In this week's issue, I'll show you how to create orchestrated Sagas using the Rebus library with RabbitMQ for message transport.]]></description>
            <content:encoded><![CDATA[<p>Designing long-lived processes in a distributed environment is an interesting engineering challenge.</p>
<p>And a well known pattern for solving this problem is a <a href="https://microservices.io/patterns/data/saga.html"><strong>Saga</strong></a>.</p>
<p>A <strong>Saga</strong> is a sequence of local transactions, where each local transaction updates the <strong>Saga</strong> state and publishes a message triggering the next step in the <strong>Saga</strong>.</p>
<p>Sagas come in two forms:</p>
<ul>
<li><strong>Orchestrated</strong></li>
<li><strong>Choreographed</strong></li>
</ul>
<p>With an orchestrated Saga, there's a central component responsible for orchestrating the individual steps.</p>
<p>In a choreographed Saga, processes work independently but coordinate with each other using events.</p>
<p>In this week's issue, I'll show you how to create an <strong>orchestrated Saga</strong> using the <strong>Rebus</strong> library with <strong>RabbitMQ</strong> for message transport.</p>
<p>Let's dive in.</p>
<h2>Rebus Configuration</h2>
<p><a href="https://github.com/rebus-org/Rebus">Rebus</a> is a free .NET “service bus”, and it's practical for implementing asynchronous messaging-based
communication between the components of an application.</p>
<p>Let's install the following libraries:</p>
<ul>
<li><code>Rebus.ServiceProvider</code> for managing the <strong>Rebus</strong> instance</li>
<li><code>Rebus.RabbitMq</code> for <strong>RabbitMQ</strong> message transport</li>
<li><code>Rebus.SqlServer</code> for <strong>SQL Server</strong> state persistence</li>
</ul>
<pre><code class="language-powershell">Install-Package Rebus.ServiceProvider -Version 8.4.0
Install-Package Rebus.RabbitMq -Version 8.0.0
Install-Package Rebus.SqlServer -Version 7.3.1
</code></pre>
<p>Inside of your <strong><a href="http://ASP.NET">ASP.NET</a> Core</strong> application you will need the following configuration.</p>
<pre><code class="language-csharp">services.AddRebus(
    rebus =&gt; rebus
        .Routing(r =&gt;
            r.TypeBased().MapAssemblyOf&lt;Program&gt;(&quot;newsletter-queue&quot;))
        .Transport(t =&gt;
            t.UseRabbitMq(
                configuration.GetConnectionString(&quot;RabbitMq&quot;),
                inputQueueName: &quot;newsletter-queue&quot;))
        .Sagas(s =&gt;
            s.StoreInSqlServer(
                configuration.GetConnectionString(&quot;SqlServer&quot;),
                dataTableName: &quot;Sagas&quot;,
                indexTableName: &quot;SagaIndexes&quot;))
        .Timeouts(t =&gt;
            t.StoreInSqlServer(
                builder.Configuration.GetConnectionString(&quot;SqlServer&quot;),
                tableName: &quot;Timeouts&quot;))
);

services.AutoRegisterHandlersFromAssemblyOf&lt;Program&gt;();
</code></pre>
<p>Unpacking the individual configuration steps:</p>
<ul>
<li><code>Routing</code> - Configures messages to be routed by their type</li>
<li><code>Transport</code> - Configures the message transport mechanism</li>
<li><code>Sagas</code> - Configures the Saga persistence store</li>
<li><code>Timeouts</code> - Configures the timeouts persistence store</li>
</ul>
<p>You also need to specify the queue name for sending and receiving messages.</p>
<p><code>AutoRegisterHandlersFromAssemblyOf</code> will scan the specified assembly and automatically register the respective message handlers.</p>
<h2>Creating The Saga With Rebus</h2>
<p>We're going to create a <strong>Saga</strong> for a newsletter onboarding process.</p>
<p>When a user subscribes to the newsletter we want to:</p>
<ul>
<li>Send a welcome email immediately</li>
<li>Send a follow-up email after 7 days</li>
</ul>
<p>The first step in creating the <strong>Saga</strong> is defining the data model by implementing <code>ISagaData</code>.
We'll keep it simple and store the <code>Email</code> for <strong>correlation</strong>, and add two flags for the distinct steps in our <strong>Saga</strong>.</p>
<pre><code class="language-csharp">public class NewsletterOnboardingSagaData : ISagaData
{
    public Guid Id { get; set; }
    public int Revision { get; set; }

    public string Email { get; set; }

    public bool WelcomeEmailSent { get; set; }

    public bool FollowUpEmailSent { get; set; }
}
</code></pre>
<p>Now we can define the <code>NewsletterOnboardingSaga</code> class by inheriting from the <code>Saga</code> class and implementing the <code>CorrelateMessages</code> method.</p>
<p>It's a best practice to use a unique value for correlation.
In our case this will be the <code>Email</code>.</p>
<p>You also configure how the <code>Saga</code> starts with <code>IAmInitiatedBy</code>, and the individual messages the <code>Saga</code> handles with <code>IHandleMessages</code>.</p>
<pre><code class="language-csharp">public class NewsletterOnboardingSaga :
    Saga&lt;NewsletterOnboardingSagaData&gt;,
    IAmInitiatedBy&lt;SubscribeToNewsletter&gt;,
    IHandleMessages&lt;WelcomeEmailSent&gt;,
    IHandleMessages&lt;FollowUpEmailSent&gt;
{
    private readonly IBus _bus;

    public NewsletterOnboardingSaga(IBus bus)
    {
        _bus = bus;
    }

    protected override void CorrelateMessages(
        ICorrelationConfig&lt;NewsletterOnboardingSagaData&gt; config)
    {
        config.Correlate&lt;SubscribeToNewsletter&gt;(m =&gt; m.Email, d =&gt; d.Email);

        config.Correlate&lt;WelcomeEmailSent&gt;(m =&gt; m.Email, d =&gt; d.Email);

        config.Correlate&lt;FollowUpEmailSent&gt;(m =&gt; m.Email, d =&gt; d.Email);
    }

    /* Handlers omitted for brevity */
}
</code></pre>
<h2>Message Types And Naming Conventions</h2>
<p>There are two types of messages you send in a <strong>Saga</strong>:</p>
<ul>
<li>Commands</li>
<li>Events</li>
</ul>
<p>Commands instruct the receiving component what to do.<br>
Think: <strong>verb, imperative.</strong></p>
<p>Events notify the Saga which process was just completed.<br>
Think: <strong>what happened, past tense.</strong></p>
<h2>Saga Orchestration With Messages</h2>
<p>The <code>NewsletterOnboardingSaga</code> starts by handling the <code>SubscribeToNewsletter</code> command, and publishes a <code>SendWelcomeEmail</code> command.</p>
<pre><code class="language-csharp">public async Task Handle(SubscribeToNewsletter message)
{
    if (!IsNew)
    {
        return;
    }

    await _bus.Send(new SendWelcomeEmail(message.Email));
}
</code></pre>
<p>The <code>SendWelcomeEmail</code> command is handled by a different component, which publishes a <code>WelcomeEmailSent</code> event when it completes.</p>
<p>In the <code>WelcomeEmailSent</code> handler we update the <code>Saga</code> state and publish a <strong>deferred message</strong> by calling <code>Defer</code>.
Rebus will persist the <code>SendFollowUpEmail</code> command, and publish it when the <strong>timeout expires</strong>.</p>
<pre><code class="language-csharp">public async Task Handle(WelcomeEmailSent message)
{
    Data.WelcomeEmailSent = true;

    await _bus.Defer(TimeSpan.FromDays(3), new SendFollowUpEmail(message.Email));
}
</code></pre>
<p>Finally, the <code>SendFollowUpEmail</code> command is handled and we publish the <code>FollowUpEmailSent</code> event.</p>
<p>We update the <code>Saga</code> state again, and also call <code>MarkAsComplete</code> to complete the <code>Saga</code>.</p>
<pre><code class="language-csharp">public Task Handle(FollowUpEmailSent message)
{
    Data.FollowUpEmailSent = true;

    MarkAsComplete();

    return Task.CompletedTask;
}
</code></pre>
<p>Completing the <code>Saga</code> will delete it from the database.</p>
<h2>Handling Commands With Rebus</h2>
<p>Here's how the <code>SendWelcomeEmail</code> command handler looks like.</p>
<pre><code class="language-csharp">public class SendWelcomeEmailHandler : IHandleMessages&lt;SendWelcomeEmail&gt;
{
    private readonly IEmailService _emailService;
    private readonly IBus _bus;

    public SendWelcomeEmailHandler(IEmailService emailService, IBus bus)
    {
        _emailService = emailService;
        _bus = bus;
    }

    public async Task Handle(SendWelcomeEmail message)
    {
        await _emailService.SendWelcomeEmailAsync(message.Email);

        await _bus.Reply(new WelcomeEmailSent(message.Email));
    }
}
</code></pre>
<p>The important thing to highlight here is that we're using the <code>Reply</code> method to send a message back.
This will reply back to the endpoint specified as the return address on the current message.</p>
<h2>In Summary</h2>
<p><strong>Sagas</strong> are practical for implementing a long-lived process in a distributed system.
Each business process is represented by a local transaction, and publishes a message to trigger the next step in the <strong>Saga</strong>.</p>
<p>Although <strong>Sagas</strong> are very powerful, they are also <em>complicated to develop, maintain and debug.</em></p>
<p>We didn't cover a few important topics in this newsletter:</p>
<ul>
<li>Error handling</li>
<li>Message retries</li>
<li>Compensating transactions</li>
</ul>
<p>I think you'll have some fun researching these on your own.</p>
<p>Take a look at the <a href="https://github.com/m-jovanovic/newsletter-orchestrated-saga"><strong>source code for the example used in this newsletter</strong></a>
if you want to learn more about implementing Sagas.</p>
<p>If you have <strong>Docker</strong> installed, you should be able to run it without a problem and try it out.</p>
<p>Thank you for reading, and have an awesome Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_031.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How To Publish MediatR Notifications In Parallel]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-publish-mediatr-notifications-in-parallel</link>
            <guid isPermaLink="false">how-to-publish-mediatr-notifications-in-parallel</guid>
            <pubDate>Sat, 25 Mar 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[MediatR is a popular library with a simple mediator pattern implementation in .NET.
Here's a definiton taken from MediatR's GitHub: "In-process messaging with no dependencies."
With the rise in popularity of the CQRS pattern, MediatR became the go-to library to implement commands and queries.
However, MediatR also has support for the publish-subscribe pattern using notifications. You can publish an INotification instance and have multiple subscribers handle the published message.
Until recently, the handlers subscribing to an INotification message could only execute serially, one by one.
In this week's newsletter, I'll show you how to configure MediatR to execute the handlers in parallel.]]></description>
            <content:encoded><![CDATA[<p><strong>MediatR</strong> is a popular library with a simple <strong>mediator pattern</strong> implementation in .NET.</p>
<p>Here's a definiton taken from MediatR's GitHub: <strong>&quot;In-process messaging with no dependencies.&quot;</strong></p>
<p>With the rise in popularity of the <strong>CQRS pattern</strong>, MediatR became the go-to library to implement commands and queries.</p>
<p>However, MediatR also has support for the <strong>publish-subscribe</strong> pattern using notifications.
You can publish an <code>INotification</code> instance and have multiple subscribers handle the published message.</p>
<p>Until recently, the handlers subscribing to an <code>INotification</code> message could only execute serially, one by one.</p>
<p>In this week's newsletter, I'll show you how to configure MediatR to <strong>execute the handlers in parallel</strong>.</p>
<p>Let's dive in.</p>
<h2>How Publish-Subscribe Works With MediatR</h2>
<p>Before I talk about notification publishing strategies, let's see how <strong>publish-subscribe</strong> works with <strong>MediatR</strong>.</p>
<p>You need a class implementing the <code>INotification</code> interface:</p>
<pre><code class="language-csharp">public record OrderCreated(Guid OrderId) : INotification;
</code></pre>
<p>Then you need a respective <code>INotificationHandler</code> implementation:</p>
<pre><code class="language-csharp">public class OrderCreatedHandler : INotificationHandler&lt;OrderCreated&gt;
{
    private readonly INotificationService _notificationService;

    public OrderCreatedHandler(INotificationService notificationService)
    {
        _notificationService = notificationService;
    }

    public async Task Handle(
        OrderCreated notification,
        CancellationToken cancellationToken)
    {
        await _notificationService.SendOrderCreatedEmail(
            notification.OrderId,
            cancellationToken);
    }
}
</code></pre>
<p>And then you simply publish a message using either <code>IMediator</code> or <code>IPublisher</code>.
I prefer using the <code>IPublisher</code> because it's more expressive:</p>
<pre><code class="language-csharp">await publisher.Publish(new OrderCreated(order.Id), cancellationToken);
</code></pre>
<p>MediatR will invoke all the respective handlers.</p>
<h2>Introducing Notification Publisher Strategies</h2>
<p>Before MediatR v12, the publishing strategy would invoke each handler individually.</p>
<p>However, there's a new interface <code>INotificationPublisher</code> controlling how the handlers are called.</p>
<p>The default implementation of this interface is <code>ForeachAwaitPublisher</code>:</p>
<pre><code class="language-csharp">public class ForeachAwaitPublisher : INotificationPublisher
{
    public async Task Publish(
        IEnumerable&lt;NotificationHandlerExecutor&gt; handlerExecutors,
        INotification notification,
        CancellationToken cancellationToken)
    {
        foreach (var handler in handlerExecutors)
        {
            await handler
                .HandlerCallback(notification, cancellationToken)
                .ConfigureAwait(false);
        }
    }
}
</code></pre>
<p>But now you can also use the <code>TaskWhenAllPublisher</code>:</p>
<pre><code class="language-csharp">public class TaskWhenAllPublisher : INotificationPublisher
{
    public Task Publish(
        IEnumerable&lt;NotificationHandlerExecutor&gt; handlerExecutors,
        INotification notification,
        CancellationToken cancellationToken)
    {
        var tasks = handlerExecutors
            .Select(handler =&gt; handler.HandlerCallback(
                notification,
                cancellationToken))
            .ToArray();

        return Task.WhenAll(tasks);
    }
}
</code></pre>
<p>Here's a comparison between these two strategies.</p>
<p><code>ForeachAwaitPublisher</code>:</p>
<ul>
<li>Invokes each handler one by one</li>
<li>Fails when an exception occurs in one of the handlers</li>
</ul>
<p><code>TaskWhenAllPublisher</code>:</p>
<ul>
<li>Invokes all the handlers at the same time</li>
<li>Executes all the handlers regardless of one of them throwing an exception</li>
</ul>
<p>If you store the task returned by <code>TaskWhenAllPublisher</code> you can access the <code>Task.Exception</code> property, which will contain an <code>AggregateException</code> instance.
You can then implement more robust exception handling.</p>
<h2>Configuring MediatR Notification Publishing Strategy</h2>
<p>How do we configure which <code>INotificationPublisher</code> strategy MediatR will use?</p>
<p>There's a new way to apply configuration options when calling the <code>AddMediatR</code> method.</p>
<p>You supply an <code>Action&lt;MediatRServiceConfiguration&gt;</code> delegate and configure the <code>MediatRServiceConfiguration</code> instance.</p>
<p>If you want to use the <code>TaskWhenAllPublisher</code> strategy, you can either:</p>
<ul>
<li>Provide a value for the <code>NotificationPublisher</code> property</li>
<li>Specify the strategy type on the <code>NotificationPublisherType</code> property</li>
</ul>
<pre><code class="language-csharp">services.AddMediatR(config =&gt; {
    config.RegisterServicesFromAssemblyContaining&lt;Program&gt;();

    // Setting the publisher directly will make the instance a Singleton.
    config.NotificationPublisher = new TaskWhenAllPublisher();

    // Seting the publisher type will:
    // 1. Override the value set on NotificationPublisher
    // 2. Use the service lifetime from the ServiceLifetime property below
    config.NotificationPublisherType = typeof(TaskWhenAllPublisher);

    config.ServiceLifetime = ServiceLifetime.Transient;
});
</code></pre>
<p>You can also implement a custom <code>INotificationPublisher</code> instance and use your own implementation instead.</p>
<h2>How Is This Useful?</h2>
<p>Being able to <strong>run notification handlers in parallel</strong> provides a significant <strong>performance improvement</strong> over the default behavior.</p>
<p>However, note that all handlers will use the same service scope.</p>
<p>If you have service instances that don't support concurrent access you may run into problems.</p>
<p>Unfortunately, one such service instance is the <strong>EF Core</strong> <code>DbContext</code>.</p>
<p>In any case, I think this is a great addition to the already amazing <strong>MediatR</strong> library.</p>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_030.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Creating Data-Driven Tests With xUnit]]></title>
            <link>https://milanjovanovic.tech/blog/creating-data-driven-tests-with-xunit</link>
            <guid isPermaLink="false">creating-data-driven-tests-with-xunit</guid>
            <pubDate>Sat, 18 Mar 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Data-driven testing is a testing method where test data is provided through some external source. Hence it's also known as parameterized testing.
A popular testing library in .NET that supports parameterized testing is xUnit. It uses attributes to define test methods. The Fact attribute defines a simple test, and the Theory attribute defines a parameterized test.
In this week's newsletter, I'm going to show you four ways to write parameterized tests with xUnit: - InlineData - MemberData - ClassData - TheoryData
And I'll discuss which approach I think is the best.]]></description>
            <content:encoded><![CDATA[<p><strong>Data-driven testing</strong> is a testing method where test data is provided through some external source.
Hence it's also known as <strong>parameterized testing</strong>.</p>
<p>A popular testing library in .NET that supports parameterized testing is <strong>xUnit</strong>.
It uses attributes to define test methods.
The <code>Fact</code> attribute defines a simple test, and the <code>Theory</code> attribute defines a parameterized test.</p>
<p>In this week's newsletter, I'm going to show you four ways to write <strong>parameterized tests</strong> with xUnit:</p>
<ul>
<li><code>InlineData</code></li>
<li><code>MemberData</code></li>
<li><code>ClassData</code></li>
<li><code>TheoryData</code></li>
</ul>
<p>And I'll discuss which approach I think is the best.</p>
<p>Let's dive in.</p>
<h2>Writing Parameterized Tests With InlineData</h2>
<p>The simplest way to write parameterized tests with xUnit is using the <code>InlineData</code> attribute.
You provide test data by passing in values to the <code>InlineData</code> constructor.</p>
<p>Here's how that would look like:</p>
<pre><code class="language-csharp">[Theory]
[InlineData(&quot;test@test.com&quot;, &quot;test.com&quot;)]
[InlineData(&quot;milan@milanjovanovic.tech&quot;, &quot;milanjovanovic.tech&quot;)]
public void EmailParser_Should_Return_Domain(string email, string expectedDomain)
{
    // Arrange
    var parser = new EmailParser();

    // Act
    var domain = parser.ParseDomain(email);

    // Assert
    Assert.Equal(domain, expectedDomain);
}
</code></pre>
<p>In this example, we provide two <code>string</code> values to the <code>InlineData</code> attribute, which represent the <code>email</code> and <code>expectedDomain</code> parameters in the test.
We can specify the <code>InlineData</code> attribute as many times as we want, to introduce more test cases.</p>
<p>The downside of this approach is that it becomes very verbose when we have many test cases.
And we are limited to only using constant data for the parameters.</p>
<h2>Writing Parameterized Tests With MemberData</h2>
<p>With the <code>MemberData</code> attribute we have the ability to programmatically provide the test data.
You can load the test data from a static property or member of a type.</p>
<p>Here's an example of using the <code>MemberData</code> attribute to load test data from a property:</p>
<pre><code class="language-csharp">[Theory]
[MemberData(nameof(EmailTestData))]
public void EmailParser_Should_Return_Domain(string email, string expectedDomain)
{
    // Arrange
    var parser = new EmailParser();

    // Act
    var domain = parser.ParseDomain(email);

    // Assert
    Assert.Equal(domain, expectedDomain);
}

public static IEnumerable&lt;object[]&gt; EmailTestData =&gt; new List&lt;object&gt;
{
    new object[] { &quot;test@test.com&quot;, &quot;test.com&quot; },
    new object[] { &quot;milan@milanjovanovic.tech&quot;, &quot;milanjovanovic.tech&quot; }
};
</code></pre>
<p>You specify the name of the member in the <code>MemberData</code> attribute, and it's a best practice to use the <code>nameof</code> operator
so that you can rename the property (or method) in the future without breaking your test.</p>
<p>The one constraint is that the property (or method) has to return <code>IEnumerable&lt;object[]&gt;</code>, so there is no strong typing.</p>
<h2>Writing Parameterized Tests With ClassData</h2>
<p>The <code>ClassData</code> attribute allows you to extract test data into its own class.
This is helpful for organizing your test data separately from your tests, and it allows for easier reuse.
You load the test from a class the inherits from <code>IEnumerable&lt;object[]&gt;</code> and implements the <code>GetEnumerator</code> method.</p>
<p>Here's an example of using the <code>ClassData</code> attribute to load test data from a class:</p>
<pre><code class="language-csharp">[Theory]
[ClassData(typeof(EmailTestData))]
public void EmailParser_Should_Return_Domain(string email, string expectedDomain)
{
    // Arrange
    var parser = new EmailParser();

    // Act
    var domain = parser.ParseDomain(email);

    // Assert
    Assert.Equal(domain, expectedDomain);
}

public class EmailTestData : IEnumerable&lt;object[]&gt;
{

    public IEnumerable&lt;object[]&gt; GetEnumerator()
    {
        yield return new object[] { &quot;test@test.com&quot;, &quot;test.com&quot; };
        yield return new object[] { &quot;milan@milanjovanovic.tech&quot;, &quot;milanjovanovic.tech&quot; };
    }

    IEnumerator IEnumerable.GetEnumerator() =&gt; GetEnumerator();
};
</code></pre>
<p>Unfortunately, this approach is complicated because you have to implement the <code>IEnumerable</code> interface.</p>
<p>It almost defeats the purpose of separating test data from the actual tests.</p>
<p>And we still suffer from lack of type-safety.</p>
<p>Is there a better solution?</p>
<h2>Writing Parameterized Tests With TheoryData</h2>
<p>Let me introduce you to <code>TheoryData</code>, which is my preferred way of providing test data for parameterized tests.
Using the <code>TheoryData</code> class you can implement a class to provide test data while having the benefit of type-safety.</p>
<p>Here's an example of using <code>TheoryData</code> in combination with <code>ClassData</code>:</p>
<pre><code class="language-csharp">[Theory]
[ClassData(typeof(EmailTestData))]
public void EmailParser_Should_Return_Domain(string email, string expectedDomain)
{
    // Arrange
    var parser = new EmailParser();

    // Act
    var domain = parser.ParseDomain(email);

    // Assert
    Assert.Equal(domain, expectedDomain);
}

public class EmailTestData : TheoryData&lt;string, string&gt;
{
    public EmailTestData()
    {
        Add(&quot;test@test.com&quot;, &quot;test.com&quot;);
        Add(&quot;milan@milanjovanovic.tech&quot;, &quot;milanjovanovic.tech&quot;);
    }
};
</code></pre>
<p>How does <code>TheoryData</code> work?</p>
<p>It's a generic class that allows us to specify the types for our parameterized test.</p>
<p>You just call the <code>Add</code> method in the <code>EmailTestData</code> constructor to provide test data for a single test case.
And introducing more test cases comes down to calling the <code>Add</code> method multiple times.</p>
<p>You can also use <code>TheoryData</code> in combination with <code>MemberData</code>, and return <code>TheoryData</code> from a property or method.</p>
<h2>Which Approach Should You Use?</h2>
<p>I showed you four approaches to write parameterized tests with xUnit:</p>
<ul>
<li><code>InlineData</code></li>
<li><code>MemberData</code></li>
<li><code>ClassData</code></li>
<li><code>TheoryData</code></li>
</ul>
<p>So which one should you use?</p>
<p>Here's my personal preference that you can follow if you want:</p>
<ul>
<li><code>InlineData</code> for simple test cases</li>
<li><code>TheoryData</code> using <code>ClassData</code> for complex test cases</li>
</ul>
<p>Thank you for reading, and have a wonderful Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_029.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Using Multiple EF Core DbContexts In a Single Application]]></title>
            <link>https://milanjovanovic.tech/blog/using-multiple-ef-core-dbcontext-in-single-application</link>
            <guid isPermaLink="false">using-multiple-ef-core-dbcontext-in-single-application</guid>
            <pubDate>Sat, 11 Mar 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Entity Framework Core (EF Core) is a popular ORM in .NET that allows you to work with SQL databases. EF Core uses a DbContext, which represents a session with the database and is responsible for tracking changes, performing database operations, and managing database connections.
It's common to have only one DbContext for the entire application.
But what if you need to have multiple DbContexts?
In this week's newsletter we're going to explore: - When you may want to use multiple DbContexts - How to create multiple DbContexts - What are the benefits of using multiple DbContexts]]></description>
            <content:encoded><![CDATA[<p><strong>Entity Framework Core (EF Core)</strong> is a popular ORM in .NET that allows you to work with SQL databases.
EF Core uses a <code>DbContext</code>, which represents a session with the database and is responsible for tracking changes,
performing database operations, and managing database connections.</p>
<p>It's common to have only one <code>DbContext</code> for the entire application.</p>
<p>But what if you need to have <strong>multiple DbContexts</strong>?</p>
<p>In this week's newsletter we're going to explore:</p>
<ul>
<li>When you may want to use multiple DbContexts</li>
<li>How to create multiple DbContexts</li>
<li>What are the benefits of using multiple DbContexts</li>
</ul>
<p>Let's dive in!</p>
<h2>Why Use Multiple DbContexts?</h2>
<p>There are a few cases where using <strong>multiple DbContexts can be useful.</strong></p>
<p><strong>Multiple Databases</strong><br>
Does your application need to work with multiple SQL databases?
Then you're forced to use multiple DbContexts, each one dedicated to a specific SQL database.</p>
<p><strong>Separating Concerns</strong><br>
If the application you're building has a complex domain model, you may see an improvement by separating concerns
between a few DbContexts, where each one is responsible for a specific area of the domain model.</p>
<p><strong>Modular Monolith</strong><br>
When you're building a <strong>Modular Monolith</strong>, using multiple DbContexts can be practical because you can configure
a different <strong>database schema</strong> per <code>DbContext</code>, giving you logical separation at the database level.</p>
<p><strong>Read Replicas</strong><br>
You can configure a separate <code>DbContext</code> instance to access the read replica of your database, and use that
<code>DbContext</code> for read-only queries.
You can also configure <code>QueryTrackingBehavior.NoTracking</code> on the <code>DbContext</code> level to improve query performance.</p>
<h2>Creating Multiple DbContexts In a Single Application</h2>
<p>Here's how you can easily configure multiple DbContexts.
Let's say we have a <code>CatalogDbContext</code> and an <code>OrderDbContext</code> in our application.
We want to configure them using the following constrainsts:</p>
<ul>
<li>Both DbContexts use the <strong>same database</strong></li>
<li>Each DbContext has a <strong>separate database schema</strong></li>
</ul>
<pre><code class="language-csharp">public class CatalogDbContext : DbContext
{
    public DbSet&lt;Product&gt; Products { get; set; }

    public DbSet&lt;Category&gt; Categories { get; set; }
}

public class OrderDbContext : DbContext
{
    public DbSet&lt;Order&gt; Orders { get; set; }

    public DbSet&lt;LineItem&gt; LineItems { get; set; }
}
</code></pre>
<p>First we need to configure the <code>CatalogDbContext</code> and <code>OrderDbContext</code> with the DI container.
You can do this by calling the <code>AddDbContext</code> method and specifying which <code>DbContext</code> is being configured,
and then using the SQL provider specific method to pass the connection string.
In this case I'm connecting to SQL Server with the <code>UseSqlServer</code> method.</p>
<pre><code class="language-csharp">using Microsoft.EntityFrameworkCore;

services.AddDbContext&lt;CatalogDbContext&gt;(options =&gt;
    options.UseSqlServer(&quot;CONNECTION_STRING&quot;));

services.AddDbContext&lt;OrderDbContext&gt;(options =&gt;
    options.UseSqlServer(&quot;CONNECTION_STRING&quot;));
</code></pre>
<p>If you just want to use both DbContexts in the same schema, then this is all the configuration you need.
You can now inject the <code>DbContext</code> instances in your application and use them.</p>
<p>However, if you want to configure a different schema for each <code>DbContext</code> then you also need to override
the <code>OnModelCreating</code> method and specify the custom schema with <code>HasDefaultSchema</code>.</p>
<pre><code class="language-csharp">public class CatalogDbContext : DbContext
{
    public DbSet&lt;Product&gt; Products { get; set; }

    public DbSet&lt;Category&gt; Categories { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.HasDefaultSchema(&quot;catalog&quot;);
    }
}

public class OrderDbContext : DbContext
{
    public DbSet&lt;Order&gt; Orders { get; set; }

    public DbSet&lt;LineItem&gt; LineItems { get; set; }

    protected override void OnModelCreating(ModelBuilder modelBuilder)
    {
        modelBuilder.HasDefaultSchema(&quot;order&quot;);
    }
}
</code></pre>
<p>Limitations with multiple DbContexts:</p>
<ol>
<li>It's not possible to do a join between different <code>DbContext</code> instances, because <strong>EF Core</strong> doesn't know if they
are using the same database</li>
<li>Transactions will only work if the DbContexts are using the same database. You have to create a new transaction
and share it between the DbContexts by <a href="/blog/working-with-transactions-in-ef-core#using-existing-transactions-with-ef-core">calling the <code>UseTransaction</code> method</a></li>
</ol>
<p><strong>Migrations History Table</strong></p>
<p>If you decide to use different schemas per <code>DbContext</code>, you will be unpleasantly surprised to learn
that the default schema doesn't apply to the migrations history table.</p>
<p>You need to configure this by calling the <code>MigrationsHistoryTable</code> method and specifying the table name
and schema where the migrations history for that context will live.
I used the <code>HistoryRepository.DefaultTableName</code> constant in this example, but you can specify a custom
table name if you want to.</p>
<pre><code class="language-csharp">using Microsoft.EntityFrameworkCore;
using Microsoft.EntityFrameworkCore.Migrations;

services.AddDbContext&lt;CatalogDbContext&gt;(options =&gt;
    options.UseSqlServer(
        &quot;CONNECTION_STRING&quot;,
        o =&gt; o.MigrationsHistoryTable(
            tableName: HistoryRepository.DefaultTableName,
            schema: &quot;catalog&quot;)));

services.AddDbContext&lt;OrderDbContext&gt;(options =&gt;
    options.UseSqlServer(
        &quot;CONNECTION_STRING&quot;,
        o =&gt; o.MigrationsHistoryTable(
            tableName: HistoryRepository.DefaultTableName,
            schema: &quot;order&quot;)));
</code></pre>
<h2>Benefits Of Using Multiple DbContexts</h2>
<p>Using multiple DbContexts can offer several benefits to your application:</p>
<ul>
<li>Separation of concerns</li>
<li>Better performance</li>
<li>More control &amp; security</li>
</ul>
<p>Each DbContext can be responsible for a specific subset of the application's data, which can help organize the code
and make it more modular.</p>
<p>When you separate data access into multiple DbContexts, the application can reduce the risk of contention and improve
concurrency, and this can improve performance.</p>
<p>And if you're using multiple DbContexts, you can configure more granular access control to improve application security.
You can also optimize performance and resource usage.</p>
<h2>In Summary</h2>
<p>Using <strong>multiple EF Core DbContexts</strong> in a single application is straightforward and has many benefits.</p>
<p>For ready-heavy applications you can configure a separate <code>DbContext</code> to turn off query tracking by default
and get improved performance.</p>
<p>Also, using multiple DbContexts is practical if you're building a <strong>Modular monolith</strong>.
You can configure the DbContexts to be in <strong>separate database schemas</strong>, giving you logical separation at the database level.</p>
<p>That's all for today.</p>
<p>See you next week.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_028.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How To Apply Functional Programming In C#]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-apply-functional-programming-in-csharp</link>
            <guid isPermaLink="false">how-to-apply-functional-programming-in-csharp</guid>
            <pubDate>Sat, 04 Mar 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Although C# is an object-oriented programming language, it received many new functional features in recent versions.
To mention just a few of these features:
- Pattern matching - Switch expressions - Records
You're probably already doing functional programming without even knowing it.
Do you use LINQ? If you do, then you're doing functional programming. Because LINQ is a functional .NET library.
In today's issue, I will show you how to refactor imperative code with functional programming.]]></description>
            <content:encoded><![CDATA[<p>Although <strong>C#</strong> is an <strong>object-oriented programming</strong> language, it received many new functional features in recent versions.</p>
<p>To mention just a few of these features:</p>
<ul>
<li>Pattern matching</li>
<li>Switch expressions</li>
<li>Records</li>
</ul>
<p>You're probably already doing <strong>functional programming</strong> without even knowing it.</p>
<p>Do you use LINQ?
If you do, then you're doing functional programming.
Because LINQ is a functional .NET library.</p>
<p>In today's issue, I will show you how to <strong>refactor</strong> some <strong>imperative code</strong> with <strong>functional programming</strong>.</p>
<p>Let's dive in.</p>
<h2>Benefits Of Functional Programming</h2>
<p>Before we take a look at some code, let's see what are the <strong>benefits</strong> of using <strong>functional programming</strong>:</p>
<ul>
<li>Emphasis on immutability</li>
<li>Emphasis on function purity</li>
<li>Code is easier to reason about</li>
<li>Less prone to bugs and errors</li>
<li>Ability to compose functions and create higher-order functions</li>
<li>Easier to test and debug</li>
</ul>
<p>From my experience, I think functional programming is more enjoyable once you get used to it.
Starting out, it feels strange because your old object-oriented programming habits will kick in.
But after a while, <strong>functional programming</strong> feels easier to work with than imperative code.</p>
<h2>Starting With Imperative Code</h2>
<p><strong>Imperative programming</strong> is the most basic programming approach.
We describe a step-by-step process to execute a program.
It's easier for beginners to reason with imperative code by following along with the steps in the process.</p>
<p>Here's an example of an <code>EmailValidator</code> class written with imperative code:</p>
<pre><code class="language-csharp">public class EmailValidator
{
    private const int MaxLength = 255;

    public (bool IsValid, string? Error) Validate(string email)
    {
        if (string.IsNullOrEmpty(email))
        {
            return (false, &quot;Email is empty&quot;);
        }

        if (email.Length &gt; MaxLength)
        {
            return (false, &quot;Email is too long&quot;);
        }

        if (email.Split('@').Length != 2)
        {
            return (false, &quot;Email format is invalid&quot;);
        }

        if (Uri.CheckHostName(email.Split('@')[1]) == UriHostNameType.Unknown)
        {
            return (false, &quot;Email domain is invalid&quot;);
        }

        return (true, null);
    }
}
</code></pre>
<p>You can clearly see the distinct steps:</p>
<ul>
<li>Check if email is null or empty</li>
<li>Check if email is not too long</li>
<li>Check if email format is valid</li>
<li>Check if email domain is valid</li>
</ul>
<p>Let's see how we can refactor this using <strong>functional programming</strong>.</p>
<h2>Applying Functional Programming</h2>
<p>The basic building block in <strong>functional programming</strong> is - <strong>a function</strong>.
And programs are written by composing function calls.
There are a few other things you need to keep in mind, like keeping your functions pure.
A function is pure if it always returns the same output for the same input.</p>
<p>We can capture each step from the imperative version of <code>EmailValidator</code> with a <code>Func</code> delegate.
To also capture the respective error message together with the validation check, we can use a tuple.
And since we know all of our validation steps, we can create an array of <code>Func</code> delegates to store all of the individual <strong>functions</strong>.</p>
<pre><code class="language-csharp">public class EmailValidator
{
    const int MaxLength = 255;

    static readonly Func&lt;string, (bool IsValid, string? Error)&gt;[] _validations =
    {
        email =&gt; (!string.IsNullOrEmpty(email), &quot;Email is empty&quot;),
        email =&gt; (email.Length &lt;= MaxLength, &quot;Email is too long&quot;),
        email =&gt; (email.Split('@').Length == 2, &quot;Email format is invalid&quot;),
        email =&gt; (
            Uri.CheckHostName(email.Split('@')[1]) != UriHostNameType.Unknown,
            &quot;Email domain is invalid&quot;)
    };

    static readonly (bool IsValid, string? Error) _successResult = (true, null);

    public (bool IsValid, string? Error) Validate(string email)
    {
        var validationResult = _validations
            .Select(func =&gt; func(email))
            .FirstOrDefault(func =&gt; !func.IsValid);

        return validationResult is { IsValid: false, Error: { Length: &gt;0 } } ?
            validationResult : _successResult;
    }
}
</code></pre>
<p>Notice that this allows us to do all sorts of interesting things with the <code>_validations</code> array.
How hard would it be to modify this function to return <em>all of the errors</em> instead of just the first one?</p>
<p>If you're thinking we can use LINQ's <code>Select</code> method somehow, you're thinking in the right direction.</p>
<h2>Further Reading</h2>
<p>We only scratched the surface of what functional programming is, and what you can do with it.
If you want to learn more, here are some learning materials:</p>
<ul>
<li><a href="https://www.manning.com/books/functional-programming-in-c-sharp">Functional Programming in C#, by Enrico Buonanno</a></li>
<li><a href="https://youtu.be/dDasAmowFts">Functional Programming With C# Using Railway-Oriented Programming</a></li>
<li><a href="https://youtu.be/zuy2j8vxgYc">How Function Composition Can Make Your Code Better</a></li>
<li><a href="https://youtu.be/AVA2mKG4WOc">Make Your ASP.NET Core API Controllers Incredibly Simple With Functional Programming</a></li>
</ul>
<p>Thank you for reading, and have a wonderful Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_027.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Outbox Pattern For Reliable Microservices Messaging]]></title>
            <link>https://milanjovanovic.tech/blog/outbox-pattern-for-reliable-microservices-messaging</link>
            <guid isPermaLink="false">outbox-pattern-for-reliable-microservices-messaging</guid>
            <pubDate>Sat, 25 Feb 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Working with Microservices, or any distributed system for that matter, is difficult. In a distributed system many things can go wrong, and there are even research papers about this. If you want to explore this topic futher, I suggest that you read about the fallacies of distributed computing.
Reducing the surface area for things to go wrong should be one of your goals, as an engineer. In this week's newsletter, we'll try to achieve exactly that using the Outbox pattern.
How can you implement reliable communication between components in a distributed system?
The Outbox pattern is an elegant solution to this problem, allowing you to achieve transactional guarantees in a single service and at-least-once message delivery to external systems.
Let's see what problems the Outbox pattern solves, and how can we implement it.]]></description>
            <content:encoded><![CDATA[<p>Working with <strong>Microservices</strong>, or any distributed system for that matter, is difficult.
In a distributed system many things can go wrong, and there are even research papers about this.
If you want to explore this topic further, I suggest that you read about the <a href="https://www.se.rit.edu/~se442/doc/fallacies.pdf">fallacies of distributed computing</a>.</p>
<p>Reducing the surface area for things to go wrong should be one of your goals, as an engineer.
In this week's newsletter, we'll try to achieve exactly that using the <strong>Outbox pattern</strong>.</p>
<p>How can you implement reliable communication between components in a distributed system?</p>
<p>The <strong>Outbox pattern</strong> is an elegant solution to this problem, allowing you to achieve transactional
guarantees in a single service and at-least-once message delivery to external systems.</p>
<p>Let's see how the <strong>Outbox pattern</strong> solves this and how can we implement it.</p>
<h2>What Problem Does The Outbox Pattern Solve?</h2>
<p>To understand what problem the <strong>Outbox pattern</strong> solves, first we need a problem, of course.</p>
<p>Here's an example of a user registration flow.
There are a few things going on here:</p>
<ul>
<li>Saving the <code>User</code> to the database</li>
<li>Sending a welcome email to the <code>User</code></li>
<li>Publishing a <code>UserRegisteredEvent</code> to a message bus</li>
</ul>
<pre><code class="language-csharp">public async Task RegisterUserAsync(User user, CancellationToken token)
{
    _userRepository.Insert(user);

    await _unitOfWork.SaveChangesAsync(token);

    await _emailService.SendWelcomeEmailAsync(user, token);

    await _eventBus.PublishAsync(new UserRegisteredEvent(user.Id), token);
}
</code></pre>
<p>In the happy path, all of the operations complete without any issues and all is well.</p>
<p>But what happens if any one of these operations fail?</p>
<ul>
<li>The database is unavailable, and saving the <code>User</code> fails</li>
<li>The email service is down and sending an email crashes</li>
<li>Publishing an event to the service bus doesn't succeed</li>
</ul>
<p>Also, imagine a situation where you manage to save a <code>User</code> to the database,
send him a welcome email, but fail to publish the <code>UserRegisteredEvent</code> to notify other services.
How are you going to recover from this scenario?</p>
<p>The <strong>Outbox pattern</strong> allows you to <strong>atommically</strong> update the database and send messages to the message bus.</p>
<h2>Implementing The Outbox Pattern</h2>
<p>The first step is to introduce a table in your database to represent the <strong>Outbox</strong>.
We can call this table <code>OutboxMessages</code>, and it's intended to store all messages that need to be delivered.
Now instead of directly making requests to external services, we simply store a message as a new row in the <strong>Outbox</strong> table.
The messages are usually stored as JSON in the database.</p>
<p>The second step is to introduce a <strong>background process</strong> that will periodically poll the <code>OutboxMessages</code> table.
If the worker process finds a row with an unprocessed message, it's going to publish that message and mark it as sent.
If publishing the message fails for some reason, the work process can <strong>retry</strong> in the next execution.</p>
<p>Notice that with retries, you now have <strong>at-least-once message delivery</strong> implemented.
The message will be published exactly once for the happy path, and more than one time in case or retries.</p>
<p>We can rewrite the <code>RegisterUserAsync</code> method from the previous example, now using an <strong>Outbox</strong>:</p>
<pre><code class="language-csharp">public async Task RegisterUserAsync(User user, CancellationToken token)
{
    _userRepository.Insert(user);

    _outbox.Insert(new UserRegisteredEvent(user.Id));

    await _unitOfWork.SaveChangesAsync(token);
}
</code></pre>
<p>The <strong>Outbox</strong> is part of the same transaction as our unit of work, so we can atomically save the <code>User</code> to the database
and also persist the <code>OutboxMessage</code>.
If saving to the database fails, the entire transaction is rolled back and no messages are sent to the message bus.</p>
<p>And since we now moved the publishing of the <code>UserRegisteredEvent</code> to the worker process, we need to add a handler
so that we can send the welcome email to the user.
Here's an example of that in the <code>SendWelcomeEmailHandler</code> class:</p>
<pre><code class="language-csharp">public class SendWelcomeEmailHandler : IHandle&lt;UserRegisteredEvent&gt;
{
    private readonly IUserRepository _userRepository;
    private readonly IEmailService _emailService;

    public SendWelcomeEmailHandler(
        IUserRepository userRepository,
        IEmailService emailService)
    {
        _userRepository = userRepository;
        _emailService = emailService;
    }

    public async Task Handle(UserRegisteredEvent message)
    {
        var user = await _userRepository.GetByIdAsync(message.UserId);

        await _emailService.SendWelcomeEmailAsync(user);
    }
}
</code></pre>
<h2>Architecture Diagram With Outbox</h2>
<p>Here's a high level overview of the system architecture with the <strong>Outbox</strong> introduced to the system.
You can see the <code>Outbox</code> table in the database.
What changes now is that you store messages to the <code>Outbox</code> table in the same transaction along with your entities.</p>
<p><img src="/blogs/mnw_026/outbox.png" alt=""></p>
<h2>Further Reading</h2>
<p>After reading this newsletter you should have a pretty good understanding of what the <strong>Outbox pattern</strong> is
and what problems it solves.
If you need to implement reliable messaging in a distributed system, it's a great solution for your problem.</p>
<p>What's missing is more details about how to implement the <strong>Outbox pattern</strong>, so here are a few videos you can watch:</p>
<ul>
<li><a href="https://youtu.be/BimfDeDV4yU">How to use the Domain Events pattern</a></li>
<li><a href="https://youtu.be/XALvnX7MPeo">How to implement the Outbox pattern</a></li>
<li><a href="https://youtu.be/xajVttkZntU">How to add retries to the Outbox with Polly</a></li>
</ul>
<p>Thanks for reading, and have an amazing Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_026.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Structured Logging In ASP.NET Core With Serilog]]></title>
            <link>https://milanjovanovic.tech/blog/structured-logging-in-asp-net-core-with-serilog</link>
            <guid isPermaLink="false">structured-logging-in-asp-net-core-with-serilog</guid>
            <pubDate>Sat, 18 Feb 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Structured logging is a practice where you apply the same message format to all of your application logs. The end result is that all your logs will have a similar structure, allowing them to be easily searched and analyzed.
Serilog is a popular logging library in .NET, packed with many features. It provides logging to files, logging to the console, and elsewhere.
However, why Serilog is unique is because it comes with support for structured logging out of the box.
Let's see how we can install Serilog and configure it an ASP.NET Core application.]]></description>
            <content:encoded><![CDATA[<p><strong>Structured logging</strong> is a practice where you apply the same message format to all of your
application logs.
The end result is that all your logs will have a similar structure, allowing them to be
easily searched and analyzed.</p>
<p><a href="https://serilog.net/">Serilog</a> is a popular logging library in .NET, packed with many features.
It provides logging to files, logging to the console, and elsewhere.</p>
<p>However, <strong>Serilog</strong> is unique because it comes with support for <strong>structured logging</strong> out of the box.</p>
<p>Let's see how we can install <strong>Serilog</strong> and configure it an <strong><a href="http://ASP.NET">ASP.NET</a> Core</strong> application.</p>
<h2>Installing Serilog</h2>
<p>To install <strong>Serilog</strong> in <strong><a href="http://ASP.NET">ASP.NET</a> Core</strong> you can add the following NuGet package:</p>
<pre><code class="language-powershell">Install-Package Serilog.AspNetCore
</code></pre>
<p>This NuGet packages comes with a simple API to integrate <strong>Serilog</strong> into your application.
You can call the <code>UseSerilog</code> method on the <code>HostBuilder</code> instance to provide a lambda
method to configure <strong>Serilog</strong>.</p>
<p>I think the most flexible way to configure Serilog is through application settings,
which is achieved by calling <code>ReadFrom.Configuration()</code>.</p>
<p>You can also call the <code>UseSerilogRequestLogging()</code> method to introduce automatic HTTP request logging
in your API.</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);

builder.Host.UseSerilog((context, configuration) =&gt;
    configuration.ReadFrom.Configuration(context.Configuration));

var app = builder.Build();

app.UseSerilogRequestLogging();

app.Run();
</code></pre>
<p>The next question is how do you provide the actual configuration values to <strong>Serilog</strong>?</p>
<h2>Configuring Serilog With <code>appsettings.json</code></h2>
<p>You need to add a <code>Serilog</code> section in your <code>appsettings.json</code> file.</p>
<p>Here you can configure, among other things:</p>
<ul>
<li>Which <strong>sinks</strong> to use with <strong>Serilog</strong></li>
<li>Override default and minimum log levels</li>
<li>Configure file logging arguments</li>
</ul>
<p>In this example, we're adding the <code>Console</code> and <code>File</code> sinks to <strong>Serilog</strong>.
And we're adding some additional configuration for the <code>File</code> sink in the <code>Serilog.WriteTo</code> configuration section.
We can configure the output path for the log files, the naming format, which formatter to use for the logs and so on.</p>
<pre><code class="language-json">&quot;Serilog&quot;: {
  &quot;Using&quot;: [ &quot;Serilog.Sinks.Console&quot;, &quot;Serilog.Sinks.File&quot; ],
  &quot;MinimumLevel&quot;: {
    &quot;Default&quot;: &quot;Information&quot;,
    &quot;Override&quot;: {
      &quot;Microsoft&quot;: &quot;Warning&quot;,
      &quot;System&quot;: &quot;Warning&quot;
    }
  },
  &quot;WriteTo&quot;: [
    { &quot;Name&quot;: &quot;Console&quot; },
    {
      &quot;Name&quot;: &quot;File&quot;,
      &quot;Args&quot;: {
        &quot;path&quot;: &quot;/logs/log-.txt&quot;,
        &quot;rollingInterval&quot;: &quot;Day&quot;,
        &quot;rollOnFileSizeLimit&quot;: true,
        &quot;formatter&quot;: &quot;Serilog.Formatting.Compact.CompactJsonFormatter, Serilog.Formatting.Compact&quot;
      }
    }
  ],
  &quot;Enrich&quot;: [ &quot;FromLogContext&quot;, &quot;WithMachineName&quot;, &quot;WithThreadId&quot; ]
}
</code></pre>
<p>You can get a more detailed overview of what's supported with the <code>Serilog.Configuration</code> library in the
<a href="https://github.com/serilog/serilog-settings-configuration">documentation</a>.</p>
<h2>Using Serilog In <a href="http://ASP.NET">ASP.NET</a> Core</h2>
<p>We managed to successfully install and configure <strong>Serilog</strong>.
But how do we actually use it?</p>
<p><strong>Serilog</strong> integrates with the <code>ILogger</code> interaface coming from the <code>Microsoft.Extensions.Logging</code> namespace.
If you're already using <code>ILogger</code> for logging, everything will continue working correctly.</p>
<p>Here's a simple example of logging inside of a Minimal API endpoint:</p>
<pre><code class="language-csharp">app.MapGet(&quot;/serilog-is-cool&quot;, (ILogger logger) =&gt;
{
    logger.LogInformation(&quot;This is a log inside of the Minimal API endpoint.&quot;);

    return Results.Ok(new { Message = &quot;success&quot; });
});
</code></pre>
<p>You just inject an <code>ILogger</code> or <code>ILogger&lt;T&gt;</code> instance and Serilog will provide its own implementation at runtime.</p>
<h2>Structured Logging Syntax</h2>
<p>The idea behind <strong>structured logging</strong> is that you can introduce additional contextual information inside
of your logs.
<strong>Serilog</strong> does this using a message template syntax, where you can specify named parameters and then
pass in their values separately.</p>
<p>Here's an example of what this message template would look like.
You specify parameters inside of curly bracers and provide a name, for example <code>{NamedParameter}</code>.
The value provided for the parameter will be serialized as a property inside of the corresponding
structured log.</p>
<pre><code class="language-csharp">var book = new { Author = &quot;Domain-Driven Design&quot;, Title = &quot;Eric Evans&quot; };
var orderNumber = 1;

log.LogInformation(
    &quot;Processing book {@Book}, order number = {@OrderNumber}&quot;,
    book,
    orderNumber);
</code></pre>
<p>There are a few things to unpack here:</p>
<ul>
<li><code>{@Book}</code> parameter which accepts an object</li>
<li><code>{OrderNumber}</code> parameter which accepts a scalar value</li>
</ul>
<p>The <code>@</code> operator in front of <code>Book</code> tells Serilog to serialize the object passed in, instead of converting
it using <code>ToString()</code>.</p>
<h2>Benefits Of Structured Logging</h2>
<p>Lastly, I want to highlight what are some of the benefits of <strong>structured logging</strong> and why you should be using it.</p>
<p>As I said at the beginning, the main idea with <strong>structured logging</strong> is that all log message follow the same
structure.
This structure can be a JSON document for example, or a row in a relational table.
Since structured logs are in a machine-readable format, it's much easier to search through them for
specific information.</p>
<p>When an error occurs, structured logs can provide more context and details about the error, making it easier to
identify the root cause and fix the problem.</p>
<p>It's very easy to start doing structured logging with <strong>Serilog</strong>, and I hope you'll give it a try.</p>
<p>See you next week, and have an excellent Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_025.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Messaging Made Easy With Azure Service Bus]]></title>
            <link>https://milanjovanovic.tech/blog/messaging-made-easy-with-azure-service-bus</link>
            <guid isPermaLink="false">messaging-made-easy-with-azure-service-bus</guid>
            <pubDate>Sat, 11 Feb 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[If you're working in a distributed system, you need to be able to communicate between multiple services. There are a few ways that you can implement this. Depending on your chosen approach, you can either introduce tight coupling between your services or stay loosely coupled.
Loose coupling is an important quality in a distributed system. It will allow you to evolve your services independently. So how do you implement loosely coupled communication between services?
You'll need a messaging system.
And Azure Service Bus is an excellent choice.
In this week's newsletter, I'll show you how to create an Azure Service Bus instance, and how to implement messaging over a queue.]]></description>
            <content:encoded><![CDATA[<p>If you're working in a <strong>distributed system</strong>, you need to be able to communicate between
multiple services.
There are a few ways that you can implement this.
Depending on your chosen approach, you can either introduce tight coupling between your
services or stay <strong>loosely coupled</strong>.</p>
<p><strong>Loose coupling</strong> is an important quality in distributed systems.
It allows you to evolve your services independently.
So how do you implement loosely coupled <strong>communication between services</strong>?</p>
<p>You need a <strong>messaging system</strong>.</p>
<p>And <a href="https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-messaging-overview">Azure Service Bus</a>
is an excellent choice.</p>
<p>In this week's newsletter, I'll show you how to create an <strong>Azure Service Bus</strong> instance,
and how to implement messaging over a <strong>queue</strong>.</p>
<p>Let's dive in.</p>
<h2>Creating An Azure Service Bus Instance</h2>
<p>You can create a new <strong>Azure Service Bus</strong> instance from the <strong>Azure</strong> portal.</p>
<p>I won't go into detail on that, since I find the <strong>Azure</strong> UI pretty intuitive.</p>
<p><img src="/blogs/mnw_024/service_bus.png" alt=""></p>
<p>After creating your <strong>Azure Service Bus</strong> instance, you'll need to do two more things:</p>
<ul>
<li>Create a new <strong>queue</strong></li>
<li>Find the <strong>connection string</strong></li>
</ul>
<p>After that you can proceed with implementing the pub-sub pattern over a queue.
And you'll use the connection string from the Azure portal for connecting to the Azure Service Bus instance.</p>
<h2>Publishing Messages To The Azure Service Bus Queue</h2>
<p>The first thing we need to do is to create the publishing side of our system, and then
we'll see how we can process messages.
We're going to use the <code>Azure.Messaging.ServiceBus</code> library to connect to the queue
running in Azure Service Bus.</p>
<p>You can install the NuGet package by running the following command:</p>
<pre><code class="language-powershell">Install-Package Azure.Messaging.ServiceBus
</code></pre>
<p>And now, let's write the code for publishing a message to an Azure Service Bus queue.</p>
<p>To work with the Azure Service Bus instance, you will use the <a href="https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.servicebusclient?view=azure-dotnet"><code>ServiceBusClient</code></a>
class.
It requires a connection string to be able to connect to the Azure Service Bus instance.</p>
<p>The <code>ServiceBusClient</code> is safe to cache and reuse in the application, so it can be registered
as a service with the singleton lifetime.</p>
<p>With the <code>ServiceBusClient</code> you can create a <a href="https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.servicebussender?view=azure-dotnet"><code>ServiceBusSender</code></a>
instance, which is responsible
for sending the actual messages.
You also need to specify which queue it will be sending messages to.
This can also be the name of a topic, if you are publishing to a topic instead.</p>
<pre><code class="language-csharp">using Azure.Messaging.ServiceBus;

await using var client = new ServiceBusClient(ConnectionString);

await using ServiceBusSender sender = client.CreateSender(QueueName);

// This will be the payload for the message ✉️
var productCreated = new ProductCreatedEvent(
    eventId: Guid.NewGuid(),
    product.Id,
    product.Name);

string json = JsonSerializer.Serialize(productCreated);

var message = new ServiceBusMessage(json);

await sender.SendMessageAsync(message);
</code></pre>
<p>As you can see, publishing a message to the queue is relatively simple.</p>
<p>We're creating a new instance of <code>ProductCreatedEvent</code>, which represents our message payload.
The payload itself is serialized into a <code>JSON</code> string, wrapped inside a <code>ServiceBusMessage</code>.</p>
<p>For publishing messages to Azure Service Bus, you simply call the <code>SendMessageAsync</code> method.</p>
<h2>Receiving Messages From The Azure Service Bus Queue</h2>
<p>Publishing messages to a queue is only half of the job.
You also need to be able to receive messages from the queue.
However, you'll see that this is very similar to the publishing side.</p>
<p>On the receiving side, you'll also need to create a <code>ServiceBusClient</code>.
And then use it to create an instance of <a href="https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.servicebusprocessor?view=azure-dotnet"><code>ServiceBusProcessor</code></a>,
which is used for consuming messages.
You need to tell the <code>ServiceBusProcessor</code> which queue it will subscribe to.</p>
<p>The <code>ServiceBusProcessor</code> exposes two events, which represent callbacks for when a message is received.
These events are <a href="https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.servicebusprocessor.processmessageasync?view=azure-dotnet"><code>ProcessMessageAsync</code></a>
and <a href="https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.servicebusprocessor.processerrorasync?view=azure-dotnet"><code>ProcessErrorAsync</code></a>.</p>
<p>You need to provide a handler for these two events, to properly consume messages from the queue.
In the example below, we're using the <code>HandleMessageAsync</code> and <code>HandleErrorAsync</code> local functions.</p>
<pre><code class="language-csharp">using Azure.Messaging.ServiceBus;

await using var client = new ServiceBusClient(ConnectionString);

await using ServiceBusProcessor processor  = client.CreateProcessor(QueueName);

processor.ProcessMessageAsync += HandleMessageAsync;

processor.ProcessErrorAsync += HandleErrorAsync;

await processor.StartProcessingAsync();

async Task HandleMessageAsync(ProcessMessageEventArgs args)
{
    string json = args.Message.Body.ToString();

    var productCreated = JsonSerializer.Deserialize&lt;ProductCreatedEvent&gt;(json);

    Console.WriteLine(productCreated);

    await args.CompleteMessageAsync(args.Message);
}

Task HandleErrorAsync(ProcessErrorEventArgs args)
{
    var exception = args.Exception;

    Console.WriteLine(exception.ToString());

    return Task.CompletedTask;
}
</code></pre>
<p>The <code>ServiceBusProcessor</code> begins listening to messages coming from the queue after calling the <code>StartProcessingAsync</code> method.</p>
<p>Notice that the success and error event handlers need to accept the
<a href="https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.processmessageeventargs?view=azure-dotnet"><code>ProcessMessageEventArgs</code></a>
and <a href="https://learn.microsoft.com/en-us/dotnet/api/azure.messaging.servicebus.processerroreventargs?view=azure-dotnet"><code>ProcessErrorEventArgs</code></a>,
respectively.</p>
<p>From the <code>ProcessMessageEventArgs</code> you can access the <code>Message.Body</code> which contains the mesage payload.</p>
<h2>Further Reading</h2>
<p><strong>Azure Service Bus</strong> is a very feature rich service cloud.
I showed you how to work with <strong>queues</strong>, which are great if you only have one publisher and one subscriber.
However, if you need the ability to have multiple subscribers to a single message, you can't achieve
this with queues.</p>
<p>You will have to use <a href="https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-queues-topics-subscriptions#topics-and-subscriptions">topics</a>,
and I invite you to research this <em>topic</em> further (pun intended).</p>
<p><strong>Azure Functions</strong> have excellent support for integrating with <strong>Azure Service Bus</strong>.
You can define a <code>QueueTrigger</code> that will run your <strong>Azure Function</strong> when you receive a message to an Azure Service Bus <strong>queue</strong>.</p>
<p>I also released a video showing how to <a href="https://youtu.be/CTKWFMZVIWA">publish and consume messages using RabbitMQ</a>,
and I think you might enjoy it after reading this newsletter.</p>
<p>Have an excellent weekend, and stay awesome!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_024.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Working With Transactions In EF Core]]></title>
            <link>https://milanjovanovic.tech/blog/working-with-transactions-in-ef-core</link>
            <guid isPermaLink="false">working-with-transactions-in-ef-core</guid>
            <pubDate>Sat, 04 Feb 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[Every software engineer working with SQL databases needs to know about transactions. And since most of the time the SQL database will be abstracted by an ORM like EF Core, it's important to understand how you can work with transactions using the available abstractions.
So today, I'll show you how to work with transactions in EF Core.
Here's what we will cover:
- Default transaction behavior - Creating transactions - Using existing transactions]]></description>
            <content:encoded><![CDATA[<p>Every software engineer working with <strong>SQL databases</strong> needs to know about <strong>transactions</strong>.
And since most of the time the <strong>SQL database</strong> will be abstracted by an ORM like <strong>EF Core</strong>,
it's important to understand how you can work with <strong>transactions</strong> using the available
abstractions.</p>
<p>So today, I'll show you how to work with <strong>transactions</strong> in <strong>EF Core</strong>.</p>
<p>Here's what we will cover:</p>
<ul>
<li>Default transaction behavior</li>
<li>Creating transactions</li>
<li>Using existing transactions</li>
</ul>
<p>Let's dive in.</p>
<h2>Default Transaction Behavior</h2>
<p>What is the <strong>default</strong> EF Core <strong>transaction behavior</strong>?</p>
<p>By default, all changes made in a single call to <code>SaveChanges</code> are applied in a
transaction. If any of the changes fail, the entire transaction is rolled back
and no changes are applied to the database. Only if all changes are successfully
persisted to the database, the call to <code>SaveChanges</code> can complete.</p>
<p>This is a wonderful feature of <strong>SQL databases</strong> and it saves us many headaches.
We don't have to think about the databases remaining in an inconsistent state,
because database <strong>transactions</strong> can do the work for us.</p>
<p>Let's take a look at an example.</p>
<pre><code class="language-csharp">using var context = new ShoppingContext();

context.LineItems.Add(new LineItem
{
    ProductId = productId,
    Quantity = quantity
});

var stock = context.Stock.FirstOrDefault(s =&gt; s.ProductId == productId);

stock.Quantity -= quantity;

context.SaveChanges();
</code></pre>
<p>Because we are adding a <code>LineItem</code>, and in the same scope reducing the <code>Stock</code>
quantity, the call to <code>SaveChanges</code> will apply both changes inside of a transaction.
We can guarantee that the database will remain in a <strong>consistent state</strong>.</p>
<h2>Creating Transactions With EF Core</h2>
<p>What if you want to have more control over <strong>transactions</strong> when working with <strong>EF Core</strong>?</p>
<p>You can manually create a transaction by accessing the <code>Database</code> facade available
on a <code>DbContext</code> instance and calling <code>BeginTransaction</code>.</p>
<p>Here's an example where we have multiple calls to <code>SaveChanges</code>. In the default
scenario, both calls would run in their own transaction. This leaves the possibility
of the second call to <code>SaveChanges</code> failing, and leaving the database in an
inconsistent state.</p>
<pre><code class="language-csharp">using var context = new ShoppingContext();
using var transaction  = context.Database.BeginTransaction();

try
{
    context.LineItems.Add(new LineItem
    {
        ProductId = productId,
        Quantity = quantity
    });

    context.SaveChanges();

    var stock = context.Stock.FirstOrDefault(s =&gt; s.ProductId == productId);

    stock.Quantity -= quantity;

    context.SaveChanges();

    // When we commit the changes, they will be applied to the databases.
    // The transaction will auto-rollback when it is disposed,
    // if any command fails.
    transaction.Commit();
}
catch (Exception)
{
    transaction.Rollback();
}
</code></pre>
<p>We call <code>BeginTransaction</code> to manually start a new <strong>database transaction</strong>.
This will create a new transaction and return it, so that we can <code>Commit</code> the
transaction when we want to complete the operation. You also want to add a
<code>try-catch</code> block around your code, so that you can <code>Rollback</code> the transaction
if there are any exceptions.</p>
<h2>Using Existing Transactions With EF Core</h2>
<p>Creating a transaction using the EF Core <code>DbContext</code> isn't the only option.
You can create a <code>SqlTransaction</code> instance and pass it to <strong>EF Core</strong>, so that
the changes applied with EF Core can be committed inside the same <strong>transaction</strong>.</p>
<p>Here's what I mean:</p>
<pre><code class="language-csharp">using var sqlConnection = new SqlTransaction(connectionString);
sqlConnection.Open();

using var transaction = sqlConnection.BeginTransaction();

try
{
    using var context = new ShoppingContext();

    // Tell EF Core to use an existing transaction.
    context.UseTransaction(transaction);

    context.LineItems.Add(new LineItem
    {
        ProductId = productId,
        Quantity = quantity
    });

    context.SaveChanges();

    var stock = context.Stock.FirstOrDefault(s =&gt; s.ProductId == productId);

    stock.Quantity -= quantity;

    context.SaveChanges();

    transaction.Commit();
}
catch (Exception)
{
    transaction.Rollback();
}
</code></pre>
<h2>In Summary</h2>
<p><strong>EF Core</strong> has excellent support for <strong>transactions</strong> and it's very easy to work with.</p>
<p>You have three options available:</p>
<ul>
<li>Rely on the default transaction behavior</li>
<li>Create a new transaction</li>
<li>Use an existing transaction</li>
</ul>
<p>Most of the time, you want to rely on the default behavior and not have to
think about it.</p>
<p>As soon as you need to perform multiple <code>SaveChanges</code> calls, you should manually
create a transaction, and manage the transaction yourself.</p>
<p>See you next week, and have an excellent Saturday.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_023.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How To Implement API Key Authentication In ASP.NET Core]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-implement-api-key-authentication-in-aspnet-core</link>
            <guid isPermaLink="false">how-to-implement-api-key-authentication-in-aspnet-core</guid>
            <pubDate>Sat, 28 Jan 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[In this week's newsletter I want to show you how to implement API Key authentication in ASP.NET Core. This authentication approach uses an API Key to authenticate the client of an API. You can pass the API Key to the API in a few ways, such as through the query string or a request header.
I will show you how to implement API Key authentication where the API key is passed in a request header. But the implementation would be similar if we were to use any other approach.
When would you want to use API Key authentication? This kind of authentication mechanism is common in Server-to-Server (S2S) communication. When your API serves request for other server-side applications to consume and integrate with. It's less common in client-server communication scenarios.]]></description>
            <content:encoded><![CDATA[<p>In this week's newsletter I want to show you how to implement <strong>API Key authentication</strong>
in <strong><a href="http://ASP.NET">ASP.NET</a> Core</strong>. This authentication approach uses an <strong>API Key</strong> to authenticate the
client of an API. You can pass the <strong>API Key</strong> to the API in a few ways, such as through
the query string or a request header.</p>
<p>I will show you how to implement <strong>API Key authentication</strong> where the <strong>API key</strong> is passed
in a request header. But the implementation would be similar if we were to use any
other approach.</p>
<p>When would you want to use <strong>API Key authentication</strong>? This kind of authentication
mechanism is common in <strong>Server-to-Server (S2S)</strong> communication. When your API serves
request for other server-side applications to consume and integrate with. It's
less common in client-server communication scenarios.</p>
<p>Let's see how we can implement <strong>API Key authentication</strong> in <a href="http://ASP.NET">ASP.NET</a> Core!</p>
<h2>Implementing API Key Authentication</h2>
<p>We will start off by creating an attribute that we can place on endpoints
where we want to apply <strong>API Key authentication</strong>. It won't be any kind of
attribute, because we will use a <code>ServiceFilterAttribute</code>.</p>
<p>What a <code>ServiceFilterAttribute</code> allows us to do is specify a type for the
filter that will be created for that attribute.
This means we can implement our authentication logic in an <code>IAuthorizationFilter</code>.
With a <code>ServiceFilterAttribute</code> we also have support for dependency injection
in our <code>IAuthorizationFilter</code> implementation.</p>
<p>Let's first define the <code>ApiKeyAttribute</code> class:</p>
<pre><code class="language-csharp">public class ApiKeyAttribute : ServiceFilterAttribute
{
    public ApiKeyAttribute()
        : base(typeof(ApiKeyAuthorizationFilter))
    {
    }
}
</code></pre>
<p>In the <code>ApiKeyAttribute</code> we specify <code>ApiKeyAuthorizationFilter</code> class as the
filter that will be resolved from the DI container. Here's what it looks like:</p>
<pre><code class="language-csharp">public class ApiKeyAuthorizationFilter : IAuthorizationFilter
{
    private const string ApiKeyHeaderName = &quot;X-API-Key&quot;;

    private readonly IApiKeyValidator _apiKeyValidator;

    public ApiKeyAuthorizationFilter(IApiKeyValidator apiKeyValidator)
    {
        _apiKeyValidator = apiKeyValidator;
    }

    public void OnAuthorization(AuthorizationFilterContext context)
    {
        string apiKey = context.HttpContext.Request.Headers[ApiKeyHeaderName];

        if (!_apiKeyValidator.IsValid(apiKey))
        {
            context.Result = new UnauthorizedResult();
        }
    }
}
</code></pre>
<p>The implementation comes down to validating the <strong>API Key</strong> obtained from
the header of the current request. If we determine that the <strong>API Key</strong>
is not valid, we set the value of <code>AuthorizationFilterContext.Result</code>
to a new instance of an <code>UnauthorizedResult</code>.</p>
<p>And lastly, all that's left for us to do is implement our custom
validation logic for the <strong>API Key</strong> inside of <code>ApiKeyValidator</code>:</p>
<pre><code class="language-csharp">public class ApiKeyValidator : IApiKeyValidator
{
    public bool IsValid(string apiKey)
    {
        // Implement logic for validating the API key.
    }
}

public interface IApiKeyValidator
{
    bool IsValid(string apiKey);
}
</code></pre>
<p>The actual implementation for validating the <strong>API Key</strong> will vary based
on your use case, and where you are storing the API keys.
For example, if you store the API keys in the database you would check
if the provided <strong>API Key</strong> exists in the database.
If it exists, then validation passes.
If it doesn't exist, then validation fails and we return an
<code>UnauthorizedResult</code>.</p>
<h2>Registering Services With Dependency Injection</h2>
<p>We have to make sure to register our <code>ApiKeyAuthorizationFilter</code> and
<code>ApiKeyValidator</code> services with the dependency injection container.</p>
<pre><code class="language-csharp">builder.Services.AddSingleton&lt;ApiKeyAuthorizationFilter&gt;();

builder.Services.AddSingleton&lt;IApiKeyValidator, ApiKeyValidator&gt;();
</code></pre>
<p>This will register them as singleton services in our application.
You can use a different service scope such as <code>Transient</code> or <code>Scoped</code>
if you need to.</p>
<h2>Applying API Key Authentication To Endpoints</h2>
<p>Finally, with our <strong>API Key authentication</strong> in place, we can apply the
<code>ApiKeyAttribute</code> attribute to our endpoints:</p>
<pre><code class="language-csharp">public class NewslettersController : ControllerBase
{
    [ApiKey]
    [HttpGet]
    public IActionResult Get()
    {
        // ...
    }
}
</code></pre>
<p>In this case I'm applying the <code>ApiKeyAttribute</code> to an endpoint, but
you can also apply it on the <code>NewslettersController</code> and it will add
authentication to all the endpoints for that controller.</p>
<h2>Next Steps</h2>
<p>Now that you know how to implement <strong>API Key authentication</strong>, I think you
should also learn how to implement <strong>JWT authentication</strong>. And while you're
at it, why not throw <strong>authorization</strong> into the mix.</p>
<p>I made a few videos about <strong>JWT authentication</strong> and <strong>permission authorization</strong>
that you should take a look at next:</p>
<ul>
<li><a href="https://youtu.be/4cFhYUK8wnc">Token Authentication In ASP.NET Core 7 With JWT</a></li>
<li><a href="https://youtu.be/PlbAuNvR16s">Introduction To Permission Authorization In ASP.NET Core 7</a></li>
<li><a href="https://youtu.be/v4vXDRJ9_sg">Managing Permissions With EF Core Migrations</a></li>
<li><a href="https://youtu.be/SZtZuvcMBA0">Implementing A Custom Authorization Handler In ASP.NET Core</a></li>
<li><a href="https://youtu.be/SUyFPp6BPV0">Using Custom JWT Claims For Authorization In ASP.NET Core</a></li>
</ul>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_022.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[C# Yield Return Statement]]></title>
            <link>https://milanjovanovic.tech/blog/csharp-yield-return-statement</link>
            <guid isPermaLink="false">csharp-yield-return-statement</guid>
            <pubDate>Sat, 21 Jan 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[In this week's newsletter I want to talk about the yield keyword in C#. I think it's a powerful C# feature and I wanted to highlight the benefits.
The yield keyword tells the compiler that the method in which it appears is an iterator block. An iterator block, or method, returns an IEnumerable as the result. And the yield keyword is used to return the values for the IEnumerable.
An interesting thing aboug IEnumerable is that it is lazily evaluted. Calling a method with an iterator block doesn't run any code. It's only when the IEnumerable is iterated over, or enumerated, that we get the actual values. I'll talk about this more later.
Let's see how we can start using the yield keyword.]]></description>
            <content:encoded><![CDATA[<p>In this week's newsletter I want to talk about the <code>yield</code> keyword in <strong>C#</strong>.
I think it's a powerful <strong>C#</strong> feature and I wanted to highlight the benefits.</p>
<p>The <code>yield</code> keyword tells the compiler that the method in which it appears
is an <strong>iterator block</strong>. An iterator block, or method, returns an <code>IEnumerable</code>
as the result. And the <code>yield</code> keyword is used to return the values for the
<code>IEnumerable</code>.</p>
<p>An interesting thing aboug <code>IEnumerable</code> is that it is lazily evaluted.
Calling a method with an iterator block doesn't run any code. It's only
when the <code>IEnumerable</code> is iterated over, or enumerated, that we get
the actual values. I'll talk about this more later.</p>
<p>Let's see how we can start using the <code>yield</code> keyword!</p>
<h2>How To Use The Yield Keyword</h2>
<p>The <code>yield</code> keyword on it's own doesn't do anything, you have to combine
it with the <code>return</code> or <code>break</code> statement:</p>
<ul>
<li><code>yield return</code> - provides the next value of the iterator</li>
<li><code>yield-break</code> - signals the end of iteration</li>
</ul>
<p>In every project I worked on, there's a piece of code similar to the
following. You create a list to hold the results, add elements to the
list, and return the list in the end.</p>
<pre><code class="language-csharp">var engineers = GetSoftwareEngineers();

public IEnumerable&lt;SoftwareEngineer&gt; GetSoftwareEngineers()
{
    var result = new List&lt;SoftwareEngineer&gt;();

    for(var i = 0; i &lt; 10; i++)
    {
        result.Add(new SoftwareEngineer
        {
            Id = i
        });
    }

    return result;
}
</code></pre>
<p>You can simplify the method using the <code>yield return</code> statement, and
completely remove the intermediate list required to hold the results.</p>
<pre><code class="language-csharp">var engineers = GetSoftwareEngineers();

public IEnumerable&lt;SoftwareEngineer&gt; GetSoftwareEngineers()
{
    for(var i = 0; i &lt; 10; i++)
    {
        yield return new SoftwareEngineer
        {
            Id = i
        };
    }
}
</code></pre>
<p>However, it's important to note these two implementation are fundamentally
different from each other. In the first example, the entire list is
populated and materialized. In the second example, the <code>IEnumerable</code>
returned will not be materialized and you have to either iterate over
it inside a <code>foreach</code> loop or call <code>ToList()</code>.</p>
<h2>Stopping Iteration With Yield Break</h2>
<p>You can use the <code>yield break</code> statement to stop iteration and exit
the iterator block. Typically you would do this when a certain
condition is met, or you only want to return a specific set of values
from the iterator block.</p>
<p>Here's an example where this would be useful:</p>
<pre><code class="language-csharp">Console.WriteLine(string.Join(&quot;, &quot;, TakeWhilePositive(new[] { 1, 2, -3, 4 })));
// Output: 1, 2

public IEnumerable&lt;int&gt; TakeWhilePositive(IEnumerable&lt;int&gt; numbers)
{
    foreach(int num in numbers)
    {
        if (num &gt; 0)
        {
            yield return num;
        }
        else
        {
            yield break;
        }
    }
}
</code></pre>
<h2>Working With IAsyncEnumerable</h2>
<p>In <strong>C# 8</strong> we got the <code>IAsyncEnumerable</code> type which allows us to
iterate over a collection asynchronously with the <code>yield</code> statement.</p>
<p>For example, this can be useful when you want to call a thid-party
API multiple times to fetch some data. A common situation is when
you get a list of users from the database, and then have to call
an external storage service to get profile picture information.</p>
<p>Without <code>IAsyncEnumerable</code> you would have to do something like this:</p>
<pre><code class="language-csharp">public async Task&lt;IEnumerable&lt;User&gt;&gt; GetUsersAsync()
{
    var users = await GetUsersFromDbAsync();

    foreach(var user in users)
    {
        user.ProfileImage = await GetProfileImageAsync(user.Id);
    }

    return users;
}

// And you would call the method like this.
var users = await GetUsersAsync();

foreach(var user in users)
{
    Console.WriteLine(user);
}
</code></pre>
<p>Now, consider this same example with the use of <code>IAsyncEnumerable</code>:</p>
<pre><code class="language-csharp">public async IAsyncEnumerable&lt;User&gt; GetUsersAsync()
{
    var users = await GetUsersFromDbAsync();

    foreach(var user in users)
    {
        user.ProfileImage = await GetProfileImageAsync(user.Id);

        yield return user;
    }
}

// And you would call the method like this.
await foreach(var user in GetUsersAsync())
{
    Console.WriteLine(user);
}
</code></pre>
<p>The second implementation will iterate over the users returned from
the database when they are yielded by the <code>IAsyncEnumerable</code>.</p>
<h2>When Should I Use Yield?</h2>
<p>I've found a few interesting practical applications for the <code>yield</code> keyword.
One example is when implementing Domain-Driven Design value objects.</p>
<p>Value objects need to support structural equality. They need to implement
a method that returns all of the equality components. Here's an example of
that using the <code>yield return</code> statement:</p>
<pre><code class="language-csharp">public class Address
{
    public string City { get; init; }

    public string Street { get; init; }

    public string Zip { get; init; }

    public string Country { get; init; }

    public IEnumerable&lt;object&gt; GetEqualityComponents()
    {
        yield return City;
        yield return Street;
        yield return Zip;
        yield return Country;
    }
}
</code></pre>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_021.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Unleash EF Core Performance With Compiled Queries]]></title>
            <link>https://milanjovanovic.tech/blog/unleash-ef-core-performance-with-compiled-queries</link>
            <guid isPermaLink="false">unleash-ef-core-performance-with-compiled-queries</guid>
            <pubDate>Sat, 14 Jan 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[In this week's newsletter I want to introduce you to an interesting feature in EF Core called Compiled Queries. If you have queries that you execute frequently in your application with a different set of parameters, it can be helpful to explicitly compile the query and reuse it throughout the lifetime of your application. Compiled Queries are more performant than standard EF queries, because they can take advantage of some additional optimizations. Let me show you how to use Compiled Queries, and how much performance improvement to expect.]]></description>
            <content:encoded><![CDATA[<p>In this week's newsletter I want to introduce you to an interesting feature
in <strong>EF Core</strong> called <strong>Compiled Queries</strong>.</p>
<p>If you have queries that you execute frequently in your application with a
different set of parameters, it can be helpful to explicitly compile the
query and reuse it throughout the lifetime of your application.</p>
<p><strong>Compiled Queries</strong> are more performant than standard EF queries, because they
can take advantage of some additional optimizations.</p>
<p>Let me show you how to use <strong>Compiled Queries</strong>, and how much performance
improvement to expect!</p>
<h2>How To Create Compiled Queries</h2>
<p>Lets define a simple class that will represent out data model that we will
use for writing <strong>EF</strong> queries:</p>
<pre><code class="language-csharp">public class Newsletter
{
    public long Id { get; init; }

    public string Title { get; init; }

    public int ReadTimeInMinutes { get; init; }
}
</code></pre>
<p>How would we write a simple query to fetch a <code>Newsletter</code> by the <code>Id</code>?
I think you can do this in your sleep.</p>
<pre><code class="language-csharp">using var dbContext = new AppDbContext();

var newsletter = dbContext.Set&lt;Newsletter&gt;().FirstOrDefault(n =&gt; n.Id == id);
</code></pre>
<p>Now, how do we convert this query into a <strong>Complied Query</strong>?</p>
<p>There are a few steps involved:</p>
<ul>
<li>Create a <strong>Compiled Query</strong> by calling <code>EF.CompileQuery</code></li>
<li>Store the <strong>Compiled Query</strong> in a static field, so that it can be reused</li>
<li>Execute the database query using the <strong>Compiled Query</strong></li>
</ul>
<p>You can define the <strong>Compiled Query</strong> in a static field inside of the <code>AppDbContext</code>.
And then expose a method that will accept an argument, and pass it to the
<strong>Compiled Query</strong> to invoke it.</p>
<pre><code class="language-csharp">using Microsoft.EntityFrameworkCore;

public class AppDbContext
{
    private static Func&lt;AppDbContext, long, Newsletter?&gt; GetNewsletter =
        EF.CompileQuery(
            (dbContext, id) =&gt;
                dbContext.Set&lt;Newsletter&gt;().FirstOrDefault(n =&gt; n.Id == id));

    public Newsletter? GetNewsletter(long id)
    {
        return GetNewsletter(this, id);
    }
}
</code></pre>
<p>This is how we would call the method which invokes the <strong>Compiled Query</strong>:</p>
<pre><code class="language-csharp">using var dbContext = new AppDbContext();

var newsletter = dbContext.GetNewsletter(id);
</code></pre>
<p>I ran some benchmarks, with the following setup:</p>
<ul>
<li><strong>EF Core 7</strong></li>
<li><strong>SQL Server 2022</strong></li>
<li>Table with 10,000 records</li>
</ul>
<p>The <strong>Compiled Query</strong> was consistently around <strong>10% faster</strong>.</p>
<p>I also tried running a no-tracking query by calling <code>AsNoTracking()</code> and
observed similar results.</p>
<h2>Why Are Compiled Queries Faster?</h2>
<p>So we can conclude that <strong>Compiled Queries</strong> are faster. But why is that?</p>
<p>Let's examine what happens when we execute an <strong>EF</strong> LINQ query. Before <strong>EF</strong>
can convert the query into valid SQL that can be executed in the database,
it needs to compile the query. The compiled query is cached and <strong>EF</strong> will be
able to reuse that cached query. In some situations the query needs to be
recompiled, introducing additional performance costs.</p>
<p>When we explicitly compile the query by calling <code>EF.CompileQuery</code>, we
can utilize some optimization techniques that aren't available at runtime.</p>
<p>Note that <strong>Compiled Queries</strong> only improve the performance of the in-memory
portion of executing an EF query. The round trip time and materializing
results from the database remain unaffected.</p>
<h2>Can We Make Compiled Queries Asynchronous?</h2>
<p>I showed you how to write a synchronous <strong>Compiled Query</strong>. But due to
performance considerations we almost always want to execute database
queries asynchronously.</p>
<p>Here's how we can create an asynchronous <strong>Compiled Query</strong>:</p>
<pre><code class="language-csharp">using Microsoft.EntityFrameworkCore;

public class AppDbContext
{
    private static Func&lt;AppDbContext, string, Task&lt;Newsletter?&gt;&gt; GetByTitle =
        EF.CompileAsyncQuery(
            (AppDbContext context, string title) =&gt;
                context.Set&lt;Newsletter&gt;().FirstOrDefault(c =&gt; c.Title == title));

    public async Task&lt;Newsletter?&gt; GetNewsletterByTitleAsync(string title)
    {
        return await GetByTitle(this, title);
    }
}
</code></pre>
<p>It's interesting that we aren't writing an asynchronous query in the expression
passed to <code>EF.CompileAsyncQuery</code>. It will be converted to an asynchronous query
during compilation.</p>
<h2>Compiled Queries Aren't a Silver Bullet</h2>
<p>You might be tempted to go and convert all of your <strong>EF</strong> queries into
<strong>Compiled Queries</strong>, to squeeze out that last little bit of performance.
I urge you not do it. <strong>Compiled Queries</strong> are a useful tool, but they
aren't the solution to all your problems.</p>
<p>Instead, I think we should use <strong>Compiled Queries</strong> sparingly, only in
situations where we really need to do these kinds of micro-optimizations.</p>
<p>Although <strong>Compiled Queries</strong> seem great, we can't deny they increase
the complexity of our code considerably. If you think the slight
performance improvement gained from using <strong>Compiled Queries</strong> justifies
the increase in complexity, then by all means, you should use them.
Otherwise, I would look for other ways to improve performance.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_020.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Adding Validation To The Options Pattern In ASP.NET Core]]></title>
            <link>https://milanjovanovic.tech/blog/adding-validation-to-the-options-pattern-in-asp-net-core</link>
            <guid isPermaLink="false">adding-validation-to-the-options-pattern-in-asp-net-core</guid>
            <pubDate>Sat, 07 Jan 2023 00:00:00 GMT</pubDate>
            <description><![CDATA[In this week's newsletter I will show you how to easily add validation the strongly-typed configuration objects injected with IOptions. The Options pattern allows us to use classes to provide strongly-typed configuration values in our application at runtime. But you have no guarantee that the configuration values injected with IOptions will be correctly read from the application settings. Let's see how we can introduce validation for our IOptions and make sure the application settings are correct.]]></description>
            <content:encoded><![CDATA[<p>In this week's newsletter I will show you how to easily add <strong>validation</strong>
to the strongly typed configuration objects injected with <code>IOptions</code>.</p>
<p>The <strong>Options pattern</strong> allows us to use classes to provide strongly typed
configuration values in our application at runtime.</p>
<p>But you have no guarantee that the configuration values injected with
<code>IOptions</code> will be correctly read from the application settings.</p>
<p>Let's see how we can introduce validation for <code>IOptions</code>, and make sure the application settings are correct.</p>
<h2>Strongly Typed Configuration</h2>
<p>I first want to define a simple class that will represent our strongly
typed configuration. Let's say we want to integrate with the <strong>GitHub API</strong>,
so we create a <code>GitHubSettings</code> class to hold our configuration:</p>
<pre><code class="language-csharp">public class GitHubSettings
{
    public string AccessToken { get; init; }

    public string RepositoryName { get; init; }
}
</code></pre>
<p>Inside of our <code>appsettings.json</code> file we need to create a section to
hold our configuration values:</p>
<pre><code class="language-json">&quot;GitHubSettings&quot;: {
    &quot;AccessToken&quot;: &quot;access-token-value&quot;,
    &quot;RepositoryName&quot;: &quot;youtube-projects&quot;
}
</code></pre>
<p>And with this in place, we can configure our <code>GitHubSettings</code>:</p>
<pre><code class="language-csharp">builder.Services.Configure&lt;GitHubSettings&gt;(
    builder.Configuration.GetSection(&quot;GitHubSettings&quot;));
</code></pre>
<p>Finally, our <code>GitHubSettings</code> is properly configured and we can inject
it with <code>IOptions&lt;GitHubSettings&gt;</code>.</p>
<h2>What Could Go Wrong?</h2>
<p>If we leave the implementation like this, we're moving the responsibility
for providing the correct configuration values to the developer. I'm not
saying we are the problem, but I've forgotten to add application settings
a few times. I'm sure this happened to you also.</p>
<p>Here are just a few things that can go wrong:</p>
<ul>
<li>Passing an incorrect section name to <code>IConfiguration.GetSection</code></li>
<li>Forgetting to add the settings values in <code>appsettings.json</code></li>
<li>Typo in a property name in the class or in the configuration</li>
<li>Unbindale properties without a setter</li>
<li>Data type mismatch resulting in incompatible values</li>
</ul>
<p>Depending on which one of these mistakes is made, the application will
behave differently at runtime.</p>
<p>The best case scenario is that the incorrect application settings cause
a runtime exception, and you realize you have a problem and fix it.</p>
<p>The worst case scenario, and this happens more often than you may think,
is that the application silently fails. The application settings aren't
correctly set on the value provided by <code>IOptions</code>, but you don't get a
runtime exception. The problem may go undetected for some time.</p>
<p>How do we solve this?</p>
<h2>Validation For The Options Pattern</h2>
<p>There is a simple way to introduce <strong>validation</strong> to the settings class
using <strong>data annotations</strong>. We just add the validation attributes that
we need to the properties of the settings class.</p>
<p>For example, we can add the <code>Required</code> attribute to the <code>GitHubSettings</code>
properties:</p>
<pre><code class="language-csharp">public class GitHubSettings
{
    [Required]
    public string AccessToken { get; init; }

    [Required]
    public string RepositoryName { get; init; }
}
</code></pre>
<p>We have to slightly change how we configure the <code>GitHubSettings</code>:</p>
<pre><code class="language-csharp">builder.Services
    .AddOptions&lt;GitHubSettings&gt;()
    .BindConfiguration(&quot;GitHubSettings&quot;)
    .ValidateDataAnnotations();
</code></pre>
<p>A few things to note here:</p>
<ul>
<li><code>AddOptions</code> - returns an <code>OptionsBuilder&lt;TOptions&gt;</code> that binds to
the <code>GitHubSettings</code> class</li>
<li><code>BindConfiguration</code> - binds the values from the configuration section</li>
<li><code>ValidateDataAnnotations</code> - enables <strong>validation</strong> using <strong>data annotations</strong></li>
</ul>
<p>With this in place, if we try to inject <code>GitHubSettings</code> with any of
the properties missing a value, we will get a runtime exception.</p>
<p>You can also define a <strong>custom delegate</strong> for the <strong>validation</strong> logic, instead
of using data annotations:</p>
<pre><code class="language-csharp">builder.Services
    .AddOptions&lt;GitHubSettings&gt;()
    .BindConfiguration(&quot;GitHubSettings&quot;)
    .Validate(gitHubSettings =&gt;
    {
        if (string.IsNullOrEmpty(gitHubSettings.AccessToken))
        {
            return false;
        }

        return true;
    });
</code></pre>
<h2>Running Validation At Application Start</h2>
<p>It would be great if we could run <strong>validation</strong> on the configuration values
when our application is starting, instead of at runtime.</p>
<p>We can do that by calling <code>ValidateOnStart</code> method when configuring
our settings class:</p>
<pre><code class="language-csharp">builder.Services
    .AddOptions&lt;GitHubSettings&gt;()
    .BindConfiguration(&quot;GitHubSettings&quot;)
    .ValidateDataAnnotations()
    .ValidateOnStart(); // 👈 the magic happens here
</code></pre>
<p>When we start the application, the validation will run on <code>GitHubSettings</code>
and an exception is thrown if validation fails. The validation exception
will look something like this:</p>
<pre><code class="language-yaml">Unhandled exception. Microsoft.Extensions.Options.OptionsValidationException:
  DataAnnotation validation failed for 'GitHubSettings' members:
</code></pre>
<p>This shortens the feedback loop, and you will know right away that you have
a problem. This is much better than finding out that you have a problem at
runtime, like in the previous examples.</p>
<h2>Closing Thoughts</h2>
<p>The <strong>Options pattern</strong> is very flexible and allows us to use strongly typed
settings in <a href="http://ASP.NET">ASP.NET</a> Core.</p>
<p>If you want to see how to implement the <a href="https://youtu.be/wxYt0motww0"><strong>Options pattern</strong></a>,
I made a <a href="https://youtu.be/wxYt0motww0"><strong>video about it where I go into the details.</strong></a>
I covered the differences between <code>IOptions</code>, <code>IOptionsSnapshot</code> and <code>IOptionsMonitor</code>.</p>
<p>And now you know how to use the <code>ValidateOnStart</code> method, which was introduced
in <strong>.NET 6</strong>, to validate your application settings on app start up. This allows
you to learn about configuration issues as soon as possible, instead of at runtime.</p>
<p>I also made a video showing how to add <a href="https://youtu.be/qRruEdjNVNE">validation to the Option pattern</a>.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_019.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How To Be a Better Software Engineer In 2023]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-be-a-better-software-engineer-in-2023</link>
            <guid isPermaLink="false">how-to-be-a-better-software-engineer-in-2023</guid>
            <pubDate>Sat, 31 Dec 2022 00:00:00 GMT</pubDate>
            <description><![CDATA[In this week's newsletter I will share 5 simple tips on how you can be a better software engineer in 2023. I find it a little amusing that the last newsletter of the year and is also coming out on the last day of the year. Here are 5 tips for being a better software engineer in 2023.]]></description>
            <content:encoded><![CDATA[<p>In this week's newsletter I will share 5 simple tips on how
you can be a better software engineer in 2023.</p>
<p>I find it a little amusing that the last newsletter of the year
and is also coming out on the last day of the year.</p>
<p>Here are 5 tips for being a better software engineer in 2023:</p>
<ul>
<li><a href="#1-keep-learning-and-acquiring-new-skills">Keep learning and acquiring new skills</a></li>
<li><a href="#2-invest-in-code-quality">Invest in code quality</a></li>
<li><a href="#3-work-on-complex-systems">Work on complex systems</a></li>
<li><a href="#4-be-comfortable-in-the-cloud">Be comfortable in the cloud</a></li>
<li><a href="#5-take-care-of-yourself">Take care of yourself</a></li>
</ul>
<p>I'm confident that you will become a better software engineer if you apply one of these, but I challenge you to work on all of them in the coming year.</p>
<p>Let's dive in.</p>
<h2>1. Keep Learning And Acquiring New Skills</h2>
<p>The field of software engineering is constantly evolving, so it's
important to stay up-to-date with <a href="https://youtu.be/dDasAmowFts"><strong>new technologies</strong></a>
and best practices.</p>
<p>We have a new .NET release every year, and it's easy to get lost
with the latest news. How I stay up to date is through online courses,
attending conferences and meetups, or working on <a href="https://youtu.be/Ru6_b50wdfo"><strong>personal projects</strong></a>.</p>
<p>Personal projects transition best to my day job, and I don't shy away
from exploring new topics. I shared many of my personal projects on <a href="https://github.com/m-jovanovic"><strong>GitHub</strong></a>,
maybe you can find some inspiration over there.</p>
<h2>2. Invest In Code Quality</h2>
<p>In addition to writing clean and readable code, it's important to also
consider the overall quality of your code.</p>
<p>This includes things like performance, security, and maintainability.
By writing high-quality code, you'll be able to build systems that are
more reliable, scalable, and easier to maintain over time.</p>
<p>One way to write clean and quality code is <a href="https://youtu.be/0nVT1gM4vPg"><strong>with the help of static code analysis</strong></a>.</p>
<p>Investing in code quality will pay dividends in the later stages of any project.</p>
<h2>3. Work On Complex Systems</h2>
<p>If you aspire to be senior engineer or software architect, you have to
work on complex systems. You need to be in a position to solve the toughest problems.</p>
<p>What do I consider to be a <a href="https://youtu.be/Ru6_b50wdfo"><strong>complex system</strong></a>?</p>
<p>That's difficult to say, but here are some rough guidelines:</p>
<ul>
<li>Microservices</li>
<li>Event-driven systems</li>
<li>High performance systems</li>
</ul>
<p>Of course, you don't need a <a href="https://youtu.be/Ru6_b50wdfo"><strong>fancy architecture</strong></a> to work in a complex system.</p>
<p>If you are working in a business domain with many domain rules,
I consider that a complex system also.</p>
<p>You should strive to always be in a position to work on a challenging project,
this will help you grow.</p>
<h2>4. Be Comfortable In The Cloud</h2>
<p>The cloud is here to stay, and you should be familiar with at least one
of the major cloud providers:</p>
<ul>
<li>Microsoft Azure</li>
<li>Amazon Web Services</li>
<li>Google Cloud Platform</li>
</ul>
<p>Most of them give you free credits to get started and explore the services they offer.</p>
<p>I stayed away from <a href="https://youtu.be/QP0pi7xe24s"><strong>cloud development</strong></a> for too long in my career, and now I wish I started sooner.</p>
<p>You want to be <a href="https://youtu.be/QP0pi7xe24s"><strong>comfortable in the cloud</strong></a> to be a better software engineer.</p>
<h2>5. Take Care Of Yourself</h2>
<p>Being a software engineer is mentally and physically demanding, and it's
important to take care of yourself in order to maintain a healthy work-life balance.</p>
<p>This includes getting enough sleep, eating well, and taking breaks when needed.</p>
<p>Taking care of yourself can also help you stay focused and productive in your work.</p>
<p>I like to take short breaks every hour or so, and not think about work for a few minutes.
This helps me replenish my energy, and allows me to continue to operate on a high level.</p>
<h2>What I'm Doing To Make 2023 Amazing</h2>
<p>What I'm Doing To Make 2023 Amazing
Making New Year's resolutions is popular at the start of the year. The problem is most people get too excited about it, but then proceed to not accomplish anything in a few months.</p>
<p>I've found the simplest rule to always make progress is taking action. I try to make a small step forward every day. And when I look back on the year, I realize I made a lot of progress.</p>
<p>Again, my 5 tips so you can be a better software engineer in 2023 are:</p>
<ul>
<li><a href="#1-keep-learning-and-acquiring-new-skills">Keep learning and acquiring new skills</a></li>
<li><a href="#2-invest-in-code-quality">Invest in code quality</a></li>
<li><a href="#3-work-on-complex-systems">Work on complex systems</a></li>
<li><a href="#4-be-comfortable-in-the-cloud">Be comfortable in the cloud</a></li>
<li><a href="#5-take-care-of-yourself">Take care of yourself</a></li>
</ul>
<p>Where I failed most in 2022 was taking care of myself, and this will be one of my main improvement points.</p>
<p>I also want to learn many new things, so I can share them with you in this newsletter and on my social media channels.</p>
<p>I wish you a very happy and prosperous New Year.
Write lots of code, squash many bugs,
and may you have green unit tests year round.</p>
<p>Stay awesome! 🎁<br>
Milan</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_018.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Clean Architecture And The Benefits Of Structured Software Design]]></title>
            <link>https://milanjovanovic.tech/blog/clean-architecture-and-the-benefits-of-structured-software-design</link>
            <guid isPermaLink="false">clean-architecture-and-the-benefits-of-structured-software-design</guid>
            <pubDate>Sat, 24 Dec 2022 00:00:00 GMT</pubDate>
            <description><![CDATA[In the world of software development, there are countless approaches and methodologies to choose from. It's easy to get swayed with the latest trends, and loose sight of architectural principles that really matter. One of the more popular ones is Clean Architecture, a design approach that prioritizes maintainability, scalability, flexibility, and productivity. In this week's newsletter, we will explore the key benefits of using Clean Architecture and how it can help your team build better software.]]></description>
            <content:encoded><![CDATA[<p>In the world of software development, there are countless approaches
and methodologies to choose from. It's easy to get swayed with the
latest trends, and loose sight of architectural principles that really matter.</p>
<p>One of the more popular ones is <strong>Clean Architecture</strong>, a design approach
that prioritizes maintainability, scalability, flexibility, and productivity.</p>
<p>In this week's newsletter, we will explore the key benefits of using
<strong>Clean Architecture</strong> and how it can help your team build better software.</p>
<p>Let's dive in.</p>
<h2>What Is Clean Architecture?</h2>
<p><strong>Clean Architecture</strong>, also known as &quot;The Onion Architecture,&quot; was first
introduced by Robert C. Martin (aka &quot;Uncle Bob&quot;) in his book
&quot;Clean Architecture: A Craftsman's Guide to Software Structure and Design&quot;.</p>
<p>At its core, <strong>Clean Architecture</strong> is a way of organizing a software system
in a way that separates the concerns of the various components,
making it easier to understand and maintain.</p>
<p>In <strong>Clean Architecture</strong>, the core of the system is the <strong>&quot;inner circle&quot;</strong>,
which contains the business rules and logic.</p>
<p>Surrounding this <strong>inner circle</strong> are layers of abstraction,
each one representing a different concern.</p>
<p><img src="/blogs/mnw_017/clean_architecture.png" alt=""></p>
<p>The typical outer layers are <strong>Infrastructure</strong> and <strong>Presentation</strong> layers.
The <strong>Infrastructure</strong> layer handles external concerns such as APIs and databases.
While the <strong>Presentation</strong> layer exposes an interface for clients to interact with
the application.</p>
<p>The key principle of <strong>Clean Architecture</strong> is that the <strong>inner circle</strong>
should not depend on the outer layers. Instead, the outer layers should
depend on the <strong>inner circle</strong>. This helps to ensure that the <strong>core</strong> of the
system is flexible and easy to modify, without worrying about the impact
on other parts of the system.</p>
<h2>Benefits Of Using Clean Architecture</h2>
<p>I want to highlight some of the key benefits of using <strong>Clean Architecture</strong>.</p>
<h3>Improved Maintainability</h3>
<p>One of the primary benefits of using <strong>Clean Architecture</strong> is improved <strong>maintainability</strong>.
By separating the concerns of the various components and enforcing the dependency rule,
it becomes much easier to understand and modify the code.
Depending on abstractions allows you to design your business logic in a flexible way,
without having to know the implementation details.</p>
<h3>Modularity and Separation of Concerns</h3>
<p><strong>Clean Architecture</strong> helps to create a clear <strong>separation of concerns</strong>
within the codebase. Each layer has a specific purpose and is decoupled
from the others, making it easier to understand and modify individual
components without affecting the rest of the system. This modularity
also makes it easier to reuse components in other projects.</p>
<h3>Testability</h3>
<p><strong>Clean Architecture</strong> also makes it <strong>easier to test</strong> and debug the code.
Because the inner circle is independent of the outer layers,
it's easier to write unit tests that focus specifically on the business
rules. This can help to catch errors early on in the development
process and reduce the overall testing effort.</p>
<h3>Loose Coupling of Components</h3>
<p><strong>Clean Architecture</strong> also promotes <strong>loose coupling</strong> between the various
components of the system. This means that it's easier to swap out
external dependencies or make other modifications without affecting
the core business logic. This can be especially useful when it comes
to upgrading or replacing technology.</p>
<h3>Increased Flexibility</h3>
<p>Another key benefit of <strong>Clean Architecture</strong> is increased <strong>flexibility</strong>.
By separating the concerns of the various components, it's easier
to modify and adapt the code to changing requirements. This can be
especially useful in fast-paced environments where requirements
are constantly evolving.</p>
<h3>Improved Team Productivity</h3>
<p><strong>Clean Architecture</strong> can help to improve team <strong>productivity</strong>.
By establishing clear separation of responsibilities and well-defined
boundaries, it's easier for team members to understand their roles
and responsibilities. This can improve communication and collaboration,
leading to more efficient and effective work.</p>
<h2>Clean Architecture In The Real World</h2>
<p>This all sounds nice in theory, but how does <strong>Clean Architecture</strong>
perform in the real world?</p>
<p>I have used <strong>Clean Architecture</strong> on roughly 10 projects in the
last 5 years, and I've had a lot of success with it. It was
easy to add new features, and scale the applications when
necessary. <strong>Clean Architecture</strong> can easily be broken down
into multiple modules or services, if performance is suffering
and there is a need to scale out.</p>
<p>One problem with the <strong>Clean Architecture</strong> is that it is <strong>easy to overengineer</strong>.</p>
<p>Dogmatism is a real issue, as I see many people with strong
opinions of what <strong>Clean Architecture</strong> should be.
I've been guilty of this myself in the past.</p>
<p>Recently, I try to be more <strong>pragmatic</strong> when using <strong>Clean Architecture</strong>.
I apply what I like, and give myself the flexibility of <strong>&quot;breaking&quot;</strong>
<strong>Clean Architecture</strong> if I think it will simplify things in the long run.</p>
<h2>Closing Thoughts</h2>
<p>By following the principles of <strong>Clean Architecture</strong>, you can create a flexible
and maintainable codebase that is well-suited to evolving requirements and technology.</p>
<p>However, it's important to be <strong>pragmatic</strong> with <strong>Clean Architecture</strong> and
allow yourself to be flexible in the design, in order to simplify things in the long run.</p>
<p>If you want to see how to apply <strong>Clean Architecture</strong> in practice,
I have a
<a href="https://youtu.be/tLk4pZZtiDY?list=PLYpjLpq5ZDGstQ5afRz-34o_0dexr1RGa"><strong>playlist with more than 20 videos on Clean Architecture</strong></a>.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_017.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Fast Document Database In .NET With Marten]]></title>
            <link>https://milanjovanovic.tech/blog/fast-document-database-in-net-with-marten</link>
            <guid isPermaLink="false">fast-document-database-in-net-with-marten</guid>
            <pubDate>Sat, 17 Dec 2022 00:00:00 GMT</pubDate>
            <description><![CDATA[Did you know you can turn PostgreSQL into a fully-fledged Document database? Marten is a .NET library that allows developers to use the PostgreSQL database as both a document database and a fully-featured event store. You don't need to install anything else to be able to use PostgreSQL as a document database, outside of the Nuget pacakge. Marten relies on the JSONB support available since PostgreSQL 9.4. In this week's newsletter, I want to introduce you to the basics of working with Marten and show you how easy it is to get started.]]></description>
            <content:encoded><![CDATA[<p>Did you know you can turn <strong>PostgreSQL</strong> into a fully-fledged <strong>Document database</strong>?</p>
<p><strong>Marten</strong> is a <strong>.NET</strong> library that allows developers to use the <strong>PostgreSQL</strong>
database as both a <strong>document database</strong> and a fully-featured <strong>event store</strong>.</p>
<p>You don't need to install anything else to be able to use <strong>PostgreSQL</strong>
as a <strong>document database</strong>, outside of the Nuget package. <strong>Marten</strong> relies
on the <strong>JSONB</strong> support available since <strong>PostgreSQL</strong> 9.4.</p>
<p>In this week's newsletter, I want to introduce you to the basics of working
with <strong>Marten</strong> and show you how easy it is to get started.</p>
<p>Let's dive in.</p>
<h2>Installing And Configuring Marten</h2>
<p>What are you going to need to start using <strong>PostgreSQL</strong> as a <strong>Document datbase</strong>?</p>
<p>Other than a running instance of <strong>PostgreSQL</strong>, of course, you will need
to install the <strong>Marten</strong> Nuget package:</p>
<pre><code class="language-csharp">dotnet add package Marten
</code></pre>
<p><strong>Marten</strong> can build the required database schema and necessary tables on the fly,
and I suggest using this approach in development.
For a production environment, you definitely want to apply schema
changes on your own with migration scripts.</p>
<p>To register <strong>Marten</strong> with dependency injection, you need to call the <code>AddMarten</code>
method.</p>
<p>Here's an example <strong>Marten</strong> configuration inside of a <strong>.NET 7</strong> application:</p>
<pre><code class="language-csharp">builder.Services.AddMarten(options =&gt;
{
    options.Connection(builder.Configuration.GetConnectionString(&quot;Marten&quot;));
});
</code></pre>
<p>This will register a few services with dependency injection:</p>
<ul>
<li><code>IDocumentStore</code> - used to create sessions, generate schema migrations, and do bulk inserts</li>
<li><code>IDocumentSession</code> - used for read and write operations</li>
<li><code>IQuerySession</code> - used for read operations</li>
</ul>
<p>Let's see how we can work with documents using <strong>Marten</strong>.</p>
<h2>Storing Documents With Marten</h2>
<p>Storing documents in the database is very straightforward. You need to create
a new <code>DocumentStore</code> instance, and open an <code>IDocumentSession</code> which exposes
methods for storing and persisting documents.</p>
<p>Let's see how we can store a <code>Product</code> document:</p>
<pre><code class="language-csharp">var store = DocumentStore.For(&quot;Connection string to PostgreSQL&quot;);

using var session = store.OpenSession();

var product = new Product
{
    Name = &quot;C# 11 and .NET 7 - Modern Cross-Platform Fundamentals&quot;,
    Price = 46.87
};

session.Store(product);

await session.SaveChangesAsync();
</code></pre>
<p>We're creating a new <code>DocumentStore</code> instance which we use to open
a session to <strong>PostgreSQL</strong>. And then we just call <code>Store</code> and pass in
the <code>Product</code> instance. Note that <strong>Marten</strong> will populate the
<code>Product.Id</code> at this point. <strong>Marten</strong> can populate keys of <code>Guid</code>,
<code>int</code>, <code>long</code>, and other data types. It uses the HiLo algorithm
for numeric keys. Finally, when we call <code>SaveChangesAsync</code> the
<code>Product</code> is serialized into <strong>JSON</strong> and persisted as a document.</p>
<p>An important thing to be aware of is that the <code>IDocumentSession</code>
created by <code>OpenSession</code> doesn't track changes on the entities automatically.
You need to create a session with dirty checking enabled by
calling <code>DirtyTrackedSession</code> on the <code>DocumentStore</code> to enable
automatic change detection.</p>
<h2>Querying Documents With Marten</h2>
<p><strong>Marten</strong> has rich support for querying documents in the database.
You can write and execute queries using <strong>LINQ</strong>, which you are
familiar with if you worked with <strong>EF Core</strong>. And you can also
write and execute <strong>SQL</strong> queries, because it's still a <strong>PostgreSQL</strong>
database underneath.</p>
<p>Here's an example query to return products that have a higher price
than the one which is specified:</p>
<pre><code class="language-csharp">var store = DocumentStore.For(&quot;Connection string to PostgreSQL&quot;);

using var session = store.QuerySession();

var products = session.Query&lt;Product&gt;().Where(p =&gt; p.Price &gt; 9.99).ToList();
</code></pre>
<p><strong>Marten</strong> also has support for:</p>
<ul>
<li>Including related documents</li>
<li>Batched queries</li>
<li>Paging</li>
<li>Full text search</li>
</ul>
<h2>Advanced Options With Marten</h2>
<p><strong>Marten</strong> can utilize the full capabilities <strong>PostgreSQL</strong> has to offer,
notably transactions and indexing. <strong>Marten</strong> sessions are transactional
by default, either all of the documents are persisted together or
none of them are. And you can configure indexes on your documents
for faster queries.</p>
<p><strong>Marten</strong> isn't just a <strong>document database</strong> on top of <strong>PostgreSQL</strong>!</p>
<p>You have fully-fledged support for <strong>event sourcing</strong> with <strong>Marten</strong>,
as well as projections. This makes it a perfect choice for
implementing <strong>CQRS</strong>. But this is a topic for a separate newsletter.</p>
<h2>Closing Thoughts</h2>
<p>I'm absolutely amazed with <a href="https://martendb.io/">Marten</a> and what it has to offer.
And <strong>PostgreSQL</strong> is also my favorite database, so it's like a match
made in heaven. I don't get too excited about learning new
technologies, but <strong>Marten</strong> has been an endless source
of joy this past week.</p>
<p>I still need to explore a few more topics before I can consider it
for production use:</p>
<ul>
<li>Schema migrations</li>
<li>Relationships and foreign keys</li>
<li>Advanced configuration options</li>
</ul>
<p>Considering that <strong>PostgreSQL</strong> is cheaper than most <strong>document databases</strong>,
I think using <strong>Marten</strong> is an interesting alternative. And if you are
familiar with <strong>SQL</strong> databases, you can still use all of that knowledge.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_016.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How To Structure Minimal APIs]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-structure-minimal-apis</link>
            <guid isPermaLink="false">how-to-structure-minimal-apis</guid>
            <pubDate>Sat, 10 Dec 2022 00:00:00 GMT</pubDate>
            <description><![CDATA[Minimal APIs were introduced to remove some of the ceremony of creating traditional APIs with controllers. To define an endpoint, you can use the new extension methods the were introduced such as MapGet to define a GET endpoint. I see one big issue with Minimal APIs, and that is the lack of clear guidance around how to structure applications built with Minimal APIs. In this newsletter, I want to offer a few solutions for that problem.]]></description>
            <content:encoded><![CDATA[<p>In this week's newsletter we are going to explore <strong>Minimal APIs</strong>,
which were introduced in <strong>.NET 6</strong>.</p>
<p><strong>Minimal APIs</strong> were introduced to remove some of the ceremony
of creating traditional APIs with controllers.
To define an endpoint, you can use the new extension
methods, such as <code>MapGet</code> to define a <strong>GET</strong> endpoint.</p>
<p>I see one big issue with <strong>Minimal APIs</strong>, and that is the lack
of clear guidance around how to structure applications
built with <strong>Minimal APIs</strong>.</p>
<p>In this newsletter, I want to offer a few solutions for that problem.</p>
<p>Let's dive in.</p>
<h2>How To Create Minimal APIs?</h2>
<p>Let's define a simple <strong>Minimal API</strong> application with two endpoints.
We're going to create one <code>GET</code> endpoint for getting a list of products.
And one <code>POST</code> endpoint for saving a product to the database.</p>
<p>We're using the powerful <strong>DI</strong> feature that allows us to inject services
as expression arguments, which you can see in the two expressions below
where we are injecting the <code>AppDbContext</code>.</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);

// Configure EF and other services...

var app = builder.Build();

app.MapGet(&quot;/products&quot;, async (AppDbContext dbContext) =&gt;
{
    return Results.Ok(await dbContext.Products.ToListAsync());
});

app.MapPost(&quot;/products&quot;, async (Product product, AppDbContext dbContext) =&gt;
{
    dbContext.Products.Add(product);

    await dbContext.SaveChangesAsync();

    return Results.Ok(product);
});

app.Run();
</code></pre>
<p>And with this we have a functioning <strong>Minimal API</strong> that we can develop further
as we continue to add more endpoints.</p>
<h2>The Problem With Maintaining Minimal APIs</h2>
<p>There is one potential problem with structuring our <strong>Minimal APIs</strong> like
in the previous example. If we keep adding the <strong>Minimal API</strong> endpoints
in the same file, our API will become hard to maintain as it grows
in complexity. How can we solve the maintance problem with <strong>Minimal APIs</strong>?</p>
<p>One solution can be to use extension methods to encapsulate the
definiton of the <strong>Minimal API</strong> endpoints.</p>
<p>Here's an example of that:</p>
<pre><code class="language-csharp">public static class ProductsModule
{
    public static void RegisterProductsEndpoints(this IEndpointRouteBuilder  endpoints)
    {
        endpoints.MapGet(&quot;/products&quot;, async (AppDbContext dbContext) =&gt;
        {
            return Results.Ok(await dbContext.Products.ToListAsync());
        });

        endpoints.MapPost(&quot;/products&quot;, async (Product product, AppDbContext dbContext) =&gt;
        {
            dbContext.Products.Add(product);

            await dbContext.SaveChangesAsync();

            return Results.Ok(product);
        });
    }
}
</code></pre>
<p>And then inside of <code>Program</code> we need to register the endpoints:</p>
<pre><code class="language-csharp">app.RegisterProductsEndpoints();
</code></pre>
<p>You can see that this simplifies our <strong>Minimal API</strong> definition,
and we also have our endpoints grouped by feature in one place.
I think this improve the maintainability of <strong>Minimal APIs</strong>,
but it comes at a cost. And that cost is having to define
extension methods for each group of endpoints you want to encapsulate,
and then you have to remember to call that extensions method in <code>Program</code>.</p>
<p>Can we do better?</p>
<h2>Structuring Minimal API Projects With Modules</h2>
<p>I want to introduce you to an interesting open source library
called <a href="https://github.com/CarterCommunity/Carter">Carter</a>,
which has a concept of modules that we can use to group endpoints.</p>
<p>Here's how we can define our <code>ProductsModule</code> with <strong>Carter</strong>:</p>
<pre><code class="language-csharp">public class ProductsModule : ICarterModule
{
    public void AddRoutes(IEndpointRouteBuilder app)
    {
        app.MapGet(&quot;/products&quot;, async (AppDbContext dbContext) =&gt;
        {
            return Results.Ok(await dbContext.Products.ToListAsync());
        });

        app.MapPost(&quot;/products&quot;, async (Product product, AppDbContext dbContext) =&gt;
        {
            dbContext.Products.Add(product);

            await dbContext.SaveChangesAsync();

            return Results.Ok(product);
        });
    }
}
</code></pre>
<p>This takes care of configuring our <strong>Minimal API</strong> endpoints, but we still
need to tell the framework to use these endpoints. We have to slightly
modify the <code>Program</code> to register the required <strong>Carter</strong> services by
calling <code>AddCarter</code>, and also map our endpoints by calling <code>MapCarter</code>.</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);

// Configure EF and other services...

builder.Services.AddCarter();

var app = builder.Build();

app.MapCarter();

app.Run();
</code></pre>
<p>When we want to define additional <strong>Minimal API</strong> endpoints we just need to
implement a new <code>ICarterModule</code>, and register our endpoints. <strong>Carter</strong> will
automatically take care of registering the new endpoints after that.</p>
<h2>Would I Use Minimal APIs In a Real Project?</h2>
<p>I think <strong>Minimal APIs</strong> have evolved nicely since they were first introduced
in <strong>.NET 6</strong>. I would be careful with using them in very large applications,
but I'm definitely going to explore options for using them on smaller projects.</p>
<p>A good use case can be for building a microservice that has a limited
number of endpoints.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_015.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Running Background Tasks In ASP.NET Core]]></title>
            <link>https://milanjovanovic.tech/blog/running-background-tasks-in-asp-net-core</link>
            <guid isPermaLink="false">running-background-tasks-in-asp-net-core</guid>
            <pubDate>Sat, 03 Dec 2022 00:00:00 GMT</pubDate>
            <description><![CDATA[In this week's newsletter we will talk about running background tasks in ASP .NET Core. After reading this newsletter, you will be able to set up a background task and have it up and running within minutes. Background tasks are used to offload some work in your application to the background, outside of the normal application flow. A typical example can be asynchronously processing messages from a queue. I will show you how to create a simple background task that runs once and completes. And you will also see how to configure a continuous background task, that repeats after a specific period.]]></description>
            <content:encoded><![CDATA[<p>In this week's newsletter we will talk about running <strong>background tasks</strong> in <strong><a href="http://ASP.NET">ASP.NET</a> Core</strong>.
After reading this newsletter, you will be able to set up a <strong>background task</strong>
and have it up and running within minutes.</p>
<p><strong>Background tasks</strong> are used to offload some work in your application to the background,
outside of the normal application flow. A typical example can be asynchronously
processing messages from a queue.</p>
<p>I will show you how to create a simple <strong>background task</strong> that runs once and completes.</p>
<p>And you will also see how to configure a continuous <strong>background task</strong>, that repeats after a specific period.</p>
<p>Let's dive in.</p>
<h2>Background Tasks With IHostedService</h2>
<p>You can define a <strong>background task</strong> by implementing the <code>IHostedService</code> interface.
It has only two methods.</p>
<p>Here's what the <code>IHostedService</code> interface looks like:</p>
<pre><code class="language-csharp">public interface IHostedService
{
    Task StartAsync(CancellationToken cancellationToken);

    Task StopAsync(CancellationToken cancellationToken);
}
</code></pre>
<p>All you have to do is implement the <code>StartAsync</code> and <code>StopAsync</code> methods.</p>
<p>Inside of <code>StartAsync</code> you would usually perform the background processing.
And inside of <code>StopAsync</code> you would perform any cleanup that is necessary,
such as disposing of resources.</p>
<p>To configure the <strong>background task</strong> you have to call the <code>AddHostedService</code> method:</p>
<pre><code class="language-csharp">builder.Services.AddHostedService&lt;MyBackgroundTask&gt;();
</code></pre>
<p>Calling <code>AddHostedService</code> will configure the <strong>background task</strong>
as a <strong>singleton</strong> service.</p>
<p>So does dependency injection still work in <code>IHostedService</code> implementations?<br>
Yes, but you can only inject <strong>transient</strong> or <strong>singleton</strong> services.</p>
<p>However, I don't like to implement the <code>IHostedService</code> interface myself.
I prefer using the <code>BackgroundService</code> class instead.</p>
<h2>Background Tasks With BackgroundService</h2>
<p>The <code>BackgroundService</code> class already implements the <code>IHostedService</code> interface,
and it has an <code>abstract</code> method that you need to override - <code>ExecuteAsync</code>.
When you are using the <code>BackgroundService</code> class, you only have to think about
the operation you want to implement.</p>
<p>Here's an example <strong>background task</strong> that runs <strong>EF</strong> migrations:</p>
<pre><code class="language-csharp">public class RunEfMigrationsBackgroundTask : BackgroundService
{
    private readonly IServiceProvider _serviceProvider;

    public RunEfMigrationsBackgroundTask(IServiceProvider serviceProvider)
    {
        _serviceProvider = serviceProvider;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        using IServiceScope scope = _serviceProvider.CreateScope();

        await using AppDbContext dbContext =
            scope.ServiceProvider.GetRequiredService&lt;AppDbContext&gt;();

        await dbContext.Database.MigrateAsync(stoppingToken);
    }
}
</code></pre>
<p>The <strong>EF</strong> <code>DbContext</code> is a <strong>scoped</strong> service, which we can't inject directly
inside of <code>RunEfMigrationsBackgroundTask</code>. We have to inject an instance of
<code>IServiceProvider</code> which we can use to create a custom service scope,
so that we can resolve the scoped <code>AppDbContext</code>.</p>
<p>I would <em>not recommend</em> running the <code>RunEfMigrationsBackgroundTask</code> in production.
<strong>EF</strong> migrations can easily fail and you'll run into problems.
However, I think it's perfectly fine for local development.</p>
<h2>Periodic Background Tasks</h2>
<p>Sometimes we want run a <strong>background task</strong> continuously, and have it
perform some operation on repeat. For example, we want consume messages
from a queue every ten seconds. How do we build this?</p>
<p>Here's an example <code>PeriodicBackgroundTask</code> to get you started:</p>
<pre><code class="language-csharp">public class PeriodicBackgroundTask : BackgroundService
{
    private readonly TimeSpan _period = TimeSpan.FromSeconds(5);
    private readonly ILogger&lt;PeriodicBackgroundTask&gt; _logger;

    public PeriodicBackgroundTask(ILogger&lt;PeriodicBackgroundTask&gt; logger)
    {
        _logger = logger;
    }

    protected override async Task ExecuteAsync(CancellationToken stoppingToken)
    {
        using PeriodicTimer timer = new PeriodicTimer(_period);

        while (!stoppingToken.IsCancellationRequested &amp;&amp;
               await timer.WaitForNextTickAsync(stoppingToken))
        {
            _logger.LogInformation(&quot;Executing PeriodicBackgroundTask&quot;);
        }
    }
}
</code></pre>
<p>We're using a <a href="https://learn.microsoft.com/en-us/dotnet/api/system.threading.periodictimer?view=net-6.0">PeriodicTimer</a>
to asynchronously wait for a given period, before executing our <strong>background task</strong>.</p>
<h2>What If You Need A More Robust Solution?</h2>
<p>It should be obvious by now that <code>IHostedService</code> is useful when you need
simple <strong>background tasks</strong> that are running while your application is running.</p>
<p>What if you want to have a scheduled <strong>background task</strong> that runs at 2AM every day?</p>
<p>You can probably build something like this yourself, but there are existing solutions
that you should consider first.</p>
<p>Here are two popular solutions for running <strong>background tasks</strong> that I worked with before:</p>
<ul>
<li><a href="https://www.quartz-scheduler.net/">Quartz</a></li>
<li><a href="https://www.hangfire.io/">Hangfire</a></li>
</ul>
<p>I also have an example of <a href="https://youtu.be/XALvnX7MPeo">using Quartz for processing Outbox messages</a>
on my YouTube channel that you can take a look at.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_014.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How To Use The New Bulk Update Feature In EF Core 7]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-use-the-new-bulk-update-feature-in-ef-core-7</link>
            <guid isPermaLink="false">how-to-use-the-new-bulk-update-feature-in-ef-core-7</guid>
            <pubDate>Sat, 26 Nov 2022 00:00:00 GMT</pubDate>
            <description><![CDATA[In this week's newsletter, we're going to explore the new ExecuteUpdate and ExecuteDelete methods that were released with EF7. ExecuteUpdate allows us to write a query and run a bulk update operation on the entities matching that query. Similarly, ExecuteDelete allows us to write a query and delete the entities matching that query. We can significantly improve performance using the new methods in some scenarios, and I'm going to show you what those scenarios are.]]></description>
            <content:encoded><![CDATA[<p>In this week's newsletter, we're going to explore the new
<code>ExecuteUpdate</code> and <code>ExecuteDelete</code> methods that were released with <strong>EF7</strong>.</p>
<p><code>ExecuteUpdate</code> allows us to write a query and run a bulk update
operation on the entities matching that query.</p>
<p>Similarly, <code>ExecuteDelete</code> allows us to write a query and delete
the entities matching that query.</p>
<p>We can significantly improve performance using the new methods in some scenarios, and I'm going to show you what those scenarios are.</p>
<p>Let's dive in.</p>
<h2>Updating And Deleting Entities Before EF Core 7</h2>
<p>If you want to update a collection of entities before <strong>EF7</strong>,
you need to load the entities into memory using the <code>DatabaseContext</code>.</p>
<p>The <strong>EF ChangeTracker</strong> will then track any changes made to these entities.
When you are ready to commit the changes to the database,
you simply call the <code>SaveChanges</code> method.</p>
<p>Here's an example where we load a few notifications,
and we want to snooze them so they aren't sent:</p>
<pre><code class="language-csharp">var notifications = dbContext
    .Notifications
    .Where(n =&gt; !n.Snoozed)
    .ToList();

foreach(var notification in notifications)
{
    notification.Snoozed = true;
}

dbContext.SaveChanges();
</code></pre>
<p><strong>EF7</strong> will generate the following <strong>SQL</strong> statement to update the records in the database:</p>
<pre><code class="language-sql">UPDATE [Notifications] n
SET n.[Snoozed] = TRUE
WHERE n.[Id] = @notificationId_1;

...

UPDATE [Notifications] n
SET n.[Snoozed] = TRUE
WHERE n.[Id] = @notificationId_N;
</code></pre>
<p>Notice that for every notification we end up with one <strong>SQL UPDATE</strong> statement.
This won't scale well as the number of notifications increases.</p>
<h2>Updating Entities With ExecuteUpdate</h2>
<p>With <strong>EF7</strong>, we now have access to the new <code>ExecuteUdpate</code> method.
It also has an asynchronous version - <code>ExecuteUpdateAsync</code>.</p>
<p>How do you use it?</p>
<p>You need to write a query that will select the records you want to update,
and then call the <code>ExecuteUpdate</code> method on the resulting <code>IQueryable</code>.</p>
<p>Let's rewrite the previous example using the new approach:</p>
<pre><code class="language-csharp">dbContext
    .Notifications
    .Where(n =&gt; !n.Snoozed)
    .ExecuteUpdate(s =&gt; s.SetProperty(
        n =&gt; n.Snoozed,
        n =&gt; true));
</code></pre>
<p>In the call to <code>ExecuteUpdate</code> we call the <code>SetProperty</code> method to specify
which properties we want to update, and what values we want to set.
The <code>SetProperty</code> method can be called multiple times, if you need to update more than one property.</p>
<p>In this case, <strong>EF7</strong> will generate the following <strong>SQL</strong> query:</p>
<pre><code class="language-sql">UPDATE n
SET n.[Snoozed] = TRUE
FROM [Notifications] AS n
WHERE n.[Snoozed] = FALSE;
</code></pre>
<p>Notice that this time we only have one <strong>SQL</strong> query being sent to the database.
This is a major performance improvement. It can be as much as 10x faster
than the old version, from my testing.</p>
<h2>Deleting Entities With ExecuteDelete</h2>
<p>Let's also see how we can do bulk deletes using the <code>ExecuteDelete</code> and <code>ExecuteDeleteAsync</code> methods.</p>
<p>Again, you have to write a query that will select the records you want to delete,
and then call the <code>ExecuteDelete</code> method on the resulting <code>IQueryable</code>.</p>
<p>If you want to delete all snoozed notifications:</p>
<pre><code class="language-csharp">dbContext
    .Notifications
    .Where(n =&gt; n.Snoozed)
    .ExecuteDelete();
</code></pre>
<p>And <strong>EF7</strong> will generate the following <strong>SQL</strong> query:</p>
<pre><code class="language-sql">DELETE FROM n
FROM [Notifications] AS n
WHERE n.[Snoozed] = TRUE;
</code></pre>
<p>I think this will be incredibly useful when you want to
delete records in the database based on a specific condition.</p>
<h2>Transactions, Change Tracking And Query Filters With Bulk Methods</h2>
<p>You need to be aware how transactions and change tracking
work with the new bulk methods. <code>ExecuteUpdate</code> and <code>ExecuteDelete</code> will
immediately go to database, and run the <strong>SQL</strong> query.</p>
<p><strong>What does this mean for transactions?</strong></p>
<p>If you want to run a bulk method together with other updates
applied with <code>SaveChanges</code>, by default they won't run in the same transaction.
You need to open an explicit <strong>transaction</strong> using the <code>DatabaseContext</code> to keep everything consistent.</p>
<p><strong>What does this mean for change tracking?</strong></p>
<p><code>ExecuteUpdate</code> and <code>ExecuteDelete</code> run directly on the database, without loading any entities into memory.
<strong>EF7</strong> will not track these entities in the <code>ChangeTracker</code>.</p>
<p>If you have any database interceptors defined, they won't execute
after calling one of the bulk update methods.
This also means that if you override <code>SaveChanges</code>
to add custom behavior, it won't be called.</p>
<p><strong>Do Query Filters still work?</strong></p>
<p>Yes, <strong>query filters</strong> will be <strong>correctly applied</strong> when calling <code>ExecuteUpdate</code> or <code>ExecuteDelete</code>.</p>
<h2>When Should You Use The New Bulk Methods?</h2>
<p>I think this is an excellent new addition to <strong>EF7</strong>,
and it solves a real problem when you need to run
a typical <strong>UPDATE</strong> or <strong>DELETE</strong> query with a
<strong>WHERE</strong> statement applied.</p>
<p>Previously, you had to write raw <strong>SQL</strong> and execute it using something like <strong>Dapper</strong>.</p>
<p>I will likely use this approach when it applies to my projects.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_013.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How To Use The Options Pattern In ASP.NET Core 7]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-use-the-options-pattern-in-asp-net-core-7</link>
            <guid isPermaLink="false">how-to-use-the-options-pattern-in-asp-net-core-7</guid>
            <pubDate>Sat, 19 Nov 2022 00:00:00 GMT</pubDate>
            <description><![CDATA[In this week's newsletter I want to show you how you can use the powerful Options pattern in ASP.NET Core 7. The options pattern uses classes to provide strongly typed settings in your application at runtime. The values for the options instance can come from multiple sources. The typical use case is to provide the settings from application configuration.]]></description>
            <content:encoded><![CDATA[<p>In this week's newsletter I want to show you how you can use
the powerful <strong>options pattern</strong> in <strong><a href="http://ASP.NET">ASP.NET</a> Core 7</strong>.</p>
<p>The <strong>options pattern</strong> uses classes to provide <strong>strongly typed settings</strong>
in your application at runtime.</p>
<p>The values for the <strong>options</strong> instance can come from multiple sources.
The typical use case is to provide the settings from application configuration.</p>
<p>You can configure the <strong>options pattern</strong> in a few different ways in <strong><a href="http://ASP.NET">ASP.NET</a> Core</strong>.
I want to discuss some of the approaches and their potential benefits.</p>
<p>Let's dive in.</p>
<h2>Creating The Options Class</h2>
<p>I want to set the stage first, by creating the <strong>options</strong> class and
explaining what settings we want to bind to it.</p>
<p>We want to configure JWT Authentication for our application,
so we decided to create the <code>JwtOptions</code> class to hold that configuration:</p>
<pre><code class="language-csharp">public class JwtOptions
{
    public string Issuer { get; init; }
    public string Audience { get; init; }
    public string SecretKey { get; init; }
}
</code></pre>
<p>And let's imagine that inside of our <code>appsettings.json</code> file
we have the following configuration values:</p>
<pre><code class="language-json">&quot;Jwt&quot;: {
    &quot;Issuer&quot;: &quot;Gatherly&quot;,
    &quot;Audience&quot;: &quot;Gatherly&quot;,
    &quot;SecretKey&quot;: &quot;dont-tell-anyone!&quot;
}
</code></pre>
<p>Alright, that's looking good. Now I want to show you a few ways to bind
the values from JSON to our <code>JwtOptions</code> class.</p>
<h2>Setting Up Options Pattern Using IConfiguration</h2>
<p>The most straightforward approach is to use the <code>IConfiguration</code> instance
that we can access while registering services.</p>
<p>We need to call the <code>IServiceCollection.Configure&lt;TOptions&gt;</code> method,
and specify the <code>JwtOptions</code> as the generic argument:</p>
<pre><code class="language-csharp">builder.Services.Configure&lt;JwtOptions&gt;(
    builder.Configuration.GetSection(&quot;Jwt&quot;));
</code></pre>
<p>It doesn't get simpler than this, does it?</p>
<p>The only downside is that we are limited to the configuration
values provided through application configuration.</p>
<p>This can be extended to include environment variables and user secrets also.</p>
<h2>Setting Up Options Pattern Using IConfigureOptions</h2>
<p>If you want a more robust approach, I have you covered.
We're going to use the <code>IConfigureOptions</code> interface to define a class
to configure our <strong>strongly typed options</strong>.</p>
<p>There are two steps that we need to follow in this case:</p>
<ul>
<li>Create the <code>IConfigureOptions</code> implementation</li>
<li>Call <code>IServiceCollection.ConfigureOptions&lt;TOptions&gt;</code> with our <code>IConfigureOptions</code>
implementation as the generic argument</li>
</ul>
<p>To start off, we will create the <code>JwtOptionsSetup</code> class:</p>
<pre><code class="language-csharp">public class JwtOptionsSetup : IConfigureOptions&lt;JwtOptions&gt;
{
    private const string SectionName = &quot;Jwt&quot;;
    private readonly IConfiguration _configuration;

    public JwtOptionsSetup(IConfiguration configuration)
    {
        _configuration = configuration;
    }

    public void Configure(JwtOptions options)
    {
        _configuration
            .GetSection(SectionName)
            .Bind(options);
    }
}
</code></pre>
<p>We wrote more code, to achieve the same thing. Was it worth it?</p>
<p>Perhaps, if you consider that we now have access to <strong>dependency injection</strong> in the <code>JwtOptionsSetup</code> class.
This means that we can resolve other services that we can use to get the configuration values.</p>
<p>We also need to tell the application to use the <code>JwtOptionsSetup</code> class:</p>
<pre><code class="language-csharp">builder.Services.ConfigureOptions&lt;JwtOptionsSetup&gt;();
</code></pre>
<p>When we try to inject our <code>JwtOptions</code> somewhere, the <code>JwtOptionsSetup.Configure</code>
method will be called first the calculate the correct values.</p>
<h2>Injecting Options With IOptions</h2>
<p>We've seen a few examples for how to configure the <strong>options pattern</strong> with the <code>JwtOptions</code> class.</p>
<p>But how do we actually use the <strong>options pattern</strong>?</p>
<p>Easy, you just need to inject <code>IOptions&lt;JwtOptions&gt;</code> from the constructor.</p>
<p>I'll just show the <code>JwtProvider</code> constructor here, for brevity.</p>
<pre><code class="language-csharp">public JwtProvider(IOptions&lt;JwtOptions&gt; options)
{
    _options = options.Value;
}
</code></pre>
<p>The actual <code>JwtOptions</code> instance is available on the <code>IOptions&lt;JwtOptions&gt;.Value</code> property.</p>
<p>The <code>IOptions</code> instance that we injected here is configured
as a <strong>Singleton</strong> in dependency injection. This is very important to be aware of.</p>
<h2>What About IOptionsSnapshot and IOptionsMonitor?</h2>
<p>If you want to use the latest configuration values every time
you inject an <strong>options</strong> class, then injecting <code>IOptions</code> won't work.</p>
<p>However, you can use the <code>IOptionsSnapshot</code> interface instead:</p>
<ul>
<li>It provides the latest configuration snapshot (cached per request)</li>
<li>It is registered as a <strong>Scoped</strong> service</li>
<li>It detects configuration changes after application start</li>
</ul>
<p>You can also use the <code>IOptionsMonitor</code> which retrieves the current
option values at any time, and it's a <strong>Singleton</strong> service.</p>
<h2>Wrapping up</h2>
<p>The <strong>options pattern</strong> gives us a way to use strongly typed configuration classes in our application.</p>
<p>We can configure the options class in a simple way with <a href="#setting-up-options-pattern-using-iconfiguration"><code>IConfiguration</code></a>,
or we can create an <a href="#setting-up-options-pattern-using-iconfigureoptions"><code>IConfigureOptions</code></a>
implementation if we need something more powerful.</p>
<p>When it comes to using the <strong>options pattern</strong>, we have three approaches:</p>
<ul>
<li><a href="https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.options.ioptions-1?view=dotnet-plat-ext-7.0"><code>IOptions</code></a></li>
<li><a href="https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.options.ioptionssnapshot-1?view=dotnet-plat-ext-7.0"><code>IOptionsSnapshot</code></a></li>
<li><a href="https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.options.ioptionsmonitor-1?view=dotnet-plat-ext-7.0"><code>IOptionsMonitor</code></a></li>
</ul>
<p>Deciding which of them to use in your application depends on what kind of behavior you want.
If you don't need to support refreshing configuration values
after application start, <a href="#injecting-options-with-ioptions"><code>IOptions</code></a> is a perfect solution.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_012.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[What's New In .NET 7?]]></title>
            <link>https://milanjovanovic.tech/blog/whats-new-in-dotnet-7</link>
            <guid isPermaLink="false">whats-new-in-dotnet-7</guid>
            <pubDate>Sat, 12 Nov 2022 00:00:00 GMT</pubDate>
            <description><![CDATA[In this week's newsletter I want to highlight a few interesting things that are now available with the release of C# 11 and .NET 7. In case you missed it, .NET 7 was released November 8th.]]></description>
            <content:encoded><![CDATA[<p>In this week's newsletter I want to highlight a few interesting things
that are now available with the release of <strong>C# 11</strong> and <strong>.NET 7</strong>.</p>
<p>In case you missed it, <strong>.NET 7</strong> was released November 8th.</p>
<p>There are many new features, and you can be sure I had a hard time choosing which ones to highlight.</p>
<p>Here's what we are going to cover:</p>
<ul>
<li><a href="#required-members">Required members</a></li>
<li><a href="#generic-attributes">Generic attributes</a></li>
<li><a href="#static-abstract-members-in-interfaces">Static abstract members in interfaces</a></li>
<li><a href="#file-keyword"><code>file</code> keyword</a></li>
<li><a href="#linq-order-and-orderdescending">LINQ Order and OrderDescending</a></li>
</ul>
<p>Let's see what the new features look like!</p>
<h2>Required Members</h2>
<p>We can now define a class member as required by using the <code>required</code> keyword.
It can be applied to a <em>field</em> or <em>property</em> and it tells the compiler
these members must be initialized by all constructors or by the object initializer.</p>
<p>Why is this useful?</p>
<p>Before <strong>C# 11</strong>, the only way to enforce a property being set was through a constructor.
If you used an object initializer you could bypass the constructor and not initialize some properties.</p>
<p>Here's how you can say that a property is required:</p>
<pre><code class="language-csharp">public class ContentCreator
{
    public required string Firstname { get; init; }
    public string? MiddleName { get; init; }
    public required string LastName { get; init; }
}
</code></pre>
<p>If you try to create a new <code>ContentCreator</code> instance without initializing
the <code>required</code> properties you get a compile error:</p>
<pre><code class="language-csharp">var creator = new ContentCreator
{
    FirstName = &quot;Milan&quot; // Error: No LastName
}
</code></pre>
<h2>Generic Attributes</h2>
<p>You can now declare a <em>generic</em> class whose base class is <code>Attribute</code>.</p>
<p>Before <strong>C# 11</strong>, if you wanted to pass in a type as a parameter
to an <code>Attribute</code> you would need to pass it through the constructor:</p>
<pre><code class="language-csharp">public class TypedAttribute : Attribute
{
    public TypedAttribute(Type t) =&gt; Param = t;

    public Type Param { get; }
}
</code></pre>
<p>And here's how you would use it with the <code>typeof</code> operator:</p>
<pre><code class="language-csharp">[TypedAttribute(typeof(int))]
public int Method() =&gt; default;
</code></pre>
<p>Using the generic attributes feature, you can now define it like this:</p>
<pre><code class="language-csharp">public class TypedAttribute&lt;T&gt; : Attribute { ... }
</code></pre>
<p>Now, we can specify the type parameter as a generic argument:</p>
<pre><code class="language-csharp">[TypedAttribute&lt;int&gt;()]
public int Method() =&gt; default;
</code></pre>
<h2>Static Abstract Members in Interfaces</h2>
<p>This is a very interesting feature that allows abstracting of static operations.
An example of this would be operators.</p>
<pre><code class="language-csharp">public interface IMonoid&lt;TSelf&gt; where TSelf : IMonoid&lt;TSelf&gt;
{
    public static abstract TSelf operator +(TSelf a, TSelf b);

    public static abstract TSelf Zero { get; }
}
</code></pre>
<p>How can we use the <code>IMonoid</code> interface?</p>
<p>It may be confusing at first, since the members are virtual
and there is no instance to call the virtual members on.
The solution is to use generics and let the compiler infer the rest.</p>
<p>Here's a simple example:</p>
<pre><code class="language-csharp">T AddAll&lt;T&gt;(params T[] elements) where T : IMonoid&lt;T&gt;
{
    T result = T.Zero;

    foreach (var element in elements)
    {
         result += element;
    }

    return result;
}
</code></pre>
<p>If you want to learn more, check out the docs on
<a href="https://learn.microsoft.com/en-us/dotnet/csharp/whats-new/tutorials/static-virtual-interface-members#static-abstract-interface-methods">static abstract interface methods</a>
and <a href="https://learn.microsoft.com/en-us/dotnet/csharp/whats-new/tutorials/static-virtual-interface-members#generic-math">generic math</a>.</p>
<h2>File Keyword</h2>
<p>With the new <code>file</code> keyword you can define a type whose scope and visibility
is restricted to the file in which it is declared.</p>
<pre><code class="language-csharp">file class HiddenClass
{
}
</code></pre>
<p>This feature is practical when used inside of source generators, to avoid collisions when naming generated types.</p>
<p>But you may be able to find a use for it in your application.</p>
<h2>LINQ Order and OrderDescending</h2>
<p>The new <code>Order</code> and <code>OrderDescending</code> methods allow us to sort an
<code>IEnumerable</code>, which simplifies the code for sorting.</p>
<p>Here's an example of ordering an array:</p>
<pre><code class="language-csharp">var array = new[] { 19, 91, 21 };

var arrayAsc = array.Order();

var arrayDesc = array.OrderDescending();
</code></pre>
<p>I want to highlight that <code>IQueryable</code> also supports the new methods.</p>
<h2>Will You Upgrade to .NET 7?</h2>
<p><strong>.NET 7</strong> is not an LTS (Long Term Support) release,
and will be in support until May 2024,
with <strong>.NET 8</strong> releasing in November 2023.</p>
<p>Here are a few reasons why you should consider upgrading:</p>
<ul>
<li>Major performance improvements</li>
<li>New features in <strong>.NET 7</strong></li>
<li>New features in <strong>EF Core 7</strong></li>
<li>Easier migration to <strong>.NET 8</strong></li>
</ul>
<p>I will be moving some of my new projects from <strong>.NET 6</strong> to <strong>.NET 7</strong>.</p>
<p>And I will also upgrade all of my YouTube content to <strong>.NET 7</strong>.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_011.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[5 Ways To Check For Duplicates In Collections, With Benchmarks]]></title>
            <link>https://milanjovanovic.tech/blog/5-ways-to-check-for-duplicates-in-collections</link>
            <guid isPermaLink="false">5-ways-to-check-for-duplicates-in-collections</guid>
            <pubDate>Sat, 05 Nov 2022 00:00:00 GMT</pubDate>
            <description><![CDATA[In this week's newsletter, we will take a look at five different ways to check if a collection contains duplicates. I'm going to explain the idea behind each algorithm, discuss the algorithm complexity (Big O Notation), and at the end, we'll look at some benchmark results.]]></description>
            <content:encoded><![CDATA[<p>In this week's newsletter, we will take a look at five different ways to check if a collection <strong>contains duplicates</strong>.</p>
<p>I'm going to explain the idea behind each <strong>algorithm</strong>, discuss the <strong>algorithm complexity</strong> (Big O Notation), and at the end, we'll look at some <strong>benchmark results</strong>.</p>
<p>The five approaches for finding a duplicate will use the:</p>
<ul>
<li><a href="#check-for-duplicates-with-foreach-loop"><code>foreach</code></a> loop</li>
<li>LINQ <a href="#check-for-duplicates-with-linq-any"><code>Any</code></a> method</li>
<li>LINQ <a href="#check-for-duplicates-with-linq-all"><code>All</code></a> method</li>
<li>LINQ <a href="#check-for-duplicates-with-linq-distinct"><code>Distinct</code></a> method</li>
<li>LINQ <a href="#check-for-duplicates-with-linq-tohashset"><code>ToHashSet</code></a> method</li>
</ul>
<p>Let's see how we can implement each approach!</p>
<h2>Check For Duplicates With ForEach Loop</h2>
<p>The first implementation will use the <code>foreach</code> loop and the <code>HashSet</code> data structure.</p>
<p>Here's the code for the <code>ContainsDuplicates</code> method:</p>
<pre><code class="language-csharp">public bool ContainsDuplicates&lt;T&gt;(IEnumerable&lt;T&gt; enumerable)
{
   HashSet&lt;T&gt; set = new();

   foreach(var element in enumerable)
   {
      if (!set.Add(element))
      {
         return true;
      }
   }

   return false;
}
</code></pre>
<p>The idea is simple:</p>
<ul>
<li>Loop through the collection</li>
<li>Add each element to the <code>HashSet</code></li>
<li>When <code>HashSet.Add</code> returns false we found a duplicate</li>
<li>If we loop through the entire collection there are no duplicates</li>
</ul>
<p>In terms of <strong>algorithm complexity</strong>, this would be <strong>O(n)</strong> or linear complexity.
This is because there's only one iteration through the collection.</p>
<p>Adding an element to a <code>HashSet</code> is a constant operation - <strong>O(1)</strong>.
So it doesn't affect the overall complexity.</p>
<h2>Check For Duplicates With LINQ Any</h2>
<p>We'll combine the idea from the previous implementation of
using the <code>HashSet</code> and pair it with the LINQ <code>Any</code>
method to iterate over the collection.</p>
<p>Here's the implementation for the <code>ContainsDuplicates</code> method:</p>
<pre><code class="language-csharp">public bool ContainsDuplicates&lt;T&gt;(IEnumerable&lt;T&gt; enumerable)
{
   HashSet&lt;T&gt; set = new();

   return enumerable.Any(element =&gt; !set.Add(element));
}
</code></pre>
<p>You can see this implementation is significantly shorter.
But it works the same as the one with the <code>foreach</code> loop.</p>
<p>If any element in the collection satisfies the specified expression,
<code>Any</code> will <em>short-circuit</em> and return <code>true</code>.
Otherwise, it will iterate over the entire collection and return <code>false</code>.</p>
<p>We're still looking at linear complexity here, <strong>O(n)</strong>.</p>
<h2>Check For Duplicates With LINQ All</h2>
<p>For our third implementation, we will use the opposite
of the LINQ <code>Any</code> method - the LINQ <code>All</code> method.</p>
<p>Here's the implementation with LINQ <code>All</code>:</p>
<pre><code class="language-csharp">public bool ContainsDuplicates&lt;T&gt;(IEnumerable&lt;T&gt; enumerable)
{
   HashSet&lt;T&gt; set = new();

   return !enumerable.All(set.Add);
}
</code></pre>
<p>The idea here is a little different than in the previous implementation.</p>
<p><code>All</code> will return <code>true</code> if all elements in a collection
satisfy the specified expression.</p>
<p>If at least one element doesn't satisfy the condition -
in our case when a <strong>duplicate</strong> is found - it will <em>short-circuit</em> and return <code>false</code>.</p>
<p>This is still linear complexity, <strong>O(n)</strong>.</p>
<h2>Check For Duplicates With LINQ Distinct</h2>
<p>So far, we've seen a few implementations using the <code>HashSet</code> data structure.
Now let's consider a different approach.</p>
<p>We can use the LINQ <code>Distinct</code> method to check for duplicates.</p>
<p>Here's the code for the <code>ContainsDuplicates</code> method:</p>
<pre><code class="language-csharp">public bool ContainsDuplicates&lt;T&gt;(IEnumerable&lt;T&gt; enumerable)
{
   return enumerable.Distinct().Count() != enumerable.Count();
}
</code></pre>
<p>The idea is first find the <code>Distinct</code> elements and <code>Count</code> them,
and then compare that to the number of all elements.</p>
<p>If the number of distinct elements is not equal to
the number of all elements, we have a <strong>duplicate</strong> value.</p>
<p>In terms of <strong>algorithm complexity</strong>, this is still linear complexity.</p>
<p>But we have at least two iterations through the collection
or three in the worst-case scenario.</p>
<p>We have one iteration for <code>Distinct</code> and one more
iteration for the call to <code>Count</code> right after that.
The last call to <code>Count</code> can return in constant time,
if the collection is an <code>array</code> or <code>List</code>.</p>
<h2>Check For Duplicates With LINQ ToHashSet</h2>
<p>For the last implementation we will use the LINQ <code>ToHashSet</code> method.</p>
<p>It takes a collection and creates a <code>HashSet</code> instance from it.</p>
<p>Here's what the <code>ContainsDuplicates</code> implementation looks like:</p>
<pre><code class="language-csharp">public bool ContainsDuplicates&lt;T&gt;(IEnumerable&lt;T&gt; enumerable)
{
   return enumerable.ToHashSet().Count != enumerable.Count();
}
</code></pre>
<p>We compare the number of elements in the <code>HashSet</code> to the number of elements in the collection.</p>
<p>If they are different, we have a <strong>duplicate</strong> value.</p>
<p>This is also linear complexity, <strong>O(n)</strong>.</p>
<h2>Benchmark Results</h2>
<p>Now that we've seen our implementations let's put them to the test.</p>
<p>I ran the benchmark for collections of varying sizes:</p>
<ul>
<li>100</li>
<li>1,000</li>
<li>10,000</li>
</ul>
<p>Each collection contains exactly one duplicate value located somewhere around the middle of the collection.</p>
<p>Here are the results:</p>
<p><img src="/blogs/mnw_010/benchmark.png" alt=""></p>
<p>The approach using the <code>foreach</code> loop comes out as the clear winner in terms of performance.</p>
<p>However, I would lean towards using the implementations with LINQ <code>Any</code> or <code>All</code> because of their simplicity.</p>
<p>You can find the <a href="https://github.com/m-jovanovic/find-duplicates-benchmark">source code for the benchmark</a>
on my GitHub. Feel free to submit a PR with a faster implementation if you can think of one.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_010.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How To Use Global Query Filters in EF Core]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-use-global-query-filters-in-ef-core</link>
            <guid isPermaLink="false">how-to-use-global-query-filters-in-ef-core</guid>
            <pubDate>Sat, 29 Oct 2022 00:00:00 GMT</pubDate>
            <description><![CDATA[In this week's newsletter, I'll show you how you can remove repetitive conditions in EF Core database queries. An example would be when you implement soft-delete, and have to check if a record was soft-deleted or not in every query. Also, it is practical if you're working in a multi-tenant system and need to specify a tenantId on every query. EF Core has a powerful feature called Query Filters that can help you remove repetitive conditions from your code.]]></description>
            <content:encoded><![CDATA[<p>In this week's newsletter, I'll show you how you can remove repetitive conditions in <strong>EF Core</strong> database queries.</p>
<p>Which kinds of queries fit this description?</p>
<p>An example would be when you implement <strong>soft-delete</strong>, and have to check if a record was <strong>soft-deleted</strong> or not in every query.</p>
<p>Also, it's practical if you're working in a multi-tenant system and need to specify a <code>tenantId</code> on every query.</p>
<p><strong>EF Core</strong> has a powerful feature that can help you remove repetitive conditions from your code.</p>
<p>I'm talking about <a href="https://learn.microsoft.com/en-us/ef/core/querying/filters">Query Filters</a>.</p>
<p>Let's see how we can implement it.</p>
<h2>How To Apply Query Filters</h2>
<p>Before introducing <strong>Query Filters</strong>, we will see how the standard approach looks.
We have an <code>Orders</code> table that supports <strong>soft-deleting</strong>.
And we never want to return <strong>soft-deleted</strong> orders.</p>
<p>We'll start with an <code>Order</code> entity that has an <code>IsDeleted</code> property.</p>
<pre><code class="language-csharp">public class Order
{
   public int Id { get; set; }
   public bool IsDeleted { get; set; }
}
</code></pre>
<p>And we have a business requirement that we can only query orders that are not deleted.</p>
<p>Here's what an <strong>EF</strong> query to get a single <code>Order</code> might look like:</p>
<pre><code class="language-csharp">dbContext
   .Orders
   .Where(order =&gt; !order.IsDeleted)
   .Where(order =&gt; order.Id == orderId)
   .FirstOrDefault();
</code></pre>
<p>This works perfectly for what we need to do.</p>
<p>However, we need to remember to apply this condition every time we want to query the <code>Order</code> entity.</p>
<p>Now, let's see how we can define a <strong>Query Filter</strong> on the <code>Order</code> entity to
apply this check when querying the database.</p>
<p>Inside of the <code>OnModelCreating</code> method on the database context, we need to
call the <code>HasQueryFilter</code> method and specify the expression we want:</p>
<pre><code class="language-csharp">modelBuilder
   .Entity&lt;Order&gt;()
   .HasQueryFilter(order =&gt; !order.IsDeleted);
</code></pre>
<p>Now we can omit the <strong>soft-delete</strong> check from the previous <strong>LINQ</strong> expression:</p>
<pre><code class="language-csharp">dbContext
   .Orders
   .Where(order =&gt; order.Id == orderId)
   .FirstOrDefault();
</code></pre>
<p>And this is the <strong>SQL</strong> that <strong>EF</strong> will generate with the <strong>Query Filter</strong>:</p>
<pre><code class="language-sql">SELECT o.*
FROM Orders o
WHERE o.IsDeleted = FALSE AND o.Id = @orderId
</code></pre>
<h2>Disabling Query Filters</h2>
<p>You may run into a situation where you need to disable <strong>Query Filters</strong> for a specific query.
Luckily, there is an easy way to do this.</p>
<p>In your <strong>LINQ</strong> expression, you need to call the <code>IgnoreQueryFilters</code> method,
and all the <strong>Query Filters</strong> configured for this entity will be disabled:</p>
<pre><code class="language-csharp">dbContext
   .Orders
   .IgnoreQueryFilters()
   .Where(order =&gt; order.Id == orderId)
   .FirstOrDefault();
</code></pre>
<p>Be careful when doing this, as you can easily introduce unwanted behavior in your application.</p>
<h2>Good Things To Know Before Using Query Filters</h2>
<p>Here are a few more details that you should know about <strong>Query Filters</strong> before using them.
Hopefully, this will save you some trouble if you decide to use them in your application.</p>
<p><strong>Configuring multiple Query Filters</strong></p>
<p>Configuring multiple <strong>Query Filters</strong> on the same entity will only apply the last one.
If you need more than one condition, you can do that with the logical <code>AND</code> operator (&amp;&amp;).</p>
<p><strong>Ignoring specific Query Filters</strong></p>
<p>If you need to ignore a specific expression in a <strong>Query Filter</strong> and leave the rest in place,
unfortunately, you can't do that. Only one <strong>Query Filter</strong> is allowed per entity type.</p>
<p>One solution is calling <code>IgnoreQueryFilters</code>, which will remove the configured <strong>Query Filter</strong>
for that entity type. And then manually apply the condition that you need for that specific query.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_009.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Introduction To Locking And Concurrency Control in .NET 6]]></title>
            <link>https://milanjovanovic.tech/blog/introduction-to-locking-and-concurrency-control-in-dotnet-6</link>
            <guid isPermaLink="false">introduction-to-locking-and-concurrency-control-in-dotnet-6</guid>
            <pubDate>Sat, 22 Oct 2022 00:00:00 GMT</pubDate>
            <description><![CDATA[In this week's newsletter, we'll see how we can work with locking in .NET 6. We won't talk about how the lock is actually implemented at the operating system level. We will focus on application-level locking mechanisms instead. Locking allows us to control how many threads can access some piece of code. Why would you want to do this?]]></description>
            <content:encoded><![CDATA[<p>In this week's newsletter, we'll see how we can work with <strong>locking</strong> in <strong>.NET 6</strong>.</p>
<p>We won't talk about how the lock is actually implemented at the operating system level.
Instead, I will focus on application-level <strong>locking</strong> mechanisms.</p>
<p><strong>Locking</strong> allows us to control how many <strong>threads</strong> can access some piece of code.
Why would you want to do this?</p>
<p>Usually because you want to protect access to <strong>expensive resources</strong>,
and you need the <strong>concurrency control</strong> that locking enables.</p>
<p>We will use a simple <code>BankAccount</code> class with a <code>Deposit</code> method
to illustrate how to implement locking.</p>
<h2>The C# Lock Statement</h2>
<p>The C# language supports locking with the <a href="https://learn.microsoft.com/en-us/dotnet/csharp/language-reference/statements/lock"><code>lock</code> statement</a>.
You can use the <code>lock</code> statement to define a code block that only one thread can access.</p>
<p>The <code>lock</code> statement acquires a mutual-exclusion lock (mutex) for a given object,
executes the statement block, and releases the lock.</p>
<pre><code class="language-csharp">lock(_lock)
{
   // Your code...
}
</code></pre>
<p>Here <code>_lock</code> is a reference type, usually an <code>object</code> instance.</p>
<p>Let's see how we can implement the <code>BankAccount</code> class using the <code>lock</code> statement:</p>
<pre><code class="language-csharp">public class BankAccount
{
   private static readonly object _lock = new();
   private decimal _balance;

   public void Deposit(decimal amount)
   {
      lock(_lock)
      {
         _balance += amount;
      }
   }
}
</code></pre>
<p>The first thread to reach and execute the <code>lock</code> statement will be allowed to update
the <code>_balance</code>. Any other threads will block until the lock is released.</p>
<h2>Locking With Semaphore</h2>
<p>The <a href="https://learn.microsoft.com/en-us/dotnet/api/system.threading.semaphore?view=net-6.0"><code>Semaphore</code></a>
class is another option we can use to achieve the same effect.</p>
<p>We'll use the <code>Semaphore</code> constructor to set the <code>initialCount</code> to 1,
which means that the <code>Semaphore</code> is open at the start.
And we will also set the <code>maximumCount</code> to 1,
which means that only one thread is allowed to enter the <code>Semaphore</code>.</p>
<p>Let's see how we can implement the <code>BankAccount</code> class using the <code>Semaphore</code>:</p>
<pre><code class="language-csharp">public class BankAccount
{
   private static readonly Semaphore _semaphore = new(
      initialCount: 1,
      maximumCount: 1);

   private decimal _balance;

   public void Deposit(decimal amount)
   {
      _semaphore.WaitOne();

      _balance += amount;

      _semaphore.Release();
   }
}
</code></pre>
<p>To enter the <code>Semaphore</code>, we have to call the <code>WaitOne</code> method.</p>
<p>If no thread was previously inside, our thread is allowed
to enter the <code>Semaphore</code> and update the balance.</p>
<p>After updating the balance, we call the <code>Release</code> method to
release the <code>Semaphore</code> for other threads that might be waiting.</p>
<h2>Asynchronous Locking With SemaphoreSlim</h2>
<p>What if we wanted to call an asynchronous method in a locked context?</p>
<p>We can't use the <code>lock</code> statement as it doesn't support asynchronous calls.
Awaiting an asynchronous call inside a <code>lock</code> statement will cause a compilation error.</p>
<p>The <code>Semaphore</code> class can solve this problem.</p>
<p>But I want to show you another option that we have, <a href="https://learn.microsoft.com/en-us/dotnet/api/system.threading.semaphoreslim?view=net-6.0"><code>SemaphoreSlim</code></a>.
It's a lightweight alternative to the <code>Semaphore</code> class and has <code>async</code> methods.</p>
<p>Let's see how we can implement the <code>BankAccount</code> class using <code>SemaphoreSlim</code>:</p>
<pre><code class="language-csharp">public class BankAccount
{
   private static readonly SemaphoreSlim _semaphore = new(
      initialCount: 1,
      maximumCount: 1);

   private decimal _balance;

   public async Task Deposit(decimal amount)
   {
      await _semaphore.WaitAsync();

      _balance += amount;

      _semaphore.Release();
   }
}
</code></pre>
<p>Notice that I updated the <code>Deposit</code> method to return a <code>Task</code>.</p>
<p>This time, we're calling <code>WaitAsync</code> to block the current
thread until it can enter the semaphore.</p>
<p>After updating the balance, we call the <code>Release</code> method
to release the <code>SemaphoreSlim</code> like in the previous example.</p>
<h2>Are There Other Options For Locking in .NET?</h2>
<p>So far I mentioned three options to implement locking:</p>
<ul>
<li><a href="#the-c-lock-statement"><code>lock</code> statement</a></li>
<li><a href="#locking-with-semaphore"><code>Semaphore</code></a></li>
<li><a href="#asynchronous-locking-with-semaphoreslim"><code>SemaphoreSlim</code></a></li>
</ul>
<p>However, <strong>.NET</strong> has other classes for <strong>concurrency control</strong> that you can
explore like
<a href="https://learn.microsoft.com/en-us/dotnet/api/system.threading.monitor?view=net-6.0"><code>Monitor</code></a>,
<a href="https://learn.microsoft.com/en-us/dotnet/api/system.threading.mutex?view=net-6.0"><code>Mutex</code></a>,
<a href="https://learn.microsoft.com/en-us/dotnet/api/system.threading.readerwriterlock?view=net-6.0"><code>ReaderWriterLock</code></a>
and many more.</p>
<p>I hope you enjoyed this brief introduction to a very complex topic.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_008.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How I Optimized an API Endpoint to Make It 15x Faster]]></title>
            <link>https://milanjovanovic.tech/blog/how-i-optimized-an-api-endpoint-to-make-it-15x-faster</link>
            <guid isPermaLink="false">how-i-optimized-an-api-endpoint-to-make-it-15x-faster</guid>
            <pubDate>Sat, 15 Oct 2022 00:00:00 GMT</pubDate>
            <description><![CDATA[Performance optimizations are my favorite thing about software engineering. Over the last 5 years, I've encountered various performance problems that taught me different ways to overcome them.]]></description>
            <content:encoded><![CDATA[<p>Performance optimization is my favorite thing about software engineering.
Over the last 5 years, I've encountered various performance problems
that taught me different ways to overcome them.</p>
<p>About a month ago, I ran into an issue with an API endpoint that wasn't scaling well.</p>
<p>This endpoint is used to calculate a report for an e-commerce web application.
It needed to talk to multiple modules (services) to gather all the necessary data,
combine it and perform the calculations.</p>
<p>I made a <a href="https://www.linkedin.com/feed/update/urn:li:activity:6966700329111310336/">post about it on LinkedIn</a>
that resonated with many people.</p>
<p><img src="/blogs/mnw_007/linkedin_post.png" alt=""></p>
<p>In this newsletter, I want to break down what I did to achieve a <strong>15x performance improvement</strong>.</p>
<h2>Focus On Bottlenecks First</h2>
<p>The first thing I do when I'm solving a performance problem
is determine where the slowest piece of the code is.
Fixing this part of the code will usually give the most significant improvement.</p>
<p>Solving one bottleneck can also reveal where the next bottleneck is.<br>
This is a continual process.</p>
<p>In my situation, there were a few bottlenecks:</p>
<ul>
<li>Calling the database from a loop</li>
<li>Calling an external service multiple times</li>
<li>Executing a complex calculation multiple times with identical parameters</li>
</ul>
<p>How can you measure performance?</p>
<p>A simple approach can be using <code>System.Timers.Timer</code> where you
manually log execution times between method calls.
Or you can use a performance profiler.</p>
<h2>Reduce The Number of Round Trips</h2>
<p>A round trip between your application and a database
(or some other service) can last 5-10ms, or more.
If you have many round trips in your flow, it's going to add up quickly.</p>
<p>Here are a few things you can do reduce the number of round trips:</p>
<ol>
<li>Don't call the database from a loop. This can usually be solved with a simple query like this:</li>
</ol>
<pre><code class="language-sql">   SELECT * FROM [TableName] WHERE Id IN (list_of_ids)
</code></pre>
<ol start="2">
<li>
<p>Use a query that returns multiple result sets from the database.
One library that supports this is <a href="https://github.com/DapperLib/Dapper">Dapper</a>, with the <code>QueryMultiple</code> method.</p>
</li>
<li>
<p>If you need to make multiple calls to another service, try to convert that into one call.
And in the service, aggregate the required data and return everything at once.</p>
</li>
</ol>
<h2>Parallelize External Calls</h2>
<p>I had a situation where I was awaiting multiple asynchronous calls from a few services.
These calls had no dependencies on each other, so I used a simple technique
to gain a significant performance improvement.</p>
<p>Let's say you're awaiting two tasks:</p>
<pre><code class="language-csharp">var task1Result = await CallService1Async();

var task2Result = await CallService2Async();

// Use the results.
</code></pre>
<p>A simple way to parallelize these calls is using the <code>Task.WhenAll</code> method:</p>
<pre><code class="language-csharp">var task1 = CallService1Async();

var task2 = CallService2Async();

await Task.WhenAll(task1, task2);

// Use the results.
task1.Result;
task2.Result;
</code></pre>
<p>Notice that I'm directly accessing the <code>Result</code> property on the tasks.
This can be <strong>detrimental</strong> if you're using it to block on an asynchronous call,
and can even lead to deadlocks.</p>
<p>However, in this situation it is perfectly safe to do,
because the two tasks will have completed after the call to <code>Task.WhenAll</code> completes.</p>
<p>Of course, whether or not these tasks will be executed in parallel
when calling <code>Task.WhenAll</code> depends on a few factors, which I won't cover here.</p>
<h2>Caching As a Last Resort</h2>
<p>I try to leave caching for the end, after I have exhausted
all other possibilities to improve performance.
While I love to use caching in general, I'm aware
it can introduce some unwanted behavior when data is stale.</p>
<p>You have to consider how long you can safely cache the data,
and how you are going to clear the cache if the underlying data changes.</p>
<p>In simple applications, I use <code>IMemoryCache</code>
that is available in <strong><a href="http://ASP.NET">ASP.NET</a> Core</strong> out of the box.
But you can also use an external cache like <a href="https://redis.io/">Redis</a>.</p>
<p>A good candidate for caching is data that is frequently accessed, but rarely modified.</p>
<h2>Closing Thoughts</h2>
<p>I think that for most Web applications,
performance optimization can be boiled down to the following approaches:</p>
<ul>
<li><a href="#focus-on-bottlenecks-first">Focus on bottlenecks first</a></li>
<li><a href="#reduce-the-number-of-round-trips">Reduce the number of round trips</a></li>
<li><a href="#parallelize-external-calls">Parallelize external calls</a></li>
<li><a href="#caching-as-a-last-resort">Caching</a></li>
</ul>
<p>I didn't talk about database optimization and indexes here,
but this should also be on your mind if the database is your bottleneck.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_007.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Decorator Pattern In ASP.NET Core]]></title>
            <link>https://milanjovanovic.tech/blog/decorator-pattern-in-asp-net-core</link>
            <guid isPermaLink="false">decorator-pattern-in-asp-net-core</guid>
            <pubDate>Sat, 08 Oct 2022 00:00:00 GMT</pubDate>
            <description><![CDATA[Let's imagine we have an existing Repository implementation, and we want to introduce caching to reduce the load on the database. How can we achieve this without changing anything about the Repository implementation? Decorator pattern is a structural design pattern that allows you to introduce new behavior to an existing class, without modifying the original class in any way. I'll show you how you can implement this with the ASP.NET Core DI container.]]></description>
            <content:encoded><![CDATA[<p>Let's imagine we have an existing <code>Repository</code> implementation, and we want to introduce caching to reduce the load on the database.</p>
<p>How can we achieve this without changing the original <code>Repository</code> implementation?</p>
<p><strong>Decorator pattern</strong> is a structural design pattern that allows you
to introduce new behavior to an existing class, without modifying the original class in any way.</p>
<p>I'll show you how you can implement this with the <strong><a href="http://ASP.NET">ASP.NET</a> Core DI</strong> container.</p>
<h2>How To Implement The Decorator Pattern</h2>
<p>We'll start with an existing <code>MemberRepository</code> implementation that implements the <code>IMemberRepository</code> interface.</p>
<p>It has only one method, which loads the <code>Member</code> from the database.</p>
<p>Here's what the implementation looks like:</p>
<pre><code class="language-csharp">public interface IMemberRepository
{
    Member GetById(int id);
}

public class MemberRepository : IMemberRepository
{
    private readonly DatabaseContext _dbContext;

    public MemberRepository(DatabaseContext dbContext)
    {
        _dbContext = dbContext;
    }

    public Member GetById(int id)
    {
        return _dbContext
            .Set&lt;Member&gt;()
            .First(member =&gt; member.Id == id);
    }
}
</code></pre>
<p>We want to introduce caching to the <code>MemberRepository</code> implementation without modifying the existing class.</p>
<p>To achieve this, we can use the <strong>Decorator pattern</strong> and create a wrapper around our <code>MemberRepository</code> implementation.</p>
<p>We can create a <code>CachingMemberRepository</code> that will have a dependency on <code>IMemberRepository</code>.</p>
<pre><code class="language-csharp">public class CachingMemberRepository : IMemberRepository
{
    private readonly IMemberRepository _repository;
    private readonly IMemoryCache _cache;

    public CachingMemberRepository(
        IMemberRepository repository,
        IMemoryCache cache)
    {
        _repository = repository;
        _cache = cache;
    }

    public Member GetById(int id)
    {
        string key = $&quot;members-{id}&quot;;

        return _cache.GetOrCreate(
            key,
            entry =&gt; {
                entry.SetAbsouluteExpiration(
                    TimeSpan.FromMinutes(5));

                return _repository.GetById(id);
            });
    }
}
</code></pre>
<p>Now I'm going to show you the power of <strong><a href="http://ASP.NET">ASP.NET</a> Core DI</strong>.</p>
<p>We will configure the <code>IMemberRepository</code> to resolve an instance of <code>CachingMemberRepository</code>,
while it will receive the <code>MemberRepository</code> instance as its dependency.</p>
<h2>Configuring The Decorator In ASP .NET Core DI</h2>
<p>For the DI container to be able to resolve <code>IMemberRepository</code> as <code>CachingMemberRepository</code>,
we need to manually configure the service.</p>
<p>We can use the overload that exposes a service provider,
that we will use to resolve the services required to construct a <code>MemberRepository</code>.</p>
<p>Here's what the configuration would look like:</p>
<pre><code class="language-csharp">services.AddScoped&lt;IMemberRepository&gt;(provider =&gt; {
    var context = provider.GetService&lt;DatabaseContext&gt;();
    var cache = provider.GetService&lt;IMemoryCache&gt;();

    return new CachingRepository(
         new MemberRepository(context),
         cache);
});
</code></pre>
<p>Now you can inject the <code>IMemberRepository</code>, and the DI will be able to resolve an instance of <code>CachingMemberRepository</code>.</p>
<h2>Configuring The Decorator With Scrutor</h2>
<p>If the previous approach seems <em>cumbersome</em> to you and like a lot of manual work - that's because it is.</p>
<p>However, there is a simpler way to achieve the same behavior.</p>
<p>We can use the <strong><a href="https://github.com/khellang/Scrutor">Scrutor</a></strong> library to register the decorator:</p>
<pre><code class="language-csharp">services.AddScoped&lt;IMemberRepository, MemberRepository&gt;();

services.Decorate&lt;IMemberRepository, CachingMemberRepository&gt;();
</code></pre>
<p><strong><a href="https://github.com/khellang/Scrutor">Scrutor</a></strong> exposes the <code>Decorate</code> method.
The call to <code>Decorate</code> will register the <code>CachingMemberRepository</code> while ensuring
that it receives the expected <code>MemberRepository</code> instance as its dependency.</p>
<p>I think this approach is much simpler, and it's what I use in my projects.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_006.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[3 Ways To Create Middleware In ASP.NET Core]]></title>
            <link>https://milanjovanovic.tech/blog/3-ways-to-create-middleware-in-asp-net-core</link>
            <guid isPermaLink="false">3-ways-to-create-middleware-in-asp-net-core</guid>
            <pubDate>Sat, 01 Oct 2022 00:00:00 GMT</pubDate>
            <description><![CDATA[In this newsletter, we'll be covering three ways to create middleware in ASP.NET Core applications. Middleware allows us to introduce additional logic before or after executing an HTTP request. You are already using many of the built-in middleware available in the framework. I'm going to show you three approaches to how you can define custom middleware: With Request Delegates, By Convention, and Factory-Based.]]></description>
            <content:encoded><![CDATA[<p>In this newsletter, we'll be covering three ways to create middleware in <strong><a href="http://ASP.NET">ASP.NET</a> Core</strong> applications.</p>
<p><strong>Middleware</strong> allows us to introduce additional logic before or after executing an HTTP request.</p>
<p>You are already using many of the built-in middleware available in the framework.</p>
<p>I'm going to show you three approaches to how you can define custom middleware:</p>
<ul>
<li><a href="#adding-middleware-with-request-delegates">With Request Delegates</a></li>
<li><a href="#adding-middleware-by-convention">By Convention</a></li>
<li><a href="#adding-factory-based-middleware">Factory-Based</a></li>
</ul>
<p>Let's go over each of them and see how we can implement them in code.</p>
<h2>Adding Middleware With Request Delegates</h2>
<p>The first approach to defining a middleware is by writing a <strong>Request Delegate</strong>.</p>
<p>You can do that by calling the <code>Use</code> method on the <code>WebApplication</code> instance
and providing a lambda method with two arguments.
The first argument is the <code>HttpContext</code> and the second argument is
the actual next request delegate in the pipeline <code>RequestDelegate</code>.</p>
<p>Here's what this would look like:</p>
<pre><code class="language-csharp">var builder = WebApplication.CreateBuilder(args);
var app = builder.Build();

app.Use(async (context, next) =&gt;
{
    // Add code before request.

    await next(context);

    // Add code after request.
});
</code></pre>
<p>By awaiting the <code>next</code> delegate, you are continuing the request pipeline execution.
You can <em>short-circuit</em> the pipeline by not invoking the <code>next</code> delegate.</p>
<p>This overload of the <code>Use</code> method is the one suggested by <strong>Microsoft</strong>.</p>
<h2>Adding Middleware By Convention</h2>
<p>The second approach requires us to create a class that will represent our middleware.
We have to follow the convention when creating this class so that we can use it as middleware in our application.</p>
<p>I'm first going to show you what this class looks like, and then explain what is the convention we are following here.</p>
<p>Here's how this class would look like:</p>
<pre><code class="language-csharp">public class ConventionMiddleware(
    RequestDelegate next,
    ILogger&lt;ConventionMiddleware&gt; logger)
{
    public async Task InvokeAsync(HttpContext context)
    {
        logger.LogInformation(&quot;Before request&quot;);

        await next(context);

        logger.LogInformation(&quot;After request&quot;);
    }
}
</code></pre>
<p>The convention we are following has a few rules:</p>
<ul>
<li>We need to inject a <code>RequestDelegate</code> in the constructor</li>
<li>We need to define an <code>InvokeAsync</code> method with an <code>HttpContext</code> argument</li>
<li>We need to invoke the <code>RequestDelegate</code> and pass it the <code>HttpContext</code> instance</li>
</ul>
<p>There's one more thing that's required, and that is to tell our application to use this middleware.</p>
<p>We can do that by calling the <code>UseMiddleware</code> method:</p>
<pre><code class="language-csharp">app.UseMiddleware&lt;ConventionMiddleware&gt;();
</code></pre>
<p>And with this, we have a functioning middleware.</p>
<h2>Adding Factory-Based Middleware</h2>
<p>The third and last approach requires us to also create a class that will represent our middleware.</p>
<p>However, this time we're going to implement the <code>IMiddleware</code> interface.
This interface has only one method - <code>InvokeAsync</code>.</p>
<p>Here's what this class would like:</p>
<pre><code class="language-csharp">public class FactoryMiddleware(ILogger&lt;FactoryMiddleware&gt; logger) : IMiddleware
{
    public async Task InvokeAsync(HttpContext context, RequestDelegate next)
    {
        logger.LogInformation(&quot;Before request&quot;);

        await next(context);

        logger.LogInformation(&quot;After request&quot;);
    }
}
</code></pre>
<p>The <code>FactoryMiddleware</code> class will be resolved at runtime from dependency injection.</p>
<p>Because of this, we need to register it as a service:</p>
<pre><code class="language-csharp">builder.Services.AddTransient&lt;FactoryMiddleware&gt;();
</code></pre>
<p>And like the previous example, we need to tell our application to use our factory-based middleware:</p>
<pre><code class="language-csharp">app.UseMiddleware&lt;FactoryMiddleware&gt;();
</code></pre>
<p>With this, we have a functioning middleware.</p>
<h2>A Word On Strong Typing</h2>
<p>I'm a big fan of <strong>strong typing</strong> whenever possible.
Out of the three approaches I just showed you, the one using the
<code>IMiddleware</code> interface satisfies this constraint the most.
This is also my <strong>preferred</strong> way to implement <strong>middleware</strong>.</p>
<p>Since we're implementing an interface, it's very easy to create
a generic solution to never forget to register your middleware.</p>
<p>You can use reflection to scan for classes implementing
the <code>IMiddleware</code> interface and add them to dependency injection,
and also add them to the application by calling <code>UseMiddleware</code>.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_005.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How To Approach Clean Architecture Folder Structure]]></title>
            <link>https://milanjovanovic.tech/blog/clean-architecture-folder-structure</link>
            <guid isPermaLink="false">clean-architecture-folder-structure</guid>
            <pubDate>Sat, 24 Sep 2022 00:00:00 GMT</pubDate>
            <description><![CDATA[Clean Architecture is a popular approach to structuring your .NET application. It's a layered architecture and splits into four layers: Domain, Application, Infrastructure, and Presentation. Each of the layers is typically one project in your solution. How do we create this in our .NET solutions?]]></description>
            <content:encoded><![CDATA[<p><strong>Clean Architecture</strong> is a popular approach to structuring your application.</p>
<p>It's a layered architecture that splits the project into four layers:</p>
<ul>
<li><a href="#domain-layer">Domain</a></li>
<li><a href="#application-layer">Application</a></li>
<li><a href="#infrastructure-layer">Infrastructure</a></li>
<li><a href="#presentation-layer">Presentation</a></li>
</ul>
<p>Each of the layers is typically one project in your solution.</p>
<p>Here's a visual representation of the <strong>Clean Architecture</strong>:</p>
<p><img src="/blogs/mnw_004/clean_architecture.png" alt=""></p>
<p>How do we create this in our .NET solutions?</p>
<h2>Domain Layer</h2>
<p>The <strong>Domain layer</strong> sits at the core of the <strong>Clean Architecture</strong>.
Here we define things like: entities, value objects, aggregates, domain events, exceptions, repository interfaces, etc.</p>
<p>Here is the folder structure I like to use:</p>
<pre><code>📁 Domain
|__ 📁 DomainEvents
|__ 📁 Entities
|__ 📁 Exceptions
|__ 📁 Repositories
|__ 📁 Shared
|__ 📁 ValueObjects
</code></pre>
<p>You can introduce more things here if you think it's required.</p>
<p>One thing to note is that the <strong>Domain layer</strong> is not allowed to reference other projects in your solution.</p>
<h2>Application Layer</h2>
<p>The <strong>Application layer</strong> sits right above the <strong>Domain layer</strong>.
It acts as an orchestrator for the <strong>Domain layer</strong>, containing the most important use cases in your application.</p>
<p>You can structure your use cases using services or using commands and queries.</p>
<p>I'm a big fan of the <strong>CQRS</strong> pattern, so I like to use the command and query approach.</p>
<p>Here is the folder structure I like to use:</p>
<pre><code>📁 Application
|__ 📁 Abstractions
    |__ 📁 Data
    |__ 📁 Email
    |__ 📁 Messaging
|__ 📁 Behaviors
|__ 📁 Contracts
|__ 📁 Entity1
    |__ 📁 Commands
    |__ 📁 Events
    |__ 📁 Queries
|__ 📁 Entity2
    |__ 📁 Commands
    |__ 📁 Events
    |__ 📁 Queries
</code></pre>
<p>In the <code>Abstractions</code> folder, I define the interfaces required for the <strong>Application layer</strong>.
The implementations for these interfaces are in one of the upper layers.</p>
<p>For every entity in the <strong>Domain layer</strong>, I create one folder with the commands, queries, and events definitions.</p>
<h2>Infrastructure Layer</h2>
<p>The <strong>Infrastructure layer</strong> contains implementations for external-facing services.</p>
<p>What would fall into this category?</p>
<ul>
<li>Databases - PostgreSQL, MongoDB</li>
<li>Identity providers - Auth0, Keycloak</li>
<li>Emails providers</li>
<li>Storage services - AWS S3, Azure Blob Storage</li>
<li>Message queues - Rabbit MQ</li>
</ul>
<p>Here is the folder structure I like to use:</p>
<pre><code>📁 Infrastructure
|__ 📁 BackgroundJobs
|__ 📁 Services
    |__ 📁 Email
    |__ 📁 Messaging
|__ 📁 Persistence
    |__ 📁 EntityConfigurations
    |__ 📁 Migrations
    |__ 📁 Repositories
    |__ #️⃣ ApplicationDbContext.cs
|__ 📁 ...
</code></pre>
<p>I place my <code>DbContext</code> implementation here if I'm using <strong>EF Core</strong>.</p>
<p>It's not uncommon to make the Persistence folder its project.
I frequently do this to have all database facing-code inside of one project.</p>
<h2>Presentation Layer</h2>
<p>The <strong>Presentation layer</strong> is the entry point to our system.
Typically, you would implement this as a Web API project.</p>
<p>The most important part of the <strong>Presentation layer</strong> is the <code>Controllers</code>, which define the API endpoints in our system.</p>
<p>Here is the folder structure I like to use:</p>
<pre><code>📁 Presentation
|__ 📁 Controllers
|__ 📁 Middlewares
|__ 📁 ViewModels
|__ 📁 ...
|__ #️⃣ Program.cs
</code></pre>
<p>Sometimes, I will move the <strong>Presentation layer</strong> away from the actual Web API project.
I do this to isolate the <code>Controllers</code> and enforce stricter constraints.
You don't have to do this if it is too complicated for you.</p>
<h2>Is This The Only Way?</h2>
<p>You don't have to follow the folder structure I proposed to the T.
<strong>Clean Architecture</strong> is very flexible, and you can experiment with it and structure it the way you like.</p>
<p>Do you like more granularity? Create more specific projects.</p>
<p>Do you dislike a lot of projects? Separate concerns using folders.</p>
<p>I'm here to give you options to explore. But it's up to you to decide what's best.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_004.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[How To Improve Performance With EF Core Query Splitting]]></title>
            <link>https://milanjovanovic.tech/blog/how-to-improve-performance-with-ef-core-query-splitting</link>
            <guid isPermaLink="false">how-to-improve-performance-with-ef-core-query-splitting</guid>
            <pubDate>Sat, 17 Sep 2022 00:00:00 GMT</pubDate>
            <description><![CDATA[I recently ran into an issue with Entity Framework Core. The query I was running was constantly timing out. So I used a new EF Core feature called Query Splitting to significantly improve my performance.]]></description>
            <content:encoded><![CDATA[<p>I recently ran into an issue with <strong>Entity Framework Core</strong>.</p>
<p>The query I was running was constantly timing out.</p>
<p>I tried to scale up the application server, and it didn't help.</p>
<p>I tried to scale up the database server, and it didn't help.</p>
<p>So how did I solve the problem?</p>
<h2>What Was The Problem With This Query?</h2>
<p>I'm working on an application in the e-commerce domain.
To be specific, it's an order management system for a kitchen cabinet manufacturer.</p>
<p>The table that I frequently query on is the <code>Orders</code> table.
The <code>Order</code> can have one or more <code>LineItems</code>.
A typical <code>Order</code> will contain 50 <code>LineItems</code>.
Also, <code>LineItems</code> have a table that contains the valid dimensions - <code>LineItemDimensions</code>.</p>
<p>This is the query I was trying to run:</p>
<pre><code class="language-csharp">dbContext
    .Orders
    .Include(order =&gt; order.LineItems)
    .ThenInclude(lineItem =&gt; lineItem.Dimensions)
    .First(order =&gt; order.Id == orderId);
</code></pre>
<p>When EF Core converts this into SQL, this is what it will send to the database:</p>
<pre><code class="language-sql">SELECT o.*, li.*, d.*
FROM Orders o
LEFT JOIN LineItems li ON li.OrderId = o.Id
LEFT JOIN LineItemDimensions d ON d.LineItemId = li.Id
WHERE o.Id = @orderId
ORDER BY o.Id, li.Id, d.Id;
</code></pre>
<p>In most cases, this query will execute just fine.</p>
<p>However, in my situation I was running into the problem of <em>Cartesian Explosion</em>.
This is mainly because of the join to the <code>LineItemDimensions</code> table.
And this is what's causing my query to fail, and time out.</p>
<p>So how did I solve this problem?</p>
<h2>Query Splitting To The Rescue</h2>
<p>With the release of <strong>EF Core 5.0</strong> we got a new feature called <strong>Query Splitting</strong>.
This allows us to specify that a given LINQ query should be split into multiple <code>SQL</code> queries.</p>
<p>To use <strong>Query Splitting</strong>, all you need to do is call the <code>AsSplitQuery</code> method:</p>
<pre><code class="language-csharp">dbContext
    .Orders
    .Include(order =&gt; order.LineItems)
    .ThenInclude(lineItem =&gt; lineItem.Dimensions)
    .AsSplitQuery()
    .First(order =&gt; order.Id == orderId);
</code></pre>
<p>In this case, EF Core will generate the following SQL queries:</p>
<pre><code class="language-sql">SELECT o.*
FROM Orders o
WHERE o.Id = @orderId;

SELECT li.*
FROM LineItems li
JOIN Orders o ON li.OrderId = o.Id
WHERE o.Id = @orderId;

SELECT d.*
FROM LineItemDimensions d
JOIN LineItems li ON d.LineItemId = li.Id
JOIN Orders o ON li.OrderId = o.Id
WHERE o.Id = @orderId;
</code></pre>
<p>Notice that for each <code>Include</code> statement we have a separate <code>SQL</code> query.
The benefit here is that we are not duplicating data when fetching from the database,
as we were in the previous case.</p>
<h2>Turning On Query Splitting For All Queries</h2>
<p>You can enable <strong>Query Splitting</strong> at the database context level.
When configuring your database context you need to call the <code>UseQuerySplittingBehavior</code> method:</p>
<pre><code class="language-csharp">services.AddDbContext&lt;ApplicationDbContext&gt;(options =&gt;
    options.UseSqlServer(
        &quot;CONNECTION_STRING&quot;,
        o =&gt; o.UseQuerySplittingBehavior(
            QuerySplittingBehavior.SplitQuery)));
</code></pre>
<p>This will cause all queries that EF Core generates to be split queries.
To revert back to a single query, you need to call the <code>AsSingleQuery</code> method:</p>
<pre><code class="language-csharp">dbContext
    .Orders
    .Include(o =&gt; o.LineItems)
    .ThenInclude(li =&gt; li.Dimensions)
    .AsSingleQuery()
    .First(o =&gt; o.Id == orderId);
</code></pre>
<h2>What You Should Know About Query Splitting</h2>
<p>Although query splitting is an excellent addition to EF Core, there are a few things you need to be aware of.</p>
<p>There is no consistency guarantee for multiple SQL queries.
You may run into a problem if you have a concurrent update going through
at the same time when you query your data.
To mitigate this, you can wrap the queries inside of a transaction, but this will only introduce performance issues elsewhere.</p>
<p>Each query will require one network round trip. This can degrade performance if your latency to the database is high.</p>
<p>Now that you are armed with this knowledge, go and make your EF queries faster!</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_003.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Records, Anonymous Types, and Non-Destructive Mutation]]></title>
            <link>https://milanjovanovic.tech/blog/records-anonymous-types-non-destructive-mutation</link>
            <guid isPermaLink="false">records-anonymous-types-non-destructive-mutation</guid>
            <pubDate>Sat, 10 Sep 2022 00:00:00 GMT</pubDate>
            <description><![CDATA[Today, I'm going to share some fascinating things you can do with records and anonymous types. I will introduce you to the concept of non-destructive mutation. And I will talk about when and why we might want to use this C# language feature.]]></description>
            <content:encoded><![CDATA[<p>Today, I'm going to share some fascinating things you can do with records and anonymous types.
I will introduce you to the concept of non-destructive mutation.
And I will talk about when and why we might want to use this C# language feature.</p>
<h2>What Is a Record?</h2>
<p>With <strong>C# 9</strong> we can use <strong>records</strong> that are a new reference type.
<strong>C# 10</strong> introduced record structs so that you can define records as value types.
Records are distinct from classes in that record types use value-based equality.</p>
<p>Let's see how we would define a <code>record</code>:</p>
<pre><code class="language-csharp">public record Food(string Name, double Price);
</code></pre>
<p>This way of declaring a <code>record</code> is called a positional record.
The constructor we have defined here is called the <strong>primary constructor</strong>.</p>
<p>The <code>Name</code> and <code>Price</code> properties are init only properties.
This means they can only be set in the constructor or using a property initializer.</p>
<p>Since our properties are init only, is there any way to change their value?</p>
<h2>Non-Destructive Mutation Using The With Expression</h2>
<p>We said we can't modify the properties of our <code>record</code>, because the properties are init only.
However, we can use the <code>with</code> expression (introduced in <strong>C# 9</strong>) to create a new instance
of our record with modified values.</p>
<p>Let's see how we would use the <code>with</code> expression:</p>
<pre><code class="language-csharp">var banana = new Food(&quot;🍌&quot;, 1.95);

var bananaOnSale = banana with
{
    Price = 0.99
};
</code></pre>
<p>It's important to highlight two things here:</p>
<ul>
<li>The original banana instance remains unchanged</li>
<li>The <code>with</code> expression creates a new record instance with only the <code>Price</code> property modified</li>
</ul>
<p>I mentioned Anonymous Types in the title, so let me show you something interesting you can do with them.</p>
<h2>Anonymous Types And Non-Destructive Mutation</h2>
<p>Did you know that you can use the <code>with</code> expression with anonymous types?</p>
<p>Just a reminder that the <code>with</code> expression is available from <strong>C# 9</strong> and later.</p>
<p>Let's create an anonymous type:</p>
<pre><code class="language-csharp">var apple = new
{
    Name = &quot;🍎&quot;,
    Price = 1.21
};
</code></pre>
<p>This is how we can modify it using the with expression:</p>
<pre><code class="language-csharp">var orange = apple with
{
    Name = &quot;🍊&quot;
};
</code></pre>
<p>And again the same rules apply:</p>
<ul>
<li>The original apple instance remains unchanged</li>
<li>The with expression creates a new anonymous type instance with only the Name property modified</li>
</ul>
<p>I found this feature useful in LINQ method chains.</p>
<p>For example, loading an anonymous type from the database where some properties have a default value.
You can then use this feature to calculate the values for these properties in memory.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_002.png" length="0" type="image/png"/>
        </item>
        <item>
            <title><![CDATA[Why I Write My LINQ Queries Tall, Not Wide]]></title>
            <link>https://milanjovanovic.tech/blog/why-i-write-tall-linq-queries</link>
            <guid isPermaLink="false">why-i-write-tall-linq-queries</guid>
            <pubDate>Sat, 03 Sep 2022 00:00:00 GMT</pubDate>
            <description><![CDATA[In this newsletter, I'll show you how you can write tall LINQ queries to improve readability and make your code easier to maintain. We are going to start from a wide LINQ query, and see how we can refactor it into a tall LINQ query.]]></description>
            <content:encoded><![CDATA[<h2>Wishing You a Warm Welcome</h2>
<p>First, I want to welcome you to the first edition of <strong>Milan's .NET Weekly</strong> newsletter.</p>
<p>I hope that this newsletter can become a positive force in the .NET community.
To bring many of us together so that we can all continue learning and improving.</p>
<p>With that out of the way, let's get into .NET!</p>
<h2>The Problem With Wide LINQ</h2>
<p>Let's consider the following LINQ expression from a code style perspective.</p>
<p>I call this a wide LINQ expression because it stretches horizontally across the entire screen.</p>
<pre><code class="language-csharp">dbContext.Animals.Where(animal =&gt; animal.HasBigEars)
    .OrderBy(animal =&gt; animal.IsDangerous).Select(
        animal =&gt; (animal.Id, animal.Name)).ToList();
</code></pre>
<ul>
<li>It is difficult to read.</li>
<li>It is difficult to reason about.</li>
<li>It is difficult to extend or maintain.</li>
</ul>
<p>To improve this, I created a simple rule that you can follow:</p>
<blockquote>
<p>When writing LINQ, try to go tall, not wide.</p>
</blockquote>
<h2>How to Write Tall LINQ</h2>
<p>So how do we write tall LINQ expressions?</p>
<p>I'm going to rewrite the previous expression, to improve it.</p>
<p>Try to follow the <em>one dot per line rule</em>:</p>
<pre><code class="language-csharp">dbContext
    .Animals
    .Where(animal =&gt; animal.HasBigEars)
    .OrderBy(animal =&gt; animal.IsDangerous)
    .Select(animal =&gt; (animal.Id, animal.Name))
    .ToList();
</code></pre>
<p>Is the new version easier to read? <strong>Yes</strong>, very much so.</p>
<p>It is easier to understand what each expression does,
and how it feeds into the next one in the chain.</p>
<p>If you are working in a team, try to propose this as a coding standard (if it isn't one already).<br>
You will see that over time this will make a noticeable difference.</p>
<hr>
]]></content:encoded>
            <author>milan@milanjovanovic.tech (Milan Jovanović)</author>
            <enclosure url="https://milanjovanovic.tech/blog-covers/mnw_001.png" length="0" type="image/png"/>
        </item>
    </channel>
</rss>