Archives
-
Messaging Patterns Explained: Pub-Sub, Queues, and Event Streams
Messaging Patterns Explained: Pub-Sub, Queues, and Event Streams
Forwarded this email? Subscribe here for moreLatest articles
If you’re not a subscriber, here’s what you missed this month.
Coupling and Cohesion: The Two Principles for Effective Architecture
OOP Design Patterns and Anti-Patterns: What Works and What Fails
The Art of REST API Design: Idempotency, Pagination, and Security
To receive all the full articles and support ByteByteGo, consider subscribing:
Modern software rarely lives on one machine anymore. Services run across clusters, web applications are dynamically rendered on the browser, and data resides on a mix of cloud platforms and in-house data centers.
In such a scenario, coordination becomes harder, latency becomes visible, and reliability is crucial. In this environment, messaging patterns become a key implementation detail.
When two services or applications need to talk, the simplest move is a direct API call. It’s easy, familiar, and synchronous: one service waits for the other to respond. But that wait is exactly where things break.
What happens when the downstream service is overloaded? Or slow? Or down entirely? Suddenly, the system starts to stall: call chains pile up, retries back up, and failures increase drastically.
That’s where asynchronous communication changes the game.
Messaging decouples the sender from the receiver. A service doesn’t wait for another to complete work. It hands off a message and moves on. The message is safely stored in a broker, and the recipient can process it whenever it's ready. If the recipient fails, the message waits. If it’s slow, nothing else stalls.
However, not all messaging systems look the same. Three main patterns show up frequently:
Message Queues: One producer, one consumer. Tasks get processed once and only once.
Publish-Subscribe: One producer, many consumers. Messages fan out to multiple subscribers.
Event Streams: A durable, replayable log of events. Consumers can rewind, catch up, or read in parallel.
Each pattern solves a different problem and comes with trade-offs in reliability, ordering, throughput, and complexity. And each maps to different real-world use cases, from task queues in background job systems to high-throughput clickstream analytics to real-time chat.
In this article, we will look at these patterns in more detail, along with some common technologies that help implement these patterns.
Core Concepts...
Continue reading this post for free in the Substack app
Like
Comment
Restack
© 2025 ByteByteGo
548 Market Street PMB 72296, San Francisco, CA 94104
Unsubscribe
by "ByteByteGo" <bytebytego@substack.com> - 11:35 - 8 May 2025 -
Unlock the full potential of your ZIP code data
Unlock the full potential of your ZIP code data
👋 Hi there! In this edition of The Geodata Insider, I'm sharing two blogs to help you get more value from your zip code data—first with smarter analysis, then by choosing the right tools. I believe you'll find these useful!
Your feedback matters
Help us tailor our newsletter to you. Take 1 minute to share your opinions and get more relevant insights each month.
3:00 minute read
📈 Extracting actionable insights from zip codes
🛠️ Choose the right tool for your zip code analysis
🗽 Major zip code boundaries update in the USA
⛴️ Update on the UNLOCODEs database
🔎 Monthly changes
🏔️ Norway's coast: A cartographer's nightmare
.png?width=100&upscale=true&name=Group%2054%20(1).png)
Extracting actionable insights from zip codes
Learn how to clean your zip code data, choose the right mapping approach, and leverage visualization tools for actionable business insights.
Choose the right tool for zip code analysis
Discover how platforms like Power BI, Tableau, or CARTO can help you clean, map, and visualize your zip code data.
Major zip code boundaries update in the USA
A full country update will be rolled out soon, affecting over 30,000 polygons across the USA. As usual, all shapes are perfectly matched and available in high resolution and visualization precision.
Update on the UNLOCODEs database
The Enhanced UNLOCODEs database includes the latest official update of 12 core columns. We have also enriched them with recommended corrections and links to real cities and regions.
Monthly changes
In April, we updated 12,536 rows in our postal database.
Full Postal & Street updates
Sweden, South Korea
Postal database patches
(small changes)
Canada (Postcodes)
Ireland (Administrative Divisions)
Italy (Postcodes)
Kyrgyzstan (Administrative Divisions name change)
EswatiniPostal boundary updates
United States of America
Postal boundary patches (small changes)
Albania, Armenia, Azerbaijan, Bahrain, Belarus, Brunei, Colombia, Costa Rica, Cyprus, Czechia, El Salvador, Faroe Islands, French Polynesia, Greece, Guadeloupe, Japan, Jordan, Kiribati, Malaysia, Malta, Mexico, Moldova, Myanmar, New Zealand, Palestine, Paraguay, Peru, Philippines, Qatar, Russia, Saudi Arabia, South Africa
Administrative boundaries patches (small changes)
Germany, United Kingdom, Malaysia
For a detailed changelog, take a look at the CSV file.
Norway's coast: A cartographer's nightmare
Welcome to the coastline paradox—a geographic brain teaser where the length of a coastline depends entirely on how closely you measure it.
Take Norway. If you measure its jagged coast with a 100 km ruler, you'll get around 25,000 km.
But switch to a 1 km ruler, and suddenly, it stretches beyond 100,000 km. That's longer than the equator!
Why? Norway's coast is full of nooks, crannies, and fjords. The smaller the measurement unit, the more detail you capture—and the longer the coastline becomes.
This occurrence isn't a math error—it's just nature being delightfully complicated.
Source: National Geographic.
Follow us on LinkedIn for more geographical facts like this!
Kind regards,
Jerome & the GeoPostcodes team
PS: Interested in previous Monthly Product Updates? Read here.
GeoPostcodes, Bld Bischoffsheim 15, 1000 Bruxelles, Belgium
by "Jérôme from GeoPostcodes" <jerome@geopostcodes.com> - 06:45 - 8 May 2025 -
88%—It’s how much IDC says we help lower downtime.
New Relic
May 2025
IDC names New Relic an observability leader in Europe. The first-ever IDC MarketScape: European Observability Market 2025 Vendor Assessment names New Relic a leader in strategic vision and capabilities. Download the report to see how our customisable platform helps leading European organisations break down silos, streamline operations, and deliver exceptional digital experiences.
Read the report 
New Relic University Observability Workshop Series We're excited to offer a series of 90 minute online workshops designed to enhance your skills and get you ready for securing the New Relic Certification of your choice. Whether you’re a seasoned professional or just starting your observability journey, our courses are designed to propel your career forward with New Relic and observability.Register now 
How Do AI and Observability Redefine Application Performance? In this episode of the Tech Transformed podcast, Manesh Tailor, EMEA Field CTO speaks with Kevin Petrie, Vice President of Research at BARC about the intersection of artificial intelligence (AI) and observability, and how this is positively changing business operations.
Listen here 
AWS Summits EMEA
We're thrilled to announce our participation in the upcoming AWS Summits happening in multiple cities across Europe and beyond! Here's where you can catch us.- Dubai, Wednesday 21st May
- Stockholm, Wednesday 4th June
- Hamburg, Thursday 5th June
- Madrid, Wednesday 11th June
We'll be showcasing our latest innovations, hosting engaging demos, and offering valuable insights into how New Relic can empower your cloud journey on AWS.
Don't miss this opportunity to connect with us and explore the future of observability and performance monitoring in the cloud!
Intelligent Observability for Business Uptime
Amidst the high cost of IT outages—up to $1.9M per hour—every minute of downtime matters. Watch our product experts cover six new innovations for increasing uptime. Watch on-demand.
New Relic Now+ On-demand
Catch up on-demand for the launch of Intelligent Observability and 20+ AI-driven innovations that redefine how IT and engineering teams power digital business. Watch here.
Speed meets precision: Watch Duty's real-time data model in wildfire emergencies
Powered by 200+ volunteers, Watch Duty delivers real-time wildfire alerts to millions of users, helping communities stay safe. Learn how their small team uses data to scale life-saving action. Learn more.
Simplify instrumentation and data management with New Relic Control
Managing observability at scale just got easier with New Relic Control—the first comprehensive observability control plane. Centrally manage agents, fleets, and telemetry all in one unified platform. Learn more.
Best answers from the Support Forum
Find curated answers from some of your most recent questions, located on the Support Forum. From Kubernetes and logs, to dashboards and queries, this trove might have the solution for you. Dive in now.
Not an existing New Relic user? Sign up for a free account to get started!
Need help? Let's get in touch.
This email is sent from an account used for sending messages only. Please do not reply to this email to contact us—we will not get your response.
This email was sent to info@learn.odoo.com Update your email preferences.
For information about our privacy practices, see our Privacy Policy.
Need to contact New Relic? You can chat or call us at +44 20 3859 9190.
Strand Bridge House, 138-142 Strand, London WC2R 1HH
© 2025 New Relic, Inc. All rights reserved. New Relic logo are trademarks of New Relic, Inc
by "New Relic" <emeamarketing@newrelic.com> - 05:14 - 8 May 2025 -
Be inspired by FairPrice Group at the Agentforce World Tour Broadcast | 29 May
Be inspired by FairPrice Group at the Agentforce World Tour Broadcast | 29 May
29 May 2025
9:00 a.m. BKT/ICT | 10:00 a.m. SGTRegister now Agentforce World Tour Broadcast
Join us for the Agentforce World Tour Broadcast, and take your business to the next level with groundbreaking innovation.
Discover how agentic AI can unlock a digital labour force for businesses and hear from inspiring Trailblazer James Huang, Head of Marketing Technology,
FairPrice Group.
You’ll find new ways to deepen customer and employee relationships, and boost growth and productivity, by activating digital labour for every team in your company.
Take away tangible insights from the Agentforce keynote and gain unique insights from our thought leaders.Our Speakers
James Huang
Head of Marketing Technology
FairPrice Group
Olivia Nguyen
Solution Engineer Lead, Slack
Salesforce
Sujith Abraham
Senior Vice President & General Manager, ASEAN
SalesforceBroadcast date and time: 29 May 2025, 09:00 a.m. BKT/ICT, 10:00 a.m. SGT
Duration: 60 mins
Experience the groundbreaking benefits of Agentforce - the agentic layer and Digital Labour Platform at Salesforce, to take your business to the next level.
It’s time to power up with AgentforceRegister now Captions will be available during the broadcast for Traditional Chinese, Thai, Vietnamese, and Bahasa Indonesia. Your unique viewing link for the broadcast event will be sent to you in a confirmation email upon registering.
© 2025 Salesforce, Inc.
Salesforce.com Singapore Pte Ltd. 5 Temasek Boulevard #13-01 Suntec Tower 5 Singapore 038985
General Inquiries: +65 6302 5700
This email was sent to info@learn.odoo.com
Manage Preferences to Unsubscribe | View as webpage | Privacy Statement
Powered by Salesforce Marketing Cloud
by "Salesforce Events" <aseanevents@salesforce.com> - 03:02 - 8 May 2025 -
Leading authentically in turbulent times
On McKinsey Perspectives
by "Only McKinsey Perspectives" <publishing@email.mckinsey.com> - 01:40 - 8 May 2025 -
Inquiry about Electrical low voltage components
Dear info,
I hope this message finds you well.
I am reaching out to inquire about several electrical components that we are currently considering for our upcoming project. Specifically, we are interested in the following items:
1. **AC Contactors** - We need detailed information on your available models, including specifications, ratings, and pricing.
2. **Current Transformers** - It would be helpful to understand the different types you offer, along with their applications and compatibility with various systems.
3. **Vacuum Circuit Breakers** - Please provide details regarding their operational characteristics, installation requirements, and best practices for maintenance.
4. **Knife Switches** - We are looking for options that meet safety standards and would like to know more about their design and functionality.
If you could provide us with catalogs, technical datasheets, and pricing information for these components, it would be greatly appreciated. Additionally, any recommendations for specific applications or projects would be invaluable.
Thank you for your assistance. I look forward to your prompt response.
Best regards,
William Chen
sales manager
Wenzhou Super Bull Electric Co., Ltd.
whatsapp and phone :+86 15167770101 & +86 15381529330
No. 20, North First Road, Zhaojiadong Village, Beibaixiang, Yueqing City, Wenzhou, Zhejiang Province, China
by "Ynae Ravanelo" <ynaeravanelo@gmail.com> - 11:20 - 7 May 2025 -
Factory price Bathroom products for you
Dear info,
I’m from Liangying Sanitary Co., Ltd and learn about your contact through the internet and know you are in the market of Bathroom Hardware products that we also specialize in. Given my familiarity with your market, I believe there is an opportunity to share market insights and potentially expand our business partnership.
If interested, we are happy to provide you with our price and product catalog.
Looking forward to hearing from you soon.
Best wishes,
by "Harikrishnan Caruso" <harikrishnancaruso613@gmail.com> - 03:37 - 7 May 2025 -
Affordable housing creates economic opportunity
Re:think
FRESH TAKES ON BIG IDEAS
ON AFFORDABLE HOUSING
Bridging the affordable housing gap and spurring economic mobility
JP JulienHousing is the biggest line item in families’ household budgets, with some spending more than half of their income to own or rent a place to live. It is often a family’s greatest source of wealth, too. But unfortunately, housing acts as a barrier instead of a gateway for far too many.
Quality housing that is affordable enables access to everything that supports economic mobility: public education, transit, employment opportunities, and physical and mental well-being, to name a few. Housing is also fundamental to having autonomy and a sense of belonging. Do families have a say about what happens to them? Are they part of a community, and do they feel valued? Housing is impossible to separate from economic mobility.
With this in mind, the McKinsey Institute for Economic Mobility examined the forces behind the 8.2-million-unit affordable housing gap in the United States, which, without action, could swell to 9.6 million units in ten years. The economic payoff from closing this gap is substantial: an estimated $2 trillion in additional output over ten years and 1.7 million new jobs. All Americans would share these benefits, but because Black Americans have been disproportionately burdened by the housing crisis, they would benefit in particular.
There are things that work to alleviate the crisis, but they happen in piecemeal and isolated ways. The push of this research was not to say, “What are the new ideas?” but rather, “What is working and what are ways to unlock barriers and help those solutions work even more effectively?”
We identified the following five documented and researched approaches—chosen from 80 that we studied—that could help alleviate the housing crisis and improve economic mobility: exploring creative incentives and partnerships to free up land, augmenting programs to unleash private capital, scaling off-site home construction, reinvesting in public housing and shared-equity models, and revamping housing-choice vouchers.
Let’s dive into zoning. It is well documented that changes in zoning have an impact on what is built, how much, how quickly, and at what cost. Solutions include creating more multifamily zoning and changing parking and lot-size requirements. On paper, these seem like easy fixes. But in practice, zoning can be incredibly challenging to change. The question is how to combine carrots and sticks to motivate jurisdictions to adopt more housing-friendly zoning solutions.“The economic payoff from closing the affordable housing gap is substantial: an estimated $2 trillion in additional output over ten years and 1.7 million new jobs.”
Another approach that stood out was off-site home construction, an idea that has been around for a long time. Modular construction, in particular, offers speed and cost-saving benefits. What was enlightening from the research was not the opportunity but rather the problem of well-meaning bureaucratic hurdles, including codes that differ from place to place that make it hard for off-site construction companies to scale production. This is an area where relatively small changes to the existing system could yield substantial gains to the cost and delivery of affordable housing.
Housing can sometimes feel like a wonky policy issue. And as such, it may not always get the airtime and attention that it warrants. But as American families know, housing is among the issues that we all deal with every day, in every community and part of the country. For change to occur, people must not lose sight of that.
Housing is a shared reality and challenge, but “zero sum” thinking has been hard to shake. While building affordable housing might be better for everyone, there are often entrenched perceived risks to neighborhood character, local traffic, and home values. This perception can impede action and create a sense of winners and losers.
But a mindset shift is possible. Housing is one of the truly universal challenges that all American communities are dealing with: rural and urban, Black, White, Latino, Asian, coastal, inland, young, and old. Solving this problem is urgent for the well-being of families everywhere and critical to unlocking the full potential of our economy. While it won’t be easy, progress is possible and necessary.
I believe that our nation can still do hard things. Systematically building and preserving more affordable housing will not be easy, but the return on investment is meaningful. Housing investments could put more Americans to work, support critical industries like the construction trades, and allow more families to invest in other products and services that improve their odds of achieving economic success.
Will the solutions we examined bridge the entire affordable housing gap? No. But reimagining and scaling them could create a better path to economic mobility for millions of Americans. If the nation commits to solving the crisis more holistically, it could do some real good.Share JP Julien’s insights
ABOUT THIS AUTHOR
JP Julien is a partner in McKinsey’s Philadelphia office and leader within the McKinsey Institute for Economic Mobility.
MORE FROM THIS AUTHOR
UP NEXT
Alex Camp on corporate culture
How do leaders measure their business cultures’ impact and make the best parts of them stick? Instead of relying on the informal transfer of rules, they should be intentional about culture.
Follow our thinking
This email contains information about McKinsey’s research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to our McKinsey Quarterly alert list.
Copyright © 2025 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey Quarterly" <publishing@email.mckinsey.com> - 02:28 - 7 May 2025 -
Enhance Your Operations with Our High-Quality Valves
Dear info,
I hope this email finds you well. I am reaching out from YAYE Industrial Technology (Zhejiang) Co., Ltd, offering a comprehensive selection of valves, including ball, gate, globe, check, forged steel, butterfly valves, filters, and special material valves.
Key Features of Our Valves:
- Precision engineering for optimal performance
- Durable construction for long-lasting reliability
- Versatile designs to suit various applications
- Rigorous testing to ensure quality and safetyIn addition to our standard offerings, we provide custom solutions to address unique challenges and specifications.
We look forward to collaborating with you.
Best regards,
Leo
YAYE Industrial Technology (Zhejiang) Co., Ltd
by "Shamira Gem" <shamiragem146@gmail.com> - 12:06 - 7 May 2025 -
The future of growth: Get the latest issue of the McKinsey Quarterly
The third of four special issues celebrating the 60th birthday of McKinsey Quarterly, “The future of growth,” explores a powerful question: how can companies achieve sustainable, long-term growth—even in uncertain times?
In “The next big arenas of competition,” McKinsey’s Chris Bradley, Michael Chui, Kweilin Ellingrud, Michael Birshan, and coauthors uncover the sectors that have historically captured outsize value—and those poised to lead in the years ahead. You’ll also find a five-pronged leadership approach to outperform, the role of shifting global demographics in shaping productivity, and other insights that together provide a holistic blueprint for consistent growth.
Since 1964, the McKinsey Quarterly has brought our firm’s most distinctive, cross-cutting insights to global executives. We’re delighted to continue a yearlong celebration commemorating the Quarterly’s 60th birthday with this third issue. Sign up for a free digital Quarterly membership today and unlock access to “Q60,” a yearlong series. Here’s what you’ll get:
•
Q60 special issues: Dive deep into themes shaping the future of business, with each issue dedicated to a critical topic.
•
Bonus digital offerings: Engage with special interactive features and explore compilations of timeless Quarterly classics, many of which were available only in print until now.
•
Immersive digital reading experience: Enjoy each Quarterly issue online.
And don’t miss the newly released interactive timeline “Revolutionary innovations propelling growth” for eight breakthroughs that are likely to drive growth in the years ahead.
A membership to the digital edition of the McKinsey Quarterly also includes downloads of more than 100 reports in The McKinsey Insights Store as well as access to previous digital issues of the Quarterly.
This email contains information about McKinsey’s research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to our McKinsey Quarterly alert list.
Copyright © 2025 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey & Company" <publishing@email.mckinsey.com> - 10:21 - 7 May 2025 -
Run global payroll the smart way with Remote
Run global payroll the smart way with Remote
Hi MD,
Managing payroll across countries shouldn’t feel like a constant scramble. But for many of our customers, fragmented systems, manual processes, and compliance hurdles were slowing them down... until they switched to Remote.
We empower you to manage payroll the smart way. Simple, accurate, and compliant from the start.
🔍 Get a first look: Book a demo to learn how Remote Payroll's latest updates can streamline global payroll for you.Here's what's new:
✅ Save time and boost accuracy by consolidating payroll data in minutes.
✅ Run payroll faster with native engines and an integrated HR platform.
✅ Stay compliant across every country without third-party add-ons.
Schedule a call with our experts to see it in action.
You received this email because you are subscribed to
News & Offers from Remote Europe Holding B.V
Update your email preferences to choose the types of emails you receive.
Unsubscribe from all future emailsRemote Europe Holding B.V
Copyright © 2025 Remote Europe Holding B.V All rights reserved.
Kraijenhoffstraat 137A 1018RG Amsterdam The Netherlands
by "Remote Team" <hello@remote-comms.com> - 03:00 - 7 May 2025 -
High-quality aluminum products supply to meet your diverse needs
Dear info,
Hello! We are a manufacturer specializing in the production of aluminum plates, aluminum bars and aluminum products with a history of 13 years. We are committed to providing high-quality aluminum materials to customers around the world and have established a good reputation in the industry.
The aluminum plates and aluminum bars we provide are suitable for multiple industries such as aviation, automobiles, electronics and construction. The products meet international standards and have reliable quality. Whether you need custom specifications or standard products, we can meet your needs.
We are very much looking forward to working with your company to promote business development together. If you have any needs or questions, please feel free to contact me.
Thank you for your valuable time!
Best regards,
Suzhou Gebizhen Metal Materials Co., Ltd.
by "Lapsb Fakv" <fakvlapsb@gmail.com> - 01:13 - 7 May 2025 -
Elevate Your Offerings with Premium Plastic Solutions
Dear info,
We are Taizhou Hongshuo Plastic Industry Co., Ltd, your trusted partner in crafting high-quality plastic lifestyle products. Our range includes plastic straw cups, folding storage boxes, color-changing cups, rice storage containers, socks, and more—all designed for durability, innovation, and everyday convenience.
With advanced manufacturing techniques and a focus on eco-friendly materials, we ensure every product meets rigorous global standards. Whether you’re sourcing for retail, hospitality, or custom OEM/ODM projects, our team provides tailored solutions to align with your brand vision and market demands.
Let’s discuss how we can enhance your product line! Reply today to request samples, catalogs, or a personalized quote. We’re here to deliver excellence at every step.
Warm regards,
Haining Mengli Knitting Co., Ltd
by "Macco Kammerer" <maccokammerer@gmail.com> - 12:27 - 7 May 2025 -
Strengthening the US healthcare system: What leaders can do
On McKinsey Perspectives
Brought to you by Alex Panas, global leader of industries, & Axel Karlsson, global leader of functional practices and growth platforms
Welcome to the latest edition of Only McKinsey Perspectives. We hope you find our insights useful. Let us know what you think at Alex_Panas@McKinsey.com and Axel_Karlsson@McKinsey.com.
—Alex and Axel
•
Living longer and healthier. Individual and collective health are critical for both personal happiness and societal progress. In the US workplace, McKinsey Senior Partner Gretchen Berlin and coauthors explain, a focus on holistic health—an integrated view of an individual’s mental, physical, spiritual, and social functioning—can also help employees be more productive. While US citizens are living longer, they aren’t necessarily living healthier: McKinsey Health Institute (MHI) analysis finds that, globally, people spend about half of their lives in moderate or poor health.
•
The way forward. To improve Americans’ holistic health, healthcare and government leaders can prioritize three critical areas. One of them involves longevity. With older adults expected to outnumber kids for the first time in US history by 2034, supporting their reentry into the workforce could address labor force challenges. It could also boost US GDP by up to 6.7%, per a joint report from the FII Institute and MHI. See all three priorities that could help leaders bolster the US healthcare system and the broader economy, and visit McKinsey’s America at 250 series for more new ideas as the nation approaches a milestone birthday.
—Edited by Belinda Yu, editor, Atlanta
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to the Only McKinsey Perspectives newsletter, formerly known as Only McKinsey.
Copyright © 2025 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "Only McKinsey Perspectives" <publishing@email.mckinsey.com> - 11:05 - 6 May 2025 -
Feeling stuck on your next digital move?
Feeling stuck on your next digital move?
Hi MD Abul,
If you're trying to figure out what to do next with your digital strategy, maybe it's new software, maybe it's AI, maybe it's just untangling the chaos, we've got something that might help.
We just launched a set of free assessments that give you a quick look at where your business stands today and what you could be doing better. These aren’t just automated results either. After you complete any of them, one of our actual consultants will follow up to talk through your results and get to know your business a bit more.
Here’s what you can check out:
Small Business Digital Assessment
Perfect for growing companies looking to get organized and future-readyDigital Transformation Readiness Assessment
Helpful if you're thinking big and want to understand how prepared you really areOrganizational Change Readiness Assessment
Focuses on the people side of change so your team doesn't get left behindThey're a great starting point if you're overwhelmed by options or just unsure what your next step should be. No sales pitch, no pressure, just some real guidance from someone who’s been there.
You can try any of them here: Explore the Assessments
Let me know if you want to chat after. We’d love to hear what you’re working on.
Mission: AI-Possible is taking off this August at Digital Stratosphere 2025.
Join us in Denver!Use the code "earlybird" for 20% off your ticket(s)!
Best regards,
Eric Kimberling
by "Eric Kimberling" <eric.kimberling@thirdstage-consulting.com> - 06:06 - 6 May 2025 -
Elevate Your Brand with Our Premium Saw Blades
Dear info,
I am Elizabeth from Wenzhou Yichuan Tools. Our company specializes in hardware tools, including jigsaw blades, reciprocating saw blades, saber saw blades, and multi-tool blades.
With two decades of industry experience and state-of-the-art production facilities, we deliver high-quality products at competitive prices. Our monthly capacity exceeds 5 million pieces.
We also provide OEM services to customize products for your brand. Would you like a quotation and samples for evaluation?
Best regards,
Elizabeth
Wenzhou Yichuan Tools Co., Ltd.
by "Odelette Hoyle" <robbie169850@gmail.com> - 05:02 - 6 May 2025 -
Greetings from Hangzhen Energy
Dear info,
I hope you are doing well. I'm with Zhejiang Hangzhen Energy Technology Company Limited, where we specialize in providing advanced solutions for the chemical, petrochemical, auxiliary machinery, and metallurgical industries.
Our offerings include steam jet vacuum pumps, Roots vacuum pump units, automatic exhaust valves, VD/VOD/RH refining furnaces, and more. , designed to optimize your processes and ensure reliable performance in demanding environments.
I'd love to discuss how our products can benefit your operations. Please let me know if we can arrange a time to talk further.
Best regards,
Xu Shangyong/Marketing Department
Zhejiang Hangzhen Energy Technology Company Limited.
Yuexiu Xinghui Middle, No. 723 Dayuan Road, Qingshanhu Street, Lin'an District, Hangzhou City, Zhejiang Province, China
www.hzvacuum.com
by "Robert Pht" <robertpht09@gmail.com> - 01:04 - 6 May 2025 -
How Halo on Xbox Scaled to 10+ Million Players using the Saga Pattern
How Halo on Xbox Scaled to 10+ Million Players using the Saga Pattern
Forwarded this email? Subscribe here for moreThe 6 Core Competencies of Mature DevSecOps Orgs (Sponsored)
Understand the core competencies that define mature DevSecOps organizations. This whitepaper offers a clear framework to assess your organization's current capabilities, define where you want to be, and outline practical steps to advance in your journey. Evaluate and strengthen your DevSecOps practices with Datadog's maturity model.
Disclaimer: The details in this post have been derived from the articles/videos shared online by the Halo engineering team. All credit for the technical details goes to the Halo/343 Engineering Team. The links to the original articles and videos are present in the references section at the end of the post. We’ve attempted to analyze the details and provide our input about them. If you find any inaccuracies or omissions, please leave a comment, and we will do our best to fix them.
In today's world of massive-scale applications, whether it’s gaming, social media, or online shopping, building reliable systems is a difficult task. As applications grow, they often move from using a single centralized database to being spread across many smaller services and storage systems. This change, while necessary for handling more users and data, brings a whole new set of challenges, especially around consistency and transaction handling.
In traditional systems, if we wanted to update multiple pieces of data (say, saving a new customer order and reducing inventory), we could easily rely on a transaction. A transaction would guarantee that either all updates succeed together or none of them happen at all.
However, in distributed systems, there is no longer just one database to talk to. We might have different services, each managing its data in different locations. Each one might be running on different servers, cloud providers, or even different continents. Suddenly, getting all of them to agree at the same time becomes much harder. Network failures, service crashes, and inconsistencies are now of everyday situations.
This creates a huge problem: how do we maintain correct and reliable behavior when we can't rely on traditional transactions anymore? If we’re booking a trip, we don’t want to end up with a hotel reservation but no flight. If we’re updating a player's stats in a game, we can't afford for some stats to update and others to disappear.
Engineers must find new ways to coordinate operations across multiple independent systems to tackle these issues. One powerful pattern for solving this problem is the Saga Pattern, a technique originally proposed in the late 1980s but increasingly relevant today. In this article, we’ll look at how the Halo Engineering Team at 343 Game Studio (now Halo Studios) used the Saga pattern to improve the player experience.
Build AI products and grow your impact (Sponsored)
Two critical themes define today's engineering landscape: building AI applications and growing your influence in an increasingly automated world. So, we teamed up with Maven to curate 6 hands-on courses on these topics, led by experienced practitioners.
Building Agentic AI Applications with Aishwarya Reganti (AWS)
Multi-Agent Applications with Amir Feizpour (AI engineer)
AI Evals with Hamel Husain (AI engineer, ex-Github, Airbnb)
Break Through to Executive with Ethan Evans (Amazon VP)
Elite Product Leadership with Amit Fulay (Microsoft VP)
Eng Management System with Gilad Naor (ex-Meta, Amazon)
Our friends at Maven said to use code BYTEBYTEGO to get $100 off any of these selected courses.
ACID Transactions
When engineers design systems that store and update data, they often rely on a set of guarantees called ACID properties. These properties make sure that when something changes in the system, like saving a purchase or updating a player's stats, it happens safely and predictably.
Here’s a quick look at each property.
Atomicity: Atomicity means "all or nothing". Either every step of a transaction happens successfully, or none of it does. If something goes wrong halfway, the system cancels everything so that it is not stuck with half-completed work. For example, either book both a flight and a hotel together, or neither.
Consistency: Consistency guarantees that the system’s data moves from one valid state to another valid state. After a transaction finishes, the data must still follow all the rules and constraints it’s supposed to. For example, after buying a ticket, the system shouldn’t show negative seat availability. The rules about what’s valid are always respected.
Isolation: Isolation ensures that transactions don’t interfere with each other even if they happen at the same time. It should feel like each transaction happens one after another, even when they’re happening in parallel. For example, if two people are trying to buy the last ticket at the same time, only one transaction succeeds.
Durability: Durability means that once a transaction is committed, it’s permanent. Even if the system crashes right after, the result stays safe and won't be lost. For example, if a user bought a ticket and the app crashes immediately after, the ticket is still booked and not lost in the system.
Single Database Model
In older system architectures, the typical way to build applications was to have a single, large SQL database that acted as the central source of truth.
Every part of the application, whether it was a game like Halo, an e-commerce site, or a banking app, would send all of its data to this one place.
Here’s how it worked:
Applications (like a game server or a website) would talk to a stateless service.
That stateless service would then communicate directly with the single SQL database.
The database would store everything: player statistics, game progress, inventory, financial transactions, all in one consistent, centralized location.
Some advantages of the single database model were strong guarantees enforced by the ACID properties, simplicity of development, and vertical scaling.
Scalability Crisis of Halo 4
During the development of Halo 4, the engineering team faced unprecedented scale challenges that had not been encountered in earlier titles of the franchise.
Halo 4 experienced an overwhelming level of engagement:
Over 1.5 billion games were played.
More than 11.6 million unique players connected and competed online.
Every match generated detailed telemetry data for each player: kills, assists, deaths, weapon usage, medals, and various other game-related statistics. This information needed to be ingested, processed, stored, and made accessible across multiple services, both for real-time feedback in the game itself and for external analytics platforms like Halo Waypoint.
The complexity further increased because a single match could involve anywhere from 1 to 32 players. For each game session, statistics needed to be reliably updated across multiple player records simultaneously, preserving data accuracy and consistency.
Inadequacy of a Single SQL Database
Before Halo 4, earlier installments in the series relied heavily on a centralized database model.
A single large SQL Server instance operated as the canonical source of truth. Application services would interact with this centralized database to read and write all gameplay, player, and match data, relying on the built-in ACID guarantees to ensure data integrity.
However, the scale required by Halo 4 quickly revealed serious limitations in this model:
Vertical Scaling Limits: While the centralized database could be scaled vertically (by adding more powerful hardware), there were inherent physical and operational limits. Beyond a certain threshold, no amount of memory, CPU power, or storage optimization could compensate for the growing volume of concurrent reads and writes.
Single Point of Failure: Relying on one database instance introduced a critical operational risk. Any downtime, data corruption, or resource saturation in that instance could bring down essential services for the entire player base.
Contention and Locking Issues: With millions of users interacting with the database simultaneously, contention for locks and indexes became a bottleneck.
Operational Complexity of Partitioning: The original centralized system was not designed for partitioned workloads. Retroactively introducing sharding or partitioning into a monolithic SQL structure would have required major rewrites and complex operational procedures, creating risks of inconsistency and service instability.
Mismatch with Cloud-native Architecture: Halo 4’s backend migrated to Azure’s cloud infrastructure, specifically using Azure Table Storage. Azure Table Storage is a NoSQL system that inherently relies on partitioned storage and offers eventual consistency. The old model of transactional consistency across a single database did not align with this partitioned, distributed environment.
Given these challenges, the engineering team recognized that continuing to rely on a monolithic SQL database would limit scalability and expose the system to unacceptable levels of risk and downtime. A transition to a distributed architecture was necessary.
Introduction to Saga Pattern
The Saga Pattern originated through a research paper published in 1987 by Hector Garcia-Molina and Kenneth Salem at Princeton University. The research addressed a critical problem: how to handle long-lived transactions in database systems.
At the time, traditional transactions were designed to be short-lived operations, locking resources for a minimal duration to maintain ACID guarantees. However, some operations, such as generating a complex bank statement, processing large historical datasets, or reconciling multi-step financial workflows, require holding locks for extended periods. These long-running transactions created bottlenecks by tying up resources, reducing system concurrency, and increasing the risk of failure.
The Saga Pattern solves these issues in the following ways:
A single logical operation is split into a sequence of smaller sub-transactions.
Each sub-transaction is executed independently and commits its changes immediately.
If all sub-transactions succeed, the Saga as a whole is considered successful.
If any sub-transaction fails, compensating transactions are executed in reverse order to undo the changes of previously completed sub-transactions.
See the diagram below that shows an example of this pattern:
Key technical points about Sagas are as follows:
Sub-transactions must be independent: The successful execution of one sub-transaction should not directly depend on the outcome of another. This independence allows for better concurrency and avoids cascading failures.
Compensating transactions are required: For every sub-transaction, a corresponding compensating action must be defined. These compensating transactions semantically "undo" the effect of their associated operations. However, the system may not always be able to return to the exact previous state; instead, it aims for a semantically consistent recovery.
Atomicity is weakened: Unlike traditional transactions, where partial updates are never visible, in a Saga, intermediate states are visible to other parts of the system. Partial results may exist temporarily until either the full sequence completes or a failure triggers rollback through compensation.
Consistency is preserved through business logic: Instead of relying on database-level transactional guarantees, Sagas maintain application-level consistency by ensuring that after all sub-transactions and compensations, the system is left in a valid and coherent state.
Failure management is built in: Sagas treat failures as an expected part of the system's operation. The pattern provides a structured way to handle errors and maintain resilience without assuming perfect reliability.
Saga Execution Models
The main aspects of the Saga execution model are as follows:
Single Database Execution
When the Saga Pattern was first introduced, it was designed to operate within a single database system. In this environment, executing a saga requires two main components:
1 - Saga Execution Coordinator (SEC)
The Saga Execution Coordinator is a process that orchestrates the execution of all the sub-transactions in the correct sequence. It is responsible for:
Starting the saga
Executing each sub-transaction one after another
Monitoring the success or failure of each sub-transaction
Triggering compensating transactions if something goes wrong
The SEC ensures that the saga progresses correctly without needing distributed coordination because everything is happening within the same database system.
2 - Saga Log
The Saga Log acts as a durable record of everything that happens during the execution of a saga. Every major event, starting a saga, beginning a sub-transaction, completing a sub-transaction, beginning a compensating transaction, completing a compensating transaction, ending a saga, is written to the log.
The Saga Log guarantees that even if the SEC crashes during execution, the system can recover by replaying the events recorded in the log. This provides durability and recovery without relying on traditional transaction locking across the entire saga.
Failure Handling
Handling failures in a single database saga relies on a strategy called backward recovery.
This means that if any sub-transaction fails during the saga’s execution, the system must roll back by executing compensating transactions for all the sub-transactions that had already completed successfully.
Here’s how the process works:
Detection of Failure: The SEC detects that a sub-transaction has failed because the database operation either returns an error or violates a business rule.
Recording the Abort: The SEC writes an abort event to the Saga Log, marking that the forward execution path has been abandoned.
Starting Compensations: The SEC reads the Saga Log to determine which sub-transactions had been completed. It then begins executing the corresponding compensating transactions in the reverse order.
Completing Rollback: Each compensating transaction is logged in the Saga Log as it begins and completes.
After all necessary compensations have been successfully applied, the saga is formally marked as aborted in the log.
Halo 4 Stats Service
Here are the key components of how Halo used the Saga Pattern:
Service Architecture
The Halo 4 statistics service was built to handle large volumes of player data generated during and after every game. The architecture used a combination of cloud-based storage and actor-based programming models to manage this complexity effectively.
The service architecture included the following major components:
Azure Table Storage: All persistent player data was stored in Azure Table Storage, a NoSQL key-value store. Each player's data was assigned to a separate partition, allowing for highly parallel reads and writes without a single centralized bottleneck.
Orleans Actor Model: The team adopted Microsoft's Orleans framework, which is based on the actor model. Actors (referred to as "grains" in Orleans) represented different logical units in the system.
Game Grains handled the aggregation of statistics for a game session.
Player Grains were responsible for persisting each player’s statistics. In Halo 4’s backend system, a Player Grain is a logical unit that represents a single player’s data and behavior inside the server-side application.
Azure Service Bus: Azure Service Bus was used as a message queue and a distributed, durable log. When a new game is completed, a message containing the statistics payload is published to the Service Bus. This acted as the start of a saga.
Stateless Frontend Services: These services accepted raw statistics from Xbox clients, published them to the Service Bus, and triggered the saga processing pipelines.
The separation into game and player grains, combined with distributed cloud storage, provided a scalable foundation that could process thousands of simultaneous games and millions of concurrent player updates.
Saga Application
The team applied the Saga Pattern to manage the complex updates needed for player statistics across multiple partitions.
The typical sequence was:
Aggregation: After a game session ended, the Game Grain aggregated statistics from the participating players.
Saga Initiation: A message was logged in the Azure Service Bus indicating the start of a saga for updating statistics for that game.
Sub-requests to Player Grains: For each player in the game (up to 32), the Game Grain sent individual update requests to their corresponding Player Grains. Each Player Grain then updated its player's statistics in Azure Table Storage.
Logging Progress: Successful updates and any errors were recorded through the durable messaging system, ensuring that no state was lost even if a process crashed.
Completion or Failure: If all player updates succeeded, the saga was considered complete. If any player update failed (for example, due to a temporary Azure storage issue), the saga would not be rolled back but would move into a forward recovery phase.
Through this structure, the team could ensure that updates were processed independently per player without relying on traditional ACID transactions across all player partitions.
Forward Recovery Strategy
Rather than using traditional backward recovery (rolling back completed sub-transactions), the Halo 4 team implemented forward recovery for their statistics sagas.
The main reasons for choosing forward recovery are as follows:
User Experience: Players who had their stats updated successfully should not see those stats suddenly disappear if a rollback occurred. Rolling back visible, successfully processed data would create a confusing and poor experience.
Operational Efficiency: Retrying only the failed player updates was more efficient than undoing successful writes and restarting the entire game processing.
Here’s how forward recovery works:
If a Player Grain failed to update its statistics (for example, due to storage partition unavailability or quota exhaustion), the system recorded the failure but did not undo any successful updates already completed for other players.
The failed update was queued for a retry using a back-off strategy. This allowed time for temporary issues to resolve without overwhelming the storage system with aggressive retries.
Retried updates were required to be idempotent. That is, repeating the update operation would not result in duplicated statistics or corruption. This was achieved by relying on database operations that safely applied incremental changes or overwrote fields as necessary.
Successful retries eventually brought all player records to a consistent state, even if it took minutes or hours to do so after the original game session ended.
By using forward recovery and designing for idempotency, the Halo 4 backend was able to achieve high availability.
Conclusion
As systems grow to support millions of users, traditional database models that rely on centralized transactions and strong ACID guarantees begin to break down.
The Halo 4 engineering team’s experience highlighted the urgent need for a new approach: one that could handle enormous scale, tolerate failures gracefully, and still maintain consistency across distributed data stores.
The Saga Pattern provided an elegant and practical solution to these challenges. By decomposing long-lived operations into sequences of sub-transactions and compensating actions, the team was able to build a system that prioritized availability, resilience, and operational correctness without relying on expensive distributed locking or rigid coordination protocols.
The lessons learned from this system apply broadly, not only to gaming infrastructure but to any domain where distributed operations must maintain reliability at massive scale.
References:
SPONSOR US
Get your product in front of more than 1,000,000 tech professionals.
Our newsletter puts your products and services directly in front of an audience that matters - hundreds of thousands of engineering leaders and senior engineers - who have influence over significant tech decisions and big purchases.
Space Fills Up Fast - Reserve Today
Ad spots typically sell out about 4 weeks in advance. To ensure your ad reaches this influential audience, reserve your space now by emailing sponsorship@bytebytego.com.
© 2025 ByteByteGo
548 Market Street PMB 72296, San Francisco, CA 94104
Unsubscribe
by "ByteByteGo" <bytebytego@substack.com> - 11:36 - 6 May 2025 -
[Save the Date] Spend Connect at SAP Connect
October 6–8, 2025 | Las Vegas, NV USA
Introducing:
Spend Connect at SAP Connect
As a spend and finance management professional, you have learned to fuel your transformation with solutions to strategically manage spend across all categories.
This year, you’ll have an opportunity to join us at a brand new event as we expand our business suite to bring out the best in your business. Experience live demos and real-world case studies tailored to finance and spend management applications, and get an inside look at how SAP helps you work smarter and adapt faster. In addition, you will see how SAP Business Suite brings AI, data, and applications together to deliver real results across your industry and organization.
Spend Connect at SAP Connect
Mark your calendar for Spend Connect at SAP Connect, taking place October 6–8, 2025 in Las Vegas, Nevada.
It’s part of SAP Connect, our new annual event designed for business leaders and practitioners looking to maximize impact and resilience across their business in an unpredictable world. You’ll connect with peers and partners, experience live demos tailored to your priorities, and discover how SAP empowers you to move forward with confidence — no matter what lies ahead.
Registration for Spend Connect at SAP Connect opens June 2025.
Sign up now to be notified about our event!
SAP (Legal Disclosure | SAP)
This e-mail may contain trade secrets or privileged, undisclosed, or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying, or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation.You are receiving this e-mail for one or more of the following reasons: you are an SAP customer, you were an SAP customer, SAP was asked to contact you by one of your colleagues, you expressed interest in one or more of our products or services, or you participated in or expressed interest to participate in a webinar, seminar, or event. SAP Privacy Statement
This e-mail was sent to you on behalf of the SAP Group with which you have a business relationship. If you would like to have more information about your Data Controller(s) please click here to contact webmaster@sap.com.
This promotional e-mail was sent to you by SAP Global Marketing and provides information on SAP's products and services that may be of interest to you. If you would prefer not to receive such e-mails from SAP in the future, please click on the Unsubscribe link.
To ensure you continue to receive SAP related information properly please add sap@mailsap.com to your address book or safe senders list.
by "SAP Connect" <sap@mailsap.com> - 10:05 - 6 May 2025
