Archives
-
Why Do We Need a Message Queue?
Why Do We Need a Message Queue?
Open in app or online This is a sneak peek of today’s paid newsletter for our premium subscribers. Get access to this issue and all future issues - by subscribing today.
Latest articles
If you’re not a subscriber, here’s what you missed this month.
"I Was Under Leveled!" — Avoiding the Tragedy of Making Only $500k a Year
Network Protocols behind Server Push, Online Gaming, and Emails
To receive all the full articles and support ByteByteGo, consider subscribing:
In this issue, we’re diving deep into a widely-used middleware: the message queue.
Message queues have a long history. They are often used for communication between different systems. Figure 1 illustrates the concept of a message queue by comparing it to how things work at Starbucks.
At Starbucks, the cashier takes the order and collects money, then they write the customer’s name on a coffee cup to hand over to the next step. The coffee maker picks up the order and the cup and makes coffee. The customer then picks up the coffee at the counter. The three steps work asynchronously. The cashier just drops the order in the form of a coffee cup and does not wait for its completion. The coffee maker just drops the completed coffee on the counter and does not wait for the customer to pick it up.
When you place an order at Starbucks, the cashier takes the order and scribbles your name on a cup and moves to the next customer. A barista then picks up the cup, prepares your drink, and leaves it for you to collect. The beauty of this process is that each step operates independently. It is much like an asynchronous system.
Figure 1 Starbucks as an analogy for message queues This asynchronous processing, where each step doesn’t have to wait for the previous one to complete, significantly increases the throughput of the system. For instance, the cashier doesn’t wait for your drink to be made before taking another order.
A Message Queue Example
Now, let’s shift our focus to a real-world example: flash sales in e-commerce. Flash sales can strain systems due to surge in user activity. Many strategies are employed to manage this demand, and message queues often play a pivotal role in backend optimizations.
A simplified eCommerce flash sale architecture is listed in Figure 2.
Steps 1 and 2: A customer places an order to the order service.
Step 3: Before processing the payment, the order service reserves the selected inventory.
Step 4: The order service then sends a payment instruction to the payment service. The payment service fans out to 3 services: payment channels, notifications, and analytics.
Steps 5.1 and 6.1: The payment service sends the payment instruction to the payment channel service. The payment channel service talks to external PSPs (Payment Service Providers) to finalize the transaction.
Steps 5.2 and 6.2: The payment service sends a notification to the notification service, which then sends a notification to the customer via email or SMS.
Step 5.3: The payment service sends transaction details to the analytics service.
Figure 2 A simplified eCommerce flash sale architecture A key takeaway here is that a seamless user experience is crucial during flash sales. To maintain service responsiveness despite high traffic, message queues can be integrated at multiple stages to ensure optimal performance.
Benefits of Message Queues
Fan-out
The payment service sends data to three downstream services for different purposes: payment channels, notifications, and analytics. This fan-out method is like someone shouting a message across a room; whoever needs to hear it, does. The producer simply drops the message on the queue, and the consumers process the message at their own pace.
Asynchronous Processing
Drawing from the Starbucks analogy, just as the cashier doesn’t wait for the coffee to be made, the order service does not wait for the payments to finalize. The payment instruction is placed on the queue, and the customer is notified once it’s finalized.
Rate Limiting
In a flash sale, there can be tens of thousands of concurrent users placing orders simultaneously. It is crucial to strike a balance between accommodating eager customers and maintaining system stability. A common approach is to cap the number of incoming requests within a specific time frame to match the capacity of the system. Excess requests might be rejected or asked to retry after a short delay. This approach ensures the system remains stable and doesn’t get overwhelmed. For requests that make it through, message queues ensure they’re processed efficiently and in order. If one part of the system is momentarily lagging, the order isn’t lost. It’s held in the queue until it can be processed. This ensures a smooth flow even under pressure.
Decoupling
Our design uses message queues in various places. The overall architecture is different from the simplified version presented in Figure 2. Services interact with each other using well-defined message interfaces rather than depending tightly on each other. Each service can be modified and deployed independently. Each component can be developed in a different programming language. This brings flexibility to the architectural design.
Horizontal Scalability
Since the services are decoupled, we can scale them independently based on demand. Each service can serve in a different capacity, so we can scale based on their planned QPS (query per second) or TPS (transaction per second).
Message Persistence
Message queues can also be used as middleware that stores messages. If the upstream service crashes, the downstream service can always pick up the messages from the message queue to process. In this way, the recovery function is moved out of each service and becomes the responsibility of the message queue.
Batch Processing
Sometimes in the processing flow, we need to batch the data to get the summary. For example, when the payment service sends updates to the analytics service, the analytics service does not need to perform real-time updates but rather set up a tumbling window to process in batches. The batch processing is the requirement of the downstream services, so there is no need for the payment service to know about it, just drop the messages into the queue.
Message Ordering
In a flash sale, there is a limited number of inventory items. For example, a flash sale offers only 10 iPhones, but there are over 10,000 users who place the order. How do we decide on the order? Having a message queue to keep all the orders will have a natural order: The first 10 in the queue will get the iPhone.
In Figure 3 we put everything together, where the services are connected via message queues and decoupled. In this way, the architecture can achieve higher throughput.
Figure 3 Use message queues for flash sale architecture... Keep reading with a 7-day free trial
Subscribe to ByteByteGo Newsletter to keep reading this post and get 7 days of free access to the full post archives.
A subscription gets you:
An extra deep dive on Thursdays Full archive Many expense it with team's learning budget Like
Comment
Restack
© 2023 ByteByteGo
548 Market Street PMB 72296, San Francisco, CA 94104
Unsubscribe
by "ByteByteGo" <bytebytego@substack.com> - 11:37 - 10 Aug 2023 -
Ship code faster with unmatched detection accuracy of security risks
New Relic
August 2023
Ship code faster with unmatched detection accuracy of security risks See and secure every application you build and run, and ship code faster with New Relic interactive application security testing (IAST). Pinpoint vulnerabilities with unmatched detection accuracy and proof-of-exploit for faster remediation. Now in public preview!Learn more 
Unlock the full potential of Kubernetes with the New Relic Kubernetes operator. Simplify complex application management, deploy, scale, and upgrade effortlessly.
Discover how New Relic empowers businesses like DAZN and Intelligent Growth Solutions with comprehensive observability solutions—improving performance and reducing costs.
Useful Reads
Learn the top five tips to ensure accurate log forwarding, gain insights, and troubleshoot effectively. Read our blog post for a seamless log monitoring experience.
Upcoming User Groups New Relic meetups are back! Tap into the expertise of our local engineers and fellow users as they show you what’s new in observability and share best practices on how they are using New Relic.
Our theme will be ‘back to school’, where we’ll give you essential tips and shortcuts to help you prepare for the peak sales period ahead.
New Relic User Meetup: Manchester
Tuesday 19th September 2023 | 2:00pm - 5:30pm (BST)Sixes Cricket, The Corn Exchange, New Cathedral St, Manchester M4 3TR
Register here.
New Relic User Meetup: London
Wednesday 20th September 2023 | 2:00pm - 5:30pm (BST)Bounce in Shoreditch, 241 Old Street, London, EC1V 9EY
Register here.
Find all upcoming user group sessions here.Webinars
Join our expert from New Relic University, Liam Hurrell, on the 17th of August at 10 am BST / 11 am CEST, for a comprehensive introduction to New Relic APM 360, the cutting-edge solution for modern distributed applications. Prevent issues before they escalate and troubleshoot faster with integrated infrastructure monitoring, error user impact analysis, and distributed tracing.
17th August, 10:00 am - 11:00 am BST/ 11:00 am - 12:00 pm CEST.
Register here
New Relic in the News
Need help? Let's get in touch.
This email is sent from an account used for sending messages only. Please do not reply to this email to contact us—we will not get your response.
This email was sent to info@learn.odoo.com Update your email preferences.
For information about our privacy practices, see our Privacy Policy.
Need to contact New Relic? You can chat or call us at +44 20 3859 9190.
Strand Bridge House, 138-142 Strand, London WC2R 1HH
© 2023 New Relic, Inc. All rights reserved. New Relic logo are trademarks of New Relic, Inc
Global unsubscribe page.
by "New Relic" <emeamaketing@newrelic.com> - 06:41 - 10 Aug 2023 -
ขยายธุรกิจของคุณด้วย Industrial Edge Computing
Schneider Electric
ขยายธุรกิจของคุณด้วย Industrial Edge ComputingDear Abul,
อุตสาหกรรม 4.0 กำลังขับเคลื่อนคลื่นเทคโนโลยีไอทีที่ไม่เคยเกิดขึ้นมาก่อนในพื้นที่อุตสาหกรรม และการปรับใช้จำนวนมากเหล่านี้จะถูกวางไว้ที่เอดจ์ เนื่องจากผู้รวมระบบอุตสาหกรรมอาจไม่มีความสามารถภายในองค์กรที่จะจัดการกับความท้าทายที่เกิดจากอุตสาหกรรมเอดจ์ได้อย่างเต็มที่ Gartner คาดการณ์ว่า "50% ของผู้ให้บริการ OT จะสร้างความร่วมมือที่สำคัญกับผู้ให้บริการด้านไอทีเป็นศูนย์กลางสำหรับการนำเสนอ IoT"
ผู้ให้บริการโซลูชันไอทีที่ช่ำชองตระหนักดีว่าการแบ่งไซโลแบบดั้งเดิมระหว่างไอทีและ OT จะสร้างโอกาสใหม่ในการสร้างความแตกต่างและขยายธุรกิจของพวกเขา
ในเซสชันนี้ ผู้เชี่ยวชาญจาก AVEVA, Hewlett Packard Enterprise และ Schneider Electric หารือเกี่ยวกับ: โอกาสสำหรับผู้ให้บริการโซลูชันด้านไอทีเอดจ์อุตสาหกรรม การประมวลผลเอดจ์อุตสาหกรรมใหม่ การออกแบบอ้างอิงเพื่อช่วยอำนวยความสะดวกในการเปลี่ยนแปลงทางดิจิทัลของลูกค้า และวิธีที่แพลตฟอร์ม Schneider Electric Exchange นำพันธมิตรด้าน IT/OT มารวมกันเพื่อให้ได้มาซึ่งธุรกิจใหม่
© 2023 Schneider Electric. All Rights Reserved. Schneider Electric is a trademark and the property of Schneider Electric SE, its subsidiaries and affiliated companies. All other trademarks are the property of their respective owners.
by "Noe Noe OO, Schneider Electric" <Marcom.thailand@se.com> - 09:04 - 9 Aug 2023 -
WooCommerce: You are now unsubscribed
WooCommerce
We have removed your email address from our list.
We're sorry to see you go.
Was this a mistake? Did you forward one of our emails to a friend, and they clicked the unsubscribe link not realizing they were in fact unsubscribing you from this list? If this was a mistake, you can re-subscribe at:
SubscribeFor questions or comments, please contact us at:
info@woocommerce.com
by "WooCommerce" <noreply@woocommerce.com> - 03:50 - 9 Aug 2023 -
Prep your business for the future
Prep your business for the future
Staff Picks
Hi there,
It’s Yida from Woo. This month, I want to share some of my favorite solutions to help you engage with, convince, and convert more shoppers. Ready to kick things up a notch in the second half of 2023? Let's go!
Yida
Woo
Popular
Tiered Pricing Table
Display tiered pricing tables to instantly show shoppers how much they can save – and close more sales.
You can offer dynamic discounts with rules for categories, selected products, or even specific customers. Compatible with Woo Subscriptions, Woo Product Bundles, and Elementor.
New to Woo
Really Simple Featured Video
Engage visitors by adding featured video support to your products, categories, and groups.
Upload videos or embed them from YouTube, Dailymotion, and Vimeo in a snap. Take control of the aspect ratio, playback, and picture-in-picture options. Lightweight and intuitive, with no unnecessary code bloat.
WooCommerce Pre-Orders
Sell items before they're available 
Must-have
Product Vendors
Create my own marketplace Popular
Datalayer for WooCommerce
Use data to optimize my store Popular
Looking for help building your dream store?
Run your store from anywhere
This email was sent to info@learn.odoo.com. You're receiving it because you signed up for updates when creating an account or purchasing a product subscription on WooCommerce.com.
No longer wish to hear from us? Unsubscribe from marketing emails or all email communication. You may still receive transactional and/or account-related emails from WooCommerce.com.
WooCommerce, Inc. is located at 60 29th St #343, San Francisco, CA 94110, U.S.A.
View an online version of this email.
© 2023 WooCommerce, Inc. – an Automattic company.
by "Yida at WooCommerce" <support+marketing@woocommerce.com> - 12:17 - 9 Aug 2023 -
[Report] Drowning in threats? Track the right vulnerability metrics
New Relic
.png)
Are you looking to enhance your vulnerability management program? Engineering teams often struggle with ineffective operational and technical metrics around security.
To address this, Gartner® published a report outlining the importance of outcome-driven metrics aligned with risk and business objectives. The report includes key findings, recommendations, and in-depth analysis. It provides valuable insights into designing vulnerability management metrics that cater to organizational stakeholders beyond security, incorporating business-specific risk and performance indicators.
By aligning metrics with threat, asset, and business context, you can improve remediation and mitigation efforts while demonstrating measurable business value.
Get the Report Need help? Let's get in touch. This email is sent from an account used for sending messages only. Please do not reply to this email to contact us—we will not get your response.
by "New Relic" <emeamarketing@newrelic.com> - 05:36 - 8 Aug 2023 -
ขยายบริการพลังงานที่มีการจัดการของคุณได้อย่างง่ายดาย
Schneider Electric
รับคำแนะนำทีละขั้นตอนสำรวจข้อเสนอซอฟต์แวร์และบริการดิจิตอลของเรา เพื่อค้นหาบริการด้านพลังงานที่มีการจัดการที่สอดคล้องกับธุรกิจของคุณได้อย่างง่ายดาย คู่มืออิเล็กทรอนิกส์ที่จำเป็นฉบับใหม่ของเราให้รายละเอียดทั้งหมด เพื่อให้คุณสามารถรวมคู่มือที่ถูกต้องได้อย่างง่ายดาย
ปรับใช้อย่างง่ายดายด้วยโซลูชันแบบเบ็ดเสร็จระบบนิเวศแบบบูรณาการของเราช่วยลดความยุ่งยากในการออกแบบ การทำงานร่วมกันของระบบ และการปรับใช้แอปพลิเคชัน Edge Computing ที่มีความสำคัญต่อธุรกิจ ทำให้ไซต์ระยะไกลหลายแห่งของคุณออนไลน์ได้ตรงเวลาและอยู่ในงบประมาณ
สร้างกระแสรายได้ที่เกิดซ้ำได้อย่างง่ายดายรับความมั่นใจในการสร้างแนวปฏิบัติ Managed Power Services ที่ประสบความสำเร็จโดยทำโปรแกรม Edge Software & Digital Services เรียนรู้ว่าข้อเสนอของเราสามารถยกระดับความสามารถของทีมและความต้องการในตลาดได้อย่างไรลดความซับซ้อนในการออกแบบและปรับใช้โซลูชันสำหรับลูกค้าของคุณลงทะเบียนสำหรับ mySchneider IT Solutions Partner Program ที่ได้รับรางวัล© 2023 Schneider Electric, All Rights Reserved. Schneider Electric trademarks are owned by Schneider Electric or its affiliated companies in the United States and other countries. All other trademarks are property of their respective owners.
by "APC by Schneider Electric" <reply@se.com> - 11:00 - 7 Aug 2023 -
Driver Monitoring System at the Wheel: Elevating Safety and Efficiency Across Industries
Driver Monitoring System at the Wheel: Elevating Safety and Efficiency Across Industries
Use Cases Across Industries
Emergency Services
Safeguarding lives, enhancing efficiency with DMS in Emergency Services.
Chemical Transportation
Proactive safety for hazardous chemical transport with real-time monitoring of drivers, and vehicles.
Transportation and Logistics
Empowering safe logistics with DMS for proactive compliance & driver safety.
To know more and dive deeper
Uffizio Technologies Pvt. Ltd., 4th Floor, Metropolis, Opp. S.T Workshop, Valsad, Gujarat, 396001, India
by "Sunny Thakur" <sunny.thakur@uffizio.com> - 08:00 - 7 Aug 2023 -
Introducing Woo Express — all-in-one ecommerce
Introducing Woo Express — all-in-one ecommerce
Try it free for 14 days, no credit card required 
Hi there,
Introducing Woo Express – everything you need for a successful online store, all in one place.
Woo Express is a WooCommerce plan with hosting, store management tools, and marketing features built in. It's all-in-one ecommerce for brands of all sizes! Launch a beautiful and functional store in days, not months, and get ready to grow.
With Woo Express, you get:
- Free priority support via email and live chat.
- Code-free site editing.
- Beautiful modern themes.
- Built-in payment processing, live shipping rates, and order tracking.
- Email, advertising, and social marketing tools.
- Plus, many more business-boosting features.
The best part? Unlike proprietary platforms that own your store and data, Woo Express is open source. Run your store however you like, customize every part of it, and even move it — it's yours.
From your first product to your 10,000th sale, Woo Express comes with everything you need to build a successful business — all in one place.
Try Woo Express for free for 14 days, no credit card required.
Get ready to own your future. Happy selling!
The Woo TeamThis email was sent to info@learn.odoo.com. You're receiving it because you opted in when creating an account or purchasing a product subscription on WooCommerce.com.
No longer wish to hear from us? Unsubscribe from marketing emails or all email communication. You may still receive transactional and/or account-related emails from WooCommerce.com.
View an online version of this email.
WooCommerce, Inc. is located at 60 29th St #343,
San Francisco, CA 94110, U.S.A.
© 2023 WooCommerce, Inc. – an Automattic company.
by "Woo" <support+marketing@woocommerce.com> - 01:50 - 7 Aug 2023 -
Hotels in the 2030s: Perspectives from Accor’s C-suite
Share this email
Share these insights
This email contains information about McKinsey’s research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to The Next Normal newsletter list.
Copyright © 2023 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey & Company" <publishing@email.mckinsey.com> - 10:27 - 7 Aug 2023 -
Taking the pulse of social media marketing: A leader’s guide
Share this email
Brought to you by Liz Hilton Segel, chief client officer and managing partner, global industry practices, & Homayoun Hatami, managing partner, global client capabilities
Social media platforms seem to be everywhere all at once, to borrow from the title of a popular movie. With an estimated 4.9 billion users around the world and younger generations using it as their shopping channel of choice, social media can be a critical growth and marketing tool for companies. Leaders who use social media are viewed favorably by employees and external stakeholders alike. Yet fewer than half of top CEOs have a strong digital presence, and despite investing heavily in social media marketing, executives are not fully convinced of its positive impact. This week, let’s explore how social media can—and cannot—benefit your organization.
About 92 percent of US companies use social media for marketing. But given a burgeoning number of social media platforms and users, organizations may find it difficult to develop a cohesive social media strategy. We find that four primary functions of social media are vital for businesses: monitor, respond, amplify, and lead consumer behavior. For example, 40 percent of consumers who make a complaint expect brands to respond within an hour. That may seem unrealistic, but companies meeting that high standard—currently, only about 50 percent of enterprises—stand a better chance of affecting consumer behavior positively. It may also be possible to lead long-term changes in consumer behavior through product launches, targeted deals and offers, and input from customers on social media.
That’s the number of new users signing up to their first social media account every second. Can your customer service organization handle that kind of volume? Challenges you may encounter include varying customer expectations across different platforms, unpredictable service demands, and skill gaps among staff. “The quality of servicing on social media can have a significant impact on brand perception,” caution McKinsey senior partner Renny Thomas and colleagues. “Social content can go viral quickly, potentially turning small or isolated customer service issues into PR disasters.” Best practices for social media servicing include developing an end-to-end service strategy, defining workflows clearly, and deploying the appropriate technologies.
That’s an observation from McKinsey researchers in 2012, when they noted that the reason for the leaders’ discomfort may have been the perception that social media initiatives lacked a clear return on investment. In the years since then, social media marketing has gone well beyond the experimental stage, but organizations remain unsure of its impact—possibly because “many brands entered the social media front lines without a clear strategy,” according to the author of a Harvard Business Review article. To ensure that social media objectives align with broader marketing strategies, companies need to “coordinate data, tools, technology, and talent across multiple functions,” suggest the McKinsey experts.
Companies spend millions on celebrity product endorsements, which appear to influence customer choices. Influencer marketing does something similar by enabling consumers to collaborate with brands to promote products to their followers on social media. “It used to be the case that you had to be a big celebrity, or you had to have a big name behind you to get one of these deals,” says Katie Freiberg, head of growth marketing at lingerie manufacturer ThirdLove, in a discussion with McKinsey partners Dianne Esber and Jane Wong. “But now anyone can be an influencer. It’s really all about having that following.” Larger companies that form influencer partnerships may find it hard to create the authenticity that smaller brands often do more easily, but managed properly, these arrangements can “supplement and sometimes overshadow traditional marketing campaigns,” according to Esber and Wong.
Negative social media content—whether the result of hacking, customer comments, or reactions to your own organization’s posts—can destroy a brand’s reputation. A strong culture of security and risk management can help counter serious cyberthreats, says Julia Houston, chief strategy and marketing officer at credit bureau Equifax, in a conversation with McKinsey senior partner Ida Kristensen. “Culture is the thing that separates good security from great security,” Houston says. On a lighter note, brands can harm themselves all on their own: beware of “boring” posts, warns the author of an Entrepreneur magazine article. “Giving a fresh perspective on an old topic or going against the status quo is what gets noticed. The last thing you want to be is forgettable.”
Lead socially.
— Edited by Rama Ramaswami, senior editor, New York
Share these insights
Did you enjoy this newsletter? Forward it to colleagues and friends so they can subscribe too. Was this issue forwarded to you? Sign up for it and sample our 40+ other free email subscriptions here.
This email contains information about McKinsey’s research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to the Leading Off newsletter.
Copyright © 2023 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey Leading Off" <publishing@email.mckinsey.com> - 02:24 - 7 Aug 2023 -
Recent reader faves
On Point
Brought to you by Liz Hilton Segel, chief client officer and managing partner, global industry practices, & Homayoun Hatami, managing partner, global client capabilities
We hope you’re enjoying On Point’s insights about what the daily news means for your business. After today, we’ll be taking a brief hiatus before returning to your inbox on August 21. In the meantime, make sure you haven’t missed these recent popular issues:
If you have friends or colleagues who might enjoy On Point, consider forwarding this email to them or sharing it on LinkedIn, Twitter, or Facebook. They can sign up for this or any of our 40+ other free email subscriptions at mckinsey.com/subscriptions. (And you might also want to revisit that page to see our full newsletter lineup.)
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to the On Point newsletter.
Copyright © 2023 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey On Point" <publishing@email.mckinsey.com> - 12:33 - 7 Aug 2023 -
Revisit our readers' favorite charts
The Week in Charts
Share this email
Thanks for reading The Week in Charts, your source for visuals that help make sense of the latest news and business trends. As summer reaches its peak on our side of the world, we’ll be taking a two-week break from our typical send schedule, returning to your inbox on August 19.
While we’re away, look back at some recent charts that resonated with our readers:If you have friends or colleagues who might enjoy The Week in Charts, consider forwarding this email to them or sharing it on LinkedIn, Twitter, or Facebook. They can sign up for this or any of our 40+ other free email subscriptions at mckinsey.com/subscriptions. (And you might also want to revisit that page to see our full newsletter lineup.)
Share these insights
Did you enjoy this newsletter? Forward it to colleagues and friends so they can subscribe too. Was this issue forwarded to you? Sign up for it and sample our 40+ other free email subscriptions here.
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to The Week in Charts newsletter.
Copyright © 2023 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey Week in Charts" <publishing@email.mckinsey.com> - 03:50 - 5 Aug 2023 -
EP71: CI/CD Pipeline Explained in Simple Terms
EP71: CI/CD Pipeline Explained in Simple Terms
Open in app or online This week’s system design refresher:
Why Google and Meta Put Billion Lines of Code In 1 Repository? (Youtube video)
CI/CD Pipeline Explained in Simple Terms
What does API gateway do?
Docker vs. Kubernetes. Which one should we use?
Cloud Native Anti Patterns
QA Wolf gets you to 80% automated test coverage in 4 months (Sponsored)
Manually end-to-end testing? Here's why you should switch to automation:
Test all your user flows in 3 minutes
Freedom to run your test suite whenever you want
Increased confidence in releases
Time to focus on other priorities
The list goes on...
So why wouldn't you? Time and resources. In-house teams typically take 2 years to reach high coverage. And you need at least a few automation engineers to build, run, and maintain a test suite.
Not anymore.
QA Wolf offers a cost-effective approach to getting 80% test coverage in just 4 months — even for the most complex web apps.
And, they include unlimited parallel runs on their testing infrastructure + 24-hour maintenance and triage. Schedule a demo to learn more.
PS: QA Wolf has a 4.8/5 ⭐️ rating on G2 - reviewed by companies with 51-1000+ employees.
Why Google and Meta Put Billion Lines of Code In 1 Repository?
CI/CD Pipeline Explained in Simple Terms
Section 1 - SDLC with CI/CD
The software development life cycle (SDLC) consists of several key stages: development, testing, deployment, and maintenance. CI/CD automates and integrates these stages to enable faster, more reliable releases.
When code is pushed to a git repository, it triggers an automated build and test process. End-to-end (e2e) test cases are run to validate the code. If tests pass, the code can be automatically deployed to staging/production. If issues are found, the code is sent back to development for bug fixing. This automation provides fast feedback to developers and reduces risk of bugs in production.
Section 2 - Difference between CI and CD
Continuous Integration (CI) automates the build, test, and merge process. It runs tests whenever code is committed to detect integration issues early. This encourages frequent code commits and rapid feedback.
Continuous Delivery (CD) automates release processes like infrastructure changes and deployment. It ensures software can be released reliably at any time through automated workflows. CD may also automate the manual testing and approval steps required before production deployment.
Section 3 - CI/CD Pipeline
A typical CI/CD pipeline has several connected stages:
- Developer commits code changes to source control
- CI server detects changes and triggers build
- Code is compiled, tested (unit, integration tests)
- Test results reported to developer
- On success, artifacts are deployed to staging environments
- Further testing may be done on staging before release
- CD system deploys approved changes to productionLatest articles
If you’re not a subscriber, here’s what you missed this month.
"I Was Under Leveled!" — Avoiding the Tragedy of Making Only $500k a Year
Network Protocols behind Server Push, Online Gaming, and Emails
To receive all the full articles and support ByteByteGo, consider subscribing:
What does API gateway do?
The diagram below shows the detail.
Step 1 - The client sends an HTTP request to the API gateway.
Step 2 - The API gateway parses and validates the attributes in the HTTP request.
Step 3 - The API gateway performs allow-list/deny-list checks.
Step 4 - The API gateway talks to an identity provider for authentication and authorization.
Step 5 - The rate limiting rules are applied to the request. If it is over the limit, the request is rejected.
Steps 6 and 7 - Now that the request has passed basic checks, the API gateway finds the relevant service to route to by path matching.
Step 8 - The API gateway transforms the request into the appropriate protocol and sends it to backend microservices.
Steps 9-12: The API gateway can handle errors properly, and deals with faults if the error takes a longer time to recover (circuit break). It can also leverage ELK (Elastic-Logstash-Kibana) stack for logging and monitoring. We sometimes cache data in the API gateway.
Over to you:What’s the difference between a load balancer and an API gateway?
Do we need to use different API gateways for PC, mobile and browser separately?
Docker vs. Kubernetes. Which one should we use?
What is Docker ?
Docker is an open-source platform that allows you to package, distribute, and run applications in isolated containers. It focuses on containerization, providing lightweight environments that encapsulate applications and their dependencies.
What is Kubernetes ?
Kubernetes, often referred to as K8s, is an open-source container orchestration platform. It provides a framework for automating the deployment, scaling, and management of containerized applications across a cluster of nodes.
How are both different from each other ?
Docker: Docker operates at the individual container level on a single operating system host.
You must manually manage each host and setting up networks, security policies, and storage for multiple related containers can be complex.
Kubernetes: Kubernetes operates at the cluster level. It manages multiple containerized applications across multiple hosts, providing automation for tasks like load balancing, scaling, and ensuring the desired state of applications.
In short, Docker focuses on containerization and running containers on individual hosts, while Kubernetes specializes in managing and orchestrating containers at scale across a cluster of hosts.
Over to you: What challenges prompted you to switch from Docker to Kubernetes for managing containerized applications?Guest post by Govardhana Miriyala Kannaiah.
Cloud Native Anti Patterns
By being aware of these anti-patterns and following cloud-native best practices, you can design, build, and operate more robust, scalable, and cost-efficient cloud-native applications.
Monolithic Architecture:
One large, tightly coupled application running on the cloud, hindering scalability and agilityIgnoring Cost Optimization:
Cloud services can be expensive, and not optimizing costs can result in budget overrunsMutable Infrastructure:
- Infrastructure components are to be treated as disposable and are never modified in place
- Failing to embrace this approach can lead to configuration drift, increased maintenance, and decreased reliabilityInefficient DB Access Patterns:
Use of overly complex queries or lacking database indexing, can lead to performance degradation and database bottlenecksLarge Containers or Bloated Images:
Creating large containers or using bloated images can increase deployment times, consume more resources, and slow down application scalingIgnoring CI/CD Pipelines:
Deployments become manual and error-prone, impeding the speed and frequency of software releasesShared Resources Dependency:
Applications relying on shared resources like databases can create contention and bottlenecks, affecting overall performanceUsing Too Many Cloud Services Without a Strategy:
While cloud providers offer a vast array of services, using too many of them without a clear strategy can create complexity and make it harder to manage the application.Stateful Components:
Relying on persistent state in applications can introduce complexity, hinder scalability, and limit fault tolerance
Over to you:
What anti-patterns have you faced in your cloud-native journey? How did you conquer them?Latest articles
Here are the latest articles you may have missed:
To receive all the full articles and support ByteByteGo, consider subscribing:
© 2023 ByteByteGo
548 Market Street PMB 72296, San Francisco, CA 94104
Unsubscribe
by "ByteByteGo" <bytebytego@substack.com> - 11:37 - 5 Aug 2023 -
The top trends in tech
Share this email
The first half of 2023 has seen a resurgence of enthusiasm about technology’s potential to catalyze progress in business and society—and generative AI (gen AI) deserves much of the credit for this revival. What are the top trends executives should pay attention to as the second half of the year unfolds? In our first featured story this month, Michael Chui, Mena Issler, Roger Roberts, and Lareina Yee—representing views from McKinsey’s Technology Council—identify the 15 most significant technology trends developing today and highlight how businesses can capitalize on them to propel innovation and achieve sustainable growth. Our second featured story presents findings from the latest annual McKinsey Global Survey on the current state of AI, which confirms the explosive growth of gen AI tools. Other highlights in this month’s issue include the following topics:
•
how gen AI and other forces are changing the way people work and the work people do
•
the effects of the COVID-19 pandemic on real estate in “superstar” cities
•
why middle managers are the true center of organizations
•
four must-dos today for retail executives
Generative AI and the future of work in America
Multiple forces are set to fuel growth in certain occupations and erode jobs in others.
Download the full report
Empty spaces and hybrid places: The pandemic’s lasting impact on real estate
Real estate in the world’s superstar cities has not kept up with shifts in behavior caused by the pandemic. The cities’ vibrancy is at risk, and they will have to adapt.
Explore the full report
Middle managers are the heart of your company
Stop thinking of middle management as a way station. Instead, make it a destination.
Offer the right rewards
Retail reset: A new playbook for retail leaders
In light of unprecedented industry disruption, a retailer’s actions today could determine whether it spends the next 20 years as a leader or a laggard. Here are four must-dos for retail executives.
Think creatively
Making sense of generational stereotypes at work
Boomers to the left of you; Gen Z to the right. Are we really so bafflingly different?
People are people
Digital and AI transformation’s new playbook
Chatbots, cloud computing, generative AI—a company’s digital transformation has never been more paramount. Are you rewired to win?
Dive deeper
McKinsey Explainers
Find direct answers to complex questions, backed by McKinsey’s expert insights.
Learn more
McKinsey Themes
Browse our essential reading on the topics that matter.
Get up to speed
McKinsey on Books
Explore this month’s best-selling business books prepared exclusively for McKinsey Publishing by Circana.
See the lists
McKinsey Chart of the Day
See our daily chart that helps explain a changing world—as we strive for sustainable, inclusive growth.
Dive in
McKinsey Classics
What’s the link between “moments of truth” and customers’ purchase decisions? Read our 2006 classic “The ‘moment of truth’ in customer service” to learn more.
Rewind
The Daily Read
Our Daily Read newsletter highlights an article a day, picked by our editors.
Subscribe now— Edited by Eleni Kostopoulos, managing editor, New York
Share these insights
Did you enjoy this newsletter? Forward it to colleagues and friends so they can subscribe too. Was this issue forwarded to you? Sign up for it and sample our 40+ other free email subscriptions here.
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you are a registered member of our Monthly Highlights newsletter.
Copyright © 2023 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey Highlights" <publishing@email.mckinsey.com> - 11:15 - 5 Aug 2023 -
The future of hotels: Customized experiences, sustainable practices
Share this email
The Next Normal | Explore the hotels of tomorrow 
Share these insights
Did you enjoy this email? Forward it to colleagues and friends so they can subscribe too. Was this alert forwarded to you? Sign up for it and sample our 40+ other free email subscriptions here.
This email contains information about McKinsey’s research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to The Next Normal newsletter list.
Copyright © 2023 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey & Company" <publishing@email.mckinsey.com> - 02:45 - 4 Aug 2023 -
Sustainable and inclusive growth: Gen AI and the future of work
Share this email
New from McKinsey & Company
Sustainable and inclusive growth: A weekly briefing
Get updated 
Share these insights
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to our McKinsey Global Institute alert list.
Copyright © 2023 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey & Company" <publishing@email.mckinsey.com> - 01:15 - 4 Aug 2023 -
Our 2023 summer reading guide is here
On Point
Brought to you by Liz Hilton Segel, chief client officer and managing partner, global industry practices, & Homayoun Hatami, managing partner, global client capabilities
•
Low levels of reading. Nearly a third of American students spend very little or no time reading for pleasure, and the average 13-year-old’s reading scores have fallen to their lowest level in about two decades, according to a recent National Assessment of Educational Progress report. Experts point to technology and social media as among the largest factors that may be contributing to a lack of reading. The average age that a child gets their first mobile phone is now 11, a 2022 Stanford Medicine study revealed. That’s an age where reading is normally encouraged. [The Hill]
•
What to read. Our world is both concerning—with executives navigating geopolitical tensions, economic setbacks, and other challenges—and exciting, with the advent of generative AI, which promises to shift the way we work. Our annual summer reading list reflects today’s evolving times, with selections by leaders from around the world, including picks from McKinsey global managing partner Bob Sternfels, chief marketing officer Tracy Francis, managing partner of global client capabilities Homayoun Hatami, and senior partner Rodney Zemmel. We also provide 11 additional curated reading lists.
— Edited by Vanessa Burke, editor, Atlanta
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to the On Point newsletter.
Copyright © 2023 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey On Point" <publishing@email.mckinsey.com> - 12:33 - 4 Aug 2023
