Archives
-
Returning to the office? Focus more on practices and less on the policy
Share this email
Brought to you by Alexis Krivkovich (Alexis_Krivkovich@McKinsey.com) and Dana Maor (Dana_Maor@McKinsey.com), global leaders of McKinsey’s People & Organizational Performance Practice
New from McKinsey Quarterly
This email contains information about McKinsey’s research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to our McKinsey Quarterly alert list.
Copyright © 2025 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey & Company" <publishing@email.mckinsey.com> - 02:54 - 20 Feb 2025 -
Non-Functional Requirements: The Backbone of Great Software - Part 2
Non-Functional Requirements: The Backbone of Great Software - Part 2
Forwarded this email? Subscribe here for moreLatest articles
If you’re not a subscriber, here’s what you missed this month.
To receive all the full articles and support ByteByteGo, consider subscribing:
Non-functional requirements (NFRs) are as critical as functional requirements because they define a system's qualities and operational parameters.
Functional requirements specify what a software product should do (for example, “users must be able to log in”). However, non-functional requirements define how well it must accomplish these tasks under real-world conditions (for example, “the login process should respond within two seconds under peak load” or “all user credentials must be encrypted and stored securely”).
Together, functional and non-functional requirements create a foundation for building great software systems.
In Part 1 of this topic, we also looked at trade-offs in non-functional requirements and the architectural impact of these requirements. Some key learning points from Part 1 were as follows:
Non-functional requirements determine a system's performance in areas such as scalability, response time, performance, security, etc., ensuring it meets quality standards and user expectations beyond mere functionality.
Functional requirements focus on what the system should do, while non-functional requirements define how well it operates under real-world conditions.
Optimizing one NFR (for example, performance) can negatively impact others (for example, maintainability), requiring a balanced approach based on project goals and constraints.
Non-functional needs (like scalability and availability) heavily influence architecture choice. For example, choosing microservices for high availability or plugin-based designs for modifiability.
In this article (Part 2), we’ll go further and look at some of the most important NFRs that should be considered while building systems.
Key NFRs to Consider
Some key non-functional requirements that should be considered while designing an application are as follows:
Response Time/Latency...
Continue reading this post for free in the Substack app
Like
Comment
Restack
© 2025 ByteByteGo
548 Market Street PMB 72296, San Francisco, CA 94104
Unsubscribe
by "ByteByteGo" <bytebytego@substack.com> - 11:34 - 20 Feb 2025 -
[FREE REGISTRATION] SAP Sapphire Virtual
Get the “best‑of” SAP Sapphire!
Register now for our free online event.Experience the best in technology, innovation, and new ideas.
Join us for SAP Sapphire Virtual to learn about our latest breakthroughs in applications, data, AI, and more.
What you can expect:
•
Livestreamed keynotes and exclusive interviews
•
Breaking news and announcements
•
Top-rated demos, road maps, and strategy talks
•
Live chats, Q&As, polls
•
And so much more!
Are you ready to
bring out the best in your business?
Looking for more information
about our SAP Sapphire event series?
SAP (Legal Disclosure | SAP)
This e-mail may contain trade secrets or privileged, undisclosed, or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying, or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation.You are receiving this e-mail for one or more of the following reasons: you are an SAP customer, you were an SAP customer, SAP was asked to contact you by one of your colleagues, you expressed interest in one or more of our products or services, or you participated in or expressed interest to participate in a webinar, seminar, or event. SAP Privacy Statement
This e-mail was sent to you on behalf of the SAP Group with which you have a business relationship. If you would like to have more information about your Data Controller(s) please click here to contact webmaster@sap.com.
This promotional e-mail was sent to you by SAP Global Marketing and provides information on SAP's products and services that may be of interest to you. If you would prefer not to receive such e-mails from SAP in the future, please click on the Unsubscribe link.
To ensure you continue to receive SAP related information properly please add sap@mailsap.com to your address book or safe senders list.
by "SAP Sapphire" <sap@mailsap.com> - 10:59 - 20 Feb 2025 -
API governance: The missing piece in your stack?
API governance: The missing piece in your stack?
Hi Md Abul,
The LEAP 2.0: API governance conference is happening next week (27 Feb), and the full agenda is now live.LEAP 2.0 is a fully curated conference featuring top experts handpicked to share valuable, real-world insights.
If you're building, scaling, or securing APIs, this free online event offers deep dives, live workshops, and case studies from Zeiss, Dynatrace, Grafana Labs, NorthWestern Mutual, Coforge, Solace, and many more.
Here’s some sessions you can’t miss:
🎤 Tyk AI studio workshop – Get hands-on with AI-powered API tools
🎤 Scaling APIM with GitOps – Multi-tenancy, Argo CD & Tyk Operator in action (Zeiss)
🎤 API observability – The missing link in your governance strategy (Dynatrace, Grafana Labs, Nivenly Foundation)
🎤 API governance & sustainability – Practical strategies for implementers🎤 Our series of lightning talks covering OpenAPI, JSON Schema, OpenID, AI, and sustainable API practices - quick and impactful!
Each session is designed to give you actionable insights that can be immediately applied to solve real-world challenges.
Plus, gain exclusive access to the API governance fundamentals certification to showcase your expertise in API management, security, and compliance.
Event details:
📅 Date: February 27, 2025
🕘 Time: 9 AM - 3 PM ET | 2 PM - 8 PM GMTSee you there!
The LEAP 2.0 team
Tyk, Huckletree 199 Bishopsgate, Broadgate, London, City of London EC2M 3TY, United Kingdom, +44 (0)20 3409 1911
by "Budhaditya Bhattacharya" <budha@tyk.io> - 06:00 - 20 Feb 2025 -
New Relic Certification is live!
New Relic
We are excited to announce that New Relic Certifications are now live for a range of professional role-based observability certifications.
Whether you're just starting your observability journey, or are a seasoned professional, we encourage you to take the certification exams available, use it as an opportunity to gain a certification, and become part of the New Relic Certified community. If successful in passing the exam you will be awarded the professional credential, and will be able to share it with your network and peers.
In preparation for the certifications, the instructor designed Certification Paths can support you to upskill on the New Relic platform before tackling the exam.
The following certifications are now available:- New Relic Verified Foundation - Foundation (NVF)
- New Relic Certified APM Practitioner - Associate (APA)
- New Relic Certified Performance Engineer - Professional (PEP)
- New Relic Certified Reliability Engineer - Professional (REP)
For more information on the exams, format and what you need to know before taking an exam, please click here.
Good Luck!
by "New Relic University" <certification@newrelic.com> - 05:07 - 20 Feb 2025 -
What are the benefits of AI guardrails?
On McKinsey Perspectives
by "Only McKinsey Perspectives" <publishing@email.mckinsey.com> - 01:48 - 20 Feb 2025 -
You still have time to register
Hi Abul, our invitation still stands.
We're excited to invite you from Odoo to our upcoming event, New Relic Now+.
This free virtual event will unveil the most transformative platform update yet, designed to enhance business impact and developer velocity.
Don’t miss out on this opportunity to gain valuable insights that could greatly benefit your organization.
ThomasConsultantNew Relic
If you don't want to hear from me again, please let me know.
by "Thomas Sampassa-Ilunga" <tilunga@newrelic.com> - 12:25 - 19 Feb 2025 -
How leaders are promoting resilience at work
On McKinsey Perspectives
Brought to you by Alex Panas, global leader of industries, & Axel Karlsson, global leader of functional practices and growth platforms
Welcome to the latest edition of Only McKinsey Perspectives. We hope you find our insights useful. Let us know what you think at Alex_Panas@McKinsey.com and Axel_Karlsson@McKinsey.com.
—Alex and Axel
•
Overcoming resistance to change. In an environment where disruptions—including the growing use of AI, geopolitical and economic shifts, and evolving employee expectations—are the norm, leaders should foster resilience and adaptability in themselves and in their employees. Yet this is easier said than done, McKinsey Senior Partners Aaron De Smet and Dana Maor and coauthors reveal. McKinsey research shows that when the path forward at work (or in life) is unclear, people tend to do what’s familiar or predictable, regardless of whether that approach is appropriate for the situation at hand.
•
The value of resilience. When a company’s people are resilient and adaptable, it can have profound effects on innovation, growth, and organizational health. Those who are both resilient and adaptable demonstrate better overall health and are more engaged at work than their peers, according to the McKinsey Health Institute. What can leaders do to foster these traits within their own workforces? See the four actions leaders can take to help their people weather the inevitable storms that lie ahead.
—Edited by Belinda Yu, editor, Atlanta
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to the Only McKinsey Perspectives newsletter, formerly known as Only McKinsey.
Copyright © 2025 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "Only McKinsey Perspectives" <publishing@email.mckinsey.com> - 01:33 - 19 Feb 2025 -
The Perfect Faceoff: NHL Teams vs. ERP Systems
The Perfect Faceoff: NHL Teams vs. ERP Systems
Hi MD Abul,
Hockey season is heating up, and just like the NHL, the ERP world is full of powerhouses, underdogs, and rising stars.
So, what if ERP systems were NHL teams? Would SAP be the Tampa Bay Lightning? Is Microsoft Dynamics 365 the Pittsburgh Penguins? And who’s the Montreal Canadiens of ERP?
We broke it all down in our latest blog, matching the biggest ERP systems with the biggest teams in hockey. Whether you're an NHL fan or just love a creative take on digital transformation, you won’t want to miss this one!
Let us know—did we get the comparisons right, or do you have a better match?
Best regards,
Eric Kimberling
by "Eric Kimberling" <eric.kimberling@thirdstage-consulting.com> - 01:07 - 19 Feb 2025 -
Let Us Handle Your Air Freight Needs with Care and Expertise From China To Mexico
Dear Mr./Ms.
We know life’s busy―so when you need to move your goods from China To Mexico, you want it fast, secure, great service and great rate.
Well, guess what? We’ve got your back!
Pls kindy find the following the Air rate for general cargo for FOB:ASLG FOB Rate - Week8 2025 AIR EXPORT Airline station Route Weight breaks FRQ T/T Remark Carrier Org Dest 100+ 300+ 500+ 1000+ HU PEK MEX PEK-MEX $3.06 $3.06 $2.72 $2.72 Daily Direct 1cbm>300kgs stackable BA/AF PEK MEX PEK-LHR/CDG-MEX $3.06 $3.06 $2.72 $2.72 Daily 4-5Days 1cbm>300kgs stackable NH PEK MEX PEK-NRT-MEX $3.06 $3.06 $2.72 $2.72 Daily 2-3Days 1cbm>300kgs stackable CZ PVG NLU PVG-NLU $4.00 $4.00 $4.00 $4.00 D146 Direct 1cbm>300kgs stackable K4 CGO NLU CGO-NLU / / / / D246 Direct 1cbm>300kgs stackable 5Y PVG GDL PVG-GDL $4.00 $4.00 $4.00 $4.00 D3 Direct 1cbm>300kgs stackable 5Y XMN NLU XMN-NLU / / / / D247 Direct 1cbm>300kgs stackable CA/CZ SZX NLU SZX-NLU $3.00 $3.00 $2.85 $2.85 D246 Direct 1cbm>300kgs stackable 5Y HKG NLU/GDL HKG-NLU/GDL $6.10 $6.10 $6.10 $6.10 TBA Direct Battery cargo CX HKG NLU/GDL HKG-NLU/GDL $6.10 $6.10 $6.10 $6.10 D24567 Direct Battery cargo CV HKG NLU/GDL HKG-NLU/GDL $6.10 $6.10 $6.10 $6.10 D1235 Direct Battery cargo
Pls noted that you have ready cargo,I will check the special air rate for case by case.Hope the above is helpful for you,if any progress,pls feel free to let me know.We are eager to explore potential collaboration and support your logistics requirements. Pls don’t hesitate to reach out for more details.
Best Regards
Vic Yang
Overseas Departments | Airsupply Logistics Limited
**Focus on international air cargo service, welcome for checking with me any time.**A: Floor 23th Bldg 13 South Park Baoneng Science and Technology Industrial Park,No.1 Qingxiang Road,Longgang Shenzhen,China
B: SZX/CAN/HKG/CSX/PVG/TSN/PEK
WCA ID: 104615 RA:28426 NVOCC: 09626
发件人： vic07@air-supply.biz发送时间： 2025-01-02 17:22收件人： info主题： Let Us Handle Your Air Freight Needs with Care and Expertise From China To MexicoDear Mr./Ms.
Happy New Year on 2025!!!Imagine this: Your critical cargo glides effortlessly from China’s busiest airports and lands precisely on schedule, handled with precision and care every step of the way, and efficient transportation with airlines.
My name is Vic from Airsupply, where we specialize in providing expert logistics solutions tailored to the needs of international trade.At Airsupply, we make this vision a reality, Here’s how we roll out the red carpet for your shipments:
Our Services:
- Extensive Network: Direct/Transfer routes from major Chinese airports (HKG/SZX/CAN/PVG/PEK/CGO) to MX/US/CA and Latin America, partnering with leading carriers (CZ/CA/HU/NH/OZ/K4/5Y/FX/CX/CV etc...).
- Global Certifications: IATA-certified, JCtrans Golden Member, WCA-approved.
- Strategic Expertise: Master loader at HKG Airport with exclusive airline advantages (CX/CV/5Y/FX/M7/OZ/KE/LO etc...).
- DG Cargo Handling: Safe and compliant transport of hazardous materials (Class 2/3/4/5/6/8/9) with IATA DGR approval.
- Integrated Logistics: Inland trucking, door-to-door DDU/DDP delivery, and seamless customs clearance for air and sea shipments.
- Value-Added Services: Cargo insurance, charter services, project cargo handling, custom brokerage, and trade show logistics expertise.
At the same time, we also need your company to provide us with the following services:
- Air/Sea Freight:Efficient transport from your country to China or other global ports.
- Inland Trucking/Delivery: Including DDU&DDP for door-to-door services.
- Customs Clearance:Seamless import and export customs clearance for both Air&Sea.
We look forward to exploring collaboration opportunities and supporting your logistics needs. Please feel free to contact me for more information.
Thank you for considering Airsupply. We hope to contribute to your global success.
If you need to learn more about us, please watch our YouTube video:https://youtu.be/euPxeTaVxv0
When you do not desire to receive future emails from us,pls let us know in time, and we will remove you from our mailing list. Thanks.
Best Regards
Vic Yang
Overseas Departments | Airsupply Logistics Limited
**Focus on international air cargo service, welcome for checking with me any time.**A: Floor 23th Bldg 13 South Park Baoneng Science and Technology Industrial Park,No.1 Qingxiang Road,Longgang Shenzhen,China
B: SZX/CAN/HKG/CSX/PVG/TSN/PEK
WCA ID: 104615 RA:28426 NVOCC: 09626
by "vic07@air-supply.biz" <vic07@air-supply.biz> - 11:23 - 18 Feb 2025 -
A healthy workplace starts with healthy workers
Five Fifty
Share this email
Enhanced employee health can boost engagement and productivity and has the potential to create up to $11.7 trillion in global economic value, say McKinsey’s Barbara Jeffery, Brooke Weddle, Jacqueline Brassey, and Shail Thaker. To learn more about how companies can support their people to help them thrive, check out the latest edition of the Five Fifty.
Share these insights
Did you enjoy this newsletter? Forward it to colleagues and friends so they can subscribe too. Was this issue forwarded to you? Sign up for it and sample our 40+ other free email subscriptions here.
This email contains information about McKinsey’s research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to our McKinsey Quarterly Five Fifty alert list.
Copyright © 2025 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey Quarterly Five Fifty" <publishing@email.mckinsey.com> - 04:23 - 18 Feb 2025 -
How Instagram Scaled Its Infrastructure To Support a Billion Users
How Instagram Scaled Its Infrastructure To Support a Billion Users
Forwarded this email? Subscribe here for moreCaching for Performance: Virtual Masterclass (Sponsored)
Insider tips for navigating caching strategies and tradeoffs
This masterclass shares pragmatic caching strategies for latency-sensitive use cases. We’ll explore a variety of caching options – how they’re architected, how that impacts their “sweet spot” and tradeoffs, as well as tips for working with them.
After this free 2-hour masterclass, you will know how to:
Decide among common caching approaches for reducing application latency
Understand the architecture decisions behind different caching approaches and how that impacts performance
Measure, monitor, and optimize the performance impact of your caching approach
Spot and diagnose the manifestations of common caching mistakes
This is a great opportunity to learn how caching strategies are evolving and how to best take advantage of them.
Disclaimer: The details in this post have been derived from Instagram Engineering Blog and other sources. All credit for the technical details goes to the Instagram engineering team. The links to the original articles are present in the references section at the end of the post. We’ve attempted to analyze the details and provide our input about them. If you find any inaccuracies or omissions, please leave a comment, and we will do our best to fix them.
Instagram was launched in 2010 by Kevin Systrom and Mike Krieger as a simple photo-sharing app. The app allowed users to instantly upload, edit, and share photos, applying filters that gave images a professional touch. Unlike other social media platforms at the time, Instagram focused exclusively on visual storytelling, making it an instant hit among smartphone users.
The platform's simplicity and mobile-first approach led to rapid adoption. Within just two months, Instagram had gained 1 million users, a milestone that took Facebook nearly a year to achieve. The app grew exponentially, reaching 10 million users in one year and 30 million users by early 2012.
By April 2012, Instagram had reached 50 million users and was attracting serious investment interest. Facebook acquired Instagram for $1 billion, a shocking move at the time, as the company had just 13 employees. However, Mark Zuckerberg saw Instagram’s potential as a mobile-first social network and wanted to integrate it into Facebook’s ecosystem while allowing it to function independently.
At the time of the acquisition, Instagram was still operating on a relatively small-scale infrastructure hosted on Amazon Web Services (AWS). The app was experiencing major scalability challenges, such as server overloads, database scaling issues, and limited engineering resources.
After joining Facebook, Instagram migrated from AWS to Facebook’s data centers, preparing it for its next phase of hypergrowth. Under Facebook’s ownership, Instagram continued to scale at an unprecedented pace. By 2014, it had 300 million users, and by 2018, it had breached the billion-user mark.
To handle billions of daily interactions, Instagram had to adopt advanced scaling strategies, including distributed databases, caching, and multi-region infrastructure. In this article, we’ll look at how Instagram scaled its infrastructure to handle this massive scale.
Instagram’s Early Infrastructure Challenges
When Instagram first launched in 2010, it was a small company with limited engineering resources.
However, with time, Instagram grew into one of the most widely used social media platforms in the world. This rapid increase in user engagement placed an enormous strain on its infrastructure, requiring continuous optimization and scaling efforts.
Some key metrics highlighting Instagram’s scale (in 2017-2018) are as follows:
400 million daily active users that open Instagram multiple times a day, generating billions of interactions.
100 million media uploads per day. Every media file must be optimized for different devices, ensuring smooth playback while minimizing bandwidth usage.
4 billion likes per day where every like generates a database write operation, which must be efficiently managed to avoid overloading storage and query systems
The platform’s rapid growth exposed significant scalability challenges, forcing engineers to adopt temporary solutions to keep the system running. However, as user activity continued to increase exponentially, these short-term fixes became unsustainable, leading to the urgent need for automation and a scalable architecture.
Some challenges faced by Instagram in its early days are as follows:
Manual Server Scaling: In the early days, Instagram’s engineering team manually added servers before weekends to prevent system failures. Traffic spikes were common on Fridays and weekends when user engagement surged due to increased social activity. Engineers would monitor CPU loads and decide if additional servers needed to be provisioned to handle the expected load.
Unreliable Performance Due to Limited Infrastructure: Instagram initially operated on Amazon Web Services (AWS), using a small number of virtual machines. The infrastructure was not optimized for handling sudden surges in traffic, leading to slow response times and occasional outages. The backend services struggled to process requests efficiently, causing delays in content loading and interactions.
Database Overload: The increasing number of users posting photos, liking content, and following accounts placed a heavy burden on Instagram’s relational database. As data volume grew, the read and write operations on the database became slower, affecting the overall performance of the platform. Engineers had to manually optimize queries and shift loads across multiple database replicas, a time-consuming process.
Lack of Automated Monitoring: Since Instagram had a small engineering team, monitoring server performance was done manually, often requiring constant attention to avoid system failures. Engineers used basic dashboards and alerts to detect issues, but there was no real-time system in place to automatically respond to performance problems.
Lack of Load Balancing: Load balancing across multiple servers was not fully automated, leading to uneven traffic distribution and bottlenecks.
Engineering Bottlenecks: Since infrastructure management was largely manual, engineers had to spend significant time scaling and maintaining servers instead of focusing on product development. As the team was small, scaling efforts often delayed feature releases and made it difficult to introduce new functionalities.
The Three Dimensions of Scaling at Instagram
As Instagram’s user base grew exponentially, the company had to develop scalable infrastructure to handle billions of daily interactions.
Scaling is not just about adding more hardware. It also involves optimizing resources and ensuring that the engineering team can operate efficiently. Instagram approached scalability in three key dimensions:
1 - Scaling Out: Expanding Infrastructure with More Servers
Scaling out refers to adding more hardware (servers, storage, and computing power) to handle increased traffic.
Some new strategies were adopted:
Instagram migrated from AWS to Facebook’s data centers, allowing horizontal scaling by distributing the workload across multiple servers. They expanded to multiple geographically distributed data centers, ensuring high availability and redundancy.
Instead of relying on a few powerful machines, Instagram distributed workloads across multiple servers and data centers.
Load balancing systems were implemented to distribute user requests across different data centers.
Instead of a single monolithic database, Instagram adopted a distributed database system, using PostgreSQL and Cassandra for high availability.
Migrating to Facebook’s data centers allowed Instagram to take advantage of Facebook’s existing tools and infrastructure, such as:
TAO: A highly efficient distributed database system for social data.
Scuba: A real-time analytics and monitoring tool for debugging performance issues.
Tupperware: Facebook’s containerized deployment system for managing large-scale applications.
However, the migration was quite challenging. Instagram needed to migrate petabytes of media files, user data, and metadata from AWS to Facebook’s data centers. Instead of a single "big bang" migration, data was incrementally copied and synchronized across both systems to ensure consistency. Facebook’s Haystack storage system was used to optimize media storage and retrieval.
Instagram’s backend services were initially built to run on AWS’s Elastic Compute Cloud (EC2). To optimize performance, these services were re-engineered to run on Facebook’s Tupperware containerization system, allowing for better resource utilization and auto-scaling.
2 - Scaling Up: Optimizing Efficiency of Each Server
Scaling up means improving the performance of existing infrastructure rather than adding more hardware. This involved reducing CPU load, optimizing queries, and using caching mechanisms to maximize server efficiency.
In Instagram’s case, adding more servers was not a sustainable long-term solution due to cost and resource limitations. Each request from a user (loading a feed, liking a post, adding a comment) resulted in multiple database queries, slowing down response times. Inefficient memory usage and unoptimized Python code caused unnecessary resource consumption.
The Instagram engineering team implemented some solutions to deal with these issues:
Optimized database queries by reducing redundant operations and batching requests.
Used Memcached and Cassandra to cache frequently accessed data, reducing direct database queries.
Profiled Python code to identify inefficient functions and replaced them with C++ implementations for better performance.
Improved server memory management, enabling a more efficient use of RAM.
3 - Scaling the Engineering Team
Scaling the engineering team means ensuring that developers can continue to work efficiently as the company grows. This involves better deployment strategies, automation, and minimizing bottlenecks in the development process.
Instagram grew from a small startup to a company with hundreds of engineers, leading to challenges in code management and deployment. More engineers working on the same codebase increased merge conflicts, regressions, and debugging complexity.
To handle these problems, the Instagram engineering team embraced continuous deployment. Instead of periodic large updates, they shifted to a rolling release model, deploying code changes 40+ times per day. This ensured that small updates could be tested, monitored, and rolled back quickly if needed.
New features were deployed gradually to a small subset of users before being rolled out globally. Engineers could monitor performance metrics and detect bugs early, preventing widespread failures. They also implemented real-time monitoring tools to detect performance regressions.
Instagram’s Backend Architecture
As Instagram grew into a platform handling billions of interactions daily, it required a robust, scalable, and efficient backend architecture.
The system needed to process a huge number of user requests, including media uploads, likes, comments, and notifications while maintaining high availability and low latency. The main components of the backend architecture are as follows:
1 - Django - The Core Web Framework
Django, a high-level Python web framework, serves as the foundation of Instagram’s backend. It provides the core structure for handling HTTP requests, user authentication, database interactions, and API endpoints.
The main advantages of Django are as follows:
Rapid development: Pre-built components allow engineers to quickly iterate on features.
Scalability: Supports ORM (Object-Relational Mapping), enabling efficient database queries.
Security: Built-in protections against common vulnerabilities like SQL injection and cross-site scripting.
2 - RabbitMQ - Message Broker
RabbitMQ acts as a message broker, facilitating communication between different services. Instead of handling everything in a single request, RabbitMQ enables asynchronous task execution, improving performance and reliability.
Instagram uses RabbitMQ to decouple components and allow different services to interact without being directly dependent on each other. It handles spikes in traffic and manages message persistence even if one part of the system temporarily fails.
3 - Celery for Async Task Processing
Celery is a distributed task queue system that works alongside RabbitMQ to execute background tasks. It ensures that long-running processes do not slow down real-time user interactions.
Instagram uses Celery to handle asynchronous tasks such as sending emails or processing large media files. Celery also has fault tolerance where a task failure results in automatic retries. Celery workers can be distributed across multiple servers, handling high workloads efficiently.
How the Components Work Together?
Instagram’s backend components operate in a coordinated manner, ensuring efficiency. Let’s understand the process when a user likes a post:
User Action: The user taps the “Like” button on a post.
Django Processes the Request: The request is sent to Django’s web server. Django updates the like count in the PostgreSQL database.
Caching and Asynchronous Processing: If the post has been liked before, the like count is retrieved from Memcached instead of querying the database. Django sends a task to RabbitMQ to notify the post owner about the like.
RabbitMQ Queues the Notification Task: Instead of handling the notification immediately, RabbitMQ queues it for later processing.
Celery Worker Sends the Notification: A Celery worker retrieves the task from RabbitMQ, generates the notification, and sends it to the recipient.
Storage Services in Instagram’s Architecture
Storage services are responsible for persisting user data, media files, and metadata across multiple distributed databases.
Instagram employs a combination of structured and distributed storage solutions to manage different types of data efficiently.
PostgreSQL: Stores structured data such as user profiles, comments, relationships, and metadata. It follows a master-replica architecture to ensure high availability and scalability. PostgreSQL also supports ACID compliance.
Cassandra: Used for storing highly distributed data, such as user feeds, activity logs, and analytics data. Unlike PostgreSQL, Cassandra follows an eventual consistency model, meaning data updates may take time to propagate across different regions. It provides high write throughput, making it ideal for operations like logging and real-time analytics.
Memcached: Used to reduce database load by caching frequently accessed data. It stores temporary copies of user profiles, posts, and like counts to prevent repeated queries to PostgreSQL or Cassandra. This helps optimize API response times
Haystack: Stores images and videos efficiently, minimizing the number of file system operations required to fetch content. It improves media loading times by serving cached versions of files through CDNs.
See the diagram below for PostgreSQL scaling out with leader-follower replication.
Also, the diagram below shows Cassandra scaling.
Since Instagram’s storage services are distributed across multiple data centers and geographic regions, maintaining data consistency is a major challenge.
The key data consistency challenges are as follows:
Replication Latency: In a distributed system, database replicas take time to sync, leading to temporary inconsistencies. For example, a user likes a post, but another user may not immediately see the updated like count because data replication has not been completed.
Eventual Consistency in NoSQL Systems: Cassandra uses an eventual consistency model, meaning updates propagate asynchronously across different database nodes. While this improves scalability, it can cause temporary discrepancies where different users see different versions of the same data.
Cache Invalidation Issues: Memcached accelerates read operations, but if an update is made to the database, the cached data must be invalidated to reflect the latest changes. For example, if a user changes their profile picture, but the cached version is not updated immediately, they may see an outdated version of their profile.
Cross-Region Data Synchronization: Since Instagram operates multiple data centers, data must be synchronized across different geographic locations. Network failures or delays in replication can lead to inconsistencies between regions.
The Instagram engineering team uses several strategies to address these consistency challenges.
For example, PostgreSQL is used for critical transactions, such as financial transactions, user authentication, and content moderation, where strong consistency is required. On the other hand, Cassandra is used for highly scalable workloads, such as activity feeds, where slight delays in consistency are acceptable. Memcached entries are automatically invalidated when new data is written to PostgreSQL or Cassandra, preventing outdated information from being served.
See the diagram below that shows how cache invalidation is handled.
Memcache Lease Mechanism
As Instagram scaled to handle billions of daily interactions, it encountered a major challenge known as the thundering herd problem.
The issue occurs for frequently accessed data (like counts, user feeds, and comments), which is stored in Memcached for fast retrieval. When a cached value expires or is invalidated due to an update, multiple web servers request the same data from the database. If thousands or millions of users request the same piece of data at the same moment, the database gets overloaded with duplicate queries.
To mitigate this, Instagram implemented the Memcache Lease mechanism, which helps prevent multiple redundant database queries by controlling how cache misses are handled.
Here’s how it works:
A user requests an Instagram feed. For each post in the feed, there is associated data such as the like counts, comments, and media metadata.
The application first checks Memcached to retrieve this data.
If the requested data is not found in Memcached, instead of immediately querying the database, the server sends a lease request to Memcached. The first server to request the data is granted a lease token, meaning it is responsible for fetching fresh data from the database.
Other servers that request the same data do not query the database. Instead, they receive a stale value from Memcached or are told to wait. This ensures that only one server fetches the updated data from the database, while the rest hold back.
The server holding the lease token queries the database and fetches the fresh data. It also updates the Memcached entry.
The first server returns the fresh data to users while the rest wait until the cache is repopulated, avoiding a direct hit to the database.
Instagram’s Deployment Model
Instagram operates on a continuous deployment model, meaning new code is pushed to production multiple times per day. This approach allows Instagram to iterate quickly, fix issues promptly, and introduce new features without causing major disruptions.
Instagram’s deployment pipeline includes code review, automated testing, canary testing, and real-time monitoring to ensure that new changes do not introduce performance regressions.
Engineers submit changes through code review to ensure quality and maintainability. Every code change must pass unit tests, integration tests, and performance benchmarks before being merged into the main branch. Once code is merged, it is automatically tested in staging environments to identify functional or security issues.
Also, instead of deploying changes to all users at once, Instagram first deploys updates to a small subset of production servers (canary servers). Engineers monitor key performance metrics (CPU usage, memory consumption, error rates) to ensure the new code does not introduce regressions.
Lastly, Instagram also monitors and optimizes CPU usage.
It uses Linux-based profiling tools to track CPU usage at the function level. Engineers analyze which parts of the codebase consume the most CPU cycles and optimize them for better efficiency. They also use profiling tools and replace CPU-intensive Python functions with optimized C++ implementations for better performance.
Conclusion
Instagram’s ability to scale while maintaining high performance and reliability is proof of its well-designed infrastructure and deployment strategies.
By leveraging Django, RabbitMQ, Celery, PostgreSQL, and Cassandra, Instagram ensures seamless processing of billions of daily interactions It has separated computing and storage services to support efficient resource utilization and minimize database load. The implementation of techniques like Memcache leasing prevents the thundering herd problem, reducing redundant queries and improving system stability.
The platform’s continuous deployment model enables rapid innovation while minimizing risk through canary testing, feature gating, and automated rollbacks. Furthermore, real-time CPU tracking, performance regression testing, and automated monitoring allow engineers to detect and resolve inefficiencies before they impact users.
These strategies have positioned Instagram as one of the most resilient and scalable social media platforms, allowing it to support millions of concurrent users, high-traffic workloads, and rapid feature development without compromising performance or stability.
References:
Instagration Part 2: Scaling Infrastructure to Multiple Data Centers
What powers Instagram: Hundreds of Instances, Dozens of Technologies
SPONSOR US
Get your product in front of more than 1,000,000 tech professionals.
Our newsletter puts your products and services directly in front of an audience that matters - hundreds of thousands of engineering leaders and senior engineers - who have influence over significant tech decisions and big purchases.
Space Fills Up Fast - Reserve Today
Ad spots typically sell out about 4 weeks in advance. To ensure your ad reaches this influential audience, reserve your space now by emailing sponsorship@bytebytego.com.
© 2025 ByteByteGo
548 Market Street PMB 72296, San Francisco, CA 94104
Unsubscribe
by "ByteByteGo" <bytebytego@substack.com> - 11:35 - 18 Feb 2025 -
15+ AI-driven innovations. It all starts 27 Feb.
New Relic
Free Virtual Event
Our most transformative software update is almost here! Join us on 27th February at New Relic Now+ to unpack the new innovations powering the Intelligent Observability Platform.
- 15+ NEW AI-driven transformative innovations
- AI-automated tasks and recommendations for immediate remediation
- Automated diagnostics and alerts to detect code issues earlier
- Multi-service trace flow maps to find and clear bottlenecks faster
Soak up the latest, and jump into guided workshops that take you closer to new certifications. See you there!Register Now
by "New Relic Inc." <emeamarketing@newrelic.com> - 04:56 - 18 Feb 2025 -
The trends behind social commerce’s rise
On McKinsey Perspectives
Brought to you by Alex Panas, global leader of industries, & Axel Karlsson, global leader of functional practices and growth platforms
Welcome to the latest edition of Only McKinsey Perspectives. We hope you find our insights useful. Let us know what you think at Alex_Panas@McKinsey.com and Axel_Karlsson@McKinsey.com.
—Alex and Axel
•
E-commerce on the rise. E-commerce, or the buying and selling of goods and services over the internet, has transformed the way that people shop since the first online transaction took place in 1994. E-commerce has grown consistently for decades, but the COVID-19 pandemic sent digital transactions soaring. In the US, e-commerce sales penetration more than doubled to about 35% in 2020, from about 16% in 2019. That increase is roughly the equivalent of ten years of growth, McKinsey Partner Dianne Esber and coauthors explain.
•
Small-business boost. For small and medium-size enterprises, the e-commerce opportunity is especially enormous—and even more so in the era of gen AI. Yet for businesses that are used to operating offline, incorporating e-commerce into the customer experience can be challenging. McKinsey analysis finds that small and medium-size retailers and brand manufacturers realize a much smaller portion of revenue from e-commerce than large retailers do. Read the McKinsey Explainer “What is e-commerce?” for more examples of how brands are using e-commerce to build value.
—Edited by Belinda Yu, editor, Atlanta
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to the Only McKinsey Perspectives newsletter, formerly known as Only McKinsey.
Copyright © 2025 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "Only McKinsey Perspectives" <publishing@email.mckinsey.com> - 01:14 - 18 Feb 2025 -
Transform talent management with AI insights
Transforming talent management requires strategies that deliver real-world results. In this live webinar, we’ll share how you can:- Pinpoint and close critical skills gaps faster
- Build a more agile, future-ready workforce
- Enhance employee engagement with personalized recommendations for learning and growth opportunities
- Improve hiring outcomes with data-driven insights
- Apply these approaches to achieve measurable success
Join experts like Dr. Katherine Gibbard, Christina Russo, and Edward Piggott, who will share actionable strategies grounded in real-world experience.
Save your spot today.
Contact us
See our complete list of local country numbers
SAP (Legal Disclosure | SAP)
This e-mail may contain trade secrets or privileged, undisclosed, or otherwise confidential information. If you have received this e-mail in error, you are hereby notified that any review, copying, or distribution of it is strictly prohibited. Please inform us immediately and destroy the original transmittal. Thank you for your cooperation.
You are receiving this e-mail for one or more of the following reasons: you are an SAP customer, you were an SAP customer, SAP was asked to contact you by one of your colleagues, you expressed interest in one or more of our products or services, or you participated in or expressed interest to participate in a webinar, seminar, or event. SAP Privacy Statement
This email was sent to info@learn.odoo.com on behalf of the SAP Group with which you have a business relationship. If you would like to have more information about your Data Controller(s) please click here to contact webmaster@sap.com.
This offer is extended to you under the condition that your acceptance does not violate any applicable laws or policies within your organization. If you are unsure of whether your acceptance may violate any such laws or policies, we strongly encourage you to seek advice from your ethics or compliance official. For organizations that are unable to accept all or a portion of this complimentary offer and would like to pay for their own expenses, upon request, SAP will provide a reasonable market value and an invoice or other suitable payment process.
This e-mail was sent to info@learn.odoo.com by SAP and provides information on SAP’s products and services that may be of interest to you. If you received this e-mail in error, or if you no longer wish to receive communications from the SAP Group of companies, you can unsubscribe here.
To ensure you continue to receive SAP related information properly, please add sap@mailsap.com to your address book or safe senders list.
by "SAP" <sap@mailsap.com> - 07:33 - 17 Feb 2025 -
Discover Reliable Welding Solutions with Unmatched Customer Support
Dear info,
At Topwill Welder, we understand the critical role that reliable welding equipment plays in your projects. We specialize in manufacturing high-performance piping tools, machine tools, and plastic pipe welding equipment tailored to your needs. Our commitment to quality and customer satisfaction is unmatched, and we are proud to support industries worldwide, from gas pipelines to environmental technologies.
Our product range includes:
- Precision butt fusion welding machines
- Versatile workshop fitting fabrication machines
- High-efficiency gas pipeline welding machines and tools
- Durable high-pressure welding machines
- A complete selection of pipe tools
With a strong presence in the USA, Europe, the Middle East, and Asia, we offer prompt after-sales repair services in Malaysia to ensure our clients' complete satisfaction. Let's discuss how we can support your projects with our exceptional products and service.
Awaiting your inquiry!
Best regards,
Adam Jiang
General Manager
Topwill Group
Head office: 1103F,Kaisa Plaza, No. 1091 Renmin East Road,Jiangyin city.
Tel:+86 510 86830885 Fax:+86 510 86805885
Website: www.topwillwelder.com info@topwillgroup.com
WhatsApp / Mobile: +86 13806163717Wechat: +86 13806163717
Topwill Group Companies:
Topwill Group Limited- HongKong
Jiangyin Topwill Welder Co Ltd- China
Jiangyin Topwill Flange Fitting Co Ltd- China
Jiangyin Topwill Hi-Precision Co Ltd - China
EverRich Flange SDN.BHD - Malaysia
by "Fatimah Tormey" <tormeyfatimah578@gmail.com> - 06:45 - 17 Feb 2025 -
High-quality health ingredients and plant extracts are now on the market
Dear info,
Have a nice day!This is Harris from Shaanxi top purity biotech ltd&sino ingredients Inc. Our company is dedicated to the field of Raw materials of health care products and plant extracts etc. Whether you are in the food and beverage industry looking for natural flavorings and colorants, or in the cosmetics and skincare sector seeking effective ingredients for anti-aging and moisturizing products, we have the perfect solution for you.
Some of our key products include: Berberine HCL Extract, NAD, Coenzyme Q10, PQQ, Sodium copper chlorophyllin etc...
Here's our advantage:
· we are familiar enough with the Chinese market to share with you the latest news on the market.
· We have a warehouse in California, USA, which greatly improves the delivery time.
· Our highly competitive prices.
· Our professional service.
By the way, Please check the attached inventory of our American warehouse. Please feel free to contact me if there are any products you are interested in.
We'd like to rely on our competitive price, professional service, good quality to reduce customers' production costs.
Looking forward to your feedback.
Best Regards,Harris Li
Mob: +86-17765028657 What's App: +86-18779653945QQ: 3109963732
Wechat: +8617765028657 TEL: +86-29-8441122
e-mail: sales08@toppuritybio.net, harrishsfbiotech@gmail.com
by "sales5" <sales5@toppuritynutrition.com> - 01:33 - 17 Feb 2025 -
residential and commercial [ projects ]
Greeting,I hope you're doing well.
As a professional estimator specializing in residential and commercial projects, I provide accurate estimates and take-offs to help you bid competitively. Without precise numbers, you risk losing projects to better-prepared competitors.
Send me your plans for a free quote—secure your next project with confidence!
Best regards,
Unsubscribe
by "Keith Bjella" <keith.construction11@gmail.com> - 11:46 - 17 Feb 2025
