Archives
-
EP66: Comparison of URL, URI, and URN
EP66: Comparison of URL, URI, and URN
Open in app or online This is a sneak peek of today’s paid newsletter for our premium subscribers. Get access to this issue and all future issues - by subscribing today.
Latest articles
If you’re not a subscriber, here’s what you missed this month:
To receive all the full articles and support ByteByteGo, consider subscribing:
This week’s system design refresher:
OAuth 2 explained in simple terms (Youtube video)
Comparison of URL, URI, and URN
Data Warehouse vs Data Lake
Twitter 1.0 Tech Stack
Sponsor us
OAuth 2 Explained in Simple Terms
URL, URI, URN - Do you know the differences?
The diagram below shows a comparison of URL, URI, and URN.
URI
URI stands for Uniform Resource Identifier. It identifies a logical or physical resource on the web. URL and URN are subtypes of URI. URL locates a resource, while URN names a resource.
A URI is composed of the following parts:
scheme:[//authority]path[?query][#fragment]
URL
URL stands for Uniform Resource Locator, the key concept of HTTP. It is the address of a unique resource on the web. It can be used with other protocols like FTP and JDBC.
URN
URN stands for Uniform Resource Name. It uses the urn scheme. URNs cannot be used to locate a resource. A simple example given in the diagram is composed of a namespace and a namespace-specific string.
If you would like to learn more detail on the subject, I would recommend W3C’s clarification.
What are the differences between a data warehouse and a data lake?
The diagram below shows their comparison.
A data warehouse processes structured data, while a data lake processes structured, semi-structured, unstructured, and raw binary data.
A data warehouse leverages a database to store layers of structured data, which can be expensive. A data lake stores data in low-cost devices.
A data warehouse performs Extract-Transform-Load (ETL) on data. A data lake performs Extract-Load-Transform (ELT).
A data warehouse is schema-on-write, which means the data is already prepared when written into the data warehouse. A data lake is schema-on-read, so the data is stored as it is. The data can then be transformed and stored in a data warehouse for consumption.
Over to you: Do you use a data warehouse or a data lake to retrieve data?
Twitter 1.0 Tech Stack
This post is based on research from many Twitter engineering blogs and open-source projects. If you come across any inaccuracies, please feel free to inform us.
Mobile: Swift, Kotlin, PWA
Web: JS, React, Redux
Services: Mesos, Finagle
Caching: Pelikan Cache, Redis
Databases: Manhattan, MySQL, PostgreSQL, FlockDB, MetricsDB
Message queues: Kafka, Kestrel
Data processing: Heron, Flume, Tableau, SummingBird, Scalding
Data storage: Hadoop, blob store
Data centers: Twitter data center, AWS, Google Cloud
Tools: Puppet, Audubon, Wilson
SPONSOR US
📈 Feature your product in the biggest technology newsletter on Substack
ByteByteGo is the biggest technology newsletter on Substack with over 450,000 readers working at companies like Apple, Meta, Amazon, Google, etc. They have the influence and autonomy to make large purchase decisions. Learn more today.
Latest articles
Here are the latest articles you may have missed:
To receive all the full articles and support ByteByteGo, consider subscribing:
Like
Comment
Restack
© 2023 ByteByteGo
548 Market Street PMB 72296, San Francisco, CA 94104
Unsubscribe
by "ByteByteGo" <bytebytego@substack.com> - 11:37 - 1 Jul 2023 -
Gen AI and the future of work
by "McKinsey Highlights" <publishing@email.mckinsey.com> - 11:15 - 1 Jul 2023 -
Last chance to register: Harness Generative AI Acceleration
Last chance to register: Harness Generative AI Acceleration
Harness Generative AI Acceleration with OpenVINO™ Toolkit
July 12, 2023; 7PM-8PM IST
Register now 
Md Abul,
Generative AI is exploding, bringing potential AI applications that could change everything we do. In this workshop, we'll delve into the world of transformer models, including Stable Diffusions and text processing, as well as explore how we've optimized these models to run on Intel’s wide variety of hardware.
What you’ll learn:
- Using Huggingface Transformers to create powerful AI solutions quickly
- Deploying stable diffusion and text processing from sophisticated Jupyter Notebooks
- AI applications can scale across GPUs and CPUs heterogeneously with Intel® Hardware
- How dynamic shape optimization maximizes Deep Learning performance
Register today
Paula Ramos
AI Software Evangelist
Dr. Ramos is a technology leader driving the intersection of AI with real-use cases and has made contributions in computer vision, computational photogrammetric, machine learning, embedded systems, edge computing, IoT systems, and cloud computing.
Anisha Udayakumar
AI Evangelist, Intel
Anisha has a passion for the OpenVINO™ toolkit that she brings to leveraging AI, Augmented Reality, Virtual Reality, Robotics, and XR to build rapid prototypes and vision-based retail solutions.Upcoming DevCon Webinars
Aug
9
Beyond the Continuum: The Importance of Quantization in Deep Learning
7PM-8PM IST
Sept
5
How To Build a Smart Queue Management System Step by Step? From Zero to Hero
7PM-8PM IST
Register early 
If you forward this email, your contact information will appear in any auto-populated form connected to links in this email.
This was sent to info@learn.odoo.com because you are subscribed to Webinars. To view and manage your marketing-related email preferences with Intel, please click here.
© 2023 Intel Corporation
Intel Corporation, 2200 Mission College Blvd., M/S RNB4-145, Santa Clara, CA 95054 USA. www.intel.com
Privacy | Cookies | *Trademarks | Unsubscribe | Manage Preferences
by "Intel Developer Zone" <intel.developer.zone@plan.intel.com> - 10:03 - 30 Jun 2023 -
How boards can build organizational resilience, revenue growth techniques for restaurants, what Gen Z wants, and more: The Daily Read weekender
Share this email
Brought to you by Liz Hilton Segel, chief client officer and managing partner, global industry practices, & Homayoun Hatami, managing partner, global client capabilities
TECH FOR EXECS
Welcome to Tech for Execs: quick primers on key technology concepts to get leaders up to speed.
What is a data mesh? For both new and experienced data practitioners, managing data is not a trivial matter. Data mesh is a relatively new approach to data management. By decentralizing data to specific business domains like sales, marketing, or customer service, teams can have more ownership over their data and easily share it across the organization.
How does it work? A data mesh is as much a cultural shift as it is a technological one. Part of this is thinking of data as a product, rather than a by-product of a process—and treating data producers as product owners, who can manage their own data and create policies with their best interests in mind.
What are the biggest benefits of a data mesh? Organizations increasingly rely on efficient data collection, storage, and access so cross-functional teams can use it effectively, across the business. A decentralized data architecture is therefore great for scalability and accessibility, especially when used alongside cloud technologies. It reduces operational bottlenecks that come with centralized systems, freeing up technical teams so they can focus on more complex systems.
Where do I start? Moving to a data mesh means overhauling technology, processes, and the way people work. So you need to start with a solid strategy. This might involve investing in DataOps to accelerate design and development, as well as fostering a data culture across the organization.QUOTE OF THE DAY
chart of the day
Ready to unwind?
—Edited by Joyce Yoo, editor, New York
Follow our thinking
Share these insights
Did you enjoy this newsletter? Forward it to colleagues and friends so they can subscribe too. Was this issue forwarded to you? Sign up for it and sample our 40+ other free email subscriptions here.
This email contains information about McKinsey’s research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to our McKinsey Quarterly Five Fifty alert list.
Copyright © 2023 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey Daily Read" <publishing@email.mckinsey.com> - 06:23 - 30 Jun 2023 -
🙌 Your exclusive invite: Remote’s Product Roadmap 2023
🙌 Your exclusive invite: Remote’s Product Roadmap 2023
.png?width=1200&upscale=true&name=Group%201%20(6).png)
Hi MD
We know that hiring efficiently and staying competitive in the global job market is important to our customers, which is why we have planned this next event specifically for you!
Your invitation: Join us for an exclusive look at Remote's 2023 Product Roadmap
Date: July 18, 2023Time: 1pm UTC
Join Remote's co-founder & CEO, Job van der Voort and Caio Tozzini, Director of Product and preview the new and exciting updates that are on our product roadmap for 2023 and beyond. We will also share what we’ve learned from years of facilitating global and remote employment and the impact it has had on our product, and how you, as customers, are the center of our product development.
We’ll conclude with a live Q&A where you can bring your questions and feedback directly to our product team.
Calendar clash?
Register for the webinar, and we’ll send you the session recording after the event.P.S. This webinar is free of charge and designed to provide you with the essential knowledge and tools to power your international expansion.
Grow your headcount
without the headaches.
You received this email because you are subscribed to Conferences & Events from Remote Technology, Inc.
Update your email preferences to choose the types of emails you receive.
Unsubscribe from all future emailsRemote Technology, Inc.
Copyright © 2023 Remote Technology, Inc. All rights reserved.
18 Bartol St. #1163 San Francisco California
by "Remote" <hello@remote-comms.com> - 08:00 - 30 Jun 2023 -
Appetite for creation
The Shortlist
Share this email
Curated by Liz Hilton Segel, chief client officer and managing partner, global industry practices, & Homayoun Hatami, managing partner, global client capabilities
Twice monthly, we bring you four new insights out of the dozens we publish that we think are worth a second glance from any business leader in any geography. In this edition, we look at the new rules for digital transformation, how to get out of the way of a good idea, and more. We hope you enjoy the read.
—Liz and Homayoun
Transformation without tears. McKinsey research finds that 90 percent of companies have tried some flavor of digital transformation, but they’re capturing only 30 percent of the value they expect. That’s why our colleagues wrote Rewired: The McKinsey Guide to Outcompeting in the Age of Digital and AI. It’s a practical guide on how to get hundreds of technologies to work together in a way that wows customers, cuts costs, and creates value. It’s the playbook we use, distilled from hundreds of transformations and packed with state-of-the-art guidance on how to make digital transformations succeed.
Make it stop. You can’t blame CEOs who are feeling dazed and confused by the unending changes shaking their organizations, from hybrid work to generative AI to quiet quitting. We documented these shifts in our April report, The State of Organizations 2023. Now, we’ve written a pocket guide for CEOs who want to future-proof their companies.
Share these insights
This email contains information about McKinsey’s research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to The Shortlist newsletter.
Copyright © 2023 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey Shortlist" <publishing@email.mckinsey.com> - 02:06 - 30 Jun 2023 -
Have you attended a concert lately? Arts institutions are navigating challenging times.
On Point
Brought to you by Liz Hilton Segel, chief client officer and managing partner, global industry practices, & Homayoun Hatami, managing partner, global client capabilities
• Jobs lost. The performing arts industry in one US state shed ten years’ worth of jobs during the COVID-19 pandemic, a recent study finds. A downturn in attendance, high inflation, and increasing rents contributed to a loss of roughly 59,000 positions from 2019 to 2021, bringing employment levels down to 2010 figures. Since the arts are so closely linked to other industries, such as restaurants and parking, the contraction could cut state and local tax revenues by $4.1 billion over the course of four years, the report noted. [LA Times]
• The arts under pressure. Before the COVID-19 pandemic, art institutions were already struggling due to increasing overheads, changing audience behaviors, and funding shifts. COVID-19-related disruptions have only compounded these trends. In the US, changing consumer behaviors decreased audience ticket demand by up to 25%. While technology can help arts institutions strengthen resilience, many may lack the capabilities, processes, and infrastructure to build data and analytics tools, McKinsey senior partner Ben Mathews and colleagues explain.
• Measuring value. Historically, art institutions have struggled to measure their value, which has limited their use of standardized performance metrics. As a result, art organizations often have an imprecise understanding of audiences, short-term views on revenue and growth, and planning methods linked to unreliable funding patterns. To help leaders increase their use of technology, McKinsey collaborated with seven leading US art institutions. Review a five-step framework leaders in the arts can use to build data and analytics capabilities.
— Edited by Belinda Yu, editor, Atlanta
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to the On Point newsletter.
Copyright © 2023 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey On Point" <publishing@email.mckinsey.com> - 10:09 - 29 Jun 2023 -
You're invited: Join us for a virtual launch of a new MGI report
Share this email
New from McKinsey Global Institute
Join us on Thursday, July 13, 2023, 9:00AM–10:00AM PT | 12:00PM–1:00PM ET | 5:00PM–6:00PM GMT, for a virtual launch event of a new McKinsey Global Institute report, Empty spaces and hybrid places: The pandemic’s lasting impact on real estate. The report models future demand for office, residential, and retail space in superstar cities around the world in several scenarios, considering long-term trends in population, employment, office attendance, housing prices, and other factors.
Following a presentation on the findings by report authors Jan Mischke, Aditya Sanghvi, and Olivia White, a CEO panel will discuss the themes and implications of the research, including:Share these insights
Did you enjoy this email? Forward it to colleagues and friends so they can subscribe too. Was this alert forwarded to you? Sign up for it and sample our 40+ other free email subscriptions here.
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to our McKinsey Global Institute alert list.
Copyright © 2023 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey Global Institute" <publishing@email.mckinsey.com> - 04:21 - 29 Jun 2023 -
Forward Thinking on bringing the joy to economics with Betsey Stevenson and Justin Wolfers
Share this email
New from McKinsey Global Institute
Forward Thinking on bringing the joy to economics with Betsey Stevenson and Justin Wolfers
Listen in Prefer audio? Listen to the podcast, and explore past episodes of the Forward Thinking Podcast. Subscribe via Google Podcasts, Apple Podcasts, Spotify, Stitcher, and Amazon Music.
‘Megathreats,’ ‘polycrises,’ and ‘doom loops’ with Nouriel Roubini
Share these insights
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to our McKinsey Global Institute alert list.
Copyright © 2023 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey & Company" <publishing@email.mckinsey.com> - 02:15 - 29 Jun 2023 -
Capacity Planning
Capacity Planning
Open in app or online This is a sneak peek of today’s paid newsletter for our premium subscribers. Get access to this issue and all future issues - by subscribing today.
Latest articles
If you’re not a subscriber, here’s what you missed this month:
To receive all the full articles and support ByteByteGo, consider subscribing:
This newsletter is written by guest author Diego Ballona, who is a senior engineering manager at Spotify.
Follow Diego for more on Twitter.
Capacity planning plays an integral role in the landscape of real-world system design. This complex exercise, far from being a mere theoretical consideration, is essential for engineers to accurately estimate the capacity needs of their proposed systems.
Several reasons underscore the significance of capacity planning in system design:
Capacity planning can tell us if a proposed design is feasible. It allows engineers to foresee potential bottlenecks in system performance or scalability, assess data storage needs, network bandwidth requirements, and predict the projected cost of resources. This verifies that aspects like latency, throughput, and availability align with the non-functional requirements.
A design might seem functionally feasible, but could have operational challenges. For example, if the system needs to handle surges in demand, it might need a lot of extra resources. This could make it prohibitively expensive to run.
Effective capacity planning shows a deep understanding of system design. It indicates an engineer’s knowledge of various design options and technologies, and the pros and cons of each. It also reflects their ability to anticipate potential problems and find appropriate solutions. This can lead to the development of systems that work better and are more efficient.
So, when designing a large-scale system, it’s important to think about capacity planning from the very beginning. By understanding what the system needs to do and how powerful it needs to be, engineers can plan and build systems that are scalable and efficient.
Now, let’s explore how capacity planning works in real-world system design.
Throughput Calculation
In designing large-scale systems, it is important to estimate the system’s scale in the beginning. Requirements might hint at the throughput, or we may need to infer it from relevant metrics like daily active users.
Throughput hint scenario
Let's consider Example 1 - Temperature sensors for a monitoring system. The system needs to measure temperature changes across a county using about 10 million sensors. These sensors report changes every 5 seconds. Analysts use the system to forecast the weather using a dashboard that displays reports.
From these requirements, we can infer a few things about the system’s capacity. We can estimate what the throughput is as follows:
In the extreme case, if all sensors are synchronized, we’d have 10 million queries per second (QPS). However, this is an unlikely edge case.
Assuming each request from a sensor to server lasts 200ms, a more realistic average would be 2 million QPS.
So, for this system, a good throughput estimate is about 2 million QPS.
Throughput through DAU scenario
Sometimes, throughput isn’t evident in the requirements. Let's look at Example 2 - Social Media News Feed:
The system allows users to post text, images, and videos on their profile timeline, which is organized chronologically. Users can follow others and see relevant posts from those they follow on their timeline.
To inform capacity planning, we could ask:
How many daily active users does this system have?
What is the expected user base growth in 2-5 years?
Then, calculate the throughput based on the feature requirements. If the most relevant posts appear in the timeline on the home page, all daily active users likely interact with it. Assume each user interacts with it a certain number of times per day (e.g., 10 times), it means 5 billion page views per day, or roughly 60k QPS.
500M users @ 10x pageviews/user = 5B timeline pageviews/day 5B pageviews / 86400 seconds (1 day = 24 * 60 * 60) = ~57.8k QPS Round up to 60k QPSProfile visits would likely be less frequent than homepage visits. If we assume that each user visits two profile pages per day, that results in an average QPS of about 12k.
Simplest way to calculate: 20% of 60k (previous example) = 12k QPS Or, to expand: 500M users @ 2x pageviews/user = 1B profile pageviews/day 1B pageviews / 86400 seconds (1 day = 24 * 60 * 60) = ~11.5k QPS Round up to 12kFor posts, let’s assume that on average, only 10% of the daily active users post once per day. This would mean an average QPS of approximately 6k.
Simplest way to calculate: 10% of 60k (previous example) = 6k QPS Or, to expand: 500M users * 10% = 50M new posts/day 50M new posts / 86400 seconds (1 day = 24 * 60 * 60) = ~5.7k QPS Round up to 6kRemember, these are rough estimates. For most system designs, this is good enough. A good tip is to think in round numbers and round up to ensure conservative estimates.
Peak QPS
Calculating peak QPS is important as it often dictates the capacity requirement of the design. Peak QPS refers to the highest rate at which a system will be expected to handle queries, often occurring during times of high usage or even traffic spikes. This can be much higher than the average rate. This is why it requires special attention.
One common method to determine peak QPS is through historical data analysis. This involves tracking the number of queries that the infrastructure handles over a specific timeframe, like days, weeks, or even months, and then choosing the highest value. This method relies on the availability of data and the system’s historical performance.
Overprovisioning the infrastructure or utilizing autoscaling features can also help handle peak QPS. These strategies allow the system to increase its capacity temporarily to deal with unexpected surges in traffic. However, they come with their own costs and need careful cost-benefit analysis.
Peak QPS could also be influenced by business requirements or predictable usage patterns. For instance:
Event-driven Peaks: A major product launch or marketing event might attract a surge of users signing up or using the system within a brief window. Predicting and preparing for these events can help maintain service quality during high demand.
Time-driven Peaks: Some systems may experience predictable daily or weekly fluctuations in usage. For instance, a business-oriented application may see significantly higher traffic during working hours compared to late evenings or early mornings.
To estimate peak QPS, we often make calculations based on expected distribution characteristics. For example, we might assume that 80% of visits per day occur within 20% of the time (a variant of the Pareto Principle). We add some buffer capacity to handle unexpected surges and provide a smooth user experience.
In Example 2 - Social Media News Feed, if we anticipate that 80% of pageviews for timelines occur within an 8-hour time span, we’d calculate the peak QPS for this period to be around 138k. However, this is just a starting point - it’s always a good practice to overprovision initially, monitor the data, and then adjust based on the actual usage patterns.
500M users @ 10x page views / user = 5B timeline pageviews/day 80% of pageviews = 4B timeline pageviews Per hour over a period of 8 hours = 4B page views / 8 hours = 500M / hour Average Peak QPS = 500M / 3600 (60 minutes in seconds) = 138kPreparing for peak QPS helps ensure that the system remains stable and responsive even under the heaviest loads. This contributes to a better user experience and system reliability.
Estimating server requirement
With the estimated throughput and response time, we can estimate the number of servers needed to run the application.
In Example 1 - Temperature sensors, the system has an average response time of 200ms and needs to handle 2M QPS, and each application server can manage 32 workers handling 160 QPS, we’d need around 12.5k server instances.
2M QPS, 200ms avg response time per request Each instance has 32 workers Each worker can handle 5 queries per second (200ms*5) Each instance can handle then 160 QPS 2M (average QPS) / 160 = 12.5k instancesNow that we estimated the scale of the overall system, let's focus on the specifics of the system we are designing, starting with request sizing.
Understanding request sizes
Assessing request sizes is crucial for determining bandwidth and storage requirements. In system design, we often need to accommodate a variety of request types that can significantly impact the load on our system. These could be as simple as GET requests retrieving data or as complex as POST requests that involve large multimedia files.
While initial requirements might not include specific request sizes, we can make informed assumptions based on the system’s functionality and the nature of data it handles
When it comes to estimating request sizes, different types of systems will naturally have different expectations. Let’s consider the two examples we’ve been discussing: Temperature Sensors and Social Media News Feed.
For a system like Example 1 - Temperature Sensors, the data sent may be relatively small. Assuming data is sent in JSON format, we can estimate the size of each field. The temperature is reported as a float (4 bytes), the sensor ID is a UUID (16 bytes), and we have three additional 4-byte fields. Accounting for the JSON format, the total request body size is less than 100 bytes. Including HTTP headers (typically between 200 to 400 bytes), we can conservatively estimate each request size to be around 0.5KB. This is quite small. However, even such small data requests can add up when dealing with millions of sensors, making this an important consideration.
For a system like Example 2 - Social Media News Feed, we’re dealing with diverse content types - text, images, and videos. While text posts might only be a few KB, images files could be several hundred KB, and video files could be several MB. In these scenarios, an average request size needs to consider the distribution and size of various content types.
Let’s make a conservative estimate of a text-only post being about 1KB, which includes a user ID (UUID) and a free-form string with 250 characters on average. For images and videos, let’s assume that for every 10 posts, there are three images (average size of 300KB after compression), and one is a video (averaging at 1MB). We can fold all these media types into an average to simplify our calculations. This leaves us with an average request size of around 200KB per post.
For 10 posts: - Every post has 1KB (text) = 10KB - Three have an image = 300KB * 3 = 900KB - One has a video = 1000KB ~191KB, round up to 200KBConsiderations for Request Sizing
Understanding request sizes is just part of the equation. We must also consider factors like data format, serialization/deserialization, and the impact of these on bandwidth and processing needs.
Data Format. The format of data being sent can significantly impact the bandwidth required. More verbose data formats, like XML, can consume more bandwidth compared to less verbose formats like JSON. Even among similar formats, options like Binary JSON (BSON) can be more efficient.
Serialization/Deserialization: Transforming data into a format that can be easily transmitted or stored (serialization) and then reverting it back to its original form (deserialization) can require significant resources. Different methods have different costs and benefits. For instance, Protobuf is faster than JSON for serialization and deserialization, but JSON is more human-readable and easier to debug.
Compression: Implementing data compression can effectively reduce request size and conserve bandwidth. While it does come with additional processing overhead, the trade-off may be acceptable or even beneficial, especially for larger payloads.
Network Protocols: The choice of network protocol (HTTP/1.1, HTTP/2, gRPC, etc.) can also affect the overall size of a request due to differences in headers, the ability to compress headers, and other factors.
Request sizing is an important aspect of capacity planning. Although it requires making assumptions, these informed guesses help anticipate the system’s needs. It ensures that we’re well-prepared to handle expected loads and traffic patterns.
Bandwidth & data in transit
Understanding each record’s cost is fundamental. We need to consider the bandwidth per operation in both directions - client to server (ingress) and server to client (egress).
Client to server (ingress)
In Example 1 - Temperature sensors, the data sent by the sensors to the application is known as ingress: incoming data to the application's network. Suppose each sensor request is 0.5KB and the average QPS is 400k, our average ingress bandwidth is approximately 200MB/second. We can simplify the discussion by leaving out the read path for dashboards, etc., as it probably won’t significantly impact capacity.
An interesting aspect to consider is how often we need to send data to the servers. It’s possible that not all data requires immediate transmission. For example, a temperature sensor might only need to report data when a significant temperature change occurs. By storing the temperature state locally on the sensor and transmitting it only upon changes, we could potentially reduce the capacity load and costs.
However, it’s important to set a limit for the maximum time a sensor can go without reporting to the server. This ensures that any malfunctioning or offline sensors are promptly identified.
Server to client (egress)
When considering a system like Example 2 - Social Networking News Feed, posting content would equate to ingress traffic. Based on our previous estimates, suppose each post is 1KB and we’ve got an average of 6k QPS, our ingress bandwidth is about 6MB/second.
The egress, or the data transmitted from the server to client, can be complex. Suppose our system paginates every 20 posts, and the server renders raw file versions each time. In this scenario, our average egress bandwidth would be very high at ~250GB/second:
500M users @ 10x pageviews/user = 5B timeline pageviews/day Average post request size: 200KB Assuming every time timelines are rendered the server sends everything to the client. 20 posts * 200 KB = 4MB 5B pageviews * 4MB = 20PB 20PB / 86400 (1 day = 24 * 60 * 60) = ~231GB/s Round up to ~250GB/sBut there are optimization strategies to reduce this egress traffic.For example:
Trimming the content: We can reduce the data load by not sending the full text of the post content by default. This could reduce the text size by half.
Image optimization: Low-quality image placeholders can be sent initially, and high-quality images can be progressively loaded based on user interactions and viewport visibility.
Video optimization: Autoplay for videos can be turned off, with only a thumbnail loaded by default. The full video can be streamed on demand, significantly reducing the initial data load.
Here’s a rough calculation:
80% of our user base won’t get past the fifth post on the timeline;
Sending only an excerpt of the text will reduce the text size by 50%;
We reduce thumbnail size to the client to 30KB as placeholders and only load on demand (300KB) for what’s visible in viewport
Video will only be loaded on play (no autoplay), and by default will load the same thumbnails as images, progressively. Only 10% of the users will play videos.
We could achieve around 67% saving in our capacity, without caching:...
Keep reading with a 7-day free trial
Subscribe to ByteByteGo Newsletter to keep reading this post and get 7 days of free access to the full post archives.
A subscription gets you:
An extra deep dive on Thursdays Full archive Many expense it with team's learning budget © 2023 ByteByteGo
548 Market Street PMB 72296, San Francisco, CA 94104
Unsubscribe
by "ByteByteGo" <bytebytego@substack.com> - 11:41 - 29 Jun 2023 -
Load Monitoring Revolution: Empowering Fleet Managers for Optimal Efficiency
Load Monitoring Revolution: Empowering Fleet Managers for Optimal Efficiency
Harness the Power of Real-Time Insights for Optimal Load Management, Enhanced Efficiency, and Regulatory Compliance.
Catch a Glimpse of What Our Software Has to Offer
Overload and Underload Prevention
The software is designed to monitor and manage load to prevent situations where a vehicle is carrying a load that exceeds or falls below recommended weight limits.
Realtime Load Monitoring
The software provides immediate visibility into load status and location. Fleet managers can track loads, optimize routes, monitor load performance, and enhance customer service.
Empower your clients with load monitoring and help them optimize load distribution
Uffizio Technologies Pvt. Ltd., 4th Floor, Metropolis, Opp. S.T Workshop, Valsad, Gujarat, 396001, India
by "Sunny Thakur" <sunny.thakur@uffizio.com> - 12:30 - 29 Jun 2023 -
Businesses face tumultuous times. What do leaders see as the ten biggest organizational shifts?
On Point
Brought to you by Liz Hilton Segel, chief client officer and managing partner, global industry practices, & Homayoun Hatami, managing partner, global client capabilities
• A new mindset. The geopolitical and economic disruptions of the past few years have drastically changed how businesses operate, innovate, and recruit talent. In an episode of The McKinsey Podcast, McKinsey senior partners Dana Maor and Patrick Simon surveyed more than 2,500 business leaders to learn what they see as the biggest organizational changes. McKinsey’s State of Organizations report reveals ten shifts affecting companies, with managing the hybrid workplace, increasing resilience, and building capabilities coming in top of mind for executives.
• Strengthening resilience. Our research has uncovered a wide range of organizations—from smaller companies with 200 employees to larger ones with 140,000 workers, in a variety of fields—that are succeeding, despite economic uncertainty. These organizations build resilience into their processes and structures by breaking down silos, communicating clear priorities to all parts of the organization, and providing upskilling opportunities for talent at all levels. Understand four ways that today’s organizations can respond to a constantly changing business environment.
— Edited by Gwyn Herbein, editor, Atlanta
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to the On Point newsletter.
Copyright © 2023 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey On Point" <publishing@email.mckinsey.com> - 12:26 - 29 Jun 2023 -
Up to 1.5x faster checkout, 5x faster order creation, 40x faster order lookup 🤯
Up to 1.5x faster checkout, 5x faster order creation, 40x faster order lookup 🤯
Here's what's new with Woo this month 
This August, we're introducing an update to speed up your store. Among other improvements, High-Performance Order Storage (HPOS) enables a faster checkout experience for your customers – and faster order processing for you.
Learn more about HPOS and how you can enable it for your store.
Get the lowdown on HPOS New in our Marketplace
Bring your electronics store to life
Built for Full Site Editing (FSE), eMart helps you quickly bring your electronics store to life, thanks to drag-and-drop blocks, multiple homepage designs, and 27+ block patterns.
Make shopping online a pleasure for your customers with this stylish theme.
Create your ideal store 
Find your ideal audience on TikTok, automatically
Save time and create smarter ads using automated campaign management from TikTok for WooCommerce.
Take the effort out of campaign creation — and watch TikTok’s machine learning build, target, and optimize ads that deliver results and grow your audience.
Optimize my advertising Bite-sized updates
- Brand stories are vital for humanizing a business or product. They're an effective way to connect with customers and show how you're different from the competition. Here's how to write a compelling brand story.
- Zapier's new Update Product Stock Quantity action allows users to edit the number of existing products or variations by SKU or ID. There are improved Authentication Dialogue instructions and dependencies too. Speed up your processes — let Zapier do the work for you.
- Earlier this month, we presented awards to five top European WooExperts for their work as creative innovators, open source champions, and top performers. Find out who took home the trophies this year (and why).
- Display events, sell tickets, and take bookings on your site with Events Manager. You have complete control, with extensive settings and customization options. Take charge of your next event.
- Now you can connect your store to Zendesk and allow customers to create support requests and interact – without leaving your site. Admins can check tickets and assign orders from their Woo dashboard. Up your support game today.
Help us build a better Woo
Exciting things are in the works and you can help them take shape. Let's talk.
7.8.1 Current WooCommerce release
67 Core translation locales
This email was sent to info@learn.odoo.com. You're receiving it because you opted in to receive updates when creating an account or purchasing a product subscription on WooCommerce.com.
No longer wish to hear from us? Unsubscribe from marketing emails or all email communication. You may still receive transactional and/or account-related emails from WooCommerce.com.
View an online version of this email.
WooCommerce, Inc. is located at 60 29th St #343,
San Francisco, CA 94110, U.S.A.
© 2023 WooCommerce, Inc. – an Automattic company.
by "Woo" <support+marketing@woocommerce.com> - 09:29 - 28 Jun 2023 -
API product trends and best practices
API product trends and best practices
The shift towards API product expansion will open new avenues for your organisation – from innovation to collaboration and revenue generation – and this newsletter might just contain the valuable nuggets of insights to help you succeed...
Firstly, we've written a whole e-book on the world of API business models and all the cases for API product growth.
Have we piqued your interest?
Well, there's more. We've also got four new expert blogs on how to make a persuasive business case for APIs and demonstrate API impact to the decision-makers in your organisation.
Throw in our upcoming events and on-demand webinars on all things APIs, and we've got something for everyone to explore the endless possibilities of API expansion and growth for your organisation.
Check them all out below!
P.S. Want to hear more from Tyk? Subscribe to our newsletter here.
Tyk in the limelight 🏆
NEW E-BOOK - The API advantage!
Get in, loser; we're taking a deep dive into the real world of API business models and their case for API product growth! This e-book, made for product managers, is an essential source of truth, so you can start seeing the world of product management and the essential role of APIs from the angle that works for you - download now!
The resource hub 💡
New blogs on the block
Building a business case for APIs
These days, you don’t have to go it alone when it comes to APIs – there are solutions and workflows to guide you every step along your API journey. You just need to present a business case to demonstrate that to your boss. Let us show you how.
Mastering API product design and delivery
The way you design and deliver your API product can make the difference between a winning product that takes off rapidly and one that never really seems to get off the ground. If you want your API to be one of the winning ones, read the blog.
How to measure the success of your API products
How do you measure the success of your API products? Doing so is key to ensuring that they are performing as they should be and that you are optimising opportunities for growth. If you want to make the most of your APIs, read on.
Demonstrating API impact
In most businesses, knowing that you’ve created a successful API product is not enough. You also need to report on that API in a way that demonstrates its impact. Read this blog to help justify ongoing investment in your API team.
Coming to a location near you 🗓️
Here are all the details on the latest and greatest events and speaker slots coming up in the Tyk universe:
Want to be the first to hear about our events? Sign up here to stay in the loop.
We also have a PLETHORA of webinars on all sorts of API-related topics you can crack into and watch on-demand here. Go onnn, you know you want to check them out.
And lastly, because we're sure you're desperate to know what an API product manager's favourite dance move is...
It's the RESTful shuffle, of course.
It involves gracefully choreographing requests and responses while maintaining a smooth flow.
Right. We're tapping out. See you next month.
Tyk, 87a Worship Street, London, City of London EC2A 2BE, United Kingdom, +44 (0)20 3409 1911
by "Budhaditya Bhattacharya" <communities@tyk.io> - 09:27 - 28 Jun 2023 -
What does Gen Z care about as employees and consumers?
On Point
Brought to you by Liz Hilton Segel, chief client officer and managing partner, global industry practices, & Homayoun Hatami, managing partner, global client capabilities
• Show me the money? In two years, Gen Z may comprise more than a quarter of the global workforce. To attract and retain these workers, employers should understand what this generation prioritizes in the workplace. For example, Gen Z is the generation least motivated by money, McKinsey senior partner Aaron De Smet and coauthors reveal. They’re the only respondents who didn’t rank compensation in their top two reasons for leaving a job. Compensation is also their least important factor for staying in a job.
• Under the (social media) influence. No generation is more attuned to social media when deciding what to buy than Gen Z, research by McKinsey senior partner Jennifer Spaulding Schmidt and colleagues finds. Nearly 40% of adult Gen Z consumers (ages 18 to 23) say that social media has the biggest influence on their purchasing decisions, according to a McKinsey survey of 10,000 US consumers. See what we’ve learned about Gen Z, including how they see themselves as consumers and what they value in a job.
— Edited by Belinda Yu, editor, Atlanta
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to the On Point newsletter.
Copyright © 2023 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey On Point" <publishing@email.mckinsey.com> - 10:06 - 27 Jun 2023
