Archives
-
$1M Saved with Data Tiering - Case Study
Sumo Logic
When you centralize the log data you supercharge the problem solving.
Trying to manage data costs?
With a 12 TB log volume per day, Infor needed a scalable log analytics solution that didn’t strain the budget.
Key savings achievements:
Impressive savings, no?
Since this case study, we've launched our industry-leading analytics-based pricing model, which includes unlimited log ingestion and indexing at $0 with unlimited users.
It’s time to re-think the economics of ingest-based pricing.Sumo Logic, Aviation House, 125 Kingsway, London WC2B 6NH, UK
© 2024 Sumo Logic, All rights reserved.Unsubscribe 
by "Sumo Logic" <marketing-info@sumologic.com> - 09:00 - 5 Jun 2024 -
What could help make life more affordable?
Only McKinsey
Brought to you by Liz Hilton Segel, chief client officer and managing partner, global industry practices, & Homayoun Hatami, managing partner, global client capabilities
—Edited by Belinda Yu, editor, Atlanta
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this newsletter because you subscribed to the Only McKinsey newsletter, formerly called On Point.
Copyright © 2024 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "Only McKinsey" <publishing@email.mckinsey.com> - 01:04 - 5 Jun 2024 -
AWS Lambda Under the Hood
AWS Lambda Under the Hood
Forwarded this email? Subscribe here for moreWorkOS: enterprise-grade auth for modern SaaS apps (Sponsored)
WorkOS helps companies become Enterprise Ready ridiculously fast.
→ It supports a complete User Management solution along with SSO, SCIM, RBAC, & FGA.
→ Unlike other auth providers that rely on user-centric models, WorkOS is designed for B2B SaaS with an org modeling approach.
→ The APIs are flexible, easy-to-use, and modular. Pick and choose what you need and integrate in minutes.
→ Best of all, User Management is free up to 1 million MAUs and includes bot protection, impersonation, MFA, & more.
Future-proof your auth stack with WorkOS
Disclaimer: The details in this post have been derived from multiple AWS reInvent talks and articles by the Amazon engineering team. All credit for information about AWS Lambda’s internals goes to the presenters and authors. The link to these videos and articles is present in the references section at the end of the post. We’ve attempted to analyze the details and provide our input about them. If you find any inaccuracies or omissions, please leave a comment, and we will do our best to fix them.
AWS Lambda is a serverless compute service that allows developers to run code without provisioning or managing servers.
Lambda functions can be triggered by various AWS services, HTTP endpoints, or custom events generated by applications. This makes it highly versatile for building event-driven architectures.
There are some interesting stats about AWS Lambda:
Processes over 10 trillion invocations per month.
More than a million customers have built applications using AWS Lambda.
The customer base includes individuals, startups, enterprises, and large organizations.
Supports various applications, such as IT automation, data processing pipelines, microservices-based applications, web applications, and machine learning workloads.
The timeline below shows the evolution of AWS Lambda over the years.
In this post, we will look at the architecture and internals of AWS Lambda.
How is a Lambda Function Invoked?
There are two types of invocation models supported by Lambda.
1 - Synchronous Invocation
In synchronous invocation, the caller directly calls the Lambda function. This can be done using the AWS CLI, SDK, or other services like the API Gateway.
The diagram below shows the entire process:
The process involves the following components:
Front-End Service: The front-end service performs authentication and authorization of the request. It also loads and caches metadata to optimize performance.
Counting Service: This service checks quota limits based on account, burst, or reserved concurrency. It ensures high throughput and low latency while operating across multiple availability zones (AZs).
Assignment Service: The assignment service routes invocation requests to available worker hosts. It coordinates the creation of secure execution environments, setting up limits such as memory and CPU allocation. We will talk more about it in a later section.
Placement Service: The assignment service communicates with the placement service to determine the best worker host to handle the new execution environment. The placement service uses machine learning models to make informed decisions about where to place new execution environments. These models take into account variables such as packing density and fleet utilization to make the decisions.
Worker Hosts: They are responsible for downloading, mounting, and running the function code within a microVM. They manage multiple runtimes (such as Node.js, Java, and Python) and Lambda agents for monitoring and operation.
When an invocation occurs, it can have a warm or cold start.
If a warm (already running) execution environment is available, the payload is sent directly to it, and the function runs immediately.
If no warm environment is available, a new execution environment is created. The process involves initializing the runtime, downloading function code, and preparing the environment for execution.
The assignment service keeps track of execution environments. If there are errors during initialization, the environment is marked unavailable. When environments are near the end of their lease, they are gracefully shut down to ensure stability and resource efficiency.
2 - Asynchronous Invocation
In asynchronous invocation, the caller doesn’t wait for the function’s response. Instead, the event is placed in an internal queue and processed separately.
The process involves:
Event Invoke Frontend Service: This service handles asynchronous requests. It authenticates and authorizes the request, then places the event in an internal SQS queue.
Polling Instances: A fleet of pollers reads messages from the SQS queues and sends them to the synchronous front-end service for processing.
Synchronous Processing: Despite being an async request, the actual function execution uses the same synchronous processing pathway as described earlier.
Event Destinations: These can be configured to handle callbacks after processing, providing notifications of success or failure.
The diagram below shows the asynchronous invocation process:
Lambda manages multiple internal queues, scaling them dynamically based on load. The queue manager oversees the creation, deletion, and monitoring of these queues to ensure efficient processing.
As far as polling is concerned, Lambda can poll from various event sources like Kinesis, DynamoDB, SQS, Kafka, and Amazon MQ. The pollers perform the following functions:
Read and Filter: Poll messages from the sources, optionally filtering them to reduce traffic and simplify function code.
Batch Processing: Batch records together before sending them to the function, optimizing performance and reducing costs.
Synchronous Invocation: Use the same synchronous pathway to execute the function, ensuring consistency.
Control plane services (such as State Manager and Stream Tracker) manage the pollers, ensuring they are assigned correctly, scaling them up or down, and handling failures gracefully.
In case a function fails, the message is returned to the queue with a visibility timeout, allowing for retries.
Latest articles
If you’re not a paid subscriber, here’s what you missed.
To receive all the full articles and support ByteByteGo, consider subscribing:
State Management in Lambda
In the earlier architecture of AWS Lambda, the Worker Manager service was responsible for coordinating the execution environments between the frontend invoke service and the worker hosts.
The worker manager performed a couple of important tasks:
State Management: The Worker Manager kept track of which execution environments were available and on which hosts they were running.
Execution Lifecycle: It managed the lifecycle of these environments, ensuring that they were ready to process invocations and handling termination when no longer needed.
There were some challenges with the worker manager:
Single Point of Failure: Each Worker Manager instance stored the state of execution environments in memory. If a Worker manager failed, the execution environments it managed could become orphaned.
Orphaned Environments: Orphaned environments could continue processing current invocations but could not be used for new invocations, leading to cold starts and inefficient resource utilization.
Zonal Failures: In case of a failure in an Availability Zone (AZ), Worker Managers in that AZ would become unavailable. This will also make the environments managed by them unavailable, resulting in capacity loss and increased cold start latencies.
Introduction of the Assignment Service
To address the challenges with Worker Manager, AWS Lambda introduced the Assignment Service which offers a more robust and resilient way to manage execution environments.
The Assignment Service is written in Rust, which is chosen for its performance, memory safety, and low latency characteristics.
The diagram below shows the high-level design of the Assignment Service
The architecture of the Assignment Service has some key properties:
1 - Partitioned Design
The Assignment Service is divided into multiple partitions. Each partition consists of one leader and two followers, spread across different Availability Zones (AZs). This ensures high availability and fault tolerance.
Multiple partitions run simultaneously, with each Assignment Service host managing several partitions. This helps with load distribution and provides redundancy.
2 - External Journal Log Service
The leader in each partition writes the state of execution environments to an external journal log service. The followers read from this log to keep their state in sync with the leader.
In case the leader fails, one of the followers can quickly take over as the new leader using the state information from the journal log.
3 - Frontend Interaction
The frontend invoke service interacts with the leader of the relevant partition to manage invocations.
The leader coordinates with the placement service to create new execution environments and writes this information to the log for followers to update their state.
4 - Failure Handling
If an AZ experiences an outage, the leader and followers in other AZs continue to operate. This means that execution environments in the unaffected AZs remain available and aren’t orphaned.
If a leader fails, a follower can take over quickly using the replicated state, ensuring that there are no interruptions in service and reducing the chances of cold starts.
The Role of Firecracker MicroVM
Firecracker is a lightweight virtual machine manager (VMM) designed specifically for running serverless workloads such as AWS Lambda and AWS Fargate. It uses Linux’s Kernel-based Virtual Machine to create and manage microVMs.
The primary goal of Firecracker is to provide secure and fast-booting microVMs.
Initially, AWS Lambda used EC2 instances to run functions, dedicating a T2 instance to each tenant. This was secure but inefficient, leading to high overhead due to the need for provisioning entire EC2 instances for each function.
In 2018, Lambda transitioned to using Firecracker for managing execution environments. This allowed for much smaller, lightweight microVMs, significantly reducing resource overhead and improving the fleet’s efficiency.
Here are some key technical details of the Firecracker implementation.
1 - MicroVMs and Isolation
Firecracker enables the creation of microVMs, which are smaller and more efficient than traditional VMs. Each microVM runs its isolated execution environment, including the runtime, function code, and any extensions.
Firecracker ensures a strong solution between microVMs, which is crucial for multi-tenant environments where different customer functions run on the same physical host.
2 - Startup Times
Firecracker microVMs are designed to boot in milliseconds, significantly reducing the cold start latency for Lambda functions.
For further optimization, Lambda uses a feature called SnapStart (initially available for the Corretto Java 11 runtime). SnapStart pre-initializes execution environments and takes snapshots, which are restored on demand when needed. We will talk more about it in the next section.
3 - Resource Utilization
By running multiple microVMs on a single physical host, Firecracker improves CPU utilization, memory, and storage resources. This allows Lambda to handle more functions with fewer resources.
Also, Firecracker supports the dynamic scaling of resources for quick adjustment to changing workloads.
4 - Storage and Networking
Lambda uses a virtual file system driver to present an EXT4 file system to the microVMs. This driver handles file system calls and maps them to the underlying storage.
Firecracker optimizes network performance by providing high-speed virtual network interfaces, ensuring fast and reliable communication.
Lambda as a Storage Service
AWS Lambda is primarily known as a serverless compute service. However, it also incorporates significant storage service functionalities.
Think of Lambda not just as a compute service but also as a storage service because it deals with storing and managing a lot of data behind the scenes.
But why is storage important for Lambda?
It’s because when you run a function on Lambda, it needs three main things:
Input Data: The information or data your function will process
Function Code: The code you wrote that tells Lambda what to do
Execution Environment: The virtual machine (VM) where your code runs.
To make all of this work smoothly, Lambda needs to manage and store these components efficiently.
1 - Storing Code and Data Efficiently
Instead of storing your entire code or data as one big piece, Lambda breaks it down into smaller pieces called chunks.
This process is known as chunking and it allows Lambda to handle large and complex functions more effectively.
Here’s how it works:
Breaking down container images: When you upload a container image (which can be up to 10 GB) to AWS Lambda, it doesn’t store the entire image as one big file. Instead, Lambda breaks this image down into smaller pieces or chunks. For example, the base OS layer is broken down into smaller pieces, the runtime layer into another set of pieces, and so on.
Storage and Access: These chunks are then stored separately, making it easier to manage and retrieve them as needed. When your function runs, Lambda only loads the chunks it needs at that moment, rather than the entire image. This is much faster and more efficient.
Retrieving Chunks: When your function is invoked, Lambda checks which parts of the container image it needs to execute the function. It then retrieves only those specific chunks from storage.
Optimizing Performance: Since only the required chunks are loaded, this reduces the amount of data that needs to be transferred and processed, leading to faster startup times and better performance. Frequently accessed chunks can be cached to avoid fetching them from the main storage repeatedly.
2 - Sharing Common Data
Lambda often runs many functions that use similar code or data such as base operating system layer or common runtime libraries.
Instead of storing multiple copies of the same thing, it stores one copy and reuses it.
To make sure the shared data is secure, Lambda uses a technique called convergent encryption.
But how does convergent encryption work?
Plaintext Chunk Creation: Lambda starts with a plaintext chunk, which is a segment of the flattened file system of a container image. Each chunk represents a specific range of the block device that makes up the file system.
Appending Extra Data: To ensure security and integrity, additional data is appended to the chunk. This extra data is generated by the Lambda service and can include metadata or unique identifiers. The purpose of this extra data is to add a layer of uniqueness to the chunk.
Hash Computation: A cryptographic hash function is used to compute a hash of the combined plaintext chunk and extra data. This hash serves as the encryption key for that specific chunk, ensuring that identical chunks produce the same hash and thus the same encryption key.
Encryption: The plaintext chunk, along with the extra data, is then encrypted using the computed hash as the encryption key.
Manifest Creation: Lambda creates a manifest file that contains the keys (hashes) and pointers to the encrypted chunks in the storage subsystem. The manifest itself is encrypted with a customer-specific AWS Key Management Service, ensuring that only authorized entities can access it.
3 - Making Initialization Faster with SnapStart
When a Lambda function is invoked and there isn’t an already running execution environment, Lambda has to create a new one. This involves several steps:
Provisioning: Setting up a new virtual machine
Downloading: Retrieving the function code from storage
Initializing: Starting the runtime and initializing the function code.
These steps can take time, especially for complex functions or runtimes like Java, which have longer startup times. This delay is known as cold start latency.
SnapStart is a feature designed to significantly reduce cold start latency by pre-initializing the execution environment and then using snapshots.
Here’s how it works:
When you update your function code or deploy a new version, Lambda performs an initial setup and creates a new execution environment.
Once the execution environment is fully set up and ready to handle invocations, Lambda takes a snapshot of this environment. The snapshot captures the entire state of the VM, including the memory state, runtime state, and initialized data. The snapshot is then broken down into smaller pieces and encrypted to ensure security.
These encrypted chunks are stored securely.
When a new invocation request arrives, Lambda first checks if there’s an already running execution environment (a warm start). If not, it proceeds to use the snapshot. The necessary chunks are loaded and decrypted and the environment is brought back to the state it was in when the snapshot was taken.
Conclusion
AWS Lambda revolutionized the way developers build and deploy applications by providing a serverless environment that abstracts away the complexities of infrastructure management.
Here are the main takeaways from the internals of AWS Lambda:
To achieve the impressive statistics associated with Lambda, AWS leverages multiple advanced techniques such as chunking and convergent encryption to manage and share common data.
The introduction of SnapStart addressed the critical challenge of cold start latency, particularly for runtimes like Java.
The transition from the Worker Manager to the Assignment Service marked a significant improvement in state management.
With Firecracker microVMs, Lambda achieves lightweight and secure virtualization.
References:
SPONSOR US
Get your product in front of more than 500,000 tech professionals.
Our newsletter puts your products and services directly in front of an audience that matters - hundreds of thousands of engineering leaders and senior engineers - who have influence over significant tech decisions and big purchases.
Space Fills Up Fast - Reserve Today
Ad spots typically sell out about 4 weeks in advance. To ensure your ad reaches this influential audience, reserve your space now by emailing hi@bytebytego.com
Like
Comment
Restack
© 2024 ByteByteGo
548 Market Street PMB 72296, San Francisco, CA 94104
Unsubscribe
by "ByteByteGo" <bytebytego@substack.com> - 11:37 - 4 Jun 2024 -
Don't forget our API platform insights online panel discussion on 18 June!
Don't forget our API platform insights online panel discussion on 18 June!
Hi Md Abul,
Quick reminder! Our API platform insights online panel discussion, in collaboration with Research HQ, is happening soon. It's all about formulating a successful API platform strategy and accelerating your API platform maturity.
📅 Date: 18th of June
🕙 Time: 10 am EDT / 3 pm BST
📍 Location: ZoomJoin industry experts Jennifer Riggins, Nicki Watt and yours truly as we delve into the API platform insights 2024 report, exploring current trends in the world of API platforms, how teams are measuring success, and the impact of AI-based tools to help platform teams boost efficiency and ROI.
Thanks,
Budha & team
Tyk, 87a Worship Street, London, City of London EC2A 2BE, United Kingdom, +44 (0)20 3409 1911
by "Budhaditya Bhattacharya" <budha@tyk.io> - 06:00 - 4 Jun 2024 -
What’s the top concern for today’s business leaders?
Only McKinsey
Brought to you by Liz Hilton Segel, chief client officer and managing partner, global industry practices, & Homayoun Hatami, managing partner, global client capabilities
•
Global tensions intensify. Rising geopolitical tensions are testing the resilience of global organizations and challenging existing growth strategies. A new global cooperation barometer released by the World Economic Forum and McKinsey in January finds that peace and security have eroded sharply since 2020, McKinsey Global Institute director Olivia White and coauthors share. With 183 active conflicts in 2023, the intensity and duration of conflicts worldwide are at their highest levels since before the end of the Cold War.
•
The year of elections. More than 60 countries are holding national elections in 2024; shifts in leadership and policy may be looming. According to our latest global economic survey about 1,000 executives, business leaders view geopolitics as the top risk to global growth and political transitions as the leading emergent risk. Businesses that systematically build geopolitical resilience—for instance, through geopolitical-scenario planning—can differentiate themselves. Consider how global companies can respond to geopolitical uncertainty.
—Edited by Belinda Yu, editor, Atlanta
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this newsletter because you subscribed to the Only McKinsey newsletter, formerly called On Point.
Copyright © 2024 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "Only McKinsey" <publishing@email.mckinsey.com> - 01:31 - 4 Jun 2024 -
Re: Weekly update Shipping information fm China
Good day dear
Feel free to let me know if you need rates
My email: overseas.12@winsaillogistics.com
My Tel/whatsapp number:+86 13660987349
by "Yori" <overseas11@gz-logistics.cn> - 06:38 - 3 Jun 2024 -
Six paths to sustainable growth: A leader’s guide
Share this email
Brought to you by Liz Hilton Segel, chief client officer and managing partner, global industry practices, & Homayoun Hatami, managing partner, global client capabilities
When it comes to goal setting, achieving company growth that’s sustainable over time sounds like a no-brainer for every organization. Yet in an era of constant turbulence and disruption, the “how” of growing and of outperforming the competition—over time, no less—may feel especially hard to pin down. A strategic approach to sustainable, inclusive growth requires a willingness to take risks, the ability to see beyond the current economic cycle, and, above all, an explicit choice to grow. This week, we explore the key components of growth that can set companies up for success in the long run.
To grow profitably, consistently, and sustainably in the long term, companies must be courageous, innovative, and bold. But only a select few are meeting this mark, according to McKinsey’s Jill Zucker, Kate Siegel, and Rebecca Doherty. After studying how more than 4,000 global companies pursue growth, they have identified six strategies that help organizations outperform over time. Among the six paths to growth, the first two (strong commitments to innovation and to sustainable, inclusive growth) foster an aspirational, growth-focused culture. Growing the core business with tech, expanding selectively into adjacent businesses, and divesting with discipline enable a company to set growth in motion. And lastly, giving employees ownership for growth initiatives helps people build the skills necessary to capture value quickly and on an ongoing basis.
That’s the share of respondents in a recent resource allocation survey who say their companies’ top management encourages high-potential, risky projects—the very sort that help spur growth. In their research, McKinsey’s Andy West and Tim Koller also found that when management does support these initiatives, respondents are more likely to report outperforming on both revenue growth and return on capital. These findings confirm that much of the choice to grow depends on whether companies are putting resources against that growth and making bold moves.
That’s senior partner Michael Birshan and his coauthors explaining why, perhaps counterintuitively, innovation is especially critical to growth during uncertain times. According to the authors, “Innovation has always been essential to long-term value creation and resilience because it creates countercyclical and noncyclical revenue streams.” So if your company’s not actively exploring new paths to growth, it’s already behind. To pave the way for outperforming over time, a company’s best innovation moves are revisiting its innovation portfolio (which should include both short-term and long-term innovations), seeking new insights and opportunities in adjacent markets, and adapting its business model.
When it comes to future-proofing the business, what does it take for companies (and their leaders) to change the game rather than just play it? According to Lucy Parker, partner at the Brunswick Group, it’s an “activist mindset”: that is, “making real the concepts that need to happen related to change.” In a McKinsey Author Talks conversation about her book on leadership for the long term, Parker describes the need for activist leadership—and the activist mindset that’s required for creating social as well as financial value. When leaders recognize their own role—and that of their companies—in tackling systemic, societal challenges that also affect the long-term prospects of their business, they’re better positioned to leave the company better than they found it. “You win by moving into the offensive and saying, ‘What’s needed here?’” Parker says. “That and then stepping toward it constitutes an activist mindset.”
If you’re pursuing growth of the horticultural kind, you’re in luck. The global network of plant influencers seems larger than ever, and some amateur plant lovers are even going pro, with book deals, stores, and their own lifestyle brands. Meanwhile, others are leaving jobs in media, fashion, and tech start-ups to care for corporate florae in New York City. Despite the postpandemic decline in office attendance, which seems like it’s here to stay, plant care jobs are in high demand among some urban dwellers who, in search of greener pastures, have traded in their laptops for watering cans.
Lead by committing to growth.
— Edited by Daniella Seiler, executive editor, Washington, DC
Share these insights
Did you enjoy this newsletter? Forward it to colleagues and friends so they can subscribe too. Was this issue forwarded to you? Sign up for it and sample our 40+ other free email subscriptions here.
This email contains information about McKinsey’s research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to the Leading Off newsletter.
Copyright © 2024 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey Leading Off" <publishing@email.mckinsey.com> - 04:49 - 3 Jun 2024 -
How business leaders can jump-start an era of abundance
Only McKinsey
Brought to you by Liz Hilton Segel, chief client officer and managing partner, global industry practices, & Homayoun Hatami, managing partner, global client capabilities
•
An abundant future. Many business leaders are uncertain about how to help their companies prosper in an era of ongoing disruption. Yet companies can upskill workers, change how their organizations operate, strive to offset higher input prices and interest rates, and invest in capital and technology. Those that do have an opportunity to accelerate growth, boost productivity, and create a more prosperous and sustainable future, McKinsey senior partner Asutosh Padhi, McKinsey partner Ezra Greenberg, and McKinsey Global Institute chair Sven Smit explain.
—Edited by Belinda Yu, editor, Atlanta
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this newsletter because you subscribed to the Only McKinsey newsletter, formerly called On Point.
Copyright © 2024 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "Only McKinsey" <publishing@email.mckinsey.com> - 11:05 - 2 Jun 2024 -
We want guest post on your site https://www.smechamber.mt/
--HyWe want guest post on your site https://www.smechamber.mt/What is it's price?We want do-follow indexed permanent links.Also tell me do you accept casino related articles?We'll provide an article also.Payment via PayPal.Let us know,waiting to hear from you.Thanks.
by "Asghar Ali" <ghulamgoesmustafa76@gmail.com> - 02:48 - 2 Jun 2024
