Archives
- By thread 3428
-
By date
- June 2021 10
- July 2021 6
- August 2021 20
- September 2021 21
- October 2021 48
- November 2021 40
- December 2021 23
- January 2022 46
- February 2022 80
- March 2022 109
- April 2022 100
- May 2022 97
- June 2022 105
- July 2022 82
- August 2022 95
- September 2022 103
- October 2022 117
- November 2022 115
- December 2022 102
- January 2023 88
- February 2023 90
- March 2023 116
- April 2023 97
- May 2023 159
- June 2023 145
- July 2023 120
- August 2023 90
- September 2023 102
- October 2023 106
- November 2023 100
- December 2023 74
- January 2024 75
- February 2024 75
- March 2024 78
- April 2024 74
- May 2024 108
- June 2024 98
- July 2024 116
- August 2024 134
- September 2024 97
-
Delivering services to the public—digitally—with Jennifer Pahlka
Listen in New from McKinsey Global Institute
Delivering services to the public—digitally—with Jennifer Pahlka
Listen in Prefer audio? Listen to the podcast, and explore past episodes of the Forward Thinking Podcast. Subscribe via Apple Podcast or Spotify.
Forward Thinking on measuring the value of the digital age with Avinash Collis
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to our McKinsey Global Institute alert list.
Copyright © 2024 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey & Company" <publishing@email.mckinsey.com> - 02:57 - 14 Jun 2024 -
The real risk of gen AI
The Shortlist
Four new insights
by "McKinsey CEO Shortlist" <publishing@email.mckinsey.com> - 04:45 - 14 Jun 2024 -
Do you know which generations are most eager to travel?
Only McKinsey
Our 2024 report on tourism and hospitality Brought to you by Liz Hilton Segel, chief client officer and managing partner, global industry practices, & Homayoun Hatami, managing partner, global client capabilities
•
Generations that prioritize travel. Global travel is back. After declining by 75% in 2020, travel is on its way to making a full recovery by the end of 2024, with domestic travel still representing the bulk of the market, McKinsey senior partner Caroline Tufft and coauthors explain. To gauge what’s on the minds of present-day travelers, McKinsey surveyed more than 5,000 of them. The results show that younger generations (Gen Zers and millennials), in particular, show significant and increasing interest in travel.
—Edited by Belinda Yu, editor, Atlanta
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this newsletter because you subscribed to the Only McKinsey newsletter, formerly called On Point.
Copyright © 2024 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "Only McKinsey" <publishing@email.mckinsey.com> - 01:35 - 14 Jun 2024 -
A Crash Course on Cell-based Architecture
A Crash Course on Cell-based Architecture
No one wants to sail in a ship that can sink because of a single hull breach. This led to the development of bulkheads, which are vertical partition walls that divide a ship’s interior into watertight compartments. Cell-based architecture attempts to follow the same concept in software development.͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ Forwarded this email? Subscribe here for moreLatest articles
If you’re not a subscriber, here’s what you missed this month.
To receive all the full articles and support ByteByteGo, consider subscribing:
No one wants to sail in a ship that can sink because of a single hull breach.
This led to the development of bulkheads, which are vertical partition walls that divide a ship’s interior into watertight compartments.
Cell-based architecture attempts to follow the same concept in software development.
In cell-based architecture, there are multiple isolated instances of a workload, where each instance is known as a cell. There are three properties of a cell:
Each cell is independent.
A cell does not share the state with other cells.
Each cell handles a subset of the overall traffic.
For example, imagine a web application that handles user requests. In a cell-based architecture, multiple cells of the same web application would be deployed, each serving a subset of the user requests. These cells are copies of the same application working together to distribute the workload.
This approach reduces the blast radius of impact. If a workload uses 5 cells to service 50 requests, a failure in only one cell means that 80% of the requests are unaffected by the failure.
In other words, failure isolation is the biggest benefit of a cell-based architecture.
In this post, we will learn about the various aspects of cell-based architecture and its various components in more detail.
What is a Workload?...
Continue reading this post for free, courtesy of Alex Xu.
A subscription gets you:
An extra deep dive on Thursdays Full archive Many expense it with team's learning budget Like Comment Restack © 2024 ByteByteGo
548 Market Street PMB 72296, San Francisco, CA 94104
Unsubscribe
by "ByteByteGo" <bytebytego@substack.com> - 11:36 - 13 Jun 2024 -
Three reasons to attend our API insights webinar on 18 June
Three reasons to attend our API insights webinar on 18 June
Learn about critical API trends, future-proof your platform, and get actionable advice. Sign up today!Hi Md Abul,
Our API platform insights 2024 webinar, in partnership with ResearchHQ, is happening next week! We believe this event is a must-attend and have outlined three reasons below why it must be on your calendar for next week.
📅 Date: 18th of June
🕙 Time: 10 am EDT / 3 pm BST
📍 Location: ZoomHere's three practical reasons why you must attend:
- Uncover hidden digital infrastructure issues. Gain insights into critical challenges and learn where to act for maximum efficiency.
- Explore new API trends. With 26% of organisations using AI and 54% prioritising automation, stay updated on the latest advancements to future-proof your platform and team.
- Seize the opportunity to learn from industry experts. 46% of organisations focus on metrics that drive business value and ROI. Get practical advice to enhance your API strategy.
Register today, and you'll receive an exclusive copy of the full report and access to the on-demand recording after the event.
Thanks,
Budha & teamTyk, 87a Worship Street, London, City of London EC2A 2BE, United Kingdom, +44 (0)20 3409 1911
by "Budhaditya Bhattacharya" <budha@tyk.io> - 06:17 - 13 Jun 2024 -
What can AI do for business?
Only McKinsey
Breaking down barriers to integrating AI Brought to you by Liz Hilton Segel, chief client officer and managing partner, global industry practices, & Homayoun Hatami, managing partner, global client capabilities
•
Expanding AI use. All kinds of businesses will find merit in AI. Using AI has matured from manufacturing and risk to marketing and sales, product and service development, and strategy and corporate finance, according to research by McKinsey senior partner and QuantumBlack, AI by McKinsey, global leader Alex Singla and coauthors. Organizations that make bigger AI investments, scale more quickly, and recruit and train top AI talent are finding a competitive edge over those that don’t. But most organizations are currently only testing the AI waters. Explore three major shifts organizations can make to scale up AI.
—Edited by Querida Anderson, senior editor, New York
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this newsletter because you subscribed to the Only McKinsey newsletter, formerly called On Point.
Copyright © 2024 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "Only McKinsey" <publishing@email.mckinsey.com> - 01:47 - 13 Jun 2024 -
Carbon removals are vital to a net-zero future
Re:think
The market for carbon removals could be significant FRESH TAKES ON BIG IDEAS
ON CARBON REMOVALS
Why more businesses are counting on carbon removalsMark Patel
Reducing carbon emissions is fundamental to reaching net-zero targets. There’s no dispute about that. But, to date, there aren’t enough solutions at scale to reduce emissions fast enough. That’s why carbon removal has emerged as a vital way to help meet climate targets, to bridge the gap between emissions reduction and a net-zero future. A look at companies that have been transparent about their path to decarbonization shows that an increasing number are relying on negative emissions.
Carbon removal entails removing CO2 from the atmosphere and storing it somewhere environmentally safe, such as in oceans or in the ground. There are both nature-based and technology-based solutions to do this. Many of these solutions have great potential, from agricultural practices that enhance CO2 uptake into soils to direct air capture, which draws CO2 directly from the atmosphere. However, these solutions are not yet available at scale. Today, although the world might be removing thousands of tons of CO2 per year, to make a real impact on climate change, the world needs to remove gigatons (billions of tons) of CO2 per year.
Scaling the carbon removals industry is going to be an enormous task. For example, the supply side requires significant investment in innovation, technology, and infrastructure to drive down costs and support project development. Our analysis estimates that, to meet net-zero targets by 2050, cumulative investment in carbon removals would have to reach $6 trillion to $16 trillion. The demand side is also quite nascent, and there’s a lot of uncertainty. Voluntary carbon credit markets create a mechanism for purchasing removals, but the reality is that the big purchases of removals are being funded by a relatively small number of companies.
However, the carbon removal industry presents a significant opportunity for businesses. Our research indicates that the market for carbon removals could be worth up to $1.2 trillion by 2050. Early buyers of carbon removals could have an opportunity to solidify a supply of reliable, high-quality carbon removal credits as demand takes off.“Our research indicates that the market for carbon removals could be worth up to $1.2 trillion by 2050.”
There are supplier companies that have the capabilities, technologies, or infrastructure to find value in scaling up removals. According to our analysis, suppliers—businesses that generate carbon credits based on removals—could earn 70 to 80 percent of the expected $300 billion to $1.2 trillion in market revenues. That’s roughly $200 billion to $950 billion by 2050. There are also companies that can reduce their carbon footprints by deploying removal technologies. For example, one removal approach is called enhanced weathering, where rock dust is spread onto fields, capturing CO2 as it weathers. For agriculture or consumer goods companies in the food value chain, this activity could help reduce their net emissions.
For the carbon removal industry to scale, a few things need to happen. For one, there are practical limitations on how quickly companies can prove their technology works at scale. And then there’s the challenge of convincing investors and other sources of capital to fund nascent technologies. Advanced market commitments are a way for buyers to purchase future carbon removals and help solidify demand for suppliers, so they have a line of sight to their end customers while they scale. Governments can also stimulate carbon removal markets, such as with tax credits, as in the United States. This kind of support can help fund R&D, move carbon removal technologies down the cost curve, and encourage more investment.
It’s still early days for the carbon removal industry. Understanding of the technologies and market potential is low across most sectors. How carbon removal fits alongside other decarbonization activities is also not well understood. But given the net-zero commitments that companies and governments have already made—not even factoring in future commitments—the market for carbon removals is likely going to exist, and it could be big. There’s always a chance that market sentiment, regulation, and voluntary corporate behavior alter the course of things. But if companies play forward today’s trends, carbon removals could be an industrial segment in its own right.ABOUT THIS AUTHOR
Mark Patel is a senior partner in McKinsey’s Bay Area office.
MORE FROM THIS AUTHOR
UP NEXT
Michael Birshan on strategic courage
It’s not enough to ride out waves of economic and political uncertainty. These are the times when leaders need to be the boldest and bravest.
This email contains information about McKinsey’s research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to our McKinsey Quarterly alert list.
Copyright © 2024 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey Quarterly" <publishing@email.mckinsey.com> - 02:32 - 12 Jun 2024 -
Packaging Services
Hi, We are a custom packaging company based in NewYork, NY. We provide packaging services, such as custom printed boxes/labels and bags. We provide these services almost worldwide. Send over your artwork and confirm the quantity so I can assist you. Thanks. Best Regards, Austin America Custom Packaging, LLC.
by austin.americacustompackaging@gmail.com - 11:10 - 12 Jun 2024 -
Your Reliable Translation Partner Mars Translation
Hello there,
It’s Mona from Mars Translation who is a professional translation & localization company. We are seeking an opportunity to work with you. Could you share with me some information about your plans or translation project?
At Mars Translation, we provide 120+ languages & 20+ industries. To build trust, we are offering a free translation of the 300-word count of your project. Only then, after your satisfaction, we ask you to proceed with our services. This way you will know exactly what you are getting into before spending any time can translate/localizing your projects exactly to your expectations.
If it sounds interesting, please respond back so that we can discuss your project details.
If you have no need to, please reply with “unsub”, we will not send emails from our side.
Mona Xiang�0�2| Global Business Developer
Phone/WhatsApp: +86�0�2178 7209 0469
Don’t want to receive any further emails? Reply with ‘Unsub'
by "Mona Xiang" <mona.xiang@marstranslation.co> - 05:36 - 12 Jun 2024 -
I need a blog post on your site. https://www.odoo.com/
Hi Sir,I need a blog post on your site. https://www.odoo.com/How much your price per post
Waiting for your replyThanks
by "Sofia Luna" <sofia.lunaseo1@gmail.com> - 02:41 - 12 Jun 2024 -
How can companies get started with generative AI?
Only McKinsey
Criteria for choosing an AI provider Brought to you by Liz Hilton Segel, chief client officer and managing partner, global industry practices, & Homayoun Hatami, managing partner, global client capabilities
•
Strategies for adopting generative AI (gen AI). Gen AI is here to stay. But what does it take for companies to create real value from it? Companies gearing up for a gen AI reset first need to determine their strategy for adopting the technology, since choosing the right approach can help create a competitive edge, explain Alex Singla, a global leader of QuantumBlack, AI by McKinsey; Rodney Zemmel, the global leader of McKinsey Digital; and coauthors. In the “taker” model, for instance, companies use preexisting tools and models with little or no customization.
•
Picking a gen AI provider. Because of the complexities of working with gen AI, which may include a lack of experience in using the technology, it’s crucial to collaborate. Teaming up with a network of gen AI providers will be critical to capturing the technology’s full value. But at the same time, selecting the right providers has become challenging as the number of AI vendors and large language models grows. Consider three criteria for choosing a gen AI provider, and visit McKinsey Digital for more on how organizations can realize value from technology.
—Edited by Belinda Yu, editor, Atlanta
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this newsletter because you subscribed to the Only McKinsey newsletter, formerly called On Point.
Copyright © 2024 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "Only McKinsey" <publishing@email.mckinsey.com> - 01:18 - 12 Jun 2024 -
Looking for a Guest Post
Hi Sir/Mam,
I have visited your website and it looks good. I am looking to publish my article on your site and need a permanent do-follow backlink. Will you accept my guest post? In case you accept guest posts, tell me the topic and criteria to write an article for you. Also, tell me the prices for general as well as casino/betting and CBD posts if you are accepting sponsored posts and link insertions.
Looking for your positive feedback.
by "Asher Owen" <asherowen929@gmail.com> - 11:47 - 11 Jun 2024 -
Re: to give the update changes of o///f from hong--ocean
HEY valued customers,
Greetings from Hongocean! We are pleased to inform you about our updated sea freight prices.
The sea freight charges from Yantian to Los Angeles are currently priced at $6100,
The shipping fees from Shanghai to New York stand at $7100.
These revised fees will remain valid until June 14, 2024.
Should you have any further queries or require assistance, could you feel gratis to contact our dedicated team. We appreciate your continued support and look forward to serving you with our enhanced sea freight services.
Warm regards
Jeffery
by "Jeffery" <azemin29@hongoceanlog.com> - 12:07 - 11 Jun 2024 -
How PayPal Scaled Kafka to 1.3 Trillion Daily Messages
How PayPal Scaled Kafka to 1.3 Trillion Daily Messages
Database Performance at Scale: A Practical Guide [FREE BOOK] (Sponsored) Discover new ways to optimize database performance – and avoid common pitfalls – in this free 270-page book. This book shares best practices for achieving predictable low latency at high throughput. It’s based on learnings from thousands of real-world database use cases – including Discord, Disney, Strava, Expedia, Epic Games & more.͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ Forwarded this email? Subscribe here for moreDatabase Performance at Scale: A Practical Guide [FREE BOOK] (Sponsored)
Discover new ways to optimize database performance – and avoid common pitfalls – in this free 270-page book.
This book shares best practices for achieving predictable low latency at high throughput. It’s based on learnings from thousands of real-world database use cases – including Discord, Disney, Strava, Expedia, Epic Games & more.
Explore often-overlooked factors that impact database performance at scale
Recognize the performance challenges teams face with different types of workloads
Select database infrastructure and topology that’s suited to your needs
Optimize how you benchmark and monitor performance
Avoid common mistakes that impact latency and throughput
Get practical advice for navigating performance tradeoffs
Disclaimer: The details in this post have been derived from the article originally published on the PayPal Tech Blog. All credit for the details about PayPal’s architecture goes to their engineering team. The link to the original article is present in the references section at the end of the post. We’ve attempted to analyze the details and provide our input about them. If you find any inaccuracies or omissions, please leave a comment, and we will do our best to fix them.
In the 2022 Retail Friday, PayPal’s Kafka setup witnessed a traffic volume of 21 million messages per second.
This was about 1.3 trillion messages in a single day.
Similarly, Cyber Monday resulted in 19 million messages per second coming to around 1.23 trillion messages in a single day.
How did PayPal scale Kafka to achieve these incredible numbers?
In this post, we will go through the complete details of PayPal’s high-performance Kafka setup that made it possible.
Kafka at PayPal
Apache Kafka is an open-source distributed event streaming platform.
PayPal adopted Kafka in 2015 and they use it for building data streaming pipelines, integration, and ingestion.
At present, PayPal’s Kafka fleet consists of over 1500 brokers hosting 20,000 topics. The 85+ clusters are expected to maintain a 99.99% availability.
Over the years, PayPal has seen tremendous growth in streaming data and they wanted to ensure high availability, fault tolerance, and optimal performance.
Some of the specific use cases where PayPal uses Kafka are as follows:
First-party Tracking: Tracking user interactions and clickstream data on PayPal’s website and mobile app for real-time analytics and personalization.
Application Health Metrics: Collecting and aggregating performance metrics from various PayPal services to monitor system health and detect anomalies.
Database Synchronization: Synchronizing data between PayPal’s primary database and secondary databases for disaster recovery and high availability.
Application Log Aggregation: Collecting and centralizing log data from different PayPal applications and services for troubleshooting, monitoring, and analysis.
Batch Processing: Processing large batches of payment transaction data using Kafka as a buffer for decoupling the data ingestion and processing stages.
Risk Detection and Management: Streaming real-time payment data through Kafka for feeding the fraud detection and risk assessment models.
Analytics and Compliance: Capturing and analyzing financial transaction data in real-time for regulatory reporting and audit purposes.
How PayPal Operates Kafka?
PayPal’s infrastructure is spread across multiple geographically distributed data centers and security zones. Kafka clusters are deployed across these zones.
There are some key differences between data centers and security zones:
A data center is a physical facility that houses computing infrastructure. A security zone is a logical partition with a data center or across data centers, created through network segmentation.
While data centers help with isolation and availability, security zones provide an additional level of security isolation beyond the physical boundaries.
Security zones are often defined based on data classification levels, such as highly sensitive, confidential, or public data.
In the context of PayPal, Kafka clusters handling sensitive payment data may be placed in a high-security zone with restricted access. Clusters processing less sensitive data may reside in a different security zone.
One thing, however, is common.
Whether it is data centers or security zones, MirrorMaker is used to mirror the data across the data centers, which helps with disaster recovery and communication across security zones.
For reference, Kafka MirrorMaker is a tool for mirroring data between Apache Kafka clusters. It leverages the Kafka Connect framework to replicate data, which improves resiliency.
See the diagram below to get an idea about PayPal’s Kafka setup across data centers and security zones:
Operating Kafka at the scale of PayPal is a challenging task. To manage the ever-growing fleet of Kafka clusters, PayPal has focused on some key areas such as:
Cluster Management
Monitoring and Alerting
Configuration Management
Enhancements and Automation
The diagram below shows a high-level view of PayPal’s Kafka Landscape:
In the subsequent sections, we will look at each area in greater detail.
Cluster Management
Cluster management deals with controlling Kafka clusters and reducing operational overhead. Some of the key improvements were done in areas like:
Kafka Config Service
ACLs
Kafka Libraries for PayPal
QA environment
Let’s look at each improvement in more detail.
Latest articles
If you’re not a paid subscriber, here’s what you missed.
To receive all the full articles and support ByteByteGo, consider subscribing:
Kafka Config Service
PayPal built a special Kafka config service to make it easy for clients to connect to the Kafka clusters.
Before the config service, clients had to hardcode the broker IPs in the connection configuration.
This created a maintenance nightmare due to a couple of reasons:
Firstly, replacing a broker for upgrades, patching, or disk failures required updates to the client. Missing to update one broker IP somewhere would often lead to multiple incidents.
Second, Kafka is configuration-heavy, making it tough for developers to figure out a suitable set of configurations. Often, the Kafka clients would override certain properties without knowing the implications, resulting in support issues due to unexpected behavior.
The Kafka config service solved these issues. The service makes it easy for the Kafka clients to follow a standard configuration. Ultimately, it reduces operational and support overhead across teams.
The diagram below shows the Kafka client retrieving bootstrap servers and configuration from the config service using the topic information.
Kafka ACLs
Initially, any PayPal application could connect to any of the existing Kafka topics. This was an operational risk for the platform considering that it streams business-critical data.
To ensure controlled access to Kafka clusters, ACLs were introduced.
For reference, ACLs are used to define which users, groups, or processes have access to specific objects such as files, directories, applications, or network resources. It’s like a table or list specifying a particular object's permissions.
With the introduction of ACLs, applications had to authenticate and authorize access to Kafka clusters and topics.
Apart from making the platform secure, ACLs also provided a record of every application accessing a particular topic or cluster.
PayPal Kafka Libraries
It was important to ensure that Kafka clusters and the clients connecting to them operate securely. Also, there was a need to ensure easy integration to multiple frameworks and programming languages. They didn’t want each engineering team to reinvent the wheel.
Supported tech stack for Kafka Libraries. Source: PayPal Tech Blog To facilitate these needs, PayPal built a few important libraries:
Resilient Client Library: When a client tries to establish a connection to the cluster, this library gets the Kafka broker details and the required configuration for the producer or consumer application.
Monitoring Library: This library publishes critical metrics for client applications, allowing applications to set alerts and get notifications in case of any issues.
Kafka Security Library: The Kafka platform supports more than 800 applications. This library takes care of the required certificates and tokens to enable SSL authentication for applications connecting to the Kafka clusters. It avoids a lot of overhead around key management, certificate updates, and key rotations.
QA Platform
One of the great things PayPal did was to set up a production-like QA platform for Kafka for developers to test changes confidently.
This is a common problem in many organizations where the testing performed by developers is hardly indicative of the production environment, resulting in issues after launch.
A dedicated QA platform solves this by providing a direct mapping between production and QA clusters.
The same security standards are followed. The same topics are hosted on the clusters with the brokers spread across multiple zones within the Google Cloud Platform.
Monitoring and Alerting
Monitoring and alerting are extremely important aspects for systems operating at a high scale. Teams want to know about issues and incidents quickly so that cascading failures can be avoided.
At PayPal, the Kafka platform is integrated with the monitoring and alerting systems.
Apache Kafka provides multiple metrics. However, they have taken out a subset of metrics that help them identify issues faster.
The Kafka Metrics library filters out the metrics and sends them to the SignalFX backend via SignalFX agents running on all brokers, Zookeepers, MirrorMakers, and Kafka clients. Individual alerts associated with these metrics are triggered whenever abnormal thresholds are breached.
Configuration Management
Operating a critical system requires one to guard against data loss. This is not only applicable to the application data but also to the infrastructure information.
What if the infrastructure gets wiped out and we’ve to rebuild it from scratch?
At PayPal, configuration management helps them store the complete infrastructure details. This is the single source of truth that allows PayPal to rebuild the clusters in a couple of hours if needed.
They store the Kafka metadata such as topic details, clusters, and applications in an internal configuration management system. The metadata is also backed up to ensure that they have the most recent data in case it’s required to re-create clusters and topics in case of a recovery.
Enhancements and Automation
Large-scale systems require special tools to carry out operational tasks as quickly as possible.
PayPal built multiple such tools for operating their Kafka cluster. Let’s look at a few important ones:
Patching Security Vulnerabilities
PayPal uses BareMetal for deploying the Kafka brokers and virtual machines for Zookeeper and MirrorMakers.
As we can expect, all of these hosts need to be patched at frequent intervals to fix any security vulnerabilities.
Patching requires BM restart which can cause partitions to lag. This can also lead to data loss in the case of Kafka topics that are configured with a replica set of three.
They built a plugin to query whether a partition was lagging before patching the host, thereby ensuring only a single broker is patched at a time with no chances of data loss.
Topic Onboarding
Application teams require topics for their application functionality. To make this process standardized, PayPal built an Onboarding Dashboard to submit a new topic request.
The diagram below shows the onboarding workflow for a topic.
A special review team checks the capacity requirements for the new topic and onboards it to one of the available clusters. They use a capacity analysis tool integrated into the onboarding workflow to make the decision.
For each new application being onboarded to the Kafka system, a unique token is generated. This token is used to authenticate the client’s access to the Kafka topic. As discussed earlier, an ACL is created for the specific application and topic based on their role.
MirrorMaker Onboarding
As mentioned earlier, PayPal uses MirrorMaker for mirroring the data from one cluster to another.
For this setup, developers also use the Kafka Onboarding UI to register their requirements. After due checks by the Kafka team, the MirrorMaker instances are provisioned.
The diagram below shows the process flow for the same:
Conclusion
The Kafka platform at PayPal is a key ingredient for enabling seamless integration between multiple applications and supporting the scale of their operation.
Some important learnings to take away from this study are as follows:
Tooling is a must when operating Kafka on a large scale. This involves operations such as cluster installation, topic management, patching VMs, etc.
Availability is as good as the ability of alerting and monitoring systems to provide timely inputs to the infrastructure team.
ACLs are a great way to have a better understanding of how the various applications are connected with Kafka.
A dedicated QA environment is critical for developers to ship changes with confidence and speed.
References
SPONSOR US
Get your product in front of more than 500,000 tech professionals.
Our newsletter puts your products and services directly in front of an audience that matters - hundreds of thousands of engineering leaders and senior engineers - who have influence over significant tech decisions and big purchases.
Space Fills Up Fast - Reserve Today
Ad spots typically sell out about 4 weeks in advance. To ensure your ad reaches this influential audience, reserve your space now by emailing hi@bytebytego.com
Like Comment Restack © 2024 ByteByteGo
548 Market Street PMB 72296, San Francisco, CA 94104
Unsubscribe
by "ByteByteGo" <bytebytego@substack.com> - 11:35 - 11 Jun 2024 -
New Relic launches Secure Developer Alliance
New Relic
May 2024New Relic launches Secure Developer Alliance The Secure Developer Alliance is a coalition of industry leaders that provides a best-of-platform approach to observable application security.
Read the blog How Skyscanner uses game days to train engineers on observabilityFlight and travel search engine Skyscanner uses OTel standards to create a new game day format to speed up training in observability using New Relic.Connecting SLOs to business metrics with Thinkific
Thinkific implements service-level objective monitoring and introduces standard KPIs for all services. Here’s how.Useful Reads2024 State of the Java Ecosystem Report
Get context and insights into the current state of the Java ecosystem. Discover the most popular versions used, the most popular frameworks, and more.
Elsevier consolidates tools and moves to the cloud
Elsevier makes several hundred thousand dollars in cost savings by reducing the number of monitoring tools and eliminating the white noise that prevented swift issue resolution.
Upcoming EventsJoin our New Relic user meetups across Europe this June
Join us at one of our user meetup events in Paris, Amsterdam, Manchester and London this June. We’ve got a jam-packed Agenda, with food and drinks, where you’ll hear from New Relic engineers and other customers in your area.
Find out what’s new at New Relic, including mobile user journeys, Session Replay, AI monitoring, and New Relic AI. We’ll also give you a sneak peek into our upcoming limited previews, as well as our roadmap.Register now New Relic University online workshops
Join our upcoming New Relic online live training workshop, "Maximising ovservability with New Relic logs," on June 27 at 10am BST / 11am CEST.
This is a 90-minute, trainer-led workshop with hands-on labs that will help you up level your observability skills.Register here New Relic End-of-Life UpdatesSupport for PromQL (July 15)
- We’re standardizing our querying experiences around NRQL by removing PromQL-styled query support.
- You can still access Prometheus metrics and events, but will need to adopt NRQL as the method for querying Prometheus data.
-
If you’re currently using PromQL-styled queries to query your data, you’ll need to adopt NRQL.
-
Prometheus metrics data will still be accessible in New Relic.
- For more information, see our documentation.
Data Explorer (September 30)
- We’re migrating the data explorer tool from the metrics and events capability to the query your data capability.
- We’re unifying querying capabilities into a single experience to streamline how customers can access their data.
-
If you want to use the data explorer tool, you’ll now find it under the query your data capability from https://one.newrelic.com > All capabilities.
-
See here for more information.
Need help? Let's get in touch.
This email is sent from an account used for sending messages only. Please do not reply to this email to contact us—we will not get your response.
This email was sent to info@learn.odoo.com Update your email preferences.
For information about our privacy practices, see our Privacy Policy.
Need to contact New Relic? You can chat or call us at +44 20 3859 9190.
Strand Bridge House, 138-142 Strand, London WC2R 1HH
© 2024 New Relic, Inc. All rights reserved. New Relic logo are trademarks of New Relic, Inc
by "New Relic" <emeamarketing@newrelic.com> - 05:23 - 11 Jun 2024 -
How does the women’s health gap affect the global economy?
Only McKinsey
View our women’s health interactive Brought to you by Liz Hilton Segel, chief client officer and managing partner, global industry practices, & Homayoun Hatami, managing partner, global client capabilities
•
Closing the women’s health gap. Because of disparities in care delivery, data, and efficacy, women spend 25% more time in poor health than men do. A recent analysis from the McKinsey Health Institute, in collaboration with the World Economic Forum, finds that closing the global women’s health gap could result in better overall health, fewer early deaths, and as much as a $1 trillion economic boost, McKinsey senior partner Lucy Pérez and coauthors share.
—Edited by Belinda Yu, editor, Atlanta
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this newsletter because you subscribed to the Only McKinsey newsletter, formerly called On Point.
Copyright © 2024 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "Only McKinsey" <publishing@email.mckinsey.com> - 11:11 - 10 Jun 2024 -
How not to be a bad boss: A leader’s guide
Bad company Brought to you by Liz Hilton Segel, chief client officer and managing partner, global industry practices, & Homayoun Hatami, managing partner, global client capabilities
Horrible Bosses. The Devil Wears Prada. Tyrannical managers are often the subject of office comedies. But behind the satire lies a darker reality. Despite some improvements in workplace practices, many leaders may still set unreasonable expectations, display favoritism, steal credit, use inappropriate language, or discriminate against employees who exhibit specific characteristics. Leaders who indulge in or condone toxic behavior can erode employee performance and morale, harm customer relationships, and expose the organization to litigation. Here are some ways to ensure that your workplace remains civilized and respectful.
“Fundamentally, leaders need to make other people better,” says business psychology professor Tomas Chamorro-Premuzic in a conversation with McKinsey senior partner Brooke Weddle and partner Bryan Hancock. “There should be less focus on individualistic, selfish, egotistical, and career-enhancing traits and dispositions and more focus on those that make other people better.” Unfortunately, some leaders may be “overconfident, narcissistic, and incompetent” because traditional corporate culture is likely to favor these traits, says Chamorro-Premuzic. To counter this tendency, bosses could consider honing their intrapersonal and interpersonal skills. “If you’re managing people, it’s important that you also manage yourself,” he says. “It’s important that you evoke and display the right emotions. It’s important that you can empathize with others and connect with them on a human level.”
That’s the number of factors that organizations may need to consider to change the behavior of managers—and increase employee satisfaction at work. For example, research shows that as people gain power, they tend to lose their ability to understand people and situations accurately, leading to a deficit in empathy. Leadership training courses that emphasize soft skills, as well as programs to encourage mindfulness and self-awareness, may counter this to some extent. Formal mechanisms that reinforce desirable behaviors could also help: for instance, employee satisfaction with an immediate boss could become part of that leader’s performance evaluation.
That’s former marketing executive Dalia Feldheim in a discussion with McKinsey on how so-called feminine leadership qualities such as empathy and intuition may help combat toxic work environments and the resulting employee burnout. In a business world that often emphasizes competition and power over people, what may be called for is “emotional bravery,” which, Feldheim says, “goes way beyond emotional intelligence—it allows emotions and empathy to come out. The role of the leader is to elicit empathy.”
During his 11-year tenure as CEO of Merck, Ken Frazier gained widespread respect for spurring organizational performance without compromising his strong personal values. “I thought it was important to be humble,” the former executive chairman says in a conversation with McKinsey senior partner Vik Malhotra and senior partner alumnus Steve Van Kuiken. “My dad was a janitor, and he was ten feet tall in my eyes. I also knew that when I was in that [CEO] chair, my job was to serve my employees and patients; it was not about me.” Rather than exerting his power, Frazier adopted an equitable approach to leadership that involved nurturing and promoting top talent, delegating key decisions, and exhorting teams to work toward the common good. “When a company is successful, the CEO gets a lot of credit for what I call the big moments, but leadership is in the many small, quiet moments with the team,” he says. “You have to assemble the right talent and figure out how to work together, making sure the company has the right intensity, operational cadence, and accountability.”
Truly toxic and destructive bosses may leave employees with no option but to quit. But not all dysfunctional leaders are inherently bad people; in such cases, it may be worthwhile for workers to adapt to their managers’ styles. Empathy, communicating clearly and frequently, and figuring out preferred work habits may help you find your groove with a difficult boss. Research led by McKinsey senior partners Aaron De Smet and Arne Gast and colleagues shows that old hierarchical models of leadership may be giving way to collaborative, connected, and compassionate ways of working that bring out everyone’s potential within the organization.
Lead by being a good boss.
— Edited by Rama Ramaswami, senior editor, New York
Share these insights
Did you enjoy this newsletter? Forward it to colleagues and friends so they can subscribe too. Was this issue forwarded to you? Sign up for it and sample our 40+ other free email subscriptions here.
This email contains information about McKinsey’s research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to the Leading Off newsletter.
Copyright © 2024 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey Leading Off" <publishing@email.mckinsey.com> - 04:24 - 10 Jun 2024 -
I am interested in your site https://besrseoskills.odoo.com/
I am interested in your site https://besrseoskills.odoo.com/ for guest posting.
Could you please provide the following information.
1: The cost per guest post on your site.
2: Do you accept non-English content and posts related to casino.
If you have additional sites available for guest posting, kindly share a list along with the respective charges.
by "Abdullah Seo" <abdullahseo83@gmail.com> - 03:36 - 10 Jun 2024 -
What future job skills might European and US employers need most?
Only McKinsey
Fastest-growing occupations through 2030 Brought to you by Liz Hilton Segel, chief client officer and managing partner, global industry practices, & Homayoun Hatami, managing partner, global client capabilities
•
Developing AI skills. US tech workers are trying to stand out in a crowded field as demand for AI skills rises, the Wall Street Journal reports. Job applicants are updating their résumés with keywords such as “ChatGPT” and “Copilot” and rushing to improve their AI capabilities by enrolling in AI boot camps. Companies are recruiting people who have experience with large language models that power sophisticated chatbots, but there’s a shortage of workers with such qualifications. [WSJ]
•
Staying competitive. Workers and organizations across the globe are focusing on AI to stay relevant, according to CNBC. A 2024 survey by LinkedIn and Microsoft has found that 55% of leaders are worried that a shortage of talent will make it tough to fill jobs. The survey, which included more than 30,000 workers in 31 countries, also found that nearly eight in ten executives think that their company needs to embrace AI to keep up with the competition. [CNBC]
•
Shifting labor demand. Europe and the US face shifts in labor demand, spurred by AI and automation. Demand for workers in health- and STEM-related fields could grow by 17 to 30% between 2022 and 2030. Demand for occupations such as office, production, and customer service workers could decline in that period. In a midpoint adoption scenario, up to 30% of current hours worked in the two regions could be automated by 2030 if accelerated by generative AI, McKinsey Global Institute chair Sven Smit and coauthors find.
•
Occupational shifts. By 2030, Europe could require up to 12 million occupational transitions. In the US, required career transitions could also reach about 12 million. In response to those shifts, organization leaders will need to upskill their workforces. Demand for both technological and social and emotional skills could rise substantially in both regions. Enhancing human capital while rapidly deploying AI could revive productivity growth. Consider four priorities that may protect organizations’ competitive advantage in an era of AI and automation.
—Edited by Belinda Yu, editor, Atlanta
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this newsletter because you subscribed to the Only McKinsey newsletter, formerly called On Point.
Copyright © 2024 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "Only McKinsey" <publishing@email.mckinsey.com> - 01:39 - 10 Jun 2024 -
The week in charts
The Week in Charts
Interest rate projections, what employees value, and more Share these insights
Did you enjoy this newsletter? Forward it to colleagues and friends so they can subscribe too. Was this issue forwarded to you? Sign up for it and sample our 40+ other free email subscriptions here.
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to The Week in Charts newsletter.
Copyright © 2024 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey Week in Charts" <publishing@email.mckinsey.com> - 03:32 - 8 Jun 2024