Archives
- By thread 3426
-
By date
- June 2021 10
- July 2021 6
- August 2021 20
- September 2021 21
- October 2021 48
- November 2021 40
- December 2021 23
- January 2022 46
- February 2022 80
- March 2022 109
- April 2022 100
- May 2022 97
- June 2022 105
- July 2022 82
- August 2022 95
- September 2022 103
- October 2022 117
- November 2022 115
- December 2022 102
- January 2023 88
- February 2023 90
- March 2023 116
- April 2023 97
- May 2023 159
- June 2023 145
- July 2023 120
- August 2023 90
- September 2023 102
- October 2023 106
- November 2023 100
- December 2023 74
- January 2024 75
- February 2024 75
- March 2024 78
- April 2024 74
- May 2024 108
- June 2024 98
- July 2024 116
- August 2024 134
- September 2024 94
-
Inquiry about your product
Hello and good morning to you!
I am Anna Nedelcu from Elowenastrid Co.,Ltd and we run a wholesale company trading on your products. We are interested in importing from your company. Please advise your very best prices and sales condition.
Furthermore, I would like to know if you will attend any exhibition trade fair to allow me to visit you there.
We look forward to hearing from you.
Thank you for your cooperation and best regards.Anna Nedelcu (Mrs)
Purchasing Manager/Import department
Elowenastrid 95 CO., LTD.
Ármin sétány 59. 40. Emelet, Hungary.
by "Anna Nedelcu" <info@grouptradingintl.com> - 03:55 - 7 Aug 2024 -
Nursery Landscape Expo 2024
Hope you are doing well.
I am just following up to see if you are interested in acquiring the Visitor/Attendees list of
Events Name: Nursery Landscape Expo 2024
Date : 14 - 16 Aug 2024
Location : Henry B. Gonzalez Convention Center, San Antonio, USA
Counts : 1,600
Each record of the list contains: Contact Name, Email Address, Company Name, URL/Website, Phone No, Title/Designation.
Let us know your thoughts on this, so that we can send you the Discounted cost and Additional Information.
Thanks & Regards,
Ava
by "Ava Max" <avamax.infodata@gmail.com> - 02:38 - 7 Aug 2024 -
Generic no more: Introducing branded real estate
Re:think
Real estate brands are rising FRESH TAKES ON BIG IDEAS
ON REAL ESTATE BRANDS
The power of branding and CX in residential real estateAlex Wolkomir
“We’ve seen real estate companies automate more than 70 percent of interactions with residents, using AI companions and other tools, with good results.”
To get started on building strong real estate brands, the first step for owners and operators is to ask, “Whom am I actually trying to serve?” The answer should go beyond “a renter” or “a buyer.” It should drill into lifestyle preferences and values—of course, in a way that is suitable and feasible for the location.
The second step is to think deeply about what matters to this type of resident. What problems can the building owner help solve, and where are the opportunities to delight the resident in ways that really resonate?
The third step is implementing a system of data and measurement. Real estate companies need to understand the ROI of each investment. For example, were tenants with 200-square-foot bedrooms more likely to renew than those with 180-square-foot bedrooms? Getting this data is one of the biggest challenges for the real estate industry today. If you don’t measure it, you can’t act on it.
In ten to 15 years, the market is going to see more new developments with national or international real estate brands, and many will be linked to existing retail, wellness, or fitness brands. There will also be more brand synergies, where residents receive targeted offers or deals because they live in a building within a brand or brand family. Strong brands will attract like-minded residents, the most satisfied of whom will spread the word, creating a marketing flywheel. In a competitive environment for capital, assets that improve the tenant experience will stand apart.
All of this will depend on modern tech stacks and data infrastructure that cannot be built overnight. Leading companies are thinking through brand architecture, asking questions such as, “Do I have multiple products under my umbrella brand?” And they are investing in tech stack transformations. Look out for some exciting, boundary-pushing built-environment experiences ahead!ABOUT THIS AUTHOR
Alex Wolkomir is a partner in McKinsey’s New York office.
MORE FROM THIS AUTHOR
McKinsey insights for the fall
What questions are CEOs asking about generative AI? What does it take to be a digital leader? What might the future hold for the space economy, the travel industry, and organizational health? McKinsey’s experts will rethink these and other crucial business and leadership topics when this column returns in September.
This email contains information about McKinsey’s research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this email because you subscribed to our McKinsey Quarterly alert list.
Copyright © 2024 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "McKinsey Quarterly" <publishing@email.mckinsey.com> - 02:38 - 7 Aug 2024 -
Get This Domain-OpenOnline.com
Great news! The domain OpenOnline.com is now available on GoDaddy!
The Internet is the most efficient way to acquire new customers.
Anytime someone types Open Online, Open Online Services, The Best Open Online Services, or any other phrase with these keywords into their browser, your site could be the first they see!
Avg Google Search Results for this domain is: 231,000,000
You can easily redirect all the traffic this domain gets to your current site!
GoDaddy.com appraises this domain at $1,096.
Priced at only $398 for a limited time! To claim it, go to Open-Online.com and select Buy Now or purchase directly at GoDaddy.
Since we're contacting multiple interested parties about this domain, quick action is recommended if you're interested.
Thank you for your time.
Narie Pante
by "Narie Pante" <yourdomaingurunarie6@gmail.com> - 05:09 - 7 Aug 2024 -
Solutions to support and grow your business
Solutions to support and grow your business
Hi there,
At CRM Management Ltd, we specialise in offering bespoke development services tailored to your company’s unique needs. If you haven’t found a solution that fits your requirements, our team of experts will work with you to design and develop the perfect system to support and grow your business.
With many years of experience across a broad range of solutions, we can create anything from a simple Customer Relationship Management system to a comprehensive multi-channel ERP system. Here are some of the solutions we offer:
- Ticketing systems
- Booking systems
- Estate Agency systems
- Recruitment systems
- Customer Relationship Management (CRM) systems
- E-commerce systems
- Enterprise Resource Planning (ERP) systems
- Bespoke systems
Do you have any projects in mind? Simply reply to this email, and we can schedule a call.
Kind regards,
Andrew Miller
CRM Management Ltd
01759 361 845
I have sent this email to info@learn.odoo.com. If you don't want to get any more emails from us, you can stop them here.
CRM Management Ltd - English House, Railway St, Pocklington, York YO42 2QZ - Company no. 10684534
by "Andrew Miller" <Andrew@crmcompany.co.uk> - 04:11 - 7 Aug 2024 -
hi dear please check my sites for link insertion or guest post
HiMy name is jimmy i am provide guest posting to our clientI am working in seo company in London i have many siteswhich you need to post your article my sites have goodsemrush traffic or also good Daplease check my some sites;sites; domain authority traffichttps://netizensreport.com/ 37 36.4khttps://celebrow.org/ 57 4.12khttps://therightmessages.org/ 65 3.01khttps://cartoonwise.com/ 57 2.40khttps://digitoont.com/ 37 1.02ki am always waiting for your responsebest regards fromjimmy
by "Jimmy Bond" <jimmybond994@gmail.com> - 02:13 - 7 Aug 2024 -
What is economic inclusion?
Only McKinsey
Where the economically excluded live •
Beyond basic needs. The world has made great strides in addressing extreme poverty, but there’s still a long way to go. True economic inclusion goes beyond what people need for basic subsistence. An inclusive economy provides opportunities to underserved people and communities, according to McKinsey senior partner Liz Hilton Segel. It also creates higher-wage, more fulfilling jobs and ensures people’s mental health needs are met.
•
Economically excluded. About 4.4 billion people that are not economically empowered live in low- and middle-income countries; nearly half are in sub-Saharan Africa and India. But more than 300 million people in high-income countries also fall into this category, including just over a quarter of the population in the US, the EU, and the UK, according to research by MGI chair Sven Smit and coauthors. Discover how the rise of generative AI could affect economic inclusion.
—Edited by Belinda Yu, editor, Atlanta
This email contains information about McKinsey's research, insights, services, or events. By opening our emails or clicking on links, you agree to our use of cookies and web tracking technology. For more information on how we use and protect your information, please review our privacy policy.
You received this newsletter because you subscribed to the Only McKinsey newsletter, formerly called On Point.
Copyright © 2024 | McKinsey & Company, 3 World Trade Center, 175 Greenwich Street, New York, NY 10007
by "Only McKinsey" <publishing@email.mckinsey.com> - 01:26 - 7 Aug 2024 -
special offer these all high quality sites only in $1000 this offer only for this month
i am a linkbuilder i am providing high traffic sites with an special offermy sites have good traffic and also good Dacheck my sites
by "Petunia Atkinson" <petuniaatkinson832@gmail.com> - 01:21 - 7 Aug 2024 -
this is an very special offer only for a week these all 40 sites just in $1000
hihi my name is pat cummins i am providing guest posting at reasonable pricesmy sites have good traffic i will give you discount if you give me bulk ordercheck my sites;site;https://itsreleased.co.uk/
https://nightcloakeddeck.co.uk/
https://uktechtimes.com
https://ps5restock.co.uk/
https://hellotostartups.com/
https://krowdapp.co.uk
https://nytconnectionshint.com/
https://mean0.com/
https://upmcshiftselect.com/
https://pircod.com/
https://implogs.com/
https://hykept.com/
dfchecking.com
https://adsswift.com/
https://clickmorestuff.com/
https://arytime.com/i am waiting for your positive responsewaiting order
by "Pat Cummins" <cumminspat626@gmail.com> - 02:23 - 6 Aug 2024 -
i am providing casino , loan , cbd and also general post
Hi SirI am providing a guest posting and link insertion service by working with me, the quality of your site will be high, I will get your site to the top of Google.I will make your site a professional siteAnd I will provide all this to you with the sale offercheck sitessites traffichttps://fabcelebbio.com/ 3.96ki am waiting orderthanks & regards
by "Ashley Haugh" <ashleyhaugh39@gmail.com> - 02:16 - 6 Aug 2024 -
i am a link builder & I've a good number of quality sites
Hii am a providing guest post and link insertioni have lots of sites with High traffic or DAi need your order on my quality sitesi accept all category general ,casino,gambling ,adult all content are acceptedplease check my some sitessites traffici am waiting order
by "alyx root" <alyxroot@gmail.com> - 02:13 - 6 Aug 2024 -
please check my site for post your article
HiHi sir and mam i am working in an guest post company and providing guesti have sites with high traffic i hope my sites inspire youand we work together for a long timealso i have full list of high traffic sitesplease check my some high traffic sitessites traffichttps://cavegreen.us/ 51.1khttps://bloggershub.org/ 14.5kkelly martin SEO
guest post provider & Guest Posting Specialist. waiting orderEmail:kellymartin7877@gmail.com-
by "Kelly Martin" <kellymartin7877@gmail.com> - 02:10 - 6 Aug 2024 -
Hi i have sites related casino ,loan,adult,and also general check my some site site traffic https://rebeldemente.net/ 1k i am waiting for your positive response best regards
Hii have sites related casino ,loan,adult,and also generalcheck my some sitessites traffici am waiting for your positive responsebest regards
by "ritu yadav" <rituya564@gmail.com> - 02:06 - 6 Aug 2024 -
Kindly check it and let me know if you like any of these sites
Hi sirHi sir, how are you? I hope you're doing well. Actually, I have a number of sites related to you. Please check and let me knowsite; traffichttps://build.com.au/ 5.38kplease check my sites and let me knowi am waiting for your order and long time business as well
by "anika deepak" <anikadeepak16@gmail.com> - 02:05 - 6 Aug 2024 -
post your article like casino,gambling,laon,cbd ,and general
Hii hope your doing wellactually i am providing guest posti have sites related casino,adult,loan,cbd,and also generali hope my site inspire youplease check some sitesite traffichttps://flashystory.com/ 3.25kwaiting replybest regards from jiya 🥰
by "Jiya Lyn" <jiyalyn01@gmail.com> - 02:02 - 6 Aug 2024 -
post your all catogary articles or link insertion
I have some very good sites related to all categories for posting your article. my sites have pure organic traffic or good da
Please check site
site traffic
https://hustlersgrip.com/ 21.6k
Let me know if you need post on above sites
Add reaction
by "Alison Whyte" <alisonwhyte502@gmail.com> - 01:58 - 6 Aug 2024 -
New Sales Order - Ref[4] - (Maqabim Distributors & British Columbia Liquor Distribution Branch)
Hello Fabien,It would be great to talk to you on phone. Hope we can get to discuss some of our mutual customers and suppliers.Ship fromShip toMaqabim DistributorsBritish Columbia Liquor Distribution BranchI am looking forward talking to you and to give you a top-class unmatched Rates with a GUARUANTEE. Our rates will be better then your carriers that you work with regularly. Just add us to your list.
RegardsScott Adams
Director National Sales
HEAD OFFICE
Phone: +1 (289) 295-1640 ext 102
Address: 1315 North Service Road East Oakville ON L6H1A
WE GOT CANADA-USA-MEXICO COVEREDAll quotes are based on Dock to Dock, Business to Business and Stackable (Unless Otherwise Specified)
Legal Disclaimer: Charges are based on the information you have provided herein. Any variation to this information may affect the charges. Rates are valid for the customer named herein and constitute a confidential rate agreement between the customer and MGC-FREIGHT and may not be disclosed to any other party. Rates displayed herein will be in effect at time of shipping unless the rate agreement is amended or expired prior to shipment date.
by "Scott Adams -MGCFREIGHT" <scott@mgcfreight.com> - 12:04 - 6 Aug 2024 -
How Facebook Syncs Time Across Millions of Servers
How Facebook Syncs Time Across Millions of Servers
FusionAuth: Auth.͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ ͏ Forwarded this email? Subscribe here for moreFusionAuth: Auth. Built for devs, by devs. (Sponsored)
Hosting Flexibility: You host or we host - the choice is yours with no loss of features
Scale Confidently: Lightning-fast performance for 10 users or 10 million users (or more)
Developer-Centric: True API first design, quick integration, built on standards, highly flexible & customizable
Total Control: Deploy on any computer, anywhere in the world and integrate easily with any tech stack
Data Isolation: Single tenant by design means your data is physically isolated from everyone else's
Unlimited: Unlimited IDPs, unlimited users, unlimited tenants, unlimited applications, always free
FusionAuth is a complete auth & user platform that has 10M+ downloads and is trusted by industry leaders!
Disclaimer: The details in this post have been derived from the Facebook/Meta Engineering Blog. All credit for the technical details goes to the Facebook engineering team. The links to the original articles are present in the references section at the end of the post. We’ve attempted to analyze the details and provide our input about them. If you find any inaccuracies or omissions, please leave a comment, and we will do our best to fix them.
A clock showing the wrong time is worse than a faulty clock.
This is the challenge Facebook had to deal with while operating millions of servers connected to the Internet and each other.
All of these devices have onboard clocks that are expected to be accurate. However, many onboard clocks contain inaccurate internal oscillators, which cause seconds of inaccuracy per day and need to be periodically corrected.
Think of these internal oscillators as the “heartbeat” of the clock. Just like how an irregular heartbeat can affect a person’s health, an inaccurate oscillator can cause the clock to gain or lose time.
Incorrect time can lead to issues with varying degrees of impact. It could be missing a simple reminder or failing a spacecraft launch.
As Facebook’s infrastructure has grown, time precision has become extremely important. For example, knowing the accurate time difference between two random servers in a data center is critical to preserving the order of transactions across these servers.
In this post, we’ll learn how Facebook achieved time precision across its millions of servers with NTP and later with PTP.
Network Time Protocol
Facebook started with Network Time Protocol (NTP) to keep the devices in sync.
NTP is a way for computers to synchronize their clocks over a network. It helps ensure that all devices on the network have the same, accurate time.
Having synchronized clocks is critical for many tasks, such as:
Scheduling events and meetings
Logging and tracking activities
Ensuring a proper sequence of transactions
Coordinating actions between different systems
NTP uses a hierarchical system of time servers where the most accurate servers are at the top. There is a 3-step process to how NTP works:
Computers on the network periodically request the current time from these servers.
The servers respond with their current time taking into account the network delay.
The requesting computer adjusts its clock based on the information received from the server.
The diagram below shows the hierarchical system of servers used by NTP.
Facebook built an NTP service at scale. They used chrony, a modern NTP server implementation. While they used the ntpd service initially, testing revealed that chrony was far more accurate and scalable.
Chrony was a fairly new daemon at the time, but it offered a chance to bring the precision down to nanoseconds. Also, from a resource consumption perspective, chrony consumed less RAM compared to ntpd.
See the diagram below that shows the difference of around 1MB when it came to RAM consumption between chrony and ntpd.
Source: Facebook Engineering Blog They designed the NTP service in four layers based on the hierarchical structure of NTP.
Stratum 0 is a layer of satellites with precise atomic clocks from a global navigation satellite system (GNSS), such as GPS, GLONASS, or Galileo.
Stratum 1 is a Facebook atomic clock synchronizing with a GNSS.
Stratum 2 is a pool of NTP servers synchronizing to the Stratum 1 devices.
Lastly, Stratum 3 is a tier of servers configured for a larger scale.
The diagram below shows the layers of Facebook’s NTP service.
Source: Facebook Engineering Blog There are a couple of interesting concepts to note over here:
Leap second
Smearing
The Earth’s rotation is not consistent and can vary slightly over time. Therefore, clocks are kept in sync with the Earth’s rotation by occasionally adding or removing a second. This is called a leap second.
While adding or removing a leap second is hardly noticeable to humans, it can cause server issues. Servers expect time to move forward continuously, and a sudden change of a second can cause them to miss important tasks.
To mitigate the impact of leap seconds on servers, a technique called “smearing” is used.
Instead of adding or removing a full second at once, the time is gradually adjusted in small increments over several hours. It’s similar to masking a train’s delay by spreading the adjustment across multiple stations.
In the case of Facebook’s NTP service, the Leap-second smearing happens at Stratum 2. The Stratum 3 servers receive smeared time and are ignorant of leap seconds.
Latest articles
If you’re not a paid subscriber, here’s what you missed.
To receive all the full articles and support ByteByteGo, consider subscribing:
The Arrival of Precision Time Protocol
NTP adoption was quite successful for Facebook. It helped them improve accuracy from 10 milliseconds to 100 microseconds.
However, as Facebook wanted to scale to more advanced systems and build the metaverse, they wanted even greater levels of accuracy.
Therefore, in late 2022, Facebook moved from NTP to Precision Time Protocol (PTP).
There were some problems with NTP, which are as follows:
NTP and Asynchronous Systems: Systems using NTP are asynchronous, meaning they work independently without a shared global clock. These systems periodically check in with each other to ensure synchronization. However, as the system grows larger, more check-ins are required, which can slow down the network.
NTP and Timekeeping Methods: NTP is susceptible to variance and latency due to its timekeeping methods of using physical clocks. In other words, NTP is like a microwave clock that keeps time on-device. If there’s a time change (e.g., daylight savings), the clock needs to be manually adjusted.
In contrast, PTP works like a smartphone clock that updates its time automatically. When there’s a time change or the phone moves to a new time zone, the clock updates itself by referencing the time over a network.
While NTP provided millisecond-level synchronization, PTP networks could hope to achieve nanosecond-level precision.
What makes PTP More Effective?
As discussed earlier, a special computer called a Stratum acts as a time reference for other computers on a network. When a computer needs to know the current time, it sends a request to the Stratum, which replies with the current time. This process is known as sync messaging.
When the Stratum sends the current time to another computer, the information travels across the network, resulting in some latency. Several factors can increase this latency, such as:
The speed at which signals travel through the fiber optic cables.
The time it takes for the network devices to convert signals.
The quality of network equipment, like routers and switches.
The time it takes for software and drivers to process the time information.
Due to latency, the time received by the other computer is no longer accurate when it arrives at the receiving computer.
The obvious solution is to measure the latency and add it to the time received by the other computer to get a more accurate time. However, measuring latency is challenging because each computer has its clock, and there is no universal clock to compare against.
To measure latency, two assumptions about consistency and symmetry are made:
The latency a packet experiences while traveling across the network is consistent.
The latency from the Stratum to the other computer is equal to the latency from the other computer back to the Stratum. In other words, the network is symmetric.
Therefore, it follows that the accuracy of time synchronization can be improved by maximizing consistency and symmetry in the network.
PTP is a solution that helps achieve this.
PTP uses hardware timestamping to improve consistency. This means that timestamps are added to the time information at the hardware level, reducing the impact of software and driver delays.
PTP also uses transparent clocks, which are special devices that measure and compensate for the time the information spends passing through network equipment.
The Need for PTP
Let’s look at a practical case where PTP is needed.
Imagine you’re using Facebook and you post a new status update. When you try to view your post, there’s a chance that your request to see the post is handled by a different server than the one that originally processed your post.
If the server handling your view request doesn’t have the latest updated data, you might not see your post. This is annoying for users and goes against the promise that interacting with a distributed system like Facebook should work the same as interacting with a single server that has all the data.
In the old solution, Facebook used to send your view request to multiple servers and wait for a majority of them to agree on the data before showing it to you. But this takes up extra computing resources and adds delay because of the back-and-forth communication over the network.
By using PTP to keep precise time synchronization across all its servers, they can simply have the view request wait until the server has caught up to the timestamp of your original post. There is no need for multiple requests and responses.
The diagram below shows this scenario.
However, this only works if all the server clocks are closely synchronized. Also, the difference between a server’s clock and the reference time needs to be known.
PTP provides this tight synchronization. It could synchronize time about 100 times more precisely than NTP, which was necessary for Facebook’s requirement.
This was just one example. There were several additional use cases where PTP excelled such as:
Event tracing
Cache invalidations
Privacy violation detection improvements
Latency compensation in the metaverse
The PTP Architecture
Facebook’s PTP architecture consists of three main components:
The PTP Rack
The PTP Network
The PTP Client
See the diagram below for a high-level view of the architecture:
Source: Facebook Engineering Blog Let’s look at each component and understand how they work together to provide precise timekeeping.
The PTP Rack
The PTP rack houses the hardware and software that serve time to clients.
It consists of critical components such as:
GNSS Antenna: This is where time originates on Earth. The antenna receives time signals from GPS, Galileo, and other satellite constellations. Facebook uses a GNSS-over-fiber technology to distribute the signal, which is more reliable and easier to install than traditional coaxial cables.
Time Appliance: This is the heart of the timing infrastructure. It disciplines the time received from the GNSS antenna using atomic clocks for improved accuracy and stability. Facebook has developed a new Time appliance that can support up to 1 million clients without compromising accuracy.
Oscillatord: This is a software component that configures and monitors the Time appliance, including the GNSS receiver and atomic clocks. It exports data that helps decide if the Time Appliance should serve clients or be taken offline.
Network Card (NIC): It’s the interface between the Time Appliance and the network. It timestamps PTP packets using its clock, which is synchronized with the Time Appliance for nanosecond-level accuracy.
Ptp4u: This is Facebook’s custom-built PTP server software that can handle over 1 million clients per server, far more than existing solutions. It runs on the Time Appliance and sends PTP messages to the clients.
The GNSS Antenna in a Datacenter (Source: Facebook Engineering Blog) The PTP Network
The PTP network is responsible for distributing time from the PTP rack to clients. Facebook uses PTP over a standard IP network with a few key enhancements:
PTP Unicast: They use unicast PTP instead of multicast for simpler network configuration and better scalability. Clients request time from servers, and servers grant requests and send PTP messages.
Transparent Clocks: Each network switch between the client and server acts as a transparent clock. It measures the time each PTP packet spends in the switch and records it in the packet. This allows clients to accurately account for network delays.
Boundary Clock Avoidance: Facebook avoided using boundary clocks, which act as both clients and servers, to reduce complexity. They rely solely on transparent clocks in the network switches.
A typical PTP unicast flow consists of the following steps:
Client Initiates Negotiation: The PTP client starts the process by requesting unicast transmission from the PTP server. It sends three types of requests:
Sync Grant Request: The client asks the server to send a specified number of Sync and Follow-Up messages per second, containing the current time, for a certain duration. This helps the client adjust its clock to match the server’s clock.
Announce Grant Request: The client requests the server to send a specific number of Announce messages per second, containing the server’s status, for a certain duration. This helps the client make sure that the server hasn’t stopped or gone haywire.
Delay Response Grant Request: The client informs the server that it will send Delay Request messages, and asks the server to respond with Delay Response packets for a specified duration. This helps the client account for any delay in communication.
Server Grants Requests: The PTP server needs to grant these requests and send corresponding grant responses to the client.
Server Sends PTP Messages: Once the requests are granted, the server starts sending the requested PTP messages
Client Sends Delay Requests: The client sends Delay Request messages to the server at the agreed-upon interval to determine the network path delay.
Client Refreshes Grant: The client needs to periodically refresh the grant by repeating the negotiation process before the current grant expires.
The diagram below shows the PTP Exchange Process
The PTP Client
The PTP client software runs on each server that needs accurate time. Facebook uses a few different components:
ptp4I: An open-source PTP client that receives PTP messages from the server and disciplines the NIC hardware clock. Facebook has made several enhancements to ptp4I to handle its scale and unique requirements.
fbclock: This is Facebook’s custom API that provides PTP time to applications. Instead of a single timestamp, it gives a “window of uncertainty” - a time range that is guaranteed to contain the true time with a high degree of certainty.
Kernel Timestamping: The Linux kernel on each server timestamps incoming and outgoing PTP packets in hardware for maximum accuracy. This relies on NIC driver support and careful configuration.
Conclusion
To conclude, Facebook’s adoption of Precision Time Protocol (PTP) across its infrastructure is a significant step forward in ensuring precise and reliable timekeeping at an unprecedented scale. By redesigning and rebuilding various components, Facebook has pushed the boundaries of what’s possible with PTP.
Also, the open-source nature of most of the work helps us learn from the PTP solution implemented by them.
References:
Like Comment Restack © 2024 ByteByteGo
548 Market Street PMB 72296, San Francisco, CA 94104
Unsubscribe
by "ByteByteGo" <bytebytego@substack.com> - 11:36 - 6 Aug 2024 -
Boost Your Website's Google Ranking with Paid Guest Posts
I hope this message finds you well.
We offer a paid guest post service designed to enhance your website's Google ranking by featuring your content on high-quality websites. We would be happy to help you achieve your SEO goals.
Please share your specific requirements with us, and we will do our best to meet them. If you are interested in this opportunity, feel free to let us know.
Here are some websites where we can place your content:
Website DA Organic Traffic Categories bitcu. co 70 39.3k Bitcoin seoaves. com 55 7.8k Business nytstartup. com 53 3k Business instamber. com 28 4.6k Business/Technology oflox. com 61 2k Business bloggershub. org 26 3.3k News and Business cryptotimes. io 48 4k Crypto finsnip. com 61 1.5k Finance/Business We look forward to your response. Thank you for considering our services.
Best regards,
Dawihubert
by "Dawid Hubert" <dawihubert@gmail.com> - 06:34 - 6 Aug 2024 -
BusinessWebsites.net
Great news! The domai BusinessWebsites.net is now available on GoDaddy!The Internet is the most efficient way to acquire new customers.
Anytime someone types Business Websites, Business Websites Online, The Best Business Websites, or any other phrase with these keywords into their browser, your site could be the first they see!
Avg Google Search Results for this domain is: 4,180,000,000
You can easily redirect all the traffic this domain gets to your current site!
GoDaddy.com appraises this domain at $1,624
Priced at only $398 for a limited time! To claim it, go to BusinessWebsites.net and select Buy Now or purchase directly at GoDaddy.
Since we're contacting multiple interested parties about this domain, quick action is recommended if you're interested.
Thank you for your time.
Marc Soabas
by "Marc Francis Soabas" <yourdomainguru11.marc@gmail.com> - 05:15 - 6 Aug 2024