People Blog History Governance Learn

Connect

Donate

The PublicNTP Blog

Follow our journey to bring accurate time to the world.

Global NTP Server Monitoring

November 18, 2019

PublicNTP has consistently grown its global deployment footprint since the day the company formed. While that’s excellent news for us, it comes with a number of unforeseen issues as a matter of scale. With 30+ servers across the globe, it’s not enough to just make sure the server and operating system are both up and running.

The “Why” Question, the Network Time Community, and You!

October 28, 2019

While Network Time Protocol (NTP) has been in development for 34 years, time refinement has been a global cause for nearly a century.

PublicNTP’s Fleet Servers and Maintenance

October 14, 2019

In the second half of 2017, a few months after PublicNTP officially became a company, we had anywhere from five to ten servers providing time. We found that our servers needed occasional maintenance but were reliable enough that we could “safely” ignore them most of the time. Our service, supported by many other surrounding stratum one/two providers, could be relied on while working and covered when down.

PublicNTP JAM

September 30, 2019

Little while back (let’s face it, a long while back), I had another opportunity to visit the team out in Salt Lake. Scott and our developers are always pleasant company when working on PublicNTP’s objectives.

Groundbreaking: Africa and the Middle East!

September 16, 2019

We’ve mentioned a few times that our cloud servers hit a speedbump around Africa due to a lack of developed infrastructure. Project: Ikenga is one of our attempts to work around that with a physical deployment.

PublicNTP July Board Meeting

August 29, 2019

PublicNTP’s board members once again met up to talk about the company’s accomplishments since our meeting last November, and make sure we all agreed on the company’s vision for the future.

GPS Generation Three Has Launched!

July 29, 2019

November 26th, 2018 saw NASA land the Mars InSight rover—an achievement that required a thousand different, precise steps to all happen perfectly in order to succeed.

F2F in Mountain Time!

July 12, 2019

PublicNTP, like many companies, is geographically distributed; the company’s four board members live in four different states.

Touching Bases With The Team (African Expansion and More!)

June 17, 2019

PublicNTP has been working to improve our footprint in Africa since before the company was legally incorporated in the state of Virginia. The company has dubbed the search for improved company presence in Africa Project Ikenga, named after an African god of time.

PNTP Open Architecture Stratum One Part 5/5: Future Investigations

June 3, 2019

In this final entry in the series about our open architecture stratum one server, we call out some of the future investigations we noted that may be worth looking into during the exploration so far.

PublicNTP Open Architecture Stratum One Part 4: Step By Step Blueprint

May 20, 2019

This next ‘article’ is for those tech-heads and time-nerds who are interested in setting up their own time server.

PNTP Open Architecture Stratum One Part 3: Technical Overview

April 22, 2019

In this third entry in the series, we get into the technical decisions we made, and the decision points we encountered along the way to the current iteration.

PublicNTP Open Architecture Stratum One Server Part 2: Rationale

April 1, 2019

As posted in PublicNTP’s previous blog article, our team has spent many hours and meetings over the past months identifying the advantages and disadvantages of our own open architecture. We’ve looked hard at utilizing tried and true NTP servers in tandem with our own established network of stratum two and, more recently, stratum one servers.

Open Stratum One - The Meinberg Solution

March 18, 2019

Since PublicNTP started in the summer of 2017, all but one of the servers we have deployed have been stratum two servers according to the Network Time Protocol.

PublicNTP’s Stratum One Deployment Checklist

March 4, 2019

Much in the same way we dove into virtual servers, we figured it would be fairly easy to find an underserved country and find a company who would lease us some power, space, and network bandwidth.

What Goes Into Deployment?

February 18, 2019

As of this writing, all of PublicNTP’s deployed time servers are virtual (i.e., running in the cloud) -- except for Salt Lake City, because Scott is a fan of turning things up to 11. This approach has served us well so far. It’s becoming clear that we’re reaching the point of diminishing returns for cloud deployments. In the underserved countries we’re now trying to target, cloud just won’t be an option.

Project Ikenga - Connecting Africa

February 4, 2019

When PublicNTP originally targeted Africa for a deployment campaign we knew that there were going to be a few challenges.

2018 PublicNTP Board Meeting - Upcoming Year

January 21, 2019

Board members look forward to what 2019 holds for PublicNTP

Year in review

January 7, 2019

PublicNTP has its annual board meeting in November, where the directors review the changes, achievements, and progress made in the previous twelve months. As always, it was a delight to get all four board members face to face to discuss what has happened.

The Time Server App Has Launched!

December 12, 2018

The Time Server app is a reflection of both PublicNTP’s vision for the future and a sizeable portion of our concerted efforts in the last two years. With Time Server, you’ll be able to have and redistribute some of the most accurate time on the planet.

A Tale of Two Meetings

September 12, 2018

A big part of keeping any company pointed in the right direction is making sure the leadership team is communicating regularly.

PublicNTP partners with Network Time Foundation

May 14, 2018

Network Time Protocol (NTP) remains one of the oldest and most fundamental internet protocols in use. It has provided a bedrock of reliable time for us since 1985. NTP started out rough, as all innovations do

Donating Time

February 24, 2018

PublicNTP receives federal public charity status

PublicNTP's Paperwork Puzzle

August 21, 2017

Part 4: Our 15 step checklist

PublicNTP’s Financial Structure

August 21, 2017

Part 3: When considering the public charity path, make sure you know how you’re reconciling your finances and how often

PublicNTP’s Organizational Structure

August 21, 2017

Part 2: We needed to establish some structure to the corporation

PublicNTP’s Dizzying Terminology

August 21, 2017

Part 1: The first steps to becoming tax-exempt

PublicNTP’s Journey to Tax-Exempt Status

August 21, 2017

The first steps to becoming tax-exempt

Global expansion and a facelift

July 8, 2017

When I first launched PublicNTP with a single NTP server in Chicago, Illinois, I had grand plans for rapid expansion, first across the US, to be quickly followed by exotic locations around the globe I’d always wanted to visit.

[{"permalink":"global-ntp-server-monitoring","timestamp":"2019-11-18T14:11:00-05:00","title":"Global NTP Server Monitoring","excerpt":"PublicNTP has consistently grown its global deployment footprint since the day the company formed. While that’s excellent news for us, it comes with a number of unforeseen issues as a matter of scale. With 30+ servers across the globe, it’s not enough to just make sure the server and operating system are both up and running.","image_urls":[{"url":"images/black-and-white-black-and-white-monochrome-707676-min.jpg","alt":"Black and White wall of clocks"},{"url":"images/black-and-white-chart-cost-241544-min.jpg","alt":"Black and white monitor with charts"},{"url":"images/black-cameras-close-up-1135453-min.jpg","alt":"Close up of black cameras"},{"url":"images/metronome-clock-music-music-production-162550.jpeg-min.jpg","alt":"Metronome"}],"content":"<h4><a href='/people.html#scott-waddell'>Scott Waddell</a></h4>PublicNTP has consistently grown its global deployment footprint since the day the company formed. While that’s excellent news for us, it comes with a number of unforeseen issues as a matter of scale. With 30+ servers across the globe, it’s not enough to just make sure the server and operating system are both up and running.We need to get a feeling for the quality and availability of the time data. It’s important to know how close each of our server estimates are to the UTC standard, and how quickly the servers are responding to requests for time (and if they ever fail to respond!).This is a natural point in the article to highlight the amazing work done by the NTP Pool.The <a href='https://www.ntppool.org/en/'>NTP Pool project</a> is an effort launched by <a href='https://groups.google.com/forum/#!topic/comp.protocols.time.ntp/cShrN7imCJ0'>Adrian von Bidder in 2003</a> that has consistently been the largest, most reliable cluster of network time servers in the world. The NTP Pool delivers time to hundreds of millions of digital clocks around the world.It’s important to note that the Pool is 100% free. Period. Nobody pays to use it, and <a href='https://www.askask.com/'>Ask Bjørn Hansen</a>, the Pool’s maintainer since 2005, pays every dollar of all hosting costs out of his own pocket.The NTP Pool includes a monitoring system to check on any submitted server. If that system indicates that a submitted server is delivering high quality time reliably, it will direct queries to the submitted server. If a participating server drifts wildly from UTC standard or is not responding to queries reliably, the Pool will remove the submitted server from service until it becomes “safe” to use again.PublicNTP proudly submits all servers we deploy for inclusion in the NTP Pool—adding the NTP Pool’s reliability data as another source of monitoring for our deployed servers.PublicNTP quickly noticed that the Pool’s monitoring data often differed from what we collected through our own monitoring systems. We found the NTP Pool monitoring system often marked our servers as far less reliable than our own data indicated.While the research effort is still very much ongoing, PublicNTP has started to be persuaded that NTP monitoring is often a function of where. By that, we mean where a server is monitored from matters—the more “wheres” you use, the better!At the time we started looking harder at the NTP Pool’s data in 2017, the NTP Pool monitoring system consisted of a single server in Los Angeles, California.It appeared that the further that a server was from Los Angeles (more a metric in terms of milliseconds of latency than physical miles—though the two are related), the more likely the Pool rated the NTP server as “low reliability”.The PublicNTP team started digging into some of the more interesting cases. One such example was the PublicNTP server in São Paulo, Brazil. The Brazil server looked terribly unreliable from the NTP Pool monitor in Los Angeles, ~10,000km/6,000 mi away.Our monitoring servers showed huge variation:<ul style='list-style-type: disc;'><li>New York, ~7,500km/4,800 mi away, showed the São Paulo server to be just as unreliable as the Pool did.</li><li>Frankfurt, Germany, 9,700 km/6,000 mi away, showed the Brazil server to be impressively accurate and reliable, though it did occasionally show a dropped response.</li><li>Singapore, 16,000 km/10,000 mi away, showed the Brazil server to be both incredibly accurate and incredibly reliable, with well over 99% of requests getting responses.</li></ul>PublicNTP is coming to the opinion that “positive” responses (i.e. the server is both accurately tracking to the UTC standard and reliably responding to NTP queries) can be given a fair bit of weight/relied upon. The issue is the same cannot be said for “negative” responses.There are a LOT of factors outside the control of an NTP server operator. One of the biggest is the networks along the path from a computer making an NTP request to the target NTP server. If NTP queries fail to even make it to the target server, it’s not accurate to “score” the NTP server as unreliable as it never got the chance to reply! And vice versa if a reply successfully sent by the NTP server fails to make it back to the NTP client.As such, the PublicNTP’s monitoring system algorithm has started taking a healthy bias approach, to try and accurately represent the health of a monitored NTP server:Minority Positive: if 20-30% of our monitoring stations around the world are getting indications of good accuracy/reliability from a server, it’s safe to assume the server itself is functioning fine and thus give the server a score of “healthy.”Majority Negative: if 80% or more of the global monitoring locations report poor accuracy or a large percentage of requests being dropped, it’s safe to assume a server is performing poorly.PublicNTP’s effort to improve our ability to ensure high-quality time data from our global fleet of servers is always ongoing, but we wanted to give readers a snapshot in time of where we are as of today. Tomorrow—who knows??? :)","tags":["NTP","sysadmin","monitoring","distributed systems","clock synchronization"],"time":"Nov 18, 2019"},{"permalink":"network-time-community-and-you","timestamp":"2019-10-28T14:11:00-05:00","title":"The “Why” Question, the Network Time Community, and You!","excerpt":"While Network Time Protocol (NTP) has been in development for 34 years, time refinement has been a global cause for nearly a century.","image_urls":[{"url":"images/fear-min.jpg","alt":"Fear Scrabble blocks"}],"content":"<h4><a href='/people.html#terry-ott'>Terry Ott</a></h4>While <a href='https://en.wikipedia.org/wiki/Network_Time_Protocol'>Network Time Protocol</a> (NTP) has been in development for 34 years, time refinement has been a global cause for nearly a century.Since NTP and <a href='https://en.wikipedia.org/wiki/Satellite_navigation'>Global Navigation Satellite Systems</a> (GNSS) matured and computer connections have enabled global public access to these systems, it has become clearer by the decade how important it is to keep track of time.PublicNTP sees global clock synchronization as an important effort to contribute to. The challenge of meeting that goal comes in many flavors -- infrastructure, server upkeep, and public access are among the most concerning. One particular challenge that many -- definitely including PublicNTP! -- seem to struggle with is responding to one simple, fundamental question:Why?Anyone involved in the network time community, from those who develop software to help synchronize digital clocks or those who run the servers distributing highly-accurate time dreads this question: “Okay, I understand what you’re doing -- but why are you doing it?”It’s a topic with multiple layers, but at some level, the person asking the question is asking you to sell them. They want to hear your pitch to find out if it’s a bigger deal than they currently understand it to be, and if so, it may motivate them to get involved in some manner.In my personal experience so far, it seems like many (most?) of the responses immediately run for “doom-and-gloom” scenarios:“Without carefully-synchronized clocks, you can’t secure your network!”“Without carefully-synchronized clocks, bank transactions will fail!”“Without carefully-synchronized clocks, the power grids will fail!”“Without carefully-synchronized clocks, THE OCEANS WILL BOIL.”Okay, well, maybe not that last one -- at least not due to lack of synchronized digital clocks. :)Jokes aside, a lot of the justifications I’ve encountered when trying to clarify the PublicNTP stance on why we put so much time, effort, and money into synchronizing digital clocks is intensely negative. I’m reluctant to use the word “fearmongering,” but a lot of the public justifications used by the network time community aren’t terribly far removed from that.There are times PublicNTP, and myself in particular, have been flirting with line between persuasive logic and fear-mongering -- I’ll be the first to say that PublicNTP is still struggling with how to positively frame the discussion about why our mission is important. That said, history has shown that trying to sway opinions to your side by telling your audience that they’d better start seeing the world the way you do or else is not a terribly effective way to change hearts and minds. Beyond that, I’d like to see PublicNTP be one of the leaders of the charge, helping show there are more effective ways to generate interest (and donations!) in our cause that don’t rely on fear-based justifications for the importance of synchronized time.The PublicNTP board has thought hard for a good while on how we’d like to talk about this topic. Our company’s position is still in its infancy, but one of the more promising threads we’re pulling on is passion.The four founders of PublicNTP didn’t form PublicNTP because we were convinced that the internet and/or civilization would collapse if we didn’t start the company. It started out as a passion driven by the fact that there’s a “right answer” (meaning an international time standard, UTC) and due to our professional training, we have the ability to help computers around the world move towards that right answer. The four of us believe accurately adhering to an international standard is good on its own merits. We love the concept of timekeeping, we love watching seconds tick by, and we innately love the idea that we can help computers around the world start ticking in close beat to each other.The four of us feel that there may be some parallels to network time community as a whole. It’s about passion first. Nothing brings a smile to my face quite like telling people that one of my most time-consuming hobbies is “distributing atomic time on a global scale.” Most people give me confused looks at first, but once I explain it, I find that my enthusiasm about the subject can be contagious, and people who were unaware of the issues, or were aware of but ambivalent to the issues, suddenly respond to my raw enthusiasm and bone-deep conviction that this all matters with enthusiasm of their own.It may sound corny but, at a pretty fundamental level, humans are explorers. Few things stifle the urge to explore as well as fear. Unfortunately, fear has been the time community’s go-to sales pitch for too long. Accurate measurement of time is both valuable and fascinating. It’s integral to our daily lives yet it’s completely arbitrary, all the units and how to measure them made up entirely by humans.Humans have agreed on a way to handle reckoning of time. PublicNTP will happily deliver our best estimate of the international time standard to absolutely any computer who wants it as a public good -- meaning for no cost and with no access restrictions. That’s the PublicNTP mission statement in a nutshell.PublicNTP can’t do it by ourselves, though. We need your help. Not only your help, your passion.We don’t want you to offer your time, expertise, or money as a direct result of PublicNTP telling you the world’s oceans could catch on fire any moment now if we don’t stay on top of things. We want your help because we love working on teams with passionate believers that love tinkering with time.Join us, won’t you?","tags":["Philosophy","Network Time Protocol","Fearmongering","Community Engagement"],"time":"Oct 28, 2019"},{"permalink":"publicntps-fleet-servers-and-maintenance","timestamp":"2019-10-14T14:11:00-05:00","title":"PublicNTP’s Fleet Servers and Maintenance","excerpt":"In the second half of 2017, a few months after PublicNTP officially became a company, we had anywhere from five to ten servers providing time. We found that our servers needed occasional maintenance but were reliable enough that we could “safely” ignore them most of the time. Our service, supported by many other surrounding stratum one/two providers, could be relied on while working and covered when down.","image_urls":[{"url":"images/bay-clouds-harbour-min.jpg","alt":"Boats in the bay at sunset."},{"url":"images/aerial-view-bay-bird-s-eye-view-min.jpg","alt":"Arial view of cargo ships in a bay."},{"url":"images/bay-beach-bird-s-eye-view-min.jpg","alt":"Arial view of boats in a line in a bay."}],"content":"<h4><a href='/people.html#terry-ott'>Terry Ott</a></h4>In the second half of 2017, a few months after PublicNTP officially became a company, we had anywhere from five to ten servers providing time. We found that our servers needed occasional maintenance but were reliable enough that we could “safely” ignore them most of the time. Our service, supported by many other surrounding stratum one/two providers, could be relied on while working and covered when down.Our “maintenance process” looked something like this: “Hey, I guess it’s been a few weeks and I need something to do during my lunch break. I should log into all the servers to make sure they’re up. While I’m there, I’ll apply the patches that have queued up.”We were moving quickly enough that we spent more time planning future deployments than providing proper care and feeding to our small fleet of existing servers.A Kurt Vonnegut quote unfortunately nails PublicNTP’s early philosophy towards server ops:“Another flaw in the human character is that everybody wants to build and nobody wants to do maintenance.”There was far more emotional payoff launching cloud servers in remote parts of the world and building relationships with other non-profits in our space. We knew maintenance was necessary, but the boring “ops” tasks always got put on the back burner -- until they suddenly became urgent (like when a friend reported a server of ours was completely offline).Skip forward to the present, where PublicNTP has 30+ servers across the globe. That laissez-faire approach to server ops doesn’t stand up well. It no longer was “a couple minutes to patch the fleet” manually -- it was quickly starting to take the better part of an hour.As we grew, like we hoped it would, the “when I am bored and need a distraction” approach to fleet ops wasn’t holding up.Luckily there are plenty of software options out there for making sure our fleet of servers keep ticking. We knew we needed to prioritize immediate alerts with rapid response times for server fixes.Since all our servers are Ubuntu-based, <a href='https://landscape.canonical.com/'>Landscape from Canonical</a> seemed our best bet. With ways of easily deploying our base cloud servers, monitors that track server resources utilization, alerts if something’s down, and the ability to apply all patches nightly is proving to be a fantastic solution for us.With Landscape helping us manage our servers, not only are we able to provide our services more reliably, but we can use this data to do better capacity planning as the fleet continues to rapidly grow.","tags":["devops","server","monitoring","patching","Ubuntu"],"time":"Oct 14, 2019"},{"permalink":"publicntp-jam","timestamp":"2019-09-30T14:11:00-05:00","title":"PublicNTP JAM","excerpt":"Little while back (let’s face it, a long while back), I had another opportunity to visit the team out in Salt Lake. Scott and our developers are always pleasant company when working on PublicNTP’s objectives.","image_urls":[{"url":"images/pntp-jam-min.jpg","alt":"Scott, Tod, Cody, and Eric at the PNTP JAM."}],"content":"<h4><a href='/people.html#terry-ott'>Terry Ott</a></h4>Little while back (let’s face it, a long while back), I had another opportunity to visit the team out in Salt Lake. Scott and our developers are always pleasant company when working on PublicNTP’s objectives.And WOW, did we get some work done.First off, Scott and I slated out an entire afternoon to tackle a problem we’ve wanted to fix for a while. Maintaining our servers in the beginning proved to be an easy ‘as-needed’ job. Until we started getting into the dozens of deployments realized that our lives in meatspace were distraction enough to miss the fact that several of our servers could stop responding without our knowing.In the span of an afternoon, we implemented Ubuntu Landscape on every stratum 2 servers in the fleet. Landscape is a wonderful tool that allows us to check and patch our entire fleet at once, as well as optionally have it automatically apply security patches on a periodic basis (we went with nightly).Our second objective was our baby: the Time Server App. We <a href='/blog/posts/bring-time-to-the-world.html'>announced it months ago</a> but this is when it all kicked off. I’ve loved time servers longer than I can remember but the thought of having an app the public could download always seemed like a fever dream before I met Cody, Eric, Tod, and Scott.Months of work later, we rallied our resources and posted it to the Play store. The app was a fun challenge but we hadn’t drafted a solid Play Store pitch.Eric and Cody both stepped up to the puzzle. Eric figured out all the hosting and submission details (for his first time doing so, he nailed it). And for the intro, Cody put together this:<iframe width='560' height='315' src='https://www.youtube.com/embed/ysnGSrYVw9M?rel=0' frameborder='0' allow='accelerometer; autoplay; encrypted-media; gyroscope; picture-in-picture' allowfullscreen></iframe>Now if that music style sounds familiar, I imagine you’ve listened to Moby! That’s right, our little non-profit reached out and got permission from the artist to use his music to promote our app.And we couldn’t have posted it at all without Tod’s rigorous QA on the app prior to launch.Looking back, it’s surprising how our plans shift but our objectives remain the same. The app was by no means perfect at launch it’s being refined with regular patches. Our servers are operating more efficiently than ever and we can react to an issue with the software immediately.Time is what we have and our time in Salt Lake was well spent.","tags":["Ubuntu","Landscape","Monitoring","Security","Time Server App","App"],"time":"Sep 30, 2019"},{"permalink":"groundbreaking-africa-and-the-middle-east","timestamp":"2019-09-16T14:11:00-05:00","title":"Groundbreaking: Africa and the Middle East!","excerpt":"We’ve mentioned a few times that our cloud servers hit a speedbump around Africa due to a lack of developed infrastructure. Project: Ikenga is one of our attempts to work around that with a physical deployment.","image_urls":[{"url":"images/abu-dhabi-architecture-buildings-1589237-min.jpg","alt":"Abu Dhabi Buildings."},{"url":"images/abu-dhabi-bridge-buildings-1660603-min.jpg","alt":"Abu Dhabi Bridge Buildings."},{"url":"images/adventure-africa-animals-861329-min.jpg","alt":"African Animals."},{"url":"images/africa-animal-animals-417142-min.jpg","alt":"African Animals 2."}],"content":"<h4><a href='/people.html#scott-waddell'>Scott Waddell</a></h4>We’ve mentioned a few times that our cloud servers hit a speedbump around Africa due to a lack of developed infrastructure. <a href='/blog/posts/Project-Ikenga.html'>Project: Ikenga</a> is one of our attempts to work around that with a physical deployment.Microsoft, Amazon, and Oracle have all recently announced plans of cloud deployments and data centers across South Africa and the Middle East. PublicNTP is excited to see them succeed as it will provide a foothold for our own efforts in the future.South African DeploymentsBack in 2017, Microsoft Azure <a href='https://blogs.microsoft.com/blog/2017/05/18/microsoft-deliver-microsoft-cloud-datacenters-africa/'>announced two datacenters</a> for cloud hosting in Cape Town and Johannesburg to be open for business in 2018. Despite missing their date, it’s a credit to Azure that this is the first time in their history that they’ve missed a go-live date. Both regions went live in <a href='https://azure.microsoft.com/en-us/updates/azure-south-africa-regions-are-now-available/'>early March 2019.</a>Hot on Microsoft’s heels, Amazon has plans to establish a cloud region consisting of <a href='https://aws.amazon.com/blogs/aws/in-the-works-aws-region-in-south-africa/'>three Availability Zones</a> in 2020.PublicNTP deployed to Azure in Johannesburg in May of 2019 and has been very pleased with our first NTP server deployment on Azure.Middle East DeploymentsBoth Microsoft Azure and AWS have announced new cloud regions in the Middle East -- United Arab Emirates and Bahrain respectively. Azure was first to open in <a href='https://azure.microsoft.com/en-us/blog/first-microsoft-cloud-regions-in-middle-east-now-available/'>June of 2019</a>. AWS Bahrain became a <a href='https://aws.amazon.com/about-aws/whats-new/2019/07/announcing-the-new-aws-middle-east--bahrain--region-/'>month later</a>.Sometimes digital development is a game of inches and other times, it leaps miles. PublicNTP is delighted watching the cloud competition heat up in some of the most underserved markets in the world!","tags":["NTP","Africa","Middle East","Cloud","Infrastructure","UAE","Bahrain","South Africa"],"time":"Sep 16, 2019"},{"permalink":"publicntp-july-board-meeting","timestamp":"2019-08-29T14:11:00-05:00","title":"PublicNTP July Board Meeting","excerpt":"PublicNTP’s board members once again met up to talk about the company’s accomplishments since our meeting last November, and make sure we all agreed on the company’s vision for the future.","image_urls":[{"url":"images/PNTP_Board_Meeting_20190713_-_Dinner.jpg","alt":"Dinner Meeting."}],"content":"<h4><a href='/people.html#dan-noland'>Dan Noland</a></h4>PublicNTP’s board members once again met up to talk about the company’s accomplishments since our meeting last November, and make sure we all agreed on the company’s vision for the future.For this meeting, we bravely took on the challenge of attempting a meeting in Georgetown (those who know the area will get that joke).Happy to report that we’ve seen great strides in our hardware, software, and the community since our last meetup!One of our major talking points was the Time Server App. Since its launch, we’ve been iterating on it, fixing bugs, and refining the quality of life aspects to better serve those who download it. The server aspect has been a riot of a puzzle considering how each model of a phone has separate protocols when it comes to serving up data — even something as simple as time. We look forward to continue cracking that code and getting the community in on the action with our open-source access.<img style='max-width:100%;' src='/images/PNTP_Board_Meeting_20190713.jpg' />Yes, Scott and Dan came to the meeting wearing the exact same shirt. No, they didn’t plan that. Or so they claim.Something Brad has been working on in the last several months is our growing concern with the costs of our physical deployments. He’s been aggressively looking at improving the cost efficiency of our Stratum 1 solution, mostly cheaper alternatives to our first iteration of the hardware. We discussed our Meinberg card and server setup, which is an excellent rig but if we’re wanting to deploy dozens of these, our server cost is going to jump to the hundreds of thousands of dollars.For a fledgling non-profit, we aren’t in a position where that’s realistic for us, so that’s why we’re looking into smaller-scale hardware for larger-scale deployment. Our various options include other open source cards. One interesting vein of research is using a raspberry pi as a host for a time serving framework. It would have to be a homebrew system — which is why we’re looking to our community for help!Terry updated the team on our stratum one deployment slated for this year in Miami. We have received all the equipment we ordered for the deployment (a new server, and a bunch of spare coaxial cable), as well as identified the hosting provider we plan to use. At this point, we’re just working final logistics with the hosting provider.After the meeting the team met up at <a href='http://www.filomena.com/'>Filomena</a>, a Georgetown institution, for a fantastic Italian dinner and a wonderful chance to catch up and get some good laughs.","tags":["Board meeting","Time Server App","nonprofits","fundrasing","stratum one","NTP"],"time":"Aug 29, 2019"},{"permalink":"gps-generation-three-has-launched","timestamp":"2019-07-29T14:11:00-05:00","title":"GPS Generation Three Has Launched!","excerpt":"November 26th, 2018 saw NASA land the Mars InSight rover—an achievement that required a thousand different, precise steps to all happen perfectly in order to succeed.","image_urls":[{"url":"images/gps-generation-three-1.jpg","alt":"Dish at dawn."},{"url":"images/gps-generation-three-2.jpg","alt":"Satellite with view of Earth."},{"url":"images/gps-generation-three-3.jpg","alt":"Rocket launch."},{"url":"images/gps-generation-three-4.jpg","alt":"Rocket pre-launch."}],"content":"<h4><a href='/people.html#dan-noland'>Dan Noland</a></h4>November 26th, 2018 saw NASA land the Mars InSight rover—an achievement that required a thousand different, precise steps to all happen perfectly in order to succeed.A month later, on December 23rd, SpaceX successfully launched the Air Force’s first third generation GPS satellite, also known as “GPS IIIA-01” on top of a Falcon 9 rocket. While unquestionably a remarkable feat of engineering, this mission succeeded against a different set of challenging hurdles—twenty years of bureaucracy.While the first GPS satellite was launched in 1978, the current Global Position System (GPS) did not reach fully operational status until 1995. While the first two GPS satellite generations served their purpose superbly for the time (compared to the previous services available), the always-marching advance of technology had progressed far beyond their estimations even then.And 1995 hasn’t even gotten us to when mass-market cell phones or high-speed internet became widely-available!In 1998, the Clinton administration recognized that a new generation of GPS satellites would be needed to keep up with the new 90’s advances. The GPS III initiative has been enthusiastically supported by every administration since its approval.Multi-billion dollar contracts were written to get the system built—from Lockheed Martin’s major contributions in building the satellites, to Raytheon building the ground control system.And some contracts changed hands. United Launch Alliance (ULA) received the launch contracts in 2012 for a launch in 2014. But due to delays in build deliveries and hiccups with late tech, the launch was postponed until 2018. In that four year span, the Air Force had sold the remaining launch contracts to SpaceX, and a later rocket-swap in 2017 handed ULA’s initial launch contract to SpaceX as well.This switcheroo, by all optimistic estimates, would not delay the launch further. After a spring scare with some technical issues, the Falcon 9 heavy rocket launched the first of ten GPS IIIA satellites—with twenty-two IIIF satellites being commissioned last September (“F” to indicate a “follow-on” series).We’re hopeful that GPS III is finally hitting its “launch stride”. The second GPS III space vehicle, GPS IIIA-02, is <a href='https://spaceflightnow.com/launch-schedule/'>scheduled to launch in August, followed by GPS IIIA-03 in December</a>.PublicNTP is extremely excited to see GPS III finally get off the ground—pun absolutely intended. The number of instruments each A2100 bus carries will be a remarkable improvement over our current system designed between 1960 and 1980.GPS III introduces <a href='https://www.gps.gov/systems/gps/modernization/civilsignals/'>three new civil signals</a> as the IIIA and IIIF blocks are launched over the next couple decades:<ul><li>L2C - A frequency designed for clear commercial use</li><li>L5 - A Safety of Life frequency designed to enhance the safety of aircrafts</li><li>L1C - A secure frequency designed to facilitate interoperability between international satellites<li></ul>Each of these advancements reinforce the technology infrastructure that the entire world has been utilizing for over 20 years. For PublicNTP, it results in additional resources to help ensure that a world with fully-synchronized time can be a reality.","tags":["GPS","NTP","Satellite","PNT"],"time":"Jul 29, 2019"},{"permalink":"f2f-in-mountain-time","timestamp":"2019-07-12T14:11:00-05:00","title":"F2F in Mountain Time!","excerpt":"PublicNTP, like many companies, is geographically distributed; the company’s four board members live in four different states.","image_urls":[{"url":"images/2019_team_meeting.jpg","alt":"2019 Team Meeting. Wylie, Scott, Terry, Eric."},{"url":"images/snowbird01.jpg","alt":"Scott and Terry at Snowbird."},{"url":"images/butterfieldcanyon.jpg","alt":"Scott and Terry at Butterfield Canyon."}],"content":"<h4><a href='/people.html#scott-waddell'>Scott Waddell</a></h4>PublicNTP, like many companies, is geographically distributed; the company’s four board members live in four different states.Terry lives out around D.C. but he and I met in Salt Lake after his family moved here from Maine in 1994. Kind of startling to realize he and I have known each other for a quarter of a century!In June, Terry was in Salt Lake City, Utah visiting family. Whenever Terry’s in town, he and I make a point to meet up and get some quality face-to-face time -- it’s amazing how much work can get knocked out in just a couple hours when you’re sitting at the same table.<a href='https://pxp200.com/team/'>Wylie Thomas</a>, the founder of <a href='https://pxp200.com/'>PXP</a>, a development contracting shop located just north of Salt Lake that PublicNTP has <a href='https://pxp200.com/portfolio/'>worked with extensively</a>, was kind enough to offer up the use of his office’s conference space for the meeting.<a href='/people.html#terry-ott'>Terry</a>, Wylie, <a href='https://www.linkedin.com/in/cody-deskins/'>Cody Deskins</a> (who had to bail just before the picture was taken), <a href='https://www.linkedin.com/in/eric-evans-8b2b9647/'>Eric Evans</a>, and I talked about the efforts we have going on the <a href='/'>PublicNTP website</a>, the <a href='https://timeserver.app/'>Time Server App</a>, and then did some brainstorming on some future projects that are in the pipeline.Terry suggested that while he was in town, we take advantage of the mountains surrounding the Salt Lake Valley to put on some altitude, as he doesn’t get much of that in Virginia. One afternoon we headed up <a href='https://goo.gl/maps/K7d7C7qNJDTGSPzD9'>Little Cottonwood Canyon</a> and took the tram to the top of <a href='https://www.snowbird.com/'>Snowbird Ski Resort</a> to check out the Salt Lake valley from the east, perched up at 11,000 feet/3,353 meters above sea level. A few days later we did a short hike from the top of <a href='https://goo.gl/maps/2v3kfDW4C4EoBwrg6'>Butterfield Canyon</a> to examine the Salt Lake Valley from a west side vantage point at 7,915 feet/2,400 meters elevation. Check out more pics on Flickr from <a href='https://flic.kr/s/aHsmEPF1UD'>Snowbird</a> and <a href='https://flic.kr/s/aHsmET2gfi'>Butterfield Canyon</a>.One of the topics Terry and I talked about in depth was how PublicNTP can better engage with the broader time community.Not unlike the physical distances between the PublicNTP board members, Terry and I agree that, as with any online community, the network time community has communication challenges. Creating opportunities to foster communication amongst all the people interested in network time is an important and ongoing challenge the group as a whole needs to constantly be working on.If you are passionate about time as we are, please consider <a href='/donate.html'>donating</a> to PublicNTP!","tags":["GPS","NTP","Satellite","PNT"],"time":"Jul 12, 2019"},{"permalink":"touching-bases-with-the-team-african-expansion-and-more","timestamp":"2019-06-17T14:11:00-05:00","title":"Touching Bases With The Team (African Expansion and More!)","excerpt":"PublicNTP has been working to improve our footprint in Africa since before the company was legally incorporated in the state of Virginia. The company has dubbed the search for improved company presence in Africa Project Ikenga, named after an African god of time.","image_urls":[{"url":"images/bay-boats-cape-town.jpg","alt":"Boats in Cape Town"},{"url":"images/altitude-blue-sky-city.jpg","alt":"Blue Sky"},{"url":"images/africa-cliff-daylight.jpg","alt":"African Cliff"},{"url":"images/africa-animal-photography-barbaric.jpg","alt":"African Elephants"}],"content":"<h4><a href='/people.html#terry-ott'>Terry D. Ott</a></h4>African Expansion: Project IkengaPublicNTP has been working to improve our footprint in Africa since before the company was legally incorporated in the state of Virginia. The company has dubbed the search for improved company presence in <a href='/blog/posts/Project-Ikenga.html'>Africa Project Ikenga</a>, named after an African god of time.The team has found Africa a challenging place to find reliable computing infrastructure at a price point that a nonprofit can sustain long term -- and we’re not alone. Problems like having all our african servers hosted with a single company go offline the same night and finding concrete proof several months later that the provider went out of business with absolutely no warning is sadly common in a lot of developing parts of the world.In early March, I was very glad to see <a href='https://azure.microsoft.com/en-us/blog/microsoft-opens-first-datacenters-in-africa-with-general-availability-of-microsoft-azure/'>Microsoft announce its Azure cloud datacenter in Johannesburg, South Africa had gone live</a>. Of the hyperscale providers -- <a href='https://aws.amazon.com/about-aws/global-infrastructure/'>Amazon</a>, <a href='https://azure.microsoft.com/en-us/global-infrastructure/regions/'>Microsoft</a>, and <a href='https://cloud.google.com/about/locations/'>Google</a> (who altogether combined hold <a href='https://www.parkmycloud.com/blog/aws-vs-azure-vs-google-cloud-market-share/'>75% of the total global cloud market share</a>), Microsoft is the first hyperscale cloud provider to open a region in the entire African continent.AWS has announced it will open a region in <a href='https://aws.amazon.com/blogs/aws/in-the-works-aws-region-in-south-africa/'>Cape Town, South Africa in early 2020</a>. But by the time AWS goes live, Microsoft will have had its African cloud region live for a full year. And in our line of work, time is literally of the essence.The <a href='https://azure.microsoft.com/en-us/global-infrastructure/regions/'>Azure</a> server in Johannesburg is still in early testing phases for the PublicNTP fleet, but early data is looking very promising as to its performance and accuracy. Expect more news soon!New NTP server software: NTPsecPublicNTP has run the <a href='https://www.nwtime.org/projects/ntp/'>NTP Reference Implementation</a> software, maintained by <a href='https://www.nwtime.org/'>Network Time Foundation</a>, on all its servers to date.While many UNIX users are used to seeing the ubiquitous and venerable ntpd in a process list, it is important to note that there are <a href='https://chrony.tuxfamily.org/comparison.html'>multiple mature software packages that will provide NTP service</a>.As part of the always-ongoing maturing of the PNTP server fleet operations, we decided as we launched the Johannesburg server, it was a good opportunity for us to start diversifying our NTP server software.PublicNTP has closely followed the <a href='https://www.ntpsec.org/'>NTPsec project</a> since its inception. NTPsec has a number of contributors, but is led by the well-known <a href='https://en.wikipedia.org/wiki/Eric_S._Raymond'>Eric S. Raymond</a> (usually shortened to “ESR”).After months of internal testing, PublicNTP was confident that NTPsec was entirely up to the challenge of running in the PublicNTP global fleet.The “Community” in “Network Time Community”I personally wanted to call out yet another example of the incredible good will in the network time community.Back in 2017, PublicNTP had a response from the Network Time Foundation within 48 hours of contacting them to let them know PublicNTP had incorporated and was looking to use NTF software in PublicNTP deployments.PublicNTP also had a very positive first encounter with the NTPsec project; within 24 hours of reaching out to make them aware PNTP was testing their software with the intent to deploy to production, we’d heard back from multiple members of the NTPsec team, all of whom were all very friendly, helpful, responsive, and encouraging.We’re excited to be cultivating such connections and are eager to grow the community by providing easier NTP access for free to the public.","tags":["Africa","Cloud","Infrastructure","Servers","NTP"],"time":"Jun 17, 2019"},{"permalink":"pntp-open-architecture-stratum-one-part-5-future-investigations","timestamp":"2019-06-03T14:11:00-05:00","title":"PNTP Open Architecture Stratum One Part 5/5: Future Investigations","excerpt":"In this final entry in the series about our open architecture stratum one server, we call out some of the future investigations we noted that may be worth looking into during the exploration so far.","image_urls":[{"url":"images/biology-close-up-instrument.jpg","alt":"Microscope"},{"url":"images/architecture-art-contemporary.jpg","alt":"Circular Window"}],"content":"<h4><a href='/people.html#brad-woodfin'>J. Brad Woodfin</a></h4>In this final entry in the series about our open architecture stratum one server, we call out some of the future investigations we noted that may be worth looking into during the exploration so far.Multi-Constellation<ul><li>Our first iteration of this architecture uses the <a href='https://www.meinbergglobal.com/english/products/pci-express-gps-clock.htm'>Meinberg GPS180PEX</a>, which can only see US <a href='https://en.wikipedia.org/wiki/Global_Positioning_System'>Global Positioning System</a> satellites.</li><li>There are several competing Global Navigation Satellite Systems (GNSS), including the EU’s <a href='https://en.wikipedia.org/wiki/Galileo_(satellite_navigation)'>Galileo</a>, Japan’s <a href='https://en.wikipedia.org/wiki/Quasi-Zenith_Satellite_System'>QZSS</a>, Russia’s <a href='https://en.wikipedia.org/wiki/GLONASS'>GLONASS</a>, and India’s <a href='https://en.wikipedia.org/wiki/Indian_Regional_Navigation_Satellite_System'>NAVIC</a>.</li><li>Using hardware that can process the signals from other constellations increases the robustness of the architecture, due to:</li><ul style='list-style-type: disc;'><li>Greater availability of satellite-based time sources becoming available</li><li>Breaking the dependency upon a single time provider (in the case of GPS, the <a href='https://www.usno.navy.mil/USNO/time/master-clock'>US Naval Observatory Master Clock</a> which is the source of <a href='https://www.usno.navy.mil/USNO/time/gps/usno-gps-time-transfer'>GPS time</a>)</li></ul></ul>Multi-Frequency<ul></ul><ul><li>The first iteration of the architecture can only receive the GPS L1 <a href='https://en.wikipedia.org/wiki/GPS_signals#Legacy_GPS_signals'>Coarse/Acquisition</a> (“L1 C/A”) signal.</li><li>Making it possible for the architecture to receive timing information on at least two different frequencies (e.g., L1 and L5) allows the receiver to compute -- and thus eliminate -- the amount of ionospheric error, which is one of the <a href='https://www.e-education.psu.edu/geog160/node/1924'>largest contributors to GPS location/timing errors</a>.</li></ul>Indoor Reception<ul><li>Newer GNSS signals, such as GPS L5 and Galileo E5a/b, were <a href='https://www.gps.gov/systems/gps/modernization/civilsignals/'>carefully designed to be easier to detect as well as broadcast from the satellites at much higher power levels</a>.</li><li>The improved signal design and strength should make it easier to maintain GPS lock for L5 versus L1 C/A, especially in situations where L1 C/A typically struggles, such as when indoors.</li></ul>Lowering Price<ul><li>At 2,700 USD, the Meinberg GPS180PEX GPS receiver is the single largest factor of the overall cost of the first iteration of the PublicNTP Open Architecture Stratum One platform.</li><li>There are numerous articles on the internet about Raspberry Pi-based GPS receiver solutions that are significantly cheaper than the Meinberg hardware.</li><li>One component typically lacking from the budget GPS receiver solutions is a precise oscillator to allow the system to “freewheel” with very low drift when the GNSS signal is not available.</li><li>We’ve been looking into much cheaper solutions for both halves (GNSS signal reception as well as the precise oscillator). It’s likely that we would have to write an operating system driver to support our oscillator solution, but that’s a challenge the PublicNTP team is eager and well-suited to take on. </li><li>The lower we can push the price point, the more people can take advantage of our work, which makes it a constant goal of the project.</li></ul>There are many different avenues to improve our first iteration on this platform; these are just the first ones we noted and remembered to write down. :)We are very interested to see where the network time community takes this, which will likely be well beyond what we have envisioned so far!","tags":["NTP","Stratum One","PublicNTP","GPS","Enhancement","Improvement","GNSS","Beidou","GLONASS","L1","L5"],"time":"Jun 3, 2019"},{"permalink":"pntp-open-architecture-stratum-one-part-4-step-by-step-blueprint","timestamp":"2019-05-20T14:11:00-05:00","title":"PublicNTP Open Architecture Stratum One Part 4: Step By Step Blueprint","excerpt":"This next ‘article’ is for those tech-heads and time-nerds who are interested in setting up their own time server.","image_urls":[{"url":"images/blueprint-designing-diagram.jpg","alt":"Blueprint Design"},{"url":"images/architecture-black-and-white-buildings.jpg","alt":"Black and White Architecture"},{"url":"images/beach-blue-boardwalk.jpg","alt":"Beach Boardwalk"}],"content":"<h4><a href='/people.html#brad-woodfin'>J. Brad Woodfin</a></h4>This next ‘article’ is for those tech-heads and time-nerds who are interested in setting up their own time server.We’re keeping it short on the site because, to be honest, it would be about four times longer than our next longest piece.But to those with a hankering for those juicy details, this document’s for you.<a href='/documents/PNTP - Projects - Open Stratum One - Meinberg GPS180PEX.pdf'>PNTP - Projects - Open Stratum One</a>In it, you’ll find:<ul style='list-style-type: disc;'><li>Links and exact specifications of our <a href='https://www.dell.com/en-us/work/shop/povw/poweredge-r240'>Dell PowerEdge R240</a> and <a href='https://www.meinbergglobal.com/english/products/pci-express-gps-clock.htm'>Meinberg GPS180PEX</a> for your own purchasing purposes.<li>Instructions for the hardware setup.</li><li>Step by step directions for installing the base software (to receive satellite signals)</li><li>Step by step directions for installing software for high quality time tracking (within 20-30 microseconds)</li><li>Step by step directions for an advanced installation of extremely accurate time tracking (down to 100 nanoseconds)</li></ul>We hope that the information both informs and inspires the network time community and that it dispels a lot of the mystery behind what’s required to serve up extremely accurate estimates of UTC.Happy timekeeping!","tags":["NTP","Stratum One","Meinberg","Open Architecture","Linux","PublicNTP","GPS","Dell"],"time":"May 20, 2019"},{"permalink":"pntp-open-architecture-stratum-one-part-3-technical-overview","timestamp":"2019-04-22T14:11:00-05:00","title":"PNTP Open Architecture Stratum One Part 3: Technical Overview","excerpt":"In this third entry in the series, we get into the technical decisions we made, and the decision points we encountered along the way to the current iteration.","image_urls":[{"url":"images/stratum_3_img_1.jpg","alt":"Satellite Terminal"},{"url":"images/stratum_3_img_2.jpg","alt":"Motherboard"},{"url":"images/stratum_3_img_3.jpg","alt":"Satellite System"}],"content":"<h4><a href='/people.html#brad-woodfin'>J. Brad Woodfin</a></h4>In our previous two articles, PublicNTP <a href='/blog/posts/open-stratum-one-the-meinberg-solution.html'>introduced our open architecture stratum one solution</a>, and then explained the rationale for <a href='/blog/posts/open-stratum-one-server-part-2-rationale.html'>why we embarked on the project in the first place</a>.In this third entry in the series, we get into the technical decisions we made, and the decision points we encountered along the way to the current iteration.Form Factor<ul>Going with a standard <a href='https://en.wikipedia.org/wiki/19-inch_rack'>19 inch/482.6mm width</a>, 1.75 inch/44.5mm high (1U) rackmount form factor was a no-brainer. We are going to install these servers in commercial datacenters and it’s easier and cheaper to buy equipment that fits in a standard datacenter equipment rack.</ul>Server chassis<ul>We looked at both <a href='https://www.dell.com/en-us/work/shop/servers-storage-networking/sf/poweredge-rack-servers'>Dell PowerEdge</a> and <a href='https://www.hpe.com/us/en/servers/rack.html'>HP Enterprise</a> (HPE) rackmount servers.Based on our board members’ collective positive personal experiences working with Dell servers, the competitive price point, and availability of spare parts, we chose the <a href='https://www.dell.com/en-us/work/shop/povw/poweredge-r240'>Dell PowerEdge R240 chassis</a>.The R240 is a single socket 1U rackmount server with 2x PCIe Gen 3.0 slots, with the ability to hold up to 64GB of memory, and has configurations with price points starting below 600 USD.</ul>Server hardware configuration<ul style='list-style-type: disc;'><li>CPU: as all modern NTP daemons are single-threaded, there is no advantage for a large number of CPU cores. We purchased the two-core/four-thread <a href='https://www.intel.com/content/www/us/en/products/processors/pentium/g5500.html'>Intel Pentium G5500</a> at 3.80 GHz with 4MB of cache.</li><li>Memory: Dedicated NTP servers require very little memory. We stayed at the chassis minimum of a single 8GB 2666MT/s DDR4 ECC DIMM</li><li>Storage: NTP servers are very light on storage requirements, including capacity, latency, and IO operations per second. A pair of 1TB 7200 RPM SATA drives in a RAID-1 configuration is entirely sufficient for our needs</li><li>Network card: NTP requires very little bandwidth (roughly 200 bytes per request/response pair), so the onboard 1Gbps NIC is sufficient</li><li>Power cord: this is very dependent upon what country the server is being installed in, so it varies wildly</li>The above configuration was able to be delivered to us with free shipping for under 700 USD.</ul>PCI Express GPS Receiver<ul>We considered offerings from:<ul style='list-style-type: disc;'><li><a href='http://connecttech.com/product-category/form-factors/pci-express/'>ConnectTech</a></li><li><a href='https://www.meinbergglobal.com/english/products/pci-express-clocks.htm'>Meinberg</a></li><li><a href='https://www.masterclock.com/products/pc-cards'>Masterclock</a></li></ul>PublicNTP settled on the <a href='https://www.meinbergglobal.com/english/products/pci-express-gps-clock.htm'>Meinberg GPS180PEX</a> PCI Express card as our GPS receiver (purchased through our partners at <a href='https://www.jtime.us/'>JTime!</a>, the exclusive distributor for Meinberg in the US).So far, the Meinberg solution has been a perfect match for our needs, primarily due to the following factors:<ul style='list-style-type: disc;'><li>Competitive price point</li><li>Well-known, highly-respected brand (Meinberg has a very strong reputation)</li><li>Three year warranty standard, extendable to ten years</li><li>Extension can be purchased at any point, during ordering or later down the road</li><li>Excellent <a href='https://www.meinbergglobal.com/english/support/tech-support.htm'>technical support</a>, including lifetime software support</li><li>Manufacturer-provided and -supported driver support for Linux, including <a href='https://www.meinbergglobal.com/english/info/ntp.htm'>out-of-the-box support</a> for the NTP reference implementation (which we run)</li><li>Meinberg-produced and -supported active (amplified) <a href='https://www.meinbergglobal.com/english/products/gps-antenna-converter.htm'>GPS antenna</a> included</li><li>Optional <a href='https://www.meinbergglobal.com/english/products/surge-voltage-protector.htm'>Surge voltage suppressor</a></li><li>Excellent documentation, particularly the <a href='https://www.meinbergglobal.com/download/docs/manuals/english/gps180pex.pdf'>user manual</a></li><li>Supports long runs from card to antenna</li><ul><li>Can do 300 meters with RG58 and 700 meters with RG213 without any additional amplification needed</li></ul></ul>The order we placed with JTime!’s <a href='https://www.meinberg-usa.com/contact/our-team.htm'>Marc Street</a> had an invoice price, including shipping, of 1,970 USD.</ul>Operating System<ul>PublicNTP primarily runs <a href='https://wiki.ubuntu.com/LTS'>Ubuntu Long Term Support</a> (LTS) on our servers, so we are currently installing <a href='http://releases.ubuntu.com/18.04/'>Ubuntu 18.04 (“Bionic Beaver”)</a>.</ul>NTP Server Software<ul>PublicNTP currently runs the Network Time Protocol <a href='http://ntp.org/'>Reference Implementation</a> (more commonly known as “ntpd”) from the <a href='https://www.nwtime.org/'>Network Time Foundation</a> on our servers.We have started internal testing with NTP server software from other vendors, including the <a href='https://www.ntpsec.org/'>NTPsec</a> project, but these efforts aren’t yet mature enough to roll out to our production servers.</ul>UTC Synchronization<ul>We configured NTPd to synchronize to UTC using both the <a href='http://doc.ntp.org/4.2.8/drivers/driver28.html'>shared memory</a> (SHM) driver and the <a href='http://doc.ntp.org/4.2.8/drivers/driver22.html'>Pulse-Per-Second</a> (“PPS”) clock discipline using pin one of the serial output of the Meinberg GPS receiver.The shared memory driver allows the server to synchronize within about ~70 microseconds (0.000070 seconds) of <a href='https://en.wikipedia.org/wiki/Coordinated_Universal_Time'>Coordinated Universal Time</a> (UTC).Supplementing the shared memory driver with the PPS clock discipline allows the server to get within ~10 microseconds (0.000010 seconds) of UTC.</ul>PublicNTP takes pride in both the work we’ve put in to this list of resources and that we are here to share it for everyone with an interest in our purpose. We hope this provides a roadmap of costs and hardware for your own time server needs. Stay tuned for our next article where we’ll be taking a deep dive into our step-by-step process.","tags":["NTP","Time","Meinberg","Stratum One","GPS","UTC","Dell","Server"],"time":"Apr 22, 2019"},{"permalink":"open-stratum-one-server-part-2-rationale","timestamp":"2019-04-01T14:11:00-05:00","title":"PublicNTP Open Architecture Stratum One Server Part 2: Rationale","excerpt":"As posted in PublicNTP’s previous blog article, our team has spent many hours and meetings over the past months identifying the advantages and disadvantages of our own open architecture. We’ve looked hard at utilizing tried and true NTP servers in tandem with our own established network of stratum two and, more recently, stratum one servers.","image_urls":[{"url":"images/analogue-antique.jpg","alt":"Ancient Map"},{"url":"images/GPS180PEX.jpg","alt":"GPS180PEX"},{"url":"images/deep-diver01.jpg","alt":"Deep Sea Diver"},{"url":"images/deep-diver02.jpg","alt":"Deep Sea Diver 2"}],"content":"<h4><a href='/people.html#brad-woodfin'>Brad Woodfin</a></h4>As posted in PublicNTP’s <a href='/blog/posts/open-stratum-one-the-meinberg-solution.html'>previous blog article</a>, our team has spent many hours and meetings over the past months identifying the advantages and disadvantages of our own open architecture. We’ve looked hard at utilizing tried and true NTP servers in tandem with our own established network of stratum two and, more recently, stratum one servers.Several people have asked why PublicNTP built its own solution, considering that fully-packaged, ready-to-deploy appliances can be easily acquired from a number of vendors.It’s an excellent question: why rebuild the wheel? We aim to answer that here.In our case, there were multiple reasons we decided to buy and build our own stratum one. In no particular order:Cost Savings<ul>After getting quotes for several entry-level NTP server appliances from many commercial companies, we became convinced that we could build our own solution for less money. Their products are impressive, but as our funding comes from donated dollars, PublicNTP must carefully ensure that each dollar goes as far as it can.</ul>Technical Challenge<ul>The members of our board and dev team had a funny reaction once we reported the results of the easier, buy-off-the-shelf research. It became a challenge to work lean and within a different set of constraints. Testing out the various options for optimal solutions was both educational and enjoyable for us!</ul>Integration Into Existing Fleet Monitoring<ul>The available appliances we looked at all came with their own closed architecture. While they did support things such as <a href='https://en.wikipedia.org/wiki/Simple_Network_Management_Protocol'>Simple Network Management Protocol</a> (SNMP), our shop is primarily <a href='https://www.ubuntu.com/'>Ubuntu</a>-oriented. We needed to ensure they were compatible with our Linux/Ubuntu monitoring structure. Obviously, we can’t run <a href='https://landscape.canonical.com/'>Canonical Landscape</a> on an NTP appliance that isn’t running Ubuntu.</ul>Extensible, Open Architecture<ul>One of the most important goals of our project is to keep this open-source. We want to help the world help itself and needed to ensure we were free to publish our architecture.Our vision is to see more and more of the community share ideas on how to improve upon PublicNTP’s work. One of the greatest examples and inspirations of community-made progress is the <a href='https://www.opencompute.org/'>OpenCompute Project</a>. The benefits of the open and free exchange of ideas are too great not to share our methodology.</ul>Dealing With Hardware Failure<ul>Despite the fidelity of hardware these days, things always break. If (and of course when) one of our stratum one deployments needs a replacement component, we want to be able to quickly source the part, ship it, and get things fixed. We’d rather not wait for a back order or pay an exorbitant amount to acquire a specific piece of hardware.All of this is simplified when using widely-available, commercial equipment instead of a custom appliance.</ul>Based on these points, our <a href='/people.html'>Board of Directors</a> decided that investing the hours and money to design, build, and test our own open architecture was the best option.Look forward to our next installment in this blog series. We’ll be tackling the technical overview of PublicNTP’s stratum one open architecture.","tags":["NTP","Time","Stratum One","GPS","UTC","Dell","Server"],"time":"Apr 1, 2019"},{"permalink":"open-stratum-one-the-meinberg-solution","timestamp":"2019-03-18T14:11:00-05:00","title":"Open Stratum One - The Meinberg Solution","excerpt":"Since PublicNTP started in the summer of 2017, all but one of the servers we have deployed have been stratum two servers according to the Network Time Protocol.","image_urls":[{"url":"images/adventure-ancient-antique.jpg","alt":"Ancient Map"},{"url":"images/business-computer-connection.jpg","alt":"Computer Connections"},{"url":"images/business-cabinet-data.jpg","alt":"Server Racks"}],"content":"<h4><a href='/people.html#brad-woodfin'>Brad Woodfin</a></h4>Since PublicNTP started in the summer of 2017, all but one of the servers we have deployed have been stratum two servers according to the Network Time Protocol.To understand what that means more fully, let’s start by explaining stratums.A stratum, in layman’s terms, is the number of “wires” between your NTP server and a high quality source of time.<ul><li>Stratum Zero refers to a direct source that accesses nanosecond or better timing data (a GPS receiver would qualify as a stratum zero source)</li><li>Stratum One is a server that is directly and physically connected to a receiver of stratum zero time -- allowing them to deliver the highest precision time possible.</li><li>Stratum Two receives its time estimates from a number of NTP servers, at least one of them being a stratum one.</li><li>Stratum Three likewise receives its time from a web of servers but the most reliable reference is stratum two.</li></ul>And it goes on ad infinitum; your server’s stratum number is one higher than the lowest stratum server used as a reference.This makes NTP incredibly resilient. If your stratum two server is connected to many highly reliable stratum two servers and a couple strong stratum ones, you can still come out with accurate time.For PublicNTP, that means that stratum two cloud servers are, in highly concentrated cities, a dandy option. But as we got into lesser-developed regions of the world, such as our South America and Africa deployments, it’s not enough. In these underserved areas, the upstream time across cloud servers is inadequate for the level of accuracy we want to provide to people who use PublicNTP servers as their source of time. In Nigeria, for example, the two upstream time sources it had access to were all in the United Kingdom or South Africa. The UK is about 5000km away and South Africa is about 4500km away. And we were seeing synchronization within 10 and 50 milliseconds of UTC. That’s a testament to how the NTP web can support global time -- but that’s approximately ten times less accurate than the rest of our servers.So we needed a stratum one server with direct access to GPS synchronization. That way we could provide the source rather than go looking for it. But our first exploration into those logistics revealed a high price tag.There are many commercially available stratum one rackmount servers. We found that much of the gear started at $5000. Which is totally understandable; it’s a niche market and that kind of price is what covers the overhead of storing a product that may not see many buyers. Combine that with precision gear like temperature-controlled crystal oscillators and it’s an expensive item right away.PublicNTP started looking for ways to build our own stratum one server for less than we could buy them from commercial vendors. That way we could base it on an open architecture and save money at the same time!After plenty of looking, we settled on a <a href='https://www.dell.com/en-us/work/shop/povw/poweredge-r240'>Dell PowerEdge R240</a> for the first part of our solution. Designed to be racked in a datacenter, with its small size and low power usage, it was a low-cost alternative to a heavier piece of equipment. Thanks to its standard equipment and components, it would also be easy to get and ship replacement parts should they fail. With Linux installed, we can run any monitoring programs we wanted (unlike being locked to proprietary software).And with Linux as our OS, we found that the <a href='https://www.meinbergglobal.com/english/products/pci-express-gps-clock.htm'>Meinberg GPS180PEX</a> was an excellent GPS receiver to parter it with. With a matched antenna and serial port output, the Meinberg solution ensures good GPS reception.With our puzzle-pieced equipment, we save enough money to essentially double our deployments. PublicNTP continues to search for host locations into 2019 and we’re excited to begin with these stratum one servers!","tags":["NTP","IT Deployments","GPS","Servers","Open Architecture"],"time":"Mar 18, 2019"},{"permalink":"publicntps-stratum-one-deployment-checklist","timestamp":"2019-03-04T14:11:00-05:00","title":"PublicNTP’s Stratum One Deployment Checklist","excerpt":"Much in the same way we dove into virtual servers, we figured it would be fairly easy to find an underserved country and find a company who would lease us some power, space, and network bandwidth.","image_urls":[{"url":"images/atlas-adventure.jpg","alt":"Atlas addventure"},{"url":"images/airbus.jpg","alt":"Airbus"},{"url":"images/do-not-enter.jpg","alt":"Do Not Enter"}],"content":"<h4><a href='/people.html#dan-noland'>Dan Noland</a></h4>The previous blog <a href='/blog/posts/What-Goes-Into-Deployment.html'>article</a> about server deployment mentioned our two physical (stratum one server) deployment projects: Project Ikenga and Project Tonatiuh.Much in the same way we dove into virtual servers, we figured it would be fairly easy to find an underserved country and find a company who would lease us some power, space, and network bandwidth.Oh, how adorably naive we were… :)After much trial and error, investigation, and a couple wake-up calls from some...let’s go with “well-traveled”...friends, we gradually came to realize that there is a lot more to nailing down a location than finding the nearest person happy to take our money.Multiple times during our search into locations, we’d find a deal-breaking detail. We then would take this new information and check to see if any of the other locations had that issue as well. During this evolution of understanding, we developed a “checklist” of sorts that contains several different dimensions of what’s involved with a deployment. It’s unlikely to find a “perfect” location, but the goal is find the locations with the most acceptable tradeoffs.The following is our current, subject-to-change, guide for locating the best cities to lay down physical gear. As we have been adding things to consider to the list since the search started, it’s likely that it will continue to change/expand as time goes on.1. IS-95A cell phone network: it turns out that in order to participate in a cellular network, the cell towers have to have hyper accurate time.. While you do need a subscription plan to send messages or make phone calls on an IS-95A network, you do not need one to read the timestamps on messages sent by the cell towers. Several stratum one NTP server vendors take advantage of the fact that just being within the broadcast range of a cell tower means you can have incredibly accurate (nanosecond-level) time accuracy.While we also have GPS-based servers on hand, we feel going for a simpler, cheaper deployment for our first one makes a lot of sense. IS-95A CDMA-based gear is affordable and transportable enough that the company has a couple sitting on a shelf in my closet.2. Datacenter Provider We Can Have Confidence In: This was a fun lesson. We were swapping emails with a potential provider in Africa (a location that we discovered would fail more than one of the tests on this list -- thankfully BEFORE we arrived!). The company had a nice website, were very responsive to our emailed questions, and seemed very eager to become our hosting partner. As we did more research on logistics, we thought to look at the location using satellite imagery, only to discover it was a literal hut with a satellite dish. Needless to say, we realized that vetting both the personalities AND the location/facility were required for future exchanges.3. Health risk: A lot of underserved countries present risk for diseases that we in the United States are lucky enough to put out of our minds Thankfully, the Center for Disease Control gives a list of recommended vaccinations for every country on the planet. It was a bit of a shock to realize that for some of the locations we were going to require vaccinations for things like <a href='https://wwwnc.cdc.gov/travel/destinations/traveler/none/nigeria'>typhoid</a>, <a href='https://wwwnc.cdc.gov/travel/destinations/traveler/none/brazil'>yellow fever</a>, <a href='https://wwwnc.cdc.gov/travel/destinations/traveler/none/ghana'>meningitis</a>, <a href='https://wwwnc.cdc.gov/travel/destinations/traveler/none/ivory-coast'>hepatitis A/B</a>, <a href='https://wwwnc.cdc.gov/travel/destinations/traveler/none/kenya'>cholera</a>, and/or <a href='https://wwwnc.cdc.gov/travel/destinations/traveler/none/senegal'>malaria</a>.Some of these vaccines require scheduling your appointment weeks or months in advance, and some take weeks or months after receiving the vaccine for the body to reach maximum immunity. All of this needs to be factored into travel planning.4. Safety risk: we leave it to the State Department’s <a href='https://travelmaps.state.gov/TSGMap/'>travel advisory information</a> to help inform this dimension. Obviously we have a strong preference for level 1 (“normal level of awareness“) but are willing to consider countries up to level 2 (“increased level of caution“). As this is a passion, not a job, we don’t see a compelling reason to disagree with the U.S. government when they encourage people to “reconsider travel” or “do not travel” to an area.5. Visas: PublicNTP has a preference for countries with no requirement for business visas to enter their country. It’s not a big deal, but it does add complications and costs.6. Accessibility: we’re not looking for an Indiana Jones-grade adventure to get to our server box. For now we’re looking for locations with a major international airport with good connectivity (if indirect) from where our Directors live: Utah, Indiana, and the Washington, DC Metro Area.7. Costs (Travel): Costs of airfare, hotel, and food need to be reasonable. When committing to a deployment, we’re wanting to remain on-site for anywhere up to a couple weeks to make sure things are up and running. More people + more time = more costs we have to anticipate.8. Costs (Datacenter): we’re not big enough or influential enough (yet?) to build our own sites and maintain them. So it’s in our best interest to work with people and businesses who already have a datacenter. Their one-time and recurring costs for installing and hosting our servers need to fit within the budget our donors make possible.9. Connectivity: we want to deploy our time servers where they will reach the widest audience possible. A big part of that is what the network connectivity from the datacenter looks like. It’s very common, for example, to see boomerang routing between locations very close to each other in South America. We’ve seen a number of internet routes from, for example, <a href='https://www.google.com/maps/place/Bogot%C3%A1,+Bogota,+Colombia/@4.6671859,-83.0888895,5z/data=!4m5!3m4!1s0x8e3f9bfd2da6cb29:0x239d635520a33914!8m2!3d4.7109886!4d-74.072092'>Bogota, Colombia</a> to <a href='https://www.google.com/maps/place/S%C3%A3o+Paulo,+State+of+S%C3%A3o+Paulo,+Brazil/@-4.5633786,-59.3301997,4.1z/data=!4m5!3m4!1s0x94ce448183a461d1:0x9ba94b08ff335bae!8m2!3d-23.5505199!4d-46.6333094'>São Paulo, Brazil</a> that route through Miami, Florida in the United States. It makes no logical sense that you’d have to go north--the exact opposite direction--before being able to go back south. These sort of network access issues affect the suitability of hosting platforms for PublicNTP.10. Import/Export Restrictions: due to US and international law, it may not be legal for us to deploy equipment in a given part of the world.As an illustration, here is the <a href='https://www.microsoft.com/en-us/exporting/exporting-information.aspx'>export list for Microsoft products</a>. Note that the XBox One has an Export Control Classification Number (<a href='https://www.bis.doc.gov/index.php/licensing/commerce-control-list-classification/export-control-classification-number-eccn'>ECCN</a>) of 5A992.c, assigned by the US Department of Commerce. As it has been classified, there are certain countries where one would have to apply for a license before taking an XBox One to those countries (in this case, the only countries it would not be legal to export the hardware to is on the Department of Commerce’s <a href='https://www.bis.doc.gov/index.php/policy-guidance/country-guidance/sanctioned-destinations'>Sanctioned Destinations</a> list).On the flip side, there are countries where it may be illegal to import the equipment. One reason for such restrictions might be that the equipment contains cryptographic capabilities. PublicNTP has retained legal counsel to help us ensure that we abide by all import/export laws of the US and any countries where we may deploy systems.This list is subject to change because we’re always learning more. Still, these basic guidelines have already helped us vet various locations, make quick decisions, stay safe, and triage/prioritize some of our first possible location.","tags":["ikenga","tonatiuh","deployments","checklist","travel","visas"],"time":"Mar 4, 2019"},{"permalink":"What-Goes-Into-Deployment","timestamp":"2019-02-18T14:11:00-05:00","title":"What Goes Into Deployment?","excerpt":"As of this writing, all of PublicNTP’s deployed time servers are virtual (i.e., running in the cloud) -- except for Salt Lake City, because Scott is a fan of turning things up to 11. This approach has served us well so far. It’s becoming clear that we’re reaching the point of diminishing returns for cloud deployments. In the underserved countries we’re now trying to target, cloud just won’t be an option.","image_urls":[{"url":"images/checklist_deployment.jpg","alt":"Checklist"}],"content":"<h4><a href='/people.html#brad-woodfin'>Brad Woodfin</a></h4>As of this writing, all of PublicNTP’s deployed time servers are virtual (i.e., running in the cloud) -- except for Salt Lake City, because Scott is a fan of turning things up to 11. This approach has served us well so far. It’s becoming clear that we’re reaching the point of diminishing returns for cloud deployments. In the underserved countries we’re now trying to target, cloud just won’t be an option.Before we talk about the deployment avenues that remain viable for us, let’s take a minute to “talk stratums.”If you read up on Network Time Protocol (NTP), you’ll quickly hit the term “stratum,” which is followed by a number. You’ll see “stratum zero” servers, “stratum one” servers, etc.What does that mean?Stratum is a term to describe how close to an authoritative NTP time source a device is.To oversimplify, you can view it as “the number of wires you need to cross before you can reach an authoritative time source.”In other words, stratum zero servers are authoritative devices like GPS and other atomic clock sources -- no wires have to be crossed to reach the high-precision time source, they ARE the time source.Devices like a Linux server connected directly to a stratum zero device is considered a stratum one device (there’s “one wire” from the server to its GPS receiver). The math follows on down the stratum tiers: if a stratum two device obtains its time from stratum one servers (usually across the Internet).The cool thing is that using NTP, devices can connect to multiple sources of time -- meaning that a stratum two servers can be querying multiple stratum one servers. That web of connections allows a device that’s two steps removed from an “authoritative” source to be much more reliable, as it can “double-check” the answers it’s getting from multiple upstream sourcesAll the PublicNTP servers (other than Utah!) are stratum two servers. We sync to at least five stratum one sources, which keeps them within 1 and 10 milliseconds of the international time standard. So far we’ve been very pleased with the results. Deploy a server in developed parts of the world with major cloud infrastructure, such as Europe or East Asia and you’ll find a plethora of upstream time sources to ensure your clocks are accurate.When PublicNTP set out to deploy in lesser-developed regions of the world, we figured that NTP would still be resilient enough to be able to work well.That was a tad optimistic -- at least with our definition of NTP when it’s “working well.” :)Turns out that absolutely everything is stacked against having highly-accurate time in much of the world. For example, we deployed a virtual server in Lagos, Nigeria. Through this process, we discovered first-hand how the severe lack of terrestrial cables across Africa turned out to be a huge obstacle. If you looked at the path of all data traveling out of Lagos through our time server, the only two paths we had access to were submarine fiber optic cabling to either South Africa OR London.When you pause to <a href='https://www.submarinecablemap.com/'>look at a map</a>, that’s an enormous distance to cross even for data moving at the speed of light. Testing quickly showed we couldn’t get more accurate than 100 milliseconds from UTC, due to lack of upstream time sources within several thousand miles from Nigeria.For humans, a tenth of a second is almost negligible. But for digital infrastructure that requires synchronization within 1/1,000th of a second or better, it was a rough situation.We quickly realized we were facing a situation that our current approach wasn’t going to work with.Enter <a href='/blog/posts/Project-Ikenga.html'>Project Ikenga</a> and Project Tonatiuh: PublicNTP’s first two physical deployment campaigns. We’ll talk specifically about both of these projects in future articles but in short, we realized that we had to bring our own stratum one sources, as there weren’t any in the region which could make stratum two servers viable. Pursuing physical deployments come with their own brain teasers which we’ve been digging into for the last several months.PublicNTP is excited to continue expanding our deployment footprint, installing our servers wherever they’re most needed. After hundreds of (combined) hours investigating physical deployments, we’ve gradually developed a checklist of sorts that we use to try and quickly vet the viability of a potential deployment location. Watch for an article coming soon that breaks that checklist down -- and consider yourself invited to help us improve it. It continues to improve, but like everything about this project, it can only get better!","tags":["NTP","IT Infrastructure","Africa","South America","Checklist","Timekeeping","Deployment","Servers"],"time":"Feb 18, 2019"},{"permalink":"Project-Ikenga","timestamp":"2019-02-04T14:11:00-05:00","title":"Project Ikenga - Connecting Africa","excerpt":"When PublicNTP originally targeted Africa for a deployment campaign we knew that there were going to be a few challenges.","image_urls":[{"url":"images/Ikenga_1.jpg","alt":"Ikenga statue."}],"content":"<h4><a href='/people.html#scott-waddell'>Scott Waddell</a></h4>The <a href='https://en.wikipedia.org/wiki/Odinani'>Odinani</a> religion of the <a href='https://en.wikipedia.org/wiki/Igbo_people'>Igbo people</a> prizes and worships spirits known as <a href='https://en.wikipedia.org/wiki/Alusi'>Alusi</a>. Alusi come in many forms and serve many functions but one of the most powerful symbols/artifacts is the carved, horned Alusi known as <a href='https://en.wikipedia.org/wiki/Ikenga%23God_of_Time'>Ikenga</a>.There are two functions of Ikenga:A God of AchievementIkenga represents achievement, success, human endeavor, and victory -- grounded in the belief that the power for man to achieve comes from his right hand.A God of TimeThe oldest concept of Ikenga is Two-faced Ikenga. Where one face looks to the old year, the other looks to the new. It is the basis of the most ancient Igbo calendar and is the god of beginnings.While time is our wheelhouse at PublicNTP, we appreciate both sentiments and felt that the name and symbolism appropriately fit our ambition for deploying accurate time servers in Africa -- firstmost considering the Nigerian area: where the Igbo people reside.The CampaignWhen PublicNTP originally targeted Africa for a deployment campaign we knew that there were going to be a few challenges:<ul><li>1) We wanted to make sure to deploy in the least-served areas of countries/continents to maximize our positive impact.</li><li>2) Basic infrastructure, including reliable internet and electricity, gets more challenging to find the further you get from major urban areas</li><li>3) Due to the nature of the NTP protocol, the further our deployment was from its upstream time sources, the time our server delivered would become less and less accurate</li></ul>We discovered quite a few more hurdles in the investigation but we knew that location, location, location would be at the fore.While we wanted to first approach Project Ikenga with cloud servers, we needed to make sure that we had a reliable server base -- so there was a bit of a balancing act that needed analyzing. If we planted down in a major city center, we’d be small fish in an already developed network: reliable but with little impact.What became a search for a server became a search for a country:Lagos, Nigeria - The Middle Point of the ContinentNigeria has the largest <a href='https://en.wikipedia.org/wiki/List_of_African_countries_by_GDP_(nominal)'>Gross Domestic Product</a> (GDP) of all African countries, plus the internet routes from Nigeria often have pass through South Africa or London. Mighty long distances for data to go.Nigeria seemed like a good central African location to start from...until we realized that the <a href='https://travel.state.gov/content/travel/en/traveladvisories/traveladvisories/nigeria-travel-advisory.html'>US State Department travel advisories</a> (marking it as a level 3/4, “reconsider travel” as of November 18, 2018) made it less attractive to fly in, vet the data center personally, and deploy ourselves.Johannesburg, South Africa - The Southern GemSouth Africa has an infrastructure that outpaces much of the underserved areas PublicNTP is looking to help and would, presumably, provide plenty of stratum 1’s for our stratum 2 to coordinate with.At the same time, being at the southern tip of Africa, our services are competing with the rest of the internet in South Africa and from its location we lose its effectiveness the further north in the continent the clients are.Lisbon, Portugal - The Mysterious Rider from the NorthThis location came up in our search recently. Because, well, we were constantly searching for ways to support Africa from Africa. We had placed ourselves in a box and hadn’t realized it.Lisbon, much like Johannesburg, provides an internet-rich environment for our time server to coordinate with. And though it suffers the problem of being quite a bit north of Africa, a notable number of fiber optic cables along the western coast of Africa connect in Lisbon:<ul><li><a href='https://www.submarinecablemap.com/#/submarine-cable/sat-3wasc'>SAT-3/WASC</a></li><li><a href='https://www.submarinecablemap.com/#/submarine-cable/africa-coast-to-europe-ace&sa='>ACE</a></li><li><a href='https://www.submarinecablemap.com/#/submarine-cable/mainone'>MainOne</a></li><li><a href='https://www.submarinecablemap.com/#/submarine-cable/west-african-cable-system-wacs'>WACS</a></li></ul><img style='max-width:100%;' src='/images/Ikenga_2.jpg' alt='Portugal cables' />Image from <a href='https://www.submarinecablemap.com'>https://www.submarinecablemap.com</a>, Copyright © PriMetrica, Inc.Enter the triple play. Why not deploy in all three? Lisbon first, to provide reliable time for Western Africa; Lagos next, where we can improve connection (likely by shipping a server to the datacenter and letting the local staff install it for us); and Johannesburg for coverage of southern and eastern Africa.It’s a plan that will take more time than our original one-stop deployment plan but PublicNTP is looking forward to the challenge, and to providing time to a continent with intense need for improvements to basic internet infrastructure.","tags":["Ikenga","Africa"],"time":"Feb 4, 2019"},{"permalink":"2018-board-meeting","timestamp":"2019-01-21T14:11:00-05:00","title":"2018 PublicNTP Board Meeting - Upcoming Year","excerpt":"Board members look forward to what 2019 holds for PublicNTP","image_urls":[{"url":"images/man-on-mountain.jpg","alt":"Man on mountain."}],"content":"<h4><a href='/people.html#scott-waddell'>Scott Waddell</a></h4>2018 was full of excitement and progress -- which has provided plenty of steam for us to face the challenges that lie ahead of us in 2019.The FormalitiesAs the year turns, it’s up to a non-profit charity to make sure its numbers and forms are all squeaky clean. Maintaining tax-exempt status requires very careful documentation.We’re happy to say that PublicNTP has seen an increase in income and <a href='/donate.html'>donations</a> in 2018 and that we have been receiving said income through multiple channels. Further expansion, though, has brought <a href='https://www.councilofnonprofits.org/tools-resources/charitable-solicitation-registration'>charitable solicitation registration</a> to our attention. We’ll need to register with Maryland, Utah, and Virginia as we have board members living in those states (Indiana does not require any particular registration). Thankfully we can register with all three using a <a href='http://multistatefiling.org/'>single form</a>.With our <a href='/documents/pntp_irs_filing_2017.pdf'>2017 IRS filing</a> under our belt, filing for our 2018 should be easier. Developing this track record is always a high priority for us around this time of year.Stratum One HorizonUp to this point, we have deployed several stratum two cloud servers. There is a cap on the usefulness of this approach, as stratum one servers are required to really sync up time-wise. This is why we’re now ready to step into that arena with some on-site equipment of our own.This last year we ordered the <a href='https://www.meinbergglobal.com/english/products/pci-express-gps-clock.htm'>Meinberg GPS180PEX</a> PCI GPS receiver card. It looks like it will integrate cleanly with ntpd through a shared memory driver. Once we’ve settled on a server case, we’ll be able to ship (or in the rare case, personally install) this gps clock card to prospective sites around the world.Stateside, we’re looking to start in more centralized, if not more saturated, locations like Los Angeles, California; Dallas, Texas; and Miami, Florida.Prospective sites outside the US include Lagos, Nigeria and Buenos Aires, Argentina for starters. But our plan is to branch out further from those central locations in order to strengthen our global web of stratum one and two servers.You can help us too! In early 2019, we’ll be releasing a step by step set of instructions, including an itemized purchase list so fellow horology enthusiasts can build and deploy their own stratum one time servers!Stratum Two InfrastructureAs always, we’re on the lookout for more ways to connect our cloud servers and 2019 has a few already slated to open including Bahrain and Hong Kong. South Africa is another we’re looking at, but we first need to apply for Azure nonprofit credits.Ways PublicNTP ImprovesOne major focus area for 2019 will be improving our server monitoring. With us deploying more servers than we ever have, the board members agreed that a more centralized monitoring/maintenance approach is now justified.We’ll be tracking health metrics (such as incoming and outgoing packets) as well as keeping a watchful eye on the fleet to make sure that required patches are applied immediately.We’re also looking at reforming the code infrastructure of our servers. That way our dev team can weed out potential issues at the source.Another point of proactive planning regards our targeted fundraising. We’ll be wanting to focus on donations from various hardware vendors (those that sell stratum one server options) as well as identifying interested companies abroad (Africa and South America being our primary targets throughout 2019).The Way ForwardWe’re always learning more here at PublicNTP. The world is vast, full of curiosities, and requires a delicate but deliberate effort to make positive change. 2018 showed us that our goal is not only useful but achievable. It will be our guiding star through 2019 -- a year we couldn’t be more excited for.","tags":["2018","Board Meeting"],"time":"Jan 21, 2019"},{"permalink":"2018-year-in-review","timestamp":"2019-01-07T14:11:00-05:00","title":"Year in review","excerpt":"PublicNTP has its annual board meeting in November, where the directors review the changes, achievements, and progress made in the previous twelve months. As always, it was a delight to get all four board members face to face to discuss what has happened.","image_urls":[{"url":"images/elephant-sunset-silhouette.jpg","alt":"Elephant Sunset."},{"url":"images/hockey_game.jpg","alt":"Enjoying a hockey game."}],"content":"<h4><a href='/people.html#terry-ott'>Terry D. Ott</a></h4>The calendar already shows November again -- does having a time-based charity somehow make time fly by even faster?PublicNTP has its annual board meeting in November, where the directors review the changes, achievements, and progress made in the previous twelve months. As always, it was a delight to get all four board members face to face to discuss what has happened.Our WebsitePublicNTP’s web presence has been under constant iteration all of 2018. Thanks to our <a href='https://pxp200.com/'>dev team</a> it’s looking better than ever. From image optimization to implementing what I can only describe as JSON wizardry for content pushing, the site has seen unexpected and remarkable progress. It’s allowed our dev team and writers to kick off our content engine with regular blog posts. But most importantly our <a href='/donate.html'>donate page</a> is up and running for international use as of this last summer!Server DeploymentsIn the last year, we’ve established many cloud servers on both Western and Eastern hemispheres.Western deployments: <ul><li>Miami, Florida</li><li>Dallas, Texas</li><li>Buenos Aires, Argentina</li><li>Santiago, Chile</li></ul>As discussed in our <a href='/blog/posts/keeping-company-pointed.html'>last board meeting article</a> our efforts to deploy in China met a steep wall and we’ve instead created servers at surrounding entry points:<ul><li>Tokyo, Japan</li><li>Seoul, Korea</li><li>Hong Kong</li><li>Singapore</li><li>India</li></ul>We’re eager to add more, including Taiwan.2018 FinancialsPublicNTP’s first full year as a recognized non-profit charity was a great success. We have several months of expenses already set aside which will allow us to further use donations toward expanding projects and increasing server stability.The best part is PublicNTP received donations from a larger number of sources compared to our 2017 financial year. Constantly expanding our donor base is a critical push for the stable financial net that all non-profits need.2018 ChallengesNot all things have been so rosy though. We’ve seen companies hosting our cloud servers go out of business with no warning, recycle their management, or simply demand a pay plan that’s not sustainable with our budget. It’s a natural progression of business, especially in underserved countries. Our infrastructure has been and always will be evolving, as we constantly explore how to maximize the impact that every donated dollar can have.Post-Meeting SocializingOnce the work was done, the four officers did dinner at <a href='https://www.bibianadc.com/'>Bibiana</a>, followed by a Washington Capitals hockey game!With the successes and learning opportunities of 2018, the coming year looms over PublicNTP with promises of even greater opportunity and challenges. We’re excited to face them and discuss them in the next article.","tags":["Year in review","2018"],"time":"Jan 7, 2019"},{"permalink":"bring-time-to-the-world","timestamp":"2018-12-12T14:11:00-05:00","title":"The Time Server App Has Launched!","excerpt":"The Time Server app is a reflection of both PublicNTP’s vision for the future and a sizeable portion of our concerted efforts in the last two years. With Time Server, you’ll be able to have and redistribute some of the most accurate time on the planet.","image_urls":[{"url":"images/blog-time-server-app-pixel_2-gps.jpg","alt":"Time Server App on Pixel 2"},{"url":"images/blog-time-server-app-meeting-room.jpg","alt":"Time Server App in Meeting Room"},{"url":"images/blog-time-server-app-outside-gps.jpg","alt":"Time Server App Outside using GPS"},{"url":"images/blog-time-server-app-showcase-server.jpg","alt":"Time Server App Showcase Image"}],"content":"<h4><a href='/people.html#terry-ott'>Terry D. Ott</a></h4>PublicNTP is thrilled to announce the release of our free Android app <a href='https://play.google.com/store/apps/details?id=app.timeserver'>Time Server</a>.The Time Server app is a reflection of both PublicNTP’s vision for the future and a sizeable portion of our concerted efforts in the last two years. With Time Server, you’ll be able to have and redistribute some of the most accurate time on the planet.<a href='https://play.google.com/store/apps/details?id=app.timeserver'><img style='width:150px;' src='/images/google-play.png' alt='Download from the play store' /></a>FeaturesOur Time Server app is a straightforward, easy to understand tool providing highly-accurate time and location information.The first function, in line with the PublicNTP mission, is time-keeping. You’ll see time down to the fraction of a second (with estimated error margin) in both <a href='https://en.wikipedia.org/wiki/Coordinated_Universal_Time'>Coordinated Universal Time</a> (“UTC”) and your local time, thanks to data directly obtained from the <a href='https://en.wikipedia.org/wiki/Global_Positioning_System'>Global Position Satellite</a> (GPS) system.As GPS serves as both a time and location reference, with Time Server you’ll be able to track where you are on the globe using four different reference systems:<ul><li><a href='https://en.wikipedia.org/wiki/Geographic_coordinate_system#Latitude_and_longitude'>Latitude/Longitude</a> (WGS84)</li><li><a href='https://en.wikipedia.org/wiki/Universal_Transverse_Mercator_coordinate_system'>Universal Transverse Mercator</a> (UTM)</li><li><a href='https://en.wikipedia.org/wiki/Military_Grid_Reference_System'>Military Grid Reference System </a>(MGRS)</li><li><a href='https://en.wikipedia.org/wiki/Open_Location_Code'>Open Location Code</a> (OLC)</li></ul>You also have the option of linking your location to Google Maps or sharing it with others.One of Time Server’s coolest functions is the ability to display the navigation satellites providing time/location information to your phone in real time.Using our compass-bound interface, you can tell where you stand in relation to the current constellation orbiting satellites along with their signal strength. Tapping displays specific information for each GPS satellite including Signal-to-Noise Ratio, Altitude Angle from your position, and its heading.Saving the best for last, the Time Server app lets you host -- you guessed it -- a time server!For a miniscule amount of battery and processing, you can become a point of light in the world’s time network. You’ll act as a small but integral part in the future of keeping the world’s digital clocks synchronized.We have strived to maintain an intuitive interface for tracking your server traffic -- allowing you to watch your contributions to the world.An Open Source CommunityPublicNTP’s mission is increasing the number of unrestricted, accurate, reliable, network-accessible digital time references around the world. To achieve our mission, PublicNTP takes maximum advantage of technology built by others who came before us and was generously open-sourced for the public good.In line with that spirit of openness, PublicNTP is dedicated to proliferating tools and information with the express hope others can and will build upon our work.PublicNTP hopes you’ll explore the <a href='https://github.com/PublicNTP/time-server-app'>Time Server’s open-source, MIT-licensed source code on GitHub</a> and contribute changes that take it far beyond PublicNTP’s original vision.PublicNTP’s Time Server is a fun, informational tool bringing accurate time and location information to the corners of the globe. Give it a try, if you’re interested <a href='https://timeserver.app/'>visit our site</a> to learn more, and if you enjoy the app be sure to give us a review in the store!","tags":["Time Server App","Android"],"time":"Dec 12, 2018"},{"permalink":"keeping-company-pointed","timestamp":"2018-09-12T14:11:00-05:00","title":"A Tale of Two Meetings","excerpt":"A big part of keeping any company pointed in the right direction is making sure the leadership team is communicating regularly.","image_urls":[{"url":"images/blog-board-restaurant.jpg","alt":"The Board"}],"content":"<h4><a href='/people.html#terry-ott'>Terry D. Ott</a></h4>A big part of keeping any company pointed in the right direction is making sure the leadership team is communicating regularly.Even though the four members of our Board of Directors are distributed across four states and two time zones, we’ve made a point to meet face to face twice since the company was founded.<h4>Sunday, November 19, 2017</h4>The four PublicNTP Directors held our 2017 annual board meeting (as captured in our <a href='/documents/bylaws.pdf'>company bylaws</a>!) in Northern Virginia. With Dan and I both living in different suburbs of Washington, DC, having everyone gather near the nation’s capital was convenient.<div class='blog__img'><img src='/images/blog-ice-meeting.jpg' />Here’s a pic of the four directors after we broke the ice (terrible pun entirely intentional) at our post-meeting hockey game (L to R: <a href='/people.html#terry-ott'>Terry Ott</a>, <a href='/people.html#dan-noland'>Dan Noland</a>, <a href='/people.html#brad-woodfin'>J. Brad Woodfin</a>, <a href='/people.html#scott-waddell'>Scott Waddell</a>)</div>The 2017 meetup was our inaugural board meeting since having formed the company -- one where we recognized and performed the official steps as the board for a public charity.Once those formalities were addressed, <a href='/people.html#scott-waddell'>Scott</a>, <a href='/people.html#brad-woodfin'>Brad</a>, <a href='/people.html#dan-noland'>Dan</a>, and I discussed where PublicNTP was headed. We had just been granted 501(c)(3) public charity status by the IRS. With that all settled, the road had opened up to us, and the pieces were in place for PublicNTP to really take off.Instead of focusing on a global web of time servers in any location, we agreed that in the short time our company had existed, our mission statement had already shifted to deploying time servers to primarily underserved countries. With that narrowing of focus, we discussed our plan of attack. I suggested we to tackle a South America deployment first with Project Tonatiuh. <a href='/people.html#scott-waddell'>Scott Waddell</a> lobbied that Project Ikenga, the plan to deploy servers in Africa, was a more pressing priority.After an energetic discussion, the group agreed Ikenga should be a higher priority due to the greater shortage of existing network time infrastructure.The rest of our meeting discussed the details of our relationship with Network Time Foundation (NTF). NTF has been the go-to name in NTP software research and development for decades. A formal partnership with one of our key software vendors (their software was running on all our deployments at the time, and it’s still true almost a year later) would provide great traction out of the starting gates. The four of us agreed to pursue a public, formal partnership with Network Time Foundation. <a href='/blog/posts/publicntp-partners-network-time-foundation.html'>You can check out our article about it here.</a><h4>Saturday, June 15, 2018</h4>In just eight months since our last board meeting, PublicNTP had made huge progress and undertaken several new initatives. In the time between meetings, the company has been heavily focused on a mobile app (to be announced later in 2018), dramatic improvements to the company’s website, and the ongoing effort to build/foster our community and the network time community at large. PublicNTP is actively seeking partners, conferences, and other authors of network time software in order to build our foundation and professional network.<div class='blog__img'><img src='/images/blog-desk-meeting.jpg' /><a href='/people.html#scott-waddell'>Scott</a> leads the presentation on the state of the mobile app with <a href='/people.html#brad-woodfin'>Brad</a> as they discuss where it should go next.</div>The board also discussed the issue of how difficult it is to find good hosting locations for time servers, and how we might address that going forward.Project Tonatiuh and Ikenga have been progressing, but slowly. Due to issues finding reliable -- but affordable! -- hosting providers, it has been difficult to establish longer-lasting cloud deployments in South America and Africa. We’ve been looking into other options, including taking trips to deploy our own hardware in those locations.During both meetings to date, the board has discussed a cohesive approach to deploying servers in Asia, especially in China. In November 2017 we were excited to explore the options. By June 2018 we had become aware of significant challenges.Doing business within China’s borders, even deploying cloud resources, requires a formal partnership with a Chinese business and filing for a business license with the Chinese government. In light of this new information, we decided it would be more effective to find places near and around China for our servers. While we were disappointed that we couldn’t hit our original goal, we found many companies have made the same realization and taken the same approachSince achieving 503(c)(3) tax-exempt status, PublicNTP has been steadily chugging along. Since November of 2017, the company deployed multiple servers in South America (Mexico, Colombia, Argentina, and Chile) and Africa (Nigeria and South Africa). We came away with a clear vision and trajectory of the company until our next annual Board of Directors meeting in November 2018.We invite you to keep an eye on our progress, and perhaps join us along the way!","tags":["Time Server Deployments","Mission Statement"],"time":"Sep 12, 2018"},{"permalink":"publicntp-partners-network-time-foundation","timestamp":"2018-05-14T14:11:00-05:00","title":"PublicNTP partners with Network Time Foundation","excerpt":"Network Time Protocol (NTP) remains one of the oldest and most fundamental internet protocols in use. It has provided a bedrock of reliable time for us since 1985. NTP started out rough, as all innovations do","image_urls":[{"url":"images/blog-guardian-of-time.jpg","alt":"Guardian of time"}],"content":"<h4><a href='/people.html#terry-ott'>Terry D. Ott</a></h4>Network Time Protocol (NTP) remains one of the oldest and most fundamental internet protocols in use. It has provided a bedrock of reliable time for us since 1985. NTP started out rough, as all innovations do, providing us with accurate time within 100 milliseconds across the few hosts on the network. With faster computers, more reliable networks, and research resulting in four different versions of the NTP protocol, the accuracy has improved exponentially.In order to emphasize the integral role that NTP plays in PublicNTP’s vision and mission, we included it in our company’s name. In order to support PublicNTP’s global deployment of NTP servers, PublicNTP has established formal ties with the source of NTP software.PublicNTP is proud to officially announce we have been accepted as an <a href='https://www.nwtime.org/current-members-donors/'>Insitutional Member</a> of the <a href='https://www.nwtime.org/'>Network Time Foundation</a> -- the non-profit public charity responsible for maintaining the source for the NTP reference implementation.Network Time Foundation (NTF) has been an IRS-recognized public charity since 2014, but came into existence several decades before then. The NTF team, led by president Harlan Stenn, has worked closely with Professor <a href='https://en.wikipedia.org/wiki/David_L._Mills'>David L. Mills</a>, the original inventor of NTP, since 1990. NTF has ensured that Professor Mills’ pioneering work to enable global, synchronized time continues to improve and meet the evolving needs of today’s increasingly-digital society.Network Time Foundation’s passion enables PublicNTP to deliver on our mission. PublicNTP is proud to partner with Network Time Foundation as an Institutional Member, and we look forward to working more closely together as a result.Image above copyrighted by Network Time Foundation, used with permission.","tags":["Time Server Deployments","Mission Statement"],"time":"May 14, 2018"},{"permalink":"donating-time","timestamp":"2018-02-24T14:11:00-05:00","title":"Donating Time","excerpt":"PublicNTP receives federal public charity status","image_urls":[{"url":"images/blog-teamwork.jpg","alt":"Teamwork"}],"content":"<h4><a href='/people.html#terry-ott'>Terry D. Ott</a></h4>In 2008, I started PublicNTP as a personal hobby project. Back then, PublicNTP consisted of a small set of cloud-based Network Time Protocol (NTP) servers, all inside the United States, that with the help of hyper-accurate time provided by atomic clocks, provided time references for computers around the world.Starting in 2016, PublicNTP added its first NTP servers outside the United States. It quickly became clear that the need to deploy NTP servers in less-developed parts of the world was orders of magnitude more urgent than inside first-world countries such as the US. Much of the world faces desperately inadequate access to any local network-based time sources--much less high-quality sources.At the same time, it became evident that PublicNTP would need to grow bigger and faster than one person could hope to do on their own. I reached out to three decades-long friends and peers, <a href='/people.html#dan-noland'>Dan Noland</a>, <a href='/people.html#brad-woodfin'>Brad Woodfin</a>, and <a href='/people.html#scott-waddell'>Scott Waddell</a>, and invited them to help PublicNTP make the transitions needed to take on a much bigger challenge.On July 26, 2017, the Commonwealth of Virginia issued PublicNTP, Inc. its <a href='/documents/certificate_of_incorp.pdf'>certificate of incorporation</a> as a nonstock corporation, with Dan, Brad, Scott, and I serving as the first four directors and officers.In early August 2017, the Internal Revenue Service granted PublicNTP, Inc. <a href='/documents/acceptance_letter.pdf'>tax-exempt public charity status</a> (commonly known as a “501(c)(3)” nonprofit corporation).As with all IRS-recognized 501(c)(3) public charities, <a href='/donate.html'>charitable contributions</a> to PublicNTP (both cash and noncash) are now tax-deductible to the <a href='https://www.irs.gov/pub/irs-pdf/p526.pdf'>limits allowed by the IRS</a>. Consult your tax advisor regarding how charitable contributions to PublicNTP can benefit your personal financial situation.PublicNTP started in 2008 with a single time server in the midwest United States. As of this writing, PublicNTP has deployed servers across 22 cities, 15 countries, and six continents. The newly-gained ability to accept <a href='/donate.html'>charitable donations</a> empowers PublicNTP to deliver on its mission statement of improving unrestricted access to no-cost, highly-accurate time sources in those parts of the world with the most dire shortage of publicly-accessible NTP servers.More developed parts of the world often take a robust infrastructure, including access to highly-accurate time references, for granted. Access to a robust infrastructure reduces the difficulties of making a wide variety of breakthroughs, technological as well as social.<a href='/donate.html'>Charitable donations</a> will enable PublicNTP to continue its decades-long effort to gradually improve the global IT infrastructure. On behalf of PublicNTP, I hope you’ll consider supporting our efforts!","tags":["Time Server Deployments","Mission Statement"],"time":"Feb 24, 2018"},{"permalink":"paperwork-puzzle","timestamp":"2017-08-21T07:32:00-04:00","title":"PublicNTP's Paperwork Puzzle","excerpt":"Part 4: Our 15 step checklist","image_urls":[{"url":"images/blog-paperwork.jpg","alt":"paperwork"}],"content":"<section class='section__puzzle'><h4><a href='/people.html#terry-ott'>Terry D. Ott</a></h4><h4>Checklist</h4><ul class='table_contents'><li>Step 1: Hire a Law Firm That Specializes In Nonprofits (Optional)</li><li>Step 2: Determine the (Possibly Virtual) Company Address</li><li>Step 3: Select the Registered Agent</li><li>Step 4: Select Directors</li><li>Step 5: Select Officers</li><li>Step 6: Write Bylaws</li><li>Step 7: Write Policies/Procedures</li><li>Step 8: Write Articles of Incorporation</li><li>Step 9: Apply for State Incorporation</li><li>Step 10: Announce First Board of Directors Meeting</li><li>Step 11: Register for Charitable Solicitation with States</li><li>Step 12: Apply for IRS Employer Identification Number (EIN)</li><li>Step 13: Apply for Federal Tax Exempt Status (501(c)(3))</li><li>Step 14: Apply for State Tax Exempt Status</li><li>Step 15: Purchase Liability Insurance</li></ul><h4>Step 1: Hire a Law Firm That Specializes In Nonprofits (Optional)</h4>Considering neither Scott nor I had attempted to form a tax-exempt nonprofit corporation before, we retained the services of a legal firm specializing in nonprofit law to increase the odds of a smooth journey to successful incorporation and declaration of tax-exempt status for PublicNTP.Scott and I feel strongly that hiring a legal team was a vital investment for PublicNTP, based on our particular situation/set of circumstances.If you are reading this and wondering if your team should retain lawyers for the purpose of incorporating and applying for tax-exemption status, please do not ask our opinion. Exactly zero of the PublicNTP directors are lawyers; you will find us exceedingly reluctant to give any opinions on legal matters! :)Your group should discuss the tradeoffs involved with retaining a legal team based on your combined levels of experience, confidence, and risk-tolerance.The only opinion I will share on this topic is I personally feel any group considering undertaking this path would benefit from at least considering scheduling an “initial consultation” with a law firm.Initial consultations are a set, fixed price -- regardless of how much time they take. Your team may find you are all very glad you made the very modest investment required for an initial consultation with a lawyer!<h4>Step 2: Determine the (Possibly Virtual) Company Address</h4>Remember that the address of the corporation will be part of the public record. If you’re comfortable publishing your address the entire world on the internet, then by all means go for it.PublicNTP chose to use the services of Traveling Mailbox, a “virtual mail service”. Mail received at our virtual address is scanned and we receive an email about new mail. I can look at the image of the envelope, and click a button to indicate “shred,” “open and scan,” or forward to my actual address. Any checks that are received can be automatically deposited in the company’s checking account.<h4>Step 3: Select the Registered Agent</h4>While you can simply list the home address of one of your director or officers, remember that if they move, you may not receive notice of being served with legal paperwork -- like a lawsuit!Legal services, such a LegalZoom, offer Registered Agent service for a yearly fee.<h4>Step 4: Select Directors</h4>Determine who you want to serve on your organization’s board. These individuals will be able to wield a tremendous amount of influence on the vision/mission of your organization, so pick carefully!<h4>Step 5: Select Officers</h4>Decide who will serve as the President, Secretary, and Treasurer of your organization.It’s very common in very small nonprofits to have members of your Board of Directors serve two roles (for example: Director and Secretary).You may want to consider ensuring that no officers have undue influence on the group. Many organizations use their bylaws to specifically prevent individuals from assuming more than one officer position at any time, or prevent certain combinations of roles to held by one person (such as preventing someone from being both the president and treasurer).<h4>Step 6: Write Bylaws</h4>At this point, it’s a very good time to <a href='/documents/bylaws.pdf'>write your bylaws</a>. There are tons of examples on the internet. While this document is terribly dry to write, invest the time this step needs.Remember, the bylaws give you a chance to develop processes that are a good fit your organization, and set rules that help protect the organization and the indiduals that are involved with it.If you hire a legal team, I’d recommend running your bylaws past your legal team before proceeding past this step. Make sure that they agree all the contents are appropriate, and that no key items are missing.<h4>Step 7: Write Policies/Procedures</h4>You’ll find you want to procrastinate on these.Don’t.Take the time up front and write them now.No, they’re absolutely no fun. They’re probably worse than writing bylaws. Trust me, if you don’t write them now, you won’t write them until after your first painful learning lesson. Get them done, be glad they’re locked up and stored safely on a shelf.<h4>Step 8: Write Articles of Incorporation</h4>This is a very short document, usually less than two pages. Lots of templates exist on the web.Again, if you have hired a legal team, have them review before you file for incorporation.<h4>Step 9: Apply for State Incorporation</h4>This is an easy step and just requires you to provide your Articles of Incorporation, provide contact info, and pay a filing fee. In Virginia, you can <a href='https://sccefile.scc.virginia.gov/'>apply online</a> and have a response within one to two days.When approved, the state will send you a Certificate of Incorporation, which is the legal proof that your company is now a legal corporate entity within the state you filed in.<h4>Step 10: Announce First Board of Directors Meeting</h4>When the company’s Certificate of Incorporation is received from the state, adhering to the corporate bylaws, find a date/time (and, if meeting in person, place) that works for the majority of the board to have their first meeting, then properly post announcement of that meeting with enough notice.The announcement should include the following agenda items:<ul><li>Articles of Incorporation<ul><li>Adopt, ratify, and affirm the Articles of Incorporation filed with Secretary of State</li><li>Authorize and direct the Secretary to insert a certified copy of the Articles of Incorporation in the Corporation’s minute book</li></ul></li><li>Bylaws<ul><li>Adopt the bylaws</li><li>Authorize and direct the Secretary to insert a certificate of the adoption of the Bylaws, and to insert a certified copy of the Bylaws, into the minute book</li></ul></li><li>Officers<ul><li>Elect (named) people to the three officer positions: President, Secretary, and Treasurer</li></ul></li><li>Corporate records and minute book<ul><li>Authorize and direct the officers to procure all corporate books and books of account that may be required by the laws of the state in connection with the business of the corporation</li><li>Authorize and direct the officers to maintain a minute book containing:<ul><li>Certified Articles of Incorporation</li><li>Bylaws and any amendments</li><li>Minutes of any and all meetings</li><li>All actions of the Board and Board committees</li></ul></li></ul></li><li>Annual Accounting Period<ul><li>Approve the date the fiscal year of the corporation ends</li></ul></li><li>Principal Executive Office<ul><li>Confirm the address of the company</li></ul></li><li>Open bank accounts for the company, apply for debit card</li> </ul><h4>Step 11: Register for Charitable Solicitation with States</h4>It’s required by law to register with each state that your charity solicits donations in.<h4>Step 12: Apply for IRS Employer Identification Number</h4>Once your Certificate for Incorporation is issued, the next step is to apply with the IRS for an <a href='https://www.irs.gov/businesses/small-businesses-self-employed/apply-for-an-employer-identification-number-ein-online'>Employer Identification Number</a> (EIN) for the company. This should be done by an officer of the company. The applicant will need to provide their own personal Social Security Number (SSN) during the application.You’ll receive an instant response, with the ability to download/print the paperwork with your company’s newly-assigned EIN.<h4>Step 13: Apply for Federal 501(c)(3) Tax-Exempt Status</h4>Now that you’ve formed your state nonprofit, it’s time to apply for your 501(c)(3) tax exempt status.Small nonprofits <a href='https://www.irs.gov/uac/newsroom/new-1023-ez-form-makes-applying-for-501c3tax-exempt-status-easier-most-charities-qualify'>may be eligible</a> to file the much shorter/easier three-page <a href='https://www.irs.gov/pub/irs-pdf/f1023ez.pdf'>1023-EZ</a> form as opposed to the full <a href='https://www.irs.gov/pub/irs-pdf/f1023.pdf'>26-page 1023 form</a>.The PublicNTP legal team was able to start, finish, and submit our 1023-EZ form -- all in under two hours of billed time. <a href='http://www.journalofaccountancy.com/issues/2016/mar/irs-form-1023-ez.html'>A March 2016 article in the Journal of Accountancy</a> indicates that processing times for the 1023-EZ form (13 days) are much faster than the 1023 (181 days), and 77% of the applications were approved for 501(c)(3) tax exemption status.PublicNTP’s legal team submitted our online 1023-EZ application on August 1, 2017. We received our 501(c)(3) acceptance letter with a date stamp of August 8, 2017.It only took the IRS five working days to approve our application (!!!).<h4>Step 14: Apply for State Tax Exemptions</h4>After you receive your 501(c)(3) acceptance letter for federal tax exemption, check to find out if you need to file any forms to receive exemption at the state level as well.State Income TaxAfter you receive your 501(c)(3) acceptance letter, check to find out if you need to file at the state level as well.Some states (such as Virginia) automatically grant state income tax exemptions to organizations who have received IRS 501(c)(3) status without needing any paperwork to be filed.Other states, however, do require paperwork to be filed which contain a copy of the 501(c)(3) letter before they grant state-level income tax exemption to the company.State Sales/Use TaxVirginia does not automatically grant 501(c)(3) organizations an exemption from state sales and use taxes. Companies are required to file paperwork with the state including copies of their most recent 990/990-EZ, articles of incorporation, etc. before being granted exemption from state sales/use tax.Make sure you do all the homework necessary to ensure your corporation is compliant! Your legal team, if you have one, can be a huge help here.<h4>Step 15: Purchase Liability Insurance</h4>A lawsuit can really ruin your whole day -- not to mention bankrupt your nonprofit.Each corporation’s board should do a cost/risk/benefit analysis to find out if any forms of liability insurance are a good fit for their needs.While (yet again) it’s likely overkill, after consulting our legal team and doing our own research into best practices, PublicNTP decided to purchase the following insurance policies at the listed coverage levels:<ul><li><a href='https://en.wikipedia.org/wiki/Commercial_general_liability_insurance'>General liability</a>: $1 million</li><li><a href='https://en.wikipedia.org/wiki/Directors_and_officers_liability_insurance'>Directors & Officers liability</a>: $1 million <ul><li>Ensures that volunteers serve as an officer or director have a legal shield protecting their personal assets </li></ul></li></ul>Some steps are harder than others but those are the fourteen steps we needed to take. And, in fact, it was a process that I thought would take perhaps a year of time and planning panned out to only be a few months of work.Now that we’re here, PublicNTP is ready to begin taking donations and deploying free, accurate digital clocks across the world.<div class='blog__nav'><a href='/blog/posts/financial-structure.html' class='blog__nav-btn'><img src='../../images/left-arrow.svg' />Financial Structure</a><div class='blog__nav-spacer'></div>","tags":["Time Server Deployments","Mission Statement","Stratum 1","Stratum 2","OCONUS (Non-US) Deployments"],"time":"Aug 21, 2017"},{"permalink":"financial-structure","timestamp":"2017-08-21T07:32:00-04:00","title":"PublicNTP’s Financial Structure","excerpt":"Part 3: When considering the public charity path, make sure you know how you’re reconciling your finances and how often","image_urls":[{"url":"images/blog-financial.jpg","alt":"Finances"}],"content":"<h4><a href='/people.html#terry-ott'>Terry D. Ott</a></h4>As mentioned in our source article, creating spotless financial records early on -- plus supporting documentation of the financial policies -- is crucial when running a tax-exempt corporation.The company is subject to audits by state or federal organizations at any point. After all, a company that’s considered a 501(c)(3) can all too easily become a fraudulent business without close monitoring.When considering the public charity path, make sure you know how you’re reconciling your finances and how often. Being a time-centric company, it should come as no surprise that PublicNTP reviews our records quite frequently!<h4>Day-To-Day Books</h4>Managing the financial books for a small nonprofit is not difficult, but it demands a lot of learning, careful attention, and constant diligence.Your treasurer can certainly learn how to do the task well by doing some brief research on the internet, reading a book or two, and by using the right tools.PublicNTP decided to outsource its bookkeeping to an accounting firm that specializes in nonprofit organizations and is staffed entirely by Certified Public Accountants (CPA’s).In addition to shifting this task to another team, our board of directors can be confident in the knowledge that our books are being worked by CPA’s, specialists in the field who are required to complete many hours of continuing education per year in order to keep up on the latest developments in the field.It’s worth the cost to know our financials are being handled by a trained, licensed professional in accordance with Generally Accepted Accounting Practices (GAAP) at every opportunity.<h4>Monthly Reconciliation and Statements</h4>At the end of each month, do a reconciliation run across the financials. Make sure that all the columns that should add up and be equal still are!Forgetting to do this over 3-4 months can make balancing the books a big challenge. Put a monthly reminder in your phone, and force yourself to take the hour to square everything up. In a couple months, it could easily end up taking a full day.A brief monthly financial statement should be created at the end of the reconciliation that should be entered by the Secretary into the organization’s formal recordkeepingsystem.<h4>Quarterly Financial Reports</h4>The bookkeeper should generate quarterly financial reports that are delivered to all members of the Board of Directors.These reports summarize income, expenses, etc. These are incredibly valuable in the event of an audit of your group’s finances. Showing that a detailed summary was generated every three months is great evidence to present during an audits. It serves as proof that the books were in order and the directors were doing all necessary due diligence in keeping current with the financial health of the organization.<h4>Yearly Financial Reports</h4>The yearly report should be generated from the quarterly reports, and formally presented at the Board of Directors meeting. By insisting on monthly and quarterly check-ups, this monumental annual task is reduced to a swift compilation of the existing monthly/quarterly reports.<h4>Board Meetings</h4>If you decide to outsource your company’s bookkeeping, require that the accountant attends (even if virtually) the annual Board of Directors meeting, so they can provide assistance to the Treasurer when financial matters are being discussed. This will remove any risk of communication problems about the initial recordings and any interpretations the directors may have.<h4>Consider Annual, Independent Financial Statement Reviews or Audits</h4>For a small nonprofit, this may well be overkill. That said, the PublicNTP leadership are big fans of overkill when it comes to financial transparency.Retaining an independent CPA to either do a full-blown audit, or a more limited “financial statement review,” is a way for small nonprofits to help give confidence to donors that the organization will handle their money properly.If your treasurer is keeping your books, hire a CPA to perform this task. If you are outsourcing your bookkeeping to an accounting firm, hire a CPA with an accounting firm completely independent of the accounting firm handling your books to generate this report.It’s a lot to process. But at its core, this financial organization simply demands diligence. You may decide on a different structure but it’s important to ensure that your process is comprehensive and accurate, whatever it is.Now that you’ve determined how finances are going to be organized for your tax-exempt company, you can progress to the last stage of this application: <a href='/blog/posts/paperwork-puzzle.html'>the stacks of paperwork that need filling out.</a><div class='blog__nav'><a href='/blog/posts/organizational-structure.html' class='blog__nav-btn'><img src='../../images/left-arrow.svg' />Organizational Structure</a><div class='blog__nav-spacer'></div><a href='/blog/posts/paperwork-puzzle.html' class='blog__nav-btn'> Paperwork Puzzle <img src='../../images/right-arrow.svg' /></a></div>","tags":["Time Server Deployments","Mission Statement","Stratum 1","Stratum 2","OCONUS (Non-US) Deployments"],"time":"Aug 21, 2017"},{"permalink":"organizational-structure","timestamp":"2017-08-21T07:32:00-04:00","title":"PublicNTP’s Organizational Structure","excerpt":"Part 2: We needed to establish some structure to the corporation","image_urls":[{"url":"images/blog-organizational.jpg","alt":"organization"}],"content":"<h4><a href='/people.html#terry-ott'>Terry D. Ott</a></h4>PublicNTP’s mission is to provide free, global, digital clocks. In order to become tax-exempt, avoid legal issues, and be a legitimate corporation, we needed to establish some structure to the corporation.For those considering creating your own tax-exempt corporation, all states require the application to include a description of some key formal positions that will need to be filled.This post discusses the PublicNTP organization structure along with descriptions.What worked for us is certainly not universal; our legal team shared their professional opinion about what the IRS “likes” to see with us as we were incorporating.Feel free to use our structure an example, but definitely go with what works for your team and company needs.Note: providing legal advice is certainly not something in the PublicNTP mission! :) If in doubt, consult legal advisors that you trust.<h2>Directors</h2>The Board of Directors is a nonprofit corporation’s governing body.Individuals serving on the Board of Directors (properly referred to as “directors”) are responsible for overseeing all the corporation’s activities.Directors ensure the corporation stays focused on its mission, strategy, and goals.The advice we received from the PublicNTP legal team was to have at least three directors on its board. We were instructed that less than three directors could potentially complicate our application for tax-exempt status with the Internal Revenue Service (IRS).<h2>Officers</h2>Officers are individuals who are selected by the Board of Directors and tasked to execute the Board’s vision and mission for the corporation.The PublicNTP legal team advised us to create the following three officer positions at an absolute minimum: <ol style='padding-left: 58px;'><li>President</li><li>Secretary</li><li>Treasurer</li></ol><h4 class='italic'>President</h4>The president is responsible for the day-to- day operation of the corporation, ensuring all other officers are fulfilling their duties.Short- and long-term planning also fall under the president’s stewardship, as he or she directs programs to further the organization’s mission.The president also oversees fundraising operations, including acting as the primary spokesperson for the organization, recruiting donors, and attending fundraising functions.He or she has the authority to sign all legal documents on behalf of the corporation.<h4 class='italic'>Secretary</h4>The secretary is responsible for maintaining all the corporation’s records, scheduling/giving proper notice for any meetings, and distributing agendas/meeting minutes.The secretary also maintains accurate documentation and enforces legal requirements, such as periodic registrations for the company (state corporation registration, charitable donation registrations with states, etc.).<h4 class='italic'>Treasurer</h4>The treasurer oversees the management and reporting of the corporation’s finances.This generally includes: <ul style='list-style-type: disc;'><li>Selecting a bank for the company’s accounts</li><li>Maintaining the company’s financial books (either personally or outsourcing</li><li>Managing cash flows</li><li>Ensuring vendors are paid promptly</li><li>Developing and tracking budgets</li><li>Delivering quarterly financial reports to the Board of Directors</li><li>Ensuring all financial filings for the corporation are filed properly and on-time (example: IRS form 990-EZ)</li></ul>Thanks to our legal team and some deliberation, PublicNTP put together a small, agile team of directors and officers to meet our particular needs.<div class='blog__nav'><a href='/blog/posts/dizzying-terminology.html' class='blog__nav-btn'><img src='../../images/left-arrow.svg' />Dizzying Terminology</a><div class='blog__nav-spacer'></div><a href='/blog/posts/financial-structure.html' class='blog__nav-btn'>Financial Structure <img src='../../images/right-arrow.svg' /></a></div>","tags":["Time Server Deployments","Mission Statement","Stratum 1","Stratum 2","OCONUS (Non-US) Deployments"],"time":"Aug 21, 2017"},{"permalink":"dizzying-terminology","timestamp":"2017-08-21T07:32:00-04:00","title":"PublicNTP’s Dizzying Terminology","excerpt":"Part 1: The first steps to becoming tax-exempt","image_urls":[{"url":"images/blog-terminology.jpg","alt":"terminology"}],"content":"<h4><a href='/people.html#terry-ott'>Terry D. Ott</a></h4>The first steps to becoming tax-exempt required we know a series of new terms that were interpreted specifically by the IRS. It was important to make sure we knew the specifics of these terms because, at first glance, some of them seem similar enough to be interchangeable. As the IRS doesn’t play in gray areas, we couldn’t either.<h4>Employer Identification Number (EIN)</h4>This one’s pretty straightforward. Any business, from individual startups to large corporations, needs one of these. It’s used to identify the company for tax-purposes. For our tax-exempt company, it would also be given to our donors for their own records when claiming the charity on their taxes. It takes about five minutes to apply for online.<h4>Non-stock Corporation</h4>A non-stock corporation is a corporation which does not issue any shares of stock. What that meant for our organization purposes was that non-stock corporations have no owners.A nonprofit is, by definition, a non-stock corporation.<h4>Non-profit/Not-for-profit Corporation</h4>There’s no gray area between these two terms; they are synonymous.Nonprofit corporations cannot distribute dividends or income to their directors, officers, or individual members, except as compensation for services rendered, or upon dissolution of the corporation.However, this doesn’t mean nonprofits can’t be profitable—they absolutely can be, and often are! The critical difference is all profits must be reinvested into the company’s operations. Individuals cannot financially benefit from profits generated by a nonprofit, only the company itself is permitted to do so.<h4>Nonprofit versus Tax-exempt</h4>All corporations exempt from paying taxes, either at the state or federal level, are non-profit corporations. However, not all nonprofit corporations are tax-exempt.<h4>Tax-exempt Status</h4>Nonprofits, depending on the purpose for which they are organized, may apply to the Internal Revenue Service and/or the state in which they incorporated for exemption from some taxes.Our target exemption status would shield PublicNTP from having to pay income tax, as well as make all donations to the corporation a legal income tax deduction for donors. Most charities calling themselves “tax-exempt” or “tax-deductible” are referring to this type of exemption status, meaning the full dollar amount of all donations can legally be deducted from donors’ income tax filings. Both individuals and for-profit corporations can legally donate to a tax-exempt corporation and benefit from the resulting income tax deductions.<h4>Registered Agent</h4>The registered agent is the point of contact for the corporation when being served when the company is being served with legal papers (such as a lawsuit).There was a bit of learning to do but once we had this vocabulary list under our belt we could move on to the stickier subjects of organizing ourselves both structurally and financially.<div class='blog__nav'><a href='/blog/posts/pntp-introduction.html' class='blog__nav-btn'><img src='../../images/left-arrow.svg' />Intro</a><div class='blog__nav-spacer'></div><a href='/blog/posts/organizational-structure.html' class='blog__nav-btn'>Organizational Structure <img src='../../images/right-arrow.svg' /></a></div>","tags":["Time Server Deployments","Mission Statement","Stratum 1","Stratum 2","OCONUS (Non-US) Deployments"],"time":"Aug 21, 2017"},{"permalink":"pntp-introduction","timestamp":"2017-08-21T07:32:00-04:00","title":"PublicNTP’s Journey to Tax-Exempt Status","excerpt":"The first steps to becoming tax-exempt","image_urls":[{"url":"images/blog-checklist.jpg","alt":"Checklist"}],"content":"<h4><a href='/people.html#terry-ott'>Terry D. Ott</a></h4>It has always been about time for me. From my nascent teenager years to booting up my first cloud server under PublicNTP. I am proud to admit to being a time dork and I’ve always been fascinated how humanity has refined our time-measuring tools. As shown on the <a href='/history.html'>timeline portion of the site</a>, you can see how we’ve gotten to the point where we can keep track of time down to nine billionths of a second.Distribution is PublicNTP’s focus in the precision time space -- providing global access to any and all who need it, free of charge.My long-time friend <a href='/people.html#scott-waddell'>Scott Waddell</a> and I decided in the summer of 2017 that it was time to get serious about this. Previously, PublicNTP had been a personal hobby, funded entirely by me. In other words, there wasn’t much “public” in PublicNTP. In the summer of 2017, Scott and I agreed that it was crucial for the long-term sustainability of PublicNTP to become a non-profit public charity, funded by donations from people around the globe.Both Scott and I had created companies before, but neither had ever created a nonprofit corporation or filed for tax exemption. We both enjoy good challenges that involve intense learning, so we decided to dive in and see what was involved!As is often the case, it turned out some things were simpler than expected...and others decidedly weren’t.This post started out in my head as a one-stop “How To Become Tax-exempt” article for other small organizations who might be considering public charity status. As I dove in and tried to capture the salient bits, I realized there was way more content than should be in a single blog post. We decided to break the content up over a series of posts, with this initial one serving as something of a “table of contents”.We spent a lot of time and effort in the early days learning the proper use of the “language” of the knowledge domain. Ranging from basic business terms like EINs to the granular legal details of what makes a company non-profit.The next major set of learning involved figuring out how to transition a sideline hobby into a proper corporation which would be taken seriously and recognized as a public charity with Directors and Officers. Making sure PublicNTP had the right people structured in the proper/correct way, and accurately capturing all of that in the paperwork proved to be a significant hurdle.We then had to decide on -- then properly document -- all the processes and best practices for publicly-supported charities. Applications for tax deductible status are heavily scrutinized by the IRS (as they absolutely should be!). It’s important to make sure from the very beginning that every financial transaction is properly recorded. This isn’t strictly required for the IRS application, but we decided that it was best to start out of the gates with spotless books, knowing that the IRS has the legal right to audit at any point for any reason. We decided to put the effort in early to ensure absolutely everything about our financial records was above-board.After we had deciphered key vocabulary terms, agreed on the company’s leadership structure, and organized our financial systems to properly record/report all financial transactions and expenditures we were (finally!) ready to pull the trigger.Our non-profit application required us to meet a number of very strict legal/financial requirements. This culminated in a paperwork puzzle that might seem insurmountable to many small, volunteer-supported organizations.PublicNTP hopes this series of articles can help illuminate how we approached the tax-exempt application process in order to show others that there is a repeatable set of steps, none of which being terribly complicated, that will likely result in the IRS approving your application as well.- Part 1: <a href='/blog/posts/dizzying-terminology.html'>PublicNTP's Dizzying Terminology</a>- Part 2: <a href='/blog/posts/organizational-structure.html'>PublicNTP's Organizational Structure</a>- Part 3: <a href='/blog/posts/financial-structure.html'>PublicNTP's Financial Structure</a>- Part 4: <a href='/blog/posts/paperwork-puzzle.html'>PublicNTP's Paperwork Puzzle</a>","tags":["Time Server Deployments","Mission Statement","Stratum 1","Stratum 2","OCONUS (Non-US) Deployments"],"time":"Aug 21, 2017"},{"permalink":"global-expansion-and-a-facelift","timestamp":"2017-07-08T17:36:00-04:00","title":"Global expansion and a facelift","excerpt":"When I first launched PublicNTP with a single NTP server in Chicago, Illinois, I had grand plans for rapid expansion, first across the US, to be quickly followed by exotic locations around the globe I’d always wanted to visit.","image_urls":[{"url":"images/blog-servers.jpg","alt":"servers"}],"content":"<h4><a href='/people.html#terry-ott'>Terry D. Ott</a></h4>When I first launched PublicNTP with a single NTP server in Chicago, Illinois, I had grand plans for rapid expansion, first across the US, to be quickly followed by exotic locations around the globe I’d always wanted to visit.As is often the case, reality ended up diverging quite a bit from the original dreams. :slightly_smiling_face: From the launch in 2008 through the summer of 2016, PublicNTP was largely in stasis. The server in Chicago was quite reliable/stable over those eight years, but the Chicago deployment was all there was to PublicNTP.While my original deployment vision centered around purchasing and installing physical servers in leased data center space, in 2012 the Chicago server was transitioned from a physical server to a cloud-based virtual machine. I was pleasantly surprised to see a virtualized environment was entirely capable of providing high-quality stratum 2 time data to clients.Many NTP information sources strongly discourage running NTPd on anything but bare metal hardware. I tend to agree that stratum 1 servers should be running directly on a physical server if possible, but I personally found virtualized NTP server deployments to be entirely well-suited for servicing stratum 2 time needs. As a nice bonus, virtualized deployments were vastly easier to maintain—not to mention significantly cheaper!In the summer of 2015, I was sitting at my desk at work looking at the AWS Global Infrastructure map, doing some investigation for my day job. While looking at the map, it hit me that the global nature of AWS data centers could provide the foundation for the first major expansion for PublicNTP.A brief investigation demonstrated that, much like I had found with the virtualized server in Chicago, (properly-sized!) EC2 instances proved to be an ideal hosting environment for PublicNTP servers.Starting in late July 2016, thanks to Amazon’s EC2 offering, PublicNTP experienced tremendous deployment expansion for the first time in its eight-year history! In just six months, PublicNTP grew from the single deployment in Chicago to 11 deployments across 5 continents:<ul><li>North America (4)<li><ul><li>The Dalles, Oregon, USA</li><li>San Jose, California, USA</li><li>Chicago, Illinois, USA</li><li>Ashburn, Virginia, USA</li></ul><li>South America (1)</li><ul><li>São Paulo, Brazil</li></ul><li>Europe (2)</li><ul><li>Dublin, Ireland</li><li>Frankfurt, Germany</li></ul><li>Asia (3)</li><ul><li>Mumbai, India</li><li>Singapore</li><li>Tokyo, Japan</li></ul><li>Australia/Oceania (1)</li><ul><li>Sydney, Australia</li></ul></ul>Starting in the second half of 2016, my long-time friend <a href='/people.html#scott-waddell'>Scott Waddell</a> and I found ourselves in the same city on a regular basis for the first time in almost twenty years. As we caught up on each others’ lives and interests, I mentioned my work with PublicNTP. Scott immediately grasped the PublicNTP mission and offered to get involved.With Scott’s support, we deployed the first-ever PublicNTP stratum 1 time server, a CDMA-based Endrun Technologies Tempus LX server, in Salt Lake City, Utah, USA. Scott also provided access to an excellent technical team, including a graphic designer and a web developer who jointly produced the massive overhaul to the PublicNTP website that you’re currently looking at!Stay tuned for more blog posts across the remainder of 2017, which will largely focus on the shared vision Scott and I have for the short- and medium-term evolution/expansion of PublicNTP!","tags":["Time Server Deployments","Mission Statement","Stratum 1","Stratum 2","OCONUS (Non-US) Deployments"],"time":"Jul 8, 2017"}]

The PublicNTP Blog

Global NTP Server Monitoring

The “Why” Question, the Network Time Community, and You!

PublicNTP’s Fleet Servers and Maintenance

PublicNTP JAM

Groundbreaking: Africa and the Middle East!

PublicNTP July Board Meeting

GPS Generation Three Has Launched!

F2F in Mountain Time!

Touching Bases With The Team (African Expansion and More!)

PNTP Open Architecture Stratum One Part 5/5: Future Investigations

PublicNTP Open Architecture Stratum One Part 4: Step By Step Blueprint

PNTP Open Architecture Stratum One Part 3: Technical Overview

PublicNTP Open Architecture Stratum One Server Part 2: Rationale

Open Stratum One - The Meinberg Solution

PublicNTP’s Stratum One Deployment Checklist

What Goes Into Deployment?

Project Ikenga - Connecting Africa

2018 PublicNTP Board Meeting - Upcoming Year

Year in review

The Time Server App Has Launched!

A Tale of Two Meetings

PublicNTP partners with Network Time Foundation

Donating Time

PublicNTP's Paperwork Puzzle

PublicNTP’s Financial Structure

PublicNTP’s Organizational Structure

PublicNTP’s Dizzying Terminology

PublicNTP’s Journey to Tax-Exempt Status

Global expansion and a facelift

PublicNTP Consent Manager