2021 has been quite a year for cryptocurrency and this morning was no exception when the value of digital coins like Bitcoin, Dogecoin and Ethereum plunged. Users flocked to Downdetector® to report problems with multiple cryptocurrency trading platforms including: Binance, Coinbase, Fidelity Investments and Revolut. Coinbase experienced the greatest number of problem reports from across the U.S. and Europe which began shortly after 5:45 a.m. PDT. Issues were reported all across the United States and in several European countries including Austria, Czechia, France, Germany, the Netherlands, Portugal, Slovakia, Spain, Switzerland and the United Kingdom. The most frequently reported issue was the inability to trade or sell coins due to Coinbase’s app and website being down. For many crypto traders trying to get out before a crash or buy during a dip, an outage like this has potentially huge ramifications.

While the spike has passed and a tweet by @coinbasesupport suggests users should no longer be experiencing issues, users were still reporting various problems on Downdetector.com as of 10:00 am PDT.

Downdetector Enterprise can help your company reduce downtime and improve your mean time to resolution. With near-real-time alerting and intelligence on customer-reported issues you’ll immediately know when your users have a problem. Contact us here to learn how your network operations, devops, engineering and customer care teams can get faster outage detection.

One of the largest content delivery network (CDN) providers, Akamai, had a service disruption this morning which affected many popular services. According to data from Downdetector®, the CDN giant started to experience elevated problem reports around 11:40 a.m. EDT. Shortly after, users flocked to Downdetector to report issues with over 48 services, including: Airbnb, Discover, Expedia, Home Depot, Salesforce and many more. Affected users were mostly within the United States and most people reported difficulty accessing the services of various companies.

As of 12:32 p.m. EDT, Akamai tweeted that they were aware of the issue and would provide an update later this afternoon. While it seems like the spike in problem reports has passed for most affected companies, users were still left frustrated when they couldn’t access their favorite services.

Downdetector Enterprise^TM can help your company monitor third-party services your business relies on, such as CDN providers, so you know immediately when they go down. Using Downdetector’s real-time alerts you can quickly adjust communications to better serve your customers. Contact us here to learn how your network operations, DevOps, engineering and customer care teams can get faster outage detection.

Cloud computing giant Google Cloud Platform, experienced issues earlier today causing users across the world to experience problems when trying to access their favorite online services. According to data from Downdetector®, Google Cloud Platform started to experience elevated problem reports around 12:42 p.m. EST. Shortly after, users flocked to Downdetector to report issues with countless services, including: Etsy, Discord, Nest, Spotify, Target, The Home Depot and more.

As of 1:21 p.m. EST, GCP Incidents tweeted that the issue was partially resolved but did not have an ETA on when there would be a full resolution. While it seems like the spike in problem reports has passed for most affected companies, users were still left frustrated when they couldn’t access their favorite services.

Downdetector Enterprise^TM can help your company monitor third-party services your business relies on, such as cloud computing services, so you know immediately when they go down. Using Downdetector’s real-time alerts you can quickly adjust communications to better serve your customers. Contact us here to learn how your network operations, DevOps, engineering and customer care teams can get faster outage detection.

Every internet user has encountered a website being down, an app not working, or even a browser failing to connect to the internet. Every day, thousands of users like you use Downdetector® to report real issues for websites and apps, but the outage you’re experiencing is often more complicated than it seems. We’re here to identify the major points of failure for the internet so you can understand what is happening and how long you might expect services to be down. Read on to find out what’s really going on during these incidents and how one service can cause critical failures across the internet.

What a large outage or incident looks like

Over the past few years, the internet has been hobbled by major outages. Some of these incidents lasted for hours (and even days) and prevented people from accessing online services like websites, streaming video platforms, social media, mobile apps, ATMs, emergency services, and more. 

On the best of days, you can simply log onto the internet with the ease of clicking open a browser. However, the internet is a complex web of interconnected parts that exist on physical servers around the world. Often, when one part of the internet has an issue, it cascades to other services and companies that depend on that particular component. For example, if your internet service provider (ISP) blinks out, you might think that the site you were just visiting is down, while others using a different ISP can use the site just fine. Similarly, when you can’t access a website or app you want to use, the root cause might not be the app or website, like when an AWS incident took out Netflix in April 2021.

These five potential points of failure can take down the internet and online services

Internet service providers (ISPs) and mobile operators connect you to the web

The simplest internet outage to understand occurs when an ISP or mobile operator goes down or can’t be accessed. This can happen for many reasons, including a severed fiber line or a power outage taking out a local cell tower. When an ISP or mobile network suffers an outage, the internet still functions correctly, but your computer or device won’t be able to access it. This is frustrating and also makes it complicated to identify what the problem even is. 

One way to figure out if your ISP or mobile provider is down is to try to access the internet from a different network (if you’re on mobile, try your computer; if you’re on your computer, try your mobile network). If the site is accessible on the other device, you may have a carrier or ISP outage. Check with your ISP on Downdetector or social media to see if they are aware of the problem, then know they are hard at work on fixing it. Usually these types of outages are resolved within a couple of hours. 

Fixed Broadband ISPs

ISPs can go down due to hardware failures, severed cable lines, natural disasters, human error, or even network cyberattacks. These are often contained to small local areas, but some ISP incidents have been nationwide.

Consumers and businesses both rely on ISPs to offer and access internet-based services. When a specific ISP does go down, cities, banks, businesses, online gaming, point-of-sale devices, and more that use single ISPs can all face outages because those services may depend on specific ISP access to the internet. But that doesn’t mean everything goes down in your area if one ISP goes down in an area; if other ISPs are operational, they’ll be able to connect to the internet. This is why some businesses use multiple ISPs to insure against outages.

If you think your ISP may be down, check Downdetector on your phone for real-time incident reports by users in your area. For those who are comfortable with running terminal commands, you can run `mtr -rn 8.8.8.8` to see which hop within your connection may be having issues. 

Mobile operators

You’ve probably seen a cell tower in your neighborhood. Mobile networks connect your device to the internet using radio waves collected by those towers and other infrastructure. When a mobile network goes down, either the equipment sending or receiving radio waves is having an issue or the network that equipment connects to is having an issue. This may be contained as a local issue with a single mobile deployment, but mobile networks aren’t immune from larger incidents. Though more rare, mobile outages can be caused by natural disasters, faulty equipment, or human error. 

For you, a mobile outage means you may not be able to text or call from your mobile network until the problem is resolved. If you think your mobile operator may be down, use your computer to check Downdetector for real-time incident reports by users in your area.

Content delivery networks (CDNs) deliver critical content

CDNs allow content (like images, videos, and other critical parts of websites) to be stored physically closer to you using networks of point of presence servers (POPs) around the world. Websites and apps use CDNs to dramatically speed up and stabilize your experience, because the information has to travel less distance and bypass potential bottlenecks on the internet. In addition, many ISPs and mobile operators have CDN POPs within their own networks, further reducing the distance a request has to travel to be fulfilled. 

CDNs are a vital part of modern internet infrastructure and are particularly helpful when a service needs to respond quickly to increased demand. Examples include a surge in online shopping on Cyber Monday or a rush of viewers streaming the next episode of that most popular TV show. Still, CDNs can experience incidents that impact their reliability. Since these are consolidation points for internet traffic, CDN outages can have huge effects on a wide range of internet services all at once. When CDNs go down, sites aren’t able to be served and apps aren’t able to receive the content stored on these networks. These failures make the sites and services appear to be down. CDN failures can stem from coding bugs or a cyberattack. CDN issues are often difficult to diagnose since they operate behind the scenes and cover specific geographies. This means a CDN network can work in one part of the world, while other parts experience a blanket outage for all the businesses that depend on it. 

One indicator that you might be dealing with a CDN outage is if a group of sites and services that rely on one CDN provider are all showing issues on Downdetector in your area, as you can see on the image above related to the Akamai service outage of June 2021. We will often flag CDN outages on Downdetector with a banner to show how trending incidents may be connected.

Domain Name System (DNS) and Border Gateway Protocol (BGP) direct internet traffic

DNS and BGP are two systems that serve as the foundation for the global internet. You could say that DNS defines the addresses to which packets of information need to navigate and BGP effectively defines the roads of the internet. Without directions from DNS and BGP, the internet would not function. DNS and BGP issues can be contained or they can cause global internet outages.

Domain Name System (DNS)

DNS provides the connection between human names and network addresses — every website has a domain name (like speedtest.net), which DNS translates to a string of numbers that forms the IP address for every site you visit. Without DNS, your browser won’t be able to determine where to send a request to, like attempting to drive to a business without being able to find their address.

DNS incidents can either be caused by a bad DNS record which sends traffic to the wrong location or if a large DNS provider has issues responding to requests. Because DNS servers are so important, there are many redundancies to prevent global failure — but they can go down due to cyber attacks, power incidents, or human error. 

If the DNS issue is on your equipment, it can sometimes be resolved by restarting your router and/or modem, clearing your device’s DNS cache, or even switching browsers. Larger DNS issues can be identified by querying a server (if you feel confident doing so), and you can also go to Downdetector to see if it’s a larger issue impacting many people. Visitors to Downdetector will often leave a comment if they’ve run a query and found a DNS issue.

Border Gateway Protocol (BGP)

BGP is the routing system for the internet, receiving routing information from other networks on the internet and using Autonomous Systems (AS) to find the fastest and most efficient path to connect your device to the destination network. BGP isn’t centralized or controlled by any party, it’s up to internet providers to provide accurate and trustworthy information to other peer networks. BGP-related outages often are due to bad routing information being shared across the internet, which causes data to be sent to incorrect networks. 

BGP issues can be harder to identify, because when BGP goes down, the route to a particular network doesn’t appear to exist even if it’s there — and information can’t automatically update because everything is decentralized. Visit Downdetector to troubleshoot your outage. You can learn more about whether the incident is caused by a BGP issue by checking the comments. Because of the decentralized nature of these outages, the issue may take a while to resolve.  Be prepared to maybe take a walk or find something to do until the engineers work it all out.

Services and their Application Programming Interfaces (API) direct programs communicating

The final type of widespread incident types on our list involves service failures. When services go down, they may impact other applications that communicate and rely upon them through an API. An API is an interface that enables two or more programs to communicate together. For example, a weather app on your mobile device may fetch forecast info from a weather station’s API. If that API is down, your weather app may display a blank content box or not work at all. 

APIs include everything from applications like Google Maps interfacing with GPS data to get you to a destination, posting a photo to Instagram or Twitter with the your device’s camera app, sending a GIF from Giphy on your messaging app, completing a transaction on a commerce site with a payment app like PayPal, and security protocol programs like Auth0 or OneLogin that provide single sign-on services for additional security to your logins.

With many companies increasingly relying on external services for payment services (Square and Stripe), security measures (OneLogin and Google Safe Browsing), programming queries (Javascript and Python), storage (Dropbox and Google Drive), and many other different service APIs like Spotify, incidents at these types of services can have huge impacts across the internet.

Diagnosing a problem caused by a service or their API isn’t straightforward. Often, service issues stem from a coding error that occurs that doesn’t allow the program to communicate information to the service’s API. Engineers at the services monitor service health carefully, but these errors may only occur in edge cases, or for a small subset of users. Even if a service doesn’t have a large amount of reports on Downdetector, check the comments to see if anyone else is reporting a similar issue to yours. 

Downdetector is your first line of support during a widespread incident

Do you think your service or internet is down? First, check your device or router and restart those. If that doesn’t work, go to Downdetector to report your incident with the internet or service you’re trying to reach and see if other users are reporting the same issues you are, or if there are other root causes that may be contributing to an incident you’re experiencing. Remember, while Downdetector can detect widespread issues, it does not diagnose them, so after logging your issue on Downdetector, check in with the service you’re having trouble with to see if they have any error reports.

Having issues with the internet can be a huge headache, so we hope this article helps you better understand what’s going on with your internet experience. Do you want to stay on top of issues that might be affecting your internet experience? Download the Downdetector app for iOS or Android.

The Federal Aviation Administration (FAA) had a major service disruption early yesterday morning, affecting many travelers in the United States. Data from Downdetector® shows all of the major airlines experienced a spike in user-reported issues around the time of the outage, with a sharp rise in problem reports just before 8:00 a.m. EST. While the FAA had a nationwide computer outage that delayed the departure of hundreds of flights, Downdetector data shows all of the major airlines in the U.S. began to experience upticks in user-reported issues (mostly delays) starting at around 7:30 a.m. EST, and peaking at about 9:30 a.m. EST, and dropping in frequency shortly after that.

At 8:50 a.m. EST, the FAA lifted the stop on flights taking off, with NBC News reporting more than 6,500 flights delayed. While the problem reports started to subside after about an hour, travelers who were flying yesterday felt the aftershocks of the FAA ground stop as airlines tried to make up for the delay and travelers scrambled to reach connecting flights on time. While this issue may have been unavoidable, data from Downdetector could have alerted aviation stakeholders much more quickly, saving time, resources, and mean-time to resolution.

Downdetector Enterprise™ can help your company monitor third-party services your business relies on, such as CDN providers, so you know immediately when they go down. Using Downdetector’s real-time alerts you can quickly adjust communications to better serve your customers. Contact us here to learn how your network operations, DevOps, engineering and customer care teams can get faster outage detection. Read more here about the different types of outages and what contributes to them.

Time is critical when your customers think your service is down. Diagnosing the issue can be especially challenging if your internal monitoring tools are showing green but users are flooding customer care and social media with problem reports. Network operations centers (NOC), systems engineering and DevOps teams are often pressed to provide answers to questions like “What happened?” and “When will the issue be resolved?” and “If all of our systems look good, why are customers reporting problems with our service?” Downdetector® can help.

Use Downdetector to diagnose the root cause when your customers are experiencing issues — especially when the root cause is a third-party system your business relies on. Downdetector Enterprise^™ aggregates consumer-submitted problem reports and other key indicators from across the web to help your business continuity teams quickly diagnose and resolve potential problems — while also providing data to customer care and marketing teams so they can swiftly communicate known issues to impacted customers.

The upcoming Ookla® webinar on April 27 will show you how to leverage user-submitted data via Downdetector Enterprise to quickly diagnose the problem and rapidly respond when customers are reporting problems.

Read on to discover eight actions you can quickly take using Downdetector Enterprise to diagnose problems and decrease your mean time to resolution (MTTR) when customers think you’re going down.

1. Receive immediate alerts when customers are impacted

As soon as a sufficient number of Downdetector users submit a problem report with your service or any service you monitor in Downdetector Enterprise, you immediately get an alert. Set specific thresholds within Downdetector Enterprise for when you would like to receive alerts to ensure the issue meets your criteria. For example, we see some Downdector users using 80 problem reports as their threshold, meaning they only receive an alert if over 80 problem reports come in.

Alerts can be customized in the Downdetector Enterprise Dashboard and are available via email, push notifications from the Downdetector mobile app, RSS feed or via the Downdetector API — all of which can easily be integrated with Slack or any other internal monitoring or communication tools.

2. Determine how many users the incident is impacting

The moment you receive an alert that customers are reporting a potential service disruption, you can begin investigating the size and duration of the incident.

This graph shows a real-time view of problem reports in the last 24 hours for your service. This view allows you to see when problem reports started escalating as well as the number of reports for each 15-minute interval.

3. See which problems customers are reporting

The Downdetector Enterprise Dashboard helps you quickly get to the root cause of an incident. Problem indicators in Downdetector show the specific features where users are reporting difficulties.

The Top Indicators page on Downdetector Enterprise shows the number of problem indicators broken down by the feature causing problems for customers. The available problem indicators customers can select from for a given company or service differs across industries. For example, problem indicators for an online gaming company might include online gaming, log-in, website or other relevant features.

4. Monitor external services that impact your business

NOC and DevOps teams can usually diagnose issues within their own service environments — but when issues are caused by an external service, internal monitoring tools may not be able to provide early alerting. Sometimes when customers report an issue with their favorite online service, they’re really experiencing an issue with their ISP or mobile network operator. Alternatively, problems may be caused by a content delivery network (CDN), cloud hosting provider or other systems that impact your service delivery.

The above view shows Reports by Provider, which displays a breakdown of which communications service providers customers are using to access your service. If a significant number of users experiencing problems are coming from a single service provider, it may indicate either an issue with that provider’s service or a network peering issue.

Downdetector Enterprise empowers Customer Care and Marketing teams to help customers resolve issues related to a given ISP or mobile network. By monitoring major communications service providers in your key markets, you can track when they go down and adjust your communications to better serve your customers. In some cases we’ve seen companies simply advise customers to switch from their cellular network to Wi-Fi to fix the problem caused by a mobile operator’s service issue.

5. See locations where users are experiencing issues

The Report Heatmap in Downdetector Enterprise shows the locations where users are experiencing issues.

Easily zoom in to see the individual locations where problem reports were submitted. This view can help you diagnose whether the issue is limited to a specific location or if it impacts users worldwide.

6. Compare your incident to those of competitors or similar companies in your market

Downtime is expensive. And in certain industries, downtime can drive customers to your competitors’ websites or services.

Above is a comparison of two service providers over time. Use this view to compare and contrast your reported outages during a specific time period against up to four other companies. If you see a correlation between spikes in problem reports, you can infer that you weren’t alone in customers believing you were down. If you see overlaps in outages across a variety of companies, it may indicate an issue with a commonly used external service such as a CDN, cloud hosting provider or telecommunications provider.

7. Access your company’s historical incident reports

To input custom data ranges and look at historical data, navigate to the upper right hand corner of the dashboard and select “Last 24 hours.” This will open a dropdown where you can enter in custom dates and times. This can be helpful when conducting internal research to start identifying any patterns in your outage.

If you’ve determined that the root cause is internal, analyzing historical outage information can spotlight patterns or surface potential issues related to seasonality or other external factors. This feature can also be helpful to pull data for reporting after customers have reported an incident.

8. Understand the conversations your customers are having about the incident

Downdetector provides a direct view into customer sentiment surrounding a perceived outage in real-time.

Downdetector Enterprise monitors tweets about potential service disruptions and comments on your company’s Downdetector page to detect trends. The aggregated word cloud feature allows you to quickly see common themes in these posts.

The webinar on Tuesday, April 27 at 7 a.m. PDT (10 p.m. EDT / 2 p.m. GMT) will show you how to rapidly respond when users report problems on Downdetector. Don’t miss it. A recording will be provided for registrants who can’t tune in to the live presentation.