Technology Advisor Blog

Internet Outages Across the Country Yesterday - What Happened?

Posted by Ann Westerheim on 11/7/17 8:36 AM

Comcast Outage - 11-6-2017 - 400.jpgYesterday afternoon users across the country lost Internet access for about two hours.  Some users had intermittent outages and slow downs, and some lost connectivity altogether.  All our clients with Comcast Internet were affected and we were on the phone non-stop talking to businesses who were disrupted by the outage.  The map above from downdetector.com shows the impact was nationwide.

Comcast Outage - Time Graph.jpgThe disruption lasted a little under two hours, but for businesses that rely heavily on the Internet for email and other mission critical business applications, that was a LONG two hours.  The outage started around 1pm and was mostly resolved a little after 3pm.  

What caused the problem? A misconfiguration at Level 3, an internet backbone company - and enterprise Internet Service Provider (ISP) - that underpins other big networks.   In our area, the affected users were all on the Comcast network, but the routing issue at Level 3 created a ripple effect, causing problems for companies like Spectrum, Verizon, Cox, and RCN, in addition to Comcast across the country.

Internet outages like this have occurred occasionally and consistently throughout the decades, but no system is fault proof.  To learn more about the details, wired.com has an excellent summary of more of the technical details.

Monday's outage shows how precarious connectivity is in our highly connected world.  When the outage first occurs, getting accurate information is challenging, and its important to know if the problem is inside or outside of your network.   

What can you do?  A secondary Internet connection can help in most cases.  Advanced business class firewalls can support two connections, and fail over as needed.  Also, you can use your phone for email and web access through your wireless carrier.  An outage like this is a good reminder to put a solid disaster recovery plan in place.  

 

Tags: Internet Outage, Comcast, disaster recovery

Disaster Recovery and Business Continuity - Optimize your Backup!

Posted by Ann Westerheim on 11/2/11 2:31 PM

WinterWeatherWe've had several days of power outages after a surprise October snow storm, and this serves as a good reminder to think about disaster planning and business continuity.  Today, businesses of all sizes need a data disaster recovery plan, which helps ensure the company will continue operations after experiencing some type of disaster due to:

  • Power outages
  • Severe Weather
  • Natural disaster
  • Hardware failure
  • Theft
  • Employee mistakes
  • A malicious act from a disgruntled employee or outsider
  • Loss of key personnel
  • Supply Chain Disruption
  • ...big disasters and small disasters...
While a snow storm may pose a few days of inconvenience, it's a good idea to plan for bigger disasters in advance.  People tend to "look on the bright side", but the statistics are scary:
  • According to the National Archives and Records Administration in Washington D.C., 93 percent of companies that lost their data center for 10 days or more due to a disaster filed for bankruptcy within one year of the disaster.

  • According to The Gartner Group, 34% of companies fail to test their tape backups, and of those that do, 77% have found tape back-up failures.

  • Data recovery services costs range from hundreds to thousands of dollars IF your hard drive can be recovered.

There are many aspects to a good disaster recovery plan to cover personnel, facilities, and technology.  One of the key areas we focus on is getting a solid data backup which is easy to do, and will alleviate a lot of pain when disaster strikes.

If you're still using tape backup, it's time to make a change RIGHT NOW.  Tape Backups are a “known evil,” and industry analysts estimate that anywhere from 40 to 70% of all attempted recoveries  from tapes fail.  Why?  Consider the tape backup process and the many points of potential failure that exist throughout:  You need to Install, update, and configure Backup Software, and determine exactly:  1) what to back up, and 2) when to back up.  Are you sure you're backing up ALL of your critical data, as often as you should?  Tapes also wear out over time .

Your backup should be:

  • Automatic - "set it and forget it" (except PLEASE check restores periodically, best practice is automatic verification)
  • Off-site - rather than relying on taking drives off site, we recommend on-line backup now that costs have come way down and make it affordable for small business.  Keeping your backup in the same building you work will leave you exposed in the event of a site disaster.  Best bet is to get your data FAR away.
  • Local copy too! - If your backup is only on-line, keep in mind that it can take a LONG time to get your data back when you need it.  With a redundant local copy, you get the best of both worlds.
  • Image based rather than file based.  If you do need to get new equipment, rather than spending all that time reloading all your applications (IF you can find the disks), and image-based backup basically serves as a clone.

For clients who have a high cost of down-time, we strongly urge them to consider an optimized backup and disaster recovery service.

For more details about putting a full plan in place, check out a great article on csoonline.com:  Business Continuity and Disaster Recovery Planning: The Basics

Tags: backup, disaster recovery, business continuity, on-line backup, image backup

Subscribe by Email

Most Popular Posts

Browse by Tag

See all tags...

Connect With Us

Older Blog Posts

For older Ekaru blog posts, go to ekaru.blogspot.com.