Explosion at The Planet’s H1 Data Center

New Entry: Jerry Seinfeld + Microsoft = Apple Killer? http://tinyurl.com/59zw2k [...]

Weblog
About
Archives
Subscribe

Explosion at The Planet’s H1 Data Center

Posted by Markus Langenfeld - 05/31/08 at 08:05:24 pm

I got an interesting email tonight, apparently there was an electrical fire at The Planet’s H1 data center in Houston Texas. According to some sources there was an actual explosion powerful enough that it knocked down walls in an equipment room. So far there are no reports of damage to servers or networking equipment but power is offline at the facility and approximately 9000 websites are affected. This will surely be all over the tech news websites by morning.

Here is the official release that was sent to The Planet clients -

Dear Valued Customers,

In the interest of keeping you updated, tonight at 5:45 p.m. here in Houston, a transformer in our H1 data in Houston caught fire, which required us to take down all generators as instructed by the fire department. All servers in the H1 data center are down, as is our ServerCommand customer portal, which are working to move to another data center.

None of our other data centers are affected.

Our management team and facilities staff are on site evaluating the situation. In our latest assessment, we have determined that networking gear has not been damaged, but we are without power so assessments continue. All disaster recovery systems are in motion, and we have teams already working in the data center.

We are posting messages in our forum, in Orbit and on our customer support phone system.

We appreciate your patience. We are working around the clock to resolve these critical issues.

Hardware, Technology, Web Hosting

18 Comments »

RSS feed for comments on this post. TrackBack URI

Theplanet has been going down the drain for quite sometime, I have 1 server that was effected but this, thank goodness I started moving my stuff to softlayer last year.

Hopefully with 9000 customers filing for SLA refunds it should give a nice financial uppercut to theplanet.

Comment by samalan — June 1, 2008 #
^^^

Shut up you idiot. Have you got any proof that ‘Illegal Mexicans’ built their data centre?

Comment by x — June 1, 2008 #
[...] Langenveld writes: “Apparently there was an electrical fire at ThePlanet’s H1 data center in Dallas [...]

Pingback by Explosion at theplanet knocks out 9000 websites | Virtual Solutions Network - — June 1, 2008 #
I am among those impacted by the outage. The irony for me in this thing is that I had a major client’s website official release today June 1st. Murphy’s Law? Act of god? Cosmic chance?

When I lay awake at night worrying it usually takes the form of something going wrong with the scripts I wrote. The phrase “bursting into flames” has always been used by me in figurative terms…NO MORE.

In my 14 years experience I have never had as much trouble as I have at this particular datacenter. Frequently I am unable to contact my server or sites on my servers. Frequently would defined as three or four times a week at times for periods as long as an hour. I had myself convinced that it must be some local DNS problem. Not so sure now as I am regularly getting complaint from clients that they can’t reach their sites.

Granted such things like this happen but in my experience they tend to happen because of poor decision making or general neglect.

“Accidents” happen when those responsible take their eyes off the road, or stop paying attention to the streets signs. I read THIS sign! It suggests a detour away from this datacenter.

Comment by Brian Marshall — June 1, 2008 #
Wow, I wasn’t aware that ThePlanet had to many problems, they are supposed to be a big player.

Comment by Markus Langenfeld — June 1, 2008 #
The geniuses kept all the ev1 DNS servers in one location.

Def. an amateur operation.

Comment by Jim — June 1, 2008 #
Interesting comments … my experiences with our leased, dedicated server are similiar to Brian’s. 1 year ago, performance was great. 6 months ago, it started going to crap - FTP uploads would hang up and crash, customer browsing sessions would hang up, etc. ThePlanet tech support claimed it was our server - they needed to “sell us more RAM”. Funny thing - the exact same server had been working flawlessly for 3 years. I monitored things and found no correlation between hang-ups and ‘available memory’ via the Linux “top” command. What I did notice is that there were major bottlenecks in the backbone links to the datacenter - 30% to 50% dropped packets at the times-of-trouble when I ran appropriate diagnostics. Tells me that they got tight on the bandwidth costs, and cut things back too far, to where peak traffic couldn’t be handled. That was not the case when it was EV1, before ThePlanet bought them out … I’ve come to the conclusion that ThePlanet is not an honest company.

Comment by John Silvey — June 1, 2008 #
That was a very stupid and idiot racist comment and should be deleted inmediately.

Comment by Juan — June 1, 2008 #
We have 12 servers at ThePlanet, and we have been complaining to them about packet loss for the past 4 months. We see the packet loss from all routes around North America. They say it’s a problem “on our end”. We have decided to stop using ThePlanet servers and moved our resources to other locations, but because of the contract with them we continue to pay for servers we don’t intend to use. I won’t be paying anymore, and will ready our lawyer to deal with it.

Comment by Allen — June 1, 2008 #
I edited his comment. Thanks for stopping.

Comment by Markus Langenfeld — June 1, 2008 #
I have been following the Planet Servers problem since saturday afternoon, More than 24 hrs, as I have a lot riding on this problem, both time and money. At first quite a few customers were posting their concern and distrust on the Planet forums pages. THEN the Planet began to Moderate the pages to prevent negative postings. THEN suspicious One Hour old Newbie Accounts started springing up posting unbelievable PRAISE for the service. OBVIOUSLY Employees and Family of the company. THEN the Planet started to sanitize the previous negative postings.

The first postings by the One in Charge at first said it will be resolved in 2 hours, Then another 3 hours, Then by Sunday Morning, Then by 10:00am Sunday, then by 4:00pm for certain THEN they promised to power up at 5:00pm on Sunday and report status, NOTHING Reported until 7:30 THEN they say WELL Monday During the day everything should be up and running again Monday night for certain. LIES LIES LIES.

They claim that only one of 4 data centers is affected HOWEVER, the forums server is running VERY SLOWLY to the point it takes pages several minutes to load. GEE they couldn’t send out an RSS feed from the other four Locations so we could get updates? How HELPLESS ARE THESE MONKEYS? Do you know the error page that comes up on an Error Page that says they have dispatched the army of Wrench Toting Monkeys? This is where the Monkeys get their training!!!

They couldn’t get a FREE GOOGLE BLOGGER ACCOUNT and create a feed with FEEDBURNER like any twelve year old could do in 5 minutes to give us real time updates?? Or move the forum to a server better than an IMB 286 PC level of Service? Oh No, SOME great Customer Service!! It’s Sunday at 10:00pm go to the Forums and the few hundred people online have bogged down the system to 1992 Dial-up Speeds.

WHAT”S UP WITH THAT? No Real Updates, Lame Excuses, Cutting off Communication, Deleting Concerned Messages. NOW go to the web pages that are actually SELLING the services of “The Planet” They load like Lighting. Read the Description about just how FIREPROOF their system is.

One of their own FORUM MODERATORS said it best. “Well Even I have a backup for my website outside the company. You really need to contract with another vendor for a backup in case something happens like this”. You shouldn’t put all your eggs in one basket. But their own website PRAISES how they are your PERFECT choice for primary AND backup service.

They might be SANITIZING their own forums of comments, but they will have all these postings to live with forever.

Sherlock

Comment by Sherlock — June 1, 2008 #
You don’t need to be Sherlock Holmes to deduce some answers read this Trade posting a few months ago;
http://www.thehostingnews.com/news-dedicated-server-firm-the-planet-data-center-manager-garners-award-4306.html It claims “Mr. Lowenberg conducted a six-month trial to reduce power consumption and increase data center operating efficiency. Initial results demonstrate that while critical server loads increased by 5 percent, power used for cooling decreased by 31 percent. Overall, the company experienced power reductions of up to 13.5 percent through a broad range of improvements. The new green initiatives were conducted across its six world-class data centers.” and also The Planet operates more than 150 30-ton computer room air conditioning (CRAC) units across its six data centers. In one data center alone, the company was able to turn off four of the units. The cooling requirement on two of the units was reduced to 50 percent of capacity, while another nine now operate at 25 percent of capacity. The company also extended the return air plenums on all of its down-flow CRAC units to optimize efficiency. ”

Wow, Let’s See Hummm… You turn off or down the air conditioning and the servers only run 5% hotter, IN THE WINTER. Hummmm…. a few Hot Summer Days….. Now how could that transformer overheated? You don’t need to be Sherlock to figure this one out!!!!!!

Comment by Sherlock — June 1, 2008 #
We utilize the planet for our shared hosting service, we have 6 servers there. Most of the are affected by this outage.

My 2 cents.

1. Of course they don’t want people making negative comments on the forum. The forum needs to be a place where updates can be obtained, you shouldn’t have to search through a bunch of negative comments to get to answers.

2. This H1 data center has been fine for us, likely the best so far. Ordered our latest server 63 days ago, it host 174 of our customers and has never crashed.

3. This is a explosion, what can you do? Just wait. Complaining about something like this is crazy.

Comment by Chad Hawkins — June 2, 2008 #
Oh yea, the data center is in Houston . . not Dallas. Hence the H in H1.

Comment by Chad Hawkins — June 2, 2008 #
http://www.syndicatedelitist.com/2008/06/theplanet-outage-update.html

http://www.syndicatedelitist.com/2008/06/theplanets-h1-datacenter-encounters.html

Both have photos and information directly provided by ThePlanet staff

Comment by Philbert — June 2, 2008 #
[...] We made it! I essentially used up my entire weekend but this website along with the others I maintain and host are all moved over to A Small Orange. I can’t say it was easy, in fact I ran into some head scratching (and hair pulling) roadblocks along the way. However I’m fairly confident that the big issues are behind us, and it’s always reassuring to know that it could be worse. [...]

Pingback by At least nothing burned down… » Markus Langenfeld — June 2, 2008 #
[...] Langenveld writes: “Apparently there was an electrical fire at ThePlanet’s H1 data center in Houston Texas. [...]

Pingback by Explosion at theplanet knocks out 9000 servers « Grumpy Boy — June 3, 2008 #
Aren’t the servers back up now?

Comment by CodyLoco — June 4, 2008 #

Explosion at The Planet’s H1 Data Center

18 Comments »

Leave a comment

Categories

Subscribe

Popular This Month

Latest Entries