S2 Games Offers Apology And Compensation To Heroes of Newerth Players

As I posted yesterday, online MMO strategy title Heroes of Newerth has been offline to to the developers running into a “critical hardware failure” which caused a server knockout. Now we’ve gotten some word from the developer on exactly what’s happened, how they plan to fix it, and the events that happened to cause the game to go offline.

S2’s CEO Marc DeForest posted on the game’s official forums an update which offered a full apology, and compensation for the loss of play time as of recently. In the post, Defrost said, “We realize there is nothing we can do to make up for the last five weeks of unacceptable performance.” S2 plans to reward their loyal fans and players with a nice package that is quite generous, in lieu of the game’s outage. Every player will be receiving:

• 2,000 Silver Coins
• Every item in the Goblin Store will be unlocked until Wednesday, June 29 at 12 PM EDT (GMT -4) for players to use in their games (This excludes Nickname Changes, Custom Account Icons, and items not currently available for purchase).
• Leaves from Tuesday, May 17 through Thursday, June 9 will be removed from all players. This is in response to numerous server issues, reconnection issues, etc. for users who should not have received leaves.
• Players can now recover all purchases made during the rollback period by going to:http://www.heroesofnewerth.com/recover Any valid purchase from June 9 through June 21 can be recovered using this page. Just enter the required information and your purchase will be recovered.

On top of the apology and compensation bonus announcement, DeFrost posted a synopsis of the events leading up to, and during the game’s outage:

Following the DDoS attacks in May, we retained a world renowned cyber security company to secure our network (this required significant software and architecture changes). After our network was secured, our then current hosting provider began experiencing connectivity issues. On June 9th, we began a transition to one of the premiere hosting companies in the world. Though the transition period was rocky, we had the system in a near 99% functional state. It was at this time that CPU upgrades to the database boxes were needed to allow for additional capacity and consistency of service. On Tuesday, June 21st, the replacement of the CPUs was started. Upon bringing the boxes back online, the engineers noticed anomalies in the raid setup, causing a total loss of data on the master and slave boxes. Following these events, they immediately attempted a restoration process on the data, it is unknown at this time if any of the data is recoverable. Concurrent to this happening, we began work to bring up our previous provider’s network that was still in place with the data from June 9th, which brings us to today. We are currently running on a temporary system while we construct our new environment at another hosting firm. Due to this system’s temporary nature, not all services may be operating at 100%.

The developer is planning for another maintenance window at which point they will switch over to the new infrastructure. During the transition, S2 intends on having the game playable, and will inform all players of the maintenance at a later date. The company takes full responsibility for the issue, stating in the post:

We hold ourselves completely accountable for the events that have transpired. Through two years of closed beta, open beta, and retail play, we provided nearly 100% up time. We are intent on restoring this level of service you have all come to expect. We will be making the necessary adjustments in our systems, processes, and management to achieve a better and more reliable quality of service!

Though the game’s outage has been an issue for many players out there. S2 is stepping up to the plate and taking the blow for the issue at hand. It’s nice to see a game developer being honest, forthcoming, and diligent about the issues that arise, as they do happen. I think that players will be happy with the new compensation bonus they are to receive, and it looks like S2 is sincerely devoted to getting the game back into players’ hands. They care about the players, and the players care about the game. As more surfaces, we will be sure to continue the updates!