Hey there, my fellow disaster recoverers. Today, I'm going to share with you the wildest roller coaster ride of my life – the day everything went down in flames and left me questioning the existence of my backups. So buckle up and get ready for a story that will make you laugh, cry, and appreciate the true power of disaster recovery plans.

The Red Alert

It all began on an ordinary day. I woke up, grabbed my favorite mug of coffee, and settled in for another day of work. Little did I know that things were about to take a turn for the worse. As I checked the company roundup, I saw a storm of red alerts flashing on Nagios, indicating that all the services were down. Panic set in, but being the cool, calm, and collected person that I am (or so I thought), I simply ignored it and went on with my coffee.

‍

The Moment of Truth

But alas, my colleagues were determined to ruin my moment of caffeine bliss. They bombarded me with messages insisting that the red alerts were still there. That's when it hit me – something was seriously wrong. I quickly verified the project website, only to witness its complete non-functionality. Desperate measures were called for, so I attempted to access the server via SSH. No response. Zilch. Nada. It was as if the world had turned against me – even the provider's website was down…

‍

Disaster Strikes

Stress levels skyrocketed, my friend. Now, it's not uncommon for Google or Microsoft to have temporary hiccups, but even they were of no help. So, I did what any self-respecting person would do – I turned to Twitter. There, in all its glory, was a message from the CEO. "IF YOU HAVE A DISASTER RECOVERY PLAN, RUN IT NOW(!!!)," it screamed at me. And that's when I realized, I was truly, ahem, screwed.

WHERE IS MY F****** BACKUP IF NOTHING WORKS?

‍

The Great Migration

With a mix of panic and determination, my plan started forming like a frenzied dance in my head. We were going to migrate to the chamber of salvation – AWS (Amazon Web Services). Fortunately, AWS was my second home, so I felt a flicker of hope. But wait, I realized I hadn't even topped off my coffee! Nevertheless, there was no time to waste. The fate of the whole team rested on my shoulders... And I'm not even wearing a cape.

‍

The Geek Squad

Amidst the chaos, phone calls started pouring in. My supervisor (here I salute @RemigiuszMyszk – what a beautiful time ;)), my colleagues, even the switchboard joined the party. It felt like an endless loop of madness. This was when the true power of teamwork revealed itself. I called for support, and I got the best team a person could ask for. Shoutout to my rockstar colleagues who stood by my side (@MaciejDaleczko, @ŁukaszParadka, @MonikaMęczekalska and of course the rest of the IT guys – you are the real MVPs! Also cheers to @MichałTomala - thx mate). You all swooped in and saved me from myself.

‍

The Pomodoro Battle

With a mountain of work ahead, I switched into Pomodoro mode – 25 minutes of intense focus, followed by a 5-minute break for some push-ups (yeah, push-ups 🙂 helped me a lot). It was during one of these short breaks that I realized I needed additional brainpower. Two heads are always better than one, right? So I made the call and gathered the reinforcements. Trust me when I say that it was a game-changer. The stress started dissipating, and with each passing hour, we could see the light at the end of the tunnel.

‍

Lessons Learned

After what felt like an eternity, the site finally sprang back to life. However, there was a catch – it was a backup from a grand two months ago. Yes, you heard that right. I had only managed to set up the backup after taking over the project in January. Talk about cutting it close!

But amidst the chaos and the sweat (or maybe tears), I learned a valuable lesson. People can be divided into two types – those who do backups and those who... WILL DO backups. I had officially joined the latter category, and let me tell you, converting to the backup-believer club is a truly enlightening experience.

So there you have it, folks. Remember, disasters do strike, but with the right preparation, a little bit of luck, and an amazing team by your side, you can turn any f***up into a triumph.

Stay hydrated, be nice to strangers and stay backed up. Cheers!