I heard a rumbling —  it was my phone frantically buzzing: calls, messages, slack notifications. The Class Central website was down. It was getting millions of hits, making it unusable for everyone.

Later, we learned that it was a ransom DDoS attack, and that at least one online course provider (that regular readers of The Report would know about) had also been attacked. The attacker asked for $5,000 in Bitcoin for a “solution file to all the problems caused.”

The timing couldn’t have been worse. I was in Guatemala for the Learning with MOOCs conference. I had just given a keynote (slides here) and was scheduled to take part in a panel a couple of hours later. And our most senior engineer was on their way to their hometown and off for the week.

Class Central was down for 36 hours as we worked to bring it back up. The attacker was watching our every move and responding to it, even sending us messages saying that our efforts were in vain.

This is the story of how we mitigated the attack, what worked and didn’t work, the SEO impact (short and long term), as well as how much it cost us.

The Attack: How and Why

As the site went down, we immediately started looking for clues. We had recently done some infrastructure improvements, so our first thought was that we might have messed up a setting.

But after looking at the logs, it became clear that we were under heavy attack, the scale of which we’d only realize after switching to Cloudflare.

The attack is a “ransom DDoS attack.” Put simply, millions of bots started visiting the Class Central homepage every hour. Our servers couldn’t differentiate between actual humans and bots, and we can only serve a fixed number of users at a time.

As a result, we were turning away real people, since we were busy serving the bots. Practically speaking, the site was unusable to everyone. To learn more, Cloudflare has some of the best explainers on this topic.

A few hours later, we realized why we were being targeted. The attacker who went by John Evie Junaid (Twitter handle @Evie24435208) had messaged me on my personal Twitter, Class Central’s Twitter, as well as our contact email. Here’s the message:

The person indicated that they’d identified an “sql injection vulnerability” and used that to “crash apache.” This was an obvious lie. The logs were clear: it was a DDoS attack. Also, Class Central doesn’t use Apache web server.

But I can imagine smaller sites with less technical resources could fall for it. As the attacker mentioned, the real threat was to “keep your site closed for months off.”  Even if you end up paying the $5,000, there’s no reason the attacker would stop or wouldn’t do the same thing again later.

18 hours later, I received another message from the attacker.

The attacker noticed that we had switched to Cloudflare (more on that later) and stuck to their original script, saying it was a “software problem.” But here, they pointed out another threat: that we might “lose Google Ranking.”

This actually happened to us before. Last year, we lost half our traffic when Google tweaked its search algorithm. A year and a half later, we still haven’t fully recovered. Early this year, when the 2U stock price dropped by half, I speculated that Google’s algorithm change might be partially responsible.

I’ve also written about the SEO Content Strategy of top platforms like Coursera, MasterClass, and edX. Even if the attacker hadn’t explicitly mentioned it, this risk was already on my mind. If you scroll down, you will find how the attack impacted our SEO.

DDoS Mitigation

Once we figured out that it was a DDoS attack, one of the first things we did was trying to find a pattern in the attack — for instance, something in the user agent or IP address — to try to distinguish real users from bots and filter out the attackers.

But we soon realized that this attack was more sophisticated and used a variety of IP addresses.

We noted that the attacker was sending all the traffic to our homepage. So we started showing a maintenance page on the homepage, and this allowed us to bring the rest of the site up.

Another trick we tried was cloning the homepage to another URL and using JavaScript to redirect genuine traffic from the maintenance page to the cloned homepage. Basically, the bots would see the maintenance page, but real users would only briefly see the maintenance page before being redirected to the alternative homepage.

But soon, the attacker realized what was happening and started attacking other pages too. That’s when we realized we were being deliberately targeted. At this point, we hadn’t seen the twitter/email messages yet.

As I mentioned earlier, at the time, I was in Guatemala for a conference and was working remotely with my colleague @suparn, who was in India. It was about 2AM in India when this all began. And 24 hours later, Suparn was supposed to give two remote presentations at the same conference. Suffice to say, he barely got any sleep.

We rapidly realized that this is more than we could handle on our own. In the world of DDoS mitigation, there is one name (in terms of public perception, at least) that stands above all: Cloudflare. After our own attempts failed, we finally decided to switch to Cloudflare.

Cloudflare Suparn to the Rescue

But even after switching to Cloudflare, the bots were still getting through. My perception until this point was that Cloudflare would magically detect all the bots and block them. I was wrong.

We had to explicitly configure Cloudflare, tailoring the setting to the attack to effectively stop all the bots. This is something both Suparn and I weren’t really familiar with. When I posted about Class Central being under attack on LinkedIn, a Cloudflare engineer reached out to me and connected me with an Account Executive.

The Account Executive directed us to some helpful resources on the Cloudflare website, with approaches we could try. When asked if they could help us configure Cloudflare, we learned that it would cost us $5,000 per month with minimum 1-year commitment. Honestly, the attacker’s price seemed more reasonable.

But not all hope was lost. When you’re in a pinch, you’re sometimes able to break your shackles and level up quickly. What’s the worst that could happen? The site was already down.

Suparn did just that. It was almost as if a Suparn from the future was currently guiding the current Suparn.

Even something like switching to Cloudflare would normally take us weeks to test and deploy. Instead, it was done in less than 30 minutes. After a lot of trial and error, a sleep-deprived Suparn figured out the correct Cloudflare configuration for blocking the bots, while allowing real humans to visit Class Central.

The secret lies in Cloudflare’s Rate Limiting and IP Access rules. Eventually, the attacker backed off. Some 36 hours after the attack had started, Class Central was back online and stable.

Costs

Class Central operational costs are relatively low compared to the volume of traffic we serve. But just in 36 hours, our Google Cloud costs doubled for the month. Basically the high volume of bot traffic caused our networking costs to skyrocket.

In short, the bot traffic cost us $1,000 per day, even though Class Central was down the majority of this time. This was one of the most scary aspects of this DDoS attack and probably also the reason why these attacks can be effective.

SEO Impact

Fortunately, there wasn’t a long-term impact to our SEO. There was a reduction in search traffic the week that immediately followed the attack, but it has since rebounded and continued to go up.

One thing I did notice was a significant drop in crawl requests from Google. It’s unclear if this is due to the attack or the switch to Cloudflare. But it’s something to keep an eye on.

DDoS attacks are something many companies prepare for. In our case, it was a trial by fire, compounded by tricky timing. We have lean operations, so a just-in-time approach worked for us. But for many startups, downtime, even when short, can have serious consequences. If you’re in that situation, preemptive measures are likely a good idea — especially if you don’t have a Suparn on your team.