Ticketmaster blames ‘bot attacks’ for Taylor Swift ticket fiasco

The fiasco around Ticketmaster's sale of tickets for Taylor Swift's upcoming tour has caused so much outrage among fans that the issue has now reached the halls of Congress and the Justice Department.

On Friday, Ticketmaster released a statement attributing the ticketing issues to bots overloading their website alongside the pop star’s adoring fans.

We want to apologize to Taylor and all of her fans – especially those who had a terrible experience trying to purchase tickets. We feel we owe it to everyone to share some information to help explain what happened: https://t.co/1Gn4kRIvq8— Ticketmaster (@Ticketmaster) November 19, 2022

“Historically, we’ve been able to manage huge volume coming into the site to shop for tickets, so those with Verified Fan codes have a smooth shopping process,” Ticketmaster said.

“However, this time the staggering number of bot attacks as well as fans who didn’t have codes drove unprecedented traffic on our site, resulting in 3.5 billion total system requests – 4x our previous peak.”

Ticketmaster requires fans to register ahead of time in an effort to manage interest in high-demand shows. The feature is specifically designed to weed out bots and limit overcrowding. More than 3.5 million people pre-registered for Taylor Swift’s tickets ahead of time, the largest in Ticketmaster’s history.

The proliferation of affordable bots-as-a-service tools has made it even more difficult for buyers of tickets and products like sneakers or Playstation 5s. Bots now beat out everyday people thanks to powerful technology made readily available by sites like Cybersole, Kodai, GaneshBot and more. Those using bots then resell the goods for a hefty profit.

The U.S. Congress passed the BOTS Act of 2016 to address purchasing and reselling concert or event tickets with bots but it has had little effect on the ability for bot runners to wipe out inventory.

'Waiting Room' failure

Jason Kent, hacker in residence for cybersecurity firm Cequence, has been at the forefront of the effort to help retailers stop bots from buying up inventory.

Kent told The Record that Ticketmaster was correct in assessing that bot attacks had caused the issues but not in the way they think.

“Often these ‘waiting room’ type mitigations create the problem. Now you have 1.5 million people with established sessions on a website and its having to manage all of the login traffic and searching, then the waiting room code has to be sent,” he said. At the end of the process, a customer has to add the ticket to the cart and purchase. A bot, though, can get to this step much faster than a human ever could.

“They operate in milliseconds. The way you stop this is to recognize them as a bot as they hit the site and throw the traffic away,” he said.

Kent explained that bots often bypass waiting room-style mitigations in several different ways but said he was surprised Ticketmaster’s system failed so spectacularly given the relatively low volume of sales.

Apparel and technology hype sales often see a much higher volume for fewer items, Kent said, in some cases 25 million transactions attempting to purchase some 1,000 items.

“The way to defeat these purchase attempts isn’t to make the humans wait in line while the bots finish buying out all of the stock. The way to defeat this is to identify the humans and block the bots. The challenge is pretty straight forward, the implementation is what matters,” he said.

Kent explained that when a bot wants to purchase a bunch of items, they figure out the sales process and find out how to bypass sitting in a waiting room.

They first attempt something called a “batch request,” where in one request they will attempt to buy up as much as possible.

Another technique bots try, according to Kent, is the “update cart” method where bots buy tickets that no one wants for another event and then they go through the motions of changing their selection to high-value tickets like those for Taylor Swift.

So the bots will get a bunch of tickets for the Ice Capades, for example, and then change the transaction to Taylor Swift tickets, bypassing the need to wait in any room and still getting their tickets purchased.

“Packing everyone in a waiting room means the waiting rooms are full while the bots are overwhelming the infrastructure and buying everything up,” Kent said. 

Kent explained that typically they partner with organizations before large sales like this and have a plan of attack beforehand involving tools like “final infrastructure alerts” that ensure mitigations are working.

“Given that TicketMaster decided to use dynamic pricing based on demand, they were doing analytics on speed and types of transactions that were happening to adjust the price of the next ticket sold. No doubt this processing caused hiccups in their Product Databases as their own systems would have created an internal denial of service while it tried to update pricing on the fly,” he explained.

“Basically Ticketmaster was using methods that worked in the '70s during the Gas Crisis but we have more advanced methods today that don’t rely on client signaling, transaction flows or anything resembling making people stand in line. The best method is to use data science and machine learning to figure out transactional intent and only let the humans buy the tickets.”

When asked what kind of actors are behind the bots Kent said they are typically not cybercriminals but people interested in buying up as many tickets and reselling them. Anyone selling $5,000 Taylor Swift tickets on resale sites like StubHub are likely “botters,” he said.  

Andrew Barratt, vice president of cybersecurity firm Coalfire, added that the bot market is “alive and well” and is a “strange sub-underground in the cybersecurity world.”

Barratt has found bots available for around $1,400 that will register many accounts and pre-fill out all the forms for verification, which is available from a well-known online bot seller.

“The only way to really avoid these kind of bots is to push for better identity verification that does more than confirm email address ownership and capture accounts,” he said.

“The challenge is the cost of this scales up quite significantly, but there are a number of identity verification as a service offerings on the market that could be leveraged during the sales and/or account registration process that would mitigate a number of these concerns. For the real super fans, the cost of paying for more official identity verification (where your documents are checked) would probably be welcome.”