Blog Article

• Posted Jun 4, 2019

Is Your Internet Booking Engine Safe from Scrapers?

This month Takeflite has implemented an important security update across all Takeflite managed airline Internet Booking Engines to combat Website Screen Scrapers.

We spoke with Takeflite's Chief Technology Officer, Mark Griffin, about how Screen Scrapers may be effecting you and your airline, plus what Takeflite is doing behind the infrastructure hood to ensure their current and future clients passengers have a seamless booking experience.

Takeflite Q&A Session

What is Website Scraping? 

Website scraping is when someone creates a process to systematically retrieve data from your website. They generate a large amount of website requests, and as we have occasionally seen, that can cause a slow or less than optimal experience for your real customers. 

What are the Scrapers accessing? 

Presently the only data being accessed by the Scrapers is the same data presented to a customer looking to make a booking, such as, published scheduled flight times and fares. 

Why are Scrapers doing this? 

There are several reasons including: 

1. Getting data to sell to interested parties 
2. A competitor analyzing your fares and route 
3. Third party re-sellers trying to on-sell a service

Why is this becoming more common?

It has been happening for many years, but Takeflite has been constantly monitoring the web services and adapting the way we vet incoming traffic, to remove unwanted requests. 

Our current solutions challenge approximately 180K attacks from rogue sources per day before they reach your website. The problem is that the tools being used by Scrapers are becoming much more sophisticated.  

Why are the Scrapers harder to detect and control? 

A couple of years ago we could quickly identify the source of a Scraper or organization as they used Data Center accounts to run the scraping. These were easy to detect and set alarms for, or automatically shut down by our rate limiting web firewall and other tools.  

In the last year this has changed, the perpetrators have new techniques that connect to many internet providers around the world. Each connection makes less than 10 requests and then logs out then back in to change their IP address, along with changing their browser profile. This randomizes their profile, so they are mixed in with all your other real customer traffic. There is no definable source or pattern to match with traditional tools. Their tools also monitor when they are being detected and will automatically change their appearance until they find a non-detectable profile. 

So what is Takeflite doing about it? 

We identified this change in behavior late last year and started a process to research how we might combat the issue. 

In doing so we wanted to ensure your customers would be able to access and buy your services with as little interference as possible. We want your site to leave them with a great experience. 

Takeflite has spent the last few months using a new solution in the background to collect the necessary data and analyzing the threats of our clients sites. We needed that time to allow the solution’s artificial intelligence to learn how sites were used, so it could then distinguish between those who are genuine customers that it should let access to the site, and those that are just scraping. 

At the same time, we have built additional functionality, understanding that while the solution is the best available, no solution is perfect.  We have added the ability to white-list organisations we want to access your data, that use similar approaches to unwanted visitors and could be accidentally challenged. The goal being to continue business from your customers. 

This new tool acts in the background, allowing us to get rid of the visual checking that currently appears, improving the customer experience and overall performance. 

Interested to know more? Drop us a line and we can talk you through how it can help you support@takeflite.freshdesk.com