Security Testing Online Travel Sites

Today I want to discuss how we test online travel sites. The approach to testing is quite similar to how we test online trading sites. Let’s review the basics of that approach:

1. Create a Threat Profile for the Travel site
2. Create the Test Plan
3. Perform the Tests
4. Prepare the Report

The Threat Profile is the list of all threats to the application. “Threat” is the goal of an adversary – it’s what he wants to achieve. As the owner of a travel site, the threats are what you are worried about. Thus, the threat profile for an online travel site like Orbitz or Travelocity would include these threats:

An adversary…

• … cancels the tickets booked by another user
• … buys tickets at a lower price than quoted
• … buys tickets on flights where no more tickets are available
• … steals the credit card details of other users
• … gets the travel itinerary of all users
• … changes the itinerary of other users
• … deletes hotels from the inventory

The Threat Profile helps us focus our testing. It helps us pursue the attacks an adversary is most interested in. [We discuss this in more detail in our post “Why we love Threat Profiles”.]

It takes us about a day or two to prepare the Threat Profile. The test engineers then share the Threat Profile with you to get your feedback: have we missed something? Have we exaggerated a threat? The Threat Profile drives the Test Plan. For an online travel site, the Threat Profile might identify about 50 threats.

Next comes the Test Plan. The test plan first maps each threat in the Threat profile to the relevant pages in the application. E.g. for the threat “an adversary cancels the tickets booked by another user”, we identify the “cancel tickets” page as the relevant page. Once the relevant pages are clear, we then decide which all attacks to try on that page to realize the threat. For instance, to test “an adversary cancels the tickets booked by another user”, we choose Variable manipulation and SQL Injection on the Cancel Tickets page. Similarly, for each Threat in the Threat Profile, we identify the relevant pages and the attacks. We put a lot of thought to create the test plan. We have a master reference checklist of all attacks to help us - we pick attacks for the Test Plan from that checklist.

After a senior reviews the Test Plan and approves it, we begin the testing. Usually we get a few new ideas once we begin the test, we then update the Test Plan and continue the Test. The tests themselves are a combination of manual and automated checks.

When we find a vulnerability, we capture step-by-step screenshots of the attack. In the final report, we walk through the attack with the aid of these screenshots. The report describes the solution for fixing each vulnerability and also provides references to good papers on the topic.

How We Test Online Trading Sites For Security

If you own an online trading site, you’ve probably thought about these threats:

Can an adversary…

• … view the portfolios of other users?
• … place trades on behalf of another user?
• … cancel trades that have already been executed?
• … steal the passwords of other users?
• … bypass login?
• … deny access to all users?

We call the list of all the threats to your application its Threat Profile.

The Threat Profile is the starting point of our tests. In the first few days, our engineers study your application and prepare its Threat Profile. We share it with you and get your feedback – have we missed something? Have we exaggerated a threat? The final Threat Profile drives the Test Plan. For an online trading application, the Threat Profile might identify 20-40 threats.

The Test Plan first maps each threat in the Threat Profile to specific pages on your site. For example: The adversary views the portfolios of other users might be mapped to the “View Portfolio” page. Next, the Test Plan identifies all the attacks to try on those pages to realize that specific threat. For example, on the “View Portfolio” page, we might decide to try a Variable Manipulation attack and a SQL Injection attack to see portfolios of other users. The Test Plan is thus built up for all the Threats in the Threat Profile. This is the most intensely “thinking” phase of the test. To assist our engineers, we have a master reference checklist of all attacks – they pick attacks for the Test Plan from that checklist.

Once the Test Plan is prepared and approved by a senior, the testing begins in earnest. The tests are a combination of manual and automated checks. The engineer adheres to the Test Plan. When he gets new ideas, he updates the Test Plan and performs the Test.

When an attack succeeds, we capture the screenshots of the attack. Our final report walks through the attack with the aid of these screenshots.

The Threat Profile to Test Plan approach helps us focus our testing on the threats that matter to you. Any large application penetration test involves hundreds of test cases, so it’s important that the engineer focus on the right set of test cases. He should, for instance, focus on whether a trade can be cancelled than on generating error messages by tampering unimportant variables.

Helping B2B applications win more customers

I’ve been working with B2B software companies for 7 years, helping them win more business by scoring high on their application’s security.

When B2B software companies sell to their clients, they almost always encounter questions on the security of the application: sometimes these are raised as objections in sales discussions, but more frequently they are asked in a security questionnaire. The larger the prospect, the longer the questionnaire. The buyers’ want to ensure the application meets or exceeds baseline security checks. These baseline security checks are sometimes defined in the buyer’s internal application security standard.

And this is true for our clients across vertical solutions for financial services, healthcare, insurance etc. and across horizontal solutions like employee productivity, human resource management, sales & marketing management etc.

Some of the basic questions our clients encounter are:

• What are the security features of your application?
• Does your application do input validation checks on every field?
• Are sensitive data protected in transmission?
• Does the session expire automatically after a period of inactivity?
• Have you had a third party security assessment of your application?

Size Matters: Larger clients have longer lists

As the size of their client increases, so does the due diligence. A third party application penetration test is an important requirement for most large companies globally. Boards, managements and security executives have to take the very best care of their customers’ data, their trade secrets, their brands, their reputation. That requires verifying the security of all applications with critical data. Whether the application is developed in-house, developed offshore, hosted internally or completely outsourced, they need to be tested thoroughly. Third-party reviews mandated by large clients is the top driver for our clients in the US, Europe and Asia.

Certification: A tangible marketing tool

A penetration test report gives a snapshot in time of the application’s security posture. To make this posture future proof (version proof), enroll the application in a continuous testing and certification program. Plynt has been certifying applications since Feb 2006. The Plynt Certification criteria specifies the minimum security requirements for an application. The certificate enables application owners to show how the application stacked up against a standard set of checks. A Certificate thus delivers real value in a very tangible form.\

Comply with Section 6.6 of PCI DSS with code reviews and get more

The PCI section 6.6 is fairly clear in what it wants from merchants.

Ensure that Web-facing applications are protected against known attacks by applying either of the following methods:

• Having all custom application code reviewed for common vulnerabilities by an organization that specializes in application security
• Installing an application layer firewall in front of Web-facing applications

The investments involved in placing application layer firewalls or doing a code reviews are not insignificant and it is essential to get value beyond the compliance tick. Coming from the code reviews corner, let me share what code reviews involve and how to get the most of your investment while complying with section 6.6 of PCI DSS.

Separating the meat balls and the gravy in your code

If we compare a typical application to meat ball gravy, we first need to figure which code is meat ball and which code is gravy. Consider a small app with 15 to 30 thousand lines of code. At 50-100 lines per class, this application has roughly 300 classes. Using scripts that scan the code for certain signatures (authentication, encryption, input validation, authorization, credit card data and more), we can identify the right 50 or so classes which are the meat balls in this code. The rest of the code is the gravy. The meat balls are the parts of the code that are most likely to have security vulnerabilities or intentional back doors. These classes should be reviewed most intensely - scanning with proprietary tools, commercial tools and most importantly manual reviews. The balance of the ‘gravy’ classes should be scanned by both proprietary and commercial tools to complete a baseline review. Experienced application security teams can review apps with <150,000 lines in 1-2 weeks.

Larger apps (with million plus lines of code) use more technologies, and usually interface with other applications. Code reviews of such apps can be completed in 2-4 weeks usually.

Getting more bang from your code review bucks

1. Do rapid risk assessments to determine which ‘money’ application should be code reviewed (Choose the right apps)
2. Use tools / processes that easily finds the meat balls (vulnerable code classes) in your thousands or millions of lines of code (Choose the right code)
3. Analyze vulnerabilities across applications, vendors to identify recurring issues, across the board solutions and effective developer training modules (Address root causes)
4. Look at trends like software-as-a-service, outsourcing & offshore centers to reduce your code review costs (Find a good deal).

Earlier Posts

• The Standard Penetration Test | 13 Mar 2008
• We are hiring | 14 Jan 2008
• OWASP Presentation on Penetration Testing Embedded Devices | 30 Dec 2007
• Penetration Testing Wireless Embedded Systems | 18 Dec 2007
• How to enforce strong SSL/TLS ciphers in IIS | 11 Dec 2007