Guide to Plynt Certification Criteria

What’s the logic behind the Plynt Certification Criteria ?
My application faces threats that you don’t specify in the criteria. How will you address those?
Why doesn’t the criteria talk about SQL Injection, Cross Site Scripting etc.?
You missed criterion xyz!
What do you mean by sensitive data in the criteria?
The criteria says that the application must not allow passwords to be stolen even after the user logs out. Am I missing something?
An application must re-authenticate the user after log out – but then, which app doesn’t do that?
You don’t mention SSL anywhere in the criteria … gotcha!
What’s with the “No sensitive data in requests to external sites”?
No sensitive data in source code available to users; ok, but how will users access any source code of a web application?
Why only these criteria? Does this cover everything?
Why a new version of the criteria?

What’s the logic behind the Plynt Certification Criteria?

The certification criteria specifies the standards an application must meet to establish that it has adequate security measures. An application must resist attacks on it as well as implement security features that enhance its security. Accordingly, the certification criteria includes two sets: the first, “Security Protection Criteria” in Section I defines threats the application must protect against. In Section II, “Security Requirements Criteria” specifies the features and characteristics of the application that enhance its security.

My application faces threats that you don’t specify in the criteria. How will you address those?

It’s quite likely that your application faces threats that are specific to its business context: for instance, a banking application needs to protect against the threat of funds being siphoned off, and a gaming application needs to defend against the rules of the game being violated. The criteria require the application to protect against these threats in the second criterion: defend against threat profile. The threat profile is the listing of all threats relevant to the application. Before the certificate is issued, the application must demonstrate through testing that it defends against these threats. The threat profile is developed by the Plynt team with inputs from the application owner; the security engineers then develop test cases to verify that the application is safe against those threats.

Why doesn’t the criteria talk about SQL Injection, Cross Site Scripting etc.?

That’s a good observation! When writing the criteria, the authors first considered listing all the attacks they wanted the application tested for. They soon discovered the pitfalls of that approach: the criteria would have to be revised with the discovery of every new attack, and there was no standard taxonomy of attacks to start with, anyway. Look at criterion 1: “Safe against popular attacks”, that’s the criterion which catches many of those attacks. Here’re some of the popular attacks the application must defend against:

Variable Manipulation	Denial of Service	Brute Force
Bypass input validation	Session Hijacking	Session Fixation
Content Spoofing	Cross-site Scripting	Cross Site Tracing
Buffer Overflow	Format String Attack	SQL Injection
OS Command Injection	LDAP Injection	XPath Injection
SSI Injection	Blind Injection attacks	Directory Indexing
Directory Traversal	Authentication Bypass	Filter evasion

You missed criterion xyz!

Thank you for pointing out. We are constantly looking at ways to improve the criteria – so your suggestions will help. While not all suggestions might make it to the criteria after evaluation, they ensure that the criteria gets the best ideas for certifying applications.

What do you mean by sensitive data in the criteria?

Sensitive data is any data that if compromised could lead to:

Loss of business in financial or reputation terms
Loss of privacy of individuals
Adversaries gaining an advantage to launch attacks

Sensitive data includes business sensitive information, authentication credentials, session token(s), authentication token(s) and any details regarding application architecture that could aid an adversary launch a successful attack against the application.

The criteria say that the application must not allow passwords to be stolen even after the user logs out. Am I missing something?

In general, the application must safeguard passwords. The note highlights that the password should be protected even when the user has logged out. Vulnerabilities in some authentication schemes enable the password to be stolen if a user who has logged out leaves the browser window open. This is described in the Paladion paper "Stealing Passwords via Browser Refresh".

An application must re-authenticate the user after log out – but then, which app doesn’t do that?

Sounds obvious, rt.? Experience, however, shows that not all applications enforce re-authentication after log out. To make matters worse, some platforms like .Net do not invalidate a user session immediately when their signout method is called. To learn more, please read the paper "ASP.Net Forms Authentication" from Foundstone. As the note points out, if an application does not invalidate a session immediately, it should protect against the session being reused. This Microsoft support document has examples of such protection.

You don’t mention SSL anywhere in the criteria … gotcha!

The criteria doesn’t mention SSL, or any specific encryption technology: that’s intentional. The criterion “Sensitive data protected in transmission” requires that the application take adequate protection to safeguard sensitive data in transit. SSL is a good example of such protection. The criteria, however, does not demand SSL. The application is free to choose the technology, so long as it protects sensitive data in transmission.

What’s with the “No sensitive data in requests to external sites”?

This is a favorite of the authors. Links to external sites are often a source of data leakage. Along with the URL, the HTTP request also includes a referrer field that contains the URL of the page the request originates from. If that URL (and the referrer field in turn) contains sensitive data, then that data will get logged in the web server logs of the external site. Administrators of the external site have access to those logs and can collect the sensitive information trivially.

No sensitive data in source code available to users; ok, but how will users access any source code of a web application?

You forget client-side javascripts. Worse, sometimes web server mis-configurations lead to server-side source code being available to adversaries. The criterion requires that any source code that’s available to users not contain sensitive data.

Why only these criteria? Does this cover everything?

The Plynt criteria focus on the high impact threats popular today. The certificate assures that an application is safe against those threats. The authors chose to omit criteria that are not essential or do not have a material impact on security. For example, the certificate does not require that the application catch all error messages. While that’s a good practice to follow, it would have been unduly stringent for the certificate. The standard only demands that no sensitive data be revealed in error messages as that has direct impact on the application’s security.

Why a new version of the criteria?

New technologies, new threats, new attacks - we live in a changing world. The Plynt team keeps reviewing the Plynt certification criteria from time to time to ensure the criteria are addressing all the new threats. This time round we decided we needed to make some important changes. We deleted a couple of criteria:

Protect against automated password guessing
New authentication token on log in

And modified these

Protect against manual password guessing
Random session token

Protect against password guessing
Random token to track authenticated sessions

Although brute forcing sounds like a common threat, we’ve not heard of many such attacks on applications. Hence we are not mandating Protect against automated password guessing anymore. But we cannot completely ignore the threat of password guessing in cases where an application asks users to set very weak passwords like a 4 digit numeric PIN. Keeping this in mind we’ve modified the criteria Protect against manual password guessing to Protect against password guessing. The risk associated with this attack actually depends on the predictability of the usernames used. Applications with hard to guess usernames can afford to have simpler passwords. We will consider this when we rate the risk of the finding.

With the criteria New authentication token on log in we were trying to protect against a complex attack - session fixation - targeted at an individual user’s session. There is little evidence that this attack is actually prevalent. So we voted to remove it. To avoid confusion between the terms - session token and authentication token we’ve modified Random session token to Random token to track an authenticated session.