Security 4 Nø0bs — Know Thy Scope!
Security 4 Nø0bs — Know Thy Scope!
He Who Has A Why To Live Can Bear Almost Any How — Friedrich Nietzsche
Hola Friends,
Kicking off the first (of many) posts in a series titled “Security 4 Nø0bs” where I will present all the lessons I’ve learned — as a newb — in web application security testing, and how I’ve applied my experience in quality assurance towards this new endeavor. Security truly puts the wind in my sails and I can’t imagine doing anything else.
To start, I’d like to share a small anecdote that got me thinking about the value of scope as it relates to projects, testing, and everything in between. If you are new to testing, or can relate, this article is for you. Nothing grinds my gears more than filing a great bug only to have it declared out of scope! When clearly it is!! But I digress …
In my neighborhood, there’s a major highway that stretches from the city running parallel to the Hudson River, past my building, and on through to the rest of New York. The highway has a few restrictions including a height limit. Some overpasses, like the one in front of my building, have an eleven-foot clearance. It never fails that, at least once a month, a truck having a twelve-foot (or higher) trailer will find itself about to get stuck, or actually get stuck and jam up the highway for a bit. Had the driver known this height restriction ahead of time they might have avoided the issue. And that’s when it hit me … scope is everything!
Scope defined
As someone who had to learn software testing the hard way, and is now applying that learning towards security, the most paramount bit of information that will determine the success of the project or testing endeavor is the scope.
Scope is the “defined features and functions of a product (1)”. The value is in information gathering for the features or functions of the project. The breakdown might include things like obtaining client information, understanding risk, timeline, resources, and success criteria.
Notice in the illustration above, scope is the largest of all the “pieces” of the project. The Project Manager, Product Owner, Tech Lead and everyone else involved set the parameters for scope as it relates to:
- Scope of Work — what the client expects and/or what is to be delivered
This can be further broken down to things like:
- Scope of Design & Development — what the designers will achieve; what the developers will do, how, and when
- Scope of Testing — what the QA & Security Team will do, how, and when
Rules of Engagement
A crucial component to scope involves the rules of engagement (ROE) — the parameters or bounds for the type of work to be performed. As it relates to testing, having an explicit ROE can mean the difference between establishing a successful testing campaign or violating the terms and conditions of the contract and facing legal issues. Consider the following scenario:
A Security Analyst is assigned to a project where they are tasked with performing a security assessment on a target site. The ROE states that testing is to be minimally intrusive, but also thorough. Security Analyst proceeds with the assignment and manages to take down the network for an extended period of time, impacting the day-to-day operations.
As a tester (Front-end, Back-end, Security, etc.) you want to know the bounds of the testing effort — namely what is permissible — and have a clearly defined plan for what is to be testing, when, and how to communicate found issues. It’s crucial to know the point-of-contact in case a serious issue is found or testing is impacted in some way.
Risk
Risk analysis is part of the scoping process that can determine what features are business-critical and which can be prioritized based on impact to the system caused by an event. I won’t delve too much into discussing risk as falls outside the bounds of this topic. The key take-away is this: it is important to have a clear understanding of what features are critical to best address these first and strategize accordingly. When testing said features, and a non-conforming requirement (aka bug!) is found, it can be triaged properly. If remote code execution is achieved by way of a shell, or data is exfiltrated, knowing the risk will help communicate the urgency of getting this feature patched.
Features & Deliverables
I remember working on a project once, some time ago, where we were on task with what we were supposed to deliver to the client from a testing perspective. Then it was discovered that key deliverables expressed in the SOW were not yet realized. A temporary win became a critical loss. The team scrambled to review what was “owed” and make haste with the work. It was another example of knowing the scope of work (SOW), not just the ROE, but also the work breakdown, schedule, and deliverables.
In the SOW, the breakdown of features to be delivered might include statements of proposed work, schedules for start/end, test effort, etc. I won’t delve into this topic either because it would deviate from the point I’m making and is relevant to the industry. A construction Scope-of-work will differ from an IT Scope-of-work, and what I’m stating might not apply.
For the purposes of this discussion, as it relates to security testing (or testing in general), we can regard the most valuable piece of information will be things that we want to hone in on, so as to stay within the bounds of what is expected. Things like:
- Target urls / IP addresses
- API endpoints
- Credentials (if applicable)
- System / Network configurations
… and so on (not an exhaustive list by any means)
Project deliverables can mean anything from the design comps, the code, test artifacts, or pen testing report. It all comes down to what is most valuable to the success of the project.
Get Permission / Sign-Off!
If you learned nothing else from this post, know this: when it comes to the overall project engagement or testing (of any kind), get the sign-off / permission.
Concerning Security testing, it is vital to get the permission from the client to proceed with testing. Once the scope has been defined, the ROE established, Risk determined, and target acquired, it is critical that as a security analyst, you get the permission to proceed.
Permission gives you — the Security Analyst — authority to interact with the system as an “attacker”. Permission acknowledges you know the scope and the bounds you are operating under. Permission establishes a trust between you and the client. Permission communicates agreement between yourself and the Client:
- You agree to act within the bounds of the ROE, escalating any data exposure or RCE / p0s immediately.
- Client agrees to let you have carde blanche with the system, operating at the predefined time, with the given credentials, as expressed in the SOW.
Challenging the “out of scope” perspective
As stated before, nothing boils my blood like a well-defined bug, with succinct steps and evidence being declared out of scope, even when everything presented states otherwise. If you know, you know!
Having a clear understanding of what the scope is will help you defend your reported issues. You can leverage statements in the scope to support your finding. If you know part of testing business logic involves testing the password strength, and you report that the Client is not compliant with password best practices, you can defend the moderation of the ticket — rejection — by pointing out that password policy is in scope because it aligns with the ROE.
Bug bounties are chronic for dismissing reported issues as being out of scope. Some other freelance testing sites operate in a similar manner. There can be a number of reasons for “why” a finding is declared out of scope, and it helps the tester have a full understanding of what the scope is, operating within the bounds of that scope, and presenting the evidence that supports why this bug is in scope.
Conclusion
Like our truck driver, it is important to have a full understanding of scope ahead of time to avoid issues (like getting stuck in an overpass) and staying within the bounds of what the engagement dictates. Knowing the “why” of the scope, and the goals thereof, can drive the engagement towards a guaranteed success.
It is imperative that you understand the ROE. Knowing what to test, how to test, and when to test will go a long way to ensuring you stay in scope. You are testing the features and/or functions expected of you, and you will not stray from your objectives.
Lastly, and I can’t stress this enough … GET PERMISSION! Before attempting any sort of security engagement (vulnerability assessment / pen testing) it is critical that you get the permission from the Client to proceed with your testing activities.
That’s all I got for now. Tune in next week where I’ll be discussing requirements gathering as part of a Security audit.
ciao for now