Security 4 Nø0bs — The 4–1–1 on Tests & Testing
Security 4 Nø0bs — The 4–1–1 on Tests & Testing
What the scientific method can teach us about testing
Hola Friends,
Up to this point in my Security 4 Nø0bs series, we’ve discuss scope, risk, understanding the rules of engagement, and the cardinal requirement of getting permission to proceed. If you’re new to the series, or have not caught up, I invite you to check these out. Comments are always welcomed.
This week, I want to discuss the purpose of what we are testing and why. The question to ask is are we even testing the right thing?
The motive for this post comes from a recent reddit post where OP stated they had an explicit scenario to test but weren’t sure about approach. Based on their post, it was clear to me that they were not addressing the principle issue nor were they asking the right questions. This lead me down a longer path and what it means to test. Let’s discuss!
Definition of Testing
What is a test?
Before we dig in, we have to define what a test even is. For that answer we can look to the scientific method.
The scientific method¹ takes a hypothesis as the prompt for investigation. Then, through observation and deductive reasoning, a conclusion is drawn.
A test is the methodical step-by-step process that serves to interrogate the subject being observed, hoping to gain more clarity and insight into it’s constitution. In other words, we can break this down to the following:
- Input — they hypothesis, question, or assumption to challenge (the subject).
- Process — placing the subject under an intense level of scrutiny, free of bias or influence, to arrive at a better level of understanding about it (the tests).
- Output — the summation and or conclusion (the results) about the subject as it relates to the hypothesis presented.
Another way to look at what a test is, or what testings is in general, comes from a premiere voice in the software testing community — Michael Bolton of “Rapid Software Testing” ²:
“Testing is acquiring competence, motivation, and credibility for creating the conditions necessary for evaluating a product through experiencing, exploring, and experimenting … so we can help our clients make informed decisions about risk.”
We are given the criteria that defines the success of the feature, then through testing we challenge that definition of success; the assertion that it works in this manner begets the question, does it? Let’s find out!!
How this applies to Security
As the name implies, the software testing process applies seamlessly to security testing (ie, a penetration test) in so many ways, starting with a given hypothesis like:
- This form takes user data and transmits it without risk of leaking PII (personally identifiable information)
From this hypothesis, we are aware of what the scope and rule of engagement is:
- Scope is to stay within the bounds of the form (url, backend requests, etc.)
- ROE — limit brute-force attacks and injections of all kinds to this form. Raise concerns to the CTO if RCE or other such compromise occurs.
We’ll take the form inputs (name, mailing address, email address, telephone number, date-of-birth, etc.) and run it through a series of tests that serve to answer the question:
- Is this form secure, or can it be compromised in some way to leak sensitive PII?
We can apply the same “inspection” efforts to API, Network, Mobile and every other domain with the expressed purpose of determining security fitness for use. As stated earlier, by assuring the security of a feature, function, or application, we help clients make informed decisions about risk.
And through our reporting, we present the impact of our findings so they can make a determination on how best to proceed. I go into more details about risk tolerance, risk acceptance, and so on in my post, The Rewards of Risk.
Security Testing
Types of Security Tests
We’ve discussed the “what” and “why” of a test. I won’t go into “how” as that falls outside the scope of this post (and is a whole other discussion unto itself), but I will discuss the test types in brevity:
- Vulnerability Assessment — A vulnerability assessment is the process of identifying, quantifying, and prioritizing the vulnerabilities in a system.
- Web Application Security Testing³ — Web application security testing aims to secure sensitive data, maintain system integrity, and safeguard against unauthorized access or malicious attacks. 3 known types of web application security testing include:
— Dynamic Application Security Testing (DAST).
— Static Application Security Testing (SAST).
— We Application Penetration Testing (WAPT). - Web API Security Testing⁴ — The practice of preventing and mitigating attacks that originate at the API level.
- Cloud Services Security Testing⁵— Identifying and mitigating security risks within an organization’s cloud infrastructure.
- Mobile App Security Testing⁶ — Testing for security compromise on mobile applications, not limited to the client-server architecture or server-side APIs.
- Network Infrastructure Testing⁷— Evaluating and validating the hardware, software, network, and other components that make up a software product’s infrastructure
- Wireless Testing⁸ — ensures the functionality, security, and performance of wireless communication systems and devices.
- Social Engineering Testing⁹ — Assesses the human element in protecting systems.
- Ransomware Testing¹⁰ — Ransomware-specific penetration testing services determine which systems and data ransomware attacks are most likely to target and fairly evaluate an organization’s ability to defend against ransomware, including its ability to recover from a successful ransomware attack.
It is important to understand the right test for the right engagement.
Definition of Done
We are at the most critical part of the test composition — the results. Up to this point we learned:
- The AC (acceptance criteria) stated the form did not disclose sensitive information. However through rigorous testing, it was discovered that not only was the form disclosing PII, but that it was susceptible to an XSS attack and we were able to gain a foothold via url path traversal.
- We also learned the form did not did not employ multi-factor authentication or some sort of CAPTCHA to prevent automated attacks.
How do we know testing is done?
Testing is never really done … but it can be determined that the first round of testing is completed when:
- Full scope of work is achieved.
- All tests for the feature or application have been employed to their fullest.
- All areas under test have been addressed and inspected/evaluated for any security concerns.
- A vulnerability assessment was completed and a report drawn up to support additional security testing efforts.
Next step is to communicate these findings in a clear and concise manner. The Report is the single most important output of any successful testing engagement. Testing for this first phase is done when the report is delivered to the client and follow-up conversations have been had.
The two most important elements of the Pen Test Report are the executive summary and the technical summary.
By way of the Executive Summary, you are communicating in a high-level manner what testing you performed, what you found, why it’s a problem, and what action to take.
The Technical Summary gives a more in-depth look at what the vulnerabilities were, their impact, how to reproduce, and how to remediate.
We’ll save the discussion of what the actual pen test report looks like for another time. There are more pieces to the report that are just as important, but that’s out of scope for this post.
Upon the completion of testing and submission of the report, if it is within the parameters of the contract that found issues are to be retested, then an additional phase of testing will occur for re-testing.
We can declare testing to be 100% done when the engagement has been completed, all found issues having been remediated, and no new issues uncovered.
Conclusion
In this post we’ve defined what a test actually is as it relates to the process of understanding the feature, functions, or application.
We learned that the discipline of testing is rooted in the scientific method.
We learned that a test serves to challenge they hypothesis given and, through thorough investigation, arrive at a better understanding.
We learned that the most import output of a test is the test report, with a summary of findings written in a specific manner based on the audience the report is meant for.
And we learned testing is never truly done, but rather completed when a specific set of conditions and obligations are met.
That’s all I have for today! Feel free to follow and comment.
Next week, we’ll take a look at the pen testing process, starting with my favorite step — reconnaissance.
Until next time,
ciao for now
- https://en.wikipedia.org/wiki/Scientific_method
- https://developsense.com/blog/2022/11/talking-about-testing
- https://www.rapid7.com/fundamentals/web-application-security-testing/
- https://www.postman.com/api-platform/api-security/
- https://www.crowdstrike.com/cybersecurity-101/cloud-security/cloud-security-assessment/
- https://mas.owasp.org/MASTG/General/0x04b-Mobile-App-Security-Testing/
- https://www.pentestpeople.com/blog-posts/guide-to-infrastructure-testing
- https://www.tessolve.com/blogs/a-comprehensive-guide-to-wireless-testing-strategies/
- https://www.stationx.net/social-engineering-penetration-testing/
- https://www.packetlabs.net/posts/guide-to-ransomware-penetration-testing/