This is the buyer's and planning guide. It covers how to decide that you need a test, which kind, what belongs in the statement of work, how to read what comes back, and how to close findings. It is not the testing methodology. How a test is performed, including how OWASP testing guidance is applied to a SaaS application, is covered in Penetration Testing Methodology for SaaS. Use that page to judge whether a provider's method is real. Use this page to scope the work and to judge the result.
A penetration test is a time-boxed, authorized attempt to break the controls you claim to have, done by a person who will try chains a scanner will not. A vulnerability scan is a useful input to that work and a poor substitute for it. If the report is a scanner export with a cover page, you bought a scan. SOC 2, HIPAA, and ISO 27001 assessments commonly expect a recent test as evidence that technical controls work. PCI DSS is the one that names penetration testing in the standard. None of them are satisfied by a tool name in a policy. An auditor or a customer will ask what you tested, when, what you found, and what you fixed.
Name the question the test is supposed to answer before you ask for a quote. "What can an outsider reach" is a different purchase from "can one customer read another's data" and from "what can someone on the office network do." A statement of work that only says "penetration test" will be answered with whatever the provider does by default.
Commission a network test when you have address space, VPNs, office networks, or exposed management services you cannot fully describe from a cloud inventory. An external test answers what an outsider can reach. An internal test answers what happens after a laptop or another foothold is inside. You want the internal test when the threat you are planning for is a phished employee or a compromised build agent. You can skip a wide network sweep when the application is the whole attack surface and the network is a default cloud network you already review as configuration. Say that in the scope so you are not billed for a sweep you did not need.
Commission an application test when customers authenticate and change data. For most SaaS products this is the test that matters. Include the roles you actually sell, at least a normal user and an administrator, and the workflows that move money, records, or permissions. A test of the marketing site is not a test of the product. Unauthenticated testing finds the front door. Authenticated testing finds broken access control, which is the class of issue that matters once the front door is ordinary. If a role is out of scope, the report should say the product was not tested as that role.
Commission API testing when mobile apps, partners, or other servers call an interface that is not just the browser talking to its own backend. Fold it into the application test when it is the same product. Split it out when partners have their own credentials, scopes, and limits. Give the tester the real client authentication flow, not one long-lived token you would never ship. If the API is the product, a test that only drives the web interface will miss it. Do not let the report imply the API was tested when the statement of work never included it.
Commission a cloud review when the account, the identity plane, storage, and the deployment pipeline sit inside the boundary customers are trusting. This is the pass that looks for a storage location opened to the world, a role that can assume far more than its job needs, a snapshot shared with another account, or a pipeline that can deploy to production with no second person. Keep it distinct from the application test unless the statement of work gives the same people enough time to do both. An application tester who is also expected to review the organization trail in the same short window will do one of them badly.
Commission social engineering when the question is whether people will hand over a credential, or whether mail and identity controls notice a phish. It is a different engagement from a technical test. It needs its own authorization, a defined group of people, a rule for how far a successful attempt is allowed to go, and a plan for telling the people who were targeted. Do not hide it in a footnote on an application test. Leave it out entirely when the only goal is a technical finding list for a questionnaire. Targeting your own staff without written scope is how a useful exercise becomes an incident with your own employees.
Commission an assessment aimed at the model when the product sends untrusted text to a system that can see customer data, call tools, or take actions. Prompt injection, leakage of data placed in the prompt or the training set, and unsafe actions from tool use are different tests from injection into a database query. Put them in the same statement of work only when the provider actually does that work. Otherwise schedule them as a follow-on, and do not let a web-only report imply the model was tested. If the model is a demo with no data and no tools, say that and spend the time on the application.
The labels describe how much the tester is allowed to know and use at the start. Pick one on purpose. Do not let the proposal pick it by omitting it.
For a product test, gray box plus an explicit unauthenticated slice is the usual fit. Choose black box when an outsider with no account is the threat. Choose white box when missing a deep flaw costs more than the access you are granting. Write the choice into the statement of work so the report cannot quietly become a skim of the login page on a product that needed a logged-in test.
Scope is a list, not a slogan. Name the environments, production or staging or both, the hostnames and accounts, the applications and APIs, the cloud accounts, and the identity providers. Name what is out: third parties you have no permission to test, corporate systems owned by another team, availability attacks, and any data the tester must not take away even as proof. If production is in scope, write what the tester may do there. A common split is limited proof against production and deeper work against staging. That split is only honest if staging enforces the same authorization as production. A staging environment with the checks turned off will waste the engagement.
Give the tester accounts you control and can revoke. Do not hand over a shared personal login. Decide whether the data they may see has to be synthetic. If production holds customer data, the rules should forbid taking that data off the system, and the report should show impact without pasting a customer's record into a document that will be forwarded around.
Time-box the work in writing. A short test of a large product follows the tester's habits, not your risks. If you cannot cover the whole product, pick the workflows that move money, data, or privilege and list the rest as not tested. An honest gap is more useful than an implied clean bill of health.
Rules of engagement are the document that keeps a test from becoming an incident you did not intend. Agree them before any testing traffic is sent.
Annual testing is the minimum for a product that customers and auditors ask about. Test again when the thing that was tested changes in a way the old report does not cover: a new authentication model, a new boundary between tenants, a new API exposed to partners, a move into a new cloud account, or a feature that handles a more sensitive class of data. Quarterly testing is the right extra cadence for a high-risk application, not a ritual to apply to every internal tool. A report about last year's architecture does not describe this year's release.
Repeating a full test of the whole product on a tight cadence is not automatically a better program. Spend the extra cycles on what changed and on the findings that were serious, and keep a lighter check, a scan plus a review of the diff, for the periods where the attack surface did not move. Write that policy down so "we test annually" is a decision you revisit. If cardholder data is in scope, follow the testing cadence PCI DSS sets for that environment, including a test after significant changes. Do not let a generic annual SaaS questionnaire schedule replace it.
Run the test when someone can fix what it finds. A report that arrives during a release freeze, or during the week an auditor shows up, becomes a PDF on a shelf. Put remediation time in the same plan as the test, before you sign.
Ask who will do the work on this engagement, and whether those are the people whose sample you are reading. A strong sales conversation and a junior-only delivery are different products. Ask for a redacted report from a similar scope and read it with the section below before you sign. Ask which methodology they follow. The methodology guide linked above is the checklist for that answer. A provider who cannot name a method, or who names only a scanner, is selling a scan.
Ask what happens after the report. A retest of the findings you fix should be either included or a priced line you can see, not a surprise. Ask how severity is assigned, and whether each finding comes with steps a developer can follow. Ask how your data and the report will be stored, and for how long. Ask whether the people doing the work hold a hands-on credential your customers already ask about. Buyers often ask about OSCP, and for some web or regulated scopes about OSWE, CREST, or GIAC credentials. Treat the letters as a filter. They do not tell you the report will be useful. A credential with no sample report is an incomplete pitch. A strong sample still needs a named person attached to this engagement.
Prefer a provider who will decline work. If the scope is too wide for the time, or if you asked for model testing and they do not do it, you want that said in the proposal. A proposal that accepts every line and asks nothing has not looked at your system yet.
Read the report against the statement of work before you read the findings. The opening should say what was in scope, what was out, which accounts were used, when the work ran, and what was not tested. If API testing was in the contract and the report never mentions the API, that part of the test did not happen. If the summary sounds calm but the method section describes only an automated scan, believe the method section.
Each finding should say where it is, what someone could do with it, how the tester showed it, how severe it is in your environment, and what to change. A finding without reproduction steps is a hypothesis your engineers cannot confirm or close. A severity with no note on exposure will be prioritized wrong: an issue on an internet-facing admin path is not the same work as the same issue behind a role you already restrict. If every item is marked critical, the scale is not helping you. Critical should mean it moves ahead of ordinary feature work.
Look for what was tried and did not succeed. A report that only lists failures, with no statement of which controls held, is harder to use in a questionnaire and easier to misread as a shallow look. You can ask for that coverage statement. It belongs in the deliverable.
Split issues you will fix from issues you will accept. Notes at the bottom of the report are not automatically free work, and they are not automatically ignore. Decide, name an owner, and write the acceptance down if you are not fixing it. An accepted item with no owner returns at the next audit as if you had never read the report.
Watch for leftovers that should not be in the document: live session tokens, customer records used as proof, internal names you did not want in a file that prospects will see. Send it back. Impact can be shown without shipping the secret.
Triage by what an attacker could do and how exposed the path is, not by the default rank a scanner printed. A missing header on a static marketing page is not the same work as a tenant boundary that fails. Put the fixes in the same tracker as product work, with the report's identifier on the ticket. Findings that live only in a PDF get lost.
Fix, then retest that finding. A retest checks that the specific issue is gone and that the obvious bypass is gone. It is not a new full test, and it should not be described as one. Do it while the original tester still knows the path. A retest long after the fact, against code that has since moved, is a new engagement. Put the retest date on the plan before the report is delivered, so remediation time exists instead of being negotiated later.
Keep the three records an auditor will ask for: the original finding, the change that addressed it, and the retest result. The original report shows that you knew. The retest shows that you acted. If you accept a finding instead of fixing it, record who accepted it, why, and when the acceptance expires. Accepted should not mean forgotten.
When a retest fails, treat that as information about the fix. The usual causes are a change in one code path and not the other, a control that applies to one role only, or a deploy that never reached the environment that was tested. Ask which of those the tester hit, and fix that cause rather than arguing the original severity.
You should be able to point at the question the test answers, the environments and roles in scope, the level of access the tester starts with, the rules of engagement, the cadence, the named people, the retest, and the internal owner who will turn findings into tickets. If one of those is missing, the proposal is not ready to sign. Add it, or reduce the claim you plan to make to customers about what was tested.
If you want a second set of eyes on a scope, a statement of work, or a report you already have, Book a Free Assessment.
A comprehensive guide to penetration testing for SaaS companies. Covers network, application, API, cloud, and AI security testing approaches.