I was working through an assessment when I reached a specific data-handling requirement: whether certain information could be deleted when required.
There was plenty of evidence in the file.
I had independent assurance, certifications, testing results, questionnaires, and other supporting documentation. Overall, it was a strong evidence package.
But I couldn’t simply send every unresolved question back to the third party. For this requirement, I had to work with what was already available.
So I went back through the evidence looking for something very specific: what actually established whether the deletion requirement was met?
That’s where I got stuck.
Some of the evidence gave me confidence in the organization’s broader security and governance environment. An audit report could tell me about the control environment, testing could tell me about technical weaknesses, and a certification could show that a management system had been independently evaluated.
All of that was useful.
But when I brought the review back to the deletion requirement, most of it didn’t answer the question in front of me.
If I’m trying to establish whether information can be deleted under a specific condition, evidence showing how that information is encrypted, monitored, or otherwise protected may tell me useful things about data security.
It doesn’t establish deletion.
A strong evidence package can make this easy to miss. The package creates confidence, and some of that confidence is justified. The problem comes when I carry that confidence into a requirement the evidence never actually addressed.
Most of the evidence in this assessment was still useful. It just didn’t resolve the deletion requirement.
For that requirement, I could only conclude:
Based on the information available, the requirement was not validated.
I couldn’t say the control had failed. I also couldn’t make a broader judgment about the third party’s security based on one unresolved requirement.
The evidence didn’t support either conclusion.
This assessment changed how I think about evidence completeness.
A file can contain a large amount of credible documentation and still leave individual requirements unresolved. So I’ve become less interested in whether an assessment simply has “enough evidence” and more interested in whether I can trace a conclusion back to something that actually supports it.
When I’m reviewing an artifact now, I try to keep the requirement in front of me instead of letting the strength of the overall package do the thinking for me.
An artifact may confirm the requirement, answer only part of it, or leave it unresolved. Even when the broader evidence package is strong, the most defensible outcome for a specific requirement may still be that it remains unvalidated.
That doesn’t diminish everything else in the file. It just sets a limit on what I can reasonably conclude from it.
I already had a good evidence package in this assessment.
What I didn’t have was support for this particular conclusion, and I couldn’t borrow confidence from everything else in the file to fill that gap.
Which conclusions are you trusting because the evidence package looks strong, rather than because the evidence actually proves them?