How to review a vendor's SOC 2 report, and spot a worthless one

Most vendor risk programs collect SOC 2 reports the way some people collect receipts: carefully filed, never looked at. The Delve allegations in March showed what that habit costs. Hundreds of companies accepted near identical reports because nobody compared two of them.

A vendor's SOC 2 report is evidence only if someone at your company reads the parts that matter, and those parts take about twenty minutes. If you don't have twenty minutes for a vendor holding your customer data, you have a bigger problem than the report.

I've reviewed vendor reports for regulated companies and prepared vendors for their own audits, so I've seen both what auditors put in these reports and what buyers skip.

What's in a SOC 2 report

Section 1 is the auditor's opinion: unqualified is what you want; qualified, adverse or a disclaimer means read on carefully. Section 2 is management's assertion. Section 3 is the system description, which defines scope. Section 4 lists each control, the auditor's test and the result, including exceptions. Some reports have a Section 5 with management's responses.

If you've ever checked the date on page one, seen the word "unqualified" and filed it, you've read about 2% of the report.

The seven red flags

Type I only, and old. A Type I report shows control design on a single date. A Type II shows operation over a period. A Type I from eighteen months ago tells you very little about today.

An auditor you can't verify. SOC 2 reports must come from a licensed CPA firm. Look the firm up on CPAverify. If you can't find it, ask the vendor why.

Scope that skips your product. Section 3 defines which systems, locations and services were examined. Vendors sometimes audit one product line and present the report for all of them.

Carved out subservice organisations. Most vendors rely on cloud providers, and the report usually carves them out, meaning they weren't tested. The report then lists complementary user entity controls, things you as the customer must do for the vendor's controls to work. If nobody at your company has read that list, part of the security model depends on controls you don't know you own.

Unread exceptions. Section 4 records where a control failed during testing. Some exceptions are trivial. Some, like terminated users keeping access for months, are exactly what you were trying to rule out. Search the PDF for "exception" and "deviation".

Stale period, no bridge letter. A report covers a period that ended in the past. If it ended more than a few months ago, ask for a bridge letter from management covering the gap.

Copy and paste descriptions. Compare a report against another vendor's. If the control descriptions, system narrative or even the typos match, you've learned something about the audit.

After the review

Write down what you checked and what you found, map any complementary controls to owners on your side, ask the vendor about exceptions that matter to you, and set a date for the next review. A vendor review that leaves no record didn't happen, as far as your own auditor is concerned.

In Steppe a vendor review is a dated record linked to the vendor, the data it touches and the next review date, so this doesn't depend on someone remembering. A spreadsheet can do it too, if somebody owns it.

A SOC 2 report is evidence only if somebody reads it, and that somebody should work for you.