Regulated firms have mature frameworks for third-party risk. Applied to an AI system, those frameworks ask the wrong questions. This is one of the challenges to be tackled in our next working session.
The double standard
Regulated firms routinely subject their software suppliers to detailed due diligence. ISO 27001 certification. Data residency questionnaires. Contractual assurances on processing, storage and sub-processing. Security reviews with named owners and renewal dates.
The same firms have, in many cases, granted access to large language models across their entire organisation on the basis of a public terms of service and a loosely worded enterprise agreement. The founders behind many AI platforms operate with a move-fast culture that sits fundamentally at odds with that of a regulated financial institution, and yet the scrutiny applied to them bears no resemblance to what is applied to any other third-party suppliers.
The question that follows is not rhetorical. Why aren’t we asking the same questions?
The regulatory position is unambiguous. Regulators do not regulate the technology. They regulate the firm. Consumer Duty and SMCR place responsibility squarely on the regulated entity to demonstrate compliance, regardless of what a provider’s terms say or do not say.
Where AI due diligence differs from vendor due diligence
A conventional third-party assessment establishes whether a supplier handles your data appropriately. An AI assessment has to do that and four other things.
It has to separate what the vendor can evidence from what the vendor asserts. Certification, audit logs and test results are evidence. A statement in a contract or sales deck is not.
It has to address behaviour after deployment. A model that performed acceptably at procurement may not perform acceptably in six months. Drift detection and the monitoring obligation belong in the assessment, not in a later review cycle.
It has to establish explainability sufficient for an external audience. The relevant test is not whether an internal committee is satisfied. It is whether a decision can be explained in terms a senior manager could relay to a regulator.
And it has to locate accountability precisely when the model belongs to someone else. Not who is responsible for the system, but who is answerable for a specific decision the system made on a specific day.
What a KYA framework should cover
A formal assessment completed before access is granted, applied consistently rather than case by case, covering PII handling, data residency, training practices, audit trail capability, certifications and contractual protections. The naming is deliberate: Know Your AI, echoing the KYC discipline every regulated firm already runs competently.
You are not relying on an assurance. You are keeping the data away from the model, and you are generating operational data that can be used as evidence.
A practical, reusable question set for AI and model vendor assessment does not currently exist in a form that firms can readily adopt, suitable for use by compliance teams, product owners and procurement functions. Building one is the agenda for the next working session of the AI & Data Leadership Forum, hosted at the FinTech-Tables Frontier AI Leadership Summit on 21st October, 2026.
If you would like to take part in this session you can register your interest here:
READ THE REPORT
FinTech-Tables is convening the follow-on session on KYA and AI due diligence at the Frontier AI Leadership Summit on Wednesday 21 October 2026.
This article draws on the AI Capability Building working session, convened by FinTech-Tables with Keeper Solutions and Higgs LLP. The full report sets out the keynotes, the three working group findings, the five key challenges and the ten takeaways in full.
