Skip to main content

06 - Compliance Toolkit: Auditing Enterprise AI Systems

A minimal kit to audit AI systems: an obligation-evidence sheet, a hash-chained event log and a disparity measure, each with its stated limits.

What do you think about this article?

06 - Compliance Toolkit: Auditing Enterprise AI Systems
AI Policy EUPhoto:Pavel DanilyukPexels

At a Glance

A minimal kit to audit AI systems: an obligation-evidence sheet, a hash-chained event log and a disparity measure, each with its stated limits.

An audit answers one question: what evidence shows each obligation is met?

Saying "my system is compliant" is not enough. Article 16, point (k), of the AI Act obliges the provider of a high-risk system, upon a reasoned request of a national competent authority, to demonstrate the conformity of the system with the requirements of Section 2 of Chapter III; the GDPR, with the accountability of Article 5(2), asks the same for personal data. An audit, internal or by third parties, is the way to build the evidence in advance: for each obligation, a document or a piece of technical evidence that can be shown.

This article does not present a product. It builds a minimal kit of three programs that you can run and adapt: an obligation-evidence sheet that can be turned into a pipeline check, an event log with a hash chain that makes tampering visible, and a calculation of disparity between groups. Then it says what existing open-source tools do and do not do. It is the last article of the series: the requirements come from the previous ones, in particular from the first on high risk and the fifth on personal data.

From the rule to the evidence sheet

The requirements of Articles 9 to 15 are seven, and each translates into an artefact that an auditor can ask for: the documented risk management system (Article 9), data governance (10), technical documentation (11), automatic recording of events (12), instructions for use for deployers (13), human oversight measures (14), results on accuracy, robustness and cybersecurity (15). The listing puts the articles, the obligation in words and the path of the file that proves it into a table, then checks which files exist.

"""Obligations -> evidence sheet. The list is an example for an Annex III system;
the articles are those of Regulation (EU) 2024/1689, consolidated text of 27.7.2026."""
OBBLIGHI = [
    ("art. 9",  "documented risk management system",   "docs/risks.md"),
    ("art. 10", "examination of biases in the data",   "docs/data.md"),
    ("art. 11", "technical documentation",             "docs/technical-file.md"),
    ("art. 12", "automatic recording of events",       "log/events.jsonl"),
    ("art. 13", "instructions for use for deployers",  "docs/instructions.md"),
    ("art. 14", "human oversight",                     "docs/oversight.md"),
    ("art. 15", "accuracy, robustness, cybersecurity", "docs/tests.md"),
]
PRESENTI = {"docs/risks.md", "docs/data.md", "log/events.jsonl", "docs/tests.md"}

mancanti = 0
for articolo, obbligo, prova in OBBLIGHI:
    ok = prova in PRESENTI
    mancanti += not ok
    print(f"{'OK     ' if ok else 'MISSING'} {articolo:<8} {obbligo:<40} {prova}")
print(f"{mancanti} missing pieces of evidence out of {len(OBBLIGHI)}")
raise SystemExit(1 if mancanti else 0)   # a CI pipeline stops if a piece of evidence is missing

# Output (python3 -I, Python 3.14.4):
# OK      art. 9   documented risk management system        docs/risks.md
# OK      art. 10  examination of biases in the data        docs/data.md
# MISSING art. 11  technical documentation                  docs/technical-file.md
# OK      art. 12  automatic recording of events            log/events.jsonl
# MISSING art. 13  instructions for use for deployers       docs/instructions.md
# MISSING art. 14  human oversight                          docs/oversight.md
# OK      art. 15  accuracy, robustness, cybersecurity      docs/tests.md
# 3 missing pieces of evidence out of 7

The structure is a list of tuples (article, obligation, evidence) and a set PRESENTI ("present") that here simulates the files found; in a real repository you replace it with an existence check on disk, and better still with a content check (for example, that the risk document has a review date). The last line makes the program exit with code 1 when a piece of evidence is missing, which is what is needed to stop a pipeline: in the reported run the output is 3 missing pieces of evidence out of 7, so the process ends with an error. The list is an example for an Annex III system and does not replace reading the article: for a deployer the obligations are different (Article 26: use according to the instructions, oversight assigned to competent persons, keeping the logs, information to the persons concerned) and the same scheme works with a different table.

A sheet like this has two limits to declare. It shows the existence of a document, not its quality: an empty risks.md file would pass. And it does not cover the provider's obligations that are not documents, such as the conformity assessment (for points 2 to 8 of Annex III based on the internal control of Annex VI, Article 43), the EU declaration of conformity (Article 47), the CE marking (Article 48) and the registration in the EU database (Article 49). Add them as separate rows, with a responsible person.

A log that makes tampering visible

Article 12 requires that high-risk systems technically allow for the automatic recording of events over the lifetime of the system, with a level of traceability appropriate to identify risk situations, facilitate post-market monitoring and monitor the operation. It does not say how, nor does it ask for immutability. If you want the log to serve as evidence, though, you must be able to say that nobody rewrote it afterwards. A hash chain makes it detectable: each entry includes the hash of the previous one.

"""Hash-chained log: every entry includes the hash of the previous one.
It makes tampering visible, it does not prevent it (Art. 12, AI Act: automatic recording of events)."""
import hashlib, json

def impronta(voce, precedente):
    corpo = json.dumps(voce, sort_keys=True, ensure_ascii=False)
    return hashlib.sha256((precedente + corpo).encode()).hexdigest()

class Registro:
    def __init__(self):
        self.voci = []   # list of (entry, hash)

    def aggiungi(self, voce):
        precedente = self.voci[-1][1] if self.voci else "0" * 64
        self.voci.append((voce, impronta(voce, precedente)))

    def verifica(self):
        precedente = "0" * 64
        for posizione, (voce, h) in enumerate(self.voci):
            if impronta(voce, precedente) != h:
                return posizione
            precedente = h
        return None

r = Registro()
r.aggiungi({"event": "input", "request": "r-001", "model": "v1.2"})
r.aggiungi({"event": "output", "request": "r-001", "score": 0.31})
r.aggiungi({"event": "human_review", "request": "r-001", "outcome": "rejected"})
print("intact:", r.verifica())
# Tampering: the outcome of the human review is changed
r.voci[2][0]["outcome"] = "approved"
print("after the change, first invalid entry:", r.verifica())

# Output (python3 -I, Python 3.14.4):
# intact: None
# after the change, first invalid entry: 2

impronta ("fingerprint") serialises the entry with sorted keys (sort_keys=True, indispensable so that two equal dictionaries produce the same text) and computes SHA-256 on the previous hash together with the entry. verifica ("verify") recomputes the chain and returns the position of the first invalid entry, or None. The experiment is in the last lines: the outcome of a human review is changed from "rejected" to "approved", and the check points to entry 2. This is the value of the mechanism, and its limit: whoever can rewrite all the entries and recompute all the hashes is not discovered. This is why the last hash must be anchored outside the system that produces it (a timestamping service, a separate logging system with different permissions), and this part is not in the listing.

There is a tension with the minimisation you saw in the fifth article: a log entry containing a person's entire input is an archive of personal data. The AI Act sets a minimum of six months of log retention "unless provided otherwise" by Union or national law, in particular on data protection (Articles 19 and 26): record what is needed to reconstruct a decision (model version, request identifier, outcome, human intervention) and consider whether the input data can be replaced by a reference or a hash.

Measuring disparity without inventing a threshold

Article 10(2), points (f) and (g), asks for an examination in view of possible biases that are likely to affect the health and safety of persons, have a negative impact on fundamental rights or lead to discrimination prohibited under Union law, and for appropriate measures to detect, prevent and mitigate them. It indicates neither a metric nor a numerical threshold. The simplest metric is the comparison of favourable-outcome rates between groups.

"""Ratio between approval rates by group, on synthetic decisions."""
from collections import defaultdict

DECISIONI = ([("A", 1)] * 62 + [("A", 0)] * 38 + [("B", 1)] * 41 + [("B", 0)] * 59)

conteggi = defaultdict(lambda: [0, 0])
for gruppo, esito in DECISIONI:
    conteggi[gruppo][0] += esito
    conteggi[gruppo][1] += 1
tassi = {g: a / n for g, (a, n) in sorted(conteggi.items())}
for g, t in tassi.items():
    print(f"group {g}: approval rate {t:.2f} over {conteggi[g][1]} decisions")
print(f"ratio B/A: {tassi['B'] / tassi['A']:.2f}")
print(f"difference A-B: {tassi['A'] - tassi['B']:.2f}")

# Output (python3 -I, Python 3.14.4):
# group A: approval rate 0.62 over 100 decisions
# group B: approval rate 0.41 over 100 decisions
# ratio B/A: 0.66
# difference A-B: 0.21

The data are 200 synthetic decisions, 100 per group. Group A has an approval rate of 0.62, group B of 0.41; the B/A ratio is 0.66 and the absolute difference 0.21. The two measures say different things: the ratio makes rates of different levels comparable, the difference says how many percentage points. With 100 decisions per group the statistical uncertainty is wide, and neither number says why the rates differ: it may be a bias of the model, a real difference in the input data or an unobserved attribute. The result is the beginning of the investigation, not a diagnosis.

A threshold such as "the ratio must not fall below 0.8" is a design choice to be justified in the risk management system, not an AI Act rule. To compute these metrics you need the attributes that define the groups; this is where Article 4a comes in, which allows providers of high-risk systems to exceptionally process special categories of data to detect and correct bias, only where strictly necessary and under the listed conditions (among them that processing other data, including synthetic or anonymised data, is not enough, technical limitations on re-use, documented access, no transmission to third parties and deletion after the correction).

What existing open-source tools offer

Mature tools exist for the parts that the kit above does not cover. I verified that the packages exist on the Python registry (PyPI) on 13 November 2026; I have not installed or run them for this article, so I do not state how they behave.

  • DVC (version 3.67.1 on the registry): versioning of data and models next to the code, useful as proof of which data produced which model.
  • MLflow (3.18.0): experiment tracking and model registry, useful for the traceability of the versions in production.
  • Fairlearn (0.15.0) and AI Fairness 360 (0.6.1): fairness metrics and mitigations, more numerous and tested than a comparison of rates.
  • deon (0.3.0): generates an ethics checklist for data science projects, to be used as a process reminder.

None of these tools certifies compliance with the AI Act: they produce technical evidence that goes into your sheet. The choice between Fairlearn and AIF360 is made by looking at which fairness definitions and which mitigation algorithms you need; the choice between DVC and MLflow depends on where your workflow lives. Links to their sites are in the sources.

When a problem becomes a report

An audit finds problems, and the question is what you are obliged to do. Article 73 requires providers of high-risk systems to report serious incidents to the market surveillance authorities: immediately after establishing a causal link with the system or the reasonable likelihood of one, and in any event not later than 15 days after becoming aware of the incident; two days in the event of a widespread infringement or of a serious incident of certain categories; ten days in the event of the death of a person, with the report made immediately once the causal link is established or merely suspected.

The definition of serious incident includes the death of a person or serious harm to a person's health, a serious and irreversible disruption of the management or operation of critical infrastructure, the infringement of obligations under Union law intended to protect fundamental rights, and serious harm to property or the environment. A bias that produces prohibited discrimination can therefore fall under the third category, but it is established by a judgement on the case, not by a threshold in the code.

For the deployer, Article 26(5) says that if it has reason to consider that use in accordance with the instructions may result in a risk, it informs without undue delay the provider or distributor and the authority and suspends the use of the system; if it identifies a serious incident, it immediately informs first the provider and then the authorities. As for users, in the text I read there is no general obligation to notify every bias to the persons concerned: there is the obligation to inform the persons subject to decisions (Article 26(11)) and the right to explanation (Article 86).

Frequently asked questions

I have a legacy system without an event log: do I have to rewrite it?

Not necessarily. Article 111(2) provides that the regulation applies to operators of high-risk systems already placed on the market or put into service before the date of application of Chapter III only if, from that date, those systems are subject to significant changes in their designs; for systems intended to be used by public authorities, compliance is due in any case by 2 August 2030. For Annex III the date of application is 2 December 2027. A significant change triggers everything, so it is better to design the log before touching the system.

Can I audit a system that is not mine, for example a cloud service?

You can verify what the provider makes available to you: the instructions for use (Article 13) with the declared performance, accuracy and limits. As a deployer you must monitor the operation on the basis of those instructions and, if you see a risk, inform the provider (Article 26(5)). You cannot verify what the contract does not show you.

Who signs the audit and what is it worth?

For Annex III systems, points 2 to 8, the conformity assessment is based on internal control (Annex VI), without a notified body, and the provider draws up the EU declaration of conformity (Article 47). A third-party audit is your own governance choice, not a requirement of those articles.

How many incidents do I have to record?

The regulation gives no number. Article 12 asks to record events relevant for identifying situations that may result in a risk, facilitating post-market monitoring and monitoring the operation; Article 73 requires reporting only serious incidents. The practical rule is to record everything that allows a decision to be reconstructed, and to report what falls within the definition.

If I discover a bias, do I have to warn the users?

The text I read contains no general obligation to notify users. There are the reporting of serious incidents to the authorities (provider, Article 73), the information and suspension for the deployer (Article 26(5)) and, if the bias comes from an unlawful processing of personal data, the GDPR obligations on personal data breaches, which I have not analysed here.

What you can do now

You can turn Articles 9 to 15 into an evidence sheet that can fail a pipeline; you can build a hash-chained log and you know what it proves and what it does not; you can compute a disparity between groups and you know why the threshold is your own justified choice; you know which open-source tools exist for the part the kit does not cover; and you know when a problem found in an audit becomes a report. If you want to pick up the thread again, the series starts from the calendar and the roles; for the organisation of models in production there is also MLOps: Deploying AI Models to Production with MLflow.

Sources

The three listings were run with Python 3.14.4, with identical output in two runs; the first ends with exit code 1 by construction. The data are synthetic. The open-source tools were neither installed nor run: only their existence and version on the registry were verified. The notion of "significant changes" in Article 111(2), which the text does not define in the passage read, was not verified.

Try These Tools

Free developer tools related to this topic.

Commenti

Caricamento commenti...

Sign in to comment

New articles and tools on Telegram too

The Telegram channel flags the site's new articles and tools.

Did you like this article?

Explore other content on the blog or discover my projects