Skip to main content
Back to blog
Best Practices

When an AI compliance tool drifts: why NIS2 asks for human oversight

How an automated tool can misclassify risks, skip controls and report "compliant" with real gaps, and what Article 20 says.

NIS2 Manager team
NIS2 Manager, BetterQA
6 min read

An automated compliance tool can run through up to 212 controls in minutes, generate reports and fill in forms. It can just as easily classify a critical risk as "medium", skip a control it thinks irrelevant, or report "compliant" when gaps exist. Unlike a human mistake, which leaves traces, a tool that strays from its task produces results that look fine. We call this drift (in English, "agent drift"): the form stays, the substance goes.

What drift looks like in compliance

  • a scanner that assesses policies by whether documents exist, not by their content;
  • a risk assessment that assigns scores from general templates, not from the organization's situation;
  • a report that fills in fields with predefined answers without checking whether the measures are applied.

An example: the risk level assessment filed with DNSC determines which measures must be implemented. A tool can calculate it from sector, size and the type of data processed. But it does not know the organization had an incident in recent months or that a critical supplier was compromised. The result would look right and be wrong, and the measures planned from it would be insufficient.

What NIS2 says about accountability

The Directive does not mention AI tools, but it is clear about who is accountable, and it is not an algorithm.

Article 20, management. Management bodies approve the risk management measures, oversee their implementation, take part in training and answer for non-compliance. If a tool produces a report and management signs it without checking, accountability stays with management.

Article 23, incident reporting. The early warning, full notification and final report have strict deadlines (24 hours, 72 hours, one month after the 72-hour notification). A tool can detect anomalies and classify an event as an incident. The decision to report calls for judgement: the real severity, the services affected, the users affected, whether the incident is isolated.

Article 21, security measures. For each control (33, 132 or 212, depending on level) the organization must show that the measure is implemented, tested and maintained. A tool can check whether a policy document exists, but not whether the policy is followed in practice.

Three situations where automation can go wrong

A superficial gap analysis. A tool compares policies with NIS2 requirements by keyword match and reports high coverage. In reality, a policy mentions "incident management" without concrete procedures, and the risk assessment document has not been updated for years.

A risk assessment on incomplete data. The tool sees the network topology and the application list, but does not know the IT team put in a new VPN without updating the documentation, or that some privileged accounts of people who left were never revoked.

Alerts filtered by history. A filter learns what a "false alarm" is from the past. A new type of attack, unlike anything in the history, can be filtered out as a false positive.

How NIS2 Manager keeps the person in the process

The platform organizes the controls for your level into a flow in which, for each control, someone assesses the state, uploads evidence and documents the measures. It does not fill in the answers for the user. The eligibility calculator asks for data at each step and does not give a yes or no from a single criterion. Actions are logged (who, when, what evidence), and alerts rest on fixed deadlines, not predictions.

Five questions for any automated tool

  1. Who validates the results?
  2. What data is the tool missing?
  3. Can you show DNSC that a person assessed each control?
  4. Who decides in atypical cases: in-house IT departments as MSPs, group membership, DORA versus NIS2?
  5. Does management understand what it approves? Signing an automated report without understanding it does not satisfy the oversight requirement in Article 20.

Tools help with speed. "The tool says it is fine" is not the same as "it is fine". Start with the eligibility check.

BetterQA does security testing and software testing from Cluj-Napoca.

Tags:
AIcomplianceNIS2human oversightautomation
Share this article:
NIS2 Manager team
NIS2 Manager, BetterQA

The NIS2 Manager team at BetterQA writes practical guidance on NIS2 compliance for essential and important entities.

Want to know if your company falls under NIS2?

Use our free calculator to check eligibility in just 3 minutes.

Check eligibility for free

Related Articles

EUR 10M
fine ceiling of at least EUR 10M or 2% of worldwide turnover for essential entities, whichever is higher; EUR 7M or 1.4% for important entities (NIS2 Directive, Article 34)
24h
to send an early warning of a significant incident (NIS2 Directive, Article 23(4)(a))
18
sectors in scope: 11 of high criticality (Annex I) and 7 other critical sectors (Annex II)

The NIS2 Directive (EU 2022/2555) entered into force on January 16, 2023, with member states required to transpose it by October 17, 2024. For essential entities, the fine ceiling is at least EUR 10 million or 2% of total worldwide annual turnover, whichever is higher (EUR 7 million or 1.4% for important entities). Romania’s DNSC annual activity report for 2024 recorded a 40.2% rise in reported cyber fraud and a 286.8% rise in malware compared with 2023 (DNSC, 2024), making compliance tools essential for the thousands of Romanian organizations affected by the directive.

BetterQA
ISO 27001 certified; NATO NCIA agreement holder
50+ Engineers
Cybersecurity & compliance specialists, clients in 20+ countries
Since 2018
Independent security testing & compliance expertise
NIS2 Ready
Full compliance lifecycle from assessment to certification