What is the most important thing companies need to understand about the EU AI Act?
– That it doesn’t regulate all AI in the same way. The requirements are primarily based on what the AI system is used for and what consequences that usage may have. The same underlying technology can therefore be subject to completely different requirements depending on the context.
An AI solution that helps an employee draft text, for example, normally presents an entirely different risk than a system that evaluates job applicants, influences who receives a loan, or is used to make decisions about people’s access to essential public services.
It is therefore not enough to ask whether the company uses AI. You need to know where AI is used, for what purpose, what data is being processed, and whether the system’s output affects people or important business decisions.
The regulation is often described as a model with four risk levels. What do they mean?
– What matters is not which AI technology a company uses, but what it is used for and what consequences that usage may have. The same technology can pose minimal risk in one context and high risk in another. The greater the impact on people’s rights, safety, and living conditions, the higher the requirements imposed by the EU AI Act.
The four risk levels can be summarized as follows:
1. Minimal risk
This includes, for example, spam filters and AI in video games. For this type of use, the EU AI Act normally contains no mandatory requirements beyond other legislation that already applies.
2. Specific transparency risks
This level covers chatbots and certain uses of generative AI, among others. Users may need to be informed that they are interacting with AI, and certain AI-generated or manipulated content must be identifiable or labeled.
3. High risk
This involves AI systems that can have significant implications for people’s rights, opportunities, or safety, such as in recruitment, education, credit assessment, and critical infrastructure. Extensive requirements are imposed here, including risk management, documentation, and human oversight.
4. Unacceptable risk
The highest risk level covers AI uses that are prohibited within the EU, such as certain forms of social scoring, manipulative AI, and emotion recognition in workplaces and schools.
– So generative AI does not automatically belong to a specific risk level. An AI service that helps an employee draft text normally poses an entirely different risk than if the same technology is used to evaluate job applicants or influence decisions about individuals. Each use case must therefore be assessed separately.
What requirements apply to high-risk AI systems?
– For high-risk AI, extensive requirements are imposed on risk management, data quality, documentation, logging, human oversight, accuracy, and cybersecurity, among other things. Those using the system must also follow applicable instructions, monitor usage, and ensure that responsible individuals have the right competencies.
A central principle is that AI must not become a black box that independently makes decisions with significant consequences. There must be people who understand the system’s role, can evaluate the output, and have the authority to intervene.
Which prohibited AI uses do companies particularly need to be aware of?
– One example is AI-based emotion recognition in workplaces, which is prohibited with limited exceptions for medical purposes or safety. This involves AI systems that use biometric data, such as facial expressions, tone of voice, or body language, to attempt to identify or draw conclusions about a person’s emotions or intentions.
The prohibition is particularly relevant for employers evaluating new digital tools for recruitment, monitoring, or employee analysis. Features presented as support for more efficient recruitment or workforce management may be based on analysis of facial expressions, tone of voice, or behaviors. The availability of a feature in a product does not automatically mean it may be used.
What do companies need to consider when using generative AI?
– The first thing is to distinguish between approved enterprise services and open consumer services. If employees enter customer data, personal information, trade secrets, source code, or other sensitive material into an AI tool, the information can end up outside the company’s control.
That’s why clear guidelines are needed for which tools may be used, what information may be entered, and how AI-generated content should be reviewed before it is used.
You also need to remember that a language model can provide a convincing but incorrect answer. AI can be a very good work tool, but the output must be assessed based on the risk of the task. A language error in an internal summary is one thing. An error in a financial report, a contract, or a decision document is something entirely different.
How does the EU AI Act affect business systems like SAP and Dynamics 365?
– AI is now rarely a separate AI project, but is increasingly being built into more and more business processes: finance, procurement, sales, HR, customer service, analytics, and planning. This makes the question particularly important for organizations using platforms like SAP or Microsoft Dynamics 365.
It’s not enough to assess a standalone AI service. The company must also understand which AI capabilities are built into the systems it already uses, which are activated, and which decisions or recommendations they influence.
An AI assistant that helps users find information is normally relatively straightforward. But if AI starts prioritizing candidates, recommending actions that affect employees, or automatically making business-critical decisions, the risk profile changes. Classification must therefore be based on the specific use, not on the product name.
Whose responsibility is this – IT’s or Compliance’s?
– It’s a shared business responsibility. IT and information security need to understand the technology, data flows, and security risks. Legal, compliance, and data protection need to assess which regulations apply and ensure that requirements are implemented in practice. HR, finance, sales, and other business functions must be able to describe how systems are used and what consequences they may have.
Leadership, meanwhile, needs to determine the company’s risk appetite – that is, which risks it is willing to accept – and establish accountability and overall governance.
If the AI question is placed solely with IT, you miss the business perspective. If it’s placed solely with legal or compliance, the work risks becoming a regulatory compliance project separate from actual usage. A model is needed where business, technology, security, legal, and compliance work together.
What should companies do now?
Before you can create control over AI, you need to have control over your information. AI doesn’t make old problems with information management less important – quite the opposite. When AI gains access to large volumes of business data, it becomes even more important to know which information is sensitive and business-critical, where it resides, who has access to it, and how it is protected.
Companies therefore need to start with some fundamental questions: What information do we have? Where is it located? Who can access it? Is it properly classified and protected? And is information that should no longer be retained being purged?
This creates the foundation for the next step: mapping where AI is used, what information AI systems have access to, and what risks the usage entails.
Then you can begin the inventory. Many organizations still lack a comprehensive picture of which AI solutions are being used, including capabilities built into existing systems and so-called shadow AI that employees have started using on their own initiative.
The next step is to describe the use cases and classify them based on risk. After that, you can determine which controls, instructions, and documentation requirements are needed.
Mats Stegemann highlights six priority actions:
- Gain control over your information. Identify which information is sensitive and business-critical, where it resides, who has access to it, and how it is classified and protected.
- Map AI usage. Identify tools, built-in capabilities, vendors, data, and use cases.
- Risk-classify use cases. Assess purpose, consequences, and which people or decisions are affected.
- Clarify accountability and decision pathways. Determine who may approve new AI solutions and who is responsible for follow-up.
- Implement practical guidelines. Make it clear which tools and data may be used and when human review is required.
- Ensure AI competency. Employees need to understand both the capabilities, limitations, and risks of the systems they use.
Is there a risk that regulation will hinder companies’ AI development?
– There is a risk that companies become so cautious that they refrain from value-creating uses. But the alternative – allowing AI to proceed without control – is far more risky. Lack of governance can lead to incorrect decisions, information leaks, discrimination, and lost trust, even when the usage doesn’t directly violate the EU AI Act.
Mats Stegemann believes that AI governance should therefore not primarily be seen as a brake.
– Good governance makes it possible to scale AI in a more secure way. When the company knows which data may be used, who is responsible for the results, and how risks should be managed, it also becomes easier to move from isolated experiments to real business value.
What is your most important advice to company leadership?
– Don’t wait for all legal and technical details to be fully investigated. Start by creating control over your information and your own usage. The biggest risk right now is often not that a company knowingly uses a prohibited AI system, but that no one has oversight of their information and which AI capabilities are already being used and which decisions they influence.
The EU AI Act makes the question more urgent, but the need is actually greater than that.
Fundamentally, it’s about trust. Customers, employees, and partners must be able to trust that AI is used securely, transparently, and with people who take responsibility for the consequences.
