Everyone is talking about the rapid advances in artificial intelligence and the famous LLMs (large language models). But do you really know what lies behind our everyday assistants?
Everyone is talking about the rapid advances in artificial intelligence and the famous LLMs (large language models). But do you really know what lies behind our everyday assistants?
To answer this question, Matteo Dora, CTO of Giskard; Amaia Cardiel, a PhD candidate at Valeo; and Yann Eric CHOHO, a PhD candidate in machine learning at Ekimetrics, gathered for a roundtable discussion during our latest AI Breakfast.
Amaia applies her research to a field where there is no room for error: autonomous vehicles. To audit her driving models, she uses a tool called “counterfactual explanation.”
How does it work? It involves finding the smallest change to make to an image or text that would cause the AI to radically change its decision. (For example: If the car is going straight, what tiny detail would have caused it to turn right?)
In her case, the self-driving car model had conflated accelerating with passing: once it exceeded a certain speed, it would systematically shift into the left lane.
Why? Because it had mindlessly memorized its training data: in reality, when you pass another vehicle, you accelerate and move to the left. On paper, the AI’s scores were perfect. In practice, it was operating for the wrong reasons. Without this analysis, the bias would have remained hidden.
Explainability made it possible to link the situation to the decision-making process, which in turn made it actionable and allowed us to resolve this problematic behavior.
To understand how AI thinks, Matteo uses a rather original method: he draws inspiration from neuroscience.
“In neuroscience, we can’t observe behavior directly inside the brain. We present tasks and measure the response. We do pretty much the same thing with models.”
This is known as the “black box” approach. We don’t know exactly what’s going on inside, so we test the model to build its profile.
For example, evaluating music theory proves much more complicated due to its unique nature.
LLMs’ capabilities in this area are also limited for the same reasons.
However, beware of the pitfall: it’s better to have no explanation at all than a misleading one. A false justification leads to blind trust on the part of the user…
Furthermore, this “black box” approach quickly reaches its limits, particularly in a legal context (such as the AI Act, for example), which requires impeccable transparency from the model.
The GDPR, meanwhile, requires explainability in the context of an automated decision; the user has the right to obtain it….
Yann Eric, a PhD student in AI, actually prefers to refer to it as a “glass box.” You can see all the numbers and parameters inside, but you simply can’t read that coded language.
To decipher how AI works, he focuses on concepts.
For an AI to recognize a bird, it looks for intermediate concepts: a beak, wings, and legs. The goal of his research is to force the AI to validate these human concepts before providing its answer. By modifying these concepts, we can correct the model’s trajectory in real time.
Because AI doesn’t think like we do: it loves co-occurrences. If you associate a soccer player with a country too often in its data, it will get the nationality wrong as soon as you change the context.
You’ve probably noticed that the new models (such as those labeled “Reasoning”) display their “chain of thought” before providing the final answer. It feels like you’re witnessing a real thought process.
Big mistake! Researchers are unanimous: this text is nothing more than a justification constructed after the fact (post hoc). It doesn’t accurately reflect what actually led the machine to choose its answer. An AI can follow a flawless line of reasoning… and still produce a completely wrong result (or vice versa).
What’s more, these models have staggering blind spots. An LLM capable of solving quantum mechanics or ultra-complex math problems can still get basic concepts wrong.
Since AI remains a massive statistical machine, it’s up to us humans to stay in control. Here are three tips from experts for your daily prompts:
Want to deploy reliable and explainable AI systems rather than opaque “black boxes”? Contact our experts!
And to make sure you don’t miss any of our news, follow us on LinkedIn !
Le cas particulier des GPAI (Modèles d'IA à usage général) : Les grands modèles de langage (LLM) comme Mistral AI, OpenAI ou Claude entrent dans un régime propre. Soumis à une application progressive, ils nécessitent des analyses d'impact approfondies pour évaluer les risques selon s’ils sont utilisés bruts, fine-tunés ou intégrés via API.
Développée par Xavier Trigano, la méthode RADAR permet à toute organisation de piloter sa mise en conformité de manière itérative :
Focus "AI by Design" - L'exemple du tri automatique de CV : Un outil RH qui exclut ou accepte des candidats de manière 100 % autonome est classé "Haut Risque", avec un coût de conformité très lourd. La méthode RADAR recommande plutôt une approche by design : modifier les fonctionnalités de l'outil pour en faire un simple système d'aide à la décision (qui extrait les compétences clés du CV mais laisse la validation finale à un recruteur humain). L'outil apporte la même valeur métier, mais bascule en risque limité, allégeant drastiquement les contraintes légales.
Comment lutter contre le Shadow AI en entreprise ?
L'interdiction pure et simple ne fonctionne pas. Pour maîtriser l'usage des LLM par les collaborateurs, la réponse doit être transverse :
L'usage des LLM (ChatGPT, Claude...) viole-t-il le RGPD ?
Ce n'est pas l'outil qui caractérise la violation, mais la finalité de l'usage. Reformuler une campagne marketing sur Claude ne présente aucun risque RGPD. En revanche, y injecter l'intégralité du fichier RH de la pyramide des âges de l'entreprise sans précaution constitue un manquement grave.
Des alternatives souveraines (hébergées on-premise ou sur des clouds français/européens) permettent de pallier les risques liés au Cloud Act américain tout en garantissant une efficacité équivalente.
Comment encadrer mes équipes dans leur utilisation de l’IA ?
L'IA Act impose une obligation de formation pour tous les utilisateurs au sein de l'organisation. L'IA pouvant se tromper ou halluciner, seul l'esprit critique de l'humain formé permet de couvrir ce risque résiduel et d'assurer un contrôle qualité efficace.
L'IA Act ne doit pas être perçu comme un frein à l’innovation, mais comme un cadre de confiance.
En intégrant la conformité dès la conception des projets, l'IA devient un levier pérenne de performance économique, d'acceptabilité sociale et de souveraineté.
Envie de développer votre agent IA sur-mesure conforme à la réglementation AI Act ? Contactez nos équipes.
Et accédez au replay de ce webinaire dès maintenant !
%20(2).png)
