Introduction

Generative models are a family of machine learning models whose purpose, as their names suggest, is to generate data. This is quite different from their use in a predictive framework.

In a predictive framework, models are trained on past data in order to learn a decision boundary or a value. In generative models, models are trained to understand the structure of the data, with the aim of producing data that resembles the training data.

Where a predictive model needs the values of the input attributes to give its prediction, a generative model needs context. This can take a variety of forms, from extracts of elements to be generated, constraints to be respected, to simply noise (which acts as a random "seed").

In the previous definitions, no details were given on the nature of the data it is capable of generating via these generative models. Actually, models have been developed to generate images, music, Excel files and so on.  In the following, we'll focus on language models, and more specifically on LLMs (Large Language Models). Having a model capable of generating text is not really new, but what makes them a real revolution compared with conventional approaches is their versatility and their quality. This has been made possible by advances in computer science research, notably by the invention of Transformer-type neural networks in 2017, and then the ability to learn from colossal volumes of data.

Large Language Model : LLM

While we see many very impressive applications of LLMs, it's important to remember that these models are designed for just one task: that of predicting the next element in a given sentence. This may seem surprising, but whether it's code generation, text summarization, medical diagnosis or a philosophy dissertation, it's all based on the ability to generate the next part of a given sentence. The model will complete the input sentence, then re-use this partially extended sentence as new input, and so on until the desired length is reached.

Most of recent LLMs are based on two main concepts: embeddings and the attention mechanism.

Embedding

Computers cannot manipulate words and their semantics directly. They can, however, compute very efficiently on numbers. We could associate each word with a unique number, but this would lead to a very inefficient system. What we'd like is for the numerical values associated with words to retain information. For example, if the word "man" is at a certain distance from the word "woman", this distance should be the same between the words "king" and "queen". This is what embeddings enable. This is a trained part of the model which, for a given word, returns a computer-usable representation that preserves the relationships between words (and even their relative position in the sentence for certain embedding type).

Attention

An LLM, such as GPT, is trained to perform a single task: predicting the next word given the beginning of a sentence. A simple approach might be to group together all the words preceding the word to be predicted, in order to create a context and learn from a training corpus which is the most likely next word. This approach appears to generate a lot of noise, as many uninformative words are taken into account. This is where the attention mechanism comes in. It learns which elements are relevant to the prediction of the next word and focuses on those, masking the others. Thanks to a particular type of embedding (positional embedding), position and semantics are taken into account. In this way, only relevant elements appear in the decision-making process, enabling the models to learn efficiently how to predict the next word to be generated.  This is clearly illustrated with images. It is indeed necessary to find a mask, then apply it to the image in order to isolate the important elements. Note that, as with text, several attention systems need to be implemented in parallel, as it may be necessary to concentrate on several parts of the image in order to interpret it correctly.

An LLM is a stack of modules

To understand the size of a language model, we need to look at a simpler example of deep learning.

The above network is made up of an input layer, through which input attribute values can be given, 2 hidden layers for learning correlations between input values, and finally an output layer for retrieving results.

Each neuron (the circles) aggregates the values of the previous layer (the x's) multiplied by an associated weight (w) and performs a calculation on this sum to produce a value that can then be used by the following layers of the network. It is these weights w that are learned during the training phase and are also called parameters.

The way in which the neurons are assembled together (their number, the links between them) is called an architecture. The attention mechanism is therefore an architecture. It's possible to build a slightly larger architecture containing an attention mechanism, the Transformers. And if we "stack" the Transformer blocks, and add the embedding block, we almost get an LLM architecture like LLaMA!  It's the number of "stacked" layers that defines the size of the model, expressed in number of parameters.

Ecosystem

The training costs, data requirements and know-how needed to create a foundation model are such that it's impossible for a non-specialist company to create one. Nevertheless, while the largest models are still only accessible via APIs (such as GPT4 or BARD), more and more models are appearing in open source. These models are available in a range of sizes (from a few billion to several hundred billion parameters), enabling the choice of LLM to be adapted to the specific use case. Indeed, putting an oversized LLM into production generates excessive energy consumption and hosting constraints.

LLMs are capable of handling a wide variety of problems. When you're only interested in solving a specific business problem, you can adapt a smaller LLM (like LLaMA2 7b, for example) to this use case. In this way, its performance on this specific use case can be made equivalent to that of a larger LLM. These techniques are known as fine-tuning. An important point about fine-tuning is that it can be carried out on standard hardware (a few GPUs), at a more reasonable cost and time. If you don't have sufficiently powerful hardware, you can also choose to lose precision in the value of the model's parameters and fit them into the GPU's memory (via quantization). The open-source distribution of models and the development of a software ecosystem now allow LLMs to be democratized and adopted by industry.

Conclusion

Navigating through the complexities of Large Language Models (LLMs), we uncover the depth of their text generation capabilities, rooted in mechanisms like embeddings and attention. While their ability to produce diverse and coherent text showcases AI's leaps in technology, LLMs also present notable challenges in areas like data labeling, deployment, and ethical use. As we stride forward in the AI domain, marrying technological advancements with ethical and responsible application will be crucial. The implementation and fine-tuning of LLMs demand a conscientious approach, ensuring their benefits permeate various domains while safeguarding ethical and operational standards.

‍

Le cas particulier des GPAI (Modèles d'IA à usage général) : Les grands modèles de langage (LLM) comme Mistral AI, OpenAI ou Claude entrent dans un régime propre. Soumis à une application progressive, ils nécessitent des analyses d'impact approfondies pour évaluer les risques selon s’ils sont utilisés bruts, fine-tunés ou intégrés via API.

‍

3. La Méthode "RADAR" pour Auditer ses Cas d'Usage

Développée par Xavier Trigano, la méthode RADAR permet à toute organisation de piloter sa mise en conformité de manière itérative :

• R – Recenser : Cartographier exhaustivement tous les cas d'usage de l'entreprise (outils internes pour les collaborateurs et solutions commercialisées).
• A – Attribuer les rôles : Identifier si l'organisation agit en tant que Fournisseur de modèle, Fournisseur de système d'IA, Intégrateur ou Déployeur (utilisateur final). Une même entreprise peut cumuler plusieurs rôles.
• D – Déterminer le risque : Qualifier le niveau de risque selon le type de cas d’usage et sa finalité (Inacceptable ? Haut risque ? Limité ? Faible ?).
• A – Appliquer les mesures : Mettre en œuvre les actions techniques et organisationnelles requises par le niveau de risque identifié.
• R – Rédiger la documentation : Constituer le registre et les preuves de conformité (similaire à la logique de l'accountability du RGPD).

Focus "AI by Design" - L'exemple du tri automatique de CV : Un outil RH qui exclut ou accepte des candidats de manière 100 % autonome est classé "Haut Risque", avec un coût de conformité très lourd. La méthode RADAR recommande plutôt une approche by design : modifier les fonctionnalités de l'outil pour en faire un simple système d'aide à la décision (qui extrait les compétences clés du CV mais laisse la validation finale à un recruteur humain). L'outil apporte la même valeur métier, mais bascule en risque limité, allégeant drastiquement les contraintes légales.

‍

4. FAQ : Shadow IT, RGPD et Souveraineté

Comment lutter contre le Shadow AI en entreprise ?

L'interdiction pure et simple ne fonctionne pas. Pour maîtriser l'usage des LLM par les collaborateurs, la réponse doit être transverse :

• Définir un catalogue d'usages autorisés sur la base d'outils sécurisés mis à disposition.
• Utiliser des solutions technologiques de DLP (Data Loss Prevention) pour bloquer le chargement de données sensibles vers des LLM tiers.
• Mettre à jour la charte informatique et le règlement intérieur.

L'usage des LLM (ChatGPT, Claude...) viole-t-il le RGPD ?

Ce n'est pas l'outil qui caractérise la violation, mais la finalité de l'usage. Reformuler une campagne marketing sur Claude ne présente aucun risque RGPD. En revanche, y injecter l'intégralité du fichier RH de la pyramide des âges de l'entreprise sans précaution constitue un manquement grave.

Des alternatives souveraines (hébergées on-premise ou sur des clouds français/européens) permettent de pallier les risques liés au Cloud Act américain tout en garantissant une efficacité équivalente.

Comment encadrer mes équipes dans leur utilisation de l’IA ?

L'IA Act impose une obligation de formation pour tous les utilisateurs au sein de l'organisation. L'IA pouvant se tromper ou halluciner, seul l'esprit critique de l'humain formé permet de couvrir ce risque résiduel et d'assurer un contrôle qualité efficace.

‍

5. Conclusion : L'IA Act, un levier de confiance et de compétitivité

L'IA Act ne doit pas être perçu comme un frein à l’innovation, mais comme un cadre de confiance.

En intégrant la conformité dès la conception des projets, l'IA devient un levier pérenne de performance économique, d'acceptabilité sociale et de souveraineté.

‍

Envie de développer votre agent IA sur-mesure conforme à la réglementation AI Act ? Contactez nos équipes.

‍

‍

Et accédez au replay de ce webinaire dès maintenant !