Industrial and responsible AI: SystemX launches the CSIA project to build trust in generative AI and hybrid AI

The ‘Trust in AI Systems’ (CSIA) project builds on the work of the Confiance.ai programme, which led to the development of a comprehensive methodology that now represents the state of the art in industrial and responsible AI engineering.

It aims to overcome R&D barriers relating to trust in generative and hybrid AI through around ten industrial use cases, with a view to developing methods and tools that comply with regulations (e.g. the AI Act) and are designed to complement those of the European Trustworthy AI Association.

SystemX, the Institute for Technological Research (IRT) dedicated to the digital engineering of future systems, announces the launch of the 48-month R&T project ‘Trust in AI Systems’. Having led the Confiance.ai programme until 2024, SystemX is continuing its research work, through the CSIA project, on the theme of industrial and responsible AI. This project currently brings together six industrial partners (Naval Group, Octopize, Safenai, Safran, Sopra Steria and Thales) and three academic partners (ONERA, Paris-Saclay University and IRT SystemX), and is open to any partner interested in joining the consortium to tackle the challenges of trust in industrial AI systems.

The Confiance.ai collaborative R&T programme has set the state of the art in industrial and responsible AI engineering by developing an end-to-end methodology and trustworthy technological components, which are now maintained by the European Trustworthy AI Association. This well-defined methodology will continue to be enhanced, notably through work carried out as part of the CSIA project, in order to incorporate solutions to emerging scientific and technological challenges, particularly those related to generative AI and hybrid AI.

The project partners have set themselves two main objectives:

• To enhance the methodology and technological components maintained by the European Trustworthy AI Association:
  • Firstly, by taking into account the specific characteristics of critical systems based on hybrid AI (such as the coupling of neural techniques with physical models, or neuro-symbolic approaches), whilst ensuring rigorous control over the data and knowledge lifecycle and guaranteeing the properties of reliability, security and explainability. The project’s industry partners also wish to be able to design trustworthy generative AI – that is, AI that is valid, transparent, secure and compliant with new regulatory frameworks and standards.
  • Finally, the catalogue of trusted open-source AI tools will also be expanded to include new tool-based concepts: OD (operational domain), ODD (operational design domain), technical formulation of the ‘Intended Purpose’, RUM (Robustness, Uncertainty, Monitoring), data anonymisation tools, etc.
• To adapt the methodology and trusted technology components to the compliance requirements of the AI Act, in particular Article 6, which concerns “high-risk AI systems” and which will come into force on 2 August 2027 – whilst aligning with ongoing standardisation initiatives, notably those led by the EUROCAE/SAE groups, to ensure alignment with emerging standards. Industrial partners will thus be able to update their development processes for trustworthy AI solutions, drawing on a combination of bespoke assets tailored to their specific use cases, thereby ensuring compliance with European regulations.

To demonstrate the relevance of the methodologies and tools developed as part of the CSIA project, as well as their applicability in industrial contexts, the partners involved will test them on around ten specific use cases, including in particular:

• Two use cases will focus on air traffic. The first will involve the implementation of a chatbot based on AI and language models, enabling users to query historical flight data, with one key objective: to demonstrate and evaluate the trust properties associated with this system. This use case falls within the theme of trustworthy AI applied to generative text models. The second will focus on creating a machine learning-enhanced air traffic management support solution. This solution provides a real-time view of arrival and departure sequences, thereby helping air traffic controllers to easily manage runway allocation and flight sequencing, which optimises runway utilisation to ensure safe, punctual and efficient operations, particularly during periods of heavy traffic.
• One use case will focus on developing a model to ensure trust in a decision-support system for generating maritime routes using Deep Reinforcement Learning (DRL).

“By leading the Confiance.ai programme, playing a key role in developing European norms and standards for assessing and certifying AI components, and establishing the European Trustworthy AI Association, SystemX has become a leading player in trustworthy AI within the French and European ecosystems. This momentum is contributing to the emergence of sovereign AI, based on the principles of accountability, explainability and transparency. In this context, European regulation serves as a key driver for developing reliable and competitive technologies. Industrial partners are joining the CSIA programme to benefit from a methodology and tools that comply with these regulations. Through this project, SystemX is also continuing its commitment by addressing the challenges associated with generative and hybrid AI,” comments Sana Tmar, CSIA Project Lead at IRT SystemX.