Evaluating the performance of hybrid AI models

One of the key challenges addressed by the LIPS platform is highlighting the strengths and weaknesses of enhanced simulators to reveal their potential and areas for improvement. The platform’s evaluation of these models is based on four categories of criteria:

• accuracy,
• ability to generalize to unseen configurations during model training,
• fulfillment of necessary prerequisites (e.g., scalability) for industrial deployment,
• and compliance with the physical constraints specific to each application.

For quick adoption, LIPS comes with three pre-integrated use cases, developed in partnership with RTE, Michelin, and Ansys, and offers users a catalog of hybrid models. These models originate from research conducted by the institute and from international competitions organized around the platform.

Results from various experiments demonstrate promising initial outcomes, adding value in several areas:

• Optimized user experience: The platform’s intuitive interface and ergonomic design ensured a rapid onboarding process for participants in various competitions relying on LIPS, facilitating smooth adoption even for new users.
• Modular architecture and integration flexibility: LIPS offers multiple levels of hybrid model integration within the evaluation pipeline:
  – Beginner level: Use existing models and features without modification.
  – Intermediate level: Customize models using the platform’s native functionalities.
  – Advanced level: Fully customize models and underlying components for fine-tuned performance optimization.
• Multi-criteria evaluation for in-depth model comparison:
  – Enhanced industrial relevance: Hybrid models outperform baselines based solely on physical simulation, providing a better balance between accuracy and computation time.
  – Strengths and weaknesses identification: Detailed performance analysis highlights the strong points and improvement areas of submitted models.
  – Diversity of optimization approaches: Results reveal various strategies to optimize one or several evaluation criteria, enriching the range of explored solutions.
• The LIPS platform embodies a new era for AI-physical hybridization, paving the way for increasingly effective solutions tailored to tomorrow’s industrial challenges. One of its future aims is to provide recommendations for model improvement paths.