EAGLE: Large-scale Learning of Turbulent Fluid Dynamics with Mesh Transformers

Steeven Janny  Aurélien Bénéteau Madiha Nadri Julie Digne Nicolas Thome Christian Wolf
LIRIS Supaero LAGEPP LIRIS ISIR Naver Labs Europe
INSA Lyon, France Toulouse, France Univ. Claude Bernard, France INSA Lyon, France Sorbonne University, France Grenoble, France





Abstract

Estimating fluid dynamics is classically done through the simulation and integration of numerical models solving the Navier-Stokes equations, which is computationally complex and time-consuming even on high-end hardware. This is a notoriously hard problem to solve, which has recently been addressed with machine learning, in particular graph neural networks (GNN) and variants trained and evaluated on datasets of static objects in static scenes with fixed geometry. We attempt to go beyond existing work in complexity and introduce a new model, method and benchmark. We propose EAGLE, a large-scale dataset of ∼1.1 million 2D meshes resulting from simulations of unsteady fluid dynamics caused by a moving flow source interacting with nonlinear scene structure, comprised of 600 different scenes of three different types. To perform future forecasting of pressure and velocity on the challenging EAGLE dataset, we introduce a new mesh transformer. It leverages node clustering, graph pooling and global attention to learn long-range dependencies between spatially distant data points without needing a large number of iterations, as existing GNN methods do. We show that our transformer outperforms state-of-the-art performance on, both, existing synthetic and real datasets and on EAGLE. Finally, we highlight that our approach learns to attend to airflow, integrating complex information in a single iteration.




Benchmark

We introduce EAGLE, a large-scale dataset for learning non-steady fluid mechanics. We accurately simulate the airflow produced by a two-dimensional unmanned aerial vehicle (UAV) moving in 2D environments with different boundary geometries. This choice has several benefits. It models the complex ground effect turbulence generated by the airflow of an UAV following a control law, and, up to our knowledge, is thus significantly more challenging than existing datasets. It leads to highly turbulent and non-periodic eddies, and high flow variety, as the different scene geometries generate completely different outcomes. At the same time, the restriction to a 2D scene (similar to existing datasets) makes the problem manageable and allows for large-scale amounts of simulations (∼1.1m meshes).

Step Triangular Spline

We simulate the complex airflow generated by a 2D unmanned aerial vehicle maneuvering in 2D scenes with varying floor profile. While the scene geometry varies, the UAV trajectory is constant: the UAV starts in the center of the scene and navigates, hovering near the floor surface. During the flight, the two propellers generate high-paced air flows interacting with each other and with the structure of the scene, causing convoluted turbulence. To produce a wide variety of different outcomes, we procedurally generate a large number of floor profiles by interpolating a set of randomly sampled points within a certain range. The choice of interpolation order induces drastically different floor profiles, and therefore distinct outcomes from one simulation to another. EAGLE contains three main types of geometry depending on the type of interpolation:

  • Step: surface points are connected using step functions (zero-order interpolation), which produces very stiff angles with drastic changes of the air flow when the UAV hovers over a step.
  • Triangular: surface points are connected using linear functions (first-order interpolation), causing the appearance of many small vortices at different location in the scene.
  • Spline: surface points are connected using spline functions with smooth boundary, causing long and fast trails of air, occasionally generating complex vortices.

Raw mesh Down-sampled mesh
Numerical simulations were carried out using the software Ansys Fluent, which solves the Reynolds Averaged Navier-Stokes (RANS) equations leveraging the Reynolds stress model for turbulence. The Reynolds Stress model uses five equations to model turbulence, which is more accurate than standard \(k\)-\(\epsilon\) or \(k\)-\(\omega\) models, leveraging solely two equations. This resulted in 3.9TB of raw simulation data, with each simulation being defined on a dynamical mesh with \(\sim\)162,760 control points. We down-sampled and compressed the data to 3,388 points in average, resulting in 270GB.
EAGLE dataset is composed of exactly 1,184 simulations of 990 time-steps (33 seconds at 30 fps). Scene geometry are arranged in three categories based on the order of the interpolation used to generate the ground structure: 197 Step scenes, 199 Triangular and 196 Spline. A geometry gives two simulations depending on whether the drone is crossing the left or the right part of the scene. A proper train/valid/test splitting is provided ensuring that each geometry type is equally represented. Train set contains 948 simulations, while test and valid set contain 118 simulations.


Geometry # simulations Avg. # of nodes Avg. # of edges Weight
Step 394 3134 6106 83 GB
Triangular 398 3372 6599 90 GB
Spline 392 3667 7172 98 GB




EAGLE Benchmark Model

We propose a new multi-scale attention-based model, which circumvents the quadratic complexity of multi-head attention by projecting the mesh onto a learned coarser representation yielding fewer but more expressive nodes. Conversely to standard approaches based on graph neural networks, we show that our model dynamically adapts to the airflow in the scene by focusing attention not only locally, but also over larger distances. More importantly, attention for specific heads seems to align with the predicted airflow, providing evidence of the capacity of the model to integrate long range dependencies in a single hop. We evaluate the method on several datasets and achieve state-of-the-art performance on two public fluid mechanics datasets (Cylinder-Flow and Scalar-Flow), and on EAGLE.

The figure below is dynamic: hover a module to get more information about its behavior.


References of baseline methods and datasets:
  • MeshGraphNet & Cylinder-Flow: Pfaff, Tobias, et al. "Learning Mesh-Based Simulation with Graph Networks." International Conference on Learning Representations.2020
  • DilResNet: Stachenfeld, Kim, et al. "Learned Simulators for Turbulence." International Conference on Learning Representations. 2021.
  • GAT: Veličković, Petar, et al. "Graph Attention Networks." International Conference on Learning Representations. 2018.
  • Scalar-Flow: Eckert, Marie-Lena, Kiwon Um, and Nils Thuerey. "ScalarFlow: a large-scale volumetric data set of real-world scalar transport flows for computer animation and machine learning." ACM Transactions on Graphics (TOG) (2019)


Attention Visualization Tool

To help you understand how our benchmark behaves, we built this visualisation tool by exporting the attentions map of four heads in our model. Use the slider to change the time step, and click on a cluster (represented by transparent square) to change the reference. This will update the attention map.

Timestep:

First Attention Layer, Head #3

Second Attention Layer, Head #1

Third Attention Layer, Head #3

Fourth Attention Layer, Head #2


Results

EAGLE
Cylinder-Flow
Scalar-Flow



Download





Citation & Contact

Do not hesitate to contact us for any questions about our project at this address : steeven.janny@insa-lyon.fr


    @inproceedings{janny2023eagle,
        title = "EAGLE: Large-scale Learning of Turbulent Fluid Dynamics with Mesh Transformers",
        author = {Steeven Janny and
                  Aurélien Benetteau and
                  Madiha Nadri and Julie Digne and Nicolas Thome and Christian Wolf},
        booktitle = "International Conference on Learning Representations (ICLR)",
        year = "2023"}
    



Funding

This project is funded by the French National Research Agency (ANR) under the program "Données et a priori, apprentissage et contrôle – DeLiCio" (ANR-19-CE23-0006) of call "CE23 - Intelligence artificielle".