|
Tue 24th of October 2006
|
|
Presentation
The MUMPS team organizes from time to time the MUMPS User Days.
This first MUMPS User Day in 2006 has been an opportunity for users to discuss their use of the MUMPS package and their requirements for the future. It was also of interest for people concerned with large-scale simulations. Part of the day has been dedicated to presentations by users from various application fields, while another part focused on research on parallel sparse direct methods and on recent and planned functionalities of the MUMPS solver (e.g., out-of-core execution, parallel analysis, processing singular matrices). It was also a good occasion for informal exchanges and discussions between users and developers of MUMPS.
Funding
Organisation
Program
| 08.30 - 08.55 | Registration and Coffee |
| 08.55 - 09.00 | Welcome and presentation of the day (Aurélia Fèvre) |
| 09.00 - 09.20 |
(MUMPS team) Short presentation of MUMPS (History, Users, Functionalities) |
| 09.20 - 09.50 |
Stéphane Pralet and Jean-Pierre Delsemme,
SAMTECH, BELGIUM Integration of MUMPS in SAMCEF Mecano |
| 09.50 - 10.20 |
Stéphane Operto,
GEOAZUR, FRANCE
Seismic wave propagation modeling using a frequency-domain finite difference method: Application to seismic imaging |
| 10.20 - 10.50 | Coffee Break |
| 10.50 - 11.10 |
(MUMPS team) Controlling MUMPS accuracy and efficiency |
| 11.10 - 11.30 |
(MUMPS team) Future functionalities and on-going projetcs |
| 11.30 - 11.55 |
Emmanuel Agullo, PhD student, ENS Lyon Out-Of-Core parallel factorization |
| 11.55 - 12.20 |
Tzvetomila Slavova, PhD student, CERFACS Out-Of-Core parallel solution |
LUNCH |
|---|---|
| 13.50 - 14.20 |
Guillaume Sylvand, EADS, FRANCE Simulation in electromagnetism at EADS-CRC using MUMPS for coupled BEM/FEM |
| 14.20 - 14.50 |
Ken Stanley, Interactive Supercomputing, Sandia National Labs, USA Power to the People: Bringing MUMPS to the masses |
| 14.50 - 15.20 |
Hong Zhang, Dept. of Computer Science, Illinois Institute of Technology and
Mathematics and Computer Science Division, Argonne National Laboratory Design, Implementation and Applications of PETSc-MUMPS Interface |
| 15.20 - 15.50 | Coffee Break |
| 15.50 - 16.20 |
(MUMPS team) Parallelism in MUMPS |
| 16.20 - 16.50 |
Luc Giraud, ENSEEIHT-IRIT, FRANCE From direct to iterative substructuring: some parallel experiences in 2 and 3D |
| 16.50 - 17.50 | General Discussion |
| 19.30 | Dinner in Lyon |
Abstracts of users' and applications presentations
Integration of MUMPS in SAMCEF Mecano (Stéphane Pralet)
SAMCEF Mecano is general finite element software that solves nonlinear structural and mechanical problems. It provides specific answers regarding your analysis: MECANO Structure, for the non-linear analysis of structures, MECANO Motion, for the analysis of flexible mechanisms and MECANO Cable, for the analysis of cables subjected to electro-dynamic efforts. A Newton-Raphson strategy is used in order to solve non-linear equations in an implicit scheme. At a given Newton iteration, a linear system of equations has thus to be solved. The 2 main linear solvers available in Mecano are BCSLIB (sequential multifrontal approach) and MUMPS (parallel multifrontal approach).
In the first part of our talk we will present the evolution of the interface between Mumps and Mecano. In a first implementation the linear solver and the problem generation were running on separate processes and data were exchanged through files. This interface has been substituted by a tighter approach where Mumps and Mecano use the same MPI processes and where data are exchanged through the main memory.
In the second part of our talk we will illustrate MUMPS advantages and we will discuss possible improvements. To our mind the strength of MUMPS is based on the alliance of a competitive parallel distributed factorization and of the availability of advanced functionalities. To show MUMPS efficiency, we will compare it with BCSLIB in the context of large test cases arising from mechanical engineering. To give an example of use of advanced functionalities, we will describe how the Schur complement feature is exploited to solve contact problems.
Seismic wave propagation modeling using a frequency-domain finite difference
method: Application to seismic imaging (Stéphane Operto)
[Slides available (.ppt.gz)]
Seismic methods are among the most powerful approaches to image the structure of the earth at different scales (from subsurface to the deep crust). Artificial sources such as explosions are used to propagate elastic waves into the basement. A receiver array records the seismic arrivals which carry indirect information on some physical properties of the basement (wave velocities, density, attenuation, anisotropy). The objective of seismic imaging is to image these physical properties (the model) from the seismic recordings (the data) using an optimization procedure (non linear inverse problem).
The non linear relationship between the seismic data and the physical properties of the basement is described by the wave equation whose heterogeneous coefficients are the medium properties. A key issue is to use an efficient tool for the numerical resolution of the wave equation. For this, we use a finite-difference frequency-domain (FDFD) method. The associated numerical problem is the resolution of a sparse system of linear equations with multiple right-hand side (RHS) terms. Each RHS term correspond to a source of the experiment, the matrix coefficients depend on the medium properties and the solutions are the modeled wavefields. The rank of the matrix is the product of the dimensions of the basement model. A specificity of our applications is the large number of RHS terms which directs us towards direct methods since the matrix factorization is independent of the RHS terms. We use MUMPS on a PC cluster to solve this system of equations with multiple RHS terms.
Several applications were finalized in 2D. The factorization and the multiple resolution phases are computed in parallel with MUMPS. We exploit the distribution of the system solutions on several processors to compute in parallel the subsequent inverse problem (which is implemented as a weighted summation over the system solutions).Abstract Presently, we are developing a 3D FDFD code for seismic wave propagation modeling based on MUMPS to assess whether representative problems can be adressed with a direct solver keeping in mind that the memory requirement will be the key issue in 3D.
Simulation in electromagnetism at EADS-CRC using MUMPS for coupled BEM/FEM (Guillaume Sylvand)
We are interested in the numerical simulation of wave propagations in electromagnetism governed by the Maxwell equations. We have to solve this kind of problem in fields such as stealth conception, antenna design and placement, etc. More precisely, we choose to solve these equations in the frequency domain and in integral form, which results in having to solve a full complex linear system. The default solver for these equations is a direct Cholesky algorithm, but iterative solvers and FMM method are available to speed-up the resolution for large systems.
However, the BEM can not simulate heterogeneous propagation media which are sometime used for realistic industrial cases. We have chosen to treat these parts of the mesh with a volumic finite element method based on a tetrahedral meshing. The system resulting of this coupling scheme is a 2x2 linear system, with one full block (on the boundary) and three sparse block. Different solvers have been tested on this system (direct, iterative, Schur complement) using MUMPS whenever the inversion of a large sparse matrix is required. Problems with several millions of unknowns (surfacic + volumic) have been solved on parallel machines.
Power to the People: Bringing MUMPS to the masses (Ken Stanley)
MUMPS is a terrific solver and can harness the power of distributed computers to solve very large sparse matrices. Programming for distributed computing, however, continues to be a time-consuming and hence expensive task. Amesos and Star-P provide interfaces which greatly reduce the cost of programming for distributed computing, making MUMPS attractive to a larger customer base.
Amesos is an object-oriented interface to sparse direct solvers within the Trilinos package. Amesos allows users easy access to several solvers and to switch solvers by changing only a couple characters. Trilinos allows users to develop robust numerical software using modern obejct-oriented techniques while leveraging the value of established numerical libraries.
Star-P allows matlab users to harness the power of distributed computing with virtually no development effort and allows users to go straight from prototype to prodcution codes.
I will describe the Amesos and Star-P and discuss how they make MUMPS easier to use.
Design, Implementation and Applications of PETSc-MUMPS Interface (Hong Zhang)
[Slides available (.ppt.gz)]
PETSc (Portable, Extensible Toolkit for Scientific Computation) is a suite of data structures and routines for the scalable solution of scientific applications modeled by partial differential equations. MUMPS is a high-performance direct sparse solver. The PETSc-MUMPS interface enables our users to easily invoke the MUMPS solver at runtime for either algorithmic study or solving computational-intensive problems under the PETSc environment.
In this talk, first, I will present the design and implementation of the PETSc-MUMPS interface. I will then demonstrate how MUMPS is used on solving large-scale eigenvalue problems arising from the nanoscale materials simulation. Finally, I will discuss what improvements can be envisaged. If time permits, I will give a short demo on using PETSc- MUMPS interface.
From direct to iterative substructuring: some parallel experiences in 2 and 3D
(Luc Giraud)
[Slides available (.pdf)]
One route to the solution of large sparse linear systems in parallel scientific computing is the use of hybrid methods that combine direct and iterative methods. These techniques inherit the advantages of each approach, namely the limited amount of memory and easy parallelization for the iterative component and the numerical robustness of the direct part.
In this talk, we will consider some parallel solution techniques for the solution of linear system arising from the discretization of PDEs. These approaches are often called the substructuring methods, referring to structural mechanics, historically its first area of application. It consists in splitting the mesh into non-overlapping sub-meshes/subdomains (or equivalently splitting the graph of the matrix), first eliminating the unknowns associated with the interior nodes from the equations associated with the nodes on the interfaces, then solving the interface problem and finally solving back the internal problems on each subdomain. The solution of the interface problem, also referred to as condensed system, can be performed using either a factorization approach or a preconditioned Krylov solver.
We will present, how some of the uniques features of MUMPS enable us to implements these schemes on parallel platforms. Numerical experiments and parallel performances will be reported for the solution on 2D problems arising in semiconductor device modelling simulation. Preliminary results on 3D heterogeneous diffusion problems will also be described.
MUMPS team presentations
Short presentation of MUMPS (History, Users, Functionalities)
General introduction and presentation of the MUMPS package.
Controlling MUMPS accuracy and efficiency
The objective of this presentation is to present the main control parameters of MUMPS and explain how default parameters can be modified to solve a numerically difficult problem more accurately, solve a huge problem more efficiently, or adapt the use of MUMPS to a particular situation resulting from the application context. The various functionalities and the effects of the different control parameters will be illustrated on practical examples.
Future functionalities and on-going projects
We will present the on-going projects and collaborations, as well as the functionalities that we plan to develop in the next few years. These developments and research work result from common needs and/or limits observed by our users when using the package in various situations.
Out-of-Core parallel factorization and solution
Memory is usually the main bottleneck to solve arbitrarily large problems with a direct method. Because demands from numerical simulation lead to huge matrices for which the physical memory of modern computers is not sufficient, we have developed a beta version of an out-of-core extension to MUMPS that manages the factors out-of-core. Two PhD students, Emmanuel Agullo (LIP-ENS Lyon) and Tzvetomila Slavova (CERFACS) are currently working on research issues on the out-of-core factorization phase and the out-of-core solution phase, respectively. The objectives are to further reduce the in-core working space of the factorization phase and improve the performance of the out-of-core solution phase.
Parallelism in MUMPS
In the parallel context, MUMPS uses a combination of static and dynamic scheduling techniques to map the computationnal tasks over the working processors. In this context, it is crucial to take into account various constraints not only linked to performance but also to memory requirements or volume of I/O (in the out-of-core context). We will describe different ways of managing parallelism and ongoing work in this topic (dynamic scheduling strategies, sophisticated static techniques, I/O constrained schedulers, ...)