MPI

In the usual sequential programming model, the sequence of instructions corresponding to the compiled code of the program executes on a single core of your processor. However, computations can be very time and memory consuming ; this is often the case for 3D computations in numerical simulation for example. In this case, parallel programming will allow us to take advantage of all available cores, and ultimately to develop efficient scientific codes that can exploit hundreds of computing nodes of a supercomputer to solve the problem at hand.

Here we will focus more precisely on code parallelization using the MPI (for Message Passing Interface) parallel programming paradigm. In this programming model, the program will be duplicated on multiple processes, which will run on the different computing cores. Each process executes a copy of the program and has access to its own memory. A process can directly access only its own memory, but can exchange messages with other processes in order to send and receive information. The communication between processes is therefore done by passing messages, through functions of the MPI library called in the program.

The first C++ MPI Hello World program below allows you to see how to compile and run an MPI program, as well as check that our working environment is functional.

#include <iostream>
#include <mpi.h>

int main() {
MPI_Init(NULL, NULL); // Initialize the MPI environment

int size;
MPI_Comm_size(MPI_COMM_WORLD, &size); // Get the number of processes

int rank;
MPI_Comm_rank(MPI_COMM_WORLD, &rank); // Get the rank of the process

std::cout << "Hello World from processor " << rank << " out of " << size << " processors !" << std::endl;

MPI_Finalize(); // Finalize the MPI environment

return 0;
}

After placing the above code in a mpi_hello_world.cpp file, you can compile it using the following command:

mpic++ mpi_hello_world.cpp -o mpi_hello_world

This command calls the C++ compiler and manages the dependencies with the MPI library.

In order to launch the parallel execution of the program with 4 processes for example, type

mpirun -np 4 ./mpi_hello_world

The output should look like this:

Hello World from processor 0 out of 4 processors !
Hello World from processor 1 out of 4 processors !
Hello World from processor 3 out of 4 processors !
Hello World from processor 2 out of 4 processors !