From Code to Performance

Exercise 2: HPC Practice Material

1. MPI Basics Practice

Objective: Learn distributed memory programming with MPI.

Exercise 1 - Hello World:

    1. Write an MPI program that prints Hello World program.
    1. Compile and run the program using 4 processes.

Exercise 2 - Sum of Numbers Across Processes:

  • Distribute the computation of the sum of numbers 1–100 across multiple processes.
  • Each process should compute a partial sum and send it to the root process. Each process computes a partial sum, and the root process prints the total sum.

Commands to Practice:

mpicc hello_mpi.c -o hello_mpi mpirun -np 4 ./hello_mpi

2. OpenMP Basics Practice

Objective: Learn shared memory parallelism.

Exercise 1 - Hello World with Threads

  • Write an OpenMP program that prints Hello World from multiple threads.
  • Compile and run the program with different thread counts (e.g., 2, 4, 8).

Exercise 2 - Parallel Loops

  • Write an OpenMP program that computes the sum of an array in parallel using a for loop.
  • Use the #pragma omp parallel for directive to parallelize the loop.

Commands to Practice:

gcc -fopenmp hello_omp.c -o hello_omp ./hello_omp

3. Hybrid Challenge

  • Use MPI to distribute an array of numbers across processes.
  • Within each process, use OpenMP to compute partial sums in parallel.
  • Use MPI to reduce partial sums and print the total sum in the root process.

4. Extra Exercises

  • Modify MPI Hello World to print process IDs in a different order.
  • Experiment with different numbers of threads in OpenMP.
  • Add timing functions to measure performance improvement with parallelism.