Lab: MPI
Table of Contents
This lab is for you to learn a little about MPI, the Message Passing Interface. We'll be using Open MPI, an implementation of MPI that has been made available for a number of languages, including Java. The documentation is strangely not online, but is available locally on any computer that has Open MPI installed. On the lab machines, it is linked here.
Hello World
Here's the code for a simple Hello World program:
import mpi.*; import java.net.*; public class HelloWorld { public static void main(String[] args) throws MPIException, UnknownHostException { MPI.Init(args); int size = MPI.COMM_WORLD.getSize(); int rank = MPI.COMM_WORLD.getRank(); String name = MPI.COMM_WORLD.getName(); String hostname = InetAddress.getLocalHost().getHostName(); System.out.println("Hello, World! I am process " + rank + " of " + size + " on " + name + " on host " + hostname); MPI.Finalize(); } }
Copy this into a file HelloWorld.java
. Compile and run it with the following
commands:
javac -cp .:/usr/local/lib/mpi.jar HelloWorld.java mpirun -n 5 java HelloWorld
You should get hellos from five distinctly numbered processes.
One of the fabulous capabilities of MPI is that in can run parallel jobs on multiple computers easily. Split up HelloWorld so that it runs on your computer as well as the computers next to you.
javac -cp .:/usr/local/lib/mpi.jar HelloWorld.java mpirun -n 8 -H cmc306-04,cmc306-05,cmc306-06 java HelloWorld
Sending and Receiving Messages
You can send and receive messages using the the send
and recv
methods of
MPI.COMM_WORLD
. These methods both take Buffer
(of which IntBuffer
is a
subclass), which is much like an array but is optimized for transfering
data. For example, here is a program in which the root process (process 0) sends
a message to process 1.
import mpi.* import java.nio.*; public class SendRecv { public static void main(String[] args) throws MPIException{ MPI.Init(args); int size = MPI.COMM_WORLD.getSize(); int rank = MPI.COMM_WORLD.getRank(); String name = MPI.COMM_WORLD.getName(); if (size < 2) { System.out.println("Too few processes!"); System.exit(1); } IntBuffer buffer = MPI.newIntBuffer(1); int numItemsToTransfer = 1; int sourceRank = 0; int destinationRank = 1; int messageTag = 0; // supplemental tag, often just not used int bufferPosition = 0; if (rank == 0) { int valueToTransmit = 5; buffer.put(bufferPosition, valueToTransmit); MPI.COMM_WORLD.send(buffer, numItemsToTransfer, MPI.INT, destinationRank, messageTag); } else if (rank == 1) { Status status = MPI.COMM_WORLD.recv(buffer, numItemsToTransfer, MPI.INT, sourceRank, messageTag); System.out.println("Process 1 received value " + buffer.get(bufferPosition) + " from process 0"); } MPI.Finalize(); } }
Exercise 1
Write a program that sends values in a circle, with each process sending a value to the next highest ranked process except for the highest-ranked process, which sends to process 0). Make sure to avoid deadlock. Whenever each process receives a value, it should print a line of text indicating which process it is, which process it received the value from, and what the value was.
One challenge you may or may not run into in the above exercise is that the
results of the output may be interleaved, sometimes out of
order. System.out.println
is synchronized and thread-safe, but here we have
multiple single-threaded processes all writing into the same output buffer, and
the timing on how the results get flushed may not happen in the order that you
expect. This problem seemed to be much more prevalent on my home Linux machine
than on the department Macs, but I've seen it at least occasionally on the lab
Macs as well. If you think that the terminal window is showing you the output in
an incorrect order, here are some tricks you can do to verify that you are
seeing the output in precisely the order it came through:
- In your
System.out.println
statements, insert a call toSystem.nanoTime()
to get a timestamp. Make sure that the timestamp is the first thing to get printed on each line. Instruct
mpirun
to output the result of each process to a separate file, so that the results of each process don't get interleaved:mpirun -n 5 -output-filename result java HelloWorld
When done, use the command-line program
sort
to concatenate together the various files in timestamp order:sort result*
Broadcast communication
MPI's collective functions allow communications among all processes in a group. Broadcast sends a message to every process. Reduce collects values from all processes and aggregates them.
Here is some sample code that uses broadcast and reduce functionality. Copy the code into a file, and run it. See if you can figure out line-by-line what it's doing. Ask questions if you have any.
import mpi.*; import java.nio.*; public class BcastReduce { public static void main(String[] args) throws MPIException { MPI.Init(args); int size = MPI.COMM_WORLD.getSize(); int rank = MPI.COMM_WORLD.getRank(); String name = MPI.COMM_WORLD.getName(); IntBuffer buffer = MPI.newIntBuffer(1); int numItemsToTransfer = 1; int sourceRank = 0; int bufferPosition = 0; int numberToPlayWith = 5; buffer.put(bufferPosition, numberToPlayWith * 3); MPI.COMM_WORLD.bcast(buffer, numItemsToTransfer, MPI.INT, sourceRank); int valueReceived = buffer.get(0); int valueModified = valueReceived + 4; buffer.put(bufferPosition, valueModified); IntBuffer finalAnswer = MPI.newIntBuffer(1); int initialValue = 0; MPI.COMM_WORLD.reduce(buffer, finalAnswer, numItemsToTransfer, MPI.INT, MPI.SUM, initialValue); if (rank == 0) { System.out.println("Final answer = " + finalAnswer.get(0)); } MPI.Finalize(); } }
Exercise 3
A classic example is computing the vaue of pi using numerical integration. Basically, we trying to calculate the area under the curve \(f(x) = 4 / (1 + x^2)\) between \(x = 0\) and \(1\), which happens to be pi. The more intervals we divide it into, the more accurate the answer. The root process broadcasts the number of intervals (n) to all processes, which compute part of the sum, then the reduce function
Here is pseudocode for the algorithm to compute pi; see if you can translate it to MPI! In case you are wondering if your solution is right, the answer I get for \(n = 1000\) is \(3.141593\).
root process only: ask the user for a value n broadcast the value n to all processes width <- 1 / n sum = 0 forall 1 <= i <= n (in parallel - divide the i values between processes somehow): sum <- sum + 4 / (1 + (width * (i - 0.5)) ** 2) mypi <- width * sum sum all processes' mypi using reduce print sum