Running mpi, jomp, and openMP from the cluster (cs.sou.edu)

1. Install putty and WinSCP3 onto your window system. The programs are free downloads. Installation is straightforward, online documentation is available. These programs allow you to remotely log into the grafiti2 system and copy files back and forth between the cluster and your windows system. You can also install cygwin as an alternative to putty. This will allow you to run a Unix-like system from within widows.
   
   
2. Log onto grafiti2 using putty in ssh mode (host: cs.sou.edu)
   (account: <your account>, password: <your password>)
   
   From cygwin: >ssh –l username –X and then enter your password
   
   
3. Commands to configure ssh (this is necessary or the mpj installation described below will not work)
   
   >ssh-keygen -t rsa
   >cat $HOME/.ssh/id_rsa.pub >> $HOME/.ssh/authorized_keys
   
   Note: -t specifies type of encryption. Then copy the public key to the authorized_keys file.
   Note: The following command might be necessary: >chmod 640 authorized_keys
   
   
4. Check to determine if the working directory (.) is in your $PATH variable (echo $PATH). If it is not, add colon dot (:.) To one of the $PATH instructions below. Otherwise you will need to type ./<progname> when to execute programs in your working directory.

Install and test mpj

1) Download from http://mpj-express.org/download.php or in grafiti2, look in /tmp/cs455

2) Installation (already done in grafiti): tar zxovf mpj-v0_27.tar.gz

3) Edit either .bashrc or .tcsrh depending on the shell you are using

edit the file: .bashrc

export MPJ_HOME=/tmp/cs455/mpj

export PATH = $PATH:$MPJ_HOME/bin

export CLASSPATH=$CLASSPATH:$MPJHOME/lib/mpj.jar

Note: if there was no $CLASSPATH variable defined, you do not need $CLASSPATH: that starts the above line.

edit the file: .tcsrh

setenv MPJ_HOME /tmp/cs455/mpj

setenv PATH ${PATH}:${MPJ_HOME}/bin
setenv CLASSPATH ${CLASSPATH}:${MPJ_HOME}/lib/mpj.jar

Note: if there was no $CLASSPATH variable, you do not need ${CLASSPATH}: that starts the above line

4) Create folder for mpj programs (ex: mpjUser)

5) You can skip this step if you are running mpj on grafiti. The edited mpj.sh file is in $MPJ_HOME/bin folder.

a. Copy the script file runmpj.sh from $MPJ_HOME/bin to the mpjUser directory.

b. Change the name of runmpj.sh that you copied in the previous step to mpj.sh in the mpjUser directory

c. Edit mpj.sh that you just copied. There are two changes to be made so the ssh environment is set when you run mpj programs

o First Edit

Change:
ssh $host "cd $dir; export MPJ_HOME=$MPJ_HOME ; \
java -cp \"$MPJ_HOME/lib/mpj.jar${CLASSPATH_SEPARATOR}$CLASSPATH\" \
-jar $name $count $conf niodev;" &

;;

To:
ssh $host

"cd $dir; \
export MPJ_HOME=$MPJ_HOME ; \
export JAVA_HOME=$JAVA_HOME; \
export PATH=$PATH:$JAVA_HOME/bin ; \
export CLASSPATH=$CLASSPATH ; \
java -jar $name $count $conf niodev;" &
;;

o Second Edit

Change:
ssh $host "cd $dir; export MPJ_HOME=$MPJ_HOME ; \
java -cp \"$MPJ_HOME/lib/mpj.jar${CLASSPATH_SEPARATOR}$CLASSPATH\" \
$name $count $conf niodev;" &
;;

To:
ssh $host \
"cd $dir; \
export MPJ_HOME=$MPJ_HOME ; \
export JAVA_HOME=$JAVA_HOME ; \
export PATH=$PATH:$JAVA_HOME/bin ; \
export CLASSPATH=$CLASSPATH ; \
java $name $count $conf niodev;" &
;;

6) Create a mpj.conf file in your MPJ directory. It is a text file with one or more lines with localhost or valid ip addresses. The following example is for parallel processing using two processes. For runs with more processors, change the number of processors, and the lines with localhost in it (one for each processor). For each processor, increment the port number by 2. For example the third processor would use port 20004.

# Number of processes
2
# Protocol switch limit
131072
# Entry in the form of machinename@port@rank
localhost@20000@0
localhost@20002@1

7) Write your program. A start up test program follows

import mpi.*;

public class World

{

public static void main(String[] args)
throws Exception

{

MPI.Init(args);

int me = MPI.COMM_WORLD.Rank();

int size = MPI.COMM_WORLD.Size();

System.out.println
("Hi from " + me + " of " + size + "\n");

MPI.Finalize();

}

}

10) Compile the code: >javac World.java

11) Run the program: > mpj.sh mpj.conf World or >mpj.sh mpj.conf ./World if the working directory is not in your $PATH variable

Install and test Jomp

1) Download from http://www2.epcc.ed.ac.uk/computing/research_activities/jomp/download.html or on grafiti2, the jomp1.0b.jar is in /tmp/cs455

2) To install JOMP, add the file jomp1.0b.jar to your CLASSPATH. Edit one of the following depending on the shell you are using

edit the file: .bashrc

export JOMP_HOME=<path to jomp jar file>

export CLASSPATH=$CLASSPATH:$JOMP_HOME/jomp1.0b.jar

edit the file: .tcshrc

setenv JOMP_HOME <path to jomp jar file>

setenv CLASSPATH ${CLASSPATH}:${JOMP_HOME}/jomp1.0b.jar

3) Write your program. A startup test program follows. Your jomp programs MUST have a .jomp extension.

import jomp.runtime.*;

public class HelloWorld

{ public static void main(String[] args)

{ int myid;

//omp parallel private(myid)

{ myid = OMP.getThreadNum();

System.out.println("Hello from " + myid);

}

}

}

4) Compile jomp directives. Note all jomp files end with .jomp, but you do not put the extension in the compile command
Write your program. A startup test program follows. Write your program. A startup test program follows.
>java jomp.compiler.Jomp MyFile
or
>java –classpath <path to jomp jar> jomp.compiler.Jomp MyFile

5) Compile the javafile created by jomp: >javac MyFile.java

6) Execute the program: >java -Djomp.threads=n MyFile

Using MPI

1) Create a machines file, which is a text file containing one or more lines: localhost or ip addresses.

2) An MPI C start up program:

/* Mpi Hello World Program - MpiHello.c */

#include <mpi.h>

#include <stdio.h>

int main(int argc, char *argv[])

{ int myRank, size;

MPI_Init(&argc, &argv);

MPI_Comm_rank(MPI_COMM_WORLD, &myRank);

MPI_Comm_size(MPI_COMM_WORLD, &size);

if (size<2) MPI_Abort(MPI_COMM_WORLD, 1);

if (myRank==0)

{ printf("Hello World - I'm the master\n");

}

else

{ printf("Hello World - I'm slave %u\n", myRank);

}

MPI_Finalize();

}

1. Edit the vi .mpd.conf file in your home directory: one line -> MPD_SECRETWORD=harveyd
2. Change permissions on the .mpd.conf file: >chmod 700 .mpd.conf
3. To Compile: mpicc -g MpiHello.c -o MpiHello
4. Start the mpi mpd daemon: mpd --daemon
5. To Execute with a default machines file: mpirun -np 4 MpiHello
   Note: the switch –machinefile <filename> uses your own machines file
   Note: You may have to type ./MpiHello if the working path is not in your $PATH variable
6. To shut down mpi: mpdallexit
   
   

Getting and installing openMP

1. You can install on your own computers using the following instructions. You need installation security privileges. This is already done on grafiti, so if running on that machine, skip to step 10.

2. Get the file Get the Omni distribution,"Omni-1.3.tar.gz"

3. unpack the distribution: tar zxovf Omni-1.3.tar.gz

4. cd <Omni unpacked directory>

5. configure

6. make

7. make install

8. Add the <install directory>/bin to your $PATH variable

9. On grafiti /tmp/cs455 contains the tar.gz file plus the Omni folder.

10. An openMP HelloWorld Program using C

#include <omp.h>

int main (int argc, char* argv[])
{

int nthreads, tid;

#pragma omp parallel private(nthreads, tid)

{

tid = omp_get_thread_num();

printf("Hello World from thread = %d\n", tid);

if (tid == 0)

{

nthreads = omp_get_num_threads();

printf("Number of threads = %d\n", nthreads);

}

}

}

1) Compile: omcc –o HelloWorld.c –o HelloWorld
Note: on grafiti, omcc is in /usr/local/bin

Appropriately set the environment variable OMP_NUM_THREADS

bash: export OMP_NUM_THREADS = 4
csh: setenv OMP_NUM_THREADS 4

2) Execute: >HelloWorld or >./HelloWorld if working directory is not in the search path