This package contains Mpi applications and packages for testing and 
evaluation.



NOTE TO RPM USERS
-------------------------------------------------------------------
If you have installed the opa-fastfabric package through your distro
you likely only have a subset of the applications required for full
testing. Certain run scripts, builds, and functionality will be 
limited. To enable full functionality, the opa-mpi-apps package
must be installed. This package is available from Intel and contained
in Intel Fabric Suite (IFS). 


MPICH_PREFIX ENVIRONMENT VARIABLE:
----------------------------------

By default, Make will build these programs with the MPI installed in
 /usr/mpi/gcc/openmpi* 

You can override this by setting MPICH_PREFIX in your .bashrc file. For
example, the following will use Intel MPI:

export MPICH_PREFIX=/opt/intel/oneapi/mpi/latest/


COMPILING THE TEST PROGRAMS:
----------------------------

The top level Makefile will build all the applications and benchmarks,
using the default (described above) or the value of MPICH_PREFIX (also
described above).

Some useful make targets include:

	clean          - remove all binaries. Useful when changing MPIs.
	all/full/quick - currently synonyms. All build all targets.

COMPILING HPL
-------------

A version of the High Performance Computing Linpack Benchmark (HPL) is
provided, but requires that the build environment have either OpenBLAS or
the Intel Math Kernel Library (MKL) available. When compiling the MPI test
programs, the Makefile will look first for OpenBlas and then for MKL. If
neither is found, compilation will fail with an error. If you wish to use
a different BLAS library, please see the documentation in the HPL source
for information on manually configuring and building HPL.


RUNNING THE TEST PROGRAMS:
--------------------------

A top level "run" script is provided to run all the applications as a
regression test. Other scripts will run individual benchmarks. These
scripts assume the existence of a local "mpi_hosts" file [alternate name can be
exported in MPI_HOSTS if desired] (in mpirun mode)
and that mpd is already running (if you're using mvapich2). For example,
to run the bandwith test, simply type:

# ./run_bw

The scripts log the output of the MPI programs to the 
/usr/src/eth/mpi_apps/logs directory.

RUNNING HPL
-----------

When running the provided version of HPL, an additional script has been
provided for configuring it with common sample problem sizes. Use the
provided "config_hpl2" script to select from one of several sample HPL
configuration files before running HPL with the provided "run_hpl2" script.

TUNING THE TEST PROGRAMS:
-------------------------

MVAPICH, MVAPICH2 and OpenMPI are all extremely tunable, and a complete
discussion of all their settings is far beyond the scope of this document.
However, these scripts to provide examples of tuning the different MPIs
as a way to get started.

When using IFS, you can tune any of the MPIs. See the files
mvapich2.params and openmpi.params for details.
Each file describes the syntax of the parameter file, provides a link to
online documentation on how to tune that MPI and several sample parameters.
