Installing Cluster OpenMP* for Intel® Compilers

Taken from Cluster OpenMP* for Intel® Compilers Website

Overview

OpenMP* is a high level, pragma-based approach to parallel application programming. Cluster OpenMP is a simple means of extending OpenMP parallelism to 64-bit Intel® architecture-based clusters. It allows OpenMP code to run on clusters of Intel® Itanium® or Intel® 64 processors, with only slight modifications. 

Prerequisite

Cluster OpenMP use requires that you already have the latest version of the Intel® C++ Compiler for Linux* and/or the Intel® Fortran Compiler for Linux*.

Benefits of Cluster OpenMP

1. Simplifies porting of serial or OpenMP code to clusters.
2. Requires few source code modifications, which eases debugging.
3. Allows slightly modified OpenMP code to run on more processors without requiring investment in expensive Symmetric Multiprocessing (SMP) hardware.
4. Offers an alternative to MPI. Is easier to learn and faster to implement.

How to Install Cluster OpenMP.

1. Installing Cluster OpenMP is simple. First you have to install Intel Compilers. For more information, see Blog Entry Free Non-Commercial Intel Compiler Download
2. After installation of the Compilers, download the Cluster OpenMP License File from Cluster OpenMP Download site
3. Place the Cluster OpenMP License file at the License Directory. Usually it is at /opt/intel/licenses
4. With the Cluster OpenMP license file in place it will make it possible to use either the “-cluster-openmp” or “-cluster-openmp-profile” compiler options with your compiler when compiling a program.

Using Intel® MKL with Threaded Applications

It seems that calling Intel MKL routines that are threaded from multiple application threads can lead to conflict (including incorrect answers or program failures) or at best longer unexpected CPU times.

A good and through description is given by the Intel Website on the issue as well as the workaround
.Using Intel® MKL with Threaded Applications.

The crux of the problem accoring to Intel is as followed:

Intel MKL can be aware that it is in a parallel region only if the threaded program and Intel MKL are using the same threading library. If the user program is threaded by some other means, Intel MKL may operate in multithreaded mode and the computations may be corrupted. Here is Intel recommendation
Here are several cases and our recommendations:

1. User threads the program using OS threads (pthreads on Linux*, Win32* threads on Windows*). If more than one thread calls Intel MKL and the function being called is threaded, it is important that threading in Intel MKL be turned off. Set OMP_NUM_THREADS=1 in the environment.
2. User threads the program using OpenMP directives and/or pragmas and compiles the program using a compiler other than a compiler from Intel. This is more problematic because setting OMP_NUM_THREADS in the environment affects both the compiler's threading library and the threading library with Intel MKL. In this case, the safe approach is to set OMP_NUM_THREADS=1.
3. Multiple programs are running on a multiple-CPU system. In cluster applications, the parallel program can run separate instances of the program on each processor. However, the threading software will see multiple processors on the system even though each processor has a separate process running on it. In this case OMP_NUM_THREADS should be set to 1.
4. If the variable OMP_NUM_THREADS environment variable is not set, then the default number of threads will be assumed 1. 

Setting the Number of Threads for OpenMP* (OMP)

The OpenMP* software responds to the environment variable OMP_NUM_THREADS:

1. Windows*: Open the Environment panel of the System Properties box of the Control Panel on Microsoft* Windows NT*, or it can be set in the shell the program is running in with the command: set OMP_NUM_THREADS=.
2. Linux*: To set and export the variableP "export OMP_NUM_THREADS=".

This is issue was mentioned by Axel Kohlmeyer at this forum on Parallization Issues