Building Chaste for high performance with the Intel compilers

Installing the compiler

See InstallIntelCompiler if you're on an Oxford University machine. If not, you'll need to download it from Intel's website yourself and get a suitable licence from somewhere.

We followed the "install using sudo" route, and installed everything into /opt/intel.

Note: to avoid needing 32-bit libraries, change the components installed to remove the IA-32 support. Select "2. Customize installation" when prompted, and then under "Target Architecture(s) of your applications:" select 1 to de-select IA-32. You can then continue with the default options.

To be able to run the compilers you need to set the INTEL_LICENSE_FILE environment variable and source Intel's setup script. You can add these two lines to the top of your .bashrc to enable the compiler for all sessions: 

source /opt/intel/bin/compilervars.sh intel64
export INTEL_LICENSE_FILE="<path for your system here>"

Building optimised dependencies

See InstallPetscAndMpiForProductionBuild for legacy instructions.

Building Chaste with the Intel compilers

Hostconfig settings

Several variables in your hostconfig file need to be set appropriately to enable building with Intel. The most important is intel_path, since this sets the path to the Intel compiler installation (i.e. the folder containing lib and bin subfolders). With the default install (pre 2016 releases) it will probably be /opt/intel/composerxe. The 2016 compiler release has changed folder structure, and you probably need something like /opt/intel/compilers_and_libraries_2016/linux.

The other variables support using a PETSc built with the Intel compilers, and possibly production builds of PETSc using Intel's MKL libraries. A particular PETSc installation may support multiple architectures/settings, by setting the PETSC_ARCH environment variable when compiling PETSc. You must tell Chaste which architecture to use by setting the petsc_build_name variables. There are 4 of these variables, allowing the use of different PETSc compilation options for different Chaste build types, and the relevant ones for Intel are:

  • petsc_build_name_production - for the IntelProduction build type;
  • petsc_build_name_optimized - for a normal Intel build, since the compiler uses optimisation by default;
  • petsc_build_name - the default if any of the above variables are not specified.

In addition, if PETSc is built to link against Intel's MKL libraries, which provide optimised blas & lapack routines, you need to specify the names of these libraries so that they are linked against in an IntelProduction build, using the blas_lapack_production variable. We use 

blas_lapack_production = ['mkl_lapack', 'mkl', 'svml']

or 

blas_lapack_production = ['mkl_intel_lp64', 'mkl_core', 'mkl_sequential']

It is possible to use PETSc and MPI built using gcc, rather than building your own with the Intel compiler (although using PETSc built with Intel is likely to be faster, so worth doing). If you use PETSc & MPI built with gcc, you must tell the mpicxx compiler wrapper to use the Intel compiler instead of the compiler it was built it (gcc). With OpenMPI, this is done by setting the OMPI_CXX environment variable, and can be configured in Chaste by using the tools dictionary in the hostconfig. You want something like 

    if build.CompilerType() == 'intel':
        tools['mpicxx'] = 'OMPI_CXX=icpc ' + tools['mpicxx']

within the Configure method in your hostconfig file.

A final variable that may be required is icpc which allows you to change the Intel compiler command, e.g. to add flags specific to 64-bit machines or specify the gcc libraries that the compiler should use. However, this is only relevant for legacy MPICH 1 installations, and should be commented out (or removed entirely) if using MPICH 2 or OpenMPI.

See ChasteGuides/HostconfigSystem? for the full reference manual for this section.

Build flags

The most important thing to remember when using the Intel compilers is to use a Chaste build type that enables Intel support. What this means is that your scons invocations should include an argument such as build=Intel (or b=Intel for short). This sets additional compiler flags (e.g. disabling some warnings) and extra library search paths for libraries shipped with the compiler. It also ensures the Intel compiler tools are used.

To run a production build, use build=IntelProduction in place of build=Intel, which will enable further optimisations and link against the MKL.

For more details about build options see ChasteGuides/DeveloperBuildGuide.