C – FASRC DOCS https://docs.rc.fas.harvard.edu Tue, 14 Oct 2025 02:52:06 +0000 en-US hourly 1 https://wordpress.org/?v=6.8.3 https://docs.rc.fas.harvard.edu/wp-content/uploads/2018/08/fasrc_64x64.png C – FASRC DOCS https://docs.rc.fas.harvard.edu 32 32 172380571 MPI (Message Passing Interface) & OpenMPI https://docs.rc.fas.harvard.edu/kb/mpi-message-passing-interface/ Fri, 31 Jan 2025 18:49:19 +0000 https://docs.rc.fas.harvard.edu/?post_type=epkb_post_type_1&p=28106

Introduction

The Message Passing Interface (MPI) library allows processes in your parallel application to communicate with one another by sending and receiving messages. There is no default MPI library in your environment when you log in to the cluster. You need to choose the desired MPI implementation for your applications. This is done by loading an appropriate MPI module. Currently the available MPI implementations on our cluster are OpenMPI and Mpich. For both implementations the MPI libraries are compiled and built with either the Intel compiler suite or the GNU compiler suite. These are organized in software modules.

Installation

MPI has many forms, we’ll list a few here, and also look at User_Codes/Parallel_Computing/MPI.

mpi4py with Python

We recommend the Mambaforge Python distribution. The latest version is available with the python/3.10.9-fasrc01 software module. Since mpi4py is not available with the default module you need to install it in your user environment.

The most straightforward way to install mpi4py in your user space is to create a new conda environment with the mpi4py package. For instance, you can do something like the below:

module load python/3.10.12-fasrc01 
mamba create -n python3_env1 python numpy pip wheel mpi4py
source activate python3_env1

This will create a conda environment named python3_env1 with the mpi4py package and activate it. It will also install a MPI library required by mpi4py. By default, the above commands will install MPICH.

For most of the cases the above installation procedure should work well. However, if your workflow requires a specific flavor and/or version of MPI, you could use pip to install mpi4py in your custom conda environment as detailed below:

Load compiler and MPI software modules:

module load gcc/12.2.0-fasrc01
module load openmpi/4.1.5-fasrc03

This will load OpenMPI in your user environment. You can also look at our user documentation to learn more about software modules on the FAS cluster.

Load a Python module:

module load python/3.10.12-fasrc01

Create a conda environment:

mamba create -n python3_env2 python numpy pip wheel

Activate the new environment:

source activate python3_env2

Install mpi4py with pip:

pip install mpi4py

For example code, see Parallel_Computing/Python/mpi4py

OpenMPI with GNU Compiler

If you want to use OpenMPI compiled with the GNU compiler you need to load appropriate compiler and MPI modules. Below are some possible combinations, check module spider MODULENAME to get a full listing of possibilities.

# GCC + OpenMPI, e.g.,
module load gcc/13.2.0-fasrc01 openmpi/5.0.2-fasrc01

# GCC + Mpich, e.g.,
module load gcc/13.2.0-fasrc01 mpich/4.2.0-fasrc01

# Intel + OpenMPI, e.g.,
module load intel/24.0.1-fasrc01 openmpi/5.0.2-fasrc01

# Intel + Mpich, e.g.,
module load intel/24.0.1-fasrc01 mpich/4.2.0-fasrc01

# Intel + IntelMPI (IntelMPI runs mpich underneath), e.g.
module load intel/24.0.1-fasrc01 intelmpi/2021.11-fasrc01

For reproducibility and consistency it is recommended to use the complete module name with the module load command, as illustrated above. Modules on the cluster get updated often so check if there are more recent ones. The modules are set up so that you can only have one MPI module loaded at a time. If you try loading a second one it will automatically unload the first. This is done to avoid dependencies collisions.

There are four ways you can set up your MPI on the cluster:

  • Put the module load command in your startup files.
    Most users will find this option most convenient. You will likely only want to use a single version of MPI for all your work. This method also works with all MPI modules currently available on the cluster.

  • Load the module in your current shell.
    For the current MPI versions you do not need to have the module load command in your startup files. If you submit a job the remote processes will inherit the submission shell environment and use the proper MPI library. Note this method does not work with older versions of MPI.

  • Load the module in your job script.
    If you will be using different versions of MPI for different jobs, then you can put the module load command in your script. You need to ensure your script can execute the module load command properly.

  • Do not use modules and set environment variables yourself.
    You obviously do not need to use modules but can hard code paths. However, these locations may change without warning so you should set them in one location only and not scatter them throughout your scripts. This option could be useful if you have a customized local build of MPI you would like to use with your applications.

Parallel HDF5

Parallel HDF5 (PHDF5) is the parallel version of the HDF5 library. It utilizes MPI to perform parallel HDF5 operations. For example, when an HDF5 file is opened with an MPI communicator, all the processes within the communicator can perform various operations on the file. PHDF5 supports file operations such as file create, open and close, as well as dataset operations such as object creation, modification and querying, all in parallel using MPI-IO. User Codes has examples are intended to illustrate the use of PHDF5 on the Cannon cluster. The specific examples are implemented in Fortran, but the could be easily translated to C or C++.

Video Training

Examples

For associated MPI examples, head over to  User_Codes/Parallel_Computing/MPI.

  • Example 1: Monte-Carlo calculation of π
  • Example 2: Integration of x2 in interval [0, 4] with 80 integration points and the trapezoidal rule
  • Example 3: Parallel Lanczos diagonalization with reorthogonalization and MPI I/O

Resources

]]> 28106 Cpp, C++ Programming Language https://docs.rc.fas.harvard.edu/kb/cpp-programming-language/ Tue, 30 Apr 2024 13:44:17 +0000 https://docs.rc.fas.harvard.edu/?post_type=epkb_post_type_1&p=26934 Description

C++ (C plus plus) is an object-oriented high-level programing language. C++ files typically have .cpp as the file extension. You can compile C++ codes with either GNU compilers (gcc) or Intel compilers (intel).

Best Practice

We recommend requesting an interactive job to compile a C++ program on a compute node (instead of a login node). The compilation could take up to few seconds to a minute and depending on the complexity of the code. Additionally, it is best to utilize the test partition to compile and test a program before executing its production run on the cluster as a batch-job.

It is best practice to compile a C++ code separately and then use the executable, generated during compilation, in the production run using the sbatch script. If possible, avoid including the compilation command in the sbatch script, which will recompile the program every time the job is submitted. If any changes are made to the source code, compile the source code separately, and then submit the production run as a batch-job.

Compilers

You can compile a C++ code using either a GNU or an Intel compiler.

GNU compiler

To use C++ with gcc on the FASRC clusters, load gcc compiler via our module system. For example, this command will load the latest gcc version:

module load gcc

If you need a specific version of R, you can search with the command

module spider gcc

To load a specific version

module load gcc/10.2.0-fasrc01

For more information on modules, see the Lmod Modules page.

To compile a code using a specific version of the GNU compiler and the O2 optimization flag, you can do the following:

module load gcc 
g++ -O2 -o sum.x sum.cpp

Intel compiler

To use C++ with Intel on the FASRC clusters, load intel compiler via our module system. For example, this command will load the latest intel version:

module load intel

If you need a specific version of R, you can search with the command

module spider intel

To load a specific version

module load intel/24.0.1-fasrc01

For more information on modules, see the Lmod Modules page.

Intel recommendations and notes

  • Intel released Intel OneAPI 23.2 with icpx, however, this version does not contain all the features, so we highly recommend using Intel 24 for icpx
  • Intel 17 is quite old. Avoid using it as it can have many incompatibilities with the current operating system
  • Intel has changed its compiler in the past few years and each module may need different flags. Below is a table of executables and possible flags
Intel module versionCommandAdditional flag
intel/17.0.4-fasrc01icpc-std=gnu++98
intel/23.0.0-fasrc01icpc
intel/23.2.0-fasrc01icpx
intel/24.0.1-fasrc01icpx

To compile using a specific version of the Intel compiler, execute:

module load intel/24.0.1-fasrc01
icpx -O2 -o sum.x sum.cpp

Examples

FASRC User Codes

]]> 26934 C Programming Language https://docs.rc.fas.harvard.edu/kb/c-programming-language/ Tue, 23 Apr 2024 10:24:01 +0000 https://docs.rc.fas.harvard.edu/?post_type=epkb_post_type_1&p=26991 Description

C is a general-purpose, procedural computer programming language supporting structured programming, lexical variable scope, and recursion, while a static type system prevents unintended operations. By design, C provides constructs that map efficiently to typical machine instructions and has found lasting use in applications previously coded in assembly language. Such applications include operating systems and various application software for computers, from supercomputers to embedded systems. Wikipedia

Best Practice

We recommend jumping to a compute node for compiling a C program as the compilation could take up to few seconds to a minute depending on the complexity of the code. Additionally, it is best to utilize the test partition to compile and test a program before executing its production run on the cluster as a batch-job.

It is best practice to compile a C code separately and then use the executable, generated during compilation, in the production run using the sbatch script. If possible, avoid including the compilation command in the sbatch script, which will recompile the program every time the job is submitted. If any changes are made to the source code, compile the source code separately, and then submit the production run as a batch-job.

Compilers

You can compile a C code using either a GNU or an Intel compiler.

GNU gcc compiler

To get a list of currently available GNU compilers on the cluster, execute: module spider gcc

The default GNU compiler is typically the latest compiler version on the cluster and can be loaded using module load gcc

To compile a code using a specific version of the GNU compiler and the O2 optimization flag, you can do the following:

module load gcc/9.5.0-fasrc01
gcc -O2 -o sum.x sum.c

Intel icc compiler

To get a list of currently available Intel compilers on the cluster, execute: module spider intel

To compile using a specific version of the Intel compiler, execute:

module load intel/23.0.0-fasrc01
icc -O2 -o sum.x sum.c

Note: If loading an intel module version, refer to the following table for compiling your C code

Intel Comiler Version C Fortran C++
Below 24.0.0 icc ifortran icpc
24.0.0 and above icx ifx icpx

If you load an Intel compiler that is lower than version 24.0.0, you might get this remark

icc: remark #10441: The Intel(R) C++ Compiler Classic (ICC) is deprecated and will be removed from product release in the second half of 2023. The Intel(R) oneAPI DPC++/C++ Compiler (ICX) is the recommended compiler moving forward. Please transition to use this compiler. Use '-diag-disable=10441' to disable this message.

This is just a warning that implies that the user of icc will be deprecated in the second half of 2023. You can quiet this warning by compiling your code using icc in the following manner:

module load intel/23.0.0-fasrc01
icc -O2 -diag-disable=10441 -o sum.x sum.c

Examples

To get started with C on the Harvard University FAS cluster you can try the example shown on our User Codes repository.

Resources

To learn and practice more in C, see the following:

]]> 26991