Lammps

Large-scale Atomic and Molecular Massively Parallel Simulator (LAMMPS)

LAMMPS (Large-scale Atomic and Molecular Massively Parallel Simulator) is a classical molecular dynamics code and has potentials for soft materials (bio-molecules, polymers) and solid-state materials (metals, semiconductors) and coarse-grained or meso-scopic systems. It can be used to model atoms or, more generically, as a parallel particle simulator at the atomic, meso, or continuum scale.

Running on Scafell Pike

Running on Power-8 Panther or Paragon

This script is part of a 3-part workflow, see example here: Panther 3-step Keras workflow. Module lammps/17Nov16 has been loaded in workflow.sh. The example below runs LAMMPS with 4 MPI ranks per node and various thread options to check scaling. Note that the GPU package has not yet been fully tested.

#!/bin/bash
#BSUB -J LAMMPS
#BSUB -o stdout.LAMMPS.txt
#BSUB -e stderr.LAMMPS.txt
#BSUB -q panther
#BSUB -R "span[ptile=4] affinity[core(1)]"
#BSUB -network "type=sn_all: protocol=mpi: mode=US: usage=shared: instance=1"
#BSUB -x
#BSUB -W 1:19
# workflow step dependency
#BSUB -data "tag:stgin_flag"

# directory should be already created and data copied in previous step
cd /gpfs/panther/local/package_build/build/rja87-build/testing/LAMMPS/bench
rm stdout.LAMMPS.txt
rm stderr.LAMMPS.txt

# count how many processors are allocated
NP=0
for TOKEN in $LSB_HOSTS
do
   ((NP++))
done

EXE=`which lmp_p8`

export XLSMPOPTS=parthds=4
export MP_TASK_AFFINITY=cpu:4
poe $EXE -sf omp -var x 2 -var y 2 -var z 1 -log panther.test.log -in in.rhodo.scaled >> echo4.txt

export XLSMPOPTS=parthds=8
export MP_TASK_AFFINITY=cpu:8
poe $EXE -sf omp -var x 2 -var y 2 -var z 2 -log panther.test.log -in in.rhodo.scaled >> echo8.txt

export XLSMPOPTS=parthds=16
export MP_TASK_AFFINITY=cpu:16
poe $EXE -sf omp -var x 4 -var y 2 -var z 2 -log panther.test.log -in in.rhodo.scaled >> echo16.txt

export XLSMPOPTS=parthds=32
export MP_TASK_AFFINITY=cpu:32
poe $EXE -sf omp -var x 4 -var y 4 -var z 2 -log panther.test.log -in in.rhodo.scaled >> echo32.txt

bstage out -tag stgout_flag -src panther.test.log

Running on Invicta and Napier

A simple job submission script for Invicta is as below, note this uses the Intel MPI and MKL libraries. Change to "ptile=24" for Napier.

On Bifort BlueGene/Q

The following script named submit_bg.sh will submit to the BG/Q.

Note that LAMMPS has been compiled with the user-omp module. This is invoked by setting "-sf omp" and "OMP_NUM_THREADS=4" . This seems to make a large improvement to the performance.

Further Information

See LAMMPS Web site: external link: http://lammps.sandia.gov .

LAMMPS can be used for a wide variety of MD simulations, see external link: http://lammps.sandia.gov/features.html . General documentation is available here: external link: http://lammps.sandia.gov/doc/Manual.html .

Some more useful information, including list of additional packages, is included here external link: http://umbc.rnet.missouri.edu/resources/How2RunLAMMPS.html . If there are error messages about missing styles etc. e.g. for long range coulombic solvers which require the kspace package to be installed, please contact hartree@stfc.ac.uk .

Benchmark Results

Please refer to on-line documentation: external link: http://lammps.sandia.gov/doc/Manual.html . The on-line documentation always reflects the current version and commands change occasionally.

For information about LAMMPS benchmarks, see external link: http://lammps.sandia.gov/bench.html .

We are running the Rhodopsin protein scaled benchmark for comparison.

Bifort:

4x processes per node and 4x threads per process.

MPI Processes124816326412825651210242048
Scaled CPU secs      108.00109.48109.67111.65126.08154.47
Scaled Efficiency            

Invicta:

4x processes per node and 4x threads per process.

MPI Processes124816326412825651210242048
Scaled CPU secs28.028.429.1229.430.7630.7330.6031.5932.04   
Scaled Efficiency10098.696.1695.2391.0391.1391.5188.6387.40