Know your Hardware
Let's review the Setonix
CPU node hardware (page 30)
We performed a range of single-node jobs of a hybrid MPI/OpenMP computer
code for calculating positron-hydrogen scattering using the Convergent
Close-Coupling (CCC) method, which is installed on Setonix and at the
Atomic, Molecular and Optical
Science Gateway.
- Connect to Setonix with ssh -X or -Y (we'll need X11 for viewing profiler output and
plotting using gnuplot)
- cd /software/projects/courses0100/igor/positron-hydrogen
- ls
- Two sets of jobs
corresponding to single and double precision.
- All jobs utilise 128 cores, but in various combinations of MPI
processes (p) and OMP threads (t) in p_t
directories. Note, pt=128
- CCC first calculates a large matrix VMAT used to define A
- Then solves resulting linear equations Ax=b using ScaLAPACK
Questions:
- What is the optimal p_t combination for the smallest execution
time?
- Which is faster single or double precision, and by how much?
- Which lines in the CCC code take the most time?
Will use the ARM profiler to answer the above questions:
- module load arm-forge gnuplot/6.0.0
- cd p_t of interest
- map *.map &