HPC

HPC-HPDA on clusters of multi-cores

Description: Programming in MPI+multithreading and use of a multi-core PC cluster for high-performance computing (HPC) and high-performance data analysis. Study of point-to-point and collective MPI communications, mechanisms for controlling the deployment of MPI+multithreading applications on multi-core node clusters. Distributed algorithms for 1D, 2D, and hypercubic problems, synchronous and asynchronous distributed algorithms. Learning how to submit distributed applications in batch mode. Review of performance metrics (speedup, efficiency, size up, scale up) and theoretical performance models; case studies. Implementation on linear algebra problems (HPC), then on a data clustering problem (HPDA), with measurement and analysis of performance and scalability.

Learning outcomes: At the end of this course, students will be able to design distributed algorithms that combine message passing between processes on different computing nodes and multithreaded programming through memory sharing within a process. They will be able to deploy these hybrid codes on multi-core node clusters, search for the most efficient deployments, and measure and analyze their performance. They will have applied this knowledge to traditional HPC calculations and to a less conventional AI problem. In both cases, they will have faced a problem of “scaling” their codes.

Evaluated skills:

  • Development
  • System

Course supervisor: Stéphane Vialle

Geode ID: SPM-INF-020

CM:

  1. Cours d’architectures distribuées (1.5 h)
  2. Programmation MPI en comm pt-a-pt (1.5 h)
  3. Programmation MPI en comm collectives (1.5 h)
  4. Déploiement d’application MPI sur cluster de multi-coeurs (1.5 h)
  5. Rappels de métriques de performance, et modèles de performances théoriques (1.5 h)
  6. Algorithmique et programmation distribuée synchrone (1.5 h)
  7. Algorithmique et programmation distribuée asynchrone (1.5 h)

TD:

  1. TD-MPI-Clustering (1.5 h)

TP:

  1. TP-MPI pb 1D synchrone en MPI+OpenMP et déploiement (3.0 h)
  2. TP-MPI pb 2D synchrone en MPI+OpenMP et déploiement (3.0 h)
  3. TP-MPI pb asynchrone en MPI+OpenMP et déploiement (3.0 h)
  4. TP-MPI-Clustering - part 1 (3.0 h)
  5. TP-MPI-Clustering - part 2 (3.0 h)