G.4 Running FHI-aims with GPU Acceleration

Compiling the FHI-aims executable with GPU acceleration support will not automatically turn on GPU acceleration. To use GPU acceleration when running FHI-aims, the user specifies which tasks should be GPU accelerated independently using the control.in keywords below:

GPU aceleration of the ELPA eigensolver is controlled by elsi_elpa_gpu. See also Section 3.9. This keyword is not yet supported by HIP.

One important keyword when running GPU-accelerated calculations is points_in_batch, which sets the targeted number of points per batch. This parameter is a trade-off: increasing the number of points per batch increases the work done by the GPU per batch, increasing the efficiency of the GPU, but it also increases the number of basis elements interacting with a batch, increasing the memory usage. Due to technical details, some of this additional work is unnecessary, as it does not appreciably add to the integrals being evaluated.

The default value for points_in_batch based on early CPU-only benchmarks was set to 100. We have found that increasing this value to 200 is a better choice for our test architecture (Kepler Tesla GPUs) when using GPU acceleration, and we have set 200 as the default value when running any GPU-accelerating any tasks involving the batch integration scheme. The user should also play around with this parameter for their own architecture, particularly if they are using a different GPU architecture.

All GPU keywords may be set independently. We have found that the charge density update shows a significantly higher GPU-accelerated speed-up than the Hamiltonian integration (c.f. Figure G.1). If the user’s architecture uses fast CPUs but slow GPUs, enabling GPU acceleration may actually slow down the calculation.