Publications

The high computational cost of Large Language Models (LLMs) requires post-training optimization for efficient deployment. This paper presents a rigorous empirical comparison of inference performance between the native PyTorch framework and the optimized ONNX format for contemporary LLMs, including Gemma-7B, Phi-3-Medium, and Llama-3.1-8B, on NVIDIA GPU hardware. Using low-level profiling with NVIDIA's Nsight Compute suite, we found that ONNX conversion yields a substantial and statistically significant performance advantage. Our results show a latency reduction of up to 27.6% and an increase in computational throughput (GFLOP/s) of up to 48.3%. We conclude that adopting an ONNX-based strategy is not merely a portability choice but a critical optimization step for deploying high-performance, efficient LLMs in production environments.

Current climate change has posed a grand challenge in the field of numerical modeling due to its complex, multiscale dynamics. In hydrological modeling, the increasing demand for high-resolution, real-time simulations has led to the adoption of GPU-accelerated platforms and performance portable programming frameworks such as Kokkos. In this work, we present a comprehensive performance study of the SERGHEI-SWE solver, a shallow water equations code, across four state-of-the-art heterogeneous HPC systems: Frontier (AMD MI250X), JUWELS Booster (NVIDIA A100), JEDI (NVIDIA H100), and Aurora (Intel Max 1550). We assess strong scaling up to 1024 GPUs and weak scaling upwards of 2048 GPUs, demonstrating consistent scalability with a speedup of 32 and an efficiency upwards of 90% for most almost all the test range. Roofline analysis reveals that memory bandwidth is the dominant performance bottleneck, with key solver kernels residing in the memory-bound region. To evaluate performance portability, we apply both harmonic and arithmetic mean-based metrics while varying problem size. Results indicate that while SERGHEI-SWE achieves portability across devices with tuned problem sizes (<70%), there is room for kernel optimization within the solver with more granular control of the architecture specifically by using Kokkos teams and architecture specific tunable parameters. These findings position SERGHEI-SWE as a robust, scalable, and portable simulation tool for large-scale geophysical applications under evolving HPC architectures with potential to enhance its performance.

The impact of high performance computing on technological innovation and scientific discovery is preponderant. Computer simulation, big data analysis, and generative artificial intelligence are often used in trailblazing products or groundbreaking findings. However, an appropriate provisioning of supercomputing resources for the entire spectrum of users is a daunting task. What should be the right hardware to satisfy future needs and demands? We set out to answer that question, particularly in academic environments, where funding schemes are based on availability of grants. The resulting machine, in those institutions, is usually a heterogeneous mix of several architectures and configurations. This paper presents a methodology to guide the next supercomputing purchase based on what is already available and what upcoming needs are anticipated. We use a collection of publicly-available benchmarks and applications to profile a machine. We then use a mathematical model, based on the current profile, to navigate the space of future configurations and suggest future investments. We applied our methodology to Kabré, a small but representative, compute cluster in an academic setting.

In this paper, the Numba, JAX, CuPy, PyTorch, and TensorFlow Python GPU accelerated libraries were benchmarked using scientific numerical kernels on a NVIDIA V100 GPU. The benchmarks consisted of a simple Monte Carlo estimation, a particle interaction kernel, a stencil evolution of an array, and tensor operations. The benchmarking procedure included general memory consumption measurements, a statistical analysis of scalability with problem size to determine the best libraries for the benchmarks, and a productivity measurement using source lines of code (SLOC) as a metric. It was statistically determined that the Numba library outperforms the rest on the Monte Carlo, particle interaction, and stencil benchmarks. The deep learning libraries show better performance on tensor operations. The SLOC count was similar for all the libraries except Numba which presented a higher SLOC count which implies more time is needed for code development.