National Center for Supercomputing Applications master calendar

View Full Calendar

NCSA staff who would like to submit an item for the calendar can email

Efficient I/O: HDF5 and DLIO Workshop

Event Type
Mar 12, 2024   1:00 - 3:00 pm  
Registration form
Bruno Abreu
Originating Calendar
NCSA Research Consulting Training Events
This workshop will include two talks about optimizing, profiling, and benchmarking input/output operations using HDF5 and DLIO.

  • Optimizing Your I/O Workload: Techniques for Effective HDF5 Usage - Scot Breitenfeld, The HDF Group
HDF5 is a widely used data model, file format, and I/O library, particularly in HPC applications for managing and storing large amounts of simulation data. This talk will focus on HDF5 usage on NCSA's Delta system and provide a brief overview of HDF5 (serial and parallel) with an emphasis on HDF5 HPC tuning techniques such as collective metadata I/O, data aggregation, asynchronous I/O, and other HDF5 tuning parameters and features. During the presentation, we will discuss various storage options available on Delta for HDF5 files and multiple post-simulation storage options, including cloud storage and other tools.
  • Deep Learning I/O: Benchmark and Profiling - Hariharan Devarajan, Lawrence Livermore National Laboratory
In this talk, we will be taking a dive into the unique features of I/O in deep learning application. We will present our ongoing efforts to understand the I/O characteristics of DL workloads using our DLIO profiler. Finally, we will present the DLIO Benchmark which is built to accurately represent the I/O characteristics in deep learning workloads.
link for robots only