Lawrence Livermore National Laboratory (LLNL)
Lawrence Livermore National Laboratory respond to fight a global pandemic
Customer
Industry
Government/Scientific Research
Challenge
To discover novel therapeutics for the COVID-19 virus in record time
Solution at-a-glance
- 2nd Gen AMD EPYC CPUs
- Cornelis™ Omni-Path™
- Supermicro rack servers and other accessories
Results
- Reduced genomic analysis from a few days to a few hours
- Accelerated COVID-19 research anti-body design throughput by a factor of eight1
Technology Partners
- AMD
- Supermicro
Meet Mammoth
-
CPU Architecture
AMD EPYCTM 7742 (2nd Gen)
-
Cores/Node
128
-
Total Cores
8,832
-
CPU Clock Speed
2.25 GHz
-
User-Available Nodes
Login Nodes – 2
Management Nodes – 1
Batch Nodes – 64
File System Router Nodes – 2
-
Memory Total
131,072 GB
-
CPU Memory/Node
2,048 GB
-
Peak Performance
294.0 TFLOPS
-
Local Storage/Node
3.84 TB SSD
-
Interconnect
Cornelis Omni-Path, 200 Gbps
-
OS
TOSS 3 / RHEL 7
-
Program
CARES Act
-
Year Commissioned
2020
Responding to a Pandemic
The global collaborative response to the COVID-19 pandemic has been unprecedented. The vast amount of research, testing, and development required to produce potential drug therapies or a vaccine usually takes years. When responding to COVID-19, time was of the essence.
In 2020, the U.S. Congress passed the Coronavirus Aid, Relief, and Economic Security (CARES) Act. This legislation provided funding to Department of Energy (DOE) national laboratories to support research and development efforts related to COVID-19. One of the centers funded was the Lawrence Livermore National Laboratory (LLNL), part of the National Nuclear Security Administration (NNSA), a semi-autonomous agency within the U.S. DOE.
At LLNL, the big memory cluster, Mammoth, is being used by scientists to perform genomics analysis and to aid the development of antiviral drugs and designer antibodies to neutralize the SARS-CoV-2 virus. To put things in perspective, LLNL researchers had to construct over 90,000 structural models that cover 14,000 human proteins and then use those models to complete extensive searches to identify functional sites (binding site pockets) in human proteins. Each of the structures are then screened against a created library of 1.9 million “pocket” templates. Mammoth is also conducting non-traditional HPC simulations and next-generation graph analytics related to genomics and other aspects of computational biology.
“The ability of large-memory systems to integrate genomic analysis with large-scale machine learning for predictive modeling of therapeutic response will be important for accelerating the development of effective new therapeutics.”
“Mammoth is integral to the development of new tools to combat COVID-19, but also for fast response in a future pandemic.”
“AMD is proud to have supported this vital research being done by the team at LLNL with Mammoth in conjunction with our technology partners at Cornelis Networks and Supermicro.”
Mammoth, a new breed of memory-dense HPC
Cornelis is excited to continue our track record of successful collaboration with the team at Lawrence Livermore. Enabling Livermore’s critical research efforts like those to eradicate COVID-19 are central to our mission as a company and we look forward to our continued partnership.
Cornelis is excited to continue our track record of successful collaboration with the team at Lawrence Livermore. Enabling Livermore’s critical research efforts like those to eradicate COVID-19 are central to our mission as a company and we look forward to our continued partnership.
Cornelis is excited to continue our track record of successful collaboration with the team at Lawrence Livermore. Enabling Livermore’s critical research efforts like those to eradicate COVID-19 are central to our mission as a company and we look forward to our continued partnership.
Cornelis is excited to continue our track record of successful collaboration with the team at Lawrence Livermore. Enabling Livermore’s critical research efforts like those to eradicate COVID-19 are central to our mission as a company and we look forward to our continued partnership.
Cornelis is excited to continue our track record of successful collaboration with the team at Lawrence Livermore. Enabling Livermore’s critical research efforts like those to eradicate COVID-19 are central to our mission as a company and we look forward to our continued partnership.
Cornelis is excited to continue our track record of successful collaboration with the team at Lawrence Livermore. Enabling Livermore’s critical research efforts like those to eradicate COVID-19 are central to our mission as a company and we look forward to our continued partnership.
Cornelis is excited to continue our track record of successful collaboration with the team at Lawrence Livermore. Enabling Livermore’s critical research efforts like those to eradicate COVID-19 are central to our mission as a company and we look forward to our continued partnership.
Cornelis is excited to continue our track record of successful collaboration with the team at Lawrence Livermore. Enabling Livermore’s critical research efforts like those to eradicate COVID-19 are central to our mission as a company and we look forward to our continued partnership.
Cornelis is excited to continue our track record of successful collaboration with the team at Lawrence Livermore. Enabling Livermore’s critical research efforts like those to eradicate COVID-19 are central to our mission as a company and we look forward to our continued partnership.
Ground-breaking results
Deploying Mammoth dramatically decreased LLNL’s standard genomic analysis run time for workflows from days to hours allowing researchers to focus on much larger workflows – using larger genomic databases or more accurate comparisons. This made those studies go from ‘major heroic effort’ to a more routine six week cycle.
Cornelis is excited to continue our track record of successful collaboration with the team at Lawrence Livermore. Enabling Livermore’s critical research efforts like those to eradicate COVID-19 are central to our mission as a company and we look forward to our continued partnership.
Cornelis is excited to continue our track record of successful collaboration with the team at Lawrence Livermore. Enabling Livermore’s critical research efforts like those to eradicate COVID-19 are central to our mission as a company and we look forward to our continued partnership.
Cornelis is excited to continue our track record of successful collaboration with the team at Lawrence Livermore. Enabling Livermore’s critical research efforts like those to eradicate COVID-19 are central to our mission as a company and we look forward to our continued partnership.
In addition, the Mammoth system was capable of running 128 Rosetta Flex calculations (code for computing binding free energies) simultaneously. This greatly accelerated research for SARS-CoV-2 antibody designs. The memory-dense clusters, along with the low latency and extreme message rates of the Cornelis Omni-Path fabric, assisted LLNL in increasing their anti-body design work throughput by a factor of eight.
The deployment of Mammoth in record time and the promising research findings it yielded were a bright spot for LLNL as the world watched the pandemic unleash itself. The big memory capacity proved to be ideal for running advanced simulations and research projects that involved large datasets to solve some of the pressing COVID-19 scientific challenges. Following these breakthroughs, LLNL is now looking at graph analytics as Mammoth’s next critical workflow to help governments to track the spread of the virus.
