Lawrence Livermore National Laboratory responds to fight a global pandemic

Customer:
Lawrence Livermore National Laboratory (LLNL)

Industry:
Government/Scientific Research

Challenge:
To discover novel therapeutics for the COVID-19 virus in record time

Solution at-a-glance:

• 2nd Gen AMD EPYC CPUs

• Cornelis™ Omni-Path™

• Supermicro rack servers and other accessories

Results:

• Reduced genomic analysis from a few days to a few hours

• Accelerated COVID-19 research anti-body design throughput by a factor of eight¹

Technology Partners:

• AMD

• Supermicro

Meet Mammoth

CPU Architecture:
AMD EPCYTM 7742 (2nd Gen)

Core/Node:
128

Total Cores:
8,832

CPU Clock Speed:
2.25 GHz

User-Available Nodes:

• Login Nodes – 2

• Management Nodes – 1

• Batch Nodes – 64

• File System Router Nodes – 2

Memory Total:
131,072 GB

CPU Memory/Node:
2,048 GB

Peak Performance:
294.0 TFLOPS

Local Storage/Node:
3.84 TB SSD

Interconnect:
Cornelis Omni-Path, 200 Gbps

OS:
TOSS 3 / RHEL 7

Program:
CARES Act

Year Commissioned:
2020

Responding to a Pandemic

The global collaborative response to the COVID-19 pandemic has been unprecedented. The vast amount of research, testing, and development required to produce potential drug therapies or a vaccine usually takes years. When responding to COVID-19, time was of the essence.

In 2020, the U.S. Congress passed the Coronavirus Aid, Relief, and Economic Security (CARES) Act. This legislation provided funding to Department of Energy (DOE) national laboratories to support research and development efforts related to COVID-19. One of the centers funded was the Lawrence Livermore National Laboratory (LLNL), part of the National Nuclear Security Administration (NNSA), a semi-autonomous agency within the U.S. DOE.

At LLNL, the big memory cluster, Mammoth, is being used by scientists to perform genomics analysis and to aid the development of antiviral drugs and designer antibodies to neutralize the SARS-CoV-2 virus. To put things in perspective, LLNL researchers had to construct over 90,000 structural models that cover 14,000 human proteins and then use those models to complete extensive searches to identify functional sites (binding site pockets) in human proteins. Each of the structures are then screened against a created library of 1.9 million "pocket" templates. Mammoth is also conducting non-traditional HPC simulations and next-generation graph analytics related to genomics and other aspects of computational biology.

AMD is proud to have supported this vital research being done by the team at LLNL with Mammoth in conjunction with our technology partners at Cornelis Networks and Supermicro. Robert Hormuth CVP Architecture & Strategy, Data Center & Embedded Solutions Group AMD

Ground-breaking results

Deploying Mammoth dramatically decreased LLNL’s standard genomic analysis run time for workflows from days to hours allowing researchers to focus on much larger workflows – using larger genomic databases or more accurate comparisons. This made those studies go from ‘major heroic effort’ to a more routine six week cycle.

AMD is proud to have supported this vital research being done by the team at LLNL with Mammoth in conjunction with our technology partners at Cornelis Networks and Supermicro. Robert Hormuth CVP Architecture & Strategy, Data Center & Embedded Solutions Group AMD

In addition, the Mammoth system was capable of running 128 Rosetta Flex calculations (code for computing binding free energies) simultaneously. This greatly accelerated research for SARS-CoV-2 antibody designs. The memory-dense clusters, along with the low latency and extreme message rates of the Cornelis Omni-Path fabric, assisted LLNL in increasing their anti-body design work throughput by a factor of eight.

The deployment of Mammoth in record time and the promising research findings it yielded were a bright spot for LLNL as the world watched the pandemic unleash itself. The big memory capacity proved to be ideal for running advanced simulations and research projects that involved large datasets to solve some of the pressing COVID-19 scientific challenges. Following these breakthroughs, LLNL is now looking at graph analytics as Mammoth’s next critical workflow to help governments to track the spread of the virus.