You are here

How to Turn a Jaguar into a Titan

November 27, 2012 - 3:02pm

Addthis

How to Turn a Jaguar into a Titan

It takes more than the side mural to turn the fastest supercomputer dedicated to Open Science into the fastest computer in the world. The same room and cabinets that held supercomputer Jaguar, now holds Titan.

Storified by Energy Department ·
Tue, Nov 27 2012 12:07:29

How did Oak Ridge National Labs turn this world-class supercomputer into an even faster, more efficient supercomputer?
It took a little more than changing the sticker. 
The semiannual TOP500 list came out earlier this month ranking the most powerful supercomputers in the world based on their Linpack benchmark performance. At the top was Oak Ridge National Lab's supercomputer Titan.
Boasting proven speeds at 17.59 petaFLOPS, or the ability to perform 17.59 quadrillion calculations per second, and theoretical speeds topping 20 petaFLOPS, Titan unseated Lawrence Livermore's IBM Sequoia supercomputer. 
Just months ago, the 200 cabinets currently housing Titan contained the brains of Jaguar, the 18,688 nodes containing dual AMD Opteron 2435 processors and 16 GB of memory. 
Jaguar's grand total: 224,256 processors that benchmarked at 1.75 petaFLOPS, and a total combined memory of 360 terabytes.
This was enough to beat out Los Alamos' Roadrunner -- the first computer ever to break into the petascale -- in 2009 and remain on top for a full year. Three years later though, the petascale is long in the rearview mirror as high-performance computing advances to the next big milestone, the exascale. 
Looking under Titan's hood, you will see the components that probably won't achieve exascale computing, but we are moving in the right direction. Here's a quick glimpse into the process of turning Jaguar into Titan:
Titan Supercomputer Installation Time-Lapse ·
techgage
The video above shows technicians pulling the blades out of the cabinets and upgrading their processors and adding GPUs. Here's a slightly slower, more in depth version from Anand Lal Shimpi. 
Oak Ridge National Laboratory Tour - Upgrading to Tesla K20 GPUs ·
anandshimpi
For a very comprehensive technical dive into what makes Titan so powerful, AnandTech posted Inside the Titan Supercomputer after a site visit at ORNL. 
As you can see above, each of the 9,344 blades have 2 nodes. Each of the 18,688 nodes each contain a 16-core AMD Opteron 6274 CPU with 32 GB of RAM and an Nvidia Tesla K20 GPU. Put together, that's a lot of computing horsepower.
Titan's grand total: 299,008 processor cores with a theoretical speed at 20 petaFLOPS, total memory at 710 TB, and power usage sits at 8.2 MW (probably 9 MW at full load), which also makes it among the most energy efficient supercomputers. 
These insane speeds are entirely made possible by graphics processing units, a piece of hardware first created for computer gaming. Here's Adam Savage and Jamie Hyneman explaining the difference between CPUs and GPUs in true Mythbusters fashion:
CPU vs. GPU ·
zackwhittaker
Got that?  So, traditional supercomputers, like the computer you're reading this on now, normally carry out instructions by performing the calculations with the CPU. Instead, Titan uses it's CPUs to guide and allocate these instructions to the GPUs, where the heavy lifting will take place. This is where the computing boost comes from, and also the need for less energy.
All of Titan's computing power will be used to carry Jaguar's torch as the World's #1 Open Science Supercomputer. 
Jaguar: The World's Most Powerful Computer. For Science! ·
nccsjaguar
The Open Science mission of Titan, and its predecessor, makes it distinct. Even if another computer takes the top spot in six months, Titan will likely still be listed as the fastest supercomputer available for science. 
Any science or engineering project in the United States can apply for time on Titan through the Innovative & Novel Computational Impact on Theory and Experiment (INCITE) program.
Titan will power the research conducted by the Consortium for Advanced Light Water Reactors (CASL) to model the behavior of neutrons in nuclear power reactors. 
CASL VERA-CS rod movement in 3x3 fuel assemblies (Problem #4) ·
Bq8Cylf_BfCgX23LJJb65A
Titan will be used to answer questions about climate change, simulate supernovae and model the next generation of combustion engines. 
Here's the competitively selected 2012 INCITE awardees and the number of processor hours their projects received:
There's a common theme to note on this page, and also in the justification and need for supercomputers, the ability to simulate reality when experiments are either cost prohibitive or impossible.
It would be really difficult to create a supernova on earth and see the results. (Let alone creating a supernova.) What if we could simulate every lepton and atom in the environment you were trying to model? We could control everything and see what happens.
It's an interesting parallel that GPUs were first created for gamers, a breed that appreciates not only the aesthetics of their games, but also how they perform. In this example, look at how the skater falls (all 50 times).
Skate 3 funny stuff compilation part 2 ·
helixsnake
He doesn't just thud to the ground. He'll bounce. Or his feet catch a rail and flip him into the ground before skidding to a seemingly painful grinding halt. These motions were created with a physics engine. This serves to illustrate a point rather than draw a direct comparison. 
Researchers are using Titan and the other supercomputers to the observe the natural world in a completely controlled (and digital) environment to further their understanding of it. They'll continue to achieve greater clarity and more realism as computing power increases.
In the meantime, Titan is at the top of the list using technology that will drive forward the industry onward toward the exascale and beyond.

Addthis