It’s almost time for SC18 and this year it’s a biggie. There will be events to mark 30 years of the event, with vendors such as the Numeral Algorithms Group celebrating longevity in the sector that few others can rival.
There have been more developments at an incredible pace over the last three decades than anybody could have predicted. We’re coming ever close to exascale as both hardware and software continue to get faster, while weather forecasting is now just one sector that relies on supercomputers to save lives.
Here is what we expect to hear about at SC18 as the Ellexus team treads the show floor.
The storage landscape diversifies
Storage is always a big topic, both on-premises and in the cloud. We will hear much more about diverse storage landscapes in general as new storage solutions appear on the market. The affordability of flash makes for a range of creative storage products, while NVMe technology is becoming more widespread.
Some vendors are entering this market for the first time and giving seasoned vendors such as DDN and Dell EMC a run for their money. Clustervision is one of the storage newcomers with ClusterVault, its own storage solution that promises scalability, industry-proven hardware and an open source storage stack. It will be interesting to listen to reactions from the market.
Others are adding I/O profiling solutions to their existing range. The amount of information that you can get from Lustre continues to become more accessible and vendors such as Cray, with the storage division that they acquired from Seagate, are not missing this opportunity.
We will certainly hear more examples of cloud migration really happening, as well as more on container use such as Singularity. Hybrid cloud is a big topic among centres that are keen to look to the future while maximizing the return on their current systems and infrastructure. Hardware may have an expiry date, but system knowledge and software infrastructure can live on for longer. The value and corporate knowledge in this should not be underestimated.
There will be people scrutinizing the pros and cons of HPC environments in the cloud and on-premise. Companies such as Alces Software that make it easy to rapidly spin up a cluster will continue to gain traction.
Anyone who fails to embrace this change will be left behind. There will always be workloads that demand very specialised kit, but cloud vendors continue to diversify their offerings, which means that set of workloads gets smaller every day.
Yet again, this will be a big topic. Each year we get closer and closer to the world being completely run by machines – perhaps not yet, but sometimes it feels scarily close.
Certainly, the dust is starting to settle on the hype now and we are seeing real use cases hit the HPC clusters. Platforms such as AWS are being used to train workloads for self-driving cars on HPC scales, but with very different I/O and compute patterns than we are used to.
The weather forecasters continue to dominate the world’s largest machines. They have some of the largest supercomputers used for a single purpose. With climate destruction becoming ever a reality, more and more of us will start to rely on those machines for our lives as well as our livelihoods.
I attended an annual update meeting at Texas Advanced Computing Centre in October and was not surprised to hear that Stamped 2 is used to top up the local weather forecast when a storm is brewing. It’s then really a race of machine against the elements to deliver a forecast on time to save lives. In the UK we rarely get hurricanes, but Western Europe still remembers the great storm of 1987 and Michael Fish will forever be known as they guy who said it wouldn’t happen.
In general, we will hear more about the convergence of HPC and big data, and the new wave of industries that are now looking to work with supercomputers. As system administrators and infrastructure providers, it’s easy to forget the impact HPC has on everyone’s lives. It’s something that I will try to remember as I take in the sights and sounds of SC18.