HPC Storage

Solutions for High Performance Parallel Environments


Overview

Leadership in High Performance Storage

High-performance computing (HPC) and the closely related field of High Performance Data Analytics (HPDA) are rapidly becoming a competitive business necessity far beyond their laboratory-based heritage. Organizations in all market sectors – both research and commercial – are now treating the ability to optimize the data-to-value cycle as a core competency which is in turn fueling a desire to acquire, refine and preserve data for both near term and future value mining activities.

HPE has a legacy of leadership in both designing and delivering enterprise-class storage in support of some of the most demanding compute environments in the world. Lustre has emerged as the dominant storage solution for HPC – and it is rapidly gaining interest from users of both Hadoop and shared memory environments as the storage foundation of high-performance analytics environments that can leverage multiple classes of data analysis techniques.

Key differences between HPC and enterprise storage

Enterprise storage HPC storage
Typical workload Transactional Job (Data Set)
Performance requirement I/O Bandwidth
Typical file size Small Large
Typical file access Random Sequential
Data movement Minimal Extensive

Intel® Enterprise Edition for Lustre®

The HPE Solution for Intel Enterprise Edition for Lustre is a high-performance compute (HPC) storage solution that includes the HPE Apollo 4520 System and Intel® Enterprise Edition for Lustre*. The Apollo 4520 is a dual-node system with up to 46 drives. Capacity can be increased by adding additional drives in a disk enclosure. The solution can scale by adding more systems in parallel to scale performance and capacity nearly linearly.

It offers a single coherent namespace across hundreds of storage servers in a single file system. Including Intel Management for Lustre* a central management and monitoring console to simplify installation, configuration and management, the HPE Solution for Intel Enterprise Edition for Lustre* solution also includes software RAID which provides resiliency, error detection and faster re-build times.

Intel® Enterprise Edition for Lustre® software (Intel® EE for Lustre software) unleashes the performance and scalability of the Lustre parallel file system for HPC workloads, including the types of technical ‘big data’ applications frequently leveraged by legacy SGI customers. It allows end-users that need the benefits of large-scale, high bandwidth storage to tap the power and scalability of Lustre, with the simplified installation, configuration, and management features provided by Intel® Manager for Lustre software, a management solution purpose-built by the Lustre experts at Intel for the Lustre file system. Further, Intel EE for Lustre software is backed by HPE’s global support team working in conjunction with Intel’s Lustre engineering team – including 24/7 service level agreement (SLA) coverage.

Intel® Enterprise Edition for Lustre® Software Components

Product Components

  • Intel® Manager for Lustre*: Intel Manager for Lustre software includes simple, but powerful, management tools that provide a unified, consistent view of Lustre storage systems and simplify the installation, configuration, monitoring, and overall management of Lustre. The manager consolidates all Lustre information in a central, browser-accessible location for ease of use.
  • Integrated Apache Hadoop® Adapter: When organizations operate both Lustre and Apache Hadoop® within a shared HPC infrastructure, there is a compelling use case for using Lustre as the file system for Hadoop analytics, as well as HPC storage.Intel Enterprise Edition for Lustre includes an Intel-developed adapter which allows users to run MapReduce® applications directly on Lustre. This optimizes the performance of MapReduce operations while delivering faster, more scalable, and easier to manage storage.

Benefits

  • Performance: Intel EE for Lustre software has been designed to enable fully parallel I/O throughput across thousands of clients, servers, and storage devices. Metadata and data are stored on separate servers to allow optimization of each system for the different workloads they present. Intel Enterprise Edition for Lustre can also scale down efficiently to provide fast parallel storage for smaller organizations.
  • Capacity: The object-based storage architecture of Intel EE for Lustre software can scale to tens of thousands of clients and petabytes of data.
  • Affordability: Intel EE for Lustre software is based on open source Lustre software, and is hardware, server, and network fabric neutral. SGI’s team of technical presales engineers can work with customers to architect systems that can meet specific customer requirements for performance and scalability. Enterprises can scale their storage deployments horizontally, yet continue to have simple-to-manage storage.
  • Maturity: Lustre has been in use in the world’s largest datacenters for over a decade and hardened in the harshest Big Data environments. Intel EE for Lustre software is rigorously tested, reliable, and backed by SGI and Intel, the leading providers of technical support for Lustre software. Intel EE for Lustre software delivers commercial- ready Lustre in a package that can scale efficiently both up and down to suit broad business workloads, with built-in manageability.

Key Features

Unleash the Lustre parallel file system as an enterprise platform-offering higher throughput and helping to prevent bottlenecks.

  • Built on open-source Lustre software
  • Intel® Manager for Lustre simplifies install and configuration
  • Enormous storage capacity and I/O
  • Open, documented interfaces for deep integration
  • Throughput in excess of 1 terabyte per second
  • Resilient, highly available storage
  • Centralized, GUI-based administration for management simplicity
  • Integrated support for Hadoop® MapReduce® applications with Lustre storage
  • Rigorously tested, stable software proven across diverse industries
  • Flexible storage solution based on Intel-enhanced open source software
  • Global 24/7 technical support

Resources

Apollo 4500

Purpose-built Systems for Big Data and HPC Storage Workloads

With the HPE Apollo 4500 Systems, HPE challenges the notion that one-size-fits all for Big Data infrastructure by creating purpose-built systems that specifically address storage and analytics workloads. For object storage, the ultra-dense HPE Apollo 4510 includes one server and up to 68 LFF drives in a 4U chassis for a maximum of 544 TB per system. For clustered storage environments, the HPE Apollo 4520 offers two servers with built-in failover capability. For Hadoop® and other Big Data solutions, the HPE Apollo 4530 uniquely offers three servers per chassis, ideal for housing three copies of data in a single system.

The HPE Apollo 4500 series allows you to realize all the value of your data at the right cost and in the least amount of space. And with HPE software tools to help you deploy, operate, and optimize your valuable data center resources, you can grow your data with confidence at any scale.