2
PARTNER BRIEF Diablo Technologies™ Memory1™ in Supermicro® SuperServers® and Hortonworks® Data Platform (HDP™) Memory1 Delivers Better Spark Performance on HDP ….at a Lower Cost Diablo 128GB Memory1 DIMMs enable Supermicro SuperServers with big memory footprints for Apache Spark™ deployments The challenge: leveraging big data to harvest the right information that will positively affect business decisions and agility… quickly. Streaming analytics engines, such as Apache Spark, allow for very fast analytics on large and complex datasets. Spark runs on Apache Hadoop YARN in HDP, which enables Spark and other applications to share common Hadoop clusters and datasets. Spark favors memory over storage as data must be close to the processors for rapid and repeated access. Supermicro Memory1 Servers use dense, high-capacity 128GB DDR4 memory DIMMs from Diablo Technologies to provide up to 2TB of application memory per server. The net result is better performance at an overall lower Total Cost of Ownership for applications including: Machine Learning, Spark SQL, Graph Analysis and other workloads on HDP. “Job-based applications such as Apache Spark running on HDFS require balancing of cost and performance. Memory1 servers have shown customers that it is entirely possible to achieve faster Spark results from an HDFS cluster while still lowering overall expenses.” Maher Amer, Chief Technology Officer, Diablo Technologies “By combining the high performance of Memory1 with the scalability and flexibility of Supermicro SuperServers, users of Apache Spark on HDP will significantly reduce job times and simultaneously improve TCO.” Don Clegg, VP of Marketing and Business Development, Super Micro Computer, Inc. More Work per Server More application data fits within memory, reducing fetches from disk, minimizing spill over to physical storage, and maximizing utilization of processor cores. The net result is more work achieved per server with less time needed to complete jobs. Reduced Cluster Size More dense computing means fewer servers required for processing and analysis. Many Spark workloads experience a 3 to 1 or even a 6 to 1 consolidation, considerably reducing the size of the overall cluster. Reduced cluster utilization per workload also allows for more applications to run concurrently. Lower Total Cost of Ownership Fewer servers achieving much faster results leads to a substantially lower TCO. Not only is CAPEX reduced with fewer servers and less rack space, but OPEX also drops drastically as less power, cooling and management are needed to maintain the cluster.
