DataPelago Accelerates Apache Spark on Amazon EKS
DataPelago Accelerator for Spark (DPA-S) is now available on Amazon EKS, promising up to 10x faster execution and up to 80% cost savings for Apache Spark workloads.
Amazon Web Services (AWS) and its partner DataPelago have introduced a solution to significantly accelerate Apache Spark workloads on Amazon Elastic Kubernetes Service (Amazon EKS). DataPelago Accelerator for Spark (DPA-S) reportedly delivers up to ten times faster execution and up to 80% cost reduction without requiring modifications to existing applications or data pipelines.
The offering targets enterprises handling large datasets and utilizing AI and analytics. Historically, Apache Spark's performance has been a bottleneck due to its design for CPU-only environments, even as modern data centers increasingly incorporate GPUs and other specialized processing units.
DataPelago, an AWS ISV Partner, integrates its acceleration capabilities directly into Amazon EKS. This enables customers in sectors such as social media, e-commerce, and financial services to enhance their data processing efficiency without recoding applications. Initial production environments have demonstrated substantial performance improvements.
AWS and DataPelago are providing customers the opportunity to trial the solution through Proof of Concept (PoC) projects within their own infrastructure. This allows for the quantification of potential benefits before wider deployment.