Pepperdata

Pepperdata Capacity Optimizer provides real-time visibility into actual CPU, GPU, and memory utilization levels, allowing the scheduler to launch more containers per node without requiring additional nodes. It enables the scheduler to see actual physical resource utilization instead of total resource allocations, optimizing resource decisions and improving hardware utilization.

For Kubernetes, it optimizes Apache Spark, Apache Flink, Apache Airflow, Jobs, JobController, CronJobs, and custom labeled apps. For YARN, it optimizes Apache Spark, MapReduce, and Apache Tez apps.

Yes, Capacity Optimizer is complementary to all other cloud cost optimization solutions. It can be implemented on top of existing tooling and processes to achieve additional cloud cost reduction.

Pepperdata's pricing is based on your usage. You can book a meeting to get started with a free trial.

No, Capacity Optimizer does not replace any of the cluster autoscaler’s standard components. It provides better information to the autoscaler to operate more efficiently.

Supported environments include Open Source Kubernetes, Amazon EKS, Google GKE, Apache Spark on Cloudera Data Engineering, Amazon EMR, Google DataProc, and Cloudera Data Platform (CDP). Supported schedulers include the default scheduler on Amazon EMR and EKS, Google GKE, and Apache YuniKorn on Amazon EKS. Supported autoscalers include Amazon EMR Managed Autoscaling, Custom Autoscaling Policy on Amazon EMR, Cluster Autoscaler and Karpenter on Amazon EKS, and Cluster Autoscaler with and without Node Auto-Provisioning (NAP) on Google GKE.

Capacity Optimizer prohibits the Cluster Autoscaler from launching new nodes until all existing nodes are fully optimized. This ensures new nodes are launched only when existing ones are fully utilized, leading to up to a 71% decrease in autoscaling usage.

As new releases of technologies are introduced, Capacity Optimizer goes through a certification process that takes approximately 30 days, depending on the changes from the previous version. Pepperdata’s Customer Success team works with you to ensure a smooth rollout of new technologies.