Amazon OpenSearch Serverless adds collection groups for cost optimization

Amazon Web Services (AWS) has announced the general availability of collection groups for Amazon OpenSearch Serverless. This new feature enables customers to reduce compute costs for multi-tenant workloads while providing secure tenant boundaries through per-tenant encryption, offering flexibility to balance cost efficiency with the required level of isolation and security.

Amazon OpenSearch Serverless is a serverless offering for Amazon OpenSearch Service, designed to eliminate infrastructure management complexity. It automatically provisions and scales resources to deliver high data ingestion rates and millisecond response times. For organizations managing multi-tenant environments, data isolation, including encryption and protection of tenant data often with unique encryption keys, is a common compliance requirement.

Previously, OpenSearch Serverless provided maximum security via physical isolation, where each AWS Key Management Service (KMS) key required dedicated OpenSearch Compute Units (OCUs). While this offered the highest level of protection, it presented challenges for scaling multi-tenant deployments. Customers managing multiple tenants with shared encryption keys benefited from efficient resource pooling. However, those managing a large number of smaller tenants, each needing its own KMS key for data isolation, faced higher costs due to the need for dedicated OCU resources per unique key, even if individual tenants required only a fraction of an OCU's capacity.

The introduction of collection groups allows multiple tenants requiring cryptographic isolation to be consolidated onto a smaller number of OCU resources. This approach improves cost efficiency, particularly for architectures with many tenants that have low individual resource demands. Customers can now achieve both strong data isolation and better cost management for their multi-tenant workloads.