Amazon S3 Directory Bucket Object Processing Automation

Amazon Web Services (AWS) has launched a new capability for automating the processing of large volumes of objects stored in Amazon S3 directory buckets, particularly within the S3 Express One Zone storage class. This new approach integrates S3 Batch Operations with AWS Lambda functions.

The capability allows for batch modification of objects, such as applying watermarks to images or adjusting audio file bitrates for optimized playback across different devices. This is facilitated by the direct integration of S3 Batch Operations with AWS Lambda, enabling custom processing logic to be applied at scale.

Previously, users could trigger AWS Lambda for large-scale data processing with Amazon S3. The S3 Batch Operations feature now provides a streamlined and efficient way to automate these tasks for millions or billions of objects, simplifying complex data workflows.

The new method, detailed in an AWS Storage Blog post, demonstrates how to configure S3 Batch Operations with AWS Lambda to modify and copy objects between S3 Express One Zone directory buckets. This is designed to accelerate processing for high-performance applications and workloads requiring extensive data manipulation.