Amazon Bedrock Model Evaluation Now Generally Available

Amazon Web Services (AWS) has made its Amazon Bedrock model evaluation capability generally available, following its preview at AWS re:Invent 2023. This new feature is designed to help developers efficiently select the best foundation models for their generative artificial intelligence applications.

According to AWS, model evaluation is a critical stage in the development process for AI applications. The new tool provides developers with the ability to experiment with different models in a playground environment, accelerate iteration through automatic evaluations, and ensure quality via human reviews before launch.

The model evaluation tool allows users to create evaluation jobs by selecting an evaluation method (automatic or human), a desired foundation model, a task type, and evaluation metrics. Automatic evaluations can utilize metrics such as accuracy, robustness, and toxicity. Users can also define custom metrics for aspects like friendliness, style, or adherence to instructions.

The service received feedback during its preview period, which AWS has used to enhance features prior to the general release. Amazon Bedrock is a cloud-based service that offers access to various AI models, enabling customers to leverage generative AI flexibly.