Amazon Bedrock to Deploy Hybrid AI Inference System for Faster Load Processing

Amazon AWS and wafer-level AI chip company Cerebras announced that they will deploy a hybrid AI inference system on the Amazon Bedrock platform to improve load processing speed. The system combines Cerebras' CS-3 system, AWS' Trainium chip, and EFA elastic network adapter.

The Trainium chip handles inference pre-filling (prompt processing), while CS-3 takes on decoding (output generation tasks), connected via EFA. Pre-filling requires large computing power and moderate memory bandwidth, while decoding needs high memory bandwidth. Using them together leverages their strengths to deliver a better user experience.