Grouped Query Attention Performance Theoretical Analysis

Sharing Key and Value Tensors for a Group of Query Tensors to Reduce Transformer Attention Layer Memory IO Pressure