This blog post focuses on an SGEMM implementation that outperforms cuBLAS with its (modified?) CUTLASS kernel across a wide range of matrix sizes. The blog delves into benchmarking code on CUDA devices and explains the algorithm's design along with optimization techniques. These include inlined PTX, asynchronous memory copies, double-buffering, avoiding shared memory bank conflicts, and efficient coalesced storage using shared memory. The code is super easy to tweak, so you can customize it for your projects with kernel fusion or just drop it into your libraries as-is. If you have any questions, feel free to comment or send me a direct message - I'd love to hear your feedback and answer any questions you may have!