Aware of the AI workload needs, in 2023, AMD announced the AI engine ML (AIE-ML),[21] the second generation of such architecture. It added support for AI-specific data types like bfloat16,[22] a common data type for deep learning applications. The version retained the same vector processing capabilities of the previous instance but with enlarged memory to support more intermediate computations.[23] From this generation, AMD integrates AI engines with other processing units like CPUs and GPUs, which are incorporated into modern Ryzen AI processors. In such systems, AI engines are usually referred to as Compute Tiles—self-contained processing blocks designed to efficiently execute AI and signal processing workloads. These blocks are integrated with other types of tiles,[17][24] namely Memory tile and Shim tile. The apparatus containing the three interconnected kinds of tiles is named XDNA,[25] and its first generation, namely XDNA 1, is released on Ryzen AI Phoenix PCs. Along with this release, AMD continues the research about programmability, releasing, as an open source tool, Riallto.[26]
On a similar path, at the end of 2023, early 2024, AMD announced the XDNA 2, along with the Strix series of Ryzen AI architectures.[27][28] Different from the first generation of XDNA architectures, the second one offers more units to target the massive workload of ML systems. Again, to keep the efforts on the programmability side, AMD released the open source Ryzen AI SW toolchain, which includes the tools and runtime libraries for optimizing and deploying AI inference on Ryzen AI PC.[25]
As neural processing and deep learning applications expand across domains, researchers and industry professionals increasingly categorize XDNA architectures as Neural processing units (NPUs). However, the term includes all those architectures specifically meant for deep learning workloads[29] and several companies, such as Huawei[30] and Tesla,[31] are proposing their own alternative.[30][31]
Hardware architecture
AI engine tile
First generation of AI engine single tile scheme, offering a vector processor capability and a 32 KB memory.
Each AI engine can execute an independent function, or multiple functions by leveraging time multiplexing. The programming structure used to describe the AI engine instantiation, placement and connection is named AIE graph. The official programming model suggested by AMD requires writing such a file in C++. However, different programming toolchains, from both companies and research, can support different alternatives to improve programmability and/or performance.[20][24]
To compile the application, the original toolchain relies on a closed-source AI engine compiler that automatically performs placement and routing, despite custom indications that can be given when writing the AIE graph.[35]
Because AI engines were initially integrated only into Versal systems, combining AI engine tiles with FPGA fabric and Network on Chip (NoC) connectivity, this layer offers direct communication with both the FPGA and the NoC. Such communication needs to be specified in both the AIE graph, to ensure a correct placement of the AI engines, and during the system-level design.[20][7]
Second generation - the AI engine ML
The second generation of AMD's AI engines, or AI engine ML (AIE-ML), provides some architectural modifications with respect to the first generation, focusing on performance and efficiency for machine learning workloads.[23]
AIE-ML possesses almost twice the density of computing per tile, improved memory bandwidth, and natively supports data types with more AI inference workload-optimized formats such as INT8 and bfloat formats. These optimizations allow the second-generation engine to deliver up to three times more TOPS per watt than the underlying AI engine, which was primarily built for DSP-heavy workloads and required explicit SIMD programming and hand-coded data partitioning.[3]
Recent publications from researchers and institutions[36] confirm that AIE-ML offers more scalable, more on-chip memory, and more computational power,[3] making it better suited for edge-based modern ML inference workloads. These advances collectively counter the limitations of the first generation.[23]
Official documentation identifies several key similarities and differences between the two architectures:[23]
XDNA 1
Simplified diagram of an AMD XDNA NPU such as found in Ryzen 7040 processors.[24]
The XDNA is the hardware layer combining three types of tiles:[24][25]
The Compute Tile (AI engine ML) is responsible for executing vector and scalar operations.
The Memory Tile is responsible for 512 KB of local memory and computes pattern-specific data movements to upstream Compute Tile fetch requests.
The ShimTile, which handles the host memory interaction, controls the data exchanges between Memory and Compute Tiles.
The XDNA architecture is combined with other architectural layers such as CPUs and GPUs, for Ryzen AI Phoenix architectures, composing the AMD product line for energy-efficient inference and AI workloads.[24]
XDNA 2
Second generation of XDNA layers is integrated within Ryzen AI Strix architecture and official documents from the producer claim it as specifically tailored for LLM inference workloads.[25]
Tools and programming model
The main programming environment for AI engine, officially supported by AMD, is the Vitis flow, which uses the Vitis toolchain to program the hardware accelerator.[33][37][7]
AMD Vivado logo.
Vitis offers support for both hardware and software developers in a unified development environment, including high-level synthesis, RTL-based flows, and domain-specific libraries.[38] Vitis enables applications to be deployed onto heterogeneous platforms, including AI engines, FPGAs, and scalar processors.[38]
1 2 3マートル、カウスタブ;タカ、エンドリ。 Arora、Aman (2025-04-15)、GAMA: AMD Versal ML に最適化された AI エンジンでの高性能 GEMM アクセラレーション、arXiv : 2504.09688
↑ Chen, Paul; Manjunath, Pavan; Wijeratne, Sasindu; Zhang, Bingyi; Prasanna, Viktor (2023-09-04). "Exploiting On-Chip Heterogeneity of Versal Architecture for GNN Inference Acceleration". 2023 33rd International Conference on Field-Programmable Logic and Applications (FPL) . IEEE. pp. 219–227 . doi : 10.1109/FPL60245.2023.00038 . ISBN979-8-3503-4151-5。
↑フローレス、フェルナンド。ペーニャ、マリア・ドロレス・バルデス。サンチェス、ホセ・マヌエル・ビジャプン。パゾ、ヘスス・マヌエル・コスタ。グラーニャ、カミロ・キンタンス（2024-11-13）。 「デジタル信号処理基本コア ユニット用の Versal インテリジェント エンジンの評価」。2024 年の第 39 回回路および集積システム設計会議 (DCIS)。 IEEE。 pp. 1–6 . doi : 10.1109/DCIS62603.2024.10769170。ISBN979-8-3503-6439-2。
1 2 「AIエンジン：次世代アプリケーションのコンピューティング需要を満たす」。
1 2 3 Menzel, Johannes; Plessl, Christian (2025-05-04). "AMD AI Engines 上での電子反発積分の効率的かつ分散的な計算". 2025 IEEE 33rd Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM) . pp. 95–104 . doi : 10.1109/FCCM62733.2025.00044 . ISBN979-8-3315-0281-2。
1 2 3 Vissers, Kees (2019-02-20). "Versal: Xilinx Adaptive Compute Acceleration Platform (ACAP)" . Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays . FPGA '19. New York, NY, USA: Association for Computing Machinery. p. 83. doi : 10.1145/3289602.3294007 . ISBN978-1-4503-6137-8。
1 2 Swarbrick, Ian; Gaitonde, Dinesh; Ahmad, Sagheer; Gaide, Brian; Arbel, Ygal (2019-02-20). "Network-on-Chip Programmable Platform in Versal ACAP Architecture" . Proceedings of the 2019 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays . FPGA '19. New York, NY, USA: Association for Computing Machinery. pp. 212–221 . doi : 10.1145/3289602.3293908 . ISBN978-1-4503-6137-8。
↑ Chhugani, Jatin; Nguyen, Anthony D.; Lee, Victor W.; Macy, William; Hagog, Mostafa; Chen, Yen-Kuang; Baransi, Akram; Kumar, Sanjeev; Dubey, Pradeep (2008-08-01). "マルチコアSIMD CPUアーキテクチャにおけるソートの効率的な実装" . Proc. VLDB Endow . 1 (2): 1313– 1324. doi : 10.14778/1454159.1454171 . ISSN 2150-8097 .
1 2 3ヘネシー、ジョン L.、パターソン、デイビッド A. (2019).コンピュータアーキテクチャ：定量的アプローチ。クルステ・アサノヴィッチ (第 6版)。マサチューセッツ州ケンブリッジ：モーガン・カウフマン出版社、エルゼビアのインプリント。ISBN978-0-12-811905-1。
1 2 Brown, Nick (2023-02-12). 「ステンシルベースの大気移流シミュレーションを高速化するためのVersal AIエンジンの探求」 . 2023年ACM/SIGDA国際フィールドプログラマブルゲートアレイシンポジウム議事録. FPGA '23. ニューヨーク州ニューヨーク、米国: Association for Computing Machinery. pp. 91–97 . arXiv : 2301.13016 . doi : 10.1145/3543622.3573047 . ISBN978-1-4503-9417-8。
↑島村、光太郎。大野あゆみ;高前田-山崎 真也 (2025-02-17)、3D ガウス スプラッティングのための Versal AI エンジンの探索、arXiv : 2502.11782
↑ Brown, Nick; Canal, Gabriel Rodríguez (2025-02-14)、「AMD AI Engines による Fortran 内部関数のシームレスな高速化」、2025 ACM/SIGDA 国際フィールドプログラマブルゲートアレイシンポジウム議事録、p. 185、arXiv : 2502.10254、doi : 10.1145/3706628.3708854、ISBN979-8-4007-1396-5
↑ Lee, Kyuho J. (2021-01-01), Kim, Shiho; Deka, Ganesh Chandra (編)、「第7章 - 深層ニューラルネットワークのためのニューラル処理ユニットのアーキテクチャ」、Advances in Computers、Hardware Accelerator Systems for Artificial Intelligence and Machine Learning、vol. 122、Elsevier、pp. 217–245、doi : 10.1016/bs.adcom.2020.11.001、2025-07-08に取得
1 2 Liao, Heng; Tu, Jiajin; Xia, Jing; Liu, Hu; Zhou, Xiping; Yuan, Honghui; Hu, Yuxing (2021-02-27). "Ascend: ユビキタス深層ニューラルネットワークコンピューティングのためのスケーラブルで統一されたアーキテクチャ: 産業トラック論文". 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA) . pp. 789–801. doi : 10.1109 /HPCA51647.2021.00071 . ISBN978-1-6654-2235-2。
1 2 Kathail, Vinod (2020-02-24). "Xilinx Vitis Unified Software Platform" . Proceedings of the 2020 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays . FPGA '20. New York, NY, USA: Association for Computing Machinery. pp. 173–174 . doi : 10.1145/3373087.3375887 . ISBN978-1-4503-7099-8。
1 2 Zhao, Zhipeng; Hoe, James C. (2017-02-22). "Using Vivado-HLS for Structural Design: A NoC Case Study (Abstract Only)" . Proceedings of the 2017 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays . FPGA '17. New York, NY, USA: Association for Computing Machinery. p. 289. doi : 10.1145/3020078.3021772 . ISBN978-1-4503-4354-1。
