v3.0: Model zoo enhancements - detailed catalog structure and model cards

.gitignore

@@ -0,0 +1,15 @@
+# MacOS temporary files
+.DS_Store
+
+# Temporary file generated after docker_run
+.confirm
+
+# IntelliJ IDEA files
+.idea
+
+# Artifact folders in the Model Zoo
+model_zoo/models/*/*/artifacts
+
+
+/docs/
+/docsrc/build/

docker_run.sh

@@ -77,6 +77,7 @@ docker_run_params=$(cat <<-END
     -v /opt/xilinx/dsa:/opt/xilinx/dsa \
     -v /opt/xilinx/overlaybins:/opt/xilinx/overlaybins \
     -e USER=$user -e UID=$uid -e GID=$gid \
+    -v /sys/kernel/debug:/sys/kernel/debug  --privileged=true \
     -v $DOCKER_RUN_DIR:/vitis_ai_home \
     -v $HERE:/workspace \
     -w /workspace \

docs/_sources/index.rst.txt

@@ -1,5 +1,5 @@
 ..
-    Copyright 2023 Advanced Micro Devices, Inc.
+    Copyright 2022 Xilinx Inc.
 
     Licensed under the Apache License, Version 2.0 (the "License");
     you may not use this file except in compliance with the License.
@@ -18,169 +18,125 @@
 Vitis AI
 ############################################################
 
-AMD Vitis™ AI is an integrated development environment that can be leveraged to accelerate AI inference on AMD platforms. This toolchain provides optimized IP, tools, libraries, models, as well as resources, such as example designs and tutorials that aid the user throughout the development process.  It is designed with high efficiency and ease-of-use in mind, unleashing the full potential of AI acceleration on AMD Adaptable SoCs and Alveo Data Center accelerator cards.
+Xilinx® Vitis™ AI is an integrated development environment that can be leveraged to accelerate AI inference on Xilinx platforms. This toolchain provides optimized IP, tools, libraries, models, as well as resources, such as example designs and tutorials that aid the user throughout the development process.  It is designed with high efficiency and ease-of-use in mind, unleashing the full potential of AI acceleration on Xilinx SoCs and Alveo™ Data Center accelerator cards.
 
 .. figure:: docs/reference/images/VAI_IDE.png
    :width: 1300
 
    Vitis AI Integrated Development Environment Block Diagram
 
-The Vitis |trade| AI solution consists of three primary components:
-
-- The Deep-Learning Processor unit (DPU), a hardware engine for optimized the inferencing of ML models
-- Model development tools, to compile and optimize ML models for the DPU 
-- Model deployment libraries and APIs, to integrate and execute the ML models on the DPU engine from a SW application
-
-The Vitis AI solution is packaged and delivered as follows:
-
-- AMD open download: pre-built target images integrating the DPU
-- Vitis AI docker containers: model development tools
-- Vitis AI github repository: model deployment libraries, setup scripts, examples and reference designs
-
 ############################################################
 Vitis AI Key Components
 ############################################################
 
-****************************
-Deep-Learning Processor Unit
-****************************
-
-The :ref:`Deep-learning Processor Unit (DPU) <workflow-dpu>` is a programmable engine optimized for deep neural networks. The DPU implements an efficient tensor-level instruction set designed to support and accelerate various popular convolutional neural networks, such as VGG, ResNet, GoogLeNet, YOLO, SSD, and MobileNet, among others. 
-
-The DPU supports on AMD Zynq |trade| UltraScale+ |trade| MPSoCs, the Kria |trade| KV260, Versal |trade| and Alveo cards. It scales to meet the requirements of many diverse applications in terms of throughput, latency, scalability, and power.
-
-AMD provides pre-built platforms integrating the DPU engine for both edge and data-center cards. These pre-built platforms allow data-scientists to start developping and testing their models without any need for HW development expertise.
-
-For embedded applications, the DPU needs to be integrated in a custom platform along with the other programmable logic functions going in the FPGA or adaptive SoC device. HW designers can :ref:`integrate the DPU in a custom platform <integrating-the-dpu>` using either the Vitis flow or the Vivado |trade| Design Suite.
-
-
-*****************
-Model Development
-*****************
-
-Vitis AI Model Zoo
-==================
-The :ref:`Vitis AI Model Zoo <workflow-model-zoo>` includes optimized deep learning models to speed up the deployment of deep learning inference on adaptable AMD platforms. These models cover different applications, including ADAS/AD, video surveillance, robotics, and data center. You can get started with these pre-trained models to enjoy the benefits of deep learning acceleration.
-
-Vitis AI Model Inspector
-========================
-The :ref:`Vitis AI Model Inspector <model-inspector>` is used to perform initial sanity checks to confirm that the operators and sequence of operators in the graph is compatible with Vitis AI. Novel neural network architectures, operators, and activation types are constantly being developed and optimized for prediction accuracy and performance. Vitis AI provides mechanisms to leverage operators that are not natively supported by your specific DPU target.
+Vitis AI is composed of the following key components:
 
-Vitis AI Optimizer
-==================
-The :ref:`Vitis AI Optimizer <model-optimization>` exploits the notion of sparsity to reduce the overall computational complexity for inference by 5x to 50x with minimal accuracy degradation. Many deep neural network topologies employ significant levels of redundancy. This is particularly true when the network backbone is optimized for prediction accuracy with training datasets supporting many classes. In many cases, this redundancy can be reduced by “pruning” some of the operations out of the graph. 
 
-Vitis AI Quantizer
-==================
-The :ref:`Vitis AI Quantizer <model-quantization>`, integrated as a component of either TensorFlow or PyTorch, converts 32-bit floating-point weights and activations to fixed-point integers like INT8 to reduce the computing complexity without losing prediction accuracy. The fixed-point network model requires less memory bandwidth and provides faster speed and higher power efficiency than the floating-point model.
+* **DPUs** - Configurable computation engines optimized for convolution neural networks. Efficient and scalable IP cores that can be customized to meet the needs of many different applications and devices.
+* **Model Zoo**  - A comprehensive set of pre-trained and pre-optimized models that are ready to deploy on Xilinx devices.
+* **Model Inspector**  - A tool and methodology through which developers can verify model architecture support.
+* **Optimizer** - An optional, commercially licensed tool that enables users to prune a model by up to 90%.
+* **Quantizer** - A powerful quantizer that supports model quantization, calibration, and fine tuning.
+* **Compiler** - Compiles the quantized model for execution on the target DPU accelerator.
+* **Runtime (VART)** - An inference runtime for Embedded applications.
+* **Profiler** - Performs an in-depth analysis of the efficiency and utilization of AI inference implementations on the DPU.
+* **Library** - Offers high-level C++ APIs for AI applications for embedded and data center use-cases.
 
-Vitis AI Compiler
-=================
-The :ref:`Vitis AI Compiler <model-compilation>` maps the AI quantized model to a highly-efficient instruction set and dataflow model. The compiler performs multiple optimizations; for example, batch normalization operations are fused with convolution when the convolution operator precedes the normalization operator. As the DPU supports multiple dimensions of parallelism, efficient instruction scheduling is key to exploiting the inherent parallelism and potential for data reuse in the graph. The Vitis AI Compiler addresses such optimizations.
 
+.. toctree::
+   :maxdepth: 3
+   :caption: Vitis AI Introduction
+   :hidden:
 
-****************
-Model Deployment
-****************
+   Overview <docs/workflow>
 
-Vitis AI Runtime
-================
-The :ref:`Vitis AI Runtime <vitis-ai-runtime>` (VART) is a set of low-level API functions that support the integration of the DPU into software applications. VART is built on top of the Xilinx Runtime (XRT) amd provides a unified high-level runtime for both Data Center and Embedded targets. Key features of the Vitis AI Runtime API include:
+.. toctree::
+   :maxdepth: 3
+   :caption: Release Notes
+   :hidden:
 
-- Asynchronous submission of jobs to the DPU.
-- Asynchronous collection of jobs from the DPU.
-- C++ and Python API implementations.
-- Support for multi-threading and multi-process execution.
+   Current Release <docs/reference/release_notes_3.0>
 
+.. toctree::
+   :maxdepth: 3
+   :caption: Installation
+   :hidden:
 
-Vitis AI Library
-================
-The :ref:`Vitis AI Library <vitis-ai-library>`  is a set of high-level libraries and APIs built on top of the Vitis AI Runtime (VART). The higher-level APIs included in the Vitis AI Library give developers a head-start on model deployment. While it is possible for developers to directly leverage the Vitis AI Runtime APIs to deploy a model on AMD platforms, it is often more beneficial to start with a ready-made example that incorporates the various elements of a typical application, including:
+   System Requirements <docs/reference/system_requirements>
+   Host Install Instructions <docs/install/install>
+   Target Setup Instructions <docs/board_setup/board_setup>
 
-- Simplified CPU-based pre and post-processing implementations.
-- Vitis AI Runtime integration at an application level.
+.. toctree::
+   :maxdepth: 3
+   :caption: Model Zoo
+   :hidden:
 
+   Pre-trained, Optimized Models <docs/workflow-model-zoo>
+   Getting started <docs/getting-started-model-zoo>
+   Models overview <docs/models-overview>
 
-Vitis AI Profiler
-=================
-The :ref:`Vitis AI Profiler <vitis-ai-profiler>` profiles and visualizes AI applications to find bottlenecks and allocates computing resources among different devices. It is easy to use and requires no code changes. It can trace function calls and run time, and also collect hardware information, including CPU, DPU, and memory utilization.
+.. toctree::
+   :maxdepth: 3
+   :caption: Model Development
+   :hidden:
 
+   Developing a NN Model for Vitis AI <docs/workflow-model-development>
 
+.. toctree::
+   :maxdepth: 3
+   :caption: Model Deployment
+   :hidden:
 
+   Deploying a NN Model with Vitis AI <docs/workflow-model-deployment>
 
 .. toctree::
    :maxdepth: 3
-   :caption: Setup and Install
+   :caption: System Integration
    :hidden:
-
-   Release Notes <docs/reference/release_notes_3.0>
-   System Requirements <docs/reference/system_requirements>
-   Host Install Instructions <docs/install/install>
+
+   Integrating the DPU <docs/workflow-system-integration>
 
 .. toctree::
    :maxdepth: 3
-   :caption: Quick Start Guides
+   :caption: Third-Party Tools
    :hidden:
 
-   Zynq™ Ultrascale+™ <docs/quickstart/mpsoc>
-   Versal™ VCK5000 Development Card <docs/quickstart/vck5000>
-   Versal™ AI Core VCK190 <docs/quickstart/vck190>
-
-
+   TVM, TensorFlow Lite, ONNX Runtime <docs/workflow-third-party>
 
 .. toctree::
    :maxdepth: 3
-   :caption: Workflow and Components
+   :caption: Release Documentation
    :hidden:
 
-   Overview <docs/workflow>
-   DPU IP Details and System Integration <docs/workflow-system-integration>
-   Vitis™ AI Model Zoo <docs/workflow-model-zoo>
-   Developing a Model for Vitis AI <docs/workflow-model-development>
-   Deploying a Model with Vitis AI <docs/workflow-model-deployment>
-
+   Formal Vitis AI Documents <docs/reference/release_documentation>
+
 .. toctree::
    :maxdepth: 3
-   :caption: Reference Designs
+   :caption: Vitis AI Tutorials
    :hidden:
 
-..   Zynq MPSoC / Kria K26 <ref_design_docs/README_DPUCZDX8G.md>
-..   Versal / VCK190 <ref_design_docs/README_DPUCVDX8G.md>
+   Vitis AI Developer Tutorials <https://github.com/Xilinx/Vitis-AI-Tutorials>
 
 .. toctree::
    :maxdepth: 3
-   :caption: Additional Information
+   :caption: Related Solutions
    :hidden:
 
-   Vitis™ AI User Guides & IP Product Guides <docs/reference/release_documentation>
-   Vitis™ AI Developer Tutorials <https://github.com/Xilinx/Vitis-AI-Tutorials>
-   Third-party Inference Stack Integration <docs/workflow-third-party>
-   IP and Tools Compatibility <docs/reference/version_compatibility>
-   Frequently Asked Questions <docs/reference/faq>
-   Branching and Tagging Strategy <docs/install/branching_tagging_strategy>
+   AMD Inference Server <https://xilinx.github.io/inference-server/>
+   Vitis Video Analytics SDK <https://xilinx.github.io/VVAS/>
+   FINN & Brevitas <https://xilinx.github.io/finn/>
+   DPU-PYNQ <https://github.com/Xilinx/DPU-PYNQ>
 
 .. toctree::
    :maxdepth: 3
    :caption: Resources and Support
    :hidden:
 
    Resources and Support <docs/reference/additional_resources>
-   
+
 .. toctree::
    :maxdepth: 3
-   :caption: Related AMD Solutions
+   :caption: FAQ
    :hidden:
 
-   DPU-PYNQ <https://github.com/Xilinx/DPU-PYNQ>
-   FINN & Brevitas <https://xilinx.github.io/finn/>
-   Inference Server <https://xilinx.github.io/inference-server/>
-   Unified Inference Frontend <https://github.com/amd/UIF>   
-   Ryzen™ AI Developer Guide ~July 29 <https://ryzenai.docs.amd.com/en/latest/>
-   Vitis™ AI ONNX Runtime Execution Provider <https://onnxruntime.ai/docs/execution-providers/community-maintained/Vitis-AI-ExecutionProvider.html>
-   Vitis™ Video Analytics SDK <https://xilinx.github.io/VVAS/>
-
-
-.. |trade|  unicode:: U+02122 .. TRADEMARK SIGN
-   :ltrim:
-.. |reg|    unicode:: U+02122 .. TRADEMARK SIGN
-   :ltrim:
+   Frequently Asked Questions <docs/reference/faq>

docs/_static/basic.css

@@ -222,7 +222,7 @@ table.modindextable td {
 /* -- general body styles --------------------------------------------------- */
 
 div.body {
-    min-width: 360px;
+    min-width: 450px;
     max-width: 800px;
 }
 
@@ -237,6 +237,16 @@ a.headerlink {
     visibility: hidden;
 }
 
+a.brackets:before,
+span.brackets > a:before{
+    content: "[";
+}
+
+a.brackets:after,
+span.brackets > a:after {
+    content: "]";
+}
+
 h1:hover > a.headerlink,
 h2:hover > a.headerlink,
 h3:hover > a.headerlink,
@@ -324,16 +334,12 @@ aside.sidebar {
 p.sidebar-title {
     font-weight: bold;
 }
-nav.contents,
-aside.topic,
 
 div.admonition, div.topic, blockquote {
     clear: left;
 }
 
 /* -- topics ---------------------------------------------------------------- */
-nav.contents,
-aside.topic,
 
 div.topic {
     border: 1px solid #ccc;
@@ -373,19 +379,13 @@ div.body p.centered {
 
 div.sidebar > :last-child,
 aside.sidebar > :last-child,
-nav.contents > :last-child,
-aside.topic > :last-child,
-
 div.topic > :last-child,
 div.admonition > :last-child {
     margin-bottom: 0;
 }
 
 div.sidebar::after,
 aside.sidebar::after,
-nav.contents::after,
-aside.topic::after,
-
 div.topic::after,
 div.admonition::after,
 blockquote::after {
@@ -428,6 +428,10 @@ table.docutils td, table.docutils th {
     border-bottom: 1px solid #aaa;
 }
 
+table.footnote td, table.footnote th {
+    border: 0 !important;
+}
+
 th {
     text-align: left;
     padding-right: 5px;
@@ -611,7 +615,6 @@ ul.simple p {
     margin-bottom: 0;
 }
 
-/* Docutils 0.17 and older (footnotes & citations) */
 dl.footnote > dt,
 dl.citation > dt {
     float: left;
@@ -629,33 +632,6 @@ dl.citation > dd:after {
     clear: both;
 }
 
-/* Docutils 0.18+ (footnotes & citations) */
-aside.footnote > span,
-div.citation > span {
-    float: left;
-}
-aside.footnote > span:last-of-type,
-div.citation > span:last-of-type {
-  padding-right: 0.5em;
-}
-aside.footnote > p {
-  margin-left: 2em;
-}
-div.citation > p {
-  margin-left: 4em;
-}
-aside.footnote > p:last-of-type,
-div.citation > p:last-of-type {
-    margin-bottom: 0em;
-}
-aside.footnote > p:last-of-type:after,
-div.citation > p:last-of-type:after {
-    content: "";
-    clear: both;
-}
-
-/* Footnotes & citations ends */
-
 dl.field-list {
     display: grid;
     grid-template-columns: fit-content(30%) auto;