diff --git a/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/_index.md b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/_index.md index b6edd39c..80bb8819 100644 --- a/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/_index.md +++ b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/_index.md @@ -1,7 +1,7 @@ --- type: docs weight: 4 -title: Welding Defect +title: Welding defect scenario using OpenVino and Kubernetes linkTitle: Welding defect scenario using OpenVino and Kubernetes summary: | The Welding Defect page provides an overview of the welding defect scenario in the Contoso Motors solution. It describes the architecture and flow of information for detecting and classifying welding defects using AI. The page also explains the steps involved in the welding defect inference process, including UI selection, RTSP video simulation, frame capturing, image pre-processing/inferencing, and post-processing/rendering. @@ -41,7 +41,7 @@ This diagram shows the welding defect inference flow, which consists of five mai 4. Transpose the dimensions of the input image from (height, width, channels) to (channels, height, width). 5. Add a new dimension to the input image at the beginning of the array to create a "batch" of images. 6. Flip the order of the color channels from RGB to BGR. - + After the pre-processing step is completed, the final frame data is sent to the OpenVINO model server for inference. This is achieved using gRPC and the [ovmsclient](https://pypi.org/project/ovmsclient/) library, which provides a convenient and efficient way to communicate with the server. The server uses the OpenVINO toolkit to perform the inference process, which involves running the input data through a trained machine learning model to generate predictions or classifications. Once the inference is complete, the results are returned to the client for further processing or display. 1. **Frame post-processing/rednering:** this is the final step and involves parsing the inference reposnse and apply the required post-process. For this welding model, the post-process involves the following transformations: @@ -53,9 +53,9 @@ This diagram shows the welding defect inference flow, which consists of five mai 5. Map the index of the highest probability value to a corresponding class label. 6. Draw the label containing the predicted class and its corresponding probability on the input image - Once the image is processed, is then served to the main application to render it to the user. Final image contains the result of the welding inference (**no weld**, **normal weld**, **porosity**) and the probability of the result. + Once the image is processed, is then served to the main application to render it to the user. Final image contains the result of the welding inference (**no weld**, **normal weld**, **porosity**) and the probability of the result. -![Welding defect uI](./img/welding_ui.png) +![Welding defect UI](./img/welding_ui.png) If you're interested in learning more about the AI inference flow, check out the [AI Inference Architecture](./ai_inferencing) page for additional information. @@ -77,6 +77,26 @@ Expected color order is BGR. The features is a blob with the shape 1, 3 containing probability scores for three output classes (**no weld**, **normal weld** and **porosity**). +## Operation technology (OT) Manager Experience + +Contoso leverages their AI-enhanced computer vision to monitor the welding process and help OT managers detect welding defects through the "Control Center" interface. + +- To access the "Control Center" interface, select the Control center [_env_] option from the _Control center_ Bookmarks folder. Each environment will have it's own "Control Center" instance with a different IP. Select one of the sites and click on the factory image to start navigating the different factory control centers. + + ![Screenshot showing the Control center Bookmark](./img/control-center-menu.png) + +- Click on the "Site" control center. + + ![Screenshot showing the two Control centers](./img/control-center-site.png) + +- Click on the "Welding" control center image. + + ![Screenshot showing the welding control center](./img/control-center-welding.png) + +- You can now see the AI-enhanced computer vision in action analyzing the video feed and showing the probability of the welding process detecting any defects. + + ![Screenshot showing the welding defect monitoring feed](./img/welding-defect.png) + ## Next steps Now that you have completed the data pipeline scenario, it's time to continue to the next scenario, [Workers safety using AI](../workers_safety/). \ No newline at end of file diff --git a/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/img/control-center-menu.png b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/img/control-center-menu.png new file mode 100644 index 00000000..4a4ad497 Binary files /dev/null and b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/img/control-center-menu.png differ diff --git a/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/img/control-center-site.png b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/img/control-center-site.png new file mode 100644 index 00000000..27e82ca2 Binary files /dev/null and b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/img/control-center-site.png differ diff --git a/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/img/control-center-welding.png b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/img/control-center-welding.png new file mode 100644 index 00000000..608e88ea Binary files /dev/null and b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/img/control-center-welding.png differ diff --git a/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/img/flow.png b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/img/flow.png index df266f50..86b9de0e 100644 Binary files a/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/img/flow.png and b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/img/flow.png differ diff --git a/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/img/welding-defect.png b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/img/welding-defect.png new file mode 100644 index 00000000..5bc9c4bd Binary files /dev/null and b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/img/welding-defect.png differ diff --git a/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/_index.md b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/_index.md index fb8f4e00..755cb6f7 100644 --- a/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/_index.md +++ b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/_index.md @@ -1,11 +1,55 @@ --- type: docs weight: 5 -title: WIP -linkTitle: WIP +title: Enabling AI at the Edge to enhance workers safety +linkTitle: Enabling AI at the Edge to enhance workers safety summary: | - WIP -serviceOrPlatform: WIP + The enabling AI at the Edge to enhance workers safety page provides an overview of how Contoso Motors uses AI to ensure workers' safety by detecting workers with no helmets on the factory floor. It describes the architecture and flow of information for detecting and classifying helmet adherence using AI. The page also explains the steps involved in the inference process, including UI selection, RTSP video simulation, frame capturing, image pre-processing/inferencing, and post-processing/rendering. +serviceOrPlatform: Manufacturing technologyStack: - WIP ---- \ No newline at end of file + - AKS + - OPENVINO + - AI + - AKS EDGE ESSENTIALS + - RTSP +--- + +# Enabling AI at the Edge to enhance workers safety + +## Overview + +Contoso Motors uses AI-enhanced computer vision to improve workers' safety by detecting workers with no helmets on the factory floor. Worker safety is one of the four computer vision use cases that Contoso Motors uses, which also include object detection, defect detection, and human pose estimation. While each use case has its own unique characteristics, they all follow the same inferencing architecture pattern and data flow. + +## Architecture + +![Workers safety](./img/flow.png) + +### Model + +#### Inputs + +#### Outputs + +## Operation technology (OT) Manager Experience + +Contoso uses AI-enhanced computer vision to monitor the safety helmet adherence for workers on the factory floor to help OT managers ensure workers safety through the "Control Center" interface. + +- To access the "Control Center" interface, select the Control center [_env_] option from the _Control center_ Bookmarks folder. Each environment will have it's own "Control Center" instance with a different IP. Select one of the sites and click on the factory image to start navigating the different factory control centers. + +![Screenshot showing the Control center Bookmark](./img/control-center-menu.png) + +- Click on the "Site" control center. + + ![Screenshot showing the two Control centers](./img/control-center-site.png) + +- Click on the "Workers safety" control center image. + + ![Screenshot showing the workers safety control center](./img/control-center-workers-safety.png) + +- You can now see the AI-enhanced computer vision in action analyzing the video feed to detect if workers are adhering to Contoso's safety helmet policies in the factory floor. + + ![Screenshot showing the helmet detection monitoring feed](./img/control-center-helmet-detection.png) + +## Next steps + +Now that you have completed the workers safety scenario, it's time to continue to the next scenario, [Infrastructure observability for Kubernetes and Arc-enabled Kubernetes](../k8s_infra_observability/). diff --git a/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/img/control-center-helmet-detection.png b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/img/control-center-helmet-detection.png new file mode 100644 index 00000000..a6cdaf8d Binary files /dev/null and b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/img/control-center-helmet-detection.png differ diff --git a/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/img/control-center-menu.png b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/img/control-center-menu.png new file mode 100644 index 00000000..4a4ad497 Binary files /dev/null and b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/img/control-center-menu.png differ diff --git a/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/img/control-center-site.png b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/img/control-center-site.png new file mode 100644 index 00000000..27e82ca2 Binary files /dev/null and b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/img/control-center-site.png differ diff --git a/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/img/control-center-workers-safety.png b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/img/control-center-workers-safety.png new file mode 100644 index 00000000..e478ce62 Binary files /dev/null and b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/img/control-center-workers-safety.png differ diff --git a/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/img/flow.png b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/img/flow.png new file mode 100644 index 00000000..40daa7fa Binary files /dev/null and b/docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/img/flow.png differ diff --git a/img/about/universe.png b/img/about/universe.png index cc9fbaa6..b5bbdcad 100644 Binary files a/img/about/universe.png and b/img/about/universe.png differ