Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ag manufacturing patches #315

Merged
merged 13 commits into from
May 17, 2024
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
type: docs
weight: 4
title: Welding Defect
title: Welding defect scenario using OpenVino and Kubernetes
linkTitle: Welding defect scenario using OpenVino and Kubernetes
summary: |
The Welding Defect page provides an overview of the welding defect scenario in the Contoso Motors solution. It describes the architecture and flow of information for detecting and classifying welding defects using AI. The page also explains the steps involved in the welding defect inference process, including UI selection, RTSP video simulation, frame capturing, image pre-processing/inferencing, and post-processing/rendering.
Expand All @@ -18,7 +18,7 @@

## Overview

Contoso Motors uses AI-enhanced computer vision to improve welding operations on its assembly lines. Welding is one of the four computer vision use cases that Contoso Motors uses, which also include object detection, human pose estimation, and safety helmet detection. While each use case has its own unique characteristics, they all follow the same inferencing architecture pattern and data flow.

Check failure on line 21 in docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/_index.md

View workflow job for this annotation

GitHub Actions / lint

[vale] reported by reviewdog 🐶 [Vale.Spelling] Did you really mean 'inferencing'? Raw Output: {"message": "[Vale.Spelling] Did you really mean 'inferencing'?", "location": {"path": "docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/_index.md", "range": {"start": {"line": 21, "column": 351}}}, "severity": "ERROR"}

Welding is a process of joining two or more metal parts by melting and fusing them together. Welding defects are flaws or irregularities that occur during or after the welding process, which can affect the quality, strength, and appearance of the weld. Welding defects can be caused by various factors, such as improper welding parameters, inadequate preparation, poor welding technique, or environmental conditions. In this scenario, an AI model is used to automatically detect and classify welding defects from a video feed. Welding defect inference can help improve the efficiency, accuracy, and safety of weld inspection and quality control.

Expand All @@ -41,7 +41,7 @@
4. Transpose the dimensions of the input image from (height, width, channels) to (channels, height, width).
5. Add a new dimension to the input image at the beginning of the array to create a "batch" of images.
6. Flip the order of the color channels from RGB to BGR.

After the pre-processing step is completed, the final frame data is sent to the OpenVINO model server for inference. This is achieved using gRPC and the [ovmsclient](https://pypi.org/project/ovmsclient/) library, which provides a convenient and efficient way to communicate with the server. The server uses the OpenVINO toolkit to perform the inference process, which involves running the input data through a trained machine learning model to generate predictions or classifications. Once the inference is complete, the results are returned to the client for further processing or display.

1. **Frame post-processing/rednering:** this is the final step and involves parsing the inference reposnse and apply the required post-process. For this welding model, the post-process involves the following transformations:
Expand All @@ -53,9 +53,9 @@
5. Map the index of the highest probability value to a corresponding class label.
6. Draw the label containing the predicted class and its corresponding probability on the input image

Once the image is processed, is then served to the main application to render it to the user. Final image contains the result of the welding inference (**no weld**, **normal weld**, **porosity**) and the probability of the result.
Once the image is processed, is then served to the main application to render it to the user. Final image contains the result of the welding inference (**no weld**, **normal weld**, **porosity**) and the probability of the result.

![Welding defect uI](./img/welding_ui.png)
![Welding defect UI](./img/welding_ui.png)

If you're interested in learning more about the AI inference flow, check out the [AI Inference Architecture](./ai_inferencing) page for additional information.

Expand All @@ -77,6 +77,26 @@

The features is a blob with the shape 1, 3 containing probability scores for three output classes (**no weld**, **normal weld** and **porosity**).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

something about this sentence seems off.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fcabrera23 can you help with explaining what this sentence should say ?


## Operation technology (OT) Manager Experience

Contoso leverages their AI-enhanced computer vision to monitor the welding process and help OT managers detect welding defects through the "Control Center" interface.

- To access the "Control Center" interface, select the Control center [_env_] option from the _Control center_ Bookmarks folder. Each environment will have it's own "Control Center" instance with a different IP. Select one of the sites and click on the factory image to start navigating the different factory control centers.

Check failure on line 84 in docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/_index.md

View workflow job for this annotation

GitHub Actions / lint

[vale] reported by reviewdog 🐶 [Vale.Spelling] Did you really mean 'env'? Raw Output: {"message": "[Vale.Spelling] Did you really mean 'env'?", "location": {"path": "docs/azure_jumpstart_ag/manufacturing/contoso_motors/welding_defect/_index.md", "range": {"start": {"line": 84, "column": 73}}}, "severity": "ERROR"}

![Screenshot showing the Control center Bookmark](./img/control-center-menu.png)

- Click on the "Site" control center.

![Screenshot showing the two Control centers](./img/control-center-site.png)

- Click on the "Welding" control center image.

![Screenshot showing the welding control center](./img/control-center-welding.png)

- You can now see the AI-enhanced computer vision in action analyzing the video feed and showing the probability of the welding process detecting any defects.

![Screenshot showing the welding defect monitoring feed](./img/welding-defect.png)

## Next steps

Now that you have completed the data pipeline scenario, it's time to continue to the next scenario, [Workers safety using AI](../workers_safety/).
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -1,11 +1,55 @@
---
type: docs
weight: 5
title: WIP
linkTitle: WIP
title: Enabling AI at the Edge to enhance workers safety
linkTitle: Enabling AI at the Edge to enhance workers safety
summary: |
WIP
serviceOrPlatform: WIP
The enabling AI at the Edge to enhance workers safety page provides an overview of how Contoso Motors uses AI to ensure workers' safety by detecting workers with no helmets on the factory floor. It describes the architecture and flow of information for detecting and classifying helmet adherence using AI. The page also explains the steps involved in the inference process, including UI selection, RTSP video simulation, frame capturing, image pre-processing/inferencing, and post-processing/rendering.
serviceOrPlatform: Manufacturing
technologyStack:
WIP
---
- AKS
- OPENVINO
- AI
- AKS EDGE ESSENTIALS
- RTSP
---

# Enabling AI at the Edge to enhance workers safety

## Overview

Contoso Motors uses AI-enhanced computer vision to improve workers' safety by detecting workers with no helmets on the factory floor. Worker safety is one of the four computer vision use cases that Contoso Motors uses, which also include object detection, defect detection, and human pose estimation. While each use case has its own unique characteristics, they all follow the same inferencing architecture pattern and data flow.

Check failure on line 21 in docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/_index.md

View workflow job for this annotation

GitHub Actions / lint

[vale] reported by reviewdog 🐶 [Vale.Spelling] Did you really mean 'inferencing'? Raw Output: {"message": "[Vale.Spelling] Did you really mean 'inferencing'?", "location": {"path": "docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/_index.md", "range": {"start": {"line": 21, "column": 383}}}, "severity": "ERROR"}

## Architecture

![Workers safety](./img/flow.png)

### Model

#### Inputs

#### Outputs

## Operation technology (OT) Manager Experience

Contoso uses AI-enhanced computer vision to monitor the safety helmet adherence for workers on the factory floor to help OT managers ensure workers safety through the "Control Center" interface.

- To access the "Control Center" interface, select the Control center [_env_] option from the _Control center_ Bookmarks folder. Each environment will have it's own "Control Center" instance with a different IP. Select one of the sites and click on the factory image to start navigating the different factory control centers.

Check failure on line 37 in docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/_index.md

View workflow job for this annotation

GitHub Actions / lint

[vale] reported by reviewdog 🐶 [Vale.Spelling] Did you really mean 'env'? Raw Output: {"message": "[Vale.Spelling] Did you really mean 'env'?", "location": {"path": "docs/azure_jumpstart_ag/manufacturing/contoso_motors/workers_safety/_index.md", "range": {"start": {"line": 37, "column": 73}}}, "severity": "ERROR"}

![Screenshot showing the Control center Bookmark](./img/control-center-menu.png)

- Click on the "Site" control center.

![Screenshot showing the two Control centers](./img/control-center-site.png)

- Click on the "Workers safety" control center image.

![Screenshot showing the workers safety control center](./img/control-center-workers-safety.png)

- You can now see the AI-enhanced computer vision in action analyzing the video feed to detect if workers are adhering to Contoso's safety helmet policies in the factory floor.

![Screenshot showing the helmet detection monitoring feed](./img/control-center-helmet-detection.png)

## Next steps

Now that you have completed the workers safety scenario, it's time to continue to the next scenario, [Infrastructure observability for Kubernetes and Arc-enabled Kubernetes](../k8s_infra_observability/).
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file modified img/about/universe.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading