Skip to content

Commit

Permalink
Merge pull request #818 from Shared-Reality-Lab/alpha-beta
Browse files Browse the repository at this point in the history
Added missing handler README files; moved deprecated handlers; added …
  • Loading branch information
jeffbl authored May 1, 2024
2 parents 7ffb744 + 9ec3a44 commit cb255e5
Show file tree
Hide file tree
Showing 60 changed files with 89 additions and 96 deletions.
91 changes: 0 additions & 91 deletions .github/workflows/hello-haptics-handler.yml

This file was deleted.

2 changes: 1 addition & 1 deletion .github/workflows/svg-od-handler.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ jobs:
- name: Install flake8
run: pip install flake8
- name: Check with flake8
run: python -m flake8 ./preprocessors/yolo --show-source
run: python -m flake8 ./handlers/svg-object-detection --show-source
build-and-push-image:
name: Build and Push to Registry
needs: lint
Expand Down
2 changes: 2 additions & 0 deletions handlers/autour-handler/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Autour Handler

Beta quality: Useful enough for testing by end-users.

![license: AGPL](https://camo.githubusercontent.com/b53b1136762ea55ee6a2d641c9f8283b8335a79b3cb95cbab5a988e678e269b8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4147504c2d73756363657373) [GitHub Container Registry Package](https://github.com/Shared-Reality-Lab/IMAGE-server/pkgs/container/image-handler-autour)

## What is this?
Expand Down
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
2 changes: 2 additions & 0 deletions handlers/hello-handler/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Hello, Handler!

Beta quality: Useful enough for testing by end-users.

## What is this?

This is an extremely basic version of a [handler](https://github.com/Shared-Reality-Lab/IMAGE-server/wiki/2.-Handlers,-Preprocessors-and-Services#handlers=) meant to illustrate its basic operation.
Expand Down
2 changes: 2 additions & 0 deletions handlers/hello-svg-handler/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Hello SVG Handler

Beta quality: Useful enough for testing by end-users.

This is a test handler that generates SVGs to demonstrate how it could be done for a visual debugger.
The [visual debugger outputs](https://github.com/Shared-Reality-Lab/IMAGE-server/issues/117) should aid
designers in understanding the outputs of various preprocessors, especially computer vision related ones.
2 changes: 2 additions & 0 deletions handlers/high-charts/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Highcharts Handler

Beta quality: Useful enough for testing by end-users.

![license: AGPL](https://camo.githubusercontent.com/b53b1136762ea55ee6a2d641c9f8283b8335a79b3cb95cbab5a988e678e269b8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4147504c2d73756363657373) [GitHub Container Registry Package](https://github.com/Shared-Reality-Lab/IMAGE-server/pkgs/container/image-handler-high-charts)

## What is this?
Expand Down
2 changes: 2 additions & 0 deletions handlers/map-tactile-svg/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Map Tactile Svg Handler

Alpha quality: Insufficiently refined to be tested by end-users.

![license: AGPL](https://camo.githubusercontent.com/b53b1136762ea55ee6a2d641c9f8283b8335a79b3cb95cbab5a988e678e269b8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4147504c2d73756363657373) [GitHub Container Registry Package](https://github.com/Shared-Reality-Lab/IMAGE-server/pkgs/container/image-handler-map-tactile-svg)

## What is this?
Expand Down
2 changes: 2 additions & 0 deletions handlers/motd/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Message of the Day Handler

Beta quality: Useful enough for testing by end-users.

![license: AGPL](https://camo.githubusercontent.com/b53b1136762ea55ee6a2d641c9f8283b8335a79b3cb95cbab5a988e678e269b8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4147504c2d73756363657373) [GitHub Container Registry Package](https://github.com/Shared-Reality-Lab/IMAGE-server/pkgs/container/image-handler-motd)

## What is this?
Expand Down
3 changes: 3 additions & 0 deletions handlers/ocr-handler/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
# OCR Handler [DEMONSTRATION]

Alpha quality: Insufficiently refined to be tested by end-users.

## What is OCR?
OCR is an acronym for Optical Character Recognition which is the process of extracting machine-readable text pictured in graphics or documents.
We expect that in the future the OCR preprocessor results will likely be rolled
Expand Down
2 changes: 2 additions & 0 deletions handlers/osm-streets-handler/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# OSM Streets

Alpha quality: Insufficiently refined to be tested by end-users.

![license: AGPL](https://camo.githubusercontent.com/b53b1136762ea55ee6a2d641c9f8283b8335a79b3cb95cbab5a988e678e269b8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4147504c2d73756363657373) [GitHub Container Registry Package](https://github.com/Shared-Reality-Lab/IMAGE-server/pkgs/container/image-handler-osm-streets)

This is a handler that displays a street-oriented view of a map, sweeping across certain streets and indicating points of interest along the way.
2 changes: 2 additions & 0 deletions handlers/photo-audio-handler/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Photo Audio Handler

Beta quality: Useful enough for testing by end-users.

![license: AGPL](https://camo.githubusercontent.com/b53b1136762ea55ee6a2d641c9f8283b8335a79b3cb95cbab5a988e678e269b8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4147504c2d73756363657373) [GitHub Container Registry Package](https://github.com/Shared-Reality-Lab/IMAGE-server/pkgs/container/image-handler-photo-audio)

## What is this?
Expand Down
2 changes: 2 additions & 0 deletions handlers/photo-audio-haptics-handler/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Photo Audio Haptics Handler

Alpha quality: Insufficiently refined to be tested by end-users.

![license: AGPL](https://camo.githubusercontent.com/b53b1136762ea55ee6a2d641c9f8283b8335a79b3cb95cbab5a988e678e269b8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4147504c2d73756363657373) [GitHub Container Registry Package](https://github.com/Shared-Reality-Lab/IMAGE-server/pkgs/container/image-handler-photo-audio-haptics)

## What is this?
Expand Down
2 changes: 2 additions & 0 deletions handlers/photo-tactile-svg/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Photo Tactile Svg Handler

Alpha quality: Insufficiently refined to be tested by end-users.

![license: AGPL](https://camo.githubusercontent.com/b53b1136762ea55ee6a2d641c9f8283b8335a79b3cb95cbab5a988e678e269b8/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f6c6963656e73652d4147504c2d73756363657373) [GitHub Container Registry Package](https://github.com/Shared-Reality-Lab/IMAGE-server/pkgs/container/image-handler-photo-tactile-svg)

## What is this?
Expand Down
3 changes: 3 additions & 0 deletions handlers/svg-action-recognition/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Alpha quality: Insufficiently refined to be tested by end-users.

This handler creates a visual debug output for debugging the action-recognition preprocessor.
3 changes: 3 additions & 0 deletions handlers/svg-depth-map/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Beta quality: Useful enough for testing by end-users.

This handler creates a visual debug output for the depth-map preprocessor.
3 changes: 3 additions & 0 deletions handlers/svg-object-detection/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Beta quality: Useful enough for testing by end-users.

This handler creates a visual debug output for object-detection preprocessors.
3 changes: 3 additions & 0 deletions handlers/svg-open-street-map/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Alpha quality: Insufficiently refined to be tested by end-users.

This handler creates a visual debug output for the open-street-map preprocessor.
3 changes: 3 additions & 0 deletions handlers/svg-semantic-seg/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
Beta quality: Useful enough for testing by end-users.

This handler creates a visual debug output for a semantic segmentation preprocessor.
2 changes: 2 additions & 0 deletions preprocessors/action-recognition/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# About

Alpha quality: Insufficiently refined to be tested by end-users.

This preprocessor is used for detecting human action from images. The preprocessor was trained using transfer learning with a pretrained DINOv2 model as a base and a linear head trained from scratch on action images. DINOv2 ViT-S/14 distilled was released by Meta AI Research (https://github.com/facebookresearch/dinov2). Output from object detection is used to find all instances of 'person' in a graphic and classify each instance individually

## Datasets Used
Expand Down
2 changes: 2 additions & 0 deletions preprocessors/autour/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
Beta quality: Useful enough for testing by end-users.

# Usage

The autour preprocessor is used to analyze an embedded google map and return locations of interest around the maps locations.
Expand Down
2 changes: 2 additions & 0 deletions preprocessors/caption-recognition/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Caption Recognition Preprocessor

Alpha quality: Insufficiently refined to be tested by end-users.

This preprocessor provides a single sentence caption for the image. Currently the code uses [BLIP](https://github.com/salesforce/BLIP), which is a huggingface based vision transformer that can generate accurate captions.

The code to use this module as an API can be found in `caption.py`. This module is fully versionned, the versions of the libraries used can be found in `requirements.txt` and in the `Dockerfile`.
Expand Down
2 changes: 2 additions & 0 deletions preprocessors/celebrity-detector/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Celebrity Detection Preprocessor

Alpha quality: Insufficiently refined to be tested by end-users.

This preprocessor detects the celebrities present in an image. The code crops individual people in an image and uses Azure Computer Vision API to determine if the cropped individual is a celebrity.
The code performing all these operations can be found in `celebrity-detector.py`

Expand Down
2 changes: 2 additions & 0 deletions preprocessors/chart-pipeline/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
Beta quality: Useful enough for testing by end-users.

# Chart data extraction pipeline

This pipeline is based on the DeepRule paper:
Expand Down
2 changes: 2 additions & 0 deletions preprocessors/clothes-detector/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Clothes Detection Preprocessor

Alpha quality: Insufficiently refined to be tested by end-users.

This preprocessor detects the type of clothing of individual people in a photograph. The following are the steps performed by the preprocessor

1. We first crop individual people from the image, then detect the clothes worn by each individual. We have used YOLOv3 in the backend to detect and localise the clothes worn by each individual. The weights were obtained from [https://github.com/simaiden/Clothing-Detection](https://github.com/simaiden/Clothing-Detection)
Expand Down
2 changes: 2 additions & 0 deletions preprocessors/collage-detector/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Per-image Classifier for Identifying Collages or Non-collages

Alpha quality: Insufficiently refined to be tested by end-users.

## Overview

This preprocessor classifies whether the input image is a collage or a non-collage. The Dockerfile builds a fully functioning IMAGE preprocessor.
Expand Down
4 changes: 2 additions & 2 deletions preprocessors/collage-detector/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
opencv-python~=4.7.0.68
numpy~=1.23.3
numpy~=1.26.4
tqdm~=4.64.1
Flask~=2.2.2
flask_api==2.0.0
jsonschema~=4.17.3
gunicorn==20.1.0
gunicorn==20.1.0
2 changes: 2 additions & 0 deletions preprocessors/content-categoriser/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
Beta quality: Useful enough for testing by end-users.

This preprocessor classifies whether the input image is a picture or a chart. The preprocessor is trained from scratch using transfer learning. The following datasets were used in training our preprocessor
## Datasets used

Expand Down
2 changes: 2 additions & 0 deletions preprocessors/depth-map-gen/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
<h1>Depth Estimation</h1>

Beta quality: Useful enough for testing by end-users.

This proprocessor will take any image as an input and output the estimated depth. The goal being that the model will allow for other preprocessors to associate depth
to different components of the image, such as objects, and provide another dimension to the sound renderings.

Expand Down
2 changes: 2 additions & 0 deletions preprocessors/emotion-recognition/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Expression Recognition Preprocessor

Alpha quality: Insufficiently refined to be tested by end-users.

This preprocessor has been designed to recognize human expressions and categorize them as `happy, neutral, or sad`. It is built on the Deepface library, which offers a variety of models like `FaceNet, Openface, DeepFace,` and more. The models can be easily switched by specifying the `detector_backend` parameter when calling the `Deepface.analyze` function. For more information on model selection, please refer to the [Deepface Readme](https://github.com/serengil/deepface).

To use this module as an API, you can follow the example provided in the `emotion-recognition.py` file. The necessary dependencies and their associated versions are clearly specified in the `requirements.txt`
Expand Down
2 changes: 2 additions & 0 deletions preprocessors/graphic-tagger/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
Beta quality: Useful enough for testing by end-users.

This preprocessor is used to determine if the image is an indoor or an outdoor image. This preprocessor could be used for getting more information about the image as well. This preprocessor has been outsourced to Azure and we access it using the API key.

According to Azure documents the preprocessor can return [86 tags](https://docs.microsoft.com/en-us/azure/cognitive-services/computer-vision/category-taxonomy), but we have limited the images to just indoor and outdoor.
Expand Down
2 changes: 2 additions & 0 deletions preprocessors/grouping/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
Beta quality: Useful enough for testing by end-users.

The grouping preprocessor tries to group in the objects detected by the Object Detection based on their type. The following libraries were used for creating the grouping preprocessor


Expand Down
3 changes: 3 additions & 0 deletions preprocessors/line-charts/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
# Line-charts Preprocessor

Beta quality: Useful enough for testing by end-users.

## The Line-charts preprocessor extracts useful information from the given graph.
This preprocessor is used to extract further insights from the given input graph (in the highcharts format). The additional keys added to the data include:
*lowerPointsOnLeft: number of continous points lower than the current point, lying on the left side of current point
Expand Down
2 changes: 2 additions & 0 deletions preprocessors/mmsemseg/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Semantic Segmentation module

Beta quality: Useful enough for testing by end-users.

This preprocessor is used for semantically segmenting images. The code for this preprocessor heavily relies on the [MMSegmentation](https://github.com/open-mmlab/mmsegmentation) framework.

The code to use this module as an API can be found in `segment.py`, additional functions are located in `utils.py`. This module is fully versionned, the versions of the libraries used can be found in `requirements.txt` and in the `Dockerfile`.
Expand Down
2 changes: 2 additions & 0 deletions preprocessors/ner/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Named Entity Recognition (NER)

Alpha quality: Insufficiently refined to be tested by end-users.

This preprocessor takes alt text and a photograph. It uses [CLIPScore](https://arxiv.org/abs/2104.08718) to determine whether or not the alt text describes the content of the photo (rather than adding additional context or the photo being for "flavor", as is often done in news articles).
If the score is above a certain threshold, the [Stanford Named Entity Recognizer](https://nlp.stanford.edu/software/CRF-NER.shtml) is used to find and tag named entities in this alt text.
The results are included in the response.
Expand Down
2 changes: 2 additions & 0 deletions preprocessors/nominatim/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Nominatim Preprocessor

Beta quality: Useful enough for testing by end-users.

This preprocessor looks for geospatial coordinates in the request (i.e., a latitude and longitude) and uses [Nominatim](https://nominatim.org) to find a common name for the location, if possible.
This name is either a civic address or the proper name of a business, landmark, organization, etc.
The public Nominatim server is used by default, but can be overridden with the `NOMINATIM_SERVER` environment variable.
2 changes: 2 additions & 0 deletions preprocessors/object-depth-calculator/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Object Depth Calculator

Beta quality: Useful enough for testing by end-users.

This preprocessor aims at combining the outputs from the depth-map-generator preprocessor and the obejct-detection preprocessor.
It calculates the depth of an object by taking the median pixel depth value within the bounding box coordinates as seen in:

Expand Down
2 changes: 2 additions & 0 deletions preprocessors/object-detection-azure/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
Beta quality: Useful enough for testing by end-users.

This preprocessor uses the object detection feature of [Azure](https://docs.microsoft.com/en-us/azure/cognitive-services/computer-vision/concept-object-detection). Once the objects are detected, they are normalised and returned back in a pre-defined formate.

We have created a docker container to ensure ease of maintenance of the code.
Expand Down
3 changes: 3 additions & 0 deletions preprocessors/ocr/README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
# OCR Preprocessor

Alpha quality: Insufficiently refined to be tested by end-users.

## What is OCR?
OCR is an acronym for Optical Character Recognition which is the process of extracting machine-readable text pictured in graphics or documents.
## About this preprocessor
Expand Down
2 changes: 1 addition & 1 deletion preprocessors/openstreetmap/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,4 @@ EXPOSE 5000

ENV FLASK_APP=main.py
USER python
CMD [ "gunicorn", "main:app", "-b", "0.0.0.0:5000", "--capture-output", "--log-level=debug" ]
CMD [ "gunicorn", "main:app", "-b", "0.0.0.0:5000", "--capture-output", "--log-level=debug" ]
4 changes: 3 additions & 1 deletion preprocessors/openstreetmap/Readme.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# Description

Alpha quality: Insufficiently refined to be tested by end-users.

This service basically involves automatic communication with the openstreetmap (OSM), based on the user's orientation and location, to extract critical map information within an optimum user's radius. The extracted data would be transformed to a suitable data framework, for effective audio and haptic rendering.

## Environment setup
Expand Down Expand Up @@ -173,4 +175,4 @@ a. Computing bounding_box-

6. https://wiki.openstreetmap.org/wiki/Overpass_API/Overpass_QL

7. https://wiki.openstreetmap.org/wiki/Overpass_API
7. https://wiki.openstreetmap.org/wiki/Overpass_API
2 changes: 2 additions & 0 deletions preprocessors/sorting/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
Beta quality: Useful enough for testing by end-users.

This preprocessor has been created in order to make sense of the outputs generated by object detection preprocessor. The object detection just detects the objects and sends them back in a json format. The sorting preprocessor arranges these objects in 3 different categories namely:

```
Expand Down
2 changes: 2 additions & 0 deletions preprocessors/yolov8/README.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
Beta quality: Useful enough for testing by end-users.

This preprocessor detects the objects present in the image. The entire code for this preprocessor has been refered from [ultralytics Repository](https://github.com/ultralytics/ultralytics).

In order to run this preprocessor we have used the [ultralytics YOLOv8 image](https://hub.docker.com/r/ultralytics/ultralytics) as a base docker image. The image is versioned with tag 'latest' and digest 'sha256:af4572bf0df164d5708e4a1f50bddb5a74cb7d793fe46b7daf42b1768fbec608'. The following commands import the base docker image:
Expand Down

0 comments on commit cb255e5

Please sign in to comment.