Skip to content

Latest commit

 

History

History
184 lines (126 loc) · 4.93 KB

Introduction.md

File metadata and controls

184 lines (126 loc) · 4.93 KB

Local Development Setup

  1. Install miniconda
  2. Clone code and checkout branch
git clone https://github.com/arylwen/fsdl_project.git fsdl_project_docker
cd fsdl_project_docker/
git checkout docker_template_exp
  1. Use environment.yml file for creating new environment in conda
make conda-update
conda activate fsdl_project

4a. (Optional, only if packages have changed)

make pip-tools-upgrade
  1. Install additional python packages in the newly created conda environment using pip-tools
make pip-tools

Running Inference

Refer tutorial jupyter notebook for running inference.

Running Flask Server

  1. Run the server
python3 semantic_search/ml_api/run.py to start the Flask Server.
  1. Goto http://localhost:5000/ui/.
  2. Goto POST /v1/predictions endpoint.
  3. Click on Try Now and enter text under the section "Text query to match an image" For eg. Type "Running dogs" and hit execute.
  4. Response will be generated under the Response Body section below.

Response

corpus_id - Pointing to Image Index Number in the preseeded Image Corpus
Score - Cosine Similarity
version - Model version 

{
  "errors": [],
  "predictions": [
    {
      "corpus_id": 2,
      "score": 0.237429678440094
    },
    {
      "corpus_id": 3,
      "score": 0.23457613587379456
    },
    {
      "corpus_id": 0,
      "score": 0.19539634883403778
    }
  ],
  "version": "0.1.0"
}

Docker Setup

Run the application manually

  1. Build Docker Container
sudo docker build -f docker/Dockerfile  -t fsdl .
  1. Run docker container[bash]
sudo docker run -it -p 5000:5000 fsdl bash

sudo docker run -it -p 5000:5000 --name ml_api docker_ml_api bash
  1. Run the flask app
python3 ml_api/run.py
  1. Goto http://localhost:5000/ui/

Run Using Docker Compose

  1. Build and Run Docker Containers
sudo docker-compose -f docker/docker-compose.yml up -d --build

  1. Ensure all 4 docker images are created and running

    1. Images corpus will be pulled 2. Model will be exposed at /v1/predictions endpoint. 2. /metric endpoint will be exposed for Prometheus server to pull monitoring data.

  2. Go to http://localhost:5000/ui/ and try sending the request by adding a text query.

  1. Ensure you get the response back as shown above in the Response body section.

Monitoring Metrics with Prometheus

  1. Go to http://localhost:3000/ and login Grafana with username admin and password foobar

  2. In the home dashboard ,click on Add Datasource and select Prometheus.

  3. Under HTTP-> URL section add http://prometheus:9090 and click Save and Test. Ensure you get a message saying "Data Source is working".

  4. Monitor Endpoints Metrics

    1. Go to Dashboards -> Browse -> New -> Import

    2. Import dashboard grafana_flask_basic_dashboard_ml_api.json" present in our repository from semantic_search/docker/config/grafana/

    3. Dashboard will be loaded which is configured to monitor two metrics : Request Latency ,Latency

      1. Built using flask_before_request and flask_after_request hooks to capture metrics with each HTTP request to the application.

    4. Try sending some requests from http://localhost:5000/ui/ , the data points will start appearing on the dashboard

Monitor Infrastructure Metrics

  1. Import dashboard basic_cadvisor_dashboard_ml_api.json" present in the repository from semantic_search/docker/config/grafana/

​

  1. Monitor ML Model Metrics

  2. Import dashboard ml_api_dashboard.json" present in the repository from semantic_search/docker/config/grafana/

Trouble Shooting

ERROR: for ml_api no such image: sha256:c3df073f09547286f8901569a25498cb09a11d94e8e2250c80074d916eee6b90: No such image: sha256:c3df073f09547286f8901569a25498cb09a11d94e8e2250c80074d916eee6b90 ERROR: The image for the service you're trying to recreate has been removed. If you continue, volume data could be lost. Consider backing up your data before continuing.

Solution :Remove stopped docker containers Go to semantic_search/docker and run: docker-compose rmand remove any stopped docker container.

Docker Commands

##List all docker containers

sudo docker container ls --all

or

sudo docker container -p

References

  1. prometheus_flask_exporter
  2. use-grafana-to-monitor-flask-apps-with-prometheus
  3. flask_prometheus_metrics