Allow data sources to be synced from the UI

.envs/.local/.django

@@ -20,3 +20,11 @@ NEW_RELIC_CONFIG_FILE=newrelic.ini
 NEW_RELIC_ENVIRONMENT=development
 
 DJANGO_CSRF_TRUSTED_ORIGINS=http://localhost:3000,
+
+MINIO_ENDPOINT=http://minio:9000
+MINIO_ROOT_USER=amistorage
+MINIO_ROOT_PASSWORD=amistorage
+MINIO_DEFAULT_BUCKET=ami
+MINIO_STORAGE_USE_HTTPS=False
+MINIO_TEST_BUCKET=ami-test
+MINIO_BROWSER_REDIRECT_URL=http://minio:9001

README.md

@@ -95,3 +95,26 @@ The following details how to deploy this application.
 ### Docker
 
 See detailed [cookiecutter-django Docker documentation](http://cookiecutter-django.readthedocs.io/en/latest/deployment-with-docker.html).
+
+
+## Project Data Storage
+
+Each project manages its own external data storage where the AMI Platform will index and process images. This is most typically a public or private S3 bucket at a cloud provider that is not AWS. For example
+the Swift object storage service at Compute Canada or a university's own storage service.
+
+To test the S3 storage backend locally, Minio is configured to run as part of the docker compose stack.
+
+To configure a project connect to the Minio service, you can use the following config:
+
+```
+Endpoint URL: http://minio:9000
+Access key: amistorage
+Secret access key: amistorage
+Public base URL: http://localhost:9000/ami/
+Bucket: ami
+```
+
+- Open the Minio web interface at http://localhost:9001 and login with the access key and secret access key.
+- Upload some test images to a subfolder in the `ami` bucket (one subfolder per deployment)
+- Give the bucket or folder anonymous access using the "Anonymous access" button in the Minio web interface.
+- You _can_ test private buckets and presigned URLs, but you will need to add an entry to your local /etc/hosts file to map the `minio` hostname to localhost.

ami/jobs/admin.py

@@ -19,6 +19,7 @@ class JobAdmin(AdminBase):
         "started_at",
         "finished_at",
         "duration",
+        "get_job_type_display",
     )
 
     @admin.action()
@@ -27,6 +28,10 @@ def enqueue_jobs(self, request: HttpRequest, queryset: QuerySet[Job]) -> None:
             job.enqueue()
         self.message_user(request, f"Queued {queryset.count()} job(s).")
 
+    @admin.display(description="Job Type")
+    def get_job_type_display(self, obj: Job) -> str:
+        return obj.job_type().name
+
     actions = [enqueue_jobs]
 
     exclude = (