Make human-friendly names available and allow multi-grid downloads #932

abyrd · 2024-02-16T15:55:03Z

The backend returns download URLs as small JSON objects with a url property. In this PR, these JSON responses are extended to also have a humanName property, which is an alternate filename that is more human readable but not guaranteed to be unique. This enables #673 (as discussed in #673 (comment)) but should be backward compatible with existing UI as responses still have the same old url property. The new humanName property can be used in various ways such as the download attribute of an HTML link or as the attachment name in a content-disposition header.

Regional result CSV download links seem to be the odd one out: they return text/plain instead of a JSON wrapped URL. Switching them over to use JSON wrapped URLs like everything else would require an accompanying change on the UI side but would also enable the CSV downloads to use human readable names. The updated code to return the JSON wrapped URL with humanName is already present in RegionalAnalysisController.getCsvResults but commented out for the time being.

This PR also adds and endpoint at /api/regional/:_id/all that will generate all available GeoTIFF rasters for a given regional analysis and store them in a persistent ZIP file, returning one of these new-style JSON wrapped download links including a human-readable name. An attempt is also made to replace any UUIDs in the names of files inside this ZIP with relevant human-readable strings.

Many of the changes here are just small changes to return types, MIME types etc. There is one big block add/remove RegionalAnalysisController which is mostly factoring the action of extracting a single grid out of the HTTP controller, so it can be called once by the existing grid-retrieval HTTP controller, but multiple times in a loop by the new bulk-zip-creating HTTP controller. The smaller targeted changes there are somewhat camouflaged/exaggerated by the indentation and movement of that big block.

this is not particularly efficient but it's only used in local operation

This enables #673, but should be backward compatible with existing UI. JSON responses containing a URL have an accompanying humanName field. This can be used as the download attribute of an HTML link, or as the attachment name in a content-disposition header. CSV download still returns text/plain (requires JSON parsing on UI side)

ansoncfit

I tested a few things with the UI locally, ensuring that pre-existing download options for opportunity datasets and regional analyses continue to work.

I would like to consider including the last six characters of UUIDs in the human readable names, which would be a consistent way for users to distinguish between inputs when names are not unique. And protip: entering part of UUID in most of the selectors in the UI works for filtering.

ansoncfit · 2024-02-28T21:53:26Z

src/main/java/com/conveyal/file/UrlWithHumanName.java

+package com.conveyal.file;
+
+/**
+ * Combines a url for downloading a file, which might include a globally unique but human-annoying UUID, with a


globally unique but human-annoying

😅

ansoncfit · 2024-02-28T21:57:40Z

src/main/java/com/conveyal/file/FileStorageFormat.java

-        String extension = filename.substring(filename.lastIndexOf(".") + 1);
-        return FileStorageFormat.valueOf(extension.toUpperCase());
+        String extension = filename.substring(filename.lastIndexOf(".") + 1).toLowerCase(Locale.ROOT);
+        for (FileStorageFormat format : FileStorageFormat.values()) {
+            if (format.extension.equals(extension)) {
+                return format;
+            }
+        }
+        return null;


What motivated this change?

Previous version did not work correctly, but it was only used for retrieving pre-existing data in the local file server implementation.

ansoncfit · 2024-02-28T22:02:55Z