RFC: Add a mechanism for passing authorization headers to child links

[john-dupuy]

I recently discovered this project and it's awesome! I am dealing with a STAC catalog that requires authentication and I noticed that there is no mechanism for me to pass an auth header for when I want to walk a STAC catalog.

As a simple scenario consider the following:

from pystac import Catalog
catalog =  Catalog.from_dict(requests.get(<stac-url-requiring-auth>, headers={"Authorization": "api-key <my-api-key>"}))
catalog.validate_all()

In this situation, we will get an exception at the first link with an exception:

urllib.error.HTTPError: HTTP Error 401: Unauthorized

coming from this line:

pystac/stac_io.py", line 292, in read_text_from_href
    with urlopen(href) as f:

It would be great if we could add an auth header when we create the STAC object. As a workaround I created a class like

class StacIOWithAuth(DefaultStacIO):
    """Override 'read_text_from_href' so that we can add authn (api-key) to the requests that pystac makes."""

    def __init__(self, api_key):
        self.api_key = api_key

    def read_text_from_href(self, href: str) -> str:
        """Reads file as a UTF-8 string.

        If ``href`` has a "scheme" (e.g. if it starts with "https://") then this will
        use :func:`urllib.request.urlopen` to open the file and read the contents;
        otherwise, :func:`open` will be used to open a local file.

        Args:

            href : The URI of the file to open.
        """
        parsed = safe_urlparse(href)
        href_contents: str
        if parsed.scheme != "":
            try:
                req = Request(href, headers={"Authorization": f"api-key {self.api_key}"})
                with urlopen(req) as f:
                    href_contents = f.read().decode("utf-8")
            except HTTPError as e:
                raise Exception("Could not read uri {}".format(href)) from e
        else:
            with open(href, encoding="utf-8") as f:
                href_contents = f.read()
        return href_contents

And then just set catalog._stac_io = StacIOWithAuth(<api_key>).

Thanks for reading - I'm happy to take this on as a first issue if no one has bandwidth to tackle it.

[gadomski]

Your approach makes sense and is the currently-supported way of doing custom IO. Note that there is a StacIO.set_default method, which you can use to enable your custom StacIO without touching the "private" _stac_io attribute.

With respect to adding a class to PySTAC that handles this authorization automatically, it's not an unreasonable request. However, we (currently) try to keep PySTAC extermely low-dependency, with only one required dependency (python-dateutil) and two optional dependencies (jsonschema and orjson). An authorization-enabled StacIO that doesn't add any extra dependencies could be useful and would be a good PR. One that uses requests would probably not be included in the current version of PySTAC. However, there are discussions (e.g. #846) around adding more functionality that uses external libraries, and so there is a possibility of adding a requests-dependent StacIO in a future (probably v2) PySTAC.

[john-dupuy]

Thanks for the reply @gadomski - good to know about that set_default method.

An authorization-enabled StacIO that doesn't add any extra dependencies could be useful and would be a good PR

I can look into doing something like this. Perhaps simply allowing users to pass in their own headers would be sufficient.

[gadomski]

Closed by #889.