[Bug]: nb_inventory failed '/api/dcim/devices/'

[wols]

Ansible NetBox Collection version

v3.20.0

Ansible version

ansible [core 2.14.16]
  config file = /home/USER/path/to/ansible.cfg
  configured module search path = ['/home/USER/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  ansible collection location = /home/USER/path/to
  executable location = /usr/bin/ansible
  python version = 3.11.2 (main, Aug 26 2024, 07:20:54) [GCC 12.2.0] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True

NetBox version

v4.1.0

Python version

3.11

Steps to Reproduce

# path/to/inventory/01_netbox.yml
---
plugin: netbox.netbox.nb_inventory

api_endpoint: http://A.B.C.D/netbox
token: 1234567890123456789012345678901234567890
validate_certs: false
config_context: false

Expected Behavior

Debian Bookworm 12.7 up to date:

The inventory works witch netbox v4.0.11 and ansible_modules v3.19.1 (despite the message Collection netbox.netbox does not support Ansible version 2.14.16).

After update to netbox v4.1.0 the error as below and ansible_module v3.20.0 brought no change.

Observed Behavior

$ ansible-inventory -i inventory/01_netbox.yml --list -vvv
ansible-inventory [core 2.14.16]
  config file = /home/USER/path/to/ansible.cfg
  configured module search path = ['/home/USER/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3/dist-packages/ansible
  ansible collection location = /home/USER/path/to
  executable location = /usr/bin/ansible-inventory
  python version = 3.11.2 (main, Aug 26 2024, 07:20:54) [GCC 12.2.0] (/usr/bin/python3)
  jinja version = 3.1.2
  libyaml = True
Using /home/USER/path/to/ansible.cfg as config file
host_list declined parsing /home/USER/path/to/inventory/01_netbox.yml as it did not pass its verify_file() method
script declined parsing /home/USER/path/to/inventory/01_netbox.yml as it did not pass its verify_file() method
[WARNING]: Collection netbox.netbox does not support Ansible version 2.14.16
Using inventory plugin 'ansible_collections.netbox.netbox.plugins.inventory.nb_inventory' to process inventory source '/home/USER/path/to/inventory/01_netbox.yml'
Fetching: http://A.B.C.D/netbox/api/status
toml declined parsing /home/USER/path/to/inventory/01_netbox.yml as it did not pass its verify_file() method
[WARNING]:  * Failed to parse /home/USER/path/to/inventory/01_netbox.yml with auto plugin: '/api/dcim/devices/'
  File "/usr/lib/python3/dist-packages/ansible/inventory/manager.py", line 293, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
  File "/usr/lib/python3/dist-packages/ansible/plugins/inventory/auto.py", line 59, in parse
    plugin.parse(inventory, loader, path, cache=cache)
  File "/home/USER/path/to/ansible_collections/netbox/netbox/plugins/inventory/nb_inventory.py", line 2179, in parse
    self.main()
  File "/home/USER/path/to/ansible_collections/netbox/netbox/plugins/inventory/nb_inventory.py", line 2018, in main
    self.fetch_api_docs()
  File "/home/USER/path/to/ansible_collections/netbox/netbox/plugins/inventory/nb_inventory.py", line 1627, in fetch_api_docs
    for p in openapi["paths"]["/api/dcim/devices/"]["get"]["parameters"]
             ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
[WARNING]:  * Failed to parse [...] AnsibleParserError('Plugin configuration YAML file, not YAML inventory')
[WARNING]:  * Failed to parse [...] AnsibleParserError(e)
[WARNING]: Unable to parse /home/wUSER/path/to/inventory/01_netbox.yml as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
{
    "_meta": {
        "hostvars": {}
    },
    "all": {
        "children": [
            "ungrouped"
        ]
    }
}

[rodvand]

What version of pynetbox do you have?

[wols]

# apt info python3-pynetbox
Package: python3-pynetbox
Version: 7.0.0-1
APT-Sources: http://ftp.de.debian.org/debian bookworm/main amd64 Packages

[rodvand]

You should upgrade to the latest pynetbox version.

[wols]

Debian wishlist item created: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1082671

[felixphew]

I'm experiencing what I think is the same error, despite having pynetbox 7.4.1 installed:

% ansible-inventory -i inventory.yml --graph -vvv          
ansible-inventory [core 2.16.12]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/home/me/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.13/site-packages/ansible
  ansible collection location = /home/me/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible-inventory
  python version = 3.13.0 (main, Oct  8 2024, 00:00:00) [GCC 14.2.1 20240912 (Red Hat 14.2.1-3)] (/usr/bin/python3)
  jinja version = 3.1.4
  libyaml = True
Using /etc/ansible/ansible.cfg as config file
host_list declined parsing /home/me/Documents/ansible-test/inventory.yml as it did not pass its verify_file() method
script declined parsing /home/me/Documents/ansible-test/inventory.yml as it did not pass its verify_file() method
Using inventory plugin 'ansible_collections.netbox.netbox.plugins.inventory.nb_inventory' to process inventory source '/home/me/Documents/ansible-test/inventory.yml'
Fetching: https://my.netbox.server/netbox/api/status
toml declined parsing /home/me/Documents/ansible-test/inventory.yml as it did not pass its verify_file() method
[WARNING]:  * Failed to parse /home/me/Documents/ansible-test/inventory.yml with auto plugin: '/api/dcim/devices/'
  File "/usr/lib/python3.13/site-packages/ansible/inventory/manager.py", line 293, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/ansible/plugins/inventory/auto.py", line 59, in parse
    plugin.parse(inventory, loader, path, cache=cache)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/me/.ansible/collections/ansible_collections/netbox/netbox/plugins/inventory/nb_inventory.py", line 2179, in parse
    self.main()
    ~~~~~~~~~^^
  File "/home/me/.ansible/collections/ansible_collections/netbox/netbox/plugins/inventory/nb_inventory.py", line 2018, in main
    self.fetch_api_docs()
    ~~~~~~~~~~~~~~~~~~~^^
  File "/home/me/.ansible/collections/ansible_collections/netbox/netbox/plugins/inventory/nb_inventory.py", line 1627, in fetch_api_docs
    for p in openapi["paths"]["/api/dcim/devices/"]["get"]["parameters"]
             ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
[WARNING]:  * Failed to parse /home/me/Documents/ansible-test/inventory.yml with yaml plugin: Plugin configuration YAML file, not
YAML inventory
  File "/usr/lib/python3.13/site-packages/ansible/inventory/manager.py", line 293, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/ansible/plugins/inventory/yaml.py", line 114, in parse
    raise AnsibleParserError('Plugin configuration YAML file, not YAML inventory')
[WARNING]:  * Failed to parse /home/me/Documents/ansible-test/inventory.yml with ini plugin: Invalid host pattern 'plugin:'
supplied, ending in ':' is not allowed, this character is reserved to provide a port.
  File "/usr/lib/python3.13/site-packages/ansible/inventory/manager.py", line 293, in parse_source
    plugin.parse(self._inventory, self._loader, source, cache=cache)
    ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/site-packages/ansible/plugins/inventory/ini.py", line 138, in parse
    raise AnsibleParserError(e)
[WARNING]: Unable to parse /home/me/Documents/ansible-test/inventory.yml as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
@all:
  |--@ungrouped:

And I think I know why.

Like @wols, our netbox instance isn't located at the root of the web server, but in a subdirectory:

api_endpoint: https://my.netbox.server/netbox

As a result, the endpoint paths returned in the API schema are prefixed with that directory:

{
    "openapi": "3.0.3",
    "info": {
        "title": "NetBox REST API",
        "version": "3.7.4 (3.7)",
        "license": {
            "name": "Apache v2 License"
        }
    },
    "paths": {
        "/netbox/api/circuits/circuit-terminations/": {
...

So, when the module goes to look for "/api/dcim/devices/", it won't find it.

Does this sound possible?

[felixphew]

Given the OpenAPI description also tells us the path on the server, we can use that to strip it out:

diff --git a/plugins/inventory/nb_inventory.py b/plugins/inventory/nb_inventory.py
index b06d0fa..baf539c 100644
--- a/plugins/inventory/nb_inventory.py
+++ b/plugins/inventory/nb_inventory.py
@@ -1627,4 +1627,7 @@ class InventoryModule(BaseInventoryPlugin, Constructable, Cacheable):
                 pass
 
+        prefix = "^/" + re.escape(openapi["servers"][0]["url"])
+        openapi["paths"] = {re.sub(prefix, "/", k): v for k, v in openapi["paths"].items()}
+
         self.api_version = version.parse(netbox_api_version)
 

this works on my setup, but it's a bit of a hack. what do you think?