ZIP loader triggers an exception when loading Velociraptor Linux collections (Ubuntu)

[Zawadidone]

Note this issue only occurs with a specific Linux collection that I made when developing #698.

Executing target-query on a Velociraptor collection of a Linux system triggers the following exception:

target-query -t Collection-Linux.zip -f hostname
Traceback (most recent call last):
  File "/dissect.target/.tox/py3/lib/python3.10/site-packages/dissect/target/target.py", line 414, in _load
    ldr.map(target)
  File "/dissect.target/.tox/py3/lib/python3.10/site-packages/dissect/target/loaders/velociraptor.py", line 130, in map
    map_dirs(target, dirs, os_type)
  File "/dissect.target/.tox/py3/lib/python3.10/site-packages/dissect/target/loaders/dir.py", line 62, in map_dirs
    dfs = ZipFilesystem(path.root.fp, path.at, alt_separator=alt_separator, case_sensitive=case_sensitive)
  File "/dissect.target/.tox/py3/lib/python3.10/site-packages/dissect/target/filesystems/zip.py", line 60, in __init__
    self._fs.map_file_entry(rel_name, file_entry)
  File "/dissect.target/.tox/py3/lib/python3.10/site-packages/dissect/target/filesystem.py", line 1231, in map_file_entry
    directory = self.makedirs(sub_dirs)
  File "/dissect.target/.tox/py3/lib/python3.10/site-packages/dissect/target/filesystem.py", line 1158, in makedirs
    if part not in directory:
TypeError: argument of type 'ZipFilesystemEntry' is not iterable

The above exception was the direct cause of the following exception:

What stands out is that the variable directory in case of th exception has the type ZipFilesystemEntry, which is weird because in all other cases it has the type dissect.target.filesystem.VirtualDirectory.

[Zawadidone]

Adding the following try/except to

dissect.target/dissect/target/filesystems/zip.py

Line 60 in a6796c4

self._fs.map_file_entry(rel_name, file_entry)

, 'fixes' the issue.

try:
    self._fs.map_file_entry(rel_name, file_entry)
except Exception:
    continue

[Zawadidone]

The Velociraptor collector was made with the following settings:

velociraptor config generate  > server.config.yaml
velociraptor --config server.config.yaml artifacts collect Server.Internal.ToolDependencies
velociraptor --config server.config.yaml artifacts collect Server.Utils.CreateCollector \
	--args OS=Linux \
	--args artifacts='["Generic.Collectors.File"]' \
	--args parameters='{"Generic.Collectors.File":{"Root":"/","collectionSpec":"Glob\netc/**\nusr/local/etc/**\nvar/log/**\nvar/spool/at/**\nvar/spool/cron/**\nvar/spool/anacron/**\nvar/lib/dpkg/status/**\nvar/audit/**\nvar/cron/**\nroot/.bash*\nroot/.zsh*\nroot/.ssh/**\nroot/.config/**\nhome/*/.bash*\nhome/*/.zsh*\nhome/*/.ssh/**\nhome/*/.config/**\nhome/*/.lastlogin\nboot/config*\nboot/efi*\nboot/grub*\nboot/init*\nvar/db/**\n"}}' \
	--args opt_filename_template="Collection-Linux-%FQDN%-%TIMESTAMP%" \
	--output linux.zip \
	--args target=ZIP \
	--args opt_prompt=N \
	--args opt_admin=Y \
	--args opt_level=0 \
	--args opt_timeout=86400 \
	--args opt_format=jsonl

[Schamper]

Maybe a path is attempted to be mapped on an already existing path (that was not detected as a directory). E.g.:

path/to/file
path/to/file/oops

path/to/file would be mapped as a ZipFilesystemEntry, which would cause this error when trying to map path/to/file/oops.

Maybe because this happens on a Linux collection, the path/to/file is actually a symlink to a directory? Zip doesn't really support symlinks AFAIK, so I'm not sure how Velociraptor deals with this. Is it possible to share an example zip file?

[Zawadidone]

Yes I have shared the file with you.

[Schamper]

Thanks, it's indeed what I expected:

checkdir error:  uploads/auto/etc/xdg/systemd/user exists but is not directory
                 unable to process uploads/auto/etc/xdg/systemd/user/sockets.target.wants/dirmngr.socket.
checkdir error:  uploads/auto/etc/xdg/systemd/user exists but is not directory
                 unable to process uploads/auto/etc/xdg/systemd/user/sockets.target.wants/gpg-agent-browser.socket.
checkdir error:  uploads/auto/etc/xdg/systemd/user exists but is not directory
                 unable to process uploads/auto/etc/xdg/systemd/user/sockets.target.wants/gpg-agent.socket.
checkdir error:  uploads/auto/etc/xdg/systemd/user exists but is not directory
                 unable to process uploads/auto/etc/xdg/systemd/user/sockets.target.wants/pk-debconf-helper.socket.

[user@localhost:/]$ ls -lah etc/xdg/systemd/
total 4.0K
drwxr-xr-x. 1 root root   8 Jan 28 17:32 .
drwxr-xr-x. 1 root root 178 Jan 28 17:33 ..
lrwxrwxrwx. 1 root root  18 Jan 22 01:00 user -> ../../systemd/user

For the time being, I think a try/except + log is indeed the correct "fix", but it should be a bit more explicit. I.e. there could be a check in makedirs that raises NotADirectoryError, which the ZipFilesystem could catch.

[Zawadidone]

How does Acquire deal with symlinks?

[Schamper]

They are stored as symlinks in the tar archive.