Eslevier: vtexXXXXX download

[ErnestaP]

Every record have for SCOAP3 has pfd and xml files.

• All fields in JSON are parsed from XML file. XML file we can find in .tar archive, for example, CERNQ000000010669A.tar
• To download one tar archive takes at least 15miin, example: workflows qa
• PDF files, for every record that was parsed from XML I mentioned above, can be found in .zip archive, for example: vtex00577479_a-2b.zip
• ZIP archive is so big that to download it takes ages, cannot even tell how long since it crashes on the end. We have extracted files saved, but not the archive. Maybe would be better to save at first the archive and then extract it, rather than read files from zip/tar and save it in the extracted dir? Might be actually faster
• Error while saving the file from ZIP:

[2023-10-30, 10:56:11 UTC] {connectionpool.py:471} WARNING - Failed to parse headers (url=https://s3.cern.ch:443/scoap3-qa-workflows-elsevier/extracted/vtex00564918_a-2b/vtex00564918_a-2b/05503213/v991sC/fp/issue_xml_xsl2881_fp.xml): [MissingHeaderBodySeparatorDefect()], unparsed data: 'HTTP/1.1 200 OK\r\nAccept-Ranges: bytes\r\nBucket: scoap3-qa-workflows-elsevier\r\nContent-Length: 0\r\nDate: Mon, 30 Oct 2023 10:56:11 GMT\r\nEtag: "2dcc3f604a9e3b2c93d9e72373fbe493"\r\nX-Amz-Request-Id: tx000009cd859128c20a345-00653f8bca-3cf7f37b-default\r\n\r\n'
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.10/site-packages/urllib3/connectionpool.py", line 469, in _make_request
    assert_header_parsing(httplib_response.msg)
  File "/home/airflow/.local/lib/python3.10/site-packages/urllib3/util/response.py", line 91, in assert_header_parsing
    raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data)
urllib3.exceptions.HeaderParsingError: [MissingHeaderBodySeparatorDefect()], unparsed data: 'HTTP/1.1 200 OK\r\nAccept-Ranges: bytes\r\nBucket: scoap3-qa-workflows-elsevier\r\nContent-Length: 0\r\nDate: Mon, 30 Oct 2023 10:56:11 GMT\r\nEtag: "2dcc3f604a9e3b2c93d9e72373fbe493"\r\nX-Amz-Request-Id: tx000009cd859128c20a345-00653f8bca-3cf7f37b-default\r\n\r\n'

• Comment about the error I found on git: MissingHeaderBodySeparatorDefect (again) minio/minio#11245 (comment)

HOW TO MAKE IT FASTER?
HOW TO HAVE THE FILES FROM .ZIP DOWNLOADED CORRECTLY?