Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

nexon.PakFile #198

Closed
1 of 2 tasks
Tracked by #194
snake-biscuits opened this issue Aug 30, 2024 · 2 comments
Closed
1 of 2 tasks
Tracked by #194

nexon.PakFile #198

snake-biscuits opened this issue Aug 30, 2024 · 2 comments
Assignees
Labels
bug Something isn't working MegaTest test every .bsp you can find need test We need to write a test for regression testing nexon.cso2 Counter-Strike: Online 2 branch script nexon.cso2_2018 Counter-Strike: Online 2 (post-2018) branch script
Milestone

Comments

@snake-biscuits
Copy link
Owner

snake-biscuits commented Aug 30, 2024

Taken from notes in #194

TODO list

  • tests.archives.nexon.test_PakFile
    • finish writing deflate test
  • tests.branches.nexon.cso2
    • test bsp.PAKFILE.as_bytes() on real .bsps

Initial Thoughts

nexon.PakFile.as_bytes() is really important to test, since it's required for NexonBsp.save_as()
We need some specific SpecialLumpClass tests for this
But afaik no public source tool for making these PakFiles exists

organnerx has some people making tools, idk how public they are

Which means the only real-world examples of the format we're going to find are inside maps
Tho we can test .from_bytes() like we do for pkware.Zip...

after this comment I made a empty test and sketched out a deflate (1 file) test

ArchiveClasses don't have to have .as_bytes() methods
Except for pkware.Zip & nexon.PakFile, since they both get used as SpecialLumpClasses

But we should keep links etc. for external tools used with each format in docs/
Something to add to the DB

After 7ba0efb

Testing nexon.PakFile via CS:O2 .bsps is failing in a really wierd way

>>> import bsp_tool
>>> bsp = bsp_tool.load_bsp("E:/Mod/CSO2/maps/gb_varena.bsp")  # map with least files in pakfile (8)
>>> bsp.headers["PAKFILE"]
<LumpHeader (offset: 587596, length: 134135, version: 0, compressed: 0, fourCC: 0)>
>>> len(bsp.PAKFILE.as_bytes())
134146  # 11 bytes over???

Really need to be testing this w/ a handmade deflate for .from_bytes()
And as a SpecialLumpClass, though that requires local test maps:

Or grabbing MegaTest files for tests.branches.nexon.cso2:

@snake-biscuits snake-biscuits added need test We need to write a test for regression testing MegaTest test every .bsp you can find labels Aug 30, 2024
@snake-biscuits snake-biscuits self-assigned this Aug 30, 2024
@snake-biscuits snake-biscuits added nexon.cso2_2018 Counter-Strike: Online 2 (post-2018) branch script nexon.cso2 Counter-Strike: Online 2 branch script bug Something isn't working labels Aug 30, 2024
@snake-biscuits
Copy link
Owner Author

Testing nexon.PakFile via CS:O2 .bsps is failing in a really wierd way

Not a bug, just failed to remember intended behaviour:
bsp_tool doesn't recompress Valve's LZMA compression when saving to bytes
nexon.PakFile uses Valve's LZMA compression on file data

The 11 byte difference is the compression savings:

>>> import bsp_tool
>>> bsp = bsp_tool.load_bsp("E:/Mod/CSO2/maps/gb_varena.bsp")
>>> {path: (lf.compressed_size, lf.uncompressed_size) for path, lf in bsp.PAKFILE.local_files.items()}
{'materials/maps/gb_varena/cubemapdefault.vtf': (196, 207),
 'materials/maps/gb_varena/c0_0_64.vtf': (0, 4401),
 'materials/maps/gb_varena/c-1536_0_64.vtf': (0, 4278),
 'materials/maps/gb_varena/c1536_0_64.vtf': (0, 4310),
 'materials/maps/gb_varena/cubemapdefault.hdr.vtf': (0, 181),
 'materials/maps/gb_varena/c0_0_64.hdr.vtf': (0, 40279),
 'materials/maps/gb_varena/c-1536_0_64.hdr.vtf': (0, 39699),
 'materials/maps/gb_varena/c1536_0_64.hdr.vtf': (0, 39702)}
>>> 207 - 196
11

nexon uses LZMA compression again inside cubemap .vtfs, in chunks
https://github.com/snake-biscuits/bsp_tool_examples/blob/master/nexon/cso2_cubemaps.py
which is likely why the larger files haven't been compressed again

@snake-biscuits
Copy link
Owner Author

This is why we need extensions.diff, to confirm data is a valid copy when bytes don't match
Flagging uncompressed data is a good idea though, so we know why it's different

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working MegaTest test every .bsp you can find need test We need to write a test for regression testing nexon.cso2 Counter-Strike: Online 2 branch script nexon.cso2_2018 Counter-Strike: Online 2 (post-2018) branch script
Projects
Status: Done
Status: Done
Development

No branches or pull requests

1 participant