Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Directory prefix Argo GDAC file system #424

Closed
wants to merge 48 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
d09c59c
[skip-ci]
gmaze Sep 3, 2024
a7cab73
Add S3 support to gdac data fetcher
gmaze Sep 13, 2024
215acf5
misc
gmaze Sep 13, 2024
2c723f1
Merge branch 'other-major-breaking-refactoring' into 302-new-data-sou…
gmaze Sep 13, 2024
55b3723
Update test_fetchers_data_gdac.py
gmaze Sep 13, 2024
8ebd896
misc
gmaze Sep 18, 2024
6747989
Merge branch 'master' into 302-new-data-source-for-gdac-from-amazon-s3
gmaze Sep 18, 2024
c812a9d
Squashed commit of the following:
gmaze Sep 25, 2024
40a92a6
clear tests data
gmaze Sep 25, 2024
2b992c3
Merge branch 'master' into 302-new-data-source-for-gdac-from-amazon-s3
gmaze Sep 25, 2024
3afafea
Create kerchunker.py
gmaze Oct 23, 2024
6b6faf9
Merge branch 'argo-kerchunk' into 302-new-data-source-for-gdac-from-a…
gmaze Oct 23, 2024
2d77293
misc
gmaze Oct 23, 2024
bfb0482
Merge branch 'master' into 302-new-data-source-for-gdac-from-amazon-s3
gmaze Oct 23, 2024
0a4b29c
Update options.py
gmaze Oct 23, 2024
040e565
Merge branch 'master' into 302-new-data-source-for-gdac-from-amazon-s3
gmaze Oct 23, 2024
0f01c3c
misc
gmaze Oct 23, 2024
c6f4186
Update kerchunker.py
gmaze Oct 24, 2024
a3a5d2d
More docstrings / docs
gmaze Oct 24, 2024
436ccc7
Update gdac_data.py
gmaze Oct 24, 2024
22411b9
Update lists.py
gmaze Oct 24, 2024
fb5e550
Update kerchunker.py
gmaze Oct 24, 2024
04aa0ed
Clear test data
gmaze Oct 24, 2024
caf0bbb
misc
gmaze Oct 24, 2024
c7220fa
misc
gmaze Oct 24, 2024
dda1eed
misc
gmaze Oct 24, 2024
6a5a532
Add support lazy http open_dataset
gmaze Oct 24, 2024
1b2ca31
Merge branch 'master' into 302-new-data-source-for-gdac-from-amazon-s3
gmaze Oct 25, 2024
dca8524
fix bug in open_dataset
gmaze Oct 25, 2024
23f5f52
Update filesystems.py
gmaze Oct 25, 2024
5afa8cd
Fix 3.10 env for CI tests
gmaze Oct 25, 2024
4186506
Improve perf and docstrings
gmaze Nov 6, 2024
de5c149
Merge branch 'master' into 302-new-data-source-for-gdac-from-amazon-s3
gmaze Dec 12, 2024
8f23323
Update check_gdac_option and check_gdac_path to accomodate s3
gmaze Dec 13, 2024
cef13d0
Make sure s3store is instanciated with the anon argument set
gmaze Dec 13, 2024
fef2e61
misc
gmaze Dec 13, 2024
f66138e
New gdacfs
gmaze Dec 16, 2024
5e0e49f
check_gdac_path/check_gdac_option to use gdacfs
gmaze Dec 16, 2024
c187e34
Update test_fetchers_data_gdac.py
gmaze Dec 16, 2024
3b2e820
Merge branch 'master' into 302-new-data-source-for-gdac-from-amazon-s3
gmaze Dec 16, 2024
4301a5e
Update gdac_index.py
gmaze Dec 17, 2024
9a9a970
New gdacfs and ArgoKerchuncker
gmaze Dec 17, 2024
9056665
Improve docstrings, doc and blackify
gmaze Dec 17, 2024
d8f48a4
New utility to list official GDAC servers
gmaze Dec 17, 2024
2240c6a
Update test_fetchers_data_gdac.py
gmaze Dec 18, 2024
578fb5b
Improve log, doc
gmaze Dec 19, 2024
68e90fb
Update filesystems.py
gmaze Dec 19, 2024
259959c
Update filesystems.py
gmaze Dec 19, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Merge branch 'master' into 302-new-data-source-for-gdac-from-amazon-s3
gmaze authored Dec 16, 2024
commit 3b2e82068ae2545554171d2c7ce3168344cf4968
4 changes: 4 additions & 0 deletions docs/whats-new.rst
Original file line number Diff line number Diff line change
@@ -14,6 +14,10 @@ Coming up next
Internals
^^^^^^^^^

- Fix bug raised when the Argo reference table 8 return by the NVS server has a missing altLabel. ID of platform types are now extracted from the NVS url ID property. :issue:`420`, (:pr:`421`) by |gmaze|.

- When argopy is sending a http request to a data server, add a custom http header ``Argopy-Version`` to ease server log analysis. (:pr:`407`) by |gmaze|.

- New :meth:`ArgoIndex.copy` method (:pr:`418`) by |gmaze|. This copy allows for a:

- deep copy, i.e. a new instance with same parameters (e.g. ``index_file``) and cleared search,
You are viewing a condensed version of this merge commit. You can view the full changes here.