Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add copy and copytree methods #108

Closed
jayqi opened this issue Nov 15, 2020 · 7 comments · Fixed by #125 or #142
Closed

Add copy and copytree methods #108

jayqi opened this issue Nov 15, 2020 · 7 comments · Fixed by #125 or #142
Labels
enhancement New feature or request good first issue Good for newcomers

Comments

@jayqi
Copy link
Member

jayqi commented Nov 15, 2020

We should have copy and copytree methods for copying from one cloud path to another, similar to how shutil's functions work.

@jayqi jayqi added enhancement New feature or request good first issue Good for newcomers labels Nov 15, 2020
@genziano
Copy link
Contributor

genziano commented Feb 8, 2021

Hello, I was building a library for the same purpose, and I discovered this very nice one. I would like to contribute; is it ok if I pick this one up?

@jayqi
Copy link
Member Author

jayqi commented Feb 8, 2021

Yes! Thank you for your interest and the offer to contribute! Let us know if you have any questions.

@pjbull
Copy link
Member

pjbull commented Feb 8, 2021

That @genziano, that would be awesome. Just as a heads up, I think we may need two code paths:

(1) that uses the _move_file method on the client, which does a copy without transiting through the machine using cloudpath lib. (e.g., here is the implementaiton for S3). This is only guaranteed to work if the client on the CloudPath is the same client (since then the creds can access both buckets).

(2) that can work across cloud providers. This will likely need to do a download then upload and transit through the machine running cloudpathlib code.


Also, there are few notes in the issue about contributing that may be useful while we haven't done full contributor instructions yet:
#25 (comment)

@genziano
Copy link
Contributor

Great, I will work on it! And thanks for the tips.

I have one initial question: to which object should the methods copy and copytree belong? Or should they be functions in a submodule that replicates the shutil API?

@pjbull
Copy link
Member

pjbull commented Feb 10, 2021

IMO on CloudPath makes the most sense. Usage would look like. @jayqi did you have other thoughts?

p1 = CloudPath("s3://b/file1.txt")

# copies the file from p1 to the passed path
p2 = p1.copy(CloudPath("s3://b/file2.txt"))

# str destination passes through to instantiation using the CloudPath factory on p1
p2 = p1.copy("s3://b/file2.txt")

# dir destination copies and keeps filename
p2 = p1.copy(CloudPath("s3://b/folder/"))

# copy on a dir raises error to use copytree if src is not a file
CloudPath("s3://b/folder/").copy("s3://b/folder2")

# copies the entire directory from folder1 to folder2
CloudPath("s3://b/folder/").copytree("s3://b/folder2")

@machielg
Copy link

would be nice to have this feature, is there a plan for releasing?

@pjbull
Copy link
Member

pjbull commented May 12, 2021

This is being worked on in PR #142 now. @BenAsaf had mentioned he may be able to take a look at implementing in the near future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request good first issue Good for newcomers
Projects
None yet
4 participants