Skip to content
This repository has been archived by the owner on Nov 1, 2024. It is now read-only.

Efficient kernel implementation for IColumn.fill_null/drop_null #64

Open
wenleix opened this issue Nov 13, 2021 · 0 comments
Open

Efficient kernel implementation for IColumn.fill_null/drop_null #64

wenleix opened this issue Nov 13, 2021 · 0 comments
Labels
native kernel

Comments

@wenleix
Copy link
Contributor

wenleix commented Nov 13, 2021 •
edited
Loading

For single colulmn, delegating to Arrow Array seems to be a good initial support.

Arrow array supports fill_null/drop_null.

So we can first call to_arrow, then calls fill_null/drop_null in Arrow array, and then converts back (from_arrow is working in progress)

This looks like a decent initial implementation.
@wenleix wenleix changed the title Delegate IColumn.fill_null/drop_null to Arrow Delegate IColumn.fill_null/drop_null to Arrow Array Nov 13, 2021
@wenleix wenleix changed the title Delegate IColumn.fill_null/drop_null to Arrow Array Efficient kernel implementation for IColumn.fill_null/drop_null Nov 25, 2021
@wenleix wenleix mentioned this issue Nov 25, 2021
Open
This was referenced Jan 18, 2022
Closed
Closed
YLGH pushed a commit to YLGH/torcharrow that referenced this issue May 7, 2022
@xing-liu @facebook-github-bot
enable FSDP (pytorch#64)
6ef3827 
Summary:
Pull Request resolved: pytorch/torchrec#64

Pull Request resolved: pytorch/torchrec#15

As title.

Reviewed By: dstaay-fb, bigrabithong, liangluofb

Differential Revision: D33712372

fbshipit-source-id: bbcd7b9adb0580f30afc765dde72df2755b09ed7
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
native kernel
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@wenleix