Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using system generated seed in RandomSampler #1441

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

ramanishsingh
Copy link
Contributor

Currently we are fixing the seed for generator in RandomSampler as 1.
This leads to the generator not changing even when torch.manual_seed() seed is changed.

For the RandomSampler in torch.utils.data.sampler, they use seed = int(torch.empty((), dtype=torch.int64).random_().item()). Using the same here.

Fixes #1440

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Feb 6, 2025
@ramanishsingh ramanishsingh marked this pull request as draft February 6, 2025 00:13
Copy link

pytorch-bot bot commented Feb 6, 2025

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/data/1441

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 14fa418 with merge base fe6b405 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@ramanishsingh ramanishsingh marked this pull request as ready for review February 6, 2025 01:02
seed = 1
torch.manual_seed(seed)
dl3 = StatefulDataLoader(self.dataset, batch_size=1, shuffle=True)
data_dl3 = []
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we call this results3 ? and similarly results1 and results2 above.


seed = 1
torch.manual_seed(seed)
dl3 = StatefulDataLoader(self.dataset, batch_size=1, shuffle=True)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can rename dl3 to dataloader3. ditto for other dataloader variables.

)

def test_seed_replicability(self):

seed = 0
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of checking for specific seeds 0 and 1, we can generalize it to two randomly generated seeds.

And also add a assert to ensure both seeds are not equal.

Copy link
Contributor

@divyanshk divyanshk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

data_source: Sized,
replacement: bool = False,
num_samples: Optional[int] = None,
generator=None,
):
if generator is None:
# Ensure that underlying sampler has something repeatable
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's remove or update this comment

Copy link
Contributor

@andrewkho andrewkho left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update/remove comment and then gogogo

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
4 participants