You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The goal here is for SDG to be able to generate data once that's usable with any student model. Today, the pretraining samples we generate have to be generated in a format that matches the chat prompt of the intended student model used for training. Instead, we want to generate data with SDG once and have training do whatever post-processing it needs to our pretrain samples to adapt them to the student model being trained.
Update data masking for agnostic pretraining data
adding @Maxusmusti and @RobotSail for more details around the requirements from the training
The text was updated successfully, but these errors were encountered: