Code

Feat ckpt#18

Code

Merged

samsja merged 20 commits intomainPrimeIntellect-ai/prime:mainfrom feat-ckptPrimeIntellect-ai/prime:feat-ckptCopy head branch name to clipboardSep 30, 2024

+554-16

Commits

Commits on Sep 29, 2024

• fix vocab size for debugmodel and real data

  samsja

  committed
  272e903

  Copy full SHA for 272e903

• add torchdata

  samsja

  committed
  5b01f13

  Copy full SHA for 5b01f13

• add ckpt v0

  samsja

  committed
  b01eb90

  Copy full SHA for b01eb90

• fix real dataset loading error

  samsja

  committed
  98e9565

  Copy full SHA for 98e9565

• ckpt save in the right step folder

  samsja

  committed
  986aa20

  Copy full SHA for 986aa20

• change total tokens diloco

  samsja

  committed
  7ccce1b

  Copy full SHA for 7ccce1b

• add tests

  samsja

  committed
  09d59f0

  Copy full SHA for 09d59f0

• refactor easier step

  samsja

  committed
  10b6215

  Copy full SHA for 10b6215

• use fsspec rsync for bbacking up the ckpt to remote

  samsja

  committed
  d7119ae

  Copy full SHA for d7119ae

• add async saving to remote

  samsja

  committed
  7622bbe

  Copy full SHA for 7622bbe

• remove unused file

  samsja

  committed
  aab5a5b

  Copy full SHA for aab5a5b

• fix rebase

  samsja

  committed
  6b63fe3

  Copy full SHA for 6b63fe3

• fix ckpt

  samsja

  committed
  00eaec7

  Copy full SHA for 00eaec7

• add diloco ckpt

  samsja

  committed
  fcbd102

  Copy full SHA for fcbd102

• save into dioco scpeific folder

  samsja

  committed
  3c052dd

  Copy full SHA for 3c052dd

Commits on Sep 30, 2024

• firemove process group

  samsja

  committed
  47c5396

  Copy full SHA for 47c5396

• firemove process group

  samsja

  committed
  e080641

  Copy full SHA for e080641

• add diloco rank

  samsja

  committed
  1c098b2

  Copy full SHA for 1c098b2

• fix ckpt issue

  samsja

  committed
  2c54c2c

  Copy full SHA for 2c54c2c

• remove ckpt tests

  samsja

  committed
  b9e6eef

  Copy full SHA for b9e6eef