Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Latpull docs, demo, browserify, countTokens, tokenStats #30

Open
wants to merge 41 commits into
base: master
Choose a base branch
from
Open

Latpull docs, demo, browserify, countTokens, tokenStats #30

Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
41 commits
Select commit Hold shift + click to select a range
473daf7
Add count tokens function
syonfox Dec 20, 2022
de619c0
add algo doc
syonfox Dec 20, 2022
14a7cfa
Fix bug with BPE cache (#1)
NickHeiner Dec 23, 2022
5f30e05
Add count tokens function (#2)
NickHeiner Dec 23, 2022
df30a04
Add fork notice
Dec 23, 2022
b779e40
Bump version
Dec 23, 2022
4054d73
Update index.d.ts
syonfox Dec 25, 2022
04cf547
Update index.js
syonfox Dec 25, 2022
ea8540d
Merge branch 'master' into syonfox-dev
Dec 25, 2022
3b496fb
Merge remote-tracking branch 'origin/patch-1' into syonfox-dev
Dec 25, 2022
bfefccc
Merge pull request #1 from NickHeiner/master
syonfox Dec 25, 2022
63260ae
Merge remote-tracking branch 'origin/patch-1' into syonfox-dev
Dec 25, 2022
65a3181
Add tokenStats some tests, and some error checks, also docs and readm…
Dec 25, 2022
d69b93e
add docs to repo for everyone else
Dec 25, 2022
4cdd31c
update readme and docs with links to pages
Dec 25, 2022
5d87dd5
update docs and bump
Dec 25, 2022
81be7f0
update docs and bump
Dec 25, 2022
d2b6dca
add a test for issue https://github.com/latitudegames/GPT-3-Encoder/i…
Dec 25, 2022
6354a3c
clean up test a bit
Dec 25, 2022
1831e5b
fix #15 in another way by removing textEncoder dependency
Dec 25, 2022
66a189f
fix #15 bump semi major version to 1.3.0 for possibly breaking change…
Jan 7, 2023
c0cd40a
update readme
Jan 7, 2023
c303e8b
bump n docs
Jan 7, 2023
3f2c339
Som more docs and also refactor so that we can use it in the browser …
Jan 8, 2023
36361c0
bump semi major version for browser support and loading change
Jan 8, 2023
a691838
added browser suport with browserify and an example
Jan 8, 2023
133a442
bump
Jan 8, 2023
5a13d99
readme and fix npm rc
Jan 8, 2023
fcbef4f
update docs ok lets test it noe
Jan 8, 2023
f324c53
hope this pushes demo to github pages
Jan 8, 2023
bc815f0
hope this pushes demo to github pages
Jan 8, 2023
fc36ade
hope this pushes demo to github pages
Jan 8, 2023
d37ff39
Some fixes to all of this hope its stable now for 1.4 release
Jan 8, 2023
c3c2e25
Some fixes to all of this hope its stable now for 1.4 release
Jan 8, 2023
70c4b30
revert npm for semi major release 1.2.0 latitudegames ... note you wi…
Jan 18, 2023
6715083
Merge branch 'master' into latpull
syonfox Jan 18, 2023
02eaadd
https://github.com/latitudegames/GPT-3-Encoder/pull/30#issuecomment-1…
hashfox Mar 8, 2023
c5a3b63
Merge branch 'master' into latpull
hashfox Mar 8, 2023
763c398
Fix token stats to run, and bump npm
hashfox Mar 8, 2023
b40302a
Rebuild docs
hashfox Mar 8, 2023
04420b0
note
syonfox Mar 8, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .github/workflows/node.js.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ jobs:

strategy:
matrix:
node-version: [12.x, 14.x]
node-version: [12.x, 14.x, 16.x, 18.x]
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

test all the versions


steps:
- uses: actions/checkout@v2
Expand All @@ -26,5 +26,6 @@ jobs:
node-version: ${{ matrix.node-version }}
- run: npm ci
- run: npm run build --if-present
- run: npm run docs --if-present
- run: npm test

3 changes: 3 additions & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1,4 @@
node_modules
.idea
.env

Loading