-
Notifications
You must be signed in to change notification settings - Fork 30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add ipex extra in pyproject.toml to use restricted transformers version #127
Comments
As a reminder, since mpt-7b CI failed, set ipex configuration to False in #125 first after verification. |
cheehook
pushed a commit
to JoshuaL3000/llm-on-ray
that referenced
this issue
Mar 8, 2024
* refactoring Signed-off-by: Jiafu Zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * refactor argument passing and model configs by using yaml Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> * debugging gpt-j-6b Signed-off-by: jiafu zhang <jiafu.zhang@intel.com> --------- Signed-off-by: Jiafu Zhang <jiafu.zhang@intel.com> Signed-off-by: jiafu zhang <jiafu.zhang@intel.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
IPEX has restriction on transformers version, but llm-on-ray doesn't have. To verify IPEX and other llm-on-ray functions in parallel in CI, we can add a new ipex extra in pyproject.toml with right transformers version. Then, add corresponding nightly CI in github workflow.
The brief steps are,
cpu = [
"transformers>=4.35.0", # some models need higher version of transformers
"intel_extension_for_pytorch==2.1.0+cpu",
"torch==2.1.0+cpu",
"oneccl_bind_pt==2.1.0+cpu"
]
**+ipex = [
+]**
add a separate dockerfile so that it can be cached properly in CI build.
copy one of Dockerfiles under dev/docker and rename it to Dockerfile.ipex. After that, replace 'pip install ...' with 'pip install .[ipex]
add nightly CI
copy the workflow-inference.yaml and rename to workflow-inference-ipex.yaml and call workflow-inference-ipex.yaml to workflow_orders_nightly.yaml.
call-inference:
uses: ./.github/workflows/workflow_inference.yml
with:
ci_type: nightly
Inside workflow-inference-ipex.yaml, add inference tests for ipex supported models.
The text was updated successfully, but these errors were encountered: