Problem installing xTuring

[heyili]

Hey guys,
I'm just starting using Xturing and have some issues installing it.
Like in the instruction I first run:
pip install xturing

Then tried with the example:

from xturing.datasets import InstructionDataset
dataset = InstructionDataset("./alpaca_data")

I am getting this error:

`Traceback (most recent call last):
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1146, in _get_module
    return importlib.import_module("." + module_name, self.__name__)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/x/miniconda3/envs/project/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/transformers/models/llama/modeling_llama.py", line 31, in <module>
    from ...modeling_utils import PreTrainedModel
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/transformers/modeling_utils.py", line 37, in <module>
    from .deepspeed import deepspeed_config, is_deepspeed_zero3_enabled
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/transformers/deepspeed.py", line 38, in <module>
    from accelerate.utils.deepspeed import HfDeepSpeedConfig as DeepSpeedConfig
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/accelerate/__init__.py", line 3, in <module>
    from .accelerator import Accelerator
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/accelerate/accelerator.py", line 35, in <module>
    from .checkpointing import load_accelerator_state, load_custom_state, save_accelerator_state, save_custom_state
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/accelerate/checkpointing.py", line 24, in <module>
    from .utils import (
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/accelerate/utils/__init__.py", line 136, in <module>
    from .launch import (
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/accelerate/utils/launch.py", line 33, in <module>
    from ..utils.other import is_port_in_use, merge_dicts
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/accelerate/utils/other.py", line 32, in <module>
    from deepspeed import DeepSpeedEngine
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/deepspeed/__init__.py", line 21, in <module>
    from . import ops
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/deepspeed/ops/__init__.py", line 6, in <module>
    from . import adam
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/deepspeed/ops/adam/__init__.py", line 6, in <module>
    from .cpu_adam import DeepSpeedCPUAdam
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/deepspeed/ops/adam/cpu_adam.py", line 8, in <module>
    from deepspeed.utils import logger
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/deepspeed/utils/__init__.py", line 10, in <module>
    from .groups import *
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/deepspeed/utils/groups.py", line 28, in <module>
    from deepspeed import comm as dist
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/deepspeed/comm/__init__.py", line 7, in <module>
    from .comm import *
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/deepspeed/comm/comm.py", line 34, in <module>
    from deepspeed.utils import timer, get_caller_func
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/deepspeed/utils/timer.py", line 31, in <module>
    class CudaEventTimer(object):
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/deepspeed/utils/timer.py", line 33, in CudaEventTimer
    def __init__(self, start_event: get_accelerator().Event, end_event: get_accelerator().Event):
                                    ^^^^^^^^^^^^^^^^^
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/deepspeed/accelerator/real_accelerator.py", line 155, in get_accelerator
    ds_accelerator = MPS_Accelerator()
                     ^^^^^^^^^^^^^^^^^
TypeError: Can't instantiate abstract class MPS_Accelerator with abstract method supported_dtypes

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/x/Desktop/project/example.py", line 1, in <module>
    from xturing.datasets import InstructionDataset
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/xturing/__init__.py", line 6, in <module>
    from .engines import (
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/xturing/engines/__init__.py", line 2, in <module>
    from xturing.engines.bloom_engine import (
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/xturing/engines/bloom_engine.py", line 4, in <module>
    from xturing.engines.causal import CausalEngine, CausalLoraEngine
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/xturing/engines/causal.py", line 15, in <module>
    from xturing.engines.lora_engine import (
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/xturing/engines/lora_engine/__init__.py", line 1, in <module>
    from xturing.engines.lora_engine.lora import (
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/xturing/engines/lora_engine/lora.py", line 30, in <module>
    from transformers import LlamaConfig, LlamaForCausalLM
  File "<frozen importlib._bootstrap>", line 1229, in _handle_fromlist
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1137, in __getattr__
    value = getattr(module, name)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1136, in __getattr__
    module = self._get_module(self._class_to_module[name])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/x/miniconda3/envs/project/lib/python3.11/site-packages/transformers/utils/import_utils.py", line 1148, in _get_module
    raise RuntimeError(
RuntimeError: Failed to import transformers.models.llama.modeling_llama because of the following error (look up to see its traceback):
Can't instantiate abstract class MPS_Accelerator with abstract method supported_dtypes
`

Does someone have similar issues ?

Best regards

[StochasticRomanAgeev]

Hi @heyili,
Just update xturing version with pip install xturing -U

[OpenSourceIronman]

I'm on a 100% fresh install of Ubuntu 22.04.3 LTS on Desktop.

I also followed the instruction and ran:
pip install xturing

However I got the following error:
/usr/lib/python3/dist-packages/requests/__init__.py:87: RequestsDependencyWarning: urllib3 (2.0.4) or chardet (4.0.0) doesn't match a supported version! warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported " [2023-09-06 14:54:47,895] [INFO] [real_accelerator.py:110:get_accelerator] Setting ds_accelerator to cuda (auto detect) Traceback (most recent call last): File "/home/saturn/MFC_GPT.py", line 1, in <module> from xturing.datasets import InstructionDataset File "/home/saturn/.local/lib/python3.10/site-packages/xturing/__init__.py", line 6, in <module> from .engines import ( File "/home/saturn/.local/lib/python3.10/site-packages/xturing/engines/__init__.py", line 2, in <module> from xturing.engines.bloom_engine import ( File "/home/saturn/.local/lib/python3.10/site-packages/xturing/engines/bloom_engine.py", line 4, in <module> from xturing.engines.causal import CausalEngine, CausalLoraEngine File "/home/saturn/.local/lib/python3.10/site-packages/xturing/engines/causal.py", line 15, in <module> from xturing.engines.lora_engine import ( File "/home/saturn/.local/lib/python3.10/site-packages/xturing/engines/lora_engine/__init__.py", line 1, in <module> from xturing.engines.lora_engine.lora import ( File "/home/saturn/.local/lib/python3.10/site-packages/xturing/engines/lora_engine/lora.py", line 45, in <module> import bitsandbytes as bnb File "/home/saturn/.local/lib/python3.10/site-packages/bitsandbytes/__init__.py", line 6, in <module> from . import cuda_setup, utils, research File "/home/saturn/.local/lib/python3.10/site-packages/bitsandbytes/research/__init__.py", line 1, in <module> from . import nn File "/home/saturn/.local/lib/python3.10/site-packages/bitsandbytes/research/nn/__init__.py", line 1, in <module> from .modules import LinearFP8Mixed, LinearFP8Global File "/home/saturn/.local/lib/python3.10/site-packages/bitsandbytes/research/nn/modules.py", line 8, in <module> from bitsandbytes.optim import GlobalOptimManager File "/home/saturn/.local/lib/python3.10/site-packages/bitsandbytes/optim/__init__.py", line 8, in <module> from .adagrad import Adagrad, Adagrad8bit, Adagrad32bit File "/home/saturn/.local/lib/python3.10/site-packages/bitsandbytes/optim/adagrad.py", line 5, in <module> from bitsandbytes.optim.optimizer import Optimizer1State File "/home/saturn/.local/lib/python3.10/site-packages/bitsandbytes/optim/optimizer.py", line 12, in <module> import bitsandbytes.functional as F File "/home/saturn/.local/lib/python3.10/site-packages/bitsandbytes/functional.py", line 12, in <module> from scipy.stats import norm ModuleNotFoundError: No module named 'scipy

and so I also ran the following command (which "pip install xturning" should probably handle).
pip install scipy

[StochasticRomanAgeev]

Hi @OpenSourceIronman,
Thanks for information, we will check why it is not installed, can you please share what version of xturing and bitsandbytes you have (because as I see error is not from xturing, but from inside of bitsandbytes)?
Also code snippet you are using will be very helpful, because in case of custom optimisers, it may require additional libraries.

[OpenSourceIronman]

Running xturing --version and pip show bitsandbytes shows that I'm running xturing version 0.1.8 and bitsandbytes version 0.41.1.

The code snippet I'm using is just the getting started example at https://github.com/stochasticai/xTuring/tree/main

# Make the necessary imports
from xturing.datasets import InstructionDataset
from xturing.models import BaseModel

# Load the desired dataset
dataset = InstructionDataset('../llama/alpaca_data')

# Load the desired model
model = BaseModel.create('gpt2')

# Run the Evaluation of the model on the dataset
result = model.evaluate(dataset)

# Print the result
print(f"Perplexity of the evalution: {result}")

[StochasticRomanAgeev]

Testing on different machines does not reproduce this issue for our library.
If issue with bitsandbytes still reproduces you should report it to bitsandbytes .
Thanks for using out library, ask if any additional help required.