From 88e9bb906888f7232d341295a6d3b7c522a73e91 Mon Sep 17 00:00:00 2001 From: erfanzar Date: Tue, 28 May 2024 16:10:22 +0330 Subject: [PATCH] fixing documentation errors --- docs/Install.rst | 24 +++++++-------- docs/contributing.rst | 31 +++++++++++++------- docs/data_processing.rst | 3 +- docs/index.rst | 53 +++++++++++++++++++++++++++------- docs/parameterquantization.rst | 16 +++++----- generate_documentations.py | 10 +++---- 6 files changed, 90 insertions(+), 47 deletions(-) diff --git a/docs/Install.rst b/docs/Install.rst index f0e113e40..c17417cf1 100644 --- a/docs/Install.rst +++ b/docs/Install.rst @@ -1,29 +1,29 @@ -# Installing EasyDeL - +Installing EasyDeL +====== EasyDeL uses FJFormer and JAX as main dependencies in order to run the scripts but there are some things that needs to be installed such as GO-lang to JAX specific platform installations, but you can simply install EasyDeL via pip: ``` pip install easydel ``` -## Installing Jax - +Installing Jax +------ JAX uses XLA to compile and run your NumPy programs on GPUs and TPUs. Compilation happens under the hood by default, with library calls getting just-in-time compiled and executed. But JAX also lets you just-in-time compile your own Python functions into XLA-optimized kernels using a one-function API, jit. you can install other version too but easydel required at least version of 0.4.16 -### TPU - +TPU +------ ```shell !pip install jax[tpu] -f https://storage.googleapis.com/jax-releases/libtpu_releases.html -q ``` -### GPU - - -#### CUDA-12 +GPU +------ +CUDA-12 +------ ```shell pip install --upgrade pip # CUDA 12 installation @@ -31,8 +31,8 @@ pip install --upgrade pip pip install --upgrade "jax[cuda12_pip]" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html ``` -#### CUDA-11 - +CUDA-11 +------ ```shell pip install --upgrade pip # CUDA 11 installation diff --git a/docs/contributing.rst b/docs/contributing.rst index d84c10b2a..14b9ba389 100644 --- a/docs/contributing.rst +++ b/docs/contributing.rst @@ -1,8 +1,10 @@ -# Contributing to EasyDeL - +Contributing to EasyDeL +========== Thank you for considering contributing to EasyDeL! We welcome your input. To ensure a smooth collaboration, please review and adhere to the following guidelines. -## How to Contribute + +How to Contribute +------ To contribute to EasyDeL, follow these steps: 1. Fork the repository. 2. Create a new branch for your feature or bug fix. @@ -10,25 +12,34 @@ To contribute to EasyDeL, follow these steps: 4. Push your changes to your branch in your forked repository. 5. Submit a pull request to the main EasyDeL repository, detailing the changes you've made and the problem it solves. -## Code of Conduct -Please adhere to the [Apache Code of Conduct](https://www.apache.org/foundation/policies/conduct.html) in all interactions related to EasyDeL. -## Reporting Bugs +Code of Conduct +------ +Please adhere to the Apache Code of Conduct_ in all interactions related to EasyDeL. + +Reporting Bugs +------ If you encounter a bug, please open an issue on the EasyDeL repository, providing a clear and detailed description of the issue, including steps to reproduce it. -## Suggesting Enhancements +Suggesting Enhancements +------ If you have ideas for enhancements, feel free to open an issue on the EasyDeL repository. Provide a clear and detailed description of your proposed enhancement. -## Development Setup +Development Setup +------ To set up EasyDeL for development, follow the instructions in the README.md file. -## Pull Request Guidelines +Pull Request Guidelines +------ When submitting a pull request, please ensure the following: - Your code follows the project's coding standards. - Your commits are accompanied by clear and descriptive messages. - Your pull request addresses a single issue or feature. -## License +License +------ By contributing to EasyDeL, you agree that your contributions will be licensed under the Apache License, Version 2.0. Thank you for your interest in contributing to EasyDeL! We appreciate your support. + +.. _Conduct https://www.apache.org/foundation/policies/conduct.html \ No newline at end of file diff --git a/docs/data_processing.rst b/docs/data_processing.rst index 2d99371a0..4200f1c04 100644 --- a/docs/data_processing.rst +++ b/docs/data_processing.rst @@ -1,4 +1,5 @@ -## Data Processing +Data Processing +====== here in this case you will see an example data required by EasyDeL to pre-train or fine-tune models diff --git a/docs/index.rst b/docs/index.rst index 88601c548..1afa9fc69 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -1,5 +1,5 @@ -## EasyDeL 🔮 - +EasyDeL 🔮 +========== EasyDeL is an open-source framework designed to enhance and streamline the training process of machine learning models. With a primary focus on Jax/Flax, EasyDeL aims to provide convenient and effective solutions for training Flax/Jax models on TPU/GPU for both serving and training purposes. @@ -43,7 +43,8 @@ models on TPU/GPU for both serving and training purposes. With its comprehensive set of features and tools, EasyDeL aims to streamline and accelerate the training and deployment of machine learning models, particularly in the domain of large language models and video-related applications. -## What Makes EasyDeL 🔮 Special +What Makes EasyDeL 🔮 Special +--------------------------------------------------------------- EasyDeL is built up on JAX and Flax and that's why EasyDeL can perform as fast and as easy as possible @@ -75,17 +76,17 @@ extensive libraries, and a wide range of pre-trained models, which can be advant Additionally, the choice of framework often depends on the specific requirements of the project and the familiarity of the team with a particular toolset. -### Hands on Code Kaggle Examples +Hands on Code Kaggle Examples +--------------------------------------------------------------- -1. [script](https://www.kaggle.com/citifer/easydel-causal-language-model-trainer-example) for mindset of using EasyDeL - CausalLanguageModelTrainer on kaggle, but you can do much more. -2. [script](https://www.kaggle.com/code/citifer/easydel-serve-example-mixtral) for using and serving LLMs with EasyDeL - JAXServer API (Mixtral Example). -3. [script](https://www.kaggle.com/code/citifer/easydel-sfttrainer-example) SuperVised Finetuning with EasyDeL. +1. for mindset of using EasyDeL CausalLanguageModelTrainer on kaggle, but you can do much more. CLMScript +2. SuperVised Finetuning with EasyDeL. SFTScript_ -## Citing EasyDeL 🥶 +Citing EasyDeL 🥶 +--------------------------------------------------------------- -#### To cite this Project +To cite this Project +--------------------------------------------------------------- ```misc @misc{Zare Chavoshi_2023, @@ -97,3 +98,33 @@ the team with a particular toolset. year={2023} } ``` + +.. toctree:: + :hidden: + :maxdepth: 1 + :caption: Getting Started + + install + contributing +.. toctree:: + :hidden: + :maxdepth: 1 + :caption: EasyDeL Magics + + easydelstate + +.. toctree:: + :hidden: + :maxdepth: 1 + :caption: Examples + + finetuning_example + lora_transferlearning_example + parameterquantization + attentionmodule_example + data_processing + + + +.. _SFTScript https://www.kaggle.com/code/citifer/easydel-sfttrainer-example +.. _CLMScript https://www.kaggle.com/citifer/easydel-causal-language-model-trainer-example \ No newline at end of file diff --git a/docs/parameterquantization.rst b/docs/parameterquantization.rst index 830722355..f64e433dd 100644 --- a/docs/parameterquantization.rst +++ b/docs/parameterquantization.rst @@ -1,5 +1,5 @@ -# What's 8-bit quantization? How does it help ? - +What's 8-bit quantization? How does it help ? +======= Quantization in the context of deep learning is the process of constraining the number of bits that represent the weights and biases of the model. @@ -8,8 +8,8 @@ Weights and Biases numbers that we need in backpropagation. In 8-bit quantization, each weight or bias is represented using only 8 bits as opposed to the typical 32 bits used in single-precision floating-point format (float32). -## Why does it use less GPU/TPU Memory? - +Why does it use less GPU/TPU Memory? +--------- The primary advantage of using 8-bit quantization is the reduction in model size and memory usage. Here's a simple explanation: @@ -35,8 +35,8 @@ To convert these to bytes (since memory is often measured in bytes): - 8-bit integer would use ( 8/8 = 1 ) bytes. - A 16-bit integer would use ( 16/8 = 2 ) bytes. -## Example of Using Parameters Quantization in EasyDeL - +Example of Using Parameters Quantization in EasyDeL +--------- in case of serving models or using them with `JAX` The Easiest and the best way you can find is EasyDeL (you can explore more if you want) you have 4 ways to use models @@ -47,8 +47,8 @@ is EasyDeL (you can explore more if you want) you have 4 ways to use models let assume we want to run a 7B model on only 12 GB of vram let just jump into codding -### Using Quantized Model via generate Function - +Using Quantized Model via generate Function +--------- let assume we want to run `Qwen/Qwen1.5-7B-Chat` ```python diff --git a/generate_documentations.py b/generate_documentations.py index ad9cd5ac1..a08ae3150 100644 --- a/generate_documentations.py +++ b/generate_documentations.py @@ -140,13 +140,13 @@ def main(): ("Home",): "index.rst", ("Install",): "Install.rst", ("Available models",): "AvailableModels.md", - ("Examples", "EasyState"): "EasyStateExample.md", + ("Examples", "EasyState"): "easydelstate.rst", ("Examples", "LoRA and Transfer Learning"): "LoRA-TransferLearningExample.rst", - ("Examples", "Fine Tuning Example"): "FineTuningExample.rst", - ("Examples", "DataProcessing"): "DataProcessing.rst", + ("Examples", "Fine Tuning Example"): "finetuning.example.rst", + ("Examples", "DataProcessing"): "data_processing.rst", ("Examples", "Easy Attention"): "AttentionModuleExample.md", - ("Examples", "Model Parameter Quantization"): "Parameter-Quantization.md", - ("Contributing",): "CONTRIBUTING.rst" + ("Examples", "Model Parameter Quantization"): "Parameter-Quantization.rst", + ("Contributing",): "contributing.rst" } cache = {("APIs",) + k: v for k, v in cache.items()}