training resources

[itlittlekou]

Hello, I would like to ask can the full training of fingpt be done on colab with A100? How long will it take.

[YangletLiu]

Hello, I would like to ask can the full training of fingpt be done on colab with A100? How long will it take.

Are you talking about "pretrain the full model", like an LLM with 7B parameters?? It is infeasible on colab. You can do fine-tuning on colab.

[itlittlekou]

Thank you so much for taking the time to answer my questions! My previous question was not accurate enough. If I want to use my own dataset to fine-tune the Fingpt-v1 version, how big a dataset do I need to use (hundreds of thousands?), and how long does it take to rent A100 training on colab to achieve a  better result?In fact, I want to know whether the fine-tuning of Fingpt-v1 on its own dataset can be completed with limited resources, that is, renting A100 on cloab, and the time it takes. Thanks again for your answer in your busy schedule 阿南 ***@***.***  

…

------------------ 原始邮件 ------------------ 发件人: "AI4Finance-Foundation/FinGPT" ***@***.***>; 发送时间: 2023年8月17日(星期四) 凌晨1:33 ***@***.***>; ***@***.******@***.***>; 主题: Re: [AI4Finance-Foundation/FinGPT] training resources (Issue #62) Hello, I would like to ask can the full training of fingpt be done on colab with A100? How long will it take. Are you talking about "pretrain the full model", like an LLM with 7B parameters?? It is infeasible on colab. You can do fine-tuning on colab. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

[oliverwang15]

Hi, itlittlekou. The training set of the FinGPT v3 series has around 70k samples, So if you want to have desirable results, maybe you should have more than 10k samples I suppose? However, you might not necessarily need to have so many samples if it's not available to you. You can do something like training more batches or use LLMs like ChatGPT to generate more samples for you.

And about the training time, if you are using LoRA and fp16, from our experience, the training time would be less than 3 hours if the epoch is set to 2 and the sample is around 70k. However, if you use different samples or different epochs the training time might also be different.