A graphical system program that allows you to quickly create your own AI VTuber for free.
https://www.youtube.com/watch?v=Hwss_p2Iroc
https://docs.google.com/document/d/16DU-DJKMaC-15K6iShLd9ioXc8VqjTLqgMPswsjPjF0/edit?usp=sharing
Python >= 3.8, install the main dependencies:
pip3 install -r requirements.txt
For the specific PyTorch packages which require a special handling, use the following command:
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
Latest GPU Driver https://www.nvidia.com.tw/Download/index.aspx?lang=tw
CUDA Toolkit 12.1.1 https://developer.nvidia.com/cuda-12-1-1-download-archive
cuDNN https://developer.nvidia.com/cudnn-downloads
Excerpt from https://github.com/openai/whisper
There are six model sizes, four with English-only versions, offering speed and accuracy tradeoffs. Below are the names of the available models and their approximate memory requirements and inference speed relative to the large model. The relative speeds below are measured by transcribing English speech on a A100, and the real-world speed may vary significantly depending on many factors including the language, the speaking speed, and the available hardware.
| Size | Parameters | English-only model | Multilingual model | Required VRAM | Relative speed |
|---|---|---|---|---|---|
| tiny | 39 M | tiny.en |
tiny |
~1 GB | ~10x |
| base | 74 M | base.en |
base |
~1 GB | ~7x |
| small | 244 M | small.en |
small |
~2 GB | ~4x |
| medium | 769 M | medium.en |
medium |
~5 GB | ~2x |
| large | 1550 M | N/A | large |
~10 GB | 1x |
| turbo | 809 M | N/A | turbo |
~6 GB | ~8x |
The .en models for English-only applications tend to perform better, especially for the tiny.en and base.en models. We observed that the difference becomes less significant for the small.en and medium.en models.
Additionally, the turbo model is an optimized version of large-v3 that offers faster transcription speed with a minimal degradation in accuracy.
Whisper's performance varies widely depending on the language. The figure below shows a performance breakdown of large-v3 and large-v2 models by language, using WERs (word error rates) or CER (character error rates, shown in Italic) evaluated on the Common Voice 15 and Fleurs datasets. Additional WER/CER metrics corresponding to the other models and datasets can be found in Appendix D.1, D.2, and D.4 of the paper, as well as the BLEU (Bilingual Evaluation Understudy) scores for translation in Appendix D.3.
