ChatGPT 开源实现

• Dolly

• ChatGLM-6B

• OpenChatKit

• Alpaca-LoRA

• GPT4All

• BELLE: Be Everyone’s Large Language model Engine

• Stanford Alpaca: An Instruction-following LLaMA Model

• 骆驼(Luotuo): Chinese-alpaca-lora

基础环境准备

1. 安装显卡驱动

# 查看系统版本
lsb_release -a

# 查看可用驱动版本
ubuntu-drivers devices

# 安装推荐驱动
sudo apt install nvidia-driver-440
# 重启系统
sudo reboot
# 重启后查看驱动
nvidia-smi

# 验证驱动是否安装
cat /proc/driver/nvidia/version

2. 安装 CUDA Toolkit

   1. 选择合适的 CUDA Toolkit 版本, 下载地址: CUDA Toolkit

   2. 运行安装

   # Ubuntu 18.04 系统
   # cuda tookit 11.8 下载地址: https://developer.download.nvidia.com/compute/cuda/11.8.0/local_installers/cuda_11.8.0_520.61.05_linux.run
   # cuda tookit 12.1 下载地址: https://developer.download.nvidia.com/compute/cuda/12.1.0/local_installers/cuda_12.1.0_530.30.02_linux.run
   wget download_url
   sudo 755 download_file
   sudo sh download_file --tookit --silent --override
   

   1. 如提示 cudnn 动态库找不到, 可参考 NVIDIA cuDNN Documentation 进行安装

3. 加速下载模型

   注意

   下载模型

   • wget

   • git lfs + clone

   git lfs install
   git clone huggingface_model_repo
   

   • huggface_hub

   from huggingface_hub import snapshot_download
   
   # bert-base-chinese 为 hugging_face  repo id
   snapshot_download(repo_id="bert-base-chinese")
   

4. 基础大模型

• Meta LLaMA Introducing LLaMA: A foundational, 65-billion-parameter large language model

  • 论文 LLaMA: Open and Efficient Foundation Language Models

  • 模型下载 LLaMA

• EleutherAI’s Pythia Pythia: Interpreting Autoregressive Transformers Across Time and Scale

  • 模型下载 Pythia Model

ChatGPT 开源实现

• ChatGLM-6B
• Dolly
• Stanford Alpaca: An Instruction-following LLaMA Model
• Alpaca-LoRA
• OpenChatKit
• GPT4All
• BELLE: Be Everyone’s Large Language model Engine