Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Linux ver python3.8 #25

Open
wants to merge 9 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
__pycache__/*
models/*
/tmp/*
/TTS/vits/*
*.log
7 changes: 4 additions & 3 deletions SocketServer.py
Original file line number Diff line number Diff line change
Expand Up @@ -62,13 +62,14 @@ def __init__(self, args):
self.addr = None
self.conn = None
logging.info('Initializing Server...')
self.host = socket.gethostbyname(socket.gethostname())
# self.host = socket.gethostbyname(socket.gethostname())
self.host = "0.0.0.0"# 所有地址上面监听
self.port = 38438
self.s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.s.setsockopt(socket.SOL_SOCKET, socket.SO_SNDBUF, 10240000)
self.s.bind((self.host, self.port))
self.tmp_recv_file = 'tmp/server_received.wav'
self.tmp_proc_file = 'tmp/server_processed.wav'
self.tmp_recv_file = '/tmp/server_received.wav'# 改为根目录
self.tmp_proc_file = '/tmp/server_processed.wav'

## hard coded character map
self.char_name = {
Expand Down
105 changes: 105 additions & 0 deletions environment.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
name: dlife
channels:
- defaults
dependencies:
- _libgcc_mutex=0.1=main
- _openmp_mutex=5.1=1_gnu
- ca-certificates=2023.01.10=h06a4308_0
- ld_impl_linux-64=2.38=h1181459_1
- libffi=3.4.4=h6a678d5_0
- libgcc-ng=11.2.0=h1234567_1
- libgomp=11.2.0=h1234567_1
- libstdcxx-ng=11.2.0=h1234567_1
- ncurses=6.4=h6a678d5_0
- openssl=1.1.1t=h7f8727e_0
- pip=23.0.1=py38h06a4308_0
- python=3.8.16=h7a1cb2a_3
- readline=8.2=h5eee18b_0
- setuptools=66.0.0=py38h06a4308_0
- sqlite=3.41.2=h5eee18b_0
- tk=8.6.12=h1ccaba5_0
- wheel=0.38.4=py38h06a4308_0
- xz=5.4.2=h5eee18b_0
- yaml=0.2.5=h7b6447c_0
- zlib=1.2.13=h5eee18b_0
- pip:
- aiohttp==3.8.4
- aiosignal==1.3.1
- anyio==3.6.2
- appdirs==1.4.4
- async-timeout==4.0.2
- async-tio==1.3.2
- attrs==23.1.0
- audioread==3.0.0
- certifi==2022.12.7
- cffi==1.15.1
- charset-normalizer==2.1.1
- cmake==3.25.0
- cn2an==0.5.19
- coloredlogs==15.0.1
- cython==0.29.34
- decorator==5.1.1
- filelock==3.9.0
- flatbuffers==23.5.9
- frozenlist==1.3.3
- fsspec==2023.5.0
- h11==0.14.0
- httpcore==0.17.0
- httpx==0.24.0
- huggingface-hub==0.14.1
- humanfriendly==10.0
- idna==3.4
- importlib-metadata==6.6.0
- jieba==0.42.1
- jinja2==3.1.2
- joblib==1.2.0
- lazy-loader==0.2
- librosa==0.10.0.post2
- lit==15.0.7
- llvmlite==0.40.0
- markupsafe==2.1.2
- mpmath==1.2.1
- msgpack==1.0.5
- multidict==6.0.4
- networkx==3.0
- numba==0.57.0
- numpy==1.24.1
- onnxruntime==1.14.1
- openai==0.27.6
- openaiauth==0.3.6
- packaging==23.1
- pillow==9.3.0
- pooch==1.6.0
- proces==0.1.4
- prompt-toolkit==3.0.38
- protobuf==4.23.0
- pycparser==2.21
- pypinyin==0.49.0
- pysocks==1.7.1
- pyyaml==6.0
- regex==2023.5.5
- requests==2.28.1
- revchatgpt==5.0.2
- scikit-learn==1.2.2
- scipy==1.10.1
- sniffio==1.3.0
- socksio==1.0.0
- soundfile==0.12.1
- soxr==0.3.5
- sympy==1.11.1
- threadpoolctl==3.1.0
- tiktoken==0.4.0
- tokenizers==0.13.3
- torch==2.0.1+cu118
- torchaudio==2.0.2+cu118
- torchvision==0.15.2+cu118
- tqdm==4.65.0
- transformers==4.29.1
- triton==2.0.0
- typeguard==2.13.3
- typing-extensions==4.4.0
- urllib3==1.26.13
- wcwidth==0.2.6
- yarl==1.9.2
- zipp==3.15.0
prefix: /home/sean/anaconda3/envs/dlife
5 changes: 3 additions & 2 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,10 @@ For other part of the project, please refer to:
## Getting stuffs ready to roll:
### Clone this repo
```bash
git clone https://github.com/zixiiu/Digital_Life_Server.git --recursive
git clone https://github.com/seanxpw/Digital_Life_Server.git -b linux_ver_python3.8 --recursive
```
### Install prerequisites
建议看[readme_detail.md](readme_detail.md)
1. install pytorch
```bash
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
Expand All @@ -24,7 +25,7 @@ git clone https://github.com/zixiiu/Digital_Life_Server.git --recursive
cd "TTS/vits/monotonic_align"
mkdir monotonic_align
python setup.py build_ext --inplace
cp monotonic_align/*.pyd .
cp monotonic_align/*.so .
```

4. Download models
Expand Down
65 changes: 29 additions & 36 deletions readme_detail.md
Original file line number Diff line number Diff line change
@@ -1,60 +1,46 @@
## 搭建”数字生命“服务:
> ⚠ 注意:
> 如果不知道你在干什么(纯小白),请在**需要存放该项目的位置**打开终端(Win11)或Powershell(win10),然后**按照下述说明逐步操作**即可
> 在进行以下操作前,请确保电脑中有Git和Python>=3.8
> 现在是linux(ubuntu)的配置。使用python3.8
### 克隆仓库
```bash
git clone https://github.com/zixiiu/Digital_Life_Server.git --recursive
git clone https://github.com/seanxpw/Digital_Life_Server.git -b linux_ver_python3.8 --recursive
cd Digital_Life_Server
```
### 保姆式配置环境
1. 使用virtualvenv建立python虚拟环境
### 配置环境
**需要显卡,且需要cuda11.8**
1. 使用conda建立python虚拟环境
```bash
python -m venv venv
conda env create -f environment.yaml
```
2. 安装pytorch于venv

> 你可以在终端(或Powershell)输入`nvcc --version`,找到输出中`Cuda compilation tools`一行来查看cuda版本

对于cuda11.8:

(默认地址,下载可能较慢)
这样会自动创建一个名字叫dlife的环境
如果已经有了dlife
```bash
.\venv\Scripts\python.exe -m pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
conda env update dlife -f environment.yaml
```
(国内加速地址,下载可能较快)
如果安装包的时候报错的话先运行下面的命令下载pytorch就好。
```bash
.\venv\Scripts\python.exe -m pip install torch==2.0.0+cu118 torchvision torchaudio -f https://mirror.sjtu.edu.cn/pytorch-wheels/torch_stable.html
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
```

对于没有Nvidia显卡的电脑:
requirements.txt是pip的,没有anaconda可以一试,不过yaml这个包好像是用conda装的
到时候自己注意点。

(默认地址,下载可能较慢)
```bash
.\venv\Scripts\python.exe -m pip install torch torchvision torchaudio
```
(国内加速地址,下载可能较快)
```bash
.\venv\Scripts\python.exe -m pip install torch==2.0.0+cpu torchvision torchaudio -f https://mirror.sjtu.edu.cn/pytorch-wheels/torch_stable.html
```
其余版本组合可以从[这个页面](https://pytorch.org/get-started/locally)获取具体的下载指令

3. 安装项目所需其它依赖项
```bash
.\venv\Scripts\python.exe -m pip install -r requirements_out_of_pytorch.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
```
4. Build `monotonic_align`
2. Build `monotonic_align`
```bash
cd "TTS/vits/monotonic_align"
mkdir monotonic_align
python setup.py build_ext --inplace
cp monotonic_align/*.pyd .
cp monotonic_align/*.so .
```

> 到这里,项目构建完毕🥰

5. 下载项目所需模型
[百度网盘](https://pan.baidu.com/s/1EnHDPADNdhDl71x_DHeElg?pwd=75gr)
3. 下载项目所需模型
[百度网盘](https://pan.baidu.com/s/1EnHDPADNdhDl71x_DHeElg?pwd=75gr)
视频简介下面也有别的网盘的链接。在里面找对应的目录就行。
注意TTS的一个模型名字应该是paimon6k_390k.pth
如果下载的是paimon6k_390000.pth请把名字修改为paimon6k_390k.pth

ASR Model:
to `/ASR/resources/models`
Sentiment Model:
Expand All @@ -67,4 +53,11 @@ to `/TTS/models`
> 启动前,不要忘记根据实际情况修改bat文件中的具体配置
```bash
run-gpt3.5-api.bat
```
```

实测python3.8会报一个版本检查的错误
```bash
dlife/lib/python3.8/site-packages/revChatGPT/__init__.py, line 23
```
把数字9改成8就行
Binary file modified requirements.txt
Binary file not shown.
16 changes: 7 additions & 9 deletions run-gpt3.5-api.bat
Original file line number Diff line number Diff line change
@@ -1,10 +1,8 @@
@echo off
set SCRIPT_NAME=SocketServer.py
set CHATVER=3
set PROXY=http://127.0.0.1:7890
set STREAM=False
set CHARACTER=paimon
set MODEL=gpt-3.5-turbo
SCRIPT_NAME=SocketServer.py
CHATVER=3
PROXY=http://127.0.0.1:7890
STREAM=False
CHARACTER=paimon
MODEL=gpt-3.5-turbo


.\venv\Scripts\python.exe %SCRIPT_NAME% --chatVer %CHATVER% --stream %STREAM% --character %CHARACTER% --model %MODEL%
python $SCRIPT_NAME --chatVer $CHATVER --stream $STREAM --character $CHARACTER --model $MODEL