问题描述
SoVITS训练终端报错
-- Process 0 terminated with the following error:
Traceback (most recent call last):
File "E:\TestProject\GPT-SoVITS-beta0706\runtime\lib\site-packages\torch\multiprocessing\spawn.py", line 69, in _wrap
fn(i, *args)
File "E:\TestProject\GPT-SoVITS-beta0706\GPT_SoVITS\s2_train.py", line 254, in run
train_and_evaluate(
File "E:\TestProject\GPT-SoVITS-beta0706\GPT_SoVITS\s2_train.py", line 352, in train_and_evaluate
y_hat_mel = mel_spectrogram_torch(
File "E:\TestProject\GPT-SoVITS-beta0706\GPT_SoVITS\module\mel_processing.py", line 135, in mel_spectrogram_torch
spec = torch.stft(
File "E:\TestProject\GPT-SoVITS-beta0706\runtime\lib\site-packages\torch\functional.py", line 641, in stft
return _VF.stft(input, n_fft, hop_length, win_length, window, # type: ignore[attr-defined]
RuntimeError: cuFFT doesn't support signals of half type with compute capability less than SM_53, but the device containing input half tensor only has SM_52
问题分析
这是由于半精度 cuFFT 需要 GPU 架构至少是 SM_53 以上，而我的GPU是TITAN X，只支持SM_52
问题诊断：
SoVITS的JSON配置中开启了对于半精度cuFFT的支持，但是你的GPU又不支持
方案尝试：
修改GPT-SoVITS-beta0706\GPT_SoVITS\configs\s2json将其中的"fp16_run": true,的true修改为false后并保存
方案检验：
终端能够正常跑动模型
INFO:mine:[2.5445058345794678, 4.0188374519348145, 9.532323837280273, 19.340923309326172, 0.0, 2.7108802795410156, 0, 9.99875e-05]
100%|██████████████████████████████████████████████████████████████████████████████████| 17/17 [00:41<00:00, 2.46s/it]
INFO:mine:====> Epoch: 1
100%|██████████████████████████████████████████████████████████████████████████████████| 17/17 [00:29<00:00, 1.71s/it]
INFO:mine:====> Epoch: 2
100%|██████████████████████████████████████████████████████████████████████████████████| 17/17 [00:28<00:00, 1.67s/it]
INFO:mine:====> Epoch: 3
100%|██████████████████████████████████████████████████████████████████████████████████| 17/17 [00:28<00:00, 1.66s/it]
INFO:mine:Saving model and optimizer state at iteration 4 to logs/mine/logs_s2\G_233333333333.pth
INFO:mine:Saving model and optimizer state at iteration 4 to logs/mine/logs_s2\D_233333333333.pth
INFO:mine:saving ckpt mine_e4:Success.
INFO:mine:====> Epoch: 4
100%|██████████████████████████████████████████████████████████████████████████████████| 17/17 [00:28<00:00, 1.69s/it]
INFO:mine:====> Epoch: 5
88%|████████████████████████████████████████████████████████████████████████▎ | 15/17 [00:25<00:03, 1.53s/it]INFO:mine:Train Epoch: 6 [88%]
INFO:mine:[2.229780435562134, 2.6936821937561035, 8.767420768737793, 14.989115715026855, 0.0, 0.5111721158027649, 100, 9.99250234335941e-05]
100%|██████████████████████████████████████████████████████████████████████████████████| 17/17 [00:29<00:00, 1.74s/it]
INFO:mine:====> Epoch: 6
100%|██████████████████████████████████████████████████████████████████████████████████| 17/17 [00:28<00:00, 1.70s/it]
INFO:mine:====> Epoch: 7
100%|██████████████████████████████████████████████████████████████████████████████████| 17/17 [00:29<00:00, 1.75s/it]
INFO:mine:Saving model and optimizer state at iteration 8 to logs/mine/logs_s2\G_233333333333.pth
INFO:mine:Saving model and optimizer state at iteration 8 to logs/mine/logs_s2\D_233333333333.pth
INFO:mine:saving ckpt mine_e8:Success.
INFO:mine:====> Epoch: 8