1 faster-whisper介紹
faster-whisper是基于OpenAI的Whisper模型的高效實現(xiàn)，它利用CTranslate2，一個專為Transformer模型設(shè)計的快速推理引擎。這種實現(xiàn)不僅提高了語音識別的速度，還優(yōu)化了內(nèi)存使用效率。faster-whisper的核心優(yōu)勢在于其能夠在保持原有模型準確度的同時，大幅提升處理速度，這使得它在處理大規(guī)模語音數(shù)據(jù)時更加高效。
項目地址：https://github.com/SYSTRAN/faster-whisper
1.1 whisper
OpenAI 的開源模型 whisper，可以執(zhí)行 99 種語言的語音識別和文字轉(zhuǎn)寫。但是 whisper 模型占用計算資源多，命令行使用門檻高。whisper 模型本身還存在一些問題，例如 模型幻聽問題（大部分的類似于不斷重復(fù)同一句話、無語音部分復(fù)讀莫名內(nèi)容等都是由于這個原因造成的）。要更好使用 whsiper 模型就需要能夠準確調(diào)試模型參數(shù)。但 whisper 模型參數(shù)眾多，且命令行使用對使用者有一定要求，而且只有 torch 版可以做到調(diào)整參數(shù)。使用 VAD 類工具也需要一定的動手能力。
1.2 faster-whisper
faster-whisper是具有完全的 whsiper 模型參數(shù)，且自帶 VAD加持的 whisper 版本，該版本使用了 CTranslate2 來重新實現(xiàn) whsiper 模型，CT2 對 transformer 類網(wǎng)絡(luò)進行了優(yōu)化，使模型推理效率更高。 相比于 openai/whisper，該實現(xiàn)在相同準確性下速度提高了 4 倍以上，同時使用的內(nèi)存更少。
所謂 VAD 即 Voice Activity Detection —— 聲音活動檢測，在語音信號處理中，例如語音增強，語音識別等領(lǐng)域有著非常重要的作用。它的作用是從一段語音（純凈或帶噪）信號中標(biāo)識出語音片段與非語音片段。在語音轉(zhuǎn)寫任務(wù)中，可以提前將語音和非語音部分分離出來，從而提升 whisper 網(wǎng)絡(luò)識別速度，并減少模型幻聽。
VAD地址：https://github.com/snakers4/silero-vad
1.3 性能對比
在性能方面，faster-whisper展現(xiàn)了顯著的優(yōu)勢。例如，在使用Large-v2模型和GPU進行13分鐘音頻的轉(zhuǎn)錄測試中，faster-whisper僅需54秒，而原始Whisper模型需要4分30秒。這一顯著的性能提升，意味著在實際應(yīng)用中，faster-whisper能夠更快地處理大量數(shù)據(jù)，特別是在需要實時或近實時語音識別的場景中。
1.4 技術(shù)優(yōu)勢
faster-whisper的技術(shù)優(yōu)勢不僅體現(xiàn)在速度上。它還支持8位量化，這一技術(shù)可以在不犧牲太多準確度的情況下，進一步減少模型在CPU和GPU上的內(nèi)存占用。這使得faster-whisper在資源受限的環(huán)境中也能高效運行，如在移動設(shè)備或嵌入式系統(tǒng)上。
aster-whisper適用于多種場景，特別是那些需要快速、準確的語音識別的應(yīng)用。例如，在客戶服務(wù)中，它可以用于實時語音轉(zhuǎn)文字，提高響應(yīng)速度和服務(wù)質(zhì)量。在醫(yī)療領(lǐng)域，faster-whisper可以輔助醫(yī)生快速轉(zhuǎn)錄病歷，提高工作效率。此外，它還適用于實時會議記錄、多語言翻譯、教育輔助等多個領(lǐng)域。
2 faster-whisper安裝及使用
2.1 conda環(huán)境下安裝
pip install faster-whisper2.2 模型下載
large-v3模型：https://huggingface.co/Systran/faster-whisper-large-v3/tree/main
large-v2模型：https://huggingface.co/guillaumekln/faster-whisper-large-v2/tree/main
large-v2模型：https://huggingface.co/guillaumekln/faster-whisper-large-v1/tree/main
medium模型：https://huggingface.co/guillaumekln/faster-whisper-medium/tree/main
small模型：https://huggingface.co/guillaumekln/faster-whisper-small/tree/main
base模型：https://huggingface.co/guillaumekln/faster-whisper-base/tree/main
tiny模型：https://huggingface.co/guillaumekln/faster-whisper-tiny/tree/main國內(nèi)模型地址：
https://aifasthub.com/models/guillaumekln2.3 使用fater-whisper進行中文語音識別
from faster_whisper import WhisperModel
model_size = "large-v3"
# Run on GPU with FP16
model = WhisperModel('../model/faster-whisper-large-v3', device="cuda", compute_type="float16")
# or run on GPU with INT8
# model = WhisperModel(model_size, device="cuda", compute_type="int8_float16")
# or run on CPU with INT8
# model = WhisperModel(model_size, device="cpu", compute_type="int8")
segments, info = model.transcribe("../data/gusuiyizhi.wav", beam_size=5)
print("Detected language '%s' with probability %f" % (info.language, info.language_probability))
for segment in segments:
print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))參數(shù)說明：
- local_files_only=True 表示加載本地模型
- model_size_or_path=path 指定加載模型路徑
- device="cuda" 指定使用cuda
- compute_type="int8_float16" 量化為8位
- language="zh" 指定音頻語言
- vad_filter=True 開啟vad
- vad_parameters=dict(min_silence_duration_ms=1000) 設(shè)置vad參數(shù)
?
運行結(jié)果顯示：文章來源：http://www.zghlxwxcb.cn/news/detail-787368.html
Detected language 'zh' with probability 0.998535
[0.00s -> 2.48s] 骨髓抑制的定義、危害和預(yù)防
[2.48s -> 4.48s] 骨髓抑制是一種病癥
[4.48s -> 8.14s] 其主要特點是骨髓中血細胞生成減少
[8.14s -> 11.62s] 這種減少會影響所有類型的血細胞
[11.62s -> 14.74s] 包括紅細胞、白細胞和血小板
[14.74s -> 15.74s] 通常
[15.74s -> 20.16s] 骨髓抑制是癌癥治療中使用的化療或放療的副作用
[20.16s -> 23.42s] 因為這些療法在殺傷癌細胞的同時
[23.42s -> 26.22s] 也會損害骨髓中的健康細胞
[26.22s -> 29.26s] 骨髓抑制可能會帶來如下危害
[29.26s -> 30.86s] 一、貧血
[30.86s -> 33.74s] 骨髓抑制導(dǎo)致紅細胞生成減少
[33.74s -> 35.64s] 可能會引起貧血
[35.64s -> 39.94s] 表現(xiàn)為疲勞、虛弱、呼吸急促和面色蒼白
[39.94s -> 44.58s] 嚴重的貧血可能導(dǎo)致心臟問題和日?；顒幽芰ο陆?[44.58s -> 46.92s] 二、中性粒細胞減少
[46.92s -> 49.52s] 骨髓抑制導(dǎo)致白細胞減少
[49.52s -> 51.96s] 尤其是中性粒細胞減少
[51.96s -> 55.94s] 患者更容易受到細菌、病毒和真菌感染
[56.22s -> 58.48s] 某些情況下可能會很嚴重
[58.48s -> 59.24s] 并危及生長
[59.24s -> 61.68s] 三、血小板減少
[61.68s -> 64.60s] 骨髓抑制導(dǎo)致血小板生成減少
[64.60s -> 67.02s] 增加出血和淤傷的風(fēng)險
[67.02s -> 71.10s] 患者可能會流鼻血、牙齦出血、容易淤傷
[71.10s -> 74.98s] 嚴重的血小板減少癥可導(dǎo)致自發(fā)性出血
[74.98s -> 78.18s] 小傷口出血時間延長和內(nèi)出血
[78.18s -> 79.72s] 這可能危及生命
[79.72s -> 81.58s] 四、治療延誤
[81.58s -> 84.06s] 如果患者的血細胞技術(shù)太低
[84.06s -> 87.24s] 則可能需要延遲化療周期或放療
[87.24s -> 88.82s] 以使骨髓功能恢復(fù)
[88.82s -> 89.22s] 三、血小板減少
[89.24s -> 90.46s] 如果延遲太久
[90.46s -> 92.38s] 可能會延誤癌癥的治療
[92.38s -> 94.38s] 造成危險的后果
[94.38s -> 96.24s] 五、劑量減少
[96.24s -> 100.06s] 骨髓抑制可能需要減少化療或放療的劑量
[100.06s -> 103.56s] 這可能會影響癌癥治療的整體有效性
[103.56s -> 105.88s] 六、延長住院時間
[105.88s -> 109.96s] 嚴重的感染或出血可能需要住院治療和監(jiān)測
[109.96s -> 114.16s] 增加了患者癌癥治療的總體時間和經(jīng)濟成本
[114.16s -> 116.44s] 七、降低生活質(zhì)量
[116.44s -> 119.22s] 疲勞、感染風(fēng)險增加和出血病毒
[119.24s -> 123.72s] 病發(fā)癥會對患者的整體健康和日常生活功能產(chǎn)生負面影響
[123.72s -> 127.84s] 預(yù)防腫瘤放化療引起的骨髓抑制雖然并不容易
[127.84s -> 132.42s] 但有一些方法可以幫助減少風(fēng)險或減輕嚴重程度
[132.42s -> 134.02s] 一、精確劑量
[134.02s -> 137.20s] 醫(yī)生會根據(jù)您的身體狀況和腫瘤大小
[137.20s -> 139.68s] 來選擇最佳的化療和放療劑量
[139.68s -> 141.70s] 以減少對骨髓的影響
[141.70s -> 143.60s] 二、藥物調(diào)整
[143.60s -> 144.48s] 如果可能
[144.48s -> 148.06s] 醫(yī)生會選擇具有較低骨髓抑制風(fēng)險的藥物
[148.18s -> 149.02s] 或調(diào)整
[149.24s -> 150.72s] 治療方案來降低風(fēng)險
[150.72s -> 152.60s] 三、治療間隔
[152.60s -> 156.16s] 醫(yī)生可能會增加治療之間的間隔時間
[156.16s -> 159.04s] 讓您的身體有更多時間恢復(fù)
[159.04s -> 160.98s] 四、生長因子
[160.98s -> 163.32s] 醫(yī)生可能會開出生長因子
[163.32s -> 165.36s] 如粒細胞刺激因子
[165.36s -> 167.88s] GCSF或紅細胞生成素
[167.88s -> 168.70s] EPO
[168.70s -> 172.48s] 以幫助身體加速產(chǎn)生白細胞和紅細胞
[172.48s -> 174.36s] 五、營養(yǎng)支持
[174.36s -> 176.30s] 保持良好的營養(yǎng)狀態(tài)
[176.30s -> 179.14s] 可以幫助身體應(yīng)對治療帶來的副作用
[179.14s -> 179.22s] 三、醫(yī)生可能會增加治療之間的間隔時間
[179.24s -> 181.38s] 注意攝入充足的蛋白質(zhì)
[181.38s -> 183.06s] 維生素和礦物質(zhì)
[183.06s -> 185.08s] 以支持骨髓功能
[185.08s -> 187.20s] 六、免疫增強劑
[187.20s -> 188.82s] 在某些情況下
[188.82s -> 191.98s] 醫(yī)生可能會推薦使用免疫增強劑
[191.98s -> 194.66s] 以提高您的免疫系統(tǒng)抵抗力
[194.66s -> 196.42s] 需要您注意的是
[196.42s -> 199.10s] 以上建議可能因個人情況而異
[199.10s -> 202.44s] 請務(wù)必與您的醫(yī)生保持緊密溝通
[202.44s -> 206.34s] 制定最適合您的個性化的防治策略
[206.34s -> 209.14s] 以應(yīng)對放化療可能引起的骨髓抑制
[209.14s -> 209.44s] 請務(wù)必與您的醫(yī)生保持緊密溝通2.4 使用fater-whisper將中文翻譯成英文輸出
from faster_whisper import WhisperModel
model_size = "large-v3"
# Run on GPU with FP16
model = WhisperModel('../model/faster-whisper-large-v3', device="cuda", compute_type="float16")
# or run on GPU with INT8
# model = WhisperModel(model_size, device="cuda", compute_type="int8_float16")
# or run on CPU with INT8
# model = WhisperModel(model_size, device="cpu", compute_type="int8")
segments, info = model.transcribe("../data/gusuiyizhi.wav", beam_size=5, language='en')
print("Detected language '%s' with probability %f" % (info.language, info.language_probability))
for segment in segments:
print("[%.2fs -> %.2fs] %s" % (segment.start, segment.end, segment.text))運行結(jié)果顯示：文章來源地址http://www.zghlxwxcb.cn/news/detail-787368.html
Detected language 'en' with probability 1.000000
[0.00s -> 2.46s] The definition of bone marrow disease, harm and prevention
[2.46s -> 4.32s] Bone marrow disease is a disease
[4.32s -> 8.06s] Its main feature is that blood cells in the bone marrow are reduced
[8.06s -> 11.60s] This reduction will affect all types of blood cells
[11.60s -> 14.56s] Including red cells, white cells and blood cells
[14.56s -> 20.08s] Usually, bone marrow disease is a side effect of chemotherapy or chemotherapy used in cancer treatment
[20.08s -> 23.44s] Because these treatments kill cancer cells at the same time
[23.44s -> 26.20s] It will also harm the healthy cells in the bone marrow
[26.20s -> 29.24s] Bone marrow disease may bring the following harm
[29.24s -> 30.86s] 1. Pneumonia
[30.86s -> 33.68s] Bone marrow disease may cause red cells to decrease
[33.68s -> 35.60s] It may cause pneumonia
[35.60s -> 39.94s] It is a sign of fatigue, weakness, respiratory cramps and pale face
[39.94s -> 44.52s] Severe pneumonia may lead to heart problems and reduced daily activity
[44.52s -> 46.90s] 2. Reducing neutral blood cells
[46.90s -> 49.48s] Bone marrow disease may lead to reduced white cells
[49.48s -> 51.94s] Especially reduced neutral blood cells
[51.94s -> 55.52s] Patients are more likely to be infected with bacteria, viruses and germs
[55.52s -> 59.02s] In some cases, it may be very serious and critical
[59.02s -> 59.22s] In some cases, it may be very serious and critical
[59.22s -> 59.70s] In some cases, it may be very serious and critical
[59.70s -> 61.68s] 3. Reducing neutral blood cells
[61.68s -> 64.54s] Bone marrow disease may cause reduced blood cells to be generated
[64.54s -> 66.98s] It may increase the risk of bleeding and bruising
[66.98s -> 71.10s] Patients may have nosebleeds, gums bleeding, and easy bruising
[71.10s -> 74.98s] Serious neutral blood cells can lead to auto-blood bleeding
[74.98s -> 78.14s] Small wound bleeding time is prolonged and internal bleeding
[78.14s -> 79.72s] This may be fatal to life
[79.72s -> 81.54s] 4. Treating delay
[81.54s -> 84.06s] If the patient's blood cell technology is too low
[84.06s -> 88.50s] It may need to delay the chemotherapy cycle or chemotherapy to restore bone marrow function
[88.50s -> 88.94s] 5. Reducing neutral blood cells
[89.22s -> 89.76s] If the patient's blood cell technology is too low
[89.76s -> 90.60s] If the patient's blood cell technology is too low
[90.60s -> 92.68s] It may hinder the treatment of cancer
[92.68s -> 94.50s] Cause danger
[94.50s -> 97.30s] 5. Reducing the amount of blood
[97.30s -> 101.14s] Bone marrow disease may require the reduction of chemotherapy or chemotherapy
[101.14s -> 103.46s] It may affect the overall effectiveness of cancer treatment
[103.46s -> 105.50s] 6. Extending the hospitalization period
[105.50s -> 108.74s] Severe infection or bleeding may require hospitalization and monitoring
[108.74s -> 113.02s] It may increase the total time and economic cost of cancer treatment for the patient
[113.02s -> 114.34s] 7. Reducing the quality of life
[114.34s -> 117.06s] 7. Reducing the quality of life
[117.06s -> 118.12s] 3. Reducing the quality of life
[118.12s -> 118.30s] 4. Reduction of life and health到了這里，關(guān)于基于OpenAI的Whisper構(gòu)建的高效語音識別模型：faster-whisper的文章就介紹完了。如果您還想了解更多內(nèi)容，請在右上角搜索TOY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！
