功能介紹：
錄音并實(shí)時(shí)獲取RAW的音頻格式數(shù)據(jù)，利用WebSocket上傳數(shù)據(jù)到服務(wù)器，并實(shí)時(shí)獲取語(yǔ)音識(shí)別結(jié)果，參考文檔使用AudioCapturer開(kāi)發(fā)音頻錄制功能(ArkTS)，更詳細(xì)接口信息請(qǐng)查看接口文檔：AudioCapturer8+和@ohos.net.webSocket (WebSocket連接)。
知識(shí)點(diǎn)：
- 熟悉使用AudioCapturer錄音并實(shí)時(shí)獲取RAW格式數(shù)據(jù)。
- 熟悉使用WebSocket上傳音頻數(shù)據(jù)并獲取識(shí)別結(jié)果。
- 熟悉對(duì)敏感權(quán)限的動(dòng)態(tài)申請(qǐng)方式，本項(xiàng)目的敏感權(quán)限為
MICROPHONE。 - 關(guān)于如何搭建實(shí)時(shí)語(yǔ)音識(shí)別服務(wù)，可以參考我的另外一篇文章：《識(shí)別準(zhǔn)確率竟如此高，實(shí)時(shí)語(yǔ)音識(shí)別服務(wù)》。
使用環(huán)境：
- API 9
- DevEco Studio 4.0 Release
- Windows 11
- Stage模型
- ArkTS語(yǔ)言
所需權(quán)限：
- ohos.permission.MICROPHONE
效果圖：
核心代碼：
src/main/ets/utils/Permission.ets是動(dòng)態(tài)申請(qǐng)權(quán)限的工具：
import bundleManager from '@ohos.bundle.bundleManager';
import abilityAccessCtrl, { Permissions } from '@ohos.abilityAccessCtrl';
async function checkAccessToken(permission: Permissions): Promise<abilityAccessCtrl.GrantStatus> {
let atManager = abilityAccessCtrl.createAtManager();
let grantStatus: abilityAccessCtrl.GrantStatus;
// 獲取應(yīng)用程序的accessTokenID
let tokenId: number;
try {
let bundleInfo: bundleManager.BundleInfo = await bundleManager.getBundleInfoForSelf(bundleManager.BundleFlag.GET_BUNDLE_INFO_WITH_APPLICATION);
let appInfo: bundleManager.ApplicationInfo = bundleInfo.appInfo;
tokenId = appInfo.accessTokenId;
} catch (err) {
console.error(`getBundleInfoForSelf failed, code is ${err.code}, message is ${err.message}`);
}
// 校驗(yàn)應(yīng)用是否被授予權(quán)限
try {
grantStatus = await atManager.checkAccessToken(tokenId, permission);
} catch (err) {
console.error(`checkAccessToken failed, code is ${err.code}, message is ${err.message}`);
}
return grantStatus;
}
export async function checkPermissions(permission: Permissions): Promise<boolean> {
let grantStatus: abilityAccessCtrl.GrantStatus = await checkAccessToken(permission);
if (grantStatus === abilityAccessCtrl.GrantStatus.PERMISSION_GRANTED) {
return true
} else {
return false
}
}
src/main/ets/utils/Recorder.ets是錄音工具類，進(jìn)行錄音和獲取錄音數(shù)據(jù)。
import audio from '@ohos.multimedia.audio';
import { delay } from './Utils';
export default class AudioCapturer {
private audioCapturer: audio.AudioCapturer | undefined = undefined
private isRecording: boolean = false
private audioStreamInfo: audio.AudioStreamInfo = {
samplingRate: audio.AudioSamplingRate.SAMPLE_RATE_16000, // 音頻采樣率
channels: audio.AudioChannel.CHANNEL_1, // 錄音通道數(shù)
sampleFormat: audio.AudioSampleFormat.SAMPLE_FORMAT_S16LE,
encodingType: audio.AudioEncodingType.ENCODING_TYPE_RAW // 音頻編碼類型
}
private audioCapturerInfo: audio.AudioCapturerInfo = {
// 音源類型，使用SOURCE_TYPE_VOICE_RECOGNITION會(huì)有減噪功能，如果設(shè)備不支持，該用普通麥克風(fēng)：SOURCE_TYPE_MIC
source: audio.SourceType.SOURCE_TYPE_VOICE_RECOGNITION,
capturerFlags: 0 // 音頻采集器標(biāo)志
}
private audioCapturerOptions: audio.AudioCapturerOptions = {
streamInfo: this.audioStreamInfo,
capturerInfo: this.audioCapturerInfo
}
// 初始化，創(chuàng)建實(shí)例，設(shè)置監(jiān)聽(tīng)事件
constructor() {
// 創(chuàng)建AudioCapturer實(shí)例
audio.createAudioCapturer(this.audioCapturerOptions, (err, capturer) => {
if (err) {
console.error(`創(chuàng)建錄音器失敗, 錯(cuò)誤碼：${err.code}, 錯(cuò)誤信息：${err.message}`)
return
}
this.audioCapturer = capturer
console.info('創(chuàng)建錄音器成功')
});
}
// 開(kāi)始一次音頻采集
async start(callback: (state: number, data?: ArrayBuffer) => void) {
// 當(dāng)且僅當(dāng)狀態(tài)為STATE_PREPARED、STATE_PAUSED和STATE_STOPPED之一時(shí)才能啟動(dòng)采集
let stateGroup = [audio.AudioState.STATE_PREPARED, audio.AudioState.STATE_PAUSED, audio.AudioState.STATE_STOPPED]
if (stateGroup.indexOf(this.audioCapturer.state) === -1) {
console.error('啟動(dòng)錄音失敗')
callback(audio.AudioState.STATE_INVALID)
return
}
// 啟動(dòng)采集
await this.audioCapturer.start()
this.isRecording = true
let bufferSize = 1920
// let bufferSize: number = await this.audioCapturer.getBufferSize();
while (this.isRecording) {
let buffer = await this.audioCapturer.read(bufferSize, true)
if (buffer === undefined) {
console.error('讀取錄音數(shù)據(jù)失敗')
} else {
callback(audio.AudioState.STATE_RUNNING, buffer)
}
}
callback(audio.AudioState.STATE_STOPPED)
}
// 停止采集
async stop() {
this.isRecording = false
// 只有采集器狀態(tài)為STATE_RUNNING或STATE_PAUSED的時(shí)候才可以停止
if (this.audioCapturer.state !== audio.AudioState.STATE_RUNNING && this.audioCapturer.state !== audio.AudioState.STATE_PAUSED) {
console.warn('Capturer is not running or paused')
return
}
await delay(200)
// 停止采集
await this.audioCapturer.stop()
if (this.audioCapturer.state.valueOf() === audio.AudioState.STATE_STOPPED) {
console.info('錄音停止')
} else {
console.error('錄音停止失敗')
}
}
// 銷毀實(shí)例，釋放資源
async release() {
// 采集器狀態(tài)不是STATE_RELEASED或STATE_NEW狀態(tài)，才能release
if (this.audioCapturer.state === audio.AudioState.STATE_RELEASED || this.audioCapturer.state === audio.AudioState.STATE_NEW) {
return
}
// 釋放資源
await this.audioCapturer.release()
}
}
還需要一些其他的工具函數(shù)src/main/ets/utils/Utils.ets，這個(gè)主要用于睡眠等待：
// 睡眠
export function delay(milliseconds : number) {
return new Promise(resolve => setTimeout( resolve, milliseconds));
}
還需要在src/main/module.json5添加所需要的權(quán)限，注意是在module中添加，關(guān)于字段說(shuō)明，也需要在各個(gè)的string.json添加：文章來(lái)源：http://www.zghlxwxcb.cn/news/detail-858148.html
"requestPermissions": [
{
"name": "ohos.permission.MICROPHONE",
"reason": "$string:record_reason",
"usedScene": {
"abilities": [
"EntryAbility"
],
"when": "always"
}
}
]
頁(yè)面代碼如下：文章來(lái)源地址http://www.zghlxwxcb.cn/news/detail-858148.html
import abilityAccessCtrl, { Permissions } from '@ohos.abilityAccessCtrl';
import common from '@ohos.app.ability.common';
import webSocket from '@ohos.net.webSocket';
import AudioCapturer from '../utils/Recorder';
import promptAction from '@ohos.promptAction';
import { checkPermissions } from '../utils/Permission';
import audio from '@ohos.multimedia.audio';
// 需要?jiǎng)討B(tài)申請(qǐng)的權(quán)限
const permissions: Array<Permissions> = ['ohos.permission.MICROPHONE'];
// 獲取程序的上下文
const context = getContext(this) as common.UIAbilityContext;
@Entry
@Component
struct Index {
@State recordBtnText: string = '按下錄音'
@State speechResult: string = ''
private offlineResult = ''
private onlineResult = ''
// 語(yǔ)音識(shí)別WebSocket地址
private asrWebSocketUrl = "ws://192.168.0.100:10095"
// 錄音器
private audioCapturer?: AudioCapturer;
// 創(chuàng)建WebSocket
private ws;
// 頁(yè)面顯示時(shí)
async onPageShow() {
// 判斷是否已經(jīng)授權(quán)
let promise = checkPermissions(permissions[0])
promise.then((result) => {
if (result) {
// 初始化錄音器
if (this.audioCapturer == null) {
this.audioCapturer = new AudioCapturer()
}
} else {
this.reqPermissionsAndRecord(permissions)
}
})
}
// 頁(yè)面隱藏時(shí)
async onPageHide() {
if (this.audioCapturer != null) {
this.audioCapturer.release()
}
}
build() {
Row() {
RelativeContainer() {
Text(this.speechResult)
.id("resultText")
.width('95%')
.maxLines(10)
.fontSize(18)
.margin({ top: 10 })
.alignRules({
top: { anchor: '__container__', align: VerticalAlign.Top },
middle: { anchor: '__container__', align: HorizontalAlign.Center }
})
// 錄音按鈕
Button(this.recordBtnText)
.width('90%')
.id("recordBtn")
.margin({ bottom: 10 })
.alignRules({
bottom: { anchor: '__container__', align: VerticalAlign.Bottom },
middle: { anchor: '__container__', align: HorizontalAlign.Center }
})
.onTouch((event) => {
switch (event.type) {
case TouchType.Down:
console.info('按下按鈕')
// 判斷是否有權(quán)限
let promise = checkPermissions(permissions[0])
promise.then((result) => {
if (result) {
// 開(kāi)始錄音
this.startRecord()
this.recordBtnText = '錄音中...'
} else {
// 申請(qǐng)權(quán)限
this.reqPermissionsAndRecord(permissions)
}
})
break
case TouchType.Up:
console.info('松開(kāi)按鈕')
// 停止錄音
this.stopRecord()
this.recordBtnText = '按下錄音'
break
}
})
}
.height('100%')
.width('100%')
}
.height('100%')
}
// 開(kāi)始錄音
startRecord() {
this.setWebSocketCallback()
this.ws.connect(this.asrWebSocketUrl, (err) => {
if (!err) {
console.log("WebSocket連接成功");
let jsonData = '{"mode": "2pass", "chunk_size": [5, 10, 5], "chunk_interval": 10, ' +
'"wav_name": "HarmonyOS", "is_speaking": true, "itn": false}'
// 要發(fā)完json數(shù)據(jù)才能錄音
this.ws.send(jsonData)
// 開(kāi)始錄音
this.audioCapturer.start((state, data) => {
if (state == audio.AudioState.STATE_STOPPED) {
console.info('錄音結(jié)束')
// 錄音結(jié)束，要發(fā)消息告訴服務(wù)器，結(jié)束識(shí)別
let jsonData = '{"is_speaking": false}'
this.ws.send(jsonData)
} else if (state == audio.AudioState.STATE_RUNNING) {
// 發(fā)送語(yǔ)音數(shù)據(jù)
this.ws.send(data, (err) => {
if (err) {
console.log("WebSocket發(fā)送數(shù)據(jù)失敗，錯(cuò)誤信息:" + JSON.stringify(err))
}
});
}
})
} else {
console.log("WebSocket連接失敗，錯(cuò)誤信息: " + JSON.stringify(err));
}
});
}
// 停止錄音
stopRecord() {
if (this.audioCapturer != null) {
this.audioCapturer.stop()
}
}
// 綁定WebSocket事件
setWebSocketCallback() {
// 創(chuàng)建WebSocket
this.ws = webSocket.createWebSocket();
// 接收WebSocket消息
this.ws.on('message', (err, value: string) => {
console.log("WebSocket接收消息，結(jié)果如下：" + value)
// 解析數(shù)據(jù)
let result = JSON.parse(value)
let is_final = result['is_final']
let mode = result['mode']
let text = result['text']
if (mode == '2pass-offline') {
this.offlineResult = this.offlineResult + text
this.onlineResult = ''
} else {
this.onlineResult = this.onlineResult + text
}
this.speechResult = this.offlineResult + this.onlineResult
// 如果是最后的數(shù)據(jù)就關(guān)閉WebSocket
if (is_final) {
this.ws.close()
}
});
// WebSocket關(guān)閉事件
this.ws.on('close', () => {
console.log("WebSocket關(guān)閉連接");
});
// WebSocket發(fā)生錯(cuò)誤事件
this.ws.on('error', (err) => {
console.log("WebSocket出現(xiàn)錯(cuò)誤，錯(cuò)誤信息: " + JSON.stringify(err));
});
}
// 申請(qǐng)權(quán)限
reqPermissionsAndRecord(permissions: Array<Permissions>): void {
let atManager = abilityAccessCtrl.createAtManager();
// requestPermissionsFromUser會(huì)判斷權(quán)限的授權(quán)狀態(tài)來(lái)決定是否喚起彈窗
atManager.requestPermissionsFromUser(context, permissions).then((data) => {
let grantStatus: Array<number> = data.authResults;
let length: number = grantStatus.length;
for (let i = 0; i < length; i++) {
if (grantStatus[i] === 0) {
// 用戶授權(quán)，可以繼續(xù)訪問(wèn)目標(biāo)操作
console.info('授權(quán)成功')
if (this.audioCapturer == null) {
this.audioCapturer = new AudioCapturer()
}
} else {
promptAction.showToast({ message: '授權(quán)失敗,需要授權(quán)才能錄音' })
return;
}
}
}).catch((err) => {
console.error(`requestPermissionsFromUser failed, code is ${err.code}, message is ${err.message}`);
})
}
}
到了這里，關(guān)于鴻蒙應(yīng)用開(kāi)發(fā)-錄音并使用WebSocket實(shí)現(xiàn)實(shí)時(shí)語(yǔ)音識(shí)別的文章就介紹完了。如果您還想了解更多內(nèi)容，請(qǐng)?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！
