当前位置：首页 > news >正文

Teams Bot机器人实时语音识别的多引擎的处理

news 2025/8/25 6:20:01

之前开发的Teams的Bot 是使用微软的实时语音识别。现在增加了一个选项，可以在Azure 和 Soniox两引擎间切换。

Azure 的实时语音识别

Azure 的实时语音识别使用 Microsoft.CognitiveServices.Speech.Transcription 进行处理。这个质量很不错，而且可以设置每个人的语音特征（wav audio file for creating voice signatures must be 16-bit, 16 kHz sample rate, in single channel (mono) format. The recommended length for each audio sample is between 30 seconds and two minutes. ）。

            byte[] fileBytes = File.ReadAllBytes(fn);var content = new ByteArrayContent(fileBytes);var client = new HttpClient();client.DefaultRequestHeaders.Add("Ocp-Apim-Subscription-Key", subscriptionKey);var response = await client.PostAsync($"https://signature.{region}.cts.speech.microsoft.com/api/v1/Signature/GenerateVoiceSignatureFromByteArray", content);var jsonData = await response.Content.ReadAsStringAsync();var result = JsonConvert.DeserializeObject<VoiceSignature>(jsonData);return JsonConvert.SerializeObject(result.Signature);

设置了语音特征后就可以在实时语音识别中，自动识别说话者。一切都好，唯一的一个问题就是不便宜，目前一个月的费用有几万美元。

Soniox的实时语音识别

Soniox的实时语音识别引擎，是用WebSocket的，这一点和Azure的差别很大。下面的代码就是初始化识别引擎连接。

 ClientWebSocket ws= new ClientWebSocket();ws.ConnectAsync(new Uri(ConversationMgr.instance.SonioxUrl), CancellationToken.None).Wait();// Send start request with correct field namesvar startMessage = Encoding.UTF8.GetBytes(Newtonsoft.Json.JsonConvert.SerializeObject(new{api_key = ConversationMgr.instance.SonioxKey,audio_format = ConversationMgr.instance.SonioxAudioFormat,sample_rate = int.Parse(ConversationMgr.instance.SonioxSampleRate),num_channels = int.Parse(ConversationMgr.instance.SonioxNumChannels),model = ConversationMgr.instance.SonioxModel,enable_speaker_diarization=true,language_hints = ConversationMgr.instance.SonioxLanguageHints.Replace(" ", "").Split(new string[] { "," }, StringSplitOptions.RemoveEmptyEntries)}));ws.SendAsync(new ArraySegment<byte>(startMessage), WebSocketMessageType.Text, true, CancellationToken.None).Wait();

初始化之后，当收到从机器人收到语音包后，就把包的时间发送到Soniox。

var audioChunk = new ArraySegment<byte>(item.Buff, 0, item.Buff.Length);
await ws.SendAsync(audioChunk, WebSocketMessageType.Binary, true, CancellationToken.None);

同时并行一个任务从 Soniox接受识别的结果：

result = await ws.ReceiveAsync(new ArraySegment<byte>(buffer), CancellationToken.None);

返回的结果如下

{"tokens":[{"text":"Wh","start_ms":540,"end_ms":540,"confidence":0.999,"is_final":false,"speaker":"1"},{"text":"at","start_ms":540,"end_ms":600,"confidence":1,"is_final":false,"speaker":"1"},{"text":" is","start_ms":660,"end_ms":720,"confidence":0.998,"is_final":false,"speaker":"1"},{"text":" y","start_ms":780,"end_ms":840,"confidence":1,"is_final":false,"speaker":"1"},{"text":"our","start_ms":840,"end_ms":900,"confidence":1,"is_final":false,"speaker":"1"},{"text":" best","start_ms":960,"end_ms":1020,"confidence":1,"is_final":false,"speaker":"1"}],"final_audio_proc_ms":0,"total_audio_proc_ms":1800}

{
"tokens": [
{
"text": "Wh",
"start_ms": 540,
"end_ms": 540,
"confidence": 0.999,
"is_final": false,
"speaker": "1"
},
{
"text": "at",
"start_ms": 540,
"end_ms": 600,
"confidence": 1,
"is_final": false,
"speaker": "1"
},
{
"text": " is",
"start_ms": 660,
"end_ms": 720,
"confidence": 0.998,
"is_final": false,
"speaker": "1"
},
{
"text": " y",
"start_ms": 780,
"end_ms": 840,
"confidence": 1,
"is_final": false,
"speaker": "1"
},
{
"text": "our",
"start_ms": 840,
"end_ms": 900,
"confidence": 1,
"is_final": false,
"speaker": "1"
},
{
"text": " best",
"start_ms": 960,
"end_ms": 1020,
"confidence": 1,
"is_final": false,
"speaker": "1"
}
],
"final_audio_proc_ms": 0,
"total_audio_proc_ms": 1800
}