Merge branch 'feat/add_two_wn' into 'master'

Feat/add two wn See merge request speech-recognition-framework/esp-skainet!34
espressif · Aug 15, 2023 · 347b434 · 347b434
2 parents a9429c3 + 1be5f97
commit 347b434
Show file tree

Hide file tree

Showing 8 changed files with 233 additions and 156 deletions.
diff --git a/components/esp-sr b/components/esp-sr
diff --git a/examples/wake_word_detection/README.md b/examples/wake_word_detection/README.md
@@ -5,40 +5,34 @@
 (See the [README.md](../README.md) file in the upper level 'examples' directory for more information about examples.)
 
 This example is used to test performance of wakenet.(the word word engine of Espressif).
+This example can load multiple models, but can only run two models at the same time
 
 
-## How to use this example
-
-### Configure, Build and Flash
-
-
-##### set-target 
+### Configure
 
+Select board and wake words
 ```
 idf.py set-target esp32s3
-```
-
-##### configure
+idf.py menuconfig
 
-Select the default sdkconfig according to the development board module
+# Select audio board
+Audio Media HAL -> Audio hardware board -> ESP32-S3-Korvo-1
 
-```
-cp sdkconfig.defaults.esp32s3 sdkconfig
-```
+# Load one wakenet model
+ESP Speech Recognition -> Select wake words -> Hi,Lexin (wn9_hilexin)
 
-Select the different wake word
-```
-idf.py menuconfig
-ESP Speech Recognition -> Wake word engine    // select the version of wakenet
-ESP Speech Recognition -> Wake word name      // select the wake word
+# Load multiple wakenet models
+ESP Speech Recognition -> Select wake words -> Hi,Lexin (wn9_hilexin) -> Load Multiple Wake Words
+ESP Speech Recognition -> Load Multiple Wake Words -> Hi,Lexin (wn9_hilexin)
+                                                   -> Hi,ESP (wn9_hiesp)
 ```
 
-##### build&flash
+### build&flash
 
 Build the project and flash it to the board, then run the monitor tool to view the output via serial port:
 
 ```
-idf.py -b 2000000 flash monitor 
+idf.py flash monitor 
 ```
 
 (To exit the serial monitor, type ``Ctrl-]``.)

diff --git a/examples/wake_word_detection/README_cn.md b/examples/wake_word_detection/README_cn.md
@@ -1,40 +1,30 @@
-# 垃圾分类例程 [[English]](./README.md)
+# 唤醒词检测 [[English]](./README.md)
 
 （参考在上一级的 `examples` 目录下的 [README.md](../README.md) 文件来获取更多信息。）
 
-在这个示例中，我们配置了 4 个命令词 ID：
 
-* 湿垃圾
-* 干垃圾
-* 可回收垃圾
-* 有害垃圾
+该示例用于展示如何使用 WakeNet 进行唤醒词检测。示例支持同时加载多个 wakenet 模型，但最多选择两个模型进行识别。
 
-我们已经通过 `menuconfig` 配置了 45 类常见的垃圾名称。
-
-## 如何使用例程
-
-### Hardware Required
-### 硬件需求
-
-- 这个示例能够在 ESP32-LyraT-Mini 开发板/ESP32-Korvo 开发板/ESP32-S3-Korvo 开发板上运行，关于开发板更多的信息，请参考 [ESP32-LyraT-Mini Getting Started Guide](https://docs.espressif.com/projects/esp-adf/en/latest/get-started/get-started-esp32-lyrat-mini.html) 或者 [ESP32-Korvo Getting Started Guide](https://github.com/espressif/esp-skainet/blob/master/docs/en/hw-reference/esp32/user-guide-esp32-korvo-v1.1.md).
-
-- 一只音箱
+### 编译和烧写
 
-### 配置工程
+开发板和唤醒词配置
 
-* 根据使用的开发板模组选择对应的 `sdkconfig`
-  
-  - 比如， 使用 ESP32， 运行 `cp sdkconfig_esp32.defaults sdkconfig`
+```
+idf.py set-target esp32s3
+idf.py menuconfig
 
-  - 比如， 使用 ESP32-S3， 运行 `cp sdkconfig_esp32s3r8_8+4.defaults sdkconfig`
+# 选择开发板
+Audio Media HAL -> Audio hardware board -> ESP32-S3-Korvo-1
 
-* 进入 `idf.py menuconfig`
+# 加载一个唤醒词
+ESP Speech Recognition -> Select wake words -> Hi,Lexin (wn9_hilexin)
 
-* 通过 `Serial Flasher Options`设置串口信息
+# 加载多个唤醒词
+ESP Speech Recognition -> Select wake words -> Hi,Lexin (wn9_hilexin) -> Load Multiple Wake Words
+ESP Speech Recognition -> Load Multiple Wake Words -> Hi,Lexin (wn9_hilexin)
+                                                   -> Hi,ESP (wn9_hiesp)
+```        
 
-* 通过`ESP Speech Recognition`添加或更改垃圾名称
-
-### 编译和烧写
 
 编译并烧写，然后运行终端监控查看打印：
 
@@ -45,84 +35,3 @@ idf.py flash monitor
 (退出窗口，请键入 ``Ctrl-]``.)
 
 参考 [Getting Started Guide](https://docs.espressif.com/projects/esp-idf/en/stable/get-started-cmake/index.html) 来获取更多使用 ESP-IDF 编译项目的细节.
-
-## 例程输出
-
-以下是例程的上电打印输出:
-
-```
-Quantized wakeNet5: wakeNet5_v1_hilexin_5_0.95_0.90, mode:0
-Quantized MN1
-I (198) MN: ---------------------SPEECH COMMANDS---------------------
-I (197) MN: Command ID0, phrase 0: wei sheng zhi
-I (197) MN: Command ID0, phrase 1: shi zhi jin
-I (207) MN: Command ID0, phrase 2: shi pin dai
-I (207) MN: Command ID0, phrase 3: can jin zhi
-I (217) MN: Command ID0, phrase 4: niao bu shi
-I (217) MN: Command ID0, phrase 5: mao sha
-I (227) MN: Command ID0, phrase 6: mao fa
-I (227) MN: Command ID0, phrase 7: yi ci xing can ju
-I (237) MN: Command ID0, phrase 8: jiu mao jin
-I (237) MN: Command ID0, phrase 9: tao ci zhi pin
-I (247) MN: Command ID0, phrase 10: bei ke
-I (247) MN: Command ID0, phrase 11: fa jiao
-I (257) MN: Command ID0, phrase 12: sao ba
-I (257) MN: Command ID0, phrase 13: da huo ji
-I (267) MN: Command ID1, phrase 14: gua zi ke
-I (267) MN: Command ID1, phrase 15: cha ye zha
-I (277) MN: Command ID1, phrase 16: ji rou
-I (277) MN: Command ID1, phrase 17: cai ye
-I (287) MN: Command ID1, phrase 18: gua guo pi
-I (287) MN: Command ID1, phrase 19: sheng fan sheng cai
-I (297) MN: Command ID1, phrase 20: zhong yao yao zha
-I (297) MN: Command ID1, phrase 21: bing gan
-I (307) MN: Command ID1, phrase 22: yu mi
-I (307) MN: Command ID1, phrase 23: ji gu tou
-I (317) MN: Command ID1, phrase 24: xi hong shi
-I (317) MN: Command ID1, phrase 25: hua sheng ke
-I (327) MN: Command ID2, phrase 26: niu kou dian chi
-I (327) MN: Command ID2, phrase 27: guo qi yao pin
-I (337) MN: Command ID2, phrase 28: lao shu yao
-I (337) MN: Command ID2, phrase 29: fei yao pin
-I (347) MN: Command ID2, phrase 30: fei you qi
-I (347) MN: Command ID2, phrase 31: you qi tong
-I (357) MN: Command ID2, phrase 32: ying guang deng
-I (367) MN: Command ID2, phrase 33: sha chong ji
-I (367) MN: Command ID3, phrase 34: su liao ping
-I (377) MN: Command ID3, phrase 35: yi la guan
-I (377) MN: Command ID3, phrase 36: kuai di zhi xiang
-I (387) MN: Command ID3, phrase 37: jiu bao zhi
-I (387) MN: Command ID3, phrase 38: guan tou he
-I (397) MN: Command ID3, phrase 39: ying su liao
-I (397) MN: Command ID3, phrase 40: bao zhuang zhi
-I (407) MN: Command ID3, phrase 41: jiu tie guo
-I (407) MN: Command ID3, phrase 42: lan qiu
-I (417) MN: Command ID3, phrase 43: bo li hu
-I (268) MN: ---------------------------------------------------------
-
-------------detect start------------
-
-```
-
-然后，说“ “Hi Lexin" ([Ləsɪ:n]) ” 来唤醒开发板，唤醒后会打印：
-
-```
-wakeword DETECTED.
------------LISTENING-----------
-```
-同时，板子上的 LED 灯会亮起，直到进入下一次等待唤醒状态。
-
-然后，板子会进入侦听状态，等到用户说出垃圾的名称。
-
-比如，用户说出：“卫生纸”，
-
-```
------------LISTENING-----------
-phrase ID: 0, prob: 0.866630
-Commands ID: 0
-干垃圾（Residual Waste）
------------awaits to be waken up-----------
-
-```
-
-喇叭会播放对应的垃圾分类种类。
diff --git a/examples/wake_word_detection/main/main.c b/examples/wake_word_detection/main/main.c
@@ -16,6 +16,7 @@
 #include "esp_mn_models.h"
 #include "esp_board_init.h"
 #include "model_path.h"
+#include "string.h"
 
 int detect_flag = 0;
 static esp_afe_sr_iface_t *afe_handle = NULL;
@@ -61,6 +62,7 @@ void detect_Task(void *arg)
 
         if (res->wakeup_state == WAKENET_DETECTED) {
             printf("wakeword detected\n");
+	        printf("model index:%d, word index:%d\n", res->wakenet_model_index, res->wake_word_index);
             printf("-----------LISTENING-----------\n");
         }
     }
@@ -77,18 +79,32 @@ void app_main()
     // ESP_ERROR_CHECK(esp_sdcard_init("/sdcard", 10));
 
     srmodel_list_t *models = esp_srmodel_init("model");
+    char *wn_name = NULL;
+    char *wn_name_2 = NULL;
+
     if (models!=NULL) {
         for (int i=0; i<models->num; i++) {
-            printf("Load: %s\n", models->model_name[i]);
+            if (strstr(models->model_name[i], ESP_WN_PREFIX) != NULL) {
+                if (wn_name == NULL) {
+                    wn_name = models->model_name[i];
+                    printf("The first wakenet model: %s\n", wn_name);
+                } else if (wn_name_2 == NULL) {
+                    wn_name_2 = models->model_name[i];
+                    printf("The second wakenet model: %s\n", wn_name_2);
+                }
+            }
         }
+    } else {
+        printf("Please enable wakenet model and select wake word by menuconfig!\n");
+        return ;
     }
-    char *wn_name = esp_srmodel_filter(models, ESP_WN_PREFIX, NULL);
 
     afe_handle = (esp_afe_sr_iface_t *)&ESP_AFE_SR_HANDLE;
     afe_config_t afe_config = AFE_CONFIG_DEFAULT();
     afe_config.memory_alloc_mode = AFE_MEMORY_ALLOC_MORE_PSRAM;
     afe_config.wakenet_init = true;
     afe_config.wakenet_model_name = wn_name;
+    afe_config.wakenet_model_name_2 = wn_name_2;
     afe_config.voice_communication_init = false;
 
 #if defined CONFIG_ESP32_S3_BOX_BOARD || defined CONFIG_ESP32_S3_EYE_BOARD

diff --git a/examples/wake_word_detection/sdkconfig.defaults.esp32s3 b/examples/wake_word_detection/sdkconfig.defaults.esp32s3
@@ -21,9 +21,15 @@ CONFIG_ESP_SYSTEM_PANIC_PRINT_REBOOT=y
 CONFIG_ESP_SYSTEM_RTC_FAST_MEM_AS_HEAP_DEPCHECK=y
 CONFIG_ESP_SYSTEM_ALLOW_RTC_FAST_MEM_AS_HEAP=y
 
-CONFIG_MODEL_IN_SPIFFS=y
-# CONFIG_MODEL_IN_SDCARD is not set
+CONFIG_USE_AFE=y
+CONFIG_AFE_INTERFACE_V1=y
 CONFIG_USE_WAKENET=y
-CONFIG_SR_WN_MODEL_WN7_QUANT8=y
-CONFIG_SR_WN_WN7_HILEXIN=y
-# CONFIG_USE_MULTINET is not set
+CONFIG_USE_MULTINET=n
+CONFIG_SR_WN_LOAD_MULIT_WORD=y
+
+#
+# Load Multiple Wake Words
+#
+CONFIG_SR_WN_WN9_HILEXIN_MULTI=y
+CONFIG_SR_WN_WN9_HIESP_MULTI=y
+