Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SpeechRecognitionAgent #353

Open
wants to merge 4 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added data.db
Binary file not shown.
39 changes: 39 additions & 0 deletions docs/agents/audioRecognitionAgent.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Audio recognition agent

This is an agent that recognizes speech spoken by the user.

**Action class:**

`action_recognize_audio`

**Parameters:**

1. `messageAddr` -- an element of `concept_message`.

**Libraries used:**

* SpeechRecognition - to recognize speech.

**Comment:**

* Internet connection is required for the agent to work.

### Examples

Examples of an input structure:

<img src="../images/audioRecognitionAgentInput.png"></img>

Examples of an outnput structure:

<img src="../images/audioRecognitionAgentOutput.png"></img>

### Agent implementation language
Python

### Result

Possible result codes:

* `SC_RESULT_OK` - agent created a structure with recognized text.
* `SC_RESULT_ERROR`- internal error.
39 changes: 39 additions & 0 deletions docs/agents/audioRecognitionAgent.ru.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Агент распознавания речи

Данный агент распознает речь, сказанную пользователем.

**Класс действий:**

`action_recognize_audio`

**Параметры:**

1. `messageAddr` -- элемент класса `concept_message`.

**Используемые библиотеки:**

* SpeechRecognition - для распознавания речи.

**Комментарий:**

* Для работы агента необходимо подключение к интернету.

### Пример

Пример входной структуры:

<img src="../images/audioRecognitionAgentInput.png"></img>

Пример выходной структуры:

<img src="../images/audioRecognitionAgentOutput.png"></img>

### Язык реализации агента
Python

### Результат

Возможные результаты:

* `SC_RESULT_OK` - агент создал структуру с распознанным текстом.
* `SC_RESULT_ERROR`- внутренняя ошибка.
Binary file added docs/agents/images/audioRecognitionAgentInput.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
* [Change interface color agent](agents/changeInterfaceColorAgent.md)
* [Find word in set by first letter agent](agents/findWordInSetByFirstLetter.md)
* [Weather agent](agents/weatherAgent.md)
* [Audio recognition agent](agents/audioRecognitionAgent.md)

- **Patterns**
* [Knowledge base patterns](patterns/kb-patterns.md)
Expand Down
1 change: 1 addition & 0 deletions docs/index.ru.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@
* [Агент смены цвета компонента интерфейса](agents/changeInterfaceColorAgent.ru.md)
* [Агент поиска слова по первой букве](agents/findWordInSetByFirstLetter.ru.md)
* [Агент прогноза погоды](agents/weatherAgent.ru.md)
* [Агент распознавания речи](agents/audioRecognitionAgent.ru.md)

- **Шаблоны**
* [Шаблоны для базы знаний](patterns/kb-patterns.ru.md)
Expand Down
Binary file added kb.bin-tests/strings1.scdb
Binary file not shown.
15 changes: 15 additions & 0 deletions kb/extra/audio_recognition_agent/temp1.scs

Large diffs are not rendered by default.

15 changes: 15 additions & 0 deletions kb/extra/audio_recognition_agent/temp2.scs

Large diffs are not rendered by default.

15 changes: 15 additions & 0 deletions kb/extra/audio_recognition_agent/temp3.scs

Large diffs are not rendered by default.

15 changes: 15 additions & 0 deletions kb/extra/audio_recognition_agent/temp4.scs

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
#include <memory>

#include "sc-memory/sc_memory.hpp"


#include "sc-memory/sc_event.hpp"




#define AssignDynamicArgumentTestAgent_hpp_15_init bool _InitInternal(ScAddr const & outputStructure = ScAddr::Empty) override \
{ \
ScMemoryContext ctx(sc_access_lvl_make_min, "AssignDynamicArgumentTestAgent::_InitInternal"); \
ScSystemIdentifierFiver fiver; \
bool result = true; \
return result; \
}


#define AssignDynamicArgumentTestAgent_hpp_15_initStatic static bool _InitStaticInternal(ScAddr const & outputStructure = ScAddr::Empty) \
{ \
ScMemoryContext ctx(sc_access_lvl_make_min, "AssignDynamicArgumentTestAgent::_InitStaticInternal"); \
ScSystemIdentifierFiver fiver; \
bool result = true; \
return result; \
}


#define AssignDynamicArgumentTestAgent_hpp_15_decl \
private:\
typedef ScAgent Super;\
protected: \
AssignDynamicArgumentTestAgent(char const * name, sc_uint8 accessLvl) : Super(name, accessLvl) {}\
virtual sc_result Run(ScAddr const & listenAddr, ScAddr const & edgeAddr, ScAddr const & otherAddr) override; \
private:\
static std::unique_ptr<ScEvent> ms_event;\
static std::unique_ptr<ScMemoryContext> ms_context;\
public: \
static bool handler_emit(ScAddr const & addr, ScAddr const & edgeAddr, ScAddr const & otherAddr)\
{\
AssignDynamicArgumentTestAgent Instance("AssignDynamicArgumentTestAgent", sc_access_lvl_make_min);\
return Instance.Run(addr, edgeAddr, otherAddr) == SC_RESULT_OK;\
}\
static void RegisterHandler()\
{\
ms_context.reset(new ScMemoryContext(sc_access_lvl_make_min, "handler_AssignDynamicArgumentTestAgent"));\
ms_event.reset(new ScEvent(*ms_context, scAgentsCommon::CoreKeynodes::question_initiated, ScEvent::Type::AddOutputEdge, &AssignDynamicArgumentTestAgent::handler_emit));\
if (ms_event.get())\
{\
SC_LOG_INFO("Register agent AssignDynamicArgumentTestAgent");\
}\
else\
{\
SC_LOG_ERROR("Can't register agent AssignDynamicArgumentTestAgent");\
}\
}\
static void UnregisterHandler()\
{\
ms_event.reset();\
ms_context.reset();\
}

#define AssignDynamicArgumentTestAgent_hpp_AssignDynamicArgumentTestAgent_impl \
std::unique_ptr<ScEvent> AssignDynamicArgumentTestAgent::ms_event;\
std::unique_ptr<ScMemoryContext> AssignDynamicArgumentTestAgent::ms_context;

#undef ScFileID
#define ScFileID AssignDynamicArgumentTestAgent_hpp

Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
#include <memory>

#include "sc-memory/sc_memory.hpp"


#include "sc-memory/sc_event.hpp"




#define CheckDynamicArgumentTestAgent_hpp_14_init bool _InitInternal(ScAddr const & outputStructure = ScAddr::Empty) override \
{ \
ScMemoryContext ctx(sc_access_lvl_make_min, "CheckDynamicArgumentTestAgent::_InitInternal"); \
ScSystemIdentifierFiver fiver; \
bool result = true; \
return result; \
}


#define CheckDynamicArgumentTestAgent_hpp_14_initStatic static bool _InitStaticInternal(ScAddr const & outputStructure = ScAddr::Empty) \
{ \
ScMemoryContext ctx(sc_access_lvl_make_min, "CheckDynamicArgumentTestAgent::_InitStaticInternal"); \
ScSystemIdentifierFiver fiver; \
bool result = true; \
return result; \
}


#define CheckDynamicArgumentTestAgent_hpp_14_decl \
private:\
typedef ScAgent Super;\
protected: \
CheckDynamicArgumentTestAgent(char const * name, sc_uint8 accessLvl) : Super(name, accessLvl) {}\
virtual sc_result Run(ScAddr const & listenAddr, ScAddr const & edgeAddr, ScAddr const & otherAddr) override; \
private:\
static std::unique_ptr<ScEvent> ms_event;\
static std::unique_ptr<ScMemoryContext> ms_context;\
public: \
static bool handler_emit(ScAddr const & addr, ScAddr const & edgeAddr, ScAddr const & otherAddr)\
{\
CheckDynamicArgumentTestAgent Instance("CheckDynamicArgumentTestAgent", sc_access_lvl_make_min);\
return Instance.Run(addr, edgeAddr, otherAddr) == SC_RESULT_OK;\
}\
static void RegisterHandler()\
{\
ms_context.reset(new ScMemoryContext(sc_access_lvl_make_min, "handler_CheckDynamicArgumentTestAgent"));\
ms_event.reset(new ScEvent(*ms_context, scAgentsCommon::CoreKeynodes::question_initiated, ScEvent::Type::AddOutputEdge, &CheckDynamicArgumentTestAgent::handler_emit));\
if (ms_event.get())\
{\
SC_LOG_INFO("Register agent CheckDynamicArgumentTestAgent");\
}\
else\
{\
SC_LOG_ERROR("Can't register agent CheckDynamicArgumentTestAgent");\
}\
}\
static void UnregisterHandler()\
{\
ms_event.reset();\
ms_context.reset();\
}

#define CheckDynamicArgumentTestAgent_hpp_CheckDynamicArgumentTestAgent_impl \
std::unique_ptr<ScEvent> CheckDynamicArgumentTestAgent::ms_event;\
std::unique_ptr<ScMemoryContext> CheckDynamicArgumentTestAgent::ms_context;

#undef ScFileID
#define ScFileID CheckDynamicArgumentTestAgent_hpp

Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
#include <memory>

#include "sc-memory/sc_memory.hpp"


#include "sc-memory/sc_event.hpp"




#define CommonModule_hpp_17_init bool _InitInternal(ScAddr const & outputStructure = ScAddr::Empty) override \
{ \
ScMemoryContext ctx(sc_access_lvl_make_min, "CommonModule::_InitInternal"); \
ScSystemIdentifierFiver fiver; \
bool result = true; \
return result; \
}


#define CommonModule_hpp_17_initStatic static bool _InitStaticInternal(ScAddr const & outputStructure = ScAddr::Empty) \
{ \
ScMemoryContext ctx(sc_access_lvl_make_min, "CommonModule::_InitStaticInternal"); \
ScSystemIdentifierFiver fiver; \
bool result = true; \
return result; \
}


#define CommonModule_hpp_17_decl \
public:\
sc_result InitializeGenerated()\
{\
if (!ScKeynodes::Init())\
return SC_RESULT_ERROR;\
if (!ScAgentInit(false))\
return SC_RESULT_ERROR;\
return InitializeImpl();\
}\
sc_result ShutdownGenerated()\
{\
return ShutdownImpl();\
}\
sc_uint32 GetLoadPriorityGenerated()\
{\
return 100;\
}

#define CommonModule_hpp_CommonModule_impl

#undef ScFileID
#define ScFileID CommonModule_hpp

46 changes: 46 additions & 0 deletions problem-solver/cxx/commonModule/generated/Keynodes.generated.hpp
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
#include <memory>

#include "sc-memory/sc_memory.hpp"


#include "sc-memory/sc_event.hpp"




#define Keynodes_hpp_13_init bool _InitInternal(ScAddr const & outputStructure = ScAddr::Empty) override \
{ \
ScMemoryContext ctx(sc_access_lvl_make_min, "Keynodes::_InitInternal"); \
ScSystemIdentifierFiver fiver; \
bool result = true; \
return result; \
}


#define Keynodes_hpp_13_initStatic static bool _InitStaticInternal(ScAddr const & outputStructure = ScAddr::Empty) \
{ \
ScMemoryContext ctx(sc_access_lvl_make_min, "Keynodes::_InitStaticInternal"); \
ScSystemIdentifierFiver fiver; \
bool result = true; \
ctx.HelperResolveSystemIdtf("action_deactivated", ScType::NodeConst, fiver);action_deactivated = fiver.addr1; result = result && action_deactivated.IsValid(); if (outputStructure.IsValid()) {ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr1);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr2);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr3);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr4);}; \
ctx.HelperResolveSystemIdtf("action_interpret_non_atomic_action", ScType::NodeConst, fiver);action_interpret_non_atomic_action = fiver.addr1; result = result && action_interpret_non_atomic_action.IsValid(); if (outputStructure.IsValid()) {ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr1);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr2);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr3);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr4);}; \
ctx.HelperResolveSystemIdtf("languages", ScType::NodeConst, fiver);languages = fiver.addr1; result = result && languages.IsValid(); if (outputStructure.IsValid()) {ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr1);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr2);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr3);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr4);}; \
ctx.HelperResolveSystemIdtf("nrel_decomposition_of_action", ScType::NodeConst, fiver);nrel_decomposition_of_action = fiver.addr1; result = result && nrel_decomposition_of_action.IsValid(); if (outputStructure.IsValid()) {ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr1);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr2);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr3);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr4);}; \
ctx.HelperResolveSystemIdtf("nrel_then", ScType::NodeConst, fiver);nrel_then = fiver.addr1; result = result && nrel_then.IsValid(); if (outputStructure.IsValid()) {ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr1);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr2);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr3);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr4);}; \
ctx.HelperResolveSystemIdtf("nrel_else", ScType::NodeConst, fiver);nrel_else = fiver.addr1; result = result && nrel_else.IsValid(); if (outputStructure.IsValid()) {ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr1);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr2);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr3);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr4);}; \
ctx.HelperResolveSystemIdtf("nrel_goto", ScType::NodeConst, fiver);nrel_goto = fiver.addr1; result = result && nrel_goto.IsValid(); if (outputStructure.IsValid()) {ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr1);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr2);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr3);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr4);}; \
ctx.HelperResolveSystemIdtf("file", ScType::NodeConst, fiver);file = fiver.addr1; result = result && file.IsValid(); if (outputStructure.IsValid()) {ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr1);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr2);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr3);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr4);}; \
ctx.HelperResolveSystemIdtf("power_of_set", ScType::NodeConst, fiver);power_of_set = fiver.addr1; result = result && power_of_set.IsValid(); if (outputStructure.IsValid()) {ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr1);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr2);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr3);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr4);}; \
ctx.HelperResolveSystemIdtf("nrel_measurement", ScType::NodeConst, fiver);nrel_measurement = fiver.addr1; result = result && nrel_measurement.IsValid(); if (outputStructure.IsValid()) {ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr1);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr2);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr3);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr4);}; \
ctx.HelperResolveSystemIdtf("lang_en", ScType::NodeConst, fiver);lang_en = fiver.addr1; result = result && lang_en.IsValid(); if (outputStructure.IsValid()) {ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr1);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr2);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr3);ctx.CreateEdge(ScType::EdgeAccessConstPosPerm, outputStructure, fiver.addr4);}; \
return result; \
}


#define Keynodes_hpp_13_decl

#define Keynodes_hpp_Keynodes_impl

#undef ScFileID
#define ScFileID Keynodes_hpp

Loading
Loading