Some full sentences are not heard by the bot

[Lovenoreus]

At 0:40 min I am starting talking while the bot is introducing itself, it does not hear this.
At 2:00 min I say a full sentence which the bot does not hear and this creates a long silence afterwards while I thought I should wait for the bot to process but it times out

complete_log_with_monitoring.txt
Transcript written before the call that i will follow t.txt

Phone_call.zip

[Lovenoreus]

I should mention that this is the latest commit 47a8387 used locally and the container is stopped (v17.0.3)

All these 3 issues have the same audio file and attachments

[Lovenoreus]

From your last comment you said that the full sentences might not be heard because of delay and anti-virus so I tested it again in the container where I produced the same problem

I made a seperate Issue for this

[clemlesne]

Seems in part related to #399.

[clemlesne]

Seeing the logs, also related of #395.

[clemlesne]

Also, from the logs, I can see that the bot answered well for multiple sentences, and stopped answering. Can you share the App Insights traces?

It is probable that you exhausted your tokens per minute quota on Azure OpenAI and that the application retried the call multiple times. Also, note that the Global Standard deployment is practical but not latency guaranteed. Sometimes, I'll be extremely fast, sometimes, it'll be much slower. To mitigate that, your company can purshase a reservation, to have guaranteed latency with enhanced performances.

[clemlesne]

Commit a6a3958 should fix the ability to cut answers and properly load the next. It may mitigate your issue. Publshed as of v17.1.0.

[Lovenoreus]

I deployed the newest commit 40d003b with 17.1.0 like I usually do and it couldn't connect to Redis

[2025-01-21 19:48:40 +0000] [8] [INFO] Application startup complete.
2025-01-21T19:48:45.023846Z [info ] Using Redis cache noreuscsgroupswe.redis.cache.windows.net:6380
2025-01-21T19:48:47.304901Z [info ] Created Search "trainings"
2025-01-21T19:49:05.023703Z [info ] Using Redis cache noreuscsgroupswe.redis.cache.windows.net:6380
2025-01-21T19:49:25.021394Z [info ] Using Redis cache noreuscsgroupswe.redis.cache.windows.net:6380
2025-01-21T19:50:25.025943Z [info ] Using Redis cache noreuscsgroupswe.redis.cache.windows.net:6380
2025-01-21T19:54:01.595343Z [error ] Error requesting Redis
Traceback (most recent call last):
File "/app/.venv/lib/python3.12/site-packages/redis/asyncio/connection.py", line 697, in _connect
reader, writer = await asyncio.open_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

redis_error_container-17-1-0.txt

[Lovenoreus]

Thank you for the information, I have changed the quota limit like you said, it was at the default value "Corresponding requests per minute (RPM) = 50 "

[Lovenoreus]

I have always removed every resource except communication service every time I have tested the application to save on cost

So when I want to test it again I do a setup ritual and I don't know if there is a more effecient way to do things

1. Overwrite config.yaml with config-remote-example.yaml and add monitoring
2. Remove the soft-deleted resources (config, multi-ai, translation, open-ai)
3. make deploy name=noreuscsgroupswe
4. Run your cosmos-permission script
5. I am quick to delete log-analytics once it deploys since it seems to be the most costly by far
6. Now I think I need to raise the quota limits

If I knew that I could let some services stay up without any meaningful charge then I would do so, otherwise I just keep doing this

[clemlesne]

I deployed the newest commit 40d003b with 17.1.0 like I usually do and it couldn't connect to Redis

The timeout is certainly an issue from your side. The dependencies nor Redis code nor the container base image changed:

• 40d003b
• 12fd8ce

[clemlesne]

I have always removed every resource except communication service every time I have tested the application to save on cost

If I knew that I could let some services stay up without any meaningful charge then I would do so, otherwise I just keep doing this

Upgrading your quota won't cause you any charge. Only when you'll use it. Azure OpenAI with Global Deployment is pay as you go.

[Lovenoreus]

I deployed it into a correct state yesterday morning with a fully filled out config.yaml file in the extended structure of a config-local-example.yaml

Then later the same day I had removed all resources but CS and redeployed using the smaller structure of config-remote-example.yaml only adding my number and monitoring. When I called the container this time it has this error.

I removed it afterwards and redeployed this morning but got same error, the only things I have changed from your repo was

image_version := 17.1.0

Removing a few letters of this name so that I don't get a name-max-length error
var aiFoundryName = '${prefix}-${openaiLocation}-foundry'
var aiFoundryName = '${prefix}-${openaiLocation}-fo'

Removing a few letters of this name so that I don't get a conflict with a "customSubDomainName" which Azure claims is in a soft-deleted state (I have double checked with commands to list any resources including soft-deleted once but it doesn't really exist)
customSubDomainName: '${prefix}-${location}-translate'
customSubDomainName: '${prefix}-${location}-transla'

Connecting...
2025-01-22T09:19:03.45835 Connecting to the container 'call-center-ai'...
2025-01-22T09:19:03.51388 Successfully Connected to container: 'call-center-ai' [Revision: 'call-center-ai--lxb5ccc-c4d4b56b-qkl8p', Replica: 'call-center-ai--lxb5ccc']
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/redis/asyncio/retry.py", line 64, in call_with_retry
raise error
File "/app/.venv/lib/python3.12/site-packages/redis/asyncio/retry.py", line 59, in call_with_retry
return await do()
^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/redis/asyncio/client.py", line 589, in _send_command_parse_response
await conn.send_command(*args)
File "/app/.venv/lib/python3.12/site-packages/redis/asyncio/connection.py", line 505, in send_command
await self.send_packed_command(
File "/app/.venv/lib/python3.12/site-packages/redis/asyncio/connection.py", line 466, in send_packed_command
await self.connect()
File "/app/.venv/lib/python3.12/site-packages/redis/asyncio/connection.py", line 296, in connect
await self.on_connect()
File "/app/.venv/lib/python3.12/site-packages/redis/asyncio/connection.py", line 365, in on_connect
auth_response = await self.read_response()
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/redis/asyncio/connection.py", line 559, in read_response
raise TimeoutError(f"Timeout reading from {host_error}")
redis.exceptions.TimeoutError: Timeout reading from noreuscsgroupswe.redis.cache.windows.net:6380
2025-01-22T09:20:03.98076 No logs since last 60 seconds

[Lovenoreus]

I should add that even though the container is failing to connect to Redis whatever I do I am still able to use the app locally

[Lovenoreus]

This is a new call I made to local and it is cutting off almost all ending text with the newest commit

I tried raising the value of "recognition_stt_complete_timeout_ms" to 200

local_call_2_17-1-0.mp4

cutting_off_local_call_2.txt

[Lovenoreus]

I was able to call the container by filling out a config-local-example.yaml and deploying all the resources from that instead, deploying from config-remote-example.yaml causes the container to immediately not be able to connect to Redis.

[Lovenoreus]

I have called the container with the latest version but it still cuts all sentences short

[clemlesne]

I have called the container with the latest version but it still cuts all sentences short

As specified in the documentation, the algorithm use the result of the transcription in best efforts. Please fine-tune the recognition timeout values (https://github.com/microsoft/call-center-ai?tab=readme-ov-file#customize-the-conversation). You can start with recognition_stt_complete_timeout_ms and vad_silence_timeout_ms.

NB, the complete recognition frequently arrives after 500-1000 ms. Enlarging vad_silence_timeout_ms will constantly have a better impact if voice are always cutted.

[clemlesne]

I was able to call the container by filling out a config-local-example.yaml and deploying all the resources from that instead, deploying from config-remote-example.yaml causes the container to immediately not be able to connect to Redis.

By default, in both configuration examples, Redis is disabled. Redis is only enabled remotely, and obviously when you sync remote config locally.

[Lovenoreus]

I was able to call the container by filling out a config-local-example.yaml and deploying all the resources from that instead, deploying from config-remote-example.yaml causes the container to immediately not be able to connect to Redis.

By default, in both configuration examples, Redis is disabled. Redis is only enabled remotely, and obviously when you sync remote config locally.

Okey, but the container has problems connecting to Redis all the time

Connecting...
2025-01-23T15:46:05.64005 Connecting to the container 'call-center-ai'...
2025-01-23T15:46:05.70511 Successfully Connected to container: 'call-center-ai' [Revision: 'call-center-ai--7ssa3sx-7675fc5d79-59tmb', Replica: 'call-center-ai--7ssa3sx']
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/redis/asyncio/retry.py", line 64, in call_with_retry
raise error
File "/app/.venv/lib/python3.12/site-packages/redis/asyncio/retry.py", line 59, in call_with_retry
return await do()
^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/redis/asyncio/client.py", line 589, in _send_command_parse_response
await conn.send_command(*args)
File "/app/.venv/lib/python3.12/site-packages/redis/asyncio/connection.py", line 505, in send_command
await self.send_packed_command(
File "/app/.venv/lib/python3.12/site-packages/redis/asyncio/connection.py", line 466, in send_packed_command
await self.connect()
File "/app/.venv/lib/python3.12/site-packages/redis/asyncio/connection.py", line 296, in connect
await self.on_connect()
File "/app/.venv/lib/python3.12/site-packages/redis/asyncio/connection.py", line 365, in on_connect
auth_response = await self.read_response()
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/app/.venv/lib/python3.12/site-packages/redis/asyncio/connection.py", line 559, in read_response
raise TimeoutError(f"Timeout reading from {host_error}")
redis.exceptions.TimeoutError: Timeout reading from noreuscsgroupswe.redis.cache.windows.net:6380
2025-01-23T15:47:06.24189 No logs since last 60 seconds
2025-01-23T15:48:06.81058 No logs since last 60 seconds
2025-01-23T15:49:07.39059 No logs since last 60 seconds
2025-01-23T15:50:07.88842 No logs since last 60 seconds

[clemlesne]

Okey, but the container has problems connecting to Redis all the time

[…] in read_response raise TimeoutError(f"Timeout reading from {host_error}") redis.exceptions.TimeoutError: Timeout reading from noreuscsgroupswe.redis.cache.windows.net:6380
2025-01-23T15:47:06.24189 No logs since last 60 seconds

The logs you are showing are from Container Apps not from your local computer, if I’m not wrong. In Azure, Redis configuration is always defined.

Your Redis instance is maybe stopped or scaled to 0. That’s not an app issue.

[Lovenoreus]

I know that this might sound unfounded but essentially what I am claiming is that even though I deployed the resources twice with the same basic config.yaml bellow, same commit, container version, same code - the application in the container did not work yesterday and it did work today.

I realize that a purpose of containers is that they are supposed to be exactly the same, but right now I am assuming that something yesterday affected the execution that made the application not able to recieve phonecalls to it.

I hope that you this consideration is not rejected out of hand since I have had considerable difficulty with unexpected behaviour when I have set up everything multiple times.

logs_working_remote.txt

2025-01-23.21-06-40.mp4

conversation:
initiate:
agent_phone_number: "+46700229902" # Phone number the bot will transfer the call to if customer asks for a human agent
bot_company: Contoso
bot_name: Amélie
lang: {}

communication_services:
phone_number: "+4532727047" # Phone number purshased from Communication Services

sms: {}

prompts:
llm: {}
tts: {}

[clemlesne]

Issue created specifically for Redis: #407.

[clemlesne]

I realize that a purpose of containers is that they are supposed to be exactly the same, but right now I am assuming that something yesterday affected the execution that made the application not able to recieve phonecalls to it.

The container is exactly the same each time, with the same configuration. I can assure you.

But, as you destroy the infrastructure and pause some of it, DNS entries can be wrong (due to TTL on the DNS), and perhaps bug in Azure as start/destroy everyday the same resources with the same names if not a quality requirement of the engineering.

To ease your life, it would be better to let everything running. Just use the company tenant with a POC or innovation budget. You'll ease your mind.