Zigbee Examples are not fully working with Hubitat and HomeAssistant with ConbeeIII zigbee dongle

[ilker-aktuna]

Board

esp32-c6 dev module

Device Description

esp32-c6 with zigbee

Hardware Configuration

zigbee end device

Version

latest master (checkout manually)

IDE Name

Arduino IDE

Operating System

Windows 11

Flash frequency

40

PSRAM enabled

yes

Upload speed

115200

Description

I am trying the Arduino zigbee examples with esp32-c6 board.
I used the on/off switch , color dimmer.
I also tried the motion sensor template from this post:
#10684

On my Hubitat C7 and C8 hubs, the device is being detected. Then sometimes, it completes the pairing , sometimes gets stuck at pairing.
If the pairing completes, sometimes I can control the device (on/off , or level commands)
Sometimes, it does not work. If I start pairing again on Hubitat, it can control the device (while pairing is active) but then starts failing.

What I see on Arduino logs is that the device restarts pairing.
if I put the log level to "verbose" , I get this line before restart:

13:08:27.383 -> Connecting to network
13:08:27.383 -> .............................[ 5289][V][ZigbeeCore.cpp:344] factoryReset(): Factory resetting Zigbee stack, device will reboot
13:08:30.300 -> .ESP-ROM:esp32c6-20220919

I see in the code:

case ESP_ZB_ZDO_SIGNAL_LEAVE:  // End Device + Router
      // Device was removed from the network, factory reset the device
      if ((zigbee_role_t)Zigbee.getRole() != ZIGBEE_COORDINATOR) {
        Zigbee.factoryReset();
      }
      break;

But Hubitat does not remove the device. So how does it go into that ?

@P-R-O-C-H-Y was trying to help me identify the problem before Christmas. But for the last 3 weeks I didn't receive anything from him.
I hope he's OK.

I also tried the same device with same examples on a friend's HomeAssistant (Raspberry PI) with Conbee III zigbee dongle.
On HE, pairing never completes. I assume these examples are not tested on 3rd party devices. So this issue is not specific to me and would occur on most of the setups with 3rd party Zigbee devices.

pairing_2.zip

Btw, @P-R-O-C-H-Y had asked me to get a zigbee sniff using another esp32-c6 and sent me a guide. Following the guide, I was able to get the attached capture. I am not familiar with Zigbee sniffs so I'm not sure if this helps.
If anything else is required, I'd be glad to capture.

About the attached traces:
Hubitat actually detects the device and pairs successfully. But the device (esp32) reports it has failed.
So the first file is capture when my hub has already paired and shows a motion sensor.
Then I removed the sensor from zigbee network (Hubitat) and restarted capture (and esp32) to record a full failing scenario.
That is the second capture file. I hope it is clear for you and you can see the problem in these traces.

pairing_2.zip

Sketch

zigbee ED examples
last tested the following one (Occupancy Sensor):
https://github.com/espressif/arduino-esp32/pull/10720/commits/77df256f5db51c139715cc73927dccd38d652cfb#diff-03e926d0f636dc753b1e0aad6f247e66376e60289d89800dde02525f747d2d62

Debug Message

-

Other Steps to Reproduce

No response

I have checked existing issues, online documentation and the Troubleshooting Guide

• I confirm I have checked existing issues, online documentation and Troubleshooting guide.

[P-R-O-C-H-Y]

Hello @ilker-aktuna, don't worry I am fine, but there were a lot of issues and ongoing tasks after Christmas vacation, so I did not get yet to your issue. Sorry about that.

I am using the HomeAssistant running on a RPi, and I am using the SONOFF Zigbee 3.0 USB Dongle Plus. I did not face any issues with this combo. Also there are other guys with other 3rd party Dongles running a Zigbee2MQTT or HA successfully.
We cannot guarantee that all combinations or providers will work 100%.

I will check the logs from the sniffer and try to figure out what is happening there.

Regarding:

But Hubitat does not remove the device. So how does it go into that ?

It must have and a leave command, otherwise this won't happen. Let me check the logs :)

[ilker-aktuna]

ok. I am here if you ask for more zigbee sniff.
Also my friend will prepare a zigbee sniff setup with his HA + Conbee III dongle.

[P-R-O-C-H-Y]

@ilker-aktuna Will you be able to try with a esp-zigbee-sdk example? To check if there is something in the Zigbee library missing (or not working) or the same behavior is when using the esp-zigbee-skd directly. You can check the examples here. Just one notice, currently the esp-zigbee-sdk latest version is 1.6.2, in Arduino we use for now the 1.6.0 so that may be the only difference. But I plan to update it soon.

[ilker-aktuna]

@P-R-O-C-H-Y
Do you want me to just use the examples from SDK ? Or do you want me to update the esp32 version on my Arduino IDE , then use them ?

[P-R-O-C-H-Y]

@ilker-aktuna Please you their example directly. So you need to install ESP-IDF and build/flash the esp-Zigbee-sdk example.

[P-R-O-C-H-Y]

@ilker-aktuna Also another test you can do is add esp_zb_secur_link_key_exchange_required_set(false); here before the esp_zb_start();

arduino-esp32/libraries/Zigbee/src/ZigbeeCore.cpp

Line 97 in 2a2b81a

ESP_ERROR_CHECK(esp_zb_start(false));

Can you give it a try? Thanks

[ilker-aktuna]

@P-R-O-C-H-Y
I am on vacation at my winterhouse and I don't have my main computer with IDF with me. I have a less powerful notebook with Arduino IDE. I can't install IDF at the moment to this notebook.
But I have my sniffer and 2 spare esp32c6 with me.
So I can do the test on your last post.
Is it ok if I just edit the cpp file ? Should I do something to compile it before building the Arduino sketch ?

Btw, you had asked me to use your unmerged code from P-R-O-C-H-Y:feat/zigbee-multi-sensor so I am still using it. Should I update ?
On my ZşgbeeCore.cpp , following line is not on 97 but on 98 :
ESP_ERROR_CHECK(esp_zb_start(false));

[P-R-O-C-H-Y]

@ilker-aktuna The best will be to use the master branch of ESP32 Arduino Core for any future ongoing work :)
There were other fixes merged to the Zigbee library already.

Is it ok if I just edit the cpp file ? Should I do something to compile it before building the Arduino sketch ?
Yes, just edit the .cpp file and build/upload through the Arduino IDE.

[ilker-aktuna]

I started with the simplest example (on/off switch)
it seems to work fine.
Moving to the dimmer...

[ilker-aktuna]

Dimmable Light works as well.
And it is stable. It works even after pairing has stopped and even after power off/on.
To see if the solution was "esp_zb_secur_link_key_exchange_required_set(false);" , I removed it and recompiled.
It then fails completig pairing.
After setting it again, it works.

I also tried color light now. On/off and level commands work but color does not.
As described in this issue:
#10669

I guess my Zigbee hub (Hubitat) sends only Hue&Sat values.
But the examples do not include anything with hue and sat. How can I make the device receive HSL ?

Do you have any update on that ?

I will try the motion sensor when I'm back home. I don't have the sensor with me because my friend is keeping it :(

[ismetfaik]

Dear @P-R-O-C-H-Y , I have also tested Arduino Zigbee examples by adding the line "esp_zb_secur_link_key_exchange_required_set(false);". All three On/Off Light, Dimmable Light and Color Dimmable Light examples are working with the setup: RPi2 + Conbee III <-> Waveshare ESP32-C6 Zero.

My question is is there any security / vulnerability consequences of adding that line?

Thank you for your efforts and help.