Skip to content

50. Howto: add a new device to tuyaDAEMON

Jump to bottom
Marco Sillano edited this page Apr 28, 2024 · 17 revisions

Howto: add a new device to tuyaDAEMON

Step-by-step guide for real and virtual devices.

The process of discovering the dP (data points) of a Tuya device is presented here. Although we refer to TuyaDAEMON and tuyaDAEMON toolkit, which simplify things a lot, the sequence of necessary steps can also be done using just node-red and the tuya-smart-device node, in a small test flow: these instructions are also useful in other contexts.

1. Preconditions:

  • Your new device MUST work in smartLife app.
  • You must know the device ID and the device Key of your new device:
    • See 'Tuyapi' for a guide (see also Issue#547).
    • See also here for an alternative method (from Issuye#104).
    • [Since v. 2.2.4] The best way to get device ID and the device Key is:
    • [Since v. 2.2.4: obsolete] I suggest the download of tuyapi and tuyapi-cli and use them in 'wizard' mode: you will get the complete list of IDs and keys and cids of all your devices. Example:
PS D:\USER\documenti\_PROGETTI\2022\tuya> tuya-cli wizard
? Do you want to use these saved API credentials? 123456789abcdef 123456789abcdef eu Yes
? Provide a 'virtual ID' of a device currently registered in the app: 123456789abcdef 
[
  {
    name: 'Zigbee Gateway',
    id: '123456789abcdef',
    key: '123456789abcdef',
    subDevices: [
      {
        name: '3 Gang ZigBee Remote',
        id: '123456789abcdef',
        cid: '123456789abcdef'
      },
. ...more...
      {
        name: 'Temperature studio',
        id: '123456789abcdef',
        cid: '123456789abcdef'
      }    ]
  },
  {
    name: 'Smart IR 2',
    id: '56685573d8bfc0508488',
    key: '95e44e12dec93db9',
    subDevices: [
      {
        name: 'Screen pc',
        id: 'bf8b4b343edcf68869otv0',
        cid: 'b77a9d87fd94a8b3'
      }
    ]
  },
... more ...
  {
    name: 'Switch Watering main',
    id: '123456789abcdef',
    key: '123456789abcdef'
  }
]
  • devices in tuyaDAEMON:
    • Your device must be a device with the MQTT always ON ('real' devices, usually WiFi devices with USB power).
    • Or your device uses a gateway: e.g. a battery-powered ZigBee sensor: The device becomes a 'subdevice' (or 'virtual' device) of the gateway.
    • If not, you can't connect the device LOW-LEVEL.
      • With TuyaDAEMON you can use it, you must create a 'mirror' device using TRIGGERS.
      • See as a guide the 'smoke detector' device (obsolete) [Since v 2.2.4] Using core_OPENAPI see the new example.
      • As an alternative, since v. 2.2.4, you can use core_OPENAPI, and the pseudoDP _APIstatus to read in polling data from Tuya Cloud.
    • note: the use of ZigBee devices (and gateways) has several advantages:
      • Transforms 'mirror' devices to 'virtual' devices, because the gateway is always active.
      • Reduces the WiFi requirements: many WiFi APs are limited to 20-30 devices, and this number is reached quickly (ZigBee gateway limits: 30..100).
      • Can extend the WiFi coverage, using a ZigBee repeater for nearby devices (see ZigBee and Wi-Fi Coexistence).
  • On node-red, install the last version of 'node-red-contrib-tuya-smart-device' module, the simple way is using 'Menage palette' on main node-red menu.
    • I recommend using a slightly modified version of node-red-contrib-tuya-smart-device: for ver. 4.1.1 see ISSUE#83, for ver. 5.0.1 see ISSUE#113. Replace only the file <nodered>\node_modules\node-red-contrib-tuya-smart-device\src\tuya-smart-device.js.

2. tuyaDAEMON.CORE flow modifications

  • A new node is added using the BASIC wiring. You can fine-tune the device behavior later with the use of subflows like: explode, RT/AVG filter, REFRESH, and dynamic retry.

  • Add (or copy) a new tuya-smart-device node.

    • Set new node properties:
      • set device id, device key, and device name (the name can be any, it is not used by tuyaDEAMON).
      • set 'interval for retry' and 'interval to find' to a big enough unique prime number, this solution reduces the load for devices that are temporarily not connected

  • Select or duplicate the node 'static retry' (static retry is a subflow, you can find it in palette:subflows):

    • Config the 'static retry' property: 'deviceId' using the new device 'id'.

  • Edit the properties of the existing switch node '*device selector':

    • Add a new rule: msg.toDev == <device_id> for the new output.
    • note: the <device_id>|<device_cid> is used as the primary key to access the devices. For this reason, it is used in several nodes, where possible in fields of type 'credentials', unfortunately not in the '*device selector' node

  • Connect the '*device selector', tuya-smart-device, and 'static retry' nodes as shown in the figure (BASIC case).

  • Later, add the new tuya-smart-device node from the list in the 'catch device and LAN errors' node, so all useless and not caught messages from devices (see tuyaDAEMON CORE) will not go to debug pad.

  • To tests if your device accepts the 'REFRESH' operation (see 14), you must add a 'REFRESH' node, see REFRESH operation.

    • Config the 'REFRESH property: 'deviceID' using the new device 'id'.

note: All tuyaDAEMON nodes having an asterisk on the name (e.g. '*real device selector') CAN/MUST be modified by the user in configuration, to add devices or for fine-tune tuyaDEAMON.

note: a device has multiple names, used in different contexts (by default they can also be the same):

  1. The SmartLife name, is used in the APP, and returned by tuya_cli wizard. Not used in TuyaDEAMON.
  2. The node-red node name, is used only by node-red to identify the node-red-contrib-tuya-smart-device node.
  3. The TuyaDAEMON user-defined name, in global.alldevices. Rules:
    • used in external outputs: global.tuyastatus, debug pad, etc..
    • used in commands, shares, etc, in place of the ID (the "ID" is not portable, the "name" is portable only if predefined).
    • can almost always be changed by the user at any time (any language accepted).
    • if a name cannot be changed (predefined or for whatever reason) that name starts with an underscore (e.g. "_system")
    • for MQTT compatibility, avoid the chars '%', '$', '+', '#' in names.

3. Connecting the device

  • If you are lucky and your device is on the list of known devices, nothing to do: jump to step 20.

  • Else you must start a long discovering process of the new device (see tuya devices capabilities as currently known for all options). You will use the tuyaDAEMON messages, and smartLife app, and, optionally, tuyadaemon-toolkit to get all useful data point of your new device:

    • tuyaDAEMON is running, all 'info' and 'trace debug' nodes are enabled.
    • Your new device looks 'connected' on the CORE flow.
    • Clear the debug pad.

  • Using smartlife send a simple command to the new device.

  • In the debug pad you must see:

    26/12/2020, 20:09:39node: updates global.tuyastatus, formats SQL'`
      function : (warn)`
        "ADD to alldevices the real deviceid: 12345678987654321abc"`
    or: 
        "ADD to alldevices the virtual device cid: 1234567321abc [gateway:12345678987654321abc]  `

4. Good, now you must add your device to global.alldevices to advance further:

  • Edit the node 'Config' and open alldevices.
  • Don't delete the actual data.
  • For real devices, you must add your device in the 'real' branch.
  • For virtual devices (i.e. using a gateway: the 'gateway' itself must be present in the real section), you must add them in the 'virtual' branch.
    • 'Id' is the device id you get doing "tuya-cli wizard" (optional, not used).
    • 'name' is your name to identify the device in TuyaDAEMON (note: the tuya-cli wizard returns the name from the SmartLife APP, not used).
    • 'cid' is the id used by the gateway. (only for 'virtual devices)
    • 'gateway' is the id of the associate gateway real device. (only for virtual devices)
Example for 'real' device in the 'real' branch:
   {
       "id": "12345678987654321abc",
       "name": "<new-device-name>"
   },

Example for a 'virtual' device in the 'virtual' branch:
   {
       "id": "12345678987654321abc",
       "name": "<new-device-name>",
       "cid": "bcdef9876543",
       "gateway": "987654321012346789ab"
   },

5. That is minimal, but enough for now.

  • Writing 'alldevices' you must pay attention to commas, and use [format ISON] to check your data. Save and Deploy._
  • Resend a command using smartlife, in the debug pad you will see:
 "In alldevices missed the '<new-device-name>':dps[<dp-x>] definition."`

[Since v. 2.2.4] Using 'core_OPENAPI' , the following steps 6, 7, and 8 are obsolete: but you can still use them if you don't want to use Tuya OpenAPI. For a faster and more reliable result, I strongly suggest using core_OPENAPI (or the standalone testing solution) and the instructions indicated in Steps 1, 2 here.

6. Now you know a dp ( = 1, or 2, or 101... ), associated with the command sent via smartlife.

  • You can edit again the node 'Config' and open alldevices: edit the list, you must add the new 'dp-x' data point (dp).
  • For 'new-x-value-name' it is better to use the same name used in _smartlife, if it exists. Use 'unknownXX' for data-point in the study phase.
  • Save, deploy.
Example for 'real' device with two data points:
  { 
      "id": "12345678987654321abc",
      "name": "<new-device-name>",
      "dps": [
          {
              "dp": "<dp-x>",
              "name": "<new-x-value-name>"
          },
          {
              "dp": "<dp-y>",
              "name": "<new-y-value-name>"
          }
      ]
  },

7. Resend the same command using smartlife

  • in the debug pad you can see the standard info from tuyaDAEMON:

note: in the case of complex objects, you can expand the result.

   27/12/2020, 13:02:09 node: info  
      ["RX: 'power strip'/'Switch1' ", "OFF"]

8. The first data point is found and added!

  • Well: now you must repeat steps 5, 6, and 7 for all commands and config options you fund in smartlife.
  • Because the device works, it is not necessary to update any time alldevices, you can only write them with the dp_nomes on a piece of paper (see next step 9).
  • "There is no way to autodetect dpIds and their functions" (says Tasmota about TuyaMCU).
  • A big help to get DP id, names, and values now (2023) can be done by Tuya: see Find Tuya data points.
  • See also this info.
  • You must keep calm: I found a TRV_Thermostatic_radiator_valve with 28 data points!
  • You can also keep running tuyaDAEMON for 24h, then you can explore the DB for all the records with 'undefined' or 'unknown' 'dp': you can find the new push-only data points.
  • The value you get can be more complicated than '7' or 'colour': maybe it is coded. Don't care about it for now.

9. Now you have a collection of 'dp' and 'dp_nomes' for your device.

Time to go further on exploration, with some helps to better profile data points capabilities. Any Tuya device is different, they are built by different constructors, so you must expect any strange behavior. The unique way is the test and reverse-engineering (see also devices capabilities):

You can do this by hand, but I make tuyaDAEMON toolkit as a helper for this phase of function discovery and reverse engineering. You can use it.

  • Using the toolkit, do STEP 1 and create the records for the new device and all found dp, using all info collected just now.

  • Create your first 'alldevices' JSON fragment for your device (STEP 3): it must be minimal, with all 'dp' but without any 'capability': for tests, we will use the defaults ('ALL' for device and 'RW' for all 'dp').

  • Use this JSON fragment to update your 'alldevices' on tuyaDAEMON: so you can test the device without warning messages.

  • Explore the data point capabilities with STEP 2. Looking also to tuyaDEAMON debug pad. You can get many different answers (see also Tuya device capabilities:

    • [test GET/SET] good answer: boolean, or plain number or string
      • simplest case: true|false or a number '7' or a string 'colour'. ok.
    • [test GET/SET] good answer: code64 string (this is the more common, but you can found also different codes)
      • A string like 'BAAAAKABpADIA94A0gTsAKU=' maybe is coded. Some tests are required to re-engineering the codification used.
      • Copy the string and use the small flow 'TEST UNKNOW RESPONSES' (in 'extra.test' flow): you can test if data are binary and coded using 'code64'.
      • Every coding type is defined by a 'typefield' and by two functions: encode() and decode().
      • Since 2.2.0, the decode() and decode() functions are in the CORE '*ENCODE/DECODE user library' node.
      • You can write your own decode() and encode() functions for a new typefield, and place it on the library.
      • Add the new typefield also in the DB 'lookupdecode' table (see tuyaDAEMON.toolkit).
Example of binary data coded 'code64': (typefield: 'STRUCTARGETTEMP') 
from 'temperature day planning', used by 'TRV_Thermostatic_radiator_valve' device:

    data: BAAAAKABpADIA94A0gTsAKU=   [base64(Uint8Array[17])]
    
    decoded, JSON: 
            {
               "count":4,
               "changes":[
                    {"time":"00:00","temp":16},
                    {"time":"07:00","temp":20},
                    {"time":"16:30","temp":21},
                    {"time":"21:00","temp":16.5}
                ]
            }

10. [test GET/SET] Bad response 1: 'data hex'

  • That minds an internal problem: tuyapi library as failed the data decoding to a JSON standard structure.
  • In all cases that have happened to me, the cause was an incorrect key. This happens also when you disconnect and reconnect a device in the SmartLife app. Redo "tuya-cli wizard" and update the key in the tuya-smart-device node. Example: see ISSUE#407.
 Example:
    function : (warn)`
        "ERROR from <new device name>: data hex:332e330019001efbfb...  (3.3��...)"

10bis. [test GET/SET] Bad response 1bis: 'devid not found'

  • [2023] In all cases that have happened to me, the cause was an incorrect cid in a command for a gateway. This happens also when you disconnect and reconnect a device in the SmartLife app. Redo "tuya-cli wizard" and update the cid in the global.alldevice JSON structure.

11. [test GET/SET] Bad response 2: 'json obj data unvalid'...

  • This message refers usually to the data provided by the device, and not to the data in input for the tuya-smart-device node. You must eval the message as 'command not implemented' by the device (see issue338).
  • Usually, it is about 'GET' and 'SCHEMA' commands, and a tuyaDEAMON user doesn't need really them: the global.tuyastatus structure stores all the last information from devices.
  • If this response is caused by a 'GET', try a 'SET' with 'NULL' data. Sometimes it works. If it is ok, you can use 'WW' or 'GW' as the capability for this 'dp': tuyaDEAMOM will replace all 'GET' with 'SET:null' (see alldevices, note#13).
  • [2023] I have also found cases in which a device hangs up for an incorrect request, and from that moment it always gives incorrect responses which generate the message 'json obj data unvalid'.

12. [test GET/SET] Bad response 3: Null response, or error timeout from the device or the gateway (see Tuya device capabilities):

  • Also consider this as 'command not implemented', See 11.
  • Using TuyaDAEMON, CORE is responsible for msg correctness, but doing tests with ad-hoc node-red flows, check the msg JSON format several times, and check that the dps is always a string, while the set value can be boolean, number, string (same as the type returned by GET) or null. The complete syntax is in CORE.'Low level IN' and CORE.'to logging' nodes; simple cases:
      {                                 //  msg for a GET
       "payload": {                    
           "cid|devId": "60a453fffe******", //cid, only for subdevices of a gateway (string)
           "operation":"GET",
           "dps":"103"                  // DP or pseudoDP (string)
           } }

       {                                //  msg for a SET
        "payload": {
           "cid|devId": "60a453fffe*****",  // cid, only for subdevices of a gateway (string)
           "dps":"103",                 // DP or pseudoDP (string)
           "set":220                    // null, BOOLEAN, INT/ENUM, STRING
           } }

Example using TuyaDAEMON:

13/2/2021, 12:26:09node: 5a17a058.f17be
msg : string[72]
"Timeout waiting for status response from device id: 554301602cf4********"

13. [test SCHEMA/MULTIPLE] good answer:

  • green button (see also at the debug pad). This is for completeness, you don't really need it using tuyaDAEMON.
  • take care of SET MULTIPLE values: they must be coded if required.

14. Some devices accept also a REFRESH command.

  • Send it and see at the debug pad: the good response is a new data message for every|some REFRESH. You can try also (but using ad hoc nodes and 'low_level_IN' input node) the messages: {operation: REFRESH, dps:20} or {operation: REFRESH, schema: true}, see ISSUE#407 and ISSUE#469

15. complete other fields for a device (see global.alldevices info):

  • On a device definition, to filter only the allowed requests, is used the "capabilities" value (see alldevices, note#7): - Array [ one or more of ('SET','GET','SCHEMA','MULTIPLE')|'NONE'|'ALL', (REFRESH)] (default ALL)
  • You can classify the real devices using the type of power supply (see alldevices, note#7): - values: 'AC'|'BAT'|'UPS' (default 'BAT') (see alldevices, note#11): - note: if missed 'power' the device is skipped by '_system' tests.
  • URLs will be encoded: in case of multiple URLs use as separators only commas ',' and no spaces.
  • Do not use the '"' (double quotation mark) char in any tuyadaemon-toolkit field. In some fields, unallowed chars are replaced automatically.

16. complete other fields for data points (see global.alldevices info):_

  • On the single dp, you define a "capability" that filters the user access (see alldevices, note#13):
    • values: 'RO' |'WO' | 'GW' |'RW' |'WW' |'PUSH'|'TRG'|'SKIP' (default RW)
      • note WW: the 'GET' is replaced by 'SET:null'. SET allowed (same as 'RW').
      • note GW: the 'GET' is replaced by 'SET:null'. Other SET not allowed (same as 'RO').
      • note PUSH: the value is only sent autonomously by the device - the user can't do 'GET'/'SET'
      • note TRG: the value is reserved to TRIGGER - the user can't do 'GET'/'SET'
      • note SKIP: do not send to the device.

17. You finish when all 'dp' are tested and doing a [auto ALL TESTS] you get all green buttons.

  • Maybe now you like to set some fields to a different 'production' value (e.g. 'WW' in place of 'RW').
    • Create your final 'global.alldevices' JSON fragment (STEP 3) and copy-paste it to do the final update of tuyaDAEMON.
    • Do 'SAVE' in tuyaDAEMON to get a backup copy of your complete 'global.alldevices' structure.

18. custom device

  • Do the same for your custom devices (see as an example _system)
  • I use tuyaDAEMON.toolkit also in the design phase of a custom device: the documentation is a good trace in implementation.

19. Share your results, on tuyaDAEMON toolkit:

  • Do STEP 3, to create the <new-device-name>.json file

  • Do STEP 4, then load the <new-device-name>.odt file in OpenOffice4, and export it as <new-device-name>.pdf file.

  • Do STEP 5, then send the zipped file. I will add it ASAP.

    Enjoy your device in tuyaDAEMON.

20. You are lucky, your device is known, and all you must do is very simple:

  • Download the <your-device>.zip file
  • Unzip it and use the ready <your-device>.json file to update your 'global.alldevices' in tuyaDAEMON.
  • If you like, use the <your-device>.sql file to update the dB and to use tuyaDAEMON toolkit on your new device.

devices

Watchdog02 (custom)

Clone this wiki locally