Unable to enter (timeout) bootloader mode on (Ryzen 3500U platform - HP Elitebook 735 G6) (ESPTOOL-144)

[vulptex]

#46 Operating system

• Win 10 64bit
• Linux 4.9.x LTS or 5.5.x

Python version

Can run python -V to check this.

• Win 10 64bit -> 3.8
• Linux 4.9.x LTS or 5.5.x -> 3.6

What Chip

ESP32-PICO-D4 and ESP32-WROOM32

This only happens with esp32, esp8266 is working.

What development board or other hardware is the chip attached to

NodeMCU, Thing32, Custom

No matter if FTDI or SiLabs CP2102. Only enterable by luck when holding boot.

Are you running esptool.py from an IDE such as Arduino or Eclipse?

No IDE, Windows command prompt, Linux shell

Full esptool.py command line that was run:

esptool.py run

Full output from esptool.py

Do you have any other information from investigating this?

Issue reproducible on two identical models of:

HP Elitebook 735 G6 (always timeout, no chance to enter bootloader mode)

On any other Laptop with Intelchipset -> 0 issues on all mentioned boards

windows.log

[projectgus]

Hi @vulptex,

That's very odd! So you've tried with different USB/serial chip types, and different OSes, and the result is always 100% the same?

Could you please run esptool.py --trace -p PORT flash_id (in Linux ideally but any OS is useful info) and post the full output?

Changes are this is nothing we can fix from the esptool or even the pyserial (esptool's serial interface) layer.

[vulptex]

Will do that. It's definitely very weird, but it is 100% reproducible. Tried on a a third HP Laptop with AMD Ryzen 3500U. I understand that you cant solve that, but i wanted to list it up here for documentation purposes. Would be great if the community can double check.

[vulptex]

Update:

Verified the issue with:

• Lenovo E495: Ubuntu 19.10 (Ryzen 5)
• Lenovo E495: Windows10
• 8266 okay, esp32 fails

This is a serious issue, as we cant flash any of our devices. AMD becomes our preferred supplier for development machines.

Log from Windows:

windows.log

[Maniac235]

Have the same issue;-)

Hardware and esp32 are now useless.
ThinkPad E495
-Win10
-Ubuntu

Urgent fix is needed. If it take long I have to change MCU type.

[vulptex]

Have the same issue;-)

Hardware and esp32 are now useless.
ThinkPad E495
-Win10
-Ubuntu

Urgent fix is needed. If it take long I have to change MCU type.

It's indeed urgent , would be great if the community can do a mass-test. Btw all other vendor boards work great: ti, nxp, stm, silabs. Hopefully that is not another minus for the esp32, I guess it's solvable as it's not happening for 8266.

[Maniac235]

Log from Ubuntu on E495 Machine:

ESP_Tool_Ubuntu_E495 trace.txt

[vulptex]

@Maniac235 I try to get my hands on a x395 as well, my Linux traces look the same. Out of curiosity, why do you give my comments a thumbsdown ? :-)

[Maniac235]

@projectgus can i help with more input?
Did have a look on the PCB (NODE-MCU ESP32-Wroom), while pushing the "BOOT" button for 2-3 sec. esptool is able to flash. Maybe a problem with DTR/RTS.

[vulptex]

I guess you guys need to dig deeper asap. I don't get why this only affects esp32? I am really willing to help you.

[projectgus]

Hi @Maniac235 @vulptex ,

There is some conflicting information in the logs but the root cause seems to be the same:

@vulptex's trace log shows that the ESP32 is resetting but it's resetting into the default "flash boot" mode not the bootloader. If you can find a way to hold GPIO0 low (either by pressing a BOOT button or just tying GPIO0 to GND) then the chip should flash.

@Maniac235's trace log shows the ESP32 isn't resetting at all, no output is being received at all. However this update from @Maniac235:

Did have a look on the PCB (NODE-MCU ESP32-Wroom), while pushing the "BOOT" button for 2-3 sec. esptool is able to flash. Maybe a problem with DTR/RTS.

... suggests that it's the same solution asfor @vulptex's problem (pressing BOOT button will pull GPIO0 low, meaning the board resets into the right mode.)

---

If that's the case (both are failing to reset into the right mode) then it is probably something to do with DTR/RTS timing as suggested. Maybe there are some additional delays between sending USB packets to change the pin state is being introduced by the USB host controller chipset.

You can try adding additional capacitance between EN and GND (maybe a 1uF capacitor or similar) and see if automatic reset starts working.

I don't get why this only affects esp32

ESP32 is a little faster to start up, so if there is a short delay between EN going high and GPIO0 going low then it can cause it to fail to reset into the right mode. Background info about boot modes is here.

---

I think to do any further debugging would require some captures from a logic analyzer on the serial lines. Capturing TX, RX, DTR and RTS as seen by the USB/serial chip would be the best place to start. Don't need a particularly fancy logic analyzer, capturing at 250K samples/sec should be enough, 1M sample/sec definitely enough.

[kiss81]

Hi @Maniac235 @vulptex ,

You can try adding additional capacitance between EN and GND (maybe a 1uF capacitor or similar) and see if automatic reset starts working.

I do have the same issue on my ryzen 3700x system and a esp32 pico d4 project.
I added a 2.2uF between the EN and GND and it solved the issue. (don't have a 1uF laying around here)
In my project I think I can swap the 100n capactor that was there with a 1uF or 2.2uF and I guess it will be fine.

It would be nice though if there is a fix possible within esptool.py as there are lot's of projects and development boards that will need a hardware fix I am afraid.

edit: I tried fidling around a bit with short sleeps in the esptool.py but no luck yet. I think I need a scope to check the timing issue and see if I can solve it in software

[Maniac235]

Hi @Maniac235 @vulptex ,

There is some conflicting information in the logs but the root cause seems to be the same:

@vulptex's trace log shows that the ESP32 is resetting but it's resetting into the default "flash boot" mode not the bootloader. If you can find a way to hold GPIO0 low (either by pressing a BOOT button or just tying GPIO0 to GND) then the chip should flash.

@Maniac235's trace log shows the ESP32 isn't resetting at all, no output is being received at all. However this update from @Maniac235:

Did have a look on the PCB (NODE-MCU ESP32-Wroom), while pushing the "BOOT" button for 2-3 sec. esptool is able to flash. Maybe a problem with DTR/RTS.

... suggests that it's the same solution asfor @vulptex's problem (pressing BOOT button will pull GPIO0 low, meaning the board resets into the right mode.)

If that's the case (both are failing to reset into the right mode) then it is probably something to do with DTR/RTS timing as suggested. Maybe there are some additional delays between sending USB packets to change the pin state is being introduced by the USB host controller chipset.

You can try adding additional capacitance between EN and GND (maybe a 1uF capacitor or similar) and see if automatic reset starts working.

I don't get why this only affects esp32

ESP32 is a little faster to start up, so if there is a short delay between EN going high and GPIO0 going low then it can cause it to fail to reset into the right mode. Background info about boot modes is here.

I think to do any further debugging would require some captures from a logic analyzer on the serial lines. Capturing TX, RX, DTR and RTS as seen by the USB/serial chip would be the best place to start. Don't need a particularly fancy logic analyzer, capturing at 250K samples/sec should be enough, 1M sample/sec definitely enough.

Hi did connect a Logic Analyzer:

1. Without connecting ESP32 NODEMCU GND to Logicanalyzer same behavior
2. With conneted GND no problem

CH1-->EN
CH2-->GPIO_0
CH3-->TX GPIO 1
CH4-->RX GPIO 3

Both logs from Consol and logic analzer attached, if you need more let me know.

Best regards
ESP32_S_NODE_MCU_RYZEN.zip

[projectgus]

Hi @Maniac235 ,

Thanks for being patient while I got time to look at the captures.

Hi did connect a Logic Analyzer:

Without connecting ESP32 NODEMCU GND to Logicanalyzer same behavior
With conneted GND no problem

I don't really understand what this means. To use a logic analyzer, the ground probe from analyzer should always be connected to the device you're testing. Are you also saying that when you connected this GND to your NODEMCU board, the problem with esptool went away?

If so, the NodeMCU board itself may be faulty (perhaps GND on the NodeMCU board is not properly connected to the USB socket ground, so instead you closed the circuit via the Logic analyzer.)

In the captures:

• "ESP32_S_NODEMCU_ERR_24 MHz,1 B Samples [5]" shows the EN pin going high to un-reset the board around 1.5ms before the GPIO0 pin goes low to select bootloader mode. These two events need to happen closer together. Later on in the capture the UART output confirms the boot mode is "SPI_FAST_BOOT" not the UART download mode:

• "ESP32_S_NODEMCU_GND_Connected_Working_24 MHz,1 B Samples [5]" this trace shows two connection attempts that both failed (same reason). I am guessing maybe one of the connection attempts after this worked, but the Logic Analyzer had stopped capturing already.

If you haven't tried adding a capacitor of approx 1uF value between the EN and GND pins of your NodeMCU, suggest that this will probably fix the problem permanently. If this works and you're able to tell the seller of the board that the design is missing this capacitor, please do!

[vulptex]

@projectgus

Thanks for your answer.

Soldering caps or HW modifications are now option for me, as the devices are already shipped to customers (> xxx). The problem is, the service team is equipped with AMD laptops now, and are unable to flash the devices anymore 😨. Luckily we are stable enough, so no need so far for on-site flashing :-).

Any other ideas? When i have access to another logic analyser (mine is broken here a @ home) i will produce timing diagrams again for my device.

[projectgus]

Hi @vulptex ,

Sorry for slow reply. I'm not sure what else can be done, although more logic analyzer traces would be invaluable as there isn't a "smoking gun" yet for whatever the AMD USB Host controller seems to do differently with respect to timing.

A USB analyzer capture might also be useful, but would need to be from a dedicated hardware device to see what's going "on the wire".

Again though, it's unlikely that it's possible to fix this from the esptool side if it's something with USB Host timing.

Are you able to post a schematic of the auto-reset & USB/serial part of your hardware, by the way?

[vulptex]

Hi @vulptex ,

Sorry for slow reply. I'm not sure what else can be done, although more logic analyzer traces would be invaluable as there isn't a "smoking gun" yet for whatever the AMD USB Host controller seems to do differently with respect to timing.

Are you able to post a schematic of the auto-reset & USB/serial part of your hardware, by the way?

Hi @projectgus,

please find attached the schematic:

btw, this is reproducible on all devkits i have, including: thing32, nodemcu v1, ...

[projectgus]

Hi @vulptex,

Thanks, that's useful. Is there anything else connected to the RESET_N net apart from the EN pin of the ESP32?

btw, this is reproducible on all devkits i have, including: thing32, nodemcu v1, ...

It would be interesting to know if any Espressif devkits exhibit the problem (or if the capacitor fix works on any of these boards). Many the third party ESP32 dev boards continue to leave off the capacitor on the EN pin that's needed to slow the rise time, because in this setup the board works with most host hardware and OS setups. But not all of them...

[vulptex]

@projectgus

[vulptex]

@projectgus Anything else i can provide?

[projectgus]

Hi @vulptex,

Sorry I didn't reply to the other schematic post.

As mentioned above, a logic analyzer trace showing the RTS/DTR signals and the corresponding EN/GPIO0 signals would be very valuable. I don't see anything in the schematic that looks like an issue.

Unfortunately thoughh, I don't think this is going to be something that can be fixed from the esptool or pyserial side.

Angus

[vulptex]

Hi @projectgus,

understood. :-(

Maybe you should think of getting rid of python at all? But judging from the latest changes in esp32-idf and the rest of your SW stacks, you are already tied to the python hell :-D. ironic

[projectgus]

Maybe you should think of getting rid of python at all?

My point isn't that this is somehow a problem with Python, quite the opposite. It looks like it's a problem with the platform USB host driver's timing behaviour (possibly interacting with the USB to serial driver). And if that's the case then nothing we change in the application software is going to be able to fix it.

[vulptex]

@projectgus i understood that 👍 . And i know its a issue with AMDs USB host somehow..but i am unable to find the root cause nor can i fix it or have the time to dig into the bootloader and timing.

To be honest, i think it is the responsibility of espressif to fix that:

1. get a AMD based system
2. reproduce
3. fix

[projectgus]

I'm sorry that your design has the automatic reset circuit from our development board designs without the manual button option, and that you're in a tough situation as a result.

However if the root cause is that some AMD chipsets have unusual USB host timing then the only fix for the automatic reset circuit used on Espressif development boards is to hold down the BOOT button while flashing them. We're not going to redesign our development boards for one buggy chipset, and even if we did then I'm afraid it wouldn't help you very much.

[vulptex]

@projectgus okay, so what is the take away? No esp32 based products with AMD based developer machines?

[vulptex]

This can be closed. We decided to switch soc manufacturer for the next generation of our products.

[kiss81]

This can be closed. We decided to switch soc manufacturer for the next generation of our products.

That's a bit more work compared changing just one capacitor... I am not sure this issue should be closed actually.

[projectgus]

For anyone still struggling with this problem, the linked IDF issue above mentions a different Ryzen chipset and that they worked around automatic reset issues by inserting an external USB2 hub between the USB3 host port and the ESP32. If you try this and experience success/failure, please report back.

(It's not clear this is exactly the same issue, a number of our developers now have Ryzen laptops with different chipsets and no problems so this specific issue may just be something about the 3500U chipset, and the linked issue may be something with Windows or WSL.)

[dobairoland]

I've tried to reproduce unsuccessfully this issue with my Lenovo Ryzen laptop which I use regularly to connect with various ESP chips.

I'm closing this because it is clearly not an ESPTOOL issue and I think nothing else could be added to this topic.

[HankLloydRight]

Sorry for the bump, but just had to post the solution that worked for me. I just got a new Ryzen 9 7940HS computer, and this bug was driving me nuts trying to flash the LOLIND32 boards I'm using. I tried all the solutions posted here on Github (1uf or 2.2uf caps on EN to GND, a dozen different USB cables, diff USB ports, and two different USB-2 hubs) -- and nothing fixed the issue.

I found a post on Reddit that suggested putting a 10uf cap between RST and GND, and that finally solved the issue for me.

I tried both 10uf ceramic and electrolytic caps and both worked, although I think the electrolytic cap allowed it to go into upload mode slightly quicker.

Reddit post: https://www.reddit.com/r/esp32/comments/rk5e56/flashing_fails_90_of_the_time

[roffe]

For anyone still struggling with this problem, the linked IDF issue above mentions a different Ryzen chipset and that they worked around automatic reset issues by inserting an external USB2 hub between the USB3 host port and the ESP32. If you try this and experience success/failure, please report back.

(It's not clear this is exactly the same issue, a number of our developers now have Ryzen laptops with different chipsets and no problems so this specific issue may just be something about the 3500U chipset, and the linked issue may be something with Windows or WSL.)

this helped me on my Ryzen 7 7800x3d with X670 motherboard.

after i built new pc auto programming just stopped working for some reason until i found this and tried a usb hub in between then it started working again <3 <3