AVRDUDE 7.1 release discussion

[mcuee]

Updated on 2-Dec-2022

It would be good to have avrdude 7.1 release within 2022.

Now we have the following few issues to sort out after @stefanrueger finished his major work on urclock support for urboot/optiboot.

• [bug #48261] STK500V2 ISP problem with ATmega8/8515/8535, ATmega16 and ATmega32 #425 --> shall we drop it from 7.1 milestone?
• [patch #9565] Patch for WiFi AVR Programmer, uPDI support for jtagmkII, and serial-over-network improvements #734 --> @dl8dtl wants to work on it. If not on time, I see no issues to postpone it.
• Consider libreadline/libedit only in interactive mode #1207 --> this is to address the remaining issue of Avrdude on macOS is built without libreadline #1173
• Added TPI support for Microchip tools #1205

Looks like the following will not be ready.

• Added get configurations fuses, lockbits and eeprom from ELF files. #1195

************** old post on 10 August 2022 **************
It seems to me that we have done a lot to improve avrdude in the recent months post 7.0 release (7.0 release is a great release).
https://github.com/avrdudes/avrdude/blob/main/NEWS

Good sign: Issue number has been reduced to double digit.

Then PR number has reduced to single digit.
I expect the following two to be merged soon by @stefanrueger .

• Implement a dry run for -U updates before opening the programmer #1056 Done
• Implement -c <wildcard>/dev_options for printing programmer entries of avrdude.conf #1059 Done

The following PR will be closed and changed to a new one as per @stefanrueger .

• Attiny85: increase delay #983 Done

The following PR is not ready anyway so probably can wait.

• Fix TPI memory writes for odd number of bytes #825 Closed as superseded by PR Fix writing of last word on DWORD TPI parts #1115 Done

Then we have the following PR which is good for 7.1 release milestone. Hopefully it can get some attention to be reviewed and merged.

• Fix permission denied error in linuxgpio programmer #917 Done

There are also two issues targeted to be sorted out for 7.1 milestone.

• [bug #48261] STK500V2 ISP problem with ATmega8/8515/8535, ATmega16 and ATmega32 #425 -- The root cause of the issue is zoomed down to a single mega commit d742827 on 15 Sept 2011. Hoepfully it can be sorted out soon.
• [patch #9565] Patch for WiFi AVR Programmer, uPDI support for jtagmkII, and serial-over-network improvements #734 (more for enhancement) -- can be delayed if needed

[mcuee]

It would be good to have a release in a few months time, say around Nov 2022 if possible (6 month from Release 7.0) or December 2022.

avrdude 6.4 -- 17 Dec 2021
avrdude 7.0 -- 8 May 2022
avrdude 7.1 -- Dec 2022 if possible

For avrdude, I think it is a relatively mature project and one year release cadence is probably good enough. However, in terms of 7.1, I think the changes so far do warranty a new release within the year of 2022.

[mcuee]

For new features, as per @stefanrueger, he is working on a nice generic bootloader solution, that will of course good to be incuded in the next release.
#983 (comment)

Just to confirm that I am queuing up work for a generic bootloader solution for all parts, large and small, in the coming weeks.

Not so sure if others have important new enhancements (30+ requests) which need to be in the next release. From what I see, many enhancements are more of a "nice to have" nature. If there are people who are actively working on them, then it will be good to include. If not, they can be delayed.
https://github.com/avrdudes/avrdude/issues?q=is%3Aopen+is%3Aissue+label%3Aenhancement

Three of the enahancement requests are also labeld as "Help Wanted". They are indeed very nice to have but probably we will need to find people who come up and help.
https://github.com/avrdudes/avrdude/issues?q=is%3Aopen+is%3Aissue+label%3A%22help+wanted%22

• Rework PICKit2 programmer to use libhidapi instead of libhid (which will make it work under macOS) #883
• Port Linux sysfs GPIO feature to libgpiod #831
• Remove all calls to exit(3) from libavrdude library functions #774

[mcuee]

For the 20+ bug fixes, if the things mentioned in the first post are sorted out, then there are also not many important bugs to be fixed.
https://github.com/avrdudes/avrdude/issues?q=is%3Aopen+is%3Aissue+label%3Abug

I think @stefanrueger has good grasps of the following issue he raised.

• -U writes sometimes more bytes to flash/memory than input file has #1050 (to be dealt with soon for 7.1 release)
• Write flash rarely works in terminal mode for classic AVRs #1020 Done
• Cannot access EEPROM on some bootloader/part combinations #970 Done

If #917 (or with some modifications) is merged, then the following issues will be fixed as well.

• Fix permission denied error in linuxgpio programmer #917 Done
• [bug #45256] Raspberry Pi sysfs bug workaround for linuxgpio driver #386 Done

[mcuee]

For the documentation side, most important is probably the CMake infrasture. But I tend to think that probably can be delayed as well if not on time.
https://github.com/avrdudes/avrdude/issues?q=is%3Aopen+is%3Aissue+label%3Adocumentation

• #CMake infrastructure does not handle documentation #947 Done

[stefanrueger]

@mcuee Great thinking and a very good summary of where we stand.

There are quite a few changes happening in the code base and otherwise. So, it is probably a good idea to set aside some period before the next release with a good and thorough program of testing (perhaps even some automated tests?); during the test period we probably should refrain from merging PRs (unless they fix problems observed at testing)

[mcuee]

Yes we can have release candidates RC1 and then RC2 if necessary and then the final release.

I am not so sure how to do automated tests (if you mean using github action) though since we may need some hardware. But yes it will be good to define some basic test cases and test matrix. And your test hex files and test scripts will definitely help.

Example from libusb-1.0.26 release.

• Misc: To release 1.0.26 libusb/libusb#1047
• 1.0.26-rc1 test matrix libusb/libusb#1099

Example from hidapi-0.12 release

• https://github.com/libusb/hidapi/projects/6
• hidapi-0.12.0 Test matrix libusb/hidapi#407

[mcuee]

Now all the relevant pull request has been closed except #825 whic is not ready anyway.

[MCUdude]

I just got my hands on an Xplained ATxmega256a3bu, an STK600 and a Power debugger. There are still a couple of ATxmega256a3bu issues to sort out, and the Power debugger has a few hardware features that's not yet available in terminal mode that I'll have a look at very soon. There's also an STK600 related issues I'll look into.

[mcuee]

Now that urclock (PR #1171) is in good shape and should be good to merge soon, I see that there are three issues slated for 7.1 release.

• [bug #48261] STK500V2 ISP problem with ATmega8/8515/8535, ATmega16 and ATmega32 #425 (tough bug to crack)
• [patch #9565] Patch for WiFi AVR Programmer, uPDI support for jtagmkII, and serial-over-network improvements #734 (@dl8dtl has the HW but may not have the time)
• Avrdude on macOS is built without libreadline #1173 (some CMake work, should not be so difficult)

I think it is good to have a release before end of the year, better as a Christmas present. It will be great to hear the opinion of @dl8dtl, @stefanrueger, @MCUdude and @mariusgreuel.

[MCUdude]

Sound good to me! #1173 is all about getting the original makefile to handle libreadline.

@mcuee one thing that we should probably do is to try and track down all documentation available for every programmer Avrdude supports and add the URLs to the wiki.

[mcuee]

@MCUdude
I can try that but I do not have the Wiki access yet. Do you have the Wiki access? I can use my fork here to host the info.

[MCUdude]

Maybe @dl8dtl can give you wiki access?

[mcuee]

Maybe @dl8dtl can give you wiki access?

Yes, I have asked for access here. (DONE)

• Wiki updates needed #1157

[mcuee]

From here: @dl8dtl is okay with a Christmas release as well.

• [patch #9565] Patch for WiFi AVR Programmer, uPDI support for jtagmkII, and serial-over-network improvements #734 (comment)

[mcuee]

@stefanrueger has merged PR #1171 and some other PRs. #1173 has also been fixed thanks to the nice libedit workaround from @dl8dtl.

Looks like avrdude is now in good shapre to prepare for a new 7.1 release, except the following two more issues with the milestone set to 7.1.

• [bug #48261] STK500V2 ISP problem with ATmega8/8515/8535, ATmega16 and ATmega32 #425 (tough bug to crack)
• [patch #9565] Patch for WiFi AVR Programmer, uPDI support for jtagmkII, and serial-over-network improvements #734 (@dl8dtl has the HW but may not have the time)

#425 is not really as bad as it sounds since you need some type of hex file to make it fail. The issue has been there for many years anyway. It also has the workaround of removing the 0xffs from the hex file. So I am thinking that we can skip it if it is not sorted out.

Of course there are other features which can be merged when they are ready before 7.1 release. @stefanrueger mentioned that he would work on the following two.

• -U writes sometimes more bytes to flash/memory than input file has #1050 (DONE)
• Fix #1168 adding the m328p spmcr address and bl section. #1178 (DONE)

[MCUdude]

@janegilruud is currently working on JTAG3 TPI support #882. I believe this is ready before the 7.1 release

[mcuee]

That will be a great enhancement to avrdude.

• jtagice3_tpi programmers not supported by Avrdude #822

[mcuee]

The pull request has been posted.

• Added TPI support for Microchip tools #1205

[mcuee]

Updated on 1-Dec-2022

So we have the following few issues to sort out now that @stefanrueger has finished his major work on urclock support.

• [bug #48261] STK500V2 ISP problem with ATmega8/8515/8535, ATmega16 and ATmega32 #425 --> shall we drop it from 7.1 milestone?
• [patch #9565] Patch for WiFi AVR Programmer, uPDI support for jtagmkII, and serial-over-network improvements #734 --> @dl8dtl wants to work on it. If not on time, I see no issues to postpone it.
• Avrdude on macOS is built without libreadline #1173 --> @dl8dtl has already good grasp of the solution, so we should be fine with it.
• Added TPI support for Microchip tools #1205

Looks like the following will not be ready.

• Added get configurations fuses, lockbits and eeprom from ELF files. #1195

[mcuee]

#1207 should address #1173 now.

[mcuee]

#1207 and #1205 have been merged.

[mcuee]

@dl8dtl

Just wondering what is the plan here to prepare for 7.1 release. It would be good to have an RC1 if possible before the formal release. We should be able to merge 4 out of the 6 PRs soon. And the two remaing PRs are not ready (PR #1195 and #1215) any time soon.

@MCUdude @stefanrueger and @mariusgreuel
.Any outstanding issues do you want to address before 7.1 release? Thanks.

Christmas is a nice time to have 7.1 release but it can be a bit later if necessary.

[mcuee]

The github error is a failure to install prerequisites for MacOS so unrelated to the PR.

I have merged PR #1240 to work-around this issue of github.

[mcuee]

@mariusgreuel

Hopefully people can post the test results for PR #1242 and then we can merge it. To me it is pretty safe to merge as it does not really touch avrdude codes other than pulling new avrdude-libftdi and changes avrdude.conf.in in a backward-compatible mode.

From my quick glance, the changes to avrdude-libftdi seems to be okay as well but I am not good at reviewing more complex code changes like that.

Other than #1242, are tthere any other things you want to be included in 7.1 release?

[mcuee]

After merging PR #1242, as far as I am concerned, it seems that we are almost there for 7.1 release.

[mcuee]

@mariusgreuel mentioned that he would like to get Issue #1186 fixed. That is a very good one to have for Windows users.

[mcuee]

There are two issues reported by @mariusgreuel.

• Micronucleus programmer shows 'error: unsupported memory type flash' #1244 -- I can reproduce the issue as well. @stefanrueger is able to quickly identify the issue and comes out a quiick solution. So this can be closed soon.

• urclock communication issues #1245 -- related to urclock under Windows. Edit: Issue was resolved as well.

[stefanrueger]

Happy to take a snapshot right after the 4 currently mature PRs have been merged. Do you want to merge @MCUdude?

[dl8dtl]

Also some nice holidays to everyone!
Alas, my holiday preparations took more time than expected, so I didn't manage a Christmas release. Promise it to still happen this year though.

[dl8dtl]

I ran a good number of tests now, primarily on older controllers and/or programming hardware *):

• STK500, ISP mode, ATmega8
• STK500, PP mode, ATmega8
• STK500, HVSP mode, ATtiny11 (a device where HVSP is the only option ;-)
• AVRISPmkII, ISP mode, ATmega8
• JTAGICEmkII, ISP mode, ATmega8
• JTAGICEmkII, JTAG mode, ATmega16
• JTAGICEmkII, PDI mode, ATxmega256A3
• USBtinyISP, ATmega8
• Matwei USBISP (STK500v2 clone), ATmega16
• JTAGICE ("mk1") clone, ATmega16
• AtmelICE, JTAG mode, ATmega16
• AtmelICE, ISP mode, ATmega16
• AtmelICE, TPI mode, ATtiny10
• Parallel-port dongle (original "bsd" one from Brian Dean), ATmega8
• "ponyser" serial-port dongle, ATmega8
• FT245R (ftdi_syncbb), ATmega8
• FT232R (ftdi_syncbb), ATmega8
• UM232H (avrftdi), ATmega16
• USBasp, ATmega8
• AVR Dragon, ISP mode, ATmega16
• AVR Dragon, JTAG mode, ATmega16

I hope I didn't forget one. ;-)

I'm quite confident with the status quo.
Anything that needs to go in (without risking new breakage) before tagging the release?

*) I think the more recent stuff has undergone quite some testing already by others.

[stefanrueger]

Excellent news!

I'd like to merge #1260 before the release, small fry (and uncontentious I should think) but neater. Can merge tonight

[mcuee]

@mariusgreuel mentioned that PR #1259 is not ready for 7.1 release.

• CMake: Include readline library for MSVC #1259

@stefanrueger has merged #1260 into git main.

@MCUdude is working on Issue #1235 and he mentioned that it will be post 7.1 release.

@dl8dtl
In that case, I think we are fine now.

[stefanrueger]

I'd give @mariusgreuel a day or so to consider the question in #1259 (comment) and then go ahead with v7.1

[dl8dtl]

fine with me

[mariusgreuel]

One more, please. Atmel ICE on Windows crashes, not sure why this pops up now.

#1269

[stefanrueger]

OK, done. 1b8a4d4 is the one...

[dl8dtl]

Fine to tag a release? I'd prefer tomorrow.

[mariusgreuel]

I ran a few quick tests and Windows looks good.

Note, CMake version needs a bump before you push the tag as the latter will trigger the Windows delivery action: https://github.com/avrdudes/avrdude/blob/main/CMakeLists.txt#L24

Makes me think, we should start on that release procedure some time...

[dl8dtl]

You mean, that one:
project(avrdude VERSION 7.0 LANGUAGES C)
Will bump that to 7.1 before tagging, sure.

[mcuee]

The other version number to update is this line in configure.ac.
https://github.com/avrdudes/avrdude/blob/main/src/configure.ac#L26

[mcuee]

@dl8dtl

What is the typical release tagging procedure for avrdude?

The following is what I can think of.

1. Update version string for CMake and Auto-tools, update NEWS

2. tag new release

3. prepare for 7.1 release (which will have CMake source tar ball) and upload Windows binary (MSVC build for x64 and x86) from github action

4. Do you want to upload the tar ball for auto-tools for 7.1 release or not? The 7.0 auto-tools tar ball is kind of broken for MSYS2 MinGW32/64 due to the missing winsock2 library.

• auto-tools build: missing ws2_32 library under Windows MSYS2 MinGW #957

[mcuee]

BTW, it seems to me only @MCUdude cares for auto-tools. But at least the auto-tools tarball will work under macOS. There will be warning messages in the linking process though.

  CCLD     avrdude
ld: warning: directory not found for option '-FCoreFoundation'
ld: warning: -undefined dynamic_lookup may not work with chained fixups
...
ld: warning: -undefined dynamic_lookup may not work with chained fixups

Using the following configure option does not help to suppress the warning messages.

 CPPFLAGS=-I/opt/homebrew/include LDFLAGS="-L/opt/homebrew/lib -Wl,-undefined,dynamic_lookup" ./configure

Ref:

• Avrdude on macOS is built without libreadline #1173 (comment)

[dl8dtl]

I'm not sure about the autotools tarball. I used to upload it to savannah.nongnu.org, but maybe we'll stop with that.

Basically, the procedure is what you described.

I just tagged the release.

[mcuee]

Great.
I have also uploaded my alternative binaries here.
https://github.com/mcuee/avrdude/releases/tag/v7.1

[mcuee]

@umbynos

BTW, I need to update one patch to get avrdude-packing to work.
https://github.com/mcuee/avrdude-packing/commit/62c0d14e9e51e6b044844f4cc212ed1f83ff69ce

With the patch it works perfectly.
https://github.com/mcuee/avrdude-packing/actions/runs/3853172768

[umbynos]

Actually, the modification to the patch does not seem to be required 🤷

[mcuee]

I see. That is good to know.

[umbynos]

you were right, it was need 🤦

By the way here it is https://github.com/arduino/avrdude-packing/releases/tag/7.1-arduino.1

[MCUdude]

Thank you @umbynos! Will the Arduino Avrdude 7.1 release be bundled with the next IDE 2.0 release and/or the next ArduinoCore-avr release? After all, it brings a bunch of new features and improvements, for instance, it now prints the actual and expected device signature and device name if the user is struggling with a faulty Arduino as ISP breadboard setup. And there are a bunch of users that are facing "Arduino as ISP issues" that will benefit from the additional information provided by Avrdude 7.1 🙂

[umbynos]

Yeah, the plan is to bundle it along with the ArduinoCore-avr and ArduinoCore-megaavr cores, some testing is required in order to make this happen. I will have to prepare an index with avrdude 7.1-arduino.1 and the two cores in order to make this new version testable by arduino users

[MCUdude]

@dl8dtl it would be great if you could update the Github pages docs and PDF for the 7.1 release!

[dl8dtl]

I plan to update the docs tonight.

[stefanrueger]

@MCUdude @dl8dtl Good point, perhaps bring authorship in line with what avdude -v says (though I realise that authorship of the docs is different to authorship of the program). Date of avrdude.1 (line 21) could be updated, too.

[dl8dtl]

Well, sorry for the man page date. We should add that to the release checklist … I don't think it's worth to re-roll the v7.1 tag just for that though.

[mcuee]

@dl8dtl

While you are at it, is it possible also to fix ##369? Thanks.

• [bug #44170] Update/Rectify/Clarify Documentation - avrftdi pin number should start with 0 #369

[stefanrueger]

Will we have a branch avrdude 7.1 as sibling branch to avrdude 7.0 in the code tab? That would be nice.