CI setup #154
CI setup #154
Conversation
|
I downloaded the linux release you created and it appears to work. I was able to downgrade and upgrade through the release manager, which is really important. Looking at the RPI portion of the README.md file, does it just run it from source? |
|
@madgrizzle Pretty much. I simply bundle the compiled firmware with it. |
|
We'd have to modify how webcontrol's ReleaseManager works to work with this approach. I'm not against it though if it all works. Part of the reason for pyinstaller was that I had to originally build opencv and numpy from source to get it to work and asking people to spend 4-hours to do that wasn't practical. But looks like it can be installed via apt now. |
|
It's not too different (still unzipping a file over that directory), but if we change some dependencies, we'll need a mechanism to install them as well. I think it might be okay to mention it in the release logs, that people need to re-run the install script. |
|
Is there harm in running the install script as part of the upgrade process, even if there aren't any changes? The idea is to make it so people don't have to do anything but hit the upgrade button. |
|
Kind of, that script needs to run as root; and it's not a good idea to run the whole WebControl as root. Altough I guess I could add something in the pre-built RaspberryPi image that runs the script with the correct permissions. |
|
Right now, we have two archs that don't build via Travis-CI, RPI and Win32, correct? These could still be manually built and uploaded to the release if needed. Perhaps that process be automated as well so they fire-off by watching the github repo or some other means of notification to build and upload a new release. I guess there aren't any cloud-based RPI and Win32 machines, huh? |
|
@madgrizzle correct. RPi would ideally use the pre-built image (or just follow the install instructions), and is Win32 really still in use? It used to be a big deal 10 years ago, but now? (Last time I dealt with Windows machines was 2013, I'm genuinely curious) |
|
Re: RPI, I'd like not to force people to change their installations if it can be avoided. I don't know how many people use webcontrol on an RPI currently, but I know its being used. I think this is something to discuss in general (maybe outside this PR) to see if we can find a good solution. Re: Win-32.. There are people trying to use old computers/laptops to run webcontrol for their maslow and this was a build that was requested. Whether or not anyone still uses it, I don't know. Is there a way to track downloads on github? Maybe we need a poll on the maslow forum. |
|
Good points. I personally am planning on running WebControl on a Raspberry Pi, but I'm really just getting started, so I don't really have an existing installation to maintain. I'll see what we can do with the upgrade process for these. Agreed about the poll; Win32 computers might have become even too old for "shop computers", but we never know. Want to launch it on the forums? In other news, the releases will look like this: https://github.com/WebControlCNC/WebControl/releases/tag/2020-05-26-1423 They're triggered by tagging a commit, like this: |
|
GitHub does count asset downloads and it is publicly accessible via their API |
|
Interesting.. I switched it to look at this repo since it has more history. Seems like maybe a few people download the win32 version. RPI is definitely in the lead. |
|
Okay, I've been working on the Raspberry Pi upgrade from pyinstaller to "regular" python (my "universal" release), and I encountered a few snags:
With that, I propose the following changes to the upgrade process:
The dependencies hook could move somewhere else, but it needs to run before restarting WebControl, and probably only for the "universal" install type. In any case, these changes to the upgrade process would help it become more interactive, and allow for an easy "undo" in case of an error at some point. |
|
I've made the proposed changes to the upgrade code in #155 |
|
On Tue, 26 May 2020, Emile Cantin wrote:
It's not too different (still unzipping a file over that directory), but if we
change some dependencies, we'll need a mechanism to install them as well. I
think it might be okay to mention it in the release logs, that people need to
re-run the install script.
let's try to make the dependencies be handled by the debian package manager.
David Lang
|
|
On Tue, 26 May 2020, madgrizzle wrote:
Re: RPI, I'd like not to force people to change their installations if it can
be avoided. I don't know how many people use webcontrol on an RPI currently,
but I know its being used.
Yes and no, we already have the docker vs pyinstaller migration and breaking
existing installations isn't good.
on the other hand, if the advantages of a new method are significant enough,
it's worth the short term pain to provide a long-term advantage
WC should be architecture independent (no compiled code, only python code), so
if we can produce a debian package, that should be able to get installed on any
debian system, including the Pis. This would let us have a repo (possibly add
something to the open build system that Suse maintains) and let WC get updated
just like any other OS package.
IMHO, that would be a significant enough advantage to be worth breaking the
existing installs (and even there, I think we may be able to bootstrap an
upgrade to switch)
David Lang
|
|
I don't think using debian package is a good idea beyond installing Python or Docker. For everything else the release process is too slow, with outdated packages, and adds a lot of bureaucracy in updating them. LTS releases rarely change. Python PIP (with PyInstaller) is much better for rapid changes as is working. |
|
docker doesn't work on windows, which is why we moved away from it to the
pyinstaller version.
not all systems are going to have Internet access, so being dependent on PIP can
be a problem (although the same can be said for being dependent on debian
packages)
I'm ok with the pyinstaller version, but others were talking about changing the
build and upgrade process.
David Lang
…On Sat, 30 May 2020, gb0101010101 wrote:
I don't think using debian package is a good idea beyond installing Python or Docker. For everything else the release process is too slow, with outdated packages, and adds a lot of bureaucracy in updating them. LTS releases rarely change.
Python PIP (with PyInstaller) is much better for rapid changes as is working.
Docker will be easy once its setup. Just stop, destroy, download, and run.
|
|
Yeah, I've slowly been thinking more and more about doing a .deb for the Raspberry Pi version; there's a lot of merit into that. Pyinstaller works well for users, but it's a pain for package builders; because you can't cross-build with it. (which IMO is pretty bad; even native code can be cross-compiled in most cases). I'd keep it for desktop systems (Win, Mac and Linux), but I'd prefer to build a .deb for everything else. And regarding internet; I think it's safe to assume they have it when they're updating through the UI. The reason we're talking about changing the build process is to move it to a cloud-based build; so we don't depend on Madgrizzle's personal machines to build. Upping the bus factor of this project in the process. |
|
I would suggest taking a look at the Suse Open Build System, they will create
packages for all the different linux flavors, including with the CI tests.
I believe that it's free for opensource software.
David Lang
…On Sat, 30 May 2020, Emile Cantin wrote:
Yeah, I've slowly been thinking more and more about doing a .deb for the Raspberry Pi version; there's a lot of merit into that.
Pyinstaller works well for users, but it's a pain for package builders; because you can't cross-build with it. (which IMO is pretty bad; even native code can be cross-compiled in most cases). I'd keep it for desktop systems (Win, Mac and Linux), but I'd prefer to build a .deb for everything else.
And regarding internet; I think it's safe to assume they have it _when they're updating through the UI_.
The reason we're talking about changing the build process is to move it to a cloud-based build; so we don't depend on Madgrizzle's personal machines to build. Upping the [bus factor](https://en.wikipedia.org/wiki/Bus_factor) of this project in the process.
|
Huh? I've been using Docker on Windows for years. |
|
ok, that was what was said about why we moved away from docker in the first
place.
David Lang
…On Sun, 31 May 2020, Zane Claes wrote:
> docker doesn't work on windows, which is why we moved away from it to the
pyinstaller version.
Huh? I've been using [Docker on Windows](https://docs.docker.com/docker-for-windows/) for years.
|
|
It was more the need to enable hyperv which windows 10 home doesn't support. There may be a way to run it in Windows home through a virtualboz VM though, but with the build problems I was having in general (and being on my own at that point) and concern regarding how complex everything was getting, I moved to getting pyinstaller to work. |
This is now a complete CI setup. It builds pyinstaller archives for all 3 major desktop platforms, as well as a more "basic", platform-agnostic package that only contains the python code + the firmware.
Unfortunately, I wasn't able to properly build a Raspberry Pi pyinstaller release, as the ARM support on Travis isn't quite there yet, and it turns out it's ARMv8 (RPi is ARMv7), so it probably won't work. To properly support the Raspberry Pi, I've created the platform-agnostic package along with an install script that should install everything WC needs to run. I've tested it on my RPi3, and I was able to flash various versions of the firmware with one slight modification to the code (fixing a path); we should add a flag somewhere to fix this. I've also amended the README instructions to match this new installation process. Overall, let me know if this is acceptable from a usability standpoint.