Discussion: Release process #139
Comments
|
Perhaps it is a matter of definition. Only at 0.932 did the even odd thing
start. I understand release to mean good for the general population while
prerelease is not fully vetted... However rigorous that is. What would be
a better description that should be used?
At what point does it move to 1.0?
|
|
No, I think that description is fine. It's just that when you go on the Releases page, everything there is labelled "pre-release", which is the default value for that flag. I think we should make use of that flag, instead of layering our own convention on top of it. |
|
Agree. Even or odd, if stable, it should be a release. If testing new or specialized features, then prerelease... is there also a major/minor release approach? |
|
I'm open to any option that works and prefer the "correct" way, whatever that is. It was easy to implement the even-odd thing to solve the problem of releasing a release that breaks webcontrol.. including upgrade/downgrade. I wanted to make it so people could opt-in to testing and those that just want stability, don't even see releases that are not "tested". As for the pre-release tag, I just never used it.. nothing against it. |
|
As of right now, all the releases are marked as "pre-release". I think it wouldn't be too hard to go through them and remove the "pre-release" flag on the stable ones, as a first step. The "isExperimental" method in the update code could then be updated to look at the The main differences would be that it'd then be possible to "promote" a release as stable (or not, no obligation), and it'd be a bit clearer for people browsing the releases page what's what. |
|
Is there any real reason to keep all the old releases? |
|
Not many. You mentioned the downgrade possibility earlier, and it could also help pinpoint when a possible regression was introduced. Having the history that goes with the releases (the description of the changes on each release) is very valuable, though. I refer to it all the time in other open-source projects. I don't think it's a significant cost to keep them, so I don't really see a great reason to get rid of them. |
|
I was just thinking of ease of moving forward using pre-release tag, but I'm sure we could just add another conditional to make sure the version number is greater than 0.9335 (or whatever) before tagging it in the release manager as an "experimental". |
|
Not sure why you'd need to do that. What I had in mind was pre-release === experimental. No need to parse the version number or anything. The only prerequisite would be to manually go into the old releases and correct the flag for all of them. There's only 18 releases right now; shouldn't take very long. I can do it myself if you give me access. |
|
I went ahead and fixed it. Regardless, you are a member of the organization, but don't have access to the WebControl repo, correct? When I brought the repo over, it brought over all the collaborators as well (which you weren't one of, I guess). Do I need to specifically add you to the WebControl repo? Or do we need to create a team or something for the organization? I'm new to all this. |
|
Awesome, thank you. And yes, that's right; I only have read access to the repo, and no, I wasn't a collaborator before (just now getting involved in this). I think creating a team would be the best approach for this. |
|
I'm creating a final release on the madgrizzle repo marked as stable (0.94, even digit). I did a PR to pull in all changes from my fork into this repo so they should be identical. I'd like to get the same release up on this repo as well and will work on modifying my build scripts to build and publish here. However, I'd like to discuss if there are alternative ways to do the builds. Right now I build for RPI, linux, Win32 and Win64. The RPI build runs on a dedicated RPI. The Win64 build runs on a win64 computer and the linux and Win32 builds run in VMs on that same win64 computer. The linux machine builds the firmwares and then makes them available on its ftp server for the RPI, Win32, and Win64 machines to download them (so they don't have to build the firmwares). I don't launch the RPI, Win32, or Win64 build scripts until the linux build script has finished with the firmware compilations. After all the builds are completed, each machine uploads them via FTP to the linux machine's "release" directory. On the linux machine, I then run a script that creates a github release and uploads the builds. Finally, I edit the release via web to add a title and description. Any idea on ways to improve this or maybe, possibly, put it all in the cloud? The builds are pyinstaller based. I'm not sure if there's a way to cross-compile, etc. so it runs on just one machine. Or do we need to maintain VMs and if so, is there a VM for an RPI? |
|
I think the first step here would be to actually publish these scripts, so people could build releases on their own. Then, we could look at integrating a continuous integration tool (like TravisCI); there are a handful out there that are free for open-source projects. This would be a "cloud build", in your words. They're usually able to build in a few different environments, Travis even has macOS environments available. I'm not really familiar with pyinstaller, so I'm not sure what it actually needs to build natively; unless you wrote some native code yourself as part of WebControl, we can probably get away with just downloading pre-built versions of every dependency. Compiling software for the Raspberry Pi is something that's pretty common; depending on our specific needs (again, need to research pyinstaller) we may have many options here. Worst case scenario would be to move all the dependency management to the pre-built image, might make things a lot simpler from the WebControl side. For the firmware build, I think that should actually live under the firmware's repos; the release script would only download the latest release binary from the firmware repo directly. That might not be possible, however, as we might not have control over all versions (notably the stock one); in these cases I think we should either maintain a fork that has the CI integration, or put in a pull request for that to Bar so we can pull from the "true" source. |
The .spec files to build the pyinstaller releases are already part of the repo. main.spec is the spec file for the singlefile variant and main-onedir.spec is for the singledirectory variant. I use the same spec files for each architecture and it seems to ignore the architecture-specific adds in them. The user will have to adjust the pathex within the file to make it work on their installation, but that should be simple enough to do. What I haven't pushed is the scripts I used to semi-automate the process. That's because its specific to how my build environment is and secondary that it includes things like FTP passwords and such.
This is the challenge because the last release was nearly a year and a half ago. Only bar can make the releases, afaik.
I really would like to see a single firmware that has both triangular calibration (i.e, stock) and holey calibration contained in one. If we do this, then I think we can convince bar to make it the official release if we demonstrate that it also works with ground control, which knows nothing of holey calibration. It's not impossible, or even hard, but its not trivial and will take time and testing. |
Afaik, you need a complete installation with all dependents installed on each architecture, to the extent that the program can actually run, in order to use pyinstaller to compile the program. I'm not sure how all this can be done in the cloud. Travis-CI is completely new to me. I understand what it does, but how to set it up is way beyond me. |
|
And so it dawned on me that I didn't create a development branch to put the recent changes into. So what's the "github" way of doing that. Just call it development, develop, devel, dev? What about the default tag? Does that make a difference. We want all PRs to be to the development branch, correct? |
|
There are many philosophies on the "proper" way of branching; and it's a subject of continuous debate, mostly because everyone has slightly different requirements, so slight different definitions of "best". One way to do this is to use git-flow, and I've used it before on very active projects, but I'd argue it's overkill for our needs. Unless you find yourself really needing it, I'd recommend keeping the |
|
Also, I've managed to build something with pyinstaller, but I'm really not sure it's the right thing. Care to share how you invoke it in your build scripts? |
|
This is all I do to build the single directory version, and create a new release file.. single file is the same, just different spec file and different renaming (there's probably a way in the spec file to get things named the proper way from the start, but this worked nevertheless).
|
|
Yeah, I've tried with the Anyway, I'm trying to set up Travis in #149, but I need you to approve access to the organization (you should have an email about that). |
|
Done.. I think |
|
Yeah, it's active, but only repo admins can add it (annoyingly). Can you add it here: https://travis-ci.org/github/WebControlCNC/WebControl ? Thanks! |
|
I logged in but says "You don't have sufficient rights to enable this repo on Travis. |
|
It's telling me the same thing, weird. |
|
Well, I went in and under my settings, I clicked on the webcontrolcnc organization and it showed two repos with switches.. i enabled the switches, but when I clicked settings, it says takes me to the travis repo and pops up an error message "Oh No! Your permissions are insufficient to access this repository's settings" |
|
Okay, I don't know what changed, but I refreshed the page and I was able to trigger a build. Success! |
|
So, can this build win32 files? I assume arm (rpi) is not feasible. |
|
Fwiw, I run a BuildKite cluster already. It should be able to handle CI
needs and I have plenty of experience implementing it..., if Travis becomes
too much of a headache lmk.
…On Fri, May 22, 2020 at 2:39 PM madgrizzle ***@***.***> wrote:
So, can this build win32 files? I assume arm (rpi) is not feasible.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#139 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAP47VP5NW2BPY54X3GOKC3RS3PHTANCNFSM4NENV64A>
.
|
|
@madgrizzle Win32 I'm not sure, but Win64 will very probably work, as well as macOS. I'm starting with the easiest part, Linux on amd64. Arm will probably work as well, depending on the specific version we need; I'll test that once I have the whole pipeline working with regular Linux. |
|
@madgrizzle if you're still here, I have one issue: I can't manage the build settings anymore, but I need to add a Github token in order to create a release from the build. (it can wait, it's getting late around here anyway).
|
|
Late here as well (east coast) you may need to talk me through it. I'm going out of town tomorrow, but not sure what time I'm leaving or (after arriving) when I'll be able to get on my laptop. |
|
East coast here as well! Québec, Canada. Go here, and create a token with the following permissions: Now go here, and in the "Environment Variables" section, add one with the name |
|
Done.. why does this need to be done for the firmware, but not the other two? |
|
And how did I manage to 'assign' this to @zaneclaes ? I don't recall doing so.. |
|
Haha, I was wondering... |
|
Not sure, but I lost access to Webcontrol as well. I managed to add the
environment variable when I created the thing on the Travis side, but when
you did something with the organization, I lost access to the settings. I
guess I probably have different permissions on the pi image repo, so I do
have access to that one.
EDIT: Nevermind, I just checked and I have access to everything now.
Le sam. 23 mai 2020 07 h 58, madgrizzle <[email protected]> a écrit :
… Done.. why does this need to be done for the firmware, but not the other
two?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
<#139 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAGYF3TDJI36Y5K2HE6PYBTRS63ADANCNFSM4NENV64A>
.
|
|
And it looks like creating a release works properly, and I get the 3 different firmwares! Just note that the git setup is a bit convoluted at the moment; we'll be able to drastically simplify this when we get the holey firmware merged in.
|
|
Amazing work! So much better than my scripts! Thank you so much for working on this. |
|
Yes, great work. Really happy to see so much activity and progress. I like the idea of using GitFlow as it corresponds with other workflow I have used before (dev, master, hotfix, feature etc). My only concern is if it makes things harder for the hobby contributor:
It looks pretty straight forward to use. Does GitFlow integrate with or extend GitHub GUI to make project management easier? |
|
There are a lot of ways we can go about this, but if we want to go towards git-flow, we have to decide a few things:
As for whether people need to use it to contribute, the answer is probably not, unless we want to maintain a clean git history tree. With the PR process that's already in place, we basically already have feature branches. In my opinion, it's a little too much work for not enough benefit to maintain 2 main branches on this repo. With most people here not having a professional software development background, I feel it's better keeping things simple. However, I'd definitely encourage everyone to use git-flow conventions when naming their branches. |
|
My two cents: The default ( I'd also argue for not maintaining releases more than one minor version away from the current SemVer HEAD — if only as a matter of expedience for a small team. The vast majority of software development hours, I've found, go toward maintenance. Until they can justify such effort (read: $), development teams have every right to insist users upgrade to the latest if they want to be supported. Speaking of backgrounds... by way of introduction, mine is that I was most recently a software engineer (3 years) and then engineering manager (3 years) for Airbnb. I worked first on the mobile apps, then on the API, and finally on DevOps & Observability. |
|
A lot of this is new to me, but what I envisioned that that we have a "stable" release and then have a series of development releases that we can test things out in.. once we are happy with what's in a development release, we do a PR and push those changes into the stable branch and create a new stable release. As for maintenance of releases, it seems like a lot of work to go back and update releases to fix issues.. I think we'd have to have a major change to think about doing that.. like a WebControl 2.0 that works differently and some people just might want to stay on WebControl 1.0. I also think new features need to be added in a manner to maintain backwards compatibility. For example, the support for R in g2/g3 and the zaxis limits should still work for people using old firmware. Doing it this way, I think, would allow people that just want stable, reliable operation to know that anytime they upgrade, it's not going to break their installation and they can then chose to use new features or not. This way they can get but fixes, but not be forced into making any other changes. |
|
Perhaps a stupid question... who are the maintainers of GroundControl? To what degree do we expect feature parity between the two? Personally, I feel that it would be better to offer the community a single piece of software. WebControl IMO is the better approach for a number of reasons, especially: (1) OS-agnostic (2) web UI development is significantly faster than native/client development (3) remote access & monitoring is a killer feature. With some investment in modern UI frameworks and some cleanup all around, I could see WebControl becoming very user-friendly and feature-rich application. psst... kinda wish it had been called AirControl... or AirCNC... but that's just because I like puns... |
|
Imagine: a built-in asset catalog that lets you upload your own designs and explore others' designs from within the app... I could imagine this project evolving to become like Fritzing.org (an app + design tool + catalog of projects). If we wanted to go that route, I'd be happy to offer my AWS Kubernetes cluster for hosting. |
|
Bar (creator of the maslowcnc) created ground control.. lots of people worked on it to improve it (including me). I think he's the only one that can create releases and though I could be wrong about it, I think one of the reasons there's not been any new releases is that he's taken a step back from maslow and is working on other projects (still CNC related). It's a stable product that people use regularly and if you keep updating it, then you really need to be involved in maintaining it.. which takes time away from other things. Bar actually proposed that webcontrol become the standard software for maslow, but it was a one man show at the time and I shied away from it. That's a big part of moving this to an org and away from my personal repo. |
|
Good ideas.. I would like to see webcontrol maintained as a user-friendly, entry level type of program. I think that's an unserved segment and is what ground control brought to maslow. Integration with Maslow community garden (or something else) would be awesome.. so would actual CAM capabilities.. but that's a huge undertaking. Bar is working on a CAM program that might be cool if I could be tied into work with webcontrol https://forums.maslowcnc.com/t/maslow-create-alpha-release/9885 |
|
Thanks for the context! One more question: what's your impression of the size of the Maslow community, and its potential to grow? I assume WebControl is permanently tied to the Maslow architecture, so the popularity will be limited to that sub-set of the CNC space. I totally agree about the user-friendly, entry-level aspect (being that I still haven't made it past my test cuts!). It seems like MaslowCreate is already doing a lot of what I'm talking about. I'd hate to see these projects bifurcate even more. All too often, with open-source, I see stuff like this. Here's my concern about what might currently happen:
I think if we're going to invest time in all this, we should strive toward a single unified experience. Speaking honestly, as a new user of Maslow, I still expect there to be some sort of coherent vision — and the quality of software, education, etc. is still lacking. I would hope that we could converge the project with Bar rather than diverging. |
|
I really don't have a handle on the size of the maslow community.. it's in the thousands, but how many I don't know. Webcontrol is currently tied to maslow, but if we could get it to support grbl, that could change. It's not hard at that point to support other CNC machines that are grbl-based and build "quick configures" for them. I think this is important thing to be able to keep webcontrol relevant. I don't know it has to be so flexible to support others (i.e. Marlin, etc). Grbl should be enough, imo. As for maslow create, I'm not sure bar's vision for it (i.e., will it include gcode sender functionality and all that entails?). How it would integrate into webcontrol, etc. would have to be figured out. Finally, having a vision for moving forward makes sense, but that's a community driven endeavour at this point. Part of the issue is people come and go from the product. I think the only really community participant that's been very active from the start is @davidelang. Most everyone comes and goes in waves. |
|
Got it, thanks again. I was looking around and found https://cnc.js.org/ today. They seem to have done a lot of work on the UI side of things. What if we approached them about adding Maslow support and merged the projects? Not that I want to throw away any work done here, but I'm in favor of having fewer tools overall. I found CNCjs because I ordered a SainSmart 3019-PRO w/ laser upgrade (for my small/detailed work and making electronics cases from acrylic), which CNCjs supports, and I was reading reviews before buying that spoke highly of this web interface. |
|
I use cnc.js myself for one of my CNC machines that runs marlin. I wouldn't consider it an entry-level CNC program like ground control or webcontrol, but it does work well. In reality, it can be used for Maslow pretty much as-is with the grbl plugin.. I think any grbl gcode sender would work with Maslow but you need to know what you are doing. You will also need a stand alone program to do calibration calculations and it would be handy to have something that assists in setting the sprockets vertical, but the latter can be done via macros. |
|
FYI PR requesting new firmware release MaslowCNC/Firmware#546 As for GitFlow/Release Process: I would like to see greater testing involvement from end users (including non coders) by making it very easy to test new features in WebControl by switching between releases with a few mouse clicks. I'm not really fussed about the underlying structure, just as long as it can do this. |
|
we don't have the manpower to maintain a lot of releases, so people should
upgrade to the latest to get all the fixes.
I would even say that we don't make changes to the stable branch, we just
promote/tag a development release to declare it stable.
…On Sun, 24 May 2020, Emile Cantin wrote:
There are a lot of ways we can go about this, but if we want to go towards git-flow, we have to decide a few things:
- What should live in the default branch (`HEAD` on github)? i.e what should people get when they clone our repo? Stable code, or more bleeding-edge code?
- This will guide whether we have `master`=stable, `develop`=bleeding-edge, OR `release`=stable and `master`=bleeding edge.
- Do we maintain old releases? Or do we expect users to update to the latest release to get all bugfixes?
- This will guide whether we need the whole hotfix part of gitflow (probably not, IMO)
|
|
On Tue, 26 May 2020, Zane Claes wrote:
I was looking around and found https://cnc.js.org/ today. They seem to have
done a lot of work on the UI side of things. What if we approached them about
adding Maslow support and merged the projects?
'Adding Maslow support' can mean a number of things.
sending gcode to the maslow is pretty trivial, just about anything could do it.
What GC and WC add are things that are specific to the Maslow compared to all
other CNC machines. Some of these things can be done by sending custom g-code,
some of them are far more complex
the simple stuff are things like manually moving each motor, telling the maslow
what the chain lengths are, setting Z=0
those could be done with macro buttons in any gcode sender
the complex stuff is things like flashing the firmware and calibration.
Calibration is the worst part, there is a lot of very hardware specific logic
that is in GC/WC for the calibration. In the long run, I think it would be best
to try to move the calibration steps into the firmware.
It's not practical to implement the calibration logic in every gcode sender.
David Lang
|
|
On Mon, 25 May 2020, madgrizzle wrote:
I really don't have a handle on the size of the maslow community.. it's in the thousands, but how many I don't know.
there was a post on how many kits had been shipped, over 3K and it may have been
6k
now, how many kits are in use, and how many of those people noticed that the
community exist are good questions.
Webcontrol is currently tied to maslow, but if we could get it to support
grbl, that could change. It's not hard at that point to support other CNC
machines that are grbl-based and build "quick configures" for them. I think
this is important thing to be able to keep webcontrol relevant. I don't know
it has to be so flexible to support others (i.e. Marlin, etc). Grbl should be
enough, imo.
70% of it is just knowing what to look for to see what's attached (the grbl0.9
that the firmware outputs at startup) and the per-line acknoledgement
20% is going to be knowing which initialization parameters to send
As for maslow create, I'm not sure bar's vision for it (i.e., will it include
gcode sender functionality and all that entails?). How it would integrate into
webcontrol, etc. would have to be figured out.
the idea is to make everything as seamless as possible, but since it is a
browser/cloud based solution, there are some headaches in having it send the
gcode to another server (it would need to do legitimate cross site scripting,
but since that's been used by the bad guys so much, it's generally blocked by
browsers). So it's going to be far easier to have WC reach out to git and fetch
the gcode than to have maslow create push it to WC
This would be a neat feature to add to WC, be able to give it a git URL and have
it pull the gcode from there and keep it synced locally.
Finally, having a vision for moving forward makes sense, but that's a
community driven endeavour at this point. Part of the issue is people come
and go from the product. I think the only really community participant that's
been very active from the start is @davidelang. Most everyone comes and goes
in waves.
and even I get busy for a couple weeks here and there. :-)
David Lang
|
|
On Tue, 26 May 2020, gb0101010101 wrote:
FYI PR requesting new firmware release MaslowCNC/Firmware#546
@BarbourSmith (Bar in Forums) is owner and only person that can do a release. Says he will try to get it out soon. GroundControl is not really being updated, the build process is apparently broken, and this is the holdup in releasing firmware as version IDs currently must match (GroundControl & Firmware).
that's a should match, not a must match.
I would advocate that we make a new GC release that at startup recommends people
switch to WC and give a pointer to a URL for instructions.
David Lang
|
|
@davidelang thanks for the notes! We've got a more formal discussion on the future of the project over here: https://github.com/orgs/WebControlCNC/teams/developers/discussions/2 |
I'd like to start a discussion regarding the release process. I know we currently use the even=stable, odd=experimental convention, but is there a reason why all of them are marked as "pre-release"?
I think it'd be way easier to actually use that pre-release feature for experimental releases. That way, we could promote experimental features to stable once they've been tested; no need to cut an additional release.
It'd also make it easier to consume the list of releases from a script. I'll probably need this as I improve on the RPI image build.
The text was updated successfully, but these errors were encountered: