Draft: Bouncer should use Balrog as a back-end

[srfraser]

No description provided.

[tomprince]

I like the idea of there being less places where we track what releases are current.

I had a couple of random thoughts on reading this:

• the election bundles are repacks that exist in bouncer, but don't currently exist in balrog
• if balrog is going to be the data source for bouncer, we should consider expanding balrog to have more of the data bouncer needs
• the outline doesn't address how the versioned bouncer entries will be handled, which I think is worth considering

[MihaiTabara]

I like the idea of there being less places where we track what releases are current.
* the election bundles are repacks that exist in bouncer, but don't currently exist in balrog

Bah, good point about this. While this one in particular might or might not fade away in the future, there could be a need for similar ones to ramp-up. Bug https://bugzilla.mozilla.org/show_bug.cgi?id=1497097 tracks automating this for bouncerworkers.

[srfraser]

* the outline doesn't address how the versioned bouncer entries will be handled, which I think is worth considering

It's something I don't currently have a solution for. Do we know where they're currently used?

[MihaiTabara]

firefox-election-bundle is a product with multiple locations associated, similarly to what we have for firefox nightlies (e.g. firefox-nightly-latest-ssl). So if we're solving that for nightlies, it should work as well for firefox-election-bundle. It could be a matter of extending the static config with more products.

[tomprince]

• the outline doesn't address how the versioned bouncer entries will be handled, which I think is worth considering

It's something I don't currently have a solution for. Do we know where they're currently used?

A bunch of them (the -Complete and -Partial and variations) are used by balrog. See here and here.

I'm not sure if the other blobs are used.

[bhearsum]

if balrog is going to be the data source for bouncer, we should consider expanding balrog to have more of the data bouncer needs

This makes me a bit uncomfortable with this idea. It's one thing to use pre-existing data that Balrog already has, but it's another to require Balrog to store data that it doesn't use for Bouncer's benefit. Perhaps we should get what we can from Balrog, and pull anything else from another source, and merge the two together? This could give us the ability to override Balrog data if it ever becomes necessary.

[bhearsum]

How will this work for releases which aren't the latest? Eg: Balrog points to versioned links for MAR files like http://download.mozilla.org/?product=firefox-62.0-complete&os=%OS_BOUNCER%&lang=%LOCALE%

If we use firefox-release to look them up, we won't get the correct set of locales.

[srfraser]

My best thought so far is to support wildcards in there and extract the version from them. I'm open to better suggestions

[bhearsum]

But if we're using Rules to find a Release, how can we find, eg: the Firefox-62.0-build1 Release?

[srfraser]

If we don't do locale validation then the only thing we need from the release blob is the version, so if it's in the query we don't have to look it up. It depends whether it's ok to redirect to a URL that will 404, by not checking locales.

[bhearsum]

That still has the problem of needing information in a blob that isn't pointed at by a Rule, I think...

[srfraser]

Should we not be using a template to generate the older URLs?

[srfraser]

There's someone working on a combined solution to this, whose github tag I don't know. The current plan is to query balrog for information about the latest version, but to have a new datastore, api and cli-tool for updating the fixed rules with specific versions and other customisations. This should be ready in Q3