Generate applications.json from software-boutique-curated-apps

[lah7]

After the "new" dedicated index repository is updated with all the latest changes in ubuntu-mate/software-boutique-curated-apps-prototype#9, we will be ready to use the new format for this version of Software Boutique until our new 2.0 version is fully developed.

This could work in one of two ways:

• Adapt ubuntu-mate-welcome to read the new JSON data.
• Amend the newly generated JSON data to the old format.

The last option would be preferred, so it doesn't break the tools or tests and we must keep the Unlisted key which is not present in the new index.

[MungoRae]

Hi. I'm looking to help out if I can.

Having looked over the related issues it seems like the ask is to build a python script to take the content of the apps directory from software-boutique-curated-apps and transfer the information contained into an applications.json file in the same format as the current one.

Given that this is slightly coupled to the related issue 9 in the other repo and someone has been assigned for that, can I just check that this is still available?

[lah7]

Hello, yes, there are two sides to this.

This one is definitely up for grabs. We can already generate the new index into a single, compressed JSON file, but to use that data here, we'll need a "converter" script to translate the new data (+ icons/screenshot filenames) from dist/ to the older applications.json format/folders in this repository.

This can be completed before ubuntu-mate/software-boutique-curated-apps-prototype#9, but bearing in mind the "new index" is outdated by a few years. The structure specification is up-to-date.

Watch out for the Unlisted key in this repository's applications.json - it doesn't exist in the new index, so maybe that could be put aside in say, unlisted.json and appended in later. New data like developer-name can be discarded for this version of Software Boutique.

---

The other task is the opposite direction - a one-time migration from this huge applications.json to the new index repository. I've asked the assignee if they're still up for the task. If we don't hear in a few days, then I'll assume the interest is lost and this can be yours for the taking too, if you're still hungry for flat data migration. 😄

I had painstakingly performed this originally back in 2017 - going through apps and updating some metadata (e.g. developer URLs). Looking back, I found I did commit a script that could be used as a base, albeit for an earlier index revision.

[MungoRae]

OK so I can just use the generated applications json from the dist directory instead of iterating through the folders. Easy enough.

I've had a play around and I get get most of the data I need. However there seems to be missing information from the new format to the old one. I was wondering if you could advise on how to proceed. 😃

• The subcategory key does not seem to appear in the new metadata, but it is wanted in the old format. How should I get a hold of this info?
• The working key also doesn't seem to have an alternative in the new format. Shall I just default to true?
• pre-install is not in the new data. It looks like most of the properties could be worked out like the ppa source, but I'm missing information for the ppa and manual methods. Will these be available once it has been updated?

Hope all this makes sense. I feel like maybe this should wait until #9 has been finished.

[lah7]

The subcategory key does not seem to appear in the new metadata, but it is wanted in the old format. How should I get a hold of this info?

Ahh shoot! 🤦 That provides the "filter" drop-down in the upper-right. This was removed in the new index since the new 2.0 Boutique doesn't intend to have an equivalent.

I guess we'll need to either bring that back in the new index temporarily, or the script could process the older applications.json "in-place" 🤔 By "in-place" I mean update the JSON if there is new data (but left blank if a new app) rather then generating from scratch every time.

---

The working key also doesn't seem to have an alternative in the new format.

The new key is now called listed 🏷️

---

pre-install is not in the new data.

Kinda, the data is more unified with the *-packages keys with differences.

Where it use to be:

"install-packages": "eviacam",
"remove-packages": "eviacam",
.....
"pre-install": {
    "xenial": {
        "method": "ppa",
        "enable-ppa": "ppa:cesar-crea-si/eviacam",
        "source-file": "cesar-crea-si-ubuntu-eviacam-CODENAME"
    },
    "all": {
        "method": "skip"
    }
},

• xenial use this PPA.
• Otherwise, all use package straight from repository (this data is also new - the new index now explicitly states the source repository - e.g. universe)

The new index is:

"apt": {
    "default": {
        "main-package": "eviacam",
        "install-packages": ["eviacam"],
        "remove-packages": ["eviacam"],
        "source": "universe"
    },
    "xenial": {
        "main-package": "eviacam",
        "install-packages": ["eviacam"],
        "remove-packages": ["eviacam"],
        "source": "ppa:cesar-crea-si/eviacam"
    }
},

• default - All releases use these packages from universe repo.
• But if xenial - use these packages from external source (PPA)

---

I suppose you're right that it would be a good idea for ubuntu-mate/software-boutique-curated-apps-prototype#9 to be completed first. 💡 Perhaps we'll need an issue/spreadsheet/script to keep track of the required data to be added afterwards, which will mainly be new applications from the older applications.json that didn't exist in 2017.