GPII-3921: Update gpii-express-user to use modern dependencies, especially gpii-json-schema.

[the-t-in-rtf]

See GPII-3921 for details.

@amb26, this should be reviewed after the related gpii-json-schema pull that fixed a blocking bug.

[the-t-in-rtf]

@sgithens, here's the pull to update gpii-express-user to support your work. I left the version of kettle alone, it still matches universal master. As I mentioned on IRC, there's a dev release for you to use:

1.0.2-dev.20190628T073950Z.6efdc50.GPII-3921

[the-t-in-rtf]

@amb26, I tried using the resource loader support for promises to bring in schema holder content. If I launch tests/js/launch-test-harness.js with debugging enabled, I get an error like the following:

14:55:58.229:  FATAL ERROR: Uncaught exception: that.generateSchema is not a function
fluid.js:256TypeError: that.generateSchema is not a function
    at gpii.schema.schemaHolder.generateIfNeeded (/Users/duhrer/Source/rtf/gpii-express-user/node_modules/gpii-json-schema/src/js/common/schemaHolder.js:22:84)

If I set a breakpoint at that line in schemaHolder.js, I can see that one of the invokers defined in that grade exists as part of that, but not the required generateSchema invoker. Please assist.

[the-t-in-rtf]

Turns out that generateSchema will exist if there's an IoC reference to it, and it also returns the schema. That appears to have got me past the error, I will test it tomorrow.

[sgithens]

@the-t-in-rtf Now that the resulting gpii-json-schema pull has been merged in, do you think we can look at merging this work back in to master?

[the-t-in-rtf]

@sgithens, thanks for reminding me here and in chat. I had actually completely forgotten that I'd already made a pass at this. It's been so long that I hadn't even checked out the repo on my new machine yet, and I've had that for months.

Anyway, @amb26, this is ready for review, and now more urgently needed. The changes are adjustments to use the new resource loader aware stuff from gpii-handlebars and gpii-json-schema, it would be good to get another pair of eyes on at least that part.

@sgithens, I did cut a dev release at the time if that helps at all, it's:

1.0.2-dev.20191014T143404Z.150969f.GPII-3921

[sgithens]

@the-t-in-rtf Awesome thanks. I might try the dev release, although it does actually work as is (but I'm just using the password unlocking, may need more of the repo soon).
The more urgent reason for getting this through is that I just want to start upgrading js libraries with security issues, which seems important since this is an account utility.

[amb26]

Be a bit clearer about the workflow here, and what object is returned, since the logic is a little complex. If the argument is a string, it is looked up, and any value other than undefined is returned, else {} .... otherwise the input argument is returned. Does this really need to be this complex? I don't see that it is used elsewhere in the pull.
In my experience, functions like this which include lots of special cases in order to be "helpful" end up causing problems down the line.

[the-t-in-rtf]

I think it was left over from a previous approach, I have deleted it.

[amb26]

Note that it might not be of type {Object} if some value other than undefined or an Object is stored in the global name.

[the-t-in-rtf]

It's gone.

[amb26]

A few of these have become further outdated since this pull was issued, e.g. grunt -> 1.1.0, eslint -> 6.8.0 (doesn't lint-all pull in its own deps anyway?)
Also please could you try updating to kettle 1.12.0 and infusion 3.0.0-dev.20200317T122331Z.1aa7ea8a1.FLUID-6145 - thanks

[amb26]

Due to some bug in Chrome and/or chromedriver 81, I am no longer able to run the webdriver tests - https://stackoverflow.com/questions/61321143/12296266720420-163936-459errorbrowser-switcher-service-cc238-xxx-init-er
I get this error, and Chrome merely starts up with the url "data:;" and then the tests hang

[gpii-bot]

CI job failed: https://ci.gpii.net/job/gpii-express-user-tests/1/

[the-t-in-rtf]

I initially got an error related to a chrome/chromedriver mismatch, just fixed that. I still see the same issue when running in Vagrant locally, although interestingly the gpii-webdriver tests pass with the same VM image and chrome/chromedriver version.

[gpii-bot]

CI job failed: https://ci.gpii.net/job/gpii-express-user-tests/3/

[the-t-in-rtf]

Now it's broken in the same way as everything else. My next ideas are:

1. Looking for errors that are preventing the tests from launching Chrome properly.
2. Rewriting these tests to use Testem (couple of hours, probably).

[amb26]

Cheers @the-t-in-rtf - I've rolled infusion 3.0.0-dev.20200512T100752Z.0d704c4.FLUID-6148 from the current head of that branch. Interesting that my choice to keep FLUID-6148 and FLUID-6145 development separated seems somewhat vindicated which otherwise might have been a pointless piece of bureaucracy ....

[amb26]

Ok - there is now 3.0.0-dev.20200514T203129Z.505489c.FLUID-6145 which fixes this issue, you should be able to update to it

[gpii-bot]

CI job failed: https://ci.gpii.net/job/gpii-express-user-tests/9/

[the-t-in-rtf]

Provisioning error in the VM creation stage. Ok to test again.

[gpii-bot]

CI job failed: https://ci.gpii.net/job/gpii-express-user-tests/10/

[the-t-in-rtf]

@amb26, just refreshing this with the newer release. It still has the circular dependency error when run locally in a VM.

[the-t-in-rtf]

So, I updated all dependencies just to make sure I hadn't had to make a code change in either gpii-handlebars or gpii-json-schema. No change. What's odd is that if I sidestep the issues in the schema holder grades, I run into the same issue with options inheritance in the test grades. I committed my work in progress so you can see the new variation.

[gpii-bot]

CI job failed: https://ci.gpii.net/job/gpii-express-user-tests/11/

[gpii-bot]

CI job failed: https://ci.gpii.net/job/gpii-express-user-tests/12/

[amb26]

Cheers - I have rolled a fresh 3.0.0-dev.20200528T154535Z.9cd9261.FLUID-6145 which contains even an even more egregious version of the hack to avoid tripping this as a circular dependency - all of this code so badly needs rewriting.
The next issue you will run into is that gpii-mail-test will require a fluid.copy on this line https://github.com/GPII/gpii-mail-test/blob/master/src/js/simpleSmtpServer.js#L30 in order to make it FLUID-6145 compliant. And then after that you will run into an issue where the test fixtures mysteriously hang just after the server starts up - but I will have to leave you in the hands of that Giant Rat of Sumatra. Unfortunately with all of the recent updates to the FLUID-6145 branch I think we will have to say that the FLUID-6148 branch is now extinct and we will have to tough it out by updating all of our ecosystem all the way if we can.

[the-t-in-rtf]

The latest failure looks like another options-munging issue where I used the old pattern of stuffing material to be inspected later on into the request:

https://github.com/the-t-in-rtf/gpii-express-user/blob/403e8ae1fe5d466c701bc7258d3fdd3466225324/tests/js/server/forgot-tests.js#L223

I'll add a member variable or other safe stash for that, and then fingers crossed.

[the-t-in-rtf]

Ugh, it was nothing nearly as innocent, it was straight-up options munging to avoid writing a proper termMap and passing direct options to incorporate into the path. Fixing it properly now.

[the-t-in-rtf]

When we switched to a FLUID-6145 variant I had to update the tests to avoid options munging. They now pass locally, fingers crossed to round this out shortly.

[gpii-bot]

CI job passed: https://ci.gpii.net/job/gpii-express-user-tests/13/

[the-t-in-rtf]

The second build is just with the full release of gpii-mail-test, I saw once that passes we're good here.

[gpii-bot]

CI job passed: https://ci.gpii.net/job/gpii-express-user-tests/14/

[amb26]

I think this TODO can be done now?

[the-t-in-rtf]

Thanks for catching this, fixing it now.

[amb26]

And this one

[the-t-in-rtf]

Last build seemed to hang and was killed off. Tests still pass locally, the TODO changes will kick of another build, let's see how it goes.

[gpii-bot]

CI job passed: https://ci.gpii.net/job/gpii-express-user-tests/15/

[the-t-in-rtf]

Ready to merge this when you are, @amb26.

[the-t-in-rtf]

Self-merged based on conversations with reviewer.