explicit output URI

[drernie]

Pull uri directly from pubDir/outDir (configurable), and validate based on output (rather than vice versa)

1. Get Params
2. Find single output URI
3. List of valid keys, in order
4. Can override via config.quilt.outputKeys
5. Infer PackageURI if not already
6. Track dest Key (always?), pass to push
7. Still track non-QuiltPath outputs for overlay and checking
8. Verify that actual outputs match the expected URI; if not, issue a warning

[drernie]

See also:

• publishDir with trailing slash creates trivial revision #191 trailing slash
• Specify dest option for S3 bucket #63 dest option
• pubDir should reuse s3 objects rather than re-copying them #115 do not recopy S3 objects
• manual configuration #185 manual configuration

[drernie]

Tower error for S3 URIs

Downloading plugin [email protected]
WARN: Unrecognized output URI: copy
ERROR ~ Cannot invoke "nextflow.quilt.nio.QuiltPath.getBucket()" because "path" is null

Local error for quilt URIs

$ make PIPELINE=rnaseq rnaseq
ERROR ~ onFilePublish: no match for udp-spec#package=nf-quilt%2frnaseq&path=pipeline_info%2fexecution_trace_2024-09-08_10-10-48.txt

[drernie]

Fixed! Creates package! Sort of:
https://nightly.quilttest.com/b/quilt-example-bucket/packages/test/nfquilt_v080_test_rnaseq_3/tree/5ff3621b86bffea280de7fae2057588dfa3b2ff65c4278b5f3d361bc7b33c4eb/

Lacks subfolders apparently created in:
https://nightly.quilttest.com/b/quilt-example-bucket/tree/test/nfquilt_v080_test_rnaseq_3/

Oddly, the source does NOT seem to have the subfolder:

Sep-10 19:50:37.620 [PublishDir-4] DEBUG nextflow.quilt.QuiltObserver - onFilePublish.Path[/quilt-example-bucket/test/nfquilt_v080_test_rnaseq_3/salmon/RAP1_UNINDUCED_REP2] <- /quilt-demos/scratch/7FSCV3ZE0whSy/a5/dd1ac988ec91d7ded99767fe231f05/RAP1_UNINDUCED_REP2

[drernie]

Success!

https://nightly.quilttest.com/b/quilt-example-bucket/packages/test/nfquilt_v080_test_rnaseq_5