-
Notifications
You must be signed in to change notification settings - Fork 302
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Rs hotfix/fix corrupt image uploads #7781
Conversation
@@ -44,7 +46,9 @@ function uploadFile() { | |||
}); | |||
|
|||
return gulp | |||
.src("content/uploads/_working-files/to-process/*") | |||
.src("content/uploads/_working-files/to-process/*", { | |||
removeBOM: true, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note
We'll have to test slides and/or spreadsheet for this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you log in to S3 you will see the PowerPoints I was using to test. Have not deleted them yet.
I tested personally,
- image and slideshow,
- just slideshow
- just image.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How I tested:
- copied test files to
content/uploads/_inbox
- ran
npx gulp upload
for each file type listed below
file | result |
---|---|
test-upload-7-12-2024.png | cannot find file error |
test-upload-7-12-2024.jpg | cannot find file error |
test-upload-7-12-2024.doc | uploads, but file is broken |
test-upload-7-12-2024.pdf | uploads, but file is broken |
test-upload-7-12-2024.xlsx | uploads, but file is broken |
static files are uploaded, accessible on aws but the file is corrupt. Test files are located at static/test-upload-7-12-2024.*
image files, both png
and jpg
are not found:
[13:56:33] Using gulpfile ~/Documents/gsa/digitalgov.gov/gulpfile.js
[13:56:33] Starting 'upload'...
[13:56:33] Starting 'fileTidy'...
Moving file from content/uploads/_inbox/test-upload-7-12-2024.png to content/uploads/_working-images/to-process/test-upload-7-12-2024.png
[13:56:33] Finished 'fileTidy' after 2.58 ms
[13:56:33] Starting 'cleanInbox'...
[13:56:33] Finished 'cleanInbox' after 7.56 ms
[13:56:33] Starting 'writeDataFile'...
[13:56:33] Finished 'writeDataFile' after 20 ms
[13:56:33] Starting 'processImages'...
[13:56:33] Finished 'processImages' after 4.43 ms
[13:56:33] Starting 'removeProcessedImage'...
file is written
[13:56:33] Finished 'removeProcessedImage' after 20 ms
[13:56:33] Starting 'determineWhichToUpload'...
[13:56:33] Finished 'determineWhichToUpload' after 6.8 ms
[13:56:33] Starting 'cleanup'...
[13:56:33] Finished 'cleanup' after 418 μs
[13:56:33] Finished 'upload' after 67 ms
node:events:498
throw er; // Unhandled 'error' event
^
<ref *2> Error: ENOENT: no such file or directory, open '/Users/nlyons/Documents/gsa/digitalgov.gov/content/uploads/_working-images/processed/test-upload-7-12-2024_w1200.png'
Emitted 'error' event on Domain instance at:
at Transform.emit (node:domain:539:12)
at ReadableState.afterDestroy (/Users/nlyons/Documents/gsa/digitalgov.gov/node_modules/streamx/index.js:493:19)
at Transform._destroy (/Users/nlyons/Documents/gsa/digitalgov.gov/node_modules/streamx/index.js:629:5)
at ReadableState.updateNonPrimary (/Users/nlyons/Documents/gsa/digitalgov.gov/node_modules/streamx/index.js:386:16)
at ReadableState.update (/Users/nlyons/Documents/gsa/digitalgov.gov/node_modules/streamx/index.js:367:71)
at ReadableState.updateReadNT (/Users/nlyons/Documents/gsa/digitalgov.gov/node_modules/streamx/index.js:536:10)
at process.processTicksAndRejections (node:internal/process/task_queues:77:11) {
errno: -2,
code: 'ENOENT',
syscall: 'open',
path: '/Users/nlyons/Documents/gsa/digitalgov.gov/content/uploads/_working-images/processed/test-upload-7-12-2024_w1200.png',
domainEmitter: <ref *1> Transform {
Note
I removed the file.upload.do
from the gulpfile.js
and can confirm that the content/uploads/_working-images/processed/
is created.
Line 40 in 4375f5f
exports.upload = series(file.prep.do, file.process.do, file.upload.do); |
I don't see those folders when I use the new code changes:
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@nick-mon1 very strange, when I run upload on a .xlsx file I see this
It opens as spreadsheet as well at this link
https://s3.amazonaws.com/digitalgov/static/test-sheet-riley-v1.xlsx
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm assuming you're testing all the files uploading at once. I'll do that as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@RileySeaburg I tested each file one at a time, did not try all of them at once since they won't typically happen
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is weird.
I'm stepping off the happy path and am going to debug a multiple upload scenario.
That's where I'm able to get errors.
Going to close this for now |
Summary
Implements the fix mentioned here
Closes #7780.
Preview
Link to Preview
Solution
How To Test
content\uploads\_inbox\__add image or static files to this folder__
Dev Checklist