Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rs hotfix/fix corrupt image uploads #7781

Closed
wants to merge 2 commits into from

Conversation

RileySeaburg
Copy link
Member

@RileySeaburg RileySeaburg commented Jul 11, 2024

Summary

Implements the fix mentioned here
Closes #7780.

Preview

Link to Preview

Solution

How To Test

  1. Add an image to content\uploads\_inbox\__add image or static files to this folder__
  2. Run npx gulp upload
  3. Click the link that appears in the generated YAML
  4. Notice the image now renders.

Dev Checklist

  • PR has correct labels
  • A11y testing (voice over testing, meets WCAG, run axe tools)
  • Code is formatted properly

@RileySeaburg RileySeaburg added bug Things that are not working as they should be Content: Images labels Jul 11, 2024
@RileySeaburg RileySeaburg self-assigned this Jul 11, 2024
Copy link

🔍 Preview in Federalist

@@ -44,7 +46,9 @@ function uploadFile() {
});

return gulp
.src("content/uploads/_working-files/to-process/*")
.src("content/uploads/_working-files/to-process/*", {
removeBOM: true,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

We'll have to test slides and/or spreadsheet for this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you log in to S3 you will see the PowerPoints I was using to test. Have not deleted them yet.

I tested personally,

  • image and slideshow,
  • just slideshow
  • just image.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How I tested:

  • copied test files to content/uploads/_inbox
  • ran npx gulp upload for each file type listed below
file result
test-upload-7-12-2024.png cannot find file error
test-upload-7-12-2024.jpg cannot find file error
test-upload-7-12-2024.doc uploads, but file is broken
test-upload-7-12-2024.pdf uploads, but file is broken
test-upload-7-12-2024.xlsx uploads, but file is broken

static files are uploaded, accessible on aws but the file is corrupt. Test files are located at static/test-upload-7-12-2024.*

image files, both png and jpg are not found:

[13:56:33] Using gulpfile ~/Documents/gsa/digitalgov.gov/gulpfile.js
[13:56:33] Starting 'upload'...
[13:56:33] Starting 'fileTidy'...
Moving file from content/uploads/_inbox/test-upload-7-12-2024.png to content/uploads/_working-images/to-process/test-upload-7-12-2024.png
[13:56:33] Finished 'fileTidy' after 2.58 ms
[13:56:33] Starting 'cleanInbox'...
[13:56:33] Finished 'cleanInbox' after 7.56 ms
[13:56:33] Starting 'writeDataFile'...
[13:56:33] Finished 'writeDataFile' after 20 ms
[13:56:33] Starting 'processImages'...
[13:56:33] Finished 'processImages' after 4.43 ms
[13:56:33] Starting 'removeProcessedImage'...
file is written
[13:56:33] Finished 'removeProcessedImage' after 20 ms
[13:56:33] Starting 'determineWhichToUpload'...
[13:56:33] Finished 'determineWhichToUpload' after 6.8 ms
[13:56:33] Starting 'cleanup'...
[13:56:33] Finished 'cleanup' after 418 μs
[13:56:33] Finished 'upload' after 67 ms
node:events:498
      throw er; // Unhandled 'error' event
      ^

<ref *2> Error: ENOENT: no such file or directory, open '/Users/nlyons/Documents/gsa/digitalgov.gov/content/uploads/_working-images/processed/test-upload-7-12-2024_w1200.png'
Emitted 'error' event on Domain instance at:
    at Transform.emit (node:domain:539:12)
    at ReadableState.afterDestroy (/Users/nlyons/Documents/gsa/digitalgov.gov/node_modules/streamx/index.js:493:19)
    at Transform._destroy (/Users/nlyons/Documents/gsa/digitalgov.gov/node_modules/streamx/index.js:629:5)
    at ReadableState.updateNonPrimary (/Users/nlyons/Documents/gsa/digitalgov.gov/node_modules/streamx/index.js:386:16)
    at ReadableState.update (/Users/nlyons/Documents/gsa/digitalgov.gov/node_modules/streamx/index.js:367:71)
    at ReadableState.updateReadNT (/Users/nlyons/Documents/gsa/digitalgov.gov/node_modules/streamx/index.js:536:10)
    at process.processTicksAndRejections (node:internal/process/task_queues:77:11) {
  errno: -2,
  code: 'ENOENT',
  syscall: 'open',
  path: '/Users/nlyons/Documents/gsa/digitalgov.gov/content/uploads/_working-images/processed/test-upload-7-12-2024_w1200.png',
  domainEmitter: <ref *1> Transform {

Note

I removed the file.upload.do from the gulpfile.js and can confirm that the content/uploads/_working-images/processed/ is created.

exports.upload = series(file.prep.do, file.process.do, file.upload.do);

Screenshot 2024-07-12 at 1 29 54 PM

I don't see those folders when I use the new code changes:

Screenshot 2024-07-12 at 2 04 11 PM

Copy link
Member Author

@RileySeaburg RileySeaburg Jul 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nick-mon1 very strange, when I run upload on a .xlsx file I see this

image

It opens as spreadsheet as well at this link

https://s3.amazonaws.com/digitalgov/static/test-sheet-riley-v1.xlsx

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm assuming you're testing all the files uploading at once. I'll do that as well.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@RileySeaburg I tested each file one at a time, did not try all of them at once since they won't typically happen

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is weird.

I'm stepping off the happy path and am going to debug a multiple upload scenario.

That's where I'm able to get errors.

@RileySeaburg
Copy link
Member Author

Going to close this for now

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Things that are not working as they should be Content: Images
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bug: Images are uploading broken
3 participants