Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: add support for extending file storage to other schemes and provide a runtime flag for the same #3348

Merged
merged 30 commits into from
Jan 30, 2024

Conversation

Chethan-rao
Copy link
Contributor

@Chethan-rao Chethan-rao commented Jan 14, 2024

Type of Change

  • Bugfix
  • New feature
  • Enhancement
  • Refactoring
  • Dependency updates
  • Documentation
  • CI/CD

Description

This PR includes refactoring for:

  • Enabling easy extension of other file storage schemes.
  • Providing a runtime flag for the file storage feature.
  • Restrict the file storage objects(structs, methods) visibility wherever possible.

Additional Changes

  • This PR modifies the API contract
  • This PR modifies the database schema
  • This PR modifies application configuration/environment variables

Motivation and Context

Closes #3338, Closes #3347

How did you test it?

  • Create a dispute and get the dispute_id

File System

  1. File uploading
curl --location --request POST 'localhost:8080/files' \
--header 'Accept: application/json' \
--header 'api-key: dev_vbE0ODzAkDHFkW2FFQZEAU4VeSk5ij4kWOZl2JXqRtYf50rVqxefvSofYCqre89Q' \
--header 'Content-Type: multipart/form-data; boundary=6467930559a8e09e-babf866111649afb-06f42165ce9a45a7-bc825186544537e1' \
--header 'Accept-Encoding: gzip' \
--form 'purpose="dispute_evidence"' \
--form 'file=@"/Users/sai.harsha/Desktop/dummy.pdf"' \
--form 'dispute_id="dp_Hv9m2MJGRjptYmEYPqFa"'

image

  1. File retrieve
curl --location --request GET 'localhost:8080/files/file_qXP0X6uAgLwSvswj8T41' \
--header 'Accept: application/json' \
--header 'api-key: dev_H3Uf7cssLge9XnWZZfV2M5xwqroMHt46cXh0EwXJQHTdYuBTVC3KVnA5Fo45hagw'

image

  1. File delete
curl --location --request DELETE 'localhost:8080/files/file_z4giuhfczJqbXwNhazqS' \
--header 'Accept: application/json' \
--header 'api-key: dev_vbE0ODzAkDHFkW2FFQZEAU4VeSk5ij4kWOZl2JXqRtYf50rVqxefvSofYCqre89Q'

image

Aws S3

  1. File uploading

image

  1. File retrieve

image

  1. File delete

image (1)

Checklist

  • I formatted the code cargo +nightly fmt --all
  • I addressed lints thrown by cargo clippy
  • I reviewed the submitted code
  • I added unit tests for my changes where possible
  • I added a CHANGELOG entry if applicable

@Chethan-rao Chethan-rao added S-waiting-on-review Status: This PR has been implemented and needs to be reviewed C-refactor Category: Refactor labels Jan 14, 2024
@Chethan-rao Chethan-rao self-assigned this Jan 14, 2024
@Chethan-rao Chethan-rao requested review from a team as code owners January 14, 2024 14:27
crates/common_utils/src/fs_utils.rs Outdated Show resolved Hide resolved
crates/common_utils/src/fs_utils.rs Outdated Show resolved Hide resolved
crates/external_services/src/file_storage/aws_s3.rs Outdated Show resolved Hide resolved
crates/external_services/src/file_storage.rs Outdated Show resolved Hide resolved
crates/external_services/src/file_storage.rs Outdated Show resolved Hide resolved
crates/external_services/src/file_storage/aws_s3.rs Outdated Show resolved Hide resolved
crates/external_services/src/file_storage/aws_s3.rs Outdated Show resolved Hide resolved
crates/external_services/src/file_storage/aws_s3.rs Outdated Show resolved Hide resolved
crates/external_services/src/file_storage/aws_s3.rs Outdated Show resolved Hide resolved
crates/external_services/src/file_storage/aws_s3.rs Outdated Show resolved Hide resolved
crates/common_utils/src/fs_utils.rs Outdated Show resolved Hide resolved
crates/external_services/src/file_storage.rs Outdated Show resolved Hide resolved
crates/external_services/src/file_storage/aws_s3.rs Outdated Show resolved Hide resolved
crates/external_services/src/file_storage/aws_s3.rs Outdated Show resolved Hide resolved
crates/external_services/src/file_storage/aws_s3.rs Outdated Show resolved Hide resolved
crates/common_utils/src/fs_utils.rs Outdated Show resolved Hide resolved
crates/router/src/routes/app.rs Outdated Show resolved Hide resolved
crates/external_services/src/file_storage.rs Outdated Show resolved Hide resolved
Base automatically changed from rename-s3-feature to main January 19, 2024 07:39
@Chethan-rao Chethan-rao requested review from a team as code owners January 19, 2024 10:15
SanchithHegde
SanchithHegde previously approved these changes Jan 24, 2024
SanchithHegde
SanchithHegde previously approved these changes Jan 28, 2024
SanchithHegde
SanchithHegde previously approved these changes Jan 29, 2024
@likhinbopanna likhinbopanna added this pull request to the merge queue Jan 30, 2024
Merged via the queue into main with commit a9638d1 Jan 30, 2024
10 of 12 checks passed
@likhinbopanna likhinbopanna deleted the runtime-flag-for-s3 branch January 30, 2024 08:03
@pixincreate pixincreate removed the S-waiting-on-review Status: This PR has been implemented and needs to be reviewed label Feb 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-refactor Category: Refactor
Projects
None yet
6 participants