Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix/non-ascii character support for derivatives & stream logging #564

Merged
merged 17 commits into from
Jan 3, 2024

Commits on May 18, 2023

  1. [DPO3DPKRT-746] fix/large file upload failure (#543)

    * switched to streaming approach for file uploads
    - removed body size check for Nginx to remove filesize cap when uploading
    - added 'Name' to WorkflowReport to assist with identification and file generation
    * cleanup of commented prior implementations for UploadWorker
    * change 'Name' property of WorkflowReport to standard varchar(512)
    EMaslowskiQ authored May 18, 2023
    Configuration menu
    Copy the full SHA
    d13240d View commit details
    Browse the repository at this point in the history

Commits on Aug 30, 2023

  1. Configuration menu
    Copy the full SHA
    bdf593c View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2023

  1. DPO3DPKRT-760/new edan publishing api (#549)

    EDAN has new endpoints that need to be reflected deprecating previous configurations.
    new: https://console.si.edu/apis/3d-api-dev/api/v1.0/admin/upsertResource
    
    * Updated GLTF dependencies and config to use new EDAN endpoints
    
    * Fixes #760: updating to new EDAN endpoints
    - updated defaults when environment variables aren't set to the EDAN
    dev environment for safety.
    
    Note: all developers need to update their environment variables and
    related scripts (e.g. env.DEV.bat) to point to the new EDAN server.
    
        PACKRAT_EDAN_3D_API = https://console.si.edu/apis/3d-api-dev/
    
    Additionally, if doing development against EDAN update the dev
    environment variables for the EDAN server.
    
        PACKRAT_EDAN_SERVER = https://edandev.si.edu/
    EMaslowskiQ authored Oct 6, 2023
    Configuration menu
    Copy the full SHA
    c055be0 View commit details
    Browse the repository at this point in the history
  2. Fixes DPO3DPKRT-764 allowing for easier local Proxy operation

    updated nginx-dev.conf to avoid CORS conflicts in modern browsers when running locally in proxy mode.
    EMaslowskiQ authored Oct 6, 2023
    Configuration menu
    Copy the full SHA
    785f292 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2023

  1. DPO3DPKRT-763/generate download scenes workaround (#552)

    * Temporarily bypass Generate Downloads while development wraps up
    Stakeholders require publishing capability but an issue with the
    generate downloads process pollutes the system with duplicates. this
    temporary fix is only for while that development is wrapped up (active)
    
    * remove additional reference to generating downloads
    * cleanup of unused variables due to temporary fix
    * fixed conditional logic when ingesting and added log output
    EMaslowskiQ authored Oct 10, 2023
    Configuration menu
    Copy the full SHA
    b474b31 View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2023

  1. Configuration menu
    Copy the full SHA
    ebbc293 View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2023

  1. DPO3DPKRT-768/Update LDAP Environment Vars (#554)

    LDAP/LDAPS endpoints have been updated to better enforce compliance and correct configurations. All environment variables (.env.dev, .env.prod, etc.) need to be updated to use the new endpoint/port for authentication against ActiveDirectory to work. No code has been committed as the environment files sit outside of source control.
    
    PACKRAT_LDAP_SERVER=ldap://si-hdc-usdc01.US.SINET.SI.EDU:389
    
    Note: that this will need to be updated further as the port (389) is not their preferred port (636), but that is currently not working as expected and will require deeper configuration and testing.
    EMaslowskiQ authored Nov 6, 2023
    Configuration menu
    Copy the full SHA
    f643eba View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2023

  1. DPO3DPKRT-767/Support Multiple Cook Resources (#555)

    * (new) table for CookResources
    * (new) default data in db for 4 Cook instances
    * (new) 'resources/cook' endpoint for getting best fit for job type. looks
    at stored support for a specific job type, how many jobs are waiting,
    and how many jobs are running. include 'job' query param. accepted
    values: inspect, scene_generation, generate_downloads, photogrammetry.
    
    * (new) centralized functionality into class 'CookResource'
    * (new) getCookEndpoint routine for returning ideal endpoint for CookJobs
    * (new) refactored getCookResources to support returning one/many results
    * (new) added async 'initialize' method to IJob interface to allow point
    for pre-execution initialization of a job. (e.g. find the best fit Cook
    resource for the job)
    * (new) log statements in OCFL path
    * (new) detailed statement on resource selection added to workflow report.
    * (new) exporting CookResourceInfo
    * (new) additional logging forCook resources and utility routine
    
    * (fix) updated 'resources/cook' endpoint to use central class
    * (fix) improved logging output
    * (fix) long timeouts for checking state of a resource
    * (fix) several errors in CookJob were not appended to any reports.
    EMaslowskiQ authored Nov 7, 2023
    Configuration menu
    Copy the full SHA
    52e6cc6 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2023

  1. DPO3DPKRT-767/support-multiple-cook-resources : jobs 'created' support (

    #557)
    
    * (new) factoring in jobs created field for better distribution across
    resources.
    EMaslowskiQ authored Nov 8, 2023
    Configuration menu
    Copy the full SHA
    9aef1e6 View commit details
    Browse the repository at this point in the history

Commits on Nov 30, 2023

  1. Updated to LDAPS. (#559)

    Environment variables need to point to ldaps://ldaps.si.edu:636
    EMaslowskiQ authored Nov 30, 2023
    Configuration menu
    Copy the full SHA
    c85c581 View commit details
    Browse the repository at this point in the history

Commits on Dec 12, 2023

  1. DPO3DPKRT-769/encoding webdav uri for voyager edit (#558)

    (new) improved name sanitizing for ingest titles to avoid conflicts with
    other systems (e.g. Voyager, Cook, etc.). replaces all special
    characters with '-' except for -, _, and .
    (new) updated name sanitizing to include all special characters
    
    (fix) switched to encodeURIComponent to handle & correctly in JobCook
    (fix) localized encodeURIComponent to right before sending to Voyager
    (fix) proper use of encodeURI vs. encodeURIComponent
    EMaslowskiQ authored Dec 12, 2023
    Configuration menu
    Copy the full SHA
    7ca9f71 View commit details
    Browse the repository at this point in the history
  2. Configuration menu
    Copy the full SHA
    c61028e View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    7a86c47 View commit details
    Browse the repository at this point in the history

Commits on Jan 3, 2024

  1. (new) remove all non-ascii characters for filename sanitization (#562)

    (new) remove all apostrophe and single-quotes for filename sanitization
    EMaslowskiQ authored Jan 3, 2024
    Configuration menu
    Copy the full SHA
    97f21ec View commit details
    Browse the repository at this point in the history
  2. DPO3DPKRT-774/streams not reporting generated errors (#563)

    * (new) streams always log any errors now
    * (new) failed stream operations now handle errors and/or return the error
    EMaslowskiQ authored Jan 3, 2024
    Configuration menu
    Copy the full SHA
    184f537 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    9721e2c View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    ae80106 View commit details
    Browse the repository at this point in the history