Skip to content

Singularity 0.19.0
Compare
Choose a tag to compare
@pschoenfelder pschoenfelder released this 21 Feb 18:44
Singularity-0.19.0
915c318

Changes in 0.19.0

Check out the 0.19.0 milestone to see new features / bugfixes in detail.

New Features

  • 1668/1650 Adds support for more overrides in run now requests. You can now override items like environment variables for individual runs of a SingularityRequest by POSTing json in the form of a SingularityRunNowRequest
  • 1690 - Adds initial support for Mesos containers, with volume sources and network mapping. These can be specified
    in the containerInfo.volumes and containerInfo.mesos.image sections of the SingularityContainerInfo in your SingularityDeploy
  • 1702 - Updates the internal locking scheme for Singularity to allow more parallel processing. As a result, the concurrency of offer and status update processing can now be tuned, with increased concurrency coming with a cost of increased memory/cpu usage by the scheduler. The following parameters in the mesos section of the SingularityConfiguration impact concurrency and tuning of the scheduler:
    • statusUpdateConcurrencyLimit - The number of status updates that can be processed in parallel. Defaults to 500 and is backed by its own cached thread pool
    • maxStatusUpdateQueueSize - A semaphore limits the number of submissions to the status update cached thread pool. If there are currently no more permits available (i.e. > statusUpdateConcurrencyLimit status updates), these are added to a queue where they wait until more capacity is available. This configuration parameter controls the max size of that queue. It is recommended that this be set a bit above the maximum number of tasks you expect to have active in Singularity at any one time, due to the fact that during reconciliation a status update for each task is sent in rapid succession.
    • offersConcurrencyLimit - The number of offer scoring calculations and checks to be done in parallel. Defaults to 100. This should generally not need to be updated.

Improvements

API/Scheduler

  • 1666 - Set AVAILABILITY_ZONE on the default task environment.
  • 1681 - Provide option to prevent emails for scale events
  • 1653 - Endpoint to allow users to delete pending on-demand tasks
  • 1657 - Include disk resources when scoring offers
  • 1683 - Support usage of DefaultServerFactory to find port
  • 1692 - Wire up overrides for the S3 uploader path.
  • 1616 - Remove swagger jackson version override
  • 1600 - Report disk usage
  • 1690 - Second pass at Mesos containers, with volume sources and network mapping
  • 1682 - Ability to update authorized groups separately from full request
  • 1695 - Allow the cors bundle to be more configurable
  • 1697 - Support uploads to GCS
  • 1703 - Better webhook auth timeouts and exception messages
  • 1699 - add support for overriding S3 endpoint URL for the downloader
  • 1707 - Enable server side encryption params for uploads
  • 1717 - Collect cpu usage for tasks that have just started

UI

  • 1654 - Ability to specify quick links for requests
  • 1636 - New dashboard in Singularity + UI performance improvements
  • 1604 - Disk usage ui
  • 1687 - Correct copy button on InfoBlocks of Task tab. Also, remove clipboard.js.
  • 1704 - Add support for configurable navbar title links.
  • 1701 - Permalinks for bounce and scale modals
  • 1705 - Capitals search on Singularity requests page

Bug Fixes

  • 1609 - Use HostAndPort#getHostText instead of HostAndPort#getHost
  • 1658 - Corrected path for executor download fallback
  • 1659 - Make sandbox logs/ dir world-readable.
  • 1685 - Allow a request in FINISHED state to be redeployed
  • 1688 - Also check pending requests on the track task endpoint
  • 1693 - Properly send task destroy message to executor
  • 1698 - Don't show s3 logs error message as a pop up
  • 1708 - Better catch for statusUpdate exceptions
  • 1718 - Account for task level overrides in usage collection
  • 1719 - Remove unneeded call to unsafeProcessStatusUpdate, fix tasksPerOfferHost check
  • 1711 - Take system usage into account when scoring offers
  • 1696 - Fix nav bar for mobile view
  • 1715 - Use total system cpus, not totalCpus in system calculation
  • 1713 - Fix mobile menu responsiveness
  • 1726 - Don't operate directly on pending tasks during statusUpdate
  • 1721 - Fix cached offer checkin in resourceOffers

Documentation

  • 1700 - Fixed task webhook docs

Thanks

Assets 2
Loading