sos-4.5.4

The sos team is pleased to announce the release of sos-4.5.4. This release focused on expanding support in policies for more and multiple package managers. The next release is slated for June 22.

Policies

• Added a snap package manager abstraction so that policies and plugins may inspect package installations using the snap manager.
• Added a MultiPackagerManager that allows policies to leverage multiple package managers on the same system - e.g. a system using both dpkg and snaps.
• The Ubuntu policy will now use both the dpkg and snap package managers, so plugins will be enabled based on the installation of a package in either dpkg or snap format.
• The dpkg package manager will now properly remove recently-uninstalled packages from the returned package list.
• The Mariner policy has been renamed to Azure Linux to match the new upstream naming.

Report Changes

• Plugins will now collect truncated strings into the report before collecting command output.

Plugin Changes

• New plugins: rhc, microk8s
• The alternatives plugin now supports Ubuntu installations.
• The dnf plugin has removed the superflous use of --assumeno from commands that cannot generate the need for user input. Command file names will change accordingly.
• The apport plugin will now collect /var/crash if --all-logs is used.
• The lxd plugin will now collect logs and configuration files from locations based on snap installations as well.
• The apt plugin will now obfuscate credentials in list files.
• The kernel plugin will now collect /var/lib/systemd/pstore.
• The powerpc plugin will now collect RMC status logs and invscout logs.
• The foreman plugin will now collect qpid-stat output.
• The landscape plugin now collects more logs.

Collect Changes

• Fixed a bug that would prevent the juju cluster from collections when a subbordinate's parent or its units were missing.
• Fixed a bug where collect could end up skipping the local node if that node was part of the cluster but was not forcibly removed via strict_node_list (primarily pacemaker clusters).

For full information on the changes contained in this release, please refer to the Git commit logs. Further release information and tarballs are available at:

https://github.com/sosreport/sos/releases/tag/4.5.4

Please report any problems to the sos-devel mailing list, or the GitHub issue tracker:

https://github.com/sosreport/sos/issues/

The team would like to thank everyone who contributed fixes, new features, testing, and feedback for this release.

sos-4.5.3

The sos team is pleased to announce the release of sos-4.5.3. This release follows our recent changes to adopt a faster release cadence, and as such is smaller in scope than previous releases for the project. The next release is slated for May 26.

Policy Changes

• Added a policy for CBL-Mariner Linux
• Fixed an issue in the ubuntu policy that could prevent archive uploads when users specified a manual --upload-url

Report Changes

• Usage of --enable-plugins, --skip-plugins, and --only-plugins from the command line will now properly override those values set by a preset if the preset added them to a conflicting option

Plugin Changes

• The lustre plugin will now collect ldiskfs information
• The zfs plugin will now collect from /proc/spl
• The firewalld plugin will now collect the ipsets, policies, and helpers subdirectories within /etc/firewalld/
• The iscsi plugin will now properly obfuscate password_in strings in collected files
• The ceph plugin will now collect a dump of mon's config database
• The powerpc plugin will now collect lsslot, amsstat, and opal elogs output
• The virsh plugin will now scrub spice passwords in virt-manager logs

Collect changes

• Added both a new transport and cluster profile for Juju environments

For full information on the changes contained in this release, please refer to the Git commit logs. Further release information and tarballs are available at:

https://github.com/sosreport/sos/releases/tag/4.5.3

Please report any problems to the sos-devel mailing list, or the GitHub issue tracker:

https://github.com/sosreport/sos/issues/

The team would like to thank everyone who contributed fixes, new features, testing, and feedback for this release.

sos-4.5.2

The sos team is pleased to announce the release of sos-4.5.2. This release follows our recent changes to adopt a faster release cadence, and as such is smaller in scope than previous releases for the project. The next release is slated for April 27.

Global Changes

• Several stale dependencies have been dropped from the rpm packaging
• sos.spec has been migrated to using SPDX license syntax
• RPM builds of sos will now include a tmpfiles configuration so skip over /var/tmp/sos*, thus avoiding constant AVC errors from the service

Report Changes

• New option journal-size: controls how large sos will collect journal output of. Default is 100MB
• New option low-priority: will now cause sos to attempt to self-constrain its impact on the system.
  • This option currently attempts to set the sos process to an idle IO class, and sets our niceness to 19
• New preset minimal: uses the low-priority option as well as reducing plugin timeouts and file collection sizes
• Archive contents are now ordered such that version.txt, manifest.json, and sos_logs/ are now first in the tarball, which should assist with any in-memory handling of tarballs via automation
• Fixed a bug that would cause double logging if a preset set verbosity levels

Plugin Changes

• The frr plugin will now collect command output from a container if that is how it is deployed

• The unpackaged plugin will now list unpackaged symlinks and not the symlink's target

• The iprconfig plugin is now gated by the sg kernel mod being loaded

• Fixed a bug that would cause the CosLogs variant to be loaded for every distribution

• The microshift plugin has been updated to use oc get commands

• Plugin.do_file_sub() for plugin post-processing is now always case-insensitive in pattern matching

Cleaner Changes

• Fix a bug where the finalizing the archive would fail if cleaner was instructed to obfuscate the keyword tmp

For full information on the changes contained in this release, please refer to the Git commit logs. Further release information and tarballs are available at:

https://github.com/sosreport/sos/releases/tag/4.5.2

Please report any problems to the sos-devel mailing list, or the GitHub issue tracker:

https://github.com/sosreport/sos/issues/

The team would like to thank everyone who contributed fixes, new features, testing, and feedback for this release.

sos-4.5.1

The sos team is pleased to announce the release of sos-4.5.1. This release follows our recent changes to adopt a faster release cadence, and as such is smaller in scope than previous releases for the project. The next release is slated for March 29.

Global Changes

• Build failures for snaps will now be available within the CI run in which a particular build failed.

Report Changes

• A plethora of new tags and changed tags have been implemented across many collections to assist with Insights inspection.
• Fixed a bug where a potential duplicate command when run in a container could result in an incorrectly handled exception within the archive. If duplicate commands are called within the same container from the same plugin, there will no longer be an error.
• The ocp preset will no longer use the --verify option

Plugin Changes

• New plugins: ceph_iscsi, microshift, microshift_ovn
• The azure plugin has been updated to use a newer endpoint for metadata retrieval
• The rhui plugin will now properly obfuscate certain sensitive keys from collections
• The composer plugin will now capture /etc/osbuild-composer
• Running an ostree fsck is now gated behind the new ostree.fsck plugin option, and not tied to --verify

For full information on the changes contained in this release, please refer to the Git commit logs. Further release information and tarballs are available at:

https://github.com/sosreport/sos/releases/tag/4.5.1

Please report any problems to the sos-devel mailing list, or the GitHub issue tracker:

https://github.com/sosreport/sos/issues/

The team would like to thank everyone who contributed fixes, new features, testing, and feedback for this release.

sos-4.5.0

The sos team is pleased to announce the release of sos-4.5.0. This is the first release of a new, faster release cadence that will see new versions cut approximately every 4 weeks going forward. This release sees design changes to plugins, most notably by now supporting the manual collection and compilation of diagnostic data not strictly limited to existing command output or file collection. Highlighted changes are noted below.

Global

• Snaps are now created automatically whenever a change is pushed to main. These snaps are available via snapcraft under latest/edge
• python3-magic is now a soft dependency, and if not present sos will use a less sophisticated method for determining if a file is binary or not
• distutils usage has been fully replaced by setuptools

Policies

• Added support for Anolis OS
• Added support for Circle Linux
• Added support for OpenCloudOS
• When loaded for an sos collect execution, a policy's remote_exec will now directly use the loaded transport's run_command functionality, rather than re-building command strings
• The Debian policy has been updated to correctly identify many more and newer Debian versions
• Fixed an issue with the RHEL policy that would prevent non-anonymous upload to the failover SFTP server if a case ID was not provided

Report

• A "tag_summary" section has been added to the report manifest.json. This is a dictionary with keys being tags that were created during collection, with values being all files sharing that tag
• sos_get_command_output() timeout handling will now properly handle the situation where a command's child process deadlocks but the timeout wrapper was able to kill the parent process, but left the child behind.
• Estimate mode for report will now report real disk usage, rather than apparent size

Plugins

• New plugins: containerd, fapolicyd
• The kernel plugin will now collect modprobe.d/*conf files
• The hpssm plugin will now collect show detail output per array and slot
• The crio plugin now supports CoS systems
• The dnf plugin will now properly obfuscate password variable values
• The flow of plugin code execution has been changed
  • setup() is now strictly for determining what collections to perform, outside of calls to collect_cmd_output() in order to build further sets of commands
  • _collect_plugin() is now used to actually perform the collections specfied by setup(). This now includes tailed file collections which were previously part of setup()
  • collect() can now be used to perform ad-hoc/manual collections that are not strictly part of command output collection. If a plugin needs to manipulate data from commands or system information and then write it out manually (such as with the rpm plugin generating package output), it should now be done inside the collect() method
• The composer plugin has been overhauled and updated for the new versions of composer
• Enablement triggers have been expanded for the xfs, nvme, firewall_tables, and krb5 plugins
• The virsh plugin will now collect more information about the host/hypervisor system
• The various ceph_* plugins have been updated to collect the appropriate data for both older and more recent versions of ceph, including traditional installations and those deployed with cephadm

Collect

• Cluster profiles may now directly specify sos options to enforce on per-node report collections
• Added a new cluster profile for Red Hat Ceph Storage 5
  • This new profile may work for other Ceph environments deployed with cephadm, but that is not tested
• Added a new saltstack transport

Cleaner|Mask

• The --domains option is now validated for items that look like a domain
• Fixed an issue where a file with encoding issues would be aborted by clean, but left in the archive. Files will now either show replaced content or be removed from the archive, rather than being left unobfuscated in any manner
• sos will no longer attempt to obfuscate the temp directory the archive is in before moving the archive at the end of cleaning
• Added a new parser to support IPv6 obfuscation

For full information on the changes contained in this release, please refer to the Git commit logs. Further release information and tarballs are available at:

https://github.com/sosreport/sos/releases/tag/4.5.0

Please report any problems to the sos-devel mailing list, or the GitHub issue tracker:

https://github.com/sosreport/sos/issues/

The team would like to thank everyone who contributed fixes, new features, testing, and feedback for this release.

sos-4.4

The sos team is pleased to announce the release of sos-4.4. This release includes a number of performance and stability fixes, as well as several developer-focused quality of life changes to various APIs. A subset of the changes in this release are highlighted below.

Global Changes

• Added a Georgian translation

Report Changes

• Fixed a bug where a plugin's default timeout was not being correctly evaluated.
• Improved binary file detection by leveraging python3-magic. SoS now has a dependency on this package/module.
• Encryption of the final archive now supports the use of the SOSENCRYPTKEY or SOSENCRYPTPASS environment variables to allow users to not pass these values via command line options.
• The --list-plugins output will now report actual, not default, option values for plugin options.
• The locale used for collections has been updated from C to C.UTF-8
• Loop devices are now more accurately enumerated on host systems.
• Network devices are now enumerated and saved to the network key for the devices dict handed to plugins.

Plugin Changes

• Added 2 new plugins: collectl, shmcli
• Container IDs may now be used with Plugin.container_exists(), not just container names.
  • This also means that Plugin.exec_cmd() can now function with IDs.
• The container-based collection paths for several OSP-related plugins has been updated.
• Plugins may now define a default set of environment variables for their collections.
• The scsi plugin will now collect SCSI persistent reserve commands.
• The openshift plugin may now reference a local kubeconfig file for authentication, via the openshift.kubeconfig option.
• The no-oc option for the openshift plugin has been renamed to with-api to be more obvious about what collections it controls.
• The openshift plugin now does not collect oc command output by default (with-api is defaulted to False).
• The yum and dnf plugins have been merged. Only the dnf plugin remains.
• The fibrechannel plugin will now collect HBA device logs.
• The stratis plugin has been updated to collect stratis 3.x commands.
• The ceph_mgr plugin will now collect ceph orchestrator commands.
• The hpssm plugin now iterates over multiple slots for collections.
• Binary files will no longer be tailed by add_copy_spec() when their size exceeds the specified sizelimit.
• add_blockdev_cmd() has been refactored into add_device_cmd() which will be used for device command iteration, only now not limited to block devices.
  • add_device_cmd() may be used to iterate over devices enumerated by sos during initialization(e.g. block, ethernet, network, and more), or manually compiled lists of devices.
• Plugins no longer log string content of tailed collections or manually compiled strings.

Policy Changes

• Fixed a bug in the Red Hat policy that would incorrectly override a provided --upload-url to Red Hat's SFTP server if incorrect credentials were given.
• Fixed usage of GenericPolicy so that it can actually be used to collect minimal reports from distributions without explicit sos policies.

Collect Changes

• New cluster profile: rhosp
  • Note: This profile is designed for use with Red Hat OpenStack environments, and as such makes some assumptions about the deployment. It may work for non-Red Hat environments that are similarly deployed, but that is not guaranteed.
• Greatly improved robustness and integration with OpenShift Container Platform to provide consistent report collections and cleanup.
• Refactored node enumeration for pacemaker clusters. Users should now see more accurate and consistent node lists from these clusters.
• By default, the ocp cluster will now attempt to use a well-known stock kubeconfig file present in order to enable collections from the oc API.
• A new with-api option for the ocp cluster has been added to control whether or not any node is used to collect API data from the cluster.
• Fixed a bug where the --image option would be ignored for OCP cluster collections using containerized hosts.
• The use of exit() is now standardized and should be used in all cases where collect needs to stop execution.

Clean|Mask Changes

• Nothing in the /etc/sos/cleaner directory will be captured in sos reports now.
• Parsers will now pre-generate regex lists for matching obfuscation patterns, rather than creating them on the fly.
  • As such, performance should be noticeably improved for most use cases.
• Fixed a bug where sos would obfuscate the tmpdir path of the final results, which would result in a "No such file or directory" error when trying to perform cleanup.
• Individual parsers (e.g. the hostname parser or ip parser) may be selectively disabled now via the --disable-parsers option.

Help Changes

• sos help was relatively unchanged this release. Various plugins, collect profiles, and other sections of the project were given extended sos help information alongside other improvements and fixes to those components.

For full information on the changes contained in this release, please refer to the Git commit logs. Further release information and tarballs are available at:

https://github.com/sosreport/sos/releases/tag/4.4

Please report any problems to the sos-devel mailing list, or the GitHub issue tracker:

https://github.com/sosreport/sos/issues/

The team would like to thank everyone who contributed fixes, new features, testing, and feedback for this release.

sos-4.3

The sos team is pleased to announce the release of sos-4.3. This release includes a number of quality-of-life changes to both end user experience and for contributors dealing with the plugin API. A subset of the changes in this release are highlighted below.

Global Changes

• Failed CI test logs will now be uploaded to GCP and available for review via the CirrusCI test page for the test that failed.
• sos is now available as a snap.

Policy Changes

• Added a policy for OpenEuler.
• Added a policy for RockyLinux.
• Sysroot determination and usage is now determined once during policy initialization, and sysroot usage is now fully consistent across sos components.
• Added a container runtime abstraction for crio.

Report Changes

• Added 6 new plugins: dellrac, gcloud, openshift_ovn, perccli, udisk, unbound
• The ceph plugin has been spit into ceph_common, ceph_mds, ceph_mgr, ceph_mon, ceph_osd, and ceph_rgw.
• A new --estimate-only option is available that will attempt to estimate the amount of disk space used by sos for a given set of options.
• Fixed a bug where host services would not be inspected properly if sos was run from a container.
• The runc plugin has been removed as it was redundant with other container plugins.
• The openvswitch plugin has been expanded to collect OpenFlow protocols.
• Fixed a bug in the unpackaged plugin that could cause the plugin to hang on recursive syminks.
• The networking plugin is now better organized when collecting from network namespaces.
• Fixed a bug in the virsh plugin that would cause no information to be collected, due to newer virsh commands requiring a TTY.
• When using --all-logs, and/or for collections where sizelimit=0, sos will now write that output directly to disk, rather than storing it in memory.

Plugin API

• Plugin options are now defined via the PluginOpt class, rather than as tuples.
  • More information for contributors is available in the wiki
• SCL plugin commands will no longer overwrite PATH.
• Renamed get_container_logs() to add_container_logs() which should be used to copy a container's stdout logs to a report archive.
• add_cmd_output() now supports a container parameter which, if set, will cause the specified commands to be executed inside the specified container(s), provided those container(s) exist.
• add_copy_spec() also now supports a container parameter to allow specified files to be copied from container(s).
  • Note that this currently does not function for crio managed containers, as crio does not support this functionality.

Clean|Mask Changes

• sos clean can now also obfuscate insights archives, as well as "plain" directories or tar archives.
• /etc/hosts is now sourced for domains to obfuscate, alongside hostname output.
• Improved username sourcing.
• The username and keyword parsers are now much more efficient when handling large lists of items to obfuscate.
• Domain and subdomain matching has been simplified.
• Improved per-parser file and keyword skipping to provide for more reliable and useful obfuscations.
• Reduced false positive matches for hostnames, domains, and mac address.
• Domain and sub-domain matching with the hostname parser should now be more reliable.

Collect Changes

• Node connections have been abstracted away from ssh directly as the remote transport. While ssh is still the default, other mechanisms may be used for remote connections.
  • This may be controlled via the --transport option.
  • Added a new oc transport for use in OCP environments. Using this transport will cause sos to create a temporary project within the OCP environment being collected from.
• The --master option has been fully removed and replaced with --primary, or the aliases --controller, or --manager.
• Cluster cleanup is now more consistently called regardless of if collect was successful or not.

Help (new!)

sos help is a new sub-command for sos with this release. Using sos help $topic will provide users with more information on specific aspects or sos, such as more detailed plugin information, than is available via normal --help output or man pages.

To see what kind of information is available, try sos help or sos help report to get started.

For full information on the changes contained in this release, please refer to the Git commit logs. Further release information and tarballs are available at:

https://github.com/sosreport/sos/releases/tag/4.3

Please report any problems to the sos-devel mailing list, or the GitHub issue tracker:

https://github.com/sosreport/sos/issues/

The team would like to thank everyone who contributed fixes, new features, testing, and feedback for this release.

sos-4.2

The sos team is pleased to announce the release of sos-4.2. This release contains numerous improvements and bug fixes to several components within sos, including an overhaul to the project's test suite and infrastructure.

Global Changes

• SoS now uses the Avocado Framework for our test suite. This has provided significantly increased testing coverage as well as providing a much easier path to writing new tests going forward.
  • For more information on how to build new test cases, see the How to Write a Test wiki page.
• Every PR now has the new test suite run against a current and previous version of Fedora and Ubuntu, as well as the current version of CentOS Stream.
  • Additional testing has been provided for The Foreman project.
• The use of the term master has been deprecated as part of a neutral language initiative. It will be removed sometime between this release and the 4.3 release.
  • As of the release of sos-4.2, the master branch on GitHub will be renamed/switched to main.
• PackIt is now enabled for all PRs to provide a copr RPM for those changes to allow for easier testing.
• Verbosity levels now have more standardized meanings:
  • 0: The default, do not print debug messages to either the logs or console
  • 1: Print debug messages to the log, but not console
  • 2: Print debug messages to both the log and console
  • 3: Enable debug messages from archive operations

Policy Changes

• Fixed an issue that would prevent uploading archives to Ubuntu/Canonical's upload server.
• Users may now toggle SSL verification on or off for HTTPS uploads of archives.
• Added support for SFTP uploads.
  • The public FTP dropbox for Red Hat systems has been removed in favor of the new SFTP upload, as the public FTP dropbox is being decomissioned.
• Removed unused IBM policy overrides.

Report Changes

• Added 13 new plugins: openstack_database, zvm, usbguard, ostree, flatpak, quacli, storcli, smclient, opensvc, pulpcore, rhui, migration_results, opencontrail
• The foreman plugin has been split into new plugins. These are now: foreman, foreman_proxy, and foreman_installer.
• Removed the upstart plugin.
• The order of entries in html and text reports will now be sorted based on the name of the entry.
• A new --cmd-timeout option has been added, that allows users to specify a maximum timeout for all commands that sos executes.
• sos report will no longer abort execution when a non-existent plugin is requested to be skipped.
• Commands that would be skipped are now reported in dry runs.
• A new --namespaces option has been added that allows users to limit the number of network namespaces that plugins will iterate over. The default value of 0 means unlimited.
  • Plugins that do iterate over network namespaces also have a plugin-specific namespaces option as well. The plugin option will override any value passed to the global --namespaces option to allow users to have fine-grained control over this kind of iteration.

Plugin Changes

• The Plugin class no longer contains/supports a version attribute, as it was unused for many years.
• Command output that has been truncated will now be written to the archive the same way truncated files are. They will be saved under sos_strings/ with a '.tailed' extension, and then symlinked to the relevant location.
• A new priority parameter has been added to add_cmd_output and add_journal that allows plugins to influence the order of command execution, regardless of when they are defined in the plugin. Lower values will run before higher values.
• Many collections now have added tags that will appear in sos_reports/manifest.json and can be used by third party utilities for better inspection of archive contents.
• By default, any services defined in a Plugin's services tuple will now automatically trigger a journal collection for those services.
• Plugins may now be enabled based upon the presence of specific running containers via the new containers tuple. This works the same as packages, services, etc...
• Wrappers for common os methods dealing with filesystem path inspection have been added to Plugin to automatically account for changes --sysroots.
  • These methods follow their os counterparts, for example os.listdir() and os.path.exists() are now exposed through Plugin.listdir() and Plugin.path_exists() respectively.
• The sssd plugin will now individually capture memory cache files, rather than globbing them together which may have hit collection size limits.
• mysql plugin collection paths have been updated for Ubuntu and Debian distributions.
• The kata_containers plugin has been updated for use with kata version 2.
• Improved the collection of all formats of sar data files for the sar plugin.
• The openstack_instack plugin will now collect config file overrides.
• Overhauled the openshift plugin to significantly expand and improve collections of OCP4 systems.

Clean|Mask Changes

• Fixed an issue where blank lines could be stripped from files that have other obfuscations occuring in them. This should help preserve the readability of obfuscated config files.
• By default, sos clean will now remove binary files in an archive, as binary files cannot be obfuscated and may contain identifying information.
  • To disable this behavior, use the new --keep-binary-files option.
• Improved the handling of symlinks that contain sensitive substrings in their paths.
• Improved the obfuscation of directory names and paths that contain sensitive substrings.
• sos clean will now use a single-process ProcessPool during the extraction process of an existing archive (e.g. sos clean /path/to/archive or sos collect --clean runs) in order to avoid potentially significant performance issues when extracting multiple large archives simultaneously.
• The username parser will now obfuscate LDAP users reported via last output.
  • Note that this means the LDAP user must have logged in to the system for their username to be obfuscated.

Collect Changes

• Added previously missing component options to the man sos-collect manpage.
• Added options to provide registry authentication credentials when pulling a container image for remote containerized hosts in a sos collect run.
• The --master option has been deprecated. Use the new --primary option or its aliases --controller, or --manager, as the deprecated form will be removed entirely in the next release.
• Fixed a bug where sos options would be clobbered across multiple nodes in the same run.
• Fixed a typo in the --plugin-option passthru option. The previous --plugin-options (with an 's') was incorrect.
• Corrected the per-node parsing of available plugins on newer remote sos installations versus older local installations.
• sos collect will no longer abort if the target for enumerating systems within a cluster does not itself has sos installed.
• Added a new cluster profile for OCP4 environments.

For full information on the changes contained in this release, please refer to the Git commit logs. Further release information and tarballs are available at:

https://github.com/sosreport/sos/releases/tag/4.2

Please report any problems to the sos-devel mailing list, or the GitHub issue tracker:

https://github.com/sosreport/sos/issues/

The team would like to thank everyone who contributed fixes, new features, testing, and feedback for this release.

sos-4.1

The sos team is pleased to announce the release of sos-4.1. This is the first minor version update to the 4.x line which introduced a new design and project direction. Included in this release are numerous bug fixes and feature enhancements focused on increasing the stability and reliability of the previous major version update.

Global Changes

• Fixed an incompatiblity with python-3.8 that prevented execution on systems using that version

• Fixed using a relative path for --tmp-dir would result in failing to build the final archive

• Fixed several issues dealing with RPM building via distutils

• Fixed a mismatch between option long form names with their internal destination names, which would prevent copy-pasting commands from sos logs

• Fixed a bug where --log-size=0 would be ignored

• SoS testing is now handled via CirrusCI on GCE, replacing the previous usage of TravisCI

• Note to contributors: as of this release, all regex substitutions being done by plugins during collection will now require an example in comments

Policy changes

• Fixed SuSE policy enablement, sos-4.x should now run properly on the latest SuSE releases
• Upload usernames and passwords may now be set via environment variables
• The docker container runtime abstraction for policies now supports docker when installed as a snap
• Red Hat systems will now automatically failback to using FTP if uploads to the Red Hat Customer Portal failed

Report

The sos report command has received a number of improvements in this minor release

• New plugins: openstack_barbican, tigervnc, bcache
• SoS will now generate sha256 checksums instead of md5
• Several plugins have been updated to enhance collection on PowerPC systems
• The snappy plugin has been renamed to snap
• The azure plugin will now capture AEM data
• Fixed an issue where the networking plugin incorrectly handled network namespaces
• The salt, keepalived, and numerous openstack_* plugins have been updated to function for Mirantis Cloud Platform
• The number of ps collections done by the process plugin has been reduced and consolidated
• Fixed enablement and collection of the kubernetes plugin for CDK environments
• Fixed an issue for SCLPlugin enablement on systems that have SCL enabled, but not for the package(s) for the SCLPlugins
• report will now collect broken symlinks properly, to show support representatives that a broken symlink exists on the system

Plugin API

• A new get_network_namespaces() method is available to assist with iterating over available network namespaces
• IndependentPlugin has been updated and revived to be used for plugins that are not-distribution specific.
  • Thus, plugins that use IndependentPlugin will now be supported on all sos-supported distributions even if they aren't specifically tagged

Clean|Mask

The sos clean or sos mask command has had significant attention paid to it to enhance the obfuscation provided for sosreports

• Added a username parser to obfuscate usernames reported by lastlog
• Improved the efficiency of the sanitize_item() entrypoint
• Fixed an issue where MAC addresses ending in punctuation weren't obfuscated
• Fixed an issue where hostnames were not being obfuscated if the host plugin wasn't run but an obfuscation mapping was available
• Fixed several reliability related issues for consistency in obfuscation
• Uppercase domains (e.g. kerberos realms) are now properly obfuscated
• cleaner is now a valid alias for sos clean

Collector

• collect will now use the sos-4.0 sos binary instead of the legacy sosreport binary if it is available
• Updated collect to use addition options added in sos-4.0
• collect may now upload in the same way sos report can
• Fixed an issue where a user-forced cluster-type would not use that cluster's node enumeration method

For full information on the changes contained in this release, please refer to the Git commit logs. Further release information and tarballs are available at:

https://github.com/sosreport/sos/releases/tag/4.1

Please report any problems to the sos-devel mailing list, or the GitHub issue tracker:

https://github.com/sosreport/sos/issues/

The team would like to thank everyone who contributed fixes, new features, testing, and feedback for this release.

sos-4.0

The sos team is pleased to announce the relase of sos-4.0. This is a major version release that represents a significant change to the sos project, new features, and bug fixes.

Major changes

Sos has been redesigned to provide functionality beyond the well known report data collection usage. It has been updated to provide
more functionality via sub-commands.

• A new sos binary has replaced the former sosreport binary as the main entry point for the utility.

  • sos report is now used to generate sosreport tarballs. A sosreport binary is maintained as a redirection point and will now invoke sos report.
  • sos collect formally brings sos-collector into the main sos project, and is used to collect sosreports from multiple nodes simultaneously. A sos-collector binary is maintained as a redirection point and will invoke sos collect.
    • This means the standalone sos-collector utility will no longer be independently developed.
  • sos clean formally brings soscleaner-like functionality into the main sos project. This sub command will perform further data obfuscation on reports, such as scrubbing IP addresses, domain names, and user-provided keywords. See below for more information.

• /etc/sos.conf has been moved to /etc/sos/sos.conf, and the layout of the config file has changed:

  • The general section has been renamed to global, and may be used to specify options that are available to all sos commands and sub-commands.
  • Each sub-command will have its own section, e.g. sos report will load options from global and from report.

• Sos is now a Python3-only utility. Python2 is no longer supported in any capacity.

• Dropped use of make, building/installing sos from source should now exclusively be done via setuptools

Report

• sos will now generate metadata and save it in sos_reports/manifest.json.

• 7 new plugins: nvmetcli, drbd, openstack_designate, pmem, containers_common, hyperv, freeipmi

• The nfsserver plugin has been merged into the nfs plugin

• sos may now be used to collect data from within a container, rather than aborting if that container was not configured to allow sos to collect information from the host

• Added support for Container-Optimized OS (COS)

• Dropped Mac OSX support

• Dropped bzip2 compression support

• Users may now use the --clean or --mask option to process a report-being-generated through sos clean at runtime.

• Size limits will now apply to add_copy_spec() calls that target a directory

• The openshift plugin has been re-written to be used for Openshift Container Platform 4

• Significantly expanded the amount of API resources the openstack_octavia plugin will collect

• The networking plugin will no longer execute ethtool -e against NICs using the bnx2x driver

• The logs plugin will now capture journal information correctly when logs are stored in-memory only

• The systemd plugin will no longer collect systemd-resolve if the service is not running

• The openvswitch plugin has been significantly updated to pull more meaningful data, and now supports OpenFlow 1.4 and 1.5

Plugin API changes

• The command execution/collection methods have been overhauled:

  • add_cmd_output() should continue to be used to specify commands that should be executed during the collection phase
  • exec_cmd() should now be used to execute commands and retrieve output during setup(), but will not save that output to the archive
  • collect_cmd_output() should be used to execute commands, retrieve output during setup() and will save that output to the archive
  • get_command_output() has been removed

• A new add_device_cmd() method is available to facilitate easier iteration of commands over a set of devices

  • add_blockdev_cmd() has been added to facilitate iteration of commands of storage devices

• A container runtime abstraction has been added that aims to standardize the discovery of a container runtime in use (e.g. docker, podman) and the retrieval of data from the runtime across plugins

Collect

sos collect is a new sub command in this release, and is an integration of the standalone sos-collector project, with the aim being to collect sosreports from multiple systems simultaneously. Note that this sub-command requires python3-pexpect to be available. If the module is not available, sos collect will abort with an appropriate error message

Compared to the standalone project, enhancements include:

• collect is now supported on all distributions that sos report supports (i.e. any distribution with a Policy defined)
• The --insecure-sudo option has been renamed to --nopasswd-sudo
• --threads in the context of the number of nodes to simultaneously connect to has been renamed to jobs
• Fixed a bug where a local node would be displayed for collection even when --no-local was used

Cleaner

sos clean, also available as sos mask, is a newly added sub-command in this release and is an implementation of the standalone soscleaner project. Its aim is to scrub potentially sensitive information from sosreports in a consistent manner, beyond the obfuscation done by plugins already.

• Support for ipv4 address/network obfuscation. Note that this will attempt to preserve topological relationships between discovered addresses
• Support for hostname, and domain name obfuscation.
• Support for user-provided keyword obfuscations
• Users may either use the --clean or --mask flag to sos report to obfuscate a report being generated, or may use sos (clean|mask) $archive to obfuscate an already existing report.
  • Using the former will result in a single obfuscated report archive, while the latter approach will result in two; an obfuscated archive and the un-obfuscated original.

For full information on the changes contained in this release, please refer to the Git commit logs. Further release information and tarballs are available at:

https://github.com/sosreport/sos/releases/tag/4.0

Please report any problems to the sos-devel mailing list, or the GitHub issue tracker:

https://github.com/sosreport/sos/issues/

The team would like to thank everyone who contributed fixes, new features, testing, and feedback for this release.