Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added PAX archive format support to minicoredumper to overcome core size limit of 8GB. #10

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

prakhar-pandey-nutanix
Copy link

PAX archive format is used to fix the below:

* Known problems:

    - If tar is active, core files larger than 8GB will be
      truncated. If it is known that the core files will be >8GB and
      the full core file is needed, tar must be disabled.

Summary

  • minicoredumper-2.0.6 lacks support for archiving > 8GB dumps.
  • patch adds support for larger cores using POSIX archive format.
  • includes sparse file support with format '1.0'.

This change contain

  • modified etc/generic.recept.json to include "use_posix_for_tar" key. (Below changes in minicoredumper-2.0.6/src)
  • prog_config.c & prog_config.h : added recept json option "use_posix_for_tar" under the "compression" entry with default value "true".
  • tar.h : contains the structs for gnu and posix tar archives.
  • added some comments in corestripper.c
  • major functions added to corestripper.c are
    • dump_compressed_posix_tar
    • write_pax_extended_entry
    • fill_name_in_pax_header
    • build_sparse_map_linked_list
  • modified the below functions in corestripper.c for PAX format support
    • check_core_size
    • dump_compressed_core
    • dump_compressed_tar

References

…rsion 2.0.6

Summary
- minicoredumper-2.0.6 lacks support for archiving > 8GB dumps.
- patch adds support for larger cores using POSIX archive format.
- includes sparse file support with format '1.0'.

This change contain
- modified etc/generic.recept.json to include "use_posix_for_tar" key.
(Below changes in minicoredumper-2.0.6/src)
- prog_config.c & prog_config.h : added recept json option
  "use_posix_for_tar" under the "compression" entry with default value
  "true".
- tar.h : contains the structs for gnu and posix tar archives.
- added some comments in corestripper.c
- major functions added to corestripper.c are
  - dump_compressed_posix_tar
  - write_pax_extended_entry
  - fill_name_in_pax_header
  - build_sparse_map_linked_list
- modified the below functions in corestripper.c for PAX format support
  - check_core_size
  - dump_compressed_core
  - dump_compressed_tar

References
- https://www.ibm.com/docs/en/zos/2.3.0?topic=ff-pax-format-pax-archives-special-header-summary-files
- https://pubs.opengroup.org/onlinepubs/9699919799/utilities/pax.html
- https://www.gnu.org/software/tar/manual/html_node/PAX-1.html#PAX-1

Signed-off-by: Prakhar Pandey <[email protected]>
Copy link

@ShaharHD ShaharHD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got an error while using the patch when building with Yocto


/* Fill in checksum and sparse map block (block 4) size in tar block 2. */
snprintf(sparse_file_hdr.numbytes, sizeof(sparse_file_hdr.numbytes),
"%011" PRIo64, strlen(sparse_map_block));
Copy link

@ShaharHD ShaharHD Jul 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got an error

'%011lo' directive output may be truncated writing between 11 and 21 bytes into a region of size 12 [-Werror=format-truncation=]

Should we just ignore the warning? or is it a possible bug? my work around:

#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wformat-truncation"
	snprintf(sparse_file_hdr.numbytes, sizeof(sparse_file_hdr.numbytes),
	         "%011" PRIo64, strlen(sparse_map_block));
#pragma GCC diagnostic pop

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants