Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add more columns to arcstat #16509

Merged
merged 1 commit into from
Sep 18, 2024
Merged

add more columns to arcstat #16509

merged 1 commit into from
Sep 18, 2024

Conversation

tkittich
Copy link
Contributor

@tkittich tkittich commented Sep 5, 2024

Motivation and Context

add more columns to arcstat. Everything from ARC structural breakdown, ARC types breakdown, ARC states breakdown in arc_summary should be in here. Also, I've modified to display decimal places if there's enough space. This will be helpful when interval is large. The values when divided by large interval would become very small. The old behavior would round down small numbers to 0.

Description

   l2wbytes : Bytes written per second to the L2ARC
      cmpsz : Compressed size
     cmpsz% : Compressed size percentage
      ovhsz : Overhead size
     ovhsz% : Overhead size percentage
      bonsz : Bonus size
     bonsz% : Bonus size percentage
      dnosz : Dnode size
     dnosz% : Dnode size percentage
      dbusz : Dbuf size
     dbusz% : Dbuf size percentage
      hdrsz : Header size
     hdrsz% : Header size percentage
      l2hsz : L2 header size
     l2hsz% : L2 header size percentage
      abdsz : ABD chunk waste size
     abdsz% : ABD chunk waste size percentage
     datatg : ARC data target
    datatg% : ARC data target percentage
     datasz : ARC data size
    datasz% : ARC data size percentage
     metatg : ARC metadata target
    metatg% : ARC metadata target percentage
     metasz : ARC metadata size
    metasz% : ARC metadata size percentage
      anosz : Anonymous size
     anosz% : Anonymous size percentage
      anoda : Anonymous data size
     anoda% : Anonymous data size percentage
      anome : Anonymous metadata size
     anome% : Anonymous metadata size percentage
      anoed : Anonymous evictable data size
     anoed% : Anonymous evictable data size percentage
      anoem : Anonymous evictable metadata size
     anoem% : Anonymous evictable metadata size percentage
      mfutg : MFU target
     mfutg% : MFU target percentage
      mfudt : MFU data target
     mfudt% : MFU data target percentage
      mfumt : MFU metadata target
     mfumt% : MFU metadata target percentage
      mfusz : MFU size
     mfusz% : MFU size percentage
      mfuda : MFU data size
     mfuda% : MFU data size percentage
      mfume : MFU metadata size
     mfume% : MFU metadata size percentage
      mfued : MFU evictable data size
     mfued% : MFU evictable data size percentage
      mfuem : MFU evictable metadata size
     mfuem% : MFU evictable metadata size percentage
     mfugsz : MFU ghost size
      mfugd : MFU ghost data size
      mfugm : MFU ghost metadata size
      mrutg : MRU target
     mrutg% : MRU target percentage
      mrudt : MRU data target
     mrudt% : MRU data target percentage
      mrumt : MRU metadata target
     mrumt% : MRU metadata target percentage
      mrusz : MRU size
     mrusz% : MRU size percentage
      mruda : MRU data size
     mruda% : MRU data size percentage
      mrume : MRU metadata size
     mrume% : MRU metadata size percentage
      mrued : MRU evictable data size
     mrued% : MRU evictable data size percentage
      mruem : MRU evictable metadata size
     mruem% : MRU evictable metadata size percentage
     mrugsz : MRU ghost size
      mrugd : MRU ghost data size
      mrugm : MRU ghost metadata size
      uncsz : Uncached size
     uncsz% : Uncached size percentage
      uncda : Uncached data size
     uncda% : Uncached data size percentage
      uncme : Uncached metadata size
     uncme% : Uncached metadata size percentage
      unced : Uncached evictable data size
     unced% : Uncached evictable data size percentage
      uncem : Uncached evictable metadata size
     uncem% : Uncached evictable metadata size percentage

How Has This Been Tested?

Tested locally.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Performance enhancement (non-breaking change which improves efficiency)
  • Code cleanup (non-breaking change which makes code smaller or more readable)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Library ABI change (libzfs, libzfs_core, libnvpair, libuutil and libzfsbootenv)
  • Documentation (a change to man pages or other documentation)

Checklist:

Copy link
Member

@amotin amotin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have no objections for this, but I was thinking that we should add much more of the ARC parameters there. At very least for completeness there should be also numbers for anon and uncached states. And then it would be good to see data/metadata distribution, which ARC collects for each state. And then may be evictable memory stats, etc.

Please see the current arc_summary output as an example of what can be reported. I think arcstat should be able to report all the same numbers to allow their analysis in dynamics.

@tkittich
Copy link
Contributor Author

tkittich commented Sep 6, 2024

I've added more columns as suggested. Let me know if anything is missing. ^^

   l2wbytes : Bytes written per second to the L2ARC
      anosz : Anonymous size
      mfusz : MFU size
      mrusz : MRU size
      uncsz : Uncached size
    anodata : Anonymous data size
    mfudata : MFU data size
    mrudata : MRU data size
    uncdata : Uncached data size
    anometa : Anonymous metadata size
    mfumeta : MFU metadata size
    mrumeta : MRU metadata size
    uncmeta : Uncached metadata size
   anoedata : Anonymous evictable data size
   mfuedata : MFU evictable data size
   mruedata : MRU evictable data size
   uncedata : Uncached evictable data size
   anoemeta : Anonymous evictable metadata size
   mfuemeta : MFU evictable metadata size
   mruemeta : MRU evictable metadata size
   uncemeta : Uncached evictable metadata size

@tkittich tkittich changed the title Update arcstat: add mfusz, mrusz, l2wbytes add more columns to arcstat Sep 6, 2024
Copy link
Member

@amotin amotin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. I can think of more counters, like for ghost states (even though they are quite different and may be confusing) or total data/metadata size, structural breakdown section of arc_summary (dnodes, etc), but this is definitely a good addition.

@tkittich
Copy link
Contributor Author

tkittich commented Sep 7, 2024

I've add more columns. Everything from ARC structural breakdown, ARC types breakdown, ARC states breakdown in arc_summary should be in here.

      cmpsz : Compressed size
     cmpsz% : Compressed size percentage
      ovhsz : Overhead size
     ovhsz% : Overhead size percentage
      bonsz : Bonus size
     bonsz% : Bonus size percentage
      dnosz : Dnode size
     dnosz% : Dnode size percentage
      dbusz : Dbuf size
     dbusz% : Dbuf size percentage
      hdrsz : Header size
     hdrsz% : Header size percentage
      l2hsz : L2 header size
     l2hsz% : L2 header size percentage
      abdsz : ABD chunk waste size
     abdsz% : ABD chunk waste size percentage
     datasz : ARC data size
    datasz% : ARC data size percentage
     metasz : ARC metadata size
    metasz% : ARC metadata size percentage
      anosz : Anonymous size
     anosz% : Anonymous size percentage
    anodata : Anonymous data size
   anodata% : Anonymous data size percentage
    anometa : Anonymous metadata size
   anometa% : Anonymous metadata size percentage
     anoeda : Anonymous evictable data size
    anoeda% : Anonymous evictable data size percentage
     anoeme : Anonymous evictable metadata size
    anoeme% : Anonymous evictable metadata size percentage
     mfudat : MFU data target
    mfudat% : MFU data target percentage
     mfumet : MFU metadata target
    mfumet% : MFU metadata target percentage
      mfusz : MFU size
     mfusz% : MFU size percentage
    mfudata : MFU data size
   mfudata% : MFU data size percentage
    mfumeta : MFU metadata size
   mfumeta% : MFU metadata size percentage
     mfueda : MFU evictable data size
    mfueda% : MFU evictable data size percentage
     mfueme : MFU evictable metadata size
    mfueme% : MFU evictable metadata size percentage
     mfugsz : MFU ghost size
     mfugda : MFU ghost data size
     mfugme : MFU ghost metadata size
     mrudat : MRU data target
    mrudat% : MRU data target percentage
     mrumet : MRU metadata target
    mrumet% : MRU metadata target percentage
      mrusz : MRU size
     mrusz% : MRU size percentage
    mrudata : MRU data size
   mrudata% : MRU data size percentage
    mrumeta : MRU metadata size
   mrumeta% : MRU metadata size percentage
     mrueda : MRU evictable data size
    mrueda% : MRU evictable data size percentage
     mrueme : MRU evictable metadata size
    mrueme% : MRU evictable metadata size percentage
     mrugsz : MRU ghost size
     mrugda : MRU ghost data size
     mrugme : MRU ghost metadata size
      uncsz : Uncached size
     uncsz% : Uncached size percentage
    uncdata : Uncached data size
   uncdata% : Uncached data size percentage
    uncmeta : Uncached metadata size
   uncmeta% : Uncached metadata size percentage
     unceda : Uncached evictable data size
    unceda% : Uncached evictable data size percentage
     unceme : Uncached evictable metadata size
    unceme% : Uncached evictable metadata size percentage

Copy link
Member

@amotin amotin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is quite a list and quite a logic. ;) Just one cosmetic comment:

cmd/arcstat.in Show resolved Hide resolved
@amotin
Copy link
Member

amotin commented Sep 9, 2024

And please squash the commits.

@tkittich
Copy link
Contributor Author

tkittich commented Sep 9, 2024

And please squash the commits.

Thank you for the review. I've added target sizes, changed column names to fit 5 chars, and squash the commits. ^^

Copy link
Member

@amotin amotin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks.

cmd/arcstat.in Outdated Show resolved Hide resolved
@tkittich tkittich force-pushed the master branch 2 times, most recently from 6e8c31c to bed3022 Compare September 10, 2024 15:35
@tkittich
Copy link
Contributor Author

I've added target size of ARC data, ARC metadata, MFU, MRU. ^^

Copy link
Member

@amotin amotin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I am slightly worrying that those grouped numbers could be confusing, showing that target is not reached, while one of their components actually is, but more is not less.

@allanjude
Copy link
Contributor

Anon_size is definitely the one I've wanted many times

@tkittich
Copy link
Contributor Author

tkittich commented Sep 12, 2024

I've shorten new column names. Also, I've modified to display decimal places when number is < 10. This will be helpful when interval is large. The values when divided by large interval would become very small. The old behavior would round down small numbers to 0.

   l2wbytes : Bytes written per second to the L2ARC
      cmpsz : Compressed size
     cmpsz% : Compressed size percentage
      ovhsz : Overhead size
     ovhsz% : Overhead size percentage
      bonsz : Bonus size
     bonsz% : Bonus size percentage
      dnosz : Dnode size
     dnosz% : Dnode size percentage
      dbusz : Dbuf size
     dbusz% : Dbuf size percentage
      hdrsz : Header size
     hdrsz% : Header size percentage
      l2hsz : L2 header size
     l2hsz% : L2 header size percentage
      abdsz : ABD chunk waste size
     abdsz% : ABD chunk waste size percentage
     datatg : ARC data target
    datatg% : ARC data target percentage
     datasz : ARC data size
    datasz% : ARC data size percentage
     metatg : ARC metadata target
    metatg% : ARC metadata target percentage
     metasz : ARC metadata size
    metasz% : ARC metadata size percentage
      anosz : Anonymous size
     anosz% : Anonymous size percentage
      anoda : Anonymous data size
     anoda% : Anonymous data size percentage
      anome : Anonymous metadata size
     anome% : Anonymous metadata size percentage
      anoed : Anonymous evictable data size
     anoed% : Anonymous evictable data size percentage
      anoem : Anonymous evictable metadata size
     anoem% : Anonymous evictable metadata size percentage
      mfutg : MFU target
     mfutg% : MFU target percentage
      mfudt : MFU data target
     mfudt% : MFU data target percentage
      mfumt : MFU metadata target
     mfumt% : MFU metadata target percentage
      mfusz : MFU size
     mfusz% : MFU size percentage
      mfuda : MFU data size
     mfuda% : MFU data size percentage
      mfume : MFU metadata size
     mfume% : MFU metadata size percentage
      mfued : MFU evictable data size
     mfued% : MFU evictable data size percentage
      mfuem : MFU evictable metadata size
     mfuem% : MFU evictable metadata size percentage
     mfugsz : MFU ghost size
      mfugd : MFU ghost data size
      mfugm : MFU ghost metadata size
      mrutg : MRU target
     mrutg% : MRU target percentage
      mrudt : MRU data target
     mrudt% : MRU data target percentage
      mrumt : MRU metadata target
     mrumt% : MRU metadata target percentage
      mrusz : MRU size
     mrusz% : MRU size percentage
      mruda : MRU data size
     mruda% : MRU data size percentage
      mrume : MRU metadata size
     mrume% : MRU metadata size percentage
      mrued : MRU evictable data size
     mrued% : MRU evictable data size percentage
      mruem : MRU evictable metadata size
     mruem% : MRU evictable metadata size percentage
     mrugsz : MRU ghost size
      mrugd : MRU ghost data size
      mrugm : MRU ghost metadata size
      uncsz : Uncached size
     uncsz% : Uncached size percentage
      uncda : Uncached data size
     uncda% : Uncached data size percentage
      uncme : Uncached metadata size
     uncme% : Uncached metadata size percentage
      unced : Uncached evictable data size
     unced% : Uncached evictable data size percentage
      uncem : Uncached evictable metadata size
     uncem% : Uncached evictable metadata size percentage

@amotin
Copy link
Member

amotin commented Sep 12, 2024

Fractional bytes or operations may weird is they are not percents or per-second. ;)

@tkittich
Copy link
Contributor Author

tkittich commented Sep 13, 2024

Fractional bytes or operations may weird is they are not percents or per-second. ;)

The latest commit has a fix that will not show decimal places when interval is 1 and value is integer. It'll also show decimal digits when value > 10 if there's enough space.^^

Signed-off-by: Theera K. <[email protected]>

arcstat: add structural, types, states breakdown

add ARC structural breakdown, ARC types breakdown, ARC states breakdown similar to arc_summary

Signed-off-by: Theera K. <[email protected]>

rename column names to fit 5 chars

Signed-off-by: Theera K. <[email protected]>

rename column names to fit 5 chars

Signed-off-by: Theera K. <[email protected]>

add data target, metadata target

Signed-off-by: Theera K. <[email protected]>

add data target, metadata target

Signed-off-by: Theera K. <[email protected]>

fix abd typo

Signed-off-by: Theera K. <[email protected]>

fix abd typo

Signed-off-by: Theera K. <[email protected]>

make column name a bit shorter

Signed-off-by: Theera K. <[email protected]>

make column name a bit shorter

Signed-off-by: Theera K. <[email protected]>

fix structural typo

Signed-off-by: Theera K. <[email protected]>

Delete cmd/arc_summary.py

incorrect file extension

Signed-off-by: Theera K. <[email protected]>

fix structural typo

Signed-off-by: Theera K. <[email protected]>

arcstat: add structural, types, states breakdown

add ARC structural breakdown, ARC types breakdown, ARC states breakdown similar to arc_summary

Signed-off-by: Theera K. <[email protected]>

Update arcstat: add Anonymous, MFU, MRU, Uncached

Signed-off-by: Theera K. <[email protected]>

Update arcstat: add Anonymous, MFU, MRU, Uncached

Signed-off-by: Theera K. <[email protected]>

Update arcstat: add Anonymous, MFU, MRU, Uncached

Signed-off-by: Theera K. <[email protected]>

Update arcstat.1 : add mfusz, mrusz, l2wbytes

Signed-off-by: Theera K. <[email protected]>

Update arcstat: add mfusz, mrusz, l2wbytes

mfusz: MFU size
mrusz: MRU size
l2wbytes: Bytes write per second to the L2ARC

Signed-off-by: Theera K. <[email protected]>

remove extra spaces

Signed-off-by: Theera K. <[email protected]>

arcstat: add target size of data, meta, MFU, MRU

arcstat: add target size of ARC data, ARC metadata, MFU, MRU

Signed-off-by: Theera K. <[email protected]>

arcstat: add target size of data, meta, MFU, MRU

arcstat: add target size of ARC data, ARC metadata, MFU, MRU

Signed-off-by: Theera K. <[email protected]>

shorten new column names; show decimal when < 10

Signed-off-by: Theera K. <[email protected]>

prettynum: add special case when interval is 1

when interval is 1 only show decimal when value is < 1

Signed-off-by: Theera K. <[email protected]>

fix scale == 100 & change prettynum()

Signed-off-by: Theera K. <[email protected]>

fix prettynum rounding

Signed-off-by: Theera K. <[email protected]>
@behlendorf behlendorf added the Status: Accepted Ready to integrate (reviewed, tested) label Sep 18, 2024
@behlendorf behlendorf merged commit 4d469ac into openzfs:master Sep 18, 2024
19 of 22 checks passed
ixhamza pushed a commit to truenas/zfs that referenced this pull request Nov 11, 2024
Add ARC structural breakdown, ARC types breakdown, ARC states
breakdown similar to arc_summary.  Additional cleanups included.

Reviewed-by: Alexander Motin <[email protected]>
Signed-off-by: Theera K. <[email protected]>
Closes openzfs#16509
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Accepted Ready to integrate (reviewed, tested)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants