Memoize _log.prefix calls

[NickLaMuro]

In situations where an EMS has a large number of hosts, the tree that gets built is quite large, and so the number of times that _log.prefix is fired is very high. This updates each one of those calls to memoize them to a instance variable, since the resulting prefix will not change.

Benchmarks

The benchmark script used here basically runs the following:

ManageIQ::Providers::Vmware::InfraManager::ProvisionWorkflow.new.init_from_dialog

And is targeting a fairly large EMS, with about 600 hosts.

	ms	queries	rows
before	15023	1961	48395
after	10056	1961	48395

Before                                                     After
------                                                     -----

Total allocated: 2069091751 bytes (23226536 objects)   |   Total allocated: 1034363078 bytes (19721183 objects)
                                                       |   
allocated objects by gem                               |   allocated objects by gem
-----------------------------------                    |   -----------------------------------
   9665227  pending                                    |      9665227  pending
   5561668  manageiq/app                               |      5561668  manageiq/app
   3513258  manageiq/lib  <<<<<<<<<<                   |      2512007  activerecord-5.0.7
   2512007  activerecord-5.0.7                         |       861272  activesupport-5.0.7
    861270  activesupport-5.0.7                        |       418737  ancestry-2.2.2
    418737  ancestry-2.2.2                             |       278793  activemodel-5.0.7
    278793  activemodel-5.0.7                          |       178419  ruby-2.3.3/lib
    178419  ruby-2.3.3/lib                             |       165577  arel-7.1.4
    165577  arel-7.1.4                                 |        52875  manageiq-providers-vmware-0be2f13a0dc9
     52875  manageiq-providers-vmware-0be2f13a0dc9     |        14424  fast_gettext-1.2.0
     14424  fast_gettext-1.2.0                         |         7903  manageiq/lib  <<<<<<<<<<
       ...                                             |          ...

Note: The benchmarks for this change do NOT include the changes from other PRs (#17354 for example). Benchmarks of all changes can be found here.

Links

• Partially addresses https://bugzilla.redhat.com/show_bug.cgi?id=1569626
• Full collection of proposed changes: git compare
• Document keeping track of overall improvements for this effort: gist
• Related PRs:
  • Prevents N+1 SQL queries in miq_request_workflow.rb #17354
    • Extracted PRs: Add MiqPreloader.polymorphic_preload_for_child_classes #17457, Use MiqPreloader.polymorphic_preload_for_child_classes in RelationshipMixin#fulltree_arranged #17460, Use MiqPreloader/Relationship improvements in MiqRequestWorkflow #17461
  • Use nested hashes instead of string keys #17357
  • Refactor get_ems_folders to create less strings #17358
  • Use OpenStruct over MiqHashStruct #17359
  • Ancestry Patches #17360
  • Load Lans effeciently in MiqProvisionVirtWorkflow #17381
  • Avoid duplicate host load in allowed_hosts_obj #17402
  • [WIP] Lighter unique Host#domain lookup for in MiqProvisionVirtWorkflow #17403

[miq-bot]

Checked commit NickLaMuro@c982f6c with ruby 2.3.3, rubocop 0.52.1, haml-lint 0.20.0, and yamllint 1.10.0
1 file checked, 0 offenses detected
Everything looks fine. 🍪

[NickLaMuro]

@miq-bot assign @gmcculloug
@miq-bot add_label performance

[simaishi]

Fine backport details:

$ git log -1
commit ef6b3f9c75781c49d77b4d71b36d57c6ed8a8d6a
Author: Brandon Dunne <[email protected]>
Date:   Fri Apr 27 18:28:23 2018 -0400

    Merge pull request #17355 from NickLaMuro/memoize_log_prefix_calls_in_miq_request_workflow
    
    Memoize _log.prefix calls
    (cherry picked from commit c2e01c9bf8426239d1f691d93fa21ef78997304f)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1593798

[simaishi]

Gaprindashvili backport details:

$ git log -1
commit a07dd77014653443a748df1045d7030279b9ba4d
Author: Brandon Dunne <[email protected]>
Date:   Fri Apr 27 18:28:23 2018 -0400

    Merge pull request #17355 from NickLaMuro/memoize_log_prefix_calls_in_miq_request_workflow
    
    Memoize _log.prefix calls
    (cherry picked from commit c2e01c9bf8426239d1f691d93fa21ef78997304f)
    
    https://bugzilla.redhat.com/show_bug.cgi?id=1593797