Bug fix for the schedule after the update_optimizer

[oguzhanbsolak]

After the update_optimizer, we need to reflect the updated optimizer to the compression scheduler. Otherwise, the scheduler uses the old optimizer, and lr is not scheduled correctly. This PR fixes the issue by traversing each policy and replacing the old optimizer object with the new one.

[rotx-eva]

please remove and fix limits in .pylintrc and .github/linters/.python-lint

[oguzhanbsolak]

Realized in the latest commit.

[rotx-eva]

Please remove line 131 and fix limits in .pylintrc and .github/linters/.python-lint instead

[rotx-eva]

Please also change .pylintrc

[rotx-eva]

Shouldn't this be:

       if compression_scheduler:
            for ep, _ in enumerate(compression_scheduler.policies):
                for pol in compression_scheduler.policies[ep]:
                    for attr_key in dir(pol):
                        attr = getattr(pol, attr_key)
                        if hasattr(attr, 'optimizer'):
                            attr.optimizer = optimizer

[rotx-eva]

Has this been tested with anything other than lr_scheduler?

[oguzhanbsolak]

Hi Robert,

Has this been tested with anything other than lr_scheduler?

I tested this change with lr_schedulers and RelationBasedKDPolicy policies. In the current train.py structure, we are only using KnowledgeDistillationPolicy, RelationBasedKDPolicy, and lr_scheduler. From them, only lr_scheduler has the "optimizer" attribute. Therefore, this change shouldn't affect other policies.

Shouldn't this be:

       if compression_scheduler:
            for ep, _ in enumerate(compression_scheduler.policies):
                for pol in compression_scheduler.policies[ep]:
                    for attr_key in dir(pol):
                        attr = getattr(pol, attr_key)
                        if hasattr(attr, 'optimizer'):
                            attr.optimizer = optimizer

When we don't define a scheduler policy, the train.py creates a default compression_scheduler. Therefore, there will be always a compression_scheduler at the state that this snippet will be run, and I believe we don't need to check if there is a compression_scheduler.

For readability your suggested version seems much better, so I am planning to push a new commit as in below:

                for pol in compression_scheduler.policies[ep]:
                    for attr_key in dir(pol):
                        attr = getattr(pol, attr_key)
                        if hasattr(attr, 'optimizer'):
                            attr.optimizer = optimizer

[ermanok]

Looks good.