Handle when backend is also in compile_kwargs

[oraluben]

cc @tohtana

[tohtana]

Hi @oraluben, Can you elaborate your goal with this PR?

[oraluben]

Hi @oraluben, Can you elaborate your goal with this PR?

Sure! In the case where "backend" is also in compile_kwargs dict, the current version fails with an error that "backend" arg is provided twice. In this PR, I use the backend arg to override possible backend value in compile_kwargs.

An example is huggingface/accelerate#3069, I have to pop backend before pass compile_kwargs to engine.compile

[tohtana]

Hi @oraluben,
Thank you for the clarification. It makes sense. At this point, it's not very clear how we benefit from treating backend separately from other compiler options, but we want to keep the option for compatibility.

I agree with the fix, but the method for merging dictionaries is not correct, as the tests have failed. Could you address this?
It would also be good if we could show a warning when backend is specified in both ways and one of them is overwritten.

[oraluben]

Is there anything wrong with the CI? My PRs keep timeout in v100.

[loadams]

Is there anything wrong with the CI? My PRs keep timeout in v100.

Yes this looks to be an issue on our side, I will take care of this.