Remove the FP32 Wrapper when evaluating (#10617) #4298

Job	Run time
Flax TPU Tests	3m 1s
ONNX CUDA Tests	4m 20s
Examples PyTorch CUDA tests on Ubuntu	32m 21s
PyTorch Compile CUDA tests	2m 50s
PyTorch xformers CUDA tests	5m 9s
Setup Torch Pipelines CUDA Slow Tests Matrix	1m 41s
Torch CUDA Tests (models)	6m 34s
Torch CUDA Tests (schedulers)	4m 58s
Torch CUDA Tests (lora)	7m 56s
Torch CUDA Tests (others)	41s
Torch CUDA Tests (single_file)	3m 47s
Torch Pipelines CUDA Tests (cogvideo)	3m 51s
Torch Pipelines CUDA Tests (controlnet)	10m 56s
Torch Pipelines CUDA Tests (ddpm)	40s
Torch Pipelines CUDA Tests (flux)	3m 39s
Torch Pipelines CUDA Tests (ip_adapters)	2m 16s
Torch Pipelines CUDA Tests (kandinsky2_2)	5m 3s
Torch Pipelines CUDA Tests (kandinsky3)	3m 35s
Torch Pipelines CUDA Tests (latent_consistency_models)	2m 52s
Torch Pipelines CUDA Tests (ltx)	2m 50s
Torch Pipelines CUDA Tests (pixart_alpha)	2m 31s
Torch Pipelines CUDA Tests (stable_diffusion)	5m 52s
Torch Pipelines CUDA Tests (stable_diffusion_2)	6m 30s
Torch Pipelines CUDA Tests (stable_diffusion_3)	1m 26s
Torch Pipelines CUDA Tests (stable_diffusion_adapter)	1m 52s
Torch Pipelines CUDA Tests (stable_diffusion_xl)	4m 32s
Torch Pipelines CUDA Tests (stable_video_diffusion)	50s
	2h 12m 33s

Provide feedback