Performance degradation with HIP memory kinds enabled in OFI conduit

[elliottslaughter]

We've had some performance issues that were fixed by rolling back that change. On a HIP system running on a single GPU the performance of GPU kernels was about a factor of 30 slower when we used tracing in the code. Interestingly the code ran with its normal performance as soon as the Legion profiler was switched on. I wonder if it has any side effect that could cause this.

Originally posted by @tukss in #27 (comment)

[elliottslaughter]

@tukss I moved this to a new issue so that we can discuss here.

I believe that enabling Legion profiling with HIP will cause Legion to synchronize the event stream so that we can determine when the kernels have finished. @eddy16112 or @seemamirch might have more information.

Why that would then make performance recover, I cannot say.

@bonachea or @PHHargrove might have other ideas to try.