Integrating SSL into SegmentationTask

[Dodekaphonia]

Hello!

I'm currently working on a model for coastline change detection along the Antarctic coastline using Sentinel-1 images.

I followed @calebrob6's tutorial on LEVIR+ and tested this approach on my relatively small dataset of labeled changes, utilizing the ResNet50_Weights.SENTINEL1_ALL_MOCO weights. The results appeared promising.

Considering further improvements, I considered the idea of pre-training a self-supervised learning (SSL) model and then using it as a backbone for the SegmentationTask. To stay within one framework, I attempted to utilize the BYOLTask, MoCoTask, and SimCLRTask. However, I encountered difficulties in finding a straightforward approach to achieve pass the '.ckpt' file into a Segmentation

So, my question is whether it is possible to pre-train one of the SSL tasks and then integrate it into the Segmentation process. Or should I explore alternative approaches?

[adamjstewart]

I feel like @isaaccorley would know a good way to do this.

[isaaccorley]

I'm working on a solution for this. Stay tuned. We have some checkpoint loading utils but I haven't used them to load a BYOL checkpoint in a few months.