Noise on my semantic segmentation

[aleguillou1]

Hello,

I've tried to make a segmentation using the same method as the book "Deep Learning for Remote sensing images with open source software". I tried a first time with the example in the book and it worked well. So I tried using my own data to perform a segmentation of buildings and roads (in a single class) but this time on a Pleiades image. I managed to train the model (using the same one as in the book: create_model4.py as I recall).

After successfully training the model on 64x64 patches, I was able to carry out the inference in the same way as in the book, but the result was rough. I'd already had this result with multi-class segmentation but I thought it was me who'd done something wrong. But as it still happens, even with binary segmentation, I'm thinking there's a problem, maybe it's the Pleiades sensor, or my training data? Or The result is normal for pléiades and maybe i need to try a better U-net Architecture ?

Here's the rough result:

My Pleiades image:

My labels :

And my point for Patch_Exctraction :

For information, I've extracted around 6000 training patches to the scale of the image and a further 3000 for validation in 64x64, the image is a 3-channel image.

Thank you in advance and have a nice day,
Adrien

[remicres]

Hello @aleguillou1 ,

The problem may come from:

• the data: did you have use 2 sources when extracting patches? a common mistake is to extract the patches for each source instead of doing that in one pass. The risk when its done in multiple passes, is that a different number of patches is extracted for each sources (e.g. different number of labels and images patches). Check that you have the same number of patches, and that they are paired correctly.
• the network: have you used normalized images? convergence can be bad when the input values are not in the [0,1] range
• the training setup: not enough epochs, wrong learning rate, etc.

Also, I would like to point you toward a more modern way to train your models. The book is a bit outdated now and TensorFlow / Keras have considerably evolved. There is also pyotb to the rescue to play with OTB apps.
Here is a lightweight tutorial showing how to do the semantic segmention like in the book, but with today's librairies versions. Also, you can take a look in the otbtf doc hosted on readthedocs that explains a lot of thing.

[aleguillou1]

Hello Rémi,

Thank you for your prompt reply.
To answer point by point:

• Yes, the images have been patched with 2 sources, so I'm integrating it into a python code to avoid errors, which gives this:

import otbApplication
app = otbApplication.Registry.CreateApplication("PatchesExtraction")
app.SetParameterStringList("source1.il", ["/home/Thèse_SSD/Unet_RS/Image/Image.tif"])
app.SetParameterInt("source1.patchsizex", 64)
app.SetParameterInt("source1.patchsizey", 64)
app.SetParameterString("source1.out", "/home/Thèse_SSD/Unet_RS/OTBTF_Unet/Patch/Patch_Image_val.tif")
app.SetParameterString("vec", "/home/Thèse_SSD/Unet_RS/OTBTF_Unet/train/val.shp")
app.SetParameterString("field", "code")
app.SetParameterStringList("source2.il" , ["/home/Thèse_SSD/Unet_RS/Image/label.tif"])
app.SetParameterInt("source2.patchsizex", 64)
app.SetParameterInt("source2.patchsizey", 64)
app.SetParameterString("source2.out" , "/home/Thèse_SSD/Unet_RS/OTBTF_Unet/Patch/Patch_Label_val.tif" )
app.SetParameterString("outlabels", "/home/Thèse_SSD/Test_Unet/Test_2/Patch2/asuppr.tif")
app.ExecuteAndWriteOutput()

for example val.shp are the validation points.

• Yes, the images are normalised before processing using OTB's Rescale tool (perhaps this isn't the best solution?).
• I'm training the U-net with 100 epochs and I quickly reach a threshold in terms of training, i.e. the scores don't evolve very quickly. Perhaps I'm in a "hole" in terms of the loss function and I should change the learning rate.

Thanks for the Pyotb tip, I've never tried it, but I will in the future.

Best regards,
Adrien

[remicres]

Hello @aleguillou1 ,

Here is a full tutorial that covers the modern way to build/train/evaluate deep nets with keras/otbtf and apply them on images with otbtf: https://orfeo-toolbox.pages.mia.inra.fr/otbtf-keras-tutorial/

In case that can help (I really recommend you to switch to keras and the new otbtf python API)

[aleguillou1]

Hi Remi,

I'm sorry, I've never relaunched on this git, but I managed to launch a semantic segmentation using pyotb without any problem. It didn't allow me to understand the original problem on my method but at least with the pyotb method everything works fine and is even more practical , thank you very much !

Thank you for the link by the way !

Adrien