Deep Learning based fusion of SAR and optical data to synthesize cloud-free and enhanced quality imagery

M-GEO
M-SE
ACQUAL
Additional Remarks

Required skills:
1) Programming skills in python.
2) Willingness to learn deep learning frameworks (TensorFlow, PyTorch)
3) Willingness to learn the manipulation of geographical data in Python (GDAL, geopandas, shapely, rasterio, etc)

Topic description

The presence of noises, clouds and another atmospheric phenomena represent an obstacle for optical remote sensing applications. A commonly used strategy is the composition of images acquired in several different dates into a single cloud-free imagery. This approach requires, in most cases, generates several unexpected artifacts and also requires a good and accurate cloud mask which, however, is not always available. Active sensors, due to the larger wavelengths used, are able to overcome the clouds and produce images at any atmospheric conditions. Despite that potential, those images are usually more complex and harder to be interpreted. Several researches have demonstrated the potential of Deep Learning methods, especially generative networks, to fuse SAR and optical imagery, synthesizing cloud free and enhanced quality images. In this sense, this MSc topic focus on designing and implementing efficient Deep Learning methods to perform this task.

Topic objectives and methodology

The student will initially revise the literature on deep generative networks, on image synthesis, and on deep learning based image fusion. The main focus will be on methods that are able to compose an efficient end-to-end trained system. The likely starting point may be the Generative Adversarial Networks (GANs), which have demonstrated substantial efficiency on image synthesis.

References for further reading

1] Bermudez, J. D., Happ, P. N., Oliveira, D. A. B., & Feitosa, R. Q. (2018). SAR TO OPTICAL IMAGE SYNTHESIS FOR CLOUD REMOVAL WITH GENERATIVE ADVERSARIAL NETWORKS. ISPRS Annals of Photogrammetry, Remote Sensing & Spatial Information Sciences, 4(1). <https://www.isprs-ann-photogramm-remote-sens-spatial-inf-sci.net/IV-1/5/2018/> [2] Meraner, A., Ebel, P., Zhu, X. X., & Schmitt, M. (2020). Cloud removal in Sentinel-2 imagery using a deep residual neural network and SAR-optical data fusion. ISPRS Journal of Photogrammetry and Remote Sensing, 166, 333-346. <https://www.sciencedirect.com/science/article/pii/S0924271620301398>