Xiaohong (Sharon) Gao

A super resolution (SR) image refers to the image with enhanced resolutions that depict increased clarity, sharpening and details (usually with four-fold (×4)) without compromising its original contents or characters. SR is employed to reveal an image in fine details. While SR image can be produced by the emerging optical devices, e.g. super-resolution microscopy (SRM) with the change of optical resolution from ~250nm to ~10nm, computational models appear to be more variable, in particular with the advances of current state of the art artificial intelligence (AI) techniques. This tutorial aims to address AI based techniques to generate SR images, with a focus on the family of generative adversarial deep learning networks (GANs). The architecture of texture-based vision transformer (TTSR) will be also discussed in conjunction with the application of detection of Human papillomavirus (HPV) from microscopic images. In computer vision field, there are broadly two ways to contend with the fundamental low-level single image super-resolution (SISR) problem. One is from theoretical point of view and another is based on subjects’ visual appearance evaluation. While SISR attempts to recover a high-resolution (HR) image from a single low resolution (LR) one, it appears that the application of deep learning neural networks can achieve state of the art results. One of these models is GAN, designating an approach to generative modelling using deep learning methods, such as convolutional neural networks (CNN). Subsequently, a number of network architectures have been proposed to improve the SR performance mainly at improving Peak Signal-to-Noise Ratio (PSNR) values. This, however, tends to be in disagreement with human observers’ evaluation as pointed out in the network of SRGAN, one of the seminal works on the improvement of visual quality of generated SR images. Towards this end, several perceptual-driven methods are advanced, including incorporation of perceptual loss to optimize SR models in a feature space instead of pixel space and segmentation of semantic images prior to recovering detailed textures. Significantly, the application of GAN in SRGAN improves the overall visual quality of reconstruction over the PSNR-oriented methods considerably by encouraging the network to advocate solutions that look more like natural images. In addition, an enhanced SRGAN (ESRGAN) further improves the visual quality by reducing generated accompanying artefacts. ESRGAN introduces relativistic GAN, in which Residual-in-Residual Dense Block (RRDB) without batch normalization is utilised as basic network building blocks whereas SRGAN is built with residual blocks, offering consistently better visual quality with more realistic and natural textures. Recently, vision transformers (ViT) are emerging and starting to show potentials by performing computer vision tasks, such as image recognition. Built upon self-attention architectures and being a leading model in natural language processing (NLP), ViT appears to demonstrate excellent performance when trained on sufficient data, outperforming comparable state-of-the-art CNNs with four times fewer computational resources. In this tutorial, the architecture of texture transformer (TTSR) as well as GAN-based networks are elaborated, in conjunction with detection of HPV like particles (HPVLPs), or HPV viral factories. The four state of the art GAN models are ESRGAN, CycleGAN, Pix2pix and Pix2pixHD.