Pseudo-panchromatic image guided transformer model for multispectral image demosaicing - Chen, Peng; Wang, Heng; Su, Xinyu; Yang, Jiangnan; Wei, Cong; Li, Shuangli (Oficyna Wydawnicza Politechniki Wrocławskiej, 2024)
The multispectral imaging system using the filter array can capture the multispectral information of the scene in one snapshot and reconstruct the complete multispectral image by demosaicing. However, the sparse sampling rate makes image captured by demosaicing a challenging problem. Although a lot of demosaicing algorithms have been developed, the existing well-performing methods have limitations in modeling non-local dependencies which lead to artifacts. To solve this problem, this paper proposes a transformer-based multispectral image demosaicing model to address the problem. The proposed model comprises a pseudo-panchromatic image generation network and a transformer-based multispectral image reconstruction network. Additionally, we designed a fusion module to combine the pseudo-panchromatic image with the raw mosaic image captured by the camera, leveraging the correlation between the band of multispectral images to improve the performance of the model. The experimental results show that the proposed method has the advantages of high reconstruction precision, strong anti-noise interference ability, and small calculation amount, which provides a better image reconstruction solution for constructing a high-quality multispectral imaging system applied to multiple scenes.
Optica Applicata, Vol. 54, 2024, nr 4, s. 551-565
Optica Applicata is an international journal, published in a non-periodical form in the years 1971-1973 and quarterly since 1973. From the beginning of the year 2008, Optica Applicata is an Open Access journal available online via the Internet, with free access to the full text of articles serving the best interests of the scientific community. The journal is abstracted and indexed in: Chemical Abstracts, Compendex, Current Contents, Inspec, Referativnyj Zhurnal, SCI Expanded, Scopus, Ulrich’s Periodicals Directory
http://opticaapplicata.pwr.edu.pl/
Keywords: optyka; transformer; deep learning; pseudo-panchromatic image; demosaicing; multispectral image