ET: Edge-Enhanced Transformer for Image Splicing Detection

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

ET: Edge-Enhanced Transformer for Image Splicing Detection

By: 
Yu Sun; Rongrong Ni; Yao Zhao

A key challenge of image splicing detection is how to localize integral tampered regions without false alarm. Although current forgery detection approaches have achieved promising performance, the integrality and false alarm are overlooked. In this paper, we argue that the insufficient use of splicing boundary is a main reason for poor accuracy. To tackle this problem, we propose an Edge-enhanced Transformer (ET) for tampered region localization. Specifically, to capture rich tampering traces, a two-branch edge-aware transformer is built to integrate the splicing edge clues into the forgery localization network, generating forgery features and edge features. Furthermore, we design a feature enhancement module to highlight the artifacts of the edge area in forgery features and assign weight values to the resulting tensor in spatial domain for vital signal strengthening and noise suppression. Extensive experimental results on CASIA v1.0, CASIA v2.0 and NC2016 demonstrate that the proposed method can accurately localize tampered regions in both pixel and edge levels and outperforms state-of-the-art methods.

In Recent years, there has been a lot of works on digital image forensics [1][7], among which image splicing detection is a hot spot. Image splicing is manipulated by copying a region from one image and pasting it into another image, and its purpose is to highlight or cover up some objects.

Many image splicing detection approaches [6][8][13] have been proposed, which can be divided into the traditional features based methods and the deep learning based methods. The traditional features based methods [6][8][14][16] depend on some specific statistical characteristics that can reveal the differences between the original region and the tampered region, such as Color Filter Array (CFA) artifacts [8], noise pattern [15] and JPEG related traces [16]. In fact, these hand-crafted methods are not robust to post-processings and difficult to generate discriminative features. With the excellent application of deep learning technology, recent methods rely on Convolutional Neural Networks (CNNs) to learn effective features. Early CNNs based detection methods learned the differences between image patches and considered abnormal patches as tampered ones. EXIF [17] employed the EXIF metadata of pairs of patches as a supervisory signal to train the model. Zhou et al. [18] made use of the triplet loss to ensure that patches from the same image share the similar features and patches from different images have big gaps in feature space. Forgery maps obtained by these patch based methods can’t localize tampered regions at pixel level. To solve this problem, some works treated spliced region localization as an image segmentation task. MFCN [19] made use of a fully convolutional network (FCN) to learn forgery features. RRU-Net [20] proposed a ringed residual network based on U-Net to detect image forgeries. More recently, manipulation detection methods, such as ManTra-Net [21], SPAN [22] and GSCNet [4], were employed to localize the manipulated regions.

SPS on Twitter

  • New SPS Webinar: On Wednesday, 8 February, join Dr. Roula Nassif for "Decentralized learning over multitask graphs"… https://t.co/GOgHb7vfAv
  • CALL FOR PAPERS: IEEE Signal Processing Magazine welcomes submissions for a Special Issue on Hypercomplex Signal an… https://t.co/UDvjUY2llT
  • New SPS Webinar: On 15 February, join Mr. Wei Liu, Dr. Li Chen and Dr. Wenyi Zhang presenting "Decentralized Federa… https://t.co/em0sQAK4V5
  • New SPS Webinar: On Monday, 13 February, join Dr. Joe (Zhou) Ren when he presents "Human Centric Visual Analysis -… https://t.co/Rc39HpkPKr
  • Help us illustrate the SPS story! In honor of our 75th anniversary, we need your support to capture the people, mem… https://t.co/MnYU9MzIok

SPS Videos


Signal Processing in Home Assistants

 


Multimedia Forensics


Careers in Signal Processing             

 


Under the Radar