ET: Edge-Enhanced Transformer for Image Splicing Detection

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

ET: Edge-Enhanced Transformer for Image Splicing Detection

Yu Sun; Rongrong Ni; Yao Zhao

A key challenge of image splicing detection is how to localize integral tampered regions without false alarm. Although current forgery detection approaches have achieved promising performance, the integrality and false alarm are overlooked. In this paper, we argue that the insufficient use of splicing boundary is a main reason for poor accuracy. To tackle this problem, we propose an Edge-enhanced Transformer (ET) for tampered region localization. Specifically, to capture rich tampering traces, a two-branch edge-aware transformer is built to integrate the splicing edge clues into the forgery localization network, generating forgery features and edge features. Furthermore, we design a feature enhancement module to highlight the artifacts of the edge area in forgery features and assign weight values to the resulting tensor in spatial domain for vital signal strengthening and noise suppression. Extensive experimental results on CASIA v1.0, CASIA v2.0 and NC2016 demonstrate that the proposed method can accurately localize tampered regions in both pixel and edge levels and outperforms state-of-the-art methods.

In Recent years, there has been a lot of works on digital image forensics [1][7], among which image splicing detection is a hot spot. Image splicing is manipulated by copying a region from one image and pasting it into another image, and its purpose is to highlight or cover up some objects.

Many image splicing detection approaches [6][8][13] have been proposed, which can be divided into the traditional features based methods and the deep learning based methods. The traditional features based methods [6][8][14][16] depend on some specific statistical characteristics that can reveal the differences between the original region and the tampered region, such as Color Filter Array (CFA) artifacts [8], noise pattern [15] and JPEG related traces [16]. In fact, these hand-crafted methods are not robust to post-processings and difficult to generate discriminative features. With the excellent application of deep learning technology, recent methods rely on Convolutional Neural Networks (CNNs) to learn effective features. Early CNNs based detection methods learned the differences between image patches and considered abnormal patches as tampered ones. EXIF [17] employed the EXIF metadata of pairs of patches as a supervisory signal to train the model. Zhou et al. [18] made use of the triplet loss to ensure that patches from the same image share the similar features and patches from different images have big gaps in feature space. Forgery maps obtained by these patch based methods can’t localize tampered regions at pixel level. To solve this problem, some works treated spliced region localization as an image segmentation task. MFCN [19] made use of a fully convolutional network (FCN) to learn forgery features. RRU-Net [20] proposed a ringed residual network based on U-Net to detect image forgeries. More recently, manipulation detection methods, such as ManTra-Net [21], SPAN [22] and GSCNet [4], were employed to localize the manipulated regions.

SPS on Twitter

  • DEADLINE EXTENDED: The 2023 IEEE International Workshop on Machine Learning for Signal Processing is now accepting…
  • ONE MONTH OUT! We are celebrating the inaugural SPS Day on 2 June, honoring the date the Society was established in…
  • The new SPS Scholarship Program welcomes applications from students interested in pursuing signal processing educat…
  • CALL FOR PAPERS: The IEEE Journal of Selected Topics in Signal Processing is now seeking submissions for a Special…
  • Test your knowledge of signal processing history with our April trivia! Our 75th anniversary celebration continues:…

SPS Videos

Signal Processing in Home Assistants


Multimedia Forensics

Careers in Signal Processing             


Under the Radar