Very Low Bitrate Semantic Compression of Airplane Cockpit Screen Content

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

Very Low Bitrate Semantic Compression of Airplane Cockpit Screen Content

By: 
Iulia Mitrica; Eric Mercier; Christophe Ruellan; Attilio Fiandrotti; Marco Cagnazzo

This paper addresses the problem of encoding the video generated by the screen of an airplane cockpit. As other computer screens, cockpit screens consist of computer-generated graphics often atop a natural background. Existing screen content coding schemes fail notably in preserving the readability of textual information at the low bitrates required in avionic applications. We propose a screen coding scheme where textual information is encoded according to the relative semantics rather than in the pixel domain. The encoder localizes textual information, and the semantics of each character are extracted with a convolutional neural network and predictively encoded. Text is then removed via inpainting, and the residual background video is compressed with a standard codec and transmitted to the receiver together with the text semantics. At the decoder side, text is synthesized using the decoded semantics and superimposed over the decoded residual video recovering the original frame. Our proposed scheme offers two key advantages over a semantics-unaware scheme that encodes text in the pixel domain. First, the text readability at the decoder is not compromised by compression artifacts, whereas the relative bitrate is negligible. Second, removal of high-frequency transform coefficients associated with the inpainted text drastically reduces the bitrate of the residual video. Experiments with real cockpit video sequences show BD-rate gains up to 82% and 69% over a reference H.265/HEVC encoder and its screen content coding extension. Moreover, our scheme achieves quasi-errorless character recognition already at very low bitrates, whereas even HEVC-SCC needs at least three or four times more bitrate to achieve a comparable error rate.

SPS on Twitter

  • SPS is now accepting nominations for several positions, including three Members-at-Large and two Regional Directors… https://t.co/y3z8Sp9Zrt
  • Many of our events, including and , have been or are in the process of being transitioned to f… https://t.co/0HDofTO8Ul
  • DEADLINE EXTENDED: The 2020 IEEE Statistical Signal Processing Workshop has extended their paper submission deadlin… https://t.co/w8lsBJupn8
  • The IEEE Journal on Selected Topics in Signal Processing welcomes submissions for a Special Issue on Reconstruction… https://t.co/YsgqicXDeu
  • SPS is now accepting proposals for Winter 2020-2021 schools. Head to our website for more information: https://t.co/RkI0Ipkqi2

SPS Videos


Signal Processing in Home Assistants

 


Multimedia Forensics


Careers in Signal Processing             

 


Under the Radar