LD-MAN: Layout-Driven Multimodal Attention Network for Online News Sentiment Recognition

You are here

Top Reasons to Join SPS Today!

1. IEEE Signal Processing Magazine
2. Signal Processing Digital Library*
3. Inside Signal Processing Newsletter
4. SPS Resource Center
5. Career advancement & recognition
6. Discounts on conferences and publications
7. Professional networking
8. Communities for students, young professionals, and women
9. Volunteer opportunities
10. Coming soon! PDH/CEU credits
Click here to learn more.

LD-MAN: Layout-Driven Multimodal Attention Network for Online News Sentiment Recognition

Wenya Guo; Ying Zhang; Xiangrui Cai; Lei Meng; Jufeng Yang; Xiaojie Yuan

The prevailing use of both images and text to express opinions on the web leads to the need for multimodal sentiment recognition. Some commonly used social media data containing short text and few images, such as tweets and product reviews, have been well studied. However, it is still challenging to predict the readers’ sentiment after reading online news articles, since news articles often have more complicated structures, e.g., longer text and more images. To address this problem, we propose a layout-driven multimodal attention network (LD-MAN) to recognize news sentiment in an end-to-end manner. Rather than modeling text and images individually, LD-MAN uses the layout of online news to align images with the corresponding text. Specifically, it exploits a set of distance-based coefficients to model the image locations and measure the contextual relationship between images and text. LD-MAN then learns the affective representations of the articles from the aligned text and images using a multimodal attention mechanism. Considering the lack of relevant datasets in this field, we collect two multimodal online news datasets, containing a total of 14,566 articles with 56,260 images and 251,202 words. Experimental results demonstrate that the proposed method performs favorably compared with state-of-the-art approaches. We will release all the codes, models and datasets to the community.

SPS on Twitter

  • SPS is proud to participate in IEEE's new Multiple Society Discount Program! Join two or more participating societi… https://t.co/BnwcM7O7iu
  • IEEE Day is October 4th. Celebrate IEEE Day by attending a local event. Visit the IEEE Day site for a complete list… https://t.co/mESJHTn7ek
  • The Biomedical Imaging and Signal Processing Webinar Series continues on Tuesday, 4 October when Selin Aviyente pre… https://t.co/Gl4bHlWbqh
  • On Wednesday, 26 October, join Dr. DeLiang Wang for a new SPS webinar, "Neural Spectrospatial Filter" - register no… https://t.co/vUkiWC4Am8
  • Join Dr. Peilan Wang and Dr Jun Fang for "Channel State Information Acquisition for Intelligent Reflecting Surface-… https://t.co/jOhyA10xuG

SPS Videos

Signal Processing in Home Assistants


Multimedia Forensics

Careers in Signal Processing             


Under the Radar