MAMFND: Multimodal Attention Mechanism for Enhanced Fake News Detection on Social Media
Abstract
In response to the growing prevalence of multimodal false information on social media platforms, traditional single-modal models and basic feature concatenation approaches in multimodal models exhibit limitations in effectively detecting fake news. Therefore, this paper presents a multimodal approach for detecting fake news, integrating a multimodal attention mechanism known as MAMFND (Multimodal Attention Mechanism for Fake News Detection). Initially, we utilize pretrained BERT (Bidirectional Encoder Representations from Transformers) and Swin Transformer (Swin Transformer: Hierarchical Vision Transformer using Shifted Windows) models to extract features from text and images, respectively. Subsequently, we introduce a fusion strategy based on attention mechanisms to integrate textual and visual features. To better capture the intrinsic relationships between text and images, we also input the textual features into a BiLSTM (Bi-directional Long Short-Term Memory) model for temporal sequence modeling, followed by an additional attention-based fusion with visual features. Finally, we extract information from the two rounds of feature fusion and input it into a fake news detection model for classification. Experimental results demonstrate that, on the Weibo and CCF competition datasets, the MAMFND model achieved average accuracy improvements of approximately 9.4% and 5.6%, respectively, compared to baseline models.
Keywords
Download Options
Introduction
Social media serves as a vital daily information source for the public, thereby making fake news detection a crucial task for enhancing the credibility of disseminated information. The continuous proliferation of fake news on social media poses a significant societal concern, underscoring the pressing need for effective detection methods. The spread of unverified and deceptive information on social media, commonly referred to as online fake news, is characterized by falsehood, exaggeration, provocation, and malicious intent. This pervasive issue extends beyond the mere disruption of social order, also resulting in considerable harm to individuals, businesses, and governments.
The onset of the COVID-19 pandemic witnessed an alarming surge in the spread of fake news across social media platforms. Amid this crisis, individuals and entities spread unsubstantiated claims, such as the efficacy of specific drugs in curing COVID19. These claims prompted many to procure and use these drugs without scientific validation, sometimes leading to delays in seeking proper medical treatment. Moreover, false assertions circulated, suggesting that the novel coronavirus could be transmitted through the air or that consuming highly alcoholic disinfectants could prevent infection. These misleading narratives not only lacked a factual and scientific basis but also posed significant threats to public safety.
Conclusion
The widespread dissemination of fake news can have serious negative impacts on society. Compared to pure text-based fake news, multimodal fake news containing both text and images is more likely to have a greater impact, as people are more easily attracted by images and tend to overlook the fake news embedded within. This can lead to misunderstandings about certain events or issues among the public, potentially causing panic, chaos, and even violent conflicts. Therefore, it is necessary to detect fake news using multimodal technology. Within this paper, we introduce a multimodal feature attention mechanism fusion model for detecting fake news. The model employs a feature extractor based on the Transformer framework to extract image and text information. It captures the interaction between modalities through attention mechanisms and further mines the potential correlations between image and text features by modeling time series on text features to enhance the role of attention mechanisms. The experimental results on relevant datasets show that the text model outperforms the baseline model in all aspects. Relevant data indicate that fake news not only includes text and images but also user comments after browsing, which contain authenticity information about fake news. Therefore, in future work, we will combine text, images, and comments to verify the authenticity of fake news.