Multimodal Fake News Detection Using Transformer Architectures and Large Language Models (LLMs)

Abstract

The proliferation of false information in digital media has evolved to include manipulated images, synthetic audio, and deepfake videos. Recent advances in generative AI have made robust multimodal detection systems an urgent necessity. This thesis proposes two novel multimodal architectures for fake news detection. The first architecture targets image–text content by extracting visual (SigLIP), textual (Longformer), frequency (DCT/FFT), and Large Language Model (LLM)-based reasoning features (Llama-3), which are fused via a Mixture-of-Experts (MoE) mechanism to produce both veracity predictions and natural language explanations. The second architecture analyzes short videos by extracting eleven feature sets across the visual, acoustic, and textual domains, utilizing cross-modal attention to combine keyframe (SigLIP), audio (Wav2Vec2), text (RoBERTa), and consistency (Phi-3 Mini) representations. Evaluated on the Fakeddit dataset, the image-text system achieves 85.03% accuracy, with visual features providing the most critical detection signal. The video system achieves 91.30% accuracy on the FakeTT dataset, where facial and temporal features prove essential. Ultimately, this work demonstrates that integrating multiple modalities and explicit LLM reasoning is essential for creating reliable, transparent, and interpretable defense mechanisms against modern digital deception