KGGLDM : Knowledge Graph Guided Diffusion Models for advanced learning

Diffusion Models (DMs) have achieved state-of-the-art performance in generative tasks across various domains. However, existing DMs lack precise control and struggle with conditional generation and task-specific adaptation. This thesis proposes a novel approach to address these limitations by incorporating Knowledge Graphs (KGs) guidance into Latent Diffusion Model (LDM). KGs provide a rich structured representation of conceptual and relational knowledge within domains. By leveraging Knowledge Graph Embedding (KGE) during the generative process, the proposed model can grasp context, relationships, and attributes encoded in the vector space. This allows for enhanced semantic control, ensuring the generated output exhibits high quality while adhering to the specified conceptual constraints from the KGs. The integration of KGs with DMs represents an unexplored direction with the potential to significantly improve the precision, controllability, and domain specificity of generative models.

This thesis explores the mathematical foundations and empirical evaluations of this innovative knowledge-guided diffusion modelling framework, demonstrating its effectiveness in enhancing the quality, diversity, and semantic consistency of generated images, particularly in capturing complex attribute combinations and relationships. Our proposed approach, Knowledge Graph Guided Latent Diffusion Model (KGGLDM) achieves superior performance with the lowest Fréchet Inception Distance (FID) score of 16.5 and high attribute consistency (55.1% accuracy across 40 features, 99.9% for prominent single attributes). Human evaluations confirms these results, with KGGLDM receiving top scores for visual quality (4.5/5) and semantic consistency (4.4/5).