A Retrieval-Augmented LLM Pipeline for Generating Cognitive Conceptualization Knowledge Graphs

Cognitive Conceptualization Diagrams (CCDs) are commonly used in Cognitive Behavioral Therapy (CBT) for representing the relationships among a patient’s personal history, beliefs, thoughts, emotions, and behaviors. Currently, CCDs are manually authored by experts, which limits the availability of structured resources for research and training. In this paper, we introduce an LLM-based framework for (i) automatically generating inferred CCDs from brief patient descriptions and psychotherapy transcripts, and (ii) representing them as Knowledge Graph (KG) that can be inspected and queried. Our method adopts a Retrieval-Augmented Generation (RAG) module, which selects semantically related transcripts from a real-world corpus to produce a CBT-oriented psychological report. We use prompt-guided LLM extraction and hierarchical machine learning classification of core beliefs to produce a detailed CCD from the psychological report. We further formalize CCDs as an RDF-based semantic model enabling structured semantic representation. Using expert-created CCDs as gold references, we generated a dataset of 99 CCDs for different patient typologies, and evaluated their quality considering semantic similarity, contradiction analysis, and agreement on major and fine-grained core-belief labels. We also performed a qualitative assessment using an LLM-as-a-judge procedure grounded in established CBT formulation criteria. Results show that our method produces CCDs that are semantically and structurally consistent with expert formulations, improves fine-grained core belief classification over different baselines, and achieves the strongest qualitative ratings among compared pipelines. These findings suggest that our system can support scalable and clinically meaningful structuring of psychotherapy case formulations. The cognitive conceptualization KG is publicly available online.