β—‘CogChat

Knowledge Graph-Augmented Conversational AI for Cognitive Grounding in Design

Anonymous Author(s)
Under review
Paper soon arXiv soon Code

TL;DR

We ground conversational AI in how a designer organizes knowledge — not just what they said most recently.

LLM chat systems keep context through recency: the last few turns in a sliding window. In design conversation this breaks down — relational meaning decays between turns (“warm” was tactile grain, not color), identical words mean different things to different designers, and the conversation loops instead of deepening. CogChat builds a personal heterogeneous knowledge graph from each designer’s input and uses a Heterogeneous Graph Transformer (HGT) to select the structurally relevant nodes for every response — and to ask probing questions that surface what the designer hasn’t yet articulated.

CogChat in action: chatting with in-context knowledge lookup, probing questions refining across turns, and the cognitive knowledge graph growing.
CogChat in action — in-context knowledge lookup, probing questions that refine across turns, and the cognitive knowledge graph growing from the designer’s input.

The problem

Recency-based context loses the relations that design reasoning runs on.

Even with the full history in the context window, the system has memory without structure. Three failures compound.

1

Information loss

Relational context decays between turns. The system forgets that “warm” referred to tactile grain, not color temperature — the situated meaning central to design.

2

Personalization failure

Identical words carry different meanings across designers — “heavy” as visual density vs. information overload — but the system processes only generic semantics.

3

Inefficiency

Without retained context or resolved meaning, designers re-explain established ground. The conversation loops and restarts rather than deepening.

These aren’t independent bugs — they’re symptoms of one absence: no persistent, structured representation of the designer’s cognitive space. Design cognition is heterogeneous: perceptual, semantic, and functional elements coexist in a single judgment. Flattening that into a token sequence is what fails.

The interface

One workspace: conversation, knowledge inspection, and probing.

Every message builds on the accumulated relational context. Hover an underlined entity below to inspect its place in the knowledge graph.

Hover an entity
I want to design an eco-friendly camping gear brand.
β—‘
I’ll update your logo by softening the recycling icon, keeping the sun motif, adding a woven fabric texture, and using layered mountains with a warm taupe tone — aiming for a modern, earthy look.

Hover a dashed entity → its typed connections appear at right.

Recycling Icon

4 connections

In-context entity inspection

    The CogChat interface, annotated: knowledge graph overview, in-context entity inspection, chat input with image generation, and probing questions.
    Figure 2. (a) Knowledge graph overview · (b) In-context entity inspection · (c) Chat input with image generation · (d) Intentional & exploratory probing questions with suggested answers.

    The pipeline

    Five stages, re-run at every turn.

    The graph and the HGT retrain incrementally each turn, so the embedding space tracks the designer’s evolving conceptual structure rather than a frozen snapshot.

    stage 01

    KG Construction

    Extract typed entities & directed relations from each utterance + response into a design-oriented ontology (concept, material, property, design_element, action).

    stage 02

    Link Set Creation

    Build positive links (co-occurring, positively-rated pairs) and hard + random negatives, then sample compact 2-hop subgraphs.

    stage 03

    HGT Embedding

    A Heterogeneous Graph Transformer with type-specific projections (3 layers × 8 heads, 512→256-d) retrains each turn on a BCE + ranking loss.

    stage 04

    Grounded Response

    Rank nodes by embedding similarity to the current message; inject only the top relevant subgraph into the LLM, tracking a topic-continuity score.

    stage 05

    Probing Questions

    Partition nodes into intentional (relevant but underspecified) and exploratory (connected but undiscussed) to generate questions. Answers feed back into extraction.

    The CogChat technical pipeline: knowledge graph construction, link-set creation and subgraph sampling, HGT embedding, context-grounded response generation, and probing question generation.
    Figure 3. The technical pipeline. (a) Entity/relation extraction · (b) Positive/negative link sets + subgraph sampling · (c) HGT embedding with type-specific projections · (d) Embedding-based ranking + context injection · (e) Intentional/exploratory probing, feeding back into extraction.

    Probing questions

    Two kinds of question, derived from the graph’s structure.

    A node’s normalized relevance score ŝ(v) decides which kind it produces. Pick an answer below — it would feed back into the next turn’s extraction.

    Pick an answer
    🎯

    Intentional

    ŝ(v) β‰₯ ΞΌ + Οƒ

    Highly relevant but underspecified concepts — the system pins down what the designer already cares about.

    What specific earthy tone should be the primary color in the logo to emphasize its eco-friendly nature?

    πŸš€

    Exploratory

    ΞΌ βˆ’ Οƒ ≀ ŝ(v) < ΞΌ + Οƒ

    Connected but undiscussed concepts — the graph topology surfaces adjacent territory the designer hasn’t mentioned yet.

    What specific eco-friendly materials would you like to incorporate into the logo symbol?

    A CogChat conversation excerpt showing probing questions across turns and the cognitive knowledge graph visualization growing from turn 3 to turn 5.
    Figure 7. A real session (P4): probing questions accumulate context across turns, and the cognitive knowledge graph visibly densifies from turn 3 (b1) to turn 5 (b2).

    Technical evaluation

    Selective HGT grounding beats both no grounding and naΓ―ve KG injection.

    All conditions share the same base LLM. KG+HGT filters to structurally relevant context; KG-only dumps the whole graph and introduces noise that compounds errors on ambiguous queries.

    Metric Baseline KG-only KG+HGT
    ASQA · QA-Hit73.381.783.3
    RewardBench · Chat72.581.291.3
    RewardBench · Chat Hard58.365.782.4
    RewardBench · Safety68.279.893.6
    RewardBench · Reasoning52.871.387.9
    Average0.6300.7450.831

    The largest gains appear where relational structure matters most: Chat Hard (+24.1 over Baseline) and Reasoning (+35.1). Gains scale with task ambiguity.

    User study

    Nine professional designers, three conditions, within-subjects.

    Designers from graphic, interior/furniture, spatial, exhibition/branding, product, and architecture practice (M = 4.78 yrs experience) compared Baseline, KG-only, and CogChat across three research questions.

    RQ1 · Context retention

    CogChat reached a satisfactory outcome in 5.1 turns vs. 9.4 for Baseline (−45.9%), in a single session instead of 2.1. Higher per-turn BERTScore and nearly double the token-level IoU show HGT reuses the designer’s own vocabulary.

    Negative feedback: 0.22 vs. 3.67 (Baseline)
    RQ2 · Personalized interpretation

    Chatbot Usability (CUQ) differed significantly across all three: 77.8 (CogChat) > 64.9 (KG-only) > 46.7 (Baseline). Probing success rate reached 70% vs. 17% for Baseline.

    Retaining entities isn’t enough — structuring their relations is what helps.
    RQ3 · Cognitive effort & depth

    Lower NASA-TLX mental demand, effort, and frustration under KG-grounded conditions. 8 of 9 participants identified CogChat as the deepest conversation, while feeling they retained directional initiative.

    Amplifies the designer’s reasoning rather than replacing it.
    Per-item survey results across NASA-TLX, intent expression, satisfaction, output quality, and agency/ownership scales for Baseline, KG-only, and CogChat.
    Figure 4. Per-item survey responses across NASA-TLX, intent expression, satisfaction, output quality, and agency & ownership. CogChat (purple) leads consistently, with the sharpest gaps on intent expression and outcome quality.
    Eight bar charts comparing Baseline, KG-only, and CogChat on CUQ, BERTScore, Token IoU, probing success rate, task completion time, turn count, and feedback counts.
    Figure 5. Per-metric study outcomes across the three conditions. (* p<.05, ** p<.01, *** p<.001)

    Reading the eight panels

    CogChat (left bar) leads on every quality metric while costing the designer less.

    a–d
    Higher is better. CUQ usability, per-turn semantic similarity (BERTScore), token-level IoU, and probing success rate — all peak for CogChat.
    e–f
    Lower is better. Task completion time and turn count drop sharply — 5.1 turns vs. 9.4 for Baseline.
    g
    Negative feedback. 0.22 dislikes/session vs. 3.67 for Baseline — context loss, not quality, drove dissatisfaction.
    h
    Positive feedback. Highest for CogChat, confirming the gains land with designers.
    Timeline of all three conditions showing user messages, system responses, image generations, probing questions, and answered/presented probing success rates per turn.
    Figure 6. Full session timelines per condition. CogChat reaches a finished design in fewer turns with a high answered/presented probing ratio, while Baseline loops and restarts.

    What the designers said

    Semi-structured interviews surfaced why the numbers moved — the relief of not re-explaining, and the surprise of being understood on their own terms.

    “The intent was preserved throughout, and subsequent requests were reflected accurately — responses became more precise as the conversation went on.”

    P2 & P5 · on context retention

    It read “water properties” not as transparency but as gentle wave texture — “actually better. I received design advice in reverse, through exploratory questions.”

    P8 · on personalized interpretation

    “I could see my own logic laid out.” Double-clicking a word to inspect its connections made the system’s understanding tangible.

    P5 · on entity inspection

    The takeaway isn’t that CogChat remembers more — Baseline already had the full history in its context window. It’s that CogChat organizes what it remembers. The bottleneck in design conversation is not memory capacity but context curation: deciding which relations to foreground at each turn. Eight of nine designers called CogChat the deepest conversation — while still feeling the design was theirs.

    BibTeX

    @inproceedings{cogchat2026,
  title     = {CogChat: Knowledge Graph-Augmented Conversational AI with Heterogeneous Graph Transformer for Cognitive Grounding in Design Generation},
  author    = {Anonymous Author(s)},
  booktitle = {},
  year      = {2026},
  publisher = {},
}