InterChat: Enhancing Generative Visual Analytics using Multimodal Interactions

Simple item page
dc.contributor.authorChen, Juntongen_US
dc.contributor.authorWu, Jiangen_US
dc.contributor.authorGuo, Jiajingen_US
dc.contributor.authorMohanty, Vikramen_US
dc.contributor.authorLi, Xuemingen_US
dc.contributor.authorOno, Jorge Piazentinen_US
dc.contributor.authorHe, Wenbinen_US
dc.contributor.authorRen, Liuen_US
dc.contributor.authorLiu, Dongyuen_US
dc.contributor.editorAigner, Wolfgangen_US
dc.contributor.editorAndrienko, Nataliaen_US
dc.contributor.editorWang, Beien_US
dc.date.accessioned2025-05-26T06:37:06Z
dc.date.available2025-05-26T06:37:06Z
dc.date.issued2025
dc.description.abstractThe rise of Large Language Models (LLMs) and generative visual analytics systems has transformed data-driven insights, yet significant challenges persist in accurately interpreting users analytical and interaction intents. While language inputs offer flexibility, they often lack precision, making the expression of complex intents inefficient, error-prone, and time-intensive. To address these limitations, we investigate the design space of multimodal interactions for generative visual analytics through a literature review and pilot brainstorming sessions. Building on these insights, we introduce a highly extensible workflow that integrates multiple LLM agents for intent inference and visualization generation.We develop InterChat, a generative visual analytics system that combines direct manipulation of visual elements with natural language inputs. This integration enables precise intent communication and supports progressive, visually driven exploratory data analyses. By employing effective prompt engineering, and contextual interaction linking, alongside intuitive visualization and interaction designs, InterChat bridges the gap between user interactions and LLM-driven visualizations, enhancing both interpretability and usability. Extensive evaluations, including two usage scenarios, a user study, and expert feedback, demonstrate the effectiveness of InterChat. Results show significant improvements in the accuracy and efficiency of handling complex visual analytics tasks, highlighting the potential of multimodal interactions to redefine user engagement and analytical depth in generative visual analytics.en_US
dc.description.sectionheadersExplainable and Generative AI
dc.description.seriesinformationComputer Graphics Forum
dc.identifier.doi10.1111/cgf.70112
dc.identifier.issn1467-8659
dc.identifier.pages12 pages
dc.identifier.urihttps://doi.org/10.1111/cgf.70112
dc.identifier.urihttps://diglib.eg.org/handle/10.1111/cgf70112
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectCCS Concepts: Human-centered computing → Interactive systems and tools; Visual analytics; Computing methodologies → Natural language processing
dc.subjectHuman centered computing → Interactive systems and tools
dc.subjectVisual analytics
dc.subjectComputing methodologies → Natural language processing
dc.titleInterChat: Enhancing Generative Visual Analytics using Multimodal Interactionsen_US
Files
Original bundle
Now showing 1 - 3 of 3
Thumbnail Image
Name:
cgf70112.pdf
Size:
4.23 MB
Format:
Adobe Portable Document Format
Download
Thumbnail Image
Name:
1138-file-i8.pdf
Size:
3.38 MB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
1138-file-i9.mp4
Size:
25.05 MB
Format:
Video MP4
Download
Collections
44-Issue 3