VISLIX: An XAI Framework for Validating Vision Models with Slice Discovery and Analysis

Simple item page
dc.contributor.authorYan, Xinyuanen_US
dc.contributor.authorXuan, Xiweien_US
dc.contributor.authorOno, Jorge Piazentinen_US
dc.contributor.authorGuo, Jiajingen_US
dc.contributor.authorMohanty, Vikramen_US
dc.contributor.authorKumar, Shekar Arvinden_US
dc.contributor.authorGou, Liangen_US
dc.contributor.authorWang, Beien_US
dc.contributor.authorRen, Liuen_US
dc.contributor.editorAigner, Wolfgangen_US
dc.contributor.editorAndrienko, Nataliaen_US
dc.contributor.editorWang, Beien_US
dc.date.accessioned2025-05-26T06:37:57Z
dc.date.available2025-05-26T06:37:57Z
dc.date.issued2025
dc.description.abstractReal-world machine learning models require rigorous evaluation before deployment, especially in safety-critical domains like autonomous driving and surveillance. The evaluation of machine learning models often focuses on data slices, which are subsets of the data that share a set of characteristics. Data slice finding automatically identifies conditions or data subgroups where models underperform, aiding developers in mitigating performance issues. Despite its popularity and effectiveness, data slicing for vision model validation faces several challenges. First, data slicing often needs additional image metadata or visual concepts, and falls short in certain computer vision tasks, such as object detection. Second, understanding data slices is a labor-intensive and mentally demanding process that heavily relies on the expert's domain knowledge. Third, data slicing lacks a human-in-the-loop solution that allows experts to form hypothesis and test them interactively. To overcome these limitations and better support the machine learning operations lifecycle, we introduce VISLIX, a novel visual analytics framework that employs state-of-the-art foundation models to help domain experts analyze slices in computer vision models. Our approach does not require image metadata or visual concepts, automatically generates natural language insights, and allows users to test data slice hypothesis interactively. We evaluate VISLIX with an expert study and three use cases, that demonstrate the effectiveness of our tool in providing comprehensive insights for validating object detection models.en_US
dc.description.sectionheadersExplainable and Generative AI
dc.description.seriesinformationComputer Graphics Forum
dc.identifier.doi10.1111/cgf.70125
dc.identifier.issn1467-8659
dc.identifier.pages12 pages
dc.identifier.urihttps://doi.org/10.1111/cgf.70125
dc.identifier.urihttps://diglib.eg.org/handle/10.1111/cgf70125
dc.publisherThe Eurographics Association and John Wiley & Sons Ltd.en_US
dc.rightsAttribution 4.0 International License
dc.rights.urihttps://creativecommons.org/licenses/by/4.0/
dc.subjectCCS Concepts: Computing methodologies → Model verification and validation; Computer vision tasks; Human-centered computing → Visual analytics; Interactive systems and tools
dc.subjectComputing methodologies → Model verification and validation
dc.subjectComputer vision tasks
dc.subjectHuman centered computing → Visual analytics
dc.subjectInteractive systems and tools
dc.titleVISLIX: An XAI Framework for Validating Vision Models with Slice Discovery and Analysisen_US
Files
Original bundle
Now showing 1 - 3 of 3
Thumbnail Image
Name:
cgf70125.pdf
Size:
12.5 MB
Format:
Adobe Portable Document Format
Download
Thumbnail Image
Name:
1168-file-i8.pdf
Size:
16.85 MB
Format:
Adobe Portable Document Format
Download
No Thumbnail Available
Name:
1168-file-i9.mp4
Size:
33.45 MB
Format:
Video MP4
Download
Collections
44-Issue 3