Skip to main content

Stanford CS448B 16 VisNLP

· 2 min read

TLDR

This article contains my notes from Stanford's CS448B (Data Visualization) course, specifically focusing on the sixteenth lecture about visualization and natural language processing. I'll discuss the importance of linking text with visualization, understanding how readers integrate charts and captions, visual question and answering, and natural language interfaces.

Original​

Download PDF.

Notes​

GVC trade grew rapidly in the 1990s but stagnated after the 2008 global financial crisis

Question: How much did the GVC share rise between 1990 and 2008?

“Words and pictures belong together.”

--Tufte, 1983

Linking text with visualization​

Facilitating Document Reading by Linking Text and Tables

--Kim et al. 2018

Stage 1: Table Structure Extraction

Stage 2: Match Sentence Text to Table Cells

Stage 3: Rule-based Refinement of Matches

Pipeline Evaluation​

Corpus

  • Pew Research Reports
  • ACL and CVPR papers
  • Kong et. al (2014)

User Study​

Hypothesis:Our interface facilitates reading documents with tables

Within-subject study
14 adult volunteers, all fluent in English
Task: Annotate references with/without our interface

Understanding how readers integrate charts and captions​

Do readers rely more on the chart or captions for their takeaways?

When text and visualization emphasis mismatch, readers rely more on the chart and can miss information in the caption

Visual question and answering​

Formative Study​

Explanations describe procedure for computing answer
Half of the explanations referred to visual features of chart

Chart QA Pipeline and Generating Explanations​

Example

User Study​

  • 16 participants
  • 20 Chart-Question-Answer-Explanation tuples (5 per explanation type)

Natural language interfaces​

Download PDF

.