Enhancing the Interactivity of Dataframe Queries by Leveraging Think Time

Understand

We propose opportunistic evaluation, a framework for accelerating interactions with dataframes.

• Interactive latency is critical for iterative, human-in-the-loop dataframe workloads for supporting exploratory data analysis.
• Opportunistic evaluation significantly reduces interactive latency by 1) prioritizing computation directly relevant to the interactions and 2) leveraging think time for asynchronous background computation for non-critical operators that might be relevant to future interactions.
• We show, through empirical analysis, that current user behavior presents ample opportunities for optimization, and the solutions we propose effectively harness such opportunities.