VP Genie
Usability Testing
Spring 2025
Overview
VP Genie is a comprehensive Gen AI-integrated solution under Interco.AI, designed to support both the creative and production aspects of filmmaking.
It targets filmmakers who have tight budgets to execute their projects, filling in the gaps in their teams and equipment and more.
We conducted this usability test on a beta version of the product.
VP Genie is a comprehensive Gen AI-integrated solution under Interco.AI, designed to support both the creative and production aspects of filmmaking.
It targets filmmakers who have tight budgets to execute their projects, filling in the gaps in their teams and equipment and more.
We conducted this usability test on a beta version of the product.
Research Background
Supervisor Kathryn Brookshier
Stakeholder Mary Rauzi, Bernard Mendez
Team Brendan Sallee, Monica Zhang, Yuan Fang, Pooja Narain
Duration 3 months
My Role UX Researcher, UX Designer
Contribution Proposal, Study Kit, Interview, Data Log, Analysis, Assess and Define, Final Presentation
How current VP Genie work?
Scope of Study
Script Creation – Generate detailed outlines, scripts, characters, locations, scenes, props, and gear needed for production.
Storyboarding – Visually map out scenes effectively.
Business Goals
🎯 Attract more users to try VP Genie’s paid subscription
🎯 Usability for All: Ensure VP Genie is effective for those still learning the craft of filmmaking.
What were our responsibilities?
🔍 Identify UX/Product Gaps in pre-production workflows
Map friction points in the current experience
Understand the pre-production workflows of user group
Understand pain points & missing features
Research Questions
Participants Criteria
We recruited 6 participants that met our requirements.
Research Methods
Standardized prompt given to participants to
generate a script
Pre-test interview questions
Task-based, moderated, within-subjects usability testing
Observed participants performing given tasks
Assessed task success/failure rate, task error (1-5 scale)
Post-test interview questions and System Usability Scale (SUS)
Task-based, moderated, within-subjects usability testing
Observed participants performing given tasks
Assessed task success/failure rate, task error (1-5 scale)
Post-test interview questions and System Usability Scale (SUS)
Study Overview
Scenario 1: Create a Script from a Prompt
Overview: The participant starts with an idea written as a prompt in a Word document and uses VP Genie to generate a script, scenes, characters, locations, and props.
![]()
Scenario 2: Edit or Regenerate Visual Elements
Overview: The participant has generated a script and now wants to edit specific characters and locations to align with their vision.
![]()
Scenario 3: Edit Storyboard
Overview: The participant refines a storyboard, making adjustments to scenes and shots.
![]()
Task Metrics
![]()
What Went Well?
![]()
Overview of Findings
![]()
Area of Improvement #1Unexpected AI Content
![]()
Area of Improvement #2 No error recovery or version control
![]()
Area of Improvement #3 Unintuitive content editing workflow
![]()
Area of Improvement #4 Unfamiliar Terminology
![]()
Area of Improvement #5 Difficulties finding essential CTAs
![]()
Area of Improvement #6 Incomplete and unstructured script export
![]()
UX Design
Due to the NDA, I cannot show more details on the website.
SUS(System Usability Scale)
Scenario 1: Create a Script from a Prompt
Overview: The participant starts with an idea written as a prompt in a Word document and uses VP Genie to generate a script, scenes, characters, locations, and props.
Scenario 2: Edit or Regenerate Visual ElementsOverview: The participant has generated a script and now wants to edit specific characters and locations to align with their vision.
Scenario 3: Edit Storyboard
Overview: The participant refines a storyboard, making adjustments to scenes and shots.
Task Metrics
What Went Well?
Overview of Findings
Area of Improvement #1Unexpected AI Content
Area of Improvement #2 No error recovery or version control
Area of Improvement #3 Unintuitive content editing workflow
Area of Improvement #4 Unfamiliar Terminology
Area of Improvement #5 Difficulties finding essential CTAs
Area of Improvement #6 Incomplete and unstructured script export
UX Design
Due to the NDA, I cannot show more details on the website.
SUS(System Usability Scale)
Our 6 participants reported they:
Did not see themselves using VP Genie often in its current state
Did not feel confident using VP Genie
Felt VP Genie was difficult to navigate
Yet did not think they would need a lot of help to learn to use VP Genie
Did not see themselves using VP Genie often in its current state
Did not feel confident using VP Genie
Felt VP Genie was difficult to navigate
Yet did not think they would need a lot of help to learn to use VP Genie
Higher values mean participants felt more positive about VP Genie in that aspect:
![]()
Takeaways
