Live Story
From static to animated storytelling:
programmatically generative tool for explainer video
Microsoft 2023 August
Team: Yu Zhang, Jiaqi Guo, Ni Shentu, Yangyu Huang, Yu Liu, Yun Wang
What is Live Story:
Live Story is a programmatically generative tool that presents solution to the challenges faced by academic professionals in creating explanatory videos for scientific contents.
Features:
-
Programmatical workflow
-
Generative assistance
-
Asynchronous tasks on single page
PROBLEM CONTEXT
In academia, visualizing research findings (knowledge) is a crucial step in the publishment and dissemination process. And the explainer video has become a popular tool for interpreting clear and engaging explanations.
However, those who frequently utilize explainer videos to interpret knowledge (the scientific academic researcher) often lack expertise in animation and video production.
Define Users
The scientific academic researchers who have the demand of making explainer videos for their research visualization.
To better understand the users' pain point, let's review the classic workflow of creating an explainer video.
User Pain Points
For our users who do not have experience in visual production:
Struggle to craft video scripts that balance video pacing with knowledge explanation
Difficulty in creating Visual that corresponding to explanation scripts.
Unable to produce graphic that has high quality movement and rhythmanimation
Define Goal and Success
Project Goal
Develop an intuitive tool that empowers researchers, typically without experience in visual creation, to easily produce motion graphic explainer videos.
Define Success
EXPLORATIONS
Explored on User habits
Visual operation tools are not for our users since...
Existed paradigm:
Complicated structure in visual resource management
Require proficiency in visual tool using (long-time experience)
Detail management can be more complicated and subtle.
What might be good?
Programmatical based tools have the advantages of..
More precise in information arrangement and elements formatting
More align to most of the researchers work style: coding base
Lower using bar without proficiency requirement
Adobe Effect interface for animation video making can be complicated even for professional users.
We got some inspiration from Overleaf, an online LaTeX tool for paper writing that has over 15 million users from research intitutions.
Design decision 1
Explore on UX architecture
Multipages structure
Based on the expected user journey, we first designed a multipages ux structure that interpreted the step by step journey of creating a explainer video
Feedback
Loose information connection between page jumping
Uneasy to iterate on previous step and get real-time update on following steps.
“Are these ‘multipages’ flows really necessary?”
User Research
What are the tools they mostly use look like?
A single functional page will consolidate all asynchronous tasks and information of a project within one comprehensive interface.
Enables easy iteration with the capability to simultaneously modify various parts of the work.
Maintains a holistic view of the entire workflow at all times.
Minimizes the risk (or concern) of data loss when navigating between pages.
Design decision 2
Design Iterations
Single Page Interface
Based on our further research on potential user and a dive-in interview on their working habit, we designed a new version of interface that consolidates asynchronous tasks into a single working interface.
Low-Fidelity
.png)
1
2
3
4
To better guide our user at the initial stage, we have an interaction sequence based on the workflow.
Walk-through
Select/ Input topic
Generate/ edit script
Generate narration
Generate frames
.png)
Feedback
A single page enable the user to iterate on previous step easily.
More convenient without jumping between pages.
" It's good that you can always monitor the change during iterations. "
Design Outcome
Hi-fi Interfaces
.png)
Design Details
Highlight editable scripts contents
Divide whole script by frames (scripts)
Overview side bar
Color notations for clearer visualization
Highlight notations between each part of the workflow, so that user can get a clearer understand of the associated relationship between script, code and frame.
Demo Video
Evaluation and Conclusion
Some Improvable Aspects
Some Take-Away
Functionality: version tracking / memory keeping in iterations
Learn from our users
Functionality: each part of the generative result could have higher quality
Learn from the feedback of collaborators
User experience: clearer instruction on script programmatical grammar that reduce learning burden
More flexible creation options
Design with the whole pipeline and schedule in mind
Index: Dev pipeline & implementation
Overall Pipeline
Details of the implementation of Design Script → Executable Script → Video:
Early stage demo record
-
Introduced a new GAIA target type “VideoScene” to accept arbitrary tag attributes generated from Python modules.
-
Define Layout on Scene level rather than distribute into each clip.
-
Run our generated SVG and Script on GAIA-UI