NASA Logo

NTRS

NTRS - NASA Technical Reports Server

Back to Results
(abstract) Synthesis of Speaker Facial Movements to Match Selected Speech SequencesWe are developing a system for synthesizing image sequences the simulate the facial motion of a speaker. To perform this synthesis, we are pursuing two major areas of effort. We are developing the necessary computer graphics technology to synthesize a realistic image sequence of a person speaking selected speech sequences. Next, we are developing a model that expresses the relation between spoken phonemes and face/mouth shape. A subject is video taped speaking an arbitrary text that contains expression of the full list of desired database phonemes. The subject is video taped from the front speaking normally, recording both audio and video detail simultaneously. Using the audio track, we identify the specific video frames on the tape relating to each spoken phoneme. From this range we digitize the video frame which represents the extreme of mouth motion/shape. Thus, we construct a database of images of face/mouth shape related to spoken phonemes. A selected audio speech sequence is recorded which is the basis for synthesizing a matching video sequence; the speaker need not be the same as used for constructing the database. The audio sequence is analyzed to determine the spoken phoneme sequence and the relative timing of the enunciation of those phonemes. Synthesizing an image sequence corresponding to the spoken phoneme sequence is accomplished using a graphics technique known as morphing. Image sequence keyframes necessary for this processing are based on the spoken phoneme sequence and timing. We have been successful in synthesizing the facial motion of a native English speaker for a small set of arbitrary speech segments. Our future work will focus on advancement of the face shape/phoneme model and independent control of facial features.
Document ID
20060037878
Acquisition Source
Jet Propulsion Laboratory
Document Type
Preprint (Draft being sent to journal)
External Source(s)
Authors
Scott, Kenneth C.
Date Acquired
August 23, 2013
Publication Date
December 5, 1994
Distribution Limits
Public
Copyright
Other
Keywords
graphics audio digitizing morphing enunciation phonemes English language human
animation imaging speech synthesis human speech video digital images computer
facial features image sequences

Available Downloads

There are no available downloads for this record.
No Preview Available