I'm working on a system to do lip synch. I already made an animation where
AL the Alien talks back in November 2000, but the system I'm working on now
is much more flexible and powerful than what I had back then.

For any lip pose you can specify the weight, so you for example can have
anything between a little "o" and a roaring "OH!" by specifying a number
from 0 to 1. It also supports weighted lip poses, so you can average
multiple phoneme poses and/or expression poses.

The lips to be animated consist of four bicubic patches. You have some
control over how the mouth looks, but not very much unfortunately. So you
can use them for certain toony characters, but you can't adapt them to any
character you might have.

The attached animation is a test of the system. I still need to implement
support for tongues (which is a major thing), but besides that the system is
fairly complete.

As always, comments are much appreciated! :)

Also, if I made an include file of this system, would anybody have a use for
it?

Rune
--
3D images and anims, include files, tutorials and more:
Rune's World:    http://rsj.mobilixnet.dk (updated Jan 2)
POV-Ray Users:   http://rsj.mobilixnet.dk/povrayusers/
POV-Ray Webring: http://webring.povray.co.uk

Post a reply to this message

Attachments:
Download 'lips1.mpg' (637 KB)

over the top excellent... keep up the progress and post results!
:o)

-peter
-- 
Current obsession: "Ballet pour ma fille."
http://www.applesnake.net

Post a reply to this message

very impressive :)

who is the voice? (and do they really sound like that?)


--

Rick

Kitty5 WebDesign - http://Kitty5.com
POV-Ray News & Resources - http://Povray.co.uk
TEL : +44 (01270) 501101 - FAX : +44 (01270) 251105 - ICQ : 15776037

PGP Public Key
http://pgpkeys.mit.edu:11371/pks/lookup?op=get&search=0x231E1CEA

Post a reply to this message

"Dearmad" wrote:
> over the top excellent...

Thank you! :)

> keep up the progress and post results!
> :o)

Will do. However, I'll stick to the same animation and only improve on that.
Making new lip synch animations takes quite a while, so I don't feel like
making additional ones right now.

Rune
--
3D images and anims, include files, tutorials and more:
Rune's World:    http://rsj.mobilixnet.dk (updated Jan 2)
POV-Ray Users:   http://rsj.mobilixnet.dk/povrayusers/
POV-Ray Webring: http://webring.povray.co.uk

Post a reply to this message

I'm really amazed.

I remember that you said, not very long ago, that you were going to make a
game using POV images. Have you finished it? Is this lip-synch test going to
be used in the game?

Congratulations on your good work!

Fernando.

Post a reply to this message

"Fernando González del Cueto" wrote:
> I'm really amazed.
>
> Congratulations on your good work!

Thank you! :)

> I remember that you said, not very long ago, that
> you were going to make a game using POV images.
> Have you finished it? Is this lip-synch test going
> to be used in the game?

The game has nothing to do with this lip-synch. The game was finished a
while ago but it's nothing special and it crashed a lot. You can read about
it in povray.general in the thread "MazeQuest - a computer game" posted
December 19 2001.

Rune
--
3D images and anims, include files, tutorials and more:
Rune's World:    http://rsj.mobilixnet.dk (updated Jan 2)
POV-Ray Users:   http://rsj.mobilixnet.dk/povrayusers/
POV-Ray Webring: http://webring.povray.co.uk

Post a reply to this message

"Rick [Kitty5]" wrote:
> very impressive :)

Thank you! :)

> who is the voice? (and do they really sound like that?)

Err... It's my voice times three - and I don't really sound like that... ;)

Rune
--
3D images and anims, include files, tutorials and more:
Rune's World:    http://rsj.mobilixnet.dk (updated Jan 2)
POV-Ray Users:   http://rsj.mobilixnet.dk/povrayusers/
POV-Ray Webring: http://webring.povray.co.uk

Post a reply to this message

Incredible work, Rune.  Your animations are just stupendous! Considering there
aren't any actual "lips" present, however, are you going to add support for
different lip sizes?  i.e. creating an ethnic variance?

Lawrence

Rune wrote:

> I'm working on a system to do lip synch. I already made an animation where
> AL the Alien talks back in November 2000, but the system I'm working on now
> is much more flexible and powerful than what I had back then.
>
> For any lip pose you can specify the weight, so you for example can have
> anything between a little "o" and a roaring "OH!" by specifying a number
> from 0 to 1. It also supports weighted lip poses, so you can average
> multiple phoneme poses and/or expression poses.
>
> The lips to be animated consist of four bicubic patches. You have some
> control over how the mouth looks, but not very much unfortunately. So you
> can use them for certain toony characters, but you can't adapt them to any
> character you might have.
>
> The attached animation is a test of the system. I still need to implement
> support for tongues (which is a major thing), but besides that the system is
> fairly complete.
>
> As always, comments are much appreciated! :)
>
> Also, if I made an include file of this system, would anybody have a use for
> it?
>
> Rune
> --
> 3D images and anims, include files, tutorials and more:
> Rune's World:    http://rsj.mobilixnet.dk (updated Jan 2)
> POV-Ray Users:   http://rsj.mobilixnet.dk/povrayusers/
> POV-Ray Webring: http://webring.povray.co.uk
>
>                 Name: lips1.mpg
>    lips1.mpg    Type: Wmplayer File (video/mpg)
>             Encoding: x-uuencode

Post a reply to this message

"Lawrence Winstead" wrote:
> Incredible work, Rune.  Your animations are just stupendous!

Thank you! :)

> Considering there aren't any actual "lips" present,
> however, are you going to add support for different
> lip sizes?  i.e. creating an ethnic variance?

You can use any texture on the mouth region, and that way "paint" any kind
of lips using pigments and normals. However, you can't make lips that
actually bulge out...

Rune
--
3D images and anims, include files, tutorials and more:
Rune's World:    http://rsj.mobilixnet.dk (updated Jan 2)
POV-Ray Users:   http://rsj.mobilixnet.dk/povrayusers/
POV-Ray Webring: http://webring.povray.co.uk

Post a reply to this message

"Rune" wrote:
> Also, if I made an include file of this system,
> would anybody have a use for it?

Maybe I should rephrase this.

Cleaning up the code in an include file and write documentation for it takes
a fair amount of time, so before doing that I'd first like to know if
there's actually anybody interested in it. Would you be interested in a lip
synch system?

Rune
--
3D images and anims, include files, tutorials and more:
Rune's World:    http://rsj.mobilixnet.dk (updated Jan 2)
POV-Ray Users:   http://rsj.mobilixnet.dk/povrayusers/
POV-Ray Webring: http://webring.povray.co.uk

Post a reply to this message

>Would you be interested in a lip synch system?

    I think I speak for everyone who could -never- accomplish this
themselves when I say YES! YES! PLEASE YES!

Post a reply to this message

Yes, I would too

Thanks Rune

--
Regards,
C.J. POV User
PRSG
http://hyperion.spaceports.com/~povstudy/
hou### [at] yahoocom

Post a reply to this message

This is another stunning piece of work!
Yes I would be interested in a lip synching system.

Generalizing this to work with an arbitrary mouth would be nice.
Perhaps your mouth presets could act as a set of influence points for any mouth
that fits within the same bounding box ( or prescale your mouth to fit theirs)
similar to a FFD box. Feed it to a custom morph macro and it could work with the
pcm package too.

Just thinking 'out loud' (so to speak)


--
light_source{0,1}#macro c(J,a)sphere{0,1pigment{rgb z}scale a translate J+O}
#end#macro B(R,V,O)c(0,4)intersection{c(V,R)difference{c(-z*4x+10)c(-z*4.1x+
10)c(0<7.5,45,5>)}}#end B(12,0z*25)B(8y*4<0,12,50>)          // Batronyx ^"^

Post a reply to this message

"Batronyx" wrote:
> This is another stunning piece of work!

Thanks!

> Yes I would be interested in a lip synching system.

I've noted this, and the other replies too. I'm glad there's some interest.
:)

> Generalizing this to work with an arbitrary mouth
> would be nice.

Depends on what you mean by arbitrary mouth. The include file uses four
bicubic patches set up in a specific way, so another arrangement of bicubic
patches doesn't work, nor does, say a mesh or mesh2. That way it's rather
limited, and impossible for me to add support for mouths made of other
objects.

However, you have a little control over the shape of the mouth. All three in
the example animation were custom defined, and the user can make other
similar, but different mouths.

> Perhaps your mouth presets could act as a set of
> influence points for any mouth that fits within
> the same bounding box

You assume a mesh, while my include file uses a completely different
approach.

Rune
--
3D images and anims, include files, tutorials and more:
Rune's World:    http://rsj.mobilixnet.dk (updated Jan 2)
POV-Ray Users:   http://rsj.mobilixnet.dk/povrayusers/
POV-Ray Webring: http://webring.povray.co.uk

Post a reply to this message

Amazing.

On my plate is the idea of coming up with a jaw/teeth/lips/cheek system for
blobs.  Perhaps this use of blobs is so much using a "hammer as a
paintbrush" I should just give up and make my blobs with b.p.'s.

Yes, please release some code, genius mon.

Post a reply to this message

> Perhaps this use of blobs is so much using a "hammer as a
> paintbrush" I should just give up and make my blobs with b.p.'s.
>

Er, I meant "make my characters' heads with bicubic patches, but keep bodies
as blobs."

Post a reply to this message