POV-Ray : Newsgroups : povray.binaries.animations : Music Video Project Server Time
23 Sep 2021 11:09:39 EDT (-0400)
  Music Video Project (Message 1 to 10 of 36)  
Goto Latest 10 Messages Next 10 Messages >>>
From: Dave Blandston
Subject: Music Video Project
Date: 10 Apr 2021 20:25:00
Message: <web.607240c8c138faf79416a1f9334df62@news.povray.org>
Greetings Friends,

More than ten years ago I made a scene that I didn't think turned out quite
right so I set it aside until recently. I re-worked some of the textures and
made some improvements. One of the improvements was to change the flag from a
flat box with fake shadows to simulate ripples (not very convincing) to a mesh2
object. Since the mesh2 is easily animatable with a simple sine function that
led to this animation:

https://www.youtube.com/watch?v=gYJac3vCHu8

The message of the image is anti-war and it's based on the Metallica song
Disposable Heroes, so I've decided to add the music to the animation and try
animating the inside (currently green) portion of the Metallica logo. My goal is
to animate the logo in such a manner that the animation is synchronized with the
music, similar to the visualizations in Windows Media Player and other music
players. It will be a big job to do by hand. One thing that would help would be
a software application that would analyze the music and generate a list of exact
times when a change occurs in the music and save the list as a text file that
POV-Ray could read, but creating a program that does that is beyond my ability
so I will try doing it by hand. There are endless possibilities so this will be
a fun project! One thing that I have working at this point is using video frames
as the pigment for one or more letters of the logo at a time, which can be
combined with procedural textures.

Kind regards,
Dave Blandston


Post a reply to this message

From: m@b
Subject: Re: Music Video Project
Date: 10 Apr 2021 21:30:14
Message: <60725126@news.povray.org>
On 11/04/2021 8:20 am, Dave Blandston wrote:
> Greetings Friends,
> 
> More than ten years ago I made a scene that I didn't think turned out quite
> right so I set it aside until recently. I re-worked some of the textures and
> made some improvements. One of the improvements was to change the flag from a
> flat box with fake shadows to simulate ripples (not very convincing) to a mesh2
> object. Since the mesh2 is easily animatable with a simple sine function that
> led to this animation:
> 
> https://www.youtube.com/watch?v=gYJac3vCHu8
> 
> The message of the image is anti-war and it's based on the Metallica song
> Disposable Heroes, so I've decided to add the music to the animation and try
> animating the inside (currently green) portion of the Metallica logo. My goal is
> to animate the logo in such a manner that the animation is synchronized with the
> music, similar to the visualizations in Windows Media Player and other music
> players. It will be a big job to do by hand. One thing that would help would be
> a software application that would analyze the music and generate a list of exact
> times when a change occurs in the music and save the list as a text file that
> POV-Ray could read, but creating a program that does that is beyond my ability
> so I will try doing it by hand. There are endless possibilities so this will be
> a fun project! One thing that I have working at this point is using video frames
> as the pigment for one or more letters of the logo at a time, which can be
> combined with procedural textures.
> 
> Kind regards,
> Dave Blandston
> 

I use Audacity to find the start time of each phrase in the song, I then 
put these times in a spreadsheet to get the clock values and frame 
numbers for POV-Ray .
See screen shots.

Best wishes,
m@


Post a reply to this message


Attachments:
Download 'aud1.gif' (42 KB) Download 'spread1.gif' (9 KB)

Preview of image 'aud1.gif'
aud1.gif

Preview of image 'spread1.gif'
spread1.gif


 

From: Dave Blandston
Subject: Re: Music Video Project
Date: 10 Apr 2021 22:25:00
Message: <web.60725d6e432b07e179416a1f9334df62@news.povray.org>
"m@b" <sai### [at] googlemailcom> wrote:
> I use Audacity to find the start time of each phrase in the song, I then
> put these times in a spreadsheet to get the clock values and frame
> numbers for POV-Ray .
> See screen shots.
>
> Best wishes,
> m@

That's almost exactly what I'm planning to do (minus the spreadsheet) but I
think it will be painstaking. How has this method worked out for you? I have
written a POV-Ray macro that will keep track of the number of frames and elapsed
time so that will help a little. Choosing a song that's over eight minutes long
might not have been the best idea for me...

Have a great day!


Post a reply to this message

From: m@b
Subject: Re: Music Video Project
Date: 11 Apr 2021 02:54:21
Message: <60729d1d$1@news.povray.org>
On 11/04/2021 10:22 am, Dave Blandston wrote:
> "m@b" <sai### [at] googlemailcom> wrote:
>> I use Audacity to find the start time of each phrase in the song, I then
>> put these times in a spreadsheet to get the clock values and frame
>> numbers for POV-Ray .
>> See screen shots.
>>
>> Best wishes,
>> m@
> 
> That's almost exactly what I'm planning to do (minus the spreadsheet) but I
> think it will be painstaking. How has this method worked out for you? I have
> written a POV-Ray macro that will keep track of the number of frames and elapsed
> time so that will help a little. Choosing a song that's over eight minutes long
> might not have been the best idea for me...
> 
> Have a great day!
> 

I don't find it takes too much time, admittedly I chose shorter and much 
  slower songs than you are planning.

If there is a constant beat throughout the song you can set up a 
variable direct from clock which can be used to animate all sorts of things.

See:
<https://www.tiktok.com/@matthew.bradwell/video/6944909548163484930>
and
<https://www.youtube.com/watch?v=djJGXZB6yRE>

Best,
m@

p.s. I just watched the song - there is a lot going on there!


Post a reply to this message

From: Dave Blandston
Subject: Re: Music Video Project
Date: 11 Apr 2021 05:00:00
Message: <web.6072ba5a432b07e179416a1f9334df62@news.povray.org>
"m@b" <sai### [at] googlemailcom> wrote:
> I don't find it takes too much time, admittedly I chose shorter and much
>   slower songs than you are planning.
>
> If there is a constant beat throughout the song you can set up a
> variable direct from clock which can be used to animate all sorts of things.
>
> See:
> <https://www.tiktok.com/@matthew.bradwell/video/6944909548163484930>
> and
> <https://www.youtube.com/watch?v=djJGXZB6yRE>
>
> Best,
> m@
>
> p.s. I just watched the song - there is a lot going on there!

I enjoyed your video very much, especially the leaning tree and the big
geographical surprise at the end! The story is very entertaining as well -
that's a great accomplishment!


Post a reply to this message

From: Kenneth
Subject: Re: Music Video Project
Date: 11 Apr 2021 10:00:00
Message: <web.60730089432b07e1d98418916e066e29@news.povray.org>
"m@b" <sai### [at] googlemailcom> wrote:

> I use Audacity to find the start time of each phrase in the song, I then
> put these times in a spreadsheet to get the clock values and frame
> numbers for POV-Ray .

I use Audacity too, a great free app. And your spreadsheet for creating
animation timings is a great idea. If I understand your spreadsheet-use
correctly, how do you ultimately output the values there into a form that
POV-ray can #read during its animation render? I assume that it would be a
simple text file-- but does the spreadsheet app automatically format its
frame-number data in the proper POV-ray-required format?

(Obviously, I'm not very familiar with spreadsheet use.)
>
> If there is a constant beat throughout the song you can set up a
> variable direct from clock which can be used to animate all sorts of things.
>

The 'constant beat' is sometimes not so constant, ha. Especially during long
songs. (Unless the beat is generated by a drum machine or click track, for
example.) Human drummers sometimes have a tendency to ever-so-slightly slow down
or speed up-- which is probably not noticed by most listeners, but might create
minor glitches when compared to computer timings.


Post a reply to this message

From: Dave Blandston
Subject: Re: Music Video Project
Date: 11 Apr 2021 22:20:00
Message: <web.6073ae0b432b07e179416a1f9334df62@news.povray.org>
"m@b" <sai### [at] googlemailcom> wrote:
> I don't find it takes too much time, admittedly I chose shorter and much
>   slower songs than you are planning.
>
> If there is a constant beat throughout the song you can set up a
> variable direct from clock which can be used to animate all sorts of things.
>
> See:
> <https://www.tiktok.com/@matthew.bradwell/video/6944909548163484930>
> and
> <https://www.youtube.com/watch?v=djJGXZB6yRE>
>
> Best,
> m@
>
> p.s. I just watched the song - there is a lot going on there!

I'm curious about a couple things, since you have already solved some problems
that I'm currently working on. Did you create one giant POV-Ray file for the
entire animation, or did you split it up into multiple files? If you split it
up, did you need to devise a method of passing information from one file to the
next? If so I'm very interested to hear the basic details of how you did this,
especially since you had to plan much farther ahead by positioning the camera
correctly. I will have to split my project into multiple files for sure and so
far I've written a macro that automatically keeps track of the current number of
frames/time elapsed but ideally it would be nice to be able to pass information
about changing textures as well. I don't think this is possible, but maybe
there's a way...

Also my original belief that it would be possible to visually examine the
waveform with a music editor and identify beats and other musical events has
turned out to be totally false. Drum beats can be identified easily enough but
bass notes are not as obvious especially when there is singing and/or other
guitar stuff going on simultaneously.

It has been brought to my attention that the mathematical method of detecting
beats/changes is by the use of Fourier transforms. So my current options are to
try to locate and adapt some Fourier transform code to POV-Ray SDL and use it to
generate a list of acoustic event times (probably beyond my ability), or find
another alternative. One option that I'll explore next is the "spectral" view.
In the past I used Magix Audio Cleaning Lab to remove tape hiss and unwanted
sounds from commercial recordings and one of the features was "spectral
cleaning" which displayed the music in a unique visual manner, which may help.
Re-installing that program and checking that option out will be my next project.

I made a five-second test animation by guessing the approximate times of the
first few bass notes and it's definitely not good enough. I posted it anyway for
an example of what not to do:

https://www.youtube.com/watch?v=Rv8kGq9PUkM

This also includes a test of using video frames in one of the characters of the
logo. I picked a random five-second segment from a concert video and it didn't
work out at all, at least not with the big "A." I think that carefully selected
video segments have potential in the smaller characters though.

Cheers,
Dave Blandston


Post a reply to this message

From: m@b
Subject: Re: Music Video Project
Date: 12 Apr 2021 04:36:28
Message: <6074068c@news.povray.org>
On 12/04/2021 10:18 am, Dave Blandston wrote:
> "m@b" <sai### [at] googlemailcom> wrote:
>> I don't find it takes too much time, admittedly I chose shorter and much
>>    slower songs than you are planning.
>>
>> If there is a constant beat throughout the song you can set up a
>> variable direct from clock which can be used to animate all sorts of things.
>>
>> See:
>> <https://www.tiktok.com/@matthew.bradwell/video/6944909548163484930>
>> and
>> <https://www.youtube.com/watch?v=djJGXZB6yRE>
>>
>> Best,
>> m@
>>
>> p.s. I just watched the song - there is a lot going on there!
> 
> I'm curious about a couple things, since you have already solved some problems
> that I'm currently working on. Did you create one giant POV-Ray file for the
> entire animation, or did you split it up into multiple files? If you split it
> up, did you need to devise a method of passing information from one file to the
> next? If so I'm very interested to hear the basic details of how you did this,
> especially since you had to plan much farther ahead by positioning the camera
> correctly. I will have to split my project into multiple files for sure and so
> far I've written a macro that automatically keeps track of the current number of
> frames/time elapsed but ideally it would be nice to be able to pass information
> about changing textures as well. I don't think this is possible, but maybe
> there's a way...
> 
> Also my original belief that it would be possible to visually examine the
> waveform with a music editor and identify beats and other musical events has
> turned out to be totally false. Drum beats can be identified easily enough but
> bass notes are not as obvious especially when there is singing and/or other
> guitar stuff going on simultaneously.
> 
> It has been brought to my attention that the mathematical method of detecting
> beats/changes is by the use of Fourier transforms. So my current options are to
> try to locate and adapt some Fourier transform code to POV-Ray SDL and use it to
> generate a list of acoustic event times (probably beyond my ability), or find
> another alternative. One option that I'll explore next is the "spectral" view.
> In the past I used Magix Audio Cleaning Lab to remove tape hiss and unwanted
> sounds from commercial recordings and one of the features was "spectral
> cleaning" which displayed the music in a unique visual manner, which may help.
> Re-installing that program and checking that option out will be my next project.
> 
> I made a five-second test animation by guessing the approximate times of the
> first few bass notes and it's definitely not good enough. I posted it anyway for
> an example of what not to do:
> 
> https://www.youtube.com/watch?v=Rv8kGq9PUkM
> 
> This also includes a test of using video frames in one of the characters of the
> logo. I picked a random five-second segment from a concert video and it didn't
> work out at all, at least not with the big "A." I think that carefully selected
> video segments have potential in the smaller characters though.
> 
> Cheers,
> Dave Blandston
> 

I put the scene all in one file, the timings go in another (.inc) file, 
I find it convenient to switch between the two when working.

Here is the layout of the timing file:

SETUP TIMES FROM THE SPREADSHEET

#declare T000 = 0.0000;  // Start
#declare T001 = 0.0667;  // Weird                   Zoom to 1
#declare T002 = 0.0833;  // Dum dum                 Rotate Florets
#declare T003 = 0.1433;  // I'm a little weird      Zoom out 1
#declare T004 = 0.2083;  // I wear a little beard   To Ladybird
#declare T005 = 0.2483;  // Durul ural um dum       Ladybird

INITIALISE ELEMENTS THAT WILL BE ACTIVATED/CHANGED DURING RENDER

#declare FlowerHead  = 0;
#declare Stalk       = 0;
#declare Grass       = 0;
#declare GroundPlane = 1;
#declare Mushroom    = 0;

USE SWITCH TO RENDER SCENE BY SCENE
See screen shot

This macro sets up some local clocks. Start runs from 1 to 0, End runs 
from 0 to 1, they both start and end slowly like an S-curve, this avoids 
jerky camera movements at each scene change. Bumper runs 0-1-0.

#macro Cosify()
	#declare Start =  (cos(1*pi*Current_clock)+1)/2;
	#declare End = (1-(cos(1*pi*Current_clock)+1)/2);
	#declare Bumper = sin(pi*Current_clock); #end

PRINT OUT SOME STATS SO i CAN FOLLOW HOW THE ANIMATION IS PROGRESSING

#debug concat( Handle_1, "   Clock = ", str(clock,0,3), " 
Current_clock = ", str(Current_clock,0,4), "    Frame = ", 
str(frame_number,0,0),  "\n")
#debug concat("Look at = ", Vstr(Camera_look_at, 0, 0), "    Camera 
location = ", Vstr(Camera_shift, 0, 0),  "    Zoom = ", str(Zoom,2,2), 
"\n")


Post a reply to this message


Attachments:
Download 'povshot1.gif' (28 KB)

Preview of image 'povshot1.gif'
povshot1.gif


 

From: m@b
Subject: Re: Music Video Project
Date: 12 Apr 2021 04:47:15
Message: <60740913@news.povray.org>
On 11/04/2021 9:59 pm, Kenneth wrote:
> "m@b" <sai### [at] googlemailcom> wrote:
> 
>> I use Audacity to find the start time of each phrase in the song, I then
>> put these times in a spreadsheet to get the clock values and frame
>> numbers for POV-Ray .
> 
> I use Audacity too, a great free app. And your spreadsheet for creating
> animation timings is a great idea. If I understand your spreadsheet-use
> correctly, how do you ultimately output the values there into a form that
> POV-ray can #read during its animation render? 

I just copy and paste manually. I am sure it would be possible to output 
the spreadsheet as a CSV file and have POV-Ray read it but I haven't yet.

>> If there is a constant beat throughout the song you can set up a
>> variable direct from clock which can be used to animate all sorts of things.
>>
> 
> The 'constant beat' is sometimes not so constant, ha. Especially during long
> songs. (Unless the beat is generated by a drum machine or click track, for
> example.) 

Good point


Post a reply to this message

From: Dave Blandston
Subject: Re: Music Video Project
Date: 12 Apr 2021 06:35:00
Message: <web.60742240432b07e179416a1f9334df62@news.povray.org>
"m@b" <sai### [at] googlemailcom> wrote:
> I put the scene all in one file, the timings go in another (.inc) file,
> I find it convenient to switch between the two when working.
>
> Here is the layout of the timing file:
>
> SETUP TIMES FROM THE SPREADSHEET
>
> #declare T000 = 0.0000;  // Start
> #declare T001 = 0.0667;  // Weird                   Zoom to 1
> #declare T002 = 0.0833;  // Dum dum                 Rotate Florets
> #declare T003 = 0.1433;  // I'm a little weird      Zoom out 1
> #declare T004 = 0.2083;  // I wear a little beard   To Ladybird
> #declare T005 = 0.2483;  // Durul ural um dum       Ladybird
>
> INITIALISE ELEMENTS THAT WILL BE ACTIVATED/CHANGED DURING RENDER
>
> #declare FlowerHead  = 0;
> #declare Stalk       = 0;
> #declare Grass       = 0;
> #declare GroundPlane = 1;
> #declare Mushroom    = 0;
>
> USE SWITCH TO RENDER SCENE BY SCENE
> See screen shot
>
> This macro sets up some local clocks. Start runs from 1 to 0, End runs
> from 0 to 1, they both start and end slowly like an S-curve, this avoids
> jerky camera movements at each scene change. Bumper runs 0-1-0.
>
> #macro Cosify()
>  #declare Start =  (cos(1*pi*Current_clock)+1)/2;
>  #declare End = (1-(cos(1*pi*Current_clock)+1)/2);
>  #declare Bumper = sin(pi*Current_clock); #end
>
> PRINT OUT SOME STATS SO i CAN FOLLOW HOW THE ANIMATION IS PROGRESSING
>
> #debug concat( Handle_1, "   Clock = ", str(clock,0,3), "
> Current_clock = ", str(Current_clock,0,4), "    Frame = ",
> str(frame_number,0,0),  "\n")
> #debug concat("Look at = ", Vstr(Camera_look_at, 0, 0), "    Camera
> location = ", Vstr(Camera_shift, 0, 0),  "    Zoom = ", str(Zoom,2,2),
> "\n")

Hey this is super interesting! We are doing some very similar things - I am also
using local "Event clocks" that go from 0 to 1 at pre-determined intervals, and
look how similar our "smoothing" functions are:

#local ClockFactor = (sin (radians (EventClock [I] * 180 - 90)) + 1) / 2; //0 ..
1
#local ClockFactor = pow (ClockFactor, 1); //Adjust for a faster or slower start

I am in luck regarding my hope of being able to see changes in the music better
than just looking at the waveform. I got the spectral display up and running and
it makes it pretty easy to spot acoustic "events." The screen shot of the first
five seconds of music that I'm using clearly shows six quick bass notes then
three slower ones then five quick ones etc. The time display is in milliseconds
which is plenty accurate. Unfortunately there are many hundreds of musical
events that I will have to log by hand. But the final product should be nearly
perfect.

Thanks for sharing your technique!


Post a reply to this message


Attachments:
Download 'screen.jpg' (397 KB)

Preview of image 'screen.jpg'
screen.jpg


 

Goto Latest 10 Messages Next 10 Messages >>>

Copyright 2003-2021 Persistence of Vision Raytracer Pty. Ltd.