Check me on this one...

I have a motion detector written in terms of a handful of pixel shaders. 
(Make it greyscale, compare this frame against two previous frames for 
differences, threshold, erode, etc.)

Now all I'm really interested in is how much motion there is.

Is there any way in a pixel shader to do something like count the number of 
white pixels, or find the bounding box around the white pixels? There's no 
way to have the calculations of one pixel or vertex dependent on the 
calculations of a different pixel in the same pass? Or any way to return an 
actual numeric value other than making a color?

I was thinking something like doing a lg_2(frame width) to propagate white 
pixels (motion) towards each edge in steps of power of two, so the leftmost 
edge would have white wherever there was any white pixel in the row and the 
topmost edge would have white where there was white anywhere in the column, 
but that was about as good as I could come up with.

I don't think a pass of a pixel shader can "see" the output of the previous 
pixel shader, right? Just possibly overwrite parts based on vertices, 
culling, alpha, and all that good stuff?

-- 
Darren New, San Diego CA, USA (PST)
   The question in today's corporate environment is not
   so much "what color is your parachute?" as it is
   "what color is your nose?"

Post a reply to this message

> Is there any way in a pixel shader to do something like count the number 
> of white pixels,

You might be able to use occlusion queries, the GPU counts all pixels that 
pass the depth test and are actually rendered.  In DirectX you can get this 
information with CreateQuery() and related functions.  You could clear your 
render buffer to 50% depth, and make the pixel shader output a depth value 
depending on whether you want it counted or not.  At the end of the pass use 
your API functions to read back the number of pixels rendered.

> There's no way to have the calculations of one pixel or vertex dependent 
> on the calculations of a different pixel in the same pass?

No because they run in parallel, and in an undefined order I expect.

> Or any way to return an actual numeric value other than making a color?

Well you can create a surface texture that is full of floats or doubles 
rather than RGBA bytes.  On later cards and APIs a pixel shader can also 
output multiple values to multiple render targets (but still limited to 4 or 
something I think, IDK the details).

> I don't think a pass of a pixel shader can "see" the output of the 
> previous pixel shader, right?

The usual method is to set the rendertarget as texture A (with input texture 
B) and run your pixel shader, then the rendertarget as texture B and use 
texture A as an input texture.  That way you get to use the previous output, 
and it isn't as slow as it sounds.

Post a reply to this message

scott wrote:
> get this information with CreateQuery() and related functions. 

Ah. Excellent. I think I saw something along those lines in the XNA stuff 
too. Thanks for the hint.

> At the end of the pass use your API functions to read back the number of 
> pixels rendered.

Hmmm. OK. I think I'm going to need to know something about where they are, 
too, but this might be a start.

>> There's no way to have the calculations of one pixel or vertex 
>> dependent on the calculations of a different pixel in the same pass?
> 
> No because they run in parallel, and in an undefined order I expect.

That's what I thought, yes.

>> Or any way to return an actual numeric value other than making a color?
> 
> Well you can create a surface texture that is full of floats or doubles 
> rather than RGBA bytes.  On later cards and APIs a pixel shader can also 
> output multiple values to multiple render targets (but still limited to 
> 4 or something I think, IDK the details).

OK. I would be willing to live with 255 values, or certainly 65K values is 
more than enough, so I probably don't have to figure out how to make a 
surface full of floats. :)

>> I don't think a pass of a pixel shader can "see" the output of the 
>> previous pixel shader, right?
> 
> The usual method is to set the rendertarget as texture A (with input 
> texture B) and run your pixel shader, then the rendertarget as texture B 
> and use texture A as an input texture.  That way you get to use the 
> previous output, and it isn't as slow as it sounds.

Right, but that isn't two "passes" in the same technique. (I've just seen 
very few techniques with multiple passes, is all, and I'm trying to figure 
out what they're so good for that you'd explicitly include the logic to 
support them all thru the IDE and languages.)

But that's a good hint, to have multiple render targets and instead of 
pulling the colors out and pushing them back in, just pick up the texture 
and feed that right back without even taking it off the card. That might 
save me some processing time.

Thanks! You've given me good stuff to look into.

-- 
Darren New, San Diego CA, USA (PST)
   The question in today's corporate environment is not
   so much "what color is your parachute?" as it is
   "what color is your nose?"

Post a reply to this message

Darren New wrote:
> Check me on this one...
> 
> I have a motion detector written in terms of a handful of pixel shaders. 
> (Make it greyscale, compare this frame against two previous frames for 
> differences, threshold, erode, etc.)
> 
> Now all I'm really interested in is how much motion there is.
> 
> Is there any way in a pixel shader to do something like count the number 
> of white pixels,

I haven't really done any shader coding myself, but IIRC the way to 
implement this would be with a summation tree.  If you're coding in a 
standard shader language instead of CUDA or OpenCL I suppose you'd do 
this in about the same way that you'd manually generate mipmaps from a 
base texture, only with a downsampling operation which sums the counts 
of the non-white pixels instead of averaging their colors.  You'll need 
to either create a surface with float precision or manually pack the 
sub-counts into the color components though.

Post a reply to this message

Kevin Wampler wrote:
> I haven't really done any shader coding myself, but IIRC the way to 
> implement this would be with a summation tree.

That was my thought. You loop thru it lg_2(width) times, say, each time 
doubling the offset, and each time setting this_pixel += pixel[offset].

Of course, I'll have to figure out if it's possible to *actually* access a 
pixel at a particular index. It only took me about 3 hours to figure out my 
erosion filter was comparing pixel[x] and pixel[x-1] and pixel[x+1], where x 
runs from 0.0 to 1.0. Sheesh.

> standard shader language instead of CUDA or OpenCL I suppose you'd do 
> this in about the same way that you'd manually generate mipmaps from a 
> base texture, only with a downsampling operation which sums the counts 
> of the non-white pixels instead of averaging their colors.  

I'll read up on that, thanks.

> You'll need 
> to either create a surface with float precision or manually pack the 
> sub-counts into the color components though.

I think I can live with one byte of precision, really. The image is coming 
off a 320x240 camera. If >256 out of 320 pixels changed, chances are they're 
moving the camera. :-)


-- 
Darren New, San Diego CA, USA (PST)
   The question in today's corporate environment is not
   so much "what color is your parachute?" as it is
   "what color is your nose?"

Post a reply to this message

> OK. I would be willing to live with 255 values, or certainly 65K values is 
> more than enough, so I probably don't have to figure out how to make a 
> surface full of floats. :)

Check out the SurfaceFormat enumeration in XNA, you can use it when creating 
a new Texture2D object.

> Right, but that isn't two "passes" in the same technique. (I've just seen 
> very few techniques with multiple passes, is all, and I'm trying to figure 
> out what they're so good for that you'd explicitly include the logic to 
> support them all thru the IDE and languages.)

Probably just a relic from the past, when you could only do a small amount 
of processing in a single pass.

> But that's a good hint, to have multiple render targets and instead of 
> pulling the colors out and pushing them back in, just pick up the texture 
> and feed that right back without even taking it off the card. That might 
> save me some processing time.

Transferring textures between the CPU and GPU on a per frame basis is never 
a good thing for performance!

Post a reply to this message

scott wrote:
> Transferring textures between the CPU and GPU on a per frame basis is 
> never a good thing for performance!

True. Right now it's just proof-of-concept and figuring out the algorithms. 
If the point wasn't a live camera, I'd let it take three seconds a frame if 
it had to. :-)

Right now I'm still in the version-zero thrashing around bit, after which 
I'll actually figure out what shape the program should have. :-)

-- 
Darren New, San Diego CA, USA (PST)
   The question in today's corporate environment is not
   so much "what color is your parachute?" as it is
   "what color is your nose?"

Post a reply to this message