Hello,

I may be writting a program that will do some sort of image recognition - or
rather image analysis.

Goal - to make "magic wand" fill like tool. Like the one in GIMP, but much
smarter.

If You take a room like http://limcore.com/misc/wand/ room1 or 3

Then, when you select few points on the wall, and few points inside unwanted
objects, then entire wall should be nicely selected by the program.

Obviously, simple "match same RGB/HSV color" is not enough here.

The match should be accurate, and using around 3..4 points "good", and just
few to mark unwanted objects must be enough to get right selection.

Images will be bigger, like 1000x1000 pixel (typical photo) and ok quality.

Quality: in room3, even if the border of photos would have the color of
wall, program should not select them. A bit of user help there may be
needed (but just showing few points, not doing it all by hand instead of
program)

So... what is interesting about it?

1) cool task \o/

2) would You take such task, and if so, then
  a) how much would it take You to write such program probably (the   
    algorithms + simple GUI to test)
  b) how much money would You want for it

3) any ideas how to implement this?

I have some interesting ideas, assuming photos will be usuall
architecture-like, then algorithms to
* detect smooth light and extend selection
* detect smooth shadows
* detect lighting changees (i.e. the window light-shadow on floor of room1)
* detect a pattern (bricks in room3) and do NOT extend if the color is same
but patter changed (i.e. if the border of the photo would be exact same
color as wall, but smooth)

Post a reply to this message

> 2) would You take such task, and if so, then
>  a) how much would it take You to write such program probably (the
>    algorithms + simple GUI to test)

My friend took around 6 months full time to do his project, but he was a 
student, so let's say 3 months in the real world ;-)

>  b) how much money would You want for it

I would expect a maximum of around 6000 GBP.  Much more than that and you 
could pay a CS graduate student to do it for you.

> 3) any ideas how to implement this?

> * detect smooth light and extend selection
> * detect smooth shadows
> * detect lighting changees (i.e. the window light-shadow on floor of 
> room1)
> * detect a pattern (bricks in room3) and do NOT extend if the color is 
> same

I would first run an algorithm to classify the image into different parts, 
like "floor", "wall", "bed", "door" etc.  There are already algorithms to do 
this (actually a friend of mine did this for his final year project at 
university).  Then using the input points from the user you can decide which 
parts to colour.

To actually classify different parts of the image, the maths is fairly 
hardcore.  I'm sure you can find lots of material online about it, I would 
say it's best to convert your image to HSV and then write some pattern 
matching algorithm to give each pixel a unique "group number" value.  You 
might want to look at fourier and wavelet transofrms to help with the 
pattern matching.

Post a reply to this message

On Mon, 31 Mar 2008 05:33:37 +0200, Rafał Maj wrote:

> Then, when you select few points on the wall, and few points inside
> unwanted objects, then entire wall should be nicely selected by the
> program.

This sounds a lot like SIOX.

Jim

Post a reply to this message

I'm a bit out of my depth here, but I might be able to give a bit of 
useful info nonetheless.  Firstly, I'd suggest that if you don't have 
some experience in computer vision already (I have no idea weather or 
not this is the case) then you hire someone who does, since vision-type 
algorithms are very easy to get very wrong unless you're implementing 
something which is already known to work.

As for some possible approaches that you might use to solve this, you 
might look at the work which had been done in image segmentation, and 
more directly relevant to your problem image matting and image colorization.

The most common approaches that I've seen in image matting and 
colorization tend to be based around defining a error measure between 
pixels defining how likely they are to be in the same region (or be the 
same color) and then based on a few marks from a user determine the 
lowest-error global matting / colorization for the entire image.  The 
min-cut algorithm is well suited to this sort of problem, but others 
have been used as well.

The problem that these methods might have in your case is that they 
generally rely on building a color model to define a region (so they 
won't work well when the region colors are very similar) or use image 
gradients to define the error metric (in which case shadows might cause 
problems).

Despite its possible drawbacks for this application, I'd say that 
looking at a gradient-based error metric is a good place to start. 
Fortunately, there's already a paper on this with respect to image 
colorization which has some Matlab code available for you to play around 
with:

http://www.cs.huji.ac.il/~yweiss/Colorization/

Your problem isn't strictly colorization, but it's closely related so 
this may well be useful.

The SIOX algorithm mentioned by Jim is a feature in GIMP version 2.4, so 
you can try it out there, but I suspect that it won't work too well for 
many of your images (just a hunch though, I could easily be wrong).

Scott's suggestion of using object recognition techniques is pretty 
reasonable, particularly since you only want this to work for a very 
constrained set of images.  The disadvantage of this is that you'll 
probably need a large set of labeled images to train a classifier on, 
but I'd be surprised if such data isn't available somewhere, and you 
could always bite the bullet and label a bunch of images yourself.  If 
you do go this route, I'd guess that you'd want to use the object 
recognition to augment a matting/colorization style algorithm so that 
it's easy for the user to correct any mistakes that the algorithm makes.

Overall, my biggest high-level suggestion would be to use an algorithm 
that runs quickly enough for a user to iteratively correct any mistakes 
that it makes, since it's not always easy to tell in advance where and 
how they will fail.  Also, I haven't done any actual programming in this 
area, so take all my advice with the appropriate grain of salt.  Good luck!

Post a reply to this message