Monday, August 17, 2009
Wednesday, November 26, 2008
Computer Vision(40): 3D Physical to 2D Projection of Depths
Sense is something that is fuzzy in humans; you know something is hot but cannot tell its exact temperature, you know something is farther away from the other but cannot tell the exact distance between the two, you know some sound is louder than the other but not by how much, etc… So long as you are interested in only the relative information our brain performs exactly but if you question its exactness it becomes relative and fuzzy.
Let me explain relative information taking the top view of the scenario as shown below.
Currently the objects are placed one beside the other (images not to scale). The relative depth between them will be zero as the distance between the objects in both the eyes will be the same. To observe relative depth between them we will have to first create physical depth between the two by pushing either one of them back. CASE 1: Let me push the rectangle back initially as shown below. ‘L’ and ‘R’ are the projected distances between the objects on the left and the right eye respectively. Whenever the object on the right is in front compared to the one on the left, the projected distance on the left eye will be greater than that on the right.
CASE 2: Now consider the image below in which the circle is pushed back. For our brain to perceive this physical relative depth the projected distance on the right eye has to greater than that on the left.
Let me bring into picture the image that actually created this doubt. If you cross view this stereogram the 2D distance between the objects that the left eye would perceive will be greater than that on the right which is CASE 1.
Saturday, August 30, 2008
Computer Vision (39): SDF continued
L - Lens.
Wednesday, August 27, 2008
Computer Vision (38): Software Defined Focus
I had already written a detailed article on what focus actually means to a camera through simple ray diagrams. Now I will take it beyond and will try to find a solution and define focus through software. I will again use simple ray diagrams to explain my observations and not the complex QED.
To revise the concepts a bit:
1. Light diverges in all possible directions from a point after getting reflected or emitted.
2. When it intersects the camera aperture, only a section of this spherical region enters to reach the camera sensor.
3. A cross section at the face of the lens wrt the point of commencement of light would give us a cone which in turn gets converged by the lens to fall on the sensor.
4. The image this cone creates on the sensor (circle or a point) depends on where along this convergence path it intersects the sensor.
Pr - real point of commencement of light.Po - some other real point.
L - Lens.
Puu - Unfocused Uncrossed Image of the point Pr with sensor at S1.
Pf - Focused Image of the point Pr with sensor at S2.
Puc - Unfocused Crossed Image of the point Pr with sensor at S3.
S1, S2 and S3 - Different positions of the sensor.
One way to get most or all the points to appear in focus is to increase the depth of focus by decreasing the aperture to as minimum (in size) as possible. This would make the light cone very narrow in width and the digital sensor would not capture it as a circle in spite of not being focused.
Now a days due to the increase in the sensor pixel density even a slight movement in the sensor from the position S2 would give an out of focused image of Pr. Though auto focus systems might be very accurate even a small movement in the camera position would out of focus the desired point in the image especially for macro shots in wide aperture. Once the image is frozen there is no way you can correct the focus except for a slight sharpening. This is because a circle is not uniquely defined by a cone. For example in the figure shown below image of the circle Ic on sensor S can be formed by any of the cones getting focused at F-1 (before the sensor), F1, F2 or F3 (after the sensor). In this format there is no way we can bring the focus back once the image is captured since we do not know the cone whose cross section this is.
Assuming even intensity distribution some people would argue that irrespective of where this point would converge we could simply take the sum of the spread intensity and put it at the location of focus. This can only be true for points on the optical axis and a single lens system. Camera lenses generally contain many groups of lenses and hence would be complex to analyze. If there are any experts who can solve this problem do let me know.Secondly, in the software solution that I am talking about as one point gets focused others at different planes should change correspondingly which is difficult in the above method. Moreover it is not possible to isolate a circle of a particular point in a natural scene where millions of points would get mixed up at the sensor. So what is the way out?
Monday, August 18, 2008
Photography and travel: Melkote
Sunday, August 17, 2008
Photography and Travel: Masinagudi
Wednesday, June 11, 2008
Photography and Travel: Savandurga
Road is pretty good at some places along the stretch as you can see below.
One can also visit the nearby dam and backwaters, which is a good spot to tent at night. On the way back you can also visit Dodda Alada Mara (The big baniyan tree). This is where one of the fight scenes of Khalnayak was shot if you remember.
Wednesday, April 2, 2008
Stargazing Olympics 2008:
Any company willing to promote its camera through this software??? Check out the online demo here: http://www.multishotimaging.com/
Friday, March 7, 2008
Voice munching
Thursday, December 27, 2007
Computer Vision (37): Sensing through Seismics, The Golden Mole
Some pythons have the ability to sense the infra red radiation from creatures and can even use it to hunt down their prey. Usually these are called pit snakes. Though not very well developed they still have eyes for vision, which leave these creatures not that special compared to the golden moles that I came across recently.
These creatures do not have eyes at all. They have extremely sensitive hearing and vibration detection, and can navigate underground with unerring accuracy. Morphological analysis of the middle ear has revealed a massive malleus which likely enables it to detect seismic cues. The make use of this seismic sensitivity to detect prey as well as to navigate when burrowing through sand. While vibrations are used over long distances to detect prey, smell is possibly used over shorter distances.FORSAKEN FANFARE
This person could embellish the algae clad wall with just a few colored chalks, and of course a lot of his esteemed abilities. People gathered to watch him chalk his imagination, but pretended not to recognize that it was not a charity show. He stood there smiling at the audience waiting to at least settle his accounts on the money he had spent for the chalks. It was shocking to see everyone disperse from there without even a single penny flying to his side. Seriously I feel that my Canon 350D failed to reproduce the shades (In fact I borrowed this snap from my friend) that he could create on such a dirty wall. With a canvas I think he will touch the skies.Here are some of the glimpses of kerala (Cochin, Attirapalli and Alleppey backwaters) through my camera: http://www.flickr.com/photos/57078108@N00/.
Tuesday, October 30, 2007
Computer Vision (36): Mechanical or Knowledge based CORRESPONDENCE
In spite of expending sleepless nights giving deep thoughts on what could be the technique behind our brain solving the problem of depth perception, my brain only gave me a drowsier day ahead. So I started to filter out the possibilities to narrow down to the solution. The question I asked to myself was; is our brain using knowledge to correspond the left and the right images, or is it something that happens more mechanically? I had tried out a lot of knowledge based approaches, but only in vain and even the discussion that we had in the earlier post concluded to nothing. I wanted to take a different route by thinking of a more mechanical and less of a knowledge based approach. My brain then pointed me to the age old theory proposed by Thomas Young to explain the wave nature (interference) of light, “The double Slit Experiment”. How could this be of use to solve a seemingly unrelated problem of depth perception? On comparing you will find a few things in common between the two setups. Both are trying to deal with light and both of them are trying to pass the surrounding light through two openings and combine them later. I excitedly thought, have I unlocked the puzzle?
Let’s analyze and understand it better to know if I really did! I am neither an expert in physics nor biology, so I can only build a wrapper around this concept and not verify its complete path.
λ is the wavelength of the light
s is the separation of the (slits/eyes)
x is the distance between the bands of light (also called fringe distance)
D is the distance from the (slits to the screen/eye and retina)
As the source starts to move away from this bisecting line the symmetry in the pattern should start to degrade.
If a light source is placed at 3 different locations equidistant from the center of the slits, the one at red would produce a symmetric pattern and the other two I guess would not. I have not experimented this and hence the letters NX (Not eXperimented). If my guess is right, a light source placed anywhere in the 3D space would produce a unique pattern on the screen!!! This means an analysis of this pattern would tell us the exact location of the source in the 3D space.
Tuesday, October 16, 2007
Photography and Travel: Kudremukha
Wednesday, October 3, 2007
Computer Vision (35): Segmentation Verses Stereo Correspondence
Hope it doesn’t get difficult for your brain at least to get the contents in the image. On observing keenly, it shows a dog testing its olfactory system to find something good for its stomach. You can almost recognize the dog as a Dalmatian. Now I bet if anyone can get me a generalized segmentation algorithm that can extract the dog from this image!!!Some people might argue that it’s almost impossible to achieve this from a 2D image, since there is no way to distinguish the plain of the dog from that of the ground. Remember, your brain has already done it! In a real scenario even if we come across such a view our stereo vision would ensure that the dog forms an image separate to the plain of the ground and hence would get segmented due to the variation in depth. Our brain can still do it from the 2D image here due to the tremendous amount of knowledge it has gathered over the years. In short, stereo vision helped us build this knowledge over the years and this knowledge is now helping us to segment objects even from a 2D image. The BIG question is, how do we do it in a computer?
Whenever I start to think about a solution to the problem of stereo correspondence the problem of segmentation would barricade it. This is why. The first step to understand or solve for stereo correspondence is to experiment with two cameras taking images at an offset. Below is a sample image.
It is very obvious that we cannot correspond the images pixel by pixel. Which blue pixel of the sky in the left image would you use to pair with a particular blue pixel in the right? Some pixels in the right would not correspond with the left and vice versa, but how do you know where these pixels are? This again loops us back to use some segmentation techniques to match similar objects in the two images, but I think we had just now concluded that segmentation was due to stereo!!!Thursday, September 20, 2007
Computer Vision and Photography (34): The Focus Story Continues…
But any point on the lens would receive light from all points visible around it. So at any point on the lens light rays will be converging from every possible angle, which leaves us with no way to pinpoint the ray that started from the optical axis.There are many more problems with this very way of thinking to solve the problem. From the perspective of the lens we never know where the real point is located on the optical axis. Different points from the surrounding space can create the same effect as though there was a real point at a different location on the optical axis. This indeed can happen continuously all along the axis! Assuming that the frequency of light reflected from a real point will almost be the same when it meets the circle and the probability of such a thing happening for a virtual point zero, the problem could be solved. But if you recall, the very reason why I started to think about this, was to get a solution to cases where there is zero contrast.
After a while I came across a theory called QED that solved a lot of these problems but kept the hardware required to achieve it out of our current technology’s reach. According to QED, a photon represents the “particle” of light, and its instantaneous phase the “wave” counterpart. This phase depends on the frequency of the light under consideration. A lens focuses light because the probability that the photons reach the focus point with the same phase is high and zero anywhere else. For more details refer to the book “QED: The Strange Theory of Light and Matter”. Putting the same theory into action for our current scenario, this would hold good only for a real particle. Since phase is something that repeats as the photon travels through space, the random points that form the virtual particle should be present at exact locations (again that can repeat in space) to meet the point “a”, all with the same phase!, which is highly improbable in a practical scenario. Now this should work for ZERO contrast!Saturday, August 25, 2007
Computer Vision and Photography (33): Capturing stereo images using a single camera
Chak De
It is simple (figure only for conceptual understanding). Like our eyes, I keep two plane glasses at an offset and direct light to the lens of my camera through a pair of pair of 45 degree mirrors as shown in the figure. So in a single sensor I capture both the images, each on one half of it.Given a camera and a requirement to take stereo images, this arrangement was anyone’s mind game. I thought simple things like these need not be documented. A few months after this finding I saw a paper on exactly the same concept from a university in Switzerland. Do we really need PhDs to build this school level optics? Now I know the reason behind the very poor numbers behind India’s contribution towards the world’s papers and patents. Are we too fainéant to put our findings on paper or do we think we are not up to the mark in creating new things when compared to others? That too when others are confident of such simple findings.
CHAK DE
Saturday, August 11, 2007
Computer Vision (32): Monocular Cues
Taking any one of the cars in this picture as reference we can very easily guess the relative position of the others in the image. This cue is called “Familiar size”. It works not only for similar objects but anything around you. Where this cue takes a beating sometimes there is another that drops in to resolve this issue; “Interposition”. On the left we have a red and a silver car projecting the same size even though both are not at the same depth. How do I know? My brain tells me that some portion of the red car is occluded by the silver one which means that the latter should be in front of it.
Bringing in the rest of the image you can see that the road in the above figure appears to get narrower farther it is considered from the camera. Taking this cue as reference you can almost separate the different regions in this image into their depth categories. The small hilly region on the right is farther away from the lake on the left. The fountain is definitely closer to the camera than the lake, etc. This is called “Linear Perspective”; the convergence of parallel lines as they move away from you.All these cues supplemented with our knowledge will always give us if not accurate a misty information about depth even in a 2D scenario.
Thursday, July 26, 2007
Ideas and Technology: More Intelligent Alarms
Sunday, July 22, 2007
Computer Vision (31): "Seeing" through ears
Till a few days back even I wasn’t aware of the existence of such creatures in Nature. I had not even thought of trying out something like this, even though it has been years getting into researching in this field. Nature again outwitted us in its design and complexity. I am actually talking about creatures having ears at a vertical offset to extract yet another dimension; depth that our ears/brain fail to solve through hearing. The Great Horned Owl (Bubo Virginianus), the Barn Owl (Tyto Alba) and the Barred Owl are some of such Nature’s selected gifted creatures. This offset helps them to hone on a creature with more sensitivity and helps them hunt down creatures even in complete darkness. With this ability they don’t even spare creatures like mice that usually hide under snow and manage to escape from their sight. Evolution has created wonders in Nature. These predators usually live in regions with long and dark winters and hence have developed the ability to “see” through their ears.
But how does it all work? With just horizontal offset our ears manage to tell us the direction of sound in the 3D space. Imagine it to be an arrow being hit in that particular direction. You don’t know the distance of the target but just fire it in that direction. The arrow actually leaves from a point which is the horizontal bisector of your ears. Applying the same concept on vertical offset there will be another arrow leaving from a point which is the vertical bisector of the ears (in the case of these specially gifted creatures). From primary school mathematics we all know that two straight non parallel lines can only meet at one point in space, which in this case happens to be the target.Even Nature can only produce best designs and not perfect ones and the Owls will definitely have to starve if their prey manages to remain silent. To make its design more reliable and worthy, Nature has never allowed a prey to have this very thought in its mind.
Saturday, July 21, 2007
Computer Vision (30): Why wasn't our face designed like this?
2. http://puneethbc.blogspot.com/2007/04/computer-vision-13.html







