« first day (72 days earlier)      last day (3173 days later) » 

2:31 AM
@rayryeng hey ray? When you come back the next day, want to exchange e-mails?
 
2:48 AM
@OneRaynyDay - ray@bublcam.com
 
thanks!
 
That's fine. My e-mail address is on my profile page.
I don't mind anyone who has it
 
oh, gotcha! I sent you an email
 
sounds good!
 
oh wait, just searched up bublcam - did you make the camera? :D
 
2:55 AM
no, but I'm the lead software engineer for the company.
 
wow - impressive :) I'd imagine the camera itself would be very difficult to make
I'm currently working at an internship for a camera chip startup
focused towards asian markets.
I have no idea what they're doing in the hardware side, and the only software developer on the team is a guy who writes C code to put onto the chip.
making a python GUI for displaying Gain vs. various factors of 4 channels & avg
 
oh :D lol
yeah I work in middleware. I have experience in hardware, but not as much as I'd like.
I'm right in the middle between the hardware where the do the image acquisition and the front-end where you actually see the output.
I focus on taking the fisheye lenses and to stitch them all together so that it appears 360 degrees.
I also focus on improving image stitching, image quality and other R & D.
 
yeah, that sounds really cool actually
but I'd imagine it's a TON of math right? haha
All I know is calculus... not even close to learning that stuff yet :'( so sad
 
3:12 AM
ah it's most linear algebra with a bit of calculus
Linear algebra because of the camera model we use... as well as solving some sparse linear system
systems
Calculus for optimizing cost functions.
 
3:25 AM
ahh right, I read up a bit on gradient descent :)
sparse linear system? Why is it sparse if every pixel has a color?
 
 
3 hours later…
6:17 AM
@rayryeng hi
 
@OneRaynyDay - Mostly for solving the alignment of two cameras... making sure that their camera parameters are optimized so that the features detected when calibrating the cameras align.
That thinking isn't currently being used in anything right now.... the pixel thinking
@legends2k - hi
 
I read your answer on machine epsilon a couple of days ago, I've some doubts on it
 
Can you ask me tomorrow? I'm about to go to bed.
it's 0219 here.
I have to work tomorrow :)
 
sure, no problem
 
ahh okay.. yeah, as you can see I have literally no idea about image processing ^_^
Alrighty, goodnight ray!
 
6:20 AM
good night!
will catch you here tomorrow
 
no worries! If you decide to go down this route, you'll get there.
 
awesome, that's reassuring to hear
 
actually, I can stay up for a bit. I have to run a couple of simulations.
what doubts do you have?
 
anyways, 2 AM? So late haha
 
eh I'm used to it
 
6:21 AM
I wrote a minifloat to understand IEEE 754s binary32 better
 
ah, coffe in the mornin I'm guessing
 
ah nah. I don't drink coffee. More of a tea person.
@legends2k - I'll try and answer your questions. From the looks of it, I may not be able to answer you
 
Isn't floating-point format a trade-off between range and resolution? Why do people call it range vs precision? Isn't precision just the number of significant digits? Moving the dot left or right shouldn't've any bearing on the precision since the mantissa bits are still the same, it's the resolution that becomes lesser, as the number becomes larger.
 
@legends2k I'm afraid I can't answer you.
 
:(
 
6:24 AM
That's a philosophical discussion that I don't have any knowledge in.
 
okay
 
sorry!
 
no issues
 
@OneRaynyDay - Imagine four cameras being oriented in a sphere. The point is to find the right camera parameters... so in our case, it's a rotation matrix and translation vector...
so that given a reference position, you figure out the right parameters to rotate and translate this reference position so that you're with respect to a camera you're looking at.
 
Hmmm.. I kind of understand
 
6:27 AM
The goal is to figure out where the cameras are oriented first.
 
do the four cameras overlap in image at all?
 
yes.
 
so you have to prune that part out?
 
they're fisheye lenses, so at least two overlap with each other.
there are no blind spots.
so let's say you choose one camera as the reference position.
the goal is to figure out the right rotation and translation parameters to move from the reference camera to another one.
once you rotate and you're viewing what the other camera sees, you have a visible field of view that sees a particular region of the world.
this field of view gets mapped back onto a 360 degree image known as an equirectangular image
Let me go find one as an example
With a sphere, there are two ways you can rotate around it.... longitudinal and latitudinal
 
The little I recall from the DSP class I audited at some point, is that a floating point representation gives you a larger dynamic range (i.e. log(max_num) - log(min num)) at the cost of not allowing you to represent all values. I remember a chart showing the density of numbers you can represent with fl. pt. which made things clear
 
6:31 AM
 
OH. This is in terms of quaternion rotation right?
 
longitudinal is lambda - latitudinal is phi
yeah so look at the equirectangular... and compare that with the sphere
 
I remember seeing this in spherical coordinate systems
 
the vertical axis in the equirectangular define the latitudinal angles that go from 0 to 180 degrees
the horizontal axis of the equirectangular map the longitudinal so from 0 to 360
each point in the equirectangular give you a unique point in the sphere to look at... assuming a unit sphere.
so think of an equirectangular as a sphere that is unfolded and placed on a piece of paper
make a slit and place it flat onto a surface.
 
oh gosh, give me one second
Trying to take this in
also my little kitty's being very disobedient haha
 
6:34 AM
so basically how the camera works is that each fisheye lens is oriented in the sphere and pointing out somewhere. This camera has a field of view and captures part of the sphere.
This part of the sphere gets projected onto the equirectangular... and now there are 4 fisheye lenses, each with equirectangular images.
so what I do is blend all of them in such a way that they become one final equirectangular that is seamless.
@legends2k - yes! That's right.
 
ohhhh I see I see
 
@legends2k - did you see what I wrote about the floating pt. representation?
 
I kind of understand now... So basically the parts of the more extreme angles
as ø becomes 90 or 270, the projection becomes wider?
 
@Dev-iL missed it then, just saw it
 
and so the slits are cut to make those places
 
6:36 AM
@Dev-iL yes, the larger the number becomes, the lesser the granularity with which you could represent numbers
 
where the ø becomes extreme(in the corners of the cameras), the slit spaces are larger
 
Yup.
the distortion gets bad at the north and south poles.
which makes sense.
 
Also the smaller...
 
so the reason why equirectangular is so important is because this is the current standard that all 360 degree players use.
 
ahh I see
 
6:37 AM
YouTube 360 accepts equirectangular.
 
so you have 4 cameras pointing NWSE?
 
all of the 360 cameras pipe out this format... Ricoh Theta, Kodak, etc.
 
And I see, yeah I've seen 360's that have this kind of format
 
@Dev-iL yes, see the output of the minifloat to see this
 
one is pointing upwards, the other three is pointing 120 degrees at the horizon.
that's the camera deconstructed.
soooo... yeah alot to take in lol.
 
6:39 AM
ah so it's all 120 degrees apart from each other in 3d space
 
Yup, with one looking upwards.
 
okay yeah haha I understand somewhat, except I have to initially deconstruct everything you say as if I'm 5 years old haha
 
the others that are separated at 120 degree separation have their latitudinal angles at roughly 107 degrees using the positive z-axis as the 0 degree inclination.
 
but I understand the idea :)
 
so they're pointing slightly downwards
so basically, we have four fisheye lenses... what we need to do is figure out where exactly each fisheye lens image gets mapped onto the equirectangular
 
6:41 AM
ahh I see, yeah I remember in chemistry that magical 107 degrees
 
there are four equirects that are generated per camera.... you blend them in such a way hat the transition is seamless.
That's the goal of calibration. What you do is take an image with each of these lenses... then detect strong features.
 
@rayryeng you know too much...
 
strong features? What do you mean by that?
@Dev-iL I second this
 
in image processing, this is what is called feature detection.
or interest point detection
 
ah, like face detection? or like areas of high contrast?
 
6:43 AM
interest points are points in an image that are repeatable and worth remembering.
yeah exactly.
edges, corners.
so first in each fisheye lens image, you figure out what the strongest interest points are.
and because there is overlap between one camera and another... there's gonna be interest points that are also visible between the two cameras.
so what you do is figure out the right rotation and translation parameters that rotate one camera with respect to another.
using these feature points as a guideline.
 
oh my gosh, so your mapping isn't always that amount of distortion to make sure the least amount of feature points are within the distortion range?
 
once you figure out these rotation parameters, this allows you to figure out where in the equirectangular you need to project to
Exactly.
 
Geez that's hard.
 
The rotation parameters are created such that the amount of error to rotate from one camera position to another, using the interest points that are common between them is minimized.
this boils down to solving a sparse linear system :)
So the camera we create... has to go through a calibration stage.
You place it in a room with a lot of activity... so that it's bound to find a lot of interest points.
you take an image so that four fisheye lens images are captured...
 
So do you use gradual change in the rotation parameters like a gradient descent way, or like calculate the answer immediately at once and rotate to that point?
 
6:46 AM
then using one camera as a reference, you look at the other three cameras and figure out what interest points are common between the reference and the other cameras.
 
I'm guessing solving the linear systemw ould give you the answer immediately though
 
Guys, does any of you want an invite for OnePlus Two? I just got it and I don't want it..
 
you figure out the right rotation and translation parameters to figure out how they're oriented in the sphere.
which then gives us where to write our image pixels into their respective equirectangulars.
you pipe out four equirectangulars... then my job after that is to merge all of them together so that the image is seamless.
I do it iteratively via non-linear least squares.
The method is called Levenberg-Marquadt
In mathematics and computing, the Levenberg–Marquardt algorithm (LMA), also known as the damped least-squares (DLS) method, is used to solve non-linear least squares problems. These minimization problems arise especially in least squares curve fitting. The LMA is used in many software applications for solving generic curve-fitting problems. However, as for many fitting algorithms, the LMA finds only a local minimum, which is not necessarily the global minimum. The LMA interpolates between the Gauss–Newton algorithm (GNA) and the method of gradient descent. The LMA is more robust than the GNA, which...
@Dev-iL - What is that!?
So... if I had to summarize... at a very high level...
 
so you're using regression to find the least amount of error bound(error bound being the interest points) or at least for a local minimum
 
Correct.
 
6:50 AM
and then you take the pixels
that aren't yet distorted
and apply the distortion, where the sparsity exists the most is where it distorts the most...?
I'm just dabbling in the dark here haha
 
(1) Take an image with lots of activity
(2) Find interest points for each fisheye lens
(3) Figure out the right camera parameters that rotate from one reference camera to another using (2)
(4) Take the image pixels for each fisheye lens and project them onto four equirectangulars - one per camera
(5) Blend the four together to make one final equirectangular.
 
and oneplus two is a phone i thiink, you need invites in order to buy it
 
OH. YEAH sure!
@OneRaynyDay - No regression. I'm minimizing a cost function where the error between rotating one camera to another's coordinate system is small
 
ahh okay gotchaa :) I think I understand it now!
oh, it says it's in a regression analysis section in the wiki so I was a bit confused
 
oh lol
You can use LM for regression for sure.
 
6:52 AM
morning
 
but I use LM for minimizing a cost function.
@OneRaynyDay - I'd like to show you a sample...of an equirectangular and how it's visualized in a sphere
and an example of our fisheye lens images
 
oh awesome please do!
I promise I'll go through it with a lot of scrutiny, I just have to take out my contacts real quick, it's getting to midnight for me haha
 
oh you don't have to lol
You seem interested with what I do so I'm sharing. :)
I think showing you images will give you a better perspective of what's going on
@OneRaynyDay - Here's an example of an image that comes straight off of our camera
 
yeah I'm really interested! I'm all ears dude :)
 
There are four fisheye lenses. The bottom right image points to the top
the other three are separated by 120 degrees.
this is what we call a multiplex format because all four images are squished together in a single 2 x 2 grid
I'm the guy in the top right image in the middle making that ridiculous face
So the goal is using one camera as a reference, you find interest points that are common between the reference camera and a target camera
you figure out the right rotation and translation parameters so that you can figure out where in the sphere the target camera is pointing to
Once you figure this out, you have a visible field of view for each camera and you know what region in the equirect this camera maps to
 

« first day (72 days earlier)      last day (3173 days later) »