Why can’t universities fund creating teaching material of this caliber?

I’m always impressed by the amount of work and quality this guy put on his posts. Especially the interactive widgets

I obviously haven&#x27;t finished reading the entire post, but I wanted to simply appreciate the hard work that has gone into this. I absolutely love interactive articles. Great job!

The first blog post of his that I read was Lights and Shadows (<a href="https:&#x2F;&#x2F;ciechanow.ski&#x2F;lights-and-shadows&#x2F;" rel="nofollow">https:&#x2F;&#x2F;ciechanow.ski&#x2F;lights-and-shadows&#x2F;</a>), and wow his blog is impressive. I love it!

Another nitpick is that the sliders are all unlabelled and rely only on their colours matching the text. I found myself having to go back and forth between the text and the sliders, and I imagine it must be worse for colour blind people.

Agree it is imperfect but there is too much Internet, too little time in life, therefore fast heuristics and patterns are useful :)

Maybe this should make you think about your &quot;worthiness&quot; scan ;)

Impressive work! Nit on design: in first ~3 attempts to scan through the page to see if it is worth reading, i found myself stopping very early when the black background started given it usually signals a footer and end of useful content. The only reason I ended up discovering there is more content was the sanity check that if it is HN front news there has to be more to it :)

And the articles renders so fast and so smooth on my laptop. In fact they render more smoothly and loads faster than 99% of all much much less technically advanced articles out there. Wow!

Added the blog to my Thunderbird RSS-Feeds :-)

I was wondering why it was lighting up as visited link. Turns out it’s same blog that posted few years ago something so special it took me entire Christmas to read!

I also like his floating point numbers&#x27; explorer
<a href="https:&#x2F;&#x2F;float.exposed&#x2F;0x3fb999999999999a" rel="nofollow">https:&#x2F;&#x2F;float.exposed&#x2F;0x3fb999999999999a</a>

The final words on the Tesseract page are lovely!&gt; I find it very inspiring that while we can’t physically experience a four dimensional space, with just a bit of ingenuity we can easily simulate how a tesseract and its shadow would look in our day-to-day world.&gt; You may find math’s indifference to the limitations of our human perception quite cruel, but I think it’s liberating. Reflecting on higher dimensions is transcendent – it removes the shackles of the physical world and allows us to explore the realms we’ll never encounter.

Regarding the lights and shadows articles, how they hell are they getting raytraced-quality lighting, with no noise, with changes to the scene in realtime?? Are these all pre-rendered PNGs?

You&#x27;d like their other work :<a href="https:&#x2F;&#x2F;ciechanow.ski&#x2F;lights-and-shadows&#x2F;" rel="nofollow">https:&#x2F;&#x2F;ciechanow.ski&#x2F;lights-and-shadows&#x2F;</a><a href="https:&#x2F;&#x2F;ciechanow.ski&#x2F;gears&#x2F;" rel="nofollow">https:&#x2F;&#x2F;ciechanow.ski&#x2F;gears&#x2F;</a><a href="https:&#x2F;&#x2F;ciechanow.ski&#x2F;tesseract&#x2F;" rel="nofollow">https:&#x2F;&#x2F;ciechanow.ski&#x2F;tesseract&#x2F;</a>Etc.Check out the whole blog! It&#x27;s amazing.These are some of the most intuitive explanations coupled with the slickest animation &#x2F; demo work I&#x27;ve ever seen. All of that put together into crystal clear educational material is such a rare gem and requires incredible talent.Impressive as hell.I hope someone is paying them to do this full time. I&#x27;d pay for this.

I wouldn&#x27;t say it&#x27;s basic at all. It&#x27;s an excellent distillation of the physics of cameras.My one minor nit-pick would be introducing the wave nature of light so early--to motivate Snell&#x27;s law and total internal reflection, but without discussing diffraction until well later and only in passing.Snell&#x27;s law can be motivated from a purely classical particle-based model as:1. Light travels slower through media other than vacuum (This is mentioned, but hand-waved as boundary conditions, which probably isn&#x27;t meaningful to the intended audience).
2. Light takes the shortest (in time) path between two points. (Light... finds a way).Either way, the audience might find the obvious &quot;why&quot; for the first point more satisfactorily addressed with a short sidebar on permittivity and permeability of materials (their electrical properties, effectively the capacitance and inductance &quot;density&quot;).That can also lead to an interesting discussion of why conductors are usually not optically transparent and insulators are.

I agree, it&#x27;s pretty basic but for people that don&#x27;t know any of this it&#x27;s an excellent explanation.

+100 because I came here to post exactly what you did. I just bookmarked this to share with my tween who&#x27;s getting interested in photography and starting to ask things like &quot;Why that camera?&quot; and &quot;Why this lens?&quot;

You&#x27;re right. Similarly, I cannot express how good this site really is in just a few words. Having been an enthusiastic amateur photographer for decades as well as having worked in electronic imaging for about as long, this is one of the best practical illustrations of the subject that I&#x27;ve come across.I&#x27;d highly recommend it to anyone who is seriously interested in the underlying &#x27;mechanics&#x27; of imaging.(Incidentally, anyone whose has read my many anti-JavaScript raves, then I&#x27;d only say this is exactly how JavaScript ought to be used.)

Although I knew most of technical aspect described having immersed myself in Photography several years ago, I cannot express properly how GOOD this article is. The sliders and diagrams are brilliant for helping a layperson understand how it all works. Kudos!

Very impressive work, the interactive examples are excellent and will help many understand the concepts.I’m a novice when it comes to anything related to web development so I’m going to go and look at how this type of this is done. Would anyone have and suggestion or recommendations for getting started?I teach fluid mechanics with Jupyter and I use pyGEL3D for inserting 3D models but it’s far from ideal. I’ve love to do something like this.

This is beautiful.I&#x27;ve been working with computer vision since university, but now days work underwater. I guess I should have a pretty good understanding of this stuff, but I still occasionally get caught out by optics.This is the best possible pre-reading for anyone wanting to jump into the field.

It&#x27;s changing the focal length of the lens (roughly the distance between focus points of the lens). If you rotate the scene, you see the lens is blue. Further in the article, they mention the focal length, using the same coloring.

It changes the focal length of the lens, although it does not seem to affect the lens (blue ring) in the left-hand pane which seems like a bug.There is a demonstration later in the article (below the 5th mention of &quot;focal length&quot;) which does affect the lens (blue ring) in the left-hand pane. Rotate the scene so that you are viewing the camera side-on, and you will see the focal length of the lens changing when you move the slider.

This is really interesting. In the first picture what does the blue slider do?

To clarify, the alternative to taking control of processing scene-referred data is to use camera’s JPEG rendering of its own idea of how the scene should look.

I think there is a fundamental difference between editor (designed to produce a deliverable) and viewer (designed for immediate experience) software. One of the things essential to the latter but really to the former is immediacy, hence I suspect that the computing powers commonly available today make it impossible for now (but likely not for much longer).

If I understand correctly, you&#x27;re looking for a simplified RAW image editor? Many digital cameras allow storing RAW images alongside JPEG. The viewer can then load the RAW images into any (web-based) image viewer&#x2F;editor that supports RAW format and have full control over tone mapping.The tool interface needs to be simplified to make it a better fit for the use-case you present but I don&#x27;t see computing power as a bottleneck.Of course they&#x27;d still be limited by the dynamic range of the camera. This can also be resolved by calculating irradiance map based on multiple RAW images taken with different exposure times.

Lately an area of interest of mine in digital photography in particular is scene-referred vs. output-referred image formats and subjective perception. The author (justifiably) walks around this topic in his camera sensor overview, but the data captured by a sensor from photons hitting it doesn’t really contain a readily viewable image.Data values captured by a modern camera far exceed the ranges that can be reproduced by output media such as paper or screens—meaning that the only way[0] to obtain an image we can communicate to someone else (or future ourselves) as a static artifact[0] is by throwing out data and conforming values to ranges that fit in the output color space, converting scene-referred (“raw”) data to output-referred.This is where subjective perception comes in: how we perceive colors and shapes in a given scene depends a lot on what we had seen prior to this scene, our general mood, and various other aspects of our mind state. It’s only by taking control of processing scene-referred data that we can use the full range of captured values to try to convey most convincingly, within the constraints of the output space, our subjective perception of the scene around the time we pressed the shutter trigger.(Naturally, further down this rabbit hole come the questions about e.g. what our conscious perception—but not the camera—was blind to in the scene, and eventually about the nature of reality itself, at which point one may feel compelled to give up and go into painting instead.)[0] This would be quite niche, but I wonder if we could develop tools for exploring raw data at viewing stage, allowing the audience to effortlessly adjust their perception of the scene (even if within ranges specified by the photographer). Such exploration would require significant computing powers, but we’ll probably be there in a couple of years.

Does Bartosz have a Patreon or something? I feel like I&#x27;m deeply indebted to him for the privilege of having read this work. Really, one of the most comprehensive and fantastic explanatory pieces I&#x27;ve ever read.I cannot imagine the effort that goes into writing and coding a post like this.

I can only echo my impressed-ness. With or without an interest in the obvious topic, it is well written, well illustrated, accurate. A superb example of the internet at its best.

I built a trinocular microscope mount for my DSLR camera out of a couple of small magnifying lenses. When I was trying to get it to work and improve the quality, I did a bunch of reading on optics. Building your own camera lens that works is pretty easy, but making it capture nice pictures is hard.Having this article back then would have been a big help! The interactive widgets are great, and it looks like they&#x27;re entirely home-grown. I am really impressed with the level of detail and work that went into this post.

Amazing, just amazing. Thanks alot. I would only suggest author to set og:image in your pages. When you share it on social media no image appears, you know how does it affect spreading, right? :)

Great work, we live in amazing times! I can&#x27;t imagine how humanity will evolve from now on if knowledge is as available and as well and accessibly presented as it is these days!

What an incredible piece of work. I can only dream of having the patience, skill and drive to make something so beautiful and informative.

I think the question is how is this drawn on a canvas. It&#x27;s not trivial, if I draw 6 shapes successively, one will be on top and another at the bottom, you would not see the tip of one at the top and the other part of the same blade at the bottom. Nicely spotted, interesting question!

Not sure I get your question, you want to know how it works in real life ? Have a look here: <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=cR7cibDvYyo" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=cR7cibDvYyo</a>

Looks like so, with a clever rotation, clipping, and drawing twice (had a cursory look at &#x27;lenses.js&#x27;, look for &#x27;draw_blade&#x27;)

Do you mean, how it was drawn using canvas?I&#x27;m guessing they drew overlapped part of the last blade twice

This is interesting. I don&#x27;t know much about html canvas, but it looks like this is handled in the author&#x27;s custom js [0] in the line that starts with: &quot;} else if (mode === &quot;blades&quot;) {&quot;[0] <a href="https:&#x2F;&#x2F;ciechanow.ski&#x2F;js&#x2F;lenses.js" rel="nofollow">https:&#x2F;&#x2F;ciechanow.ski&#x2F;js&#x2F;lenses.js</a>

It may not seem interesting at first, but I am intrigued by how they managed to draw the overlapping blades of the IRIS.&gt; In real camera lenses an adjustable aperture is often constructed from a set of overlapping blades that constitute an iris.If you naively create 6 blades and rotate them around the pivot (the tiny screw), they will never overlap EACH OTHER. One will end up at the very top, and one at the very bottom.If you know how this was created, please let me know.

It&#x27;s truly amazing to me how someone can be this talented. The text, the animations, the style. It&#x27;s all high quality.I&#x27;m in awe.

I&#x27;ve seen a few of these kinds of interactive articles on hackernews but this is on a whole new level. Wow. One of the best explanations of just about anything I have ever seen.

It&#x27;s really fascinating - I had always assumed digital cameras had different sensors for different light colours like eyes do.

Awesome article and blog, as an amateur photographer this is highly educational and fun to read!

I picked up a Fuji last Christmas, and it’s one of the best purchases I’ve made in the last decade. Their X-line takes incredible pictures, and is so approachable. If anyone has been considering more serious photography, or is having a child and wants to capture more unique shots of them, I can’t recommend them highly enough.

Which is still pretty much a Bayer sensor, just with a different filter layout. You still have to demosaic to get a usable file. And a well-tuned RAW convertor to do so.

Fuji also use what they call an X-Trans sensor.<a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Fujifilm_X-Trans_sensor" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Fujifilm_X-Trans_sensor</a>

That debayering sensor mod looks really interesting :) Nice way to quadruple the effective resolution of your camera, if you don&#x27;t care about colors.

In Rawtherappee you can choose which demosaicing algorithm to use.<a href="http:&#x2F;&#x2F;rawpedia.rawtherapee.com&#x2F;Demosaicing" rel="nofollow">http:&#x2F;&#x2F;rawpedia.rawtherapee.com&#x2F;Demosaicing</a>

While the mod is interesting and a way to get into monochrome photography, it should be noted that the mod has disadvantages. Mainly, you end up removing the microlenses. Which seems to cancel out any gains unfortunately.source: I&#x27;ve done this mod on a couple cameras and wouldn&#x27;t recommend it for performance gains alone. It&#x27;s a fun project, not much more.

Bryce Bayer was my father. Hopefully I have a few more decades before I&#x27;m &quot;debayered&quot;.

FYI there are digital sensors which aren&#x27;t using bayer filters. Some are classic sensors missing the bayer filter (leica monochrome for example) other are based on different tech like Foveon sensors [0]Many people also debayer their cameras themselves, especially in astro photography [1][0] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Foveon_X3_sensor" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Foveon_X3_sensor</a>[1] <a href="https:&#x2F;&#x2F;stargazerslounge.com&#x2F;topic&#x2F;166334-debayering-a-dslrs-bayer-matrix&#x2F;" rel="nofollow">https:&#x2F;&#x2F;stargazerslounge.com&#x2F;topic&#x2F;166334-debayering-a-dslrs...</a>

It is just the same with your eye - and camera response curves are optimized to be similar to your eye.
There is metamerism - many spectral distributions will look the same to your eye as well as to a camera. See <a href="https:&#x2F;&#x2F;en.m.wikipedia.org&#x2F;wiki&#x2F;Metamerism_(color)" rel="nofollow">https:&#x2F;&#x2F;en.m.wikipedia.org&#x2F;wiki&#x2F;Metamerism_(color)</a>

Thank you for your great explanation. The overlap was the missing piece in my mental model ...

&gt; The wavelength of the light cannot be measured directly, only inferred by the intensity of the pixels with the different color filters.Well, depending how you think &quot;directly&quot; but you can get pretty far with spectrometer, i.e. device that splits light and measures the intensity spatially to collect a spectrum. It&#x27;s not impossible thought to build camera based on that principle, just need to sample the light in an array to make pixels

You are right, this is a challenge. The wavelength of the light cannot be measured directly, only inferred by the intensity of the pixels with the different color filters. On the other side, most reproductions of photos are reproducing the original frequencies either. A computer screen has red, green and blue dots, which produce light at the corresponding wave lenghts. So if you have orange light, you have a signal on the green and red pixels and the green and red dots on your screen will light up. Which will be detected by the sensors for red and green light in your eyes. No where in the chain, not even in your eye, is a sensor for &quot;orange&quot; directly, it is just the mixture of the red and green sensitivity.It is important to note, that neither the sensor pixels nor your eyes have a complete separate reaction to a wavelength. The sensitive area strongly overlaps. So for hues of green which have rather long wavelengths, you get some reaction on the red pixels, which gets stronger as you move towards orange, where both red and green pixels detect until it gets more and more red and less green. The exact absorption curve of the sensore color filters matters here, that is one reason, different manufacturers have slightly different color rendition. On top of that is calibration, when converting the raw image into a proper RGB-image, one can further balance the response. For that, the color calibration targets are used, which have like 24 patches of different colors. Taking a phone of this target, the calibration software can both calibrate for the light illuminating the target as well as the color response of your camera.A common reason for red-green colorblindness is that the affected persons have the sensitivity between the red and green colors overlapping too strongly, so they loose the ability to differentiate. A green creates almost as strong a signal in the &quot;red&quot; cells. A way to improve the color vision for those people are glasses which increase that separation by absorbing the frequencies between the red and green colors.

That is correct. The sensitivity of the &quot;red&quot; and &quot;green&quot; pixels overlaps in the orange light frequencies.

That is in the rgb color model. In physical world, there are (almost) infinitely many different spectra that could be perceived as same &quot;orange&quot;. It is honestly pretty amazing how well human color vision works despite that

Not an expert but: orange color code is FFA500.
It has red and green. I expect when orange hits the sensor, it will register as X amount of read and less than X green?

They work exactly the same as a screen. Orange is not pure red, it is red + yellow, and yellow is red + green. So orange is red + _some_ green. The in-camera processing will render the image based on the sensor input and the known properties of the filter (think of it as a color profile, a mapping between what the sensors read and the color). And the processing includes color interpolation for each pixel, as each pixel (photosite) only has one color filter, but the resulting image pixel has all three colors; these are calculated based on the neighbor photosites.Different sensors&#x2F;cameras have different filters, and combined with the manufacturer specific post-processing this gives different cameras&#x2F;manufacturers a different color rendition and feel.

&gt;&gt; So e.g. when the light is pure orange, it falls down into red wavelength range and is counted as red&#x27;Pure&#x27; orange light isn&#x27;t red. It has a wavelength of 590–620 nm. Red is 625–740 nm and Green is 495–570, so &#x27;orange&#x27; is between red and green. The sensor filters each allow a range of wavelengths through. So green is triggered as well as the red filter. In RGB terms orange is 255, 127, 0, i.e. with a strong red component and a smaller green component.White balance is computed downstream from the sensor, and is used to resolve the effect that a coloured light source creates a colour cast on objects, most noticeably on white ones. The human visual system auto-compensates for this but cameras require special processing, sometimes done using presets for different types of light (sunlight, shade, tungsten etc).

Filters allow rather large families of wavelengths through. You don&#x27;t have just red, you have a fraction of green. Because your eyes only have three types of photoreceptors they can be fooled by playing back a similarly large family of wavelengths back.

It is because our eyes are also &quot;incorrect&quot;. The cameras and processing and displays were engineered to match our eyes expectation. If some alien race with a sufficiently different physiology would look at our pictures they could tell that it is off. (and it wouldn&#x27;t even mean that their eyes are more correct, just that their model is different.)

It works because the system is faulty in the same way our eyes are faulty, we basically have sensors for three main wavelengths and derive every color we perceive from a mix of those three, 
check this out:<a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Color_vision#Physiology_of_color_perception" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Color_vision#Physiology_of_col...</a><a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Trichromacy" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Trichromacy</a>

&gt; So e.g. when the light is pure orange, it falls down into red wavelength range and is counted as red.If you only work with one photon at a time, then yes, you don&#x27;t know if the photon passing through the filter was red, or orange, or with smaller probability even green or blue. But when you have trillions of photons passing through, you can &quot;see&quot; the difference by the relative intensity of light.Remember that at the quantum level, things don&#x27;t happen deterministically; you have to consider the probability that a given outcome occurs. So the photon has a certain probability to hit the filter and get absorbed, a certain probability to pass through the entire depth of the filter, a certain probability to hit the sensor without generating an event, a certain probability to hit the sensor and initiate a chemical reaction (for film or biological eyes) or an electron cascade (for CCD sensors), a certain probability to quantum tunnel to the other side of the universe...So getting back to your question, when pure orange photons hit a red filter, many of them will make it through the filter, but not as many as if they were pure red photons. When pure orange photons hit a green filter, some of them will make it through, but not as many as if they were pure green photons. So if your brain knows what the &quot;white point&quot; of a given environment is, relative to that white color it&#x27;ll see a specific combination of &quot;some red, some green&quot; as orange. (Of course, if known-to-be-white objects are already orange--like when you put on amber ski googles--your brain will eventually adjust and recalibrate to perceive that color as something else... perception is tricky!)

If you tried to turn the sensor data into light again, there would not be enough information to do so accurately. Everything is built around human perception of color. When a photon hits your eye, it produces a &quot;tristimulus value&quot; for your brain. (The tristimulus is produced by &quot;S&quot;, &quot;M&quot;, and &quot;L&quot; sensitive-cones; these roughly correspond to blue, green, and red; that&#x27;s why those colors are what we use for base colors. But, you can use other colors, and you could use more than 3 if you wanted to. There is no law of the universe that splits colors into red&#x2F;green&#x2F;blue parts... that&#x27;s just a quirk of human anatomy.)The goal of a digital camera is to be sensitive to colors in the same way that your eyes are. If that tristimulus can be recorded and played back, your brain won&#x27;t know the difference. The colors your monitor emits when viewing a photograph could be totally unrelated to what was in the original scene, but since your brain is just looking for a tristimulus, as long as that same tristimulus is produced, you won&#x27;t be able to tell the difference.(Fun fact -- there are colors you can see that don&#x27;t correspond to a wavelength of light. There is no single wavelength of light that fully stimulates your S and L cones, but plenty of things are magenta.)TL;DR: computerized color is basically hacking your brain, and it works pretty well!

Your eyes cannot see orange directly, but as an overlap of different intensities of the stimulation of different receptors in the base colors.So as long as the filters in the camera have roughly the same transmission curve as the sensitivity curve of the color receptors, all is good.However, to animals (or hypothetical aliens) with other color receptors, the images produced by photo prints and screens would look quite weird, with colors all wrong.

I have a question for the more knowledgeable:Color filters on the sensor split the light into three wavelength ranges - red, green, blue. Then photosites measure intensity of light, which means that sensor only knows that the incoming photon is &quot;roughly green&quot;, but doesn&#x27;t actually recognize its precise wavelength.So e.g. when the light is pure orange, it falls down into red wavelength range and is counted as red.Based on this I would expect that cameras would often produce pretty incorrect colors, but they are usually pretty good (after correcting for stuff like white balance).

Look into light field photography tech. It is possible to capture a ”volume of light”, within which bokeh &amp; more can be adjusted after the fact. Issue is the amount of data generated and complexity of tech versus getting a ”good enough for most situations” image via simpler means (regular photo). Regular + depth images (Apple LiDAR etc) with help of AI can create something vaguely similar to actual bokeh, but they’re missing a lot of source data.In the world of 3D rendering (content created from scratch) very advanced &amp; realistic bokeh effects are possible, as an example see <a href="http:&#x2F;&#x2F;lentil.xyz" rel="nofollow">http:&#x2F;&#x2F;lentil.xyz</a> for the Arnold renderer.

Not sure but - as film e.g. captures actual photons from the scene, probably some kind of information is encoded through that.Bokeh is a kind of of space representation, similar to how you can basically &quot;see&quot; through hearing a sound stage of instruments separated properly when someone has a really good sound system, or how dogs have &quot;5.1&#x2F;7.1&quot; sense of smell.How does one encode that I have no idea.

If instead of resolving points of light on the image sensor, you use a group of pixels to resolve an entire tiny image you can effectively also see around things. You end up with a picture of many small sections of the large image each at a different angle. The image on the far left of the image would see a different angle than the image on the far right. This is exactly what the Lytro camera did and it&#x27;s why you can take the picture first and focus later. Of course you sacrifice overall imagine resolution quite severely:* <a href="https:&#x2F;&#x2F;www.researchgate.net&#x2F;figure&#x2F;a-b-The-first-and-second-generation-Lytro-cameras-c-The-microlens-array-raw-image_fig9_283646161" rel="nofollow">https:&#x2F;&#x2F;www.researchgate.net&#x2F;figure&#x2F;a-b-The-first-and-second...</a>

A large lens captures information over an area, and so to a certain extent can &quot;see around&quot; out of focus objects. A selective blur of a fully focused scene captured from a single viewpoint (i.e. a small lens) can only approximate this effect, because it simply doesn&#x27;t have access to the same information. Even with a perfect depth map, you still don&#x27;t know what&#x27;s behind occluded objects.

Sidenote, it appears the author works for Apple.

Phone cameras are extremely wide-angle so everything is in focus and there is no natural bokeh. To add bokeh, you have to separate the subject from the background, and then also determine how far different parts of the background are. This requires very advanced AI for non-trivial images (see the imperfections in Photoshop&#x27;s &quot;select subject&quot; tool), which Apple is actually still doing (that&#x27;s what portrait mode is). But if it&#x27;s not perfect, it quickly becomes worthless, so in short, they are doing it, but only the most advanced companies can try.

Here&#x27;s another interesting paper [1] on that topic. It shows that synthetically blurred images are significantly more realistic if they&#x27;re based on recovered radiance maps (HDR).[1] <a href="https:&#x2F;&#x2F;people.eecs.berkeley.edu&#x2F;~malik&#x2F;papers&#x2F;debevec-malik97.pdf" rel="nofollow">https:&#x2F;&#x2F;people.eecs.berkeley.edu&#x2F;~malik&#x2F;papers&#x2F;debevec-malik...</a>

One of the limiting factors for modeling bokeh and flare-like effects is dynamic range limitation. You need extreme HDR capturing to accurately reproduce these effects, as they often play the largest part with bright, especially colored, light sources. I did work on flare simulation and while many effects can be modeled by a rather simple convolution (in the spectral space of course -- you cannot make a rainbow out of RGB straightforwardly), the problem is that kernels (PSFs, point spread functions) for these convolutions have very long tails and it&#x27;s the shape of these tails that gives most of the &#x27;natural&#x27; artistic feel.The thing is, these tails become apparent only when you convolve with a very very bright source -- which on a typical 12-bit level linear raw image would amount to something like 10⁵-10⁶, i.e. needing 4-8 additional bits of HDR.Here are some useful links on the topic of flare simulation, I believe bokeh has mahy similar aspects:<a href="https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20200119024053&#x2F;http:&#x2F;&#x2F;simonwinder.com&#x2F;projects&#x2F;simulation-of-lens-flare-in-the-human-eye&#x2F;" rel="nofollow">https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20200119024053&#x2F;http:&#x2F;&#x2F;simonwinde...</a><a href="http:&#x2F;&#x2F;resources.mpi-inf.mpg.de&#x2F;hdr&#x2F;TemporalGlare&#x2F;" rel="nofollow">http:&#x2F;&#x2F;resources.mpi-inf.mpg.de&#x2F;hdr&#x2F;TemporalGlare&#x2F;</a>

As a hobby photographer, this is simply amazing and the most intuitive article I’ve come across. This is a must read.I am curious, however, why we still can’t digitally reproduce bokeh. Apple is getting close. I thought LiDAR would theoretically solve that and could yield indistinguishable renders compared to analog lenses. That would be a game changer in my view and why I would like to see Apple develop a full-frame sensor coupled with their technology.

In the article it is clearly acknowledged that there are different types. When you create a introduction like this you kind of have to limit your scope

The article is very good but you have to consider the existence of other filters, not only the Bayer one.And the Canon Dual Pixel AF that works a bit different than the article explains <a href="https:&#x2F;&#x2F;www.canonwatch.com&#x2F;dual-pixel-af-has-become-a-canon-standard-heres-how-it-works&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.canonwatch.com&#x2F;dual-pixel-af-has-become-a-canon-...</a> (but the substance is the same)

Sliders work perfectly for me on latest FF on latest LineageOS, which suggests that it&#x27;s not a problem with the website, rather with either FF or Windows on your device.

It works flawlessly on my iPad in Safari.

I did some more testing and I think it&#x27;s due to how Windows interact with the website slider.I&#x27;ve tried with Chrome on Windows 10 and find that the slider works but only if I touch on the line the slider is sitting on to set a value. Attempting to tap on the round circle and pulling it to slide does not work.I tried again on Firefox and if I very carefully tap on the line, I can set the slider. But touching the circle and sliding does not work. So, Firefox is more sensitive than Chrome when it comes to determining whether I am touching the line or not.I presume this is due to how Windows interprets touch events and passes them on to the browser, making the sliders hard to work with in my case.

Firefox on Android here, every animation works flawlessly.

The website sliders don&#x27;t work well with a touch interface. If I use a mouse, I can smoothly move the slider values. But attempting to tap and slide them either has no response or makes the slide move to the extreme left or right.I&#x27;m using Firefox (Nightly) on a Windows 10 tablet (Microsoft Surface Go).

I’ve been an amateur photographer for over 10 years now, and I’ve never seen such an easy-to-understand but still mathematically in-depth explanation of depth of field until now. Bravo. This is excellent.

Wow. What is this tool used to make these animations?

Sure! &quot;Color&quot; vs &quot;Black and White&quot; vs &quot;Sepia&quot; is primarily a function of the photosensitive (aka film or digital sensor) layer.

Is it possible to make a color pinhole camera?

I&#x27;m sure there&#x27;s a simple explanation if I look at the code, but how he did the z-order for the fins on that aperture is breaking my brain.

I&#x27;m speechless, even in a mid-end android phone every animation is smooth, is like dark magic to me.

Looking at the page source I&#x27;m doubly impressed there seems to be no use of third party libraries. I personally happily use three.js, but still, wow.<a href="https:&#x2F;&#x2F;ciechanow.ski&#x2F;js&#x2F;base.js" rel="nofollow">https:&#x2F;&#x2F;ciechanow.ski&#x2F;js&#x2F;base.js</a>
<a href="https:&#x2F;&#x2F;ciechanow.ski&#x2F;js&#x2F;lenses.js" rel="nofollow">https:&#x2F;&#x2F;ciechanow.ski&#x2F;js&#x2F;lenses.js</a>

This is so good!I wonder if the author publishes the source code for the interactive bits somewhere (I could of course look at them from the presented JS, but still). I&#x27;ve been looking for a non-fan-inducing way to generate a DOF-effect, and the one in the first example is definitely good enough for my use case.

Brilliant work.It really helps building an intuitive foundation of these phenomena.With such intuitive foundation, one has confidence to look into the math behind the curtain, even if this math gets complex, you will still feel confident.IMO, this kind of explanations is the correct way to start teaching anything.

I&#x27;ve seen this kind of essay described as an &quot;explorable explanation&quot; - I absolutely love them. There&#x27;s a directory of other examples of this kind of thing here: <a href="https:&#x2F;&#x2F;explorabl.es&#x2F;" rel="nofollow">https:&#x2F;&#x2F;explorabl.es&#x2F;</a>

While this is a very long article I just want to say great webpage, I&#x27;m saving this for an seo reference file.

The diffraction bands are approximately the Fourier transform of the slit function.Is this correct from a physics standpoint? Feynman had a lot to say about light in his book (QED <a href="https:&#x2F;&#x2F;www.amazon.com&#x2F;QED-Strange-Theory-Light-Matter&#x2F;dp&#x2F;0691024170" rel="nofollow">https:&#x2F;&#x2F;www.amazon.com&#x2F;QED-Strange-Theory-Light-Matter&#x2F;dp&#x2F;06...</a>) and never framed it this way. At one point he remarked that the explanations in the book were related to diffraction, and the explanation there was very different from the Fourier transform.

Indeed! We like to think of light as photons, traveling in rays in straight lines. That&#x27;s geometric optics and works great for most purposes in photography.Physical optics takes into account the wave nature of light. This becomes important when the size of the lens becomes small (eg pinholes) ... there&#x27;s diffraction around edges and pixels receive contributions from many points in space.Geometric optics lets you model using ray-tracing, reflections, and Snell&#x27;s law refraction.Physical optics uses tools such as Fourier transforms, convolutions, and sinc functions.Understanding a simple lens system? Geometric optics is your friend. Building an astronomical telescope? Check into physical optics.

I suspect the GP comment is referring in context to the use of the Fourier transform to efficiently implement convolution.

Pinhole lenses create visible diffraction effects pretty fast. Actually, even normal lenses cause diffraction at small ~f&#x2F;11-f&#x2F;22 apertures.

&gt;That &quot;infinitely many overlaps&quot; I was talking about happens with lenses too and is essentially a convolution where you convolve the image with itself (actually many different perspectives of itself if I am correct). Which is just the Fourier transform.That statement is a bit muddled, let me unpack it. The infinitely many overlaps thing can be expressed mathematically as the convolution of the image with a function that&#x27;s one where the aperture is open, and zero where it&#x27;s not. The thing about the Fourier transform is actually related to a different phenomenon. When the slit is really small, you start getting diffraction effects. The diffraction bands are approximately the Fourier transform of the slit function. However that is not significant unless the slit is extraordinarily tiny.

Here&#x27;s a paper on depth: <a href="http:&#x2F;&#x2F;groups.csail.mit.edu&#x2F;graphics&#x2F;CodedAperture&#x2F;" rel="nofollow">http:&#x2F;&#x2F;groups.csail.mit.edu&#x2F;graphics&#x2F;CodedAperture&#x2F;</a>Stanford has a class on computational imaging: <a href="http:&#x2F;&#x2F;stanford.edu&#x2F;class&#x2F;ee367&#x2F;" rel="nofollow">http:&#x2F;&#x2F;stanford.edu&#x2F;class&#x2F;ee367&#x2F;</a>I think these slides should give you a pointer to enough papers to fill a lifetime: <a href="https:&#x2F;&#x2F;web.stanford.edu&#x2F;class&#x2F;ee367&#x2F;slides&#x2F;lecture9.pdf" rel="nofollow">https:&#x2F;&#x2F;web.stanford.edu&#x2F;class&#x2F;ee367&#x2F;slides&#x2F;lecture9.pdf</a>Georgia Tech also has a class, that&#x27;s on youtube and Udacity: <a href="https:&#x2F;&#x2F;www.udacity.com&#x2F;course&#x2F;computational-photography--ud955" rel="nofollow">https:&#x2F;&#x2F;www.udacity.com&#x2F;course&#x2F;computational-photography--ud...</a>

&gt; captures depth information and you can use it for 3d reconstruction, refocusing etcThat’s crazy cool! Got any good links for reading about how to do those things that way; 3d reconstruct and refocus?

The pattern of holes technique in that link (MURA) is also used in UV lithography for IC fabrication (example: <a href="https:&#x2F;&#x2F;youtu.be&#x2F;8eT1jaHmlx8?t=1543" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;8eT1jaHmlx8?t=1543</a>).

The only thing better than a pinhole camera is a pinhole camera with multiple holes.Let me explain:When you have two holes (apertures), their images would likely overlap (depending on the arrangement).Imagine you took two photos from a few millimetres apart to the left and to the right, then you superimposed them. Some areas would look ok (as both views saw the same distant object) and some would be less ok (of objects that are closer).If you have a DSLR take the lens off and put a sheet with holes in front. Just try it.Now as you add more holes you get more and more overlapping images on the same area of the sensor and eventually end up with a blur.This blur is the same blur you see with a single large hole. It&#x27;s just that with a large hole you had infinitely many overlaps!Even cooler, if you happen to arrange the holes in a specific pattern you could capture images with different combinations of perspectives from different holes and you may even undo the overlaps. This is called coded aperture imaging:<a href="https:&#x2F;&#x2F;www.paulcarlisle.net&#x2F;codedaperture&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.paulcarlisle.net&#x2F;codedaperture&#x2F;</a>This doesn&#x27;t just solve the biggest problem (limited light) of a single hole, but also captures depth information and you can use it for 3d reconstruction, refocusing etc.One final bit, with a warning of a deep rabbit hole:That &quot;infinitely many overlaps&quot; I was talking about happens with lenses too and is essentially a convolution where you convolve the image with itself (actually many different perspectives of itself if I am correct). Which is just the Fourier transform.

The amount of total light is the same, but the smaller the points of focus, the easier it is to distinguish image elements from each other because each point overlaps less with its neighbors.Here’s a metaphor. Let’s say you have 10 glasses, with 10 different amounts of water in them. When you hit them with a spoon, each glass makes a different musical note: 10 different notes.If you pour the water from each glass individually into 10 new glasses, you preserve those 10 notes.If you combine every two glasses into one, now you have 5 glasses, and only 5 notes. It’s the same total amount of water but you lost information.

If you take up a larger part of the sensor, then the distance between any two details must be larger so the areas that their light covers over the sensor don&#x27;t overlap. If they overlap, you have no clue where the information is coming from.Look up circle of confusion, Rayleigh resolution criterion and sparrows resolution criterion for the technical details of resolution.

It&#x27;s more that the rays from a point on an object converge to the smallest point when in focus? The rays from the object as a whole are spread out over an object-shaped are on the image plane, unless the object is (effectively) at infinity, when they will converge to a point...

Ideally, the mapping from object to image is injective. What you&#x27;re proposing will lead to &quot;hash collisions&quot; aka blurriness, since each point in the object will bleed colors into neighboring points in the image.The entire object doesn&#x27;t get collapsed to a single point. Rather, a single point of the object radiates light in all directions. A lens then captures a fraction of that radiation and collapses it back down to a single point. Then we iterate over each point in the scene with a &quot;for each&quot;.

Great article. One thing that&#x27;s always stumped me as an amateur photographer is how an object is &quot;in focus&quot; when the rays from that object coverage to the smallest point, thus the smallest amount of pixels are getting hit. It would seem to me you would want to collect the max amount of information and thus use more of the sensor. How does converging to less of the sensor give you a sharper photo?

I was 45 years old when I learned why squinting your eyes make you see sharper. Never too old to learn!

Looks like the author works at Apple:<a href="https:&#x2F;&#x2F;patents.justia.com&#x2F;inventor&#x2F;bartosz-ciechanowski" rel="nofollow">https:&#x2F;&#x2F;patents.justia.com&#x2F;inventor&#x2F;bartosz-ciechanowski</a>

&quot;By default all animations are enabled, but if you find them distracting, or if you want to save power, you can globally pause all the following demonstrations.&quot;I love this writer &#x2F; developer

In terms of sharpness, each lens is different. And the sharpness varies depending on the focal length (zoom), aperture and the sensor.A great way to visualize this, is to take a look at the sharpness of the nifty fifty canon lens<a href="https:&#x2F;&#x2F;www.imaging-resource.com&#x2F;lenses&#x2F;canon&#x2F;ef-50mm-f1.8-stm&#x2F;review&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.imaging-resource.com&#x2F;lenses&#x2F;canon&#x2F;ef-50mm-f1.8-s...</a>Click into the heat map <a href="https:&#x2F;&#x2F;www.imaging-resource.com&#x2F;lenses&#x2F;canon&#x2F;ef-50mm-f1.8-stm&#x2F;blur&#x2F;canon-7d&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.imaging-resource.com&#x2F;lenses&#x2F;canon&#x2F;ef-50mm-f1.8-s...</a>Move the aperture from 1.8 to 2.8, and note how the sharpness increases, starting in the center of the lens.Then at about 16, diffraction kicks in and things start getting less sharp again.Things get even wilder when you introduce variable zoom and different camera bodies.It&#x27;s a great way to visualize the good&#x2F;fast&#x2F;cheap pick two tradeoff&#x27;s of lenses.

&quot;Dual native ISO&quot; sensors don&#x27;t perform different amplification before the read out amplifier or anything like that.With an image sensor you get to decide between high sensitivity and high capacity (how much light it can detect before saturating). So a highly sensitive sensor can handle less light before clipping (which makes sense).This trade-off can be tweaked by changing the size of the photodiode (bigger diode -&gt; more sensitive), but also electrically - simply add capacitance in parallel to the photodiode. This requires more energy, i.e. more photons, to change the voltage over the photodiode, which is what the readout measures.Dual native ISO sensors simply add another transistor (as a switch) in series to extra capacitance; this allows them to switch between low-capacitance (high sensitivity, but saturates faster) and high-capacitance (lower sensitivity, can handle more light).Edit: Haven&#x27;t watched the whole video but his core explanation in there seems to be that Dual ISO means the sensors has a PGA (programmable gain amplifier) -- almost all sensors use that approach, and that&#x27;s what is controlled by the ISO setting in them.

There are also &quot;dual native ISO&quot; sensors. Still different apmplification levels, but in this case it is before ADC.See <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=g8hHFt3ChZ8" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=g8hHFt3ChZ8</a>

Perhaps what we&#x27;re seeing here is that quantization errors for the non-amplified signal become larger than the noise introduced by the amplification?

&gt; Sometimes it&#x27;s better to take a deliberately underexposed photo and tweak regions of it in post to achieve the desired exposure instead of introducing global noise from in-camera amplification.That is definitely not my experience for Canon&#x27;s in low light situations. Here [1] is an example shot at ISO 100 but pushed by 5EV to match the brightness of the same scene shot at ISO 3200. The noise is much more tolerable for the latter.[1] <a href="https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;Ca9ccbK" rel="nofollow">https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;Ca9ccbK</a>P.S.: it&#x27;s raw, no in-camera JPEG denoising in place.

&gt; Alternatively, we could increase the sensitivity of the sensor which is described using the ISO rating.Umm well, not really.The sensitivity of a digital CCD or CMOS sensor never changes; it has one &#x27;native ISO&#x27; equivalent sensitivity and that&#x27;s all. It creates the same output signal in response to a photon every time.Varying the pseudo-ISO simply varies the gain of the downstream post-readout amplifier. That&#x27;s where noise is introduced.Sometimes it&#x27;s better to take a deliberately underexposed photo and tweak regions of it in post to achieve the desired exposure instead of introducing global noise from in-camera amplification.

Refraction goes beyond electrical fields. An intuitive feeling for refraction can be obtained from:- water: when waves go from, say, deeper to shallower water, their speed changes. If they are moving at an angle, refraction is seen. This can be explained by drawing wavefronts. The wavefront doesn&#x27;t hit the shallower water all at the same time, and so does not slow down all at once, but gradually.- wheels: when rolling axle with two wheels goes from pavement to grass on an angle, its direction will bend direction, due to the change in speed, which is experienced by one wheel ahead of the other.

How is this the first time I have gained an intuitive feeling for refraction? Instead I&#x27;ve been taught to simply accept Snell&#x27;s law. It feels very powerful that a single hypothesis - and a reasonable one at that (namely that the electrical field is continuous at the junction between materials) is enough to explain refraction!

This article is stunning. The design, the way it presents the information, everything. I don&#x27;t think I&#x27;ve ever seen such a good webpage.Edit: Now I&#x27;ve seen some of his other pages... This man is a genius.

this is quite a beautiful page, and in the first few seconds you can immediately grasp focus &amp; depth-of-field&#x2F;bokke.  But then it keeps going!

The pinhole diameter slider is backwards; the objects should appear sharper as the aperture shrinks. Also it&#x27;d be good if you showed the effect of diminished light-gathering power at the same time, to emphasize the tradeoff.All of the sliders should be labeled.All in all, this is a good conceptual effort, but the details aren&#x27;t strong.

Anyone else came here expecting a Haskell post?

Wow - just wow - such a great resource.  This is what I love about www - exposure to such enthusiasm and expertise.

You can go up or down (probably sideways if you can account for gravity, eg. through rotary motion). Up is more common since you only need significant equipment on one side and access requirements mean that keeping the bulk of it &#x27;down&#x27; is easier. No polishing is necessary. However, I recall reading that some polymers can be polished if you want, but IIRC it depends heavily on the polymer, the abrasive, intermediate fluids in use and result desired. Re: Source, I raided libgen for thermoforming over the last few months, visited about 10 equipment factories and studied their mechanisms and control systems, but actually learned this tidbit from 1000 Australian Inventions or some such book which I bought for my daughter before our family returned to China... to keep inventing ;)

Do you mean, like creating a bubble of plastic fed from below where surface tension holds it in a shape like a bubble? And then letting it harden while in that shape? Does the air pressure you talk about mean blowing a stream down onto the lens?And then, no further polishing is necessary? I&#x27;m fascinated to learn more if you have a source to read more from!

This works for some lens shapes. For most though, molding is necessary.

For a markedly less engineering oriented introduction see <a href="https:&#x2F;&#x2F;en.wikibooks.org&#x2F;wiki&#x2F;Modern_Photography&#x2F;The_camera" rel="nofollow">https:&#x2F;&#x2F;en.wikibooks.org&#x2F;wiki&#x2F;Modern_Photography&#x2F;The_camera</a>One interesting thing I learned recently is how they make cheap plastic lenses, and the fact that plastic lenses were invented by an Australian. They are created by choosing a polymer with good optical properties and then thermoforming it by carefully applying air pressure to raise a rounded shape to a dome after clamping the edges, then holding the air pressure constant until it has cooled. Because nothing actually touches the lens portion, optical properties are preserved.

As someone who spends most of his time either shooting, teaching or hustling, this truly is phenomenal. It makes some of the more mind-bending aspects of photography perfectly clear. Great post!

If I&#x27;m conversing with a Japanese photographer in Japanese, what word do I use for &quot;that Japanese word that American photographers use for a special kind of blur?&quot;It can&#x27;t be &quot;bokeh&quot; because that&#x27;s just an everyday word that means blur; for instance &quot;mae no shashin ga boketeru kara, keshimashou ka&quot; (the previous picture is blurry, so shall I erase it?)Also, since that term came into use in around 1997, if I could have a conversation with an English-speaking photographer via time telephone to 1990, what would be the wording used?

Cameras and Lenses