Is it possible to 'unblur' a photo? - technical explanation of the issue of image blur (film, 10m)
Captain Disillusion recently challenged the internet to unblur a heavily blurred image to reveal a secret pattern of numbers lying underneath. He promised a video shout-out to the first person who could accomplish this feat. While many people attempted to tackle this challenge, only one person seemed to succeed—although there’s surely more to the story! This leads to an interesting discussion about how to restore seemingly degraded or missing information in images. How does image blurring work in the first place? Images consist of pixels, which have sets of color values. Since these values can be altered through arithmetic, the image can be made brighter or darker depending on the adjustments made to these pixel values. This begins the exploration of point spread functions, or PSFs. For example, by averaging the values of neighboring pixels, one can create an effect of blurring. While that is the simplest form of blur, it is nonetheless a beginning. By using different distributions in this 'kernel' of values, we can achieve more sophisticated blurs known as Gaussian blur.
However, advanced image processing is not limited to just blurring. Captain Disillusion mentions that when an image is large, the convolution process can become extremely time-consuming. He admits that there are tricks one can use to speed up this process with separable functions. If the function happens to be separable, one can convolve one-dimensional slices of the kernel instead of the entire image. The captain also introduces Fourier Transform as a method of spectral decomposition of signals where any periodic function can be expressed as sums of sine and cosine functions. This concept is pivotal as the Fast Fourier Transform allows for efficient computation with images, emphasizing its importance in modern image processing.
In this video, Captain Disillusion not just illustrates how to 'unblur' images but also teaches how mathematics can fundamentally change our perception and processing of visuals. The central theme he conveys is that any image can be transformed into a matrix of frequency values, enabling deeper analysis. Convolution in the frequency domain makes operations on images easier while reducing computational complexity. He makes it clear through examples that understanding the challenges that arise is crucial when attempting to deconvolve an image.
Nevertheless, it's not as simple as it might seem. Captain highlights that without knowing the exact kernel function, one cannot simply 'unblur' an image. Any alteration to the blurred image can incorrectly affect its relation to the kernel, complicating matters further. Approaches such as Wiener deconvolution aim to tackle noise in the frequency domain, and Captain invites viewers to explore further into the topic, even using a unique analogy with making borscht to illustrate
Ultimately, the Captain humorously suggests that while one can recreate an image based on educated guesses, this is not genuine unblurring for unique images. Perhaps, however, some viewers will utilize these techniques to delve into image creation more creatively. By watching this 17:12-minute video, audiences can gain fresh perspectives on image processing, especially as the current view count stands at 1,110,277 with 95,153 likes. This piece of content is definitely worth checking out!
Toggle timeline summary
-
The speaker challenges the internet to unblur a heavily blurred image.
-
Only one person successfully unblurred the image, prompting further investigation.
-
The speaker questions the process of restoring missing information in images.
-
An exploration of how images can be blurred is introduced.
-
Pixels in images consist of color values influenced by arithmetic changes.
-
More complex image processing techniques can blur images based on surrounding pixels.
-
The Gaussian blur technique is explained as a smoother alternative.
-
Convolution is introduced as a significant process in image manipulation.
-
The speaker mentions techniques to speed up convolution using separable kernels.
-
The Fourier transform is discussed as a method for efficiently calculating filters.
-
Images can be broken down into component frequencies with the Fourier transform.
-
The speaker contrasts spatial and frequency domains in image processing.
-
Wiener deconvolution is mentioned for tackling noise challenges.
-
A clever but limited method of reproduction is described, emphasizing that it's not true unblurring.
-
The speaker notes advancements in deconvolution software while reflecting on personal shout outs.
-
The speaker humorously acknowledges the numerous creative individuals contributing to the field.
Transcription
I challenged the internet to unblur this heavily blurred image to reveal the secret pattern of numbers written underneath, and promised a video shout-out to whoever could do it first! Many a catgirl tried, but only one actually pulled it off! Or did she? How does one bring back seemingly degraded, missing information in a picture? Is it even possible? Are the world's Japanese vloggers safe? For that matter, how does one blur an image in the first place? What is blur? Where am I? Images are made of pixels, which are a set of three color values, each of which is just a number of some range. And numbers can be altered via something philosophers call arithmetic. Add an amount to all the pixels, the picture gets brighter. Subtract an amount, the picture gets darker. Multiply by 2, twice as bright, if you're in linear gamma. Multiply by 0.4, it's darker again, I got you! But that's basic adjustments. We can do more elaborate image processing by changing the value of each pixel in a unique way based on the pixels around it, using another bunch of pixels, a pattern or function of points that are spread out. A point spread function, a PSS. For example, add the values of all the immediate neighbors. Wait, first divide all of them by 9, because there's 9 of them, and then add them. Okay, it didn't do much, but if we march the point spread function box across every pixel in the image and do that, we get box blur! The crappiest kind of blur, but it's a start. We can use different distributions in this, let's just call it kernel, of values, as long as they add up to 1. If we arrange it in an unpopped pimple pattern, we get a much nicer, smoother kind of blur. Gaussian blur. By the way, you don't have to think of it as marching the kernel across. It's also kind of like putting kernels over all the pixels, multiplying each one by the pixel, then adding them all together. And this process is not just for blurs, it works for all sorts of image filters. You can sharpen, find edges, and just imagine the trippy possibilities when the function in the kernel varies depending on what part of the image it's in! Whoa. When we mush the image with the kernel this way, in terms of numbers, what's happening is called convolution. But I'll let the math people explain that. All I know is it's a tedious, computationally expensive process. If the image is big, and especially if the kernel is big, and the kernel is allowed to be as big or bigger than the image, it takes frickin' forever to convolve. Humans only live like 700,000 hours. There are tricks we can do to speed it up. For instance, if the function happens to be separable, we could take just a one-dimensional slice of the kernel, convolve that, then take the other dimension and convolve the convoluted thing with that. So it's like doing just twice instead of square of the n root... I'll let the math people... It's way less work is the point. Still, when I go to filter, blur, blur, I don't just want it to happen pretty fast, I want it now! Luckily, there's an even more efficient way to calculate a filter. It involves a furrier, I mean fourier, I mean fourier, transform. Joseph Gordon Fourier was an 18th century French guy obsessed with hotness of things. Somehow, this led him to discover that any periodic function is actually a sum of a series of basic sine and cosine functions. Later, other people figured out that actually any non-repeating wave is actually made up of a bunch of sine waves, actually. There's literally no squiggle you can draw that couldn't be transformed into these. That's crazy. An image is a 2D function. A digital image is a discrete sampling of the function at the pixel locations, and this can be handled super efficiently by a discrete version of the fourier transform called discrete fourier transform, which we can do fast using an algorithm called fast fourier transform. Well, the math people can. The devil numbers! But once we get them to get a computer to do it for us, we can break any picture down into its component frequencies, which allows us to accomplish things we never dreamed of before. If you wish to make a bowl of borscht less salty, you must first invent a universe in which you've already done that. But what if we could put the bowl through a mathematical machine? One that separates back out the cabbage, the beets, the bouillon cubes, even the not yet butchered cow that made the sour cream. In this ingredient domain, reducing the amount of salt could be done with a flick of the wrist. Oh no! Square me! Someone help! What was a tedious convolution process in the spatial domain, in the frequency domain becomes simple multiplication. The kernel can be applied to the image everywhere all at once, just like that. But what good is it unless we can turn it back into the original form of the signal, a viewable picture? Well, we can, through an inverse of a discrete fourier transform process called... Wait. Wait. If multiplying the fourier transform of the image by the fourier transform of the kernel blurs the image, then wouldn't taking the fourier transform of that blurred image and doing some hypothetical operation with the kernel that's like the opposite of multiplication undo the blur? If only there was a... Ah, so this is who I obligated myself to give a shout-out to. M. Nurson, huh? What a special kitty, they broke the mold when they made her- oh wait, never mind, there's a bunch of them. So did this kid just perform sophisticated image deconvolution with her bare hands? Because yes, in principle it does work the way I explained, but the reason you don't see everything getting easily unblurred all the time is because it's not that easy in practice. For one thing, you need to know the exact kernel function and its size, and the blurred image can't be altered in any way. The slightest rescaling or crop changes its relationship to the kernel and throws the whole process off. Even imperceptible tweaks to the blurred image can ruin it. The frequency domain is like an opposite world. Big broad contrasts are barely noticeable, while tiny subtle details are huge. Microscopic noise or compression artifacts end up massive and can overwhelm the deconvoluted image. Of course, there are mathematical approaches to tackling these challenges. There's Wiener deconvolution, which tries to cleverly attenuate the noise in the frequency domain. Oh look, it's the limit of my ability to comprehend and explain things. It actually exists right here. If you need more, if you're so smart and have the time and energy to dive deeper, then you should have no problem continuing onward. Be my guest. Go frolic with your math people friends. Imagine imaginary numbers together. Good luck. For the rest of us, there's always cheating. Like what Emerson did. That's right. You can get the details on her channel, but basically she recreated my grid of numbers pixel for pixel and used a boosted up difference mat to manually try out different digits in each spot until each blurry blob matched. Is it an extremely clever approach to this specific challenge? Yeah, but it's not unblurring. It's a reproduction based on a guess, and it was only possible because I provided every bit of helpful data. This method could never be used to unblur a unique image like for example, this go ahead. I dare you internet. It's a very compromising picture and a clue to my mysterious origins. There are brilliant people out there teaching these concepts, advancing them, making free open source deconvolution software while giving away files for 3d printing your own microscope at home. And I'm supposed to give a personal shout out to some Photoshop savvy, furry geometry dash addict. I don't think so. If I shouted out every cat child I come across, we'd be here all day. I'd have to shout out Sonia and Riesel and Mochi and Fluffles and Phantom and Sonic and Squirrel and Sophia and Archer and Chopper and Masha and Charlie and Ash and Risa and Bessie and Simba and Sima and