Image Sensors

Reference: Digital Image Processing (ECE 6258) from Prof. Ghassan Alregib

---

• Representing 3D to the 2D images would have artifacts.
• For example, motion blur could be happen.
  • Motion blur happens when the object moves during the exposure time.
  • Reduce the exposure time for preventing the motion blur - but the image could become dark.
• In addition, it would be hard to formulate and digitize in 2D for the occluded area of the object.

 

• Each light source has different kinds of SPD (Spectral power distribution).
• SPD describes the power per unit area per unit wavelength of an illumination.

 

• There are three color components.
• Brightness refers to how much light appears to shine from something. Brightness is a subjective sensation of an object.
• Hue indicates the attribute of the color itself. The term is also used to refer to colors that have no added tint or shade.
• More white added, the saturation could be different.

 

• CCD (Charge-Coupled Devices) is consisted with diode. It sends the electrical signal directly.
• CMOS (Complementary Metal-Oxide Semiconductor) sends the transformation form(voltage) of the electron from the light source.
• While transforming, the loss could be appear. So CCD outpass the CMOS in terms of Image quality.
• Both sensors transmit the illuminance signal(electric charge) to the processor.
• CMOS > CCD
  • Fast
  • Sensitivity
  • Efficiency 1/3 (consume power only when the circuit is on)
  • Simple
  • Noisy (CCD has only one circuit board for accessing the voltage, indicating the consistent quality)
• CCD > CMOS
  • Cost
  • Image quality
  • Represents the input signal as it stands.
• How CCD / CMOS could be operated: https://youtu.be/nxUDHcZl1uo

• Increasing the number of megapixels results in higher resolution.
• But if the sensor size remains the same, it results in smaller photosites.
• Photosite size is reffered to as the pixel pitch.
• Smaller photosites gather less light, which means less signal strength.

• Nominal height resolution = Actual Height(cm) / the number of pixel in height
• Nominal width resolution = Actual Width(cm) / the number of pixel in width

• After being captured from the camera, the image is transformed to the digital image that computer could understand.
• Frame grabber transforms the video signal from the camera into the digital image data. 

• After measuring the intensity of the light, the filter layer(color filter) is applied in order to obtain colored image.
• The most common format of the filter layer would be 'Bayer filter' (BGRG).

 

• Mobile phone typically use CMOS for portable issue.

• CMOS use rolling shutter, which would be 

 

what impact?

CMOS problem

1 line by line

speed of the motion is not enoughly sampled by the camera.

skewing happen - cmos artifacts

needs correction!

 

how can we recover the image would be really challenging. (denoise, restore,.. )

 

CCD: global shutter

blur

Reference : https://smartphones.gadgethacks.com/how-to/everything-you-should-know-about-rolling-shutter-your-phones-camera-0196244/

Everything You Should Know About Rolling Shutter on Your Phone's Camera

 

- photometric distortion : color, 

- geometric distortion: zoom in - pixels are not in align.

autonomous vehicle : fish-eye view(FOV bigger) -> how can machine learning algorithm understand the distortion?(tough problem)

ex 1. pincushion : middle uniform/

ex 2. barrel: 

 

- spatial digitization+time rate, 

higher resolution(frame rate)-> higher fidelity -> higher quality -> but difficult to process

- amplitude:quantization

 

- super resolution,

- interporlation

sampling took place (tv, youtube, .. 

 

- match the details with the image

 

- 72 dpi has a lot of distortion

- 150 dpi has distortion in the edge.

1250 vs. 300 has some differences but not much difference.

but the number 1250 vs. 300 is huge difference.

How can we get such an image efficiently? (how do we mimic human vision system)

Mathematical formulation

quality assessment