The HDR/Tonemapping/Image Quality/RAW vs JPG Experiment

I have been meaning to do an experiment like this for a while. I went on a hike and decided that today is the day I am going to do it.

The Experiment:
Create an HDR image using different file sizes and type (RAW) and compare the differences to see if it is actually worth bumping up the quality.  Also; I will be comparing tonemapping.  I have been dabbling in HDR photos for a while now and have concluded that creating an HDR image using 3 images vs just one does not seem to make any significant difference...  I will find out if it does or not. In the process of all of this, I am also going to see if there is any noticeable difference using RAW vs JPG.

The Setup:
For this I setup my camera on a tripod and took a picture of a landscape in a canyon.  The camera I use is a Canon EOS 20D.  It is a pretty outdated camera but for the purpose of this experiment should work fine.  I took 6 different shots (of the same thing).  I took 2 sets of bracketed photos.  I took 3 pictures on the highest quality jpg + RAW and 3 pictures of medium + RAW.

Things to be tested:

• If there is a significant improvement using the highest quality vs medium.
• If there is a significant improvement shooting RAW vs jpg.
• If creating an HDR image and doing tonemapping is actually better using 3 bracketed photos vs using only one.
• If tonemapping has any significant improvement using RAW vs jpg

 Procedure and Observations:

Firstly, I might as well point out that the sole purpose of this experiment is because as a photographer, it is important to do things in a time efficient manner; saving hard drive space is also a bonus (not to mention editing smaller files is much quicker).

File Size

Does highest quality vs medium vs RAW actually make a significant difference?  We will compare 9 images.  With HDR photography, it is most common to take three different images at different exposures. Underexposed (dark), normal, and overexposed (light).  We will be comparing RAW, highest quality jpg (HQ jpg) and medium quality jpg (MQ jpg).

Overexposed RAW = 10.9MB
Normal RAW = 9.1MB
Underexposed RAW = 7.4MB

Overexposed HQ jpg = 5MB
Normal HQ jpg = 4.6MB
Underexposed HQ jpg =  3.3MB


Overexposed MQ jpg = 1.6MB
Normal MQ jpg = 1.4MB
Underexposed MQ jpg = 1MB

Observations - I was surprised to see that the lighter the photo was, the larger the file size.  I always thought it was the other way around.  Good to know.  Comparing file sizes...  we will use the underexposed image to make the math easy for me.  The medium quality was 1MB.  Highest quality is 3 times that. RAW is 7 times that. It is pretty easy to say that you can get twice as many photos for the same space if you are willing to go down in quality...   below we will see if it is worth it to bump up the quality.



The Image:

Here is the image.  This is the normal exposure with medium quality.

To better contrast with high quality, we will zoom in to the knot on the closest log.  To be fair to the medium quality image, we will use the Gimp and scale the image up to be the same size as the larger, highest quality image.  Here is the result:

This is the high quality log.

This is the medium quality log.

Is there a noticeable difference?  Yes.  Is it a big difference?  Debatable.  Is the high quality file 3 times better than the medium quality?  Certainly not.  The difference to me is not very noticeable...  I really don't see any reason to shoot highest quality pictures for casual photography.  It is sad that the difference is so minor.  If I had to guess, I would say this is a limitation of the sensor...  it is almost as if the high quality is a blown up version of the medium quality.  Not much improvement; just bigger.

What about raw you ask?  Let's see:

RAW log.

The only difference I notice is that the raw image seems to be a bit darker.  Perhaps this is better?

OK.  Now on to HDR and tonemapping.

First; I am going to use a single image to set my tonemapping settings.  I have noticed that I have been getting similar or better results using only a single image for HDR rather than 3 images with different exposures.  Let's find out which is really better!

The software I use for HDR and tonemapping is called Luminance HDR (formerly QTPFSGUI)  http://qtpfsgui.sourceforge.net/

I will use the high quality normal exposure image to set the standard and settings.

Settings used: Matiuk '06 - Contrast Factor = .2 Saturation Factor = 1.6 Detail Factor = 7

Now we will compare the before and after:

Before

After (Tonemapped)

Most of the differences are pretty subtle.  The tonemapped image has a darker mountain in the background.  Also; if you look to the logs (especially to the left) you will notice the shadows pop out a bit more. The sky looks a bit weird; quite wispy.

Now let's look at the difference if we use three images using the same settings.

Tonemapped using 3 bracketed photos for actual HDR image.

So there are some differences that I like; some I don't.  Firstly; the colors seem to be more vivid with the actual HDR photo. Also; the sky is bluer, but still kind of wispy.  It seems though that the foreground is a bit darker...  it seems to take away from some of the detail...

Now; what about a single raw image? Apparently raw is supposed to be better for HDR as it retains more color info.  Let's try it out!

Single raw image. Tonemapped.

Wow!  I would say the difference using a raw image is pretty significant!  The sky looks much better.  The colors seem to be just as vivid as using 3 images.

Now let's try 3 raw images.  You can't get more HDR than this...  this should be the best image for sure.

True HDR using 3 bracketed raw images.

Interesting. I could not notice any difference with using 1 raw image as opposed to 3 raw images until I flipped between the two.  The HDR image using the 3 bracketed images is slightly darker. I don't think I prefer it though.

That was a lot of work!  I think I learned a lot of great stuff here.  I think I will change the way I take some pictures, but continue to do some things the same way.

Conclusion

File Size: If you are at all concerned about file size, go ahead and use medium quality. Taking raw or high quality is not going to make a significant difference for casual photography if you are just going to keep it on your computer.  If you are going to do a paying gig, or want to print things out, go ahead and bump up the quality to be on the safe side. It probably won't make a huge difference, but if someone is paying you... 

HDR: This is a pretty time consuming process.  The results suprised me a bit.  If you are taking pictures using jpgs, if you want the extra pop in your HDR images, you should probably take the bracketed photos.  It might not make a huge difference on all images though.  Using raw images for HDR DOES in fact make a positive difference...   though I wouldn't say it is necessary to take 3 bracketed images. I think I will take raw pictures to save some time and the quality seems to be the best.

Limitations of this test: Admittedly, my test photos were not the best or prettiest for measuring.  They worked sufficient for the test. You should keep in mind that pictures with more sky/rock/water etc might have varying results. Also; I kept the settings the same throughout.  Perhaps you could get improved quality by editing these settings.

How this changes the way I do things:  Raw. I will use it more.  If I see a great HDR shot, I will probably go ahead, turn on the manual settings and get a raw picture or two.  If I am just doing casual pictures (not HDR) I will continue to shoot using medium quality jpgs.

This was an interesting experiment.  Expect to see some more in depth (and more specific) tests for quality differences and HDR.