diff --git a/README.md b/README.md index f044c821..a110abd8 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,129 @@ CUDA Denoiser For CUDA Path Tracer -================================== +================ -**University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 4** +**University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 2** -* (TODO) YOUR NAME HERE -* Tested on: (TODO) Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab) +* RHUTA JOSHI + * [LinkedIn](https://www.linkedin.com/in/rcj9719/) + * [Website](https://sites.google.com/view/rhuta-joshi) -### (TODO: Your README) +* Tested on: Windows 10 - 21H2, i7-12700 CPU @ 2.10 GHz, NVIDIA T1000 4096 MB +* GPU Compatibility: 7.5 -*DO NOT* leave the README to the last minute! It is a crucial part of the -project, and we will not be able to grade you without a good README. +## Introduction + +[Link to full path tracer project](https://github.com/rcj9719/GPU-Programming-CUDA-Path-Tracer) + +[Link to denoiser branch for denoiser code](https://github.com/rcj9719/GPU-Project4-CUDA-Denoiser/tree/denoiser) + +Since we cast many rays per pixel in order to get enough light information, we can get effects like caustics, soft shadows, anti-aliasing, and depth of field. Since this technique involves computing a large number of rays independently, it can be highly parallelized to converge images incredibly faster on a GPU as compared to a path tracer implementation on CPU. In this project, I have used CUDA to compute intersections and shading per iteration for multiple rays parallelly. + +In this branch, I have explained how I enhanced my path tracer by implementinf a pathtracing denoiser that uses geometry buffers (G-buffers) to guide a smoothing filter. + +|Before denoising(10 iterations)|After denoising (10 iterations)| +|---|---| +|![](/img/denoiser/nondenoise_10_time_776dot137.png)|![](/img/denoiser/denoise_10_16_65_time_798dot5116.png)| +|Time taken: 776.137ms|Time taken: 798.5116ms| + +## Technical Background + +This implemetation is based on the paper ["Edge-Avoiding A-Trous Wavelet Transform for fast Global Illumination Filtering,"](https://jo.dreggn.org/home/2010_atrous.pdf) by Dammertz, Sewtz, Hanika, and Lensch. + +Denoisers can help produce a smoother appearance in a pathtraced image with fewer samples-per-pixel/iterations, although the actual improvement often varies from scene-to-scene. Smoothing an image can be accomplished by blurring pixels - a simple pixel-by-pixel blur filter may sample the color from a pixel's neighbors in the image, weight them by distance, and write the result back into the pixel. + +### A-Trous filter +A simple Gaussian blur may look something like this: +``` +for each pixel being blurred: + sample every pixel in blur radius + multiply by a weight in a weight distribution called a "filter," or h + sum results and write result into result buffer +``` +![](img/denoiser/atrous.png) + +However, this can be computationally expensive. For a 16x16 blur width/filter, this is 256 pixel reads for each pixel blurred. So instead, we use ATrous filter. Atrous filter reduces the number of pixels to be taken into consideration for blur, by iteratively spreading a small mask/filter spatially or in other words, by applying sparse blurs of increasing size. + +![](img/denoiser/sparsefilter.png) + +### G-buffers + +Simply applying the filter blurs the entire image without taking features like edges, positions, depth and normals into consideration. + +|ATrous simple blur| Atrous filter with edge avoidance| +|---|---| +|![](img/denoiser/atrous.png)|![](img/denoiser/atrousedgeavoid.png)| + +To detect and avoid edges while applying the filter, we make use of deferred shading, in which we first store the geometry information such as position, normals and depth in a buffer and based on weights assigned to each buffer, tune our denoiser for better results. + +|Position G-buffer|Normal G-buffer|Depth G-buffer| +|---|---|---| +|![](img/denoiser/position.png)|![](img/denoiser/normal.png)|![](img/denoiser/depth.png)| + + +## Denoising Results and Analysis + +### Changing blur radius vs changing filter size + +The following chart shows time taken for denoising an image of 10 iterations with a filter size of 5 and varying blur radius. + +![](img/denoiser/kernelsizeanalysis.png) + +We can see that as the size of blur radius increases time taken to denoise the image also increases. +For a fixed filter size of 5, we can increase the blur radius to get better denoised images as you can see below: + +### Gaussian vs ATrous filter + +Since ATrous filter spreads out the filter sparsely and calculates the sam enumber of pixels per iteration, it performs much more effectively than Gaussian filter of same size as the blur radius. Visually, Gaussian may generate better blur images. + +![](img/denoiser/filtersize.png) + +### Changing image resolution + +Not surprisingly, as the resolution of our render increases, time taken to denoise it also increases because of increased number of pixels to be denoised. + +While the denoiser may add a small overhead, it generates equally acceptable images in very less iterations, thus making it an effective feature for a path tracer. For 10 iterations, filter width 5, blur width 17, we see the results below. + +![](img/denoiser/resolutionanalysis.png) + +|Resolution|Time taken| Render| +|---|---|---| +|400 x 400|3.28918 ms |![](img/denoiser/400.png)| +|600 x 600|5.72173 ms|![](img/denoiser/600.png)| +|800 x 800|13.4084 ms|![](img/denoiser/800.png)| +|1000 x 1000|17.682 ms|![](img/denoiser/1000.png)| + +### Denoiser on different materials + +This denoiser does not always appear to give best visual approximation especially for specular transmissive materials as you can see below + +**Diffused materials** - Diffused materials generally look good and do not lose out on any specific visual features like color, positions etc because of geometry buffers passed in. +![](img/denoiser/diffuse_10_16_65.png) + +**Reflective** - Reflections can also get blurred, as edge avoidance does not avoid edges inside reflections, making it appear like a microfacet reflecting material with increased roughness instead of purely reflective specular material. +![](img/denoiser/reflective_10_16_65.png) + +**Transmissive** - Transmissive materials may also lose some prominent visible features such as caustics. +![](img/denoiser/transmissive_10_16_65.png) + +### Obtaining visually smooth results + +|Status|Iterations|Time taken|Render| +|---|---|---|---| +|Denoiser on|10|0.798 seconds|![](img/denoiser/denoise_10_16_65_time_798dot5116.png)| +|Denoiser off|200|13.96 seconds|![](img/denoiser/nondenoise_200_time_13961dot3.png)| + +We can see that while we make lose some details on specular reflections, a denoiser can give us fairly smooth images in very few iterations. + +## References + +1. ["Edge-Avoiding A-Trous Wavelet Transform for fast Global Illumination Filtering,"](https://jo.dreggn.org/home/2010_atrous.pdf) by Dammertz, Sewtz, Hanika, and Lensch. +2. UPenn CIS 565 Course Notes and presentation slides + + +## Bloopers + +![](img/denoiser/blooper1.png) +![](img/denoiser/blooper2.png) +![](img/denoiser/blooper3.png) +![](img/denoiser/blooper4.png) diff --git a/img/denoiser/1000.png b/img/denoiser/1000.png new file mode 100644 index 00000000..db8e4f3a Binary files /dev/null and b/img/denoiser/1000.png differ diff --git a/img/denoiser/10_16_65.png b/img/denoiser/10_16_65.png new file mode 100644 index 00000000..df4e6892 Binary files /dev/null and b/img/denoiser/10_16_65.png differ diff --git a/img/denoiser/10_2_9.png b/img/denoiser/10_2_9.png new file mode 100644 index 00000000..338b2b94 Binary files /dev/null and b/img/denoiser/10_2_9.png differ diff --git a/img/denoiser/10_4_17.png b/img/denoiser/10_4_17.png new file mode 100644 index 00000000..21b07f16 Binary files /dev/null and b/img/denoiser/10_4_17.png differ diff --git a/img/denoiser/10_8_33.png b/img/denoiser/10_8_33.png new file mode 100644 index 00000000..d3d9c3b0 Binary files /dev/null and b/img/denoiser/10_8_33.png differ diff --git a/img/denoiser/10iter_17by17.png b/img/denoiser/10iter_17by17.png new file mode 100644 index 00000000..21b07f16 Binary files /dev/null and b/img/denoiser/10iter_17by17.png differ diff --git a/img/denoiser/1200.png b/img/denoiser/1200.png new file mode 100644 index 00000000..908d6a01 Binary files /dev/null and b/img/denoiser/1200.png differ diff --git a/img/denoiser/400.png b/img/denoiser/400.png new file mode 100644 index 00000000..85f6d5ef Binary files /dev/null and b/img/denoiser/400.png differ diff --git a/img/denoiser/600.png b/img/denoiser/600.png new file mode 100644 index 00000000..2a8191c3 Binary files /dev/null and b/img/denoiser/600.png differ diff --git a/img/denoiser/800.png b/img/denoiser/800.png new file mode 100644 index 00000000..cf6c358e Binary files /dev/null and b/img/denoiser/800.png differ diff --git a/img/denoiser/Annotation 2022-10-18 180741.png b/img/denoiser/Annotation 2022-10-18 180741.png new file mode 100644 index 00000000..cabecd3a Binary files /dev/null and b/img/denoiser/Annotation 2022-10-18 180741.png differ diff --git a/img/denoiser/Annotation 2022-10-20 190634.png b/img/denoiser/Annotation 2022-10-20 190634.png new file mode 100644 index 00000000..b31dd58a Binary files /dev/null and b/img/denoiser/Annotation 2022-10-20 190634.png differ diff --git a/img/denoiser/Annotation 2022-10-20 233427.png b/img/denoiser/Annotation 2022-10-20 233427.png new file mode 100644 index 00000000..4a38ab6d Binary files /dev/null and b/img/denoiser/Annotation 2022-10-20 233427.png differ diff --git a/img/denoiser/Annotation 2022-10-21 171205.png b/img/denoiser/Annotation 2022-10-21 171205.png new file mode 100644 index 00000000..42398cb7 Binary files /dev/null and b/img/denoiser/Annotation 2022-10-21 171205.png differ diff --git a/img/denoiser/Annotation 2022-10-21 235302.png b/img/denoiser/Annotation 2022-10-21 235302.png new file mode 100644 index 00000000..193e620e Binary files /dev/null and b/img/denoiser/Annotation 2022-10-21 235302.png differ diff --git a/img/denoiser/atrous.png b/img/denoiser/atrous.png new file mode 100644 index 00000000..32e4a706 Binary files /dev/null and b/img/denoiser/atrous.png differ diff --git a/img/denoiser/atrousedgeavoid.png b/img/denoiser/atrousedgeavoid.png new file mode 100644 index 00000000..9d893279 Binary files /dev/null and b/img/denoiser/atrousedgeavoid.png differ diff --git a/img/denoiser/blooper1.png b/img/denoiser/blooper1.png new file mode 100644 index 00000000..fc488452 Binary files /dev/null and b/img/denoiser/blooper1.png differ diff --git a/img/denoiser/blooper2.png b/img/denoiser/blooper2.png new file mode 100644 index 00000000..df26f863 Binary files /dev/null and b/img/denoiser/blooper2.png differ diff --git a/img/denoiser/blooper3.png b/img/denoiser/blooper3.png new file mode 100644 index 00000000..576e5f77 Binary files /dev/null and b/img/denoiser/blooper3.png differ diff --git a/img/denoiser/blooper4.png b/img/denoiser/blooper4.png new file mode 100644 index 00000000..496e595a Binary files /dev/null and b/img/denoiser/blooper4.png differ diff --git a/img/denoiser/cornell.png b/img/denoiser/cornell.png new file mode 100644 index 00000000..bc60c2d7 Binary files /dev/null and b/img/denoiser/cornell.png differ diff --git a/img/denoiser/cornellceiling.png b/img/denoiser/cornellceiling.png new file mode 100644 index 00000000..5185b967 Binary files /dev/null and b/img/denoiser/cornellceiling.png differ diff --git a/img/denoiser/denoise_10_16_65_time_798dot5116.png b/img/denoiser/denoise_10_16_65_time_798dot5116.png new file mode 100644 index 00000000..036cdeb0 Binary files /dev/null and b/img/denoiser/denoise_10_16_65_time_798dot5116.png differ diff --git a/img/denoiser/denoise_25_time_1847dot42.png b/img/denoiser/denoise_25_time_1847dot42.png new file mode 100644 index 00000000..423e4076 Binary files /dev/null and b/img/denoiser/denoise_25_time_1847dot42.png differ diff --git a/img/denoiser/depth.png b/img/denoiser/depth.png new file mode 100644 index 00000000..f299f491 Binary files /dev/null and b/img/denoiser/depth.png differ diff --git a/img/denoiser/diffuse_10_16_65.png b/img/denoiser/diffuse_10_16_65.png new file mode 100644 index 00000000..fd609ac1 Binary files /dev/null and b/img/denoiser/diffuse_10_16_65.png differ diff --git a/img/denoiser/filtersize.png b/img/denoiser/filtersize.png new file mode 100644 index 00000000..aa708f07 Binary files /dev/null and b/img/denoiser/filtersize.png differ diff --git a/img/denoiser/kernelsizeanalysis.png b/img/denoiser/kernelsizeanalysis.png new file mode 100644 index 00000000..2f8a3c8d Binary files /dev/null and b/img/denoiser/kernelsizeanalysis.png differ diff --git a/img/denoiser/nondenoise_100_time_ 6999dot79.png b/img/denoiser/nondenoise_100_time_ 6999dot79.png new file mode 100644 index 00000000..7481d25f Binary files /dev/null and b/img/denoiser/nondenoise_100_time_ 6999dot79.png differ diff --git a/img/denoiser/nondenoise_10_time_776dot137.png b/img/denoiser/nondenoise_10_time_776dot137.png new file mode 100644 index 00000000..6e71795e Binary files /dev/null and b/img/denoiser/nondenoise_10_time_776dot137.png differ diff --git a/img/denoiser/nondenoise_200_time_13961dot3.png b/img/denoiser/nondenoise_200_time_13961dot3.png new file mode 100644 index 00000000..57516d0a Binary files /dev/null and b/img/denoiser/nondenoise_200_time_13961dot3.png differ diff --git a/img/denoiser/normal.png b/img/denoiser/normal.png new file mode 100644 index 00000000..7bf5ddfb Binary files /dev/null and b/img/denoiser/normal.png differ diff --git a/img/denoiser/performanceanalysis.txt b/img/denoiser/performanceanalysis.txt new file mode 100644 index 00000000..a7e6b04f --- /dev/null +++ b/img/denoiser/performanceanalysis.txt @@ -0,0 +1,22 @@ + + + +10_5_17 + +3.28918 + 5.72173 +13.4084 + 17.682 + 25.1773 + + + + + + + +5x5 - 4.70522ms +9x9 - 9.19005ms +17x17 - 13.3842ms +33x33 - 17.8359ms +65x65 - 21.9197ms \ No newline at end of file diff --git a/img/denoiser/position.png b/img/denoiser/position.png new file mode 100644 index 00000000..7551b26b Binary files /dev/null and b/img/denoiser/position.png differ diff --git a/img/denoiser/reflective_10_16_65.png b/img/denoiser/reflective_10_16_65.png new file mode 100644 index 00000000..f9ffb60a Binary files /dev/null and b/img/denoiser/reflective_10_16_65.png differ diff --git a/img/denoiser/resolutionanalysis.png b/img/denoiser/resolutionanalysis.png new file mode 100644 index 00000000..7cf6b5b1 Binary files /dev/null and b/img/denoiser/resolutionanalysis.png differ diff --git a/img/denoiser/sparsefilter.png b/img/denoiser/sparsefilter.png new file mode 100644 index 00000000..24c59cc3 Binary files /dev/null and b/img/denoiser/sparsefilter.png differ diff --git a/img/denoiser/transmissive_10_16_65.png b/img/denoiser/transmissive_10_16_65.png new file mode 100644 index 00000000..de45434c Binary files /dev/null and b/img/denoiser/transmissive_10_16_65.png differ diff --git a/img/denoiser/transmissive_10_4_17.png b/img/denoiser/transmissive_10_4_17.png new file mode 100644 index 00000000..9a06d738 Binary files /dev/null and b/img/denoiser/transmissive_10_4_17.png differ