Project4: Zhangkaiwen Chu

.ipynb_checkpoints/Untitled-checkpoint.ipynb

@@ -0,0 +1,6 @@
+{
+ "cells": [],
+ "metadata": {},
+ "nbformat": 4,
+ "nbformat_minor": 5
+}

README.md

@@ -1,13 +1,104 @@
 CUDA Denoiser For CUDA Path Tracer
 ==================================
 
-**University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 4**
+**University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 3**
 
-* (TODO) YOUR NAME HERE
-* Tested on: (TODO) Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab)
+* Zhangkaiwen Chu
+  * [LinkedIn](https://www.linkedin.com/in/zhangkaiwen-chu-b53060225/)
+* Tested on: Windows 10, R7-5800H @ 3.20GHz 16GB, RTX 3070 Laptop GPU 16310MB (Personal Laptop)
 
-### (TODO: Your README)
+This project implement a pathtracing denoiser that uses geometry buffers to guide a smoothing filter, which is based on the paper ["Edge-Avoiding A-Trous Wavelet Transform for fast Global Illumination Filtering"](https://jo.dreggn.org/home/2010_atrous.pdf). This paper use wavelet to approximmate Gaussion filter. By guiding the filter with edge-stopping function, the denoiser will not make the whole image blurry.
 
-*DO NOT* leave the README to the last minute! It is a crucial part of the
-project, and we will not be able to grade you without a good README.
+| raw pathtraced image | simple blur | blur guided by G-buffers |
+|---|---|---|
+|![](img/origin.png)|![](img/blur.png)|![](img/denoise.png)
+| SSIM = 0.5651 | SSIM = 0.8352 | SSIM = 0.9704 |
 
+The G-buffer contains position and surface normal informations.
+| scene | position | surface normal |
+|---|---|---|
+|![](img/scene.png)|![](img/position.png)|![](img/normal.png)|
+
+## Performance Analysis
+
+To measure the performance, a ground truth image is used to compare with denoised image. The ground truth image is generated by the pathtracer with 5000 iterations, and the metric is Structural Similarity Index (SSIM).
+
+
+### Runtime Analysis
+
+**Runtime with Different Resolutions**
+
+![](img/plot2.png)
+
+
+The denoiser is using a 80*80 filter size. As expected, the runtime of both the pathtracer and the denoiser scale linearly with the number of pixels when the number of pixels is not too small. When the number of pixels is small, other overheads may dominate the runtime. 
+
+**Runtime with Different Filter Size**
+![](img/plot3.png)
+
+The number of A-Trous iterations determines the actual filter size. The runtime scales linearly with the number of A-Trous iterations. This is because each iteration the work load of the A-Trous algorithm is the same.
+
+
+### Image Quality Analysis
+
+**Denoising with Different # of Iterations**
+
+| 1 iteration | position | surface normal |
+|---|---|---|
+|![](img/10itorigin.png)|![](img/1it.png)|![](img/10it.png)|
+| SSIM = 0.4597 | SSIM = 0.9668 | SSIM = 0.9704 |
+
+![](img/plot1.png)
+
+With a denoiser, the pathtracer can reach 0.97 SSIM with only 7 iterations, which needs about 1500 iterations without denoiser, so the denoiser can greatly reduce the number of iterations needed to get an acceptly smooth result. 
+
+**Denoising with Different Filter Size**
+
+| 0 A-Trous iteration |  1 A-Trous iteration |  2 A-Trous iteration |
+|---|---|---|
+|![](img/0.png)|![](img/1.png)|![](img/2.png)|
+| SSIM = 0.4597 | SSIM = 0.7744 | SSIM = 0.9261 |
+
+| 3 A-Trous iteration |  4 A-Trous iteration |  5 A-Trous iteration |
+|---|---|---|
+|![](img/3.png)|![](img/4.png)|![](img/5.png)|
+| SSIM = 0.9653 | SSIM = 0.9704 | SSIM = 0.9704 |
+
+The number of A-Trous iterations determines the actual filter size. The image quality didn't scale uniformly with the filter size. This is because when the filter size is small, increase the filter size will increase the number of pixels having similar color, position and normal greatly, while when the filter size is large, continue increasing the filter size will only add pixels far from the center to the filter, and these newly added pixels are likely to have different position, color anr surface norm, thus contribute little to the result.
+
+**Denoising with Different Scene**
+
+| cornell box with ceiling light | cornell box |
+|---|---|
+|![](img/denoise.png)|![](img/cornell_denoise.png)
+
+The denoising performance also greatly depend on scene. The result of denoising the original cornell box is apperantly worse than denoising the cornell box with ceiling light. With a smaller light, fewer number of rays can finally pass the light, so the image is much noisier with the same number of iterations. 
+
+**Denoising with Different Material**
+
+| Denoised Result|
+|---|
+|![](img/denoise.png)
+
+The denoiser is effective on difussive materials, and less effective on reflective materials, especially on reflective sphere. On difussive materials, the color is likely to be uniform, so the denoiser can work well. But on reflective materials, the color likely vary a lot among pixels. Also, the surface normal changes a lot among the pixels on sphere, so the weight on neighboring pixels will be low, thus the denoising will be less effective.
+
+### Compare with Gaussian Filter
+
+The generating kernel is used to approximate gaussian filters. By comparing the A-Trous filter and gaussian filter, we can know whether the approximation is successful.
+
+| A-Trous Filter | Gaussian Filter |
+|---|---|
+|![](img/denoise.png)|![](img/gaussian.png)
+| SSIM = 0.9704 | SSIM = 0.9559 |
+
+However, the gaussian filter have a worse performance with the same filter size as the A-Trous filter. Gaussian filter blurred the reflective sphere. A possible reason is the A-Trous filter have several iterations, and it uses the edge-stopping function at each iteration, so the edge-stopping function actually works for multiple times, and thus the weighted average is more accurate. Another hypothesis is there are still better parameters for Gaussian filter. I choose variance = 30, sigma_c = 5.41, sigma_n = 0.30, sigma_x = 4.94. By tuning variance and weights, we may achieve a better result. However, the runtime of Gaussian filter is not acceptable. With filtersize = 65, the runtime of Gaussian filter is 71ms, while the runtime of A-Trous filter is only 6.15ms. This is because the A-Trous filter's time complexity is O(nlog(k)), while the Gaussian filter's time complexity is O(nk^2), where n is the pixel count and k is the filter size. 
+
+### G-Buffer Optimization
+
+Given the depth, the camera position and the pixel index, we can calculate the position of the object. Thus, we only need to store the depth information in the G-buffer instead of the whole position. This [paper](http://jcgt.org/published/0003/02/01/paper.pdf) also provides a method to encode a 3-d normal vector with oct. With this two method, we can half the size of the G-buff. Here is the runtime
+
+|              | no optimization | optimize normal | optimize position | optimize both |
+|--------------|-----------------|-----------------|-------------------|---------------|
+| time elapsed (ms) | 6.15            | 6.258           | 6.104             | 6.517         |
+
+It shows that use depth to store the position information can reduce the runtime slightly, while oct-encode the normal will enlarge the runtime. This may because we didn't futher encode the oct-encoded normal to a single float. Also, the computation for oct-encoding and decoding is a little complex, so the increased computation time is larger than the saved I/O time.

scenes/cornell.txt

@@ -52,7 +52,7 @@ EMITTANCE   0
 CAMERA
 RES         800 800
 FOVY        45
-ITERATIONS  5000
+ITERATIONS  10
 DEPTH       8
 FILE        cornell
 EYE         0.0 5 10.5

scenes/cornell_ceiling_light.txt

@@ -52,7 +52,7 @@ EMITTANCE   0
 CAMERA
 RES         800 800
 FOVY        45
-ITERATIONS  10
+ITERATIONS  30
 DEPTH       8
 FILE        cornell
 EYE         0.0 5 10.5

src/main.cpp

@@ -45,6 +45,14 @@ int iteration;
 int width;
 int height;
 
+cudaEvent_t start;
+cudaEvent_t stop;
+
+float t;
+float t_denoise = 0;
+float t_pathtrace = 0;
+float count_denoise = 0;
+
 //-------------------------------
 //-------------MAIN--------------
 //-------------------------------
@@ -91,6 +99,9 @@ int main(int argc, char** argv) {
     // Initialize CUDA and GL components
     init();
 
+    cudaEventCreate(&start);
+    cudaEventCreate(&stop);
+
     // GLFW main loop
     mainLoop();
 
@@ -102,6 +113,8 @@ void saveImage() {
     // output image file
     image img(width, height);
 
+    update(ui_denoise);
+
     for (int x = 0; x < width; x++) {
         for (int y = 0; y < height; y++) {
             int index = x + (y * width);
@@ -150,6 +163,7 @@ void runCuda() {
     // No data is moved (Win & Linux). When mapped to CUDA, OpenGL should not use this buffer
 
     if (iteration == 0) {
+        t_pathtrace = 0.f;
         pathtraceFree();
         pathtraceInit(scene);
     }
@@ -162,12 +176,41 @@ void runCuda() {
 
         // execute the kernel
         int frame = 0;
+
+        cudaEventRecord(start);
+
         pathtrace(frame, iteration);
+
+
+
+        cudaEventRecord(stop);
+        cudaEventSynchronize(stop);
+        cudaEventElapsedTime(&t, start, stop);
+        t_pathtrace += t;
+        if (iteration == ui_iterations){
+            printf("pathtrace time:%f\n", t_pathtrace / iteration);
+        }
     }
 
-    if (ui_showGbuffer) {
+    if (ui_denoise) {
+        cudaEventRecord(start);
+
+        denoise(ui_colorWeight, ui_normalWeight, ui_positionWeight, ui_filterSize);
+
+        cudaEventRecord(stop);
+        cudaEventSynchronize(stop);
+        cudaEventElapsedTime(&t, start, stop);
+        t_denoise += t;
+        count_denoise++;
+
+        showDenoisedImage(pbo_dptr, iteration);
+    } else if (ui_showGbuffer) {
+      count_denoise = 0.f;
+      t_denoise = 0.f;
       showGBuffer(pbo_dptr);
     } else {
+      count_denoise = 0.f;
+      t_denoise = 0.f;
       showImage(pbo_dptr, iteration);
     }
 
@@ -192,6 +235,9 @@ void keyCallback(GLFWwindow* window, int key, int scancode, int action, int mods
       case GLFW_KEY_S:
         saveImage();
         break;
+      case GLFW_KEY_I:
+        printf("denoise count:%f,denoise time:%f\n", count_denoise,t_denoise / count_denoise);
+        break;
       case GLFW_KEY_SPACE:
         camchanged = true;
         renderState = &scene->state;
@@ -203,13 +249,15 @@ void keyCallback(GLFWwindow* window, int key, int scancode, int action, int mods
 }
 
 void mouseButtonCallback(GLFWwindow* window, int button, int action, int mods) {
+    return;
   if (ImGui::GetIO().WantCaptureMouse) return;
   leftMousePressed = (button == GLFW_MOUSE_BUTTON_LEFT && action == GLFW_PRESS);
   rightMousePressed = (button == GLFW_MOUSE_BUTTON_RIGHT && action == GLFW_PRESS);
   middleMousePressed = (button == GLFW_MOUSE_BUTTON_MIDDLE && action == GLFW_PRESS);
 }
 
 void mousePositionCallback(GLFWwindow* window, double xpos, double ypos) {
+    return;
   if (xpos == lastX || ypos == lastY) return; // otherwise, clicking back into window causes re-start
   if (leftMousePressed) {
     // compute new camera parameters