Project 5: Liam Dugan

README.md

@@ -3,26 +3,65 @@ WebGL Clustered and Forward+ Shading
 
 **University of Pennsylvania, CIS 565: GPU Programming and Architecture, Project 5**
 
-* (TODO) YOUR NAME HERE
-* Tested on: (TODO) **Google Chrome 222.2** on
-  Windows 22, i7-2222 @ 2.22GHz 22GB, GTX 222 222MB (Moore 2222 Lab)
+* Liam Dugan -- Fall 2018
+  * [LinkedIn](https://www.linkedin.com/in/liam-dugan-95a961135/), [personal website](http://liamdugan.com/)
+* Tested on: **Google Chrome 71.0** Windows 10, Intel(R) Xeon(R) CPU E5-2687W v3 @ 3.10GHz 32GB, TITAN V 28.4GB (Lab Computer)
+
+
+![](images/final.gif)
 
 ### Live Online
 
-[![](img/thumb.png)](http://TODO.github.io/Project5B-WebGL-Deferred-Shading)
+[Click here to see the project in action!](http://liamdugan.github.io/Project5-WebGL-Clustered-Deferred-Forward-Plus)
+
+What are Forward+ and Clustered Rendering?
+=============
+
+Forward+ and Clustered are special rendering techniques used to optimize lighting calculations in scenes with many moving light sources.
+
+![](images/MoneyShot.bmp)
+
+Usually, the way light calculations are handled is to loop over all lights in the scene for each fragment 
+(i.e. pixel) and calculate the effect each and every light has on the pixel for every frame.
+This is clearly not the most efficient way, since there will be many distant lights with little to no effect
+on the pixel taking up valuable processing time.
+
+**Forward+ Rendering** (Or sometimes called **Tiled Rendering**), splits the screen up into individual tiles and keeps track of a 
+data structure that denotes which lights are within which tiles. 
+
+<img align="center" src="images/9-Forward-Plus.png">
+
+This data structure is then passed to the GPU so that each pixel only must loop over the lights in their specific
+cluster and not all lights in the scene.
+
+**Deferred Clustered Rendering** takes this one step further and involves the depth of each fragment as well to further 
+cull unnecessary light sources. In addition, in order to avoid having to compute the fragment transforms multiple times, the rendering is broken up into two stages. The first stage computes the vertex transforms and stores all of the 
+data into a G-buffer, then the second stage uses that information and the cluster data structure to compute the lighting.
+
+Performance Analysis
+================
+
+As we can see from the graph below, the performance data shows that for our small scene, the simpler algorithm is best
+even at a somewhat large number of lights. This may also be due to the power and memory capability of the GPU that I tested on.
+I tested on a TITAN V, which has plenty of computation capability and so the lack of memory reads that the 
+Foward rendering technique had probably gave it a very large advantage over the very read/write heavy Forward+ and Clustered.
 
-### Demo Video/GIF
+<img align="center" src="images/Analysis1.png">
 
-[![](img/video.png)](TODO)
+To help combat this bottleneck I implemented a packing optimization to reduce the total number of G-buffer 
+spaces used, but it only had a very minor effect towards the end.
 
-### (TODO: Your README)
+<img align="center" src="images/Analysis2.png">
 
-*DO NOT* leave the README to the last minute! It is a crucial part of the
-project, and we will not be able to grade you without a good README.
+One of the other main reasons why I suspect that my code is running slow is the lack of precision when 
+checking sphere frustum intersections. I check if the center of the light is within frustum orthocenter to corner + light radius units 
+of the frustum center, which is simply an approximation of the actual intersection algorithm (which involves checking for 6 planes).
 
-This assignment has a considerable amount of performance analysis compared
-to implementation work. Complete the implementation early to leave time!
+Since the GPU is much better at handling computational work than memory reads and since the lack of precision likely amounted to 
+a larger cluster to light data structure, this would slow the algorithm down a lot since it has to read and write more data. 
 
+I suspect that if I were to have implemented this optimization then we would see the Clustered rendering perform well on very closed but long scenes  with many light sources and
+we would see the forward+ rendering work well for the more open and far away scenes with many light sources. (Forward would of course do better on scenes with fewer lights).
 
 ### Credits
 

src/init.js

@@ -1,5 +1,5 @@
 // TODO: Change this to enable / disable debug mode
-export const DEBUG = true && process.env.NODE_ENV === 'development';
+export const DEBUG = false && process.env.NODE_ENV === 'development';
 
 import DAT from 'dat.gui';
 import WebGLDebug from 'webgl-debug';
@@ -16,7 +16,7 @@ export function abort(message) {
 
 // Get the canvas element
 export const canvas = document.getElementById('canvas');
-
+ 
 // Initialize the WebGL context
 const glContext = canvas.getContext('webgl');
 
@@ -60,7 +60,7 @@ stats.domElement.style.top = '0px';
 document.body.appendChild(stats.domElement);
 
 // Initialize camera
-export const camera = new PerspectiveCamera(75, canvas.clientWidth / canvas.clientHeight, 0.1, 1000);
+export const camera = new PerspectiveCamera(75, canvas.clientWidth / canvas.clientHeight, 0.1, 100);
 
 // Initialize camera controls
 export const cameraControls = new OrbitControls(camera, canvas);

src/renderers/base.js

@@ -1,6 +1,7 @@
 import TextureBuffer from './textureBuffer';
 
 export const MAX_LIGHTS_PER_CLUSTER = 100;
+export const PI = 3.14159265359;
 
 export default class BaseRenderer {
   constructor(xSlices, ySlices, zSlices) {
@@ -15,16 +16,71 @@ export default class BaseRenderer {
     // TODO: Update the cluster texture with the count and indices of the lights in each cluster
     // This will take some time. The math is nontrivial...
 
+    // view matrix takes things in view space and transforms them into world space 
+    // now the question is: are the lights in view space?
+
+    // okay so we know that the lights are in world space. Now the question is: 
+    // how large are these slices (especially the z one). We can assume the 
     for (let z = 0; z < this._zSlices; ++z) {
       for (let y = 0; y < this._ySlices; ++y) {
         for (let x = 0; x < this._xSlices; ++x) {
           let i = x + y * this._xSlices + z * this._xSlices * this._ySlices;
+
+          // number of lights in the cluster starts out as 0
+          let numLightsInCluster = 0;
+
+          // get the position of the cluster's center in camera space
+          // as well as the height width and depth
+          let fovRadians = camera.fov * (Math.PI / 180.0); 
+          let frustumCenterZ = ((z + 0.5)  / this._zSlices) * (camera.far - camera.near);
+
+          let frustumDepth = (camera.far - camera.near) / this._zSlices; 
+          let screenHeight = 2.0 * (frustumCenterZ * Math.tan(fovRadians / 2));
+          let screenWidth = screenHeight * camera.aspect;
+          let frustumHeight = screenHeight / this._ySlices;
+          let frustumWidth = screenWidth / this._xSlices;
+
+          let frustumCenterY = (y + 0.5) * frustumHeight;
+          let frustumCenterX = (x + 0.5) * frustumWidth;
+
+          let frustumDiagonalLength = Math.sqrt(Math.pow((frustumWidth / 2.0), 2) + 
+                                     Math.pow((frustumHeight / 2.0), 2) + 
+                                     Math.pow((frustumDepth / 2.0), 2));
+
+          // now we loop through each light and see if it's within the cluster (using the view matrix)
+          for (let j = 0; j < scene.lights.length; ++j) {
+
+            // TODO: Get the w for perspective divide***
+            // get the light's position in camera space by *manually* multiplying the view matrix *sigh*
+            let lightCenterX = viewMatrix[0] * scene.lights[j].position[0] +
+                              viewMatrix[4] * scene.lights[j].position[1] +
+                              viewMatrix[8] * scene.lights[j].position[2] +
+                              viewMatrix[12] * 1;
+            let lightCenterY = viewMatrix[1] * scene.lights[j].position[0] +
+                              viewMatrix[5] * scene.lights[j].position[1] +
+                              viewMatrix[9] * scene.lights[j].position[2] +
+                              viewMatrix[13] * 1;
+            let lightCenterZ = viewMatrix[2] * scene.lights[j].position[0] +
+                              viewMatrix[6] * scene.lights[j].position[1] +
+                              viewMatrix[10] * scene.lights[j].position[2] +
+                              viewMatrix[14] * 1;
+
+            let distance = Math.sqrt(Math.pow(frustumCenterX - lightCenterX, 2) + 
+                                    Math.pow(frustumCenterY - lightCenterY, 2));
+
+            // If the sphere intersects the cluster 
+            if (distance < (scene.lights[j].radius*10 + frustumDiagonalLength) && numLightsInCluster < MAX_LIGHTS_PER_CLUSTER)
+            {
+              numLightsInCluster += 1;
+              this._clusterTexture.buffer[this._clusterTexture.bufferIndex(i, Math.floor(numLightsInCluster/4)) + (numLightsInCluster % 4)] = j;
+            } 
+          }
+
           // Reset the light count to 0 for every cluster
-          this._clusterTexture.buffer[this._clusterTexture.bufferIndex(i, 0)] = 0;
+          this._clusterTexture.buffer[this._clusterTexture.bufferIndex(i, 0)] = numLightsInCluster;        
         }
       }
     }
-
     this._clusterTexture.update();
   }
 }

src/renderers/clustered.js

@@ -9,7 +9,7 @@ import fsSource from '../shaders/deferred.frag.glsl.js';
 import TextureBuffer from './textureBuffer';
 import BaseRenderer from './base';
 
-export const NUM_GBUFFERS = 4;
+export const NUM_GBUFFERS = 3;
 
 export default class ClusteredRenderer extends BaseRenderer {
   constructor(xSlices, ySlices, zSlices) {
@@ -29,8 +29,9 @@ export default class ClusteredRenderer extends BaseRenderer {
       numLights: NUM_LIGHTS,
       numGBuffers: NUM_GBUFFERS,
     }), {
-      uniforms: ['u_gbuffers[0]', 'u_gbuffers[1]', 'u_gbuffers[2]', 'u_gbuffers[3]'],
-      attribs: ['a_uv'],
+      uniforms: ['u_viewProjectionMatrix', 'u_lightbuffer', 'u_gbuffers[0]', 'u_gbuffers[1]', 'u_gbuffers[2]','u_clusterbuffer', 
+      'u_screenWidth', 'u_screenHeight', 'u_far', 'u_near', 'u_xSlices', 'u_ySlices', 'u_zSlices', 'u_cameraPos'],
+      attribs: ['a_position'],
     });
 
     this._projectionMatrix = mat4.create();
@@ -81,7 +82,6 @@ export default class ClusteredRenderer extends BaseRenderer {
     // Tell the WEBGL_draw_buffers extension which FBO attachments are
     // being used. (This extension allows for multiple render targets.)
     WEBGL_draw_buffers.drawBuffersWEBGL(attachments);
-
     gl.bindFramebuffer(gl.FRAMEBUFFER, null);
   }
 
@@ -102,7 +102,7 @@ export default class ClusteredRenderer extends BaseRenderer {
     if (canvas.width != this._width || canvas.height != this._height) {
       this.resize(canvas.width, canvas.height);
     }
-
+ 
     // Update the camera matrices
     camera.updateMatrixWorld();
     mat4.invert(this._viewMatrix, camera.matrixWorld.elements);
@@ -127,6 +127,9 @@ export default class ClusteredRenderer extends BaseRenderer {
     // Draw the scene. This function takes the shader program so that the model's textures can be bound to the right inputs
     scene.draw(this._progCopy);
 
+    // Update the clusters for the frame
+    this.updateClusters(camera, this._viewMatrix, scene);
+
     // Update the buffer used to populate the texture packed with light data
     for (let i = 0; i < NUM_LIGHTS; ++i) {
       this._lightTexture.buffer[this._lightTexture.bufferIndex(i, 0) + 0] = scene.lights[i].position[0];
@@ -141,9 +144,6 @@ export default class ClusteredRenderer extends BaseRenderer {
     // Update the light texture
     this._lightTexture.update();
 
-    // Update the clusters for the frame
-    this.updateClusters(camera, this._viewMatrix, scene);
-
     // Bind the default null framebuffer which is the screen
     gl.bindFramebuffer(gl.FRAMEBUFFER, null);
 
@@ -155,8 +155,29 @@ export default class ClusteredRenderer extends BaseRenderer {
 
     // TODO: Bind any other shader inputs
 
+    gl.activeTexture(gl.TEXTURE0);
+    gl.bindTexture(gl.TEXTURE_2D, this._lightTexture.glTexture);
+    gl.uniform1i(this._progShade.u_lightbuffer, 0);
+
+    // Set the cluster texture as a uniform input to the shader
+    gl.activeTexture(gl.TEXTURE1);
+    gl.bindTexture(gl.TEXTURE_2D, this._clusterTexture.glTexture);
+    gl.uniform1i(this._progShade.u_clusterbuffer, 1);
+
+    gl.uniformMatrix4fv(this._progShade.u_viewProjectionMatrix, false, this._viewProjectionMatrix);
+
+    gl.uniform1f(this._progShade.u_far, camera.far);
+    gl.uniform1f(this._progShade.u_near, camera.near);
+    gl.uniform1f(this._progShade.u_screenHeight, canvas.height);
+    gl.uniform1f(this._progShade.u_screenWidth, canvas.width);
+    gl.uniform1f(this._progShade.u_xSlices, this._xSlices);
+    gl.uniform1f(this._progShade.u_ySlices, this._ySlices);
+    gl.uniform1f(this._progShade.u_zSlices, this._zSlices);
+    gl.uniform3f(this._progShade.u_cameraPos, camera.position.x, camera.position.y, camera.position.z);
+
+
     // Bind g-buffers
-    const firstGBufferBinding = 0; // You may have to change this if you use other texture slots
+    const firstGBufferBinding = 2; // You may have to change this if you use other texture slots
     for (let i = 0; i < NUM_GBUFFERS; i++) {
       gl.activeTexture(gl[`TEXTURE${i + firstGBufferBinding}`]);
       gl.bindTexture(gl.TEXTURE_2D, this._gbuffers[i]);

src/renderers/forwardPlus.js

@@ -13,11 +13,11 @@ export default class ForwardPlusRenderer extends BaseRenderer {
 
     // Create a texture to store light data
     this._lightTexture = new TextureBuffer(NUM_LIGHTS, 8);
-
     this._shaderProgram = loadShaderProgram(vsSource, fsSource({
       numLights: NUM_LIGHTS,
     }), {
-      uniforms: ['u_viewProjectionMatrix', 'u_colmap', 'u_normap', 'u_lightbuffer', 'u_clusterbuffer'],
+      uniforms: ['u_viewProjectionMatrix', 'u_colmap', 'u_normap', 'u_lightbuffer', 'u_clusterbuffer', 
+      'u_screenWidth', 'u_screenHeight', 'u_far', 'u_near', 'u_xSlices', 'u_ySlices', 'u_zSlices'],
       attribs: ['a_position', 'a_normal', 'a_uv'],
     });
 
@@ -76,6 +76,13 @@ export default class ForwardPlusRenderer extends BaseRenderer {
     gl.uniform1i(this._shaderProgram.u_clusterbuffer, 3);
 
     // TODO: Bind any other shader inputs
+    gl.uniform1f(this._shaderProgram.u_far, camera.far);
+    gl.uniform1f(this._shaderProgram.u_near, camera.near);
+    gl.uniform1f(this._shaderProgram.u_screenHeight, canvas.height);
+    gl.uniform1f(this._shaderProgram.u_screenWidth, canvas.width);
+    gl.uniform1f(this._shaderProgram.u_xSlices, this._xSlices);
+    gl.uniform1f(this._shaderProgram.u_ySlices, this._ySlices);
+    gl.uniform1f(this._shaderProgram.u_zSlices, this._zSlices);
 
     // Draw the scene. This function takes the shader program so that the model's textures can be bound to the right inputs
     scene.draw(this._shaderProgram);