The final project for my Deep Learning course completed in collaboration with Ben Fisk.

This project revolved around implementing Generative Adversarial Networks (GANs) to produce synthetic images which look visually similar to the training data. GANs are a type of generative deep model which aim to replicate training data by simultaneously training a generator and a discriminator. The two components of the GAN play a two-player mini-max game where the generator aims to trick the discriminator into believing its generated examples are part of the real dataset while the discriminator is trying to maximize its ability to correctly distinguish between real and fake data. For more information on how GANs work, I recommend reading the original paper by Goodfellow et. al. (2014).

Image stitching is the process of combining several images of a scene taken from approximately the same viewpoint but at different angles to create a larger resulting image. One of the underlying assumptions of this process is that the fields of view of the images to be stitched slightly overlap. This assumption allows for the detection of common features within the images. The relative positions of the features allows for the calculation of a matrix transformation which describes how to warp one image to fit the coordinate system of another. Once the coordinate transformation is determined, the images can be warped, combined, blended together, and then cropped to produce a larger, panoramic image of the scene.

A simple game made with Unity attempting to recreate the Wii Play Tanks mini game as part of the final project of my Game Design and Development course completed in just a couple days.

Available to play online here.

Use WASD/Arrow keys to move your tank. Point and shoot down all enemy tanks to progress to the next level. There are 5 levels in total. Note: you’ll need to turn on sound, enter full screen mode, then click to play! The game was not made to originally be published online so the WebGL version lacks some of the lighting of the original and seems to have floaty controls.

The following are a few of the most interesting projects completed for the Intro to Computer Vision course which implement several basic computer vision tasks. The results seen in each of these reports were produced by implementing the relevant algorithms from scratch using NumPy. OpenCV was only used for loading images or drawing on top of images.

Canny Edge Detection

The purpose of this project was to implement the Canny Edge detection algorithm which consists of 5 main steps: gaussian smoothing, calculating the image gradient, selecting thresholds for the gradient magnitude, thinning the gradient magnitudes by suppressing non-maxima, and finally, linking strong edges along weaker edges.

This was the final project for a Computational Optics seminar completed in collaboration with Claire Paré. Made in a jupyter notebook with numpy and matplotlib, the first aim of this project was to create a simplified version of this 2D ray-optics simulator. The initial goal was to show the paths of light rays through transparent spheres with various indices of refraction and accurately model Fresnel effects to showcase how the amount of reflected and refracted light depends on the index of refraction and incoming ray angle. Next, we implemented continuous refraction, showcasing the bent paths light rays take as they travel through objects with spatially varying refractive indices. Our implementation of this was based off the paper Path Tracing Estimators for Refractive Radiative Transfer by Pediredle et. al., from whom we also got the idea of modelling Luneberg lenses.

This project was part of an independent study where I, along with a few other students, were extending a pre-existing ray tracer made by a PhD student, Kelly Jiang. What follows is the report I submitted for the project that runs through the different camera systems I implemented with a brief explainer and example results of each.

Introduction

Ray tracing has the unique advantage of easily simulating many different types of cameras. All we need is to provide the appropriate outgoing ray from the camera system and let the ray tracer take care of the rest. The ability to simulate different types of cameras opens up many artistic possibilities when making rendered images or animations. For instance, one of my favorite artistic uses of a realistic camera system can be seen in Toy Story 4. In this movie they modeled several different types of cameras including the simpler spherical lens camera (which are discussed later in this report) as well as more complex split-diopter (fig. 1) and anamorphic lenses. The YouTube channel Nerdwriter has a good explainer on the artistic impact of these lenses in this video. Ofcourse, realistic camera simulations can also be useful for research. For example, the Dahl lab at Northwestern is currently in the process of simulating images from their bubble chamber. The cameras peer into the bubble chamber through a series of relay lenses which need to be accurately simulated for correct results.