Possible Computing RET Projects
Research Project Catalog:
Representative research projects related to the three threads described in Figure 1, below (technical organization of the RET site, image sources noted in figure) are summarized below. (Others are possible as well.) Note that as part of the RET experience, participants will develop materials to be integrated into high school classes that are based on this research – yet that are suitable for high school students.
THREAD #1: Information processing via neural networks on CPUs, GPUs
Deep learning for visual object recognition:
This project incorporates elements of computer vision, machine learning, and neuroscience. An ultimate goal might be to expose students to the basics of visual processing for object recognition. The mammalian visual ventral stream (the part of the brain responsible for vision) is organized as a hierarchy of areas, each constructing increasingly complex representations of a stimulus observed by the retina. Decisions about the nature of the stimulus are made at the highest levels of the hierarchy. Emerging models (like convolutional neural networks from the research area of deep learning) mimic this process to achieve excellent performance on real-world computer vision tasks – e.g., like identifying people in social media photos. By learning different network parameters, for a host of different network architectures, and from large collections of labeled images from the Internet, we can work to teach a computer to understand images like humans do. This project will likely incorporate the basics of GPU programming and vector programming on conventional CPUs to simulate deep networks.
HS biology teachers can use this knowledge to describe the concept of vision, and simulate basic models in the classroom that can recognize objects in real-life scenes. In artificial neural networks, HS computer science and math teachers will find a straightforward algorithm for introducing the concepts of statistical parameter estimation, experimental error rates, and learning from data.
Searching and retrieving data:
New information is being produced and curated at an increasing rate, and as a result, economies and industries will depend heavily on the effective storage and retrieval of that information. Teachers will work closely with faculty and graduate students to learn how search engines like Google, Bing and Yahoo perform keyword searches. To that end, they will improve their understanding on how terms and documents are represented and compared digitally, how massive storage systems quickly retrieve results, and how everyday users naturally parse and refine complex questions into a handful of key words using models created by recurrent and convolutional neural networks. In addition, teachers will be able to apply these techniques to create innovative science fair projects that encourage students to independently investigate specific tasks in computing. As such, lessons gained from the RET experience could also be used to mentor science fair projects in computing at respective high schools.
Recurrent neural networks (RNNs):
At present, there are many types of neural networks being studied/developed to perform various image processing tasks, etc. One type of network – a recurrent neural network (or RNN) has been shown to be highly effective for processing natural language text, medical images, computer code, and music. However, the nature of the memory that these networks possess is still poorly understood. Participants will study several variants of RNNs to analyze, visualize, and understand their inner-workings to accomplish application-level tasks like those described above. They will also compare RNN-based solutions to other algorithms that might address similar problems (i.e., to understand tradeoffs over metrics like computational latency, energy, and accuracy), and work to understand how human beings might accomplish similar tasks. Teachers will have the opportunity to consider said networks in the context of the Python programming language.
Understanding wireless network performance in crowded venues with deep learning:
One of the most challenging aspects for wireless networks is scaling when there are large numbers of people in close proximity to one another. For anyone who has tried to use cellular data in a crowded area or even casually browsing data via WiFi in a busy coffee shop, the results are often unsatisfactory. Over the past year, researchers at Notre Dame have been gathering wireless network performance data from a wide cross-section of students and individuals with regards to network performance. The challenge now is to “make” sense of the data and to determine the extent to which hidden patterns with respect to network performance might be present within our large dataset. The focus of this RET is to apply techniques such as deep learning and other biologically-inspired algorithms to analyze this large corpus of data.
THREAD #2: Non-Boolean, neuromorphic architectures
Cellular neural networks:
Notre Dame faculty are targeting hardware realizations of analog computing elements called cellular neural networks (CeNNs). A CeNN is a spatially parallel computing paradigm consisting of identical processing elements (which again, are often analog in nature) that are connected to their nearest neighbors. CeNNs can significantly improve both the power and performance of various computation-intensive information processing applications – e.g., image processing, pattern recognition, etc. Quantitatively, for complex, two-dimensional image processing functions, a CeNN-based processor with an area of 1.4 cm2 and a power budget of just 4.5 W could match the performance of an IBM supercomputer (i.e., traditional programmable hardware) – with an area of ~7 m2 and a power budget of 491 kW. Teachers working on this project will not just learn about CeNN fundamentals, but will also consider how new devices could lead to improved CeNNs, how CeNNs can be employed to solve complex problems (e.g,. targeting tracking, digit recognition, etc.), and will benchmark CeNN-based computing against more conventional algorithms for common problems.
(For an example of the capabilities of a CeNN for hand-written digit recognition, please see the video at
https://www.youtube.com/watch?v=BTXx11HKlac. Note that the camera in this video includes a CeNN chip that performs computations to classify the written digit – see computer-based readout in lower left hand corner – “class” refers to the classification decision.)
For this project, there are natural connections to existing HS physics courses (e.g., current/voltage/resistance), digital electronics courses, as well as mathematics courses.
Cellular neural network-inspired diffusion networks:
As present, ND faculty are working to develop CeNN-inspired architectures, specifically diffusion networks based on emerging transistor technologies. Diffusion networks (see Figure 2a) are comprised of a grid of resistors and capacitors, and the diffusion process can be used to approximate a Gaussian smoothing filter that is widely used in image processing. This may be for something as simple as a de-noising operation that precedes a task such as edge detection (see Figure 2b), or hardware diffusion kernels can play an integral role in CeNN-based target tracking – see previous project description.) Research has shown that diffusion networks based on emerging transistor technologies could improve device count, power, and processing time by at least an order of magnitude when compared to hardware implementations based on conventional CMOS technology. If diffusion operations comprise the bulk of a given image processing task, the potential benefits could be substantial. Teachers working on this task will have the opportunity to design and simulate hardware based on new technologies, will study how to “program” diffusion-based tasks for a given image processing task, and/or perform design space explorations to determine how such hardware could accelerate a given computation. As one can ultimately think about a diffusion network as a resistive-capacitive network, there are natural connections to HS physics and digital electronics courses.
Brain-inspired spiking neural networks:
At present, researchers at Notre Dame are working to solve various medical imaging problems (i.e., identifying and classifying disease/cancer cells for diagnosis). Algorithmic solutions have been based on various types of neural networks – e.g., fully convolutional networks, recurrent neural networks, etc. There is also interest in mapping said problems to brain-inspired spiking neural networks – i.e., where information is encoded in spatial-temporal streams of spikes. These “spike trains” are then used to process information, perform image recognition tasks, etc. Moreover, recently, IBM introduced a new chip (TrueNorth) that realizes a digital spiking neural network (see http://www.research.ibm.com/articles/brain-chip.shtml for more details). Notre Dame researchers are currently working with IBM researchers to consider said mappings. A participant in this project will have the opportunity to explore how different types of neural networks impact three important design metrics for problems of interest: energy, delay, and accuracy.
THREAD #3: Physics-inspired information processing
Optically-inspired computing paradigms:
This project considers optically-inspired computational hardware that is realized via magnetic (or “spin”)-based devices. Spinwaves are magnetic excitations in magnetic micro- and nano-structures. They have a small wavelength (10 nm—100 nm), enabling dense integration, and they can be created in a straightforward manner on a microprocessor chip. They can also be converted back to the electrical domain, which suggests that magnetic hardware is compatible with conventional electronics. For small amplitudes, they behave as linear waves, and for larger amplitudes, nonlinearities appear, which may also be exploited. By properly configuring a spinwave-based system (Figure 3), different types of computational kernels can be realized. For example, a well-known and often exploited property of the optical lens is that it can perform a Fourier transform (FT). Additional spintronic devices situated on top of the film may provide opportunities for reconfigurability – i.e., to implement different lenses. A wavefront generated at the inputs undergoes propagation and interference, and can be read out by magnetoresistive structures.
This work is particularly amenable to a research experience for HS physics teachers – where a teacher may simulate a magnetic system to realize other computationally interesting functions (i.e., besides a FT), quantify any errors that may occur in a spin-based computation (as compared to an all-digital approach), etc. As both optical phenomena and magnetics are integral parts of the HS physics curriculum, curricular elements from the RET will allow students to learn about new applications of fundamental physical principles. There are also opportunities for cross-linkages to additional projects (i.e., the computing in music project) as the mathematics associated with computational kernels for signal processing might also be accomplished via physical systems like those described above. Opportunities to study tradeoffs between potentially lower energy (but also lower precision) optically inspired computations vs. higher energy (but more higher precision) digital computations are of interest.
Computing with coupled oscillators:
The goal of this project is to study the collective dynamics of physical devices, and investigate non-Boolean information processing architectures that fundamentally embrace the “let the physics do the computing” paradigm. Participants will explore the fundamental limits of computational complexity that are associated with such devices, and consider how practical issues such as oscillation frequency, synchronization latency, inherent device-to-device variability, etc. can impact an application-level task. Note that such systems might ultimately be capable of providing solutions to problems that are deemed to be “computationally hard” – i.e., they cannot be solved in polynomial time (where time is a polynomial function of input size), and typically require exponential time. This makes addressing large input sizes – which are typically representative of the problems of greatest interest – quite challenging. Examples of such problems include assigning colors to nodes of a graph such that two connected nodes do not have the same color. (This problem has direct applicability to various resource allocation problems – e.g., airline scheduling, etc.) While there is obvious applicability to physics-based courses, there are also direct linkages to various math courses. As an example, one might have students determine what polynomial function, exponential function, etc. describes the complexity of a problem, and then use these expressions to compare run times, etc.
THREAD #4: Cross-cutting projects
Computing and music:
Participants in this project will learn the basics of how to use digital signal processing to generate and manipulate sound waves to produce music. The project will begin by using the Mozzi sound synthesis library for Arduino, and then look at how similar operations may be performed more efficiently by a custom processor using a field programmable gate array (FPGA). Participants will also learn how custom musical instrument enclosures can be fabricated using either 3D printing or laser cutting. Applications of this research were used in the production of the musical work "Wild Sound," composed by Glenn Kotche of the band Wilco for Notre Dame artists-in-residence Third Coast Percussion, which has been performed at venues in the U.S. and Europe including the NY Metropolitan Museum of Art, St. Paul Chamber Orchestra, and SF Jazz Center.
Applying deep-learning to robotics:
There is growing interest in applying deep learning techniques to robotics applications. This project will consider learning visual-motor-tactile sensory robotic processes (visual servo-ing) using deep neural networks. Participants will investigate how to train a robotic system to grasp different kind of objects.
Natural language generation for software documentation & accessibility techniques:
The objective of this project is to automatically generate English descriptions of software source code – e.g., to create comments for source code that does not already have comments in it. The descriptions have two uses: one, to include with software to help programmers understand code, and two, as tools to help visually impaired programmers and students navigate code more easily. The strategy that the project will follow is: 1) perform empirical study to determine the process that human programmers use to write descriptions, 2) design algorithms that mimic a portion of that process, and 3) evaluate the effectiveness of those algorithms. Teachers will work alongside Notre Dame faculty and graduate students. [Note that we are still working to determine if this project will be offered. However, if you are interested in it, please make note of this in your application.]