Contributor
Prasanna Rangarajan, Murali Balaji Madabhushi
Subject Area
Computer Engineering, Computer Science, Neuroscience
Abstract
This thesis explores the potential of Spiking Neural Networks (SNNs) in processing event sensor data and generating high-fidelity activity maps. Event sensors capture asynchronous binary events with high dynamic range, but traditional processing methods often fail to leverage their advantages fully. SNNs, with their asynchronous, event-driven nature, offer a promising alternative.
A Spiking Autoencoder (SAE) was employed in this thesis to handle the stochastic and sparse event data, integrating deep dictionary learning to enhance the feature space and improve activity map quality. The encoder, modeled after the VGG network, extracts features from event streams generated by speckle patterns, which are decoded and reconstructed within a dictionary learning framework.
Various loss functions/smoothness penalties—Sobel, Laplace, Total Variation, and Gradient Magnitude—were tested on different datasets accumulated over two temporal resolutions. Results showed that incorporating smoothness penalties improved visual quality but increased training time. Higher temporal resolution data presented challenges in feature definition and artifact removal, while lower resolution data produced clearer images but struggled with artifacts.
To test the versatility of our network, we examined its performance on different embedded objects: a music note, power symbol, location symbol, and speaker symbol. These tests revealed the network's ability to handle straight lines and rounded shapes effectively but also highlighted difficulties in distinguishing multiple objects and reconstructing complex shapes. Sobel loss was particularly adaptable across different feature sets.
Quantitative analyses revealed that filter-based methods, particularly Sobel and Laplace losses, outperformed gradient approximation methods in fidelity and structural quality. However, gradient methods offered faster convergence, making them suitable for speed-critical applications.
Our findings demonstrate advancements in using SNNs for event-based data processing. Integrating dictionary learning into SAE architectures enhances feature space and provides a robust mechanism for handling high-dimensional event data. This thesis highlights the potential of fully-spiking architectures in processing and generating high-quality outputs from event-based sensor data, paving the way for innovations in neuromorphic engineering.
Degree Date
Summer 8-6-2024
Document Type
Thesis
Degree Name
M.S.
Department
Electrical and Computer Engineering
Advisor
Prasanna Rangarajan
Second Advisor
Joseph Camp
Third Advisor
Carlos Davila
Number of Pages
46
Format
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial 4.0 License
Recommended Citation
easton, jack, "Activity Map Generation and Event-Based Sensor Processing with Spiking Autoencoders and Sparse Dictionary Learning" (2024). Computer Science and Engineering Theses and Dissertations. 40.
https://scholar.smu.edu/engineering_compsci_etds/40