Research

“Little strokes fell great oaks.”
- Benjamin Franklin

My research has wandered from machine learning applications in neuroscience toward the theory and methodology of deep learning. Across both areas, I’m interested in how learning systems pick up useful patterns, and when those patterns carry over to new people, data, or tasks.

See my publication list for papers, preprints, presentations, and datasets, with filters by topic and type.

At Penn, I worked with electroencephalography (EEG)—electrical recordings of brain activity—and used machine learning to ask whether brain signals could help predict behavior, like whether someone would remember an item. We’re still a ways off from mind-reading, but it’s pretty cool stuff. These days I’m interested in why deep learning works so well, where it runs into trouble, and whether a better theoretical understanding can help us build models that are more reliable and interpretable.

Theory of deep learning

Right now I’m especially interested in how training procedures shape the solutions neural networks learn, and what those solutions can tell us about generalization.

Inductive bias / implicit regularization

Paper: Estimating Implicit Regularization in Deep Learning

One possible explanation for why neural networks generalize well is that they have some kind of inductive bias that encourages them to learn generalizing solutions (perhaps e.g. a simplicity bias that prevents overfitting). Lots of theory has been devoted to studying how our training methods (e.g. stochastic gradient descent) implicitly regularize models’ effective loss landscape such a way.

The animation shows what happens to a simple two-parameter model as we turn up \(\ell_2\) regularization. The red surface is the original loss, the blue contours show the regularizer, and the gray surface is the combined objective. As the regularization gets stronger, the minimum moves toward the point preferred by the regularizer.

The new solution isn’t the same as the original one: regularization nudges it to a different point in parameter space. I’m interested in using shifts like this to reverse-engineer the inductive biases that training introduces.

EEG analysis and machine learning applications

Working towards foundation models for neural data

For my master’s thesis, I trained deep neural networks to predict behavior (especially memory) from neural data collected across different people. This is a kind of transfer learning called domain adaptation: the hope is that a model can pick up patterns shared across brains and use them to make predictions for someone it hasn’t seen before.

The work received a departmental award, though we didn’t end up with a paper. Since then, similar approaches have shown promise for tasks with clearer neural signals than memory, like motor activity, sleep stages, and stress. I’m not working on this now, but I still think it’s a really promising direction.

Decoding brain states and improving memory

Paper: Decoding EEG for optimizing naturalistic memory, Journal of Neuroscience Methods

• In this project we asked whether using machine learning to optimize the timing of item presentations during learning could improve memory performance. Presented as a poster at the Cognitive Neuroscience Society (CNS) annual meeting, Context and Episodic Memory Symposium, and MathPsych in spring/summer 2022.

Oscillatory biomarkers of memory

Paper: Hippocampal theta and episodic memory, Journal of Neuroscience

• I investigate how a method of distinguishing pink noise in brain recordings from true brain rhythms helps us understand what patterns of brain activity actually relate to successful memory encoding and retrieval. Presented at the Context and Episodic Memory Symposium in August 2021 and Computational and Systems Neuroscience (COSYNE) in March 2022.

EEG pre-processing methods

Undergraduate research project: Optimal EEG Referencing Schemes for Brain State Classification. The project compares reference schemes for EEG electrodes, explains how they act as spatial filters, and evaluates their utility for classifying brain state and memory success.

Sports analytics

In my spare time, I like to dabble in sports analytics. In 2022, my team of Penn grad students was named a finalist for the NFL Big Data Bowl! You can check out our Kaggle notebook as well as the NFL’s press release announcing the finalists and my team’s video presentation of our project.

Our project used high-resolution player-tracking data to predict kick-return outcomes, then built a framework for finding optimal return paths and evaluating players’ decisions.