T. Anderson Keller

@andykeller.bsky.social

373 Followers 372 Following

Postdoctoral Fellow at Harvard Kempner Institute. Trying to bring natural structure to artificial neural representations. Prev: PhD at UvA. Intern @ Apple MLR, Work @ Intel Nervana

Tables from the paper comparing wave based models and baselines (CNNs and U-Nets) on a variety of semantic segmentation tasks
Visualization of the neural wave machine dynamics on the multi-MNIST dataset.
Video of the LSTM hidden state after being trained to semantically segment the tetronimoes dataset. We see despite having no prior bias, the model learns to use wave dynamics to propagate information.
Plot of five representative frequency bins from the FFT of the dynamics of our wave-RNN on the shape task. We see different shapes pop out in different bins, indicating that they 'sound' different, and allowing the model to uniquely classify each shape. On the right we plot the average FFT for each pixel, separated by each shape, over the whole dataset, showing that different shapes do have measurably different frequency spectra, even in this average case.
Visualization of the learned dynamics in the hidden state of a wave-based RNN learning to semantically segment different polygon shapes. The outline of the input stimulus is plotted as a dashed white line, but this information is only added for the viewer's convenience and is not present in the raw data. We see the model uses differing natural frequencies between inside and outside the shape to induce soft boundaries, causing reflections, and ultimately yielding shape-specific wave patterns.
Visualization of the input stimuli to our network (left) and the target segmentation labels by color (right). The receptive field of the final layer neurons in our model is plotted as the yellow box, demonstrating that a single neuron has no way to know what shape it may be a part of simply from its local neighborhood, and therefore will require global integration of information over time to solve the task.
Visualization of the same wave-based RNN on two drums of different sizes (13 and 33 side length respectively). In the middle (in purple) we show the displacement of the drum head at a point just off the center, and (in red) the theoretical fundamental frequency of vibration that we can analytically derive for a square of side length L plotted. On the right we show the Fourier transform of these time-series dynamics, showing the frequency peak in the expected location. This validates we can estimate the size of a drum head from the frequency spectrum of vibration at any point.
Visualization of a wave-based RNN simulating the standard wave equation on a square drum surface of side length 43.
Visualization of the hidden state of a wave-based RNN processing an image of a hexagon. The model learns to use wave dynamics to compute a unique frequency-space representation of the shape, allowing it to distinguish it from other shapes in the dataset.