Domain Theory of Neural
Networks
Computational domain theory can be used to analyze the memory capacity of neural networks:
- MSc Thesis: The Storage Capacity of Forgetful Neural Networks
Derives a two-stage algorithm — using domain theory and chaotic dynamical systems to model the decay of memorised patterns, then a solved Ising model from statistical mechanics — to compute the storage capacity of any smooth forgetful neural network, showing it caps at 0.0489585N patterns, for N neurons.
- A Smooth Approximation on the Edge of Chaos
Shows that for certain hyperbolic iterated function systems with probabilities, close to a degenerate fixed-point case, the chaotic invariant measure can be approximated by a smooth probability density, and applies the result to forgetful neural networks.