Shared structure of Partial Differential Equations (PDEs)
During my Master's thesis, I started a project with Ion Stamatescu (Prof. of Theoretical Physics in Heidelberg, Germany) and James Owen Weatherall (Prof. of Philosophy of Physics in Irvine, California, USA) to pursue the philosophically motivated question “How to describe the shared structure of classical theories of physics?”.
I created a mathematical framework to compare systems of nonlinear partial differential equations (PDEs) that can be used to compare classical field theories in a precise way. During my PhD, I developed those concepts, until they were published in a dedicated book chapter titled A Geometric Framework to Compare PDEs and Classical Field Theories.
Our framework facilitates to transfer solutions and generalizes Bäcklund transformations that allow to solve some difficult nonlinear PDEs. Furthermore, I implemented an algorithm that can determine the minimal number of consistency conditions that must be fulfilled for two field theories to share structure. Our theory involves tools from the algebro-geometric theory of jet spaces, PDEs and Lie groups, the differential-topological theory of transversality and the cohomological theory of formal integrability.
Shared structure of data
In my PhD with Jürgen Jost (Honorary Professor of the Department of Mathematics and Retired Director of the Max Planck Institute for Mathematics in the Sciences, Leipzig, Germany), I explore the question how to identify and generalize patterns in data.
Manifold learning approach
In our first approach, we contributed to manifold learning, where one assumes that the data is sampled from a distribution that is concentrated around a manifold which represents the shared structure of the data. Our approach builds up on UMAP, a scheme that makes use of the category theory of fuzzy simplicial sets. In contrast to ordinary simplicial sets that play a central role in algebraic topology and homology, fuzzy simplicial sets can resolve geometric information because they represent entire filtrations, related to those in topological data analysis and persistent homology.We developed this theory in several publications (Fuzzy simplicial sets and their application to geometric data analysis and IsUMap: Manifold Learning and Data Visualization Leveraging Vietoris-Rips Filtrations and Merging Hazy Sets with m-Schemes) and combined all results with an extended introduction to geometry and category theory in our book Data Visualization with Category Theory and Geometry. Apart from deriving new relationships between well-known manifold learning schemes like UMAP and Isomap, we published an improved algorithm that we call IsUMap on github.
Information theoretic approach
Since there are algorithmic patterns that are not well extrapolated by a manifold, I also formulated an information theoretic notion of shared structure in my thesis. Kolmogorov complexity is decomposed into shared and individual structure, building up on the minimum description length principle. In contrast to Shannon-information based decomposition, the approach defines shared structure algorithmically.I used this notion to investigate posterior consistency results in the space of all computable distributions, expanding on the notion of Solomonoff induction. With a Master student I supervised, we developed approximation algorithms for computing shared structure with deep neural networks. Generalizing ideas from compressed sensing to the non-linear setting, we created algorithms that quickly find good local optima of the provably NP-hard problem of \ell_0-regularized optimization and published the results in two articles (Probabilistic and nonlinear compressive sensing and Efficient compression of neural networks and datasets). Both for the traditional compressive sensing problem as well as for nonlinear neural network compression (image classifiers and language models), we achieved lower generalization error and better compression ratios than previous methods. We published all of our code on github.
Compositional Optimization
During a self-organized research visit at the Topos Institute in Berkeley, California, I collaborated with David Spivak (Senior Scientist and Institute Fellow at Topos Institute), who has developed mathematical constructions in the category of polynomial functors that can serve as a rich basis to describe the compositional behavior of interacting dynamical systems. The aim of the collaboration is to combine this theory with my knowledge about shared structure, information and optimization theory to develop a compositional theory of optimization with applications to multi-agent (reinforcement) learning and alignment research. We managed to prove that under certain conditions on the optimization objectives of a collection of suitably defined agents, there is a well-defined rule that makes their composition again into a (bigger) agent that optimizes a shared objective. We presented these result at the Berkeley seminar and work on developing them further.Synthesis and future work
Though the above projects are diverse, they are united by the aspiration to find precise notions of shared structure (which facilitate solution transfer, generalization or alignment). Apart from applications to PDE theory and machine learning, there are also relations to social sciences like philosophy of mind, Gestalt psychology, language processing and semantics.
The goal for my future research is to refine the notion of shared computational structure to study the emergence of meaning and reasoning in multi-agent systems. The process of abstraction itself consists of the extraction of the shared from the parts. During communication, which serves to achieve shared objectives, semantic meaning then emerges. I believe there is a lot of potential in a formalization based on shared structure and causal inference in the context of compositional multi-agent systems.
Realizing this aim would enable very interesting applications in statistical inference and alignment research. My aim is not just to develop theory but concrete implementations as well to improve upon the human condition.