Projects
First Contact with ML Research - Ablating Regularizers on Spectral Koopman Attention
I documented my first-ever machine learning project, which involved proposing and ablating orthogonal and spectral regularizers for an architecture which replaces attention patterns with potentially unstable sufficient statistics. Turns out, gradient descent ensured the stability independent of our regularizers!