Residual Entropy paper now on arXiv
It's an idea I've been thinking about for a long while, and over the last few months I finally carved out some time to put down some initial results in a paper. The central idea is to use the fact that a form of entropy can be defined for any probability distribution, and when fitting models to data we should prefer residuals that plausibly arise from a high entropy distribution. Such residuals are uncorrelated. Residuals may not always be uncorrelated in practice, although this is a standard assumption in Least Squares fitting. I think this assumption is both too strong, and throws away useful information about the quality of modelling.
A preprint of the paper is now available on arXiv at https://arxiv.org/abs/1907.03888, and I would welcome any thoughts or discussion. There is a ton more still to do...!