Residual Entropy paper now on arXiv
It's an idea I've been thinking about for a long while, and over the last few months I finally carved out some time to put down some initial results in a paper. The central idea is to use the fact that a form of entropy can be defined for any probability distribution, and when fitting models to data we should prefer residuals that plausibly arise from a high entropy distribution. Such residuals are uncorrelated. Residuals may not always be uncorrelated in practice, although this is a standard assumption in Least Squares fitting. I think this assumption is both too strong, and throws away useful information about the quality of modelling.
A preprint of the paper is now available on arXiv at https://arxiv.org/abs/1907.03888, and I would welcome any thoughts or discussion. There is a ton more still to do...!
![](https://1aa7869417.cbaul-cdnwnd.com/40bc19234e04681015c4f48e6ac1172a/200000010-4769548606/residualentropy.png?ph=1aa7869417)