When we are doing weighted least squares how do we find the weights? Where ever I see tutorials are just using $w_i = \frac{1}{(sigma)i^2}$ and doing it with basic data. But I want to know how to find the weights for real data. Is it always the inverse of the square of variance?
Topic linear-regression regression machine-learning
Category Data Science
No, the weight does not always has to be equal to the inverse of the square of variance. There is no universally accepted scheme. Different weighting schemes are used, such as $1/x$, $1/x^2$, $1/\sqrt{x}$, where $x$ are your data. The weighting scheme depends on many things: the distribution of your data, their meaning, and in which regions you want to fit them more closely (whether small data values should have lower error), etc.
This is a paper that compares different weighting schemes for data in chemistry: [1]. The introduction in that paper has references to many other papers discussing different weighting schemes.
[1] Ram B. Jain, Comparison of three weighting schemes in weighted regression analysis for use in a chemistry laboratory, Clinica Chimica Acta, Volume 411, Issues 3–4, 2010, Pages 270-279, ISSN 0009-8981, https://doi.org/10.1016/j.cca.2009.11.021.
Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.