Learn and talk about Method of moments (statistics), Fitting probability distributions

This article does not cite any references or sources. Please help improve this article by adding citations to reliable sources. Unsourced material may be challenged and removed. (December 2009)

See method of moments (probability theory) for an account of a technique for proving convergence in distribution.

In statistics, the method of moments is a method of estimation of population parameters such as mean, variance, median, etc. (which need not be moments), by equating sample moments with unobservable population moments and then solving those equations for the quantities to be estimated.

1 Methodology
2 Example
3 Advantages and disadvantages of this method
4 See also

Methodology[edit]

Suppose that the problem is to estimate $p$ unknown parameters $\theta_{1}, \theta_{2}, \dots, \theta_{p}$ characterizing a distribution $f_{W}(w; \theta)$ . Suppose $p$ of the moments of the true distribution can be expressed as functions of the $\theta$ s:

$\mu_{1} \equiv E[W^1]=g_{1}(\theta_{1}, \theta_{2}, \dots, \theta_{p}) ,$

$\mu_{2} \equiv E[W^2]=g_{2}(\theta_{1}, \theta_{2}, \dots, \theta_{p}) ,$

$\vdots$

$\mu_{p} \equiv E[W^p]=g_{p}(\theta_{1}, \theta_{2}, \dots, \theta_{p}) .$

Let $\hat{\mu_{j}}=(\Sigma_{i=1}^{n} w_{i}^{j})/n$ be the j-th sample moment corresponding to the population moment $\mu_{j}$ . The method of moments estimator for $\theta_{1}, \theta_{2}, \dots, \theta_{p}$ denoted by $\hat{\theta}_{1}, \hat{\theta_{2}}, \dots, \hat{\theta}_{p}$ is defined by the solution (if there is one) to the equations:^{[citation needed]}

$\hat \mu_{1} = g_{1}(\hat{\theta}_{1}, \hat{\theta_{2}}, \dots, \hat{\theta}_{p}) ,$

$\hat \mu_{2} = g_{2}(\hat{\theta}_{1}, \hat{\theta_{2}}, \dots, \hat{\theta}_{p}) ,$

$\vdots$

$\hat \mu_{p} = g_{p}(\hat{\theta}_{1}, \hat{\theta_{2}}, \dots, \hat{\theta}_{p}) .$

Example[edit]

Suppose X₁, ..., X_n are independent identically distributed random variables with a gamma distribution with probability density function

${x^{\alpha-1} e^{-x/\beta} \over \beta^\alpha\, \Gamma(\alpha)} \,\!$

for x > 0, and 0 for x < 0.

The first moment, i.e., the expected value, of a random variable with this probability distribution is

$\operatorname{E}(X_1)=\alpha\beta\,$

and the second moment, i.e., the expected value of its square, is

$\operatorname{E}(X_1^2)=\beta^2\alpha(\alpha+1).\,$

These are the "population moments".

The first and second "sample moments" m₁ and m₂ are respectively

$m_{1} = {X_1+\cdots+X_n \over n} \,\!$

and

$m_{2} = {X_1^2+\cdots+X_n^2 \over n}.\,\!$

Equating the population moments with the sample moments, we get

$\alpha\beta = m_{1} \,\!$

and

$\beta^2\alpha(\alpha+1) = m_{2}.\,\!$

Solving these two equations for α and β, we get

$\alpha={ m_{1}^2 \over m_{2} - m_{1}^2}\,\!$

and

$\beta={ m_{2} - m_{1}^2 \over m_{1}}.\,\!$

We then use these 2 quantities as estimates, based on the sample, of the two unobservable population parameters α and β.

Advantages and disadvantages of this method[edit]

In some respects, when estimating parameters of a known family of probability distributions, this method was superseded by Fisher's method of maximum likelihood, because maximum likelihood estimators have higher probability of being close to the quantities to be estimated.

However, in some cases, as in the above example of the gamma distribution, the likelihood equations may be intractable without computers, whereas the method-of-moments estimators can be quickly and easily calculated by hand as shown above.

Estimates by the method of moments may be used as the first approximation to the solutions of the likelihood equations, and successive improved approximations may then be found by the Newton–Raphson method. In this way the method of moments and the method of maximum likelihood are symbiotic.

In some cases, infrequent with large samples but not so infrequent with small samples, the estimates given by the method of moments are outside of the parameter space; it does not make sense to rely on them then. That problem never arises in the method of maximum likelihood. Also, estimates by the method of moments are not necessarily sufficient statistics, i.e., they sometimes fail to take into account all relevant information in the sample.

When estimating other structural parameters (e.g., parameters of a utility function, instead of parameters of a known probability distribution), appropriate probability distributions may not be known, and moment-based estimates may be preferred to Maximum Likelihood Estimation.

See also[edit]

Generalized method of moments

Statistics

Descriptive statistics

Continuous data

Location	Mean (Arithmetic, Geometric, Harmonic) Median Mode

Dispersion	Range Standard deviation Coefficient of variation Percentile Interquartile range

Shape	Variance Skewness Kurtosis Moments L-moments

Count data

Index of dispersion

Summary tables

Dependence

Statistical graphics

Data collection

Designing studies	Effect size Standard error Statistical power Sample size determination

Survey methodology	Sampling Stratified sampling Opinion poll Questionnaire

Controlled experiment	Design of experiments Randomized experiment Random assignment Replication Blocking Factorial experiment Optimal design

Uncontrolled studies	Natural experiment Quasi-experiment Observational study

Statistical inference

Statistical theory	Sampling distribution Order statistics Sufficiency Completeness Exponential family Permutation test (Randomization test) Empirical distribution Bootstrap U statistic Efficiency Asymptotics Robustness

Frequentist inference	Unbiased estimator (Mean unbiased minimum variance, Median unbiased) Biased estimators (Maximum likelihood, Method of moments, Minimum distance, Density estimation) Confidence interval Testing hypotheses Power Parametric tests (Likelihood-ratio, Wald, Score)

Specific tests	Z (normal) Student's t-test F Chi-squared Signed-rank (1-sample, 2-sample, 1-way anova) Shapiro–Wilk Kolmogorov–Smirnov

Bayesian inference	Bayesian probability Prior Posterior Credible interval Bayes factor Bayesian estimator Maximum posterior estimator

Correlation and regression analysis

Correlation	Pearson product–moment correlation Partial correlation Confounding variable Coefficient of determination

Regression analysis	Errors and residuals Regression model validation Mixed effects models Simultaneous equations models

Linear regression	Simple linear regression Ordinary least squares General linear model Bayesian regression

Non-standard predictors	Nonlinear regression Nonparametric Semiparametric Isotonic Robust

Generalized linear model	Exponential families Logistic (Bernoulli) Binomial Poisson

Partition of variance	Analysis of variance (ANOVA) Analysis of covariance Multivariate ANOVA Degrees of freedom

Categorical, multivariate, time-series, or survival analysis

Categorical data

Multivariate statistics

Time series analysis

General	Decomposition Trend Stationarity Seasonal adjustment

Time domain	ACF PACF XCF ARMA model ARIMA model Vector autoregression

Frequency domain	Spectral density estimation

Survival analysis

Applications

Biostatistics	Bioinformatics Clinical trials & studies Epidemiology Medical statistics

Engineering statistics	Chemometrics Methods engineering Probabilistic design Process & Quality control Reliability System identification

Social statistics	Actuarial science Census Crime statistics Demography Econometrics National accounts Official statistics Population Psychometrics

Spatial statistics	Cartography Environmental statistics Geographic information system Geostatistics Kriging

Original courtesy of Wikipedia: http://en.wikipedia.org/wiki/Method_of_moments_(statistics) — Please support Wikipedia.
A portion of the proceeds from advertising on Digplanet goes to supporting Wikipedia.

Method of moments (statistics)

Contents

Methodology[edit]

Example[edit]

Advantages and disadvantages of this method[edit]

See also[edit]

Talk About Method of moments (statistics)

Support Wikipedia