Learn and talk about Poisson regression, Categorical data, Econometrics, Generalized linear models, Regression analysis

Regression analysis

Models
Linear regression Simple regression Ordinary least squares Polynomial regression General linear model
Generalized linear model Discrete choice Logistic regression Multinomial logit Mixed logit Probit Multinomial probit Ordered logit Ordered probit Poisson
Multilevel model Fixed effects Random effects Mixed model
Nonlinear regression Nonparametric Semiparametric Robust Quantile Isotonic Principal components Least angle Local Segmented
Errors-in-variables
Estimation
Least squares Ordinary least squares Linear (math) Partial Total Generalized Weighted Non-linear Iteratively reweighted Ridge regression LASSO
Least absolute deviations Bayesian Bayesian multivariate
Background
Regression model validation Mean and predicted response Errors and residuals Goodness of fit Studentized residual Gauss–Markov theorem
Statistics portal
v t e

In statistics, Poisson regression is a form of regression analysis used to model count data and contingency tables. Poisson regression assumes the response variable Y has a Poisson distribution, and assumes the logarithm of its expected value can be modeled by a linear combination of unknown parameters. A Poisson regression model is sometimes known as a log-linear model, especially when used to model contingency tables.

Poisson regression models are generalized linear models with the logarithm as the (canonical) link function, and the Poisson distribution function.

1 Regression models
2 Maximum likelihood-based parameter estimation
3 Poisson regression in practice
4 Extensions
- 4.1 Regularized Poisson regression
5 Implementations
6 See also
7 References

Regression models [edit]

If $x \in \mathbb{R}^n$ is a vector of independent variables, then the model takes the form

$\log (\operatorname{E}(Y|x))=a' x + b,\,$

where $a \in \mathbb{R}^n$ and $b \in \mathbb{R}$ . Sometimes this is written more compactly as

$\log (\operatorname{E}(Y|x))=\theta' x,\,$

where x is now an (n + 1)-dimensional vector consisting of n independent variables concatenated to some constant, usually 1. Here θ is simply a concatenated to b.

Thus, when given a Poisson regression model θ and an input vector $x$ , the predicted mean of the associated Poisson distribution is given by

$\operatorname{E}(Y|x)=e^{\theta' x}.\,$

If Y_i are independent observations with corresponding values x_i of the predictor variable, then θ can be estimated by maximum likelihood. The maximum-likelihood estimates lack a closed-form expression and must be found by numerical methods. The probability surface for maximum-likelihood Poisson regression is always convex, making Newton–Raphson or other gradient-based methods appropriate estimation techniques.

Maximum likelihood-based parameter estimation [edit]

Given a set of parameters θ and an input vector x, the mean of the predicted Poisson distribution, as stated above, is given by

$\operatorname{E}(Y|x)=e^{\theta' x}\,$ ,

and thus, the Poisson distribution's probability mass function is given by

$p(y|x;\theta) = \frac{[\operatorname{E}(Y|x)]^y \times e^{-\operatorname{E}(Y|x)}}{y!} = \frac{e^{y \theta' x} e^{-e^{\theta' x}}}{y!}$

Now suppose we are given a data set consisting of m vectors $x_i \in \mathbb{R}^{n+1}, \, i = 1,...,m$ , along with a set of m values $y_1,\ldots,y_m \in \mathbb{R}$ . Then, for a given set of parameters θ, the probability of attaining this particular set of data is given by

$p(y_1,\ldots,y_m|x_1,\ldots,x_m;\theta) = \prod_{i=1}^m \frac{e^{y_i \theta' x_i} e^{-e^{\theta' x_i}}}{y_i!}.$

By the method of maximum likelihood, we wish to find the set of parameters θ that makes this probability as large as possible. To do this, the equation is first rewritten as a likelihood function in terms of θ:

$L(\theta|X,Y) = \prod_{i=1}^m \frac{e^{y_i \theta' x_i} e^{-e^{\theta' x_i}}}{y_i!}$ .

Note that the expression on the right hand side has not actually changed. A formula in this form is typically difficult to work with; instead, one uses the log-likelihood:

$\ell(\theta|X,Y) = \log L(\theta|X,Y) = \sum_{i=1}^m \left( y_i \theta' x_i - e^{\theta' x_i} - \log(y_i!)\right)$ .

Notice that the parameters θ only appear in the first two terms of each term in the summation. Therefore, given that we are only interested in finding the best value for θ we may drop the y_i! and simply write

$\ell(\theta|X,Y) = \sum_{i=1}^m \left( y_i \theta' x_i - e^{\theta' x_i}\right)$ .

To find a maximum, we need to solve an equation $\frac{\partial \ell(\theta|X,Y)}{\partial \theta} = 0$ which has no closed-form solution. However, the negative log-likelihood, $-\ell(\theta|X,Y)$ , is a convex function, and so standard convex optimization techniques such as gradient descent can be applied to find the optimal value of θ.

Poisson regression in practice [edit]

Poisson regression may be appropriate when the dependent variable is a count, for instance of events such as the arrival of a telephone call at a call centre.^[1]^:pp.740-752 The events must be independent in the sense that the arrival of one call will not make another more or less likely, but the probability per unit time of events is understood to be related to covariates such as time of day.

"Exposure" and offset [edit]

Poisson regression may also be appropriate for rate data, where the rate is a count of events occurring to a particular unit of observation, divided by some measure of that unit's exposure. For example, biologists may count the number of tree species in a forest, and the rate would be the number of species per square kilometre. Demographers may model death rates in geographic areas as the count of deaths divided by person−years. More generally, event rates can be calculated as events per unit time, which allows the observation window to vary for each unit. In these examples, exposure is respectively unit area, person−years and unit time. In Poisson regression this is handled as an offset, where the exposure variable enters on the right-hand side of the equation, but with a parameter estimate (for log(exposure)) constrained to 1.

$\log{(\operatorname{E}(Y|x))} = \log{(\text{exposure})} + \theta' x$

which implies

$\log{(\operatorname{E}(Y|x))} - \log{(\text{exposure})} = \log{\left(\frac{\operatorname{E}(Y|x)}{\text{exposure}}\right)} = \theta' x$

Offset in the case of a GLM in R can be achieved using the offset() function:

glm.fit <- glm(y ~ offset(log(exposure)) + x, family=poisson(link=log) )

Overdispersion [edit]

A characteristic of the Poisson distribution is that its mean is equal to its variance. In certain circumstances, it will be found that the observed variance is greater than the mean; this is known as overdispersion and indicates that the model is not appropriate. A common reason is the omission of relevant explanatory variables, or dependent observations. Under some circumstances, the problem of overdispersion can be solved by using a negative binomial distribution instead.^[2]^[3]

Another common problem with Poisson regression is excess zeros: if there are two processes at work, one determining whether there are zero events or any events, and a Poisson process determining how many events there are, there will be more zeros than a Poisson regression would predict. An example would be the distribution of cigarettes smoked in an hour by members of a group where some individuals are non-smokers.

Other generalized linear models such as the negative binomial model or zero-inflated model may function better in these cases.

Use in survival analysis [edit]

Poisson regression creates proportional hazards models, one class of survival analysis: see proportional hazards models for descriptions of Cox models.

Tests of over-dispersion [edit]

One method for testing for over dispersion in the data is to regress a variable z_i against the predicted values of t estimated from the Poisson regression.^[4] This test has three steps.

1. Estimate a Poisson regression of y_i on x_i and generate the predicted values ( t_i )

2. Calculate the z_i variable

$z_i = \frac{ ( y_i - t_i )^2 - y_i } { t_i \sqrt{2} }$

3. Regress z_i against t_i with ordinary least squares. In symbols

$z_i = a t_i + e_i \,$

where a is a constant and e_i is a random variable with an expectation of zero.

The null hypothesis being tested here is that the data are Poisson-distributed: in this case a = 0.

Extensions [edit]

Regularized Poisson regression [edit]

When estimating the parameters for Poisson regression, one typically tries to find values for θ that maximize the likelihood of an expression of the form

$\sum_{i=1}^m \log(p(y_i;e^{\theta' x})),$

where m is the number of examples in the data set, and $p(y_i;e^{\theta' x})$ is the probability mass function of the Poisson distribution with the mean set to $e^{\theta' x}$ . Regularization can be added to this optimization problem by instead maximizing

$\sum_{i=1}^m \log(p(y_i;e^{\theta' x})) - \lambda \left\|\theta\right\|_2^2$ ,

for some positive constant $\lambda$ . This technique, similar to ridge regression, can reduce overfitting.

Implementations [edit]

Some statistics packages include implementations of Poisson regression.

MATLAB Statistics Toolbox: Poisson regression can be performed using the "glmfit" and "glmval" functions.^[5]
Microsoft Excel: Excel is not capable of doing Poisson regression by default. One of the Excel Add-ins for Poisson regression is XPost
R: The function for fitting a generalized linear model in R is glm(), and can be used for Poisson Regression
SAS: Poisson regression in SAS is done by using GENMOD
SPSS: In SPSS, Poisson regression is done by using the GENLIN command
Stata: Stata has a procedure for Poisson regression named "poisson", and for panel data "xtpoisson"
mPlus: mPlus allows for Poisson regression using the command COUNT IS when specifying the data

References [edit]

Cameron, A.C. and P.K. Trivedi (1998). Regression analysis of count data, Cambridge University Press. ISBN 0-521-63201-3

Christensen, Ronald (1997). Log-linear models and logistic regression. Springer Texts in Statistics (Second ed.). New York: Springer-Verlag. pp. xvi+483. ISBN 0-387-98247-7. MR 1633357.

Hilbe, J. M. (2007). Negative Binomial Regression, Cambridge University Press. ISBN 978-0-521-85772-7

^ Greene, William H., Econometric Analysis, fifth edition, Prentice-Hall, 2003.
^ Paternoster R, Brame R (1997). "Multiple routes to delinquency? A test of developmental and general theories of crime". Criminology 35: 45–84.
^ Berk R, MacDonald J (2008). "Overdispersion and Poisson regression". Journal of Quantitative Criminology 24: 269–284.
^ https://files.nyu.edu/mrg217/public/count.pdf
^ http://www.mathworks.com/help/toolbox/stats/glmfit.html

Statistics

Descriptive statistics

Continuous data

Location	Mean (Arithmetic, Geometric, Harmonic) Median Mode

Dispersion	Range Standard deviation Coefficient of variation Percentile Interquartile range

Shape	Variance Skewness Kurtosis Moments L-moments

Count data

Index of dispersion

Summary tables

Dependence

Statistical graphics

Data collection

Designing studies	Effect size Standard error Statistical power Sample size determination

Survey methodology	Sampling Stratified sampling Opinion poll Questionnaire

Controlled experiment	Design of experiments Randomized experiment Random assignment Replication Blocking Factorial experiment Optimal design

Uncontrolled studies	Natural experiment Quasi-experiment Observational study

Statistical inference

Statistical theory	Sampling distribution Order statistics Sufficiency Completeness Exponential family Permutation test (Randomization test) Empirical distribution Bootstrap U statistic Efficiency Asymptotics Robustness

Frequentist inference	Unbiased estimator (Mean unbiased minimum variance, Median unbiased) Biased estimators (Maximum likelihood, Method of moments, Minimum distance, Density estimation) Confidence interval Testing hypotheses Power Parametric tests (Likelihood-ratio, Wald, Score)

Specific tests	Z (normal) Student's t-test F Chi-squared Signed-rank (1-sample, 2-sample, 1-way anova) Shapiro–Wilk Kolmogorov–Smirnov

Bayesian inference	Bayesian probability Prior Posterior Credible interval Bayes factor Bayesian estimator Maximum posterior estimator

Correlation and regression analysis

Correlation	Pearson product–moment correlation Partial correlation Confounding variable Coefficient of determination

Regression analysis	Errors and residuals Regression model validation Mixed effects models Simultaneous equations models

Linear regression	Simple linear regression Ordinary least squares General linear model Bayesian regression

Non-standard predictors	Nonlinear regression Nonparametric Semiparametric Isotonic Robust

Generalized linear model	Exponential families Logistic (Bernoulli) Binomial Poisson

Partition of variance	Analysis of variance (ANOVA) Analysis of covariance Multivariate ANOVA Degrees of freedom

Categorical, multivariate, time-series, or survival analysis

Categorical data

Multivariate statistics

Time series analysis

General	Decomposition Trend Stationarity Seasonal adjustment

Time domain	ACF PACF XCF ARMA model ARIMA model Vector autoregression

Frequency domain	Spectral density estimation

Survival analysis

Applications

Biostatistics	Bioinformatics Clinical trials & studies Epidemiology Medical statistics

Engineering statistics	Chemometrics Methods engineering Probabilistic design Process & Quality control Reliability System identification

Social statistics	Actuarial science Census Crime statistics Demography Econometrics National accounts Official statistics Population Psychometrics

Spatial statistics	Cartography Environmental statistics Geographic information system Geostatistics Kriging

Least squares and regression analysis

Computational statistics

Correlation and dependence

Regression analysis

Regression as a
statistical model

Linear regression	Simple linear regression Ordinary least squares Generalized least squares Weighted least squares General linear model

Predictor structure	Polynomial regression Growth curve Segmented regression Local regression

Non-standard	Nonlinear regression Nonparametric Semiparametric Robust Quantile Isotonic

Non-normal errors	Generalized linear model Binomial Poisson Logistic

Decomposition of variance

Model exploration

Background

Design of experiments

Numerical approximation

Applications

Original courtesy of Wikipedia: http://en.wikipedia.org/wiki/Poisson_regression — Please support Wikipedia.
A portion of the proceeds from advertising on Digplanet goes to supporting Wikipedia.

Poisson regression

Contents