Estimation Theory

In my work trying to work through the proofs in Lin¹ and Moran², I’ve run into some confusion in parameter estimation. In this post, I’ll cover the relevant sections in Tsiatis’ Semiparametric Theory and Missing Data³ and Newey and McFadden’s Large Sample Estimation and Hypothesis Testing⁴ in my journey to answering some of my questions:

1. What is the efficient information matrix?
2. When exactly are the quasi-likelihood estimators asymptotically normal and consistent?
3. Are any regularity conditions violated in the variance componenet estimation scheme for generalized linear mixed models?

---

Set-Up

We assume we have independent and identically distributed random variables, $X_1, \dots, X_n$, arranged into a vector $\mathbf{X}$ defined on the probability space $(\mathcal{X}, \mathcal{A}, P)$. We assume $X_i \sim P$ where $P \in \mathcal{P}$ for some family of distributions, $\mathcal{P}$. We also assume that $P$ is defined by the values of parameters, $\theta$, where some coordinates are of interest and some are a nuisance (i.e. we can partition the parameter space). We use $\beta$ and $\eta$ to denote the parameters of interest and the nuisance parameters, respectively. We have:

\[\theta = (\beta^\top, \eta^\top)^\top \in \mathbb{R}^{p \times 1} \hspace{2mm} (p = q + r), \hspace{8mm} \beta \in \mathbb{R}^{q \times 1}, \hspace{8mm} \eta \in \mathbb{R}^{r \times 1} \label{eq:parameter-vector}\]

As in Tsiatis’ book, we’ll restrict the parameter space, $\Omega$, to subspaces of linear vector spaces, which themselves could be finite- or infinite-dimensional (depending on whether there exist a finite number of elements in it that span it). In most applications/examples, this assumption will not be too restrictive since we will usually be working in Euclidean spaces.

Notation

Uppercase letters denote random variables, and lowercase letters denote realizations of these variables. Boldface indicates a vector or matrix, exactly which will be clear from context.

We will indicate parameters that are functions with parenthesis (e.g. $\gamma(\cdot)$), and we’ll use $\hat{\gamma}_n$ to denote an estimator of parameter $\gamma$. We’ll use a subscript of $0$ to denote the true parameter values (i.e. $\theta_0 = (\beta_0^\top, \eta_0^\top)^\top$).

$\nu_X(x)$ denotes the dominating measure on which densities for $\mathbf{X}$ are defined. That is, the dominating measure is any measure $\mu(\cdot)$ such that the density $f_\mathbf{X}(\mathbf{x})$ exists and such that $\mu’(A) = 0$ implies $\mu(A) = 0$. For continuous $X$, we usually use the Lebesgue measure, and for discrete $X$, we use the counting measure.

I am pretty sure all of the expectations are taking with respect to the true data-generating distribution, $P_X(x, \theta_0)$. I’ve denoted this with a subscript of $\theta_0$ on the expectation, but I may have missed a few.

Convergence in probability and convergence in distribution when the density of the random variable $X$ is equal to $p_X(x; \beta, \eta)$ are denoted by, respectively:

\[\xrightarrow{P\{ \beta, \eta \}}, \hspace{10mm} \xrightarrow{D\{ \beta, \eta \}}\]

General convergence in probability and distribution are denoted by:

\[\overset{p}{\rightarrow}, \hspace{10mm} \rightsquigarrow\]

Hilbert Spaces

Before we dive into our discussion of semiparametric inference, we need to first look at some geometric definitions.

Consider the space of functions of the form $h(X)$ where $h: \mathcal{X} \rightarrow \mathbb{R}^q$, are measurable (w.r.t. the probability space we described earlier), and are such that:

\[\mathbb{E}[h(X)] = 0, \hspace{10mm} \mathbb{E}\left[ h^\top(X) h(X)\right] < \infty\]

That is, we consider the space of all measurable, $q$-dimensional random functions with mean zero and finite variance. Notice that this space is a linear space. Let $\mathbf{0}$ (the constant function outputting a $q$-dimensional vector of zeros) denote the origin. Though these are random functions, we will sometimes drop the $(X)$ from our notation to just write $h$.

Later we will concern ourselves with Hilbert spaces of these types of functions. We define these formally for completeness. Recall that a Hilbert space is a complete normed linear vector space with an inner product.

Our Hilbert spaces of interest (defined above) are dependent upon the true value $\theta_0$, and this is the value with respect to which we take the expectation in the inner product. Thus, the space will change if $\theta_0$ changes.

We come to a theorem that will be helpful later in some definitions and results.

A version of the Pythagorean theorem and the Cauchy-Schwarz inequality can be derived as a result of the Projection theorem.

---

Extremum Estimators

Before we dive into more specifics, let’s define a few different classes of estimators.

The name comes from the fact that we are maximizing the objective (i.e. finding an extreme value). An example is the maximum likelihood estimator (MLE), which has the (normalized) log-likelihood as its objective function:

\[\hat{Q}_n(X, \theta) = \frac{1}{n} \sum_{i = 1}^n \log (p_X(x_i; \theta))\]

In certain cases, the maximization can be done by taking the derivative and setting this equal to zero. However, we should be cautious as there are cases when there are many such solutions. In this case, a solution to this first-order condition may not be the global maximum of $\hat{Q}_n(X, \theta)$ (i.e. a solution may be a local maximum). Luckily, if the extremum estimator is consistent and $\theta_0$ is on the interior of $\Omega$, then the extremum estimator will be included in the set of solutions to setting the derivative equal to zero.

The extremum estimator class is quite broad. A subclass of extremum estimator is the $m$-estimator.

Put simply, an extremum estimator that maximizes a sample average is an $m$-estimator⁴. Before we look at an example, let’s define the score.

The MLE is also an example of an $m$-estimator. Consider the function $m(X, \theta) = \log (p_X(x; \theta))$, the log-likelihood. We can maximize the log-likelihood by taking the derivative with respect to $\theta$ and setting this equal to zero. This is called the score equation in $\theta$:

\[\sum_{i = 1}^n U_\theta(X_i, \theta) = 0\]

Influence Functions

Some estimators (asymptotically linear ones…to be defined later) can be analyzed with respect to a particular defining function, called the influence function.

Note that the influence function is always defined with respect to the true data-generating distribution; it is evaluated at $\theta_0$, and the expectations are always taken with respect to the parameter being $\theta_0$. Thus, we don’t really need to write $\psi(X, \theta_0)$, since the influence function is not a function of $\theta$ (it is just dependent on $\theta_0$). However, I keep the full notation to avoid confusion.

We can also define special properties of estimators to help us decribe the ones that are desirable. We say that $\hat{\theta}_n$ is asymptotically linear if it has an influence function(s). Thus, the maximum likelihood estimator is asymptotically linear.

The influence function(s) provides us a clear way to analyze the behavior of our (asymptotically linear) estimator as $n \rightarrow \infty$:

\[\begin{aligned} \frac{1}{\sqrt{n}}\sum_{i = 1}^n \psi(X_i, \theta_0) &\rightsquigarrow \mathcal{N}(\mathbf{0}_{q \times q}, \mathbb{E}_{\theta_0}\left[ \psi(X, \theta_0) \psi(X, \theta_0)^\top \right]) & \left(\text{CLT}\right) \\ \sqrt{n}(\hat{\theta}_n - \theta_0) &\rightsquigarrow \mathcal{N}(\mathbf{0}_{q \times q}, \mathbb{E}_{\theta_0}\left[ \psi(X, \theta_0) \psi(X, \theta_0)^\top \right]) & \left(\text{Slutsky's theorem}\right) \end{aligned} \label{eq:al-dist}\]

Eq. \eqref{eq:al-dist} implies that the asymptotic variance of the estimator is the variance of its influence function. Furthermore, an asymptotically linear estimator is (effectively) uniquely identified by its influence function.

The score vector also satisfies nice properties, which we summarize in the following theorem. First, let’s define what it means for an estimator to be “regular”.

In many cases, the maximum likelihood estimator is regular. When an estimator is both asymptotically linear and regular, we say it is RAL.

Note that the influence functions for RAL estimators are in the subspace of elements of $\mathcal{H}$ (the Hilbert space of mean-zero measurable random functions) satisfying Eq. \eqref{eq:corollary-1}. Similarly, each element in that subspace is the influence function for some RAL estimator.

From this geometric perspective, we can see that the asymptotic variance of an RAL estimator is basically the squared distance between the origin and its influence function in our special Hilbert space.

---

Consistency

We can show under what conditions extremum estimators are consistent ($\hat{\theta}_n \overset{p}{\rightarrow} \theta_0$). This is called the Basic Consistency Theorem by Newey and McFadden. In what follows, we introduce the probability limit, $Q_0(X, \theta)$, of $\hat{Q}_n(X, \theta)$, which is the quantity described in the appendix.

Relaxations.

The authors note that the some of the conditions can be relaxed. Instead of assuming that $\hat{\theta}_n$ maximizes $\hat{Q}_n(X, \theta)$, we can instead assume that it "nearly" maximizes it: $$ \hat{Q}_n(\hat{\theta}_n) \geq \underset{\theta \in \Omega}{\sup} \hat{Q}_n(\theta) + o_p(1) $$ The second condition can be relaxed if the objective function, $\hat{Q}_n(X, \theta)$, is concave. Then the assumption of compactness of the parameter space, $\Omega$, can be exchanged for just convexity.
In addition, the third condition can be relaxed to just upper semi-continuity rather than continuity proper. That is, we assume that, for any $\theta \in \Omega$ and any $\epsilon > 0$, there exists an open subset $\mathcal{B} \subset \Omega$ such that $\theta \in \mathcal{B}$ and such that: $$ Q_0(X, \theta') < Q_0(X, \theta) + \epsilon \hspace{2mm} \forall \theta' \in \mathcal{B} $$ The fourth condition can be changed to just require that: $$ \hat{Q}_n(X, \theta_0) \overset{p}{\rightarrow} Q_0(\theta_0), \hspace{5mm} \text{and} \hspace{5mm} \hat{Q}_n(X, \theta) < Q_0(X, \theta) + \epsilon \hspace{2mm} \forall \epsilon > 0, \hspace{1mm} \forall \theta \in \Omega \hspace{2mm} \text{with probability approaching one} $$ If we make the stronger assumption that: $$ \underset{\theta \in \Omega}{\sup} \big\rvert \hat{Q}_n(X, \theta) - Q_0(X, \theta) \big\rvert \overset{as}{\rightarrow} 0 $$ instead of the fourth condition, then we have that $\hat{\theta}_n \overset{as}{\rightarrow} \theta_0$ (i.e. $\hat{\theta}_n$ is strongly consistent).

Showing Consistency

In order to use Theorem 2.1, one must be able to show that the conditions (or their relaxations) hold. This can be difficult in practice, we often try to show some other property that are sufficient for the conditions. Newey and McFadden call these primitive conditions.

The above lemma can be used for sample averages, which is exactly what we deal with in maximum likelihood estimation (and several other estimation settings). This leads us to the next theorem that states that, under certain conditions, the MLE is consistent:

---

Asymptotic Normality

To construct confidence intervals, we often rely upon asymptotical normality of an estimator.

Applied to maximum likelihood estimators, Theorem 3.1 gives:

\[\sqrt{n}(\hat{\theta}_n - \theta_0) \rightsquigarrow \mathcal{N}(\mathbf{0}, \mathcal{I}^{-1}(\theta_0))\]

Intuition.

The more expansive explanation of the intuition is provided at the beginning of Section 3 in Newey and McFadden, but we summarize it here for convenience.
For MLE, $\hat{Q}_n(\theta) = \frac{1}{n} \sum_{i = 1}^n \log(p_X(x_i, \theta))$. Suppose that the log-likelihood is differentiable and the maximum likelihood estimate, $\hat{\theta}$, is on the interior of the parameter space. This implies that the gradient of the log-likelihood will be equal to $0$ when evaluated at $\hat{\theta}_n$: $$ \left. \left[ \frac{1}{n}\sum_{i = 1}^n \frac{\partial \log(p_X(x_i, \theta))}{\partial \theta}\right] \right\vert_{\theta = \hat{\theta}_n} = 0 \nonumber $$ Suppose also that the log-likelihood is twice differentiable and the second derivative is continuous. We can apply the mean value theorem (MVT) to each element of the gradient to rewrite the first-order condition. Let $\bar{\theta}$ be the vector of intermediate points between each coordinate of $\hat{\theta}_n$ and $\theta_0$ that are used in the MVT. The $(i,j)$-th element is: $$ \left. \left[ \frac{1}{n} \sum_{i = 1}^n \frac{\partial^2 \log(p_X(x_i, \theta))}{\partial \theta_i \partial \theta_j} \right] \right\vert_{\theta = \bar{\theta}} = \frac{1}{(\hat{\theta}_n)_j - (\theta_0)_j} \left( \left. \left[ \frac{1}{n}\sum_{i = 1}^n \frac{\partial \log(p_X(x_i, \theta))}{\partial \theta_i}\right] \right\vert_{\theta = \hat{\theta}_n} - \left. \left[ \frac{1}{n}\sum_{i = 1}^n \frac{\partial \log(p_X(x_i, \theta))}{\partial \theta_i}\right] \right\vert_{\theta = \theta_0}\right) \nonumber $$ Using the fact that the gradient of the log-likelihood evaluated at the maximum likelihood estimate is $0$: $$ ((\hat{\theta}_n)_j - (\theta_0)_j) \left. \left[ \frac{1}{n} \sum_{i = 1}^n \frac{\partial^2 \log(p_X(x_i, \theta))}{\partial \theta_i \partial \theta_j} \right] \right\vert_{\theta = \bar{\theta}} = - \left. \left[ \frac{1}{n}\sum_{i = 1}^n \frac{\partial \log(p_X(x_i, \theta))}{\partial \theta_i}\right] \right\vert_{\theta = \theta_0} \nonumber $$ And rearranging terms: $$ (\hat{\theta_j} - (\theta_0)_j) \left. \left[ \frac{1}{n} \sum_{i = 1}^n \frac{\partial^2 \log(p_X(x_i, \theta))}{\partial \theta_i \partial \theta_j} \right] \right\vert_{\theta = \bar{\theta}} + \left. \left[ \frac{1}{n}\sum_{i = 1}^n \frac{\partial \log(p_X(x_i, \theta))}{\partial \theta_i}\right] \right\vert_{\theta = \theta_0} = 0 \nonumber $$ Rewriting in matrix notation: $$ \left. \left[ \frac{1}{n} \sum_{i = 1}^n \frac{\partial^2 \log(p_X(x_i, \theta))}{\partial \theta \partial \theta^\top} \right] \right\vert_{\theta = \bar{\theta}}(\hat{\theta}_n - (\theta_0)) + \left. \left[ \frac{1}{n}\sum_{i = 1}^n \frac{\partial \log(p_X(x_i, \theta))}{\partial \theta}\right] \right\vert_{\theta = \theta_0} = \mathbf{0} \nonumber $$ Multiplying by $\sqrt{n}$ and rearranging again: $$ \sqrt{n}(\hat{\theta_j} - (\theta_0)_j) = - \left[ \left. \left[ \frac{1}{n} \sum_{i = 1}^n \frac{\partial^2 \log(p_X(x_i, \theta))}{\partial \theta \partial \theta^\top} \right] \right\vert_{\theta = \bar{\theta}} \right]^{-1} \left. \left[ \frac{\sqrt{n}}{n}\sum_{i = 1}^n \frac{\partial \log(p_X(x_i, \theta))}{\partial \theta_i}\right] \right\vert_{\theta = \theta_0} \nonumber $$ As the score, the second term has zero mean. In addition, it is the sum of (functions of) i.i.d. random variables, so we can apply the central limit theorem to see that: $$ \left. \left[ \frac{\sqrt{n}}{n} \sum_{i = 1}^n \frac{\partial \log(p_X(x_i, \theta))}{\partial \theta_i}\right] \right\vert_{\theta = \theta_0} \rightsquigarrow \mathcal{N}\left(\mathbf{0}, \mathcal{I}(\theta_0)\right) \nonumber $$ where $\mathcal{I}(\theta_0) = \text{Var}\left( \left. \left[ \frac{1}{n} \sum_{i = 1}^n \frac{\partial \log(p_X(x_i, \theta))}{\partial \theta_i}\right] \right\vert_{\theta = \theta_0} \right)$.
Since $\bar{\theta}$ is just a function of $\hat{\theta}_n$ (since $\theta_0$ is fixed), if $\hat{\theta}_n$ is consistent, then $\bar{\theta}$ should also be. Supposing we can use a uniform law of large numbers (see Lemma 2.4), then: $$ \left. \left[ \frac{1}{n} \sum_{i = 1}^n \frac{\partial^2 \log(p_X(x_i, \theta))}{\partial \theta \partial \theta^\top} \right] \right\vert_{\theta = \bar{\theta}} \overset{p}{\rightarrow} \mathbb{E}_{\theta_0}\left[ \left. \left[ \frac{1}{n} \sum_{i = 1}^n \frac{\partial^2 \log(p_X(x_i, \theta))}{\partial \theta \partial \theta^\top} \right] \right\vert_{\theta = \theta_0} \right] = H \nonumber $$ where the righthand side is the expected Hessian, denoted by $H$.
Since we assumed the log-likelihood is twice continuously differentiable, the inverse function theorem gives us: $$ -\left[\left. \left[ \frac{1}{n} \sum_{i = 1}^n \frac{\partial^2 \log(p_X(x_i, \theta))}{\partial \theta \partial \theta^\top} \right] \right\vert_{\theta = \bar{\theta}} \right]^{-1} \overset{p}{\rightarrow} -H^{-1} \nonumber $$ Since $-H^{-1}$ is a constant, by Slutsky's theorem: $$ \sqrt{n}(\hat{\theta} - \theta_0) \rightsquigarrow \mathcal{N}\left(\mathbf{0}, H^{-1}\mathcal{I}(\theta_0) H^{-1}\right) $$ Under our regularity conditions, the negative expected Hessian is the variance of the score, so: $$ -H^{-1} = \mathcal{I}^{-1}(\theta_0) \implies \mathcal{N}\left(\mathbf{0}, H^{-1}\mathcal{I}(\theta_0) H^{-1}\right) = \mathcal{N}\left(\mathbf{0}, \mathcal{I}^{-1}(\theta_0)\right) \nonumber $$

Newey and McFadden make some important points about the conditions stated in Theorem 3.1. Perhaps most notable is the need for the true parameter value to be located in the interior of the parameter space. When this condition is not met ($\hat{\theta}_n$ is on the boundary even as $n \rightarrow \infty$), the asymptotic normality result is no longer guaranteed (though it could still be true). This comes up in variance estimation, since we usually constrain our space to positive reals. We also need the average score over the sample to satisfy a central limit theorem (this gives us the “base” Normal distribution) and the inverse Hessian needs to converge to a constant (so we can apply Slutsky’s theorem).

Estimating Asymptotic Variance

For confidence intervals, we need a consistent estimator of the asymptotic variance, $H^{-1} \Sigma H^{-1}$, of the estimator. Usually we do this by estimating the components and then plugging these in; i.e. we find $\hat{H}^{-1}$ and $\hat{\Sigma}$ and use $\hat{H}^{-1} \hat{\Sigma} \hat{H}^{-1}$

This estimator is sometimes called a sandwich estimator since we have the asymptotic variance “sandwiched” between the same term twice.

For maximum likelihood estimators, one can estimate the asymptotic variance with any consistent estimator of the inverse Fisher information matrix. Some examples provided by Newey and McFadden are to use the negative Hessian (if regularity conditions are met) or the sample average of the outer product of the score. However, one should use caution because, under model misspecification, estimators of the inverse information matrix may not be consistent.

---

Efficiency

We can compare different asymptotically normal estimators by their efficiency. But before we begin, we need to define a few new quantities.

We can define a similar space when the parameter vector can be partitioned as $\theta = (\beta^\top, \eta^\top)^\top$. The nuisance tangent space is the linear subspace of $\mathcal{H}$ spanned by the nuisance score vector $U_\eta(X, \theta_0)$:

\[\Lambda = \left\{ B U_\eta(X, \theta_0): B \in \mathbb{R}^{q \times r}\right\} \label{eq:nuisance-space}\]

The tangent space generated by $U_\beta(X, \theta_0)$ is:

\[\mathcal{T}_\beta = \left\{ B U_\beta(X, \theta_0): B \in \mathbb{R}^{q \times p} \right\} \label{eq:interest-space}\]

Notably, the direct sum of these two spaces equals the tangent space generated by the entire score vector:

\[\mathcal{T} = \mathcal{T}_\beta \oplus \Lambda\]

We can also construct the set of elements that are orthogonal to $\Lambda$ are then given by $h - \Pi(h \rvert \Lambda)$ for all $h \in \mathcal{H}$ where $\Pi(h \rvert \Lambda)$ is the residual of $h$ after projecting onto $\Lambda$.

Efficient Influence Functions

We call the influence function with the smallest variance matrix the efficient influence function. In the case of RAL estimators, we say that $\psi^{(1)}(X, \theta_0)$ has smaller (asymptotic) variance than $\psi^{(2)}(X, \theta_0)$ if:

\[\text{Var}(\psi^{(1)}(X, \theta_0)) \leq \text{Var}(\psi^{(2)}(X, \theta_0)) \iff \text{Var}(a^\top \psi^{(1)}(X, \theta_0)) \leq \text{Var}(a^\top \psi^{(2)}(X, \theta_0)), \hspace{10mm} \forall q \times 1 \text{ constant vectors } a\]

Implications.

Since influence functions have zero mean, the righthand side of the above condition is equivalent to: $$ a^\top \mathbb{E}_{\theta_0}[\psi^{(1)}(X, \theta_0) (\psi^{(1)}(X, \theta_0))^\top]a \leq a^\top \mathbb{E}_{\theta_0}[\psi^{(2)}(X, \theta_0) (\psi^{(2)}(X, \theta_0))^\top]a , \hspace{10mm} \forall q \times 1 \text{ constant vectors } a $$ which itself is equivalent to: $$ a^\top \left[ \mathbb{E}_{\theta_0}[\psi^{(2)}(X, \theta_0) (\psi^{(2)}(X, \theta_0))^\top] - \mathbb{E}_{\theta_0}[\psi^{(1)}(X, \theta_0) (\psi^{(1)}(X, \theta_0))^\top] \right] a \geq 0, \hspace{10mm} \forall q \times 1 \text{ constant vectors } a $$

This brings us to a theorem that defines the subspace of influence functions.

This result states that we can construct all influence functions of RAL estimators by taking an arbitrary influence function and adding any element from the orthogonal complement of the tangent space to it. Theorem 3.4 can be used to define the efficient influence function.

Efficient Score

We can define an efficient version of the score if we are in the setting where we can partition $\theta = (\beta^\top, \eta^\top)^\top$.

In this setting, we can construct the efficient influence function as:

\[\psi_\text{eff}(X, \theta_0) = \left[ \mathbb{E}_{\theta_0}\left[ U_\text{eff}(X, \theta_0) U^\top_\text{eff}(X, \theta_0) \right] \right]^{-1} U_\text{eff}(X, \theta_0) \label{eq:partitioned-efficient-influence}\]

and has variance equal to:

\[\text{Var}(\psi_{\text{eff}}(X, \theta_0)) = \left[ \mathbb{E}_{\theta_0} \left[ U_\text{eff}(X, \theta_0) U^\top(X, \theta_0) \right] \right]^{-1}\]

which is the inverse variance matrix of the efficient score. If we partition the variance of the score vector as:

\[\text{Var}(U_\theta(X, \theta_0)) = \mathcal{I} = \begin{bmatrix} \mathcal{I}_{\beta, \beta} = \mathbb{E}_{\theta_0}\left[ U_\beta(X, \theta_0) U^\top_\beta(X, \theta_0) \right] & \mathcal{I}_{\beta, \eta} = \mathbb{E}_{\theta_0}\left[ U_\beta(X, \theta_0) U^\top_\eta(X, \theta_0) \right] \\ \mathcal{I}_{\eta, \beta} = \mathbb{E}_{\theta_0}\left[ U_\eta(X, \theta_0) U^\top_\beta(X, \theta_0) \right] & \mathcal{I}_{\eta, \eta} = \mathbb{E}_{\theta_0}\left[ U_\eta(X, \theta_0) U^\top_\eta(X, \theta_0) \right] \end{bmatrix}\]

Then we can use the Schur complement formula to get the varaince of the efficient influence function is:

\[\text{Var}(\psi_{\text{eff}}(X, \theta_0)) = \left[\mathcal{I}_{\beta, \beta} - \mathcal{I}_{\beta, \eta} \mathcal{I}^{-1}_{\eta, \eta} \mathcal{I}^\top_{\beta, \eta} \right]^{-1} \label{eq:var-eff-influence}\]

Note that in Lin¹, Eq. \eqref{eq:var-eff-influence} is called the efficient information matrix.

---

Appendix

Auxiliary Definitions

Auxiliary Results

The following result is called the Information inequality and states that if $\theta_0$ can be identified using a MLE, then the limiting objective function will have a unique maximum at the true value.

---

References