Non-asymptotic Confidence Estimation of the Autoregressive Parameter in AR(1) Process with an Unknown Noise Variance

The paper considers the estimation problem of the autoregressive parameter in the first-order autoregressive process with Gaussian noises when the noise variance is unknown. We propose a non-asymptotic technique to compensate the unknown variance, and then, to construct a point estimator with any prescribed mean square accuracy. Also a fixed-width confidence interval with any prescribed coverage accuracy is proposed. The results of Monte-Carlo simulations are given.


Introduction
The problem of constructing a fixed-width confidence interval with any prescribed accuracy by a finite sample size might be complicated enough even for the process with independent observations. For the case of Gaussian independent variables with an unknown variance, Stein (1945) proposed a two-stage sequential procedure to construct such interval for an unknown mean.
The models of stochastic processes described by a stochastic difference and stochastic differential are widely used in the problems of optimal control and prediction, in finance mathematics, in the time series analysis. To estimate unknown parameters, the maximum likelihood method and the least squares method are used. The quality of the obtained estimators is studied usually in the asymptotic statement, when the number of observations tends to infinity.
The problem of estimation with any prescribed accuracy of the first-order autoregressive process parameter was considered in Borisov and Konev (1977). A sequential estimator was proposed. The choice of a stopping instant guarantees the upper bound of the mean square accuracy. To construct this estimator one needs to know the variance of the noises. Note that Lai and Siegmund (1983) proposed a similar procedure but only asymptotic properties of the estimator were investigated. The sequential estimator of an unknown parameter of a diffusion-type process with any prescribed mean square accuracy was described in Novikov (1972). In Dmitrienko and Konev (1994), a two-stage procedure to construct the estimator of an unknown parameter if the noise variance is unknown was introduced. At the first stage, the upper bound of the variance is obtained. It should be noted that if the absolute value of the autoregressive parameter is close to unity, the estimator Dmitrienko and Konev (1994) exceeds manifold the variance; hence, the estimation time increases dramatically.
In Konev and Vorobeychikov (2017), the sequential estimation procedure of Borisov and Konev (1977) was modified; it allows obtaining a point autoregressive parameter estimator with a non-asymptotic Gaussian distribution and constructing a fixed-width confidence interval with any prescribed probability of coverage. We propose to use this estimator to construct a modified three-stage estimation procedure for AR(1) process with an unknown noise variance. Unlike Dmitrienko and Konev (1994), we use an additional stage to obtain an estimator of an unknown autoregressive parameter and then improve the upper bound of an unknown variance. It leads to the decrease of the estimation time as compared with Dmitrienko and Konev (1994).
The confidence interval for the parameter in the autoregressive process was proposed in Chow and Robbins (1965), Lee and Sriram (1999), Sriram (2001), Wei, Hao, and Ching (2018). The properties of the estimators were investigated in the asymptotic statement, as the number of observations tends to infinity. In our paper, we construct a fixed-size confidence interval with any prescribed coverage probability. The presented results of the simulation demonstrate a good quality of the algorithm.

Problem statement
Consider the first-order autoregressive model AR(1) defined as follows: where θ and b are unknown real parameters, ε k are independent identically distributed random variables, Eε k = 0, Eε 2 k = 1. The first problem is to construct a point estimator for θ with the prescribed mean-square deviation on the basis of the observations {x k }. The second problem is to construct a fixed-width confidence interval with any prescribed coverage accuracy using this point estimator. We solve the problem under certain constraints imposed on the noise density function f ε (x), which are specified in the next section.

Three-stage sequential point estimator
We propose a three-stage procedure to estimate the parameter the θ in model (1), based on the modified sequential least-squares method. At the first stage, we estimate the autoregressive parameter using the least squares method. At the second stage, we construct a special multiplier to compensate an unknown noise variance. At the third stage, we estimate the autoregressive parameter.
To compensate the unknown noise variance, we need a pilot estimator of the unknown autoregressive parameter θ; we use the usual least-squares estimator (2) The compensating factor has the following form where the factor C f (l) is determined by the density function of the noise In the case of the Gaussian distribution, C f (l) = 1/(l − 2); so, according to Dmitrienko and Konev (1994) the parameter l should be not less that 3, to provide the limited expectation of the multiplier 1/Γ l . However, we recommend to take l ≥ 10, which makes the estimator (3) more stable.
The following theorem establishes an important property of the multiplier Γ resulting in the non-asymptotic properties of the proposed estimator.
Theorem 1. Let the noise density function, f ε (y), be a symmetric function, P roof. To prove the theorem, we need an auxilary result first presented in Anderson (1956).
Theorem 2. Let C be a convex set, symmetric about the origin. Let f (y) ≥ 0 be a function such that: Introduce the following notation for the sum in (5) Let F N be a sigma-algebra generated by {x 0 , ε 1 , ..., ε N }, and F N K be a sigma-algebra generated by {ε K , ..., ε N }; then, using properties of the conditional expectation together with (1), we have Note that S t+l−1 and (θ −θ)x t+l−1 are adapted to the σ−algebra F t+l−1 , whereas ε t+l does not depend on the σ−algebra. Let us introduce the notations Here C is a convex. If the noise density function f ε (t) meets the conditions of Theorem 1 then it satisfies the conditions of Theorem 2. Applying Theorem 2 for k = 0, we obtain Now we introduce the notation V N K = N k=K b 2 ε 2 k and prove that for any K = t + l, ..., t, We have just obtained this result for K = t + l. Suppose the statement is true for K = t + l, ..., M + 1 and consider it for K = M .
As (θ −θ)x M −1 and S M −1 are adapted to F M −1 and V t+l M +1 is adapted to F t+l M +1 , we can apply Theorem 2 and obtain It implies (8) for K = M ; hence, it is true for any K which implies the Theorem.
The following theorem provides an important property of the compensating factor Γ (3) which allows us to bound from above the standard deviation of the proposed estimator.
Theorem 3. Let the noise density function f ε (y) be a symmetric function, f ε (t) = f ε (−t), and let f ε (t) decrease in the interval [0, +∞). Then for the compensating factor Γ (3) Proof. Let us estimate E 1 Γ l . Using (3) and (7), we have Using properties of the expectation, we obtain According the result of Theorem 2 Using this result in (10), we obtain which implies the Theorem.
At the third stage we construct an estimator for the parameter θ on the basis of the improved sequential point estimator proposed in Konev and Vorobeychikov (2017), which is a special modification of the least squares (maximum likelihood) estimators. For each H > 0 we introduce the stopping instance and define a sequential estimator by the following formula where β k = 1 if k < τ and β τ = α τ , α τ is the correction factor, 0 < α τ ≤ 1, uniquely defined by the equation Theorem 4. In the conditions of Theorem 1, stopping instant (11) is finite with the probability one. The mean square deviation of estimator (12) is bounded from above Proof. Using (1) in (12), we obtain Note that, according to Theorem 6.1 in Konev and Vorobeychikov (2017), the conditional distribution of m(H) subject to F τ +l is the standard Gaussian distribution. Taking into account inequalities (9) and H ≤H we obtain Hence the Theorem.
Finally, we construct a fixed-width confidence interval covering the parameter θ with the prescribed probability α.
Proof. Consider the deviation θ * (H) − θ. Using (14) one has For the confidence interval of the deviation, we have AsH ≥ H, we have Let us denote the distribution function of Γ l /b 2 as G l (y). Then According to Theorem 1, As ε k are standard Gaussian random variables, the sum of their squares has the χ 2 (l) distribution. According to Theorem 6.1 in Konev and Vorobeychikov (2017), where Φ(x) is the standard Gaussian distribution function. This fact allows constructing the confidence interval for θ where Γ(l/2) is the gamma-function. This implies the Theorem.

Simulation results
In this section, we report and discuss the results of Monte Carlo experiments. Table 1 presents selected data obtained by the simulations. For our study, we set θ = ±0.1, ±0.3, ±0.5, ±0.7, ±0.9, ±0.99.  For each θ, 1000 replications were run. The quantities recorded in Table 1 are: H -the threshold in the sequential sampling rule; t -the number of observations to obtain a pilot estimator of θ; θ -the autoregressive parameter; l -the number of observations to estimate the unknown variance; ∆ 2 -the mean square deviation for θ * (l, H); τ 2 -the mean numbers of observations. We also compared our results with the estimator described in Dmitrienko and Konev (1994), here ∆ 1 -the mean square deviation for the estimator Dmitrienko and Konev (1994); τ 1 -the mean numbers of observations.