A Parsimonious Hedonic Distributional Regression Model for Large Data with Heterogeneous Covariate Effects

Abstract

Modeling real estate prices in the context of hedonic models often involves fitting a Generalized Additive Model, where only the mean of a (lognormal) distribution is regressed on a set of variables without taking other parameters of the distribution into account. Thus far, the application of regression models that model the full conditional distribution of the prices, has been infeasible for large data sets, even on powerful machines. Moreover, accounting for heterogeneity of effects regarding time and locale, is often achieved by naive stratification of the data rather than on a model basis. 
A novel batchwise backfitting algorithm is applied in the context of a structured additive distributional regression model, which enables us to efficiently model all distributional parameters of the price distribution. Using a large German dataset of apartment asking prices with over one million observations, we employ a model-based clustering algorithm to capture the heterogeneity of covariate effects on the parameters with respect to dwelling locale. We thus identify clusters that are homogeneous with respect to the influence of dwelling locale on price. A boosting type algorithm of the batchwise backfitting algorithm is then used to automatically determine the variables relevant for modelling the location and scale parameters in each regional cluster. This allows for a different influence of variables on the distribution of prices depending on the locale and price segment of the dwelling.