Resampling a normally distributed dataset for regression problems?

Question

I have a dataset from an operating process having 5 measurements and 1 outcome. All values are normally distributed. When I train a regression model on the dataset it performs good on the majority of the dataset - the default operating condition of the process. It performs much worse though on other than default operating conditions, values distant from the mean. If it were a classification problem I would treat this as class imbalance and perform some resampling technique to get balanced classes. How do I treat this for a regression problem?

Answer 1

One option is Bayesian regression. Instead of estimating a point estimate for each regression coefficient, Bayesian regression estimate a posterior distribution. A posterior distribution will be better model the entire empirical range of values.