Oblique Bayesian additive regression trees

Abstract : Current implementations of Bayesian Additive Regression Trees (BART) are based on axis-aligned decision rules that recursively partition the feature space using a single feature at a time. Several authors have demonstrated that oblique trees, whose decision rules are based on linear combinations of features, can sometimes yield better predictions than axis-aligned trees and exhibit excellent theoretical properties. We develop an oblique version of BART that leverages a data-adaptive decision rule prior that recursively partitions the feature space along random hyperplanes. Using several synthetic and real-world benchmark datasets, we systematically compared our oblique BART implementation to axis-aligned BART and other tree ensemble methods, finding that oblique BART was competitive with - and sometimes much better than - those methods.

A pre-print is available here.

Recommended citation: Nguyen, P-H.V., Yee, R., Deshpande, S.K. (2024+). "Oblique Bayesian additive regression trees."