Identifying Interactions among Categorical Predictors with Monte-Carlo Tree Search
Identifying interpretable interactions among categorical predictors for predictive modeling is crucial in various research fields. Recent studies have examined interpretable interactions using decision tree (DT) learning methods, which construct DTs by greedy rules due to the high memory and time complexity of building and evaluating DTs, resulting in a local optimal solution. This paper formulates the selection of quadratic and higher order interactive terms into a LASSO problem and then relaxes it into multiple DT learning problems. A Monte Carlo Tree Search-based interaction selection (MCTs-IS) method is proposed to identify the optimal DT in an online learning manner. A DT pruning strategy is developed based on LASSO that can easily be applied to MCTs. We prove that MCTs-IS converges with high probability to the optimal solution of the DT learning problem. Extensive experiments have been conducted to demonstrate the effectiveness of the proposed algorithm on real-world datasets.
PDF Abstract