v1.2.2

A v1.2 in-line release with upstream scikit-learn.

This version is backwards compatible with v1.2 of scikit-learn with the exception of the changelog mentioned in the README. These changes are mostly to private API with the exception of the max_bins parameter added to the Forest based models in sklearn/ensemble/. Currently the max_bins feature on this fork is experimental and should be used with caution.

The details of the changes are listed below.

Cython Internal Private API:

Note, the Cython API for scikit-learn is still not a publicly supported API, so it may
change without warning.

• leaf and split nodes: These nodes are treated the same way and there is no internal
  API for setting them differently. Quantile trees and causal trees inherently generalize
  how leaf nodes are set.
• Criterion class: The criterion class currently assumes a supervised learning interface.
  • Our fix: We implement a BaseCriterion object that provides an abstract API for unsupervised criterion.
• Splitter class: The splitter clas currently assumes a supervised learning interface and
  does not provide a way of generalizing the way split candidates are proposed.
  • Our fix: We implement a BaseSplitter object that provides an abstract API for unsupervised splitters and also implement an API to allow generalizations of the SplitRecord struct and Splitter.node_split function. For example, this enables oblique splits to be considered.
• Tree class: The tree class currently assumes a supervised learning interface and does not
  provide a way of generalizing the type of tree.
  • Our fix: We implementa BaseTree object that provides an abstract API for general tree models and also implement an API that allows generalization of the type of tree. For example, oblique trees are trivially implementable as an extension now.
• stopping conditions for splitter: Currently, the Splitter.node_split function has various
  stopping conditions for the splitter based on hyperparameters. It is plausible that these conditions
  may be extended. For example, in causal trees, one may want the splitter to also account for
  a minimal degree of heterogeneity (i.e. variance) in its children nodes.

Python API:

• sklearn.tree.BaseDecisionTree assumes the underlying tree model is supervised: The y
  parameter is required to be passed in, which is not necessary for general tree-based models.
  For example, an unsupervised tree may pass in y=None.
  • Our fix: We fix this API, so the BaseDecisionTree is subclassable by unsupervised tree models that do not require y to be defined.
• sklearn.tree.BaseDecisionTree does not provide a way to generalize the Criterion, Splitter
  and Tree Cython classes used: The current codebase requires users to define custom
  criterion and/or splitters outside the instantiation of the BaseDecisionTree. This prevents
  users from generalizing the Criterion and Splitter and creating a neat Python API wrapper.
  Moreover, the Tree class is not customizable.
  • Our fix: We internally implement a private function to actually build the entire tree, BaseDecisionTree._build_tree, which can be overridden in subclasses that customize the criterion, splitter, or tree, or any combination of them.
• sklearn.ensemble.BaseForest and its subclass algorithms are slow when n_samples is very high. Binning
  features into a histogram, which is the basis of "LightGBM" and "HistGradientBoostingClassifier" is a computational
  trick that can both significantly increase runtime efficiency, but also help prevent overfitting in trees, since
  the sorting in "BestSplitter" is done on bins rather than the continuous feature values. This would enable
  random forests and their variants to scale to millions of samples.
  • Our fix: We added a max_bins=None keyword argument to the BaseForest class, and all its subclasses. The default behavior is no binning. The current implementation is not necessarily efficient. There are several improvements to be made. See below.

Overall, the existing tree models, such as sklearn.tree.DecisionTreeClassifier <https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html>_
and sklearn.ensemble.RandomForestClassifier <https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier>_ all work exactly the same as they
would in scikit-learn main, but these extensions enable 3rd-party packages to extend
the Cython/Python API easily.