scikit-learn 1.4版本发布亮点

很高兴地宣布scikit-learn1.4版本的发布!此版本包含了许多bug修复和改进,还引入了一些关键的新特性。以下是这个版本的主要新特性。有关所有更改的详尽列表,请参考发布说明。

要安装最新版本,可以使用pip:

pip install --upgrade scikit-learn

或者使用conda:

conda install -c conda-forgescikit-learn

直方图梯度提升原生支持DataFrame中的分类数据类型

现在,ensemble.HistGradientBoostingClassifierensemble.HistGradientBoostingRegressor直接支持包含分类特征的DataFrame。这里有一个包含分类和数值特征的数据集:

from sklearn.datasets import fetch_openml X_adult, y_adult = fetch_openml( "adult", version=2, return_X_y=True) # 移除冗余和非特征列 X_adult = X_adult.drop(["education-num", "fnlwgt"], axis="columns") X_adult.dtypes age int64 workclass category education category marital-status category occupation category relationship category race category sex category capital-gain int64 capital-loss int64 hours-per-week int64 native-country category dtype: object

通过设置categorical_features="from_dtype",梯度提升分类器将把具有分类数据类型的列作为算法中的分类特征处理:

from sklearn.ensemble import HistGradientBoostingClassifier from sklearn.model_selection import train_test_split from sklearn.metrics import roc_auc_score X_train, X_test, y_train, y_test = train_test_split( X_adult, y_adult, random_state=0) hist = HistGradientBoostingClassifier( categorical_features="from_dtype") hist.fit(X_train, y_train) y_decision = hist.decision_function(X_test) print(f"ROC AUC score is {roc_auc_score(y_test, y_decision)}")

ROC AUC得分是0.9281447774661015。

在set_output中支持Polars输出

scikit-learn的转换器现在支持使用set_output API输出Polars。

import polars as pl from sklearn.preprocessing import StandardScaler from sklearn.preprocessing import OneHotEncoder from sklearn.compose import ColumnTransformer df = pl.DataFrame({ "height": [120, 140, 150, 110, 100], "pet": ["dog", "cat", "dog", "cat", "cat"]}) preprocessor = ColumnTransformer([ ("numerical", StandardScaler(), ["height"]), ("categorical", OneHotEncoder(sparse_output=False), ["pet"]), ], verbose_feature_names_out=False,) preprocessor.set_output(transform="polars") df_out = preprocessor.fit_transform(df) print(f"Output type: {type(df_out)}")

输出类型:class 'polars.dataframe.frame.DataFrame'

随机森林支持缺失值

ensemble.RandomForestClassifierensemble.RandomForestRegressor现在支持缺失值。在训练每棵单独的树时,分割器会评估每个潜在的阈值,并将缺失值分配到左右节点。更多细节请参考用户指南。

import numpy as np from sklearn.ensemble import RandomForestClassifier X = np.array([0, 1, 6, np.nan]).reshape(-1, 1) y = [0, 0, 1, 1] forest = RandomForestClassifier(random_state=0) forest.fit(X, y) forest.predict(X)

输出:[0, 0, 1, 1]

在基于树的模型中添加对单调约束的支持

在scikit-learn 0.23中为直方图梯度提升添加了对单调约束的支持,现在为所有其他基于树的模型也添加了这一特性,包括树、随机森林、额外树和精确梯度提升。这里展示了随机森林在回归问题上使用这一特性的例子。

import matplotlib.pyplot as plt from sklearn.inspection import PartialDependenceDisplay from sklearn.ensemble import RandomForestRegressor n_samples = 500 rng = np.random.RandomState(0) X = rng.randn(n_samples, 2) noise = rng.normal(loc=0.0, scale=0.01, size=n_samples) y = 5 * X[:, 0] + np.sin(10 * np.pi * X[:, 0]) - noise rf_no_cst = RandomForestRegressor().fit(X, y) rf_cst = RandomForestRegressor(monotonic_cst=[1, 0]).fit(X, y) disp = PartialDependenceDisplay.from_estimator(rf_no_cst, X, features=[0], feature_names=["feature 0"], line_kw={"linewidth": 4, "label": "unconstrained", "color": "tab:blue"},) PartialDependenceDisplay.from_estimator(rf_cst, X, features=[0], line_kw={"linewidth": 4, "label": "constrained", "color": "tab:orange"}, ax=disp.axes_,) disp.axes_[0, 0].plot(X[:, 0], y, "o", alpha=0.5, zorder=-1, label="samples", color="tab:green") disp.axes_[0, 0].set_ylim(-3, 3) disp.axes_[0, 0].set_xlim(-1, 1) disp.axes_[0, 0].legend() plt.show()

增强的估计器显示

估计器的显示已经增强:如果查看上面定义的forest,在Jupyter环境中,请重新运行此单元格以显示HTML表示,或信任笔记本。在GitHub上,HTML表示无法渲染,请尝试使用nbviewer.org加载此页面。

from sklearn.base import clone clone(forest)

克隆的模型尚未拟合。

元数据路由支持

许多元估计器和交叉验证例程现在支持元数据路由,这些在用户指南中列出。例如,可以使用样本权重和GroupKFold进行嵌套交叉验证:

import sklearn from sklearn.metrics import get_scorer from sklearn.datasets import make_regression from sklearn.linear_model import Lasso from sklearn.model_selection import GridSearchCV, cross_validate, GroupKFold # 默认情况下元数据路由被禁用,需要显式启用。 sklearn.set_config(enable_metadata_routing=True) n_samples = 100 X, y = make_regression(n_samples=n_samples, n_features=5, noise=0.5) rng = np.random.RandomState(7) groups = rng.randint(0, 10, size=n_samples) sample_weights = rng.rand(n_samples) estimator = Lasso().set_fit_request(sample_weight=True) hyperparameter_grid = {"alpha": [0.1, 0.5, 1.0, 2.0]} scoring_inner_cv = get_scorer("neg_mean_squared_error").set_score_request(sample_weight=True) inner_cv = GroupKFold(n_splits=5) grid_search = GridSearchCV(estimator=estimator, param_grid=hyperparameter_grid, cv=inner_cv, scoring=scoring_inner_cv,) outer_cv = GroupKFold(n_splits=5) scorers = {"mse": get_scorer("neg_mean_squared_error").set_score_request(sample_weight=True)} results = cross_validate(grid_search, X, y, cv=outer_cv, scoring=scorers, return_estimator=True, params={"sample_weight": sample_weights, "groups": groups},) print("cv error on test sets:", results["test_mse"]) # 将标志设置为默认的`False`以避免与其他脚本的干扰。 sklearn.set_config(enable_metadata_routing=False)

测试集上的cv错误:[-0.38473211 -0.14486073 -0.2538949 -0.33416118 -0.38869178]

PCA现在能够原生处理稀疏矩阵,对于arpack求解器,通过利用scipy.sparse.linalg.LinearOperator来避免在执行数据集协方差矩阵的特征值分解时显式化大型稀疏矩阵。

from sklearn.decomposition import PCA import scipy.sparse as sp from time import time X_sparse = sp.random(m=1000, n=1000, random_state=0) X_dense = X_sparse.toarray() t0 = time() PCA(n_components=10, svd_solver="arpack").fit(X_sparse) time_sparse = time() - t0 t0 = time() PCA(n_components=10, svd_solver="arpack").fit(X_dense) time_dense = time() - t0 print(f"Speedup: {time_dense / time_sparse:.1f}x")

加速:4.4x

沪ICP备2024098111号-1
上海秋旦网络科技中心:上海市奉贤区金大公路8218号1幢 联系电话:17898875485