LightGBM - 特徵互動約束

當LightGBM完成對資料集上整合樹的訓練後，每個節點都表示由特徵值確定的條件。當使用單個樹進行預測時，我們從根節點開始，並將節點中給定的特徵條件與我們的樣本特徵值進行比較。我們根據樣本中的特徵值和樹的條件做出決策。這允許我們透過採取特定路徑到達樹的葉節點來生成最終預測。預設情況下，對哪些節點可以具有哪些功能沒有限制。

這種透過迭代樹的節點並分析特徵條件來生成最終決策的方法被稱為特徵互動，因為預測器在評估前一個節點的狀態後到達特定節點。LightGBM允許我們決定哪些特徵可以相互互動。我們可以定義一組索引，只有這些特徵才能相互互動。這些特徵將無法與其他特徵互動，並且在訓練階段生成樹時將強制執行此限制。

我們已經展示瞭如何在LightGBM中對估計器強制執行特徵互動約束。LightGBM估計器有一個名為interaction_constraints的引數，它接受一個列表的列表，每個列表都包含可以相互互動的引數的索引。

示例1

這是一個關於如何在LightGBM估計器中強制執行特徵互動約束的示例。

來自sklearn.datasets的load_boston函式在某些版本的scikit-learn中可能已棄用。如果出現任何錯誤，您可以從外部資源載入資料集或使用替代資料集。

# Import necessary libraries
import lightgbm as lgb
from sklearn.datasets import load_boston
from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score

# Load the Boston housing dataset
boston = load_boston()

# Split the data into training and testing sets
X_train, X_test, Y_train, Y_test = train_test_split(boston.data, boston.target, train_size=0.90, random_state=42)

# Print the size of the training and testing sets
print("Sizes of Train or Test Datasets : ", X_train.shape, X_test.shape, Y_train.shape, Y_test.shape, "\n")

# Create LightGBM datasets
train_dataset = lgb.Dataset(X_train, Y_train, feature_name=boston.feature_names.tolist())
test_dataset = lgb.Dataset(X_test, Y_test, feature_name=boston.feature_names.tolist())

# Train the LightGBM model
booster = lgb.train({
    "objective": "regression",
    "verbosity": -1,
    "metric": "rmse",
    'interaction_constraints': [[0,1,2,11,12], [3,4], [6,10], [5,9], [7,8]]
    },
    train_set=train_dataset,
    valid_sets=(test_dataset,),
    num_boost_round=10
)

# Make predictions
test_preds = booster.predict(X_test)
train_preds = booster.predict(X_train)

# Calculate and print R2 scores
print("\nR2 Test Score : %.2f" % r2_score(Y_test, test_preds))
print("R2 Train Score : %.2f" % r2_score(Y_train, train_preds))

輸出

這將生成以下結果

Sizes of Train or Test Datasets :  (455, 13) (51, 13) (455,) (51,)

[1]	valid_0's rmse: 7.50225
[2]	valid_0's rmse: 7.01989
[3]	valid_0's rmse: 6.58246
[4]	valid_0's rmse: 6.18581
[5]	valid_0's rmse: 5.83873
[6]	valid_0's rmse: 5.47166
[7]	valid_0's rmse: 5.19667
[8]	valid_0's rmse: 4.96259
[9]	valid_0's rmse: 4.69168
[10]	valid_0's rmse: 4.51653

R2 Test Score : 0.67
R2 Train Score : 0.69

示例2

以下程式碼訓練一個LightGBM模型，使用波士頓資料集預測房價。訓練後，它將使用R²得分計算模型在訓練資料和測試資料上的效果。

# Import necessary libraries
import lightgbm as lgb
from sklearn.datasets import load_boston
from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score

# Load the Boston housing dataset
boston = load_boston()

# Split the dataset into training and testing sets
X_train, X_test, Y_train, Y_test = train_test_split(boston.data, boston.target, test_size=0.2, random_state=42)

# Print the size of the training and testing sets
print("Sizes of Training and Testing Datasets : ", X_train.shape, X_test.shape, Y_train.shape, Y_test.shape)

# Create a LightGBM model with interaction constraints and 10 estimators
booster = lgb.LGBMModel(objective="regression", n_estimators=10,
                        interaction_constraints=[[0,1,2,11,12], [3, 4], [6,10], [5,9], [7,8]])

# Train the model on the training set and validate it on the test set
booster.fit(X_train, Y_train,
            eval_set=[(X_test, Y_test)],
            eval_metric="rmse")

# Make predictions on both the test and training sets
test_preds = booster.predict(X_test)
train_preds = booster.predict(X_train)

# Calculate and print the R2 score for the test and training sets
print("\nR2 Test Score : %.2f" % r2_score(Y_test, test_preds))
print("R2 Train Score : %.2f" % r2_score(Y_train, train_preds))

輸出

這將產生以下結果

Sizes of Training and Testing Datasets :  (379, 13) (127, 13) (379,) (127,)
[1]	valid_0's rmse: 8.97871	valid_0's l2: 80.6173
[2]	valid_0's rmse: 8.35545	valid_0's l2: 69.8135
[3]	valid_0's rmse: 7.93432	valid_0's l2: 62.9535
[4]	valid_0's rmse: 7.61104	valid_0's l2: 57.9279
[5]	valid_0's rmse: 7.16832	valid_0's l2: 51.3849
[6]	valid_0's rmse: 6.93182	valid_0's l2: 48.0501
[7]	valid_0's rmse: 6.57728	valid_0's l2: 43.2606
[8]	valid_0's rmse: 6.41497	valid_0's l2: 41.1518
[9]	valid_0's rmse: 6.13983	valid_0's l2: 37.6976
[10]	valid_0's rmse: 5.9864	valid_0's l2: 35.837

R2 Test Score : 0.60
R2 Train Score : 0.69

列印頁面