Python 3.x 使用xgb和xgb分类器的CPU速度比GPU快_Python 3.x_Gpu_Cpu_Xgboost

Python 3.x 使用xgb和xgb分类器的CPU速度比GPU快

python-3.x

Python 3.x 使用xgb和xgb分类器的CPU速度比GPU快,python-3.x,gpu,cpu,xgboost,Python 3.x,Gpu,Cpu,Xgboost,我先道歉，因为我是初学者。我正在使用xgb和XGBclassifier使用XGBoost测试GPU和CPU。结果如下： passed time with xgb (gpu): 0.390s passed time with XGBClassifier (gpu): 0.465s passed time with xgb (cpu): 0.412s passed time with XGBClassifier (cpu): 0.421s 我想知道为什么CPU看起来比GPU

我先道歉，因为我是初学者。我正在使用xgb和XGBclassifier使用XGBoost测试GPU和CPU。结果如下：

   passed time with xgb (gpu): 0.390s
   passed time with XGBClassifier (gpu): 0.465s
   passed time with xgb (cpu): 0.412s
   passed time with XGBClassifier (cpu): 0.421s

我想知道为什么CPU看起来比GPU性能更好。这是我的设置：

Python 3.6.1
操作系统：Windows 10 64位
GPU:NVIDIA RTX 2070 Super 8gb vram（驱动程序更新至最新版本）
CUDA 10.1已安装
CPU i7 10700 2.9Ghz
在Jupyter笔记本上运行
通过pip安装xgboost 1.2.0的夜间版本

**还尝试使用预先构建的二进制轮子安装的xgboost版本，使用pip:相同问题

下面是我正在使用的测试代码（摘自）：

我曾尝试合并Sklearn grid搜索，看看是否能在GPU上获得更快的速度，但结果比CPU慢得多：

passed time with XGBClassifier (gpu): 2457.510s
Best parameter (CV score=0.490):
{'xgbclass__alpha': 100, 'xgbclass__eta': 0.01, 'xgbclass__gamma': 0.2, 'xgbclass__max_depth': 5, 'xgbclass__n_estimators': 100}


passed time with XGBClassifier (cpu): 383.662s
Best parameter (CV score=0.487):
{'xgbclass__alpha': 100, 'xgbclass__eta': 0.1, 'xgbclass__gamma': 0.2, 'xgbclass__max_depth': 2, 'xgbclass__n_estimators': 20}

我使用的数据集有75k个观测值。你知道为什么我用GPU没有加速吗？数据集是否太小，无法从使用GPU中获得收益

任何帮助都将不胜感激。多谢各位

选择CPU vs GPU

神经网络的复杂性还取决于输入特征的数量，而不仅仅是隐藏层中的单位数量。如果您的隐藏层有50个单位，并且数据集中的每个观测值都有4个输入特征，那么您的网络很小（约200个参数）。如果每个观察结果都有5M个输入特征，就像在一些大的上下文中一样需要处理，那么你的网络在参数数量上是相当大的

根据我的观察，上面有一些参数需要处理，因此在GPU中需要花费大量时间

根据我个人的经验：

我曾经用CNN算法在GPU和CPU中训练一些图像进行预测 CPU在整个数据集上生成经过训练的模型所需的处理时间较低，但GPU需要更多的时间

访问

有趣的问题。正如您所注意到的，Github和官方网站上都有一些这样的例子：

还有其他人也提出了类似的问题：

看看这个

有几件事需要检查。文件指出：

树的构建（训练）和预测可以通过具有CUDA功能的GPU

1.您的GPU CUDA是否已启用？

2.您使用的参数是否会受到GPU使用的影响？请记住，只有某些参数受益于使用GPU。这些是：

是的。其中大部分都包含在超参数集中，这是一件好事

{subsample, sampling_method, colsample_bytree, colsample_bylevel, max_bin, gamma, gpu_id, predictor, grow_policy, monotone_constraints, interaction_constraints, single_precision_histogram}

3.您是否正在配置参数以使用GPU支持？如果你看一下，你会发现其他方面可能有助于改善你的时间。例如，

updater

可以设置为

grow\u gpu\u hist

，这（注意，这是没有意义的，因为您设置了

tree\u方法

，但对于注释）：

生长gpu历史：使用gpu生长树

在参数页面的底部，有启用的

gpu\u hist

的其他参数，特别是

deterministic\u直方图

（注意，这是没有意义的，因为它默认为

True

）：

确定地在GPU上建立直方图。柱状图建筑不是由于浮点的非关联性，因此具有确定性总和我们采用预取整程序来缓解问题，这可能会导致精度稍低。设置为false可禁用它

4.数据我用一些数据做了一些有趣的实验。因为我无法访问您的数据，所以我使用了生成数据的

sklearn

我对您的脚本做了一些更改，但没有注意到任何更改：我更改了gpu与cpu示例上的超参数，我运行了100次，并取得了平均结果，等等。对我来说，似乎没有什么突出的地方。我记得我曾经使用

XGBoost

GPU vs CPU功能来加速一些分析，然而，我正在处理一个更大的数据集。

我稍微编辑了您的脚本以使用此数据，并开始更改数据集中

样本

和

特征

的数量（通过

n_样本

和

n_特征

参数）以观察对运行时的影响。似乎GPU将显著改善高维数据的训练时间，但多样本的批量数据并没有显著改善。请参见下面的脚本：

import xgboost as xgb, numpy, time from sklearn.datasets import make_classification from sklearn.model_selection import train_test_split xgb_gpu = [] xgbclassifier_gpu = [] xgb_cpu = [] xgbclassifier_cpu = [] n_samples = 75000 n_features = 500 for i in range(len(10)): n_samples += 10000 n_features += 300 # Make my own data since I do not have the data from the SO question X_train2, y_train = make_classification(n_samples=n_samples, n_features=n_features*0.9, n_informative=n_features*0.1, n_redundant=100, flip_y=0.10, random_state=8) # Keep script from OP intact param = {'max_depth':5, 'objective':'binary:logistic', 'subsample':0.8, 'colsample_bytree':0.8, 'eta':0.5, 'min_child_weight':1, 'tree_method':'gpu_hist', 'gpu_id': 0 } num_round = 100 dtrain = xgb.DMatrix(X_train2, y_train) tic = time.time() model = xgb.train(param, dtrain, num_round) print('passed time with xgb (gpu): %.3fs'%(time.time()-tic)) xgb_gpu.append(time.time()-tic) xgb_param = {'max_depth':5, 'objective':'binary:logistic', 'subsample':0.8, 'colsample_bytree':0.8, 'learning_rate':0.5, 'min_child_weight':1, 'tree_method':'gpu_hist', 'gpu_id':0} model = xgb.XGBClassifier(**xgb_param) tic = time.time() model.fit(X_train2, y_train) print('passed time with XGBClassifier (gpu): %.3fs'%(time.time()-tic)) xgbclassifier_gpu.append(time.time()-tic) param = {'max_depth':5, 'objective':'binary:logistic', 'subsample':0.8, 'colsample_bytree':0.8, 'eta':0.5, 'min_child_weight':1, 'tree_method':'hist'} num_round = 100 dtrain = xgb.DMatrix(X_train2, y_train) tic = time.time() model = xgb.train(param, dtrain, num_round) print('passed time with xgb (cpu): %.3fs'%(time.time()-tic)) xgb_cpu.append(time.time()-tic) xgb_param = {'max_depth':5, 'objective':'binary:logistic', 'subsample':0.8, 'colsample_bytree':0.8, 'learning_rate':0.5, 'min_child_weight':1, 'tree_method':'hist'} model = xgb.XGBClassifier(**xgb_param) tic = time.time() model.fit(X_train2, y_train) print('passed time with XGBClassifier (cpu): %.3fs'%(time.time()-tic)) xgbclassifier_cpu.append(time.time()-tic) import pandas as pd df = pd.DataFrame({'XGB GPU': xgb_gpu, 'XGBClassifier GPU': xgbclassifier_gpu, 'XGB CPU': xgb_cpu, 'XGBClassifier CPU': xgbclassifier_cpu}) #df.to_csv('both_results.csv')
我在同一个数据集上分别和一起运行了这项更改（示例、功能）。见以下结果：

| Interval | XGB GPU | XGBClassifier GPU | XGB CPU | XGBClassifier CPU | Metric | |:--------:|:--------:|:-----------------:|:--------:|:-----------------:|:----------------:| | 0 | 11.3801 | 12.00785 | 15.20124 | 15.48131 | Changed Features | | 1 | 15.67674 | 16.85668 | 20.63819 | 22.12265 | Changed Features | | 2 | 18.76029 | 20.39844 | 33.23108 | 32.29926 | Changed Features | | 3 | 23.147 | 24.91953 | 47.65588 | 44.76052 | Changed Features | | 4 | 27.42542 | 29.48186 | 50.76428 | 55.88155 | Changed Features | | 5 | 30.78596 | 33.03594 | 71.4733 | 67.24275 | Changed Features | | 6 | 35.03331 | 37.74951 | 77.68997 | 75.61216 | Changed Features | | 7 | 39.13849 | 42.17049 | 82.95307 | 85.83364 | Changed Features | | 8 | 42.55439 | 45.90751 | 92.33368 | 96.72809 | Changed Features | | 9 | 46.89023 | 50.57919 | 105.8298 | 107.3893 | Changed Features | | 0 | 7.013227 | 7.303488 | 6.998254 | 9.733574 | No Changes | | 1 | 6.757523 | 7.302388 | 5.714839 | 6.805287 | No Changes | | 2 | 6.753428 | 7.291906 | 5.899611 | 6.603533 | No Changes | | 3 | 6.749848 | 7.293555 | 6.005773 | 6.486256 | No Changes | | 4 | 6.755352 | 7.297607 | 5.982163 | 8.280619 | No Changes | | 5 | 6.756498 | 7.335412 | 6.321188 | 7.900422 | No Changes | | 6 | 6.792402 | 7.332112 | 6.17904 | 6.443676 | No Changes | | 7 | 6.786584 | 7.311666 | 7.093638 | 7.811417 | No Changes | | 8 | 6.7851 | 7.30604 | 5.574762 | 6.045969 | No Changes | | 9 | 6.789152 | 7.309363 | 5.751018 | 6.213471 | No Changes | | 0 | 7.696765 | 8.03615 | 6.175457 | 6.764809 | Changed Samples | | 1 | 7.914885 | 8.646722 | 6.997217 | 7.598789 | Changed Samples | | 2 | 8.489555 | 9.2526 | 6.899783 | 7.202334 | Changed Samples | | 3 | 9.197605 | 10.02934 | 7.511708 | 7.724675 | Changed Samples | | 4 | 9.73642 | 10.64056 | 7.918493 | 8.982463 | Changed Samples | | 5 | 10.34522 | 11.31103 | 8.524865 | 9.403711 | Changed Samples | | 6 | 10.94025 | 11.98357 | 8.697257 | 9.49277 | Changed Samples | | 7 | 11.80717 | 12.93195 | 8.734307 | 10.79595 | Changed Samples | | 8 | 12.18282 | 13.38646 | 9.175231 | 10.33532 | Changed Samples | | 9 | 13.05499 | 14.33106 | 11.04398 | 10.50722 | Changed Samples | | 0 | 12.43683 | 13.19787 | 12.80741 | 13.86206 | Changed Both | | 1 | 18.59139 | 20.01569 | 25.61141 | 35.37391 | Changed Both | | 2 | 24.37475 | 26.44214 | 40.86238 | 42.79259 | Changed Both | | 3 | 31.96762 | 34.75215 | 68.869 | 59.97797 | Changed Both | | 4 | 41.26578 | 44.70537 | 83.84672 | 94.62811 | Changed Both | | 5 | 49.82583 | 54.06252 | 109.197 | 108.0314 | Changed Both | | 6 | 59.36528 | 64.60577 | 131.1234 | 140.6352 | Changed Both | | 7 | 71.44678 | 77.71752 | 156.1914 | 161.4897 | Changed Both | | 8 | 81.79306 | 90.56132 | 196.0033 | 193.4111 | Changed Both | | 9 | 94.71505 | 104.8044 | 215.0758 | 224.6175 | Changed Both |
无变化
线性增加特征计数
线性增加样本数
线性增加样本+特征
随着我开始更多的研究；这是有道理的众所周知，GPU可以很好地与高维数据进行缩放，如果您的数据是高维的，那么您将看到训练时间的改进是有意义的。请参见以下示例：

虽然我们不能确定如果不访问您的数据，GPU的硬件功能似乎可以在您的数据支持下显著提高性能，考虑到您拥有的数据的大小和形状，情况似乎并非如此。
这显然与您使用的学习问题和超参数有关。我可以验证我有两个不同的数据集，其中一个数据集在GPU上的训练速度更快，另一个数据集的训练速度较慢
具体来说，xgboost提供了一个基准

python测试/benchmark/benchmark\u tree.py--tree\u method=gpu\u hist python测试/benchmark/benchmark\u tree.py--tree\u method=hist
在我的硬件上，
gpu\hist
比
hist
快4倍

| Interval | XGB GPU | XGBClassifier GPU | XGB CPU | XGBClassifier CPU | Metric | |:--------:|:--------:|:-----------------:|:--------:|:-----------------:|:----------------:| | 0 | 11.3801 | 12.00785 | 15.20124 | 15.48131 | Changed Features | | 1 | 15.67674 | 16.85668 | 20.63819 | 22.12265 | Changed Features | | 2 | 18.76029 | 20.39844 | 33.23108 | 32.29926 | Changed Features | | 3 | 23.147 | 24.91953 | 47.65588 | 44.76052 | Changed Features | | 4 | 27.42542 | 29.48186 | 50.76428 | 55.88155 | Changed Features | | 5 | 30.78596 | 33.03594 | 71.4733 | 67.24275 | Changed Features | | 6 | 35.03331 | 37.74951 | 77.68997 | 75.61216 | Changed Features | | 7 | 39.13849 | 42.17049 | 82.95307 | 85.83364 | Changed Features | | 8 | 42.55439 | 45.90751 | 92.33368 | 96.72809 | Changed Features | | 9 | 46.89023 | 50.57919 | 105.8298 | 107.3893 | Changed Features | | 0 | 7.013227 | 7.303488 | 6.998254 | 9.733574 | No Changes | | 1 | 6.757523 | 7.302388 | 5.714839 | 6.805287 | No Changes | | 2 | 6.753428 | 7.291906 | 5.899611 | 6.603533 | No Changes | | 3 | 6.749848 | 7.293555 | 6.005773 | 6.486256 | No Changes | | 4 | 6.755352 | 7.297607 | 5.982163 | 8.280619 | No Changes | | 5 | 6.756498 | 7.335412 | 6.321188 | 7.900422 | No Changes | | 6 | 6.792402 | 7.332112 | 6.17904 | 6.443676 | No Changes | | 7 | 6.786584 | 7.311666 | 7.093638 | 7.811417 | No Changes | | 8 | 6.7851 | 7.30604 | 5.574762 | 6.045969 | No Changes | | 9 | 6.789152 | 7.309363 | 5.751018 | 6.213471 | No Changes | | 0 | 7.696765 | 8.03615 | 6.175457 | 6.764809 | Changed Samples | | 1 | 7.914885 | 8.646722 | 6.997217 | 7.598789 | Changed Samples | | 2 | 8.489555 | 9.2526 | 6.899783 | 7.202334 | Changed Samples | | 3 | 9.197605 | 10.02934 | 7.511708 | 7.724675 | Changed Samples | | 4 | 9.73642 | 10.64056 | 7.918493 | 8.982463 | Changed Samples | | 5 | 10.34522 | 11.31103 | 8.524865 | 9.403711 | Changed Samples | | 6 | 10.94025 | 11.98357 | 8.697257 | 9.49277 | Changed Samples | | 7 | 11.80717 | 12.93195 | 8.734307 | 10.79595 | Changed Samples | | 8 | 12.18282 | 13.38646 | 9.175231 | 10.33532 | Changed Samples | | 9 | 13.05499 | 14.33106 | 11.04398 | 10.50722 | Changed Samples | | 0 | 12.43683 | 13.19787 | 12.80741 | 13.86206 | Changed Both | | 1 | 18.59139 | 20.01569 | 25.61141 | 35.37391 | Changed Both | | 2 | 24.37475 | 26.44214 | 40.86238 | 42.79259 | Changed Both | | 3 | 31.96762 | 34.75215 | 68.869 | 59.97797 | Changed Both | | 4 | 41.26578 | 44.70537 | 83.84672 | 94.62811 | Changed Both | | 5 | 49.82583 | 54.06252 | 109.197 | 108.0314 | Changed Both | | 6 | 59.36528 | 64.60577 | 131.1234 | 140.6352 | Changed Both | | 7 | 71.44678 | 77.71752 | 156.1914 | 161.4897 | Changed Both | | 8 | 81.79306 | 90.56132 | 196.0033 | 193.4111 | Changed Both | | 9 | 94.71505 | 104.8044 | 215.0758 | 224.6175 | Changed Both |