前言
XGBoost、LightGBM与CatBoost是主流的梯度提升决策树算法，它们在CPU与GPU上的表现如何呢？
他人观点
Szilard Pafka
on CPU, the numbers have changed very little. The top performers are still XGBoost and LightGBM
on GPU XGBoost became even faster (2x on larger data and even more than 2x on smaller data) (it already was the best performer, so now even more so)
Mark Tenenholtz
XGBoost when you have a GPU
LightGBM when you’re only using a CPU
CatBoost with a lot of categorical features
You don’t need Catboost. Really, you don’t. It’s slower than LightGBM on the CPU, much worse than XGBoost on the GPU, no more accurate than either.
我的观点
-
CPU选LightGBM
-
GPU选XGBoost
-
默认选XGBoost
XGBoost在CPU与GPU上的表现
Python代码
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
|
import xgboost as xgb
from sklearn.datasets import load_diabetes
from sklearn.model_selection import train_test_split
import time
# 加载数据
data = load_diabetes()
X, y = data.data, data.target
# 划分训练集和测试集
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
# 参数设定
params = {
'objective': 'reg:squarederror',
'max_depth': 6,
'min_child_weight': 1,
'learning_rate': 0.1,
'n_estimators': 100,
'tree_method': 'hist', # 使用直方图优化方法
'device': 'cpu' # 默认使用CPU
}
# 使用CPU训练
start_time = time.time()
model_cpu = xgb.XGBRegressor(**params)
model_cpu.fit(X_train, y_train)
cpu_time = time.time() - start_time
# 更新参数为GPU训练
params['device'] = 'cuda' # 设置为CUDA设备
start_time = time.time()
model_gpu = xgb.XGBRegressor(**params)
model_gpu.fit(X_train, y_train)
gpu_time = time.time() - start_time
print(f"CPU运行时间: {cpu_time:.4f}秒")
print(f"GPU运行时间: {gpu_time:.4f}秒")
print("##########")
# pip install psutil py-cpuinfo GPUtil
import platform
import psutil
import cpuinfo
import GPUtil
# 获取CPU信息
cpu_info = cpuinfo.get_cpu_info()
cpu_name = cpu_info['brand_raw']
cpu_freq = psutil.cpu_freq().max
memory = psutil.virtual_memory().total / (1024**3) # 转换为GB
# 获取GPU信息
gpus = GPUtil.getGPUs()
gpu_name = gpus[0].name if gpus else 'No GPU found'
gpu_memory = gpus[0].memoryTotal if gpus else 'No GPU found'
# 打印硬件信息
print(f"操作系统: {platform.system()} {platform.release()}")
print(f"处理器: {cpu_name}, 频率: {cpu_freq:.2f} MHz")
print(f"内存总量: {memory:.2f} GB")
print(f"GPU型号: {gpu_name}")
print(f"GPU显存: {gpu_memory/1024} GB")
|
运行结果
1
2
3
4
5
6
7
8
|
CPU运行时间: 1.9740秒
GPU运行时间: 0.3300秒
##########
操作系统: Windows 10
处理器: 12th Gen Intel(R) Core(TM) i9-12900HX, 频率: 2300.00 MHz
内存总量: 31.73 GB
GPU型号: NVIDIA GeForce RTX 3080 Ti Laptop GPU
GPU显存: 16.0 GB
|
