Python 如何让嵌套交叉验证在多个核上运行?
继昨天的问题()之后,我尝试在并行处理中运行模型,但是我的代码是否可以检查这些模型(每个模型只需要一个内核就可以运行)是否可以在内核上并行运行,或者使用此代码是否总是一个内核一次只运行模型1 我可以访问更高性能的计算,在那里我可以要求6个核心,每个模型一个核心,但是,我的代码不确定这些模型是否实际分配了自己的核心并并行运行。如果这是有道理的(抱歉,如果这是完全错误的信息,因为我试图学习,因为我去,任何帮助将不胜感激)。在网格搜索中,我的每个模型的n_jobs=1,我可以进一步指定每个模型有自己的核心吗 这是我的密码:Python 如何让嵌套交叉验证在多个核上运行?,python,keras,parallel-processing,scikit-learn,nested,Python,Keras,Parallel Processing,Scikit Learn,Nested,继昨天的问题()之后,我尝试在并行处理中运行模型,但是我的代码是否可以检查这些模型(每个模型只需要一个内核就可以运行)是否可以在内核上并行运行,或者使用此代码是否总是一个内核一次只运行模型1 我可以访问更高性能的计算,在那里我可以要求6个核心,每个模型一个核心,但是,我的代码不确定这些模型是否实际分配了自己的核心并并行运行。如果这是有道理的(抱歉,如果这是完全错误的信息,因为我试图学习,因为我去,任何帮助将不胜感激)。在网格搜索中,我的每个模型的n_jobs=1,我可以进一步指定每个模型有自己的
dataset= pd.read_csv('data.csv')
data = dataset.drop(["gene"],1)
df = data.iloc[:,0:24]
df = df.fillna(0)
X = MinMaxScaler().fit_transform(df)
le = preprocessing.LabelEncoder()
encoded_value = le.fit_transform(["certain", "likely", "possible", "unlikely"])
Y = le.fit_transform(data["category"])
sm = SMOTE(random_state=100)
X_res, y_res = sm.fit_resample(X, Y)
seed = 7
logreg = LogisticRegression(penalty='l1', solver='liblinear',multi_class='auto')
LR_par= {'penalty':['l1'], 'C': [0.5, 1, 5, 10], 'max_iter':[500, 1000, 5000]}
rfc =RandomForestClassifier()
param_grid = {'bootstrap': [True, False],
'max_depth': [10, 20, 30, 40, 50, 60, 70, 80, 90, 100, None],
'max_features': ['auto', 'sqrt'],
'min_samples_leaf': [1, 2, 4,25],
'min_samples_split': [2, 5, 10, 25],
'n_estimators': [200, 400, 600, 800, 1000, 1200, 1400, 1600, 1800, 2000]}
mlp = MLPClassifier(random_state=seed)
parameter_space = {'hidden_layer_sizes': [(10,20), (10,20,10), (50,)],
'activation': ['tanh', 'relu'],
'solver': ['adam', 'sgd'],
'max_iter': [10000],
'alpha': [0.1, 0.01, 0.001],
'learning_rate': ['constant','adaptive']}
gbm = GradientBoostingClassifier(min_samples_split=25, min_samples_leaf=25)
param = {"loss":["deviance"],
"learning_rate": [0.15,0.1,0.05,0.01,0.005,0.001],
"min_samples_split": [2, 5, 10, 25],
"min_samples_leaf": [1, 2, 4,25],
"max_depth":[10, 20, 30, 40, 50, 60, 70, 80, 90, 100, None],
"max_features":['auto', 'sqrt'],
"criterion": ["friedman_mse"],
"n_estimators":[200, 400, 600, 800, 1000, 1200, 1400, 1600, 1800, 2000]
}
svm = SVC(gamma="scale", probability=True)
tuned_parameters = {'kernel':('linear', 'rbf'), 'C':(1,0.25,0.5,0.75)}
def baseline_model(optimizer='adam', learn_rate=0.01):
model = Sequential()
model.add(Dense(100, input_dim=X_res.shape[1], activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(50, activation='relu')) #8 is the dim/ the number of hidden units (units are the kernel)
model.add(Dense(4, activation='softmax'))
model.compile(loss='categorical_crossentropy', optimizer=optimizer, metrics=['accuracy'])
return model
keras = KerasClassifier(build_fn=baseline_model, batch_size=32, epochs=100, verbose=0)
learn_rate = [0.001, 0.01, 0.1, 0.2, 0.3]
optimizer = ['SGD', 'RMSprop', 'Adagrad', 'Adadelta', 'Adam', 'Adamax', 'Nadam']
kerasparams = dict(optimizer=optimizer, learn_rate=learn_rate)
inner_cv = KFold(n_splits=10, shuffle=True, random_state=seed)
outer_cv = KFold(n_splits=10, shuffle=True, random_state=seed)
models = []
models.append(('GBM', GridSearchCV(gbm, param, cv=inner_cv,iid=False, n_jobs=1)))
models.append(('RFC', GridSearchCV(rfc, param_grid, cv=inner_cv,iid=False, n_jobs=1)))
models.append(('LR', GridSearchCV(logreg, LR_par, cv=inner_cv, iid=False, n_jobs=1)))
models.append(('SVM', GridSearchCV(svm, tuned_parameters, cv=inner_cv, iid=False, n_jobs=1)))
models.append(('MLP', GridSearchCV(mlp, parameter_space, cv=inner_cv,iid=False, n_jobs=1)))
models.append(('Keras', GridSearchCV(estimator=keras, param_grid=kerasparams, cv=inner_cv,iid=False, n_jobs=1)))
results = []
names = []
scoring = 'accuracy'
X_train, X_test, Y_train, Y_test = train_test_split(X_res, y_res, test_size=0.2, random_state=0)
for name, model in models:
nested_cv_results = model_selection.cross_val_score(model, X_res, y_res, cv=outer_cv, scoring=scoring)
results.append(nested_cv_results)
names.append(name)
msg = "Nested CV Accuracy %s: %f (+/- %f )" % (name, nested_cv_results.mean()*100, nested_cv_results.std()*100)
print(msg)
model.fit(X_train, Y_train)
print('Test set accuracy: {:.2f}'.format(model.score(X_test, Y_test)*100), '%')
print("Best Parameters: \n{}\n".format(model.best_params_))
print("Best CV Score: \n{}\n".format(model.best_score_))
编辑:我现在尝试了for循环,如下所示:
def run_models(models):
nested_cv_results = model_selection.cross_val_score(model, X_res, y_res, cv=outer_cv, scoring=scoring)
results.append(nested_cv_results)
names.append(name)
msg = "Nested CV Accuracy %s: %f (+/- %f )" % (name, nested_cv_results.mean()*100, nested_cv_results.std()*100)
print(msg)
model.fit(X_train, Y_train)
print('Test set accuracy: {:.2f}'.format(model.score(X_test, Y_test)*100), '%')
#print("Best Estimator: \n{}\n".format(model.best_estimator_))
print("Best Parameters: \n{}\n".format(model.best_params_))
print("Best CV Score: \n{}\n".format(model.best_score_))
if __name__ == '__main__':
pool = Pool() # Create a multiprocessing Pool
pool.map(run_models, models)
然而,这似乎无限期地运行,没有错误,但没有输出
编辑:在尝试将多处理更改为非空闲时,我尝试了:
def run_models(models):
nested_cv_results = model_selection.cross_val_score(model, X_res, y_res, cv=outer_cv, scoring=scoring)
results.append(nested_cv_results)
names.append(name)
msg = "Nested CV Accuracy %s: %f (+/- %f )" % (name, nested_cv_results.mean()*100, nested_cv_results.std()*100)
return msg, model.best_params_
for name, model in models:
if __name__ == '__main__':
pool = Pool() # Create a multiprocessing Pool
pool.map(run_models, model)
但是,这会产生错误:
TypeError: 'GridSearchCV' object is not iterable
看一看。谢谢你,它看起来非常适合我的问题,我会尝试应用它,看看它是如何运行的。我现在已经尝试了应用它,但它似乎没有给我任何输出,虽然我知道它可能只需要更多的时间,有没有办法确认它是否正确应用了多处理?你是如何运行它的?我知道它在空闲时的行为很奇怪。您还可以从函数返回值并在主循环中进行打印。谢谢您的回答,我已经尝试按照您的建议进行操作,现在将这项工作放在我的问题中(尽管很抱歉,因为我是初学者,所以这可能不是很好)如果你有任何进一步的建议,请让我知道。看看。谢谢你,它看起来非常适合我的问题,我会尝试应用它,看看它是如何进行的。我现在已经尝试了应用它,但它似乎没有给我任何输出,尽管我知道它可能需要更多的时间,有没有办法确认它是否正确应用了多重处理?您是如何运行它的?我知道它在空闲时的行为很奇怪。您也可以从函数中返回值并在主循环中进行打印。谢谢您的回答,我已经按照您的建议进行了尝试,现在将这项工作放在我的问题中(尽管很抱歉,因为我是初学者,所以这可能不是很好)。如果您有任何进一步的建议,请告诉我。