Tensorflow 使用Talos将单个超参数集与多个超参数集配合使用
我正在尝试做一个小作业项目，我正在尝试使用暹罗网络做说话人识别。我正在尝试使用talos来寻找最好的超参数。当尝试一组超参数时，代码通过了OK，但当我给出一组超参数时，我得到了MemoryError 背景（不确定是否需要，但为了方便起见，请允许我解释）Tensorflow 使用Talos将单个超参数集与多个超参数集配合使用,tensorflow,keras,deep-learning,siamese-network,talos,Tensorflow,Keras,Deep Learning,Siamese Network,Talos,我正在尝试做一个小作业项目，我正在尝试使用暹罗网络做说话人识别。我正在尝试使用talos来寻找最好的超参数。当尝试一组超参数时，代码通过了OK，但当我给出一组超参数时，我得到了MemoryError 背景（不确定是否需要，但为了方便起见，请允许我解释） 将音频分割为5秒，每个块 计算每个音频样本的MFCC和MFCC_del以及MFCC_del_del，并将其全部合并，以便将每个音频样本转换为（24x939）数据点/矩阵 通过两组矩阵创建暹罗网络，其中一组是所有“相似”成对扬声器音频样本，另一组
- 将音频分割为5秒，每个块
- 计算每个音频样本的MFCC和MFCC_del以及MFCC_del_del，并将其全部合并，以便将每个音频样本转换为（24x939）数据点/矩阵
- 通过两组矩阵创建暹罗网络，其中一组是所有“相似”成对扬声器音频样本，另一组是“不相似扬声器”音频样本
- 总共6000个测试点和14886个训练点（现在不使用验证将在以后使用）
Model: "model_6"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_11 (InputLayer) (None, 24, 939) 0
__________________________________________________________________________________________________
input_12 (InputLayer) (None, 24, 939) 0
__________________________________________________________________________________________________
sequential_6 (Sequential) (None, 10) 45580 input_11[0][0]
input_12[0][0]
__________________________________________________________________________________________________
lambda_6 (Lambda) (None, 1) 0 sequential_6[1][0]
sequential_6[2][0]
==================================================================================================
Total params: 45,580
Trainable params: 45,544
Non-trainable params: 36
__________________________________________________________________________________________________
Epoch 1/5
2481/2481 [==============================] - 29s 12ms/step - loss: 0.0056 - accuracy: 0.9986 - val_loss: 0.8333 - val_accuracy: 0.1667
Epoch 2/5
2481/2481 [==============================] - 28s 11ms/step - loss: nan - accuracy: 0.9993 - val_loss: nan - val_accuracy: 0.8333
Epoch 3/5
2481/2481 [==============================] - 28s 11ms/step - loss: nan - accuracy: 1.0000 - val_loss: nan - val_accuracy: 0.8333
Epoch 4/5
2481/2481 [==============================] - 28s 11ms/step - loss: nan - accuracy: 1.0000 - val_loss: nan - val_accuracy: 0.8333
Epoch 00004: early stopping
(<keras.callbacks.callbacks.History at 0x2969fa3fd88>,
<keras.engine.training.Model at 0x2969f2a17c8>)
---------------------------------------------------------------------------
MemoryError Traceback (most recent call last)
<ipython-input-9-df856388a4bb> in <module>
1 #t=ta.Scan(x=xtrain_np_img1_img2,y=y_train_numpy,x_val=xtest_np_img1_img2,y_val=y_test_numpy,model=siamese,params=p,experiment_name='exp_1')
2
----> 3 t=ta.Scan(x=[xtrain_np_img1,xtrain_np_img2],y=y_train_numpy,x_val=[xtest_np_img1,xtest_np_img2],y_val=y_test_numpy,model=siamese,params=p,experiment_name='exp_1')
~\anaconda3\envs\MyEnv\lib\site-packages\talos\scan\Scan.py in __init__(self, x, y, params, model, experiment_name, x_val, y_val, val_split, random_method, seed, performance_target, fraction_limit, round_limit, time_limit, boolean_limit, reduction_method, reduction_interval, reduction_window, reduction_threshold, reduction_metric, minimize_loss, disable_progress_bar, print_params, clear_session, save_weights)
194 # start runtime
195 from .scan_run import scan_run
--> 196 scan_run(self)
~\anaconda3\envs\MyEnv\lib\site-packages\talos\scan\scan_run.py in scan_run(self)
7
8 from .scan_prepare import scan_prepare
----> 9 self = scan_prepare(self)
10
11 # initiate the progress bar
~\anaconda3\envs\MyEnv\lib\site-packages\talos\scan\scan_prepare.py in scan_prepare(self)
28 round_limit=self.round_limit,
29 time_limit=self.time_limit,
---> 30 boolean_limit=self.boolean_limit
31 )
32
~\anaconda3\envs\MyEnv\lib\site-packages\talos\parameters\ParamSpace.py in __init__(self, params, param_keys, random_method, fraction_limit, round_limit, time_limit, boolean_limit)
42
43 # create the parameter space
---> 44 self.param_space = self._param_space_creation()
45
46 # handle the boolean limits separately
~\anaconda3\envs\MyEnv\lib\site-packages\talos\parameters\ParamSpace.py in _param_space_creation(self)
133 if len(self.param_index) > 100000:
134
--> 135 final_grid = list(it.product(*self._params_temp))
136 out = np.array(final_grid, dtype='object')
137
MemoryError:
---------------------------------------------------------------------------
MemoryError回溯（上次最近调用）
在里面
1#t=ta.Scan（x=xtrain\u np\u img1\u img2，y=y train\u numpy，x\u val=xtest\u np\u img1\u img2，y\u val=y\u test\u numpy，model=暹罗，params=p，exp\u name='exp\u 1'）
2.
---->3 t=ta.Scan（x=[xtrain\u np\u img1，xtrain\u np\u img2]，y=y\u train\u numpy，x\u val=[xtest\u np\u img1，xtest\u np\u img2]，y\u val=y\u test\u numpy，model=暹罗，params=p，exp\u name='exp\u 1'）
初始化中的~\anaconda3\envs\MyEnv\lib\site packages\talos\scan\scan.py__（self，x，y，params，model，experiment\u name，x\u val，y\u val，val\u split，random\u method，seed，performance\u target，fraction\u limit，round\u limit，time\u limit，boolean\u limit，reduce\u method，reduce\u interval，reduce\u window\u threshold，reduce\u threshold\u，reduce\u metric\u，reduction\u，reduction\u，reduction\u metric\u，最小化\
194#启动运行时
195从。扫描运行导入扫描运行
-->196扫描运行（自我）
~\anaconda3\envs\MyEnv\lib\site packages\talos\scan\scan\u run.py in scan\u run（self）
7.
8从。扫描\准备导入扫描\准备
---->9 self=扫描准备（self）
10
11#启动进度条
~\anaconda3\envs\MyEnv\lib\site packages\talos\scan\scan\u prepare.py in scan\u prepare（self）
28舍入极限=自我舍入极限，
29时间限制=自我时间限制，
--->30布尔值限制=自布尔值限制
31 )
32
~\anaconda3\envs\MyEnv\lib\site packages\talos\parameters\ParamSpace.py in\uuuuuu init\uuuu（self、params、param\u键、随机方法、分数限制、舍入限制、时间限制、布尔限制）
42
43#创建参数空间
--->44 self.param_space=self._param_space_creation（）
45
46#分别处理布尔限制
~\anaconda3\envs\MyEnv\lib\site packages\talos\parameters\ParamSpace.py in\u param\u space\u creation（self）
133如果len（自身参数指数）>100000：
134
-->135最终网格=列表（it.product（*自身参数温度））
136 out=np.array（最终网格，dtype='object'）
137
记忆错误：
嗨，Saurabh，这是很多代码，供大家查看和理解。您应该尝试在尽可能少的代码行中重现相同的错误，以便人们可以轻松地关注导致问题的特定部分。请参阅测试用例缩减的说明。与“我的代码不起作用，为什么不起作用？”不同，您应该努力识别导致错误的特定条件，并发布“为什么x条件导致tensorflow、Siamse网络中的y错误”谢谢FinleyGibson谢谢你的回复我会删除部分代码，因为它可能不相关我不确定是否有人会感兴趣以防万一。然而，我想说的是，我已经介绍了两套代码。第一种情况下，我使用单组参数，效果很好；第二种情况下，我使用超参数优化工具talos和多组参数，我得到MemoryError，所以我想知道我的代码或talos更改标题是否有问题；Fit_生成器使用单组超参数，但MemoryError和多组参数与fit_generator相比，使用TalosI的hyperparameter检查了几个类似的问题，很少有人使用DASK数据帧指出问题？但我不明白的是，fit_生成器在处理单组超参数时运行良好，为什么它不能处理多组参数？Talos会保留每次迭代使用的所有内存吗？仔细查看错误后，我意识到如果len（self.param\u index）>100000:134-->135 final\u grid=list（it.product（*self.param\u temp）），Talos会在创建（self）133中显示~\anaconda3\envs\MyEnv\lib\site packages\Talos\parameters\ParamSpace.py136 out=np.array（final_grid，dtype='object'）137 MemoryError：所以我检查了我给它的3188646000000个参数集的总数，这些参数集是YYY。所以我只是减少了参数的数量，MemoryError消失了，但是现在一些其他错误，这些参数是模型中的权重，内存不足
Model: "model_6"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_11 (InputLayer) (None, 24, 939) 0
__________________________________________________________________________________________________
input_12 (InputLayer) (None, 24, 939) 0
__________________________________________________________________________________________________
sequential_6 (Sequential) (None, 10) 45580 input_11[0][0]
input_12[0][0]
__________________________________________________________________________________________________
lambda_6 (Lambda) (None, 1) 0 sequential_6[1][0]
sequential_6[2][0]
==================================================================================================
Total params: 45,580
Trainable params: 45,544
Non-trainable params: 36
__________________________________________________________________________________________________
Epoch 1/5
2481/2481 [==============================] - 29s 12ms/step - loss: 0.0056 - accuracy: 0.9986 - val_loss: 0.8333 - val_accuracy: 0.1667
Epoch 2/5
2481/2481 [==============================] - 28s 11ms/step - loss: nan - accuracy: 0.9993 - val_loss: nan - val_accuracy: 0.8333
Epoch 3/5
2481/2481 [==============================] - 28s 11ms/step - loss: nan - accuracy: 1.0000 - val_loss: nan - val_accuracy: 0.8333
Epoch 4/5
2481/2481 [==============================] - 28s 11ms/step - loss: nan - accuracy: 1.0000 - val_loss: nan - val_accuracy: 0.8333
Epoch 00004: early stopping
(<keras.callbacks.callbacks.History at 0x2969fa3fd88>,
<keras.engine.training.Model at 0x2969f2a17c8>)
p = {
'filter1':[1,2,4,6,8,12,16,24],
'kernel_size1':[2,4,6,8,12],
'filter3' : [1,2,4,6,8],
'kernel_size3' : [1,2,4,6,8,12],
'decay' :[.1,0.01,.001,.0001,.00001],
'droprate1' :[.1,.2,.3],
'filter2':[1,2,4,6,8],
'kernel_size2':[2,4,6,8,12],
'droprate4' : [.1,.2,.3],
'droprate2' :[.1,.2,.3],
'unit1': [10,24,36,64,128,256],
'droprate3': [.1,.2,.3],
'lr' :[(.1,0.01,.001,.0001,.00001)],
'batch_size' : [1,2],
'epochs': [4,8,10] }
def siamese(generator,validation_data,params):
W_init = tf.keras.initializers.he_normal(seed=100)
b_init = tf.keras.initializers.he_normal(seed=50)
input_shape = (24,939)
left_input = Input(input_shape)
right_input = Input(input_shape)
encoder = Sequential()
encoder.add(Conv1D(filters=(params['filter1']),kernel_size=(params['kernel_size1']), padding='same', activation='relu',input_shape=input_shape,kernel_initializer=W_init, bias_initializer=b_init))
encoder.add(BatchNormalization())
encoder.add(Dropout((params["droprate1"])))
encoder.add(MaxPool1D())
encoder.add(Conv1D(filters=(params["filter2"]),kernel_size=(params['kernel_size2']), padding='same', activation='relu'))
encoder.add(BatchNormalization())
encoder.add(Dropout((params["droprate2"])))
encoder.add(MaxPool1D())
encoder.add(Conv1D(filters=(params["filter3"]),kernel_size=(params['kernel_size3']), padding='same', activation='relu'))
encoder.add(BatchNormalization())
encoder.add(Dropout((params['droprate3'])))
encoder.add(MaxPool1D())
encoder.add(Flatten())
encoder.add(Dense((params['unit1']),activation='relu'))
encoder.add(Dropout((params['droprate4'])))
encoded_l = encoder(left_input)
encoded_r = encoder(right_input)
distance = Lambda(euclidean_distance, output_shape=eucl_dist_output_shape)([encoded_l, encoded_r])
adam = optimizers.Adam(lr=params['lr'], beta_1=0.1, beta_2=0.999,decay=params['decay'], amsgrad=False)
earlyStopping = EarlyStopping(monitor='loss',min_delta=0,patience=3,verbose=1,restore_best_weights=False)
callback_early_stop_reduceLROnPlateau=[earlyStopping]
model = Model([left_input, right_input], distance)
model.compile(loss=contrastive_loss, optimizer=adam,metrics=[accuracy])
model.summary()
#history = model.fit([(x_train[:,:,:,0]).astype(np.float32),(x_train[:,:,:,1]).astype(np.float32)],y_train, validation_data=([(x_val[:,:,:,0]).astype(np.float32),(x_val[:,:,:,1]).astype(np.float32)], y_val) ,batch_size=params['batch_size'],epochs=params['epochs'],callbacks=callback_early_stop_reduceLROnPlateau)
history=model.fit_generator(generator=train_generator,validation_data=val_generator,steps_per_epoch=2481,epochs=5,validation_steps=1000,verbose=1,callbacks=callback_early_stop_reduceLROnPlateau,use_multiprocessing=False,workers=0)
return history,model
t=ta.Scan(x=[xtrain_np_img1,xtrain_np_img2],y=y_train_numpy,x_val=[xtest_np_img1,xtest_np_img2],y_val=y_test_numpy,model=siamese,params=p,experiment_name='exp_1')
---------------------------------------------------------------------------
MemoryError Traceback (most recent call last)
<ipython-input-9-df856388a4bb> in <module>
1 #t=ta.Scan(x=xtrain_np_img1_img2,y=y_train_numpy,x_val=xtest_np_img1_img2,y_val=y_test_numpy,model=siamese,params=p,experiment_name='exp_1')
2
----> 3 t=ta.Scan(x=[xtrain_np_img1,xtrain_np_img2],y=y_train_numpy,x_val=[xtest_np_img1,xtest_np_img2],y_val=y_test_numpy,model=siamese,params=p,experiment_name='exp_1')
~\anaconda3\envs\MyEnv\lib\site-packages\talos\scan\Scan.py in __init__(self, x, y, params, model, experiment_name, x_val, y_val, val_split, random_method, seed, performance_target, fraction_limit, round_limit, time_limit, boolean_limit, reduction_method, reduction_interval, reduction_window, reduction_threshold, reduction_metric, minimize_loss, disable_progress_bar, print_params, clear_session, save_weights)
194 # start runtime
195 from .scan_run import scan_run
--> 196 scan_run(self)
~\anaconda3\envs\MyEnv\lib\site-packages\talos\scan\scan_run.py in scan_run(self)
7
8 from .scan_prepare import scan_prepare
----> 9 self = scan_prepare(self)
10
11 # initiate the progress bar
~\anaconda3\envs\MyEnv\lib\site-packages\talos\scan\scan_prepare.py in scan_prepare(self)
28 round_limit=self.round_limit,
29 time_limit=self.time_limit,
---> 30 boolean_limit=self.boolean_limit
31 )
32
~\anaconda3\envs\MyEnv\lib\site-packages\talos\parameters\ParamSpace.py in __init__(self, params, param_keys, random_method, fraction_limit, round_limit, time_limit, boolean_limit)
42
43 # create the parameter space
---> 44 self.param_space = self._param_space_creation()
45
46 # handle the boolean limits separately
~\anaconda3\envs\MyEnv\lib\site-packages\talos\parameters\ParamSpace.py in _param_space_creation(self)
133 if len(self.param_index) > 100000:
134
--> 135 final_grid = list(it.product(*self._params_temp))
136 out = np.array(final_grid, dtype='object')
137
MemoryError: