4.3 训练模型
history = model.fit(x_train, y_train, epochs=1000, validation_data=(x_test, y_test))
Train on 489 samples, validate on 164 samples Epoch 1/1000 489/489 [==============================] - 0s 487us/step - loss: 14.4564 - acc: 0.5951 - val_loss: 3.3778 - val_acc: 0.7256 Epoch 2/1000 489/489 [==============================] - 0s 110us/step - loss: 6.0909 - acc: 0.6012 - val_loss: 2.0924 - val_acc: 0.7195 Epoch 3/1000 489/489 [==============================] - 0s 195us/step - loss: 2.4527 - acc: 0.6074 - val_loss: 1.0763 - val_acc: 0.7378 Epoch 4/1000 489/489 [==============================] - 0s 183us/step - loss: 1.0751 - acc: 0.6585 - val_loss: 0.8990 - val_acc: 0.7134 Epoch 5/1000 489/489 [==============================] - 0s 155us/step - loss: 1.2669 - acc: 0.6503 - val_loss: 1.8094 - val_acc: 0.6585 Epoch 6/1000 489/489 [==============================] - 0s 202us/step - loss: 3.6742 - acc: 0.6892 - val_loss: 1.1836 - val_acc: 0.4573 Epoch 7/1000 489/489 [==============================] - 0s 166us/step - loss: 1.7544 - acc: 0.7301 - val_loss: 2.0060 - val_acc: 0.4573 Epoch 8/1000 489/489 [==============================] - 0s 185us/step - loss: 1.4768 - acc: 0.6605 - val_loss: 0.8917 - val_acc: 0.5427 Epoch 9/1000 489/489 [==============================] - 0s 163us/step - loss: 1.6829 - acc: 0.6667 - val_loss: 4.7695 - val_acc: 0.4573 Epoch 10/1000 489/489 [==============================] - 0s 157us/step - loss: 8.4323 - acc: 0.7239 - val_loss: 2.0879 - val_acc: 0.7439 …… Epoch 1000/1000 489/489 [==============================] - 0s 97us/step - loss: 0.0272 - acc: 0.9877 - val_loss: 2.2746 - val_acc: 0.8049
4.4 分析模型
history.history.keys()
dict_keys(['val_loss', 'val_acc', 'loss', 'acc'])
plt.plot(history.epoch, history.history.get('val_acc'), c='r', label='val_acc') plt.plot(history.epoch, history.history.get('acc'), c='b', label='acc') plt.legend()
<matplotlib.legend.Legend at 0x7fd417ff5978>
model.evaluate(x_train, y_train)
489/489 [==============================] - 0s 37us/step [0.021263938083038197, 0.9897750616073608]
model.evaluate(x_test, y_test)
164/164 [==============================] - 0s 46us/step [2.274633582976715, 0.8048780560493469]
过拟合：在训练数据正确率非常高， 在测试数据上比较低
5. 使用Dropout抑制过拟合
5.1 构建神经网络
model = keras.Sequential() model.add(layers.Dense(128, input_dim=15, activation='relu')) model.add(layers.Dropout(0.5)) model.add(layers.Dense(128, activation='relu')) model.add(layers.Dropout(0.5)) model.add(layers.Dense(128, activation='relu')) model.add(layers.Dropout(0.5)) model.add(layers.Dense(1, activation='sigmoid'))
model.summary()
Model: "sequential_3" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= dense_9 (Dense) (None, 128) 2048 _________________________________________________________________ dropout_1 (Dropout) (None, 128) 0 _________________________________________________________________ dense_10 (Dense) (None, 128) 16512 _________________________________________________________________ dropout_2 (Dropout) (None, 128) 0 _________________________________________________________________ dense_11 (Dense) (None, 128) 16512 _________________________________________________________________ dropout_3 (Dropout) (None, 128) 0 _________________________________________________________________ dense_12 (Dense) (None, 1) 129 ================================================================= Total params: 35,201 Trainable params: 35,201 Non-trainable params: 0 ________________________________________________________________
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['acc'] )
5.2 训练模型
history = model.fit(x_train, y_train, epochs=1000, validation_data=(x_test, y_test))
Train on 489 samples, validate on 164 samples Epoch 1/1000 489/489 [==============================] - 1s 1ms/step - loss: 41.6885 - acc: 0.5378 - val_loss: 9.9666 - val_acc: 0.6768 Epoch 2/1000 489/489 [==============================] - 0s 298us/step - loss: 53.1358 - acc: 0.5358 - val_loss: 11.0265 - val_acc: 0.6951 Epoch 3/1000 489/489 [==============================] - 0s 173us/step - loss: 36.9899 - acc: 0.5828 - val_loss: 11.6578 - val_acc: 0.6890 Epoch 4/1000 489/489 [==============================] - 0s 177us/step - loss: 43.3404 - acc: 0.5808 - val_loss: 7.5652 - val_acc: 0.6890 Epoch 5/1000 489/489 [==============================] - 0s 197us/step - loss: 23.3085 - acc: 0.6196 - val_loss: 7.9913 - val_acc: 0.6890 Epoch 6/1000 489/489 [==============================] - 0s 254us/step - loss: 24.1833 - acc: 0.6155 - val_loss: 5.5747 - val_acc: 0.7073 Epoch 7/1000 489/489 [==============================] - 0s 229us/step - loss: 19.7051 - acc: 0.5890 - val_loss: 5.5711 - val_acc: 0.7012 Epoch 8/1000 489/489 [==============================] - 0s 180us/step - loss: 22.1131 - acc: 0.5849 - val_loss: 7.0290 - val_acc: 0.6890 Epoch 9/1000 489/489 [==============================] - 0s 172us/step - loss: 23.2305 - acc: 0.6115 - val_loss: 4.2624 - val_acc: 0.6951 Epoch 10/1000 …… Epoch 1000/1000 489/489 [==============================] - 0s 137us/step - loss: 0.3524 - acc: 0.8200 - val_loss: 0.7290 - val_acc: 0.7012
5.3 分析模型
model.evaluate(x_train, y_train)
489/489 [==============================] - 0s 41us/step [0.3090217998422728, 0.8548057079315186]
model.evaluate(x_test, y_test)
164/164 [==============================] - 0s 64us/step [0.7289713301309725, 0.7012194991111755]
plt.plot(history.epoch, history.history.get('val_acc'), c='r', label='val_acc') plt.plot(history.epoch, history.history.get('acc'), c='b', label='acc') plt.legend()
<matplotlib.legend.Legend at 0x7fd4177c87b8>
6. 正则化
l1：loss = s*abs(w1 + w2 + …) + mse
l2：loss = s*(w12 + w22 + …) + mse
from keras import regularizers
6.1 神经网络太过复杂容易过拟合
#神经网络太过复杂容易过拟合 model = keras.Sequential() model.add(layers.Dense(128, kernel_regularizer=regularizers.l2(0.001), input_dim=15, activation='relu')) model.add(layers.Dense(128, kernel_regularizer=regularizers.l2(0.001), activation='relu')) model.add(layers.Dense(128, kernel_regularizer=regularizers.l2(0.001), activation='relu')) model.add(layers.Dense(1, activation='sigmoid'))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['acc'] )
history = model.fit(x_train, y_train, epochs=1000, validation_data=(x_test, y_test))
Train on 489 samples, validate on 164 samples Epoch 1/1000 489/489 [==============================] - 0s 752us/step - loss: 22.2560 - acc: 0.5910 - val_loss: 9.1111 - val_acc: 0.6524 Epoch 2/1000 489/489 [==============================] - 0s 137us/step - loss: 6.8963 - acc: 0.6217 - val_loss: 3.2886 - val_acc: 0.4573 Epoch 3/1000 489/489 [==============================] - 0s 161us/step - loss: 5.0407 - acc: 0.6830 - val_loss: 1.1973 - val_acc: 0.7256 Epoch 4/1000 489/489 [==============================] - 0s 218us/step - loss: 6.6088 - acc: 0.6421 - val_loss: 7.4651 - val_acc: 0.7012 Epoch 5/1000 489/489 [==============================] - 0s 233us/step - loss: 8.3945 - acc: 0.6973 - val_loss: 2.5579 - val_acc: 0.7317 Epoch 6/1000 489/489 [==============================] - 0s 192us/step - loss: 7.0204 - acc: 0.6196 - val_loss: 3.6758 - val_acc: 0.6829 Epoch 7/1000 489/489 [==============================] - 0s 152us/step - loss: 3.9961 - acc: 0.7382 - val_loss: 1.6183 - val_acc: 0.7317 Epoch 8/1000 489/489 [==============================] - 0s 94us/step - loss: 2.3441 - acc: 0.6237 - val_loss: 1.1523 - val_acc: 0.7256 Epoch 9/1000 489/489 [==============================] - 0s 114us/step - loss: 1.8178 - acc: 0.6442 - val_loss: 1.3449 - val_acc: 0.7073 Epoch 10/1000 489/489 [==============================] - 0s 157us/step - loss: 1.6122 - acc: 0.7117 - val_loss: 1.2869 - val_acc: 0.6646 …… Epoch 1000/1000 489/489 [==============================] - 0s 130us/step - loss: 0.1452 - acc: 0.9775 - val_loss: 1.0515 - val_acc: 0.7866 model.evaluate(x_train, y_train)
model.evaluate(x_train, y_train)
489/489 [==============================] - 0s 34us/step [0.17742264538942426, 0.9611452221870422]
model.evaluate(x_test, y_test)
164/164 [==============================] - 0s 77us/step [1.0514701096023, 0.7865853905677795]
6.2 太简单容易欠拟合
#太简单容易欠拟合 model = keras.Sequential() model.add(layers.Dense(4, input_dim=15, activation='relu')) model.add(layers.Dense(1, activation='relu')) model.add(layers.Dense(1, activation='sigmoid'))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['acc'] )
history = model.fit(x_train, y_train, epochs=1000, validation_data=(x_test, y_test))
Train on 489 samples, validate on 164 samples Epoch 1/1000 489/489 [==============================] - 0s 502us/step - loss: 0.6932 - acc: 0.4765 - val_loss: 0.6931 - val_acc: 0.6341 Epoch 2/1000 489/489 [==============================] - 0s 91us/step - loss: 0.6931 - acc: 0.5174 - val_loss: 0.6930 - val_acc: 0.6341 Epoch 3/1000 489/489 [==============================] - 0s 107us/step - loss: 0.6931 - acc: 0.5174 - val_loss: 0.6924 - val_acc: 0.6341 Epoch 4/1000 489/489 [==============================] - 0s 91us/step - loss: 0.6930 - acc: 0.5174 - val_loss: 0.6916 - val_acc: 0.6341 Epoch 5/1000 489/489 [==============================] - 0s 101us/step - loss: 0.6930 - acc: 0.5174 - val_loss: 0.6914 - val_acc: 0.6341 Epoch 6/1000 489/489 [==============================] - 0s 113us/step - loss: 0.6930 - acc: 0.5174 - val_loss: 0.6914 - val_acc: 0.6341 Epoch 7/1000 489/489 [==============================] - 0s 147us/step - loss: 0.6929 - acc: 0.5174 - val_loss: 0.6908 - val_acc: 0.6341 Epoch 8/1000 489/489 [==============================] - 0s 166us/step - loss: 0.6929 - acc: 0.5174 - val_loss: 0.6905 - val_acc: 0.6341 Epoch 9/1000 489/489 [==============================] - 0s 162us/step - loss: 0.6929 - acc: 0.5174 - val_loss: 0.6904 - val_acc: 0.6341 Epoch 10/1000 489/489 [==============================] - 0s 129us/step - loss: 0.6928 - acc: 0.5174 - val_loss: 0.6901 - val_acc: 0.6341 …… Epoch 1000/1000 489/489 [==============================] - 0s 86us/step - loss: 0.6926 - acc: 0.5174 - val_loss: 0.6849 - val_acc: 0.6341
model.evaluate(x_train, y_train)
489/489 [==============================] - 0s 43us/step [0.6925447341854587, 0.5173823833465576]
model.evaluate(x_test, y_test)
164/164 [==============================] - 0s 39us/step [0.684889389247429, 0.6341463327407837]
6.3 选取适当的神经网络
# 选取适当的神经网络 model = keras.Sequential() model.add(layers.Dense(4, input_dim=15, activation='relu')) model.add(layers.Dense(4, activation='relu')) model.add(layers.Dense(1, activation='sigmoid'))
model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['acc'] )
history = model.fit(x_train, y_train, epochs=1000, validation_data=(x_test, y_test))
Train on 489 samples, validate on 164 samples Epoch 1/1000 489/489 [==============================] - 0s 575us/step - loss: 40.1825 - acc: 0.5317 - val_loss: 17.6376 - val_acc: 0.6098 Epoch 2/1000 489/489 [==============================] - 0s 104us/step - loss: 30.0785 - acc: 0.5337 - val_loss: 12.6986 - val_acc: 0.6159 Epoch 3/1000 489/489 [==============================] - 0s 148us/step - loss: 20.0469 - acc: 0.5112 - val_loss: 8.3732 - val_acc: 0.5671 Epoch 4/1000 489/489 [==============================] - 0s 151us/step - loss: 12.5171 - acc: 0.4908 - val_loss: 3.8925 - val_acc: 0.5061 Epoch 5/1000 489/489 [==============================] - 0s 113us/step - loss: 4.4324 - acc: 0.4294 - val_loss: 0.9156 - val_acc: 0.4573 Epoch 6/1000 489/489 [==============================] - 0s 79us/step - loss: 1.0313 - acc: 0.5419 - val_loss: 0.9974 - val_acc: 0.4695 Epoch 7/1000 489/489 [==============================] - 0s 88us/step - loss: 1.0071 - acc: 0.5562 - val_loss: 0.8852 - val_acc: 0.5183 Epoch 8/1000 489/489 [==============================] - 0s 88us/step - loss: 0.9085 - acc: 0.5808 - val_loss: 0.7934 - val_acc: 0.5366 Epoch 9/1000 489/489 [==============================] - 0s 107us/step - loss: 0.8235 - acc: 0.5992 - val_loss: 0.7390 - val_acc: 0.5366 Epoch 10/1000 489/489 [==============================] - 0s 114us/step - loss: 0.7711 - acc: 0.5971 - val_loss: 0.7174 - val_acc: 0.5366 …… Epoch 1000/1000 489/489 [==============================] - 0s 141us/step - loss: 0.3095 - acc: 0.8732 - val_loss: 0.3971 - val_acc: 0.8537
model.evaluate(x_train, y_train)
489/489 [==============================] - 0s 68us/step [0.30120014958464536, 0.8813905715942383]
model.evaluate(x_test, y_test)
164/164 [==============================] - 0s 45us/step [0.39714593858253666, 0.8536585569381714]
