任務(wù)
使用啟發(fā)式優(yōu)化算法遺傳算法對(duì)多層感知機(jī)中中間層神經(jīng)個(gè)數(shù)進(jìn)行優(yōu)化，以提高模型的準(zhǔn)確率。
待優(yōu)化的模型：
基于TensorFlow2實(shí)現(xiàn)的Mnist手寫數(shù)字識(shí)別多層感知機(jī)MLP
# MLP手寫數(shù)字識(shí)別模型，待優(yōu)化的參數(shù)為layer1、layer2
model = tf.keras.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28, 1)),
tf.keras.layers.Dense(layer1, activation='relu'),
tf.keras.layers.Dense(layer2, activation='relu'),
tf.keras.layers.Dense(10,activation='softmax') # 對(duì)應(yīng)0-9這10個(gè)數(shù)字
])
查看當(dāng)前的準(zhǔn)確率情況
設(shè)置隨機(jī)樹種子，避免相同結(jié)構(gòu)的神經(jīng)網(wǎng)絡(luò)其結(jié)果不同的影響。
random_seed = 10
np.random.seed(random_seed)
tf.random.set_seed(random_seed)
random.seed(random_seed)
model = tf.keras.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28, 1)),
tf.keras.layers.Dense(128, activation='relu'),
tf.keras.layers.Dense(32, activation='relu'),
tf.keras.layers.Dense(10,activation='softmax') # 對(duì)應(yīng)0-9這10個(gè)數(shù)字
])
optimizer = tf.keras.optimizers.Adam()
loss_func = tf.keras.losses.SparseCategoricalCrossentropy()
model.compile(optimizer=optimizer,loss=loss_func,metrics=['acc'])
history = model.fit(dataset, validation_data=test_dataset, epochs=5, verbose=1)
score = model.evaluate(test_dataset)
print("測試集準(zhǔn)確率:",score) # 輸出 [損失率，準(zhǔn)確率]
Epoch 1/5
235/235 [==============================] - 1s 5ms/step - loss: 0.4663 - acc: 0.8703 - val_loss: 0.2173 - val_acc: 0.9361
Epoch 2/5
235/235 [==============================] - 1s 4ms/step - loss: 0.1882 - acc: 0.9465 - val_loss: 0.1604 - val_acc: 0.9521
Epoch 3/5
235/235 [==============================] - 1s 4ms/step - loss: 0.1362 - acc: 0.9608 - val_loss: 0.1278 - val_acc: 0.9629
Epoch 4/5
235/235 [==============================] - 1s 4ms/step - loss: 0.1084 - acc: 0.9689 - val_loss: 0.1086 - val_acc: 0.9681
Epoch 5/5
235/235 [==============================] - 1s 4ms/step - loss: 0.0878 - acc: 0.9740 - val_loss: 0.1077 - val_acc: 0.9675
40/40 [==============================] - 0s 2ms/step - loss: 0.1077 - acc: 0.9675
測試集準(zhǔn)確率: [0.10773944109678268, 0.9674999713897705]
準(zhǔn)確率為96.7%
使用遺傳算法進(jìn)行優(yōu)化
使用scikit-opt提供的遺傳算法庫進(jìn)行優(yōu)化。（pip install scikit-opt）
官方文檔：https://scikit-opt.github.io/scikit-opt/#/zh/README
# 遺傳算法調(diào)用
ga = GA(func=loss_func, n_dim=2, size_pop=4, max_iter=3, prob_mut=0.15, lb=[10, 10], ub=[256, 256], precision=1)
優(yōu)化目標(biāo)函數(shù)loss_func：1 - 模型準(zhǔn)確率（求優(yōu)化目標(biāo)函數(shù)的最小值）
待優(yōu)化的參數(shù)數(shù)量n_dim：2
種群數(shù)量size_pop：4
迭代次數(shù)max_iter：3
變異概率prob_mut：0.15
待優(yōu)化兩個(gè)參數(shù)的取值范圍均為10-256
精確度precision：1
# 定義多層感知機(jī)模型函數(shù)
def mlp_model(layer1, layer2):
model = tf.keras.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28, 1)),
tf.keras.layers.Dense(layer1, activation='relu'),
tf.keras.layers.Dense(layer2, activation='relu'),
tf.keras.layers.Dense(10,activation='softmax') # 對(duì)應(yīng)0-9這10個(gè)數(shù)字
])
optimizer = tf.keras.optimizers.Adam()
loss_func = tf.keras.losses.SparseCategoricalCrossentropy()
model.compile(optimizer=optimizer,loss=loss_func,metrics=['acc'])
return model
# 定義損失函數(shù)
def loss_func(params):
random_seed = 10
np.random.seed(random_seed)
tf.random.set_seed(random_seed)
random.seed(random_seed)
layer1 = int(params[0])
layer2 = int(params[1])
model = mlp_model(layer1, layer2)
history = model.fit(dataset, validation_data=test_dataset, epochs=5, verbose=0)
return 1 - history.history['val_acc'][-1]
ga = GA(func=loss_func, n_dim=2, size_pop=4, max_iter=3, prob_mut=0.15, lb=[10, 10], ub=[256, 256], precision=1)
bestx, besty = ga.run()
print('bestx:', bestx, '\n', 'besty:', besty)
結(jié)果：
bestx: [165. 155.]
besty: [0.02310002]
通過迭代，找到layer1、layer2的最好值為165、155，此時(shí)準(zhǔn)確率為1-0.0231=0.9769。
查看每輪迭代的損失函數(shù)值圖（1-準(zhǔn)確率）：
Y_history = pd.DataFrame(ga.all_history_Y)
fig, ax = plt.subplots(2, 1)
ax[0].plot(Y_history.index, Y_history.values, '.', color='red')
Y_history.min(axis=1).cummin().plot(kind='line')
plt.show()
上圖為三次迭代種群中，種群每個(gè)個(gè)體的損失函數(shù)值（每個(gè)種群4個(gè)個(gè)體）。
下圖為三次迭代種群中，種群個(gè)體中的最佳損失函數(shù)值。
可以看出，通過遺傳算法，其準(zhǔn)確率有一定的提升。文章來源：http://www.zghlxwxcb.cn/news/detail-412307.html
增加種群數(shù)量及迭代次數(shù)效果可更加明顯。文章來源地址http://www.zghlxwxcb.cn/news/detail-412307.html
完整代碼
# python3.6.9
import tensorflow as tf # 2.3
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import random
from sko.GA import GA
# 數(shù)據(jù)導(dǎo)入，獲取訓(xùn)練集和測試集
(train_image, train_labels), (test_image, test_labels) = tf.keras.datasets.mnist.load_data()
# 增加通道維度
train_image = tf.expand_dims(train_image, -1)
test_image = tf.expand_dims(test_image, -1)
# 歸一化 類型轉(zhuǎn)換
train_image = tf.cast(train_image/255, tf.float32)
test_image = tf.cast(test_image/255, tf.float32)
train_labels = tf.cast(train_labels, tf.int64)
test_labels = tf.cast(test_labels, tf.int64)
# 創(chuàng)建Dataset
dataset = tf.data.Dataset.from_tensor_slices((train_image, train_labels)).shuffle(60000).batch(256)
test_dataset = tf.data.Dataset.from_tensor_slices((test_image, test_labels)).batch(256)
# 定義多層感知機(jī)模型函數(shù)
def mlp_model(layer1, layer2):
model = tf.keras.Sequential([
tf.keras.layers.Flatten(input_shape=(28, 28, 1)),
tf.keras.layers.Dense(layer1, activation='relu'),
tf.keras.layers.Dense(layer2, activation='relu'),
tf.keras.layers.Dense(10,activation='softmax') # 對(duì)應(yīng)0-9這10個(gè)數(shù)字
])
optimizer = tf.keras.optimizers.Adam()
loss_func = tf.keras.losses.SparseCategoricalCrossentropy()
model.compile(optimizer=optimizer,loss=loss_func,metrics=['acc'])
return model
# 定義損失函數(shù)
def loss_func(params):
random_seed = 10
np.random.seed(random_seed)
tf.random.set_seed(random_seed)
random.seed(random_seed)
layer1 = int(params[0])
layer2 = int(params[1])
model = mlp_model(layer1, layer2)
history = model.fit(dataset, validation_data=test_dataset, epochs=5, verbose=0)
return 1 - history.history['val_acc'][-1]
ga = GA(func=loss_func, n_dim=2, size_pop=4, max_iter=3, prob_mut=0.15, lb=[10, 10], ub=[256, 256], precision=1)
bestx, besty = ga.run()
print('bestx:', bestx, '\n', 'besty:', besty)
到了這里，關(guān)于【優(yōu)化算法】使用遺傳算法優(yōu)化MLP神經(jīng)網(wǎng)絡(luò)參數(shù)（TensorFlow2）的文章就介紹完了。如果您還想了解更多內(nèi)容，請(qǐng)?jiān)谟疑辖撬阉鱐OY模板網(wǎng)以前的文章或繼續(xù)瀏覽下面的相關(guān)文章，希望大家以后多多支持TOY模板網(wǎng)！
