搜罗到两种方案，经测试都可正常运行。这两种方案各有利弊，可根据实际需求选择。
- nssm的方案
将tensorflow模型的推理逻辑制作成flask服务，假设文件为app.py。其中的model_predict需要换成用户自己的推理模块。
# app.py文件 from flask import Flask, request import numpy as np from tensorflow.python.saved_model import tag_constants from tensorflow.contrib.tensor_forest.python import tensor_forest from tensorflow.python.ops import resources import tensorflow.compat.v1 as tf import json from gevent import pywsgi import multiprocessing from multiprocessing import freeze_support from datetime import datetime import platform app = Flask(__name__) class predict(): def __init__(self, model_path): # with tf.Session() as self.sess: self.sess = tf.Session() meta_graph_def = tf.saved_model.loader.load(self.sess, [tag_constants.SERVING], model_path + '/001/') signature = meta_graph_def.signature_def self.x = signature['prediction'].inputs['input'].name self.result = signature['prediction'].outputs['output'].name def run(self, input_data): _input_data = [] _input_data.append(input_data) y = self.sess.run(self.result, feed_dict={self.x: _input_data}) return y @app.route('/') def hello(): return 'hello world' @app.route('/predict', methods=['POST']) def model_predict(): input_json = request.get_json() method = input_json['method'] input_data = input_json['data'] if method != "inference": results = {'ret_code':101,'ret_message':'字段错误'} return json.dumps(results,ensure_ascii=False) input_arr = np.array(input_data) try: result = pred_infer.run(input_arr)[0] if (result[1] > 0.5): ret_status = 'good' else: ret_status = 'bad' ret_code = 100 results = {'ret_code':ret_code,'ret_message':'处理成功','result':result.tolist(),'ret_status':ret_status} except: ret_code = 201 results = {'ret_code':ret_code,'ret_message':'参数错误'} results_json = json.dumps(results,ensure_ascii=False) return results_json #model_path = '.\\models\\healthy\\model_state' model_path = 'D:\\YourModelPath\\models\\model_state' pred_infer = predict(model_path) def MyServer(host, port): server = pywsgi.WSGIServer((host, port), app) server.serve_forever() if __name__ == '__main__': MyServer('0.0.0.0', 8088)
- 将python文件打包成exe文件。
D:\Python36\Scripts\pyinstaller.exe -F .\app.py #dist目录下生成app.exe
- 命令行测试app.exe能否正常运行，提供推理服务。
下载nssm，使用nssm实现注册/开启/关闭/更新/移除服务。
nssm\win32\nssm.exe install appServer #注册服务，appServer是服务名 nssm\win32\nssm.exe start appServer #开启服务，appServer是服务名
- pywin32的方案
将tensorflow模型的推理逻辑改写成flask服务，假设文件为app.py(推理模块)和server.py(服务模块)。
# app.py文件 from flask import Flask, request import numpy as np from tensorflow.python.saved_model import tag_constants from tensorflow.contrib.tensor_forest.python import tensor_forest from tensorflow.python.ops import resources import tensorflow.compat.v1 as tf import json from gevent import pywsgi import multiprocessing from multiprocessing import freeze_support from datetime import datetime import platform app = Flask(__name__) class predict(): def __init__(self, model_path): # with tf.Session() as self.sess: self.sess = tf.Session() meta_graph_def = tf.saved_model.loader.load(self.sess, [tag_constants.SERVING], model_path + '/001/') signature = meta_graph_def.signature_def self.x = signature['prediction'].inputs['input'].name self.result = signature['prediction'].outputs['output'].name def run(self, input_data): _input_data = [] _input_data.append(input_data) y = self.sess.run(self.result, feed_dict={self.x: _input_data}) return y @app.route('/') def hello(): return 'hello world' @app.route('/predict', methods=['POST']) def model_predict(): input_json = request.get_json() method = input_json['method'] input_data = input_json['data'] if method != "inference": results = {'ret_code':101,'ret_message':'字段错误'} return json.dumps(results,ensure_ascii=False) input_arr = np.array(input_data) try: result = pred_infer.run(input_arr)[0] if (result[1] > 0.5): ret_status = 'good' else: ret_status = 'bad' ret_code = 100 results = {'ret_code':ret_code,'ret_message':'处理成功','result':result.tolist(),'ret_status':ret_status} except: ret_code = 201 results = {'ret_code':ret_code,'ret_message':'参数错误'} results_json = json.dumps(results,ensure_ascii=False) return results_json #model_path = '.\\models\\healthy\\model_state' model_path = 'D:\\YourModelPath\\models\\model_state' pred_infer = predict(model_path)
- 就是把WSGIServer调用的部分放到server.py中。拆分的原因很明显，解耦合，方便其他模型做服务时，只在app.py内改动。特别注意， 模型的路径需要用绝对路径，相对路径可以注册服务，但无法正常启动服务(闪退)。
# server.py文件 import win32serviceutil from gevent.pywsgi import WSGIServer from app import app class Service(win32serviceutil.ServiceFramework): # 服务名 _svc_name_ = "flask_gevent_service_test" # 显示服务名 _svc_display_name_ = "flask gevent service test display name" # 描述 _svc_description_ = "flask gevent service test description" def __init__(self, *args): super().__init__(*args) # host和ip绑定 self.http_server = WSGIServer(('127.0.0.1', 8088), app) self.SvcStop = self.http_server.stop self.SvcDoRun = self.http_server.serve_forever if __name__ == '__main__': win32serviceutil.HandleCommandLine(Service)
- 使用python自带的pythonServer实现注册/开启/关闭/更新/移除服务。
python server.py install #注册服务 python server.py start #开启服务
- 总结：在某些情况下无法使用nssm的方案，比如防火墙拦截等，这时可选择第二种方案。当然第二种方案的执行命令仍然需要python环境包，可以在此基础上将app.py和server.py两个文件打包成一个exe，方便移植。这部分操作读者可以参考第一种方案中的打包方法自行验证。
