开发者社区> taro_秋刀鱼> 正文
阿里云
为了无法计算的价值
打开APP
阿里云APP内打开

Tensorflow Serving部署模型与调用

简介: 本文以mnist为数据集,使用keras 构建CNN网络,将训练获取的模型通过Tensorflow Serving方式部署提供Rest Full接口,分别使用PostMan和Python调用服务,代码编辑调试使用阿里云PAI DSW实例,模型部署使用阿里云ECS虚拟机。
+关注继续查看

一、模型训练Code

import matplotlib.pyplot as plt
import time
from numpy import asarray
from numpy import unique
from numpy import argmax
from tensorflow.keras.datasets.mnist import load_data
from tensorflow.keras import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras.layers import Conv2D
from tensorflow.keras.layers import MaxPool2D
from tensorflow.keras.layers import Flatten
from tensorflow.keras.layers import Dropout

#load MNIST dataset
(x_train, y_train), (x_test, y_test) = load_data()
print(f'Train: X={x_train.shape}, y={y_train.shape}')
print(f'Test: X={x_test.shape}, y={y_test.shape}')

# reshape data to have a single channel
x_train = x_train.reshape((x_train.shape[0], x_train.shape[1], x_train.shape[2], 1))
x_test = x_test.reshape((x_test.shape[0], x_test.shape[1], x_test.shape[2], 1))

# normalize pixel values
x_train = x_train.astype('float32') / 255.0
x_test = x_test.astype('float32') / 255.0

# set input image shape
input_shape = x_train.shape[1:]

# set number of classes
n_classes = len(unique(y_train))

# define model
model = Sequential()
model.add(Conv2D(64, (3,3), activation='relu', input_shape=input_shape))
model.add(MaxPool2D((2, 2)))
model.add(Conv2D(32, (3,3), activation='relu'))
model.add(MaxPool2D((2, 2)))
model.add(Flatten())
model.add(Dense(50, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(n_classes, activation='softmax'))

# define loss and optimizer
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])

# fit the model
model.fit(x_train, y_train, epochs=10, batch_size=128, verbose=1)

# evaluate the model
loss, acc = model.evaluate(x_test, y_test, verbose=0)
print('Accuracy: %.3f' % acc)

#save model
ts = int(time.time())
file_path = f"./img_classifier/{ts}/"
model.save(filepath=file_path, save_format='tf')
  • 直接运行代码训练获取的模型:

图片.png


二、部署模型

  • 2.1 安装docker

CentOS Docker 安装

  • 2.2 Installing Tensorflow Serving
docker pull tensorflow/serving

图片.png

  • 2.3 部署模型
docker run -p 8501:8501 --name tfserving_classifier --mount type=bind,source=/root/demo/img_classifier/,target=/models/img_classifier -e MODEL_NAME=img_classifier -t tensorflow/serving

三、模型调用

  • 3.1 服务地址
http://《部署服务的机器IP》:8501/v1/models/img_classifier:predict

注意:如果机器有内外地址,测试机器和服务部署机器内外可达,建议使用内外地址,如果测试机器与服务机器内外不可达,建议可以使用部署服务机器的公网IP地址。

  • 3.2 Python测试Code

import requests
import json
from tensorflow.keras.datasets.mnist import load_data

#load MNIST dataset
(_, _), (x_test, y_test) = load_data()

# reshape data to have a single channel
x_test = x_test.reshape((x_test.shape[0], x_test.shape[1], x_test.shape[2], 1))
# normalize pixel values
x_test = x_test.astype('float32') / 255.0

#server URL
url = 'http://192.168.0.***:8501/v1/models/img_classifier:predict'

def make_prediction(instances):
   data = json.dumps({"signature_name": "serving_default", "instances": instances.tolist()})
   headers = {"content-type": "application/json"}
   json_response = requests.post(url, data=data, headers=headers)
   predictions = json.loads(json_response.text)['predictions']
   return predictions

xx = x_test[0:1]
predictions = make_prediction(xx)

print(predictions)
  • 3.3 Post Man测试

图片.png

  • 测试Body Json文本
{"signature_name": "serving_default", "instances": [[[[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.3294117748737335], [0.7254902124404907], [0.6235294342041016], [0.5921568870544434], [0.23529411852359772], [0.1411764770746231], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.8705882430076599], [0.9960784316062927], [0.9960784316062927], [0.9960784316062927], [0.9960784316062927], [0.9450980424880981], [0.7764706015586853], [0.7764706015586853], [0.7764706015586853], [0.7764706015586853], [0.7764706015586853], [0.7764706015586853], [0.7764706015586853], [0.7764706015586853], [0.6666666865348816], [0.20392157137393951], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.26274511218070984], [0.4470588266849518], [0.2823529541492462], [0.4470588266849518], [0.6392157077789307], [0.8901960849761963], [0.9960784316062927], [0.8823529481887817], [0.9960784316062927], [0.9960784316062927], [0.9960784316062927], [0.9803921580314636], [0.8980392217636108], [0.9960784316062927], [0.9960784316062927], [0.5490196347236633], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.06666667014360428], [0.25882354378700256], [0.054901961237192154], [0.26274511218070984], [0.26274511218070984], [0.26274511218070984], [0.23137255012989044], [0.08235294371843338], [0.9254902005195618], [0.9960784316062927], [0.4156862795352936], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.32549020648002625], [0.9921568632125854], [0.8196078538894653], [0.07058823853731155], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.08627451211214066], [0.9137254953384399], [1.0], [0.32549020648002625], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.5058823823928833], [0.9960784316062927], [0.9333333373069763], [0.1725490242242813], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.23137255012989044], [0.9764705896377563], [0.9960784316062927], [0.24313725531101227], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.5215686559677124], [0.9960784316062927], [0.7333333492279053], [0.019607843831181526], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.03529411926865578], [0.8039215803146362], [0.9725490212440491], [0.22745098173618317], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.4941176474094391], [0.9960784316062927], [0.7137255072593689], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.29411765933036804], [0.9843137264251709], [0.9411764740943909], [0.2235294133424759], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.07450980693101883], [0.8666666746139526], [0.9960784316062927], [0.6509804129600525], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0117647061124444], [0.7960784435272217], [0.9960784316062927], [0.8588235378265381], [0.13725490868091583], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.14901961386203766], [0.9960784316062927], [0.9960784316062927], [0.3019607961177826], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.12156862765550613], [0.8784313797950745], [0.9960784316062927], [0.45098039507865906], [0.003921568859368563], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.5215686559677124], [0.9960784316062927], [0.9960784316062927], [0.20392157137393951], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.239215686917305], [0.9490196108818054], [0.9960784316062927], [0.9960784316062927], [0.20392157137393951], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.4745098054409027], [0.9960784316062927], [0.9960784316062927], [0.8588235378265381], [0.1568627506494522], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.4745098054409027], [0.9960784316062927], [0.8117647171020508], [0.07058823853731155], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]], [[0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0], [0.0]]]]}


更多参考

How to Serve Machine Learning Models With TensorFlow Serving and Docker

版权声明:本文内容由阿里云实名注册用户自发贡献,版权归原作者所有,阿里云开发者社区不拥有其著作权,亦不承担相应法律责任。具体规则请查看《阿里云开发者社区用户服务协议》和《阿里云开发者社区知识产权保护指引》。如果您发现本社区中有涉嫌抄袭的内容,填写侵权投诉表单进行举报,一经查实,本社区将立刻删除涉嫌侵权内容。

相关文章
TensorFlow笔记--Deep Dream模型(上)
TensorFlow学习笔记--Deep Dream模型
92 0
Tensorflow V2 图像识别模型训练流程
这里以 TinyMind 《汉字书法识别》比赛数据为例,展示使用 Tensorflow V2.0 进行图像数据分类模型训练的整个流程。
173 0
tensorflow serving api
背景介绍  tensorflow serving 在客户端和服务端之间的通信采用的是RPC/REST协议。在TFS提供的REST协议接口存在一定的局限性,REST和RPC对比如下: 1、REST在实际应用中不能支持运行过程动态模型发布。
2331 0
TensorFlow的新型模型优化工具包可使模型速度提高3倍
本周TensorFlow推出了一个新的模型优化工具包。这套技术同时适用于新老开发员以优化机器学习模型,特别是运行TensorFlowLite的开发人员。任何现有TensorFlow模型均适用。
1079 0
用Tensorflow搭建预测磁盘性能的机器学习模型
收集足够多的磁盘性能原始数据,选择合适的机器学习模型来仿真性能数据。
1478 0
[译] 写给机器学习工程师:如何测试Tensorflow模型
## 引言 这篇文章来自斯坦福大学计算与数学工程所(Institute for Computational & Mathematical Engineering)博士生Guillaume Genthial的[博客](https://guillaumegenthial.github.io/testing.html)。主要介绍了如何将工程界里已经得到充分认可的单元测试实践应用到算法建模的领域中,
1783 0
我用 tensorflow 实现的 “一个神经聊天模型”
这个工作尝试重现这个论文的结果 A Neural Conversational Model (aka the Google chatbot). 它使用了循环神经网络(seq2seq 模型)来进行句子预测。它是用 python 和 TensorFlow 开发。
1629 0
TensorFlow中滑动平均模型介绍
内容移至:https://oldpan.me/archives/tensorflow-movingaverage 内容总结于《TensorFlow实战Google深度学习框架》 不知道大家有没有听过一阶滞后滤波法...
1271 0
+关注
taro_秋刀鱼
博客园主页:https://home.cnblogs.com/u/taro/
249
文章
28
问答
来源圈子
更多
文章排行榜
最热
最新
相关电子书
更多
低代码开发师(初级)实战教程
立即下载
阿里巴巴DevOps 最佳实践手册
立即下载
冬季实战营第三期:MySQL数据库进阶实战
立即下载