Tutorial: Implementation of Siamese Network on Caffe, Torch, Tensorflow

简介: Tutorial: Implementation of Siamese Network on Caffe, Torch, Tensorflow    1. caffe version:     If you want to try this network, just do as the o...

Tutorial: Implementation of Siamese Network on Caffe, Torch, Tensorflow 

 


  1. caffe version: 

    If you want to try this network, just do as the offical document said, like the following codes:    

  1 ---
  2 title: Siamese Network Tutorial
  3 description: Train and test a siamese network on MNIST data.
  4 category: example
  5 include_in_docs: true
  6 layout: default
  7 priority: 100
  8 ---
  9 
 10 # Siamese Network Training with Caffe
 11 This example shows how you can use weight sharing and a contrastive loss
 12 function to learn a model using a siamese network in Caffe.
 13 
 14 We will assume that you have caffe successfully compiled. If not, please refer
 15 to the [Installation page](../../installation.html). This example builds on the
 16 [MNIST tutorial](mnist.html) so it would be a good idea to read that before
 17 continuing.
 18 
 19 *The guide specifies all paths and assumes all commands are executed from the
 20 root caffe directory*
 21 
 22 ## Prepare Datasets
 23 
 24 You will first need to download and convert the data from the MNIST
 25 website. To do this, simply run the following commands:
 26 
 27     ./data/mnist/get_mnist.sh
 28     ./examples/siamese/create_mnist_siamese.sh
 29 
 30 After running the script there should be two datasets,
 31 `./examples/siamese/mnist_siamese_train_leveldb`, and
 32 `./examples/siamese/mnist_siamese_test_leveldb`.
 33 
 34 ## The Model
 35 First, we will define the model that we want to train using the siamese network.
 36 We will use the convolutional net defined in
 37 `./examples/siamese/mnist_siamese.prototxt`. This model is almost
 38 exactly the same as the [LeNet model](mnist.html), the only difference is that
 39 we have replaced the top layers that produced probabilities over the 10 digit
 40 classes with a linear "feature" layer that produces a 2 dimensional vector.
 41 
 42     layer {
 43       name: "feat"
 44       type: "InnerProduct"
 45       bottom: "ip2"
 46       top: "feat"
 47       param {
 48         name: "feat_w"
 49         lr_mult: 1
 50       }
 51       param {
 52         name: "feat_b"
 53         lr_mult: 2
 54       }
 55       inner_product_param {
 56         num_output: 2
 57       }
 58     }
 59 
 60 ## Define the Siamese Network
 61 
 62 In this section we will define the siamese network used for training. The
 63 resulting network is defined in
 64 `./examples/siamese/mnist_siamese_train_test.prototxt`.
 65 
 66 ### Reading in the Pair Data
 67 
 68 We start with a data layer that reads from the LevelDB database we created
 69 earlier. Each entry in this database contains the image data for a pair of
 70 images (`pair_data`) and a binary label saying if they belong to the same class
 71 or different classes (`sim`).
 72 
 73     layer {
 74       name: "pair_data"
 75       type: "Data"
 76       top: "pair_data"
 77       top: "sim"
 78       include { phase: TRAIN }
 79       transform_param {
 80         scale: 0.00390625
 81       }
 82       data_param {
 83         source: "examples/siamese/mnist_siamese_train_leveldb"
 84         batch_size: 64
 85       }
 86     }
 87 
 88 In order to pack a pair of images into the same blob in the database we pack one
 89 image per channel. We want to be able to work with these two images separately,
 90 so we add a slice layer after the data layer. This takes the `pair_data` and
 91 slices it along the channel dimension so that we have a single image in `data`
 92 and its paired image in `data_p.`
 93 
 94     layer {
 95       name: "slice_pair"
 96       type: "Slice"
 97       bottom: "pair_data"
 98       top: "data"
 99       top: "data_p"
100       slice_param {
101         slice_dim: 1
102         slice_point: 1
103       }
104     }
105 
106 ### Building the First Side of the Siamese Net
107 
108 Now we can specify the first side of the siamese net. This side operates on
109 `data` and produces `feat`. Starting from the net in
110 `./examples/siamese/mnist_siamese.prototxt` we add default weight fillers. Then
111 we name the parameters of the convolutional and inner product layers. Naming the
112 parameters allows Caffe to share the parameters between layers on both sides of
113 the siamese net. In the definition this looks like:
114 
115     ...
116     param { name: "conv1_w" ...  }
117     param { name: "conv1_b" ...  }
118     ...
119     param { name: "conv2_w" ...  }
120     param { name: "conv2_b" ...  }
121     ...
122     param { name: "ip1_w" ...  }
123     param { name: "ip1_b" ...  }
124     ...
125     param { name: "ip2_w" ...  }
126     param { name: "ip2_b" ...  }
127     ...
128 
129 ### Building the Second Side of the Siamese Net
130 
131 Now we need to create the second path that operates on `data_p` and produces
132 `feat_p`. This path is exactly the same as the first. So we can just copy and
133 paste it. Then we change the name of each layer, input, and output by appending
134 `_p` to differentiate the "paired" layers from the originals.
135 
136 ### Adding the Contrastive Loss Function
137 
138 To train the network we will optimize a contrastive loss function proposed in:
139 Raia Hadsell, Sumit Chopra, and Yann LeCun "Dimensionality Reduction by Learning
140 an Invariant Mapping". This loss function encourages matching pairs to be close
141 together in feature space while pushing non-matching pairs apart. This cost
142 function is implemented with the `CONTRASTIVE_LOSS` layer:
143 
144     layer {
145         name: "loss"
146         type: "ContrastiveLoss"
147         contrastive_loss_param {
148             margin: 1.0
149         }
150         bottom: "feat"
151         bottom: "feat_p"
152         bottom: "sim"
153         top: "loss"
154     }
155 
156 ## Define the Solver
157 
158 Nothing special needs to be done to the solver besides pointing it at the
159 correct model file. The solver is defined in
160 `./examples/siamese/mnist_siamese_solver.prototxt`.
161 
162 ## Training and Testing the Model
163 
164 Training the model is simple after you have written the network definition
165 protobuf and solver protobuf files. Simply run
166 `./examples/siamese/train_mnist_siamese.sh`:
167 
168     ./examples/siamese/train_mnist_siamese.sh
169 
170 # Plotting the results
171 
172 First, we can draw the model and siamese networks by running the following
173 commands that draw the DAGs defined in the .prototxt files:
174 
175     ./python/draw_net.py \
176         ./examples/siamese/mnist_siamese.prototxt \
177         ./examples/siamese/mnist_siamese.png
178 
179     ./python/draw_net.py \
180         ./examples/siamese/mnist_siamese_train_test.prototxt \
181         ./examples/siamese/mnist_siamese_train_test.png
182 
183 Second, we can load the learned model and plot the features using the iPython
184 notebook:
185 
186     ipython notebook ./examples/siamese/mnist_siamese.ipynb
View Code

  

   

  

  If you want to shown the neural network in a image. first, you should install the following softwares: 

    1. sudo apt-get install graphviz 

    2. sudo pip install pydot2 

  then, you can draw the following graph using tool provided by python files. 

    

  

   

  


 

    If you want to know how to implement this on your own data. You should: 

    1. Preparing your data:

      ==>> positive and negative image pairs and corresponding label (1 and -1).

    2. Convert the files into lmdb files

    3. then just do as above mentioned. 

 

  ==>>  But  I am still feel confused about how to deal with this whole process.

      Will fill with this part later. 

      

 

 


 

  2. Siamese Lasagne Theano version :   

 

  1 # Run on GPU: THEANO_FLAGS=mode=FAST_RUN,device=gpu,floatX=float32 python mnist_siamese_graph.py
  2 from __future__ import print_function
  3  
  4 import sys
  5 import os
  6 import time
  7 import numpy as np
  8 import theano
  9 import theano.tensor as T
 10 import lasagne
 11 import utils 
 12 from progressbar import AnimatedMarker, Bar, BouncingBar, Counter, ETA, \
 13     FileTransferSpeed, FormatLabel, Percentage, \
 14     ProgressBar, ReverseBar, RotatingMarker, \
 15     SimpleProgress, Timer
 16 import matplotlib.pyplot as plt
 17 from matplotlib import gridspec
 18 import cPickle as pickle
 19 import time
 20 from sklearn import metrics
 21 from scipy import interpolate
 22 from lasagne.regularization import regularize_layer_params_weighted, l2, l1
 23 from lasagne.regularization import regularize_layer_params
 24   
 25 NUM_EPOCHS = 40
 26 BATCH_SIZE = 100
 27 LEARNING_RATE = 0.001
 28 MOMENTUM = 0.9
 29  
 30 # def build_cnn(input_var=None):
 31 #     net = lasagne.layers.InputLayer(shape=(None, 1, 64, 64),
 32 #                                         input_var=input_var)
 33 #     cnn1 = lasagne.layers.Conv2DLayer(
 34 #             net, num_filters=96, filter_size=(7, 7),
 35 #             nonlinearity=lasagne.nonlinearities.rectify,
 36 #             W=lasagne.init.GlorotNormal())
 37 #     pool1 = lasagne.layers.MaxPool2DLayer(cnn1, pool_size=(2, 2))
 38 #     cnn2 = lasagne.layers.Conv2DLayer(
 39 #             pool1, num_filters=64, filter_size=(6, 6),
 40 #             nonlinearity=lasagne.nonlinearities.rectify,
 41 #             W=lasagne.init.GlorotNormal())
 42 #     fc1 = lasagne.layers.DenseLayer(cnn2, num_units=128)
 43 #     # network = lasagne.layers.FlattenLayer(fc1)
 44 #     return fc1
 45  
 46 def build_cnn(input_var=None):
 47     net = lasagne.layers.InputLayer(shape=(None, 1, 64, 64),
 48                                         input_var=input_var)
 49     cnn1 = lasagne.layers.Conv2DLayer(
 50             net, num_filters=96, filter_size=(7, 7),
 51             nonlinearity=lasagne.nonlinearities.rectify,
 52             stride = (3,3),
 53             W=lasagne.init.GlorotNormal())
 54     pool1 = lasagne.layers.MaxPool2DLayer(cnn1, pool_size=(2, 2))
 55     cnn2 = lasagne.layers.Conv2DLayer(
 56             pool1, num_filters=192, filter_size=(5, 5),
 57             nonlinearity=lasagne.nonlinearities.rectify,
 58             W=lasagne.init.GlorotNormal())
 59     pool2 = lasagne.layers.MaxPool2DLayer(cnn2, pool_size=(2, 2))
 60     cnn3 = lasagne.layers.Conv2DLayer(
 61             pool2, num_filters=256, filter_size=(3, 3),
 62             nonlinearity=lasagne.nonlinearities.rectify,
 63             W=lasagne.init.GlorotNormal())
 64     # fc1 = lasagne.layers.DenseLayer(cnn2, num_units=128)
 65     network = lasagne.layers.FlattenLayer(cnn3)
 66     return network
 67  
 68 def init_data(train,test):
 69     dtrain = utils.load_brown_dataset("/home/vassilis/Datasets/"+train+"/")
 70     dtest = utils.load_brown_dataset("/home/vassilis/Datasets/"+test+"/")
 71  
 72     dtrain['patches'] = dtrain['patches'].astype('float32')
 73     dtest['patches'] = dtest['patches'].astype('float32')
 74  
 75     dtrain['patches'] /= 255
 76     dtest['patches'] /= 255
 77  
 78     mu = dtrain['patches'].mean()
 79     dtrain['patches'] = dtrain['patches'] - mu
 80     dtest['patches'] = dtest['patches'] - mu
 81     return dtrain,dtest
 82  
 83 def eval_test(net,d):
 84     bs = 100
 85     pb = np.array_split(d['patches'],bs)
 86     descrs = []
 87     for i,minib in enumerate(pb):
 88         dd = lasagne.layers.get_output(net,minib).eval()
 89         descrs.append(dd)
 90  
 91     descrs = np.vstack(descrs)
 92     dists = np.zeros(100000,)
 93     lbls = np.zeros(100000,)
 94      
 95     for i in range(100000):
 96         idx1 = d['testgt'][i][0]
 97         idx2 = d['testgt'][i][1]
 98         lbl = d['testgt'][i][2]
 99         dists[i] = np.linalg.norm(descrs[idx1]-descrs[idx2])
100         lbls[i] = lbl
101         #print(dists[i],lbls[i])
102     fpr, tpr, thresholds = metrics.roc_curve(lbls, -dists, pos_label=1)
103     f = interpolate.interp1d(tpr, fpr)
104     fpr95 = f(0.95)
105     print('fpr95-> '+str(fpr95))
106  
107 def main(num_epochs=NUM_EPOCHS):
108     widgets = ['Mini-batch training: ', Percentage(), ' ', Bar(),
109              ' ', ETA(), ' ']
110     print("> Loading data...")
111     dtrain,dtest = init_data('liberty','notredame')
112     net = build_cnn()
113  
114     dtr = utils.gen_pairs(dtrain,1200000)
115     ntr = dtr.shape[0]
116  
117     X = T.tensor4()
118     y = T.ivector()
119     a = lasagne.layers.get_output(net,X)
120  
121     fx1 = a[1::2, :]
122     fx2 = a[::2, :]
123     d = T.sum(( fx1- fx2)**2, -1)
124  
125     l2_penalty = regularize_layer_params(net, l2) * 1e-3
126  
127     loss = T.mean(y * d +
128                   (1 - y) * T.maximum(0, 1 - d))+l2_penalty
129  
130     all_params = lasagne.layers.get_all_params(net)
131     updates = lasagne.updates.nesterov_momentum(
132         loss, all_params, LEARNING_RATE, MOMENTUM)
133  
134     trainf = theano.function([X, y], loss,updates=updates)    
135  
136     num_batches = ntr // BATCH_SIZE
137     print(num_batches)
138     print("> Done loading data...")
139     print("> Started learning with "+str(num_batches)+" batches")
140  
141     shuf = np.random.permutation(ntr)
142  
143     X_tr = np.zeros((BATCH_SIZE*2,1,64,64)).astype('float32')
144     y_tr = np.zeros(BATCH_SIZE).astype('int32')
145  
146     for epoch in range(NUM_EPOCHS):
147         batch_train_losses = []
148         pbar = ProgressBar(widgets=widgets, maxval=num_batches).start()
149         for k in range(num_batches):
150             sh = shuf[k*BATCH_SIZE:k*BATCH_SIZE+BATCH_SIZE]
151             pbar.update(k)
152             # fill batch here
153             for s in range(0,BATCH_SIZE*2,2):
154                 # idx1 = dtrain['traingt'][sh[s/2],0]
155                 # idx2 = dtrain['traingt'][sh[s/2],1]
156                 # lbl = dtrain['traingt'][sh[s/2],2]
157  
158                 idx1 = dtr[sh[s/2]][0]
159                 idx2 = dtr[sh[s/2]][1]
160                 lbl = dtr[sh[s/2]][2]
161                  
162                 X_tr[s] = dtrain['patches'][idx1]
163                 X_tr[s+1] = dtrain['patches'][idx2]
164                 y_tr[s/2] = lbl
165  
166             batch_train_loss = trainf(X_tr,y_tr)
167             batch_train_losses.append(batch_train_loss)
168         avg_train_loss = np.mean(batch_train_losses)
169         pbar.finish()
170         print("> Epoch " + str(epoch) + ", loss: "+str(avg_train_loss))
171  
172         eval_test(net,dtest)
173  
174         with open('net.pickle', 'wb') as f:
175             pickle.dump(net, f, -1)
176      
177         # netlayers = lasagne.layers.get_all_layers(net)
178         # print(netlayers)
179         # layer = netlayers[1]
180         # print(layer)
181         # print(layer.num_filters)
182         # W = layer.W.get_value()
183         # b = layer.b.get_value()
184         # f = [w + bb for w, bb in zip(W, b)]
185         # gs = gridspec.GridSpec(8, 12)
186         # for i in range(layer.num_filters):
187         #     g = gs[i]
188         #     ax = plt.subplot(g)
189         #     ax.grid()
190         #     ax.set_xticks([])
191         #     ax.set_yticks([])
192         #     ax.imshow(f[i][0])
193         # plt.show()
194          
195  
196 if __name__ == '__main__':
197    main(sys.argv[1])
View Code

 

 

  3. Tensorflow version :

    Github link: https://github.com/ywpkwon/siamese_tf_mnist 

    

 

 

 

  

相关文章
|
6月前
|
机器学习/深度学习 PyTorch TensorFlow
一文带你了解 三种深度学习框架(Caffe,Tensorflow,Pytorch)的基本内容、优缺点以及三者的对比
一文带你了解 三种深度学习框架(Caffe,Tensorflow,Pytorch)的基本内容、优缺点以及三者的对比
418 1
|
机器学习/深度学习 TensorFlow 算法框架/工具
一图看懂深度学习框架对比----Caffe Torch Theano TensorFlow
  Caffe Torch Theano TensorFlow Language C++, Python Lua Python Python Pretrained Yes ++ Yes ++ Yes (Lasagne) Inception...
1264 0
|
5月前
|
机器学习/深度学习 人工智能 算法
海洋生物识别系统+图像识别+Python+人工智能课设+深度学习+卷积神经网络算法+TensorFlow
海洋生物识别系统。以Python作为主要编程语言,通过TensorFlow搭建ResNet50卷积神经网络算法,通过对22种常见的海洋生物('蛤蜊', '珊瑚', '螃蟹', '海豚', '鳗鱼', '水母', '龙虾', '海蛞蝓', '章鱼', '水獭', '企鹅', '河豚', '魔鬼鱼', '海胆', '海马', '海豹', '鲨鱼', '虾', '鱿鱼', '海星', '海龟', '鲸鱼')数据集进行训练,得到一个识别精度较高的模型文件,然后使用Django开发一个Web网页平台操作界面,实现用户上传一张海洋生物图片识别其名称。
175 7
海洋生物识别系统+图像识别+Python+人工智能课设+深度学习+卷积神经网络算法+TensorFlow
|
5月前
|
机器学习/深度学习 人工智能 算法
【乐器识别系统】图像识别+人工智能+深度学习+Python+TensorFlow+卷积神经网络+模型训练
乐器识别系统。使用Python为主要编程语言,基于人工智能框架库TensorFlow搭建ResNet50卷积神经网络算法,通过对30种乐器('迪吉里杜管', '铃鼓', '木琴', '手风琴', '阿尔卑斯号角', '风笛', '班卓琴', '邦戈鼓', '卡萨巴', '响板', '单簧管', '古钢琴', '手风琴(六角形)', '鼓', '扬琴', '长笛', '刮瓜', '吉他', '口琴', '竖琴', '沙槌', '陶笛', '钢琴', '萨克斯管', '锡塔尔琴', '钢鼓', '长号', '小号', '大号', '小提琴')的图像数据集进行训练,得到一个训练精度较高的模型,并将其
67 0
【乐器识别系统】图像识别+人工智能+深度学习+Python+TensorFlow+卷积神经网络+模型训练
|
2月前
|
机器学习/深度学习 数据挖掘 TensorFlow
解锁Python数据分析新技能,TensorFlow&PyTorch双引擎驱动深度学习实战盛宴
在数据驱动时代,Python凭借简洁的语法和强大的库支持,成为数据分析与机器学习的首选语言。Pandas和NumPy是Python数据分析的基础,前者提供高效的数据处理工具,后者则支持科学计算。TensorFlow与PyTorch作为深度学习领域的两大框架,助力数据科学家构建复杂神经网络,挖掘数据深层价值。通过Python打下的坚实基础,结合TensorFlow和PyTorch的强大功能,我们能在数据科学领域探索无限可能,解决复杂问题并推动科研进步。
53 0
|
2月前
|
机器学习/深度学习 数据挖掘 TensorFlow
从数据小白到AI专家:Python数据分析与TensorFlow/PyTorch深度学习的蜕变之路
【9月更文挑战第10天】从数据新手成长为AI专家,需先掌握Python基础语法,并学会使用NumPy和Pandas进行数据分析。接着,通过Matplotlib和Seaborn实现数据可视化,最后利用TensorFlow或PyTorch探索深度学习。这一过程涉及从数据清洗、可视化到构建神经网络的多个步骤,每一步都需不断实践与学习。借助Python的强大功能及各类库的支持,你能逐步解锁数据的深层价值。
57 0
|
3月前
|
持续交付 测试技术 jenkins
JSF 邂逅持续集成,紧跟技术热点潮流,开启高效开发之旅,引发开发者强烈情感共鸣
【8月更文挑战第31天】在快速发展的软件开发领域,JavaServer Faces(JSF)这一强大的Java Web应用框架与持续集成(CI)结合,可显著提升开发效率及软件质量。持续集成通过频繁的代码集成及自动化构建测试,实现快速反馈、高质量代码、加强团队协作及简化部署流程。以Jenkins为例,配合Maven或Gradle,可轻松搭建JSF项目的CI环境,通过JUnit和Selenium编写自动化测试,确保每次构建的稳定性和正确性。
56 0
|
3月前
|
测试技术 数据库
探索JSF单元测试秘籍!如何让您的应用更稳固、更高效?揭秘成功背后的测试之道!
【8月更文挑战第31天】在 JavaServer Faces(JSF)应用开发中,确保代码质量和可维护性至关重要。本文详细介绍了如何通过单元测试实现这一目标。首先,阐述了单元测试的重要性及其对应用稳定性的影响;其次,提出了提高 JSF 应用可测试性的设计建议,如避免直接访问外部资源和使用依赖注入;最后,通过一个具体的 `UserBean` 示例,展示了如何利用 JUnit 和 Mockito 框架编写有效的单元测试。通过这些方法,不仅能够确保代码质量,还能提高开发效率和降低维护成本。
49 0
|
3月前
|
UED 开发者
哇塞!Uno Platform 数据绑定超全技巧大揭秘!从基础绑定到高级转换,优化性能让你的开发如虎添翼
【8月更文挑战第31天】在开发过程中,数据绑定是连接数据模型与用户界面的关键环节,可实现数据自动更新。Uno Platform 提供了简洁高效的数据绑定方式,使属性变化时 UI 自动同步更新。通过示例展示了基本绑定方法及使用 `Converter` 转换数据的高级技巧,如将年龄转换为格式化字符串。此外,还可利用 `BindingMode.OneTime` 提升性能。掌握这些技巧能显著提高开发效率并优化用户体验。
56 0