1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
7 章 Ceph 整合OpenStack 遇到问题解决
7.1  一个日志引发的错误追踪
1 )  Ceph 问题起因 http: / / bbs.ceph.org.cn / question / 161  错误日志
  
 
2 )  找到 vim nova / virt / libvirt / driver.py 代码处  3090 
* * * * * * * * * * * * * * * * * * * * * * * *
     def  _get_guest_disk_config( self , instance, name, disk_mapping, inst_type,
                                image_type = None ):
         if  CONF.libvirt.hw_disk_discard:
             if  not  self ._host.has_min_version(MIN_LIBVIRT_DISCARD_VERSION,
                                               MIN_QEMU_DISCARD_VERSION,
                                               REQ_HYPERVISOR_DISCARD):
                 msg  =  (_( 'Volume sets discard option, but libvirt %(libvirt)s'
                          ' or later is required, qemu %(qemu)s'
                          ' or later is required.' %
                       { 'libvirt' : MIN_LIBVIRT_DISCARD_VERSION,
                        'qemu' : MIN_QEMU_DISCARD_VERSION})
                 raise  exception.Invalid(msg)
             else :
                 pass
         image  =  self .image_backend.image(instance,
                                          name,
                                          image_type)
         disk_info  =  disk_mapping[name]
         return  image.libvirt_info(disk_info[ 'bus' ],
                                   disk_info[ 'dev' ],
                                   disk_info[ 'type' ],
                                   self .disk_cachemode,
                                   inst_type[ 'extra_specs' ],
                                   self ._host.get_version())
* * * * * * * * * * * * * * * * * * * * * * * *
对比之后的代码修改
  
* * * * * * * * * * * * * * * * * * * * * * * *
     def  _get_guest_disk_config( self , instance, name, disk_mapping, inst_type,
                                image_type = None ):
         image  =  self .image_backend.image(instance,
                                          name,
                                          image_type)
         disk_info  =  disk_mapping[name]
         return  image.libvirt_info(disk_info[ 'bus' ],
                                   disk_info[ 'dev' ],
                                   disk_info[ 'type' ],
                                   self .disk_cachemode,
                                   inst_type[ 'extra_specs' ],
                                   self ._host.get_version())
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
 
3 )  继续创建云主机,看是否报其他错误信息
2015 - 08 - 11  01 : 47 : 10.456  82044  ERROR nova.virt.libvirt.driver [req - 27f7ade9 - 3142 - 4ec6 - 815d - 84488c6e0201  -  -  -  -  - ] Error launching a defined domain with XML: <domain  type = 'kvm' >
   <name>instance - 000000d9 < / name>
   <uuid> 4b525800 - 3e00 - 4b48 - a997 - c104f919cde3< / uuid>
   <metadata>
     <nova:instance xmlns:nova = "http://openstack.org/xmlns/libvirt/nova/1.0" >
       <nova:package version = "2015.1.0-3.el7" / >
       <nova:name>exta< / nova:name>
       <nova:creationTime> 2015 - 08 - 10  17 : 47 : 09 < / nova:creationTime>
       <nova:flavor name = "linux-8-8-50" >
         <nova:memory> 8192 < / nova:memory>
         <nova:disk> 120 < / nova:disk>
         <nova:swap> 0 < / nova:swap>
         <nova:ephemeral> 50 < / nova:ephemeral>
         <nova:vcpus> 8 < / nova:vcpus>
       < / nova:flavor>
       <nova:owner>
         <nova:user uuid = "95a96f0ddcf449239c6682a3c310857e" >root< / nova:user>
         <nova:project uuid = "be27eb2862904a0f9c636c337f66709c" >admin< / nova:project>
       < / nova:owner>
       <nova:root  type = "image"  uuid = "59e1c70b-70c8-4c22-9253-fc889f94d891" / >
     < / nova:instance>
   < / metadata>
   <memory unit = 'KiB' > 8388608 < / memory>
   <currentMemory unit = 'KiB' > 8388608 < / currentMemory>
   <vcpu placement = 'static'  cpuset = '0,2,4,6,8,10,12,14,16,18,20,22,24,26,28,30,32,34,36,38' > 8 < / vcpu>
   <cputune>
     <shares> 8192 < / shares>
   < / cputune>
     <sysinfo  type = 'smbios' >
       <system>
         <entry name = 'manufacturer' >Fedora Project< / entry>
         <entry name = 'product' >OpenStack Nova< / entry>
         <entry name = 'version' > 2015.1 . 0 - 3.el7 < / entry>
         <entry name = 'serial' > 09c6f9d1 - 825f - 43e2 - 8774 - 5ed6705af12b < / entry>
         <entry name = 'uuid' > 4b525800 - 3e00 - 4b48 - a997 - c104f919cde3< / entry>
       < / system>
     < / sysinfo>
   <os>
     < type  arch = 'x86_64'  machine = 'pc-i440fx-rhel7.0.0' >hvm< / type >
     <boot dev = 'hd' / >
     <smbios mode = 'sysinfo' / >
   < / os>
   <features>
     <acpi / >
     <apic / >
   < / features>
   <cpu mode = 'host-model' >
     <model fallback = 'allow' / >
     <topology sockets = '8'  cores = '1'  threads = '1' / >
   < / cpu>
   <clock offset = 'utc' >
     <timer name = 'pit'  tickpolicy = 'delay' / >
     <timer name = 'rtc'  tickpolicy = 'catchup' / >
     <timer name = 'hpet'  present = 'no' / >
   < / clock>
   <on_poweroff>destroy< / on_poweroff>
   <on_reboot>restart< / on_reboot>
   <on_crash>destroy< / on_crash>
   <devices>
     <emulator> / usr / libexec / qemu - kvm< / emulator>
     <disk  type = 'network'  device = 'disk' >
       <driver name = 'qemu'  type = 'raw'  cache = 'writeback' / >
       <source protocol = 'rbd'  name = 'vms/4b525800-3e00-4b48-a997-c104f919cde3_disk' >
         <host name = '192.168.103.211'  port = '6789' / >
         <host name = '192.168.103.212'  port = '6789' / >
         <host name = '192.168.103.214'  port = '6789' / >
       < / source>
       <target dev = 'vda'  bus = 'virtio' / >
       <address  type = 'pci'  domain = '0x0000'  bus = '0x00'  slot = '0x04'  function = '0x0' / >
     < / disk>
     <disk  type = 'network'  device = 'disk' >
       <driver name = 'qemu'  type = 'raw'  cache = 'writeback' / >
       <source protocol = 'rbd'  name = 'vms/4b525800-3e00-4b48-a997-c104f919cde3_disk.local' >
         <host name = '192.168.103.211'  port = '6789' / >
         <host name = '192.168.103.212'  port = '6789' / >
         <host name = '192.168.103.214'  port = '6789' / >
       < / source>
       <target dev = 'vdb'  bus = 'virtio' / >
       <address  type = 'pci'  domain = '0x0000'  bus = '0x00'  slot = '0x05'  function = '0x0' / >
     < / disk>
     <controller  type = 'usb'  index = '0' >
       <address  type = 'pci'  domain = '0x0000'  bus = '0x00'  slot = '0x01'  function = '0x2' / >
     < / controller>
     <controller  type = 'pci'  index = '0'  model = 'pci-root' / >
     <interface  type = 'bridge' >
       <mac address = 'fa:16:3e:19:60:f5' / >
       <source bridge = 'br100' / >
       <model  type = 'virtio' / >
       <filterref  filter = 'nova-instance-instance-000000d9-fa163e1960f5' / >
       <address  type = 'pci'  domain = '0x0000'  bus = '0x00'  slot = '0x03'  function = '0x0' / >
     < / interface>
     <serial  type = 'file' >
       <source path = '/data/nova/instances/4b525800-3e00-4b48-a997-c104f919cde3/console.log' / >
       <target port = '0' / >
     < / serial>
     <serial  type = 'pty' >
       <target port = '1' / >
     < / serial>
     <console  type = 'file' >
       <source path = '/data/nova/instances/4b525800-3e00-4b48-a997-c104f919cde3/console.log' / >
       <target  type = 'serial'  port = '0' / >
     < / console>
     < input  type = 'tablet'  bus = 'usb' / >
     < input  type = 'mouse'  bus = 'ps2' / >
     < input  type = 'keyboard'  bus = 'ps2' / >
     <graphics  type = 'vnc'  port = '-1'  autoport = 'yes'  listen = '0.0.0.0'  keymap = 'en-us' >
       <listen  type = 'address'  address = '0.0.0.0' / >
     < / graphics>
     <video>
       <model  type = 'cirrus'  vram = '16384'  heads = '1' / >
       <address  type = 'pci'  domain = '0x0000'  bus = '0x00'  slot = '0x02'  function = '0x0' / >
     < / video>
     <memballoon model = 'virtio' >
       <address  type = 'pci'  domain = '0x0000'  bus = '0x00'  slot = '0x06'  function = '0x0' / >
       <stats period = '10' / >
     < / memballoon>
   < / devices>
< / domain>
 
2015 - 08 - 11  01 : 47 : 10.457  82044  ERROR nova.compute.manager [req - 27f7ade9 - 3142 - 4ec6 - 815d - 84488c6e0201  -  -  -  -  - ] [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3] Instance failed to spawn
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3] Traceback (most recent call last):
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]    File  "/usr/lib/python2.7/site-packages/nova/compute/manager.py" , line  2442 in  _build_resources
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]      yield  resources
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]    File  "/usr/lib/python2.7/site-packages/nova/compute/manager.py" , line  2314 in  _build_and_run_instance
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]     block_device_info = block_device_info)
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]    File  "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py" , line  2354 in  spawn
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]     block_device_info = block_device_info)
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]    File  "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py" , line  4380 in  _create_domain_and_network
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]     power_on = power_on)
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]    File  "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py" , line  4311 in  _create_domain
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]     LOG.error(err)
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]    File  "/usr/lib/python2.7/site-packages/oslo_utils/excutils.py" , line  85 in  __exit__
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]     six.reraise( self .type_,  self .value,  self .tb)
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]    File  "/usr/lib/python2.7/site-packages/nova/virt/libvirt/driver.py" , line  4301 in  _create_domain
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]     domain.createWithFlags(launch_flags)
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]    File  "/usr/lib/python2.7/site-packages/eventlet/tpool.py" , line  183 in  doit
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]     result  =  proxy_call( self ._autowrap, f,  * args,  * * kwargs)
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]    File  "/usr/lib/python2.7/site-packages/eventlet/tpool.py" , line  141 in  proxy_call
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]     rv  =  execute(f,  * args,  * * kwargs)
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]    File  "/usr/lib/python2.7/site-packages/eventlet/tpool.py" , line  122 in  execute
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]     six.reraise(c, e, tb)
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]    File  "/usr/lib/python2.7/site-packages/eventlet/tpool.py" , line  80 in  tworker
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]     rv  =  meth( * args,  * * kwargs)
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]    File  "/usr/lib64/python2.7/site-packages/libvirt.py" , line  996 in  createWithFlags
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]      if  ret  = =  - 1 raise  libvirtError ( 'virDomainCreateWithFlags() failed' , dom = self )
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3] libvirtError: 内部错误:无法获得对 ACL 技术驱动程序  'ebiptables'  的访问
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3] 
2015 - 08 - 11  01 : 47 : 10.460  82044  INFO nova.compute.manager [req - 210e07f7 - cba3 - 4dec - b2d5 - b7be95c2f559  95a96f0ddcf449239c6682a3c310857e  be27eb2862904a0f9c636c337f66709c  -  -  - ] [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3] Terminating instance
 
问题分析: 在程序中打断点,手动启动云主机:
vim nova / virt / libvirt / driver.py: 4284
  * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
     def  _create_domain( self , xml = None , domain = None ,
                        instance = None , launch_flags = 0 , power_on = True ):
         """Create a domain.
 
         Either domain or xml must be passed in. If both are passed, then
         the domain definition is overwritten from the xml.
         """
         import  ipdb;ipdb.set_trace()
         err  =  None
         try :
             if  xml:
                 err  =  _LE( 'Error defining a domain with XML: %s' %  xml
                 domain  =  self ._conn.defineXML(xml)
 
             if  power_on:
                 err  =  _LE( 'Error launching a defined domain with XML: %s' ) \
                           %  encodeutils.safe_decode(domain.XMLDesc( 0 ),
                                                     errors = 'ignore' )
                 domain.createWithFlags(launch_flags)
 
             if  not  utils.is_neutron():
                 err  =  _LE( 'Error enabling hairpin mode with XML: %s' ) \
                           %  encodeutils.safe_decode(domain.XMLDesc( 0 ),
                                                     errors = 'ignore' )
                 self ._enable_hairpin(domain.XMLDesc( 0 ))
 
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
关闭nova - compute 服务器,启用ipdb
  service opensack - nova - compute stop
  ipdb  / usr / bin / nova - compute  - - config - file = / etc / nova / nova.conf 
 
获取UUID ,查看ceph vms pool 池,是否创建images
 
[root@athcontroller103210 nova] # rbd ls vms 
175dc9db - 2409 - 4b34 - b6ca - efc0a1788687_disk
175dc9db - 2409 - 4b34 - b6ca - efc0a1788687_disk.local
5220f145 - 73d1 - 4831 - 872f - cfd32b09dd20_disk
5220f145 - 73d1 - 4831 - 872f - cfd32b09dd20_disk.local
847c6dea - a887 - 4fad - 8acd - 36f13bc29b57_disk
847c6dea - a887 - 4fad - 8acd - 36f13bc29b57_disk .local
d0adb3ea - 7c4d - 44d5 - b4d6 - a9a02a3c3468_disk
d0adb3ea - 7c4d - 44d5 - b4d6 - a9a02a3c3468_disk.local
  
  
 
手动启动云主机,进入 / data / nova / instances / UUID /
virsh create libvirt.xml 
 
virsh create  启动报错
 
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3]      if  ret  = =  - 1 raise  libvirtError ( 'virDomainCreateWithFlags() failed' , dom = self )
2015 - 08 - 11  01 : 47 : 10.457  82044  TRACE nova.compute.manager [instance:  4b525800 - 3e00 - 4b48 - a997 - c104f919cde3] libvirtError: 内部错误:无法获得对 ACL 技术驱动程序  'ebiptables'  的访问 
根据错误信息提示,去除nwfilter 
去除instanes nwfilter 问题,写入log,后期解决ebiptables 功能
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
vim nova / virt / libvirt / config.py :  1196  行     
1196          if  self .filtername  is  not  None :
1197              filter  =  etree.Element( "filterref" filter = self .filtername)
1198              for  in  self .filterparams:
1199                  filter .append(etree.Element( "parameter" ,
1200                                              name = p[ 'key' ],
1201                                              value = p[ 'value' ]))
1202              #dev.append(filter)
1203              LOG.info( "Add William: Delete nwfilter rule %s"  % filter )   
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
     
 
重新进入ipdb 模式,创建 3 台太云主机
 
2015 - 08 - 11  03 : 06 : 39.834  122763  INFO nova.scheduler.client.report [ - ] Compute_service record updated  for  ( 'athcontroller103210.sjz.autohome.com.cn' 'athcontroller103210.sjz.autohome.com.cn' )
2015 - 08 - 11  03 : 06 : 39.956  122763  INFO nova.scheduler.client.report [ - ] Compute_service record updated  for  ( 'athcontroller103210.sjz.autohome.com.cn' 'athcontroller103210.sjz.autohome.com.cn' )
2015 - 08 - 11  03 : 06 : 40.064  122763  INFO nova.scheduler.client.report [ - ] Compute_service record updated  for  ( 'athcontroller103210.sjz.autohome.com.cn' 'athcontroller103210.sjz.autohome.com.cn' )
2015 - 08 - 11  03 : 07 : 23.798  122763  INFO nova.virt.libvirt.config [req - 0de0fe4b - 7c21 - 4e4f - 9172 - a63c65c26bd8  -  -  -  -  - ] Add William: Delete nwfilter rule <Element filterref at  0x4eb3fa0 >
2015 - 08 - 11  03 : 07 : 23.814  122763  INFO nova.virt.libvirt.firewall [req - 0de0fe4b - 7c21 - 4e4f - 9172 - a63c65c26bd8  -  -  -  -  - ] [instance:  175dc9db - 2409 - 4b34 - b6ca - efc0a1788687] Called setup_basic_filtering  in  nwfilter
2015 - 08 - 11  03 : 07 : 23.815  122763  INFO nova.virt.libvirt.firewall [req - 0de0fe4b - 7c21 - 4e4f - 9172 - a63c65c26bd8  -  -  -  -  - ] [instance:  175dc9db - 2409 - 4b34 - b6ca - efc0a1788687] Ensuring static filters
/ usr / lib / python2. 7 / site - packages / nova / virt / libvirt / driver.py( 4292 )_create_domain()
    4291          import  ipdb;ipdb.set_trace()
- 4292          err  =  None
    4293          try :
 
ipdb> c
2015 - 08 - 11  03 : 08 : 02.499  122763  INFO nova.compute.resource_tracker [req - 28fc26e5 - 9ded - 4394 - a211 - 494c3cd20d87  -  -  -  -  - ] Auditing locally available compute resources  for  node athcontroller103210.sjz.autohome.com.cn
2015 - 08 - 11  03 : 08 : 02.922  122763  INFO nova.compute.resource_tracker [req - 28fc26e5 - 9ded - 4394 - a211 - 494c3cd20d87  -  -  -  -  - ] Total usable vcpus:  40 , total allocated vcpus:  8
2015 - 08 - 11  03 : 08 : 02.923  122763  INFO nova.compute.resource_tracker [req - 28fc26e5 - 9ded - 4394 - a211 - 494c3cd20d87  -  -  -  -  - ] Final resource view: name