报错信息
[root@controller ~]# openstack-service status | grep failed MainPID=0 Id=neutron-server.service ActiveState=failed [root@controller ~]# systemctl restart neutron-server.service Job for neutron-server.service failed because a timeout was exceeded. See "systemctl status neutron-server.service" and "journalctl -xe" for details.
查看网络服务状态及报错信息查询位置
[root@controller ~]# systemctl status neutron-server.service ● neutron-server.service - OpenStack Neutron Server Loaded: loaded (/usr/lib/systemd/system/neutron-server.service; enabled; vendor preset: disabled) Active: failed (Result: timeout) since Tue 2020-10-27 19:41:09 EDT; 11s ago Process: 3691 ExecStart=/usr/bin/neutron-server --config-file /usr/share/neutron/neutron-dist.conf --config-dir /usr/share/neutron/server --config-file /etc/neutron/neutron.conf --config-file /etc/neutron/plugin.ini --config-dir /etc/neutron/conf.d/common --config-dir /etc/neutron/conf.d/neutron-server --log-file /var/log/neutron/server.log (code=killed, signal=TERM) Main PID: 3691 (code=killed, signal=TERM) Oct 27 19:39:39 controller systemd[1]: Starting OpenStack Neutron Server... Oct 27 19:39:39 controller neutron-server[3691]: Guru mediation now registers SIGUSR1 and SIGUSR2 by default for backward comp...ports. Oct 27 19:39:40 controller neutron-server[3691]: Option "verbose" from group "DEFAULT" is deprecated for removal. Its value m...uture. Oct 27 19:39:40 controller neutron-server[3691]: Option "notification_driver" from group "DEFAULT" is deprecated. Use option "...ions". Oct 27 19:41:09 controller systemd[1]: neutron-server.service start operation timed out. Terminating. Oct 27 19:41:09 controller systemd[1]: Failed to start OpenStack Neutron Server. Oct 27 19:41:09 controller systemd[1]: Unit neutron-server.service entered failed state. Oct 27 19:41:09 controller systemd[1]: neutron-server.service failed. Hint: Some lines were ellipsized, use -l to show in full.
[root@controller ~]# journalctl -xe Oct 27 19:40:42 controller cinder-api[3093]: 2020-10-27 19:40:42.657 3473 INFO cinder.volume.api [req-a9b66df5-7289-4b8f-9873-6131c60d2 Oct 27 19:40:42 controller cinder-api[3093]: 2020-10-27 19:40:42.658 3473 INFO cinder.api.openstack.wsgi [req-a9b66df5-7289-4b8f-9873-6 Oct 27 19:40:42 controller cinder-api[3093]: 2020-10-27 19:40:42.659 3473 INFO eventlet.wsgi.server [req-a9b66df5-7289-4b8f-9873-6131c6 Oct 27 19:40:42 controller cinder-api[3093]: 2020-10-27 19:40:42.781 3473 INFO cinder.api.openstack.wsgi [req-482fcd46-4e9e-459b-9f3d-6 Oct 27 19:40:42 controller cinder-api[3093]: 2020-10-27 19:40:42.809 3473 WARNING cinder.quota [req-482fcd46-4e9e-459b-9f3d-6e300aa1527 Oct 27 19:40:42 controller cinder-api[3093]: 2020-10-27 19:40:42.810 3473 WARNING cinder.quota [req-482fcd46-4e9e-459b-9f3d-6e300aa1527 Oct 27 19:40:42 controller cinder-api[3093]: 2020-10-27 19:40:42.835 3473 INFO cinder.api.openstack.wsgi [req-482fcd46-4e9e-459b-9f3d-6 Oct 27 19:40:42 controller cinder-api[3093]: 2020-10-27 19:40:42.837 3473 INFO eventlet.wsgi.server [req-482fcd46-4e9e-459b-9f3d-6e300a Oct 27 19:41:00 controller glance-registry[1250]: /usr/lib/python2.7/site-packages/oslo_db/sqlalchemy/enginefacade.py:1056: OsloDBDepre Oct 27 19:41:00 controller glance-registry[1250]: expire_on_commit=expire_on_commit, _conf=conf) Oct 27 19:41:02 controller cinder-scheduler[3103]: 2020-10-27 19:41:02.761 3103 ERROR oslo.messaging._drivers.impl_rabbit [req-8d112d4f Oct 27 19:41:09 controller systemd[1]: neutron-server.service start operation timed out. Terminating. Oct 27 19:41:09 controller systemd[1]: Failed to start OpenStack Neutron Server. -- Subject: Unit neutron-server.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit neutron-server.service has failed. -- -- The result is failed. Oct 27 19:41:09 controller systemd[1]: Unit neutron-server.service entered failed state. Oct 27 19:41:09 controller systemd[1]: neutron-server.service failed. Oct 27 19:41:09 controller polkitd[3134]: Unregistered Authentication Agent for unix-process:3686:304822 (system bus name :1.12, object Oct 27 19:41:28 controller cinder-scheduler[3103]: 2020-10-27 19:41:28.812 3103 ERROR oslo.messaging._drivers.impl_rabbit [req-8d112d4f
查看报错日志信息
[root@controller ~]# cat /var/log/neutron/server.log | grep -i error 2020-10-27 19:39:41.394 3691 ERROR oslo.messaging._drivers.impl_rabbit [req-096f3c0d-1e1e-459f-946f-adca961a0364 - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds. 2020-10-27 19:39:42.399 3691 ERROR oslo.messaging._drivers.impl_rabbit [req-096f3c0d-1e1e-459f-946f-adca961a0364 - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 2 seconds. 2020-10-27 19:39:44.406 3691 ERROR oslo.messaging._drivers.impl_rabbit [req-096f3c0d-1e1e-459f-946f-adca961a0364 - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 4 seconds. 2020-10-27 19:39:48.415 3691 ERROR oslo.messaging._drivers.impl_rabbit [req-096f3c0d-1e1e-459f-946f-adca961a0364 - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 6 seconds. 2020-10-27 19:39:54.428 3691 ERROR oslo.messaging._drivers.impl_rabbit [req-096f3c0d-1e1e-459f-946f-adca961a0364 - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 8 seconds. 2020-10-27 19:40:02.450 3691 ERROR oslo.messaging._drivers.impl_rabbit [req-096f3c0d-1e1e-459f-946f-adca961a0364 - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 10 seconds. 2020-10-27 19:40:12.476 3691 ERROR oslo.messaging._drivers.impl_rabbit [req-096f3c0d-1e1e-459f-946f-adca961a0364 - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 12 seconds. 2020-10-27 19:40:24.513 3691 ERROR oslo.messaging._drivers.impl_rabbit [req-096f3c0d-1e1e-459f-946f-adca961a0364 - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 14 seconds. 2020-10-27 19:40:38.543 3691 ERROR oslo.messaging._drivers.impl_rabbit [req-096f3c0d-1e1e-459f-946f-adca961a0364 - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 16 seconds. 2020-10-27 19:40:54.573 3691 ERROR oslo.messaging._drivers.impl_rabbit [req-096f3c0d-1e1e-459f-946f-adca961a0364 - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 18 seconds. 2020-10-27 19:46:47.542 4105 ERROR oslo.messaging._drivers.impl_rabbit [req-8d29ab36-53cf-4801-b76c-fb49e728ad9d - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds. 2020-10-27 19:46:48.549 4105 ERROR oslo.messaging._drivers.impl_rabbit [req-8d29ab36-53cf-4801-b76c-fb49e728ad9d - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 2 seconds. 2020-10-27 19:46:50.569 4105 ERROR oslo.messaging._drivers.impl_rabbit [req-8d29ab36-53cf-4801-b76c-fb49e728ad9d - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 4 seconds. 2020-10-27 19:46:54.580 4105 ERROR oslo.messaging._drivers.impl_rabbit [req-8d29ab36-53cf-4801-b76c-fb49e728ad9d - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 6 seconds. 2020-10-27 19:47:00.594 4105 ERROR oslo.messaging._drivers.impl_rabbit [req-8d29ab36-53cf-4801-b76c-fb49e728ad9d - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 8 seconds. 2020-10-27 19:47:08.615 4105 ERROR oslo.messaging._drivers.impl_rabbit [req-8d29ab36-53cf-4801-b76c-fb49e728ad9d - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 10 seconds. 2020-10-27 19:47:18.634 4105 ERROR oslo.messaging._drivers.impl_rabbit [req-8d29ab36-53cf-4801-b76c-fb49e728ad9d - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 12 seconds. 2020-10-27 19:47:30.666 4105 ERROR oslo.messaging._drivers.impl_rabbit [req-8d29ab36-53cf-4801-b76c-fb49e728ad9d - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 14 seconds. 2020-10-27 19:47:44.697 4105 ERROR oslo.messaging._drivers.impl_rabbit [req-8d29ab36-53cf-4801-b76c-fb49e728ad9d - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 16 seconds. 2020-10-27 19:48:00.741 4105 ERROR oslo.messaging._drivers.impl_rabbit [req-8d29ab36-53cf-4801-b76c-fb49e728ad9d - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 18 seconds. 2020-10-27 19:48:36.754 4145 ERROR oslo.messaging._drivers.impl_rabbit [req-1ec56f6b-41a2-4800-9c14-a7bf0d5cb5fc - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 1 seconds. 2020-10-27 19:48:37.761 4145 ERROR oslo.messaging._drivers.impl_rabbit [req-1ec56f6b-41a2-4800-9c14-a7bf0d5cb5fc - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 2 seconds. 2020-10-27 19:48:39.769 4145 ERROR oslo.messaging._drivers.impl_rabbit [req-1ec56f6b-41a2-4800-9c14-a7bf0d5cb5fc - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 4 seconds. 2020-10-27 19:48:43.779 4145 ERROR oslo.messaging._drivers.impl_rabbit [req-1ec56f6b-41a2-4800-9c14-a7bf0d5cb5fc - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 6 seconds. 2020-10-27 19:48:49.791 4145 ERROR oslo.messaging._drivers.impl_rabbit [req-1ec56f6b-41a2-4800-9c14-a7bf0d5cb5fc - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 8 seconds. 2020-10-27 19:48:57.809 4145 ERROR oslo.messaging._drivers.impl_rabbit [req-1ec56f6b-41a2-4800-9c14-a7bf0d5cb5fc - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 10 seconds. 2020-10-27 19:49:07.837 4145 ERROR oslo.messaging._drivers.impl_rabbit [req-1ec56f6b-41a2-4800-9c14-a7bf0d5cb5fc - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 12 seconds. 2020-10-27 19:49:19.863 4145 ERROR oslo.messaging._drivers.impl_rabbit [req-1ec56f6b-41a2-4800-9c14-a7bf0d5cb5fc - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 14 seconds. 2020-10-27 19:49:33.893 4145 ERROR oslo.messaging._drivers.impl_rabbit [req-1ec56f6b-41a2-4800-9c14-a7bf0d5cb5fc - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 16 seconds. 2020-10-27 19:49:49.942 4145 ERROR oslo.messaging._drivers.impl_rabbit [req-1ec56f6b-41a2-4800-9c14-a7bf0d5cb5fc - - - - -] AMQP server on controller:5672 is unreachable: [Errno 111] ECONNREFUSED. Trying again in 18 seconds.
查看rabbitmq-server.service服务状态
[root@controller ~]# systemctl status rabbitmq-server.service ● rabbitmq-server.service - RabbitMQ broker Loaded: loaded (/usr/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset: disabled) Active: failed (Result: exit-code) since Tue 2020-10-27 18:49:52 EDT; 1h 13min ago Process: 1782 ExecStop=/usr/lib/rabbitmq/bin/rabbitmqctl stop (code=exited, status=0/SUCCESS) Process: 1235 ExecStart=/usr/lib/rabbitmq/bin/rabbitmq-server (code=exited, status=1/FAILURE) Main PID: 1235 (code=exited, status=1/FAILURE) Oct 27 18:49:52 controller rabbitmqctl[1782]: attempted to contact: [rabbit@controller] Oct 27 18:49:52 controller rabbitmqctl[1782]: rabbit@controller: Oct 27 18:49:52 controller rabbitmqctl[1782]: * unable to connect to epmd (port 4369) on controller: address (cannot connect t.../port) Oct 27 18:49:52 controller rabbitmqctl[1782]: current node details: Oct 27 18:49:52 controller rabbitmqctl[1782]: - node name: 'rabbitmq-cli-53@controller' Oct 27 18:49:52 controller rabbitmqctl[1782]: - home dir: /var/lib/rabbitmq Oct 27 18:49:52 controller rabbitmqctl[1782]: - cookie hash: SLEbktPcRQQidx2ktup2gw== Oct 27 18:49:52 controller systemd[1]: Failed to start RabbitMQ broker. Oct 27 18:49:52 controller systemd[1]: Unit rabbitmq-server.service entered failed state. Oct 27 18:49:52 controller systemd[1]: rabbitmq-server.service failed. Hint: Some lines were ellipsized, use -l to show in full.
重启并查看rabbitmq-server服务
[root@controller ~]# systemctl restart rabbitmq-server.service [root@controller ~]# systemctl status rabbitmq-server.service ● rabbitmq-server.service - RabbitMQ broker Loaded: loaded (/usr/lib/systemd/system/rabbitmq-server.service; enabled; vendor preset: disabled) Active: active (running) since Tue 2020-10-27 20:03:14 EDT; 2s ago Process: 1782 ExecStop=/usr/lib/rabbitmq/bin/rabbitmqctl stop (code=exited, status=0/SUCCESS) Main PID: 4368 (beam.smp) Status: "Initialized" CGroup: /system.slice/rabbitmq-server.service ├─4368 /usr/lib64/erlang/erts-7.3.1.2/bin/beam.smp -W w -A 64 -P 1048576 -t 5000000 -stbt db -K true -- -root /usr/lib64/... ├─4588 inet_gethost 4 └─4589 inet_gethost 4 Oct 27 20:03:11 controller systemd[1]: Starting RabbitMQ broker... Oct 27 20:03:13 controller rabbitmq-server[4368]: RabbitMQ 3.6.5. Copyright (C) 2007-2016 Pivotal Software, Inc. Oct 27 20:03:13 controller rabbitmq-server[4368]: ## ## Licensed under the MPL. See http://www.rabbitmq.com/ Oct 27 20:03:13 controller rabbitmq-server[4368]: ## ## Oct 27 20:03:13 controller rabbitmq-server[4368]: ########## Logs: /var/log/rabbitmq/rabbit@controller.log Oct 27 20:03:13 controller rabbitmq-server[4368]: ###### ## /var/log/rabbitmq/rabbit@controller-sasl.log Oct 27 20:03:13 controller rabbitmq-server[4368]: ########## Oct 27 20:03:13 controller rabbitmq-server[4368]: Starting broker... Oct 27 20:03:14 controller systemd[1]: Started RabbitMQ broker. Oct 27 20:03:14 controller rabbitmq-server[4368]: completed with 0 plugins.
重启并查看网络服务
[root@controller ~]# systemctl restart neutron-server [root@controller ~]# systemctl status neutron-server ● neutron-server.service - OpenStack Neutron Server Loaded: loaded (/usr/lib/systemd/system/neutron-server.service; enabled; vendor preset: disabled) Active: active (running) since Tue 2020-10-27 20:03:31 EDT; 9s ago Main PID: 4607 (neutron-server) CGroup: /system.slice/neutron-server.service ├─4607 /usr/bin/python2 /usr/bin/neutron-server --config-file /usr/share/neutron/neutron-dist.conf --config-dir /usr/shar... ├─4618 /usr/bin/python2 /usr/bin/neutron-server --config-file /usr/share/neutron/neutron-dist.conf --config-dir /usr/shar... ├─4619 /usr/bin/python2 /usr/bin/neutron-server --config-file /usr/share/neutron/neutron-dist.conf --config-dir /usr/shar... ├─4620 /usr/bin/python2 /usr/bin/neutron-server --config-file /usr/share/neutron/neutron-dist.conf --config-dir /usr/shar... ├─4621 /usr/bin/python2 /usr/bin/neutron-server --config-file /usr/share/neutron/neutron-dist.conf --config-dir /usr/shar... ├─4622 /usr/bin/python2 /usr/bin/neutron-server --config-file /usr/share/neutron/neutron-dist.conf --config-dir /usr/shar... └─4623 /usr/bin/python2 /usr/bin/neutron-server --config-file /usr/share/neutron/neutron-dist.conf --config-dir /usr/shar... Oct 27 20:03:28 controller systemd[1]: Starting OpenStack Neutron Server... Oct 27 20:03:29 controller neutron-server[4607]: Guru mediation now registers SIGUSR1 and SIGUSR2 by default for backward comp...ports. Oct 27 20:03:29 controller neutron-server[4607]: Option "verbose" from group "DEFAULT" is deprecated for removal. Its value m...uture. Oct 27 20:03:29 controller neutron-server[4607]: Option "notification_driver" from group "DEFAULT" is deprecated. Use option "...ions". Oct 27 20:03:31 controller systemd[1]: Started OpenStack Neutron Server. Hint: Some lines were ellipsized, use -l to show in full.
至此，排错完成！
