I have setup Openstack (Newton) with Nova (Compute) on Ubuntu 16.04.1 LTS Desktop as a virtual machine guest (Oracle VM VirtualBox ver.5.1.8) on Windows 10 host.
The installation was successful and could access the Horizon after the login process via http://192.168.56.102/dashboard/ (from the Host) as shown in the figure below. (Can also access via http://10.0.2.15/dashboard/ from the Guest but not shown here)
When the VM was shut down and later switched on again, the access to Openstack using Horizon’s dashboard was lost. “Error: Unable to retrieve usage information” is shown on the dashboard (as in the top right position of the figure below). This issue is a hard problem and could be replicated all the time.
The below is the snipped of error shown in /var/log/apache2/horizon_error.log file when Computer->Overview option was selected from the Horizon’s dashboard after the login process (that produces Figure 2 above).
2016-12-02 13:03:08.786092 Recoverable error: HTTPConnectionPool(host='10.0.2.15', port=8774): Max retries exceeded with url: /v2.1/extensions (Caused by NewConnectionError('<requests.packages.urllib3.connection.HTTPConnection object at 0x7f3436ebeb10>: Failed to establish a new connection: [Errno 111] Connection refused',))
2016-12-02 13:03:08.804729 DEBUG:keystoneauth.session:REQ: curl -g -i -X GET http://10.0.2.15/identity/v3/users/9af3721f2d8e41cc9342b9941bafc5d1/projects -H "User-Agent: python-keystoneclient" -H "Accept: application/json" -H "X-Auth-Token: {SHA1}67ffd6b14a7e8891ae7cc8b29a639b81822c7b45"
2016-12-02 13:03:08.935723 DEBUG:keystoneauth.session:RESP: [200] Date: Fri, 02 Dec 2016 13:03:08 GMT Server: Apache/2.4.18 (Ubuntu) Vary: X-Auth-Token x-openstack-request-id: req-fa398fe1-c5c9-4ce6-986b-cec266278f04 Content-Length: 966 Keep-Alive: timeout=5, max=100 Connection: Keep-Alive Content-Type: application/json
2016-12-02 13:03:08.935791 RESP BODY: {"links": {"self": "http://10.0.2.15/identity/v3/users/9af3721f2d8e41cc9342b9941bafc5d1/projects", "previous": null, "next": null}, "projects": [{"is_domain": false, "description": "Bootstrap project for initializing the cloud.", "links": {"self": "http://10.0.2.15/identity/v3/projects/4350966521b54001aeb666b6e55b7deb"}, "enabled": true, "id": "4350966521b54001aeb666b6e55b7deb", "parent_id": "default", "domain_id": "default", "name": "admin"}, {"is_domain": false, "description": "", "links": {"self": "http://10.0.2.15/identity/v3/projects/43dba94541ba4e8e83d5528038346ea9"}, "enabled": true, "id": "43dba94541ba4e8e83d5528038346ea9", "parent_id": "default", "domain_id": "default", "name": "demo"}, {"is_domain": false, "description": "", "links": {"self": "http://10.0.2.15/identity/v3/projects/be3ee2f8cb074628ac24c8be6e4c7160"}, "enabled": true, "id": "be3ee2f8cb074628ac24c8be6e4c7160", "parent_id": "default", "domain_id": "default", "name": "alt_demo"}]}
2016-12-02 13:03:08.935809
2016-12-02 13:03:09.018515 Call to list enabled services failed. This is likely due to a problem communicating with the Neutron endpoint. Load Balancers panel will not be displayed.
2016-12-02 13:03:09.024775 Call to list enabled services failed. This is likely due to a problem communicating with the Neutron endpoint. Firewalls panel will not be displayed.
2016-12-02 13:03:09.029002 Call to list enabled services failed. This is likely due to a problem communicating with the Neutron endpoint. VPN panel will not be displayed.
2016-12-02 13:03:09.035255 Call to list supported extensions failed. This is likely due to a problem communicating with the Nova endpoint. Host Aggregates panel will not be displayed
As the error about indicated problem in communicating with the endpoint. Verified if the endpoint for the services is ok in the VM, unfortunately it is not.
svanan@svanan:~/devstack$ openstack endpoint list
Password:
Missing parameter(s):
Set a username with --os-username, OS_USERNAME, or auth.username
Set an authentication URL, with --os-auth-url, OS_AUTH_URL or auth.auth_url
The below is the original output in the VM when before it was shutdown.
svanan@svanan:~/devstack$ openstack endpoint list
+----------------------------------+-----------+--------------+----------------+---------+-----------+-----------------------------------------+
| ID | Region | Service Name | Service Type | Enabled | Interface | URL |
+----------------------------------+-----------+--------------+----------------+---------+-----------+-----------------------------------------+
| 07480eca2071416c8dec30bafa95ff8f | RegionOne | neutron | network | True | admin | http://10.0.2.15:9696/ |
| 0aa32dc14c194561816b6d1b60d04404 | RegionOne | glance | image | True | admin | http://10.0.2.15:9292 |
| 0e5365a8ab3744c780feb25ecffc2c93 | RegionOne | keystone | identity | True | internal | http://10.0.2.15/identity |
| 15d8608b021f42e2a3b0402889a9064f | RegionOne | neutron | network | True | public | http://10.0.2.15:9696/ |
| 289a99bb83e846b2bfc5eabfb800d86d | RegionOne | glance | image | True | internal | http://10.0.2.15:9292 |
| 41397886443d44b7b6d857fccb34d55d | RegionOne | neutron | network | True | internal | http://10.0.2.15:9696/ |
| 4458eed550ae44479feca25f9bec96ce | RegionOne | nova_legacy | compute_legacy | True | public | http://10.0.2.15:8774/v2/$(project_id)s |
| 51be7c90e6c4437fbb75c3a40857774f | RegionOne | keystone | identity | True | public | http://10.0.2.15/identity |
| 7526c5a7e5084a07b1d1c7affe261ae4 | RegionOne | nova_legacy | compute_legacy | True | admin | http://10.0.2.15:8774/v2/$(project_id)s |
| 7c1e026c1c2d4f0b9dfdfe15860d6c5b | RegionOne | keystone | identity | True | admin | http://10.0.2.15/identity_v2_admin |
| 8641d88381fc499688162e8bcde5456c | RegionOne | cinderv2 | volumev2 | True | internal | http://10.0.2.15:8776/v2/$(project_id)s |
| 8a194a2954a943568ccaf8bce5eb9400 | RegionOne | cinder | volume | True | internal | http://10.0.2.15:8776/v1/$(project_id)s |
| 8c0b4d932da34c5990e43c0877146819 | RegionOne | nova | compute | True | admin | http://10.0.2.15:8774/v2.1 |
| 94e8cae799954ba09abcd3b31b1c926e | RegionOne | nova | compute | True | public | http://10.0.2.15:8774/v2.1 |
| a1527968e91d4df9b98fe09039e80bed | RegionOne | cinderv2 | volumev2 | True | public | http://10.0.2.15:8776/v2/$(project_id)s |
| a3e47c8f82d546c8917a8223b8b81a74 | RegionOne | cinderv3 | volumev3 | True | internal | http://10.0.2.15:8776/v3/$(project_id)s |
| b39a19918016462db99cf1de151fc870 | RegionOne | cinderv3 | volumev3 | True | public | http://10.0.2.15:8776/v3/$(project_id)s |
| d17514aa0eec4752848df9599dd8cd00 | RegionOne | cinder | volume | True | admin | http://10.0.2.15:8776/v1/$(project_id)s |
| db570fe7343b461a95cdd4179e1b28d9 | RegionOne | glance | image | True | public | http://10.0.2.15:9292 |
| eb4aa71c03b0418eadfed00715744b07 | RegionOne | cinderv3 | volumev3 | True | admin | http://10.0.2.15:8776/v3/$(project_id)s |
| ee474205e562411abc1842dc9bccbb5d | RegionOne | nova_legacy | compute_legacy | True | internal | http://10.0.2.15:8774/v2/$(project_id)s |
| f383e0696ee647f79f6fd3a3d7ac5dff | RegionOne | cinder | volume | True | public | http://10.0.2.15:8776/v1/$(project_id)s |
| f4239a1f86144d8dab3ed94237d7c10f | RegionOne | cinderv2 | volumev2 | True | admin | http://10.0.2.15:8776/v2/$(project_id)s |
| fe679fcbc79e4664b7897a381e90dd78 | RegionOne | nova | compute | True | internal | http://10.0.2.15:8774/v2.1 |
+----------------------------------+-----------+--------------+----------------+---------+-----------+-----------------------------------------+
Verified ifconfig at VM, it has the enp0s3, enp0s8, lo, virbr0 interfaces configured except the br-ex which was there before the shutdown as below:
br-ex Link encap:Ethernet HWaddr 1a:ba:3a:4a:d5:4f
inet addr:172.24.4.1 Bcast:0.0.0.0 Mask:255.255.255.0
inet6 addr: fe80::18ba:3aff:fe4a:d54f/64 Scope:Link
inet6 addr: 2001:db8::2/64 Scope:Global
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:15 errors:0 dropped:0 overruns:0 frame:0
TX packets:79 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:776 (776.0 B) TX bytes:10328 (10.3 KB)
Not sure if the absense of this bridge (br-ex) could be the cause Horizon (from the Host) could not connect with the services in the Guest. Tried to configure statically in etc/network/interfaces as below to get the same above br-ex output.
auto br-ex
iface br-ex inet static
hwaddress 1a:ba:3a:4a:d5:4f
address 172.24.4.1
broadcast 0.0.0.0
netmask 255.255.255.0
iface br-ex inet6 static
pre-up modprobe ipv6
address 2001:db8::2
netmask 64
The br-ex output I got after
svanan@svanan:~$ sudo systemctl restart networking
svanan@svanan:~$ ifconfig
br-ex Link encap:Ethernet HWaddr 7a:07:1d:1d:1a:40
inet addr:172.24.4.1 Bcast:0.0.0.0 Mask:255.255.255.0
inet6 addr: fe80::18ba:3aff:fe4a:d54f/64 Scope:Link
inet6 addr: 2001:db8::2/64 Scope:Global
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:126 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1
RX bytes:0 (0.0 B) TX bytes:19334 (19.3 KB)
As can be seen above I am unable to configure HWaddr to 1a:ba:3a:4a:d5:4f instead 7a:07:1d:1d:1a:40 showing up.
Questions:
- Could the absense of br-ex after the VM shutdown and switched on sequence could be the reason Horizon’s failure to retrieve the information from its services.
- How to statically set the HWaddr to 1a:ba:3a:4a:d5:4f for br-ex? The syntax hwaddress 1a:ba:3a:4a:d5:4f does not seem to work.
- If you faces similar problem, can you share how did you solve it.
Is it true, one need to run ./unstack.sh and ./stack.sh ( ./rejoin-stack.sh has been depreciated in Newton) everytime a VM is restarted as mentioned in No rejoin-stack.sh script in my setup .This approach takes very long time as shown below,
=========================
DevStack Component Timing
=========================
Total runtime 1777
Hope you can assist to guide me in this. Thanks.
Saravanan Kandasamy
