Specify IP for VR in shared networks #4503

Pearl1594 · 2020-11-25T08:24:46Z

Description

This PR enables admins to specify IP for a VR in a shared network

Types of changes

Breaking change (fix or feature that would cause existing functionality to change)
New feature (non-breaking change which adds functionality)
Bug fix (non-breaking change which fixes an issue)
Enhancement (improves an existing feature and functionality)
Cleanup (Code refactoring and cleanup, that may add test cases)

Feature/Enhancement Scale or Bug Severity

Feature/Enhancement Scale

Major
Minor

Bug Severity

Screenshots (if appropriate):

How Has This Been Tested?

Pearl1594 · 2020-11-25T08:28:48Z

@blueorangutan package

blueorangutan · 2020-11-25T08:29:26Z

@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-11-25T08:42:06Z

Packaging result: ✖centos7 ✖centos8 ✖debian. JID-2431

Pearl1594 · 2020-11-25T09:20:23Z

@blueorangutan package

blueorangutan · 2020-11-25T09:21:21Z

@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-11-25T09:54:03Z

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2432

Pearl1594 · 2020-11-27T05:54:28Z

@blueorangutan package

blueorangutan · 2020-11-27T05:55:32Z

@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-11-27T06:28:47Z

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2439

Pearl1594 · 2020-11-27T06:29:18Z

@blueorangutan test

blueorangutan · 2020-11-27T06:30:29Z

@Pearl1594 a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

api/src/main/java/org/apache/cloudstack/api/command/admin/network/CreateNetworkCmdByAdmin.java

sureshanaparti · 2020-11-27T10:48:36Z

...hestration/src/main/java/org/apache/cloudstack/engine/orchestration/NetworkOrchestrator.java


                        if (network.getPvlanType() != null) {
-                            NetworkDetailVO detailVO = new NetworkDetailVO(networkPersisted.getId(), ApiConstants.ISOLATED_PVLAN_TYPE, network.getPvlanType().toString(), true);
+                                NetworkDetailVO detailVO = new NetworkDetailVO(networkPersisted.getId(), ApiConstants.ISOLATED_PVLAN_TYPE, network.getPvlanType().toString(), true);


extra tab space, may be not required.

...hestration/src/main/java/org/apache/cloudstack/engine/orchestration/NetworkOrchestrator.java

server/src/main/java/com/cloud/vm/UserVmManagerImpl.java

sureshanaparti · 2020-11-27T13:49:12Z

server/src/main/java/com/cloud/network/NetworkServiceImpl.java

+            if (!NetUtils.isValidIp6(routerIpv6)) {
+                throw new CloudRuntimeException("Router IPv6 IP provided is of incorrect format");
+            }
+            if (!NetUtils.isIp6InRange(routerIp, ipv6Range)) {


routerIp => routerIpv6

server/src/main/java/com/cloud/network/NetworkServiceImpl.java

blueorangutan · 2020-11-27T16:46:16Z

Trillian test result (tid-3272)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 34888 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4503-t3272-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_password_server.py
Intermittent failure detected: /marvin/tests/smoke/test_router_dhcphosts.py
Intermittent failure detected: /marvin/tests/smoke/test_routers_network_ops.py
Smoke tests completed. 84 look OK, 2 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_03_deploy_and_upgrade_kubernetes_cluster	`Failure`	214.96	test_kubernetes_clusters.py
test_01_RVR_Network_FW_PF_SSH_default_routes_egress_true	`Failure`	80.30	test_routers_network_ops.py

Pearl1594 · 2020-11-30T05:21:20Z

@blueorangutan package

blueorangutan · 2020-11-30T05:22:42Z

@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2020-11-30T06:17:17Z

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2443

Pearl1594 · 2020-11-30T10:31:36Z

@blueorangutan test

blueorangutan · 2020-11-30T10:32:43Z

@Pearl1594 a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2020-11-30T20:38:14Z

Trillian test result (tid-3287)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 34337 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4503-t3287-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_kubernetes_clusters.py
Intermittent failure detected: /marvin/tests/smoke/test_hostha_kvm.py
Smoke tests completed. 85 look OK, 1 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File
test_03_deploy_and_upgrade_kubernetes_cluster	`Failure`	264.94	test_kubernetes_clusters.py

server/src/main/java/com/cloud/network/NetworkServiceImpl.java

…arvin

Pearl1594 · 2021-02-16T11:54:06Z

@blueorangutan package

Pearl1594 · 2021-02-16T12:28:25Z

@blueorangutan package

blueorangutan · 2021-02-16T12:29:37Z

@Pearl1594 a Jenkins job has been kicked to build packages. I'll keep you posted as I make progress.

blueorangutan · 2021-02-16T13:06:48Z

Packaging result: ✔centos7 ✔centos8 ✔debian. JID-2715

Pearl1594 · 2021-02-16T13:42:06Z

@blueorangutan test

blueorangutan · 2021-02-16T13:42:38Z

@Pearl1594 a Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests

blueorangutan · 2021-02-16T23:40:09Z

Trillian test result (tid-3553)
Environment: kvm-centos7 (x2), Advanced Networking with Mgmt server 7
Total time taken: 33853 seconds
Marvin logs: https://github.com/blueorangutan/acs-prs/releases/download/trillian/pr4503-t3553-kvm-centos7.zip
Intermittent failure detected: /marvin/tests/smoke/test_templates.py
Smoke tests completed. 86 look OK, 0 have error(s)
Only failed tests results shown below:

Test	Result	Time (s)	Test File

rohityadavcloud · 2021-02-18T08:23:50Z

Tests LGTM both marvin tests and from @vladimirpetrov - I'll merge this.

weizhouapache · 2021-02-19T08:19:24Z

@rhtyd @Pearl1594 @DaanHoogland
it looks recent travis test failed due to this pr.

see
https://travis-ci.com/github/apache/cloudstack/jobs/484387022
https://travis-ci.com/github/apache/cloudstack/jobs/484391002

error logs below

Currently running test: smoke/test_network
/home/travis/.local/lib/python2.7/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
  from cryptography.hazmat.backends import default_backend
Port Forwarding Rule is deleted
NATRule is deleted
Router is deleted
Deleting Public IP : c582d08b-5207-4e9f-8281-8edce8c76d71
Port Forwarding Rule is deleted
Port Forwarding Rule is deleted
====Trying SSH Connection: Host:192.168.2.6 User:root                                   Port:22 RetryCnt:2===
SshClient: Exception under createConnection: ['Traceback (most recent call last):\n', '  File "/home/travis/.local/lib/python2.7/site-packages/marvin/sshClient.py", line 122, in createConnection\n    allow_agent=False)\n', '  File "/home/travis/.local/lib/python2.7/site-packages/paramiko/client.py", line 349, in connect\n    retry_on_signal(lambda: sock.connect(addr))\n', '  File "/home/travis/.local/lib/python2.7/site-packages/paramiko/util.py", line 283, in retry_on_signal\n    return function()\n', '  File "/home/travis/.local/lib/python2.7/site-packages/paramiko/client.py", line 349, in <lambda>\n    retry_on_signal(lambda: sock.connect(addr))\n', '  File "/usr/lib/python2.7/socket.py", line 228, in meth\n    return getattr(self._sock,name)(*args)\n', 'timeout: timed out\n']
Traceback (most recent call last):
  File "/home/travis/.local/lib/python2.7/site-packages/marvin/sshClient.py", line 122, in createConnection
    allow_agent=False)
  File "/home/travis/.local/lib/python2.7/site-packages/paramiko/client.py", line 349, in connect
    retry_on_signal(lambda: sock.connect(addr))
  File "/home/travis/.local/lib/python2.7/site-packages/paramiko/util.py", line 283, in retry_on_signal
    return function()
  File "/home/travis/.local/lib/python2.7/site-packages/paramiko/client.py", line 349, in <lambda>
    retry_on_signal(lambda: sock.connect(addr))
  File "/usr/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
timeout: timed out
====Trying SSH Connection: Host:192.168.2.6 User:root                                   Port:22 RetryCnt:1===
SshClient: Exception under createConnection: ['Traceback (most recent call last):\n', '  File "/home/travis/.local/lib/python2.7/site-packages/marvin/sshClient.py", line 122, in createConnection\n    allow_agent=False)\n', '  File "/home/travis/.local/lib/python2.7/site-packages/paramiko/client.py", line 349, in connect\n    retry_on_signal(lambda: sock.connect(addr))\n', '  File "/home/travis/.local/lib/python2.7/site-packages/paramiko/util.py", line 283, in retry_on_signal\n    return function()\n', '  File "/home/travis/.local/lib/python2.7/site-packages/paramiko/client.py", line 349, in <lambda>\n    retry_on_signal(lambda: sock.connect(addr))\n', '  File "/usr/lib/python2.7/socket.py", line 228, in meth\n    return getattr(self._sock,name)(*args)\n', 'timeout: timed out\n']
Traceback (most recent call last):
  File "/home/travis/.local/lib/python2.7/site-packages/marvin/sshClient.py", line 122, in createConnection
    allow_agent=False)
  File "/home/travis/.local/lib/python2.7/site-packages/paramiko/client.py", line 349, in connect
    retry_on_signal(lambda: sock.connect(addr))
  File "/home/travis/.local/lib/python2.7/site-packages/paramiko/util.py", line 283, in retry_on_signal
    return function()
  File "/home/travis/.local/lib/python2.7/site-packages/paramiko/client.py", line 349, in <lambda>
    retry_on_signal(lambda: sock.connect(addr))
  File "/usr/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
timeout: timed out
====Trying SSH Connection: Host:192.168.2.6 User:root                                   Port:22 RetryCnt:0===
Executing command '{'endip': '172.16.179.21', 'acltype': 'domain', 'routerip': '172.16.179.15', 'name': 'MySharedNetwork - Test', 'networkofferingid': u'f160b481-ef28-4afd-8999-a94b180a74b3', 'physicalnetworkid': u'3a52e342-1cd1-4d5e-b2f4-d9a2efa253a6', 'startip': '172.16.179.2', 'vlan': 212, 'netmask': '255.255.255.0', 'displaytext': 'MySharedNetwork', 'scope': 'all', 'gateway': '172.16.179.1'}'

and

Printing marvin logs /tmp//MarvinLogs/test_network_AUK59S/failed_plus_exceptions.txt :
2021-02-18 23:08:13,330 - CRITICAL - EXCEPTION: test_01_deployVMInSharedNetwork: ['Traceback (most recent call last):\n', '  File "/usr/lib/python2.7/unittest/case.py", line 329, in run\n    testMethod()\n', '  File "/home/travis/build/apache/cloudstack/test/integration/smoke/test_network.py", line 1988, in test_01_deployVMInSharedNetwork\n    host = self.get_router_host(router)\n', '  File "/home/travis/build/apache/cloudstack/test/integration/smoke/test_network.py", line 1944, in get_router_host\n    host.user, host.password = get_host_credentials(self.config, host.ipaddress)\n', '  File "/home/travis/.local/lib/python2.7/site-packages/marvin/lib/utils.py", line 231, in get_host_credentials\n    raise KeyError("Please provide the marvin configuration file with credentials to your hosts")\n', "KeyError: 'Please provide the marvin configuration file with credentials to your hosts'\n"]
2021-02-18 23:08:18,539 - CRITICAL - EXCEPTION: test_02_verifyRouterIpAfterNetworkRestart: ['Traceback (most recent call last):\n', '  File "/usr/lib/python2.7/unittest/case.py", line 329, in run\n    testMethod()\n', '  File "/home/travis/build/apache/cloudstack/test/integration/smoke/test_network.py", line 2016, in test_02_verifyRouterIpAfterNetworkRestart\n    host = self.get_router_host(router)\n', '  File "/home/travis/build/apache/cloudstack/test/integration/smoke/test_network.py", line 1944, in get_router_host\n    host.user, host.password = get_host_credentials(self.config, host.ipaddress)\n', '  File "/home/travis/.local/lib/python2.7/site-packages/marvin/lib/utils.py", line 231, in get_host_credentials\n    raise KeyError("Please provide the marvin configuration file with credentials to your hosts")\n', "KeyError: 'Please provide the marvin configuration file with credentials to your hosts'\n"]
2021-02-18 23:08:19,602 - CRITICAL - FAILED: test_03_destroySharedNetwork: ['Traceback (most recent call last):\n', '  File "/usr/lib/python2.7/unittest/case.py", line 329, in run\n    testMethod()\n', '  File "/home/travis/build/apache/cloudstack/test/integration/smoke/test_network.py", line 2025, in test_03_destroySharedNetwork\n    self.fail("Failed to destroy the shared network")\n', '  File "/usr/lib/python2.7/unittest/case.py", line 410, in fail\n    raise self.failureException(msg)\n', 'AssertionError: Failed to destroy the shared network\n']
2021-02-18 23:08:19,638 - CRITICAL - EXCEPTION: test_03_destroySharedNetwork: ['Traceback (most recent call last):\n', '  File "/home/travis/.local/lib/python2.7/site-packages/nose/suite.py", line 228, in run\n    self.tearDown()\n', '  File "/home/travis/.local/lib/python2.7/site-packages/nose/suite.py", line 351, in tearDown\n    self.teardownContext(ancestor)\n', '  File "/home/travis/.local/lib/python2.7/site-packages/nose/suite.py", line 367, in teardownContext\n    try_run(context, names)\n', '  File "/home/travis/.local/lib/python2.7/site-packages/nose/util.py", line 471, in try_run\n    return func()\n', '  File "/home/travis/build/apache/cloudstack/test/integration/smoke/test_network.py", line 1902, in tearDownClass\n    raise Exception("Warning: Exception during cleanup : %s" % e)\n', "Exception: Warning: Exception during cleanup : Execute cmd: deletenetworkoffering failed, due to: errorCode: 431, errorText:Can't delete network offering 22 as its used by 1 networks. To make the network offering unavaiable, disable it\n"]

Pearl1594 · 2021-02-19T08:21:04Z

Yes @weizhouapache - I'm working on fixing it. Thanks!

weizhouapache · 2021-02-19T08:40:44Z

Yes @weizhouapache - I'm working on fixing it. Thanks!

@Pearl1594 good.

Pearl1594 added this to the 4.16.0.0 milestone Nov 25, 2020

Pearl1594 mentioned this pull request Nov 25, 2020

Support for specifying VR IP in shared networks apache/cloudstack-primate#872

Draft

Pearl1594 force-pushed the vr-ip-shared-net branch from 4fbf893 to cd2e9db Compare November 25, 2020 08:57

Pearl1594 force-pushed the vr-ip-shared-net branch from cd2e9db to e987ec3 Compare November 27, 2020 05:49