Tweak gate hooks scripts to handle both functional and api jobs
With this change both the API and Functional job will benefit
from the use of gate hooks. This is especially useful to make
the API job emit the test results the same way other jobs do.
NOTE: Changes are applied to both contrib directories under
neutron/tests and neutron/tests/functional; these two directories
are supposed to be in sync until change [1] merges, after which
we can drop neutron/tests/functional/contrib, in favor of
neutron/tests/contrib. This was done because of the symlink issue
caused by [2].
Sam Betts [Mon, 20 Oct 2014 12:26:33 +0000 (13:26 +0100)]
Ensure test_metaplugin handles random hashseeds
2 tests fail in test_metaplugin when using hashseed 2701526934 this is
down to the nature of using dictionaries and sets in Python having
unpredictable ordering when retrieving data stored in them. This patch
ensures that no matter the order fake1 and fake2 get placed into
self.plugins that the test can assert both possible scenarios.
Kevin Benton [Sun, 4 Jan 2015 09:47:01 +0000 (01:47 -0800)]
Ignore non-existent ports during OVS intf list
A recent commit[1] to pass the list of port names directly to
ovs-vsctl during a list operation introduced a new possible
failure condition where one of the names might refer to a port
which no longer exists. By default this causes ovs-vsctl to quit
in a fit of rage[2].
Previously, all interfaces were retrieved and the ones that were a
subset of the name list were processed. The name list could contain
extra non-existent names (e.g. recently deleted interfaces).
This patch just passes the '--if-exists' flag to the 'list Interface'
command to match the same previous behavior.
Angus Lees [Tue, 23 Dec 2014 00:13:27 +0000 (11:13 +1100)]
Enable the "not-callable" pylint check
This check catches attempts to call variables that pylint believes are
not functions. A trivial example would be:
# Trivial example caught by this check:
foo = dict()
print foo('bar') # <- oops, meant foo['bar']
This change enables the "not-callable" pylint check, after disabling a
few cases where the alert triggers but the usage was intended (defining
decorators).
Angus Lees [Mon, 22 Dec 2014 04:41:28 +0000 (15:41 +1100)]
ovs_dvr: Use lazy logging interpolation
There are a small number of examples of "eager" interpolation in
neutron:
logging.debug("foo %s" % arg)
These should be converted to perform the interpolation lazily within
the logging function, since if the severity is below the logging level
then the interpolation can be skipped entirely.
This change addresses all such examples found in ovs agent via a pylint
test. Other occurrences are addressed elsewhere.
Maru Newby [Tue, 28 Oct 2014 19:50:31 +0000 (19:50 +0000)]
Add a constant for router interface device owners
In the absense of a port object that includes a check for whether a
given port is implementing a router interface, this change adds the
ROUTER_INTERFACE_OWNERS tuple containing the relevant DEVICE_OWNER_*
constants.
This change was suggested by https://review.openstack.org/#/c/129865/
Stale VXLAN & GRE tunnel endpoint deletion from DB
Description:
Stale GRE and VXLAN tunnel endpoints persists in neutron db this should be
deleted from the database. Also, if local_ip of L2 agent changes the
stale tunnel ports and flows persists on br-tun on other Compute Nodes and
Network Nodes for that remote ip this should also be removed.
Implementation
Plugin changes:
Added host column in 'ml2_vxlan_endpoints' and 'ml2_gre_endpoints' table.
Added delete_endpoint method for deleting the stale endpoints from db.
Modified tunnel_sync() method to accommodate these changes.
Modified testcases in test_type_vxlan.py
Modified testcases in test_type_gre.py
Agent changes:
Added tunnel_delete rpc for removing stale ports and flows from br-tun.
tunnel_sync rpc signature upgrade to obtain 'host'.
Added testcases for TunnelRpcCallbackMixin().
This patch-set only deals with plugin side changes.
Irena Berezovsky [Thu, 18 Dec 2014 04:34:59 +0000 (06:34 +0200)]
Add support for flat networks in SRIOV Mechanism Driver
Allow flat network segment to be bound by SRIOV Mechanism Driver.
Populate vif_details with vlan_id = 0 to support flat networking
(according to man page of ip command).
Terry Wilson [Fri, 11 Jul 2014 23:55:30 +0000 (17:55 -0600)]
Retry on unassigned ofport instead of treating it as a failure
Open vSwitch will return '[]' when querying an interface's ofport
when the ofport has not yet been assigned. This doesn't signal a
failure, but the get_port_ofport code was treating it as such.
This patch uses a decorator from python-retrying which has been
added as a dependency of oslo_concurrency and therefore packaged
everywhere. The call to fetch the ofport is retried until the
vsctl_timeout is reached and, on failure, INVALID_OFPORT is
returned.
The add_port function will attempt to delete the port if
INVALID_OFPORT is returned from get_port_ofport. add_port is also
extended to take optional Interface options so that the
add_tunnel_port and add_patch_port functions can reuse it instead
of just duplicating its functionality.
abhishekkekane [Tue, 21 Oct 2014 11:15:15 +0000 (04:15 -0700)]
Eventlet green threads not released back to pool
Presently, the wsgi server allows persist connections. Hence even after
the response is sent to the client, it doesn't close the client socket
connection. Because of this problem, the green thread is not released
back to the pool.
In order to close the client socket connection explicitly after the
response is sent and read successfully by the client, you simply have to
set keepalive to False when you create a wsgi server.
Added a parameter to take advantage of the new(ish) eventlet socket timeout
behaviour. Allows closing idle client connections after a period of time, eg:
$ time nc localhost 8776
real 1m0.063s
Setting 'client_socket_timeout = 0' means do not timeout.
Terry Wilson [Sat, 20 Dec 2014 02:01:36 +0000 (19:01 -0700)]
Don't unnecessarily loop through all ports/interfaces
The ovs-vsctl 'list' command can take a list of records as an
argument, so there is no need to manually loop through all records
discarding the ones with names that don't match the bridge's
port name list.
Also, since the data is returned as json, the ofport returned isn't
a string, so we don't have to convert it to int for testing.
Terry Wilson [Fri, 26 Dec 2014 20:06:20 +0000 (13:06 -0700)]
Set type=internal as part of port creation
When creating an internal device with ovs-vsctl, the 'set'
operation must occur as part of the transaction that creates the
port, otherwise the device creation will fail and an ofport will
not be assigned.
fujioka yuuichi [Fri, 31 Jan 2014 01:54:34 +0000 (10:54 +0900)]
Allow to specify IP address of floating ip
IP address of floating ip will be automatically allocated.
There are cases where users need to specify a floating IP address.
This patch addresses the problem.
The feature is limited by "create_floatingip:floating_ip_address" in
"policy.json".
By default, it needs an admin role.
Numan Siddique [Mon, 1 Dec 2014 13:19:30 +0000 (18:49 +0530)]
Do not count dvr agents while creating HA ports
Presently dvr agents are not excluded when a
ha router interface is created, because of which
an interface is created even in the dvr agent namespace.
This patch fixes the issue by supporting a filter - 'agent_modes'
in the function L3AgentSchedulerDbMixin.get_l3_agents.
If this filter is defined, only the l3 agents whose modes
defined in this filter are returned
Cedric Brandily [Mon, 24 Nov 2014 15:53:04 +0000 (15:53 +0000)]
Do not run neutron-ns-metadata-proxy as root on L3 agent
Currently neutron-ns-metadata-proxy runs with root permissions when
namespaces are enabled on the l3 agent because root permissions are
required to "enter" in the namespace. But neutron-ns-metadata-proxy
permissions should be reduced as much as possible because it is
reachable from vms.
This change allows to change neutron-ns-metadata-proxy permissions
after its startup through the 2 new options metadata_proxy_user and
metadata_proxy_group which allow to define user/group running metadata
proxy after its initialization. Their default values are
neutron-l3-agent effective user and group.
Permissions drop is done after metadata proxy daemon writes its
pid in its pidfile (it could be disallowed after permissions drop).
Using nobody as metadata_proxy_user/group (more secure) is currently
not supported because:
* nobody has not the permission to connect the metadata socket,
* nobody has not the permission to log to file because neutron uses
WatchedFileHandler (which requires read/write permissions after
permissions drop).
This limitation will be addressed in a daughter change.
Cedric Brandily [Fri, 19 Dec 2014 14:05:53 +0000 (15:05 +0100)]
get_binary_name should returns strings without spaces
Iptables does not support chain names with spaces. It implies
get_binary_name should return strings without spaces (they are used as
chain name prefix). But currently 'python -m unittest $module' implies
spaces in get_binary_name() result, it disallows to use it when $module
is a functional test module. This change replaces spaces with
underscores in get_binary_name results.
Angus Lees [Wed, 22 Oct 2014 10:20:23 +0000 (21:20 +1100)]
Enable pylint checks for "anomalous" string escapes
Escapes in python string literals are well defined, but can be
confusing. These pylint checks look for backslash escapes in strings
that might be mistakes. Two code refactors were required to satisfy
these tests:
1. midonet_lib.py used \**kwargs in docstrings.
There doesn't seem to be a sphinx standard for kwargs, so this change
simply replaces them with "kwargs".
2. Regex literals containing escapes replaced with r''.
The assumption with this change (and the underlying pylint
check) is that r'' literals are more straightforward for regular
expressions, where every backslash is important.
While looking at these regexes, this change also removes a few
unnecessary "\-" escapes.
Angus Lees [Fri, 28 Nov 2014 01:23:47 +0000 (12:23 +1100)]
Tighten dnsmasq version regex
The previous regex forgot to escape the '.', resulting a more liberal
match than intended. Luckily it continued to work, since the dnsmasq
version is the first number that appears in the --version output.
This change improves the regex to correctly escape the '.' (as
presumably originally intended) and to look for the prefix "version ".
Angus Lees [Fri, 28 Nov 2014 01:18:49 +0000 (12:18 +1100)]
Remove unnecessary regex grouping
Several regexes in this class used had unnecessary (\s+) grouping. This
change makes the regexes slightly cheaper by removing the need for the
regex engine to remember the whitespace that was matched.
Robert Pothier [Mon, 22 Dec 2014 16:42:35 +0000 (11:42 -0500)]
ML2 UT: Fix incorrect mock return value
In the UT for ML2 Cisco Nexus MD,
in the function test_ncclient_version_detect()
The value being passed into the mock is incorrect
to mock the ncclient connect object.
Ihar Hrachyshka [Thu, 27 Nov 2014 14:43:48 +0000 (15:43 +0100)]
ipv6: set OtherConfig flag for DHCPv6 stateless subnets
In case of DHCPv6 stateless subnets, we should inform DHCP clients about
other configuration values available from DHCP server. This is done by
setting O (other) flag in RAs, which is controlled by AdvOtherConfigFlag
setting in radvd case.
Since radvd configuration file becomes quite complex, migrated its
generation to Jinja2.
Added a basic unit test that checks that flag is set for stateless mode
and not SLAAC. For stateful, it doesn't really matter whether other flag
is set, so no need to expect any value of it.
No more unit tests seem to be needed: conditional prefix generation is
already covered in test_l3_agent, and other statements are common for
all ipv6_ra_modes.
Fawad Khaliq [Sun, 21 Dec 2014 20:57:27 +0000 (12:57 -0800)]
PLUMgrid plugin: Fix for delete subnet with admin context
When delete call using admin for a subnet created from a
non-admin project is made, the tenant_id passed to backend
happened to be of admin project. This commit fixes the issues
by getting the correct tenant_id.
Angus Lees [Mon, 22 Dec 2014 04:41:01 +0000 (15:41 +1100)]
brocade: Use lazy logging interpolation
There are a small number of examples of "eager" interpolation in
neutron:
logging.debug("foo %s" % arg)
These should be converted to perform the interpolation lazily within
the logging function, since if the severity is below the logging level
then the interpolation can be skipped entirely.
This change addresses all such examples found in brocade
l3_router_plugin via a pylint test. Other occurrences are addressed
elsewhere.
Angus Lees [Mon, 22 Dec 2014 04:40:32 +0000 (15:40 +1100)]
linuxbridge: Use lazy logging interpolation
There are a small number of examples of "eager" interpolation in
neutron:
logging.debug("foo %s" % arg)
These should be converted to perform the interpolation lazily within
the logging function, since if the severity is below the logging level
then the interpolation can be skipped entirely.
This change addresses all such examples found in linuxbridge agent via a
pylint test. Other occurrences are addressed elsewhere.
Angus Lees [Mon, 22 Dec 2014 04:40:03 +0000 (15:40 +1100)]
embrane: Use lazy logging interpolation
There are a small number of examples of "eager" interpolation in
neutron:
logging.debug("foo %s" % arg)
These should be converted to perform the interpolation lazily within
the logging function, since if the severity is below the logging level
then the interpolation can be skipped entirely.
This change addresses all such examples found in embrane via a pylint
test. Other occurrences are addressed elsewhere.
Angus Lees [Mon, 22 Dec 2014 04:38:40 +0000 (15:38 +1100)]
bigswitch: Use lazy logging interpolation
There are a small number of examples of "eager" interpolation in
neutron:
logging.debug("foo %s" % arg)
These should be converted to perform the interpolation lazily within
the logging function, since if the severity is below the logging level
then the interpolation can be skipped entirely.
This change addresses all such examples found in bigswitch via a pylint
test. Other occurrences are addressed elsewhere.
Angus Lees [Mon, 22 Dec 2014 04:35:32 +0000 (15:35 +1100)]
Use lazy logging interpolation
There are a small number of examples of "eager" interpolation in
neutron:
logging.debug("foo %s" % arg)
These should be converted to perform the interpolation lazily within
the logging function, since if the severity is below the logging level
then the interpolation can be skipped entirely.
This change addresses all such current examples found in neutron core
via a pylint test. Vendor plugins and services are fixed elsewhere.
As written, msg is a tuple, and the various logging lines below print
the tuple members without properly expanding the format string as
intended.
The format in msg should have been expanded using % - as was presumably
the intention. This change fixes this and a similar example elsewhere
in this file.
Assaf Muller [Sun, 21 Dec 2014 14:15:12 +0000 (16:15 +0200)]
Update L3 agent drivers singletons to look at new agent
L3 agent drivers are singletons. They're created once, and hold
self.l3_agent. During testing, the agent is tossed away and
re-built, but the drivers singletons are pointing at the old
agent, and its old configuration.
This is necessary because pbr is stupid enough not to be
able to handle symlinks. The initial symlink was necessary to
avoid breaking the functional job. Once change [1] merges we
can drop the old contrib directory.
Russell Bryant [Tue, 9 Dec 2014 17:30:53 +0000 (17:30 +0000)]
Scope dhcp rpc api using a messaging namespace
This patch updates the rpc API used by the DHCP agent to make calls
back into the neutron plugin to use the 'dhcp' namespace instead of
the default namespace. The reason is that this API is exposed over
the 'q-plugin' topic along with several other interfaces. Without the
use of namespaces, all of the interfaces are effectively treated as
one by oslo.messaging. When a namespace is used, the interface can be
versioned independently and when a method is called, the only class
considered for fulfilling the request is the one that claims to
implement the 'dhcp' namespace.
While we're here, add documentation to both the client and server side
of this interface that indicates where the other side is located.
Kevin Benton [Wed, 18 Jun 2014 19:03:01 +0000 (12:03 -0700)]
Allow setting a tenant router's external IP
Adds an external_ip option to the router creation
and update operations to set the IP address the router
will try to use as its fixed IP on the external network
it's attached to. By default this is restricted to an
admin-only operation by policy.json.
Salvatore [Wed, 17 Dec 2014 22:09:24 +0000 (23:09 +0100)]
Remove NSX 'service' plugin
Following the spin-off for Neutron advanced services, this plugin
has become non-functional.
This patch removes the plugin, the service drivers, the database
models which kept tracking of resource associations, exceptions,
and obviously unit tests.
As there were some extensions which were leveraged only by this
plugin, they are being removed as well. In particular, one of
these extensions, 'routed-service-insertion' was in the
neutron.extensions package rather than neutron.plugins.vmware
package. This was for historical reasons. As no other plugin is
using this extension, it is being removed as well with this
patch.
By removing this plugin vmware the temporary skips applied to
neutron unit tests can be lifted. This patch does this.
Code Review / openstack-build / neutron-build.git / log