Kevin Benton [Tue, 30 Sep 2014 03:21:23 +0000 (20:21 -0700)]
ML2: move L3 cleanup out of network transaction
Move _process_l3_delete out of the delete_network
transaction to eliminate the semaphore deadlock that
occurs when it tries to delete the ports associated
with existing floating IPs.
It makes more sense to live outside of the transaction
anyway because the operations it performs cannot be
rolled back only in the database if the L3 plugin makes
external calls for floating IP creation/deletion.
e.g. if delete_floatingip is successful, it may have
deleted external resources and restoring the DB records
would make things inconsistent.
If a failure to delete the network does occur, any cleanup
done by _process_l3_delete will not be reversed.
Note: There are several other unrelated unit tests that also break with a
randomized PYTHONHASHSEED, but they are not addressed here. They will be
addressed in separate patches.
Kevin Benton [Sat, 20 Sep 2014 07:17:58 +0000 (00:17 -0700)]
Fix broken port query in Extraroute test case
One of the queries in an extra route test case tries
to filter based on the port owner, but the _list_ports
method it calls doesn't take a device_owner parameter.
This can cause failures if a DHCP port is created on
the same subnet.
The patch being reverted here addresses an issue that can no longer be
reproduced, in that under no circumstances, I can make the FIP lie around
before deleting a router (which can only be done after all FIP have been
disassociated or released).
Unless we have more clarity as to what the initial commit was really meant
to fix, there is a strong case for reverting this patch at this point.
Yong Sheng Gong [Mon, 30 Jun 2014 07:01:17 +0000 (15:01 +0800)]
Deletes floating ip related connection states
When a floating ip is dissociated with a port, the current
connection with the floating ip is still working. This patch
will clear the connection state and cut off the connection
immediately.
Since conntrack -D will return 1, which is not an error code,
so add extra_ok_codes argument to execute methods.
Brian Haley [Thu, 18 Sep 2014 01:48:53 +0000 (21:48 -0400)]
Do not lookup l3-agent for floating IP if host=None, dvr issue
If a floating IP has been associated with a port, but the port
has not been associated with an instance, attempting to lookup
the l3-agent hosting it will cause an AgentNotFoundByTypeHost
exception. Just skip it and go onto the next one.
Carl Baldwin [Fri, 19 Sep 2014 17:37:17 +0000 (17:37 +0000)]
Remove RPC notification from transaction in create/update port
Removing notifications to the L3 agent from within the transaction in
create_port and update_port eliminates many lock wait timeouts in the
dvr check queue job and in scale testing locally.
Since this patch leaves context unused in _process_port_binding, the
argument is removed from the method.
Jacek Swiderski [Wed, 6 Aug 2014 09:23:16 +0000 (11:23 +0200)]
Do not assume order of body and tags elements
This fixes the l2gateway unit test that breaks with a randomized PYTHONHASHSEED
(see the bug report).
The test assumed that the body dict from self._create_expected_req_body
had elements (including contents of tags list) in a particular order.
Found with PYTHONHASHSEED=2455351445.
The fix ensures that body is in predictable order.
Partial-bug: #1348818
Note: There are several other unrelated unit tests that also break with a
randomized PYTHONHASHSEED, but they are not addressed here. They will be
addressed in separate patches.
Elena Ezhova [Tue, 19 Aug 2014 11:54:36 +0000 (15:54 +0400)]
Forbid regular users to reset admin-only attrs to default values
A regular user can reset an admin-only attribute to its default
value due to the fact that a corresponding policy rule is
enforced only in the case when an attribute is present in the
target AND has a non-default value.
Added a new attribute "attributes_to_update" which contains a list
of all to-be updated attributes to the body of the target that is
passed to policy.enforce.
Changed a check for whether an attribute is explicitly set.
Now, in the case of update, the function should not pay attention
to a default value of an attribute, but check whether it was
explicitly marked as being updated.
Gary Kotton [Mon, 22 Sep 2014 17:03:37 +0000 (10:03 -0700)]
Security groups: prevent race for default security group creation
When a VM is booted via the Nova the client connection is created
with an admin user. This causes problems when creating the neutron
port. That is, there may be a race for the creation of the default
security group for the tenant.
The problem was introduced by commit acf44dba26ca8dca47bfb5fb2916807f9f4e2060
If the firewall rules are not shared and if they belong to different
tenants, then admin should not be able to create a policy using
these rules and he should not be able to insert such rules into
policies. An exception should be raised in such case. Added new
exception “FirewallRuleConflict” to handle such conditions.
Neutron cannot possibly be passing tests under Python 2.6, as
neutron/tests/unit/test_api_v2.py is referencing
collections.OrderedDict, which does not exist in Python 2.6.
Since there is no reason to use an OrderedDict in this case,
this replaces it with a simple dict.
Kevin Benton [Sat, 20 Sep 2014 17:48:22 +0000 (10:48 -0700)]
Mock out all RPC calls with a fixture
Mock out the rpc proxy calls used by various agents to
prevent unit tests from blocking for 10+ seconds while waiting
for a timeout. This happened with the OVS agent unit tests
recently in Change-ID Idd770a85a9eabff112d9613e75d8bb524020234a.
This change results in a reduction from 330.8 seconds to 2.7 seconds
for the neutron.tests.unit.openvswitch.test_ovs_neutron_agent
test module.
Elena Ezhova [Tue, 26 Aug 2014 15:22:20 +0000 (19:22 +0400)]
Add logging for enforced policy rules
There are a lot of policy rules which should not necessarily
be explicitly specified in policy.json to be checked while enforcement.
There should be a way for an operator to know which policy rules are
actually being enforced for each action.
Kevin Benton [Thu, 18 Sep 2014 11:21:05 +0000 (04:21 -0700)]
Remove unnecessary _make_port function in BSN UTs
The Big Switch unit tests had unnecessary copies of the
_make_port function to allow the binding:host_id field to
be set. This was already possible with the existing _make_port
call through the use of kwargs so the extra function wasn't
necessary.
This patch has a few benign changes that should be easily reviewed.
The purpose of this patch is to allow me to make cleaner edits in
follow on patches so that they're more easily reviewed in their
specific contexts.
Indicate the begin and end of the sync process to EOS
Send a trigger to EOS when a sync operation is initiated, and,
another trigger when the sync operation is complete.
Additionally, sync_interval value (from ml2_conf_arista.ini)
is passed down to EOS. This is used by EOS to timeout the
transaction.
Bradley Jones [Wed, 6 Aug 2014 13:16:23 +0000 (14:16 +0100)]
Do not assume order of device_ids set elements
This fixes the test_ancillary_bridges_multiple unit test that breaks with a randomized
PYTHONHASHSEED (see the bug report).
The test assumed that the device_ids set had
elements in a particular order. Found with PYTHONHASHSEED=2455351445.
The fix refactors the pullup_side_effect function so that it checks if the
device_id exists before returning the bridge.
Note: There are several other unrelated unit tests that also break with a
randomized PYTHONHASHSEED, but they are not addressed here. They will be
addressed in separate patches.
Only setup dhcp interface if dhcp is not active on network
When enabling (DhcpLocalProcess.enable()) dhcp for a network the agent
first sets dhcp interface, then checks if dhcp is curently active and
if it's true then the agent restarts dhcp.
Restart (DhcpBase.restart()) first disables dhcp and then enables it again
by calling DhcpLocalProcess.enable() recursively which in turn sets
dhcp interface again (it doesn't see the port created earlier as network
is not re-fetched from db). This leads to duplicate dhcp interface
for the network.
The fix is to only setup dhcp interface if dhcp is not active.
HA routers master state now distributed amongst agents
We're currently running with no pre-emption, meaning that
the first router in a cluster to go up will be the master,
regardless of priority. Since the order in which we sent
notifications was constant, the same agent hosted the
master instances of all HA routers, defeating the idea
of load sharing.
Paul Michali [Tue, 16 Sep 2014 15:22:17 +0000 (11:22 -0400)]
Rework and enable VPNaaS UT for Cisco CSR REST
The Cisco CSR REST client library unit tests were developed in
Icehouse, using the httmock library. However, the community did
not want to add this library to global requirements, as there was
a similar httpretty library available (albeit with some short-
comings). As a result, the test module was renamed with a "no"
prefix, to prevent inclusion in automated tests.
Since then, a new library, requests-mock, has been added to global
requirements, to replace httpretty, and is being used on several
other projects.
This commit reworks the unit test to use requests-mock, instead of
httmock. The functionality is the same, but the mechanism (a
fixture with URI registration vs context manager) is different.
This commit provides coverage for the REST client code, by using a
mock for the Cisco CSR VM. The unit test module can be subclassed,
and used with a real CSR VM, for 3rd party CI testing, in the
future.
The functional job was breaking due to the interaction between
devstack installing neutron system-wide in editable mode (pip install
-e) and tox packaging in the same path. Installing in editable
mode meant that neutron.egg-info/PKG_INFO could be updated (in this
case by tox) to change the installed version of the neutron package
without updating scripts that depended on the installed version
(e.g. rootwrap). This fix is to set the dsvm-functional env to
use system packages and avoid having tox update PKG_INFO.
Kevin Benton [Thu, 18 Sep 2014 20:46:51 +0000 (13:46 -0700)]
Delete a broken subnet delete unit test
A test to delete a subnet in use was incorrectly
calling 'subnet' instead of 'subnets' in the API request
and asserting a 404 instead of a 409. Even the correct
version of this test is already covered by the
'test_port_prevents_subnet_deletion' method so this
commit just removes the broken test.
Fix to delete user and group association in Nuage Plugin
After a router delete operation, the attached zone to that
router is also deleted. Got rid of code that tried to do
a get operation on the nuage_zone after router delete
operation.
Closes-Bug: #1367864
Kevin Benton [Wed, 17 Sep 2014 00:29:51 +0000 (17:29 -0700)]
Delete DB records instead of tables to speedup UT
Now that the schema is fixed for all of the plugins,
there isn't a need to delete and recreate the entire
schema for every unit test.
This patch clears the tables at the end of each test
instead of deleting them. This eliminated overhead seems
to save 10%+ execution time of the entire set of unit
tests.
Example of performance gain from tox -epy27 tests.unit.ml2:
Before: Ran 2495 tests in 284.186s
After: Ran 2495 tests in 223.299s
A recent change has made this attribute required for nova
integration.
This patch adds this attribute to responses generated by the NSX
plugin, and also ensures relevant unit tests are executed for the
vmware NSX plugin.
Paul Michali [Fri, 12 Sep 2014 18:16:35 +0000 (14:16 -0400)]
Access correct key for template name
When an exception occurs while loading the config agent driver, the
handler tries to log a message, but accesses the wrong key to get the
template name. This causes another exception, which masks the original
exception.
This change accesses the correct key and performs logging inside a
with block to (defensively) preserve the exception context.
Carl Baldwin [Tue, 11 Feb 2014 00:58:42 +0000 (00:58 +0000)]
Rename workers to api_workers and simplify code
Refactor a few ugly aspects of the multiple API worker patch to make
way for multiple rpc workers. This came up as I was trying to add
multiple RPC workers using similar patterns and remembering that some
things were left in a rather awkward state.
This fix ensures that DHCP Ports that are
available on DVR routed subnets, are serviced
by DVR neutron infrastructure.
Here servicing by DVR means, creation of
DVR namespaces on such nodes holding DHCP
Ports and also applying DVR specific OVS
Rules to the br-int and br-tun bridges on
such nodes, to enable traffic to be routed
via DVR to such DHCP Ports.
Currently, there is no check which validates the values of
tunnel range for VXLAN/GRE networks. The VXLAN VNI is 24 bit
which have range between 1 to 2^24 - 1. Similarly, GRE key field
is 32 bit which have range between 1 to 2^32 - 1.
Code Review / openstack-build / neutron-build.git / log