Carl Baldwin [Thu, 9 Jul 2015 22:05:05 +0000 (22:05 +0000)]
Create dvr base class and stop passing around snat_ports
The one thing that I see that the two dvr classes have in common is
the ability to map internal ports to snat ports. The dvr local router
needs it to set up a redirect to the central part. The central part
needs it to create the port for the internal network.
This change renames the mapping method to something more logical and
removes snat_ports as an argument to two methods because it is a quick
O(1) operation to get it from the router dict and passing it around
just tangles things up.
In some cases, logging can grab locks and thusly attempt to reschedule,
which will fail in signal handlers. this patch removes the actions
from the signal handers, just set a flat if it got a signal, then in
rpc_loop, check the flag and perform appropriate actions.
Cedric Brandily [Tue, 16 Jun 2015 07:42:35 +0000 (09:42 +0200)]
Galera multi-writers compliant sync_allocations
Currently sync_allocations[1] uses with_lockmode('update) which implies
possible deadlocks with Galera multi-writers. This change decorates the
method in order to catch and retry sync_allocations.
The change I81806a43ecc6f0a7b293ce3e70d09d1e266b9f02
effectively removed _delete_port from NeutronDbPluginV2.
Unfortunately, it's still used directly by l3_dvr_db.
Fix update_port_postcommit and port not found with DVR
Updating DVR Router interface ports was throwing
errors in the l2pop mechanism drivers function
update_port_postcommit.
PortContext's portbinding information does not show
the status of the ports. For DVR Router interface
ports the DVRPortbinding table contains the status
information for the ports.
In the case of the update_port method, there was
no code related to DVR that retreives the port
binding information from the DVRPortBinding table.
This was working before, since in the driver_context,
the PortContext was just returning the port status for
all router interfaces.
With the recent refactor to the driver_context, this
behavior changed and the PortContext was returning the
_binding.status for the DVR router interface ports and
the _port.status for the non DVR ports.
When the update_port function calls update_port_postcommit
with PortContext for DVR router interface ports, l2pop
was throwing an error saying that Portbinding does not
have the attribute 'status'.
This was causing addition of any second subnet to the
same network with respect to IPv6 to fail.
Because in the case of IPv6, when you add additional
subnets to the existing network, it just updates the port
with the IPv6 prefix instead of creating additional port.
In the case of IPv4 still we could see that there are
two different ports created for each subnet we try to
add.
This patch fixes the above issue in l2pop and allows the
DVR router interface ports to be successfull.
Also the _find_ipv6_router_port_by_network was returning
all the ports for DVR including the DVR CSNAT internal
ports which are not part of the router interface ports.
This patch also fixes this problem by returning false,
when it finds a DVR SNAT port.
oslo_utils raise ImportError if import fails. We should propagate other
failures to callers. Otherwise we may hide issues.
Also report exact failure from import_object in case L3 agent fails to
import interface_driver.
As part of the job, consolidated code to load interface driver into
common function.
Also, stopped checking for specific log messages in dhcp and l3 agent
unit tests: it's too fragile and actually not something we need a unit
test for.
Not to introduce more work for people who handle py3 porting effort,
added the unit test into the list of those that are executed for py34
job until the whole suite is ready for python3.
Ihar Hrachyshka [Mon, 22 Jun 2015 14:23:36 +0000 (16:23 +0200)]
[neutron-db-manage] support separate migration branches
New migration rule scheme is introduced. Now, changes are classified
into the following branches:
- expand (additive changes only)
- contract (contraction changes, including data migration)
Make 'neutron-db-manage revision' generate two separate migration
scripts, one per branch.
Now that we support multiple heads, renamed HEAD file in HEADS. We still
don't allow more branching, so keep validation for the number of
branches.
For backwards compatibility, calling to 'upgrade head' applies both
branches in proper order.
Note that advanced services will be moved to the new migration scheme in
separate patches for respective repos.
This patch does not introduce autogenerate support for multiple branches
for 'revision' command (that depends on a new alembic version yet
unreleased; but alembic/master already has everything to support us).
The patch does not implement 'expand' or 'contract' commands that are
anticipated by the spec proposal. Those will come in consequent patches.
Upgrade impact is backwards compatible: those who are interested in
reducing downtime while applying some migration rules can opt in it by
modifying their upgrade practices, while everything should still work
the old way for those who don't care.
At now, global-requirements has an entry:
Routes!=2.0,!=2.1,>=1.12.3
and this leads to Routes < 2.0 which is not compatible
with Python 3. This blocks updating of global requirements [1].
To avoid the situation, this commit disables python 3 related
error due to Routes version.
Closes-Bug: #1474266
Related to blueprint neutron-python3
Fix typo 'adress_scope' to 'address_scope'
in neutron/extensions/address_scope.py.
Fix typo 'test_remove_adresses_by_interface' to
'test_remove_addresses_by_interface'
in neutron/tests/unit/agent/linux/test_keepalived.py
Kevin Benton [Mon, 13 Jul 2015 22:27:54 +0000 (15:27 -0700)]
Remove db-access semaphore in ML2
With the addition of the deadlock retry decorator in
I635cc49ca69f589f99ab145d4d51e511b24194d2 and the change
to instantaneous deadlock feedback under the pymysql driver,
this semaphore should no longer be necessary. Removing it
will allow us to benefit from processing multiple requests
simultaneously.
Carl Baldwin [Thu, 9 Jul 2015 21:26:49 +0000 (21:26 +0000)]
Remove self.snat_ports, a dvr thing, from router base class
This was overlooked in splitting out dvr code from the router base
class. It is dvr only. Also, it doesn't need to be an instance
variable and was not use consistently.
Description:
Presently Neutron doesn't validate the address family of floating IP
addresses or the internal addresses they are associated with. It merely
associates the first IP of the floating IP's port with the first IP of
the internal port, unless a specified fixed IP is specified. This can
lead to incorrect or poorly defined behavior when IPv6 is present.
The existing L3 agent implementation only manages IPv4 NAT rules. While
IPv6 NAT and NAT protocol translation are possible, the existing
implementation does not support these configurations.
Presently a floating IP can be created on an IPv6 only external network
or associated with an IPv6 fixed IP, but the L3 agent is unable to bind
these configurations.
Implementation:
When creating and updating a floating IP, only consider IPv4 addresses
on both the floating IPs port and the internal port he floating IP is
associated with. Additionally disallow creating floating IPs on networks
without any IPv4 subnets, since these floating IPs could not be
allocated an IPv4 address.
Enforce specific order for firewall.(un)filtered_ports and devices
Lots of tests in the file rely on specific order of devices and ports
with which they are iterated thru inside firewall implementation. This
is needed to match a regexp against iptables output generated by the
firewall driver.
In production code, those .(un)filtered_ports dictionaries are
unordered, and it would be not wise to enforce the order for them just
for the sake of those unit tests.
Instead, we 'patch' the agent firewall with ordered versions of dict for
those attributes.
Also enforce specific order for device_info dictionaries we pass into
firewall.
The failure was easily reproducible with PYTHONHASHSEED=111, and after
the fix, it's gone.
While at it, stop making assumptions about stable order of dict.values()
between multiple dictionaries with the same keys. It may actually work
for now, but it seems fragile. Overall simplified regex construction
code a bit, f.e. killing some dead or redundant code.
Carl Baldwin [Thu, 9 Jul 2015 21:14:39 +0000 (21:14 +0000)]
Use _is_this_snat_host and remove _get_gw_port_host
The _is_this_snat_host utility should be used consistently. After
replacing a call to _get_gw_port_host with _is_this_snat_host,
_get_gw_port_host is left as just a simple helper method for
_is_this_snat_host. The indirection is excessive and so they are
combined.
Dan Prince [Thu, 9 Jul 2015 19:31:13 +0000 (15:31 -0400)]
get_vif_ports: ignore non-Interface ports
This patch updates get_vif_ports so that it skips
ports which aren't in the 'Interfaces' table.
This fixes an issue where neutron-ovs-cleanup would
fail if any sort of OVS bond was on the bridge getting
cleaned up. This is because bonds don't have the same
attributes as ports, and thus fail subsequent ovs-vsctl
queries.
Pavel Bondar [Mon, 6 Jul 2015 15:36:22 +0000 (18:36 +0300)]
Add Pluggable IPAM Backend Part 1
Add methods for allocating/deallocating ips using IPAM driver.
Methods are covered by unit tests and currently used only by them.
For pluggable IPAM case ipam driver may execute calls to third-party servers.
It means we can't rely on database transaction rollback in case of failure.
So if any bulk ip allocation/deallocation fails rollback should be done
on third-party servers as well.
Any completed ip allocation should be explicitly deallocated in case of
failure, and vise versa for failure on deallocation.
Try-except block is used to do manual rollback actions.
After rollback actions are done, exception is reraised and local db
transaction rollback occurs.
Pluggable IPAM was divided into two parts to keep review size small.
Following patches are expected to use these methods for ip address
allocation.
Wei Wang [Thu, 25 Sep 2014 09:49:59 +0000 (17:49 +0800)]
Fix duplicate entry catch for allowed address pairs
If None is submitted as a MAC address in an allowed_address_pair,
the port MAC will be used. So if two entries are submitted with the
same IP and one's MAC is None while the others is the port's MAC,
they will pass the API duplication check and fail to insert into the DB
due to a unique constraint violation.
This patch catches the db error and turns it into the same exception
the API uses on duplicate entries.
The new version of mock is now released which has the fix to
make mock.patch.stopall behave deterministically. The code to
stop double-mocking is no longer required.
Other fixes:
'assert_has_calls' that have changed behavior in the new version
(no longer accepts single calls).
Calls to non-existent assert methods that did not exist and were
silently passing.
Use of autospec on a class with decorated functions.
Arista ML2 Mech driver for VLANs presently does
not filter out non-vlan type networks (e.g vxlan).
This fix will simply ignore the request if a
non-vlan based network request is seen.
Sean M. Collins [Fri, 26 Jun 2015 17:14:41 +0000 (11:14 -0600)]
Make IPAM more pythonic
__init__.py is usually an empty file, or contains platform specific
code to make the package function. One more use is to have __init__.py
contain import statements, to import functions and classes, for convenience.
get_snat_candidates() is about DVR routers scheduling and hence
should be a method of L3_DVRsch_db_mixin.
This commit does not modify anything, just moves.
Pavel Bondar [Wed, 4 Feb 2015 13:13:28 +0000 (16:13 +0300)]
Allow IPAM backend switch
Changed inheritance chain for NeutronDbPluginV2 to allow switching from
non-pluggable to pluggable IPAM implementation.
IpamNonPluggableBackend methods are called on it's instance,
instead of previous way where IpamNonPluggableBackend was parent for
NeutronDbPluginV2.
It allows switching IPAM implementation in set_ipam_backend
(IpamNonPluggableBackend to IpamPluggableBackend).
All methods that became public in IpamNonPluggableBackend were renamed.
This is refactoring step before Pluggable IPAM can be applied.
Move update_security_group_on_port to SecurityGroupDbMixin
Previously this function lived in SecurityGroupServerRpcMixin. This
patch moves this function to SecurityGroupDbMixin so plugins who don't
use SecurityGroupServerRpcMixin can leverage this function. This patch
has no affect to anyone using SecurityGroupServerRpcMixin as that inherits
from SecurityGroupDbMixin.
In Arista ML2 driver Reconfigure VLAN on VM migration
Whenever a VM moves from one compute node to the other, the VLAN on the old
switch interface was not removed and the VLAN was not being provisioned on the
new switch interface. With this patch, the VLANs are provisioned correctly.
Add sub-project lieutenant for networking-plumgrid
Since networking-plumgrid has been approved [1] in
governance to be part of Neutron. So as per the
concept of Lieutenants for Neutron, this patch defines
clear point of contact for networking-plumgrid.
Carl Baldwin [Tue, 7 Jul 2015 16:41:03 +0000 (16:41 +0000)]
Add IP_ANY dict to ease choosing between IPv4 and IPv6 "any" address
I'm working on a new patch that will add one more case where we need
to choose between 0.0.0.0/0 and ::/0 based on the ip version. I
thought I'd add a new constant and simplify a couple of existing uses.
The result of get_sorts was a 'zip object' in Python 3, and it was later used
as a list, which fails. Just cast the result to a list to fix this issue.
Code Review / openstack-build / neutron-build.git / log