In Python 3:
>>> None > 0
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: unorderable types: NoneType() > int()
This commit does not change the behaviour of the affected code, but fixes the
TypeError that is raised in Python 3.
Some tests have been added to the py34 section of tox.ini, but are currently
commented out: these are tests that work with this patch and a fixed version of
Routes.
Carl Baldwin [Mon, 13 Jul 2015 18:44:38 +0000 (18:44 +0000)]
Make external_gateway_nat_rules easier to understand
Future patches will change these snat rules for address scopes. I
found it much easier to understand the changes needed after formatting
these like this.
Carl Baldwin [Thu, 9 Jul 2015 19:19:00 +0000 (19:19 +0000)]
Remove perform_snat_action indirection
This indirection seems complicated to me. I don't know the history
behind it but it made some of the address scope work more difficult
than I think it needs to be.
This patch introduces API and DB plugin for flavor framework.
API adds Flavors and Service Profiles which are resources
available only for admins to operate.
This framework then should be leveraged by advanced services.
Yusuke Ide [Sat, 4 Jul 2015 06:38:58 +0000 (15:38 +0900)]
Add breakages in public API of devref
In wiki it is intended to keep track of neutron changes that may affect
external plugins and drivers. The information should be kept in a living
document in the devref area of the neutron code tree.
Kevin Benton [Mon, 15 Jun 2015 09:18:36 +0000 (02:18 -0700)]
Network RBAC DB setup and legacy migration
This patch implements the database model required for the network
RBAC work. In addition it migrates the current network and subnet
'shared' attributes to leverage the new table.
'shared' is no longer a property of the DB model because its status
is based on the tenant ID of the API caller. From an API perspective
this is the same (tenants will see networks as 'shared=True' if the
network is shared with them). However, internal callers (e.g. plugins,
drivers, services) will not be able to check for the 'shared' attribute
on network and subnet db objects any more.
This patch just achieves parity with the current shared behavior so it
doesn't add the ability to manipulate the RBAC entries directly. The
RBAC API is in the following patch.
Kevin Benton [Thu, 16 Jul 2015 09:07:48 +0000 (02:07 -0700)]
Add oslo db retry decorator to non-CRUD actions
The previously added decorators to the create and update handlers
in the API layer only applied to actions that followed the standard
create/update path. However, for API operations like add_router_interface,
a different path is followed that wasn't covered by a retry decorator.
This patch adds the decorator to handle deadlocks in those operations as
well.
Carl Baldwin [Thu, 9 Jul 2015 22:05:05 +0000 (22:05 +0000)]
Create dvr base class and stop passing around snat_ports
The one thing that I see that the two dvr classes have in common is
the ability to map internal ports to snat ports. The dvr local router
needs it to set up a redirect to the central part. The central part
needs it to create the port for the internal network.
This change renames the mapping method to something more logical and
removes snat_ports as an argument to two methods because it is a quick
O(1) operation to get it from the router dict and passing it around
just tangles things up.
In some cases, logging can grab locks and thusly attempt to reschedule,
which will fail in signal handlers. this patch removes the actions
from the signal handers, just set a flat if it got a signal, then in
rpc_loop, check the flag and perform appropriate actions.
Cedric Brandily [Tue, 16 Jun 2015 07:42:35 +0000 (09:42 +0200)]
Galera multi-writers compliant sync_allocations
Currently sync_allocations[1] uses with_lockmode('update) which implies
possible deadlocks with Galera multi-writers. This change decorates the
method in order to catch and retry sync_allocations.
The change I81806a43ecc6f0a7b293ce3e70d09d1e266b9f02
effectively removed _delete_port from NeutronDbPluginV2.
Unfortunately, it's still used directly by l3_dvr_db.
Fix update_port_postcommit and port not found with DVR
Updating DVR Router interface ports was throwing
errors in the l2pop mechanism drivers function
update_port_postcommit.
PortContext's portbinding information does not show
the status of the ports. For DVR Router interface
ports the DVRPortbinding table contains the status
information for the ports.
In the case of the update_port method, there was
no code related to DVR that retreives the port
binding information from the DVRPortBinding table.
This was working before, since in the driver_context,
the PortContext was just returning the port status for
all router interfaces.
With the recent refactor to the driver_context, this
behavior changed and the PortContext was returning the
_binding.status for the DVR router interface ports and
the _port.status for the non DVR ports.
When the update_port function calls update_port_postcommit
with PortContext for DVR router interface ports, l2pop
was throwing an error saying that Portbinding does not
have the attribute 'status'.
This was causing addition of any second subnet to the
same network with respect to IPv6 to fail.
Because in the case of IPv6, when you add additional
subnets to the existing network, it just updates the port
with the IPv6 prefix instead of creating additional port.
In the case of IPv4 still we could see that there are
two different ports created for each subnet we try to
add.
This patch fixes the above issue in l2pop and allows the
DVR router interface ports to be successfull.
Also the _find_ipv6_router_port_by_network was returning
all the ports for DVR including the DVR CSNAT internal
ports which are not part of the router interface ports.
This patch also fixes this problem by returning false,
when it finds a DVR SNAT port.
oslo_utils raise ImportError if import fails. We should propagate other
failures to callers. Otherwise we may hide issues.
Also report exact failure from import_object in case L3 agent fails to
import interface_driver.
As part of the job, consolidated code to load interface driver into
common function.
Also, stopped checking for specific log messages in dhcp and l3 agent
unit tests: it's too fragile and actually not something we need a unit
test for.
Not to introduce more work for people who handle py3 porting effort,
added the unit test into the list of those that are executed for py34
job until the whole suite is ready for python3.
Ihar Hrachyshka [Mon, 22 Jun 2015 14:23:36 +0000 (16:23 +0200)]
[neutron-db-manage] support separate migration branches
New migration rule scheme is introduced. Now, changes are classified
into the following branches:
- expand (additive changes only)
- contract (contraction changes, including data migration)
Make 'neutron-db-manage revision' generate two separate migration
scripts, one per branch.
Now that we support multiple heads, renamed HEAD file in HEADS. We still
don't allow more branching, so keep validation for the number of
branches.
For backwards compatibility, calling to 'upgrade head' applies both
branches in proper order.
Note that advanced services will be moved to the new migration scheme in
separate patches for respective repos.
This patch does not introduce autogenerate support for multiple branches
for 'revision' command (that depends on a new alembic version yet
unreleased; but alembic/master already has everything to support us).
The patch does not implement 'expand' or 'contract' commands that are
anticipated by the spec proposal. Those will come in consequent patches.
Upgrade impact is backwards compatible: those who are interested in
reducing downtime while applying some migration rules can opt in it by
modifying their upgrade practices, while everything should still work
the old way for those who don't care.
At now, global-requirements has an entry:
Routes!=2.0,!=2.1,>=1.12.3
and this leads to Routes < 2.0 which is not compatible
with Python 3. This blocks updating of global requirements [1].
To avoid the situation, this commit disables python 3 related
error due to Routes version.
Closes-Bug: #1474266
Related to blueprint neutron-python3
Fix typo 'adress_scope' to 'address_scope'
in neutron/extensions/address_scope.py.
Fix typo 'test_remove_adresses_by_interface' to
'test_remove_addresses_by_interface'
in neutron/tests/unit/agent/linux/test_keepalived.py
Kevin Benton [Mon, 13 Jul 2015 22:27:54 +0000 (15:27 -0700)]
Remove db-access semaphore in ML2
With the addition of the deadlock retry decorator in
I635cc49ca69f589f99ab145d4d51e511b24194d2 and the change
to instantaneous deadlock feedback under the pymysql driver,
this semaphore should no longer be necessary. Removing it
will allow us to benefit from processing multiple requests
simultaneously.
Carl Baldwin [Thu, 9 Jul 2015 21:26:49 +0000 (21:26 +0000)]
Remove self.snat_ports, a dvr thing, from router base class
This was overlooked in splitting out dvr code from the router base
class. It is dvr only. Also, it doesn't need to be an instance
variable and was not use consistently.
Description:
Presently Neutron doesn't validate the address family of floating IP
addresses or the internal addresses they are associated with. It merely
associates the first IP of the floating IP's port with the first IP of
the internal port, unless a specified fixed IP is specified. This can
lead to incorrect or poorly defined behavior when IPv6 is present.
The existing L3 agent implementation only manages IPv4 NAT rules. While
IPv6 NAT and NAT protocol translation are possible, the existing
implementation does not support these configurations.
Presently a floating IP can be created on an IPv6 only external network
or associated with an IPv6 fixed IP, but the L3 agent is unable to bind
these configurations.
Implementation:
When creating and updating a floating IP, only consider IPv4 addresses
on both the floating IPs port and the internal port he floating IP is
associated with. Additionally disallow creating floating IPs on networks
without any IPv4 subnets, since these floating IPs could not be
allocated an IPv4 address.
Lower log level of errors caused by user requests to INFO
When webob exception is returned, error level log is
always recorded even though we return 4xx (client error)
to users. 4xx error in response code means errors due to
user requests, so error level log is not appropriate.
This commit lowers the log level to INFO for such events.
Currently only case I see is webob.exc.HTTPNotFound returned
by the policy engine when a user requests an operation
prohibited by the policy, but I think there is no reason
we deal with 404 specially in neutron.api.v2.resource.
Enforce specific order for firewall.(un)filtered_ports and devices
Lots of tests in the file rely on specific order of devices and ports
with which they are iterated thru inside firewall implementation. This
is needed to match a regexp against iptables output generated by the
firewall driver.
In production code, those .(un)filtered_ports dictionaries are
unordered, and it would be not wise to enforce the order for them just
for the sake of those unit tests.
Instead, we 'patch' the agent firewall with ordered versions of dict for
those attributes.
Also enforce specific order for device_info dictionaries we pass into
firewall.
The failure was easily reproducible with PYTHONHASHSEED=111, and after
the fix, it's gone.
While at it, stop making assumptions about stable order of dict.values()
between multiple dictionaries with the same keys. It may actually work
for now, but it seems fragile. Overall simplified regex construction
code a bit, f.e. killing some dead or redundant code.
Carl Baldwin [Thu, 9 Jul 2015 21:14:39 +0000 (21:14 +0000)]
Use _is_this_snat_host and remove _get_gw_port_host
The _is_this_snat_host utility should be used consistently. After
replacing a call to _get_gw_port_host with _is_this_snat_host,
_get_gw_port_host is left as just a simple helper method for
_is_this_snat_host. The indirection is excessive and so they are
combined.
Code Review / openstack-build / neutron-build.git / log