rajeev [Mon, 13 Oct 2014 20:25:36 +0000 (16:25 -0400)]
Fix race condition on processing DVR floating IPs
Fip namespace and agent gateway port can be shared by multiple dvr routers.
This change uses a set as the control variable for these shared resources
and ensures that Test and Set operation on the control variable are
performed atomically so that race conditions do not occur among
multiple threads processing floating IPs.
Limitation: The scope of this change is limited to addressing the race
condition described in the bug report. It may not address other issues
such as pre-existing issue with handling of DVR floatingips on agent
restart.
Moves the HA resource creations outside of transaction
Currently the HA resources are created in the
_create_router_db which includes calls to
the plugin and generates RPC calls. Even if the
resource creations are outside of any transaction
from the _create_router_db point of view, this
method is called in a transaction from the
create_router method.
This patch moves the resource creations to the
create_router method outside the transaction.
The failures are handled as previously with
a try/expect.
Romil Gupta [Mon, 30 Jun 2014 12:35:08 +0000 (05:35 -0700)]
Schema enhancement to support MultiSegment Network
Description:
Currently, there is nothing in the schema that ensures segments
for a network are returned in the same order they were specified
when the network was created, or even in a deterministic order.
Solution:
We need to add another field named 'segment_index' in
'ml2_network_segment' table containing a numeric position index.
With segment_index field we can retrieve the segments in the
order in which user created.
This patch set also fixes ML2 invalid unit test case in
test_create_network_multiprovider().
Remove redundant initialization and check from DVR RPC mixin
The initialization, as well as the recurrent checks on whether the agent
is able to handle DVR requests, are not necessary; this is because the
dvr_agent is going to be set at the time the L2 agent starts receiving
RPC messages.
Kevin Benton [Wed, 24 Sep 2014 12:23:32 +0000 (05:23 -0700)]
Improve performance of security group DB query
The _select_ips_for_remote_group method was joining the
IP allocation, port, allowed address pair, and security group tables
together in a single query. Additionally, it was loading all of
the port columns and using none of them. This resulted in a
very expensive query with no benefit.
This patch eliminates the unnecessary use of the port table by joining
the IP allocation table directly to the security groups and allowed
address pairs tables. In local testing of the method, this sped it up
by an order of magnitude.
Ihar Hrachyshka [Tue, 14 Oct 2014 13:05:20 +0000 (15:05 +0200)]
Updated cache module and its dependencies
This is to avoid cache module dependency on timeutils that are now moved
to oslo.utils.
The following changes are included:
* neutron/openstack/common/cache/_backends/memory.py 6ff6b4b Switch oslo-incubator to use oslo.utils and remove old modules 2bedce3 Fix MemoryBackend not purging item from _keys_expired on delete
* neutron/openstack/common/cache/backends.py 39625e1 Set pbr 'warnerrors' option for doc build
* neutron/openstack/common/cache/cache.py 9c683be fix small typo
* neutron/openstack/common/lockutils.py 5d40e14 Remove code that moved to oslo.i18n 7209975 Always log the releasing, even under failure bbb266c Clarify logging in lockutils 942e1aa Use file locks by default again ac995be Fix E126 pep8 errors 15b8352 Remove oslo.log from lockutils
Ihar Hrachyshka [Tue, 14 Oct 2014 12:36:06 +0000 (14:36 +0200)]
Updated service.py and its dependencies
This is to avoid service module dependency on importutils that are now
moved to oslo.utils.
The following changes are included:
* neutron/openstack/common/eventlet_backdoor.py 5d40e14 Remove code that moved to oslo.i18n 90ae24b Remove redundant default=None for config options fcf517d Update oslo log messages with translation domains
* neutron/openstack/common/loopingcall.py 5d40e14 Remove code that moved to oslo.i18n e377393 Changes calcuation of variable delay ab5d5f1 Use timestamp in loopingcall bc48099 Log the function name of looping call fb4e863 Remove deprecated LoopingCall fcf517d Update oslo log messages with translation domains
* neutron/openstack/common/service.py 5d40e14 Remove code that moved to oslo.i18n 6ede600 rpc, notifier: remove deprecated modules
* neutron/openstack/common/systemd.py 17c4e21 Fix docstring indentation in systemd
* neutron/openstack/common/threadgroup.py 5a1a016 Make stop_timers() method public fdc8883 Add graceful stop function to ThreadGroup.stop 2d06d6c Simple typo correction 4d18b57 threadgroup: use threading rather than greenthread
Ihar Hrachyshka [Tue, 14 Oct 2014 12:22:09 +0000 (14:22 +0200)]
Updated fileutils and its dependencies
This is to avoid fileutils dependency on excutils that are now moved to
oslo.utils.
The following changes are included:
* neutron/openstack/common/__init__.py 6b048e7 Let oslotest manage the six.move setting for mox
* neutron/openstack/common/_i18n.py 9ce1d96 Fix i18n import 5d40e14 Remove code that moved to oslo.i18n
* neutron/openstack/common/fileutils.py 6ff6b4b Switch oslo-incubator to use oslo.utils and remove old modules 2b966f9 Fix deletion of cached file for policy enforcer 9c88dc3 file_open: fixed docstring to refer to open() instead of file() 6c7407b fileutils: port to Python 3 fcf517d Update oslo log messages with translation domains
* neutron/openstack/common/log.py 6c706c5 Delete graduated serialization files 5d40e14 Remove code that moved to oslo.i18n 6ff6b4b Switch oslo-incubator to use oslo.utils and remove old modules aa74411 log: add missing space in error message 037dee0 Set stevedore log level to WARN by default 37c0091 Add unicode coercion of logged messages to ContextFormatter 6614413 Correct coercion of logged message to unicode 1188d88 Except socket.error if syslog isn't running ac995be Fix E126 pep8 errors 631f880 Set keystonemiddleware and routes.middleware to log on WARN level 726d00a Adjust oslo logging to provide adapter is enabled for 433fa0b Make logging_context_format_string optional in log.set_defaults ac92c06 Add default log level for websocket 5fd77eb Ability to customize default_log_levels for each project 4d9328c Python 3: enable tests/unit/test_log.py cb5a804 Move `mask_password` to strutils
Note: cb5a804 is partially included; that's ok because we don't use the
moved function in Neutron.
Abhishek Raut [Wed, 8 Oct 2014 00:06:10 +0000 (17:06 -0700)]
Cisco N1kv: Fix update network profile for add tenants
This patch makes sure that while updating network profile to add new
tenants, it does not delete the older profile bindings and maintains
them, while adds only the new tenant ids to the bindings.
Isaku Yamahata [Thu, 16 Oct 2014 06:31:03 +0000 (15:31 +0900)]
l3_agent: avoid name conflict with context
module name, context, conflicts with argument name in many place in
l3_agent. In order to avoid such conflict, import context as n_context
following Neutron practice.
Numan Siddique [Sat, 11 Oct 2014 12:08:05 +0000 (17:38 +0530)]
Fix KeyError in dhcp_rpc when plugin.port_update raise exception
KeyError exception is seen because of following reasons
* DhcpRpcCallback._port_action() is called by two functions
- DhcpRpcCallback.create_dchp_port()
- DhcpRpcCallback.update_dhcp_port()
* When create_dhcp_port() function calls _port_action(), the
function argument 'port' will have the body as
{'port': {'network_id': foo_network_id, 'fixed_ips': [..] ...}
* When update_dhcp_port() function calls _port_action(), the
function argument 'port' will have the body as
{'id': port_id, 'port': {{'port': {'network_id': foo_network_id,
'fixed_ips': [..] ...}}
* If an exception occurs when _port_action() calls plugin.create_port(),
network id is accessed as
net_id = port['port']['network_id']
* If an exception occurs when _port_action() calls plugin.update_port(),
network id is accessed as
net_id = port['port']['network_id']
which is causing the KeyError. network_id should have been accessed as
net_id = port['port']['port']['network_id']
This patch fixes the issue by making the _port_action() take the
same port body. update_dhcp_port() insteading of passing the port_id
and port information in a single argument, it now adds port_id
in the port body itself.
Kevin Benton [Sat, 11 Oct 2014 10:42:47 +0000 (03:42 -0700)]
Call DVR VMARP notify outside of transaction
The dvr vmarp table update notification was being called inside
of the delete_port transaction in ML2, which can cause a yield
and lead to the glorious mysql/eventlet deadlock.
This patch moves it outside the transaction and adjusts it to
use an existing port dictionary rather than re-looking it up since
the port is now gone from the DB by the time it is called.
With l2pop enabled, race exists in delete_port_postcommit
when both create/update_port and delete_port deal with
different ports on the same host, where such ports are
either the first (or) last on same network for that host.
This race happens outside the DB locking zones in
the respective methods of ML2 plugin.
To fix this, we have moved determination of
fdb_entries back to delete_port_postcommit and removed
delete_port_precommit altogether from l2pop mechanism
driver. In order to accomodate dvr interfaces, we
are storing and re-using the mechanism-driver context
which hold dvr-port-binding information while
invoking delete_port_postcommit. We loop through
dvr interface bindings invoking delete_port_postcommit
similar to delete_port_precommit.
Kevin Benton [Tue, 14 Oct 2014 06:40:36 +0000 (23:40 -0700)]
Minor: remove unnecessary intermediate variable
Removes an unnecessary intermediary variable and an
unnecessary list extend operation that implied previous
list members where there weren't any. There should be no
functional change. This just improves readability slightly.
Xu Han Peng [Fri, 20 Jun 2014 06:59:53 +0000 (14:59 +0800)]
Use EUI64 for IPv6 SLAAC when subnet is specified
This commit uses EUI64 for SLAAC and stateless IPv6 address
when subnet id in fixed_ip is specified.
After this patch, all the ports created on a subnet which has
ipv6_address_mod=slaac or ipv6_address_mod=dhcpv6-stateless
will use EUI64 as the address.
This patch also checks if fixed IP address is specified
for a IPv6 subnet with address mode slaac or dhcpv6-stateless
during creating or updating a port. If yes, raise InvalidInput
error to stop the port creation or update.
Remove unit test test_generated_duplicate_ip_ipv6 because
fixed_ip should not be specified for a slaac subnet.
Arista L3 Ops is success if it is successful on one peer
This fix checks to see if Arista HW is
configured in MLAG (redundant) mode. If yes,
as long as operation is successful on one of the
paired switches, consider it successful.
first_ip, allocation_pool_id and last_ip, allocation_pool_id
should be unique in the table.
These constraints are essential to detect concurrent modifications
of the IpAvailabilityRange table if the SELECT ... FOR UPDATE
lock is removed
Andrew Boik [Fri, 10 Oct 2014 17:13:45 +0000 (13:13 -0400)]
Update VPN logging to use new i18n functions
For log messages in neutron/services/vpn and neutron/db/vpn, replace
_() marker functions with log-level-specific marker functions: _LI(),
_LW(), _LE() from oslo.i18n.
Also, remove _() functions for debug log messages as debug level log
messages should not be translated.
Jacek Swiderski [Tue, 23 Sep 2014 12:35:06 +0000 (14:35 +0200)]
mock.assert_called_once() is not a valid method
mock.assert_called_once() is a no-op that tests nothing. Instead
mock.assert_called_once_with() should be used (or use
assertEqual(1, mock_obj.call_count) if you don't want to check
parameters).
Borrowed HACKING rule from Davanum Srinivas's nova patch to
prevent it from appearing again.
Angus Lees [Thu, 28 Aug 2014 04:37:34 +0000 (14:37 +1000)]
Compare subnet length as well when deleting DHCP entry
When searching for the DHCP subnet entry to delete, the existing code
compares only the subnet prefix and ignores subnet length. This could
delete the wrong entry/entries if nested subnets are present.
Angus Lees [Wed, 20 Aug 2014 03:38:14 +0000 (13:38 +1000)]
Add pylint tox environment and disable all existing warnings
pylintrc update disables all warnings that currently trigger on neutron
code. The rough plan is to slowly re-enable warning categories as we
clean up code in question.
This change also includes a few ultra-trivial syntax cleanups where it
allowed the check to be immediately enabled for the rest of the
codebase:
- Added missing trailing newlines in several files
(db/migration/__init__.py, nuage/{nuagedb,syncmanager,common/config}.py)
- Renamed self to cls in @classmethods
(cisco/db/l3/device_handling_db.py)
- Removed whitespace around '=' in a kwarg
(cisco/db/l3/device_handling_db.py, cisco/db/n1kv_db_v2.py)
- Updated deprecated pylint 'disable-msg' directive to newer 'disable'
(cisco/extensions/qos.py)
- File-specific disable for too-many-format-args pending further
investigation of alternatives
(ml2/drivers/arista/arista_l3_driver.py)
- Import module rather than object and avoid long line
(services/l3_router/l3_arista.py)
Within ovs agent daemon loop, prepare_devices_filter will impose heavy workloads
to neutron server in order to retrieve the security groups message to apply
firewall rules. If agent is configured to use Noopfirewall driver or security
groups are disabled, there is no need for loading the rules from server and
refreshing the firewalls. This will reduce the number of db calls and improve
performance for neutron server in this case.
NSX: drop support to deprecated dist-router extension
The NSX plugin originally implemented its own 'dist-router' extension.
During the Juno timeframe the DVR extension was introduced and the NSX
plugin was ported to support both. At the same time 'dist-router' was
marked for removal in Kilo.
Now that Kilo opened, we can drop the deprecated one.
Carl Baldwin [Thu, 9 Oct 2014 20:33:23 +0000 (20:33 +0000)]
Remove an argument that is never used
This code was creating a dict with a gw_exists key. I was curious to
know who was interested in receiving this information down the line.
However, no one is ever interested in that key. It took me some time
to follow this through wondering what was going on and found only dead
ends.
Code Review / openstack-build / neutron-build.git / log