Kevin Benton [Wed, 24 Sep 2014 12:23:32 +0000 (05:23 -0700)]
Improve performance of security group DB query
The _select_ips_for_remote_group method was joining the
IP allocation, port, allowed address pair, and security group tables
together in a single query. Additionally, it was loading all of
the port columns and using none of them. This resulted in a
very expensive query with no benefit.
This patch eliminates the unnecessary use of the port table by joining
the IP allocation table directly to the security groups and allowed
address pairs tables. In local testing of the method, this sped it up
by an order of magnitude.
Numan Siddique [Sat, 11 Oct 2014 12:08:05 +0000 (17:38 +0530)]
Fix KeyError in dhcp_rpc when plugin.port_update raise exception
KeyError exception is seen because of following reasons
* DhcpRpcCallback._port_action() is called by two functions
- DhcpRpcCallback.create_dchp_port()
- DhcpRpcCallback.update_dhcp_port()
* When create_dhcp_port() function calls _port_action(), the
function argument 'port' will have the body as
{'port': {'network_id': foo_network_id, 'fixed_ips': [..] ...}
* When update_dhcp_port() function calls _port_action(), the
function argument 'port' will have the body as
{'id': port_id, 'port': {{'port': {'network_id': foo_network_id,
'fixed_ips': [..] ...}}
* If an exception occurs when _port_action() calls plugin.create_port(),
network id is accessed as
net_id = port['port']['network_id']
* If an exception occurs when _port_action() calls plugin.update_port(),
network id is accessed as
net_id = port['port']['network_id']
which is causing the KeyError. network_id should have been accessed as
net_id = port['port']['port']['network_id']
This patch fixes the issue by making the _port_action() take the
same port body. update_dhcp_port() insteading of passing the port_id
and port information in a single argument, it now adds port_id
in the port body itself.
Kevin Benton [Sat, 11 Oct 2014 10:42:47 +0000 (03:42 -0700)]
Call DVR VMARP notify outside of transaction
The dvr vmarp table update notification was being called inside
of the delete_port transaction in ML2, which can cause a yield
and lead to the glorious mysql/eventlet deadlock.
This patch moves it outside the transaction and adjusts it to
use an existing port dictionary rather than re-looking it up since
the port is now gone from the DB by the time it is called.
With l2pop enabled, race exists in delete_port_postcommit
when both create/update_port and delete_port deal with
different ports on the same host, where such ports are
either the first (or) last on same network for that host.
This race happens outside the DB locking zones in
the respective methods of ML2 plugin.
To fix this, we have moved determination of
fdb_entries back to delete_port_postcommit and removed
delete_port_precommit altogether from l2pop mechanism
driver. In order to accomodate dvr interfaces, we
are storing and re-using the mechanism-driver context
which hold dvr-port-binding information while
invoking delete_port_postcommit. We loop through
dvr interface bindings invoking delete_port_postcommit
similar to delete_port_precommit.
Kevin Benton [Tue, 14 Oct 2014 06:40:36 +0000 (23:40 -0700)]
Minor: remove unnecessary intermediate variable
Removes an unnecessary intermediary variable and an
unnecessary list extend operation that implied previous
list members where there weren't any. There should be no
functional change. This just improves readability slightly.
Xu Han Peng [Fri, 20 Jun 2014 06:59:53 +0000 (14:59 +0800)]
Use EUI64 for IPv6 SLAAC when subnet is specified
This commit uses EUI64 for SLAAC and stateless IPv6 address
when subnet id in fixed_ip is specified.
After this patch, all the ports created on a subnet which has
ipv6_address_mod=slaac or ipv6_address_mod=dhcpv6-stateless
will use EUI64 as the address.
This patch also checks if fixed IP address is specified
for a IPv6 subnet with address mode slaac or dhcpv6-stateless
during creating or updating a port. If yes, raise InvalidInput
error to stop the port creation or update.
Remove unit test test_generated_duplicate_ip_ipv6 because
fixed_ip should not be specified for a slaac subnet.
Arista L3 Ops is success if it is successful on one peer
This fix checks to see if Arista HW is
configured in MLAG (redundant) mode. If yes,
as long as operation is successful on one of the
paired switches, consider it successful.
first_ip, allocation_pool_id and last_ip, allocation_pool_id
should be unique in the table.
These constraints are essential to detect concurrent modifications
of the IpAvailabilityRange table if the SELECT ... FOR UPDATE
lock is removed
Andrew Boik [Fri, 10 Oct 2014 17:13:45 +0000 (13:13 -0400)]
Update VPN logging to use new i18n functions
For log messages in neutron/services/vpn and neutron/db/vpn, replace
_() marker functions with log-level-specific marker functions: _LI(),
_LW(), _LE() from oslo.i18n.
Also, remove _() functions for debug log messages as debug level log
messages should not be translated.
Angus Lees [Wed, 20 Aug 2014 03:38:14 +0000 (13:38 +1000)]
Add pylint tox environment and disable all existing warnings
pylintrc update disables all warnings that currently trigger on neutron
code. The rough plan is to slowly re-enable warning categories as we
clean up code in question.
This change also includes a few ultra-trivial syntax cleanups where it
allowed the check to be immediately enabled for the rest of the
codebase:
- Added missing trailing newlines in several files
(db/migration/__init__.py, nuage/{nuagedb,syncmanager,common/config}.py)
- Renamed self to cls in @classmethods
(cisco/db/l3/device_handling_db.py)
- Removed whitespace around '=' in a kwarg
(cisco/db/l3/device_handling_db.py, cisco/db/n1kv_db_v2.py)
- Updated deprecated pylint 'disable-msg' directive to newer 'disable'
(cisco/extensions/qos.py)
- File-specific disable for too-many-format-args pending further
investigation of alternatives
(ml2/drivers/arista/arista_l3_driver.py)
- Import module rather than object and avoid long line
(services/l3_router/l3_arista.py)
NSX: drop support to deprecated dist-router extension
The NSX plugin originally implemented its own 'dist-router' extension.
During the Juno timeframe the DVR extension was introduced and the NSX
plugin was ported to support both. At the same time 'dist-router' was
marked for removal in Kilo.
Now that Kilo opened, we can drop the deprecated one.
Carl Baldwin [Thu, 9 Oct 2014 20:33:23 +0000 (20:33 +0000)]
Remove an argument that is never used
This code was creating a dict with a gw_exists key. I was curious to
know who was interested in receiving this information down the line.
However, no one is ever interested in that key. It took me some time
to follow this through wondering what was going on and found only dead
ends.
Carl Baldwin [Thu, 2 Oct 2014 20:35:21 +0000 (20:35 +0000)]
Refactor _process_routers to handle a single router
The method _process_routers no longer handles multiple routers. The
only caller of this method would construct a list of exactly one
router in order to make the call. This made the for loop unnecessary.
The method's logic is too heavy for its current purpose. This commit
removes much of the weight.
The use of the sets in this method is also no longer necessary. It
became clear that all of it boiled down to "if the router is not
compatible with with this agent but it is known in router_info from
before then we need to remove it." This is an exceptional condition
that shouldn't be handled in this method so I raise an exception and
handle it in process_router_update where other router removal is
handled. Logging was added for this exceptional condition.
The eventlet pool was also obsolete. It was used to spawn two methods
and there was a waitall at the end. The other refactoring made it
clear that the two spawns were mutually exclusive. There was only one
thread spawned for any given invocation of the method and the eventlet
pool is overkill.
Mark McClain [Wed, 8 Oct 2014 18:49:20 +0000 (18:49 +0000)]
Add database relationship between router and ports
Add an explicit schema relationship between a router and its ports. This
change ensures referential integrity among the entities and prevents orphaned
ports.
Henry Gessau [Wed, 8 Oct 2014 00:38:38 +0000 (20:38 -0400)]
Disable PUT for IPv6 subnet attributes
In Juno we are not ready for allowing the IPv6 attributes on a subnet
to be updated after the subnet is created, because:
- The implementation for supporting updates is incomplete.
- Perceived lack of usefulness, no good use cases known yet.
- Allowing updates causes more complexity in the code.
- Have not tested that radvd, dhcp, etc. behave OK after update.
Therefore, for now, we set 'allow_put' to False for the two IPv6
attributes, ipv6_ra_mode and ipv6_address_mode. This prevents the
modes from being updated via the PUT:subnets API.
Note: There are several other unrelated unit tests that also break with a
randomized PYTHONHASHSEED, but they are not addressed here. They will be
addressed in separate patches.
Assaf Muller [Tue, 7 Oct 2014 19:45:41 +0000 (22:45 +0300)]
Forbid update of HA property of routers
While the HA property is update-able, and resulting router-get
invocations suggest that the router is HA, the migration
itself fails on the agent. This is deceiving and confusing
and should be blocked until the migration itself is fixed
in a future patch.
Brian Haley [Fri, 3 Oct 2014 21:32:01 +0000 (17:32 -0400)]
Teach DHCP Agent about DVR router interfaces
When DVR is enabled and enable_isolated_metadata=True,
the DHCP agent should only inject a metadata host route
when there is no port with the gateway IP address configured
on the subnet. Add a check for DEVICE_OWNER_DVR_INTERFACE
when we look at each port's device_owner field, otherwise
it will always add this route to the opts file when DVR
is enabled.
Miguel Angel Ajo [Fri, 19 Sep 2014 16:59:58 +0000 (18:59 +0200)]
Modify the ProcessMonitor class to have one less config parameter
It's a follow up patch, as agreed on the ProcessMonitor review
patch to coalesce the check_child_processes parameter into
check_child_process_interval.
When this parameter is set to 0, the feature is disabled.
Kevin Benton [Tue, 7 Oct 2014 11:34:41 +0000 (04:34 -0700)]
Big Switch: Don't clear hash before sync
This patch removes the step of clearing the consistency
hash from the DB before a topology sync. This will ensure
that inconsistency will be detected if the topology sync
fails.
This logic was originally there to make sure the hash header
was not present on the topology sync call to the backend.
However, the hash header is ignored by the backend in a sync
call so it wasn't necessary.
John Schwarz [Tue, 23 Sep 2014 12:24:47 +0000 (15:24 +0300)]
Divide _cleanup_namespaces for easy extensibility
This division of the function to 2 different functions allows for
easier overwriting in the l3 test agent used by the HA functional
tests, and later by the integration tests.
John Schwarz [Tue, 23 Sep 2014 11:41:54 +0000 (14:41 +0300)]
L3 Agent should generate ns_name in a single place
Currently the l3 agent has 2 places where it allows generating ns_name
of specific router_ids (ie. qrouter-<router_id>): in the RouterInfo's
constructor, and in _cleanup_namespaces. This patch proposes a
unification of this creation code with a property which lives in
RouterInfo's namespace. A simpler fix was also made for snat_ns_name.
This patch also offers a single way to initialize a new RouterInfo.
John Schwarz [Mon, 29 Sep 2014 13:28:18 +0000 (16:28 +0300)]
L3 agent should always use a unique CONF object
The l3 agent accepts an oslo configuration in its constructor and uses
it throughout the code, but there are some references to the global
configuration object held by the oslo library. Since HA functional
tests need to create two agents, the configuration should be consistent
throughout the code.
The important difference between the agents is their 'host' value so
that they create different namespaces, and 'state_path' value so the
agents get their own filesystem root.
Code Review / openstack-build / neutron-build.git / log