Carl Baldwin [Thu, 6 Mar 2014 23:57:11 +0000 (23:57 +0000)]
L3 agent prefers RPC messages over full sync
When the L3 agent starts up and runs the sync task it doesn't process
any incoming RPC events until the sync task is complete.
This change combines the work from _rpc_loop and _sync_routers_task in
to a single loop called _process_routers_loop. This loop spawns
threads that pull from a priority queue. The queue ensures that RPC
messages are handled before _process_routers_loop. The latter is
generally maintenance tasks triggered by the agent rather than user
triggered tasks.
Synchronization between RPC and sync routers loops is no longer
necessary since they both feed in to a single queue. There were
places where it was necessary to reorder some things to allow for the
lack of synchronization. For example, it is necessary to list
namespaces before fetching the full list of routers to ensure that it
doesn't delete a new namespace that gets created after listing
namespaces. The lack of the need for synchronization between loops is
probably the main strength of this patch.
With multiple worker threads, need to handle the case where an RPC
message came in while a thread was working on a router. Another
thread should not handle the same router that is already in progress.
Adds a mechanism to signal to the working thread that an update came
in for the router it is working on. The original thread will repeat
processing the router when it is finished to get the update.
Multiple rapid updates to the same router will be consolidated.
Essentially, there is still synchronization of work for a given router
but not between routers. Much better than before.
Maruti [Sat, 7 Jun 2014 12:42:45 +0000 (05:42 -0700)]
Fixed audit notifications for l3-agent-router ops
l3-agent-router-add and l3-agent-router-remove do not
generate audit notifications which are used for security
compliance.
CRUD operations of core network resources are
handled by neutron/api/v2/base.py. In base.py, each of
create(), update(), delete() methods makes calls to
oslo.messaging.Notifier.info() to generate these notifications.
In the proposed fix, it is fixed in a similar fashion in
extensions/l3agentscheduler.py inside create() and delete()
methods by introducing info() method calls inside them.
Steven Hillman [Tue, 8 Jul 2014 22:35:10 +0000 (15:35 -0700)]
Send network name and uuid to subnet create
Added the network segment name and uuid parameters to the infomation sent to
Cisco N1kV during subnet creation to allow for proper association of the
subnet and network segment.
Robert Kukura [Wed, 12 Mar 2014 01:54:35 +0000 (21:54 -0400)]
ML2: Bind ports outside transactions
The ML2 plugin now calls the bind_port() operation on the registered
mechanism drivers outside of any enclosing DB transaction. Ports are
created or updated in one transaction, then a binding is established
if possible, and finally a second transaction commits the binding
result.
With [re]binding moved outside the DB transaction that triggered it,
it is now possible that multiple threads or processes will
concurrently try to bind the same port, or that the port will be
updated between transactions. Concurrent attempts to bind the same
port are allowed to proceed, which results are used is resolved in the
second transaction, and binding is retried if necessary.
Improvements to the Cisco Nexus driver and unit tests from Rich Curran
needed due to the binding changes are also included.
Jakub Libosvar [Thu, 26 Jun 2014 14:56:32 +0000 (16:56 +0200)]
Freeze models for healing migration
Take a snapshot of all models from the code base at the time when the
healing migration merges. The healing migration needs this frozen view
of the models to be available (even as the models change in the future)
to compare with the current DB schema. The healing migration will use
this comparison to heal the schema.
While adding iptables rule, cidr is added as destination for both ingress
and egress directions. Modified code to add -s for egress and -d for ingress.
Assaf Muller [Sun, 4 May 2014 15:26:43 +0000 (18:26 +0300)]
Change all occurences of no_delete to do_delete
Previously, ports, networks and subnets had a do_delete=True
parameter. By default, these resources were deleted at the
end of the context manager scope. All other resources used
a different semantic: no_delete=False.
This caused confusing situations such as:
with self.subnet(network, do_delete=False) as subnet:
with self.security_group(no_delete=True) as sg:
pass
Now all resources use the same do_delete semantic.
Joe Gordon [Fri, 11 Jul 2014 18:02:54 +0000 (18:02 +0000)]
Revert "VPNaaS REST Client UT Broken"
I8badc249ad021fdbdb2367b5416c72435ed58994 causes anything importing
neutron/tests/unit/services/vpn/device_drivers/_test_cisco_csr_rest.py
to exit since httmock isn't a dependency.
Fix all hacking issues in addition to the revert, as this patch fixes 'tox -epep8'.
And just a reverting the patch will cause pep8 to fail.
If a model mentions 'default' it is better to set 'server_default'
as well because this 'defaut' can be reached only via code. This
change adds 'server_default' parameter to models and in database,
so if someone will use SQL query it will also use default value.
For model Router added parameter server_default. In migration it was
added by this change https://review.openstack.org/105212.
For model MeteringLabelRule added parameter server_default.
In migration it will be added by heal script due to problems
with Metering plugin and grenade.
db_base_plugin_v2 imports too much modules that are not necessary
usually, so extract CommonDBMixin in different file.
Plus using db_base_plugin_v2 for some types of modules can lead to
cycles in imports, this refactoring should resolve the issue.
NSX: neutron router-interface-add should clear security-groups
NSX does not support security groups on router ports so in the case
where someone uses a port that has a security group on it as the router
port we need to clear the security group off the port.
Currently treat_devices_added_or_updated in the OVS agent skips
processing devices which disappeared from the integration bridge
during the agent loop.
This is fine, however the agent should not mark these devices as
processed. Otherwise they won't be processed, should they appear
again on the bridge.
This patch ensures these devices are not added to the current
device set.
The patch also changes treat_devices_added_or_updated. The
function now will return the list of skipped devices and not
anymore a flag signalling whether a resync is required.
With the current logic a resync would be required if retrieval
of device details fails. With this change, the function
treat_devices_added_or_updated will raise in this case and the
exception will be handled in process_network_ports.
For the sake of consistency, this patch also updates the
similar function treat_ancillary_devices_added in order to
use the same logic.
Finally, this patch amends an innaccurate related comment.
Ann Kamyshnikova [Fri, 21 Mar 2014 08:33:30 +0000 (12:33 +0400)]
Fix 'server_default' parameter usage in models
In ml2 models parameter 'default' is used for vnic_type, profile and
vif_details, but in migrations 27cc183af192_ml2_vnic_type,
157a5d299379_ml2_binding_profile and
50d5ba354c23_ml2_binding_vif_details is used 'server_default' parameter.
Usage 'default' and 'server_default' should be equal in models and
migration. So models in models is added 'server_default' parameter.
Paul Michali [Mon, 30 Jun 2014 22:31:42 +0000 (18:31 -0400)]
VPNaaS REST Client UT Broken
When the H302 work was done, it broke two modules used for UT of
Cisco CSR VPNaaS REST client. Since the UT is currently disabled
(because the httmock library was not approved for test-requirements),
the error was not detected.
This fixes the files, so that they can still be used manually (by
locally importing httmock or its source file, and renaming the
module to remove the "_" prefix).
In the long term, the UT needs to be reworked to use httpretty,
which uses a register based mechanism, instead of a context
manager based mechanism, so this will take some effort. In addition.
there is discussion of switching to a new mock-request package,
instead of httpretty.
Ihar Hrachyshka [Wed, 18 Jun 2014 14:56:25 +0000 (16:56 +0200)]
Avoid notifying while inside transaction opened in delete_port()
delete_port() calls to disassociate_floatingips() while in transaction.
The latter method sends RPC notification which may result in eventlet
yield. If yield switches a thread to another one that tries to access
the same floating IP object in db as disassociate_floatingips() method
does, we're locked and get db timeout.
We should avoid calling to notifier while under transaction.
To achieve this, I introduce a do_notify argument that controls whether
notification is done by disassociate_floatingips() itself or delegated
to caller. Callers that call to disassociate_floatingips() from under
transactions should handle notifications on their own. For this,
disassociate_floatingips() returns a set of routers that require
notification.
Updated drivers to reflect new behaviour. Added unit test.
CANARY_TABLE is for br-int, not for br-tun. Grouping it with
constants for br-tun tables together is misleading. Move it into
a separate section and comment appropriately. No functional changes.
Fix UnboundLocalError raised during L3 router sync task
This can be fixed in a number of ways: a) consolidating the
two except clauses into one; b) adding a 'return' after the
last except clause c) by calling the cleanup method only on
success; d) initializing 'routers' before usage.
Approach c) has the benefit of stating the developer's intent
more explicitly and minimize chances of regression.
self.assertIsInstance is preferred over assert(isinstance(..)). See feedback in:
https://review.openstack.org/#/c/102754/1/neutron/tests/unit/test_extension_security_group.py
Previously, sudo-requiring functional tests hardcoded the use of
'sudo' as the root helper. Devstack gate jobs do not allow
password-less invocation of 'sudo', though, so such tests were unable
to run in the gate. This patch adds the ability to configure the
rootwrap command installed by devstack by setting the OS_ROOTWRAP_CMD
environment variable in the test execution environment, allowing
sudo-requiring tests to run.
Fix OVSBridge.get_port_ofport to handle empty output
Previously ovs_lib.OVSBridge.get_port_ofport() was not handling the
case where db_get_val() was returning None when no output was recieved
from ovs-vsctl. This patch ensures that the TypeError that results
from this condition is appropriately handled.
Henry Gessau [Tue, 1 Jul 2014 21:56:49 +0000 (17:56 -0400)]
Ignore variable column widths in ovsdb functional tests
The ovsdb tests check the output of "ovsdb-client monitor" before and after
some operations. The first line of output is a header row with spaces for
column separation. The column widths can vary depending on the data in the
columns, so compress multiple spaces to one for testing.
Kevin Benton [Tue, 1 Jul 2014 13:03:19 +0000 (06:03 -0700)]
BSN: Remove db lock and add missing contexts
Adds context tracking decorators that were missing
from router interface methods. Without them, new
sessions were being created instead of using the
existing context which was causing transaction
issues.
Modifies the servermanager to store context references
as weakrefs so if multiple functions are called before
the rest functions are called, the first one doesn't steal
the only context reference with a pop() call.
Removes a DB lock for update in the server manager that occured
during rest calls that was triggering deadlocks due to the
file lock synchronization for the rest calls.
Ensure the floating IP status is put ACTIVE or DOWN when it
is associated or disassociated.
This is done directly on the plugin class as the NSX backend
applies relevant NAT rules synchronously.
Ralf Haferkamp [Mon, 23 Jun 2014 12:27:55 +0000 (14:27 +0200)]
Fix example for running individual tests
The ':' syntax doesn't seem to work anymore. Additionally fix the "tox" call to
not try to run it from the current working directory. It's usually installed
somewhere in $PATH.
Edgar Magana [Fri, 30 May 2014 23:27:35 +0000 (16:27 -0700)]
Stop the dhcp-agent process when dnsmasq version is not determined
When it is not possible to determinate the dnsmasq version
or dnsmasq is not installed, neutron shows only a warning message.
With this commit the dhcp-agent process will be stopped if the
dnsmasq service is not installed.
Modifies the dhcp version unit test with the expected SystemExit exception
Assaf Muller [Sat, 3 May 2014 14:05:04 +0000 (17:05 +0300)]
Replace occurences of 'test_tenant' with 'test-tenant' in tests
tenant_id will now be consistent among all resources created.
This is important when getting or listing resources by tenant_id.
Most resources are created under the 'test-tenant', while
metering and security groups were created under 'test_tenant'.
Code Review / openstack-build / neutron-build.git / log