Cedric Brandily [Mon, 16 Jun 2014 20:43:13 +0000 (22:43 +0200)]
Add partial specs support in ML2 for vlan provider networks
ML2 provider networks partial specs let admins choose some provider
network attributes and let neutron choose remaining attributes. This
change provides the implementation for VLAN provider networks.
In practice, for VLAN provider networks provider:physical_network
and provider:segmentation_id choices can be delegated to neutron,
in such case neutron will try to find a network in tenant network
pools which respects provided provider attributes.
DocImpact
Related to blueprint provider-network-partial-specs
Partial-Bug: #1330562
Background:
ML2 plugin sometimes uses truncated port uuids.
For example, in the case of ofagent and linuxbridge,
if port id is 804ceaa1-0e3e-11e4-b537-08606e7f74e7,
an agent would send "tap804ceaa1-0e" to the plugin.
ML2 plugin's _device_to_port_id() would restore it to
"804ceaa1-0e". While it's still truncated, ML2 plugin's
get_port() handles that by using "startswith".
The recently merged DVR change (https://review.openstack.org/#/c/102332/)
assumes that port_id is always a complete uuid (it's the case
for openvswitch) and fails to handle the above mentioned case.
This commit fixes the regression.
This patch introduces the RPC contract changes
required for both the server (plugin) and agent
to propagate and retrieve additional information
about Distributed Routers, like MAC addresses
and Port Bindings.
Allow to import _LC, _LE, _LI and _LW functions directly
_LC, _LE, _LI, _LW are translation helper functions. This change updates
hacking rules to allow importing them directly:
from neutron.openstack.common.gettextutils import _LC
from neutron.openstack.common.gettextutils import _LE
from neutron.openstack.common.gettextutils import _LI
from neutron.openstack.common.gettextutils import _LW
neutron.policy module has been updated as an example of _LE, _LI and
_LW uses.
This patch introduces the models, the DB migrations
and the config options required by the L2 layer to
support DVR east/west traffic.
These changes will be used by the control-plane made
of ML2, L2pop and L2 agent.
Two new configuration options have been introduced:
'dvr_base_mac' is used to set DVR MAC addresses apart
from tenant ones (every distributed router will have
ports being created on compute hosts) and
'enable_distributed_routing' is used to enable dvr
support in the L2 agent. This gives the capability of
rolling out the dvr functionality in stages.
Kevin Benton [Mon, 9 Jun 2014 19:20:18 +0000 (12:20 -0700)]
Big Switch: Remove consistency hash on full sync
Clears the consistency hash before performing a full
backend synchronization. Since all of the data is being
replaced, there is no reason to include this and it
currently requires special-casing to handle.
Paul Michali [Mon, 23 Jun 2014 20:42:41 +0000 (16:42 -0400)]
VPNaaS: separate out validation logic for ref impl
Modify the persistence logic for VPNaaS to separate out the
validation for the reference implmentation. This allows
providers to override/extend the validation, as needed.
Additional commits will address the separation for other
providers and for other L3 services.
In addition, the logic that sets default values for optional
attributes is also moved into separate methods to allow
providers to override the defaults used.
Only attributes that have multiple validations are considered
at this time, but this can be extended, as needed.
Currently, one UT fails due to an oslo.messaging bug that
has been fixed, but not yet released for Neutron use.
As a workaround, the affected test case has been commented
out. This test verifies that the right (only, in this case)
device driver is used for validation. Once the issue with
releasing oslo.messaging 1.4.0.0a2 or newer is resolved,
this can be uncommented.
In the future, TaskFlow may be used for the L3 services workflow,
and these changes will provide validation modularization for that
effort.
This patch introduces the model and extension
framework for implementing distributed virtual
routing on top of Open vSwitch.
A new admin-only 'distributed' (as opposed to a
'centralized' or legacy-mode) attribute is added
to the API router resource. It is possible to convert
an existing (centralized) router to a distributed
one; the opposite conversion, even though allowed by
the API, may not be honored by the underlying
plugin implementation and an appropriate error will
be reported.
When creating a router (regardless of the user role),
Neutron will rely on a system wide configuration, whose
default currently allows to create 'centralized' routers.
Tests are added for basic unit coverage; when the first
building blocks for neutron-testing-refactor
are complete, functional testing will be added.
This is because we should be moving away from how
extension tests have been done up until now.
Paul Michali [Mon, 7 Jul 2014 22:57:43 +0000 (18:57 -0400)]
VPNaaS Cisco REST client enhance CSR create
For the create of the REST client object that represents a Cisco CSR,
all of the info needed were passed in as separate parameters. This
change just uses a dict instead, so that additional parameters can
be added w/o changing the API.
Updated the currently unused UT module, just so that it can be used
locally and stays up-to-date until it can be converted to use the
new requests-mock package.
Fixes port update failure when device ID is not updated
The updation was failing because the changed_device_id variable
was undeclared.This fix declares the variable with a default value
at the beginning of the method.
Carl Baldwin [Thu, 6 Mar 2014 23:57:11 +0000 (23:57 +0000)]
L3 agent prefers RPC messages over full sync
When the L3 agent starts up and runs the sync task it doesn't process
any incoming RPC events until the sync task is complete.
This change combines the work from _rpc_loop and _sync_routers_task in
to a single loop called _process_routers_loop. This loop spawns
threads that pull from a priority queue. The queue ensures that RPC
messages are handled before _process_routers_loop. The latter is
generally maintenance tasks triggered by the agent rather than user
triggered tasks.
Synchronization between RPC and sync routers loops is no longer
necessary since they both feed in to a single queue. There were
places where it was necessary to reorder some things to allow for the
lack of synchronization. For example, it is necessary to list
namespaces before fetching the full list of routers to ensure that it
doesn't delete a new namespace that gets created after listing
namespaces. The lack of the need for synchronization between loops is
probably the main strength of this patch.
With multiple worker threads, need to handle the case where an RPC
message came in while a thread was working on a router. Another
thread should not handle the same router that is already in progress.
Adds a mechanism to signal to the working thread that an update came
in for the router it is working on. The original thread will repeat
processing the router when it is finished to get the update.
Multiple rapid updates to the same router will be consolidated.
Essentially, there is still synchronization of work for a given router
but not between routers. Much better than before.
Bertrand Lallau [Wed, 11 Jun 2014 13:46:58 +0000 (15:46 +0200)]
Dnsmasq config files syntax issue when dhcp_domain is empty
When using dhcp-agent with the following property dhcp_domain=""
addn_hosts and host files genererated for Dnsmasq have wrong syntax
concerning hostname parameter (dot char at the end).
As described in RFC-952 the hostname grammar is as follows:
<hname> ::= <name>*["."<name>]
<name> ::= <let>[*[<let-or-digit-or-hyphen>]<let-or-digit>]
Hence it can't be conclude with a dot char.
Dnsmasq process is waiting for a hostname (2nd parameter) in the
following files:
"addn_hosts" file contains:
20.0.0.3 host-20-0-0-3. host-20-0-0-3
"host" file contains:
fa:16:3e:bf:e1:e4,host-20-0-0-3.,20.0.0.3
With the patch you will get:
"addn_hosts":
20.0.0.3 host-20-0-0-3 host-20-0-0-3
"host":
fa:16:3e:bf:e1:e4,host-20-0-0-3,20.0.0.3
Maruti [Sat, 7 Jun 2014 12:42:45 +0000 (05:42 -0700)]
Fixed audit notifications for l3-agent-router ops
l3-agent-router-add and l3-agent-router-remove do not
generate audit notifications which are used for security
compliance.
CRUD operations of core network resources are
handled by neutron/api/v2/base.py. In base.py, each of
create(), update(), delete() methods makes calls to
oslo.messaging.Notifier.info() to generate these notifications.
In the proposed fix, it is fixed in a similar fashion in
extensions/l3agentscheduler.py inside create() and delete()
methods by introducing info() method calls inside them.
Steven Hillman [Tue, 8 Jul 2014 22:35:10 +0000 (15:35 -0700)]
Send network name and uuid to subnet create
Added the network segment name and uuid parameters to the infomation sent to
Cisco N1kV during subnet creation to allow for proper association of the
subnet and network segment.
Robert Kukura [Wed, 12 Mar 2014 01:54:35 +0000 (21:54 -0400)]
ML2: Bind ports outside transactions
The ML2 plugin now calls the bind_port() operation on the registered
mechanism drivers outside of any enclosing DB transaction. Ports are
created or updated in one transaction, then a binding is established
if possible, and finally a second transaction commits the binding
result.
With [re]binding moved outside the DB transaction that triggered it,
it is now possible that multiple threads or processes will
concurrently try to bind the same port, or that the port will be
updated between transactions. Concurrent attempts to bind the same
port are allowed to proceed, which results are used is resolved in the
second transaction, and binding is retried if necessary.
Improvements to the Cisco Nexus driver and unit tests from Rich Curran
needed due to the binding changes are also included.
Jakub Libosvar [Thu, 26 Jun 2014 14:56:32 +0000 (16:56 +0200)]
Freeze models for healing migration
Take a snapshot of all models from the code base at the time when the
healing migration merges. The healing migration needs this frozen view
of the models to be available (even as the models change in the future)
to compare with the current DB schema. The healing migration will use
this comparison to heal the schema.
While adding iptables rule, cidr is added as destination for both ingress
and egress directions. Modified code to add -s for egress and -d for ingress.
Assaf Muller [Sun, 4 May 2014 15:26:43 +0000 (18:26 +0300)]
Change all occurences of no_delete to do_delete
Previously, ports, networks and subnets had a do_delete=True
parameter. By default, these resources were deleted at the
end of the context manager scope. All other resources used
a different semantic: no_delete=False.
This caused confusing situations such as:
with self.subnet(network, do_delete=False) as subnet:
with self.security_group(no_delete=True) as sg:
pass
Now all resources use the same do_delete semantic.
Edgar Magana [Thu, 5 Jun 2014 23:33:16 +0000 (16:33 -0700)]
Validate expected parameters in add/remove router interfaces
The add and remove router interface methods check that interface_info
is not empty but don't check if it contains any of expected parameters:
port_id and subnet_id
This patch adds a helper method to validate that interface_info contains
at least one of the expected parameters
Include a unit test for the empty port_id and subnet_id case
Joe Gordon [Fri, 11 Jul 2014 18:02:54 +0000 (18:02 +0000)]
Revert "VPNaaS REST Client UT Broken"
I8badc249ad021fdbdb2367b5416c72435ed58994 causes anything importing
neutron/tests/unit/services/vpn/device_drivers/_test_cisco_csr_rest.py
to exit since httmock isn't a dependency.
Fix all hacking issues in addition to the revert, as this patch fixes 'tox -epep8'.
And just a reverting the patch will cause pep8 to fail.
If a model mentions 'default' it is better to set 'server_default'
as well because this 'defaut' can be reached only via code. This
change adds 'server_default' parameter to models and in database,
so if someone will use SQL query it will also use default value.
For model Router added parameter server_default. In migration it was
added by this change https://review.openstack.org/105212.
For model MeteringLabelRule added parameter server_default.
In migration it will be added by heal script due to problems
with Metering plugin and grenade.
Kevin Benton [Thu, 10 Jul 2014 19:18:54 +0000 (12:18 -0700)]
Add BSN plugin to agent migration script
Adds the Big Switch plugin to the migration script that creates
the agents table. A recent commit (d3be7b040eaa61a4d0ac617026cf5c9132d3831e)
added it to the agent tables but it missed this one so an error was thrown
during migration when it reached a table that depended on the agents table.
db_base_plugin_v2 imports too much modules that are not necessary
usually, so extract CommonDBMixin in different file.
Plus using db_base_plugin_v2 for some types of modules can lead to
cycles in imports, this refactoring should resolve the issue.
Shiv Haris [Fri, 20 Jun 2014 22:05:10 +0000 (15:05 -0700)]
Added support for NOS version 4.1.0, 5.0.0 and greater
NETCONF temaplates for NOS version greater than 4.1.0 are slightly
different (argh). An init time check of the NOS version is done
to enable selection of the correct NETCONF templates.
NSX: neutron router-interface-add should clear security-groups
NSX does not support security groups on router ports so in the case
where someone uses a port that has a security group on it as the router
port we need to clear the security group off the port.
The functional gate jobs (*-neutron-dsvm-functional) need special
configuration (e.g. OS_SUDO_TESTING, OS_ROOTWRAP_CMD) to run tests.
This patch adds a new gate-specific tox env (tox -e dsvm-functional)
that sets the appropriate values and allows easy duplication of the
results of the upstream job on a host that has successfully run
devstack with neutron enabled.
A new configuration option, OS_FAIL_ON_MISSING_DEPS, is also set by
the dsvm-functional tox env to ensure that dependency failures will
result in failures rather than skips when run in the gate.
Currently treat_devices_added_or_updated in the OVS agent skips
processing devices which disappeared from the integration bridge
during the agent loop.
This is fine, however the agent should not mark these devices as
processed. Otherwise they won't be processed, should they appear
again on the bridge.
This patch ensures these devices are not added to the current
device set.
The patch also changes treat_devices_added_or_updated. The
function now will return the list of skipped devices and not
anymore a flag signalling whether a resync is required.
With the current logic a resync would be required if retrieval
of device details fails. With this change, the function
treat_devices_added_or_updated will raise in this case and the
exception will be handled in process_network_ports.
For the sake of consistency, this patch also updates the
similar function treat_ancillary_devices_added in order to
use the same logic.
Finally, this patch amends an innaccurate related comment.
Ann Kamyshnikova [Fri, 21 Mar 2014 08:33:30 +0000 (12:33 +0400)]
Fix 'server_default' parameter usage in models
In ml2 models parameter 'default' is used for vnic_type, profile and
vif_details, but in migrations 27cc183af192_ml2_vnic_type,
157a5d299379_ml2_binding_profile and
50d5ba354c23_ml2_binding_vif_details is used 'server_default' parameter.
Usage 'default' and 'server_default' should be equal in models and
migration. So models in models is added 'server_default' parameter.
Code Review / openstack-build / neutron-build.git / log