Ihar Hrachyshka [Wed, 13 Jan 2016 11:37:21 +0000 (12:37 +0100)]
Make security_groups_provider_updated work with Kilo agents
Initially, we bumped the required version for the agent endpoint from
1.1 (the initial version that implemented security groups) to 1.3
without considering that the code should work with old agents that do
not yet know about the new devices_to_update argument.
Actually, there was no need to bump the version: old agent side code
already captures all unknown arguments that could be passed from the
server, ignoring them:
(^ the link shows the latest Kilo code as of writing)
Note: some people may argue that the approach that is taken in Neutron
to support backwards compatibility for server notifications is wrong,
and we instead should adopt some stricter mechanism like nova version
pinning. While that is a noble thing to do, it's out of scope for the
patch that is designed to be easily backportable to stable/liberty.
Note: some people may also argue that the patch should go straight into
stable/liberty because we don't claim support for rolling upgrade
scenarios that span multiple releases. That's indeed true, though my
take on it is that if we have a way to handle more unofficial scenarios
without more coding effort, it's worth doing it.
Oleg Bondarev [Thu, 17 Dec 2015 11:57:21 +0000 (14:57 +0300)]
Introduce new query to return all hosts for DVR router
When we remove explicit binding of dvr routers to compute nodes
we'll need a way to know all hosts where a dvr router should be
hosted in order to send notifications.
This patch adds such a query and updates l3 rpc notifier to use it.
lzklibj [Wed, 28 Oct 2015 09:02:11 +0000 (17:02 +0800)]
fix get_ha_sync_data_for_host for non-dvr agent
get_ha_sync_data_for_host will call _get_dvr_sync_data
even given agent is not in DVR agent_mode.
_get_dvr_sync_data has additional processing for dvr
like:
dvr_router_ids = set(router['id'] for router in routers
if is_distributed_router(router))
floating_ip_port_ids = [fip['port_id'] for fip in floating_ips
if fip['router_id'] in dvr_router_ids]
which should only work for DVR scenario.
That will increase neutron-server processing time for
non-DVR scenario. Adding logic to judge whether given
agent is in DVR agent_mode, call get_sync_data directly.
Ihar Hrachyshka [Thu, 14 Jan 2016 15:06:20 +0000 (16:06 +0100)]
dhcp: handle advertise_mtu=True when plugin does not set mtu values
DHCP agent may be used by plugins that don't set mtu value for networks.
Handle the case by not passing the DHCP option when network does not
have the value set.
Most plugins do set the value though, since it's enforced in base db
plugin class.
Kevin Benton [Fri, 15 Jan 2016 04:23:32 +0000 (20:23 -0800)]
Retry port create/update on duplicate db records
Even though we have a duplicate IP check before creating an IP
allocation, there is a window between when that check is completed
and when the record is committed to the database. During this window,
another thread or server may use the same IP address, which will result
in the database commit failing with a DBDuplicateEntry exception.
This patch adds a decorator to convert the exception into a
RetryRequest, which will be caught at the API layer to start the
operation over, at which point the normal duplicate check will find
the IP and return the correct error to the user.
This was done instead of trying to convert the DBDuplicate to the
correct exception since it's on commit and there are possibly many
different sources of the duplicate record beyond the IP allocation
(depending on what mechanism drivers did during pre-commit, etc).
By retrying the request, we ensure that the prechecks run again that
will raise the appropriate exception.
Kevin Benton [Fri, 15 Jan 2016 00:15:38 +0000 (16:15 -0800)]
Make neutron pecan server an option instead of binary
Launch pecan server instead of home-grown wsgi server using
a new config option. This will make it easier to test out
pecan without invasive changes to devstack.
Related Blueprint: wsgi-pecan-switch
Change-Id: I99261e6bfc9b16c0d601828f97553a9192804216
Added Keystone and RequestID headers to CORS middleware
CORS middleware's latent configuration feature, new in 3.0.0,
allows adding headers that apply to all valid origins.
This patch adds headers commonly used in openstack to neutron's paste
pipeline, so that operators do not have to be aware of additional
configuration magic to ensure that browsers can talk to the API.
For more information:
http://docs.openstack.org/developer/oslo.middleware/cors.html#configuration-for-pastedeploy
lzklibj [Sun, 27 Dec 2015 06:08:36 +0000 (14:08 +0800)]
Unify exceptions for assign router to dvr agent
validate_agent_router_combination use two different exceptions
for assigning a router to an agent in 'dvr' mode:
RouterL3AgentMismatch: assign dvr router to legacy agent.
DVRL3CannotAssignToDvrAgent: assign dvr router to (another) dvr agent.
This should be unified to one single exception, for routers on agent in
'dvr' mode should be only scheduled, not allowed to be manually assigned.
Dustin Lundquist [Wed, 30 Dec 2015 23:59:36 +0000 (15:59 -0800)]
Do not prohibit VXLAN over IPv6
9fc45cee in introduced a regression prohibiting using VXLAN over
IPv6. Relax restriction on local_ip, but validate that both
local_ip and vxlan_group of the same address family. Move existing
validation of vxlan_group into validate_vxlan_group_with_local_ip()
method and refactor existing tests for that validation.
Ihar Hrachyshka [Wed, 13 Jan 2016 12:32:15 +0000 (13:32 +0100)]
tests: stop validating neutronclient in neutron-debug tests
In neutronclient 4.0.0, any command executed triggers DeprecationWarning
on stderr (to be fixed by I77f168af92ae51ce16bed4988bbcaf7c18557727 and
a new client release including it).
The test cases assumed that if command is successful, it never writes to
stderr though. Making the test failing when using the latest client.
Instead of fixing the test class not to assume there is no output on
stderr, remove it because we are not meant to validate neutronclient in
neutron gate at all and should rely on the library as shipped. Client
should already have reasonable coverage for its CLI.
ML2: verify if required extension drivers are loaded
This change ensures extension drivers required by service plugins are loaded
when using ML2 plugin: we check that ML2 loads QoS extension driver when QoS
service plugin is enabled.
We switched to constrained jobs a while back, but these links were
showing the non constrained ones, making these graphs useless.
This patch updates them to reflect the jobs that are currently
running, however the docs job is left for later as right now
switching would make graphite fail with:
'TypeError: reduce() of empty sequence with no initial value'
I suspect that's because the job has never failed so far.
Static routes not added to qrouter namespace for DVR
Today static routes are added to the SNAT namespace
for DVR routers. But they are not added to the qrouter
namespace.
Also while configuring the static routes to SNAT
namespace, the router is not checked for the existence
of the gateway.
When routes are added to a router without a gateway the
routes are only configured in the router namespace, but
when a gateway is set later, those routes have to be
populated in the snat_namespace as well.
chen-li [Wed, 13 Jan 2016 10:31:57 +0000 (18:31 +0800)]
Pass environment variables of proxy to tox
When a development environment is under a proxy, tox is failed
even if environment variables of the proxy are set. This is a
new feature above tox 2.0, you can check it from [1].
This patch fixes quota management APIs in the Pecan framework.
To this aim:
1) an ad-hoc pair of collection/item controllers are introduced
for the quota resource; as the new controllers have been added
in a separate module, the neutron.pecan_wsgi.controllers.utils
module has been added as well for helpers, routines and classes
used by all pecan controllers;
2) the quota API extension is made pecan-aware, meaning that it
simply returns a Pecan controller instance rather than deferring
the task to the startup process that builds controllers using the
home-grown WSGI framework ext manager;
3) the quota resource is now "almost" a standard neutron resource;
unfortunately since it does not yet have its own service plugin a
special provision is made in the attribute population hook in
order to ensure the object is loaded for allowing correct
policy enforcement.
4) Functional tests for the quota controller have been added.
Pecan: Fixes and tests for the policy enforcement hook
As PolicyNotAuthorizedException is raised in a hook, the
ExceptionTranslationHook is not invoked for it; therefore a 500
response is returned whereas a 403 was expected. This patch
explicitly handles the exception in the hook in order to ensure
the appropriate response code is returned.
Moreover, the structure of the 'before' hook prevented checks
on DELETE requests from being performed. As a result the check
was not performed at all (checks on the 'after' hook only pertain
GET requests). This patch changes the logic of the 'before' hook
by ensuring the item to authorize acces to is loaded both on PUT
and DELETE requests.
This patch also adds functional tests specific for the policy
enforcement hook.
Using model_query in the operation for retrieving tenant limits
will spare the need for explicit authorization check in the
quota controller. This is particularly relevant for the pecan
framework where every Neutron API call undergoes authZ checks
in the same pecan hook.
This patch will automatically adapt by eventuals changes
introducing "un-scoped" contexts.
Oleg Bondarev [Tue, 15 Dec 2015 14:58:51 +0000 (17:58 +0300)]
Use admin context when removing DVR router on vm port deletion
In case non-admin tenant removes last VM on a shared network (owned
by admin) connected to a DVR router (also owned by admin) we need
to remove the router from the host where there are no more dvr
serviceable ports. Commit edbade486102a219810137d1c6b916e87475d477
fixed logic that determines routers that should be removed from host.
However in order to actually remove the router we also need admin
context.
This was not caught by unit tests and one reason for that is so called
'mock everything' approach which is evil and generally useless.
This patch replaces unit tests with functional tests that we able
to catch the bug.
Paul Michali [Tue, 12 Jan 2016 21:44:47 +0000 (21:44 +0000)]
Provide kwargs for callback abort
The callback mechanism allows notifiers to provide keyword args in
the notification. If this is a create callback, and there is an
exception, then the notifier will call an abort callback.
However, currently, the keyword arguments are not provided to the
abort callback. This information could be useful for the callbacks,
and would make the mechanism consistent. This commit provides that
information.
Brandon Logan [Tue, 29 Dec 2015 08:41:48 +0000 (02:41 -0600)]
Pecan controller loads service plugins
The pecan controllers were not parsing out the case when a service
plugin is being used that changes the URI to
/v2.0/service_plugin/resource
This will check to see if the first resource after v2.0 is an
extension to a service plugin, and if it is it'll check the next
resource for a controller.
Terry Wilson [Wed, 6 Jan 2016 07:19:31 +0000 (01:19 -0600)]
Make sure datapath_type is updated on bridges changed
When changing datapath_type in the config, physical and tunnel bridges
do not have their datapath_type updated. Calling create() on already
created bridges should be safe as it passes '--may-exist' when adding
the bridge, which will do nothing if the bridge already exists, but
the second part of the transaction will still update things like
datapath_type.
It should be noted that ancillary bridges (like br-ex) are not
modified by this patch as datapath_type was never applied to them to
begin with.
Incidentally, the native and vsctl versions behaved slightly
differently when handling datapath_type: vsctl builds the multi-cmd
transaction with add-br ... -- set ..., so that the second cmd would
actually complete. The native just bailed if may_exist and the bridge
existed. This is fixed as part of this patch.
Hynek Mlnarik [Tue, 12 Jan 2016 08:10:25 +0000 (09:10 +0100)]
Log INFO message when setting admin state up flag to False for OVS port
Disabling admin state can cause confusion in OVS agent logs as to why
VLAN 4095 is set to a particular port. This commit adds an extra INFO
log message when admin state is disabled. There is no log emitted
upon enabling admin state as there is already a detailed log entry
logged in treat_devices_added_or_updated() function.
Assaf Muller [Tue, 12 Jan 2016 02:58:30 +0000 (21:58 -0500)]
Fix regression with unbound ports and l2pop
When l2pop is enabled and an unbound port is deleted l2pop mech
driver raises an exception as a result of patch:
https://review.openstack.org/#/c/263471/
As a result of the same patch, when an unbound port's IP
address is changed l2pop sends a fanout RPC message needlessly.
Oleg Bondarev [Tue, 13 Oct 2015 09:45:59 +0000 (12:45 +0300)]
L3 agent: paginate sync routers task
In case there are thousands of routers attached to thousands of
networks, sync_routers request might take a long time and lead to timeout
on agent side, so agent initiate another resync. This may lead to an endless
loop causing server overload and agent not being able to sync state.
This patch makes l3 agent first check how many routers are assigned to
it and then start to fetch routers by chunks.
Initial chunk size is set to 256 but may be decreased dynamically in case
timeouts happen while waiting response from server.
This approach allows to reduce the load on server side and to speed up
resync on agent side by starting processing right after receiving
the first chunk.
Oleg Bondarev [Wed, 9 Dec 2015 14:58:14 +0000 (17:58 +0300)]
Remove duplicate for check_ports_exist_on_l3agent
This removes check_ports_on_host_and_subnet which mostly duplicates what
check_ports_exist_on_l3agent is doing.
Also rename check_ports_exist_on_l3agent to check_dvr_serviceable_ports_on_host
for more clarity.
Currently 'flows' is being checked for empty list in
a non standard way 'if flows == []:'. This patch
simplifies logic so that above check is unnecessary.
This patch simplifies the process for validating the request body for
POST and PUT requests and preparing it for dispatching to plugins.
In particular it removes the need for parsing the URL to find the
resource identifier, and instead leverages Pecan's routing engine
to extract it. As a result the attribute population hook now simply
deals with request body validation, and has been renamed accordingly.
The logic for loading the current state of the object from the plugin
in order to enforce authZ policy on PUT requests has been moved to
the appropriate hook, PolicyEnforcementHook.
The logic for managing plural/singular names for resource has also been
improved, and two helper functions to retrieve a resource's attributes
have been added to neutron.api.v2.attributes
The logic for aborting requests with unsupported HTTP methods has also
been moved to the REST controllers. It used to be in PolicyEnforcementHook,
which was probably not the right thing to do.
This patch also unskips a functional test concerning ownership checks,
and add functional tests for verifying correct request processing.
Unit tests for the newly added helper functions are also provided.
Kevin Benton [Thu, 7 Jan 2016 23:40:27 +0000 (15:40 -0800)]
make floating IP specification test robust to races
The get_unused_ip function returns an address that isn't in use on
the network at the time it's called. However, by the time the caller
uses it, it may have been allocated to another concurrent test if
the network is shared or external.
The test_create_floatingip_with_specified_ip_address test creates
a floating IP with an unused IP on the external network, which has
lots of activity on it from other tests so it would occasionally
have its address taken by another test.
This patch just catches the in use error and tries again with a
new IP.
Oleg Bondarev [Wed, 30 Dec 2015 17:24:55 +0000 (20:24 +0300)]
Fix get_subnet_for_dvr() to return correct gateway mac
Fix filters to get the right gateway port for a subnet.
Wrong filters led to random port's mac being returned which may cause
side effects. See bug for details.
Code Review / openstack-build / neutron-build.git / log