Maru Newby [Sat, 23 Aug 2014 06:21:07 +0000 (23:21 -0700)]
Add hook scripts for the functional infra job
Infra jobs support hooks to customize behavior. This change adds
scripts that are intended to be called by the neutron-dsvm-functional
jobs as hooks. Defining hook behavior in-tree instead of directly in
the job definition has the advantage of allowing changes in hook
behavior to be tested before merge.
The gate_hook.sh script added by this change installs a rootwrap
filter to allow functional tests to destroy python processes that
they have spawned. The tests use the python executable provided by
their tox env, and the default rootwrap configuration does not
support killing processes launched by non-system python executables.
This change will have to merge to master and stable/icehouse before
the supporting infra config change can be merged:
Kevin Benton [Tue, 19 Aug 2014 22:15:09 +0000 (15:15 -0700)]
Join tables in query for down L3 agents
Join the agent table together with the L3 agent scheduler
table. This fixes an issue where the query would return
all admin_state_up L3 agents with router bindings when
any agent in the database was dead (missed heartbeats)
instead of just returning the dead L3 agents with bindings.
Carl Baldwin [Tue, 19 Aug 2014 03:58:22 +0000 (03:58 +0000)]
Rename range to avoid shadowing the builtin
This refactor supports a patch I was reviewing [1]. I thought it
better to apply this consistently in this file as a separate refactor
so that that patch could be done cleaner.
The ML2 RPC version 1.1 breaks the hyper-v agent and
consequentially the RPC version in the Hyper-V agent needs
to be set to 1.1 to match the ML2 RPC base API version.
Carl Baldwin [Tue, 19 Aug 2014 05:07:47 +0000 (05:07 +0000)]
Remove unused network parameter from _allocate_ips_for_port
I found this in the context of a patch that I was reviewing [1]. I
found that patch left a few loose ends and so I thought it would be
better to remove the parameter as a separate refactor so that it can
be carefully reviewed. The other patch [1] should be rebased to this
one.
* This patch adds support for subnets created with 'ipv6_address_mode'
set to 'dhcpv6-stateful' or 'dhcpv6-stateless' by dnsmasq.
* If no dnsmasq process for subnet's network is launched, Neutron
will launch new dnsmasq process on subnet's dhcp port in 'qdhcp-'
namespace. If previous dnsmasq process is already launched,
restart dnsmasq with new configuration.
* Neutron will update dnsmasq process and restart it when subnet
gets updated.
* This patch enforces the version check of dnsmasq. dhcp-agent will
fail to start if version of dnsmasq<2.63.
Implements securitygroup extension for nuage plugin
In Nuage VSP, the scope of vport-tag(Neutron securitygroup equivalent) is either
per router or per subnet whereas securitygroup scope in Neutron is per tenant.
Because of this, the mapping between neutron and VSP resource always happens at
the port create or update time, such that port's router/subnet is known and
thus sg attachment point in VSP is known.
On port-update --security-group sg1, if this is the first port getting attached
to that security-group, corresponding vport-tag(for sg) and rules( for sg rules)
are created on VSP. Subsequent port-update for the same sg1 will just update the
port to vport-tag binding. When security-group and security-group-rules are
deleted on neutron, corresponding vport-tag and rules are deleted on the VSP.
Kris Lindgren [Wed, 20 Aug 2014 01:33:22 +0000 (19:33 -0600)]
Fix bigswitch setup.cfg lines
Bigswitch was not correctly creating the bigswitch/ssl/ca_certs and
host_certs directories when running the setup.py install step. This
makes it difficult to correctly package the plugin because it has
missing files/directories.
Jakub Libosvar [Thu, 7 Aug 2014 08:35:07 +0000 (10:35 +0200)]
Ensure ip6tables are used only if ipv6 is enabled in kernel
On systems where ipv6 module is not loaded in kernel we need to avoid
usage of ip6tables. This patch reads
/proc/sys/net/ipv6/conf/default/disable_ipv6 file and if ipv6 is
disabled then ip6tables are not used in IptablesManager
Eugene Nikanorov [Wed, 13 Aug 2014 11:10:45 +0000 (15:10 +0400)]
Reorder operations in create_vip
Previously VIP's port creation was moved outside the transaction
to avoid deadlocks related to rpc calls.
It led to small chance of VIP being fetched by rpc code with
port being still None.
This patch reorders VIP association with the pool so it's done
only after port has been created.
This patch also leaves small possibility of uncaught exception in
case of pool being concurrently deleted in the process of VIP creation
Elena Ezhova [Tue, 12 Aug 2014 15:43:09 +0000 (19:43 +0400)]
Send HTTP exceptions in the format expected by neutronclient
Neutron client for the v2 API expects exceptions to have 'type',
'message' and 'detail' attributes. That is why they need to be
included in the body of HTTP Exceptions.
Remove auto-generation of db schema from models at startup
This patch removes the Neutron capability of creating database tables
from sqlalchemy models for all those model classes for which
a table is not found in the database schema.
Migrations should be the official and only solution for creating and
managing the Neutron db schema.
This patch also adapts unit tests in order to ensure test schemas
are still correctly created.
Carl Baldwin [Thu, 10 Jul 2014 19:41:40 +0000 (19:41 +0000)]
Implement namespace cleanup for new DVR namespaces
DVR introduces a new namespace type called snat-<router_id>. These
namespaces are not properly cleaned up when found stale after an agent
restart. This patch fixes that.
The method for setting the floating IP status in the NSX plugin
has a flaw that causes in some cases the status to be reset.
This patch removes this bug and also protects against regression
adding status check in the unit test covering floating ip
creation.
This patch also moves the above mentioned unit test in a more
suitable place in the test same module.
Akihiro Motoki [Thu, 14 Aug 2014 06:53:47 +0000 (15:53 +0900)]
Skip FWaaS config mismatch check if RPC method is unsupported
In this commit FWaaS config check skip is skipped
if neutron server does not support this API.
Commit d6f014d introduced FWaaS config mismatch check between
server and agent. It added a new RPC method get_service_plugin_list
and bumped l3-agent RPC version to 1.3, but this version RPC is
only supported by L3 router service plugin and
it breaks existing plugins using L3 router mixin.
Bumping l3-agent RPC version requires detailed investigation on all
affected plugins and it can be done by plugin maintainer later.
Ihar Hrachyshka [Thu, 7 Aug 2014 22:04:44 +0000 (00:04 +0200)]
Use jsonutils instead of stdlib json
jsonutils provides multiple benefits in comparison to pure stdlib json
(like using simplejson on Python 2.6).
Similar patch was already merged before [1], but since it lacked hacking
rule to enforce jsonutils usage, new occurrences of stdlib json module
usage were introduced.
This patch switches all the code to using jsonutils and adds a hacking
rule to enforce the rule.
The hacking rule requires that jsonutils module does not mimic as 'json'
thru using import renames, so the code was updated not to rename the
module when doing import.
The hacking rule was shamelessly copied from the corresponding nova
review [2].
rajeev [Tue, 12 Aug 2014 00:42:18 +0000 (20:42 -0400)]
Fix interface add for dvr with gateway
when an interface is added after router gateway set, external
connectivity using snat fails. Instead of just adding the snat port for
the new subnet, method internal_network_added(..) incorrectly re-adds
all the snat ports with wrong cidr.
Stephen Ma [Wed, 6 Aug 2014 22:33:32 +0000 (22:33 +0000)]
Delete DVR namespaces on node after removing last VM
After removing the last VM using a distributed router,
the router's namespaces are still present on the VM host
The problem is that the neutron API server sent the router
remove notification to the L3 agent using the name of the
host running the L3 agent instead of the agent's uuid. This
caused an error when sending the notification. So the L3
agent never had the chance to cleanup the namespace.
This problem is fixed here.
Afterwards, it was found that the notification was still not
sent. The reason is that the router/L3-agent binding has
already been deleted before the routine that sends the
router removed notification was called. The notifier routine
errored out when it tried to delete the same router/L3 agent
binding. This problem is fixed in this patch by removing the
binding removal step from the DVR scheduler.
Fix PortNotFound error during update_device_up for DVR
An agent's request to update the ARP entry for a VM port
may come after a deletion request has been processed,
resulting in a PortNotFound exception being raised.
This patch takes care of this condition. A test has
been added, which required a minor refactoring of the
test case class, in order to accommodate the use of
side effects for the objects being mocked.
Kevin Benton [Wed, 30 Jul 2014 22:49:59 +0000 (15:49 -0700)]
Option to remove routers from dead l3 agents
Add a configuration-enabled periodic check to examine the
status of all L3 agents with routers scheduled to them and
admin_state_up set to True. If the agent is dead, the router
will be rescheduled to an alive agent.
Neutron considers and agent 'dead' when the server doesn't
receive any heartbeat messages from the agent over the
RPC channel within a given number of seconds (agent_down_time).
There are various false positive scenarios where the agent may
fail to report even though the node is still forwarding traffic.
This is configuration driven because a dead L3 agent with active
namespaces forwarding traffic and responding to ARP requests may
cause issues. If the network backend does not block the dead
agent's node from using the router's IP addresses, there will be
a conflict between the old and new namespace.
This conflict should not break east-west traffic because both
namespaces will be attached to the appropriate networks and
either can forward the traffic without state. However, traffic
being overloaded onto the router's external network interface
IP in north-south traffic will be impacted because the matching
translation for port address translation will only be present
on one router. Additionally, floating IPs associated to ports
after the rescheduling will not work traversing the old
namespace because the mapping will not be present.
Remove SELECT FOR UPDATE use in ML2 tunnel driver add_endpoint
SELECT FOR UPDATE expression, which is triggered with the use of the
SQLAlchemy Query object's with_lockmode('update') method, is
detrimental to performance and scalability of the database
performance code in Neutron due to the lock contention it produces.
SELECT FOR UPDATE can be entirely avoided in add_endpoint methods
with the use of single-shot SELECT and INSERT expressions and the
correction of VxlanEndpoint primary key: indeed previously it was not
possible to create multiple endpoints with the same ip, now the model
primary key constraint ensures it.
Method sync_routers is used by the L3 agent to query
routers it knows about. Routers and GW ports lists
are populated in two different times, which means that
they can be interleaved by a delete request which
results in gateway ports being missing in one of the
two data structures.
Paul Michali [Wed, 13 Aug 2014 01:26:27 +0000 (21:26 -0400)]
VPNaaS: Cisco fix validation for GW IP
The validation to check that the router has a GW IP
for the Cisco service driver was trying to directly
get the gw_port information, which is not available, as
the validator only has a dict of the vpn_service info.
Modified validator to get the router object (not a
dict representation), using the vpn_service's router_id
and then check the attributes for the router directly.
Carl Baldwin [Fri, 25 Jul 2014 03:57:40 +0000 (03:57 +0000)]
Preserve link local IP allocations for DVR fip ns across restart
The L3 agent allocates link local address pairs used in connecting the
routers to the floating ip namespace. When those allocations are
forgetten by restarting the L3 agent they all get rewired on restart.
This change preserves the allocations using a file in the local file
system. Storing them in the database would be overkill and would
affect system performance.
The metadata agent was unable to find networks
attached to the DVR router because it was only
filtering ports for 'centralized' routers.
To fix the issue, this patch expands the search
filters to include DVR router interfaces during
the network lookup operation.
The extra filter cause no evident performance
loss while serving the request; a different
approach would require to pass the router type
around to narrow down the search filter, but it
sounds like an overkill.
Wei Wang [Tue, 12 Aug 2014 14:59:18 +0000 (22:59 +0800)]
SecurityGroupRuleExists should point out rule id inseand of group id
When we get a SecurityGroupRuleExists, we want to get the rule id
instaed of group id, and this is what the fucntion
_check_for_duplicate_rules() does.
Praneet Bachheti [Thu, 29 May 2014 23:11:39 +0000 (16:11 -0700)]
Opencontrail plug-in implementation for core resources
Adds Opencontrail plug-in implementation with unit tests
This patch has no dependency on any other blueprints
The link below describes how to install VIF driver for opencontrail
https://github.com/Juniper/contrail-controller/wiki/
OpenContrail-bring-up-and-provisioning
- The contrail_plugin_core.py is the main interface for neutron common
infrastructure. It relays API requests to the opencontrail controller
Note: There are several other unrelated unit tests that also break with a
randomized PYTHONHASHSEED, but they are not addressed here. They will be
addressed in separate patches.
Make plugin and l3plugin available as mixin's properties
This makes the code simpler and slightly more efficient.
This is a minor refactoring related to a race condition
observed when the agent is processing a router while its
interfaces are being removed.
Code Review / openstack-build / neutron-build.git / log