Andrew Boik [Mon, 23 Mar 2015 15:21:11 +0000 (11:21 -0400)]
Support multiple IPv6 prefixes on internal router ports
(Patch set #3 for the multiple-ipv6-prefixes blueprint)
Provides support for adding multiple IPv6 subnets to an internal router
port. The limitation of one IPv4 subnet per internal router port
remains, though a port may contain one IPv4 subnet with any number of
IPv6 subnets.
This changes the behavior of both the router-interface-add and
router-interface-delete APIs. When router-interface-add is called with
an IPv6 subnet, the subnet will be added to an existing internal port
on the router with the same network ID if the existing port already has
one or more IPv6 subnets. Otherwise, a new port will be created on the
router for that subnet. When calling the router-interface-add with a
port (one that has already been created using the port-create command),
that port will be added to the router if it meets the following
conditions:
1. The port has no more than one IPv4 subnet.
2. If the port has any IPv6 subnets, it must not have the same
network ID as an existing port on the router if the existing
port has any IPv6 subnets.
If the router-interface-delete command is called with a subnet, that
subnet will be removed from the router port to which it belongs. If the
subnet is the last subnet on a port, the port itself will be deleted
from the router. If the router-interface-delete command is called with
a port, that port will be deleted from the router.
This change also allows the RADVD configuration to support advertising
multiple prefixes on a single router interface.
The ovsdb monitor test was using a timeout of 60s for monitor start.
This change sets the timeout to the global timeout value if it is
greater (it's 90s currently).
Paul Michali [Thu, 26 Mar 2015 12:01:58 +0000 (08:01 -0400)]
Refactoring of L3 agent notifications for router
The goal of this refactoring is to reduce duplication by
replacing the L3EventObservers mechanism (a specific
mechanism for L3 agent notifications), with the
CallbacksManager mechanism (a more general mechanism
currently in use), so that there is one method
used.
This is the first part of refactoring the L3 agent so that
it uses the new neutron.callbacks mechanism. To do this,
duplicate calls will be made for notifications related to
the router, only using the new callback mechanism.
This commit does two things. First, it puts in place the
notifiers for the new callback mechanism. Second, it updates
the metatdata proxy agent (which is in the same repo) to
use the new callback mechanism.
Later commits will update other repos from the old to new
callback mechanism, and to then remove the old callback
mechanism, once no longer used.
Currently if the quota_port, quota_network, quota_subnet values
in the neutron.conf are set to a negative value not equal to -1,
neutron reports the values as is to consumers like Nova.
Nova treats -1 as the infinite quota indicator and doesn't expect
neutron to return any other non-negative value.
The fix allows the flexibility of having any negative number for the
quota parameters in the neutron.conf file and allows the nova boot
to succeed subsequently. The fix would report any negative value
as -1 for port, subnet and network.
According to changes [1,2], API tests' new home is under neutron/tests/api.
Change 92d2054f8a slipped through the cracks. It seems also that wrong
imports lead to tests silently dropped (i.e. not executed). This patch
rectifies the issue.
Cedric Brandily [Tue, 3 Mar 2015 22:26:52 +0000 (22:26 +0000)]
Allow metadata proxy to log with nobody user/group
Currently metadata proxy cannot run with nobody user/group as
metadata proxy (as other services) uses WatchedFileHandler handler to
log to file which does not support permissions drop (the process must
be able to r/w after permissions drop to "watch" the file).
This change allows to enable/disable log watch in metadata proxies with
the new option metadata_proxy_log_watch. It should be disabled when
metadata_proxy_user/group is not allowed to read/write metadata proxy
log files. Option default value is deduced from metadata_proxy_user:
* True if metadata_proxy_user is agent effective user id/name,
* False otherwise.
When log watch is disabled and logrotate is enabled on metadata proxy
logging files, 'copytruncate' logrotate option must be used otherwise
metadata proxy logs will be lost after the first log rotation.
A recent change added a new api test to the old location that is no
longer used for discovery. This change moves it to
neutron/tests/api/admin to ensure that it can be discovered and run.
Carl Baldwin [Thu, 26 Mar 2015 18:10:10 +0000 (18:10 +0000)]
Implement default subnet pool configuration settings
The default_ipv6_subnet_pool option was added [1] as an integration
point between prefix delegation work and subnet allocation work. This
patch completes the integration with subnet allocation. This
addresses the use case where a deployer wants all ipv6 addresses to
come -- by default -- from a globally routable pool of ipv6 addresses.
In a deployment with this option set, an API user can still access the
old behavior by passing None explicitly as subnetpool_id when creating
a subnet.
This patch also adds the default_ipv4_subnet_pool for completeness.
Kyle Mestery [Fri, 13 Mar 2015 14:54:37 +0000 (14:54 +0000)]
Update core reviewer responsibilities
This patch more clearly lays out who can merge code into the plethora
of Neutron repositories. It also clarifies a few things with the
existing text in places.
Remove "Arguments dropped when creating context" logging
This log was previously reduced from warning to debug.
Cinder removed it entirely in:
https://bugs.launchpad.net/cinder/+bug/1329156
The root cause is this:
Agent heartbeats use an admin context. The context is serialized
with its to_dict method, which exposes 'tenant' and 'project_name'
(These are properties of the class that are calculated from other
attributes). In the controller, this dict is used to initialize a
ContextBase, which does not accept tenant and project_name as arguments,
de facto sending those values as key word arguments.
We can either handle 'tenant' and 'project_name' specially, fix
it any other way, or drop the logging entirely. Is this logging
ever useful?
Henry Gessau [Fri, 27 Mar 2015 02:54:21 +0000 (22:54 -0400)]
Modify a different agent in test_update_agent_description
API test_update_agent_description modifies an agent's description, and
test_list_agent assumes the first agent is never modified. We make
sure that an agent other than the first one is modified.
Maru Newby [Tue, 24 Mar 2015 16:21:57 +0000 (16:21 +0000)]
Move API tests to neutron.test.api
To make api test development simpler, move the tests to
neutron.tests.api. The neutron.tests.tempest subtree will remain
while work continues to transition the required functionality to
tempest-lib.
Ryan Tidwell [Mon, 16 Mar 2015 18:02:13 +0000 (11:02 -0700)]
Simple subnetpool allocation quotas
Enables enforcement of allocation quotas on subnet pools. The quota
is pool-wide, with the value of allocation_quota applied to every
tenant who uses the pool. allocation_quota must be non-negative,
and is an optional attribute. If not supplied, no quotas are
enforced. Quotas are measured in prefix space allocated. For IPv4
subnet pools, the quota is measured in units of /32 ie each tenant
can allocate up to X /32's from the pool. For IPv6 subnet pools, the
quota is measured in units of /64 ie each tenant can allocate up to
X /64's from the pool. For backward-compatibility, allocation quotas
are not applied to the implicit (AKA null) pool. Standard subnet
quotas will continue to be applied to all requests.
Ryan Tidwell [Thu, 19 Feb 2015 23:29:08 +0000 (15:29 -0800)]
Subnet allocation from a subnet pool
Contains API changes, model changes, and logic required to enable a subnet to
be allocated from a subnet pool. Users can request a subnet allocation by
supplying subnetpool_id and optionally prefixlen or cidr. If cidr is
specified, an attempt is made to allocate the given CIDR from the pool. If
prefixlen is specified, an attempt is made to allocate any CIDR with the
given prefix length from the pool. If neither is specified, a CIDR is chosen
from the pool using the default prefix length for the pool.
Maru Newby [Tue, 24 Mar 2015 01:30:11 +0000 (01:30 +0000)]
Simplify retargetable test framework
The retargetable testing prototype previously relied on each test case
defining the 'scenarios' attribute used to parametize testing with
testscenarios. Anticipating the requirement to retrofit the imported
tempest api test cases, this change moves scenario definition to a
base class since scenarios are common across all api tests.
This change also sets the retargetable test to skip when invoked
against rest. Tempest uses class-level setup for auth and this needs
to be broken out into fixtures before the retargetable testing will
work again.
Itsuro Oda [Wed, 25 Feb 2015 04:34:04 +0000 (13:34 +0900)]
Make floatingip reachable from the same network
The problem is that if one tries to communicate from a tenant network
to floatingip which attached to a port on the same network, the
communication fails.
This problem is a regression cased by [1].
[1] https://review.openstack.org/131905/
Before [1] SNAT rule is as follows:
-s %(internal_cidr)s -j SNAT --to-source ...
(for each internal interface)
After [1] SNAT rule is as follows:
-o %(interface_name)s -j SNAT --to-source ...
(for an external interface)
The new rule was considered a super-set of the packets going out to
the external interface compared to the old rules. This is true but
there is a lack of consideration.
Note that the packet is 'going out to external interface' OR 'DNATed'
at this point since the rule:
! -o %(interdace_name)s -m conntrack ! --ctstate DNAT -j ACCEPT
was applied already. So we should consider the following three cases.
1) going out to external interface
should be SNATed. It is OK under the new rule but there was a lack
of rules for packets from indirectly connected to the router under the
old rules. ([1] fixed this.)
2) DNATed (and going out to internal interface)
2-1) came in from internal interface
should be SNATed because the return traffic needs to go through the
router to complete the conntrack association and to reverse the effect
of DNAT on the return packets. If a packet is not SNATed, the return
packet may be sent directly to the private IP of the initiator.
The old rules done SNAT in this case but the new rule doesn't.
2-2) came in from external interface
nothing to do.
This patch adds a rule for the case 2-1).
This patch also adds mangle rules to examine whether a packet came from
external interface.
Allow router-gateway-set to work even without an assigned
subnet with the net_id so as to enable IPv6 L3 routing
using the assigned LLA for the gateway.
The goal is to allow for IPv6 routing using just
the allocated LLA address for the gateway port to be
used as the external gateway to connect to the upstream
router. For this purpose router-gateway-set no
longer has a requirement of an assigned subnet.
A new config has also been added to the l3_agent.ini
to allow the user to set a valid ipv6_gateway address
to be used as the gateway for the default ::/0 route
If the ipv6_gateway config is not set and a gateway
is still created without a subnet, the gateway interface
will be configured to accept router advertisements (RAs)
from the upstream router so as to build the default route.
Unit test changes and additions reflect these changes.
Ihar Hrachyshka [Wed, 18 Mar 2015 13:21:57 +0000 (14:21 +0100)]
tests: don't rely on configuration files outside tests directory
etc/... may be non existent in some build environments. It's also pip
does not install those files under site-packages neutron module, so
paths relative to python files don't work.
So instead of using relative paths to etc/... contents, maintain our own
version of configuration files. It means we need to maintain tests only
policy.json file too, in addition to neutron.conf.test and
api-paste.ini.test.
Ideally, we would make etc/policy.json copied under site-packages in
addition to /etc/neutron/. In that way, we would not maintain a copy of
policy.json file in two places.
Though it seems that setuputils does not have a good way to install
files under site-packages that would consider all the differences
between python environments (specifically, different prefixes used in
different systems).
Note: it's not *absolutely* needed to update the test policy.json file
on each next policy update, though it will be needed in cases when we
want to test policy changes in unit tests. So adding a check to make
sure files are identical.
Kevin Benton [Wed, 17 Sep 2014 03:36:42 +0000 (20:36 -0700)]
Set floating IP port status to "N/A"
The status of the port associated with a floating IP
would always show as DOWN. This caused confusion to
operators that weren't aware that this is expected behavior
since the port is only used for an IP allocation.
This commit sets the port status to "N/A" to reflect the fact
that the port associated with a floating IP has no operational
status.
Kevin Benton [Sat, 28 Mar 2015 06:18:08 +0000 (23:18 -0700)]
Fix error raising in security groups method
In case there were security groups not belonging to tenant on port
_get_security_groups_on_port would try to raise exception but fail
trying to index set.
This patch simply joins the whole set as a string and inserts it
into the standard SecurityGroupNotFound exception.
No new exception types, no string freeze violations.
Co-Author: watanabe.isao <zou.yun@jp.fujitsu.com>
Co-Author: Jacek Swiderski <jacek.swiderski@codilime.com>
Andrew Boik [Wed, 4 Mar 2015 03:39:57 +0000 (22:39 -0500)]
Auto-update gateway port after subnet-create
(Patch set #6 for the multiple-ipv6-prefixes blueprint)
In the multi-prefix scenario, one can add two subnets
to an external gateway port by adding the two subnets
to the external network and using router-gateway-set.
However, if there is only one subnet on the port and
the user wishes to add another later, it is desirable
to have the newly-created external subnet automatically
added to the port. This patch adds this functionality.
Andrew Boik [Fri, 27 Feb 2015 23:48:29 +0000 (18:48 -0500)]
Allow update of ext gateway IP's w/out port delete
(Patch set #5 for the multiple-ipv6-prefixes blueprint)
Updating an external gateway port currently triggers a port-delete
followed by a port-create. In the multi-prefix case, if a second
subnet is added to an external gateway port, the port will be
deleted, freeing the original IP allocation, and then the port will
be recreated with new IP allocations from the two subnets. This is
undesirable as the port can't keep the same IP address from the
original subnet.
This patch modifies the behavior so that a fixed-ip change on an
external gateway port will cause a port-update instead of a
delete/create. If the gateway port network id has changed, however,
the port will be deleted and recreated as before.
Dane LeBlanc [Wed, 18 Mar 2015 20:38:57 +0000 (16:38 -0400)]
Support Dual-Stack Gateway Ports on Neutron Routers
(Patch set #2 for multiple-ipv6-prefixes blueprint)
This patchset adds support for dual-stack gateway ports on Neutron
routers. Some background on the changes included in this patchset:
- The L3 driver's init_l3() method has been changed to accept a list
of gateway IPs, rather than a single gateway IP.
- The Neutron port dictionary's singular 'subnet' entry has been
replaced with a 'subnets' list, since ports can now be associated
with multiple subnets.
- The Neutron port dictionary no longer has a (singular) 'ip_cidr'
entry, since a port can now be associated with multiple IP CIDRs
(e.g. up to one IP CIDR per IP family on gateway ports).
Instead, a 'prefixlen' entry has been added to the Neutron
fixed_ips dictionary, so that the port's (multiple) IP CIDRs can
be derived from the matching 'ip_address' and 'prefixlen' pairs
in the port's fixed_ips.
Kevin Benton [Sat, 21 Mar 2015 01:56:51 +0000 (18:56 -0700)]
Remove auto deletion of routers in unit tests
Remove the automatic deletion behavior of the router
context manager in the L3 unit tests. Any tests that
depend on the router being deleted should do so
explicitly.
It additionally removes the logic from the test_l3_plugin
unit tests that was just related to tearing down enough
stuff to allow the context managers to exit. It was code
that distracted from what the tests were actaully verifying.
All of the context managers for port, network, and subnet
do not auto delete by default and that will be extended to
the L3 constructs as well. The patch that did this for
ports/subnets/networks is here:
https://review.openstack.org/#/c/102465/
Dane LeBlanc [Wed, 18 Mar 2015 16:41:25 +0000 (12:41 -0400)]
No allocation needed for specific IPv6 SLAAC addr assignment
(Patch set #7 for the multiple-ipv6-prefixes blueprint)
On internal router ports, Neutron allows for an address to
be assigned for an IPv6 SLAAC subnet that is not necessarily
EUI-64. This makes it easier for subnet create, since a
convenient address, e.g. one ending in ::1, can be used as
the subnet gateway IP address.
Currently, when an internal router port is created with a specific
(non-EUI-64) address for a SLAAC subnet, the call flow includes
a call to _allocate_specific_ip. This call is not necessary,
since we're not allocating an address from a pool (and
recalibrating availability ranges, etc.).
This patch set prevents the call to _allocate_specific_ip for
this scenario.
Co-Authored-By: Baodong (Robert) Li <baoli@cisco.com>
Change-Id: I2533ee82980bb602faa663b875787ca50b268b34
Partially-implements: blueprint multiple-ipv6-prefixes
Maru Newby [Fri, 27 Mar 2015 17:39:41 +0000 (17:39 +0000)]
Remove neutron.tests.sub_base
Change Ifca5615680217818b8c5e8fc2dee5d089fbd9532 was intended to
remove the neutron.tests.sub_base module, but a bad rebase means that
it was left in the tree.
Kevin Benton [Fri, 27 Mar 2015 15:13:58 +0000 (08:13 -0700)]
Fix test case for DHCP agent interface restart
One of the new test cases in the recent DHCP
interface patch[1] was supposed to confirm that
the driver wouldn't be restarted if the IP address
stayed the same. However, it wasn't matching the
device ID of the agent so it was never making it
to that conditional.
This patch just fixes that UT so it's exercising
the right code path.
abhishek60014726 [Wed, 25 Mar 2015 11:20:55 +0000 (04:20 -0700)]
Test to verify shared attribute of network
Add Funtion to create a shared network
Add function to create a shared network in bulk
Add a test to create and update a shared network
Add a test to create a port in a shared network using non admin tenant
Add test to create shared networks in bulk
Add function to list and show shared network
Add test to list and show the shared network by admin and non admin
Miguel Angel Ajo [Tue, 24 Mar 2015 13:10:37 +0000 (13:10 +0000)]
Enable Process Monitor by default.
Process monitor is enabled by default by this patch,
with a default 60 second monitoring interval, this
interval was calculated early in the development
process to scale to 1000s of processes with light load.
We believe it's important to have it enabled to get
user feedback as we release kilo.
Process monitor is sucessfully enabled and backported
to Red Hat D/S distributions from icehouse to juno
without any issue.
Specific process monitor functional tests provide
coverage, also keepalived checks that it can be
properly respawned.
We should follow up with dhcp and l3 agent functional
testing for killing and checking their processes
correctly respawned. Normal process start/stop is
already validated by other functional tests and tempest.
Kevin Benton [Fri, 27 Mar 2015 02:52:23 +0000 (19:52 -0700)]
Don't eagerly load ranges from IPAllocationPool
The subnet object eagerly loads the IPAllocationPools
associated with it. Each of these was eagerly loading
the IPAvailabilityRange objects associated with it.
On a large subnet with lots of churn, this could be
thousands of records. All of these records were being
loaded for every call to get_subnet, which means all
get_subnets, get_networks, and so-on. icky
This patch changes the relationship between IPAllocationPool
and available_ranges to a 'select' load, so they won't be
loaded until referenced. On my test system with a subnet
that contained 10k ports, this changed the subnet-show time
from 4.7 seconds to 0.56 seconds.
There is no performance downside to this in the upstream
code. At the time of this patch, there were no references
to 'available_ranges' on an IPAllocationPool result. The
logic that deals with the available ranges queries them
explicitly using join statements.
Itsuro Oda [Thu, 8 Jan 2015 23:47:56 +0000 (08:47 +0900)]
Enable services on agents with admin_state_up False
Previously when admin_state_up of an agent is turned to False,
all services on it will be disabled.
This fix makes existing services on agents with admin_state_up
False keep available.
To keep current behavior available the following configuration
parameter added.
* enable_services_on_agents_with_admin_state_down
If the parameter is True, existing services on agents with admin_state_up
False keep available. No more service will be scheduled to the agent
automatically. But adding a service to the agent manually is available.
i.e. admin_state_up: False means to stop automatic scheduling under the
parameter is True.
The default of the parameter is False (current behavior).
Code Review / openstack-build / neutron-build.git / log