Since we don't want rule_types API to misbehave by claiming support for
QoS for linuxbridge ml2 driver, let's trade API test for its
service correctness.
Note that we cannot enforce the supported rule_types for actual CRUD
operations because then we would need to disable the whole API test
coverage for rules, and we don't want it.
This suggests we should get Linux Bridge support in some way or another.
Ihar Hrachyshka [Fri, 7 Aug 2015 11:50:07 +0000 (13:50 +0200)]
Clean up QoS rules first, then QoS policies
Since policy deletion kills all rules by CASCADE, cleaning rules up
after policies is basically no-op. Instead, let's swap the order and
in that way implicitly cover rule deletions.
Initialize ancillary_port_info dict as blank in OVS agent
The first assignment of ancillary_port_info was from the scan_ancillary_ports
function which could result in an exception and result in ancillary_port_info
being unbound for the port stats scan below.
This patch just initializes ancillary_port_info as an empty dict so the port
stats will always have an input.
Assaf Muller [Thu, 7 May 2015 06:02:59 +0000 (09:02 +0300)]
Enable fullstack multinode tests, add L3 HA test exemplar
* Created a 'resources' subdir and moved all fixture files
to it.
* Split ML2ConfigFixture to the server-side ml2 configuration
fixture, and the OVS agent configuration fixture.
* Neutron process logs were using H:M:S format as their file name,
but when starting multiple agents of the same type my machine
was fast enough to do that in the same second so that different
processes were outputting to the same log file. No good!
Added ms to the log name format. I also changed the log time
from UTC to local timezone.
* Renamed and moved 'FullstackFixture' to neutron/tests/fullstack/
resources/environment.Environment
* Added a 'Host' abstraction that groups agents that report with
the same 'host' value. Hosts may be interconnected by the
environment via shared bridges.
* The 'Environment' class will accept global
attributes (This will be later filled with stuff like tunneling,
l2pop or other environment-level flags), and in this patch accepts
a list of host attributes (Configuration that may differ between
hosts like the l3 agent mode [legacy, dvr, dvr_snat]).
* Made OVS agent and L3 agent fixtures expose their bridges
so that I could interconnect them.
* Added a super simple L3 HA test to show that this entire thing
works.
Moshe Levi [Sun, 2 Aug 2015 09:58:54 +0000 (12:58 +0300)]
SR-IOV: Update eswitch manager to support rate
This patch update the eswitch manager to support
max rate on VF, moreover it updates the eswitch
manager to be singleton so it can be called from the
SR-IOV qos driver.
Ihar Hrachyshka [Tue, 4 Aug 2015 13:29:37 +0000 (15:29 +0200)]
Follow up with some cleanup for agent qos_driver
Removed error handling for missing rule type handler since the rule type
is already filtered thru supported types for the backend, so in case the
handler is really not present, that's a huge bug in the qos driver
extension and should not be handled gracefully.
John Schwarz [Mon, 3 Aug 2015 15:33:44 +0000 (18:33 +0300)]
Gracefully handle duplicate rule creation
Previously, creating a second bandwidth limit rule for a policy raised
an uncaught exception, which eventually caused 'ServerFault' on the
client side. This patch replaces this exception with a NeutronException
which leads to a more correct 'Conflict' error instead.
Note that the code is implemented in the base object class. This means
that future versioned objects will also feature this restriction if
their database implies that no duplicate entries can be created.
John Davidge [Thu, 16 Jul 2015 17:26:24 +0000 (18:26 +0100)]
DB, IPAM & RPC changes for IPv6 Prefix Delegation
This patch includes the DB, IPAM & RPC changes needed for the IPv6 Prefix
Delegation feature.
To enable this feature, the subnetpool_id attribute of subnets has been
modified to allow for a special subnetpool identifier - "prefix_delegation".
WORKFLOW:
1. Admin sets default_ipv6_subnet_pool in neutron.conf to "prefix_delegation"
2. User creates a new IPv6 subnet without a CIDR or subnetpool ID
3. User creates an interface between this subnet and a router with an existing
external interface
The agent-side changes will follow in separate patches.
A documentation patch is up for review here:
https://review.openstack.org/#/c/178739
Video guides for configuring and using this feature are available on
YouTube:
Cyril Roelandt [Wed, 5 Aug 2015 09:39:50 +0000 (11:39 +0200)]
Python 3: convert dict_keys object to list
This makes sure we use the same types in Python 2 and 3, prevents TypeErrors
from happening, and should have no performance impact since the the lists are
quite small anyway.
Henry Gessau [Wed, 5 Aug 2015 01:29:46 +0000 (21:29 -0400)]
Remove VPN from API tests
Change Idd5524ed0b602408be5a53830981a8ab974b390c removed the
VPN service from devstack installations in the gate, so now we
can no longer test VPN APIs from neutron. The tests will be
added to the vpnaas repo's API test suite.
Moshe Levi [Tue, 28 Jul 2015 12:46:10 +0000 (15:46 +0300)]
Update OVS driver to work with objects
This patch updates the QoS OVS driver to work with policy NeutronObjects
that are passed by the agent extension manager, instead of lists of rule
dicts, as we originally expected. It also adds validation that the
rules that are sent by the neutron-server are actually supported by the
backend.
Finally, port dict was not really enough to determine the name of the
port in ovsdb. 'name' field is not really present in all port dicts, and
does not reflect what is known to ovs anyway. So instead, we should
rely on vif_port object to determine the ovs port name. Since ovs agent
only added the vif_port value to details dict when binding was desired,
I made adding the vif_port object unconditional, and covered that fact
with unit tests.
With this patch in place, I was able to get policy rules applied to a
port in devstack installation. Functional tests will belong to a
follow-up.
Cyril Roelandt [Tue, 4 Aug 2015 12:27:48 +0000 (14:27 +0200)]
Python 3: fix test_ovs_tunnel
In Python 3, this happens:
>>> d = {}
>>> a = d.values()
>>> b = d.values()
>>> a == b
False
And anyway we're not really willing to pass dict_values objects around; we are
expecting lists, just like in Python 2, so let's just do the conversion.
John Schwarz [Mon, 3 Aug 2015 12:49:13 +0000 (15:49 +0300)]
Add API tests for non-accessible policies
Tests which dealt with creating a rule for a policy that the tenant has
no access to, or for a policy which does not even exist, were missing.
This patch adds them.
John Schwarz [Mon, 3 Aug 2015 13:56:27 +0000 (16:56 +0300)]
Gracefully handle fetching nonexistent rule
Currently, if we invoke the API for 'show rule' but the rule does not
exist, an exception is raised from deep within Neutron. This in turns
causes an uncaught exception and the user will see 'ServerFault'. This
patch proposes a fix for this scenario - the case where the rule does
not exist is handled and a NeutronException is caused, causing a proper
'NotFound' error on the client side instead.
John Schwarz [Mon, 27 Jul 2015 09:09:10 +0000 (12:09 +0300)]
Replace to_dict() calls with a function decorator
Up until now, API server functions would need to return simple iterable
objects, such as dicts and lists of dicts. This patch introduces a
decorator which allows such functions to return non-simple objects (as
long as the returned object implements the 'to_dict()' method, or is a
list of such objects) and converts them on its own, simplifying the
user's code and removing code duplication.
Enable set DNS and DHCP log of dnsmasq for dhcp agent
Add a new configuration named 'dnsmasq_base_log_dir'
in dhcp_agent.ini.
This entry should be a path of log file. It should
like this:
dnsmasq_base_log_dir=/tmp
And the DNS and DHCP log will be written into the file
"/tmp/aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa/dhcp_dns_log".
The dir path will be created if the given path doesn't exists.
Install arp spoofing protection flow after setting port tag
when ovs-agent set a tag for a port, it will first remove all
flows on this port, because it should guarantee that no drop_port
flow installed by port_dead remains, so arp spoofing protection
flow must be installed after it.
Move 1c844d1677f7 expand migration to appropriate branch
Commit cb60d0bb4e0cc0cba68f59fdf5f4e89d6ec52950 added an expand
migration. Howewer it indicated a contract migration as its
ancestor (2a16083502f3). As a result the migration was treated
by alembic as a contract one, even if it was defined in the
'expand' directory.
This patch 'rebases' this migration on top of the expand branch
HEAD. This might impact trunk chasers which have already executed
migration 1c844d1677f7 as a part of the contract branch.
shihanzhang [Mon, 1 Jun 2015 08:17:37 +0000 (16:17 +0800)]
Fix ipset can't be destroyed when last rule is deleted
when it deletes a security group all rules, it should
include this sg information in RPC method
'security_group_info_for_devices', otherwise the ports
in this sg can't corrcectly update their iptables and
ipset sets.
Ihar Hrachyshka [Sun, 2 Aug 2015 20:40:40 +0000 (22:40 +0200)]
Cleaned up some TODO comments for feature/qos that do not apply anymore
1. Removed comment to add tests for invalid input for attach/detach
methods for QosPolicy. Those tests are already implemented in the test
class, so the TODO became obsolete.
2. Removed TODO to use a constant for rule 'type' field. There is no
'type' field in rules anymore, so it does not apply any more.
Move away nested transaction from _ensure_default_security_group
This patch remove the nested transaction started in
_ensure_default_security_group, before calling create_security_group.
Instead, a nested transaction will be started inside
create_security_group.
The purpose of this change is to reduce the time period in which the db
transaction is open (to avoid races, db lock timeouts, etc), when creating
the default security-group.
OVSBridge was inheriting db_list from BaseOVS, which was
returning the information of all the ports on the machine,
not only the ones belonging to the bridge.
The OVSNeutronAgent was using that method with the assumption
that ports were filtered by bridge.
To avoid confusion, this patch add a new method to OVSBridge
get_ports_attributes to query the info for all the ports
belonging to the bridge.
db_list is removed from BaseOVS since that method is already
available in ovsdb/api.py
ovs_lib methods that use db_list are refactored accordingly.
Also renamed service_notification_drivers into notification_drivers
since it's clear where it belongs anyway (it's in neutron.conf meaning
it's a server side configuration value).
The only values in qos_rules table are: type, id and qos_policy_id. Both
id fields point to qos_*_rules and qos_policies objects.
Type is redundant since qos_rule and qos_*_rule objects maintain 1-to-1
relationship.
Keeping a separate table just to link qos_*_rule and qos_policy objects
has no meaning. At the same time, it complicates the code for rule
objects significantly.
So instead of copying with all those issues, we just squash the tables
into single one. It allows us to reuse all base methods from
NeutronObject for rules.
LOC stats for the patch clearly shows the point:
65 insertions(+), 267 deletions(-)
And no actual functionality is lost.
While at it, the following changes were applied:
- some base tests are reimplemented to test objects in a more explicit
way;
- fields_no_update class attribute is now actually enforced in base
object class.
John Schwarz [Sun, 26 Jul 2015 13:00:12 +0000 (16:00 +0300)]
Fix accessing shared policies, add assoc tests
This patch is two-fold:
1. Previously, policies that were created using the 'shared=True' flag
were not accessible to other tenants, since the context used to
search the policies was not elevated. This patch elevates the context
prior to retrieving the policy, and if a match was found, makes sure
that the user has permissions to access it (either the policy is
shared or it's from the same tenant id).
2. Tests for both associations and disassociations of policies to both
networks and ports are added in this patch, to make sure coverage is
good and that the problem is fixed.
Code Review / openstack-build / neutron-build.git / log