Cedric Brandily [Wed, 27 May 2015 18:30:28 +0000 (20:30 +0200)]
Do not assume order of security group rules
This fixes the unit tests[1] that breaks with a randomized
PYTHONHASHSEED (see the bug report).
The test assumed that the security_group_rules_for_devices method from
neutron.agent.securitygroups_rpc returned security group rules in a
particular order. Found with PYTHONHASHSEED=2.
The fix refactors the test case to handle unsorted security group rules.
Note: There are several other unrelated unit tests that also break with
a randomized PYTHONHASHSEED, but they are not addressed here. They will
be addressed in separate patches.
Note: There are several other unrelated unit tests that also break with
a randomized PYTHONHASHSEED, but they are not addressed here. They will
be addressed in separate patches.
Kevin Benton [Wed, 27 May 2015 21:52:06 +0000 (14:52 -0700)]
Remove time formatting in agent clock error
This removes time formatting that may be hiding timezone
issues that are leading to a delta being calculated between
the agent and the server even when it shows none. It also
adds logging of the difference so we can see how far off it
thinks they are.
Example message:
during the registration of Open vSwitch agent has a timestamp:
2015-05-19T18:15:27Z. This differs from the current server
timestamp: 2015-05-19T18:15:27Z by more than the threshold agent
downtime: 75.
Note that the timestamps are exactly the same after formatting.
Kyle Mestery [Wed, 27 May 2015 17:26:00 +0000 (17:26 +0000)]
Flesh out the new RFE process and set deadlines for it's use
The new RFE process is great in concept, but as was discovered in the
first neutron-drivers meeting where we discussed these, there exist
some rough edges. Specifically around deadlines and the conversion to
using RFEs, the gray area was very obvious. This patch attempts to put
a stake in the ground for when we transition fully to this new model,
including distinct timelines.
Given that we will need to work with people during the transition,
what is proposed is a way to let us do that while not blocking existing
specs and work.
Cedric Brandily [Wed, 27 May 2015 17:57:04 +0000 (19:57 +0200)]
Do not assume order of dictionary elements in init_l3
This fixes the test_interface unit tests[1] that breaks with a
randomized PYTHONHASHSEED (see the bug report).
The test assumed that the init_l3 method from
neutron.agent.linux.interface had dictionary elements in a particular
order. Found with PYTHONHASHSEED=2.
The fix refactors the test case to handle unsorted dictionaries in
init_l3.
Note: There are several other unrelated unit tests that also break with
a randomized PYTHONHASHSEED, but they are not addressed here. They will
be addressed in separate patches.
As Neutron has grown, the need to continually evolve the way we land
code has changed. As the project has grown, it's become obvious we
need to subdivide responsibilites and merge rights across the
repository. The advanced services split [1] was one way in which we did
this, and it's proven effective. Plugin decomposition was another [2].
As a next step to evolve things, this changes the core reviewer
paradigm into a system of Lieutenants, all reporting to the PTL in the
project. The Lieutenants will be able to maintain their own core
review team. The idea of "super cores" will fade away. And we begin to
see the benefits of a layered, structured approach to managing
Neutron.
This proposal makes an attempt to comment on existing core reviewers.
While the idea of existing core reviewers being labeled as "supercores"
has been floated, the reality is a bit more nuanced than that, and I've
taken steps to address that with this patch. It should be noted that like
all things in Neutron governance, the system is based on a mutual trust.
Cedric Brandily [Wed, 27 May 2015 12:54:35 +0000 (14:54 +0200)]
Do not assume order of convert_kvp_list_to_dict method responses
This fixes the test_attributes unit tests[1] that breaks with a
randomized PYTHONHASHSEED (see the bug report).
The test assumed that the convert_kvp_list_to_dict method from
neutron.api.v2.attributes returned elements in a particular
order. Found with PYTHONHASHSEED=1.
The fix refactors the test case to handle unsorted responses from
convert_kvp_list_to_dict.
Note: There are several other unrelated unit tests that also break with
a randomized PYTHONHASHSEED, but they are not addressed here. They will
be addressed in separate patches.
Cedric Brandily [Tue, 26 May 2015 17:54:22 +0000 (19:54 +0200)]
Do not assume order of iptables_firewall method responses
This fixes the iptables_firewall group unit tests[1] that breaks with
a randomized PYTHONHASHSEED (see the bug report).
The test assumed that the _get_remote_sg_ids and
_determine_remote_sgs_to_remove methods from
neutron.agent.linux.iptables_firewall returned elements in a particular
order. Found with PYTHONHASHSEED=1.
The fix refactors the test case to handle unsorted responses from
_get_remote_sg_ids and _determine_remote_sgs_to_remove.
Note: There are several other unrelated unit tests that also break with
a randomized PYTHONHASHSEED, but they are not addressed here. They will
be addressed in separate patches.
Cedric Brandily [Tue, 26 May 2015 22:23:09 +0000 (00:23 +0200)]
Do not assume order of get_sync_data_metering response elements
This fixes the test_add_metering_label_rpc_call[1] unit tests that breaks
with a randomized PYTHONHASHSEED (see the bug report).
The test assumed that the get_sync_data_metering[2] had response
elements in a particular order. Found with PYTHONHASHSEED=1.
The fix refactors the test_add_metering_label_rpc_call test case to handle
an unsorted get_sync_data_metering response. The fix defines the class
UnorderedList[3] which is equal to any permutation of itself.
Note: There are several other unrelated unit tests that also break with
a randomized PYTHONHASHSEED, but they are not addressed here. They will
be addressed in separate patches.
Cedric Brandily [Tue, 26 May 2015 13:14:11 +0000 (13:14 +0000)]
Force order of dhcp.needs_resync_reasons dictionary elements
This fixes the test_periodoc_resync_helper unit test that breaks with
a randomized PYTHONHASHSEED (see the bug report).
The test assumed that the dhcp.needs_resync_reasons dictionary from
neutron.agent.dhcp.agent had elements in a particular order. Found with
PYTHONHASHSEED=2.
The fix refactors the test case to force a sorted dhcp.needs_resync_reasons
dictionary.
Note: There are several other unrelated unit tests that also break with
a randomized PYTHONHASHSEED, but they are not addressed here. They will
be addressed in separate patches.
Henry Gessau [Mon, 25 May 2015 22:00:58 +0000 (18:00 -0400)]
Skip external tables for neutron-db-manage --autogenerate
DB tables that do not have models in the neutron tree cause
neutron-db-manage --autogenerate to create commands to drop the
tables. This fix hooks into alembic's environment with a include_object
callback that ignores external tables.
We already had a list of external tables for use by the migration tests,
so re-use them for --autogenerate.
John Schwarz [Thu, 16 Apr 2015 09:01:26 +0000 (12:01 +0300)]
Add callback prior to deleting a subnet
When using LBaaS and trying to delete a subnet, neutron has no way of
knowing if the subnet is associated to some pool. As a result, the
subnet is deleted but the pool remains associated to the (now
nonexistent) subnet_id. This patch lays the ground-work for adding a
check in LBaaS' side to prevent such cases.
OVS-agent: Separate ovs-ofctl using code as a driver
This is a preparation to introduce another Ryu-based implementation.
The aim is to replace this with the new Ryu-based implementation
eventually.
Add a config option for OVS-agent which selects the implementation.
Currently, the only available choice is 'ovs-ofctl'.
Also, this commit simplifies DVR logics by reducing duplications
and makes some of DVR UTs actually check the flows rather than just
"add_flow is called".
Anand Shanmugam [Sat, 23 May 2015 08:22:23 +0000 (01:22 -0700)]
Ensure mac address added to iptables is always in unix format
When a allowed address pair entry is added with a mac format
other than unix format the ovs-vs agent keeps on restarting as
it is not able to save the proper iptables due to the error
"Error while processing VIF ports". This fix makes sure
that the mac address sent to the iptables firewall is always
in the unix format
ankitagrawal [Thu, 14 May 2015 11:08:36 +0000 (04:08 -0700)]
Remove use of contextlib.nested
Removed use of contextlib.nested call from codebase, as it has been
deprecated since Python 2.7.
There are also known issues with contextlib.nested that were addressed
by the native support for multiple "with" variables. For instance, if
the first object is created but the second one throws an exception,
the first object's __exit__ is never called. For more information see
https://docs.python.org/2/library/contextlib.html#contextlib.nested
contextlib.nested is also not compatible with Python 3.
Multi-patch set for easier chunks. This one addresses the
neutron/plugins/ml2 directory.
Line continuation markers (e.g. '\') had to be used or syntax
errors were thrown. While using parentheses is the preferred way
for multiple line statements, but in case of long with statements
backslashes are acceptable.
Anand Shanmugam [Thu, 21 May 2015 09:03:33 +0000 (02:03 -0700)]
Adding loadbalanacerv2 device owner constant to neutron constants
The neutron constants doesn't have the constant for device owner
lbaasv2. This fix adds the constant. This is needed for the bug 1430394
as we need to check the device owner when the port is to be deleted.
Partial-Bug: #1430394
Sripriya [Thu, 21 May 2015 00:24:16 +0000 (17:24 -0700)]
Fix minor errors in the Vyatta L3 Plugin:
update management_network to management_network_id in vrouter.ini
Fix copyright header to refer to Brocade in vrouter_neutron_plugin.py
Fix neutron.service_plugins brocade_vyatta_l3 entry in setup.cfg
Ihar Hrachyshka [Wed, 20 May 2015 21:17:19 +0000 (23:17 +0200)]
Remove middleware oslo-incubator module
The module was used during Kilo cycle to provide backwards compatibility
for users that upgrade to the release without updating their
api-paste.ini. We have issued the deprecation warning for a cycle now,
so we should be ok to just drop the compatibility layer.
Note that the change may require a notion in release notes to make sure
everyone is notified, even if they don't look through their logs.
Kevin Benton [Sat, 16 May 2015 02:44:16 +0000 (19:44 -0700)]
Match order of iptables arguments to iptables-save
The way we were forming our iptables rules was not matching
the output of iptables-save. This caused the logic that preserves
counters to miss many of the rules.
This patch corrects the order for the comments and the allowed address
pairs to match the output order of iptables-save.
Gary Kotton [Fri, 15 May 2015 15:12:54 +0000 (08:12 -0700)]
VMware NSXV: update configuration file
Update the configuration file to show the variables for configuring
the Edge username and password. This is very useful for administrators
when they wish to debug issues.
An alternate pluggable IPAM implementation from the built-in one
in db_base_plugin_v2.
Modifies IPAM interface to allow passing context to driver and
introduces new interface method 'associate_neutron_subnet'.
The function `xrange` was renamed to `range` in Python 3.
* Remove `xrange` occurences so that Python 3 tests can pass. Use
`six.moves.range` instead to get the right function in both cases.
* Generalize the use of the efficient `range` (ex-`xrange`) in
critical sections (when iterating over large lists).
* Simplify code.
* Add a hacking check to prevent future usage of `xrange`.
Cedric Brandily [Fri, 15 May 2015 16:10:09 +0000 (18:10 +0200)]
Use namespace names in NetcatTester
Currently NetcatTester requests namespace IPWrapper instances as
client/server_namespace arguments but functional tests commonly use
namespace names as arguments not IPWrapper instances (because IPWrapper
is cheap to instantiate) and NetcatTester needs only namespace names.
This change requests names as NetcatTester client/server_namespace
arguments in order to simplify NetcatTester and its use.
Assaf Muller [Fri, 15 May 2015 21:58:13 +0000 (17:58 -0400)]
Optimize IptablesManager._find_last_entry
As it turns out calling .strip() thousands of times can be expensive.
I'll defer to security groups and iptables experts to try and find ways
to call the method less often, cache the results, or any other clever
trick.
Moving strip to the return statement speeds up the method by more than
x2.
Take Daemon stdin/stdout/stderr args as file objects
Previously Daemon constructor took stdin/stdout/stderr as
paths (defaulting to '/dev/null') and opened them as regular files.
This greatly limits the type of filehandles supported (no pipes, for
example), and doesn't allow simple things like reusing existing fds.
This change switches to accepting file objects rather than strings,
and uses a sentinal value to represent the previous "open /dev/null"
default behaviour.
John Schwarz [Thu, 7 May 2015 12:22:41 +0000 (15:22 +0300)]
Support for concurrent full-stack tests
This patch introduces the last step of complete test separation for
full-stack tests - separate rabbitmq queues - in the form of rabbitmq
virtual hosts.
* This patch also renames EnvironmentFixture to FullstackFixture, for
clarity.
Jeremy Stanley [Thu, 14 May 2015 21:38:20 +0000 (21:38 +0000)]
Replace ci.o.o links with docs.o.o/infra
The http://ci.openstack.org/ documentation site has been deprecated,
replaced by redirects to corresponding paths within
http://docs.openstack.org/infra/ where other Project Infrastructure
documentation already resides.
Jakub Libosvar [Tue, 5 May 2015 12:32:21 +0000 (14:32 +0200)]
Refactor initialize() of sriov mech driver
This patch rewrites checking correctness of supported_pci_vendor_devs
config value from C-style to Python-style. Patch also adds some tests
for wrong values passed.
Oleg Bondarev [Thu, 14 May 2015 12:03:54 +0000 (15:03 +0300)]
Don't pass namespace name in disable_isolated_metadata_proxy
It's not always possible/convenient to get namespace name
when need to disable some process (like metadata process for stale
router, see related bug). Since namespace name is not required
for process manager to disable process we can remove this parameter
from disable_isolated_metadata_proxy()
Moshe Levi [Wed, 22 Apr 2015 11:17:28 +0000 (14:17 +0300)]
Add client id option support to dhcp agent
According to the dnsmasq man client id option should be
written to dhcp-hostsfile and not to the dhcp-optsfile.
Also this patch update the dhcp_release command to take
into account the client id when releasing old leases.
ankitagrawal [Thu, 14 May 2015 09:06:39 +0000 (02:06 -0700)]
Remove use of contextlib.nested
Removed use of contextlib.nested call from codebase, as it has been
deprecated since Python 2.7.
There are also known issues with contextlib.nested that were addressed
by the native support for multiple "with" variables. For instance, if
the first object is created but the second one throws an exception,
the first object's __exit__ is never called. For more information see
https://docs.python.org/2/library/contextlib.html#contextlib.nested
contextlib.nested is also not compatible with Python 3.
Multi-patch set for easier chunks. This one addresses the
neutron/tests/unit/agent/test_securitygroups_rpc.py tests.
Line continuation markers (e.g. '\') had to be used or syntax
errors were thrown. While using parentheses is the preferred way
for multiple line statements, but in case of long with statements
backslashes are acceptable.
Code Review / openstack-build / neutron-build.git / log