Russell Bryant [Fri, 21 Nov 2014 18:50:30 +0000 (18:50 +0000)]
Drop RpcProxy usage from cisco.l3
Drop usage of the RpcProxy compatibility class from cisco.l3. This
was the last of RpcProxy usage in the cisco plugin. The equivalent
APIs from oslo.messaging are now used instead.
Russell Bryant [Fri, 21 Nov 2014 14:43:26 +0000 (14:43 +0000)]
Drop RpcProxy usage from brocade plugin
This patch drops the usage of the RpcProxy compatibility class from
the brocade plugin. The equivalent usage of oslo.messaging APIs is
now used instead.
Note that there is one very minor functional change included. The
previous code set the base version to '1.1'. It should be '1.0'. The
proper pattern to use is to set the base version on the client side to
be the initial version of the API (X.0). Then, any methods that
require a newer version need to specify it.
Romil Gupta [Sun, 26 Oct 2014 13:18:53 +0000 (06:18 -0700)]
Update i18n translation for ML2 plugin log msg's
All the existing LOG.info, LOG.warning, LOG.error and LOG.critical
messages should have _LI, _LW, _LE and _LC respectively.
Also, debug level log shouldn't be translated.
This patch set will cover the ml2 directory under neutron/plugins.
Russell Bryant [Thu, 20 Nov 2014 22:22:41 +0000 (22:22 +0000)]
Drop RpcProxy usage from bigswitch plugin
This patch drops the usage of the RpcProxy compatibility class from
the bigswitch plugin. The equivalent usage of oslo.messaging APIs is
now used instead.
Note that there is one very minor functional change included. The
previous code set the base version to '1.1'. It should be '1.0'. The
proper pattern to use is to set the base version on the client side to
be the initial version of the API (X.0). Then, any methods that
require a newer version need to specify it. In this case, verison
'1.1' was the security group methods, and those specify '1.1' as you
would expect.
Implement migration of legacy routers to distributed
This patch addresses the necessary changes required
for the router migration or conversion from legacy
to distributed.
The _update_router_db() method was modified to add
logic to support the SNAT functionality of DVR by
calling create_snat_intf_ports_if_not_exists(). A
call to _unbind_router() was also called to force
the scheduling of the router with 'distributed'
enabled.
Right now we will be supporting migration from legacy
to distributed only.
Closes-bug: #1348309
Partially-implements: blueprint neutron-ovs-dvr Co-Authored-By: Michael Smith <michael.smith6@hp.com>
Change-Id: I6f252271bb5b52ce57184ad7b0bf1ce280b965fb
Russell Bryant [Thu, 20 Nov 2014 21:01:14 +0000 (21:01 +0000)]
Show progress output while running unit tests
This patch imports some magic to enable showing test output as the
tests run. Seeing output showing tests as they execute is reassuring
that the test run is continuing to make progress. This originated in
tempest and was then brought over to Nova in this commit:
https://review.openstack.org/#/c/121914/
Note the justification in that commit to just copy it instead of move
it to a shared repo.
To get the old behavior where you only get output when something
fails, you can set the TRACE_FAILONLY env var. That originated in
this commit in Nova:
Cedric Brandily [Tue, 11 Nov 2014 16:01:54 +0000 (17:01 +0100)]
Enforce log hints in neutron.services.loadbalancer
This change enforces log hints use and removes debug level log
translation, modifications are validated through a hacking rule and
the change respects loggging guidelines.
Validate that hacking rules apply to directories:
- neutron/services/loadbalancer
abhishek.talwar [Tue, 18 Nov 2014 09:50:51 +0000 (15:20 +0530)]
Check for default sec-group made case insensitive
Currently creating another security group with name
"default" is not allowed, however we can create
another security group with name "DEFAULT"
(or any other CASE pattern).
When trying to boot a VM without specifying
a security group it should always pick the
"default" security group. However, if another
security group of the name DEFAULT is present,
the VM gets associated with the wrong
security group (i.e DEFAULT and not default).
So, updated the code so that a security group
with name DEFAULT (or any other case pattern)
is not allowed.
Closes-Bug: #1384505
Brian Haley [Wed, 19 Nov 2014 17:10:57 +0000 (12:10 -0500)]
Fix a race condition adding a security group rule
setup_port_filters() needs to grab self.devices_to_refilter
before it calls prepare_devices_filter(), else it could skip
processing a device if an RPC arrives while it's processing
new devices. That device will now be handled the next time
it's called.
Russell Bryant [Tue, 18 Nov 2014 14:46:06 +0000 (14:46 +0000)]
Drop RpcProxy usage from FWaaS code
This patch removes the usage of the RpcProxy compatibility class from
the FWaaS code. The equivalent direct usage of oslo.messaging APIs
are now used instead.
Russell Bryant [Fri, 14 Nov 2014 14:21:05 +0000 (14:21 +0000)]
Drop RpcProxy usage from neutron.agent.rpc.PluginApi
This patch removes the usage of the RpcProxy compatibility class from
the neutron.agent.rpc.PluginApi class. The equivalent use of
oslo.messaging APIs have been put in place instead. This simple
conversion had a pretty wide impact on unit tests, as well.
The security groups API was converted in this patch as well. It was
necessary because the security group class is used as a mixin, so it
must be implemented the same way. Unfortunately, the way this is used
as a mix-in is not consistent, so for now it's only conditionally
converted.
Finally, some other miscellaneous plugin specific interfaces were
converted as well. Again, these were methods mixed-in for certain
plugins.
Note that there's one very minor functional difference in this patch.
The previous code set the base version to be '1.1'. The right pattern
is for this to be set to '1.0'. This version is the default version
specified by the client, telling the server that it must implement at
least this version to satisfy the request. The default should be
'1.0' and methods that require higher than that should specify it.
From looking at other parts of the code, '1.0' vs '1.1' is not
actually important, as '1.1' was actually the addition of some
security group methods defined elsewhere. The correction is more
about establishing the right pattern to follow.
Russell Bryant [Mon, 17 Nov 2014 21:33:06 +0000 (21:33 +0000)]
Fix a copy/pasted test mistake
Fix a minor test mistake that was copied around a few times. In mock
setup, the intention was to only return a value of the rpc method used
was a 'call', not 'cast'. Unfortunately, the variable checked was
method instead of rpc_method, so this expected return was always set
to None.
This also points out that what's being tested here isn't what *really*
was intended. It ends up testing that the code returns what we mocked
it to return, which isn't all that useful. However, at least with
this fix it's a bit more clear what it's trying to do.
Russell Bryant [Thu, 13 Nov 2014 21:31:10 +0000 (21:31 +0000)]
Drop several uses of RpcCallback
This patch drops several uses of the RpcCallback compatibility class.
All of these were trivial and straight forward conversions so I
batched them up.
There are still several other uses of RpcCallback, but the conversions
are were not necessarily trivial, and may be broken in one way or
another, so I wanted to address them separately. In particular, the
use of mixin classes means that there could be cases where the version
declaration is being stepped on, so they need to be investigated more
closely.
Russell Bryant [Wed, 12 Nov 2014 21:37:24 +0000 (21:37 +0000)]
Add some basic rpc api docs
The devref docs had a placeholder file for rpc API docs. Now that
both a client side and server side interface have been converted from
the rpc compat layer to oslo.messaging APIs, add some docs that give
an overview of what the client and server sides look like.
Also include a section that describes the code layout of where you can
find client and server api implementations in the neutron code base.
It starts by discussing the DHCP agent related APIs.
Russell Bryant [Tue, 11 Nov 2014 19:23:15 +0000 (14:23 -0500)]
Drop RpcProxy usage from PluginReportStateAPI
Drop the usage of the RpcProxy compatibility class from the
PluginReportStateAPI rpc client class. It now uses the appropriate
oslo.messaging APIs directly, instead.
John Perkins [Mon, 6 Oct 2014 21:24:57 +0000 (16:24 -0500)]
Fix hostname regex pattern
Current hostname_pattern regex complexity grows exponentially
when given a string of just digits, which can be exploited to
cause neutron-server to freeze.
Eugene Nikanorov [Mon, 17 Nov 2014 16:49:09 +0000 (20:49 +0400)]
Catch DBReferenceError exception during binding a router
In some cases (Concurrent API tests) router can be deleted
before it is bound to an agent by scheduler.
That may lead to traces on L3 agent side returned via RPC.
Need to handle this case in more graceful way.
Itsuro Oda [Wed, 29 Oct 2014 23:00:07 +0000 (08:00 +0900)]
Enable default SNAT from networks connected to a router indirectly
Make outgoing packets to an external interface SNATed regardless
of source address of the packets. As a result of deep review,
any problem was not found with this change.
Kevin Benton [Fri, 26 Sep 2014 04:42:39 +0000 (21:42 -0700)]
BSN: Optimistic locking strategy for consistency
Summary:
Adds an optimistic locking strategy for the Big Switch
server manager so multiple Neutron servers wanting to
communicate with the backend do not receive the consistency
hash for use simultaneously.
The bsn-rest-call semaphore is removed because serialization
is now provided by the new locking scheme.
A new DB engine is added because the consistency hashes
need a life-cycle with rollbacks and other DB operations
than cannot impact or be impacted by database operations
happening on the regular Neutron objects.
Unit tests are included for each of the new branches
introduced.
Problem Statement:
Requests to the Big Switch controllers must contain the
consistency hash value received from the previous update.
Otherwise, an inconsistency error will be triggered which
will force a synchronization. Essentially, a new backend
call must be prevented from reading from the consistency
hash table in the DB until the previous call has updated
the table with the hash from the server response.
This can be addressed by a semaphore around the rest_call
function for the single server use case and by a table lock
on the consistency table for multiple Neutron servers.
However, both solutions are inadequate because a single
Neutron server does not scale and a table lock is not
supported by common SQL HA deployments (e.g. Galera).
This issue was previously addressed by deploying servers
in an active-standby configuration. However, that only
prevented the problem for HTTP API calls. All Neutron
servers would respond to RPC messages, some of which would
result in a port update and possible backend call which
would trigger a conflict if it happened at the same time
as a backend call from another server. These unnecessary
syncs are unsustainable as the topology increases beyond
~3k VMs.
Any solution needs to be back-portable to Icehouse so new
database tables, new requirements, etc. are all out of the
question.
Solution:
This patch stores the lock for the consistency hash as a part
of the DB record. The guaruntees the database offers around
atomic insertion and constrained atomic updates offer the
primitives necessary to ensure that only one process/thread
can lock the record at once.
The read_for_update method is modified to not return the hash
in the database until an identifier is inserted into the
current record or added as a new record. By using an UPDATE
query with a WHERE clause restricting to the current state,
only one of many concurrent callers to the DB will successfully
update the rows. If a caller sees that it didn't update any
rows, it will start the process over of trying to get the
lock.
If a caller observes that the same ID has the lock for
more than 60 seconds, it will assume the holder has
died and will attempt to take the lock. This is also done
in a concurrency-safe UPDATE call since there may be many
other callers may attempt to do the same thing. If it
fails and the lock was taken by someone else, the process
will start over.
Some pseudo-code resembling the logic:
read_current_lock
if no_record:
insert_lock
sleep_and_retry if constraint_violation else return
if current_is_locked and not timer_exceeded:
sleep_and_retry
if update_record_with_lock:
return
else:
sleep_and_retry
Code Review / openstack-build / neutron-build.git / log