Cinder v2 API allows creating a volume with a quoted (single or double
quotes or even unbalanced number of quotes) display name. But when we
try to get info for such volume, we end up getting an error message
saying that no volume with such a name or ID exists. This error is due
to the inadvertent stripping of quotes from the filter in the api layer.
The api call eventually comes to check_volume_filters() in
cinder/volume/api.py. The invocation of ast.literal_eval() inside this
method strips the quotes for certain quoted strings leading to this
incorrect filtering. ast.literal_eval() is used to convert string
representations into python objects which are then used to frame the
SQL queries in the db layer. For example, the string "[1,2]" for a
filter (not the display name filter) gets converted to a list object
and results in an "IN" operation being emitted in the SQL query as
opposed to an exact match.
When display_name does not contain any quotes or contains an unbalanced
number of quotes, then ast.literal_eval() throws (just like the Python
interpreter would throw for an unquoted string literal or one with
unbalanced number of quotes). We handle this by ignoring the exception
and using the raw input value as the filter and moving on. For string
containing balanced number of quotes, such as, '"foo"',
ast.literal_eval() succeeds and returns the input with the surrounding
quotes stripped (just like how the python interpreter strips quotes
from a string literal to initialize a string var's value in memory).
To always use the raw user input string as the filter value, we can
either not pass string inputs to ast.literal_eval() or encode the
string using encode("string-escape") so that we get the original string
back after passing through ast.literal_eval(). We choose the former as
the latter buys us nothing.
Andreas Jaeger [Fri, 26 Feb 2016 18:44:07 +0000 (19:44 +0100)]
always use pip constraints
This flips the tox.ini to always use constraints installation for
all targets. It drops the extra -constraints targets in the process.
This makes it so there is no developer change required to operate in our
new world order.
Evgeny Antyshev [Sat, 27 Feb 2016 12:14:37 +0000 (12:14 +0000)]
Misprint in policy.json
Erroneous space in "rule: admin_api" was introduced in change Ibbd6f47c370d8f10c08cba358574b55e3059dcd1
oslo_policy regards space as a separator,
and fails to parse this rule: http://paste.openstack.org/show/488452/
Wilson Liu [Thu, 25 Feb 2016 03:27:19 +0000 (11:27 +0800)]
Huawei: delete_snapshot need not return any value
Currently delete_snapshot return True or False to indicate
whether the snapshot is really deleted on the array. This
is useless and just write for unit tests. Now we will remove
the return value and add another unit test for delete_snapshot.
Wilson Liu [Sat, 23 Jan 2016 02:58:46 +0000 (10:58 +0800)]
Add description when rename LUN in Huawei driver
Sometimes when we rename a LUN on the array, we should
add some description to record some information like
volume id on cinder side. This patch will enable this.
Helen Walsh [Wed, 18 Nov 2015 15:40:11 +0000 (15:40 +0000)]
EMC VMAX - Changing PercentSynced to CopyState in isSynched
In SE 8.x PercentSynced no longer works for VMAX3 as a criteria to
determine whether a device is synched or not. Luckily 'CopyState'
works for both VMAX2 and VMAX3 and is backward compatible with 7.6.x.
John Griffith [Wed, 3 Feb 2016 16:11:58 +0000 (16:11 +0000)]
Replication v2.1 (Cheesecake)
This focuses the replication work on a specific use case,
and eliminates some of the ambiguity in earlier versions.
Additionally this implementation addresses needs for
devices that do replication based on the whole backend-device
or on Pools.
Use case:
DR scenario, where a storage device is rendered inoperable.
This implementation allows the preservation of user data
for those volumes that are of type replication-enabled.
The goal is NOT to make failures completely transparent
but instead to preserve data access while an Admin tries
to rebuild/recover his/her cloud.
It's very important to note that we're no longer interested in
dealing with replication in Cinder at a Volume level. The concept
of have "some" volumes failover, and "others" left behind, proved
to not only be overly complex and difficult to implement, but we
never identified a concrete use-case where one would use failover
in a scenario where some volumes would stay and be accessible on
a primary but other may be moved and accessed via a secondary.
In this model, it's host/backend based. So when you failover,
you're failing over an entire backend. We heavily leverage
existing resources, specifically services, and capabilities.
Victor Stinner [Wed, 24 Feb 2016 17:34:43 +0000 (18:34 +0100)]
Enable all unit tests on Python 3.4
* Remove tests-py3.txt: all unit tests now pass on Python 3
* tox.ini: remove specific command for py34 test environment, it now
inherits testenv commands (same command than Python 2.7)
Michał Dulko [Fri, 26 Feb 2016 08:29:24 +0000 (09:29 +0100)]
Move deprecation release notes to correct section
During the cycle reno was updated to include "deprecations" section. We
should use this one instead of "upgrades" to add release notes about
deprecations of features or config options.
This commit moves deprecation relase notes added during the cycle to
this section.
Yucong Feng [Fri, 19 Feb 2016 21:25:19 +0000 (22:25 +0100)]
Return all target_wwpns for FC storwize
Storwize FC terminate_connection was using 'lsfabric' to lookup
target_wwpns, active on the fabric, to be used to terminate
the connection. However during a delete of a NPIV VM, the target_wwpns
is no longer active, it is not logged onto the fabric. The target list
will return empty. Fix is to return all the available wwpns
from 'storage_nodes'.
Anthony Lee [Thu, 14 Jan 2016 18:50:04 +0000 (10:50 -0800)]
LeftHand: Updating minimum client version
Updated HPE LeftHand driver to have a minimum client
version of 2.1.0.
This is a follow up to the following patches:
https://review.openstack.org/#/c/255002/
https://review.openstack.org/#/c/255544/
https://review.openstack.org/#/c/255015/
Adriano Rosso [Wed, 24 Feb 2016 13:13:38 +0000 (10:13 -0300)]
Fix HNAS iSCSI driver attachment
Currently, the HNAS iSCSI driver returns the logical unit number
related to the iSCSI target as a string while initializing the
connection.
This is breaking the attachment feature because os-brick expects
a numeric type.
This patch converts the LUN from string to int in order to solve
this problem.
Chuck Fouts [Thu, 4 Jun 2015 14:28:23 +0000 (10:28 -0400)]
NetApp: Support iSCSI CHAP Uni-directional Auth
This change adds iSCSI CHAP uni-directional authentication support for
NetApp cDOT and 7-Mode iSCSI driver.
Enabling CHAP authentication does not impact an existing iSCSI session.
The iSCSI session needs to be reestablished before CHAP authentication
is initiated.
Davanum Srinivas [Wed, 24 Feb 2016 14:48:05 +0000 (06:48 -0800)]
Avoid hardcoding value from oslo library
In I65b40b9906b75146a0085bbe168f1e6bcae82f21, the
wsgi_default_pool_size was lowered from 1000 to
100. We should not be using internal details of
the oslo libraries.
Many changes to the Cinder REST API require changes to the consumers of the API.
For example, If we need to add a required parameter to a method that is called
by Nova, we'd need both the Nova calling code and the cinderclient that
Nova uses to change. But newer Cinder versions with the change must work with
older Nova versions, and there is no mechanism for this at the moment. Adding
microversions will solve this problem.
With microversions, the highest supported version will be negotiated by a field
in the HTTP header that is sent to the Cinder API. In the case where the field
'versions' is not sent (i.e. clients and scripts that pre-date this change),
then the lowest supported version would be used. In order to ensure that the
API consumer is explicitly asking for a microversioned API, a new endpoint v3
is added, which is identical to API version v2. This means that our new
Cinder API v3 would be the default, and consumers of the API that wished to
use a newer version could do so by using that endpoint and a microversion in
the HTTP header.
New tests for microversioned API features on endpoint /v3 should be added to
cinder/tests/unit/api/v3/ directory. Existing functionality will be tested via
the .../v2/ unit tests.
Victor Stinner [Mon, 15 Feb 2016 15:51:20 +0000 (16:51 +0100)]
Port netapp dataontap driver to Python 3
With this change, all unit tests on volume drivers pass on Python 3.
Changes:
* Replace a/b with a//b to get an integer on Python 3.
* Replace map(...) and filter(...) with a list-comprehension to get a
list on Python 3.
* Replace dict.keys() with list(dict.keys()) to get a list
on Python 3.
* Replace (str, int, float, long) with
six.integer_types + (str, float): long type was removed
from Python 3.
* decode_base32_to_hex(): on Python 3, decode encode_hex_to_base32()
to return a Unicode string.
* convert_es_fmt_to_uuid(): encode/decode on Python 3 to pass the
right types to base64.b32decode() (bytes) and uuid.UUID()
(Unicode).
* fakes.py: Change type of XML document from Unicode to bytes: add
b prefix to literal strings (b'...').
* Replace range(a, b) with list(range(a, b)) to get a list
on Python 3.
* tests-py3.txt: add cinder.tests.unit.volume.drivers, all volume
drivers tests now pass on Python 3.
LisaLi [Fri, 19 Feb 2016 08:28:36 +0000 (09:28 +0100)]
Scalable backup service - Liberty compatibility
To support rolling upgrades we need to make sure that Mitaka's services
are running fine with Liberty's. It gets complicated with backups as
we've strongly reworked them. Main difference is that Mitaka c-bak can
handle backup/restore of any volume and Liberty was restricted to
operate only on volumes placed on the same node.
Now when running in version heterogeneous environment we need to use old
way of backup jobs scheduling and switch to new one (round robin) only
when everything is running Mitaka.
This commit implements that by adding a dummy backup RPC API version
(1.3) that marks the beginning of scalable backups era. Jobs are
scheduled the new way only if every c-bak reports that (or higher)
version.
There are also small changes to volume.rpcapi - to fail fast if some
c-vol services aren't supporting new calls required by scalable backups
feature. This allows us to error out backups with proper message when
upgrade was done in an improper way (in Mitaka we require c-vols to be
upgraded before c-baks).
This commit also includes small changes to CinderObjectSerializer to
block tries to "forwardport" an object when sending it over RPC. If a
service receives an older object it should handle it explicitly.
Related-Blueprint: scalable-backup-service Co-Authored-By: Michal Dulko <michal.dulko@intel.com>
Change-Id: I45324336ba00726d53cfa012e8bd498868919a8c
Wilson Liu [Wed, 24 Feb 2016 07:48:56 +0000 (15:48 +0800)]
Huawei: Log the IP when login fails
Now we support configuring multi-ips,
if one ip login fails, we will try to
connect to other ips. The problem is,
if one ip login fails, the log message
did not tell users which ip fails.
Wilson Liu [Sat, 23 Jan 2016 02:32:43 +0000 (10:32 +0800)]
Huawei: Don't fail when port group does not exist
Currently when we request a port group but the
port group does not exist, an error will be raised.
This is not appropriate, because port group not
exists is a normal case.
Tom Barron [Tue, 23 Feb 2016 02:17:25 +0000 (21:17 -0500)]
Fix invalid uuid warnings in object unit tests
Currently 'tox -epy27 -- --regex cinder.tests.unit.objects' emits
74 FutureWarnings from oslo.versionedobjects about invalid uuids
as documented here[1].
This commit changes these tests to use valid uuids so that when
they are run these warnings are no longer emitted.
Helen Walsh [Tue, 16 Feb 2016 22:33:24 +0000 (22:33 +0000)]
EMC VMAX - not cleaning up HW Resource WWPN initiators
On cleanup, the last volume is unmapped and the masking view and
storage group are deleted. Now the initiator group and the associated
StorageHardwareIDs are also deleted as long as that initiator group
is no longer contained in another masking view.
Code Review / openstack-build / cinder-build.git / log