Ollie Leahy [Thu, 1 Oct 2015 17:21:52 +0000 (18:21 +0100)]
encryption_api_url requires a version
If the value configured for encryption_api_url does not include the
barbican API version, then some calls from cinder will fail. This can
mean that encrypted volumes cannot be deleted. To prevent this happening
raise an exception if the configured value for encryption_api_url does not
include the barbican version.
Victor Stinner [Wed, 7 Oct 2015 15:38:12 +0000 (17:38 +0200)]
Port WSGI tests to Python 3
* Replace dict.keys()[0] with list(data.keys())[0]. On Python 3,
dict.keys() now returns a view which is not indexable.
* Skip SSL tests on Python 3. Tests hang for an unknown reason, they
must be fixed later.
* Fix unit tests: HTTP body type is bytes, not Unicode.
* Debug.print_generator(): on Python 3, write into sys.stdout.buffer
instead of sys.stdout, because HTTP body type is bytes not Unicode.
* ResponseObject: encode serializer output to UTF-8 if it's Unicode.
* tox.ini: add the following tests to Python 3.4
Note: Ignore pylint error E1101 on sys.stdout.buffer. pylint on
Python 2 complains that the buffer attribute doesn't exist, whereas
the code is only executed on Python 3 and the attribute exists on
Python 3.
Use lvm_conf_file directory for LVM_SYSTEM_DIR value.
Currently when LVM_SYSTEM_DIR is added to LVM.LVM_CMD_PREFIX it is
hardcoded as /etc/cinder.
However the configuration option lvm_conf_file could specify a file
located in an arbitrary directory, not just /etc/cinder, so we should
be using the path to the directory containing the specified lvm.conf
file as the value for LVM_SYSTEM_DIR.
John Griffith [Fri, 9 Oct 2015 23:31:15 +0000 (17:31 -0600)]
Update config format for replication_devices
The first pass at setting up config entries for
replication targets was a hacky custom string parser
with its own unique syntax.
A better option is to use oslo.cfg's MultiOpt in conjunction
with Dict types so we can eliminate quite a bit of custom
parsing and most importantly use a standard syntax for conf
entries to make things easy for admins and reduce the
probability of syntax errors and mistakes in config setttings.
There is a strict rule about constructing status line for HTTP:
'...Status-Line, consisting of the protocol version followed by a
numeric status code and its associated textual phrase, with each
element separated by SP characters'
(http://www.faqs.org/rfcs/rfc2616.html)
This patch coerces filling associated textual phrase.
Also removed unused code from cinder/tests/unit/test_exception.py
Change-Id: Ia9099fb5020cee02bfee2cd0e8e111845918c36c
Closes-Bug: #1496055 Co-Authored-By: Marian Horban <mhorban@mirantis.com>
Victor Stinner [Mon, 12 Oct 2015 08:35:25 +0000 (10:35 +0200)]
Fix test_misc for WebOb 1.5
WebOb 1.5 was released at 2015-10-11. With this new version,
webob.exc.WSGIHTTPException() constructor now fails with a KeyError
when the HTTP status code is 0.
test_exceptions_raise() of test_misc tries to instanciate all
exceptions of cinder.exception. The problem is that
ConvertedException uses a default HTTP status code of 0.
Modify the default HTTP status code of ConvertedException to 400 to
fix the unit test. The bug is only in the test,
cinder/api/openstack/wsgi.py copies an existing HTTP code:
Alex O'Rourke [Fri, 9 Oct 2015 18:10:38 +0000 (11:10 -0700)]
Fix error string format for replication API calls
enable_replication, disable_replication, and failover_replication all
raise exceptions if the volume they are being called on are not in
valid states. When that is done, the error message is not properly
formatted and looks as such:
"ERROR: Invalid volume: (u'Invalid status to enable replication. valid
states are: %(valid_states)s, current replication-state is:
%(curr_state)s.', {'curr_state': u'enabled', 'valid_states':
['disabled']}) (HTTP 400)"
The reason for this is the msg string is using a ',' instead of a '%'
to evaluate the string. When fixed, the error is displayed correctly:
"ERROR: Invalid volume: Invalid status to enable replication. valid
states are: ['disabled'], current replication-state is: enabled.
(HTTP 400)"
Victor Stinner [Wed, 7 Oct 2015 16:02:43 +0000 (18:02 +0200)]
Port IBM flashsystem to Python 3
* Replace dict.keys()[0] with list(dict.keys())[0]. On Python 3,
dict.keys() now returns a view which is not indexable.
* tests-py3.txt: add cinder.tests.unit.test_ibm_flashsystem
Victor Stinner [Wed, 7 Oct 2015 16:07:38 +0000 (18:07 +0200)]
Port ceph driver to Python 3
* Replace xrange() with range(). cinder/volume/utils.py uses
"from six.moves import range", so "range" is xrange on Python 2.
* Replace '' with b'' for image content to get a bytes string on
Python 3.
Note: _transfer_data() of cinder.volume.utils is tested by
test_backup_ceph which is already run on Python 3, but the test only
failed with python3 run with -bb. This bug is now fixed.
Tom Barron [Thu, 8 Oct 2015 01:10:48 +0000 (21:10 -0400)]
Remove references to Swift in chunked driver
After the chunked/Swift backup driver refactor in commit 2f701a39208f9396893cd8a57165e37cfeb6fc2f
some references to 'Swift' remained in code comments and
log messages in code that no longer lives in the Swift driver
proper.
Tom Barron [Fri, 2 Oct 2015 20:49:46 +0000 (16:49 -0400)]
Add insecure option for swift backup
When running backups with the swift backup driver and an
https url for the swift endpoint, the config option
backup_swift_ca_cert_file must point to a valid server side
cert.
For test purposes, it is useful to be able to ignore
cert errors while still making a SSL/TLS connection, the way
one can with https:// urls and the curl command and the
swift client itself.
This commit adds a boolean CONFIG option, 'backup_swift_auth_insecure',
whose default value is False, in which case there is no change from
current behavior. If this option is set to True, however, cert validation
is skipped for SSL/TLS.
Xiaoqin Li [Sun, 4 Oct 2015 17:59:37 +0000 (10:59 -0700)]
Add fast format option for thick volume creation
Add option for user to disable fast format at thick
provisioned volume creation in IBM Storwize cinder.
The default value is False and clients who do not want
to do Fast Format for thick volume, they can set this
parameter to True.
DocImpact: Add fast format option
Change-Id: Ib627696b91c336d09de6066ee70ee518fca5a7a9
Closes-Bug: 1502404
Xi Yang [Tue, 1 Sep 2015 10:05:15 +0000 (06:05 -0400)]
Retype enhancement for EMC VNX cinder driver
VNX driver will trigger lun migration when retyping a thin/thick
volume to a compressed volume and it is time-consuming. To enhance
the performance, this patch will directly turn on the compression
on a volume instead of migrating it.
John Griffith [Tue, 6 Oct 2015 23:15:58 +0000 (17:15 -0600)]
Verify volume is replication capable
The V1 implementation of replication added a replication_status
field to the Volume object and set it to disabled by default on
volume creation.
This is problematic, because there's no way for the API or any
caller to know if the status is disabled because it was
explicitly set that way, or if it is because the back end or
volume-type do not support replication.
This results in cases like enable replication (which does a status
check on disabled) to be called inappropriately on devices that
don't support replication.
This patch adds a decorator to the new replication methods which will
check that the volume is of type with replication_enabled=True before
attempting any replication related operations.
Victor Stinner [Wed, 7 Oct 2015 17:01:57 +0000 (19:01 +0200)]
Port violin driver to Python 3
* Replace dict.items()[0] with list(dict.items())[0], same for
dict.values()
* Replace dict.iteritems() with dict.items()
* Replace dict.itervalues() with dict.values()
* Replace a/b with a//b to get integer on Python 3.
* test_get_active_fc_targets(): ignore order when comparing active
FC targets. On Python 3, the hash function is randomized and so
dictionaries values are returned in a random order.
Victor Stinner [Wed, 7 Oct 2015 15:57:00 +0000 (17:57 +0200)]
Port EMC scaleio to Python 3
* Import urllib modules using six.moves.urllib
* MockHTTPSResponse: encode HTTP body to UTF-8 if it's Unicode,
replace six.string_types with (bytes, six.text_type).
* _id_to_base64(): catch also binascii.Error when calling
b16decode(); encode name to UTF-8 before calling b64encode();
decode b64encode() from ASCII on Python 3 to get Unicode.
* tox.ini: add cinder.tests.unit.volume.drivers.emc.scaleio
to Python 3.4
John Griffith [Tue, 6 Oct 2015 23:39:24 +0000 (17:39 -0600)]
Fix SolidFire target composition
We added the target composition a while back but never
really finished it. We created a set of interface methods
and had them call the target class mostly thinking we'd
want this when adding FC.
It turns out that doing a regular composition and using
get_attr will actually work fine when the FC target is
added, so re factor this and make it a true composition
where both objects inherit attributes from each other.
Victor Stinner [Wed, 7 Oct 2015 15:14:54 +0000 (17:14 +0200)]
Port targets test_iet_driver to Python 3
* Replace StandardError with MemoryError: StandardError was removed
in Python 3
* Replace __builtin__ with six.moves.builtins to mock open()
* Mock the logger in test_create_iscsi_target() because the test
injects errors in open() and the logger uses open() to format the
traceback
* tox.ini: add cinder.tests.unit.targets.test_iet_driver to Python 3.4
Eric Harney [Tue, 6 Oct 2015 19:31:03 +0000 (15:31 -0400)]
test_backup_nfs: Set volume id per test
Since some of these tests trigger operations (such as
progress reporting) that can happen as an asyncronous
task, set a different ID for each test so it's possible
to identify where notifications are coming from.
Eric Harney [Tue, 6 Oct 2015 18:14:00 +0000 (14:14 -0400)]
test_backup_swift: Set volume id per test
Since some of these tests trigger operations (such as
progress reporting) that can happen as an asyncronous
task, set a different ID for each test so it's possible
to identify where notifications are coming from.
rajinir [Mon, 5 Oct 2015 20:56:40 +0000 (15:56 -0500)]
Dell Eqlx: Support over subscription in thin provisioning
Dell eqlx driver reports the following new stats:
* provisioned_capacity
* max_over_subscription_ratio
* thin_provisioning_support (True or False)
* thick_provisioning_support (True or False)
Kendall Nelson [Tue, 15 Sep 2015 01:37:00 +0000 (20:37 -0500)]
Hacking check for opt name registration
Depending on how opts are registered (either with register_opt() or
register_opts()), the name needs to be singular or plural to match
the method. This patch adds a hacking check to make sure the names
of the opts and opt lists (or tuples) are correct given how they are
being registered. The check also verifies that a single option is
sent when register_opt() is used and a list is used when using
register_opts().
Includes fixes to files that don't meet the naming convention and a
addition to the generate_cinder_opts.py file in order to skip
checks.py in the generation of the opts.py file.
John Griffith [Mon, 5 Oct 2015 23:22:30 +0000 (17:22 -0600)]
Add ability to set prefix in SolidFire Volume name
The SolidFire creates names for Volumes on the backend using
the format UUID-<cinder-id>
This works well, but it turns out that there is a possibility
of a UUID collision for customers running multiple clouds against
the same cluster. To eliminate that possibility we should allow
a config option to set the prefix to be whatever an admin chooses.
This patch introduces the sf_volume_prefix config option to
address this bug. The default still uses the 'UUID-' prefix.
DocImpact
Changing this setting on an existing deployment will
result in the existing volumes being inaccessible. To introduce
this change to an existing deployment it's recommende to
add the Cluster as if it were a second backend and disable
new deployments to the current backend.
Gorka Eguileor [Fri, 2 Oct 2015 13:08:31 +0000 (15:08 +0200)]
Detach volume on device validation failure
If device validation fails when attaching a volume for some driver
operation (copy_volume_data, copy_image_to_volume, copy_volume_to_image)
we may end up with an attached volume that we don't cleanup.
This patch tries to detach the volume if we fail when validating the
device after we have attached the volume.
This happens for example on multipath when we have properly detected the
paths but they are all in a failed state when we try to read from the
device on validation.
Wilson Liu [Tue, 22 Sep 2015 12:52:36 +0000 (20:52 +0800)]
Fix capacity report error in Huawei driver
Currently the capacity report is incorrect due
to wrong way code clear.
The previously right way is:
result = var / 1024.0 / 1024.0 / 2
But currently we use a constant CAPACITY_UNIT to make the code cleaner
like this:
CAPACITY_UNIT = 1024.0 / 1024.0 / 2
result = var / CAPACITY_UNIT
We want the CAPACITY_UNIT work like a *real* constant like C
language, but in python, it is not a *real* constant, it's
just a variable. It will calculate the CAPACITY_UNIT first,
so the finally result is:
result = var / 0.5
And this is wrong.
Xing Yang [Thu, 1 Oct 2015 02:28:56 +0000 (22:28 -0400)]
VMAX Target iSCSI IP Address
In VMAX iSCSI driver, the iscsi_ip_address was hardcoded
in cinder.conf. This may have issues with multi-portgroup
environment. If a customer has multiple portgroups
containing different ports with different iSCSI IP addresses,
then ‘iscsiadm‘ command cannot use more than the one hardcoded
IP address in its sendtargets operation.
This patch addresses this by examining the ports in the
portgroup of the masking view used in the attach operation.
Each port has a corresponding iSCSI IP address. A portgroup
with only one port will have one IP address; a portgroup with
multiple ports will have multiple IP addresses. Even if there
is more than one IP address, the first in the list is likely to
result in a successful iscsiadm -sendtargets.
Code Review / openstack-build / cinder-build.git / log