Some of the unit tests use mox instead of mock. Also, there are
cases where a single test tests multiple cases and methods. This
patch refactors the unit tests for the following methods in the
vmdk module to fix these issues:
Alex O'Rourke [Fri, 9 Oct 2015 18:10:38 +0000 (11:10 -0700)]
Fix error string format for replication API calls
enable_replication, disable_replication, and failover_replication all
raise exceptions if the volume they are being called on are not in
valid states. When that is done, the error message is not properly
formatted and looks as such:
"ERROR: Invalid volume: (u'Invalid status to enable replication. valid
states are: %(valid_states)s, current replication-state is:
%(curr_state)s.', {'curr_state': u'enabled', 'valid_states':
['disabled']}) (HTTP 400)"
The reason for this is the msg string is using a ',' instead of a '%'
to evaluate the string. When fixed, the error is displayed correctly:
"ERROR: Invalid volume: Invalid status to enable replication. valid
states are: ['disabled'], current replication-state is: enabled.
(HTTP 400)"
Victor Stinner [Wed, 7 Oct 2015 16:02:43 +0000 (18:02 +0200)]
Port IBM flashsystem to Python 3
* Replace dict.keys()[0] with list(dict.keys())[0]. On Python 3,
dict.keys() now returns a view which is not indexable.
* tests-py3.txt: add cinder.tests.unit.test_ibm_flashsystem
Victor Stinner [Wed, 7 Oct 2015 16:07:38 +0000 (18:07 +0200)]
Port ceph driver to Python 3
* Replace xrange() with range(). cinder/volume/utils.py uses
"from six.moves import range", so "range" is xrange on Python 2.
* Replace '' with b'' for image content to get a bytes string on
Python 3.
Note: _transfer_data() of cinder.volume.utils is tested by
test_backup_ceph which is already run on Python 3, but the test only
failed with python3 run with -bb. This bug is now fixed.
Tom Barron [Thu, 8 Oct 2015 01:10:48 +0000 (21:10 -0400)]
Remove references to Swift in chunked driver
After the chunked/Swift backup driver refactor in commit 2f701a39208f9396893cd8a57165e37cfeb6fc2f
some references to 'Swift' remained in code comments and
log messages in code that no longer lives in the Swift driver
proper.
Tom Barron [Fri, 2 Oct 2015 20:49:46 +0000 (16:49 -0400)]
Add insecure option for swift backup
When running backups with the swift backup driver and an
https url for the swift endpoint, the config option
backup_swift_ca_cert_file must point to a valid server side
cert.
For test purposes, it is useful to be able to ignore
cert errors while still making a SSL/TLS connection, the way
one can with https:// urls and the curl command and the
swift client itself.
This commit adds a boolean CONFIG option, 'backup_swift_auth_insecure',
whose default value is False, in which case there is no change from
current behavior. If this option is set to True, however, cert validation
is skipped for SSL/TLS.
Xiaoqin Li [Sun, 4 Oct 2015 17:59:37 +0000 (10:59 -0700)]
Add fast format option for thick volume creation
Add option for user to disable fast format at thick
provisioned volume creation in IBM Storwize cinder.
The default value is False and clients who do not want
to do Fast Format for thick volume, they can set this
parameter to True.
DocImpact: Add fast format option
Change-Id: Ib627696b91c336d09de6066ee70ee518fca5a7a9
Closes-Bug: 1502404
Xi Yang [Tue, 1 Sep 2015 10:05:15 +0000 (06:05 -0400)]
Retype enhancement for EMC VNX cinder driver
VNX driver will trigger lun migration when retyping a thin/thick
volume to a compressed volume and it is time-consuming. To enhance
the performance, this patch will directly turn on the compression
on a volume instead of migrating it.
John Griffith [Tue, 6 Oct 2015 23:15:58 +0000 (17:15 -0600)]
Verify volume is replication capable
The V1 implementation of replication added a replication_status
field to the Volume object and set it to disabled by default on
volume creation.
This is problematic, because there's no way for the API or any
caller to know if the status is disabled because it was
explicitly set that way, or if it is because the back end or
volume-type do not support replication.
This results in cases like enable replication (which does a status
check on disabled) to be called inappropriately on devices that
don't support replication.
This patch adds a decorator to the new replication methods which will
check that the volume is of type with replication_enabled=True before
attempting any replication related operations.
Victor Stinner [Wed, 7 Oct 2015 17:01:57 +0000 (19:01 +0200)]
Port violin driver to Python 3
* Replace dict.items()[0] with list(dict.items())[0], same for
dict.values()
* Replace dict.iteritems() with dict.items()
* Replace dict.itervalues() with dict.values()
* Replace a/b with a//b to get integer on Python 3.
* test_get_active_fc_targets(): ignore order when comparing active
FC targets. On Python 3, the hash function is randomized and so
dictionaries values are returned in a random order.
Victor Stinner [Wed, 7 Oct 2015 15:57:00 +0000 (17:57 +0200)]
Port EMC scaleio to Python 3
* Import urllib modules using six.moves.urllib
* MockHTTPSResponse: encode HTTP body to UTF-8 if it's Unicode,
replace six.string_types with (bytes, six.text_type).
* _id_to_base64(): catch also binascii.Error when calling
b16decode(); encode name to UTF-8 before calling b64encode();
decode b64encode() from ASCII on Python 3 to get Unicode.
* tox.ini: add cinder.tests.unit.volume.drivers.emc.scaleio
to Python 3.4
John Griffith [Tue, 6 Oct 2015 23:39:24 +0000 (17:39 -0600)]
Fix SolidFire target composition
We added the target composition a while back but never
really finished it. We created a set of interface methods
and had them call the target class mostly thinking we'd
want this when adding FC.
It turns out that doing a regular composition and using
get_attr will actually work fine when the FC target is
added, so re factor this and make it a true composition
where both objects inherit attributes from each other.
Victor Stinner [Wed, 7 Oct 2015 15:14:54 +0000 (17:14 +0200)]
Port targets test_iet_driver to Python 3
* Replace StandardError with MemoryError: StandardError was removed
in Python 3
* Replace __builtin__ with six.moves.builtins to mock open()
* Mock the logger in test_create_iscsi_target() because the test
injects errors in open() and the logger uses open() to format the
traceback
* tox.ini: add cinder.tests.unit.targets.test_iet_driver to Python 3.4
Eric Harney [Tue, 6 Oct 2015 19:31:03 +0000 (15:31 -0400)]
test_backup_nfs: Set volume id per test
Since some of these tests trigger operations (such as
progress reporting) that can happen as an asyncronous
task, set a different ID for each test so it's possible
to identify where notifications are coming from.
Eric Harney [Tue, 6 Oct 2015 18:14:00 +0000 (14:14 -0400)]
test_backup_swift: Set volume id per test
Since some of these tests trigger operations (such as
progress reporting) that can happen as an asyncronous
task, set a different ID for each test so it's possible
to identify where notifications are coming from.
rajinir [Mon, 5 Oct 2015 20:56:40 +0000 (15:56 -0500)]
Dell Eqlx: Support over subscription in thin provisioning
Dell eqlx driver reports the following new stats:
* provisioned_capacity
* max_over_subscription_ratio
* thin_provisioning_support (True or False)
* thick_provisioning_support (True or False)
Kendall Nelson [Tue, 15 Sep 2015 01:37:00 +0000 (20:37 -0500)]
Hacking check for opt name registration
Depending on how opts are registered (either with register_opt() or
register_opts()), the name needs to be singular or plural to match
the method. This patch adds a hacking check to make sure the names
of the opts and opt lists (or tuples) are correct given how they are
being registered. The check also verifies that a single option is
sent when register_opt() is used and a list is used when using
register_opts().
Includes fixes to files that don't meet the naming convention and a
addition to the generate_cinder_opts.py file in order to skip
checks.py in the generation of the opts.py file.
John Griffith [Mon, 5 Oct 2015 23:22:30 +0000 (17:22 -0600)]
Add ability to set prefix in SolidFire Volume name
The SolidFire creates names for Volumes on the backend using
the format UUID-<cinder-id>
This works well, but it turns out that there is a possibility
of a UUID collision for customers running multiple clouds against
the same cluster. To eliminate that possibility we should allow
a config option to set the prefix to be whatever an admin chooses.
This patch introduces the sf_volume_prefix config option to
address this bug. The default still uses the 'UUID-' prefix.
DocImpact
Changing this setting on an existing deployment will
result in the existing volumes being inaccessible. To introduce
this change to an existing deployment it's recommende to
add the Cluster as if it were a second backend and disable
new deployments to the current backend.
Gorka Eguileor [Fri, 2 Oct 2015 13:08:31 +0000 (15:08 +0200)]
Detach volume on device validation failure
If device validation fails when attaching a volume for some driver
operation (copy_volume_data, copy_image_to_volume, copy_volume_to_image)
we may end up with an attached volume that we don't cleanup.
This patch tries to detach the volume if we fail when validating the
device after we have attached the volume.
This happens for example on multipath when we have properly detected the
paths but they are all in a failed state when we try to read from the
device on validation.
Wilson Liu [Tue, 22 Sep 2015 12:52:36 +0000 (20:52 +0800)]
Fix capacity report error in Huawei driver
Currently the capacity report is incorrect due
to wrong way code clear.
The previously right way is:
result = var / 1024.0 / 1024.0 / 2
But currently we use a constant CAPACITY_UNIT to make the code cleaner
like this:
CAPACITY_UNIT = 1024.0 / 1024.0 / 2
result = var / CAPACITY_UNIT
We want the CAPACITY_UNIT work like a *real* constant like C
language, but in python, it is not a *real* constant, it's
just a variable. It will calculate the CAPACITY_UNIT first,
so the finally result is:
result = var / 0.5
And this is wrong.
Xing Yang [Thu, 1 Oct 2015 02:28:56 +0000 (22:28 -0400)]
VMAX Target iSCSI IP Address
In VMAX iSCSI driver, the iscsi_ip_address was hardcoded
in cinder.conf. This may have issues with multi-portgroup
environment. If a customer has multiple portgroups
containing different ports with different iSCSI IP addresses,
then ‘iscsiadm‘ command cannot use more than the one hardcoded
IP address in its sendtargets operation.
This patch addresses this by examining the ports in the
portgroup of the masking view used in the attach operation.
Each port has a corresponding iSCSI IP address. A portgroup
with only one port will have one IP address; a portgroup with
multiple ports will have multiple IP addresses. Even if there
is more than one IP address, the first in the list is likely to
result in a successful iscsiadm -sendtargets.
Masaki Kanno [Sun, 4 Oct 2015 06:57:22 +0000 (15:57 +0900)]
Delete a temporary volume in DB with admin context
Non-disruptive backup creates a temporary volume. Information
of the volume will be deleted from DB if the volume cannot be
created by drivers. volume_destroy method is used in order to
delete the information. We must elevate the context to admin
context because the method requests admin context.
Cinder raises an exception when performing an update quota
operation in a subproject in which its parent's quotas weren't
explicitly updated.
This is because cinder will try to find the parent's quotas
in order to update the allocated value. But, since the parent's
quotas are the default quotas (i.e. there are no entries for them
in the database), the operation will raise an error.
Steps to reproduce:
1. Create a project A in keystone;
2. Create a project B in keystone with A as its parent;
3. Try to update the quota value of any resources of project B
(e.g. openstack quota set <id of B> --volumes 2)
This patch fix this bug by adding entries to the database when
performing an update quota operation in a subproject and there
are still no quota associated with the parent project.
Victor Stinner [Fri, 2 Oct 2015 15:25:50 +0000 (17:25 +0200)]
Port test_quobyte to Python 3
Modify assertRaisesAndMessageMatches() to tolerate subclasses. On
Python 3, a permission error now raises an exception PermissionError
which is a subclass of OSError, whereas the test expected exactly the
OSError type.
Jay S Bryant [Fri, 2 Oct 2015 17:44:13 +0000 (12:44 -0500)]
Remove unused 'deprecated' option from auth_strategy opt
Noticed that the auth_strategy option in
cinder/common/config.py has a 'deprecated' option. I
can only see that 'no_auth' and 'keystone' are used anywhere
in the code. Looking at the git logs it appears that
the 'deprecated' option may have been pulled over when we split
from Nova. This patch removes the unused option.
Victor Stinner [Fri, 2 Oct 2015 15:35:37 +0000 (17:35 +0200)]
Port test_netapp to Python 3
* Add close() method to FakeHttplibSocket,
FakeDirectCmodeHTTPConnection, FakeDirect7modeHTTPConnection
* Replace StringIO() with BytesIO() for socket buffers
* makefile(): ignore optional arguments after mode
* Fix bytes versus Unicode: use bytes for HTTP body. Encode HTTP body
to UTF-8 and HTTP headers to Latin1.
* tox.ini: add cinder.tests.unit.test_netapp to Python 3.4
Xing Yang [Fri, 2 Oct 2015 14:17:40 +0000 (10:17 -0400)]
VMAX Truncate Storage Group Name
In the VMAX driver, it generates a storage group name using
the short host name, pool name, etc. However the storage
group name has an upper limit of 64 characters. So it fails
if the length of the name exceeds the limit. This patch
fixes it by truncating the name to fix within the limit.
Adriano Rosso [Mon, 24 Aug 2015 17:13:30 +0000 (14:13 -0300)]
HNAS iSCSI manage does not work with spaces
Currently, HNAS iSCSI driver does not work when trying to manage a
volume that contains one or more space in the name.
This patch fixes this functionality and performs some other minor
adjustments.
Victor Stinner [Fri, 2 Oct 2015 08:24:54 +0000 (10:24 +0200)]
Port scheduler host manager to Python 3
* Rewrite ReadOnlyDict using collections.Mapping: collections.Mapping
has no method to modify the dictionary and is available on Python 2
and Python 3. Add also a __repr__() method to ease debug. Remove
ReadOnlyDict.update() method: a read-only dictionary must not be
modifiable.
* test_host_manager.py: sort dictionaries using a key function to
have a reliable order for the list of dictionaries. On Python 3,
the hash function is now randomized.
* tox.ini. add cinder/tests/unit/scheduler/ tests to Python 3.4
Code Review / openstack-build / cinder-build.git / log