Xi Yang [Tue, 1 Sep 2015 10:05:15 +0000 (06:05 -0400)]
Retype enhancement for EMC VNX cinder driver
VNX driver will trigger lun migration when retyping a thin/thick
volume to a compressed volume and it is time-consuming. To enhance
the performance, this patch will directly turn on the compression
on a volume instead of migrating it.
rajinir [Mon, 5 Oct 2015 20:56:40 +0000 (15:56 -0500)]
Dell Eqlx: Support over subscription in thin provisioning
Dell eqlx driver reports the following new stats:
* provisioned_capacity
* max_over_subscription_ratio
* thin_provisioning_support (True or False)
* thick_provisioning_support (True or False)
Kendall Nelson [Tue, 15 Sep 2015 01:37:00 +0000 (20:37 -0500)]
Hacking check for opt name registration
Depending on how opts are registered (either with register_opt() or
register_opts()), the name needs to be singular or plural to match
the method. This patch adds a hacking check to make sure the names
of the opts and opt lists (or tuples) are correct given how they are
being registered. The check also verifies that a single option is
sent when register_opt() is used and a list is used when using
register_opts().
Includes fixes to files that don't meet the naming convention and a
addition to the generate_cinder_opts.py file in order to skip
checks.py in the generation of the opts.py file.
John Griffith [Mon, 5 Oct 2015 23:22:30 +0000 (17:22 -0600)]
Add ability to set prefix in SolidFire Volume name
The SolidFire creates names for Volumes on the backend using
the format UUID-<cinder-id>
This works well, but it turns out that there is a possibility
of a UUID collision for customers running multiple clouds against
the same cluster. To eliminate that possibility we should allow
a config option to set the prefix to be whatever an admin chooses.
This patch introduces the sf_volume_prefix config option to
address this bug. The default still uses the 'UUID-' prefix.
DocImpact
Changing this setting on an existing deployment will
result in the existing volumes being inaccessible. To introduce
this change to an existing deployment it's recommende to
add the Cluster as if it were a second backend and disable
new deployments to the current backend.
Gorka Eguileor [Fri, 2 Oct 2015 13:08:31 +0000 (15:08 +0200)]
Detach volume on device validation failure
If device validation fails when attaching a volume for some driver
operation (copy_volume_data, copy_image_to_volume, copy_volume_to_image)
we may end up with an attached volume that we don't cleanup.
This patch tries to detach the volume if we fail when validating the
device after we have attached the volume.
This happens for example on multipath when we have properly detected the
paths but they are all in a failed state when we try to read from the
device on validation.
Xing Yang [Thu, 1 Oct 2015 02:28:56 +0000 (22:28 -0400)]
VMAX Target iSCSI IP Address
In VMAX iSCSI driver, the iscsi_ip_address was hardcoded
in cinder.conf. This may have issues with multi-portgroup
environment. If a customer has multiple portgroups
containing different ports with different iSCSI IP addresses,
then ‘iscsiadm‘ command cannot use more than the one hardcoded
IP address in its sendtargets operation.
This patch addresses this by examining the ports in the
portgroup of the masking view used in the attach operation.
Each port has a corresponding iSCSI IP address. A portgroup
with only one port will have one IP address; a portgroup with
multiple ports will have multiple IP addresses. Even if there
is more than one IP address, the first in the list is likely to
result in a successful iscsiadm -sendtargets.
Masaki Kanno [Sun, 4 Oct 2015 06:57:22 +0000 (15:57 +0900)]
Delete a temporary volume in DB with admin context
Non-disruptive backup creates a temporary volume. Information
of the volume will be deleted from DB if the volume cannot be
created by drivers. volume_destroy method is used in order to
delete the information. We must elevate the context to admin
context because the method requests admin context.
Cinder raises an exception when performing an update quota
operation in a subproject in which its parent's quotas weren't
explicitly updated.
This is because cinder will try to find the parent's quotas
in order to update the allocated value. But, since the parent's
quotas are the default quotas (i.e. there are no entries for them
in the database), the operation will raise an error.
Steps to reproduce:
1. Create a project A in keystone;
2. Create a project B in keystone with A as its parent;
3. Try to update the quota value of any resources of project B
(e.g. openstack quota set <id of B> --volumes 2)
This patch fix this bug by adding entries to the database when
performing an update quota operation in a subproject and there
are still no quota associated with the parent project.
Victor Stinner [Fri, 2 Oct 2015 15:25:50 +0000 (17:25 +0200)]
Port test_quobyte to Python 3
Modify assertRaisesAndMessageMatches() to tolerate subclasses. On
Python 3, a permission error now raises an exception PermissionError
which is a subclass of OSError, whereas the test expected exactly the
OSError type.
Victor Stinner [Fri, 2 Oct 2015 15:35:37 +0000 (17:35 +0200)]
Port test_netapp to Python 3
* Add close() method to FakeHttplibSocket,
FakeDirectCmodeHTTPConnection, FakeDirect7modeHTTPConnection
* Replace StringIO() with BytesIO() for socket buffers
* makefile(): ignore optional arguments after mode
* Fix bytes versus Unicode: use bytes for HTTP body. Encode HTTP body
to UTF-8 and HTTP headers to Latin1.
* tox.ini: add cinder.tests.unit.test_netapp to Python 3.4
Xing Yang [Fri, 2 Oct 2015 14:17:40 +0000 (10:17 -0400)]
VMAX Truncate Storage Group Name
In the VMAX driver, it generates a storage group name using
the short host name, pool name, etc. However the storage
group name has an upper limit of 64 characters. So it fails
if the length of the name exceeds the limit. This patch
fixes it by truncating the name to fix within the limit.
Adriano Rosso [Mon, 24 Aug 2015 17:13:30 +0000 (14:13 -0300)]
HNAS iSCSI manage does not work with spaces
Currently, HNAS iSCSI driver does not work when trying to manage a
volume that contains one or more space in the name.
This patch fixes this functionality and performs some other minor
adjustments.
Victor Stinner [Fri, 2 Oct 2015 08:24:54 +0000 (10:24 +0200)]
Port scheduler host manager to Python 3
* Rewrite ReadOnlyDict using collections.Mapping: collections.Mapping
has no method to modify the dictionary and is available on Python 2
and Python 3. Add also a __repr__() method to ease debug. Remove
ReadOnlyDict.update() method: a read-only dictionary must not be
modifiable.
* test_host_manager.py: sort dictionaries using a key function to
have a reliable order for the list of dictionaries. On Python 3,
the hash function is now randomized.
* tox.ini. add cinder/tests/unit/scheduler/ tests to Python 3.4
Victor Stinner [Fri, 2 Oct 2015 07:43:44 +0000 (09:43 +0200)]
Fix various Python 3 issues
* Drop exceptions import. Replace exceptions.OSError with OSError.
The exceptions module was removed in Python 3.
* Replace 8l with 8
* Replace generator.next() with next(generator)
* convert_uuid_to_es_fmt(): use bytes
* Replace function.func_name with function.__name__
This change helps to run tests using "testr run" instead of
testtools.run.
Victor Stinner [Thu, 1 Oct 2015 16:04:30 +0000 (18:04 +0200)]
Fix volume throttling to Python 3
BlkioCgroup._set_limits(): sort devices before iterating on them to
have a reliable behaviour.
The devs variable is a dictionary. On Python 3, the hash function is
now randomized, so iterating on a dictionary gives items in a random
order. Use sorted() to iterate on the list of sorted devices instead.
tox.ini: add cinder.tests.unit.test_volume_throttling to Python 3.
In both enable_replication and disable_replication,
it tests to see the replication_state is in the correct
state. When it's not it was raising InvalidVolume()
InvalidVolume requires a reason text message.
This patch adds the missing reason=msg when creating
the InvalidVolume exception
Sean McGinnis [Thu, 1 Oct 2015 14:18:43 +0000 (09:18 -0500)]
Cleanup orphaned code from cinder root directory
There were several methods in the root directory source files
that were no longer being used anywhere. While some of these
have potential for being useful, they are not in use and should
not be sitting around the codebase waiting for someone to need them.
This patch removes unused methods from utils as well as their
associated unit tests.
Method _devices_sizes() in Block Device driver invokes N times
_get_device_size(device) to get devices sizes. It can be done by
executing one command: blockdev --getsize64 /dev/loop1 /dev/loop2.
In such a way we take the devices' sizes in bytes and convert them
to Megabytes.
Obtain target authentication from database same as LIO target
Currently, tgt, iet and cxt obtain user and password for iSCSI
target by analyzing configuration file.
However this information is already stored in DB and LIO obtains
these authentication from provider_auth in DB.
This way is simple and robust instead of analyzing configuration
file directly.
This patch proposes these two changes:
- Change the way to obtain authentication from configuration
file to DB at _get_target_chap_auth().
- Move _get_target_chap_auth() into iscsi.py and inherit
the method at tgt, iet and cxt target because they can use
same implementation to get authentication from DB.
dell_storagecenter_api.py was looking for the cgsnapshot for
cgsnapshot-delete via the wrong REST API call. It was also only
looking for the first volume's snapshot rather than all snapshots
associated with a cgsnapshot. It now locates the snapshots properly
and expires the associated replays on cgsnapshot-delete.
Victor Stinner [Fri, 21 Aug 2015 16:47:50 +0000 (09:47 -0700)]
Port test_volume to Python 3
* don't use hasttr() to check if a lazy SQLAlchemy is loaded or not: use
the obj_attr_is_set() method instead. On Python 3, hasattr() pass
through SQLAlchemy exceptions which is unexpected.
* Replace a/b with a//b to use integer division, not float division.
* Replace filter() with list-comprehension using an if.
* Replace file() with open() and open /dev/null in binary mode (not text
mode, so Unicode on Python 3).
* test_volume require "import cinder.volume.targets.tgt" on Python 3
* Use a key function to sort a list of dictionaries. Dictionaries are no
more comparable on Python 3.
* tox.ini: add the following tests to Python 3.4.
Wilson Liu [Thu, 24 Sep 2015 10:05:11 +0000 (18:05 +0800)]
Fix update Huawei driver issue
If a volume is created with old version of Huawei driver,
there is no pool info related to the volume, but the new
version of Huawei driver assumes that the volume have the
pool info and will read the pool info from the volume,
and then the read fails. This will happen when we call
create_volume_from_snapshot() in new version of Huawei
driver when the volume is created by old version of Huawei
driver.
Victor Stinner [Tue, 18 Aug 2015 23:03:51 +0000 (16:03 -0700)]
Fix Python 3 issues in wsgi
Fix unicode versus bytes issues in wsgi code, a HTTP body is a bytes
string. Changes:
* On Python 3, encode JSON to UTF-8.
* Replace file() with open(); file() was removed in Python 3.
* Use literal byte strings (b'...') for HTTP body.
* Use binary instead of text mode for HTTP body when getting a file from
a socket (sock.makefile).
Victor Stinner [Wed, 26 Aug 2015 22:13:51 +0000 (00:13 +0200)]
py3: Port pure driver test to Python 3
* Replace func.func_name with func.__name__: the func_name attribute
was removed in Python 3, whereas the __name__ attribute exists on
Python 2 and Python 3
* Create INITIATOR_TARGET_MAP using list(set()) to get the same order
than the tested code. The exact order is not reliable, it depends
on the hash function which is now randomized by default on
Python 3. The hash function is also different between Python 2.7
and 3.4.
* tox.ini: add test_pure to Python 3.4
Wilson Liu [Thu, 24 Sep 2015 08:52:31 +0000 (16:52 +0800)]
Report *real* free capacity in Huawei driver
When we query capacity of a storage pool on Huawei storage,
there are two items in the result: one is "USERFREECAPACITY"
and the another one is "DATASPACE". In fact the "DATASPACE"
is the *real* space we can use to create LUN, the
"USERFREECAPACITY" contains some metadata that we can not use
to store user data.
Dave Chen [Mon, 3 Aug 2015 09:01:22 +0000 (17:01 +0800)]
Show image metadata
This patch implements the API to show the glance image metadata
for a specific volume.
DocImpact: May need to document this API.
APIImpact: Provided the API to show image metadata.
Partially implements: bp support-modify-volume-image-metadata
Currently the volume related operation in CloudByte driver does not:
* Update the ISCSI Initiator Group during volume deletion operation,
and
* During retry of any volume operation it does not catches the exact
cause of the failure happened.
So to deal with the following issues two changes has been done in
CloudByte Driver:
* Added a new method "_update_initiator_group"
ISCSI Initiator Group is one of the parameter added to the volumes
at the time of creation in CloudByte Storage. Using this the external
source can access the volumes easily. By default, Initiator Group
value is set to 'ALL', but it should be changed to 'None' before
deletion. Since the following is not happening, thus volume delete
operation fails from OpenStack. In order to avoid this the
"delete_volume" method now uses the "_update_initiator_group" which
update the volume ISCSI Initiatior Groupof the volumes to 'None'
before deletion, and thus volume delete operation will have no issues
while running from OpenStack.
* Added a check to "_retry_volume_operation"
The _retry_volume_operation method was not able to check the exact
cause of the failure happened during volume operation in CloudByte
Storage, which arises if some invalid parameters are provided in
"cinder.conf" against "cloudbyte backend" or if some anomaly happens
at CloudByte Storage during any volume operation from OpenStack. Due
to this, tracking the ERROR becomes a very difficult task to do. Thus
to resolve this issue a "check" has been added to the method which
will raise the exact cause of the error and thus making it easier to
solve.
Code Review / openstack-build / cinder-build.git / log