Anthony Lee [Fri, 26 Jun 2015 17:38:33 +0000 (10:38 -0700)]
Improve 3PAR driver VLUN creation and deletion
This patch improves the creation and deletion logic for VLUNs
for the 3PAR drivers. The changes will help to prevent VLUNs
from being left behind after volume detaches. It also prevents
extra VLUNs being created when initialize_connection is called
multiple times for the same volume and host.
gate-cinder-python27 gate-cinder-python34 is reporting failures where it
wasn't before. This is all due to new mock version 1.1.0. This patch
fixed the tests to pass the gate.
Some of these errors are actual errors in our test code, while other are
changes to Mock syntax.
Matt Riedemann [Fri, 19 Jun 2015 18:25:43 +0000 (11:25 -0700)]
Set encrypted key in connection_info during initialize
There are encrypted volume tests in Tempest which are passing
in the ceph job but the encryption providers in nova (luks and
cryptsetup) are not actually being called in nova because the
'encrypted' key isn't being set in connection_info for the Rbd
volume driver, and nova is keying off that value to determine
if it needs to run it's encryption providers.
So the tests are passing in the ceph job but it's a total false
positive and the API should actually fail - either the cinder
API to create the encrypted volumes if the Rbd volume driver in
Cinder doesn't support encryption, or the volume attach call in
nova if it can't encrypt the connected volume, either way the
test should fail until that support is baked into nova.
I'm not aware of any kind of 'supports_encryption' or 'encryptable'
capability flag for cinder volume drivers, but that might be
needed for the volume create API to fail if trying to create a
volume from an encrypted volume type where the volume driver
itself doesn't support encryption - but maybe it's only a nova
problem since nova has the encryption providers.
This is fixed generically in the VolumeManager's
initialize_connection method where we check for the encrypted
flag being set in the volume driver method and if not, the
manager sets it based on the volume.encryption_key_id value.
We do this globally since there are other volume drivers besides
rbd that aren't setting the encrypted key in connection_info,
but we test rbd in the gate via the ceph job so most of the
bug tracking is around validation of the ceph job.
Related Nova change: I8efc2628b09d4e9e59831353daa080b20e17ccde
DocImpact: Attaching 'encrypted' RBD volumes to a Nova server
instance created from the libvirt virt driver will fail
since RBD volume encryption is not currently supported in
Nova's libvirt driver.
Vilobh Meshram [Mon, 22 Jun 2015 21:45:02 +0000 (14:45 -0700)]
Nested Quota: Set default values to subproject
In case of hierarchical projects set the default value
(for resources who have the resource flag set) for a
sub-project to zero. This patch adds this functionality for
hierarchical projects, while keeping the functionality
intact for non-hierarchical project. At the top level,
project resources will get the default values as imposed
by the "default" class settings.
Co-Authored-By: Erickson Santos <erickson@lsd.ufcg.edu.br>
Change-Id: I3b357464d5e5e0aa065506ac1e9d908e87f45c63
Partially-Implements: bp cinder-nested-quota-driver
Bob Callaway [Fri, 8 May 2015 19:33:41 +0000 (12:33 -0700)]
Fix NetApp cDOT driver use of Glance locations
As of Kilo, Glance returns a list of locations associated with an
image, but the NetApp cDOT driver only operated on the first one.
In addition, Glance has started enforcing a schema on the metadata
that can be added to images stored in the file glance_store. This
change modifies the cDOT driver to iterate through the list of
locations, using the correct schema-compliant fields, and uses the
first one with a valid IP address for our backend for optimizing
the copy of images into Cinder volumes.
Co-authored-by: Tom Barron <tbarron@dyncloud.net>
Closes-Bug: #1461116
This patch makes sure that the entire host is used
in the consistency group host entry. Previously,
whatever host entry was in the database was being
overwritten by the volume manager's self.host entry,
which doesn't contain a pool.
Alex Meade [Mon, 6 Jul 2015 14:17:43 +0000 (10:17 -0400)]
Add tracing facility for drivers
It is useful for engineering to be able to see the API traffic
to and from the storage controllers, as well as to see the
progression of method calls through the driver.
This work will add a simple config option to enable various
debug tracing types to be written to the debug log at runtime.
Use of the decorator has been added to NetApp drivers.
Drivers can make use of this facility by adding the
@cinder.utils.trace_method decorator to any method
calls they would like to allow tracing for or use trace_flags
for any other developer targeted debug logs. The trace_method
decorator can easily be added to every function of a class by
using the cinder.utils.TraceWrapperMetaclass or
cinder.utils.TraceWrapperWithABCMetaclass metaclasses. It is
advised to use the added decorators as the outer most decorator
for a function in order to avoid side-effects with other
decorators.
The @cinder.utils.trace_api method should be used to log
requests to storage backends by wrapping a method that
has the request as an argument and returns the response.
For example:
@six.add_metaclass(cinder.utils.TraceWrapperMetaclass)
class MyClass(object):
pass
Simon Dodsley [Tue, 7 Jul 2015 14:06:37 +0000 (10:06 -0400)]
Fix error message in Pure driver with correct text
The error message was specifically mentioning the PureISCSIDriver but
now there is a PureFCDriver which could call the error message. This
change gets the name of the driver and uses that to ensure
the error message refers to the correct driver.
wanghao [Tue, 26 May 2015 07:32:42 +0000 (15:32 +0800)]
Notify the transfer volume action in cinder
Now when we transfer a volume, there is not corresponding action
notification that is sent to ceilometer. Include the actions of create,
accept and delete.
The bp that this patch implements, proposes adding those action to enrich
the notification in cinder.
Notify is added in Create, Accept, and Delete actions.
TaoBai [Tue, 30 Jun 2015 07:03:38 +0000 (00:03 -0700)]
Storwize Driver zone removing
Storwize driver may remove zone even when the zone is still in use
by an attached volume. To fix this issue, we need only report initiator
and target map when there are no volumes attached.
tswanson [Mon, 29 Jun 2015 19:21:45 +0000 (14:21 -0500)]
Dell SC: Add support for consistency groups
Added support for consistency groups. Associated tests added to
test_dellsc.py and test_dellscapi.py. create_consistency_group_from_src
is not implemented in this patch.
Tom Barron [Sat, 20 Jun 2015 08:57:54 +0000 (04:57 -0400)]
Port NetApp E-Series iSCSI driver to ABC model
This commit modifies the NetApp E-Series volume driver to the
new ABC driver model. The corresponding FibreChannel driver
already uses that model since it was recently implemented and
was written with an appropriate ABC class signature in the first
place. A corresponding class signature is now used for iSCSI.
This commit also modifies a couple doc strings to match the
corresponding FibreChannel driver.
xiaolei hu [Thu, 25 Jun 2015 02:22:50 +0000 (10:22 +0800)]
Separate FlashSystem FC and iSCSI common code
The patch is mainly to split original FC driver code into common
code and FC driver for IBM FlashSystem. iSCSI driver merged in L-1
has been inherited from original FC driver for common functions
already.
In this patch,
* Separate FC driver into new driver and common code.
* Modify iSCSI driver to inherite the common code.
* Common function _get_node_data in iSCSI driver is moved into
common code.
* Add locks in initialize_connection and terminate_connection
for iSCSI driver.
* Remove flashsystem_multipath_enabled to use Cinder iSCSI
multipath support.
Alberto Murillo [Tue, 30 Jun 2015 21:12:55 +0000 (16:12 -0500)]
Update expected error message from lvs
drivers/lvm.py uses _volume_not_present() to avoid deleting a snapshot
when it does not exist in the lvm backend. This functions uses brick/
local_dev/lvm.py get_lv_info() which expects "not found" as error message.
Since LVM2 2.102.112, The error message returned by lvs changed
From:
One or more specified logical volume(s) not found
To:
Failed to find logical volume \"%s/%s\"
Vincent Hou [Thu, 7 May 2015 05:53:25 +0000 (13:53 +0800)]
Implement the update_migrated_volume for the drivers
This patch implements update_migrated_volume for LVM, Storwize
and updates Dell, StorPool and Infortrend drivers accordingly. It
makes sure that after a successful volume migration, the cinder
volume name(id) is the same as the backend volume name(id). Other
back-end drivers can take this patch as a reference to implement
update_migrated_volume.
PS: Not applicable to multi-attached volumes, since we need to wait
until the multi-attach lands in Nova.
This patch also adds a unit test for the StorPool driver's
update_migrated_volume() implementation.
Co-Authored-By: Peter Penchev <openstack-dev@storpool.com>
Change-Id: I69707340ddf2b55286ff0d84319529b2f502cefa
Partial-Bug: #1450649
Andrey Pavlov [Fri, 19 Jun 2015 15:03:19 +0000 (18:03 +0300)]
Avoid race condition at snapshot deletion stage
Snapshot`s list method can raise SnapshotNotFound exception.
It happens because of race condition:
1. List method gets all snapshots from DB.
2. At the same time one of the snapshots is being deleted from DB.
3. List method gets snapshot metadata from DB and checks
that snapshot exists in DB and raises the exception.
This patchset changes behaviour of getting snapshot metadata from DB.
Code now gets metadata from db_snapshot object that was queried
by joined query from DB instead of second query for each snapshot
for metadata.
And I removed checking of snapshot existence for getting
metadata for private method because all public methods already have
such decorator. Using second decorator will slow public method.
Victor Stinner [Tue, 30 Jun 2015 14:47:15 +0000 (16:47 +0200)]
Fix Python 3 issues in cmd
* Replace filter() with a list-comprehension using if to get a list on
Python 3.
* Get the mock module from the stdlib unittest module on Python 3.3 and
newer, or fallback to the third-party mock module.
* Replace __builtin__ with six.moves.builtins.
* tox.ini: add the following tests for Python 3.4
Victor Stinner [Tue, 30 Jun 2015 14:06:28 +0000 (16:06 +0200)]
Port image/glance.py to Python 3
* Fix usage of six.reraise(): new_exc is the exception value, not the
exception type
* Replace __builtin__ with six.moves.builtins.
* test_extracting_v2_boot_properties(): set config.glance_num_retries to
fix the test on Python 3 (comparison between mock and int now raises
a TypeError on Python 3).
* TestGlanceSerializer: use dictionaries with only one key to have a
reliable output even if the hash is randomized (hash randomization is
now enabled by default in Python 3).
* tox.ini: add cinder.tests.unit.image.test_glance to Python 3.4.
liyingjun [Mon, 29 Jun 2015 01:49:34 +0000 (09:49 +0800)]
Validate maximum limit for quota
Currently, when updating quota using limit larger than 0x7FFFFFFF,
there will be a DBError. The maximum limit needs to be validated.
And the maximum value is defined by SQL integer type INT, which
is a general value:
http://dev.mysql.com/doc/refman/5.0/en/integer-types.html
http://www.postgresql.org/docs/9.1/static/datatype-numeric.html
Commit Ibaf43858d60e1320c339f2523b5c09c7f7c7f91e caused new problem with
cross thread communication. According to Python documentation, code can
lead to a deadlock if the spawned thread directly or indirectly attempts
to import a module. python-rados spawns new thread to connect to
cluster. So I removed new spawning new thread to connect to rados. All
long-running operations calls whith python-rbd are still implemented in
native Python threads to block eventlet loop.
Victor Stinner [Mon, 29 Jun 2015 13:33:12 +0000 (15:33 +0200)]
Port dothill to Python 3
* Replace urllib2 with six.moves.urllib
* On Python 3, encode string to UTF-8 to hash it using MD5
* On Python 3, decode base64 from ASCII to get Unicode
* Replace "pattern in exc" with "pattern in exc.args"
* test_initialize_connection(): fix get_active_iscsi_target_portals()
mock. Use the return_value attribute to return a dictionary, instead
of returning a single IP address.
* tox.ini: add cinder.tests.unit.test_dothill to Python 3.4
Michal Dulko [Wed, 1 Jul 2015 06:41:03 +0000 (08:41 +0200)]
Mock socket.gethostbyaddr in test_v7000_fcp
Currently test_v7000_fcp doesn't mock socket.gethostbyaddr and
hardcodes expected response to be google-public-dns-a.google.com
(as it is querying for 8.8.8.8 as IP). This introduces dependency on
external DNS lookup to pass unit tests and causes problems with
running tests behind a proxy.
This commit mocks socket.gethostbyaddr in the problematic test.
Rakesh H S [Tue, 23 Jun 2015 14:06:24 +0000 (19:36 +0530)]
Return 404 if volume type encryption is not found
This commit adds a check for volume type encryption existence before
marking it as deleted. If it isn't present call returns 404 error
code so that it can be handled properly in services that use the
REST API (like Heat).
APIImpact
HTTP 404 will be returned if volume type encryption is not found.
Dmitry Guryanov [Wed, 10 Jun 2015 20:01:55 +0000 (23:01 +0300)]
smbfs: fix invalid check for smbfs_used_ratio correctness
The check allowed to specify values greater than 1
because of a error in condition. So I've fixed the
condition and updated unit test, so it would catch
the bug.
Victor Stinner [Tue, 30 Jun 2015 14:14:09 +0000 (16:14 +0200)]
Fix Python 3 issues in targets unit tests
* Replace StandardError with ZeroDivisionError: StandardError was
removed in Python 3.
* Replace __builtin__ with six.moves.builtins.
* Open text mode with "w" mode (instead of "wb").
* tox.ini: add targets unit tests to Python 3.4
Note: cinder.tests.unit.targets.test_iet_driver still fails on Python 3,
it will be fixed in a different change.
Victor Stinner [Mon, 29 Jun 2015 13:53:44 +0000 (15:53 +0200)]
Port drbdmanagedrv driver to Python 3
* Replace map() with a list-comprehension where a list is expected
* Replace apply(fn, args) with fn(*args)
* Use str.replace() to remove "{" and "}" characters in _clean_uuid(),
instead of using str.translate()
* Use literal syntax to create a new dictionary instead of using
dict() + dict.items()
* tox.ini: add cinder.tests.unit.test_drbdmanagedrv to Python 3.4
Victor Stinner [Mon, 29 Jun 2015 13:11:12 +0000 (15:11 +0200)]
Port test_db_api to Python 3
* _dict_from_object(): on dictionaries, call the items() method instead
of iteritems() method. oslo.db objects have no items() method yet, so
check the object type to decide which method should be called.
* test_db_api: fix usage of db.quota_reserve(), quotas dictionary values
must be integers (hard limit of quotas), not Quota instances of
cinder.db.sqlalchemy.models.
* tox.ini: add the following tests to Python 3.4
Victor Stinner [Mon, 29 Jun 2015 20:20:16 +0000 (22:20 +0200)]
Port hitachi driver to Python 3
* On Python 3, shlex expects Unicode, not bytes: don't encode
* Replace map() with list-comprehension where a list is expected.
* tox.ini: add cinder.tests.unit.test_hitachi_hbsd_horcm_fc to Python 3.4
wanghao [Tue, 23 Jun 2015 08:38:29 +0000 (16:38 +0800)]
Do not allow to modify access for public volume type
Now public volume type can be added/removed project access, but when
listing access, cinder returns 'Access list not available for public
volume types.' It's weird for users experience.
We should check if a type is public and do not allow public volume type
to modify project access.
APIImpact
When modifying access for public volume type, API will return 400 and message:
"Invalid volume type: Type access modification is not applicable to public
volume type."
Victor Stinner [Mon, 29 Jun 2015 20:31:45 +0000 (22:31 +0200)]
Port huawei driver to Python 3
* On Python 3, encode text to UTF-8 before encoding it to base64
* On Python 3, decode encoded base64 from ASCII to get Unicode
* Fix JSON in unit test: COUNT must be an integer, not a string
* tox.ini: add cinder.tests.unit.test_huawei_18000 to Python 3.4
Code Review / openstack-build / cinder-build.git / log