chenzongliang [Thu, 4 Jun 2015 02:43:07 +0000 (10:43 +0800)]
Refactor Huawei Volume driver
This patch attempts to refactor Huawei volume driver in liberty.
We add a base driver to implement the basic functions.
The sub-class will inherit from the base driver according to different
storages.
The following changes were made in this refactor:
1. Abstract a base class named HuaweiBaseDriver to make Huawei driver more
universal. You can find it in the huawei_driver.py.
2. Put all static variables into the constants.py.
3. Rename rest_common.py to rest_client.py. rest_client.py stores the
relevant methods implemented for Huawei driver.
4. Migrate some public methods from rest_client.py to huawei_utils.py,
such as parse_xml_file(), _get_volume_type() and so on.
5. This refactor only involves structural adjustment and does not involve
functional changes.
Matt Riedemann [Thu, 16 Jul 2015 20:51:23 +0000 (13:51 -0700)]
scality: add export and name keys to os-initialize_connection info
Nova is standardizing on some of the common code for file system type
volume drivers in blueprint consolidate-libvirt-fs-volume-drivers and
expects the 'export' and 'name' keys to be in connection_info['data']
returned from the os-initialize_connection Cinder API.
The Scality volume driver wasn't providing these keys so those are added
here.
Related Nova change: I7db834956c67383e63be2fe2c465118e00273e4b
Jinru Yan [Thu, 9 Jul 2015 01:30:45 +0000 (09:30 +0800)]
Report capability of multiattach for FlashSystem
This patch enables the multiattach capability reporting in
_update_volume_stats. This will allow nova to attach and detach
a volume to multiple virtual machine instances.
John Griffith [Tue, 7 Jul 2015 22:26:21 +0000 (16:26 -0600)]
Handle volume not found on zfssa volume delete
The zfssa driver doesn't handle cases where a volume may not
exist on the backend when a delete call is sent. The result
is a stack-trace in the logs, sometimes a crash of the service
and an inability to clean up volumes from Cinder's perspective.
This patch doesn't address the root cause of the issue, but
it does handle the case more gracefully by catching the
exception, logging an error and returning.
This way the volume can be cleaned up on the Cinder side and
the operator still has an indication that something went wrong.
This is a common pattern for most of the drivers in Cinder.
If backup cannot be found while resetting its status, Cinder
exception (BackupNotFound) is raised instead of HTTP exception.
This patch handles the BackupNotFound exception and raises
HTTPNotFound exception.
APIImpact
HTTPNotFound exception (404) is raised instead of BackupNotFound
if backup cannot be found during backup reset-status.
Patrick East [Mon, 13 Jul 2015 20:42:46 +0000 (13:42 -0700)]
Preserve mock side_effect’s in test_pure
The helper method to test if exceptions are propagated or handled would
unset the mock objects side_effect value when its done. This overrides
the original value making the mock not behave as expected as soon as it
is passed into assert_error_propagates.
This changes the helper to store the original side effect and then put
it back once we are done making it throw an exception.
Peter Penchev [Fri, 12 Jun 2015 07:41:52 +0000 (10:41 +0300)]
StorPool: clean up the last uses of str.format()
Replace three logging uses of str.format() with parameters passed
directly to LOG.error(), simplifying them in the process by
avoiding the need to call six.text_type() ourselves.
Replace the remaining use with string concatenation.
Suggested by: Sean McGinnis in review 188725
Change-Id: Ie53c394d7f31fa1dccfa4ed5c4b514539c77aa05
Sean McGinnis [Wed, 1 Jul 2015 20:14:57 +0000 (15:14 -0500)]
Remove useless logging from unit tests
It has been discussed that there really isn't much point in having
unit tests making any kind of logger calls. Some usage has already
been cleaned up. This patch removes the remaining instances of log
calls under the cinder/tests directory.
Also cleaned up a lot of cases where the source files would import
oslo_logging and declare a LOG variable which was never actually
used.
Leaving logging in the cinder/tests/unit/integrated tree for now
until a plan is decided as to what to do with all of these type of
tests.
Added hacking check to prevent new instances from slipping by code
reviews.
Gorka Eguileor [Fri, 26 Jun 2015 14:05:04 +0000 (16:05 +0200)]
Fix saving tz aware datetimes in Versioned Objects
Currently we cannot save date fields in Versioned Objects because they
are timezone aware and our database is not.
If we look at our objects definition, we are not giving any value to
tzinfo_aware argument, so it takes DateTime default value (True).
In our sqlalchemy model definitions we are defining our datetime fields
as Column(DateTime) which will take sqlalchemy default: class
sqlalchemy.types.DateTime(timezone=False)
In most cases we are accessing those fields directly through the DB
methods, so we don't see the problem, but if we were to modify any of
those fields using versioned object, for example scheduled_at, and then
save the object we'll get an exception:
File ".../sqlalchemy/sql/type_api.py", line 278, in compare_values
return x == y
TypeError: can't compare offset-naive and offset-aware datetimes
Because we are trying to save a timezone aware date to a timezone
unaware DB field.
Following Dan's advice we create a specific cinder_obj_get_changes,
instead of overwriting obj_get_changes, that returns all datefields as
timezone naive UTC datetimes ready for saving.
Dave Chen [Fri, 16 Jan 2015 09:59:18 +0000 (17:59 +0800)]
set/unset volume image metadata
This patch implements:
- Create, delete, update APIs for modifying volume image metadata.
- Refactoring in the volume API to accommodate both user and
image metadata.
- All of the necessary testcases needed for the changes.
DocImpact
APIImpact
Partially implements: bp support-modify-volume-image-metadata
Change-Id: I22792ef7bd49c763d7c130aa8cac9abc3fff2d4c
Jay S. Bryant [Fri, 10 Jul 2015 21:38:15 +0000 (16:38 -0500)]
Fix not implemented wording in update_migrated_volume
It was recommended in review https://review.openstack.org/#/c/180873
that the message sent to the NotImplementedError exception be reworded.
The recommendation was made late in the review process and was not
corrected. This patch implements the recommendation.
wanghao [Fri, 20 Mar 2015 07:51:57 +0000 (15:51 +0800)]
Add support for force-delete backups
1. Add _get and _delete method to BackupAdminController.
2. Add force=False arg to delete method in backup api.
3. If force=True, then allow the deletion of backup, which
can be in any state.
4. For chunkeddriver based backup, check the status of backup
from DB in backup loop, if it has changed to deleting, set the
flag is_backup_canceled and clear the data to avoid leaving orphan
object in the object store.
5. Add the flag: support_force_delete=False in BackupDriver. That
indicates if the backup driver supports force deletion.
It should be set to True if the driver that inherits from BackupDriver
supports the force deletion function.
6. If the backup driver doesn't support this feature, and it will return
405 when calling API of force delete.
Anthony Lee [Fri, 26 Jun 2015 17:38:33 +0000 (10:38 -0700)]
Improve 3PAR driver VLUN creation and deletion
This patch improves the creation and deletion logic for VLUNs
for the 3PAR drivers. The changes will help to prevent VLUNs
from being left behind after volume detaches. It also prevents
extra VLUNs being created when initialize_connection is called
multiple times for the same volume and host.
gate-cinder-python27 gate-cinder-python34 is reporting failures where it
wasn't before. This is all due to new mock version 1.1.0. This patch
fixed the tests to pass the gate.
Some of these errors are actual errors in our test code, while other are
changes to Mock syntax.
Xing Yang [Fri, 19 Jun 2015 13:47:59 +0000 (09:47 -0400)]
Support SMI-S provider v8.0.3 in VMAX driver
This patch made changes in the VMAX driver to support
interface changes in SMI-S provider v8.0.3.
* Accommodate System Name prefix change from 'SYMMETRIX+' to
'SYMMETRIX-+-'.
* Delete composite volume - changed from using
ReturnElementsToStoragePool to EMCReturnToStoragePool.
* Extend volume - removed InPool parameter from EMCUnBindElements
method.
* Create snapshot/clone with multi meta members -
EMC_PartialAllocOfConcreteExtentCIM was deprecated;
used 'GetCompositeElements' instead and changed how we get meta
members' capacities.
* Create CG snapshot - A new WaitForCopyState parameter was added.
* Added helper function to determine SMI-S provider and SE version
Note: SMI-S provider is bundled with SE (Solutions Enabler) and
they have the same version number starting from v8.0.
The EMC SMI-S Provider supports the SNIA Storage Management Initiative
(SMI), an ANSI standard for storage management. It supports the VMAX
storage system. SE is a software product that discovers and manages
VMAX storage system.
Add ability to override OpenStack privileged user auth url
Introduce os_privileged_user_auth_url config to give the ability
to override the auth_url used when authenticating
the OpenStack privileged user and bypass use of catalog found in token.
DocImpact: New os_privileged_user_auth_url config
Closes-bug: #1473206
Change-Id: I4ffca8df0eb38fe41264439ae5bb93e025c808ff
Ryan Lucio [Thu, 12 Mar 2015 23:12:10 +0000 (16:12 -0700)]
VMEM v6000: Fix export verify routines
New 'state' nodes in vmos6 are reflecting the kernel/system state
indicating lun accessibility on the data plane. The nodes used
previously were only reflecting success on the mgmt plane, meaning
that the routines were sometimes returning before the target/export
was actually available for client use.
Also, switch to using one global iscsi target group (backend
container that groups target IPs) to avoid a backend export bug
exposed by the changes mentioned above.
Matt Riedemann [Fri, 19 Jun 2015 18:25:43 +0000 (11:25 -0700)]
Set encrypted key in connection_info during initialize
There are encrypted volume tests in Tempest which are passing
in the ceph job but the encryption providers in nova (luks and
cryptsetup) are not actually being called in nova because the
'encrypted' key isn't being set in connection_info for the Rbd
volume driver, and nova is keying off that value to determine
if it needs to run it's encryption providers.
So the tests are passing in the ceph job but it's a total false
positive and the API should actually fail - either the cinder
API to create the encrypted volumes if the Rbd volume driver in
Cinder doesn't support encryption, or the volume attach call in
nova if it can't encrypt the connected volume, either way the
test should fail until that support is baked into nova.
I'm not aware of any kind of 'supports_encryption' or 'encryptable'
capability flag for cinder volume drivers, but that might be
needed for the volume create API to fail if trying to create a
volume from an encrypted volume type where the volume driver
itself doesn't support encryption - but maybe it's only a nova
problem since nova has the encryption providers.
This is fixed generically in the VolumeManager's
initialize_connection method where we check for the encrypted
flag being set in the volume driver method and if not, the
manager sets it based on the volume.encryption_key_id value.
We do this globally since there are other volume drivers besides
rbd that aren't setting the encrypted key in connection_info,
but we test rbd in the gate via the ceph job so most of the
bug tracking is around validation of the ceph job.
Related Nova change: I8efc2628b09d4e9e59831353daa080b20e17ccde
DocImpact: Attaching 'encrypted' RBD volumes to a Nova server
instance created from the libvirt virt driver will fail
since RBD volume encryption is not currently supported in
Nova's libvirt driver.
Vilobh Meshram [Mon, 22 Jun 2015 21:45:02 +0000 (14:45 -0700)]
Nested Quota: Set default values to subproject
In case of hierarchical projects set the default value
(for resources who have the resource flag set) for a
sub-project to zero. This patch adds this functionality for
hierarchical projects, while keeping the functionality
intact for non-hierarchical project. At the top level,
project resources will get the default values as imposed
by the "default" class settings.
Co-Authored-By: Erickson Santos <erickson@lsd.ufcg.edu.br>
Change-Id: I3b357464d5e5e0aa065506ac1e9d908e87f45c63
Partially-Implements: bp cinder-nested-quota-driver
Tom Swanson [Wed, 8 Jul 2015 15:46:09 +0000 (10:46 -0500)]
Dell SC: Add support for ManageableVD
Added support for the ManageableVD volume driver class. This also
meant the driver changed from inheriting the volumedriver class to
inheriting the assorted *VD classes.
A few things about our ManageableVD support. We only allow management
of a volume if it is sized in multiples of 1GB. We only allow
management of volumes that are not currently mapped to a server on the
Storage Center. id-types of source-id (DeviceId on the Storage Center)
and source-name (Volume Name on the Storage Center) are supported.
Bob Callaway [Fri, 8 May 2015 19:33:41 +0000 (12:33 -0700)]
Fix NetApp cDOT driver use of Glance locations
As of Kilo, Glance returns a list of locations associated with an
image, but the NetApp cDOT driver only operated on the first one.
In addition, Glance has started enforcing a schema on the metadata
that can be added to images stored in the file glance_store. This
change modifies the cDOT driver to iterate through the list of
locations, using the correct schema-compliant fields, and uses the
first one with a valid IP address for our backend for optimizing
the copy of images into Cinder volumes.
Co-authored-by: Tom Barron <tbarron@dyncloud.net>
Closes-Bug: #1461116
This patch makes sure that the entire host is used
in the consistency group host entry. Previously,
whatever host entry was in the database was being
overwritten by the volume manager's self.host entry,
which doesn't contain a pool.
Alex Meade [Mon, 6 Jul 2015 14:18:18 +0000 (10:18 -0400)]
NetApp ONTAP: Add debug tracing
Add debug tracing to the NetApp ONTAP drivers using the
utils.trace_method and utils.trace_api decorators. This
is enabled in the driver via the 'trace_flags' configuration
option.
Alex Meade [Mon, 6 Jul 2015 14:17:43 +0000 (10:17 -0400)]
Add tracing facility for drivers
It is useful for engineering to be able to see the API traffic
to and from the storage controllers, as well as to see the
progression of method calls through the driver.
This work will add a simple config option to enable various
debug tracing types to be written to the debug log at runtime.
Use of the decorator has been added to NetApp drivers.
Drivers can make use of this facility by adding the
@cinder.utils.trace_method decorator to any method
calls they would like to allow tracing for or use trace_flags
for any other developer targeted debug logs. The trace_method
decorator can easily be added to every function of a class by
using the cinder.utils.TraceWrapperMetaclass or
cinder.utils.TraceWrapperWithABCMetaclass metaclasses. It is
advised to use the added decorators as the outer most decorator
for a function in order to avoid side-effects with other
decorators.
The @cinder.utils.trace_api method should be used to log
requests to storage backends by wrapping a method that
has the request as an argument and returns the response.
For example:
@six.add_metaclass(cinder.utils.TraceWrapperMetaclass)
class MyClass(object):
pass
Simon Dodsley [Tue, 7 Jul 2015 14:06:37 +0000 (10:06 -0400)]
Fix error message in Pure driver with correct text
The error message was specifically mentioning the PureISCSIDriver but
now there is a PureFCDriver which could call the error message. This
change gets the name of the driver and uses that to ensure
the error message refers to the correct driver.
wanghao [Tue, 26 May 2015 07:32:42 +0000 (15:32 +0800)]
Notify the transfer volume action in cinder
Now when we transfer a volume, there is not corresponding action
notification that is sent to ceilometer. Include the actions of create,
accept and delete.
The bp that this patch implements, proposes adding those action to enrich
the notification in cinder.
Notify is added in Create, Accept, and Delete actions.
TaoBai [Tue, 30 Jun 2015 07:03:38 +0000 (00:03 -0700)]
Storwize Driver zone removing
Storwize driver may remove zone even when the zone is still in use
by an attached volume. To fix this issue, we need only report initiator
and target map when there are no volumes attached.
tswanson [Mon, 29 Jun 2015 19:21:45 +0000 (14:21 -0500)]
Dell SC: Add support for consistency groups
Added support for consistency groups. Associated tests added to
test_dellsc.py and test_dellscapi.py. create_consistency_group_from_src
is not implemented in this patch.
Tom Barron [Sat, 20 Jun 2015 08:57:54 +0000 (04:57 -0400)]
Port NetApp E-Series iSCSI driver to ABC model
This commit modifies the NetApp E-Series volume driver to the
new ABC driver model. The corresponding FibreChannel driver
already uses that model since it was recently implemented and
was written with an appropriate ABC class signature in the first
place. A corresponding class signature is now used for iSCSI.
This commit also modifies a couple doc strings to match the
corresponding FibreChannel driver.
Xi Yang [Wed, 1 Jul 2015 07:50:27 +0000 (03:50 -0400)]
Fix getting out-of-date volume operation state issue for VNX
The driver tries to get a volume's operation state with no polling
in the function wait_until_lun_ready_for_ops, that means the driver
may get out-of-date data. For example, if a volume has just changed
to 'Preparing' state, the function may get the state as 'Ready' and
report that the volume is ready to use. The fix is to get the volume
state with polling at the first time, then the driver would report
the right state of the volume.
xiaolei hu [Thu, 25 Jun 2015 02:22:50 +0000 (10:22 +0800)]
Separate FlashSystem FC and iSCSI common code
The patch is mainly to split original FC driver code into common
code and FC driver for IBM FlashSystem. iSCSI driver merged in L-1
has been inherited from original FC driver for common functions
already.
In this patch,
* Separate FC driver into new driver and common code.
* Modify iSCSI driver to inherite the common code.
* Common function _get_node_data in iSCSI driver is moved into
common code.
* Add locks in initialize_connection and terminate_connection
for iSCSI driver.
* Remove flashsystem_multipath_enabled to use Cinder iSCSI
multipath support.
Code Review / openstack-build / cinder-build.git / log