Xing Yang [Mon, 22 Jun 2015 21:46:32 +0000 (17:46 -0400)]
Non-disruptive backup
This patch adds support for non-disruptive backup for
volumes in 'in-use' status as follows:
Adds a force flag in create backup API when backing up
an 'in-use' volume.
For the default implementation in volume/driver.py:
* Create a temporary volume from the original volume
* Backup the temporary volume
* Clean up the temporary volume
For the LVM driver:
* Create a temporary snapshot
* Obtain local_path for the temporary snapshot
* Backup the temporary snapshot
* Cleanup the temporary snapshot
Attach snapshot will be implemented in another patch.
Philipp Marek [Wed, 22 Jul 2015 11:36:24 +0000 (13:36 +0200)]
DRBD: Rename a constant to a better name
As requested in a Icb7347d55dc5b22720269405b6d2b4053c2ad819 review
(https://review.openstack.org/#/c/178573/19..20/cinder/volume/drivers/drbdmanagedrv.py)
constants should consist of uppercase letters only.
Tom Swanson [Tue, 14 Jul 2015 17:21:19 +0000 (12:21 -0500)]
Dell SC: Fix Consistency Group issues
The Dell Storage Center driver was not honoring the consisgroup_id
in the volume object in volume creation. Updated create_volume,
create_cloned_volume and create_volume_from_snapshot to honor this
value.
The driver was also not checking that volumes were initialized prior
to snapshotting a consistency group. The Dell backend does not
actually create the volumes until there is data or the volumes are
mapped to a server.
On create_volume_from_snapshot the driver was not checking for
cgsnapshot_id before snapshot_id.
Changed from calling the db to update snapshots to retrieving
the SnapshotList() object.
In addition some tests were not checking results. Fixed that.
Anton Arefiev [Wed, 17 Jun 2015 15:57:34 +0000 (18:57 +0300)]
Add drivers list generator
It is useful to have a maintained (in source control) list
of drivers that exist in Cinder. It could be used in docs
and unit tests to check method impl on backend drivers.
This change add tool for generate list of drivers based on
existing BaseVD class in class hierarchy of volume drivers.
Test test_create_volume_with_consistencygroup_invalid_type in
tests.unit.api.v2.test_volumes is verifying consistency group
logic in cinder.volumes.api. This patch moves this test to
tests.unit.test_volume.py.
Philipp Marek [Mon, 20 Jul 2015 20:23:19 +0000 (22:23 +0200)]
DRBD: Define a separate prefix for snapshots
As snapshots and volumes are done via one namespace (DRBD resources),
having distinct names helps during debugging - it's easy to see which
type of Cinder element is.
Patrick East [Mon, 13 Jul 2015 22:49:40 +0000 (15:49 -0700)]
Prevent missing Purity hosts from raising errors
Occasionally a Purity host can be deleted out from underneath Cinder.
When this happens if cinder is attempting to disconnect a volume from
that host we should treat it as a ‘success’ since if the host is gone,
it is disconnected already (as far as Purity is concerned).
To prevent one cause of this, the disconnect method will now be locked
with the connect ones so that can’t delete a host we intend to re-use.
chenzongliang [Thu, 4 Jun 2015 02:43:07 +0000 (10:43 +0800)]
Refactor Huawei Volume driver
This patch attempts to refactor Huawei volume driver in liberty.
We add a base driver to implement the basic functions.
The sub-class will inherit from the base driver according to different
storages.
The following changes were made in this refactor:
1. Abstract a base class named HuaweiBaseDriver to make Huawei driver more
universal. You can find it in the huawei_driver.py.
2. Put all static variables into the constants.py.
3. Rename rest_common.py to rest_client.py. rest_client.py stores the
relevant methods implemented for Huawei driver.
4. Migrate some public methods from rest_client.py to huawei_utils.py,
such as parse_xml_file(), _get_volume_type() and so on.
5. This refactor only involves structural adjustment and does not involve
functional changes.
Backups: allow name to be specified during restore
This commit adds the ability to specify a `name` attribute during backup
restore. This will create a volume with the specified name during
restore, instead of the default pattern of the form
`restore-backup-<id>`. In case volume id is specified the `name`
attribute is ignored.
Matt Riedemann [Thu, 16 Jul 2015 20:51:23 +0000 (13:51 -0700)]
scality: add export and name keys to os-initialize_connection info
Nova is standardizing on some of the common code for file system type
volume drivers in blueprint consolidate-libvirt-fs-volume-drivers and
expects the 'export' and 'name' keys to be in connection_info['data']
returned from the os-initialize_connection Cinder API.
The Scality volume driver wasn't providing these keys so those are added
here.
Related Nova change: I7db834956c67383e63be2fe2c465118e00273e4b
Anna Sortland [Thu, 16 Jul 2015 14:33:46 +0000 (09:33 -0500)]
Remove "volume:services" rule from policy.json
The rule "volume:services": "rule:admin_api" does not seem to be used.
The cinder/api/contrib/services.py is using "volume_extension:services".
This change deletes "volume:services" rule from policy.json.
Jinru Yan [Thu, 9 Jul 2015 01:30:45 +0000 (09:30 +0800)]
Report capability of multiattach for FlashSystem
This patch enables the multiattach capability reporting in
_update_volume_stats. This will allow nova to attach and detach
a volume to multiple virtual machine instances.
John Griffith [Tue, 7 Jul 2015 22:26:21 +0000 (16:26 -0600)]
Handle volume not found on zfssa volume delete
The zfssa driver doesn't handle cases where a volume may not
exist on the backend when a delete call is sent. The result
is a stack-trace in the logs, sometimes a crash of the service
and an inability to clean up volumes from Cinder's perspective.
This patch doesn't address the root cause of the issue, but
it does handle the case more gracefully by catching the
exception, logging an error and returning.
This way the volume can be cleaned up on the Cinder side and
the operator still has an indication that something went wrong.
This is a common pattern for most of the drivers in Cinder.
If backup cannot be found while resetting its status, Cinder
exception (BackupNotFound) is raised instead of HTTP exception.
This patch handles the BackupNotFound exception and raises
HTTPNotFound exception.
APIImpact
HTTPNotFound exception (404) is raised instead of BackupNotFound
if backup cannot be found during backup reset-status.
Ivan Kolodyazhny [Tue, 14 Jul 2015 15:24:29 +0000 (18:24 +0300)]
Remove obsolete API from documentation
Cinder doesn't support EC2 API and doesn't contain cinder.api.ec2
module.
Also references to cinder.api.openstack.auth and cinder.api.cloud were
removed too.
Eric Harney [Tue, 30 Jun 2015 20:23:51 +0000 (16:23 -0400)]
Tests: test_volume mock conversion
The goal here is to try to drive down complexity in
our tests. Note that this file is massive and this
will be a gradual effort, not something that will
be completed in one patchset.
osprofiler is mocked out in all unit tests now, so
remove that from here.
Make a first pass at converting these tests over to
mock and strengthen the assertions for calling methods
that were previously mocked with stubs.
Remove a bunch of "fake" methods, using mock instead.
Switch to mock decorators for some tests where it
makes the code easier to follow. This helps because
it separates test setup from the test itself more
clearly.
Fix restore point if backup base is diff-format in ceph
In Ceph when backup base is diff-format and use this backup
to restore volumes not rbd, need to find out the correct restore
point.
This is fixed in cinder/backup/drivers/ceph.py _diff_restore_allowed.
Before the change: when backup is diff-format, and destination volume
is not rbd, the function returns restore_point as None. As a result,
backup restore uses wrong image/snapshot to do restore.
With the change, we always get the right restore point, regardless
of the kind of restore we'll do differential or not.
peter_wang [Tue, 30 Jun 2015 02:56:40 +0000 (22:56 -0400)]
Add white list support for target ports in VNX driver
Currently, VNX cinder driver registers host initiator to all
configured iSCSI/FC ports on array and chooses ports from
all available ports to return in initialize_connection.
By introducing a new driver option io_port_list:
1. driver will only register initiator to ports in io_port_list
2. driver will only return ports in io_port_list from initialize_connection
Patrick East [Mon, 13 Jul 2015 20:42:46 +0000 (13:42 -0700)]
Preserve mock side_effect’s in test_pure
The helper method to test if exceptions are propagated or handled would
unset the mock objects side_effect value when its done. This overrides
the original value making the mock not behave as expected as soon as it
is passed into assert_error_propagates.
This changes the helper to store the original side effect and then put
it back once we are done making it throw an exception.
Peter Penchev [Fri, 12 Jun 2015 07:41:52 +0000 (10:41 +0300)]
StorPool: clean up the last uses of str.format()
Replace three logging uses of str.format() with parameters passed
directly to LOG.error(), simplifying them in the process by
avoiding the need to call six.text_type() ourselves.
Replace the remaining use with string concatenation.
Suggested by: Sean McGinnis in review 188725
Change-Id: Ie53c394d7f31fa1dccfa4ed5c4b514539c77aa05
Sean McGinnis [Wed, 1 Jul 2015 20:14:57 +0000 (15:14 -0500)]
Remove useless logging from unit tests
It has been discussed that there really isn't much point in having
unit tests making any kind of logger calls. Some usage has already
been cleaned up. This patch removes the remaining instances of log
calls under the cinder/tests directory.
Also cleaned up a lot of cases where the source files would import
oslo_logging and declare a LOG variable which was never actually
used.
Leaving logging in the cinder/tests/unit/integrated tree for now
until a plan is decided as to what to do with all of these type of
tests.
Added hacking check to prevent new instances from slipping by code
reviews.
Gorka Eguileor [Fri, 26 Jun 2015 14:05:04 +0000 (16:05 +0200)]
Fix saving tz aware datetimes in Versioned Objects
Currently we cannot save date fields in Versioned Objects because they
are timezone aware and our database is not.
If we look at our objects definition, we are not giving any value to
tzinfo_aware argument, so it takes DateTime default value (True).
In our sqlalchemy model definitions we are defining our datetime fields
as Column(DateTime) which will take sqlalchemy default: class
sqlalchemy.types.DateTime(timezone=False)
In most cases we are accessing those fields directly through the DB
methods, so we don't see the problem, but if we were to modify any of
those fields using versioned object, for example scheduled_at, and then
save the object we'll get an exception:
File ".../sqlalchemy/sql/type_api.py", line 278, in compare_values
return x == y
TypeError: can't compare offset-naive and offset-aware datetimes
Because we are trying to save a timezone aware date to a timezone
unaware DB field.
Following Dan's advice we create a specific cinder_obj_get_changes,
instead of overwriting obj_get_changes, that returns all datefields as
timezone naive UTC datetimes ready for saving.
Dave Chen [Fri, 16 Jan 2015 09:59:18 +0000 (17:59 +0800)]
set/unset volume image metadata
This patch implements:
- Create, delete, update APIs for modifying volume image metadata.
- Refactoring in the volume API to accommodate both user and
image metadata.
- All of the necessary testcases needed for the changes.
DocImpact
APIImpact
Partially implements: bp support-modify-volume-image-metadata
Change-Id: I22792ef7bd49c763d7c130aa8cac9abc3fff2d4c
Jay S. Bryant [Fri, 10 Jul 2015 21:38:15 +0000 (16:38 -0500)]
Fix not implemented wording in update_migrated_volume
It was recommended in review https://review.openstack.org/#/c/180873
that the message sent to the NotImplementedError exception be reworded.
The recommendation was made late in the review process and was not
corrected. This patch implements the recommendation.
wanghao [Fri, 20 Mar 2015 07:51:57 +0000 (15:51 +0800)]
Add support for force-delete backups
1. Add _get and _delete method to BackupAdminController.
2. Add force=False arg to delete method in backup api.
3. If force=True, then allow the deletion of backup, which
can be in any state.
4. For chunkeddriver based backup, check the status of backup
from DB in backup loop, if it has changed to deleting, set the
flag is_backup_canceled and clear the data to avoid leaving orphan
object in the object store.
5. Add the flag: support_force_delete=False in BackupDriver. That
indicates if the backup driver supports force deletion.
It should be set to True if the driver that inherits from BackupDriver
supports the force deletion function.
6. If the backup driver doesn't support this feature, and it will return
405 when calling API of force delete.
Anthony Lee [Fri, 26 Jun 2015 17:38:33 +0000 (10:38 -0700)]
Improve 3PAR driver VLUN creation and deletion
This patch improves the creation and deletion logic for VLUNs
for the 3PAR drivers. The changes will help to prevent VLUNs
from being left behind after volume detaches. It also prevents
extra VLUNs being created when initialize_connection is called
multiple times for the same volume and host.
gate-cinder-python27 gate-cinder-python34 is reporting failures where it
wasn't before. This is all due to new mock version 1.1.0. This patch
fixed the tests to pass the gate.
Some of these errors are actual errors in our test code, while other are
changes to Mock syntax.
Anton Arefiev [Tue, 31 Mar 2015 16:44:57 +0000 (19:44 +0300)]
Fix properties extracting from image with glance api v2
When a volume is created using the glance v1 API, the non based
properties places in 'properties' dict, and nova is expected
to find them here. Otherwise api's v2 return custom image properties
as base properties, so nova can't find them in volume 'image
properties'.
This change add new glance call to get image schema, it allows
extract custom properties from image.
Xing Yang [Fri, 19 Jun 2015 13:47:59 +0000 (09:47 -0400)]
Support SMI-S provider v8.0.3 in VMAX driver
This patch made changes in the VMAX driver to support
interface changes in SMI-S provider v8.0.3.
* Accommodate System Name prefix change from 'SYMMETRIX+' to
'SYMMETRIX-+-'.
* Delete composite volume - changed from using
ReturnElementsToStoragePool to EMCReturnToStoragePool.
* Extend volume - removed InPool parameter from EMCUnBindElements
method.
* Create snapshot/clone with multi meta members -
EMC_PartialAllocOfConcreteExtentCIM was deprecated;
used 'GetCompositeElements' instead and changed how we get meta
members' capacities.
* Create CG snapshot - A new WaitForCopyState parameter was added.
* Added helper function to determine SMI-S provider and SE version
Note: SMI-S provider is bundled with SE (Solutions Enabler) and
they have the same version number starting from v8.0.
The EMC SMI-S Provider supports the SNIA Storage Management Initiative
(SMI), an ANSI standard for storage management. It supports the VMAX
storage system. SE is a software product that discovers and manages
VMAX storage system.
Add ability to override OpenStack privileged user auth url
Introduce os_privileged_user_auth_url config to give the ability
to override the auth_url used when authenticating
the OpenStack privileged user and bypass use of catalog found in token.
DocImpact: New os_privileged_user_auth_url config
Closes-bug: #1473206
Change-Id: I4ffca8df0eb38fe41264439ae5bb93e025c808ff
Ryan Lucio [Thu, 12 Mar 2015 23:12:10 +0000 (16:12 -0700)]
VMEM v6000: Fix export verify routines
New 'state' nodes in vmos6 are reflecting the kernel/system state
indicating lun accessibility on the data plane. The nodes used
previously were only reflecting success on the mgmt plane, meaning
that the routines were sometimes returning before the target/export
was actually available for client use.
Also, switch to using one global iscsi target group (backend
container that groups target IPs) to avoid a backend export bug
exposed by the changes mentioned above.
Code Review / openstack-build / cinder-build.git / log