Xing Yang [Wed, 11 Mar 2015 04:42:16 +0000 (00:42 -0400)]
Sort snapshots in create CG from CG snapshot
This problem was discovered while testing create CG from CG snapshot.
In create_consistencygroup_from_src() in manager.py, it calls the driver
method create_consistencygroup_from_src() and passes in parameters
including source snapshots and target volumes. The problem is that the
snapshots list is not sorted to be in the same order as their corresponding
volumes. So they may be in the wrong order.
In this patch, the source snapshots were sorted before being passed to
the driver.
Tom Barron [Thu, 15 Jan 2015 23:03:16 +0000 (18:03 -0500)]
Refactor Swift backup driver and introduce chunking driver
Abstract out a "chunked" backup driver class between the top
abstract backup driver class and the Swift backup driver class
which implements common functionality for backup drivers that
store volume data in multiple "chunks" in a backup repository
when the size of the backed up cinder volume exceeds the size
of a backup repository "chunk."
The Swift driver itself is reworked to extend the chunked
backup driver and implement abstract methods defined in the
chunked backup driver. It is expected that posix filesystem
and NFS backup drivers will also extend the chunked backup
driver.
Co-authors: Kevin Fox <kevin@efox.cc>
Tom Barron <tpb@dyncloud.net>
Inspired by Dave Chen's change request on keystone about this, I
decided to check in cinder code to see if there were many cases.
Since unittest provides assertTrue and assertFalse, it is better
to use assertTrue(var) instead of assertEqual(True, var) and the same
thing applies to assertFalse.
I used these commands to search the occurrences:
* grep -PRIn "assertEqual\(True"
* grep -PRIn "assertEqual\(False"
* grep -PRIn "assertEqual\(\w+\s*,\s*True"
* grep -PRIn "assertEqual\(\w+\s*,\s*False"
Mike Perez [Mon, 9 Mar 2015 21:38:50 +0000 (14:38 -0700)]
Update Datera's export to avoid deprecated keys
The export call has been updated in Datera. The old location for the IQN
has moved. This updates the Kilo driver with support of looking in the
new location. The Juno driver will remain using the old location.
Erlon R. Cruz [Mon, 12 Jan 2015 18:59:10 +0000 (16:59 -0200)]
Adds pool aware scheduling for HNAS drivers
This commit adds the pool aware feature for HNAS drivers. Now, each service
label in HNAS XML config will be considered as a pool unit. In order to be able
to cast a volume to a service, the admin must create 'volume_type's and each
volume type need to have the 'service_label' with the same tag set in the XML
config, and then the user can create a volume with the desired volume_type.
Patrick East [Fri, 13 Feb 2015 21:35:51 +0000 (13:35 -0800)]
PureISCSIDriver consistency group updates.
This will enable the driver to add and remove volumes from Purity
protection groups that correspond to Cinders consistency groups, as well
as create a consistency group from a cgsnapshot.
Mike Perez [Sat, 7 Mar 2015 01:24:00 +0000 (17:24 -0800)]
Don't fail target_delete if ACL's don't exist
Seen in the gate a bit where a target delete is called and fails because
the ACL's don't exist for that target. If we're deleting a target and
the ACL's don't exist, it's probably safe to move on.
This patch adds a check for this specific case, logs a warning, and
continues rather than raising an exception.
Glenn M. Gobeli [Mon, 23 Feb 2015 12:35:34 +0000 (04:35 -0800)]
Add Manage/Unmanage support to NetApp NFS drivers
This change adds support for the Cinder Manage and Unmanage API calls
to the NetApp NFS drivers. This does not add the support for the generic
NFS driver.
Thang Pham [Thu, 5 Mar 2015 23:18:33 +0000 (18:18 -0500)]
Use snapshot object in create_volume flow
create_volume flow was making a call to db.snapshot_get to get
a snapshot dict and passing it onto drivers in
create_volume_from_snapshot() instead of an object. This causes
a conflict for some drivers, since create_snapshot and
create_volume_from_snapshot methods have two different snapshot
types (object vs. dict). The following patch fixes this by using
objects in create_volume flow and allows drivers to be updated
appropriately to use snapshot objects.
The strings can be got from the [DEFAULT] section or a driver instance group
section.
If the string can not be found in the cinder.conf, it will try to find a
default filter_fuction or a default goodness_function which can be specified
in the driver code. HP3par drivers don't provide default values. However if
other driver want to provide default values, it can be done by overwriting
get_default_filter_function and get_default_goodness_function.
2. Also added goodness_function and filter_function to the
_update_volume_stats in ISCSIDriver, FibreChannelDriver, FakeISCSIDriver
and LVMVolumeDriver.
3. Added 2 stats total_volumes and capacity_utilization so they can be
used as the part of the formula in filter_function or goodness_function
string.
4. Fixed free_capacity for limited cpg to include SDUsage's usedMiB as part of
the calcuation so it is consistent with unlimited cpg and capacity_utilization
looks better.
5. Added total_volumes capability to the LVM driver.
Xing Yang [Mon, 12 Jan 2015 19:16:28 +0000 (14:16 -0500)]
Add support to incremental backups in cinder
1. Added a new option to create backup to support incremental backups.
New CLI option added for "cinder backup": --incremental or --incr
By default, it is full backup.
Add the following to API: 'incremental': True or False
(False is the default)
2. swift.py creates sha256file for every backup to calculate deltas
Erlon Cruz [Thu, 12 Jun 2014 10:55:38 +0000 (07:55 -0300)]
Adds SSH communication to HNAS drivers
This commit adds SSH communication to HNAS iSCSI and NFS drivers. The user now
has the option to export the server public keys to HNAS and configure the
authentication via public keys. This avoid exposing HNAS passwords in the plain text
config file and exposing the private HNAS internal network.
Patrick East [Fri, 20 Feb 2015 02:49:23 +0000 (18:49 -0800)]
Add CHAP support to PureISCSIDriver
This change adds support for iSCSI CHAP with randomly generated
secrets for each initiator. Due to some restrictions with storing meta
data in the backend these secrets need to be stored someplace accessible
from the driver. We will use the driver specific initiator data in the
Cinder DB to store them.
John Griffith [Tue, 3 Mar 2015 21:27:27 +0000 (21:27 +0000)]
Make objects behave more like our old dictionaries
Prior to Cinder Objects we passed a dictionary representation
of a Cinder resource around. There are a few places where we
abuse this a bit and do things like "ref_dict.get('some-key', None)"
to reuse some code and determine if for example the object is
a snapshot or a volume (based on it's keys). This is probably
a pretty lazy/bad idea, but we have cases where we do it,
such as in the SolidFire driver and it looks like maybe some
calls in the profiler.
For now, we just make the compatibility shim in Object base truly
compatible and return None for non-existent attributes rather
than raising. Follow up work would be to clean up all of our
usage to strictly use objects and get rid of the compatibility
shim altogether.
nikeshmahalka [Wed, 4 Mar 2015 09:04:42 +0000 (14:34 +0530)]
Two choices for iscsi_helper is missing
In the cinder/volume/driver.py, scstadmin and ietadm were missing from
the choices for iscsi_helper. This will cause SCST and IET target helper
functionality to not work properly.
Mike Perez [Sat, 28 Feb 2015 07:48:46 +0000 (23:48 -0800)]
Update Datera's Authentication method
Previous auth-token/client verification auth method is deprecated. It's
supported until Liberty release. This also supports the new method,
which is using san_login and san_password, which will fetch an
auth-token to use.
Tom Swanson [Wed, 4 Mar 2015 19:56:13 +0000 (13:56 -0600)]
Dell SC API change fails snapshot creation
The Dell Storage Center driver is failing snapshot creation on
later versions of the Dell REST API.
The doNotExpire option was removed in API 15.3.1.110. We were
setting this so that the replays are not expired and removed when
Cinder expects them to still be there. This has been replaced by
setting expiretime to 0. We were doing this already so the fix
was to simply remove doNotExpire from the REST request payload.
This will fail on every future release of the Dell Storage Center
so this is a critical fix.
Jeegn Chen [Wed, 4 Mar 2015 13:10:34 +0000 (21:10 +0800)]
More validation logic for VNX CG creation
In VNX array, compressed LUNs cannot be added into a Consistency Group.
VNX Cinder Driver should check the volume types.
If storagetype:provisioning=compressed is specified,
VNX Cinder Driver should reject the CG creation operation.
When parsing group['volume_type_id'], the trailing comma should be
considered.
Patrick East [Fri, 22 Aug 2014 23:43:20 +0000 (16:43 -0700)]
Add DB table for driver specific data
This will allow drivers to store key/value pairs in the database for any
needs they have requiring persisted storage for initiator information
across cinder service hosts.
Jay S. Bryant [Tue, 3 Mar 2015 03:31:46 +0000 (21:31 -0600)]
Move oslo.messaging to the oslo_messaging namespace
This is the sixth in a series of changes to move to using
the new oslo_<library> namespace that is being used for
oslo libraries.
There is currently a shim in place that is allowing the old
oslo.<library> imports to work, but we need to be prepared for
when the shims go away. Thus we need patches like this one to
move to the new namespace.
This patch also updates our hacking check to ensure that no
instances of oslo.messaging sneak back in.
Xing Yang [Tue, 24 Feb 2015 02:38:14 +0000 (21:38 -0500)]
Create Consistency Group from CG Snapshot API
This patch addressed the following:
* Added a new create Consistency Group from CG Snapshot API.
- Note this is separate from the Create Consistency Group
API which requires volume types as the input.
* Added a corresponding driver API.
Xing Yang [Wed, 31 Dec 2014 02:27:56 +0000 (21:27 -0500)]
Modify Consistency Group API
This patch addressed the following:
* Modify Consistency Group
* Added an API that supports adding existing volumes to CG and removing
volumes from CG after it is created. It also allows the name and the
description to be modified.
* Added a volume driver API accordingly.
Guan Qiang [Mon, 9 Feb 2015 06:18:35 +0000 (14:18 +0800)]
Remove useless storage_availability_zone import
Config 'storage_availability_zone' have been removed from cinder.volume.manager
long time ago, but Still been imported by cinder.volume.api and
cinder.consistencygroup.api.
Tomoki Sekiyama [Tue, 9 Dec 2014 22:09:43 +0000 (17:09 -0500)]
Failover to alternative iSCSI portals on login failure
When the main iSCSI portal is unreachable by network failure etc.,
volume attach will fail even if the array has the other portal
addresses alive. To enable nova-compute and brick (with corresponding
patches) to fall-back to alternative portals, this patch makes Cinder
to contains the multiple portal addresses and corresponding IQNs and
LUNs in connection_info returned by initialize_connection API.
The main portal information is also returned in target_portal,
target_iqn, target_lun for backward compatibility. For example:
This patch makes LVM iSCSI backend return multiple portals,
iqns, and luns when iscsi_secondary_ip_addresses is specified in
cinder.conf.
It also make brick/initiator/connector.py to failover to alternative
paths, if specified, when it fails to establish the main path.
In addition, to unify the data structure, target_portal, target_iqn,
target_lun will also be included even if 'multipath=True' is specified
in connector information.
Thang Pham [Mon, 10 Nov 2014 22:38:42 +0000 (17:38 -0500)]
Snapshot and volume objects
Abstract volumes and volume snapshots into objects.
Get, create, and delete snapshot APIs were changed to
use new snapshot objects. A skeleton volume object was
created, but cinder internals were not changed to use
the volume object, although volume is referenced and
used by the snapshot object. The internals will be
changed to use volume object in a subsequent patch.
Thang Pham [Mon, 10 Nov 2014 22:02:02 +0000 (17:02 -0500)]
Cinder objects base
This patch is to insert the objects code base, currently based
on nova.objects. It will be later based on
oslo_versionedobjects once it is available. This is done
to make progress on changing cinder internals to use objects,
since nova.objects and oslo.versionedobjects are similar.
Inserted objects code base into cinder and modified cinder services
to use object-aware serializer.
Tom Swanson [Fri, 27 Feb 2015 17:25:38 +0000 (11:25 -0600)]
Dell Storage Center Driver API bug
In later versions of the Dell Storage Center REST API
the index attribute is removed from the ScVolume object.
Referencing this caused an exception that failed volume
creation. Replaced with instanceId.
Also _init_volume was mapping/unmapping to every server
that was currently up on the Storage Center rather than
a single server. While this would certainly initialize
the volume it would create a lot of extra work on a busy
system. We now break after the first map/unmap.
Changed copyright to 2015 to match when it will actually
be released.
Clear migration_status from a destination volume if migration fails
When some error happens during volume migration, cinder-volume
keeps remaining a destination volume because the volume copy may be
in the completing phase and a source volume may be deleted already.
In this case, the migration_status of the destination volume in
data base should be cleared. As a result, the volume can be handled
properly even if volume migration is failed. If the migration is
in the migrating phase, the volume should be deleted and an entry
of database is also destroyed because the source volume is remaining.
Also there is an another problem that a source volume is remaining
as migrating status if create_export raises exception. In this case,
source volume can't be deleted even if it will be unnecessary.
Jon Bernard [Tue, 24 Feb 2015 21:16:26 +0000 (16:16 -0500)]
RBD: Query volume features from ceph.conf
This patch enabled RADOSClient to query the local ceph.conf for rbd
features configuration, falling back to layering if nothing is found.
This allows users to disable default features or to opt in to upcoming
features that have not yet become default.
Xing Yang [Fri, 27 Feb 2015 05:45:09 +0000 (00:45 -0500)]
Correct a few changes in the VMAX driver
The following patch made a few wrong changes in the VMAX driver.
https://review.openstack.org/#/c/145780/
The following two files contain the wrong changes:
https://review.openstack.org/#/c/145780/15/cinder/tests/
test_emc_vmax.py line 604
https://review.openstack.org/#/c/145780/15/cinder/volume/drivers/
emc/emc_vmax_https.py
Erlon R. Cruz [Thu, 27 Nov 2014 18:44:23 +0000 (16:44 -0200)]
Fix HNAS driver parsing errors
Depending on some configurations of HNAS array, the driver returns an error
during volume creation or driver initialization. This patch adds some checks
to fix those parsing errors.
Jeegn Chen [Fri, 27 Feb 2015 06:59:48 +0000 (14:59 +0800)]
More validation logic for VNX CG creation
In VNX array, compressed LUNs cannot be added into a Consistency Group.
VNX Cinder Driver should check the volume types.
If storagetype:provisioning=compressed is specified,
VNX Cinder Driver should reject the CG creation operation.
Mike Perez [Thu, 26 Feb 2015 20:07:22 +0000 (12:07 -0800)]
Update v1 deprecation warnings
As discussed on IRC and mailing lists, we're going to give more time for
deprecation of the v1 API. Since we introduced the deprecation of the v1
API in Juno, we can remove it after EOL of Juno, which will be the
release of Liberty.
Review 145780 broke the lefthand driver. The author and reviewers
did not even bother to look at the 3rd party CI results that clearly
showed that this patch broke the hplefthand driver, due to changing
the imports. The change removed an import to one of the hplefthandclient
modules, which was then later accessed.
Reviewers, please check 3rd party CI when a cinder patch touches a driver.
Authors of patches, if you touch a driver, you need to check the 3rd party
CI failures and make sure your patch changes aren't causing the failures.
Steven Kaufer [Tue, 10 Feb 2015 21:12:16 +0000 (21:12 +0000)]
Replication status periodic task optimization
A periodic task exists to update the replication status for all volumes.
Currently, this task executes for all drivers and always retrieves all
volumes for the current host from the DB.
This patch set:
* Ensures that the periodic task is only activated if the driver actually
supports replication
* Only retrieves volumes from the DB if 'replication_status'!='disabled' in
the periodic task
Also, the driver documentation in cinder.volume.driver.VolumeDriver.
get_volume_stats() is updated to reflect that the 'replication' key indicates
that the driver supports replication; this is the key that was actually
implemented in the drivers that support replication.
Code Review / openstack-build / cinder-build.git / log