Jay S. Bryant [Thu, 12 Mar 2015 18:34:16 +0000 (13:34 -0500)]
Remove strutils from oslo-incubator
This change goes along with the work to get away from
oslo-incubator modules to using the new oslo libraries.
The strutils module moved to the oslo_utils library and
should no longer be used from oslo-incubator.
Additionally, merging this change removes a dependency on
gettextutils, allowing us to remove another deprecated
module.
Jay S. Bryant [Thu, 12 Mar 2015 15:06:23 +0000 (10:06 -0500)]
Update hacking check for oslo_log
With commit 5980da5d we moved Cinder to using the olso.log
library. We also moved to the new oslo_log namespace. With that
change we should have also updated the hacking checks to include
looking for old uses of oslo.log.
Vincent Hou [Wed, 29 Oct 2014 08:40:31 +0000 (01:40 -0700)]
Remove the useless next link for volumes, transfers and backups
Two cases this patch will resolve:
1) Currently if the number of items equals the osapi_max_limit or
the last page of items equals the osapi_max_limit without the
parameter limit set in the user request, a next link is generated
in the response, though this next link will return empty volume
list. In fact it is unnecessary to generate the next link in
this case.
2) If the number of items equals the osapi_max_limit and limit is
greater than osapi_max_limit, a next link is generated. Actually,
the next link does not need to be generated, because it is certain
that there is no more volumes left in the database.
The method _get_collection_links has been called in volumes,
volume_transfers and backups. The patch can only affect the next
link generation for three of them. However, other lists like
consistency groups, qos specs, cgsnapshots have not implemented the
generation for the next link. Potentially this can be a wishlist
item for them.
Mike Perez [Wed, 11 Mar 2015 19:27:32 +0000 (12:27 -0700)]
Unset auth token before trying to login to Datera
After a token expires, the Datera driver would try to login, and pass
along the expired token. This would fail because the receiving server
will authorize anytime it's given an auth token, even for endpoints that
don't require it. This change unsets the auth token when logging in, so
it's not passed a long.
Tom Barron [Mon, 19 Jan 2015 17:58:02 +0000 (12:58 -0500)]
NFS backup driver
Implements an NFS backup service in accord with the NFS backup driver
blueprint spec (https://review.openstack.org/#/c/130858).
Currently the NFS backup driver inherits directly from the abstract
chunked backup driver. It should be straightforward to abstract out
of this code a posix filesystem backup driver, without dynamic mount
capabilities, which itself inherits from the chunked backup driver.
When that is done, most of the code in this implementation can be
removed and it can be modified to inherit from the posix filesystem
backup driver itself.
This patch is a follow up to the multiattach patch that adds
the ability to the scheduler to be able to filter capabilities
on the volume multiattach flag. When a volume creation request
comes in with the multiattach flag set on a volume, there was no
way to filter out backends that don't support attaching a volume
to more than one host. By defaut, no host supports it. When
The Nova side lands, we can adjust drivers to add the new
multiattach:True capability reporting.
Eric Harney [Tue, 10 Mar 2015 20:34:16 +0000 (16:34 -0400)]
Tests: Harden fake_notifier asserts
This is a continuation of 8740e063 - Make test_create_delete_snapshot more robust
The same issue cropped up for me in
test_create_consistencygroup_from_src().
Add some more specific asserts to try to reveal what
is going wrong here. The idea is to assert more
specific things (examining individual notifications)
before more general asserts (count of notifications) when
possible, to fail the test on the check that will provide
the most information back.
Zhiteng Huang [Tue, 24 Feb 2015 09:05:46 +0000 (17:05 +0800)]
Allow scheduler to receive volume stats when starting service
Filter scheduler relies on volume services to report their stats in
order to make decisions - this nature makes scheduler service a bit
special when it starts because it can't really do anything without
knowning stats of avilable volume backend. So the original
implementation of Filter Scheduler added a hook in init_host() to
ask all volume services to send their stats immediately (volume RPC
call 'publish_service_capabitlies()'), this reduced the chance of a
new volume being set to 'error' state because scheduler has no
knowledge of backends to minimum.
However, commit 65fa80c361f71158cc492dfc520dc4a63ccfa419 moved
init_host() ahead of service RPC initialization because for volume
services, service initialization should be done before the service
starts accepting RPC requests. This change unfortunately invalidated
scheduler service's init_host() because that needs to be done *AFTER*
RPC is ready (a queue is bound to scheduler fanout exchange). The
result is scheduler asks vol service to send update, but it misses
the information it asked for because its queue hasn't been created
and bound to the fanout queue yet. So scheduler always has to wait
for the next update from volume services to be able to work properly.
This change adds a new hook to Manager class called init_hook_with_rpc()
to allow services like scheduler to do something once RPC is ready.
This should restore scheduler's behavior before commit 65fa80c361f71158cc492dfc520dc4a63ccfa419.
Stefan Amann [Mon, 26 Jan 2015 12:49:16 +0000 (13:49 +0100)]
Adjust Cinder to support FCP on System z systems
There are some platform specific changes needed to allow Cinder to
support FCP on System z: the System z specific format of the device file
paths, adding and removing devices explicitely, and ignoring vHBAs which
are offline.
Tom Barron [Thu, 15 Jan 2015 23:03:16 +0000 (18:03 -0500)]
Refactor Swift backup driver and introduce chunking driver
Abstract out a "chunked" backup driver class between the top
abstract backup driver class and the Swift backup driver class
which implements common functionality for backup drivers that
store volume data in multiple "chunks" in a backup repository
when the size of the backed up cinder volume exceeds the size
of a backup repository "chunk."
The Swift driver itself is reworked to extend the chunked
backup driver and implement abstract methods defined in the
chunked backup driver. It is expected that posix filesystem
and NFS backup drivers will also extend the chunked backup
driver.
Co-authors: Kevin Fox <kevin@efox.cc>
Tom Barron <tpb@dyncloud.net>
Jay S. Bryant [Fri, 20 Feb 2015 21:21:55 +0000 (15:21 -0600)]
Namespace updates for _i18n and imageutils & fileutils
The _i18n and imageutils modules only have changes
to move oslo.utils to the oslo_utils namespace and
oslo.i18n to oslo_i18n. So, I have lumped these
two small changes into this one patch.
Current HEAD in OSLO:
---------------------
commit e589dde0721a0a67e4030813e582afec6e70d042
Date: Wed Feb 18 03:08:12 2015 +0000
Merge "Have a little fun with release notes"
Changes merged with this patch:
---------------------
_i18n.py 0cc741a9 - switch to oslo_i18n
imageutils.py ac17de97 - Use oslo_utils instead of deprecated oslo.utils
fileutils.py ac17de97 - Use oslo_utils instead of deprecated oslo.utils
Sean McGinnis [Tue, 10 Mar 2015 16:20:25 +0000 (11:20 -0500)]
Dell FC driver inheritance order causing failures
After commit 610c672998c87e2bf2f919a4fdea86749c748a1f merged
the third party CI test runs for the Dell Storage Center fibre
channel driver started failing. This appeared to be a change
in get_volume_stats calls.
After isolating the cause of this problem it was identified
that the order of the inherited classes for the driver was
resulting in the wrong overridden get_volume_stats call to be
called.
This patch fixes this be just switching the order of the two
parent classes DellCommonDriver and FibreChannelDriver. Swapping
these two results in the correctly overridden call to be made
and CI tests to pass successfully.
This patch includes the Cinder changes needed
to support volume multiple attaches. Nova and
python-cinderclient also need patches associated
to provide support for multiple attachments.
This adds the multiattach flag to volumes. When a
volume is created, a multiattach flag can be set,
which allows a volume to be attached to more than
one Nova instance or host. If the multiattach flag is
not set on a volume, it cannot be attached to more
than one Nova instance or host
Each volume attachment is tracked in a
new volume_attachment table. The attachment id is
the unique identifier for each attachment to an
instance or host.
When a volume is to be detached the attachment
uuid must be passed in to the detach call in
order to determine which attachment should be
removed. Since a volume can be attached to an
instance and a host, the attachment id is used
as the attachment identifier.
Inspired by Dave Chen's change request on keystone about this, I
decided to check in cinder code to see if there were many cases.
Since unittest provides assertTrue and assertFalse, it is better
to use assertTrue(var) instead of assertEqual(True, var) and the same
thing applies to assertFalse.
I used these commands to search the occurrences:
* grep -PRIn "assertEqual\(True"
* grep -PRIn "assertEqual\(False"
* grep -PRIn "assertEqual\(\w+\s*,\s*True"
* grep -PRIn "assertEqual\(\w+\s*,\s*False"
Mike Perez [Mon, 9 Mar 2015 21:38:50 +0000 (14:38 -0700)]
Update Datera's export to avoid deprecated keys
The export call has been updated in Datera. The old location for the IQN
has moved. This updates the Kilo driver with support of looking in the
new location. The Juno driver will remain using the old location.
Erlon R. Cruz [Mon, 12 Jan 2015 18:59:10 +0000 (16:59 -0200)]
Adds pool aware scheduling for HNAS drivers
This commit adds the pool aware feature for HNAS drivers. Now, each service
label in HNAS XML config will be considered as a pool unit. In order to be able
to cast a volume to a service, the admin must create 'volume_type's and each
volume type need to have the 'service_label' with the same tag set in the XML
config, and then the user can create a volume with the desired volume_type.
Patrick East [Fri, 13 Feb 2015 21:35:51 +0000 (13:35 -0800)]
PureISCSIDriver consistency group updates.
This will enable the driver to add and remove volumes from Purity
protection groups that correspond to Cinders consistency groups, as well
as create a consistency group from a cgsnapshot.
Mike Perez [Sat, 7 Mar 2015 01:24:00 +0000 (17:24 -0800)]
Don't fail target_delete if ACL's don't exist
Seen in the gate a bit where a target delete is called and fails because
the ACL's don't exist for that target. If we're deleting a target and
the ACL's don't exist, it's probably safe to move on.
This patch adds a check for this specific case, logs a warning, and
continues rather than raising an exception.
Glenn M. Gobeli [Mon, 23 Feb 2015 12:35:34 +0000 (04:35 -0800)]
Add Manage/Unmanage support to NetApp NFS drivers
This change adds support for the Cinder Manage and Unmanage API calls
to the NetApp NFS drivers. This does not add the support for the generic
NFS driver.
Thang Pham [Thu, 5 Mar 2015 23:18:33 +0000 (18:18 -0500)]
Use snapshot object in create_volume flow
create_volume flow was making a call to db.snapshot_get to get
a snapshot dict and passing it onto drivers in
create_volume_from_snapshot() instead of an object. This causes
a conflict for some drivers, since create_snapshot and
create_volume_from_snapshot methods have two different snapshot
types (object vs. dict). The following patch fixes this by using
objects in create_volume flow and allows drivers to be updated
appropriately to use snapshot objects.
The strings can be got from the [DEFAULT] section or a driver instance group
section.
If the string can not be found in the cinder.conf, it will try to find a
default filter_fuction or a default goodness_function which can be specified
in the driver code. HP3par drivers don't provide default values. However if
other driver want to provide default values, it can be done by overwriting
get_default_filter_function and get_default_goodness_function.
2. Also added goodness_function and filter_function to the
_update_volume_stats in ISCSIDriver, FibreChannelDriver, FakeISCSIDriver
and LVMVolumeDriver.
3. Added 2 stats total_volumes and capacity_utilization so they can be
used as the part of the formula in filter_function or goodness_function
string.
4. Fixed free_capacity for limited cpg to include SDUsage's usedMiB as part of
the calcuation so it is consistent with unlimited cpg and capacity_utilization
looks better.
5. Added total_volumes capability to the LVM driver.
Xing Yang [Mon, 12 Jan 2015 19:16:28 +0000 (14:16 -0500)]
Add support to incremental backups in cinder
1. Added a new option to create backup to support incremental backups.
New CLI option added for "cinder backup": --incremental or --incr
By default, it is full backup.
Add the following to API: 'incremental': True or False
(False is the default)
2. swift.py creates sha256file for every backup to calculate deltas
Erlon Cruz [Thu, 12 Jun 2014 10:55:38 +0000 (07:55 -0300)]
Adds SSH communication to HNAS drivers
This commit adds SSH communication to HNAS iSCSI and NFS drivers. The user now
has the option to export the server public keys to HNAS and configure the
authentication via public keys. This avoid exposing HNAS passwords in the plain text
config file and exposing the private HNAS internal network.
Patrick East [Fri, 20 Feb 2015 02:49:23 +0000 (18:49 -0800)]
Add CHAP support to PureISCSIDriver
This change adds support for iSCSI CHAP with randomly generated
secrets for each initiator. Due to some restrictions with storing meta
data in the backend these secrets need to be stored someplace accessible
from the driver. We will use the driver specific initiator data in the
Cinder DB to store them.
John Griffith [Tue, 3 Mar 2015 21:27:27 +0000 (21:27 +0000)]
Make objects behave more like our old dictionaries
Prior to Cinder Objects we passed a dictionary representation
of a Cinder resource around. There are a few places where we
abuse this a bit and do things like "ref_dict.get('some-key', None)"
to reuse some code and determine if for example the object is
a snapshot or a volume (based on it's keys). This is probably
a pretty lazy/bad idea, but we have cases where we do it,
such as in the SolidFire driver and it looks like maybe some
calls in the profiler.
For now, we just make the compatibility shim in Object base truly
compatible and return None for non-existent attributes rather
than raising. Follow up work would be to clean up all of our
usage to strictly use objects and get rid of the compatibility
shim altogether.
nikeshmahalka [Wed, 4 Mar 2015 09:04:42 +0000 (14:34 +0530)]
Two choices for iscsi_helper is missing
In the cinder/volume/driver.py, scstadmin and ietadm were missing from
the choices for iscsi_helper. This will cause SCST and IET target helper
functionality to not work properly.
Mike Perez [Sat, 28 Feb 2015 07:48:46 +0000 (23:48 -0800)]
Update Datera's Authentication method
Previous auth-token/client verification auth method is deprecated. It's
supported until Liberty release. This also supports the new method,
which is using san_login and san_password, which will fetch an
auth-token to use.
Tom Swanson [Wed, 4 Mar 2015 19:56:13 +0000 (13:56 -0600)]
Dell SC API change fails snapshot creation
The Dell Storage Center driver is failing snapshot creation on
later versions of the Dell REST API.
The doNotExpire option was removed in API 15.3.1.110. We were
setting this so that the replays are not expired and removed when
Cinder expects them to still be there. This has been replaced by
setting expiretime to 0. We were doing this already so the fix
was to simply remove doNotExpire from the REST request payload.
This will fail on every future release of the Dell Storage Center
so this is a critical fix.
Jeegn Chen [Wed, 4 Mar 2015 13:10:34 +0000 (21:10 +0800)]
More validation logic for VNX CG creation
In VNX array, compressed LUNs cannot be added into a Consistency Group.
VNX Cinder Driver should check the volume types.
If storagetype:provisioning=compressed is specified,
VNX Cinder Driver should reject the CG creation operation.
When parsing group['volume_type_id'], the trailing comma should be
considered.
Patrick East [Fri, 22 Aug 2014 23:43:20 +0000 (16:43 -0700)]
Add DB table for driver specific data
This will allow drivers to store key/value pairs in the database for any
needs they have requiring persisted storage for initiator information
across cinder service hosts.
Jay S. Bryant [Tue, 3 Mar 2015 03:31:46 +0000 (21:31 -0600)]
Move oslo.messaging to the oslo_messaging namespace
This is the sixth in a series of changes to move to using
the new oslo_<library> namespace that is being used for
oslo libraries.
There is currently a shim in place that is allowing the old
oslo.<library> imports to work, but we need to be prepared for
when the shims go away. Thus we need patches like this one to
move to the new namespace.
This patch also updates our hacking check to ensure that no
instances of oslo.messaging sneak back in.
Xing Yang [Tue, 24 Feb 2015 02:38:14 +0000 (21:38 -0500)]
Create Consistency Group from CG Snapshot API
This patch addressed the following:
* Added a new create Consistency Group from CG Snapshot API.
- Note this is separate from the Create Consistency Group
API which requires volume types as the input.
* Added a corresponding driver API.
Xing Yang [Wed, 31 Dec 2014 02:27:56 +0000 (21:27 -0500)]
Modify Consistency Group API
This patch addressed the following:
* Modify Consistency Group
* Added an API that supports adding existing volumes to CG and removing
volumes from CG after it is created. It also allows the name and the
description to be modified.
* Added a volume driver API accordingly.
Guan Qiang [Mon, 9 Feb 2015 06:18:35 +0000 (14:18 +0800)]
Remove useless storage_availability_zone import
Config 'storage_availability_zone' have been removed from cinder.volume.manager
long time ago, but Still been imported by cinder.volume.api and
cinder.consistencygroup.api.
Tomoki Sekiyama [Tue, 9 Dec 2014 22:09:43 +0000 (17:09 -0500)]
Failover to alternative iSCSI portals on login failure
When the main iSCSI portal is unreachable by network failure etc.,
volume attach will fail even if the array has the other portal
addresses alive. To enable nova-compute and brick (with corresponding
patches) to fall-back to alternative portals, this patch makes Cinder
to contains the multiple portal addresses and corresponding IQNs and
LUNs in connection_info returned by initialize_connection API.
The main portal information is also returned in target_portal,
target_iqn, target_lun for backward compatibility. For example:
This patch makes LVM iSCSI backend return multiple portals,
iqns, and luns when iscsi_secondary_ip_addresses is specified in
cinder.conf.
It also make brick/initiator/connector.py to failover to alternative
paths, if specified, when it fails to establish the main path.
In addition, to unify the data structure, target_portal, target_iqn,
target_lun will also be included even if 'multipath=True' is specified
in connector information.
Code Review / openstack-build / cinder-build.git / log