Bob-OpenStack [Mon, 19 Jan 2015 01:05:14 +0000 (17:05 -0800)]
Huawei driver check before associating LUN to a LUN group
This commit fixes the following problems:
* Change in huawei_t.py removes the duplicated wwns
returned from the array.
* Change in rest_common.py adds log print for response
data from the array.
* Change in rest_common.py does a check before associating
a LUN to a LUN group.
* Change in rest_common.py fixes the REST request.
* Change in ssh_common.py checks whether the hostlunid
have already been assigned.
Mitsuhiro Tanino [Thu, 26 Feb 2015 00:16:39 +0000 (19:16 -0500)]
Implement IET target driver
When introducing Change-Id: I43190d1dac33748fe55fa00f260f32ab209be656,
IET driver was not implemented. This patch adds IET target for new
iscsi target driver model.
Part of the process of accepting the Cinder multiattach
patch was to create a follow up patch that fixes some of
the inconsistencies with order of parameters in the new
unit tests. This patch fixes those assertX calls and
parameter ordering to be in the style of:
expected, actual
Xing Yang [Fri, 20 Feb 2015 06:38:09 +0000 (01:38 -0500)]
Fixed a concurrency issue in VMAX driver
This patch fixed the following problem:
When trying to add a second volume to the same masking view,
the first volume got removed at the same time, causing
the operation on the second volume to fail.
When two attach requests happen at the same time on the same
volume, the second one will fail.
Also fixed a W503 pep8 issue (line break before binary operator)
in emc_vmax_common.py.
Duncan Thomas [Tue, 13 Jan 2015 16:41:13 +0000 (18:41 +0200)]
Fix LVM thin pool creation race
In the event that two copied of the LVM driver get init called at
the same time (e.g. cinder-volume and cinder-backup getting
started in parallel, on the same host), it is possible for the
thin pool check/create to race. Add a simple recheck if the create
fails, to cover this window.
peter_wang [Fri, 13 Mar 2015 06:27:17 +0000 (02:27 -0400)]
Fix ArgsAlreadyParsedError in emc_vnx_cli.py
This error was introduced after adding following:
logging.register_options(CONF)
in https://review.openstack.org/#/c/157441/
So, this patchset is to remove this.
Jay S. Bryant [Wed, 11 Mar 2015 01:15:00 +0000 (20:15 -0500)]
Move to the oslo.middleware library
This patch moves Cinder to using olso.middleware, updates us so
we are using the oslo_middleware namespace and syncs the latest
middleware code from oslo-incubator to support grenade jobs.
The details for the middleware sync from oslo-incubator are as follows:
Current HEAD in OSLO:
---------------------
commit e589dde0721a0a67e4030813e582afec6e70d042
Date: Wed Feb 18 03:08:12 2015 +0000
Merge "Have a little fun with release notes"
Changes merged with this patch:
---------------------
__init__.py 4ffc4c87 - Add middleware.request_id shim for Kilo 4504e4f4 - Remove middleware
catch_errors.py a01a8527 - Use oslo_middleware instead of deprecated oslo.middleware ce8f8fa4 - Add middleware.catch_errors shim for Kilo 4504e4f4 - Remove middleware 5d40e143 - Remove code that moved to oslo.i18n 76183592 - add deprecation note to middleware 463e6916 - remove oslo log from middleware fcf517d7 - Update oslo log messages with translation domains
request_id.py a01a8527 - Use oslo_middleware instead of deprecated oslo.middleware 66d8d613 - Fix oslo.middleware deprecation error 4ffc4c87 - Add middleware.request_id shim for Kilo 4504e4f4 - Remove middleware 76183592 - add deprecation note to middleware d7bd9dc3 - Don't store the request ID value in middleware as class variable
Some notes on this change. It is based on the change made in Nova:
https://review.openstack.org/#/c/130771 and is the recommended method
for cleaning up the unused portions of middleware from oslo-incubator,
moving to the oslo.middleware library and not breaking grenade in the
gate.
Sean McGinnis [Tue, 24 Feb 2015 15:24:52 +0000 (09:24 -0600)]
Remove use of contextlib.nested
The contextlib.nested call has been deprecated
in Python 2.7. This causes DeprecationWarning
messages in the unit tests.
There are also known issues with contextlib.nested
that were addressed by the native support for
multiple "with" variables. For instance, if the
first object is created but the second one throws
an exception, the first object's __exit__ is never
called.
Since Cinder no longer supports 2.6 we can remove
the use of these contextlib.nested calls.
Added hacking check to catch if any new instances
are added to the codebase.
Note: line continuation markers (e.g. '\') had to
be used or syntax errors were thrown. While using
parentheses is the preferred way for multiple line
statements it is not a requirement.
Jay S. Bryant [Thu, 12 Mar 2015 18:34:16 +0000 (13:34 -0500)]
Remove strutils from oslo-incubator
This change goes along with the work to get away from
oslo-incubator modules to using the new oslo libraries.
The strutils module moved to the oslo_utils library and
should no longer be used from oslo-incubator.
Additionally, merging this change removes a dependency on
gettextutils, allowing us to remove another deprecated
module.
Michal Dulko [Thu, 12 Mar 2015 16:24:09 +0000 (17:24 +0100)]
Add waiting for the driver to SchedulerManager
This patch adds _wait_for_scheduler method before serving any request.
Method waits till scheduler.is_ready() returns true or
CONF.periodic_interval seconds passed from service startup.
Jay S. Bryant [Thu, 12 Mar 2015 15:06:23 +0000 (10:06 -0500)]
Update hacking check for oslo_log
With commit 5980da5d we moved Cinder to using the olso.log
library. We also moved to the new oslo_log namespace. With that
change we should have also updated the hacking checks to include
looking for old uses of oslo.log.
Michal Dulko [Thu, 12 Mar 2015 15:07:03 +0000 (16:07 +0100)]
Add is_ready method to scheduler driver
This commit adds is_ready method to scheduler driver to indicate
if driver is ready to accept requests. Scheduler is considered ready
when host_manager received capabilities for all active volume services.
Vincent Hou [Wed, 29 Oct 2014 08:40:31 +0000 (01:40 -0700)]
Remove the useless next link for volumes, transfers and backups
Two cases this patch will resolve:
1) Currently if the number of items equals the osapi_max_limit or
the last page of items equals the osapi_max_limit without the
parameter limit set in the user request, a next link is generated
in the response, though this next link will return empty volume
list. In fact it is unnecessary to generate the next link in
this case.
2) If the number of items equals the osapi_max_limit and limit is
greater than osapi_max_limit, a next link is generated. Actually,
the next link does not need to be generated, because it is certain
that there is no more volumes left in the database.
The method _get_collection_links has been called in volumes,
volume_transfers and backups. The patch can only affect the next
link generation for three of them. However, other lists like
consistency groups, qos specs, cgsnapshots have not implemented the
generation for the next link. Potentially this can be a wishlist
item for them.
Mike Perez [Wed, 11 Mar 2015 19:27:32 +0000 (12:27 -0700)]
Unset auth token before trying to login to Datera
After a token expires, the Datera driver would try to login, and pass
along the expired token. This would fail because the receiving server
will authorize anytime it's given an auth token, even for endpoints that
don't require it. This change unsets the auth token when logging in, so
it's not passed a long.
Tom Barron [Mon, 19 Jan 2015 17:58:02 +0000 (12:58 -0500)]
NFS backup driver
Implements an NFS backup service in accord with the NFS backup driver
blueprint spec (https://review.openstack.org/#/c/130858).
Currently the NFS backup driver inherits directly from the abstract
chunked backup driver. It should be straightforward to abstract out
of this code a posix filesystem backup driver, without dynamic mount
capabilities, which itself inherits from the chunked backup driver.
When that is done, most of the code in this implementation can be
removed and it can be modified to inherit from the posix filesystem
backup driver itself.
This patch is a follow up to the multiattach patch that adds
the ability to the scheduler to be able to filter capabilities
on the volume multiattach flag. When a volume creation request
comes in with the multiattach flag set on a volume, there was no
way to filter out backends that don't support attaching a volume
to more than one host. By defaut, no host supports it. When
The Nova side lands, we can adjust drivers to add the new
multiattach:True capability reporting.
Eric Harney [Tue, 10 Mar 2015 20:34:16 +0000 (16:34 -0400)]
Tests: Harden fake_notifier asserts
This is a continuation of 8740e063 - Make test_create_delete_snapshot more robust
The same issue cropped up for me in
test_create_consistencygroup_from_src().
Add some more specific asserts to try to reveal what
is going wrong here. The idea is to assert more
specific things (examining individual notifications)
before more general asserts (count of notifications) when
possible, to fail the test on the check that will provide
the most information back.
marcusvrn [Thu, 5 Mar 2015 18:52:54 +0000 (15:52 -0300)]
Error trying to delete snapshots on Hitachi driver
Patch #133566 (Change-Id: I387018e80c8539565e99454db65d976030002c0f) partially
changed Volumes and Volume Snapshots to objects and modified their attributes.
This change broke HBSD driver, once the 'snapshot_metadata' attribute was
replaced to 'metadata'.
Michal Dulko [Wed, 11 Mar 2015 09:36:38 +0000 (10:36 +0100)]
Remove global mocking from test_pure.py
Tests for pure driver are patching cinder.utils.retry globally. All
tests running after test_pure got imported have retry decorator already
mocked. This creates problems when writing new unit tests involving the
decorator. This change unpatches cinder.utils.retry after test_pure
imports pure driver.
Zhiteng Huang [Tue, 24 Feb 2015 09:05:46 +0000 (17:05 +0800)]
Allow scheduler to receive volume stats when starting service
Filter scheduler relies on volume services to report their stats in
order to make decisions - this nature makes scheduler service a bit
special when it starts because it can't really do anything without
knowning stats of avilable volume backend. So the original
implementation of Filter Scheduler added a hook in init_host() to
ask all volume services to send their stats immediately (volume RPC
call 'publish_service_capabitlies()'), this reduced the chance of a
new volume being set to 'error' state because scheduler has no
knowledge of backends to minimum.
However, commit 65fa80c361f71158cc492dfc520dc4a63ccfa419 moved
init_host() ahead of service RPC initialization because for volume
services, service initialization should be done before the service
starts accepting RPC requests. This change unfortunately invalidated
scheduler service's init_host() because that needs to be done *AFTER*
RPC is ready (a queue is bound to scheduler fanout exchange). The
result is scheduler asks vol service to send update, but it misses
the information it asked for because its queue hasn't been created
and bound to the fanout queue yet. So scheduler always has to wait
for the next update from volume services to be able to work properly.
This change adds a new hook to Manager class called init_hook_with_rpc()
to allow services like scheduler to do something once RPC is ready.
This should restore scheduler's behavior before commit 65fa80c361f71158cc492dfc520dc4a63ccfa419.
Stefan Amann [Mon, 26 Jan 2015 12:49:16 +0000 (13:49 +0100)]
Adjust Cinder to support FCP on System z systems
There are some platform specific changes needed to allow Cinder to
support FCP on System z: the System z specific format of the device file
paths, adding and removing devices explicitely, and ignoring vHBAs which
are offline.
Tom Barron [Thu, 15 Jan 2015 23:03:16 +0000 (18:03 -0500)]
Refactor Swift backup driver and introduce chunking driver
Abstract out a "chunked" backup driver class between the top
abstract backup driver class and the Swift backup driver class
which implements common functionality for backup drivers that
store volume data in multiple "chunks" in a backup repository
when the size of the backed up cinder volume exceeds the size
of a backup repository "chunk."
The Swift driver itself is reworked to extend the chunked
backup driver and implement abstract methods defined in the
chunked backup driver. It is expected that posix filesystem
and NFS backup drivers will also extend the chunked backup
driver.
Co-authors: Kevin Fox <kevin@efox.cc>
Tom Barron <tpb@dyncloud.net>
Jay S. Bryant [Fri, 20 Feb 2015 21:21:55 +0000 (15:21 -0600)]
Namespace updates for _i18n and imageutils & fileutils
The _i18n and imageutils modules only have changes
to move oslo.utils to the oslo_utils namespace and
oslo.i18n to oslo_i18n. So, I have lumped these
two small changes into this one patch.
Current HEAD in OSLO:
---------------------
commit e589dde0721a0a67e4030813e582afec6e70d042
Date: Wed Feb 18 03:08:12 2015 +0000
Merge "Have a little fun with release notes"
Changes merged with this patch:
---------------------
_i18n.py 0cc741a9 - switch to oslo_i18n
imageutils.py ac17de97 - Use oslo_utils instead of deprecated oslo.utils
fileutils.py ac17de97 - Use oslo_utils instead of deprecated oslo.utils
Sean McGinnis [Tue, 10 Mar 2015 16:20:25 +0000 (11:20 -0500)]
Dell FC driver inheritance order causing failures
After commit 610c672998c87e2bf2f919a4fdea86749c748a1f merged
the third party CI test runs for the Dell Storage Center fibre
channel driver started failing. This appeared to be a change
in get_volume_stats calls.
After isolating the cause of this problem it was identified
that the order of the inherited classes for the driver was
resulting in the wrong overridden get_volume_stats call to be
called.
This patch fixes this be just switching the order of the two
parent classes DellCommonDriver and FibreChannelDriver. Swapping
these two results in the correctly overridden call to be made
and CI tests to pass successfully.
This patch includes the Cinder changes needed
to support volume multiple attaches. Nova and
python-cinderclient also need patches associated
to provide support for multiple attachments.
This adds the multiattach flag to volumes. When a
volume is created, a multiattach flag can be set,
which allows a volume to be attached to more than
one Nova instance or host. If the multiattach flag is
not set on a volume, it cannot be attached to more
than one Nova instance or host
Each volume attachment is tracked in a
new volume_attachment table. The attachment id is
the unique identifier for each attachment to an
instance or host.
When a volume is to be detached the attachment
uuid must be passed in to the detach call in
order to determine which attachment should be
removed. Since a volume can be attached to an
instance and a host, the attachment id is used
as the attachment identifier.
Inspired by Dave Chen's change request on keystone about this, I
decided to check in cinder code to see if there were many cases.
Since unittest provides assertTrue and assertFalse, it is better
to use assertTrue(var) instead of assertEqual(True, var) and the same
thing applies to assertFalse.
I used these commands to search the occurrences:
* grep -PRIn "assertEqual\(True"
* grep -PRIn "assertEqual\(False"
* grep -PRIn "assertEqual\(\w+\s*,\s*True"
* grep -PRIn "assertEqual\(\w+\s*,\s*False"
Mike Perez [Mon, 9 Mar 2015 21:38:50 +0000 (14:38 -0700)]
Update Datera's export to avoid deprecated keys
The export call has been updated in Datera. The old location for the IQN
has moved. This updates the Kilo driver with support of looking in the
new location. The Juno driver will remain using the old location.
Erlon R. Cruz [Mon, 12 Jan 2015 18:59:10 +0000 (16:59 -0200)]
Adds pool aware scheduling for HNAS drivers
This commit adds the pool aware feature for HNAS drivers. Now, each service
label in HNAS XML config will be considered as a pool unit. In order to be able
to cast a volume to a service, the admin must create 'volume_type's and each
volume type need to have the 'service_label' with the same tag set in the XML
config, and then the user can create a volume with the desired volume_type.
Patrick East [Fri, 13 Feb 2015 21:35:51 +0000 (13:35 -0800)]
PureISCSIDriver consistency group updates.
This will enable the driver to add and remove volumes from Purity
protection groups that correspond to Cinders consistency groups, as well
as create a consistency group from a cgsnapshot.
Mike Perez [Sat, 7 Mar 2015 01:24:00 +0000 (17:24 -0800)]
Don't fail target_delete if ACL's don't exist
Seen in the gate a bit where a target delete is called and fails because
the ACL's don't exist for that target. If we're deleting a target and
the ACL's don't exist, it's probably safe to move on.
This patch adds a check for this specific case, logs a warning, and
continues rather than raising an exception.
Glenn M. Gobeli [Mon, 23 Feb 2015 12:35:34 +0000 (04:35 -0800)]
Add Manage/Unmanage support to NetApp NFS drivers
This change adds support for the Cinder Manage and Unmanage API calls
to the NetApp NFS drivers. This does not add the support for the generic
NFS driver.
Thang Pham [Thu, 5 Mar 2015 23:18:33 +0000 (18:18 -0500)]
Use snapshot object in create_volume flow
create_volume flow was making a call to db.snapshot_get to get
a snapshot dict and passing it onto drivers in
create_volume_from_snapshot() instead of an object. This causes
a conflict for some drivers, since create_snapshot and
create_volume_from_snapshot methods have two different snapshot
types (object vs. dict). The following patch fixes this by using
objects in create_volume flow and allows drivers to be updated
appropriately to use snapshot objects.
The strings can be got from the [DEFAULT] section or a driver instance group
section.
If the string can not be found in the cinder.conf, it will try to find a
default filter_fuction or a default goodness_function which can be specified
in the driver code. HP3par drivers don't provide default values. However if
other driver want to provide default values, it can be done by overwriting
get_default_filter_function and get_default_goodness_function.
2. Also added goodness_function and filter_function to the
_update_volume_stats in ISCSIDriver, FibreChannelDriver, FakeISCSIDriver
and LVMVolumeDriver.
3. Added 2 stats total_volumes and capacity_utilization so they can be
used as the part of the formula in filter_function or goodness_function
string.
4. Fixed free_capacity for limited cpg to include SDUsage's usedMiB as part of
the calcuation so it is consistent with unlimited cpg and capacity_utilization
looks better.
5. Added total_volumes capability to the LVM driver.
Xing Yang [Mon, 12 Jan 2015 19:16:28 +0000 (14:16 -0500)]
Add support to incremental backups in cinder
1. Added a new option to create backup to support incremental backups.
New CLI option added for "cinder backup": --incremental or --incr
By default, it is full backup.
Add the following to API: 'incremental': True or False
(False is the default)
2. swift.py creates sha256file for every backup to calculate deltas
Code Review / openstack-build / cinder-build.git / log