Julia Varlamova [Thu, 27 Feb 2014 14:31:29 +0000 (18:31 +0400)]
Replace tearDown with addCleanup - Part 5
Replace tearDown with addCleanup in:
- cinder/test.py
- cinder/tests/test_migrations.py
Remove empty tearDown methods from:
- cinder/tests/test_hp3par.py
- cinder/tests/test_hplefthand.py
- cinder/tests/test_ibmnas.py
since it does not add any functionality (and empty setUp methods as well).
Infra team has indicated that tearDown should not be used and should
be replaced with addCleanup in all places.
Li Min Liu [Tue, 8 Apr 2014 02:18:56 +0000 (10:18 +0800)]
Storwize/SVC driver crashes when check volume copy status
Storwize/SVC driver does not clear up property of _vdiskcopyops
when deleting a volume. So if a volume which under async copy is
deleted before sync completed, the driver will crash.
When delete a volume, add a check for _vdiskcopyops, if volume
is in the dict, remove it.
The volume manager should call driver.create_export()
and driver.remove_export() with an elevated context.
This is already done for remove_export() in some cases
but not in initialize_connection error paths, or for
terminate_connection.
This will at a minimum cause issues with the LVM LIO
driver as its create/remove_export methods uses database
queries requiring admin access (volume_get_iscsi_target_num,
iscsi_target_count_by_host).
John Griffith [Thu, 10 Apr 2014 00:10:08 +0000 (00:10 +0000)]
Check for silent failure of tgtadm remove
In order to work around a failure to remove targets that
had a reconnect the force flag was added to the tgt delete cmd.
(https://bugs.launchpad.net/cinder/+bug/1159948)
It turns out there's a bug in tgt where some versions will
sometimes silently fail when using the force flag.
(https://bugs.launchpad.net/ubuntu/+source/tgt/+bug/1305343)
The problem is that in the gates since we merged the force change
there's a very high number of cases where lvremove fails because
the device is still active. It appears that this is a result of
the silent force failures in target.
This patch adds a simple check after the force target removal,
if it detects the target is still present it reverts back to the old
non-force method to catch the cases that don't actually need the
force option.
This is a work-around until the version of target in the distros
is updated and can be used reliably, but closes the gate issue
bug that was reported.
GlusterFS: Delete active snapshot file on volume delete.
If a snapshot is taken of a volume that is attached to an active
instance, the volume file used by the instance will be switched to
the new snapshot file that is created. When you delete the
snapshot, the base volume file will be merged with the snapshot
file and the base volume is deleted. Upon a deleting the active
volume, the active snapshot file is not deleted because it does not
have the expected name that cinder is looking for, i.e.
volume-<uuid>. Instead, the snapshot file has the name
volume-<uuid>.<snapshot-uuid>. This patch looks at the volume info
file to find any active snapshot file and properly delete it when
the volume is deleted.
Eric Harney [Tue, 8 Apr 2014 17:55:25 +0000 (13:55 -0400)]
Log initialize_connection error before remove_export
The way this code is currently structured, if
self.driver.remove_export() raises an exception, you see the
error message logged for that but the original exception
from initialize_connection() never gets logged.
Since force delete volume apis are only admin apis, force detach volume api
should also be an admin only api. Currently, the force detach api,
which uses the default rule in policy.json, can be called by admins and owners.
This patch make force detach volume api an admin only api like force
delete volume.
Li Min Liu [Tue, 1 Apr 2014 09:08:37 +0000 (17:08 +0800)]
Change iogrp property when retyping for Storwize/SVC
If need migration when retyping, Storwize/SVC driver will add a copy
of the volume first, and then start a task to check volume sync
progress which will run each 10 mins. At last delete the origin copy
when sync finished.
But iogrp property is not a parameter of addvdiskcopy, so the iogrp
of the copy volume is the same as the origin. If iogrp property is
diff between the old and new volume type, the property will not
change when retyped.
Mike Perez [Thu, 3 Apr 2014 00:52:11 +0000 (17:52 -0700)]
Allow deprecated volume update keys in v2
Keys like display_name and display_description were deprecated for other
actions in v2 like creating volumes, so for consistency they should work
with updating.
Chris Buccella [Thu, 3 Apr 2014 04:02:45 +0000 (04:02 +0000)]
_translate_from_glance() can cause an unnecessary HTTP request
After returning from a get() call to python-glanceclient, cinder runs a
translation function on the returned Image to get the data it wants. Part of
this process is checking for an expected set of attributes, one of which is
the deletion time ('deleted_at'). However, if the image has not been deleted,
deleted_at key will not exist. This forces another call to glance to occur for
the same image. A similar problem exists for the checksum attribute, which does
not exist before an image is active. The fix here is to only consider
deleted_at and checksum if they are expected to be present.
Bryan D. Payne [Mon, 31 Mar 2014 22:50:43 +0000 (15:50 -0700)]
Adds ionice command permutations to rootwrap filters
When setting volume_clear_ionice, the ionice command is prepended to a dd
command before executing it. Previously, this would have failed because it
wasn't allowed through the default rootwrap filters. This commit fixes this
problem.
Note that due to the way that the RegExpFilter works, we need three different
expressions to cover all cases. This is because the various options result
in a different number of arguments. And this filter will fail immediately
if the number of arguments is incorrect.
Sasikanth [Thu, 27 Mar 2014 15:14:50 +0000 (20:44 +0530)]
Append nas_opts to IBMNAS_NFSDriver configuration
'nas_opts' were added in the NFS Driver (nfs.py), but were not appended
to the IBMNAS driver configuration, causing failure with the multi-backend
configuration of driver.
Steven Kaufer [Mon, 31 Mar 2014 20:32:39 +0000 (20:32 +0000)]
GET details REST API next link missing 'details'
When executing a pagination query a "next" link is included in the
API reply when there are more items then the specified limit.
See pagination documentation for more information:
http://docs.openstack.org/api/openstack-compute/2/content/
Paginated_Collections-d1e664.html
The caller should be able to invoke the "next" link (without
having to re-format it) in order to get the next page of data.
The documentation states "Subsequent links will honor the
initial page size. Thus, a client may follow links to traverse
a paginated collection without having to input the marker parameter."
The problem is that the "next" link is always scoped to the non-
detailed query.
For example, if you execute "/v2/<tenant>/volumes/detail?limit=1",
the "next" link does not have the URL for a detailed query and is
formatted as "/v2/<tenant>/volumes?limit=1&marker=<marker>". In this
case the "next" link needs to be scoped to "/v2/<tenant>/volumes/detail".
The user could work around this issue my manually inserting '/details'
into the "next" link URL.
Test code is included to verify that the '/details' URL is correctly added
when the "next" link is included in a detailed pagination query. Also,
existing tests were changed to ensure that the correct controller function
(ie, 'index' vs. 'detail) are invoked for the appropriate query
(ie, non-detailed or detailed) -- 'index' was previously alwayed invoked
for detailed URL requests.
Thang Pham [Mon, 31 Mar 2014 19:22:47 +0000 (15:22 -0400)]
GlusterFS: Delete volume-<uuid>.info file when volume is deleted.
glusterfs creates a file named volume-<uuid>.info to hold volume
data, mapping cinder snapshot UUIDs and filenames. When the
volume and its associated snapshots are both deleted, this file
remains. This patch deletes the volume-<uuid>.info (if one exists)
when the volume is deleted.
Andreas Jaeger [Sat, 29 Mar 2014 05:38:52 +0000 (06:38 +0100)]
Fix Jenkins translation jobs
The jobs cinder-propose-translation-update and
cinder-upstream-translation-update do not update from
transifex since our po files contain duplicate entries where
obsolete entries duplicate normal entries.
Lucian Petrut [Fri, 28 Mar 2014 08:47:07 +0000 (10:47 +0200)]
Fixes cinder volume delete on Windows
Because of the fact that an iSCSI target is not created anymore
at volume creation time, not all volumes will have a corresponding
target. For this reason, when removing an iSCSI target we must
first check if the target actually exists.
Lucian Petrut [Fri, 28 Mar 2014 08:57:13 +0000 (10:57 +0200)]
Fixes cinder volume attach on Windows
The initiator name and the target name are inverted when passing
the arguments to the method which associates the iSCSI target to
an initiator. For this reason, this operation will fail.
Also, the connection to the iSCSI target cannot be initialized
properly as the method which gets portal information is missing
the return value.
Thierry Carrez [Thu, 27 Mar 2014 10:14:52 +0000 (11:14 +0100)]
Open Juno development
Bump preversion to 2014.2 to formally open Juno development.
The Icehouse release branch (milestone-proposed) will be cut
from the last commit before this one.
Glenn M. Gobeli [Fri, 21 Mar 2014 14:05:07 +0000 (10:05 -0400)]
Changes to correct name of missing NetApp license.
Change to have the NetApp ZAPI error of "SIS clone not licensed"
refer to the correct NetApp license of "FlexClone not licensed".
This code path is encountered when the NetApp 7mode driver uses
the clone interface for snapshotting or cloning.
Alex Meade [Thu, 20 Mar 2014 19:44:44 +0000 (15:44 -0400)]
NetApp cmode nfs: Fix QOS extra spec
This patch fixes the incorrect behavior where the NetApp cmode
nfs driver will choose to create a volume on a flexVol that has
a QOS policy group that matches the specified QOS policy group
intended for the cinder volume. The correct behavior is to ignore
the QOS policy group of the flexVol and instead assign the QOS policy
group to the newly created cinder volume.
Alex Meade [Wed, 19 Mar 2014 18:03:15 +0000 (14:03 -0400)]
NetApp cmode iscsi: Fix QOS extra spec
This patch fixes the incorrect behavior where the NetApp cmode
iscsi driver will choose to create a volume on a flexVol that has
a QOS policy group that matches the specified QOS policy group
intended for the cinder volume. The correct behavior is to ignore
the QOS policy group of the flexVol and instead assign the QOS policy
group to the newly created cinder volume.
Xing Yang [Sat, 22 Mar 2014 19:05:42 +0000 (15:05 -0400)]
Fixes a problem in attach volume in EMC driver.
This patch fixes a problem in attach volume in EMC SMI-S driver.
The existing logic checks if a volume is already attached to any host,
but it doesn't check whether a volume is already attached to the specific
host that nova wants cinder to attach. As a result, initialize_connection
could return success (thinking it is already attached), but nova will
fail to discover the LUN later and fail the attach.
This patch adds a check to see if a volume is already attached to a
specific host. If not, it will do the attach. The reason that the
volume being attached already could be due to a nova live-migration
use case. Cinder doesn't support multiple attaches currently, but
allows a volume to be attached multiple times from nova during
live-migration.
john-griffith [Wed, 26 Mar 2014 22:47:21 +0000 (16:47 -0600)]
Update config generator from OSLO
Cinders sample config is out of date with recent
updates needed for keystone, however we had an
old version of the oslo generator so there was
no way to pick these changes up locally by running
the generate tool.
This pulls the latest generator.py from oslo, and
we run the update script to get us back in sync.
Chris Buccella [Wed, 26 Mar 2014 19:48:16 +0000 (19:48 +0000)]
Import request_id middleware bug fix from oslo
There is a bug in request-id middleware that a subsequent API request
will overwrite a request-id of a previous request when multiple API
calls are processed in parallel in request_id middleware. The fix is
a drop-in replacement; cinder code does not need to be modified to
use the updated request_id middleware.
Jeff Applewhite [Mon, 24 Mar 2014 21:37:22 +0000 (17:37 -0400)]
Netapp iscsi: allow snapshots with unspecified block range.
Snapshots were failing to create in all cases in the NetApp iscsi
driver. The clone lun function was never issuing the zapi call
because the block count was set to 0.
An API invocation with a stale session returns an empty response. In order
to distinguish it from an API returning valid empty response, the session
is recreated and the API is retried. If an empty response is received even
after the retry, it can be assumed that the API response is actually empty.
But this behavior results in authentication error from the VMware server
when there is an active session and the API response is actually empty.
This change fix this behavior by using the SessionIsActive check to
identify stale session.
wanghong [Thu, 20 Mar 2014 02:39:37 +0000 (10:39 +0800)]
init_host should be called before RPC consumer is created
Currently, the init_host method is called after RPC consumer is
created. This behavior will lead to a bug that when a rpc request
is received the manager can not handle it because the driver has not
been initialized!
john-griffith [Wed, 19 Mar 2014 22:19:28 +0000 (16:19 -0600)]
Add RequestContextSerializer for rpc notifications
RequestContext should be serialized when sent via oslo.messaging. The
serializer is correctly used for the general RPC mechanism, but has been
forgotten in the notifier. This patch fixes that.
Thanks goes to eharney for noticing this and pointing it out!!!
Andrew Kerr [Tue, 11 Mar 2014 14:28:55 +0000 (10:28 -0400)]
Allow NetApp iSCSI driver to sub-clone large volumes
The NetApp zapi used during certain extend operations has several limits
imposed on it. Each block-range provided can only be 2^24 in size, and
there can only be 32 block-ranges per zapi call. This fix allows the
NetApp iSCSI driver to send multiple zapi calls if necessary, to allow
for extend operations on volumes of an arbitrary size.
lirenke [Tue, 4 Mar 2014 12:59:07 +0000 (04:59 -0800)]
Can't force-create snapshot by an non-exist error volume
If we create an LVM volume whose size larger than
rest space in VG, the volume would not exist with
error status.
But then we still can force create a snapshot
by this volume, and the snapshot's status is available.
In the code,I found in create_lv_snapshot,
it return False when fail to get the volume.
However, raising an exception is only way to be
handle outside.
So, we should raise exception instead of return False.
Mike Perez [Mon, 17 Mar 2014 05:24:38 +0000 (22:24 -0700)]
Simplify test force delete snapshot unit test
Make this test simple in just verifying if we have a snapshot and we do
a force delete that we get back an ok response from the controller.
Removing out RPC call and verifying snapshot was deleted since that's
out of the scope here.
Xiao Chen [Thu, 6 Mar 2014 10:13:46 +0000 (18:13 +0800)]
resolve KeyError for IBM Storwize/SVC driver
When using iSCSI protocol to conenct IBM v7000, and compression
is not enabled for the system, KeyError may occur because of the response
item's lack of 'license_compression_enclosures'. So add a check to
resolve it.
The pbm_default_policy config option is used to associate a storage
policy with the volume backing if there is no associated volume_type
or the volume_type doesn't have the storage_profile option. A better
approach is to set the storage_profile option in the default_volume_type.
This change removes the pbm_default_policy and queries the policy
using the storage_profile option in the given volume_type.
Avishay Traeger [Wed, 26 Feb 2014 09:27:06 +0000 (11:27 +0200)]
Fix create_export/remove_export in driver.py
1. There was a call to rpcapi.create_export which does not have a
matching volume manager function, causing volume migration to crash.
This call was not necessary because there is already an
initialize_connection RPC call, which calls the driver's create_export
function. Removed the create_export RPC call and function. Added better
error handling to that code in _attach_volume in driver.py as well.
2. The manager called remove_export from detach_volume, which was not
being called by these functions. I believe it makes more sense to call
it from terminate_connection. Moved it there, and fixed up the
corresponding code in _detach_volume in driver.py.
3. Remove leftovers of export code from create_volume manager flow.
Code Review / openstack-build / cinder-build.git / log