John Griffith [Fri, 20 Sep 2013 20:55:46 +0000 (20:55 +0000)]
Fix issues with failed lvremove
There are some race conditions that
can cause problems with lvremove commands. In
most cases these seem to recover nicely just
with a simple retry of the lvremove. Adding
a udev settle seems to elimate the rest of them.
This is a difficult issue to reproduce, and there's
a suspiscion that it relates to failed target
creeates.
The patch adds a catch on the lvremove failure,
followed by a udevadm settle and a retry of the
lvremove. With the setup that I've been able
to reproduce this issue these changes have eliminated
any force delete failures.
The other option that had been proposed was using dmsetup remove
but there are concerns that this may cause problems.
Eric Harney [Mon, 9 Sep 2013 22:47:00 +0000 (18:47 -0400)]
GlusterFS: Copy snap from correct source file
The GlusterFS driver's copy_volume_from_snapshot was previously
not parsing the snapshot information to determine which backing
file to copy from, instead copying from the active file at all
times. This would result in the wrong data being supplied when
a volume is cloned from a snapshot.
Add a test for the copy_volume_from_snapshot method.
Eric Harney [Sun, 8 Sep 2013 14:18:18 +0000 (10:18 -0400)]
GlusterFS: Use image_utils for qemu-img calls
Code added for GlusterFS snapshot support in Havana included
new code to parse qemu-img output. This removes this code
and uses image_utils qemu_img_info instead.
Add test to image_utils for parsing a different style of
qemu-img info output.
Improve GlusterFS driver test coverage with tests for cloning,
create from snapshot, and initialize_connection.
John Griffith [Fri, 20 Sep 2013 02:47:22 +0000 (20:47 -0600)]
Add a retry to create_iscsi_target for LVM
There's a bug in the tgt driver where under certain
conditions where a race condition is present that can
result in tgtadm attempting re-use a target ID.
Testing this it seems that a retry will address this nicely
and cause tgtd to grab a new target ID and things are ok.
This patch adds a simple check/retry mechanism around the
iscsi_target_create in the LVM driver to catch this issue and
deal with it.
There are a number of bugs that it appears can be attributed to
this, but this patch is associated with the most predominant
clear cut version.
Jim Branen [Fri, 20 Sep 2013 15:41:44 +0000 (08:41 -0700)]
Fix HP3PAR iSCSI path connection
The iSCSI paths are not correctly identified as ‘active’
when port information is requested from the hp3parclient
because the ‘active’ value is a Boolean not the string
‘true’.
Mike Perez [Tue, 17 Sep 2013 09:41:59 +0000 (02:41 -0700)]
Add volume metadata to v2
This exposes volume metadata in the Cinder v2 API. This already exists
in v1 and was originally suppose to exist back with Grizzly, but appears
to have slipped.
This updates the volume manager and the
backup manager to enforce that the driver
has been initialized.
The managers call the driver in many places
without ensuring that the driver has been
properly initialized. When the driver fails
inside of do_setup(), the managers shouldn't
call the driver's set_initialized() method.
The managers now dump out the exception, and
exit the init_host, leaving the driver in
an uninitialized state.
VolumeNotFoundForInstance class is used only in test_xiv_ds8k.py. I
think the exception class that is used only in test code should not be
defined in cinder/exception.py.
Luis A. Garcia [Fri, 6 Sep 2013 21:24:33 +0000 (21:24 +0000)]
Sync gettextutils from oslo
Some Messages, such as those created from Invalid exceptions, use a
Message within a Message, and we were only translating the base Message
but not the Message substitution within.
A mapping key of 'netloc' is always ignored. Thus information in
'netloc' is lost. But because it is included in 'error_msg', I remove
'netloc=netloc'.
A mapping key of 'volume_id' is ignored when we use the non keyword
argument. Thus information in 'volume_id' is lost. Similarly, in the
case of exception.SnapshotNotFound, information in 'snapshot_id' is
lost.
Previously the row for the temporary migration volume was deleted from
the database, which caused a foreign key violation for its admin
metadata. Regardless, the volume should have been soft deleted from the
database anyway, and this patch takes care of that.
Victor Rodionov [Tue, 17 Sep 2013 21:22:05 +0000 (01:22 +0400)]
Fix __init__ methods of brick initiator connectors
Wrong order of positional arguments in RemoteFsConnector __init__ call.
This only in RemoteFsConnector.__init__, but I decide also update
__init__ methods of other connectors to avoid errors in future.
This change modifies the behaviour of the secure delete for thick
LVM snapshots to wipe the underlying COW of the snapshot LV
instead of the snapshot LV itself.
This change is necessary because the snapshot LV does not contain
exactly the same number of writable blocks as the original LV. The
COW includes header information per COW block that identifies the
device as a COW device as well as the source and destination blocks
for the changed item. The amount of metadata contained in the COW is
variable based on I/O performed on the snapshot.
This does not change the behavior of secure deletes on thin LVs
or secure deletes on the thick LV snapshot origin.
Replace "params[params.index('-lun')]" with
"params[params.index('-lun') + 1]" to get lun ID correctly.
Replace the invalid controller ID "N" with "B".
Dict CLONED_LUN_INFO has no key provider_location, delete it.
Moreover, simplify the "if..., elif..." module with "getattr"
method in fake recv function.
According to the comment in the source code, the snapshot-update of
v2 would be able to allow either "name" or "display_name" parameter.
And if both paramters are specified, "name" parameter is effective
over "display_name" like "description" parameter.
However "display_name" parameter is ignored now, because cinder writes
over empty dict(update_dict['display_name']).
Jay S. Bryant [Mon, 16 Sep 2013 18:21:34 +0000 (13:21 -0500)]
Set rootwrap_config path to rootwrap.conf
Other components set
rootwrap_config = /etc/<component>/rootwrap.conf by default.
This commit brings cinder up-to-date with what other
components are doing by setting
rootwrap_config = /etc/cinder/rootwrap.conf by default.
A couple of test cases needed to be updated to handle the change
from 'None'.
Not counting this as a documentation impact as the documentation I
can find already lists /etc/cinder/rootwrap.conf as the default.
Eric Harney [Sat, 7 Sep 2013 03:26:46 +0000 (23:26 -0400)]
Do not use qemu-img --backing-chain or --output=json
Initial code for GlusterFS snapshot support in Havana
used qemu-img arguments which are too new to run in many
environments. Run qemu-img using only arguments which
will exist on older platforms.
VMware driver: Fix for invalid datastore selection
Added filter to ignore datastores that are not mounted, in-accessible
and read-only for volume creation. Without this VMware's VC/ESX driver can
choose such datastore with higher invalid capacity value as returned by the
server for volume creation.
Also using parent's self.mox object in the test cases rather than creating
a new mox.Mox obj everytime.
Fixes ceph volume restore to larger image than source
If a backup is restored to a volume that is bigger than the
backup itself, which could be the case if --volume-id is
supplied, we need the option to set extraneous extents to zero
i.e. set extents after those from the backup to zero.
Huawei driver failed to parse ssh result when firstly logging
in storage system on paramiko 1.8.0. The update of paramiko
make the order of ssh returns changed. This patch improves
the parsing way of ssh returns.
This patch also does some changes for log punctuations.
Fixes the use of exception.InvalidInput with the wrong arguments
A mapping key of 'message' in exception.InvalidInput is 'reason'.
cinder/volume/drivers/netapp/iscsi.py and
cinder/volume/drivers/netapp/utils.py had used 'data'.
Michael Kerrin [Mon, 9 Sep 2013 08:12:27 +0000 (08:12 +0000)]
Add default_availability_zone configuration option to cinder
This option is used to specify the default AZ for the whole service. So when
creating a new volume this value is used to specify the AZ of a volume.
Previously we were overloading the use of the storage_availability_zone option
which is also used to specify what AZ a service belongs to. We still fail back
on the storage_availability_zone when creating a new volume if the
default_availability_zone is not set.
Zhi Yan Liu [Tue, 3 Sep 2013 14:40:49 +0000 (22:40 +0800)]
Turn db model object into a primitive object to avoid error
In volume API controller we need turn db model object into a primitive
object to avoid error when it adding visible administrator metadata to
volume's metadata dict.
When the driver copy_image_to_volume fails it can
at the current time raise more than just CinderException
as its root exception type. This causes rescheduling due
to the blacklisted exception list that is used to determine
if a exception is 'bad enough' to trigger rescheduling or
should the volume creation action just set the volume to
error state.
To avoid the situation where this would cause a rescheduling
we should make sure (for now) that any exception that is
emitted on copying an image to a volume is translated
into a image copy failure and reraised.
Fixing UnicodeEncodeError against volume creating function
When creating a volume with non-English characters and there is
not enough space for it, then an UnicodeEncodeError occurs. So
we need convert any coded exception message to unicode string
for logging.
Previously deleting a key in certain qos specs was accomplished via
'update' API. Unfortunately, 'update' isn't able to tell the
difference between setting a key with no value and deleting a key (and
its value).
This change adds an new API 'delete_keys' to qos_specs API extension.
'delete_keys' API allows client to specify a list of to-be-deleted keys
in one single request (batch mode!), which can be handy when removing
multiple keys in a qos specs.
Example URL and request body for 'delete_keys' API:
PUT to http://127.0.0.1:8776/v2/qos-specs/QOS_SPECS_UUID/delete_keys
with body: {'keys': ['foo', 'bar', 'zoo']}
Above example request will result in 'foo', 'bar', 'zoo' key/value
pairs of QOS_SPECS_UUID be marked as deleted in DB. If QOS_SPECS_UUID
doesn't exist, a 404 error will return; if any key in 'foo', 'bar',
'zoo' couldn't be found in QOS_SPECS_UUID, a 400 error will return with
error message telling which key couldn't be found. Note that Cinder
will puke 400 and stop trying the rest once it encounters a
non-existing (or deleted) key amoung the given list of keys.
This change also fixes 'list'/'show' API includes deleted keys in
result.
Jay S. Bryant [Mon, 12 Aug 2013 17:55:48 +0000 (12:55 -0500)]
Add check for qemu-img to image_utils fetch_to_raw
Some platforms, particularly PowerPC systems running RHEL,
do not have qemu-img installed by default and do not
support image formats other than RAW. For these systems,
functions/drivers that use fetch_to_raw() currently fail
because it is assumed that qemu-img will always be installed.
This change updates fetch_to_raw() to function more like the
upload_volume() function which checks the image format and skips
any conversion if the image is already in the RAW format. To
minimize the impact upon platforms that have qemu-img installed
this is being implemented by checking to see if qemu-img is
installed. If it is installed, no changes are made to the flow
through fetch_to_raw. If qemu-img is not available and the image
format is not already RAW, an exception is thrown. Otherwise,
the image is downloaded and we do not progress to the portions of
fetch_to_raw that require qemu-img to be installed.
This commit also updates the test cases that touch this code.
Fixed erroneous force full copy in ceph backup driver
The following steps should not result in a forced full copy:
1. create vol
2. create backup (works fine i.e. uses incremental diff)
3. delete backup
4. create new backup (full copy forced because volume backup-snap exists and is not ignored)
cinder.volume.rpcapi.create_volume does not convert the request_spec to
primitive before casting the request. This makes requests containing non
primitive types to fail. For example:
This will create a new database record and call create_volume on
volume.rpcapi. This will fail because VolumeMetadata won't be
serialized correctly when calling cast. This, however, is not True when
--source-volid is not passed because in such case,
scheduler.rpcpai.create_volume will be called, which converts
request_spec to primitive correctly.
Fix a corner case in associating QoS Specs with Volumt types that was
not handled well: Raise HTTPBadRequest() to client if associate volume
type, which is already associated one qos specs other than specified
qos specs. Previously such operation will proceed without error which
results overwriting original association.
Also refactor DB API volume_type_qos_specs_get() to return qos specs
in consistent dict format as other qos specs related APIs.
Code Review / openstack-build / cinder-build.git / log