When doing a volume restore with Ceph backend to a non-rbd volume
that is greater in size that the backup volume, we write zeroes
the extraneous extents but we are not zeroing the final few bytes
in the case where the destination volume is non-rbd.
Also fixed up internationalisation on a number of log messages.
This change modifies the behaviour of the secure delete for thick
LVM snapshots to wipe the underlying COW of the snapshot LV
instead of the snapshot LV itself.
This change is necessary because the snapshot LV does not contain
exactly the same number of writable blocks as the original LV. The
COW includes header information per COW block that identifies the
device as a COW device as well as the source and destination blocks
for the changed item. The amount of metadata contained in the COW is
variable based on I/O performed on the snapshot.
This does not change the behavior of secure deletes on thin LVs
or secure deletes on the thick LV snapshot origin.
Jay S. Bryant [Mon, 16 Sep 2013 18:21:34 +0000 (13:21 -0500)]
Set rootwrap_config path to rootwrap.conf
Other components set
rootwrap_config = /etc/<component>/rootwrap.conf by default.
This commit brings cinder up-to-date with what other
components are doing by setting
rootwrap_config = /etc/cinder/rootwrap.conf by default.
A couple of test cases needed to be updated to handle the change
from 'None'.
Not counting this as a documentation impact as the documentation I
can find already lists /etc/cinder/rootwrap.conf as the default.
Eric Harney [Sat, 7 Sep 2013 03:26:46 +0000 (23:26 -0400)]
Do not use qemu-img --backing-chain or --output=json
Initial code for GlusterFS snapshot support in Havana
used qemu-img arguments which are too new to run in many
environments. Run qemu-img using only arguments which
will exist on older platforms.
Fixes ceph volume restore to larger image than source
If a backup is restored to a volume that is bigger than the
backup itself, which could be the case if --volume-id is
supplied, we need the option to set extraneous extents to zero
i.e. set extents after those from the backup to zero.
Huawei driver failed to parse ssh result when firstly logging
in storage system on paramiko 1.8.0. The update of paramiko
make the order of ssh returns changed. This patch improves
the parsing way of ssh returns.
This patch also does some changes for log punctuations.
Fixes the use of exception.InvalidInput with the wrong arguments
A mapping key of 'message' in exception.InvalidInput is 'reason'.
cinder/volume/drivers/netapp/iscsi.py and
cinder/volume/drivers/netapp/utils.py had used 'data'.
Michael Kerrin [Mon, 9 Sep 2013 08:12:27 +0000 (08:12 +0000)]
Add default_availability_zone configuration option to cinder
This option is used to specify the default AZ for the whole service. So when
creating a new volume this value is used to specify the AZ of a volume.
Previously we were overloading the use of the storage_availability_zone option
which is also used to specify what AZ a service belongs to. We still fail back
on the storage_availability_zone when creating a new volume if the
default_availability_zone is not set.
Zhi Yan Liu [Tue, 3 Sep 2013 14:40:49 +0000 (22:40 +0800)]
Turn db model object into a primitive object to avoid error
In volume API controller we need turn db model object into a primitive
object to avoid error when it adding visible administrator metadata to
volume's metadata dict.
When the driver copy_image_to_volume fails it can
at the current time raise more than just CinderException
as its root exception type. This causes rescheduling due
to the blacklisted exception list that is used to determine
if a exception is 'bad enough' to trigger rescheduling or
should the volume creation action just set the volume to
error state.
To avoid the situation where this would cause a rescheduling
we should make sure (for now) that any exception that is
emitted on copying an image to a volume is translated
into a image copy failure and reraised.
Fixing UnicodeEncodeError against volume creating function
When creating a volume with non-English characters and there is
not enough space for it, then an UnicodeEncodeError occurs. So
we need convert any coded exception message to unicode string
for logging.
Previously deleting a key in certain qos specs was accomplished via
'update' API. Unfortunately, 'update' isn't able to tell the
difference between setting a key with no value and deleting a key (and
its value).
This change adds an new API 'delete_keys' to qos_specs API extension.
'delete_keys' API allows client to specify a list of to-be-deleted keys
in one single request (batch mode!), which can be handy when removing
multiple keys in a qos specs.
Example URL and request body for 'delete_keys' API:
PUT to http://127.0.0.1:8776/v2/qos-specs/QOS_SPECS_UUID/delete_keys
with body: {'keys': ['foo', 'bar', 'zoo']}
Above example request will result in 'foo', 'bar', 'zoo' key/value
pairs of QOS_SPECS_UUID be marked as deleted in DB. If QOS_SPECS_UUID
doesn't exist, a 404 error will return; if any key in 'foo', 'bar',
'zoo' couldn't be found in QOS_SPECS_UUID, a 400 error will return with
error message telling which key couldn't be found. Note that Cinder
will puke 400 and stop trying the rest once it encounters a
non-existing (or deleted) key amoung the given list of keys.
This change also fixes 'list'/'show' API includes deleted keys in
result.
Jay S. Bryant [Mon, 12 Aug 2013 17:55:48 +0000 (12:55 -0500)]
Add check for qemu-img to image_utils fetch_to_raw
Some platforms, particularly PowerPC systems running RHEL,
do not have qemu-img installed by default and do not
support image formats other than RAW. For these systems,
functions/drivers that use fetch_to_raw() currently fail
because it is assumed that qemu-img will always be installed.
This change updates fetch_to_raw() to function more like the
upload_volume() function which checks the image format and skips
any conversion if the image is already in the RAW format. To
minimize the impact upon platforms that have qemu-img installed
this is being implemented by checking to see if qemu-img is
installed. If it is installed, no changes are made to the flow
through fetch_to_raw. If qemu-img is not available and the image
format is not already RAW, an exception is thrown. Otherwise,
the image is downloaded and we do not progress to the portions of
fetch_to_raw that require qemu-img to be installed.
This commit also updates the test cases that touch this code.
Fixed erroneous force full copy in ceph backup driver
The following steps should not result in a forced full copy:
1. create vol
2. create backup (works fine i.e. uses incremental diff)
3. delete backup
4. create new backup (full copy forced because volume backup-snap exists and is not ignored)
cinder.volume.rpcapi.create_volume does not convert the request_spec to
primitive before casting the request. This makes requests containing non
primitive types to fail. For example:
This will create a new database record and call create_volume on
volume.rpcapi. This will fail because VolumeMetadata won't be
serialized correctly when calling cast. This, however, is not True when
--source-volid is not passed because in such case,
scheduler.rpcpai.create_volume will be called, which converts
request_spec to primitive correctly.
Fix a corner case in associating QoS Specs with Volumt types that was
not handled well: Raise HTTPBadRequest() to client if associate volume
type, which is already associated one qos specs other than specified
qos specs. Previously such operation will proceed without error which
results overwriting original association.
Also refactor DB API volume_type_qos_specs_get() to return qos specs
in consistent dict format as other qos specs related APIs.
Victor Rodionov [Fri, 6 Sep 2013 22:18:48 +0000 (02:18 +0400)]
Fix bug in Nexenta NFS driver _do_create_volume
Fix bug in _do_create_volume method of Nexenta NFS volume driver.
NMS call nms.folder.create doesn't support third argument, that is why
this call changed to nms.folder.create_with_opts.
If cinder-backup service is not enabled, "cinder backup-create"
command fails like the following. As the result, the volume status
has been changed to "backing-up" in spite of not backing-up.
$ cinder backup-create f48aa6ae-4c35-4a6a-a393-5a5cf610945a
ERROR: Service cinder-backup could not be found.
$ cinder list
+--+------------+--------------+------+-------------+----------+-------------+
|ID| Status | Display Name | Size | Volume Type | Bootable | Attached to |
+--+------------+--------------+------+-------------+----------+-------------+
|..| backing-up | vol-test | 1 | None | False | |
+--+------------+--------------+------+-------------+----------+-------------+
$
To avoid this situation, this patch moves the cinder-backup check
before changing a volume status to "backing-up".
John Griffith [Fri, 6 Sep 2013 19:55:39 +0000 (13:55 -0600)]
Do not attempt vg.update on uninitialized vg
It's possible for the scheduler to send the periodic
stats update request *before* the volume service/driver
has finished initializing, resulting in an exception trace
being logged.
This change adds a check to verify that the vg is actually
initialized (completed check_for_setup_error) and it also
removes all of the duplication of each LVM class driver.
To deal with other drivers that may have similar issues,
add initialized member to base driver, that member is
set False on __init__ and is updated by the manager
after running check_for_setup_error. We also skip
update_stats if initialization isn't set True.
Rather than have a separate copy of update_stats in every
driver, we have one in the base LVM class and we set the two
unique variables as member parameters.
Zhongyue Luo [Fri, 6 Sep 2013 04:10:28 +0000 (12:10 +0800)]
Fixes misuse of assertTrue in test scripts
Misuse of assertTrue in places where assertEqual should be used.
If assertTrue is used instead of assertEquals, the test will always pass
if the first argument's boolean cast is True. The second argument
passed to assertTrue() will be regarded as the error message
when the assertion fails. Using assertEqual will actually check
if two arguments are same value.
Zhongyue Luo [Fri, 6 Sep 2013 04:01:26 +0000 (12:01 +0800)]
Utilizes assertIsNotNone
Using assertTrue and the 'is not' operator to test if an element
is not None is too python2.4. Our unit testing framework supports
assertIsNotNone, included from python 2.7, which was created
for these types of tests.
Zhongyue Luo [Fri, 30 Aug 2013 11:50:26 +0000 (20:50 +0900)]
Utilize assertIsInstance
Using assertTrue and the 'isinstance' function to test
if an object is in an instance of some class is too python2.4.
Our unit testing framework supports assertIsInstance which was created
for these types of tests. Let's use assertIsInstance for these tests.
Up till now we only had copy-on-write for cloning from snapshot. This
change optionally allows clone from volume to use copy-on-write
instead of a doing a full copy each time. This should increase speed
and reduce nearterm storage consumtion but could introduce some new
risks e.g. excessively long clone chains and flatten storms. To avoid
this, a new config option has been providedons are provided -
rbd_max_clone_depth - which allows the user to limit the depth of a
chain of clones i.e.
a->b->c->d as opposed to a->b
->c
->d
This will avoid flatten storms by breaking chains as they are formed
and at an early, predefined stage.
A second option - rbd_clone_from_volume_force_copy - allows the user
to use a full copy as before i.e. disable COW for volume clones.
VMDK copy_image_to_volume and copy_volume_to_image
Implemented copy_image_to_volume that creates a new volume backing (vm) and
copies over the vmdk file from the glance image. Only glance images of disk
format 'vmdk' can be used to create a volume using this driver.
Also implemented copy_volume_to_image that creates a new glance image using
the volume's vmdk file. The steps involved are to take a snapshot of volume
vmdk, take a copy of this read only file and upload to glance.
Cleaned up docstrings as pointed out in earlier reviews.
Added timeout to avoid blocking wait on read/write threads and also fixed a
bug to upload image from copy of vmdk instead of from original vmdk during
upload to glance.
Kurt Martin [Wed, 4 Sep 2013 22:08:51 +0000 (15:08 -0700)]
Validate VV Set exists in 3PAR drivers
The 3PAR drivers must first create a volume and then add that
volume to a virtual volume set on the backend for QoS support.
If that predefined virutal volume set does not exists the
volume would not be associated with the correct QoS settings.
This patch will now look to see if the virtaul volume set
exists and if not will delete the volume and notify the user
that the VV Set does not exists.
We need to let folks know where brick is
currently being maintained for the Havana
release and where to file defects.
Once this simple patch lands, we'll pull this into
the Nova patch.
(fix hacking violation in commit message for the 2nd
time on this patch. To prevent it from blocking the
merge queue on feature freeze day.)
PLEASE DON'T +A PATCHES UNTIL THEY PASS JENKINS CHECK
The exc_info is just a tuple (captured in taskflow)
and isn't itself a callable object so we should not
need to use it like one to extract the exception type
and value.
Running pep8 tests in Nova gives a violation
in lvm.py for not importing a module. This
is a simple fix to import the module itself
instead of only importing a function in the
module.
Kun Huang [Wed, 4 Sep 2013 16:36:48 +0000 (00:36 +0800)]
fix inconsistent i18n log message
An inconsistent example:
imessage = _("Invalid metadata") + ": %(reason)s"
This is not consistent i18n usage. In most of cases, we use _() for
whole string instead of leaving something out.
More than inconsistent, some text in leaving strings may be not
translated, for example, ':' in English is different '：' in Chinese.
Code Review / openstack-build / cinder-build.git / log