LVM: Support efficient data copy using "dd" for create_cloned_volume
The create_cloned_volume uses dd command for data copy, but the
copy always copy full blocks even if the source data contains
many null or zero blocks. When we use thin provisioned LVM,
blocks are not pre-allocated, so unused region returns zero.
If we copy full block for destination volume, unnecessary blocks
will be allocated and the usage will be 100%.
The dd command has conv=sparse option in order to copy data more
efficiently. This patch enables conv=sparse option as an argument
of dd command for create_cloned_volume when we use thin provisioned
LVM.
[NOTE]
This patch only enables conv=sparse parameter of dd command for
create_cloned_volume() path of LVM driver. There are some places
using dd in Cinder, but we should carefully consider to apply
this parameter for other places because misuse of this parameter
causes security issues or data corruptions. Also we DO NOT use this
parameter for volume wiping case because the volume is not cleared
at all.
John Griffith [Tue, 12 May 2015 14:03:27 +0000 (08:03 -0600)]
Check type match on create from source/snap
We used to allow creating from source/snap and specifying a
different type than the originating resource when doing so.
Once we started getting more drivers and more multi-backend
configurations, we implemented a check in volume.api that
took this away (broke it). There have been a number of
arguments about whether this should be allowed or not, and
that it could fail after the rpc call leaving the user with
nothing more than a "failed" volume and no explanation as to
why.
This patch allows the capability, but checks validity at the
API layer before issuing the create call. There are two
requirements for the new type specification to be valid:
1. There is only one backend (cinder-volume) topic configured
2. Both types in question specify the same volume_backend_name
If neither of these requirements are met, the user will receive
an "invalid type" error explaining that the type combination is
not compatible and that they should omit the type argument altogether.
Wikipedia's list of common misspellings [1] has a machine-readable
version. This patch fixes those misspellings mentioned in the list
which don't have multiple right variants (as e.g. "accension", which can
be both "accession" and "ascension"), such misspellings are left
untouched. The list of changes was manually re-checked for false
positives.
GlusterFS: Renaming test case to test_mount_glusterfs
With the patch [1], GlusterFS is now using mount method in
RemoteFsClient. That method mounts a gluster share only if it has
not already been mounted.
So, having test case name as
test_mount_glusterfs_should_mount_correctly, doesn't make sense now.
This patch changes its name to 'test_mount_glusterfs'.
John Griffith [Mon, 4 May 2015 17:31:15 +0000 (11:31 -0600)]
Add new exception to retryables in SolidFire driver
Newer versions of SolidFire Element OS introduce an
internal snapshot mechanism during cloning that also
includes a new exception (xNotReadyForIO).
This can be encountered when programmatically cloning
hundreds of large volumes. Typically this will succeed
on a retry of the clone command, so we should add this
new exception to our list of retryable exceptions in
the SolidFire driver.
Tom Swanson [Thu, 7 May 2015 21:39:26 +0000 (16:39 -0500)]
Dell: Added support for update_migrated_volume
Added support for updated_migrated_volume to common driver.
Added rename_volume to dell_storagecenter_api to support it.
Added tests to test_dellsc.py and test_dellapi.py.
The only thing we do in this rename the newly migrated volume
destination volume on the Dell Storage center to be the same
as the original volume ID. Since we identify the volumes
by volume ID this prevents them from being lost.
Matt Riedemann [Thu, 7 May 2015 20:39:33 +0000 (13:39 -0700)]
Log command failure details before raising ISCSITargetDetachFailed
In the case that there is some useful information in the
ProcessExecutionError (like stderr) when trying to terminate the
connection in the lio target, also log the error.
Eqlx: Fixes the retries on Network Connection Error
Volume creations were failing due to ssh connection
errors throwing ProcessExecutionError
and retries were not happening. Made a fix
to retry commands on both network errors and
backend api exceptions.
The parameter eqlx_cli_max_retries determines
the number of retry attempts made.
ankitagrawal [Mon, 16 Mar 2015 13:01:59 +0000 (06:01 -0700)]
Fix missing translations for log messages
Fixed log translations in sqlalchemy module for error, info and warning
messages with appropriate marker function according to the logging
standards [1].
As LOG.warn has deprecated [2] so I have changed LOG.warn to
LOG.warning.
If volume is in 'in-use' state or invalid instance UUID
is passed to cinder os-attach api, then it returns 500 error
because InvalidVolume and InvalidUUID exceptions are raised
from manager, but not caught at api level.
Caught RemoteError exception in the api and returned 400 error
code if user passes invalid information of the volume or invalid
instance UUID.
If invalid 'attachment_id' is passed to the os-detach
api, then it returns 500 error because even though
VolumeAttachmentNotFound exception is raised from manager,
it gets converted to RemoteError due to rpcapi call.
Caught RemoteError exception in the api and returned 400
error to the user if exception type is VolumeAttachmentNotFound
OR InvalidVolume. In other cases 500 error is raised.
Sean McGinnis [Tue, 5 May 2015 20:57:30 +0000 (15:57 -0500)]
Logging not using oslo.i18n guidelines (zonemgr)
Multi-patch set for easier chunks. This one addresses
the zonemanager cinder directory.
There have been quite a few instances found where the
i18n guidelines are not being followed. I believe this
has helped lead to some of the confusion around how to
correctly do this. Other developers see this code and
assume it is an example of the correct usage.
This patch attempts to clean up most of those violations
in the existing codebase to hopefully help avoid some of
that confusion in reviews.
Some issues address:
* Correct log translation markers for different log levels
* Passing format values as arguments to call, not preformatting
* Not forcing translation via six.text_type and others
Guidelines can be found here:
http://docs.openstack.org/developer/oslo.i18n/guidelines.html
Hacking checks will not be able to identify all violations of
the guidelines, but it could be useful for catching obvious
one such as LOG.info("No markers!").
Some of unit tests print logs to stdout. Tests output should be clear
and contain only necessary information. Debug output should not be mixed
with tests output.
Gorka Eguileor [Mon, 9 Mar 2015 18:39:11 +0000 (19:39 +0100)]
Preserve usage and reservations on quota deletion
Current API deletes quota usage and reservations on quota limit
deletion.
According to API documentation what should only happen is that quotas
limits revert to default values by deleting tenant/user limits.
This patch fixes this issue.
APIImpact: Delete on os-quota-sets will no longer remove usage and
reservation quotas. Those quotas are handled by Cinder
service.
UpgradeImpact: There is no upgrade impact afaik.
Closes-Bug: #1410034
Change-Id: I9340b6f78623cfa5b505886ad75b8e4d3cd6131b
Lucian Petrut [Tue, 5 May 2015 08:12:50 +0000 (11:12 +0300)]
Windows iSCSI: remove ensure_export
The ensure_export method is called by the manager when the service
is initialized, ensuring that in-use volumes are properly exported.
iSCSI targets exported by WinTarget persist after host reboot.
For this reason, the ensure_export method can simply pass, thus
simplifying the iSCSI target creation logic.
The patch set depending on this introduces CHAP credentials support.
If the iSCSI target is accidentaly deleted, the CHAP credentials
will change, so the volume won't be accessible anyway if the target
is recreated using ensure_export.
Julien Danjou [Thu, 15 Jan 2015 11:40:38 +0000 (12:40 +0100)]
tests: replace mox by mox3, clean out mox usage
This patch replaces mox3 by mox, so we can drop mox from requirements
and bring Python 3 compatibility. It also clear some mox usage and
use the fixture brought by oslotest.
Jamie Lennox [Tue, 5 May 2015 01:50:21 +0000 (11:50 +1000)]
Catch additional type conversion errors
When converting strings to integers in quota we catch ValueError which
will handle an input which is not a valid number, eg 'abc'. Extend this
to covert TypeErrors which will handle things like a None value being
passed.
Currently vCenter certificate is not verified during connection
establishment. This patch adds a config option to specify a CA
bundle file to verify vCenter server certificate.
Xing Yang [Sat, 25 Apr 2015 02:42:28 +0000 (22:42 -0400)]
Clean up failed clones in VMAX driver
If the clone operation fails, clean up is not done properly in
the VMAX driver. This patch catches the exception during the
clone operation, removes the target volume, and then re-throw
the exception.
Jay S. Bryant [Fri, 1 May 2015 21:05:25 +0000 (16:05 -0500)]
Correct directories check for N327 hacking check
For some reason the N327 hacking heck was looking for
cinder.unit/tests and cinder.functional/tests in the filename
that is sent in for the hacking check. This should be
cinder/tests/unit and cinder/tests/functional.
This concern was originally raised in this review:
https://review.openstack.org/175604
John Griffith [Fri, 1 May 2015 16:35:30 +0000 (10:35 -0600)]
Fake out sleeps in unit tests
Just a minor cleanup, had a few tests that were doing sleeps
(or looping calls for those that prefer it) that were being
executed to test timeouts in the unit tests. No value in
actually *waiting* for these, so this patch just mocks them
out so we still test the procedure but don't actually have
to wait.
This shaves about 30 seconds or so off of the unit test run,
not a huge deal, but every little bit helps IMO.
Also noticed a LOG.debug message in the test_extensions file
that wasn't picked up in other cleanup work so I removed that
debug log call as well.
Tom Barron [Sat, 18 Apr 2015 13:42:10 +0000 (09:42 -0400)]
Fix range check for NFS used ratio
The nfs_used_ratio configuration parameter is supposed to be
greater than zero and less than or equal to one. However,
current configuration checks actually allow values greater
than one because of a logic error. Unit tests fail to detect
this issue because they find the exception that they expect
to find, but it is actually being thrown for another reason.
This commit fixes the logic error in the nfs driver code.
It also reworks all the unit tests for the do_setup method since
they suffer from the same issue that led to failure to detect
this error in the first place - they fail to do proper mocks of
submethods, and when these run the expected exception gets triggered
for the wrong reason. We have moved these tests into their own test
class, where mock is used rather than mox. Over time it is likely
that new tests will be implemented, or old tests reworked, with mock
instead of mox. These can go in the new test class, which can be
renamed if that is appropriate.
When attach is called twice for the same volume and instance the
attach_volume checks if the attachment already exists and returns.
This leaves the volume in 'Attaching' state. The volume status should
be reset to 'in-use' if we see the attachment already exists.
John Griffith [Wed, 29 Apr 2015 23:25:12 +0000 (17:25 -0600)]
Add retry to lvm delete
Seems we have another issue related to lvm and
snapshots, but this time it's on the delete side.
We used a simple retry mechanism for snapshot create
here:
https://review.openstack.org/#/c/149360/8
I'm hesitant to just add another retry without looking
at dm issues, but this should address the problem
temporarily and give us a chance to collect some
data on the issue.
wuyuting [Fri, 13 Feb 2015 02:16:24 +0000 (10:16 +0800)]
Admin extends tenant's volume but change admin's quota
When admin extends a tenant's volume, the tenant's
quota-usage doesn't change. However, admin's quota-usage
has changed. The right practice is: extend whose volume,
whose quota-usage should be changed.
The Oslo libraries have moved all of their code out of the 'oslo'
namespace package into per-library packages. The namespace package was
retained during kilo for backwards compatibility, but will be removed by
the liberty-2 milestone. This change removes the use of the namespace
package, replacing it with the new package names.
The patches in the libraries will be put on hold until application
patches have landed, or L2, whichever comes first. At that point, new
versions of the libraries without namespace packages will be released as
a major version update.
Please merge this patch, or an equivalent, before L2 to avoid problems
with those library releases.
Code Review / openstack-build / cinder-build.git / log