Tom Swanson [Tue, 2 Jun 2015 19:20:55 +0000 (14:20 -0500)]
Dell SC: update_volume_stats could use uninitialized vars
In _update_volume_stats in dell_storagecenter_common.py totalcapacitygb
and freespacegb are not necessarily initialized in the LOG.debug call.
Changed to data['total_capacity_gb'] and data['free_capacity_gb'].
Also initialized to total_capacity_gb and free_capacity_gb dict entries
to 'unavailable' which allowed the removal of two conditionals in the
same function.
Patrick East [Tue, 14 Apr 2015 20:45:22 +0000 (13:45 -0700)]
Refactor PureISCSIDriver into base and iSCSI classes.
This adds in a new class called PureBaseVolumeDriver that contains
all of the generic shared methods that utilize the management REST API.
The PureISCSIDriver now inherits from the base driver and implements
the functionality required for iSCSI hosts/initiators.
As part of this we’ve cleaned up some of the logging where it would
previously have hard coded the drivers class name in it, and moved some
static non-class helper methods into the base class.
These changes will ease future work to add additional drivers or split
them into separate files.
Gorka Eguileor [Mon, 1 Jun 2015 16:30:01 +0000 (18:30 +0200)]
Fix cinder concurrency issues on rtstool
Since there's no synchronized access to configfs in rtslib it can happen
that rtstool or rtslib access an element that no longer exists because
it has been removed just in the middle of a loop by another Cinder
request.
This results in quite a different number of exceptions:
- .dump()
- KeyError
- IOError
- RTSLibError on storage_object
This patch synchronizes access to all rtstool calls that access or
modify configfs using utils.synchronized decorator.
John Griffith [Fri, 29 May 2015 23:41:25 +0000 (23:41 +0000)]
Use SolidFire snapshots for Cinder snapshots
This patch just replaces the use of clones on the SolidFire
backend for SolidFire Snapshot objects. This isn't necessarily
an issue, but it is important necessary to do things like
consistency group snaps.
Note, that we have to keep compatability for things like
create_from_snapshot where the system may have clones that
were used for snapshots prior to updating the driver.
While we're here, we also removed the logging calls in the test
which shouldn't really be there, as well as the overly chatty
debug messages like "entering/leaving". More cleanup work
can be done to logs but should be in their own patch.
wanghao [Wed, 6 May 2015 04:08:00 +0000 (12:08 +0800)]
Notification with volume and snapshot metadata
Now notify message to ceilometer do not contain volume
metadata/glance_metadata and snapshot metadata, which are
useful to bill and enrich resource information in ceilometer.
This feature will add the metadata/glance_metadata of volume
and metadata of snapshot to notification usage info.
John Griffith [Fri, 29 May 2015 18:57:59 +0000 (12:57 -0600)]
Add volume ID to fake volumes in Gluster tests
This adds the volume ID field to the fake volume objects
in the GlusterFS tests. Turning up concurrency exposes some
issues where access of this field is attempted but doesn't
exist.
Xing Yang [Wed, 27 May 2015 16:58:04 +0000 (12:58 -0400)]
Fix capacity filter to allow oversubscription
Currently the capacity filter does not allow oversubscrition
if the free physical capacity is less than the new volume size.
This was based on the assumption that a new thinly provisioned
volume will be consumed right away. This assumption does not
allow us to take full advantage of thin provisioning.
The fix is to allow oversubscription if thin provisioning is
supported and max_over_subscription_ratio is greater than or
equal to 1. The free physical capacity will not be used to
compare with the new volume size for a backend that supports
oversubscription in thin provisioning. The oversubscription
ratio and virtual free capacity will still be used to decide
whether a backend can be chosen for thin provisioning.
yogeshprasad [Fri, 8 May 2015 10:37:22 +0000 (16:07 +0530)]
Add chap support to CloudByte cinder driver
CloudByte iSCSI cinder driver currently does not support
iSCSI CHAP security setting. This opens a security vulnerability
whereby any host machine can mount a CloudByte iSCSI volumes
created via OpenStack.
This patch adds CHAP authentication support for the CloudByte iSCSI
cinder driver.
Admin has to manually add the auth group and chap username/password in
CloudByte storage. Driver uses the first user found in auth group.
Clinton Knight [Wed, 15 Apr 2015 14:50:49 +0000 (10:50 -0400)]
NetApp FC driver shims missing manage/unmanage
The NetApp FC drivers' passthrough shim layers are missing the
manage/unmanage methods, causing the manage/unmanage functions
to fail. The necessary functionality is already present in
the underlying library layer, so the trivial fix is to add the
functions to the passthrough layers. This patch also adds a
unit test to ensure the four block storage driver interfaces
remain in sync.
Daniel Allegood [Wed, 27 May 2015 17:27:06 +0000 (10:27 -0700)]
Updating cmd/manage.py get_arg_string() argument parser and adding unit test
Looks like get_arg_string() would improperly chop off the first two characters
of an argument that was passed in with one dash. For example,
passing '-option foo' would return 'tion foo' instead of 'option foo'.
Tom Barron [Mon, 23 Feb 2015 13:51:13 +0000 (08:51 -0500)]
Add standard QoS spec support to cDOT drivers
This commit adds support for standard cinder QoS specs to
NetApp cDOT drivers, alongside our pre-existing support for
externally provisioned QoS policy groups via qualified
extra specs.
Previously, the logic to calculate the LUN ID only considered a single attached
host. Because multiple hosts can be attached to a LUN, the logic needs to
consider multiple hosts. This patchset proposes to choose non-conflicting
LUN IDs across the whole array. This patch first checks for an empty LUN ID
across the array. If it doesn't find one, then it looks for the least used
LUN ID across the whole array.
The value of MAX_LUNS_PER_HOST has been changed here as the range function
iterates from 0 to n-1 LUN IDs. With Eseries backend, a maximum of 256 LUN_IDS
are supported, thus a value of 256 MAX_LUNS_PER_HOST is required.
Eric Brown [Tue, 5 May 2015 01:42:48 +0000 (18:42 -0700)]
VMware: insecure option should be exposed
Commit Ida730db66b154a4d445f7a91bccb9ca5b5a26f5e introduced
certificate verification but did not expose the option to turn
on/off verification using the insecure boolean of oslo.vmware.
Gorka Eguileor [Wed, 4 Mar 2015 13:24:25 +0000 (14:24 +0100)]
Create iSCSI lio portals with right IPs and port
Currently lioadm helper creates portals with fixed IP addresses 0.0.0.0
and ::0 and also uses fixed standard port 3260 instead of using store
configuration (iscsi_ip_address, iscsi_secondary_ip_addresses,
iscsi_port) from the configuration as it should.
This could lead to problems if a different port was defined in the
configuration as the target would be available only on 3260 and other
services would look for it at configured port.
With this fix we now create the right portals with the right port.
Since now we pass list of ips and port to helpers cxt helper has also
been updated to use these parameters instead of going directly to the
configuration.
Tina [Thu, 12 Mar 2015 00:34:36 +0000 (20:34 -0400)]
Create consistgroup from cgsnapshot support in VNX driver
Changed VNX Cinder driver to support creating a consistency group from
a consistency group snapshot and refactored the migration lun code for
reusing the code in create cg from cgsnapshot feature.
wanghao [Mon, 9 Mar 2015 10:49:31 +0000 (18:49 +0800)]
Fix response when querying host detail by host name
When querying host detail by host name like
'wanghao-devstack-snapshot@lvmdriver-1'
returned by host list command, cinder returns
incorrect result in 'total' block.
"resource": {
"volume_count": "0",
"total_volume_gb": "0",
"total_snapshot_gb": "0",
"project": "(total)",
"host": "wanghao-devstack-snapshot@lvmdriver-1",
"snapshot_count": "0"
}
This issue is caused by 'volume_data_get_for_host'
in sqlalchemy. This has nothing to do with multibackend,
this is due to the introduction of pool support for single
backend.
Fix this by querying volume filter by host or host name
like 'host#%' in volume_data_get_for_host.
Gorka Eguileor [Tue, 26 May 2015 14:39:28 +0000 (16:39 +0200)]
Display NOTIFICATIONS on assert failure
From time to time we have itermitent errors regarding the
fake_notifier.NOTIFICATIONS count.
This patch does two things to try to find out which notifications are
being fired unexpectedly:
- Make sure that fake_notifier is reset in cleanup regardless of
exception raised by shutil.rmtree
- Display fake_notifier.NOTIFICATIONS whenever count assertion fails.
Jeegn Chen [Fri, 8 May 2015 07:41:23 +0000 (15:41 +0800)]
Address 0x712d8e0e error in VNX Cinder Driver
In original implementation, 0x712d8e0e error is not considered.
Thus, some snapshot creation and volume extending operations may fail
when the system is in a transition state.
This patch is to add some waiting and retry logic when 0x712d8e0e error
comes out so that the system is more error-tolerant.
ChangBo Guo(gcb) [Wed, 24 Dec 2014 14:42:58 +0000 (22:42 +0800)]
Leverage dict comprehension in PEP-0274
PEP-0274 introduced dict comprehensions to replace dict constructor
with a sqeuence of key-pairs[1], these are benefits:
First, it makes the code look neater.
Second, it gains a micro-optimization.
Cinder dropped python 2.6 support in Kilo, we can leverage this now.
Note: This commit doesn't handle dict constructor with kwargs.
This commit also adds a hacking rule.
Jeegn Chen [Wed, 20 May 2015 14:00:07 +0000 (22:00 +0800)]
Add missing '-o' CLI option to VNX Cinder Driver
In some NaviSecCLI command, if -o is not given, the command may prompt
some message and ask for 'y' or 'n' for confirmation in some special
scenario, which is not working for Cinder.
This patch is to add the missing '-o' option to avoid these failures.
Eric Harney [Fri, 15 May 2015 17:23:08 +0000 (13:23 -0400)]
Add ability for drivers to copy data preserving sparseness
Drivers can set self._sparse_copy_volume_data to opt in
to copying data in a way that preserves sparseness.
This is likely to be desired elsewhere, but currently this
patch only enables it for the NFS driver and provides a
method for other drivers to opt in, in case there are
unexpected issues in some drivers.
On the NFS driver, this prevents volume migration from
changing a volume file from sparse to fully allocated.
Tom Barron [Tue, 21 Apr 2015 15:07:30 +0000 (11:07 -0400)]
Use nfs_oversub_ratio when reporting pool capacity
Currently NetApp NFS drivers do not make use of the nfs_oversub_ratio
configuration parameter to enable OpenStack administrators to report
anything other than the most conservative capacity and usage information
up to the scheduler
This commit:
* modifies the NetApp drivers to use the nfs_oversub_ratio as
documented.
* uses direct API with filers to gather capacity information rather
than stat and du commands
* brings our reporting of reserved percentage in line with the way
the scheduler actually makes use of this statistic
* simplifies and cleans up the way we gather and report pool statistics
Note that this fix addresses an in-the-field bug report from juno
and is intended to be the basis for backported fixes.
In kilo, a more general approach to overprovisioning using
max_oversubscription_ratio
was introduced via commit 3548a4bc9edbb26b248b5af5ecc2145f2c6f7481.
In this newer approach, the scheduler computes "virtual" free space
rather than the driver reporting "apparent" free space.
This fix to bring NetApp NFS drivers into conformity with the generic
NFS driver is not intended as a substitute for introducing support
in our drivers for the newer approach to overprovisioning. Rather,
it is a bug-fix for an existing failure to honor the nfs_oversub_ratio.
Mitsuhiro Tanino [Wed, 13 May 2015 15:59:13 +0000 (11:59 -0400)]
LVM: Pass volume size in MiB to copy_volume() during volume migration
Currently migrate_volume() in lvm.py does not pass volume size in
MiB but passes volume size in GiB to copy_volume(). The size argument
of copy_volume() requires MiB value. As a result, if the volume size
is 1GiB, copy_volume() copies only 1MiB head of 1GiB volume to the
destination volume, and then the volume data is corrupted.
This patch fixes to pass volume size in MiB.
LVM: Support efficient data copy using "dd" for create_cloned_volume
The create_cloned_volume uses dd command for data copy, but the
copy always copy full blocks even if the source data contains
many null or zero blocks. When we use thin provisioned LVM,
blocks are not pre-allocated, so unused region returns zero.
If we copy full block for destination volume, unnecessary blocks
will be allocated and the usage will be 100%.
The dd command has conv=sparse option in order to copy data more
efficiently. This patch enables conv=sparse option as an argument
of dd command for create_cloned_volume when we use thin provisioned
LVM.
[NOTE]
This patch only enables conv=sparse parameter of dd command for
create_cloned_volume() path of LVM driver. There are some places
using dd in Cinder, but we should carefully consider to apply
this parameter for other places because misuse of this parameter
causes security issues or data corruptions. Also we DO NOT use this
parameter for volume wiping case because the volume is not cleared
at all.
Xing Yang [Thu, 2 Apr 2015 05:31:56 +0000 (01:31 -0400)]
Fix a problem with FAST support in VMAX driver
The VMAX driver doesn't distinguish between storage groups for
masking views (used for attachment) and storage groups for volumes
under FAST policy. This causes problems during attach volume.
This patch fixed the problem.
Sean McGinnis [Sat, 25 Apr 2015 00:15:37 +0000 (19:15 -0500)]
Fix incorrect reraising of exceptions
There are several places in the code where exception handling raises
the exception variable rather than just calling raise. This results
in the traceback being incorrect.
Had considered adding a hacking check for this, but that becomes a
little tricky. There are valid places where "raise ex" is used that
would prevent a simple check.
Thang Pham [Sun, 1 Mar 2015 22:46:02 +0000 (17:46 -0500)]
Switch to oslo_versionedobjects
oslo_versionedobjects was not made available until very
late in the Kilo cycle (i.e. near the end of kilo-3).
In order to make progress on cinder objects, a fork of
nova objects was made, so that proper trial and testing
could be done. The following patch makes the switch to
use oslo_versionedobjects.
If invalid 'attachment_id' is passed to the os-force_detach
api, then it returns 500 error because even though
VolumeAttachmentNotFound exception is raised from manager,
it gets converted to RemoteError due to rpcapi call.
Caught RemoteError exception in the api and returned 400
error to the user if exception type is VolumeAttachmentNotFound
or InvalidVolume. In other cases 500 error is raised.
Code Review / openstack-build / cinder-build.git / log