Nathaniel Potter [Mon, 28 Sep 2015 22:18:35 +0000 (17:18 -0500)]
Add backup_swift_auth_url to swift backup driver
Currently the config parameter backup_swift_url expects a
keystone endpoint url in the swift backup driver if you're
using single_user mode, but if you're in per_user mode it
expects a swift endpoint url. This patch adds a parameter
called backup_swift_auth_url to differentiate the two.
Unit tests are also provided for testing
backup_swift_auth_url.
DocImpact: The config options "backup_swift_auth_url" and
"keystone_catalog_info" will need to be added to the Cinder
docs.
Change-Id: Iee466fe1404eef9402a19d529dba19c7d5a4ef2b
Closes-Bug: #1449972
Dave Chen [Mon, 3 Aug 2015 09:01:22 +0000 (17:01 +0800)]
Show image metadata
This patch implements the API to show the glance image metadata
for a specific volume.
DocImpact: May need to document this API.
APIImpact: Provided the API to show image metadata.
Partially implements: bp support-modify-volume-image-metadata
Currently the volume related operation in CloudByte driver does not:
* Update the ISCSI Initiator Group during volume deletion operation,
and
* During retry of any volume operation it does not catches the exact
cause of the failure happened.
So to deal with the following issues two changes has been done in
CloudByte Driver:
* Added a new method "_update_initiator_group"
ISCSI Initiator Group is one of the parameter added to the volumes
at the time of creation in CloudByte Storage. Using this the external
source can access the volumes easily. By default, Initiator Group
value is set to 'ALL', but it should be changed to 'None' before
deletion. Since the following is not happening, thus volume delete
operation fails from OpenStack. In order to avoid this the
"delete_volume" method now uses the "_update_initiator_group" which
update the volume ISCSI Initiatior Groupof the volumes to 'None'
before deletion, and thus volume delete operation will have no issues
while running from OpenStack.
* Added a check to "_retry_volume_operation"
The _retry_volume_operation method was not able to check the exact
cause of the failure happened during volume operation in CloudByte
Storage, which arises if some invalid parameters are provided in
"cinder.conf" against "cloudbyte backend" or if some anomaly happens
at CloudByte Storage during any volume operation from OpenStack. Due
to this, tracking the ERROR becomes a very difficult task to do. Thus
to resolve this issue a "check" has been added to the method which
will raise the exact cause of the error and thus making it easier to
solve.
This patch cleanly reverts the changes made via the
commit: e681ba2a99995dcd999e6539bb1222f8a1ac8adc
cleanly and mitigates the conflicts that would
occur with git-revert on the said commit.
The revert is solely for changes pertaining to the use
of the external library, netapp_lib. Minor code refactors
from the prior change are retained.
Unit test coverage has been increased for ZAPI and REST
interface code in netapp/dataontap/client/api.py and
netapp/eseries/client.py.
Vincent Hou [Tue, 1 Sep 2015 03:17:04 +0000 (11:17 +0800)]
Remove the destination volume check in delete_volume
* If the volume migration fails in the middle of the progress,
the destination volume will be cleaned up via
rpcapi.delete_volume. The method delete_volume in manager
should allow us to clean up the database record for the
destination volume.
* The variable update in migrate_volume_completion has been
assigned twice. Remove the second one.
Wilson Liu [Wed, 23 Sep 2015 07:04:33 +0000 (15:04 +0800)]
Huawei driver add check before use a QoS
One QoS in Huawei storage array can contain at most 64 LUNs.
Do a check before adding LUN to QoS, if the number of the
LUNs in the QoS is already equal or larger than 64, then
this QoS should not be used.
Kendall Nelson [Mon, 21 Sep 2015 18:11:19 +0000 (13:11 -0500)]
Missing configuration opts from cinder.sample.conf
The logic for writing opts registered with register_opt() to the
auto-generated opts.py file was flawed in that the directories where
there were opts being registered with this method would be imported,
but the actual opt would be missed when looking through the file for
the name of the opt being registered. Now the singular opts are being
caught and written to opts.py for the oslo-config-generator to process.
The 'backend' section was added to generate_cinder_opts.py
due to some of the missing opts being a part of that section.
Also the way some of the opts that were being registered with groups
were incorrect and so this addresses those in both how they were
processed in generate_cinder_opts.py and the files in which they are being
registered.
There is also one change to the name of an opt in cinder/volume/api.py.
Instances such as this one will be caught by the in-progress hacking
check patch: https://review.openstack.org/#/c/223375/
Tina [Thu, 17 Sep 2015 04:04:42 +0000 (00:04 -0400)]
Create volume in cg enhancement in VNX driver
VNX driver queries whether a LUN is in CG after adding it into
CG. However, as long as the LUN is added successfully, the LUN
will be in CG. The additional query is useless. This patch removes
the additional query.
Eric Harney [Mon, 21 Sep 2015 18:29:19 +0000 (14:29 -0400)]
Allow c-vol backends to start when some backends fail to load
Currently, if a c-vol backend fails to load due to an error in
the module startup process (like a volume driver's __init__ or
dependency import failure), the c-vol process halts completely.
Instead, catch that backend failure, log it, and allow other
backends in enabled_backends to still start up and run.
Note that this is not applicable for failures that allow
drivers to load and remain "uninitialized" -- this is for
failures that occur prior to that point.
When using NetApp drivers, all available flexvols on
a datastore are being reported to the Cinder Scheduler
as available storage pools. This allows Cinder volumes
to be created on flexvols not designated for OpenStack.
This commit fixes the problem by delimiting which pools are
eligible for provisioning via a configuration parameter
'netapp_pool_name_search_pattern'. This allows
the Cinder drivers to operate against a restricted
set of storage containers.
Tom Barron [Mon, 14 Sep 2015 21:38:40 +0000 (17:38 -0400)]
NetApp volume/snapshot delete performance fix
cDOT NFS NetApp drivers back cinder volumes with array-side files and
use array-side cloning techniques when creating snapshots and when
creating volumes from glance images. Customers have experienced
performance issues on the array when deleting many of these in
in quick succession, especially for large volumes.
This commit addresses these performance issues by overriding parent
class volume and snapshot delete methods, which use OS "rm" command to
delete volume backing files, with methods that use DOT API to delete
these files, and which invoke the optimized file deletion engine
available in DOT 8.3.
Michal Dulko [Mon, 21 Sep 2015 13:58:01 +0000 (15:58 +0200)]
Replace soft_delete in volume_type_access_remove
This commit replaces oslo.db's soft_delete used in
volume_type_access_remove in db.api with a simple update setting
deleted column to True.
As deleted column is boolean it is represented in the DB by TINYINT,
which max value is 127. This introduces problems when removing records
with id higher than that, because soft_delete sets deleted column to the
value of id. This commit fixes the issue and adds unit tests for such
case.
Tina [Wed, 16 Sep 2015 03:38:03 +0000 (23:38 -0400)]
Fixing create CG from Cgsnapshot bug in VNX driver
When creating a consistency group from a cgsnapshot, VNX driver
didn't force a poll to the VNX system. This caused the creation
to fail because the out-of-date status was returned. This patch
forces a poll to the VNX system to get rid of this issue.
Kendall Nelson [Thu, 13 Aug 2015 15:17:36 +0000 (10:17 -0500)]
Dynamically create cinder.conf.sample
As it stands, the opts.py file that is passed into
oslo-config-generator isn't being generated dynamically
and the old way of generating the cinder.conf.sample is
dependent on oslo-incubator which Cinder is trying to
move away from. oslo-config-generator works differently
than oslo-incubator so a number of changes had to be made
in order to make this switch.
This patch adds the config directory to Cinder and in it
are two files:
-generate_cinder_opts.py that will take the
results of a grep command to create the opts.py
file to be passed into oslo-config-generator.
-cinder.conf which is the new configuration for
oslo-config-generator. The file is inside the config
directory to be consistent with other projects.
Some changes were made to the generate_sample.sh file in
order to give the base directories and target directories
to the generate_cinder_opts.py program.
tox.ini was edited to remove the checkonly option because
all that needs to happen in check_uptodate.sh is a check to
ensure that the cinder.conf.sample is actually being
generated with no issues.
All options were removed from the check_uptodate.sh
because they were unnecessary given the new, more simple
way of generating the cinder.conf.sample.
setup.cfg was also edited in order to add information
oslo-config-generator needs to run.
Co-Authored By: Jay Bryant <jsbryant@us.ibm.com>
Co-Authored By: Jacob Gregor <jgregor@us.ibm.com>
Ryan McNair [Thu, 27 Aug 2015 20:51:14 +0000 (20:51 +0000)]
Fix MITM vulnerability for Brocade FC SAN lookup
Refactor the Brocade FC SAN lookup service implementation to use
common SSH utilities which already avoid MITM vulnerabilities.
This is a follow-up to https://review.openstack.org/#/c/138526/
which reverted an incomplete fix for the MITM issues for Brocade.
Michael Price [Tue, 25 Aug 2015 22:36:44 +0000 (17:36 -0500)]
NetApp: Fix volume extend with E-Series
Thin provisioned volumes were being converted to thick provisioned
volumes by extend_volume. When thin provisioning support was added,
the extend_volume method was not updated to use the correct API
for performing the capacity expansion operation.
This patch changes the volume extension strategy when a volume is
thin provisioned to use the correct API. With a thick provisioned
volume, a clone will be created with the new capacity, and the old
volume removed, but a thin provisioned volume's capacity can simply
be expanded directly, because that is merely the capacity that the user
sees, as opposed to the backend provisioned capacity.
Michael Price [Wed, 2 Sep 2015 18:48:05 +0000 (13:48 -0500)]
Fix netapp_enable_multiattach default for E-Series
The E-Series driver erroneously changed the netapp_enable_multiattach
option to default to True. This option should default to False to
ensure backwards-compatibility with upgrades.
Michal Dulko [Tue, 15 Sep 2015 13:52:57 +0000 (15:52 +0200)]
Check for None on service's updated_at
We weren't checking if service's updated_at is None when passing it to
timeutils.normalize_time which caused an exception. This commit adds
the check to cinder-manage and cinder-api (services and hosts). Also
regression unit tests are added.
peter_wang [Fri, 21 Aug 2015 05:33:44 +0000 (01:33 -0400)]
Fix issue of volume after host-assisted migration
In host-assisted migration, Cinder framework currently doesn't
update name_id and provider_location when the driver doesn't provide
name_id or provider_location in update_migrated_volume
This changes previous Cinder behavior which will update name_id and
provider_location anyway. Any operation against this volume will not
be successful; for example, volume deletion cannot delete it's
underlying LUN.
This fix will update name_id and provider_location if driver doesn't
return them.
Mike Rooney [Mon, 31 Aug 2015 15:29:53 +0000 (11:29 -0400)]
Local img-cache files ignored for image transfers
The cDOT NFS driver will only attempt to use the local img-cache-*
files if the copy offload path is defined in the cinder.conf file.
This should be attempted regardless of copy-offload's configuration.
As per:
http://lists.openstack.org/pipermail/openstack-dev/2015-August/073338.html
The point behind the addition of the stevedore.sphinxext extension is
"to document drivers and other types of plugins to make the available
sets built into projects easier to discover" (dhellman).
The VMDK driver config options 'vmware_ca_file' and 'vmware_insecure'
are ignored while copying sparse or preallocated vmdk images to vCenter
datastores for volume creation. Therefore, the vCenter certificate is
not verified during image copy.
This patch sets appropriate parameters in the oslo.vmware API (for
copying images) call to enable vCenter certificate verification.
Tom Barron [Thu, 10 Sep 2015 23:53:29 +0000 (19:53 -0400)]
Fix NetApp clone from glance failure
When glance and cinder are backed by the same NetApp DOT flexvol
container, exposed as an NFS share, a direct clone is attempted
of the backing files rather than a copy. With the advent of
kilo 2015.1.1, this clone operation fails with the message:
"Image cloning unsuccessful ... need more than 2 values to unpack."
The issue here is that the clone path code is calling the parent
nfs class _is_share_eligible() method to see if there is enough
space on the destination flexvol and this method calls _get_capacity_info()
expecting a three-tuple to be returned. However, commit e52f304313efc695f7dd89c222041bffd53c131a deliberately changed the return
values of _get_capacity_info() within the NetApp drivers to return
a 2-tuple, based on API calls to obtain capacity information from
the NetApp storage arrays. That commit missed that this method
would also be invoked by the parent nfs class _is_share_eligible()
method, which expects a different signature.
The fix in this commit is to replace the call to the parent
_is_share_eligible() method with a more appropriate NetApp class
specific _share_has_space_for_clone() method. This new method
uses the NetApp version of _get_capacity_info to get space information
from the arrays via NetApp apis and expects a 2-tuple to be returned
from this method.
John Griffith [Sun, 13 Sep 2015 14:52:53 +0000 (08:52 -0600)]
Check for empty attributes on SF volume
This should not happen, but; we should be more strict
about our dict reading/checking and always make sure
the key item actually comes back and exists.
In the case of volume['attributes'] we were always
assuming this to be present which should be "ok", but
it's better to verify if rather than assuming, so this
patch adds a simple verification step before trying to
fetch on volume['attributes'].
Code Review / openstack-build / cinder-build.git / log