John Griffith [Fri, 11 Sep 2015 06:21:40 +0000 (06:21 +0000)]
Don't rely on provider_id for resource deletion
Provider_id info isn't necessarily reliable when dealing
with failure cases. There are a number of cases where the
info in the Cinder database may not be updated.
This change uses the old delete routines instead of trying
to use the provider_id info. There are ways to do both,
however it's probably not that beneficial for now.
Patrick East [Fri, 11 Sep 2015 21:12:03 +0000 (14:12 -0700)]
Fix Pure get pgroup volume snapshot name
When creating a volume from a consistency group snapshot we have to go
look up the volume level snapshot from the group. Previously we could
just directly get to the cgsnapshot model from the snapshot. After we
switched to using the snapshot objects however we cannot do that (at
least not until we have cgsnapshot objects too).
As an interim solution we did a query to the REST API on the array to
just list all of the snapshots. Unfortunately that doesn’t work on all
versions of the Purity REST API. To have a more usable solution we need
to do a db lookup to get the pgsnapshot.
It’s not ideal, and will be removed ASAP, but for now we need to at
least not regress in functionality.
This patch fixes invalid access to the options that live
in the dothill_common. They were recently renamed from
common_opt to common_opts, but the drivers weren't updated.
This patch fixes access to the options defined in
hpmsa_common. They were recently renamed from common_opt
to common_opts, but the drivers weren't updated.
The VMDK driver configuration uses invalid product name "VC"
instead of "vCenter". This patch replaces "VC" with "vCenter"
in help messages and code documentation. This change is
consistent with the rest of driver documentation.
DocImpact
Replaced "VC" with "vCenter" in driver config option help
messages
When transferring db volume module through RPC call, it becomes
dict objects during RPC call. The attributes/properties
of the type can't be used.
As a result, to get volume['name'] raises KeyError.
The issue is fixed by retrieving volume from db.
Using volume version object will fix everything but it is
too late in Liberty. To keep code consistency, db.get_volume
is used in the fix.
Xing Yang [Thu, 10 Sep 2015 19:31:39 +0000 (15:31 -0400)]
Add ScaleIO Cinder driver commands
ScaleIO Cinder driver commands are currently missing from
Cinder's rootwrap filters. Since we are not using os-brick's
rootwrap filters in Liberty, these missing commands need to be
added to Cinder's rootwrap filters.
John Griffith [Thu, 10 Sep 2015 00:59:38 +0000 (00:59 +0000)]
SolidFire provider_id for snapshots on init
We've added the provider_id field to the database for both
volumes and snapshots. This however couldn't be used because
there was no upgrade/migration path included for backends to
populate these fields (ie upgrade from K to L).
This patch fixes the provider_id columns for snapshots and volumes
both, and updates these fields on backend init. This fixes the
issue of not being able to "rely" on the database entries, and it
also fixes some issues with inefficiencies in how we were deleting
resources.
Now we can just use the provider_id info from the reference passed
in on the delete calls.
Anthony Lee [Tue, 8 Sep 2015 22:29:17 +0000 (15:29 -0700)]
LeftHand Add update_migrated_volume to drivers
The LeftHand drivers were missing the method
update_migrated_volume, which was causing the volume IDs
to no longer be accurate after a volume was migrated into
a LeftHand array.
Tom Barron [Mon, 24 Aug 2015 17:21:48 +0000 (13:21 -0400)]
Fix NetApp loop in clone of NFS backed images
If there is a list of locations for glance images and an unsuitable
location is encountered when iterating through the list, current code
breaks out of the loop, failing to find later, valid locations.
Here we fix this issue by ignoring invalid locations and continuing
through the loop rather than breaking out when we find invalid locations.
Tom Barron [Wed, 26 Aug 2015 20:07:00 +0000 (16:07 -0400)]
backup init_host cleanup exception handling
Current cleanup of leftover backups, volumes, and temporary
volumes and snapshots is not resilient in the face of exceptions
such as failures in detach operations.
This commit adds appropriate try/except logic to this section
of code, thereby ensuring that the backup service can start even
if exceptions are encountered when attempting to cleanup detritus
from earlier failed backup operations.
Kendall Nelson [Wed, 9 Sep 2015 13:21:25 +0000 (08:21 -0500)]
Making opt names consistent
Wherever there are lists of options being registered, the name of
the list needs to be plural to be consistent with all other opt lists
being registered elsewhere. This change is necessary for the dynamic
generation of cinder.sample.conf.
Co-Authored-By: Jacob Gregor <jgregor@us.ibm.com>
Change-Id: I1434a195016a2a595d0205d73a78a60cd94137f8
Michal Dulko [Wed, 9 Sep 2015 15:20:18 +0000 (17:20 +0200)]
Add ConsistencyGroup object entries to linstack.py
Change I3febb95bb2b3dd8f10da4f01ffa705e39cb20520 wasn't passing pylint
gate because of missing objects_ignore_messages entries in linstack.py.
This commits adds these entries.
John Griffith [Wed, 9 Sep 2015 15:15:31 +0000 (15:15 +0000)]
Pass in snapshot refs for host on provider_update
Now that snapshot_get_by_host is fixed we can and should
pass in a list of snapshot-refs to the drivers update_provider
method (just like we do with volumes).
John Griffith [Tue, 8 Sep 2015 21:29:05 +0000 (15:29 -0600)]
Return a tuple from SolidFire update_provider_info
The SolidFire update_provider_info is only returning
a list of updated volumes, but the manager call (and
the method def in the reference) calls for a tuple of
volumes and snapshots.
This patch fixes the return to be a tuple, a follow on
will be needed to add snapshot updates.
Eric Harney [Tue, 8 Sep 2015 20:23:45 +0000 (16:23 -0400)]
Add unmanage default implementation to VolumeDriver
Currently, only drivers that inherit from ManageableVD
get the default unmanage() implementation.
This is not correct, because drivers can support unmanage
without driver code. Fix unmanage() in VolumeDriver to
support the LVM driver's unmanage operation.
(Regression from the refactoring done when introducing
abc classes.)
Eric Harney [Tue, 8 Sep 2015 17:31:31 +0000 (13:31 -0400)]
Add migrate_volume to BaseVD
migrate_volume() only exists in the deprecated VolumeDriver()
class and not BaseVD. This means that drivers inheriting from
*VD hit attribute errors when trying to call driver.migrate_volume().
I suspect this indicates that we should rework some of our
abc class structure a little bit, but this seems like a
reasonable fix for now.
This patch disables the 3PAR drivers if an admin tries to
enable the generic image volume cache capability.
The 3PAR drivers don't work with this currently because of the way volume
cloning works in the drivers. When a volume is cloned, it's done so
in the background, which makes the clone available immediately for attaching.
The problem is, a background cloning also locks the original volume from
being modified. The generic image cache does a clone, and then an extend
volume, which fails for the 3PAR drivers currently.
Sean McGinnis [Sat, 5 Sep 2015 18:04:05 +0000 (13:04 -0500)]
Add os-brick's scsi_id command to Cinder rootwrap
FC drivers are failing on calls to scsi_id. This command
is in the os-brick rootwrap filters, but that file is not
being pulled in to Cinder yet.
As a temporary fix until that work can be completed, this
patch adds the missing command directly to volume.filters
to allow this command to run. Follow up work will be needed
to remove this and other legacy brick commands from the
volume.filters file and instead pull in os-brick's
os-brick.filters values.
Jacob Gregor [Fri, 4 Sep 2015 21:30:12 +0000 (16:30 -0500)]
Error message in update_migrated_volume was incorrect
If an error was encountered in self.vg.rename_volume the error
message tried to use volume['name'], which is an invalid key.
The invalid key error would then cause the retype/migration to
abort in the middle of the process and leave original and temporary
volume in place. This patch fixes this bug by providing a valid key.
Fix HDS HNAS driver logging password as plain text
HDS HNAS driver logs every command that is sent to HNAS. Some
commands need the HNAS password, so the driver ends up logging
this password as plain text. This patch changes these commands
syntax so that oslo_utils.strutils.mask_password can successfully
mask this password.
John Griffith [Wed, 2 Sep 2015 20:08:49 +0000 (20:08 +0000)]
Add mechanism to update snapshot provider_id
During the L release we added the ability to store
provider_id info for volumes and snapshots. We did
not however add a mechanism for this info to be updated
on upgrades.
This patch adds on to the volume update mechanism and
gives a driver the opportunity to set provider_id columns
that aren't set on snapshots just as it does for volumes.
Commit 212aff327a36f925be3be69b9b54b946dbdd5c2a replaced
'cinder.openstack.common.fileutils.file_open' with 'open'
from builtins. It also updated unit tests to patch 'six.
moves.builtins.open' instead of 'cinder.openstack.common.
fileutils.file_open'. This resulted in intermittent failure
of test 'test_create_backing_from_stream_optimized_file'
in VMwareVcVmdkDriverTestCase. The failure happens because
the test patches 'open' from builtins and tests code which
throws an exception in a 'with open(...)' block. This
exception is handled in excutils.save_and_reraise_exception
which uses 'with open(...)' and the exception handling
fails due to the global patching of 'open'.
This patch replaces the global patching of builtins 'open'
with a patch decorator which binds 'open' to the namespace
of the code under testing.
Volume creation will fail if the VMDK driver selects
an ESX host in maintenance mode for volume creation.
This may happen even if there are usable ESX hosts
in vCenter. This patch fixes the problem by ignoring
ESX hosts in maintenance mode during volume creation.
The 3PAR drivers were missing the method
update_migrated_volume, which would then cause
all sorts of problems after a volume was migrated
into a 3PAR array.
Switch SVC driver to use lsportfc to determine FC target WWPNS
Switched the startup code within SVC driver to use the
SVC lsportfc command rather than the lsnode command to determine the
target WWPNs for an SVC node, and updated SVC testcases.
The SVC lsportfc command returns all of the FC ports so that
all of the FC ports can be zoned and thus utilized by
the VM that a volume is attached to.
Where the lsnode command is documented to only report the
first 4 FC ports.
Patrick East [Wed, 2 Sep 2015 20:58:02 +0000 (13:58 -0700)]
Use consolidated update for failover_replication
Right now it’s using a special built volume_updates dictionary that
contains some fields *and* a model update. We then glom all of them
together into an update dictionary for the volume model.
This change simplifies them into a single model update more in-line with
how the other volume API’s work for drivers.
It also fixes a bug where the model_update property of the volume_update
was not being applied to the db model.
Currently some of the exceptions defined in the VMware driver
emit unformatted error messages. This is because they are
not overriding the correct message format variable. This
patch fixes the issue by overriding the correct variable
defined in the exception super class.
Edmund Rhudy [Thu, 27 Aug 2015 17:49:58 +0000 (13:49 -0400)]
Adds allow_availability_zone_fallback option to Cinder
This change adds allow_availability_zone_fallback, which allows Cinder
to fall back to an already configured availability zone if the
requested availability zone is unavailable.
Michael Price [Sun, 16 Aug 2015 14:58:41 +0000 (10:58 -0400)]
NetApp E-Series over-subscription support
Add support to the E-Series driver for
over-subscription/overprovisioning. This allows the Cinder scheduler
to more accurately represent the remaining space on a pool when
thin provisioned Cinder volumes are in use on an E-Series backend.
Co-Authored-By: Tom Barron <tpb@dyncloud.net>
DocImpact
Partially-implements: blueprint netapp-block--over-subscription-thin-provisioning
Change-Id: I5fb8f480168b9195221412d70f187c5ddfc4f527
Abhiram Moturi [Mon, 10 Aug 2015 14:23:09 +0000 (14:23 +0000)]
ZFSSA driver to return project 'available' space
This fix allows the iSCSI driver to return the 'available' space
property at project level instead of the pool level which is more
accurate in cases when storage is not thin provisioned.
Code Review / openstack-build / cinder-build.git / log