Michal Dulko [Tue, 15 Sep 2015 13:52:57 +0000 (15:52 +0200)]
Check for None on service's updated_at
We weren't checking if service's updated_at is None when passing it to
timeutils.normalize_time which caused an exception. This commit adds
the check to cinder-manage and cinder-api (services and hosts). Also
regression unit tests are added.
Mike Rooney [Mon, 31 Aug 2015 15:29:53 +0000 (11:29 -0400)]
Local img-cache files ignored for image transfers
The cDOT NFS driver will only attempt to use the local img-cache-*
files if the copy offload path is defined in the cinder.conf file.
This should be attempted regardless of copy-offload's configuration.
Tom Barron [Thu, 10 Sep 2015 23:53:29 +0000 (19:53 -0400)]
Fix NetApp clone from glance failure
When glance and cinder are backed by the same NetApp DOT flexvol
container, exposed as an NFS share, a direct clone is attempted
of the backing files rather than a copy. With the advent of
kilo 2015.1.1, this clone operation fails with the message:
"Image cloning unsuccessful ... need more than 2 values to unpack."
The issue here is that the clone path code is calling the parent
nfs class _is_share_eligible() method to see if there is enough
space on the destination flexvol and this method calls _get_capacity_info()
expecting a three-tuple to be returned. However, commit e52f304313efc695f7dd89c222041bffd53c131a deliberately changed the return
values of _get_capacity_info() within the NetApp drivers to return
a 2-tuple, based on API calls to obtain capacity information from
the NetApp storage arrays. That commit missed that this method
would also be invoked by the parent nfs class _is_share_eligible()
method, which expects a different signature.
The fix in this commit is to replace the call to the parent
_is_share_eligible() method with a more appropriate NetApp class
specific _share_has_space_for_clone() method. This new method
uses the NetApp version of _get_capacity_info to get space information
from the arrays via NetApp apis and expects a 2-tuple to be returned
from this method.
John Griffith [Sun, 13 Sep 2015 14:52:53 +0000 (08:52 -0600)]
Check for empty attributes on SF volume
This should not happen, but; we should be more strict
about our dict reading/checking and always make sure
the key item actually comes back and exists.
In the case of volume['attributes'] we were always
assuming this to be present which should be "ok", but
it's better to verify if rather than assuming, so this
patch adds a simple verification step before trying to
fetch on volume['attributes'].
John Griffith [Fri, 11 Sep 2015 19:57:32 +0000 (19:57 +0000)]
Fix volume lookups in SolidFire template caching
There's a number of issues with the SolidFire template caching
code. There are a number of metadata items that aren't consistent
between Glance API V1 and V2, there's also some problems with how
the code is dealing with it's internal volume and just flat out
wrong account checking.
This patch focuses mainly on fixing up the account and volumeID
issues. The _do_clone method currently expects to find the src
volume under the same account as the dest volume. That's fine
for most cases, but doesn't work for either the SolidFire template
caching, or for the use of generic image caching in Cinder.
We fix that easily by modifying the _get_sf_volume call to have an
option to search all active volumes on the Cluster instead of just
those for a specified account. For now we only change the clone
methods, as they're the only case where we fall into the case of
cross-ownership of volumes in the same context.
The other issue is around the cleanup/delete in the case of a
failed template-volume create. In this case we were passing in
the Cinder UUID to a direct SolidFire API delete cmd, which
is wrong. The direct call for the SolidFire API call requires
that the actual SolidFire ID be used. So we leverage the modified
_get_sf_volume call here as well.
Another issue that was introduced was the introduction of
a connector object to the intialize_connection call. We
got around this once already by just passing in a fake
connector that was actually a string. That was fine, but
the initialize_connection routine also now checks the connector
for multi_attach, so it fails the _get call on a string. We
fix that by just building a dict and using that instead of
a fake string.
Finally, the method of relying on the Glance metadata virt size
has been problematic, it's not set by default, it's not accurate
and it's a bit inefficient. There have been a number of issues
with this method of size determination, so we'll dump that and
just leverage the image_utils methods directly.
John Griffith [Fri, 11 Sep 2015 06:21:40 +0000 (06:21 +0000)]
Don't rely on provider_id for resource deletion
Provider_id info isn't necessarily reliable when dealing
with failure cases. There are a number of cases where the
info in the Cinder database may not be updated.
This change uses the old delete routines instead of trying
to use the provider_id info. There are ways to do both,
however it's probably not that beneficial for now.
Patrick East [Fri, 11 Sep 2015 21:12:03 +0000 (14:12 -0700)]
Fix Pure get pgroup volume snapshot name
When creating a volume from a consistency group snapshot we have to go
look up the volume level snapshot from the group. Previously we could
just directly get to the cgsnapshot model from the snapshot. After we
switched to using the snapshot objects however we cannot do that (at
least not until we have cgsnapshot objects too).
As an interim solution we did a query to the REST API on the array to
just list all of the snapshots. Unfortunately that doesn’t work on all
versions of the Purity REST API. To have a more usable solution we need
to do a db lookup to get the pgsnapshot.
It’s not ideal, and will be removed ASAP, but for now we need to at
least not regress in functionality.
This patch fixes invalid access to the options that live
in the dothill_common. They were recently renamed from
common_opt to common_opts, but the drivers weren't updated.
This patch fixes access to the options defined in
hpmsa_common. They were recently renamed from common_opt
to common_opts, but the drivers weren't updated.
The VMDK driver configuration uses invalid product name "VC"
instead of "vCenter". This patch replaces "VC" with "vCenter"
in help messages and code documentation. This change is
consistent with the rest of driver documentation.
DocImpact
Replaced "VC" with "vCenter" in driver config option help
messages
When transferring db volume module through RPC call, it becomes
dict objects during RPC call. The attributes/properties
of the type can't be used.
As a result, to get volume['name'] raises KeyError.
The issue is fixed by retrieving volume from db.
Using volume version object will fix everything but it is
too late in Liberty. To keep code consistency, db.get_volume
is used in the fix.
Xing Yang [Thu, 10 Sep 2015 19:31:39 +0000 (15:31 -0400)]
Add ScaleIO Cinder driver commands
ScaleIO Cinder driver commands are currently missing from
Cinder's rootwrap filters. Since we are not using os-brick's
rootwrap filters in Liberty, these missing commands need to be
added to Cinder's rootwrap filters.
John Griffith [Thu, 10 Sep 2015 00:59:38 +0000 (00:59 +0000)]
SolidFire provider_id for snapshots on init
We've added the provider_id field to the database for both
volumes and snapshots. This however couldn't be used because
there was no upgrade/migration path included for backends to
populate these fields (ie upgrade from K to L).
This patch fixes the provider_id columns for snapshots and volumes
both, and updates these fields on backend init. This fixes the
issue of not being able to "rely" on the database entries, and it
also fixes some issues with inefficiencies in how we were deleting
resources.
Now we can just use the provider_id info from the reference passed
in on the delete calls.
Anthony Lee [Tue, 8 Sep 2015 22:29:17 +0000 (15:29 -0700)]
LeftHand Add update_migrated_volume to drivers
The LeftHand drivers were missing the method
update_migrated_volume, which was causing the volume IDs
to no longer be accurate after a volume was migrated into
a LeftHand array.
zhaohua [Wed, 19 Aug 2015 07:57:48 +0000 (15:57 +0800)]
Huawei: fix multi REST-URLs bug
Huawei driver now supports multi REST-URLs, which means users can
write more than one REST-URLs in the config file(seperated by ";")
and driver can choose the usable one to set up a connection(see
function "login" in rest_client.py of Huawei driver).
But if the chosen REST-URL has some failures, Huawei driver can
not change to another REST-URL automatically.
To resolve this, rewrite the "call" function in rest_client.py of
Huawei driver. If REST command failed to send, driver will do a
"login" process to set up another connection, and send the failed
command to a new REST-URL again.
And change REST command timeout to 52s, change login timeout to 4s,
to avoid long time waiting for failure.
Tom Barron [Mon, 24 Aug 2015 17:21:48 +0000 (13:21 -0400)]
Fix NetApp loop in clone of NFS backed images
If there is a list of locations for glance images and an unsuitable
location is encountered when iterating through the list, current code
breaks out of the loop, failing to find later, valid locations.
Here we fix this issue by ignoring invalid locations and continuing
through the loop rather than breaking out when we find invalid locations.
Tom Barron [Wed, 26 Aug 2015 20:07:00 +0000 (16:07 -0400)]
backup init_host cleanup exception handling
Current cleanup of leftover backups, volumes, and temporary
volumes and snapshots is not resilient in the face of exceptions
such as failures in detach operations.
This commit adds appropriate try/except logic to this section
of code, thereby ensuring that the backup service can start even
if exceptions are encountered when attempting to cleanup detritus
from earlier failed backup operations.
Kendall Nelson [Wed, 9 Sep 2015 13:21:25 +0000 (08:21 -0500)]
Making opt names consistent
Wherever there are lists of options being registered, the name of
the list needs to be plural to be consistent with all other opt lists
being registered elsewhere. This change is necessary for the dynamic
generation of cinder.sample.conf.
Co-Authored-By: Jacob Gregor <jgregor@us.ibm.com>
Change-Id: I1434a195016a2a595d0205d73a78a60cd94137f8
Anthony Lee [Wed, 19 Aug 2015 22:48:33 +0000 (15:48 -0700)]
Fix QoS keys not being available to scheduler
There is a qos_specs property contained within the properties
passed to filters and weighers during volume placement decision
time. Currently qos_specs is always set to None even when a
QoS spec is associated with a volume type.
This patch exposes the QoS keys by placing them in the
qos_specs property. When no QoS spec is associated with a volume
the qos_specs property would still be set to None.
Michal Dulko [Wed, 9 Sep 2015 15:20:18 +0000 (17:20 +0200)]
Add ConsistencyGroup object entries to linstack.py
Change I3febb95bb2b3dd8f10da4f01ffa705e39cb20520 wasn't passing pylint
gate because of missing objects_ignore_messages entries in linstack.py.
This commits adds these entries.
John Griffith [Wed, 9 Sep 2015 15:15:31 +0000 (15:15 +0000)]
Pass in snapshot refs for host on provider_update
Now that snapshot_get_by_host is fixed we can and should
pass in a list of snapshot-refs to the drivers update_provider
method (just like we do with volumes).
John Griffith [Tue, 8 Sep 2015 21:29:05 +0000 (15:29 -0600)]
Return a tuple from SolidFire update_provider_info
The SolidFire update_provider_info is only returning
a list of updated volumes, but the manager call (and
the method def in the reference) calls for a tuple of
volumes and snapshots.
This patch fixes the return to be a tuple, a follow on
will be needed to add snapshot updates.
Eric Harney [Tue, 8 Sep 2015 20:23:45 +0000 (16:23 -0400)]
Add unmanage default implementation to VolumeDriver
Currently, only drivers that inherit from ManageableVD
get the default unmanage() implementation.
This is not correct, because drivers can support unmanage
without driver code. Fix unmanage() in VolumeDriver to
support the LVM driver's unmanage operation.
(Regression from the refactoring done when introducing
abc classes.)
Eric Harney [Tue, 8 Sep 2015 17:31:31 +0000 (13:31 -0400)]
Add migrate_volume to BaseVD
migrate_volume() only exists in the deprecated VolumeDriver()
class and not BaseVD. This means that drivers inheriting from
*VD hit attribute errors when trying to call driver.migrate_volume().
I suspect this indicates that we should rework some of our
abc class structure a little bit, but this seems like a
reasonable fix for now.
This patch disables the 3PAR drivers if an admin tries to
enable the generic image volume cache capability.
The 3PAR drivers don't work with this currently because of the way volume
cloning works in the drivers. When a volume is cloned, it's done so
in the background, which makes the clone available immediately for attaching.
The problem is, a background cloning also locks the original volume from
being modified. The generic image cache does a clone, and then an extend
volume, which fails for the 3PAR drivers currently.
Sean McGinnis [Sat, 5 Sep 2015 18:04:05 +0000 (13:04 -0500)]
Add os-brick's scsi_id command to Cinder rootwrap
FC drivers are failing on calls to scsi_id. This command
is in the os-brick rootwrap filters, but that file is not
being pulled in to Cinder yet.
As a temporary fix until that work can be completed, this
patch adds the missing command directly to volume.filters
to allow this command to run. Follow up work will be needed
to remove this and other legacy brick commands from the
volume.filters file and instead pull in os-brick's
os-brick.filters values.
chenying [Tue, 25 Aug 2015 02:35:37 +0000 (10:35 +0800)]
Fix order of arguments in assertEqual
Some tests used incorrect order assertEqual(observed, expected).
The correct order expected by testtools is
assertEqual(expected, observed).
This patch does not cover v1 api tests and vendor unit tests.
Code Review / openstack-build / cinder-build.git / log