Tom Barron [Mon, 24 Aug 2015 17:21:48 +0000 (13:21 -0400)]
Fix NetApp loop in clone of NFS backed images
If there is a list of locations for glance images and an unsuitable
location is encountered when iterating through the list, current code
breaks out of the loop, failing to find later, valid locations.
Here we fix this issue by ignoring invalid locations and continuing
through the loop rather than breaking out when we find invalid locations.
Kendall Nelson [Wed, 9 Sep 2015 13:21:25 +0000 (08:21 -0500)]
Making opt names consistent
Wherever there are lists of options being registered, the name of
the list needs to be plural to be consistent with all other opt lists
being registered elsewhere. This change is necessary for the dynamic
generation of cinder.sample.conf.
Co-Authored-By: Jacob Gregor <jgregor@us.ibm.com>
Change-Id: I1434a195016a2a595d0205d73a78a60cd94137f8
John Griffith [Wed, 9 Sep 2015 15:15:31 +0000 (15:15 +0000)]
Pass in snapshot refs for host on provider_update
Now that snapshot_get_by_host is fixed we can and should
pass in a list of snapshot-refs to the drivers update_provider
method (just like we do with volumes).
John Griffith [Tue, 8 Sep 2015 21:29:05 +0000 (15:29 -0600)]
Return a tuple from SolidFire update_provider_info
The SolidFire update_provider_info is only returning
a list of updated volumes, but the manager call (and
the method def in the reference) calls for a tuple of
volumes and snapshots.
This patch fixes the return to be a tuple, a follow on
will be needed to add snapshot updates.
Eric Harney [Tue, 8 Sep 2015 20:23:45 +0000 (16:23 -0400)]
Add unmanage default implementation to VolumeDriver
Currently, only drivers that inherit from ManageableVD
get the default unmanage() implementation.
This is not correct, because drivers can support unmanage
without driver code. Fix unmanage() in VolumeDriver to
support the LVM driver's unmanage operation.
(Regression from the refactoring done when introducing
abc classes.)
This patch disables the 3PAR drivers if an admin tries to
enable the generic image volume cache capability.
The 3PAR drivers don't work with this currently because of the way volume
cloning works in the drivers. When a volume is cloned, it's done so
in the background, which makes the clone available immediately for attaching.
The problem is, a background cloning also locks the original volume from
being modified. The generic image cache does a clone, and then an extend
volume, which fails for the 3PAR drivers currently.
Sean McGinnis [Sat, 5 Sep 2015 18:04:05 +0000 (13:04 -0500)]
Add os-brick's scsi_id command to Cinder rootwrap
FC drivers are failing on calls to scsi_id. This command
is in the os-brick rootwrap filters, but that file is not
being pulled in to Cinder yet.
As a temporary fix until that work can be completed, this
patch adds the missing command directly to volume.filters
to allow this command to run. Follow up work will be needed
to remove this and other legacy brick commands from the
volume.filters file and instead pull in os-brick's
os-brick.filters values.
Jacob Gregor [Fri, 4 Sep 2015 21:30:12 +0000 (16:30 -0500)]
Error message in update_migrated_volume was incorrect
If an error was encountered in self.vg.rename_volume the error
message tried to use volume['name'], which is an invalid key.
The invalid key error would then cause the retype/migration to
abort in the middle of the process and leave original and temporary
volume in place. This patch fixes this bug by providing a valid key.
Fix HDS HNAS driver logging password as plain text
HDS HNAS driver logs every command that is sent to HNAS. Some
commands need the HNAS password, so the driver ends up logging
this password as plain text. This patch changes these commands
syntax so that oslo_utils.strutils.mask_password can successfully
mask this password.
John Griffith [Wed, 2 Sep 2015 20:08:49 +0000 (20:08 +0000)]
Add mechanism to update snapshot provider_id
During the L release we added the ability to store
provider_id info for volumes and snapshots. We did
not however add a mechanism for this info to be updated
on upgrades.
This patch adds on to the volume update mechanism and
gives a driver the opportunity to set provider_id columns
that aren't set on snapshots just as it does for volumes.
Commit 212aff327a36f925be3be69b9b54b946dbdd5c2a replaced
'cinder.openstack.common.fileutils.file_open' with 'open'
from builtins. It also updated unit tests to patch 'six.
moves.builtins.open' instead of 'cinder.openstack.common.
fileutils.file_open'. This resulted in intermittent failure
of test 'test_create_backing_from_stream_optimized_file'
in VMwareVcVmdkDriverTestCase. The failure happens because
the test patches 'open' from builtins and tests code which
throws an exception in a 'with open(...)' block. This
exception is handled in excutils.save_and_reraise_exception
which uses 'with open(...)' and the exception handling
fails due to the global patching of 'open'.
This patch replaces the global patching of builtins 'open'
with a patch decorator which binds 'open' to the namespace
of the code under testing.
Volume creation will fail if the VMDK driver selects
an ESX host in maintenance mode for volume creation.
This may happen even if there are usable ESX hosts
in vCenter. This patch fixes the problem by ignoring
ESX hosts in maintenance mode during volume creation.
The 3PAR drivers were missing the method
update_migrated_volume, which would then cause
all sorts of problems after a volume was migrated
into a 3PAR array.
Switch SVC driver to use lsportfc to determine FC target WWPNS
Switched the startup code within SVC driver to use the
SVC lsportfc command rather than the lsnode command to determine the
target WWPNs for an SVC node, and updated SVC testcases.
The SVC lsportfc command returns all of the FC ports so that
all of the FC ports can be zoned and thus utilized by
the VM that a volume is attached to.
Where the lsnode command is documented to only report the
first 4 FC ports.
Patrick East [Wed, 2 Sep 2015 20:58:02 +0000 (13:58 -0700)]
Use consolidated update for failover_replication
Right now it’s using a special built volume_updates dictionary that
contains some fields *and* a model update. We then glom all of them
together into an update dictionary for the volume model.
This change simplifies them into a single model update more in-line with
how the other volume API’s work for drivers.
It also fixes a bug where the model_update property of the volume_update
was not being applied to the db model.
Currently some of the exceptions defined in the VMware driver
emit unformatted error messages. This is because they are
not overriding the correct message format variable. This
patch fixes the issue by overriding the correct variable
defined in the exception super class.
Edmund Rhudy [Thu, 27 Aug 2015 17:49:58 +0000 (13:49 -0400)]
Adds allow_availability_zone_fallback option to Cinder
This change adds allow_availability_zone_fallback, which allows Cinder
to fall back to an already configured availability zone if the
requested availability zone is unavailable.
Michael Price [Sun, 16 Aug 2015 14:58:41 +0000 (10:58 -0400)]
NetApp E-Series over-subscription support
Add support to the E-Series driver for
over-subscription/overprovisioning. This allows the Cinder scheduler
to more accurately represent the remaining space on a pool when
thin provisioned Cinder volumes are in use on an E-Series backend.
Co-Authored-By: Tom Barron <tpb@dyncloud.net>
DocImpact
Partially-implements: blueprint netapp-block--over-subscription-thin-provisioning
Change-Id: I5fb8f480168b9195221412d70f187c5ddfc4f527
Abhiram Moturi [Mon, 10 Aug 2015 14:23:09 +0000 (14:23 +0000)]
ZFSSA driver to return project 'available' space
This fix allows the iSCSI driver to return the 'available' space
property at project level instead of the pool level which is more
accurate in cases when storage is not thin provisioned.
Patrick East [Wed, 2 Sep 2015 20:51:42 +0000 (13:51 -0700)]
Get full volume model in Replication manager API’s
Currently we pass in a dictionary of the volume properties, but not the
actual volume model. This dictionary is missing some of the properties
like ‘name’ which backends may required for volume operations.
Fix problem of efficient volume copy for migration
After introducing commit f586043fa969b9d1dcf4933aacbf615f53691093,
new volume copy method _copy_volume_data() was added into
manager.py. Originally driver.py had this method and it was copied
into manager.py. However new _copy_volume_data() lost efficient
volume copy logic during the reimplementation.
This patch simply add efficient volume copy logic again into new
_copy_volume_data() to fix the problem.
Patrick East [Wed, 8 Jul 2015 00:23:58 +0000 (17:23 -0700)]
Generic image-volume cache
This introduces a new feature for backends to be able to use cached
glance images when creating volumes from images.
If enabled, and the cinder internal tenant is configured, it will
create a minimal sized clone of a volume the first time it is
downloaded from glance. A new db table ‘image_volume_cache’ tracks
these volumes, which are owned by the Cinder internal tenant. The
original will then be extended to full size. Any subsequent requests to
create a volume from an image will be able to do a volume clone from
the image-volume cache entry instead of downloading the image data from
glance again.
In the future we could create an entry upon creating an image from a
volume right before uploading the volume to glance. This version
however does not aim to do more than add the initial framework and
help with subsequent image downloads. There are certainly more
improvements that can be done over time on top of this.
These image-volumes are host specific, so each backend may end up with
its very own image-volume to do clones from.
The cache is limited in size by number of entries allowed and size in
gb. When creating a new entry if space is needed the last used entries
will be evicted to make room.
In the current implementation the image-volumes will be viewable by a
cloud admin in the volume list, and show up as owned by the Cinder
internal tenant. They can safely be deleted manually by an admin, this
will cause the entry to be removed from the cache. These volumes will
use quota for the internal tenant.
Cache actions will post notifications to Ceilometer. There are
notifications like ‘image_volume_cache.miss’, ‘image_volume_cache.hit’,
and ‘image_volume_cache.evict’. A change will be required to the
event_definitions.yaml to see them nicely. Until then you only need to
add a new event type 'image_volume_cache.*' and look for the ‘image_id’
and ‘host’ fields.
DocImpact: We probably want some instructions on restrictions of the
cache and how to use it. The three new config options should also be
documented somewhere: ‘image_volume_cache_enabled’,
’image_volume_cache_max_size_gb’, ‘image_volume_cache_max_size_count’
Michael Price [Wed, 5 Aug 2015 20:36:59 +0000 (15:36 -0500)]
Implement thin provisioning support for E-Series
Implement a new extra spec, 'netapp_thin_provisioned', that will allow
users to define thin provisioned Cinder volumes on E-Series storage,
alongside pre-existing extra specs such as
'netapp_eseries_data_assurance', 'netapp_eseries_flash_reach_cache',
'netapp_raid_type', etc.
We have a followup patch, https://review.openstack.org/#/c/215801/ ,
that reports 'thin_provisioning_support=True/False' and same for
'thick_provisioning_support', in accord with the scheduler-based
over-subscription support added in Kilo.
We are not yet attempting to implement the get_capabilities()
feature just merged into Liberty
(https://review.openstack.org/#/c/201243/)
but fully intend to do so in a way that conforms with that plan of
record.
John Griffith [Wed, 2 Sep 2015 15:56:15 +0000 (15:56 +0000)]
Remove useless response checks in SolidFire driver
The main issue_api_request method in the SolidFire driver
has had a mechanism to check validity of response data for
some time now. It checks the response and if there isn't
a valid result in the response it raises.
The problem is that we still have some cruft from the old
code that does local checking/verfication. These checks
won't ever actually be called because of the exception, and
they create some confusion.
This patch just removes those useless checks and cleans up
the api_request calls a bit.
Tina [Thu, 6 Aug 2015 03:11:13 +0000 (23:11 -0400)]
Fix the virtual port support in VNX driver
VNX support virtual port on each physical iSCSI port. The
driver used some incorrect command which cannot get the virtual
port id information. This causes the driver to not be able to
tell whether the registered port is a virtaul port or not.
This change fixes the driver to use correct command and handle
the virtual port id correctly.
This patch changes how the Dothill driver finds the
parent volume id during create_cloned_volume time.
The proper place to get it is from the source volume
dict/object being passed in.
This patch fixes where we are getting the source volume id
from. Previously we were reading the source volume id from
the destination volume object in volume['source_volid']. There are
cases where this value doesn't exist. We are now reading the value
from the src_vref['id'] instead.
The CloudByte driver is reading the parent's volume id from the
new volume's source_volid. There are some cases when this isn't set.
The source volume object/dict is passed in, so just access the id
from there.
Tom Barron [Mon, 17 Aug 2015 18:08:28 +0000 (14:08 -0400)]
Scheduler-based over-subscription for NFS drivers
Add support to the NFS drivers for over-subscription/overprovisioning.
This allows the Cinder scheduler to more accurately represent the
remaining space on a pool when thin-provisioned Cinder Volumes are in
use on an NFS backend.
DocImpact Co-Authored-By: Clinton Knight <cknight@netapp.com> Co-Authored-By: Mike Rooney <rooneym@netapp.com>
Implements: blueprint nfs-over-subscription-thin-provisioning
Xing Yang [Wed, 19 Aug 2015 02:16:03 +0000 (22:16 -0400)]
ScaleIO driver should use os-brick connector
Currently the ScaleIO Cinder volume driver contains the connect_volume logic
inside the driver itself. This code is redundant with the code in the os-brick
connector. This patch modifies the driver to use the os-brick connector and
removes the redundant code.
Thang Pham [Fri, 28 Aug 2015 14:02:50 +0000 (07:02 -0700)]
Sync volume versionedobject to ORM
The following patch syncs the volume object to match the
ORM. More specifically, it adds metadata, admin_metdata,
volume_attachment, and volume_type to the volume object.
These fields are required to convert cinder internals to
use volume versionedobject. It was originally part of
a large patch, but is broken up to make reviews more
bearable.
When importing backup metadata only required fields were being imported,
and all other fields were being ignored.
This led to some serious problems when importing backups back, since
you would lose not only the original Volume ID, but also relevant parent
id information needed to restore incremental backups.
This patch fixes this by importing everything back and creating backup
records with the right ID.
Code Review / openstack-build / cinder-build.git / log