John Griffith [Wed, 20 Feb 2013 04:34:01 +0000 (21:34 -0700)]
Handle maxclonepervolume/node limits in SF driver.
The SolidFire cluster has a max simultaneous clones per volume and node
limt that we weren't accounting for. Add some logic to detect this situation
and loop until enough simultaneous clones are completed to allow processing
of the next one in line.
Mark McLoughlin [Sun, 10 Feb 2013 23:55:24 +0000 (18:55 -0500)]
Use oslo-config-2013.1b3
The cfg API is now available via the oslo-config library, so switch to
it and remove the copied-and-pasted version.
Add the 2013.1b3 tarball to tools/pip-requires - this will be changed
to 'oslo-config>=2013.1' when oslo-config is published to pypi. This
will happen in time for grizzly final.
Add dependency_links to setup.py so that oslo-config can be installed
from the tarball URL specified in pip-requires.
Remove the 'deps = pep8==1.3.3' from tox.ini as it means all the other
deps get installed with easy_install which can't install oslo-config
from the URL.
Retain dummy cfg.py file until keystoneclient middleware has been
updated (I18c450174277c8e2d15ed93879da6cd92074c27a).
Kurt Martin [Wed, 13 Feb 2013 21:55:43 +0000 (13:55 -0800)]
HP 3PAR Fibre Channel Driver and iSCSI Updates
This is the initial FC driver for the HP 3PAR array, along with some
3PAR iSCSI driver cleanup and changes in the HP 3PAR common class.
Summary of changes:
* Initial FC Driver for the Grizzly release (depends on the FC base class
and nova FC changes https://review.openstack.org/#/c/19992/ &
https://review.openstack.org/#/c/20003/)
* Updates to iSCSI Driver include volume type and create clone volume support
as well as general cleanup(comments, copyright, minor bug fixes).
* The tests have been updated to add FC test and the file was renamed
to reflect both FC and iSCSI 3PAR tests.
* Volume types added for the Grizzly release include cpg, snap_cpg, persona
and provisioning(thin vs. full)
Kurt Martin [Wed, 13 Feb 2013 19:30:17 +0000 (11:30 -0800)]
Fibre Channel base class for Cinder drivers
This is the base class, FibreChannelDriver, for all fibre channel
drivers to extend. Currently, it has only initialize_connection
with comments, but this will be a place to put common functions
similar to the iSCSI base class.
This patch depends on the following nova and devstack changes
to be functional(both still in the review process but approved
for Grizzly):
https://review.openstack.org/#/c/19992/
https://review.openstack.org/#/c/20003/
This patch does not support copy_image_to_volume and
copy_volume_to_image in the Grizzly release. We'll revisit
this in Havana and add that support.
Dan Prince [Mon, 4 Feb 2013 02:54:33 +0000 (21:54 -0500)]
Add a safe_minidom_parse_string function.
Adds a new utils.safe_minidom_parse_string function and
updates external API facing Cinder modules to use it.
This ensures we have safe defaults on our incoming API XML parsing.
Internally safe_minidom_parse_string uses a ProtectedExpatParser
class to disable DTDs and entities from being parsed when using
minidom.
During creating a volume from an image, the volume status changed
from creating to active, skipping the download phase. This bug was most
likely introduced during:
https://review.openstack.org/20514
This patch fixes the issue by change image_location back to image_id at
the correct place.
This patch also fixes an error in the exception handling, and the update
of the volume status after creation.
XenAPINFS did not implement the get_volume_stats method, thus
CapacityFilter fitered out the host, thus no volume was scheduled. This
fix is working around this problem quickly, by reporting an 'unknown'
capacity.
Mike Perez [Mon, 18 Feb 2013 19:17:26 +0000 (11:17 -0800)]
Fix undef function call in test_migrations for py26
unittest does not have this function until later. This uses the already
imported cinder.test.skip_unless to accomplish the same result without
outputted error.
Michael Basnight [Mon, 11 Feb 2013 21:54:45 +0000 (15:54 -0600)]
Update cinder-volume to enable multi volume support
* Added the service wrapper for multi binscripts
* Added a service_name to manager
* Added the configuration wrapper so the impls do not need
to know if they are accessing config values from a option
group or from the DEFAULT group
* Updated drivers to use the new configuration wrapper
* Added an example config set for multi backend
* Modified service to comply w/ common
* Added get_volume_stats to iscsi
* Added config for volume_backend_name
Chuck Short [Mon, 18 Feb 2013 14:03:32 +0000 (08:03 -0600)]
Install rtslib when installing cinder
It doesnt make sense to put rtslib in tools/test-requires
since it is not directly using the python module while running
the tests, since the tests use cinder-rtstool.
Change-Id: Ib39b91e25f22b4943ef17eee50967828f7460b25 Signed-off-by: Chuck Short <chuck.short@canonical.com>
Mark McLoughlin [Mon, 21 Jan 2013 16:25:38 +0000 (16:25 +0000)]
Sync latest cfg and log from oslo-incubator
Main cfg change is:
c5984ba Move logging config options into the log module
Logging changes include:
c5984ba Move logging config options into the log module 751c35b Verbose should not enable debug level logging edcdd25 Improve millisecond logging 9e5912f Fix pep8 E125 errors. edf14e0 Enable millisecond logging by default 9b81289 Allow nova and others to override some logging defaults 90ada0e update deprecated stanza efba202 Adjust the logging_context_format_string. c8cf6f7 move nova.common.deprecated to openstack-common
Fixes bug #989269
Use new LOG.deprecated() in place of cinder.common.deprecated.warn().
Also pull this cinder/context.py change from nova:
Zhiteng Huang [Sun, 17 Feb 2013 08:32:04 +0000 (16:32 +0800)]
Handle 'infinite' and 'unknown' capacity in CapacityWeigher
This patch updates CapacityWeigher to transform 'infinite' and
'unknown' capacity reported by back-ends to float('inf'). They
are considered the same for sorting purpose.
john-griffith [Sat, 16 Feb 2013 16:58:46 +0000 (09:58 -0700)]
Add get_cluster_stats to SolidFire driver
With the filter scheduler addition we added get_stats to the
drivers. This provides the required data for the filtering
mechanisms and is called by the scheduler.
The change adds the required call in the driver and also adds
an update routine to it's init to populate this info on startup.
This also adds the fake api response so that the tests can perform
the init routine and get the data, while giving the added bonus of
testing the new functions at the same time.
Matthew Treinish [Fri, 15 Feb 2013 19:14:54 +0000 (14:14 -0500)]
Add an update option to run_tests.sh
This commit adds an update option to run_tests.sh. This option
is used to rerun install_venv.py on an already installed venv.
This will then just update out of date packages with pip.
Xing Yang [Wed, 6 Feb 2013 18:32:12 +0000 (13:32 -0500)]
Update EMC SMI-S Driver
1. Merged with Avishay's changes(https://review.openstack.org/#/c/
19808/3) and removed copy volume<->image from my driver.
2. Also did some refactoring based on Mike's comments
last time (See https://review.openstack.org/#/c/19979/).
Made the following changes in emc_smis_common:
- Added a member variable conn to save get_ecom_connection
and moved exceptions inside the helper function get_ecom_connection.
- Also moved exceptions inside find_pool and get_storage_type.
- Added test cases so get_ecom_connection, find_pool, and
get_storage_type will be covered.
3. Made changes in emc_smis_common to handle both iscsi and fc.
Eric Harney [Wed, 6 Feb 2013 15:26:45 +0000 (10:26 -0500)]
Add GlusterFS volume driver
This driver enables use of GlusterFS in a similar fashion
as the NFS driver. It supports basic volume operations,
and like NFS, does not support snapshot/clone.
To enable, set volume_driver to
cinder.volume.drivers.glusterfs.GlusterfsDriver
Note that this requires a Nova libvirt GlusterFS driver
as well.
Adds config options: glusterfs_shares_config,
glusterfs_mount_point_base, glusterfs_disk_util, and
glusterfs_sparsed_volumes.
John Griffith [Fri, 15 Feb 2013 03:41:15 +0000 (03:41 +0000)]
Fix ordering of function args
Previous change swapped context and type-id params in function
declaration. Put it back.
Also, no reason for storing the qos dict in the parameters, if
it's stored then we continuously have to update it, remove that
for now and come up with a cleaner method later.
john-griffith [Thu, 14 Feb 2013 19:00:47 +0000 (12:00 -0700)]
Add an ID to temporary volume snapshot object
The create_cloned_volum method creates a temp snapshot and
when it's done it uses the standard snapshot_delete method
in the LVM driver.
A change was added that logs a statement using the object[id],
but we weren't actually setting this on our temp object so we
would get a trace fo key DNE. This change adds an id key to
our temp object.
Zhiteng Huang [Fri, 25 Jan 2013 18:18:03 +0000 (02:18 +0800)]
Allow create_volume() to retry when exception happened
Due to the fact that certain volume back-ends cannot easily report
simple total_capacity_gb/free_capacity_gb for their internal
implementation complexity, scheduler is updated to let those back-ends
who report unclear capacity pass capacity filter, thus there is chance
create_volume() request would fail. In a more general case, when a
volume back-end failed to serve create_volume request for whatever reason
it'd be good that we have a mechanism to 'retry' the request.
So the idea is when volume manager catches the exception from
driver.create_volume() call, it checks if the request is allowed to be
rescheduled (requests that are not: clone volume and create volume from
snapshot while 'snapshot_same_host' option is true), it composes a new
request back to scheduler with additional information to mark this specific
back-end has been tried (so that scheduler may choose to skip this back-end
if needed). Scheduler is (filter scheduler only, simple and chance
scheduler is not supported) is updated as well so that it only retry
scheduler_max_attempts times. In order to skip/rule out previously tried
back-ends in next schedule task, a new RetryFilter is added.
Changes:
1) volume RPC API create_volume() is updated with new parameters
to save original request information in case rescheduling is needed.
This bumps volume RPC API to 1.4.
2) implementation of create_volume() method in volume API is
refactored in order to distinguish if a request is allowed to
do reschedule (i.e. currently create volume from source volume
bypasses scheduler, not rescheduling is allowed).
3) add reschedule functionality in create_volume() of volume
manager so that it's able to send the request back to scheduler.
4) add schedule_max_attempts config option in scheduler/driver.py
5) add RetryFitler
6) change scheduler_driver default option to FilterScheduler
Mike Perez [Tue, 12 Feb 2013 06:49:26 +0000 (22:49 -0800)]
Update snapshot rest api to be consistent with volumes
display_name becomes name and display_description becomes description in
both parameters to pass in requests and keys returned from responses.
This does not update the snapshot model as it takes the same approach
with updating the volume rest api.
Mike Perez [Sun, 10 Feb 2013 20:34:44 +0000 (12:34 -0800)]
v2 volume/snapshot create will correctly give a 202 response
Before it was giving a 200 response. Since the request was received fine
but still has to process things, this is a more correct response. In
addition this is more consistent with other OpenStack project create
rest api calls.
Sean Dague [Sat, 9 Feb 2013 15:10:03 +0000 (10:10 -0500)]
make test_databases instance variable
eventually we'll need to do this to allow for testr testing, but in
doing so this also exposed that sqlite downgrade of source_volid
didn't work when data was in the table (sqlite just happily ignored
the drop, then failed when trying to added the column for a second
time during the snake walk.)
This fix also adds the correct sqlite downgrade path for migration
005 to actually drop source_volid, but ensure we don't loose any
other data in the process.
Avishay Traeger [Thu, 7 Feb 2013 09:11:03 +0000 (11:11 +0200)]
Move create_cloned_volume() to LVMVolumeDriver.
The implementation of create_cloned_volume() that was in driver.py
relied on functions that aren't declared there, such as
_create_volume(), _copy_volume(), and _sizestr(). Moved the function
to LVMVolumeDriver in lvm.py, where those functions exist.
Avishay Traeger [Wed, 6 Feb 2013 21:04:15 +0000 (23:04 +0200)]
Allow disabling of long-lived SSH connections.
Currently, SSHPool connections are kept alive indefinitely. This can
be good for many cases, but sometimes it is not desirable. For
example, this causes unit tests which use SSH to not complete because
of the open connections. This patch allows users to revert back
to paramiko's regular timeout policy.
Avishay Traeger [Wed, 23 Jan 2013 11:55:51 +0000 (13:55 +0200)]
Don't require importing paramiko for error.
The _run_ssh function raises a paramiko exception, which requires
functions that use it to import the paramiko class. This should be
hidden away, and so I created a new exception in its place. Nobody
but the Storwize/SVC driver use it anyway, and I am updating that
driver in a separate patch.
Adds a new config option:
nfs_mount_options=None (Default)
When not None, anything set here will be passed as -o <options> to
the mount command. The default behavior is to do exactly as we are
doing today and relying on the OS/kernel defaults.
Sean Dague [Tue, 5 Feb 2013 12:28:22 +0000 (07:28 -0500)]
add data injection on migrations
bring over the data injection from migrations from nova, which allows
us to inject sample data at every migration in the walk_versions, and
run a check after the migration to make sure that things look as we
expected.
Mate Lakat [Fri, 25 Jan 2013 14:44:08 +0000 (14:44 +0000)]
XenAPINFS: Copy image from glance
related to blueprint xenapinfs-glance-integration
This change is using the glance xenapi plugin to copy a glance image to
a xenapinfs backed volume. It slightly differs from the reference
implementation, as this implementation is composed of three steps:
- create volume
- overwrite the created volume with the one downloaded from glance
- resize the volume
As opposed to fill the bytes from glance to the created image. This
approach works more effective with XenServer type semi ovf tgz -ed vhds.
Rushi Agrawal [Fri, 1 Feb 2013 18:16:58 +0000 (23:46 +0530)]
Fix inability to delete volumes in error state for NetApp driver
While using NetApp 7-mode storage box to create volumes, if the volume
is not created on the storage box (possibly due to wrong/missing
options in cinder.conf file), the volume goes in error state, and when
one tries to delete the volume, an exception was raised which made
the volume go in error_deleting state, which is unrecoverable. This
commit quiesces that exception, resulting in successful removal of the
volume entry from the volume table.
John Griffith [Tue, 5 Feb 2013 00:54:56 +0000 (17:54 -0700)]
Copy glance_image_metadata when cloning volumes.
When cloning a volume we were not capturing/copying
the glance_image_metadata from the source volume.
This change implements a copy_image_metadata_from_vol_to_vol.
So now we can clone bootable volumes without going back to glance
or messing with them otherwise, they're just ready to be booted.
John Griffith [Mon, 4 Feb 2013 23:18:29 +0000 (16:18 -0700)]
Add volume_glance_metadata to volume.api.get
The dict(rv.iteritems()) agains the volume object from
volume.api.get does NOT set the volume_glance_metadata k/v.
This is used to show bootable in the show and list api calls,
the result was that list properly reflected the bootable status
of a volume, however show would always describe the flag as False.
So we simply check if the rv includes the glance_metadata and if
so go ahead and create the entry in the volume dict.
Thierry Carrez [Tue, 29 Jan 2013 13:58:07 +0000 (14:58 +0100)]
Import Oslo's common rootwrap to Cinder
Import oslo-incubator's rootwrap, which contains all the new
features and bugfixes that were pushed to nova-rootwrap earlier
in this cycle, including logging support and path search.
Code Review / openstack-build / cinder-build.git / log