john-griffith [Sat, 16 Feb 2013 16:58:46 +0000 (09:58 -0700)]
Add get_cluster_stats to SolidFire driver
With the filter scheduler addition we added get_stats to the
drivers. This provides the required data for the filtering
mechanisms and is called by the scheduler.
The change adds the required call in the driver and also adds
an update routine to it's init to populate this info on startup.
This also adds the fake api response so that the tests can perform
the init routine and get the data, while giving the added bonus of
testing the new functions at the same time.
Matthew Treinish [Fri, 15 Feb 2013 19:14:54 +0000 (14:14 -0500)]
Add an update option to run_tests.sh
This commit adds an update option to run_tests.sh. This option
is used to rerun install_venv.py on an already installed venv.
This will then just update out of date packages with pip.
Xing Yang [Wed, 6 Feb 2013 18:32:12 +0000 (13:32 -0500)]
Update EMC SMI-S Driver
1. Merged with Avishay's changes(https://review.openstack.org/#/c/
19808/3) and removed copy volume<->image from my driver.
2. Also did some refactoring based on Mike's comments
last time (See https://review.openstack.org/#/c/19979/).
Made the following changes in emc_smis_common:
- Added a member variable conn to save get_ecom_connection
and moved exceptions inside the helper function get_ecom_connection.
- Also moved exceptions inside find_pool and get_storage_type.
- Added test cases so get_ecom_connection, find_pool, and
get_storage_type will be covered.
3. Made changes in emc_smis_common to handle both iscsi and fc.
Eric Harney [Wed, 6 Feb 2013 15:26:45 +0000 (10:26 -0500)]
Add GlusterFS volume driver
This driver enables use of GlusterFS in a similar fashion
as the NFS driver. It supports basic volume operations,
and like NFS, does not support snapshot/clone.
To enable, set volume_driver to
cinder.volume.drivers.glusterfs.GlusterfsDriver
Note that this requires a Nova libvirt GlusterFS driver
as well.
Adds config options: glusterfs_shares_config,
glusterfs_mount_point_base, glusterfs_disk_util, and
glusterfs_sparsed_volumes.
John Griffith [Fri, 15 Feb 2013 03:41:15 +0000 (03:41 +0000)]
Fix ordering of function args
Previous change swapped context and type-id params in function
declaration. Put it back.
Also, no reason for storing the qos dict in the parameters, if
it's stored then we continuously have to update it, remove that
for now and come up with a cleaner method later.
Zhiteng Huang [Fri, 25 Jan 2013 18:18:03 +0000 (02:18 +0800)]
Allow create_volume() to retry when exception happened
Due to the fact that certain volume back-ends cannot easily report
simple total_capacity_gb/free_capacity_gb for their internal
implementation complexity, scheduler is updated to let those back-ends
who report unclear capacity pass capacity filter, thus there is chance
create_volume() request would fail. In a more general case, when a
volume back-end failed to serve create_volume request for whatever reason
it'd be good that we have a mechanism to 'retry' the request.
So the idea is when volume manager catches the exception from
driver.create_volume() call, it checks if the request is allowed to be
rescheduled (requests that are not: clone volume and create volume from
snapshot while 'snapshot_same_host' option is true), it composes a new
request back to scheduler with additional information to mark this specific
back-end has been tried (so that scheduler may choose to skip this back-end
if needed). Scheduler is (filter scheduler only, simple and chance
scheduler is not supported) is updated as well so that it only retry
scheduler_max_attempts times. In order to skip/rule out previously tried
back-ends in next schedule task, a new RetryFilter is added.
Changes:
1) volume RPC API create_volume() is updated with new parameters
to save original request information in case rescheduling is needed.
This bumps volume RPC API to 1.4.
2) implementation of create_volume() method in volume API is
refactored in order to distinguish if a request is allowed to
do reschedule (i.e. currently create volume from source volume
bypasses scheduler, not rescheduling is allowed).
3) add reschedule functionality in create_volume() of volume
manager so that it's able to send the request back to scheduler.
4) add schedule_max_attempts config option in scheduler/driver.py
5) add RetryFitler
6) change scheduler_driver default option to FilterScheduler
Mike Perez [Sun, 10 Feb 2013 20:34:44 +0000 (12:34 -0800)]
v2 volume/snapshot create will correctly give a 202 response
Before it was giving a 200 response. Since the request was received fine
but still has to process things, this is a more correct response. In
addition this is more consistent with other OpenStack project create
rest api calls.
Sean Dague [Sat, 9 Feb 2013 15:10:03 +0000 (10:10 -0500)]
make test_databases instance variable
eventually we'll need to do this to allow for testr testing, but in
doing so this also exposed that sqlite downgrade of source_volid
didn't work when data was in the table (sqlite just happily ignored
the drop, then failed when trying to added the column for a second
time during the snake walk.)
This fix also adds the correct sqlite downgrade path for migration
005 to actually drop source_volid, but ensure we don't loose any
other data in the process.
Avishay Traeger [Thu, 7 Feb 2013 09:11:03 +0000 (11:11 +0200)]
Move create_cloned_volume() to LVMVolumeDriver.
The implementation of create_cloned_volume() that was in driver.py
relied on functions that aren't declared there, such as
_create_volume(), _copy_volume(), and _sizestr(). Moved the function
to LVMVolumeDriver in lvm.py, where those functions exist.
Avishay Traeger [Wed, 6 Feb 2013 21:04:15 +0000 (23:04 +0200)]
Allow disabling of long-lived SSH connections.
Currently, SSHPool connections are kept alive indefinitely. This can
be good for many cases, but sometimes it is not desirable. For
example, this causes unit tests which use SSH to not complete because
of the open connections. This patch allows users to revert back
to paramiko's regular timeout policy.
Avishay Traeger [Wed, 23 Jan 2013 11:55:51 +0000 (13:55 +0200)]
Don't require importing paramiko for error.
The _run_ssh function raises a paramiko exception, which requires
functions that use it to import the paramiko class. This should be
hidden away, and so I created a new exception in its place. Nobody
but the Storwize/SVC driver use it anyway, and I am updating that
driver in a separate patch.
Adds a new config option:
nfs_mount_options=None (Default)
When not None, anything set here will be passed as -o <options> to
the mount command. The default behavior is to do exactly as we are
doing today and relying on the OS/kernel defaults.
Sean Dague [Tue, 5 Feb 2013 12:28:22 +0000 (07:28 -0500)]
add data injection on migrations
bring over the data injection from migrations from nova, which allows
us to inject sample data at every migration in the walk_versions, and
run a check after the migration to make sure that things look as we
expected.
Mate Lakat [Fri, 25 Jan 2013 14:44:08 +0000 (14:44 +0000)]
XenAPINFS: Copy image from glance
related to blueprint xenapinfs-glance-integration
This change is using the glance xenapi plugin to copy a glance image to
a xenapinfs backed volume. It slightly differs from the reference
implementation, as this implementation is composed of three steps:
- create volume
- overwrite the created volume with the one downloaded from glance
- resize the volume
As opposed to fill the bytes from glance to the created image. This
approach works more effective with XenServer type semi ovf tgz -ed vhds.
Rushi Agrawal [Fri, 1 Feb 2013 18:16:58 +0000 (23:46 +0530)]
Fix inability to delete volumes in error state for NetApp driver
While using NetApp 7-mode storage box to create volumes, if the volume
is not created on the storage box (possibly due to wrong/missing
options in cinder.conf file), the volume goes in error state, and when
one tries to delete the volume, an exception was raised which made
the volume go in error_deleting state, which is unrecoverable. This
commit quiesces that exception, resulting in successful removal of the
volume entry from the volume table.
John Griffith [Tue, 5 Feb 2013 00:54:56 +0000 (17:54 -0700)]
Copy glance_image_metadata when cloning volumes.
When cloning a volume we were not capturing/copying
the glance_image_metadata from the source volume.
This change implements a copy_image_metadata_from_vol_to_vol.
So now we can clone bootable volumes without going back to glance
or messing with them otherwise, they're just ready to be booted.
John Griffith [Mon, 4 Feb 2013 23:18:29 +0000 (16:18 -0700)]
Add volume_glance_metadata to volume.api.get
The dict(rv.iteritems()) agains the volume object from
volume.api.get does NOT set the volume_glance_metadata k/v.
This is used to show bootable in the show and list api calls,
the result was that list properly reflected the bootable status
of a volume, however show would always describe the flag as False.
So we simply check if the rv includes the glance_metadata and if
so go ahead and create the entry in the volume dict.
Thierry Carrez [Tue, 29 Jan 2013 13:58:07 +0000 (14:58 +0100)]
Import Oslo's common rootwrap to Cinder
Import oslo-incubator's rootwrap, which contains all the new
features and bugfixes that were pushed to nova-rootwrap earlier
in this cycle, including logging support and path search.
Michael J Fork [Mon, 4 Feb 2013 14:51:29 +0000 (14:51 +0000)]
Mark password config options with secret
Config object supports masking values when writing out if the secret
flag is set on the option definition. This change flags all cinder
options containing a password.
Chuck Short [Sun, 3 Feb 2013 01:22:18 +0000 (19:22 -0600)]
Skip tests if cinder is not installed
The test_capacity_filter_passes_infinite and test_capacity_filter_passes_unknown
tests were failing when cinder is not installed. Skip the tests like the other
tests for test_host_filters.
Change-Id: I5ebfa2dca05b6e89a12e3153598bf4699da888de Signed-off-by: Chuck Short <chuck.short@canonical.com>
john-griffith [Sat, 2 Feb 2013 02:21:18 +0000 (19:21 -0700)]
Fix provider_location column add for PSQL
Migration 006 (commit 690cae58e6bbac5758ea2f7b60774c797d28fba5)
didn't work properly for postgres,
this patch corrects the upgrade by ensuring the execute
is performed and the value is initialized to None.
Since we haven't released a milestone etc with this migration in the
code it should be safe to just fix it here and submit.
Kurt Martin [Sat, 12 Jan 2013 01:13:43 +0000 (17:13 -0800)]
Update 3PAR driver
Added support for get_volume_stats()
Add sanity checks for:
Ensure the CPG lives in the Domain that's configured.
On 3PAR systems, the create volume from snapshot has to be
the same size. The driver now checks to make sure that they
are the same size.
Now using volume and snapshot id's instead of the names.
Checking for optional fields before using them(i.e. description).
Added a new method get_ports() to gather the active array ports.
Fixed inline comments, added a space between the comment and # sign
john-griffith [Fri, 1 Feb 2013 05:38:51 +0000 (22:38 -0700)]
Fix the generalized copy_image_to_volume operation.
The Generis iSCSI copy volume<->image patch
(change: Iff097629bcce9154829a7eb5aee0ea6302338b26) did
not account for the fact that the existing volume reference
passed in to the copy method would not have the updated
provider_location and iSCSI info available.
To address this, we simply just get a new ref from the DB
after creation if a copy image is requested.
Also, the list of acceptable formats was not fully inclusive,
not only that, but there's no reason to not let the qemu-convert
handle any errors here.
Zhiteng Huang [Wed, 19 Dec 2012 06:56:11 +0000 (14:56 +0800)]
Pull cfg module from Oslo and update cinder-manage accordingly
New cfg module abandons previous disable_interspersed_args() and
recommand to use add_subparsers from argparser module instead.
This patch pull cfg module from Oslo and update the affected
cinder-manage utils.
When we are cloning a volume, we now set the source volume's status
to 'in use'. This prevents a user from deleting the source volume while the
backend copies/clones the contents to the new volume.
When the cloning is complete, we reset the status to it's original
state.
Zhiteng Huang [Mon, 28 Jan 2013 17:12:31 +0000 (01:12 +0800)]
Allow volume back-end to report 'infinite' or 'unknown' as capacity
For some reason, some volume back-ends cannot report actual capacity
that is available or even total capacity (e.g. thin-provisioning),
Cinder should allow such capability report and make placement/schedule
decision based on these status.
In particular, we agree to use 'infinite' to mark unlimited space and
'unknown' to mark unknown/unclear space. Host manager and CapacityFilter
are updated accordingly to allow such capacity. For those back-ends
report 'infinite' or 'unknown' capacity, CapacityFilter will let them
pass to give them a chance to try. Even if failure happened, the retry
mechanism should take care of the case.
Dan Prince [Mon, 28 Jan 2013 15:44:57 +0000 (10:44 -0500)]
Update osapi_volume_extension default.
Updates the Cinder config default for osapi_volume_extension
so that it only uses the new cinder.api.contrib.standard_extensions
loader.
Previously both the new and old extension loader path were listed
which can cause extensions to be loaded twice thus causing
log WARNINGs to appear in the Cinder API log file.
Also, regenerates the cinder.conf.sample to reflect this change.
John Griffith [Fri, 11 Jan 2013 21:38:23 +0000 (14:38 -0700)]
Implement LVM thin provisioning support.
As of LVM2 version 2.02.89 the ability to do thin provisioning
was made available in LVM, this provides some cool new features
but also addresses some problems with things like terrible
LVM snapshot performance.
Currently the version of LVM in Ubuntu 12.04 does NOT support LVM thin,
however an experimental PPA from brightbox which is a backport from
Quantal has been proposed to Cannonical to be pulled in. For some
users the experimental PPA is a better option than dealing with some of
the current issues in the standard LVM2 version of Precise (including
the dd hangs on secure delete). See BP for more info.
This change will create the thin pool if it doesn't exist
using pool_size flag, or by default using entire VG space.
Code Review / openstack-build / cinder-build.git / log