David Ripton [Tue, 9 Jul 2013 17:32:08 +0000 (13:32 -0400)]
Make String column creation compatible with SQLAlchemy 0.8
3 migration scripts used the convert_unicode, unicode_error, and
_warn_on_bytestring arguments when creating String columns. These
seem to cause problems with SQLAlchemy 0.8. Nova excised all use
of such arguments in commit 93dec58156e when squashing migrations
for Grizzly, and seems to have no problems with SQLAlchemy 0.8.
Last usage of suds was removed in
https://review.openstack.org/#/c/33168/. suds is
meanwhile unmaintained and has pending security issues,
so better remove references to it.
Key functions include efficient creation of snapshots and clones using
file system level copy-on-write mechanism. Copying of image blocks is
also avoided in case Glance backend repository happens to be on GPFS.
This change is required for future works in the SolidFire driver.
New SolidFire features (such as extend) are only available in
the latest version of the SolidFire API.
Default to '1.0' for backward compatibility. SolidFire still supports
the '1.0' version in its latest release.
Refactor reschedule in exception handling of volume manager
Previous exception handling has a pitfall that may potentially clear
the sys.exc_info() (by calling SQLalchmey to update db). This patch
refactors some part of exception handling in create_volume() to make
sure sys.exc_info() is retrieved so that we can log and reraise it;
also we make sure exception be reraised at the end of exception handling
no matter the request is rescheduled or not.
As a side effect, we fixed a bug in unittest which didn't provide
correct argument to db API but previously this exception has been wrongly
consumed by volume manager's exception handling (not reraise exception
when request is rescheduled).
Roman Podolyaka [Thu, 27 Jun 2013 15:34:21 +0000 (18:34 +0300)]
Don't pass 'session' arg to public DB API methods
DB API is an abstraction layer, which is used to make it
possible to switch DB backends easily (though we've got only
SQLAlchemy backend at the moment).
Public methods of DB API should not accept any backend-specific
arguments (i. e. a Session instance, that is an SQLAlchemy entity
to work with DB transactions).
This patch removes 'session' argument from all DB API public methods
(except volume_data_get_for_project() and snapshot_data_get_fro_project(),
which are a bit tricky and will be fixed by another patch).
If a DB API method must be called by another one in the context of a started
transaction, a private method is used. It accepts the same arguments as the
corresponding public method plus one additional argument to pass the transactional
context (in case of SQLAlchemy backend it's a Session instance).
Marc Koderer [Thu, 27 Jun 2013 10:05:56 +0000 (12:05 +0200)]
Add interface class for backup drivers
This fix introduces an interface class to define the structure of all backup
drivers. It also renames backup/service to backup/driver.
To be backward compatible a mapping functionality is introduced to map old
backup services to backup drivers.
Current codes in create_volume() may exists a scenario that
a context which is not elevated but after it fails in volume
creation, it becomes elevated. This patch saves original
context, so that if it fails in volume creation simply use the
original context for further scheduling.
John Griffith [Tue, 25 Jun 2013 22:52:21 +0000 (16:52 -0600)]
Fix extent size issue when creating thin pool.
The LVM create thin pool rounds up extents, this
change rounds the free space down to an int to avoid
this issue and also cleans up some comments and
code around how PV's are set up and used.
John Griffith [Thu, 4 Jul 2013 15:25:03 +0000 (09:25 -0600)]
Add check for snapshot to Brick LVM.
Adds a check to the LVM code in brick to see if
a snapshot exists. This implements the incorrectly
named "is_busy" method that exists in the cinder LVM driver.
Replace FLAGS with cfg.CONF
Rename modules fake_flags to conf_fixture, test_flags to test_conf, declare_flags to declare_conf, runtime_flags to runtime_conf
Renamed cinder.flags, because exactly the same was done in the glance and nova
John Griffith [Tue, 2 Jul 2013 19:16:28 +0000 (13:16 -0600)]
Add execute wrapper to brick LVM code.
Add an execute wrapper to the brick lvm code and
enable the ability to pass in a desired executor other
than just using the default common/processutils executor
if there's some reason to do so.
This will enable the switch of the cinder/volume/drivers/lvm.py
driver to continue using cinder/utils execute until we switch over
to common/processutils and also helps immensely with the tests.
CoraidDriver: Create_volume_from_snapshot of a different size
When you create a Volume from a Snapshot, the 'size' argument for the
new volume isn't used at all.
So, the new volume created will be created (successfully but) with the
same size as the snapshot size.
Cinder think the volume has been created with the right size.
We just need to call Coraid ESM LV Resize with the REST API after
the volume has been created.
Updates the os-services API extension so that it is consistent
internally (index and update return similar formats), and so
that it works with the following cinderclient code changes which sends
the the following request body format:
{"binary": "cinder-volume", "host": "host1"}
It addresses a similar issue which happens in nova os-services extention
before. https://bugs.launchpad.net/nova/+bug/1147746
This change added 'binary' key in request and 'status' key in response
while still keeping 'service' ane 'disabled' key for API compatibility
sake.
Viraj Hardikar [Tue, 25 Jun 2013 22:37:09 +0000 (15:37 -0700)]
Removes 3PAR domain option from cinder config file
The HP 3PAR domain is no longer required to be specified in the
cinder configuration file. This patch automatically looks up the
domain using the Common Provisioning Group (CPG) specified.
hp3par_domain will remain in the config for now and will be
logged as deprecated. It will be removed in the I release.
Added new Ceph backup service to allow backup
of Cinder volumes to a Ceph object store. This
driver is compatible with the existing backup
interface provided by the Swift backup service.
Brian Waldon [Wed, 26 Jun 2013 16:43:48 +0000 (09:43 -0700)]
Add os-availability-zone extension
* Query /os-availability-zone to get an object representing the configured
availability zones and their state
* This implements a subset of Nova's os-availability-zone extension
Dirk Mueller [Fri, 14 Jun 2013 21:37:11 +0000 (23:37 +0200)]
Run flake8 also on cinder/*/openstack
exclude=openstack also matches cinder/api/openstack,
which is however not merged from oslo-incubator.
Exclude=common instead, and explicitely include
cinder/common
Cory Stone [Wed, 22 May 2013 15:31:13 +0000 (10:31 -0500)]
Quotas by Volume Type
The VolumeTypeQuotaEngine creates a volume, snapshot, and gigabyte
quota for each volume type that exists, in addition to the existing
global quotas for each. These are queried every time a quota operation
happens.
The resources for creating quotas are named: volumes_<vtype>,
gigabytes_<vtype>, and snapshots_<vtype>.
Another patch will have changes to cinderclient for setting project
quotas by volume type.
Mate Lakat [Mon, 24 Jun 2013 13:37:08 +0000 (14:37 +0100)]
xenapi: implement xenserver image to volume
Amend image_utils, so that it could handle xenserver type images (vhd
chains inside a tgz archive). This requires the vhd-util binary, if the
image contains a vhd chain (e.g.: snapshot)
DocImpact
- Copy a XenServer type OpenStack image (tgz -ed vhd chain) is now
supported.
- To use the above functionality, the cinder box must have access to the
vhd-util binary. In ubuntu, this is done by apt-get installing
blktap-utils.
Vincent Hou [Thu, 27 Jun 2013 07:43:00 +0000 (15:43 +0800)]
Save some more image attributes to volume_glance_metadata.
Current issue: When copying an image to a volume, attributes like size, disk_format, container_format,
checksum, min_disk and min_ram have not been copied so far. When booting a VM from a volume, we are
not able to check these atrributes without giving the image id. Furthermore, if the original
image is deleted, we will lose all the information.
This patch saves these attributes into volume_glance_metadata. We can still check these attributes
without giving the image id when booting a VM from a volume.
Kai Zhang [Thu, 27 Jun 2013 07:31:36 +0000 (00:31 -0700)]
Fix check_for_setup_error for sheepdog driver
Current implementation of check_for_setup_error() of sheepdog driver
cannot work correctly with the latest version of sheepdog cluster.
This patch fixes it and adds tests for both old and new version.
This patch adds the new brick initiator connector object
which contains the code to do volume attach and detach to a host
machine. It includes hooks to work in both cinder and nova.
Nova has different exec wrapper and it also needs to talk to a
hypervisor at certain points during detach.
This patch also pulls the copy/pasted code from nova in the base
ISCSIDriver's _attach_volume method to use this new brick library.
This patch also includes a fix in the ISCSIDriver's
copy_volume_to_image code that didn't actually detach a volume
when it was done with the operation.
Bug 1194962
This patch also includes a fix for iSCSI detaches where the
iSCSI LUN wasn't being removed from the system until the very
last detach issues an iscsiadm logout.
Bug 1112483
This patch includes a fix for iSCSI multipath detaches where
the multipath device and the iSCSI LUNs for the multipath device
were never removed from the kernel.
Bug 1112483
Zhiteng Huang [Tue, 18 Jun 2013 20:43:26 +0000 (04:43 +0800)]
Calculate count for customized dd blocksize
In previous commit we introduce a configurable blocksize option for
'dd' command, however the 'count' value wasn't updated accordingly.
As a result, count could be too small which is bug. This patch
introduce a private method to 1) validate custom dd blocksize; 2)
calculate correct count value.
Marc Koderer [Wed, 19 Jun 2013 14:07:44 +0000 (16:07 +0200)]
Content-length missing in put_object
Radosgw returns a "411 Length Required" in case the content-length is not
specified. The parameter content_type is not set which leads to the problem
that content_length cannot be determined. Since the content length is already
computed in backup/services/swift it shouldn't be a problem to set this
parameter.
John Griffith [Wed, 19 Jun 2013 05:22:48 +0000 (23:22 -0600)]
Replace glance_metadata check with bootable column.
This patch adds a column to indicate if a volume is bootable or not.
Cinder API V1 was using get.volume_glance_metadata to determine
if a volume was bootable for translate_view.
This is fine until you put a heavy load on the system and things
can fall apart, particularly the backref of the glance meta is no longer
available and the call blows up. Also this results in a bit of extra
unnecessary data being passed around with every create and get call.
This is especially an issue when creating hundreds or thousands of volumes.
Even better we can now reliably create thousands of volumes.
Sergey Vilgelm [Mon, 24 Jun 2013 11:19:50 +0000 (15:19 +0400)]
Do not raise NEW exceptions
Raising NEW exception is bad practice, because we lose TraceBack.
So all places like:
except SomeException as e:
raise e
should be replaced by
except SomeException:
raise
If we are doing some other actions before reraising we should
store information about exception then do all actions and then
reraise it. This is caused by eventlet bug. It lost information
about exception if it switch threads.
Code Review / openstack-build / cinder-build.git / log