John Griffith [Thu, 15 Aug 2013 02:06:05 +0000 (20:06 -0600)]
Replace os.unlink with delete_if_exists
Shouldn't care when doing unlink on our temp files
if they exist or not. In fact this causes problems
when you do things like with tempfile/dir and happen
to try and unlink after it's already been removed.
This replaces these calls with the safer
common.fileutils.delete_if_exists which will
ignore the os exception of the object DNE.
Zhiteng Huang [Tue, 13 Aug 2013 04:57:17 +0000 (12:57 +0800)]
Raise exception when Glance metadata not found.
It'd be better to raise exception when trying to copy Glance metadata
from source (volume/snapshot) to destination (volume/snapshot) rather
than silent failure, which is exactly the reason there's unspotted error
inside glance_meta unittest but it was able to pass.
With this fix, one should _not_ directly call glance_metadata_copy()
without looking at source's bootable flags. This patch also refactors
_create_volume_from_snapshot() and _create_from_source_volume() to only
do Glance metadata copy when needed (bootable is True).
XueChendi [Mon, 12 Aug 2013 16:25:43 +0000 (00:25 +0800)]
Interprete scoped key as nested tags
Current codes in xml_util.py does not support REST xml
so well when one tagname contains delimiter, it fails
to be interpreted. So this patch is to interprete one
tagname like "<a:b>1</a:b>" as a nested tag like
"<a><b>1</a></b>".
Kurt Martin [Tue, 13 Aug 2013 20:51:15 +0000 (13:51 -0700)]
Adding the -online option to the 3PAR clone
The 3PAR drivers had to wait while the clone was being performed
and this would take a considerable amount of time for large volumes.
This patch takes advantage of the 3PAR backend by using the -online
option in the command that we were calling to perform the copy. This
allows us to remove the sleep in the driver. Using the -online
option forced us to change some of the delete_volume code because
the 3PAR backend will not allow a volume that was copied to be added
to a virtual volume set.
This patch removes the brick iser.py's dependency
on cinder exceptions and volume_utils.
This required moving some exceptions out of cinder's
exception.py that the iser.py raises. Also had
to create a BrickException and refactor existing
brick exceptions to use the new BrickException model.
Jay S. Bryant [Mon, 12 Aug 2013 16:46:06 +0000 (11:46 -0500)]
Fix handling ImageUnacceptable in create_volume
In the create_volume flow ImageUnacceptable exceptions are
not properly handled. _copy_image_to_volume can receive an
ImageUnacceptable exception from copy_image_to_volume if fetch_to_raw
is used. Currently the ImageUnacceptable exception is changed to
a generic ImageCopyFailure exception which does not make the cause
of the excpetion clear.
This change adds handling and raising ImageUnacceptable exceptions
in _copy_image_to_volume. It also adds the exception to
no_reschedule_types as it doesn't make sense to keep retrying the
copy if the image was found to be unacceptable.
Eric Harney [Sat, 10 Aug 2013 20:12:59 +0000 (16:12 -0400)]
Fix signature of _create_volume() in ThinLVMVolumeDriver
create_volume_from_snapshot will call _create_volume() with the
wrong parameters when using ThinLVMVolumeDriver. It should be
compatible with LVMVolumeDriver's _create_volume() method.
Seif Lotfy [Fri, 9 Aug 2013 13:37:38 +0000 (13:37 +0000)]
Fix ratelimiting
Current master does now respect ratelimiting, since parsing of the
api-paste.ini was faulty. api-paste.ini limited user limiting by
setting a line as follows:
user:<user-id>:(GET, *, ".*", 4, minute) which was passed to the
Limiter as kwargs with "user" as a key. Thus multiple user limiting
was not possible as well as extracting the id of the user was bound
to fail, since we checked on the key with startswith("user:")
An example config in the api-paste.ini has to look as follows:
limits = (POST, "*", .*, 10, MINUTE)
limits.<user-id1>:(GET, "*", .*, 4, minute)
limits.<user-id2>:(GET, "*", .*, 2, minute)
Ignore purge_props for v2 Glance api and fix upload
The V2 Glance API image schema does not contain
purge_props. While this may be a bug in Glance,
we will ignore this property in cinder when
glance_api_version=2. This will not change behaviour
since Glance defaults this property to True, Cinder
never sets it to False and the v2 client appears to
ignore it anyway.
Also fixed image upload which is a seperate client
call to update in v2 (v1 update does both).
Luis A. Garcia [Mon, 8 Jul 2013 23:11:05 +0000 (23:11 +0000)]
Add support for API message localization
Add support for doing language resolution for a request, based on the
Accept-Language HTTP header.
Using the lazy gettext functionality from oslo gettextutils, it is now
possible to use the resolved language to translate an exception message
to the user requested language and return that translation from the API.
Kurt Martin [Thu, 8 Aug 2013 21:14:56 +0000 (14:14 -0700)]
3PAR drivers creating incorrect comment data
The 3PAR volumes created from a shapshot had incorrect comment data
that gets added in the comment section of the volumes on the backend.
This patch fixes the display name and description by getting the correct
data when the volume is created. This patch also fixes the inconsistency
in how the keys (name vs. display_name) were used in the comments.
Resource usage sync functions was declared in cinder/quota.py, and
using db.api public methods. This functions was moved to database
backend implementation, so now sync functions can use private
methods of database backend, and session attribute can be removed
from this public methods.
Joshua Harlow [Thu, 1 Aug 2013 19:08:04 +0000 (12:08 -0700)]
Refactoring of create_volume to use taskflow.
Move the create_volume workflow to using taskflow and
split that workflow into three major pieces (each with
there own workflow) and create tasks that perform the
individual required actions to accomplish the pieces
desired outcome.
1. An api workflow composed of the following tasks:
- Extracting volume request (which checks types, values) and creates a
standard output for other tasks to work on (allowing further tasks to be
plugged in the chain without having to worry about other tasks output
formats).
- Quota reservation (rolled back on failure).
- Database entry creation.
- Quota committing.
- Volume RPC casting to volume scheduler or to targeted volume manager.
2. A scheduler workflow composed of the following tasks:
- Extracting scheduler request specification for further tasks to use.
- Change status & notify (activated only on failure).
- Create volume scheduler driver call (which will itself RPC cast to a
targeted volume manager).
3. A manager workflow composed of the following tasks:
- Extract volume request specification from incoming request for
further tasks to use. This also breaks up the incoming request into the 4
volume types that can be created later.
- Change status & notify on failure or reschedule on failure, this is
dependent on if rescheduling is enabled *and* which exception types are
thrown from the volume creation code.
- Create volume from specification
- This contains the code to create from image, create raw volume, create
from source volume, create from snapshot using the extracted volume
specification.
- Change status & notify success.
Key benefits:
- Handled exceptions in a easier to understand, easier to review and more
reliable way than they are currently being handled.
- Rescheduling is now easier to understand.
- Easier to understand structure with tasks that consume inputs, take some
action on them and produce outputs and revert on subsequent failure using
whatever they produced to know how to revert.
- Ability to add new unit tests that can test individual task actions by
providing mock task inputs and validating expected task outputs.
Future additions:
- Eventual addition of resumption logic to recover from operations stopped
halfway through.
- Ability to centrally orchestrate the tasks and pick and choice how
reconciliation of failures based on code or policies.
Lakhinder Walia [Thu, 1 Aug 2013 23:43:45 +0000 (16:43 -0700)]
Add minimum features in HDS driver (for Havana & Icehouse)
* Add create_cloned_volume() api.
* Add extend_volume() api.
* Reorganized some connection state keeping into _loc_info() function.
Earlier this logic was spread out in various calls.
* New self tests for #1 and #2 above.
The glanceclient supports a timeout for http/https
connections but this is not configurable in cinder.
This patch adds a glance_request_timeout
option to cinder.conf. If unset, None is applied thus
allowing glanceclient to use its default value (600).
Kurt Martin [Tue, 6 Aug 2013 20:25:36 +0000 (13:25 -0700)]
3PAR volumes created from snaps failed to attach
An error would occur if you attempted to attach a volume that
was created from a snapshot. The 3PAR backends does not have a
'userCPG' entry in the volume data on the backend, it instead
has a 'snapCPG' field. This patch will now look for the 'snapCPG'
entry and use that CPG when attaching the volume created from a
snapshot.
John Griffith [Tue, 6 Aug 2013 20:58:02 +0000 (14:58 -0600)]
Move volume_clear and clear_size opts up to driver
The block_device driver pulled in the clear and clear_size
options for it's own use and broke the inheritance model needed
for multiple backend configs.
This change moves the two configs up higher in the heirarchy to
driver.py so that it can be shared/used by all drivers if they desire.
This means that they are now set properly in volume_opts. At some
point we shoudl restructure this a bit to allow multi-backends to
specify independent settings for these configs.
Kurt Martin [Mon, 5 Aug 2013 21:23:52 +0000 (14:23 -0700)]
Set the concurrent connections on the 3PAR array
Currently, the 3PAR backend has a limit to the number of concurrent
connections that can be made to the WS API Server. This patch is
setting this limit to 15 connections, the highest possible.
Joel Coffman [Tue, 30 Jul 2013 14:06:28 +0000 (10:06 -0400)]
Create key manager interface
This interface provides a thin wrapper around an underlying key
management implementation such as Barbican or a KMIP server. The key
manager interface is used by the volume encryption code to retrieve
keys for volumes.
This change is a copy-paste of the key manager interface accepted by
Nova. The major modifications are to the module imports (i.e.,
nova -> cinder).
The RBD driver supports two methods of image cloning;
thin copy i.e. copy-on-write (requires v2 Glance API)
and full copy (default if v2 Glance API is not available).
clone_image() was failing in the Glance V1 API case because
it was not returning a tuple, as expected by the volume
manager. Glance V2 support works fine.
Code Review / openstack-build / cinder-build.git / log