John Griffith [Mon, 26 Aug 2013 18:23:49 +0000 (12:23 -0600)]
Add automated check of conf sample
This adds check_uptodate to run_tests and
tox.ini checks.
This also requires that we add novaclient to the
test-requires, since we try and poke at cinder/compute/nova.
There is another implementation in use to do this that
pulls in openstack.common.config.generator,
(see ceilometer project for a pretty slick impl), however
adding that also pulls in a rather large list of
updates and dependencies that quite frankly Cinder
doesn't need and the ones that we do use break
a number of things in Cinder.
There's also a number of additional options added
to generate_sample that I don't think we need right now.
In the future we'll probably want to go this route
but for now I'd just like to get automated checking
in place.
John Griffith [Sat, 24 Aug 2013 16:23:29 +0000 (10:23 -0600)]
Sync up with global requirements
This is a pull from openstack/requirements.
Note that pep8, pyflakes and flake8 are now specified
as deps in hacking, so we can remove those here. Also
if we don't remove them then we run into version conflicts
when hacking tries to set things up.
John Griffith [Sat, 24 Aug 2013 15:57:40 +0000 (09:57 -0600)]
Remove print statement in db api test
Remove the encryption related print statements in
test_dp_api, this should bring us up to speed
now for updated requirements and take care of the
py3.3 incompat print issues.
Zhongyue Luo [Mon, 19 Aug 2013 04:03:35 +0000 (12:03 +0800)]
Sync execute() related exceptions with oslo
Cinder has its own ProcessExecutionError exception defined which
already exists in processutils.py. This was done to have minimal
editing when applying processutils.execute() to Cinder. As a result,
new instances are being created for each exception raised during
cli processing using execute()
As for the first step to remove the redundant excptions, this
patch syncs ProcessExecutionError and UnknownArgumentError
with that in processutils to ease the transition.
Michael Kerrin [Fri, 23 Aug 2013 14:42:46 +0000 (14:42 +0000)]
The DB migration shouldn't populate types table
It is up to the system adminisator to do this via an API call.
Also fixes up a few cases of dict() being used rather than the prefered {}
(https://github.com/openstack-dev/hacking/blob/master/HACKING.rst#dictionarieslists)
Avishay Traeger [Thu, 22 Aug 2013 20:17:57 +0000 (23:17 +0300)]
Storwize/SVC: allow setting of I/O group
Allow setting the I/O group via config option or volume type. Also
updated comments and checks regarding iSCSI multipath, which is in Nova
and is not controlled by the driver.
Zhiteng Huang [Sat, 18 May 2013 14:21:28 +0000 (22:21 +0800)]
Implement QoS support for volumes
This patch is to implement generic Quality-of-Service(QoS) support for volumes.
The goal is to add an interface so that cloud/Cinder admins can use to set
volume QoS, which can be enforced either in hypervisor or on Cinder back-end
or both. QoS specifications are added as a standalone (only visible to admin)
entity. So admin can create/update/delete and associate/disassociate QoS
specifications to other entities, in this case volume types.
Note that while it's possible for Cinder to set the granularity of QoS control
to every single volume, this patch puts the control granularity to the level
of volumes of the same type to minimize the impact of other Cinder parts.
In other words, the design is to bond QoS with volume types. So Cinder admin
can associate volume types with QoS specifications, and volumes of same volume
type share the same QoS specifications.
QoS can mean a lot different things that it's unlikely we can come up with a
interpretation that all vendors can agree on. So the approach this
implementation takes is to make Quality-of-Service specs as free-from, i.e.
expressed as key/value pairs.
Changes:
- Add a quality_of_service_specs table, using adjacency list relation to store
a specs entry and its detailed specs in key/values. Note that to be able to
distinguish where should the QoS specs be consumed, each QoS specs entity
will have a 'consumer' (i.e. fixed key) with the value of where admin would
like the QoS policy to be enforced/consumed, currently these three values are
considered valid: 'front-end' (Nova Compute), 'back-end' (Cinder back-end),
'both'. The default value for 'consumer' is 'back-end';
- Add a new API extension 'qos_specs_manage' to allow list/create/update/
delete/associate/disassociate of QoS specs;
- Add volume/qos_specs internal API for qos specs manipulation;
- Add 'qos_specs' info to data structure when
initialize_connection() is called.
- Add 'qos_specs' to request_specs and filter properties for
a volume create request.
TODO
- Modify 'type_manage' API extension to be able to accept qos info.
- Modify volume_types.create() to accept qos info and do the checks.
Joshua Harlow [Wed, 21 Aug 2013 00:47:12 +0000 (17:47 -0700)]
Move the frequently injection task to the base folder.
Key/value injection is a typical way to bootstrap a
dependency based system (as there needs to be an
initial set of values to depend on). Since this inject
task is frequently used to accomplish this we should
just move it to the base file.
Ronen Kat [Thu, 8 Aug 2013 09:12:37 +0000 (12:12 +0300)]
Backup driver for IBM Tivoli Storage manager (TSM)
An implementation of Cinder backup driver using TSM as a backend for
Cinder backups. The driver is a wrapper for the TSM command line
utility dsmc, and uses TSM image backup and restore.
John Griffith [Wed, 21 Aug 2013 19:05:30 +0000 (13:05 -0600)]
Dont crash service if sf cluster isnt available
Currently if the SolidFire driver is configured but the cluster
isn't available for some reason the ensure_export call will raise
an unhandled exception and crash the volume-service.
We should be able to handle things like loosing connectivity to
a single back-end without impacting the other volume-service
backends.
We'll wrap the ensure_export call in a try block here and
return None in the case that the connection can't be made.
This will keep the service from crashing and log an error
message that the connection timed out.
Additional work would include adding a periodic retry task
to the manager to try and start the backend service for us
on some regular interval incase the device comes back.
Kurt Martin [Thu, 22 Aug 2013 16:28:37 +0000 (09:28 -0700)]
3PAR driver add missing domain check on QOS cmd
The 3PAR drivers need to support CPGs that are not part of a
virtual domain on the 3PAR backend. This patch now checks
if the domain is set before calling the createvvset command.
This instance was overlooked in the following patch that has
already been merged: https://review.openstack.org/#/c/42697/
Julia Varlamova [Mon, 12 Aug 2013 08:49:25 +0000 (12:49 +0400)]
Refactor cinder/tests/test_volume.py
Refactor test_volume.py to remove code duplication.
- remove iterant definition of the 'expected' dict from
'test_create_delete_volume' and 'test_create_delete_snapshot'
method;
- move 'test_copy_volume_to_image_status_available',
'test_copy_volume_to_image_status_use' and
'test_copy_volume_to_image_exception' methods to new class
CopyVolumeToImageTestCase;
- move 'test_volume_get_active_by_window' and
'test_snapshot_get_active_by_window' to new class
GetActiveByWindowTestCase;
- create base class BaseVolumeTestCase for TestCase classes.
bp cinder-tests-improvement
This patch adds some cinder utils functions
that wrap calls into brick to automatically
populate cinder's root wrap helper. This
prevents propogating the recreation of the
sudo cinder-rootwrap string.
John Griffith [Mon, 12 Aug 2013 21:50:53 +0000 (21:50 +0000)]
Fix python 3 pep8 errors for print
The new auto-pulled requirements files exposed
python 3 compat issues in a number of modules.
We added these to tox.ini ignore temporarily,
this change updates the print routines in bin/cinder
and removes the debug cruft that was in the unit tests.
Will update requirements and test-requirements
appropriately in a follow up patch.
Jay S. Bryant [Wed, 21 Aug 2013 21:39:04 +0000 (16:39 -0500)]
Fix incorrect msgstr's to avoid translation errors
There are numerous msgstr's in the cinder.po files for en_US,
en_AU and en_GB that are incorrect translations of the associated
msgid. It appears that there are a number that are the result of
problems with past fuzzy translations gone wrong and some may also be the
result of copy/paste mistakes. Regardless of the origin it makes debug and
development very difficult if the string that is coming out in the
logs doesn't match what is expected. This patch fixes the numerous
bad msgstr's.
Once this commit is merged, Transifex will catch the change to
the Cinder po files and then spin a new Transifex Translations import.
Bill Owen [Fri, 16 Aug 2013 22:49:06 +0000 (15:49 -0700)]
GPFS use clone_image for creating volumes
If both source and target of gpfs create volume from image operation
are backed by gpfs storage use clone_image method for implementing
the move of image data to the new volume.
The copy_image_to_volume method is used only if this is not true,
and uses image_utils.fetch_to_raw to move image data to the new
volume.
Kurt Martin [Wed, 21 Aug 2013 16:38:54 +0000 (09:38 -0700)]
3PAR driver terminate connection host validation
The 3PAR backend does not allow FQDN host names (i.e. foo.rose.hp.com),
instead it requires just foo without the rose.hp.com. This patch will
now validate the host name in terminate connection by calling
_safe_host just as it was doing in initialize connection when creating
the host on the 3PAR backend.
Michael Kerrin [Wed, 14 Aug 2013 10:16:42 +0000 (10:16 +0000)]
Re-enable a lot of cinder scheduler tests
This highlight and fixes a NoneType exception in the schedulers
_get_weighted_candidates method
This was originally added to get around the fact that the filter tests
required that cinder was correctly installed on the system. But the
is_cinder_installed condition always returning False and this hid
problems of old packages been installed in our system. Also 930f5891b0815e1b49b9b2cc840e0c24b2796e84 added the setup.py automatically
to run_tests.sh so we can remove this.
See https://review.openstack.org/#/c/20213/ for history.
Kui Shi [Wed, 21 Aug 2013 11:07:43 +0000 (19:07 +0800)]
emit warning while running flake8 without virtual env
run_tests.sh -N -p
it will call the flake8 installed on your host to detect PEP8, and
the flake8 plugin "OpenStack hacking" may not installed on your
host, so this command may not detect the OpenStack Style Commandment
supplied by hacking(e.g H202).
run_tests.sh -p
it will call the flake8 from virtual env, flake8 plugin "OpenStack
hacking" installed in virtual env will be triggered.
The result from "run_tests.sh -p" should be trusted, and jenkins uses
virtual env to run flake8 too.
When "-N" is enabled, emit warning to remind user.
xiaoxi_chen [Mon, 12 Aug 2013 11:25:59 +0000 (19:25 +0800)]
Set bootable flag for volume cloned from image
In previous code we don't set the bootable flag for volume
cloned from image. This does not appear to break booting
from volume at present but the status displayed by the cinder
client is broken.
The bug is reported and fixed several months ago(bug #1185533)
but the author doesn't provide unit test with the patch. Now it
has been broken again by somebody else.The patch re-fixed this
bug together with unit tests.
Also add a function named _handle_bootable_volume_glance_meta,
it is a combination of enable_bootable_flag and glance_metadata
handling. There are 3 kinds of volume creation tasks may required
to copy/fetch the glance metadata. In previous code every kind of
task have its own handling code, this patch consolidate them together
to share some common code.
Ken'ichi Ohmichi [Tue, 20 Aug 2013 03:26:36 +0000 (12:26 +0900)]
Allow to delete a volume in error_extending status
We must be able to remove "error_extending" volume, because we
cannot retrieve it without changing database status.
If trying to remove the volume, now it fails like the following:
$ cinder delete vol-test01
ERROR: Invalid volume: Volume status must be available or error,
but current status is: error_extending
$
"error_extending" also would be error status, so it is good to
remove the volume.
If volume_glance_metadata does not exist for a certain volume,
it returns an exception and does not delete volume from db.
Avoid deleting it if it does not exist.
Luis A. Garcia [Tue, 20 Aug 2013 16:47:38 +0000 (16:47 +0000)]
Use system locale when Accept-Language header is not provided
Remove en_US as the default language when no header is provided, and use
None instead. Upon translation None will be defaulted to system as it
was before the translation changes.
Ronen Kat [Wed, 7 Aug 2013 12:37:07 +0000 (15:37 +0300)]
Generic backup_volume and restore_backup functions
Add implementation for backup_volume and restore_backup to the
VolumeDriver class, which uses brick for attach and detach.
Add default NotImplmeneted implmentation for non block drivers:
remotefs, coraid, gpfs, scality, sheepdog and sm.
Joel Coffman [Tue, 20 Aug 2013 17:02:24 +0000 (13:02 -0400)]
Relax policy so owner can access encryption info
The admin only policy is too restrictive to allow Nova to access a
volume's encryption metadata using the owner's request context. Hence,
this commit relaxes the policy for the volume encryption metadata
API extension so the metadata is accessible to the volume's owner.
Avishay Traeger [Tue, 20 Aug 2013 15:53:47 +0000 (18:53 +0300)]
Fix Fibre Channel attach for single WWN
The code allowed for only a string of the WWN or a list of them.
Unfortunately unicode is also returned, for which the attach fails.
This patch allows for unicode as well.
John Griffith [Fri, 16 Aug 2013 23:21:21 +0000 (17:21 -0600)]
Add accept_transfer to solidfire driver
The SolidFire cluster is tenant aware and as such
needs to change account association on volume transfer.
We use the project_id to build this account, so read that
in here and create a new account if necessary and re-assign
the existing volume to the new tenant account.
John Griffith [Fri, 16 Aug 2013 23:12:11 +0000 (17:12 -0600)]
Added need info to accept_transfer
Drivers that implement accept_transfer will need
things like the new user_id and project_id. We were
not including this in the original add, in order for
drivers that are tenant aware this information will be
necessary.
Also the get_volume call in the transfer was using
the new user context, so the volume would never be
found. We fix this here by providing an elevated
context to the get_volume call when accepting the
transfer.
Alan Jiang [Tue, 20 Aug 2013 05:22:44 +0000 (00:22 -0500)]
Allow volume create from source unless in error status
This patch restores the source volume status checking behavior
to allow create_volume function to proceed unless it is in
error status. In FibreChannel attached environment, it is typical
to have a volume attached before the volume clone starts. The
operating system owns the volume will ensure the consistency.
Code Review / openstack-build / cinder-build.git / log