Eric Harney [Thu, 18 Jun 2015 22:06:54 +0000 (18:06 -0400)]
Tests: Make fake_notifier per-instance
Tests have been failing because multiple threads modify the
same global NOTIFICATIONS list. Move the notification storage
to be per-test instance, which is what the tests assume.
This has been broken for a while, but made more obvious by
recent changes to parallelize tests.
Also remove all the extra reset calls we added to work around
this problem over time.
Matt Riedemann [Thu, 18 Jun 2015 15:16:52 +0000 (08:16 -0700)]
Use a hard-coded project_id in racy cinder.tests.unit.test_volume tests
Since concurrent unit test runs have been turned on with ostestr there
are races with the notifications causing the assertions in
test_volume.py tests to fail.
A few attempts have been made at writing an elastic-recheck query to
identify the failures but they are too generic, e.g. querying for
"fake_notifier.NOTIFICATIONS" or "MismatchError 0 != ", which could hit
on anything that fails with those, e.g. anything that uses
"self.assertEqual(0, " in the code could hit that kind of query - and we
want to avoid those types of queries running against changes in the
check queue since they could mask legitimate test failures.
So we add something unique in these tests to identify them from other
test failures in Cinder by using a hard-coded uuid for the project_id so
that if we have an assertion failure with notifications, the
notification payload is dumped (due to commit 4bbe1c87e26150c62e1dc8a2a5b8979896c1ddd9) and we can query on the
assertion failure plus the hard-coded project_id used only in these
tests.
Michal Dulko [Thu, 18 Jun 2015 07:27:26 +0000 (09:27 +0200)]
Add missing Jinja2 to requirements.txt
Recently merged commit I5bbdc0f97db9b0ebd7b48e50ab7869e2ca33aead got in
a dependency on Jinja2 library without adding it to requirements.txt.
The gate was running fine, because Nova already has it in it's
requirements. Problem uncovers when we install DevStack with
--use-venv=True option.
This patch adds reporting of the multiattach capability
in the lvm driver. Currently the Cinder scheduler does filtering
on hosts at create time for the --allow-multiattach option being
passed into the create request from the client. In order for someone
to be able to create an LVM volume that can do multiattach, the LVM
driver needs to report the capability.
This patch's purpose is to allow LVM volumes to be multiattachable
for the folks working on the Nova patches to get Nova to do multiattach.
Without this patch, they can't do end to end testing.
Lee [Fri, 24 Apr 2015 07:50:20 +0000 (15:50 +0800)]
Add volume drivers for Infortrend Storage
Infortrend implement ISCSI and FC volume drivers for
EonStor DS product.
It manages storage by Infortrend CLI tool.
common_cli.py implements the basic Cinder Driver API.
infortrend_fc_cli.py and infortrend_iscsi_cli.py use them to
provide FC and iSCSI specific support.
Support features:
- Volume Create/Delete
- Volume Attach/Detach
- Snapshot Create/Delete
- Create Volume from Snapshot
- Get Volume Stats
- Copy Image to Volume
- Copy Volume to Image
- Clone Volume
- Extend Volume
Rakesh Mishra [Tue, 26 May 2015 20:54:20 +0000 (02:24 +0530)]
Add config option to set max_volume_size_limit
There is a need to limit maximum size of a volume to levels
that the storage infrastructure can handle.
Setting a maximum limit on size of a volume also prevents
a tenant from creating large volumes that have not been tested
and certified to satisfy SLA objectives.
This feature allows admin to set volume size limit for a tenant.
Get default value for volume size limit via config.
The defaults will either come from the default values
set in the quota configuration option or via cinder.conf
if the user has configured default values for quotas there.
The per_volume_size_limit defaults to -1["No Limit"] always
unless changed in cinder.conf by admin
Victor Stinner [Mon, 25 May 2015 15:08:00 +0000 (17:08 +0200)]
Replace xrange() with six.moves.range()
Add "from six.moves import range" to replace the builtin range()
function with six.moves.range() to always create an iterator, instead of
creating a temporary list.
Replace "xrange" with "range".
Don't add the import for ranges of 1024 items or less.
wanghao [Thu, 14 May 2015 07:42:44 +0000 (15:42 +0800)]
Introduce Guru Meditation Reports into Cinder
This commit integrates functionality from the
`openstack.common.report` module into Cinder.
This enables Cinder services to receive SIGUSR1
and print a Guru Meditation Report to stderr.
The required modules were added to
'openstack-common.conf' as well.
It is essentially a copy from implementation of
nova side.
Co-Authored-By: Gauvain Pocentek<gauvain.pocentek@objectif-libre.com> Co-Authored-By: Walter A. Boring IV<walter.boring@hp.com>
Change-Id: Iaf70ade96d4ad4234cc7e88277ed7b52cf458c2a
Kurt Martin [Thu, 11 Jun 2015 20:29:11 +0000 (13:29 -0700)]
Update 3PAR user config help strings
The 3PAR drivers no longer requires a user to have the 'super'
access role. This patch simply updates the help strings for the
3PAR username and password to reflect that an 'edit' role will
be sufficient.
Victor Stinner [Fri, 12 Jun 2015 11:28:38 +0000 (13:28 +0200)]
Fix Python 3 issues in utils
* Replace string.split(key_file, ',') with key_file.split(','):
the string.split() function was removed in Python 3, use the
str.split() method instead
* Fix conf_fixture: import backup_enable_progress_timer and
backup_swift_enable_progress_timer options. This change is required
to run tests using testtools (it's now possible yet to run tests on
Python 3 using testr).
* hash_file(): use b'' (bytes) for the sentinel, not '' (unicode), when
reading the input binary file
* test_hash_file() requires a binary file (bytes), not a text file
(unicode)
* Get the builtin open() function from six.moves.builtins instead of
getting it from __builtin__ to mock the open() function. The
__builtin__ module was renamed to builtins in Python 3.
* sanitize_hostname(): on Python 3, decode back the hostname from Latin1
to work on a native string (Unicode)
* replace reduce() with six.moves.reduce(): the Python 2 builtin
reduce() function was moved to the functools module in Python 3
* convert_version_to_str(): fix integer division, use a//b, not a/b
* convert_version_to_str(): replace reduce() with ''.join() with map(str)
John Griffith [Thu, 11 Jun 2015 14:52:38 +0000 (08:52 -0600)]
Update SolidFire to use target driver model
A while back we broke the inheritance chain of transport and backend
by moving the target drivers outside of the drivers.
This means you can instantiate and use any target (data transport)
method that your device supports without requiring a separate
driver (again, if your device can support it). This was
particularly important for the LVM ref driver and the 5 iSCSI
options that go with it.
This patch converts the SolidFire driver to use the same model. For
the most part it's pretty simple:
* Create an internal class for the target driver
* Instantiate the target driver that you want to use
* wrap the old transport calls that come to the driver to call
the new target driver objects
This exposed some issues with the abc work that was being done.
There are a number of items that are marked as required but are
really only necessary for LVM based drivers. So this also
creates a san/3'rd party iSCSI subclass as well that has the
more appropriate abc settings for such devices.
Mitsuhiro Tanino [Sat, 13 Jun 2015 04:23:40 +0000 (00:23 -0400)]
Wait until service thread is done on service stop
In order to be able to perform graceful shutdown of services, we need
to wait for all the current service threads to finish before exiting
from the service process.
Oslo.service provides this facility through it's wait() method, so we
need to call it in Cinder Service stop() method which gets called when
graceful shutdown is requested by sending the process SIGINT or SIGTERM.
This patch enables the reporting of the
multiattach capability in get_volume_stats.
The scheduler currently filters on the capability
for drivers that support/don't support that ability
to do more than one attachment for a volume.
By reporting multiattach = True, it lets the scheduler
know this driver allows attaching a volume more than once.
Victor Stinner [Fri, 12 Jun 2015 13:21:09 +0000 (15:21 +0200)]
Replace dit.itervalues() with dict.values()
This change adds Python 3 compatibility to the modified code.
The itervalues() method of Python 2 dictionaries was renamed to values() on
Python 3. As discussed on the openstack-dev mailing list, itervalues() must be
replaced with values(), six.itervalues() should not be used. In OpenStack, the
overhead of creating a temporary list with dict.values() on Python 2 is
negligible.
Victor Stinner [Fri, 12 Jun 2015 12:55:37 +0000 (14:55 +0200)]
Replace dict(obj.iteritems() with dict(obj)
Since oslo.db 1.10, it's possible to convert directly a SQLAlchemy
object to a dictionary using dict(obj) instead of having to write
dict(obj.iteritems()).
dict(obj) works also on regular dictionaries to duplicate a dictionary.
Victor Stinner [Fri, 12 Jun 2015 12:53:05 +0000 (14:53 +0200)]
Replace dict.iteritems() with dict.items()
This change adds Python 3 compatibility to the modified code.
Replace also six.iteritems(obj) with obj.items().
The iteritems() method of Python 2 dictionaries was renamed to items()
on Python 3. As discussed on the openstack-dev mailing list, iteritems()
must be replaced with items(), six.iteritems() should not be used. In
OpenStack, the overhead of creating a temporary list with dict.items()
on Python 2 is negligible.
Enables native support for HGST Solutions software Spaces as
Cinder volumes or snapshots.
Each Cinder Volume or Snapshot is mapped to a single HGST Space.
This space may be named differently from the actual volume/snap ID
and so we store the Space Name<->ID mapping in the volume provider_id.
Snapshots are not supported with the current HGST Solutions
software, so they are implemented as heavyweight copies in the
driver.
All Spaces are made visible on the Cinder host for speed of access,
and only the spaces requested by Nova instances are actually made
visible on other members of the cluster.
Not all nodes need SSD storage to take advantage of these volumes,
cinder.conf entries specify which nodes share their local SSDs.
Prerequisites:
HGST Solutions is a software-SAN-like package which allows local
SSDs in a cluster to be combined into a single storage pool.
The driver has a series of configuration options which must be set
in the cinder.conf, prefixed with hgst_*.
Nodes should have the HGST software installed and connected to the
HGST domain prior to rolling out Nova nodes using this storage.
Additional patches required for full functionality (being tracked
under the same blueprint):
OS-brick patch @ https://review.openstack.org/#/c/186588/
Nova patch @ https://review.openstack.org/#/c/186594/
Nova patch required until os-brick<->nova connection finalized
in https://review.openstack.org/#/c/175569/
Code Review / openstack-build / cinder-build.git / log