Victor Stinner [Tue, 9 Feb 2016 10:47:31 +0000 (11:47 +0100)]
Port backup drivers to Python 3
* PosixBackupDriver: open file in binary mode (to read/write).
Update test_backup_posix.py for that.
* test_backup_nfs: buffer() doesn't exist and is no more needed on
Python 3, only use buffer() on Python 2.
* tests-py3.txt: add cinder.tests.unit.backup
scottda [Mon, 8 Feb 2016 23:59:34 +0000 (16:59 -0700)]
Volume create fails with FakeISCSIDriver
Volume create fails when the FakeISCSIDriver is used, because the
scheduler capacity filter detects that capacity on the fake backend
is zero. Change the total_capacity and free_capacity_gb for the
FakeISCSIDriver from 0 to infinite.
Victor Stinner [Tue, 26 Jan 2016 14:48:34 +0000 (15:48 +0100)]
Port objects unit tests to Python 3
* BaseObjectsTestCase._compare(): replace hasattr() with getattr()
and a white list of expected exceptions. On Python 2, hasattr()
ignores *all* exceptions and so may hide real bugs.
* Mocks: except a call to obj.__bool__() rather than
obj.__nonzero__() on Python 3. bool(obj) now calls obj.__bool__()
on Python 3.
* Replace dict.keys() with list(dict.keys()) to get a list on Python
3. On Python 3, dict.keys() now returns a view.
* JSON/base64:
* Replace jsonutils.dumps() with jsonutils.dump_as_bytes() to get
JSON as bytes
* Replace base64.encodestring() with oslo_serialization.base64.encode_as_text() to get
base64 as bytes
* Replace base64.decodestring() with
oslo_serialization.base64.decode_as_text() to accept Unicode and
return Unicode
Anthony Lee [Wed, 9 Dec 2015 00:15:42 +0000 (16:15 -0800)]
LeftHand: Implement un/manage snapshot support
Implements support for managing and unmanaging snapshots to
the HPE LeftHand driver.
This patch now allows snapshots to be removed from OpenStack
management but still left on the LeftHand backend. Snapshots
on the LeftHand backend can also be managed by OpenStack.
Patrick East [Tue, 2 Feb 2016 19:08:39 +0000 (11:08 -0800)]
Allow for eradicating Pure volumes on Cinder delete
This change adds in a config option which, when enabled, will cause
the Pure Storage volume drivers to eradicate volumes, snapshots, and
protection groups on delete for their Cinder counterpart.
With this config enabled an admin will NOT be able to recover data once
objects are deleted. By default the new config option is disabled and
volumes, snapshots, and protection group snapshots will be retained
pending eradication where they can be recovered for a period of time.
Philipp Marek [Thu, 4 Feb 2016 13:47:39 +0000 (14:47 +0100)]
DRBD: Compat for current DRBDmanage versions.
The to-be-1.0 versions may provide additional data via a DM_INFO
tag; but as older versions don't have that (yet), we need to be
careful when asking for its value.
Accela Zhao [Thu, 4 Feb 2016 02:11:09 +0000 (18:11 -0800)]
Fix variable scope issue in try-except
Below piece of code in cinder.backup.api.API.create does not handle
the scope of variable 'backup' properly. In general variable used in
except clause has larger scope than what is in try clause.
The first problem: if object.Backup.__init__ raises exception,
variable 'backup' won't be assigned. However the cleanup code invokes
backup.destroy(), which results in an UnboundLocalError. The original
exception of __init__ is dropped, instead UnboundLocalError is raised
up-level. I know that for now objects.Backup.__init__ doesn't have
chance to raise exception, but when writing code we should not rely
on such assumption.
The second problem: backup.create() invokes DB. If an DB exception is
raised, backup.id won't be assigned. Any access to backup.id will
result in an "NotImplementedError: Cannot load 'id' in the base
class". However, when cleaning up, the backup.destroy() does access
backup.id. So we have the similar problem as the first one, and the
original exception is overrode by NotImplementedError.
This patch fixes it by adding a first-assignment outside try clause,
and add a safe check in except clause. The added test cases, if run
on the original code, it will show how the above problem occurs.
Sheel Rana [Wed, 3 Feb 2016 18:30:11 +0000 (00:00 +0530)]
Bootable filter for listening volumes from CLI
Currently cinder treats all values passed to it as True
for bootable filter in volume GET call.
For ex, if bootable=invalid, it converts it into True being
bootable as boolean value in db table 'volume'.
Need to update filtering functionality in cinder to address
'True' and 'False' values seperately.
Existing behavior will be kept same.
Only True/False/true/false input handling is done to treat
true/false seperately
(In current behavior, whatever input is passed for boolean
parameter, cinder convetrs it to True)
Tom Barron [Wed, 3 Feb 2016 22:14:23 +0000 (17:14 -0500)]
mock time.sleep in Broadcom unit test
TestBrcdHttpFCZoneClient.test_post_zone_data() invokes
BrcdHTTPFCZoneClient.post_zone_data(), which in turn
has a loop around its connect() method inside which there
is a sleep(). Without time.sleep() mocked out, the test
runs 7s on my laptop.
This change mocks out time.sleep() in the test so that
it now runs in .6s.
Accela Zhao [Wed, 3 Feb 2016 07:19:45 +0000 (23:19 -0800)]
Don't create cgsnapshot if cg is empty
If we create cgsnapshot from an empty consistencygroup, it fails
with "Consistency group is empty. No cgsnapshot will be created".
However, a cgsnapshot object is still created in DB.
This is because in cinder/consistencygroup/api.py::_create_cgsnapshot,
we create cgsnapshot object before we test whether the corresponding
consistencygroup is empy.
This patch fixes this problem by moving the creation of cgsnapshot
object to after the test of whether consistencygroup is empty.
This patch fixes the problem with the 3PAR not
working with the generic image cache.
In order to do the image cache, the 3PAR has to create a
temporary snapshot of the volume being cloned. Then the
background cloning is against the temporary snapshot.
The problem is, the temporary snapshot stays around.
We try to account for this temporary snapshot in
the delete volume action.
We also bump the minimum required python-3parclient
version to 4.1.0, which has the new getVolumeSnapshots
API call required to make this all work.
Currently the volume manager doesn't log the instance ID during
volume detach. This is missing in cinder-api log as well. This
patch adds a log statement to print a message with volume ID and
instance ID before calling driver detach.
Wilson Liu [Sat, 23 Jan 2016 02:21:41 +0000 (10:21 +0800)]
Remove useless unit tests mock in Huawei driver
There are some useless ut mocks in Huawei driver,
we'd better to cleanup them.
Also we did some other code cleanup:
1. Cleanup the return value when delete_volume and extend_volume.
2. Check whether the LUN exists in a LUN group firstly besause
this case is more likely to happen than other cases.
Helen Walsh [Wed, 11 Nov 2015 22:09:59 +0000 (22:09 +0000)]
EMC VMAX - Fix for last volume in VMAX3 storage group
This patch fixes bug in the deletion the last volume in a
storage group for the VMAX3. If a masking view did not exist
we tried to get an instance of it which threw an exception.
Now we handle both scenarios correctly.
Michał Dulko [Thu, 14 Jan 2016 19:57:12 +0000 (20:57 +0100)]
Pin RPC and object version to lowest running
This commit adds detection and pinning to lowest RPC API version to all
rpcapi modules. The version pin is determined by a DB call done once per
service lifetime thanks to caching. Handling the compatibility is
guaranteed by oslo.messaging and our shims in rpcapi modules.
To achieve o.vo compatibility, a similar approach is implemented. Custom
oslo.messaging serializer is implemented that backports objects to the
lowest running version on sending.
During the process of upgrade it may happen that manager receives an
object in the version lower than current one. Handling of such
situations is up to the manager and it should do that explicitely by
checking obj.VERSION.
The patch also adds required methods to db.api and Service object.
Michał Dulko [Thu, 14 Jan 2016 18:37:18 +0000 (19:37 +0100)]
Report RPC and objects versions
While doing a rolling upgrade we will have services running in various
versions. In order to determine how to downgrade the RPC request and
payload (objects) to the lowest common version we need to actually
report versions of RPC servers (managers).
This commit implements such reporting in generic cinder.service module.
It is using DB columns that were merged in Liberty to save this
information.
To have a single version string identify a set of o.vo versions we need
to have dictionary with objects versions history. In that purpose a
dict-like CinderObjectVersionsHistory class and OBJ_VERSIONS instance of
it is added to cinder.objects.base. A unit test enforcing bumping the
versions is also included with the patch.
Michał Dulko [Wed, 13 Jan 2016 15:18:45 +0000 (16:18 +0100)]
Add missing RPC calls versions to rpcapi modules
Some RPC calls in rpcapi modules are not passing version indicators to
the RPC client. The RPC call version should be indicated explicitely
because otherwise it gets defaulted to the latest one or '1.0'. This
creates a set of problem in maintaining RPC API backward compatibility.
This commit changes these calls to point to the right versions according
to git history and version comments in rpcapi files.
Wilson Liu [Thu, 24 Dec 2015 04:57:56 +0000 (12:57 +0800)]
Huawei: Balanced FC port selection when zoning
Currently we always select the first two FC ports
from every controller when zoning, the rest FC ports
of the controller will never be selected. This patch
will calculate each port's load and select the first
two smallest load ports.
VNX cinder driver also supports failover back and forth
NOTE: you can only place one replication_device for each backend since
VNX driver only supports 1:1 fanout ratio. Only synchronized replication
is supported in this patch.
replication_device describes the target array for replication. This can
be either another managed cinder backend or one not managed by cinder.
The above settings will cause a protection group to be created between
the two flash arrays. It will replicate any volumes placed in it with
settings that cause a volume to be replicated every 15 mins (900s). All
snapshots will be kept for 4 hours (14400s). Then 3 snapshots per day
will be kept for 7 days.
Note that replication-failover is supported from primary to any of
multiple secondary arrays, but subsequent replication-failover is
only supported back to the original primary.
DocImpact
Implements: bp pure-v2-replication
Change-Id: I69e567143b65e92f89969ff5036c4f242c17a1dd
Tom Barron [Tue, 19 Jan 2016 13:56:51 +0000 (08:56 -0500)]
Remove deprecated options from NFS driver
In the Liberty release, the NFS driver deprecated the 'nfs_used_ratio'
and 'nfs_oversubscription_ratio' configuration options since equivalent
common functionality is now provided across drivers by the scheduler-aware
'reserved_percentage' and 'max_over_sucscription_ratio' configuration
options.
This commit removes these deprecated options, tests for same, and
some Liberty code that proviced backwards compatibility with respect to
these options.
Erlon R. Cruz [Tue, 17 Nov 2015 17:43:01 +0000 (15:43 -0200)]
Fixing HNAS XML parser
HNAS XML parser does not handles situations where the user
could accidentally insert a blank space in the parameter value
breaking the driver in execution time. With this fix, an
error is raised in the driver setup.
jiamin [Wed, 20 Jan 2016 09:24:41 +0000 (02:24 -0700)]
IBM XIV/DS8K: Implements Replication V2
This patch implements cinder Replication version 2 as proposed in
https://blueprints.launchpad.net/cinder/+spec/replication-v2
the changes are:
1. the ability to enable replication on the specified volume
2. the ability to disable replication on the specified volume.
3. the ability to return the list of replication targets for a volume
4. the ability to force failover to a secondary replication target.
Mitsuhiro Tanino [Wed, 16 Dec 2015 01:19:35 +0000 (20:19 -0500)]
Support cinder_img_volume_type in image metadata
This patch adds a feature to specify Cinder volume type
via cinder_img_volume_type parameter in glance image metadata.
In some cases, between image and hypervisor, storage backend
are tightly connected in each other. For example, if users
want to boot a VMware instance from an image, this image has
to be configured some VMware specific parameters in previous,
and then Nova can deploy an instance on to proper hypervisor
based on the image metadata properties.
Currently, Cinder handles few image metadata properties but
volume type is not included. If volume type can be retrieved
from image metadata which is configured by cloud admin, user
doesn't need to care volume type and also storage backends.
And then appropriate volume type will be chosen automatically
based on the information of cinder_img_volume_type in the
image metadata during volume creation.
If user has enough knowledge about image, hypervisor and
storage backend, user also can specify 'volume_type' via
CLI or API as in the past.
Priority of volume type related parameters are shown below.
1. volume_type (via API or CLI)
2. cinder_img_volume_type (via glance image metadata)
3. default_volume_type (via cinder.conf)
DocImpact: Add usage of this parameter to 'Manage volumes'
section in Cloud Administrator Guide
Change-Id: I62f02d817d84d3a7b651db36d7297299b1af2fe3
Angela Smith [Mon, 9 Nov 2015 18:14:35 +0000 (10:14 -0800)]
Adds friendly zone name support
Added user friendly names for zones to include host and storage
names along with ids to easily identify the host and storage port
details.
This is done by extracting the host and storage name from connection
info object in the zone manager. The host and storage names are passed
to the zone drivers to form a friendly name. The method signature has
been changed which accepts host and storage name, with default as None.
Also added test cases to test this support. Changes have been made to
zone manager test code to accommodate the change in signature of some
methods.
Moved get_friendly_zone_name method to utils.py so that it can be used
by both cisco and brocade drivers
Changed cisco driver to accommodate host_name and storage_system
parameters in add/delete connection.
Code Review / openstack-build / cinder-build.git / log