Michał Dulko [Fri, 5 Feb 2016 14:41:01 +0000 (15:41 +0100)]
Add SIGHUP handlers to reset RPC version pins
Adding SIGHUP handlers (by implementing reset from oslo.service) to
cinder-scheduler, cinder-backup and cinder-volume that reset cached RPC
version pins. This is to avoid the need to restart all the services when
upgrade of the deployment is completed.
Some changes go a little deep into the stack, because to reload all the
pins we need to recreate <service>.rpcapi.<service>API objects that are
stored in the memory.
Please note that SIGHUP signal is handled by oslo.service only when
service runs in daemon mode (without tty attached). To test this commit
in DevStack you need to add "&" to the end of the command that starts
the service.
Situation is more complicated with the API service, so we're leaving it
with restart required for now. In the deployments with HA cinder-api is
typically behind a load balancer, so restarting individual nodes
one-by-one should be easy.
DocImpact: Add information on rolling upgrades procedures to the docs.
Implements: blueprint rpc-object-compatibility
Gorka Eguileor [Thu, 18 Feb 2016 12:51:40 +0000 (13:51 +0100)]
Rtstool sets wrong exception message on save
Rtstool does not properly set the message on the RtstoolError exception
on the save_to_file method (save operation) because it's passing
formatting arguments as if it were a LOG call instead of formatting
those same arguments with
%.
There are 2 cases where this can happen, when saving to the default
location on a system that has no target.service installed and the
creation of the default directory fails, and when saving the file
actually fails.
This patch properly formats the message before passing it to the
instantiation of RtstoolError exception.
Nate Potter [Mon, 4 Jan 2016 23:10:19 +0000 (23:10 +0000)]
Roll back reservations quota in RPC if necessary
Currently the latest version of the volume rpcapi passes along
old_reservations in retype whereas older versions do not. In the
case where old_reservations aren't passed, they will be checked again
in c-vol. This patch rolls back the quotas that were checked for
old_reservations if the client can't send the latest version in
the RPCAPI so that they aren't reserved twice.
This is an amendment to this recently merged patch:
Iba24edd8ad824837028353b52c90742df55c9173
LisaLi [Wed, 30 Dec 2015 07:03:10 +0000 (15:03 +0800)]
Scaling backup service
Currently the cinder backup service is tightly coupled to the cinder
volume service in ways that prevent scaling out backup services
horizontally across multiple physical nodes.
This patch is to loosen this coupling to enable backup processes
to run on multiple nodes without having to be colocated with volume
services.
The following works are not included in this patch:
1. Remote attach snapshot.
2. Vendor specific work.
3. Remove current backup_volume in driver.
4. Rolling upgrades.
DocImpact
Change-Id: I743e676372703e74178c79683dd622d530981e04
Partial-Implements: bp scalable-backup-service Co-Authored-By: Tom Barron <tpb@dyncloud.net>
Victor Stinner [Tue, 9 Feb 2016 13:36:45 +0000 (14:36 +0100)]
Port API v1 and v2 to Python 3
* Replace jsonutils.dumps() with jsonutils.dump_as_bytes() since
output is used for the HTTP body, and HTTP body type is bytes, not
Unicode. On Python 3, jsonutils.dumps() returns Unicode.
* FakeHttplibSocket: replace io.StringIO with io.BytesIO, HTTP body
type is bytes. Encode Unicode to UTF-8.
* Fix FakeHttplibSocket.makefile() API: only the first parameter is
mandatory. On Python 3, it's only called with the first 'mode'
parameter.
* Use bytes strings to test HTTP bodies (JSON or XML).
* Replace dict.values() with list(dict.values()) to get a list on
Python 3.
* Replace error.message with encodeutils.exception_to_unicode(error),
exceptions loose their message attribute in Python 3.
* Update JSON serializer test in test_wsgi.py to use bytes.
* tests-py3.txt: add cinder.tests.unit.api.v1 and
cinder.tests.unit.api.v2
Victor Stinner [Tue, 9 Feb 2016 17:40:21 +0000 (18:40 +0100)]
Port API contribs to Python 3
* Replace dict.iteritems() with dict.items(), dict.iteritems() was
removed in Python 3.
* Replace dict.values() with list(dict.values()) to get a list on
Python 3.
* HTTP body type must be bytes:
- Fix unit tests to use bytes strings for HTTP body
- Encode XML to UTF-8 on Python 3.
* Use "%r" instead of "%s" to format request and response in test to
avoid BytesWarning on Python 3.
* tests-py3.txt: add cinder.tests.unit.api.contrib
LisaLi [Thu, 24 Dec 2015 03:16:26 +0000 (11:16 +0800)]
Add restore_volume_id in backup
This patch is to add restore_volume_id in backup object.
When restoring a volume from a backup, it saves the
volume in backup object.
Currently volume service and backup service are in same host.
When backup service starts, it does cleanup tasks on both
backups and volumes on current host.
But with bp scalable-backup-service, backup service and
volume services can run on different hosts. When doing cleanup
tasks, we need to find out backing-up and restoring volumes
related to the backups on current host. Backing-up volumes can
be found with field backup.volume_id. Restoring volumes are found
by new field backup.restore_volume_id.
Change-Id: I757be7a5e47fc366c181400587b5a61fe3709a0b
Partial-Implements: bp scalable-backup-service Co-Authored-By: Tom Barron <tpb@dyncloud.net>
Tom Barron [Fri, 12 Feb 2016 16:48:16 +0000 (11:48 -0500)]
Fix 'asert' typo in unit test
The dellscapi unit tests contain two occurences of an
apparent typo, 'mock_get_volume_list.asert_called_once_with'.
Because mock will magically create an 'asert_called_once_with'
method, this typo masks a problem with the test_manage_unmanage
test where the wrong set of arguments is expected.
This commit substitutes 'assert' for 'asert' in these occurences
and fixes up the test_manage_unmanage test.
Cory Stone [Mon, 8 Feb 2016 17:22:59 +0000 (11:22 -0600)]
Fix dynamic import of CONF.volume_api_class
Modifies the import to actually happen when API() is called instead of
when the module is loaded.
This is necessary because CONF.volume_api_class is always evaluated as
the default value since the config file hadn't been parsed yet when the
module is loaded.
Nova fixed this problem a while ago for the compute driver in
https://review.openstack.org/#/c/14353/.
Javeme [Thu, 11 Feb 2016 12:09:41 +0000 (20:09 +0800)]
Don't use Mock.called_once_with that does not exist
class mock.Mock does not exist method called_once_with, it just exists
method assert_called_once_with. Currently there are still ome places
where we use called_once_with method, this patch let's correct it.
NOTE: called_once_with() does nothing because it's a mock object.
Helen Walsh [Fri, 15 Jan 2016 15:27:11 +0000 (15:27 +0000)]
EMC VMAX - necessary updates for CG changes
Some of the stubs for the Consistency Group functionality
have changed to remove the db object. As a consequence the
VMAX implementation for these operations needs to be
changed.
Eric Harney [Wed, 10 Feb 2016 19:27:49 +0000 (14:27 -0500)]
Profiler: make it possible to run without loading osprofiler
This makes it possible to run Cinder without having
osprofiler installed. This is primarily to make it easier
to debug issues and ensure that osprofiler is not
introducing problems when debugging CI failures.
Note that this requires disabling osprofiler in api-paste.ini
as well.
Sean McGinnis [Wed, 10 Feb 2016 04:13:19 +0000 (22:13 -0600)]
Remove Cisco FC Zone Manager Driver
After many attempts to get a Cisco CI running to validate the Cisco FCZM
driver over a few releases there is still no official third party CI
reporting. This was covered for some time due to the support of it being
covered by another CI, but that is no longer the case.
We can't include drivers that do not have test coverage and no report of
whether it is working as expected. Until there is test coverage, we will
need to remove this driver due to lack of CI.
Clinton Knight [Mon, 11 Jan 2016 21:13:03 +0000 (16:13 -0500)]
Add variable QoS to NetApp cDOT drivers
Most QoS implementations involve fixed limits, such as maxIOPS
or maxBPS. But there are clouds that offer QoS limits that are
partly based on capacity of the underlying resource. This commit
adds two new QoS flags to the NetApp cDOT drivers, maxIOPSperGiB
and maxBPSperGiB, which implement this capability. A light
refactor of the snapshot delete paths was required to separate
those from the volume delete paths, since only the latter should
involve the QoS logic.
Victor Stinner [Tue, 9 Feb 2016 17:38:11 +0000 (18:38 +0100)]
py3: Fix usage of JSON in API contrib
Replace json.dumps() and jsonutils.dumps() with
jsonutils.dump_as_bytes() to get bytes on Python 3. The result is
used as HTTP body, and the body type must be bytes.
Eric Harney [Tue, 9 Feb 2016 16:57:32 +0000 (11:57 -0500)]
Log stack trace for middleware faults
If an error occurs in the middleware layer,
such as an exception generated by keystoneclient,
there is no stack trace provided, leaving the
issue difficult to debug.
Log a stack trace here with LOG.exception instead.
Victor Stinner [Tue, 9 Feb 2016 10:47:31 +0000 (11:47 +0100)]
Port backup drivers to Python 3
* PosixBackupDriver: open file in binary mode (to read/write).
Update test_backup_posix.py for that.
* test_backup_nfs: buffer() doesn't exist and is no more needed on
Python 3, only use buffer() on Python 2.
* tests-py3.txt: add cinder.tests.unit.backup
Vincent Hou [Tue, 2 Feb 2016 19:20:02 +0000 (14:20 -0500)]
Storwize: Implement v2 replication
Storwize supports three major types for volume replications:
split IO, global mirror and metro mirror.
This patch is dedicated to implement the replication for
the modes of global mirror and metro mirror. Mirror
establishes a Global/Metro Mirror relationship between
two volumes of equal size. The volumes in a Mirror
relationship are referred to as the primary volume and
the replica volume. The replication_mode in
replication_device must be set to global or metro.
The volume type needs to associate with the extra spec
with 'replication_enabled' equaling to "<is> True", and
'replication_type' equaling to '<in> global' or '<in>
metro'.
What is supported with replication:
* create volume
* create volume from snapshot
* clone a volume
When a volume is created and replication is enabled, the
replica volume is also created on the back-end enabled
for replicas.
What is not supported with replication yet:
* volume migration
* volume retype
The replica volume will not be created or moved after migration
or retype of a replicated volume. Admins should be aware,
when migrating or retyping a volume to a type with replication
enabled, that the replica will not be automatically created.
The replication can be configured via either multi-backend
on one cinder volume node, or on separate cinder volume
nodes.
Options to be put in cinder.conf, where the primary back-end
is located:
scottda [Mon, 8 Feb 2016 23:59:34 +0000 (16:59 -0700)]
Volume create fails with FakeISCSIDriver
Volume create fails when the FakeISCSIDriver is used, because the
scheduler capacity filter detects that capacity on the fake backend
is zero. Change the total_capacity and free_capacity_gb for the
FakeISCSIDriver from 0 to infinite.
Code Review / openstack-build / cinder-build.git / log