John Griffith [Wed, 3 Feb 2016 16:11:58 +0000 (16:11 +0000)]
Replication v2.1 (Cheesecake)
This focuses the replication work on a specific use case,
and eliminates some of the ambiguity in earlier versions.
Additionally this implementation addresses needs for
devices that do replication based on the whole backend-device
or on Pools.
Use case:
DR scenario, where a storage device is rendered inoperable.
This implementation allows the preservation of user data
for those volumes that are of type replication-enabled.
The goal is NOT to make failures completely transparent
but instead to preserve data access while an Admin tries
to rebuild/recover his/her cloud.
It's very important to note that we're no longer interested in
dealing with replication in Cinder at a Volume level. The concept
of have "some" volumes failover, and "others" left behind, proved
to not only be overly complex and difficult to implement, but we
never identified a concrete use-case where one would use failover
in a scenario where some volumes would stay and be accessible on
a primary but other may be moved and accessed via a secondary.
In this model, it's host/backend based. So when you failover,
you're failing over an entire backend. We heavily leverage
existing resources, specifically services, and capabilities.
Anthony Lee [Thu, 14 Jan 2016 18:50:04 +0000 (10:50 -0800)]
LeftHand: Updating minimum client version
Updated HPE LeftHand driver to have a minimum client
version of 2.1.0.
This is a follow up to the following patches:
https://review.openstack.org/#/c/255002/
https://review.openstack.org/#/c/255544/
https://review.openstack.org/#/c/255015/
Adriano Rosso [Wed, 24 Feb 2016 13:13:38 +0000 (10:13 -0300)]
Fix HNAS iSCSI driver attachment
Currently, the HNAS iSCSI driver returns the logical unit number
related to the iSCSI target as a string while initializing the
connection.
This is breaking the attachment feature because os-brick expects
a numeric type.
This patch converts the LUN from string to int in order to solve
this problem.
Chuck Fouts [Thu, 4 Jun 2015 14:28:23 +0000 (10:28 -0400)]
NetApp: Support iSCSI CHAP Uni-directional Auth
This change adds iSCSI CHAP uni-directional authentication support for
NetApp cDOT and 7-Mode iSCSI driver.
Enabling CHAP authentication does not impact an existing iSCSI session.
The iSCSI session needs to be reestablished before CHAP authentication
is initiated.
Davanum Srinivas [Wed, 24 Feb 2016 14:48:05 +0000 (06:48 -0800)]
Avoid hardcoding value from oslo library
In I65b40b9906b75146a0085bbe168f1e6bcae82f21, the
wsgi_default_pool_size was lowered from 1000 to
100. We should not be using internal details of
the oslo libraries.
Many changes to the Cinder REST API require changes to the consumers of the API.
For example, If we need to add a required parameter to a method that is called
by Nova, we'd need both the Nova calling code and the cinderclient that
Nova uses to change. But newer Cinder versions with the change must work with
older Nova versions, and there is no mechanism for this at the moment. Adding
microversions will solve this problem.
With microversions, the highest supported version will be negotiated by a field
in the HTTP header that is sent to the Cinder API. In the case where the field
'versions' is not sent (i.e. clients and scripts that pre-date this change),
then the lowest supported version would be used. In order to ensure that the
API consumer is explicitly asking for a microversioned API, a new endpoint v3
is added, which is identical to API version v2. This means that our new
Cinder API v3 would be the default, and consumers of the API that wished to
use a newer version could do so by using that endpoint and a microversion in
the HTTP header.
New tests for microversioned API features on endpoint /v3 should be added to
cinder/tests/unit/api/v3/ directory. Existing functionality will be tested via
the .../v2/ unit tests.
Victor Stinner [Mon, 15 Feb 2016 15:51:20 +0000 (16:51 +0100)]
Port netapp dataontap driver to Python 3
With this change, all unit tests on volume drivers pass on Python 3.
Changes:
* Replace a/b with a//b to get an integer on Python 3.
* Replace map(...) and filter(...) with a list-comprehension to get a
list on Python 3.
* Replace dict.keys() with list(dict.keys()) to get a list
on Python 3.
* Replace (str, int, float, long) with
six.integer_types + (str, float): long type was removed
from Python 3.
* decode_base32_to_hex(): on Python 3, decode encode_hex_to_base32()
to return a Unicode string.
* convert_es_fmt_to_uuid(): encode/decode on Python 3 to pass the
right types to base64.b32decode() (bytes) and uuid.UUID()
(Unicode).
* fakes.py: Change type of XML document from Unicode to bytes: add
b prefix to literal strings (b'...').
* Replace range(a, b) with list(range(a, b)) to get a list
on Python 3.
* tests-py3.txt: add cinder.tests.unit.volume.drivers, all volume
drivers tests now pass on Python 3.
LisaLi [Fri, 19 Feb 2016 08:28:36 +0000 (09:28 +0100)]
Scalable backup service - Liberty compatibility
To support rolling upgrades we need to make sure that Mitaka's services
are running fine with Liberty's. It gets complicated with backups as
we've strongly reworked them. Main difference is that Mitaka c-bak can
handle backup/restore of any volume and Liberty was restricted to
operate only on volumes placed on the same node.
Now when running in version heterogeneous environment we need to use old
way of backup jobs scheduling and switch to new one (round robin) only
when everything is running Mitaka.
This commit implements that by adding a dummy backup RPC API version
(1.3) that marks the beginning of scalable backups era. Jobs are
scheduled the new way only if every c-bak reports that (or higher)
version.
There are also small changes to volume.rpcapi - to fail fast if some
c-vol services aren't supporting new calls required by scalable backups
feature. This allows us to error out backups with proper message when
upgrade was done in an improper way (in Mitaka we require c-vols to be
upgraded before c-baks).
This commit also includes small changes to CinderObjectSerializer to
block tries to "forwardport" an object when sending it over RPC. If a
service receives an older object it should handle it explicitly.
Related-Blueprint: scalable-backup-service Co-Authored-By: Michal Dulko <michal.dulko@intel.com>
Change-Id: I45324336ba00726d53cfa012e8bd498868919a8c
Wilson Liu [Wed, 24 Feb 2016 07:48:56 +0000 (15:48 +0800)]
Huawei: Log the IP when login fails
Now we support configuring multi-ips,
if one ip login fails, we will try to
connect to other ips. The problem is,
if one ip login fails, the log message
did not tell users which ip fails.
Wilson Liu [Sat, 23 Jan 2016 02:32:43 +0000 (10:32 +0800)]
Huawei: Don't fail when port group does not exist
Currently when we request a port group but the
port group does not exist, an error will be raised.
This is not appropriate, because port group not
exists is a normal case.
Tom Barron [Tue, 23 Feb 2016 02:17:25 +0000 (21:17 -0500)]
Fix invalid uuid warnings in object unit tests
Currently 'tox -epy27 -- --regex cinder.tests.unit.objects' emits
74 FutureWarnings from oslo.versionedobjects about invalid uuids
as documented here[1].
This commit changes these tests to use valid uuids so that when
they are run these warnings are no longer emitted.
Helen Walsh [Tue, 16 Feb 2016 22:33:24 +0000 (22:33 +0000)]
EMC VMAX - not cleaning up HW Resource WWPN initiators
On cleanup, the last volume is unmapped and the masking view and
storage group are deleted. Now the initiator group and the associated
StorageHardwareIDs are also deleted as long as that initiator group
is no longer contained in another masking view.
We had the call in the manager as part of a periodic,
it did nothing at all. Some drivers had comments "no idea
what to do with this", even better we had unit tests that
faked out a call that returned nothing and asserted it was
nothing. ?!?
Mitsuhiro Tanino [Fri, 22 Jan 2016 16:31:25 +0000 (11:31 -0500)]
[LVM] Restore target config during ensure_export
If server crash or reboot happened, LIO target configuration
will be initialized after boot up a server. Currently, Cinder
has a functionality to save LIO target configuration to save
file at several checkpoint.
If LIO target service is configured properly, LIO target
configuration is restored by this service during boot up
a server, but if not, existing in-use volumes would become
inconsistent status after c-vol service starts.
If there is no iSCSI target configuration during
ensure_export, LIO dirver should restore the saved
configuration file to avoid the problem.
Kurt Martin [Wed, 17 Feb 2016 21:58:41 +0000 (13:58 -0800)]
3PAR get host by WWN now handles mixed cases
The 3PAR driver comparison of WWNs will be able to handle
WWNs with upper/lower case differences. For example, the
driver code can now locate a 3PAR host with a WWN of 123abc6789012345 or 123ABD6789012345.
Michael Price [Fri, 23 Oct 2015 20:52:12 +0000 (13:52 -0700)]
NetApp: E-Series remove snapshot limitations
The E-Series backend does not allow snapshots defined under a snapshot
group to be deleted out of order. Only the oldest snapshot image may
be deleted. In order to work around this behavior, we would only define
a single snapshot on each snapshot group. Unfortunately, this would
limit the total number of snapshots to a total of 4 per volume.
In order to remove this limitation without causing deletion errors
when a newer snapshot is deleted, a soft-delete was implemented that
will remove the snapshot from Cinder management and purge the
snapshot from the backend once other dependent snapshots have been
deleted.
In order to ensure that our clone operations have a snapshot group
available, one of the 4 snapshot groups that can be defined on a
volume will be reserved for clone operations and will not be
allowed to be utilized for snapshot creation. This will limit the
total number of snapshots supported per volume to a maximum of 96.
The new 1.4 version of the WebServices proxy is required in order
to support the full snapshot behavior. Since snapshots are such
an integral part of the driver, 1.4 is now required for running
the driver with this release.
Co-Authored-By: Erik Johannes <erik.johannes@netapp.com>
Change-Id: Ie097743b167c1086ecf0536ad696786bdcd241b2
Michał Dulko [Fri, 5 Feb 2016 14:41:01 +0000 (15:41 +0100)]
Add SIGHUP handlers to reset RPC version pins
Adding SIGHUP handlers (by implementing reset from oslo.service) to
cinder-scheduler, cinder-backup and cinder-volume that reset cached RPC
version pins. This is to avoid the need to restart all the services when
upgrade of the deployment is completed.
Some changes go a little deep into the stack, because to reload all the
pins we need to recreate <service>.rpcapi.<service>API objects that are
stored in the memory.
Please note that SIGHUP signal is handled by oslo.service only when
service runs in daemon mode (without tty attached). To test this commit
in DevStack you need to add "&" to the end of the command that starts
the service.
Situation is more complicated with the API service, so we're leaving it
with restart required for now. In the deployments with HA cinder-api is
typically behind a load balancer, so restarting individual nodes
one-by-one should be easy.
DocImpact: Add information on rolling upgrades procedures to the docs.
Implements: blueprint rpc-object-compatibility
Gorka Eguileor [Thu, 18 Feb 2016 12:51:40 +0000 (13:51 +0100)]
Rtstool sets wrong exception message on save
Rtstool does not properly set the message on the RtstoolError exception
on the save_to_file method (save operation) because it's passing
formatting arguments as if it were a LOG call instead of formatting
those same arguments with
%.
There are 2 cases where this can happen, when saving to the default
location on a system that has no target.service installed and the
creation of the default directory fails, and when saving the file
actually fails.
This patch properly formats the message before passing it to the
instantiation of RtstoolError exception.
Victor Stinner [Tue, 24 Nov 2015 16:49:21 +0000 (17:49 +0100)]
Test middleware test_faults to Python 3
* XMLDictSerializer: sort dictionary items to get a reliable XML
output. On Python 3, dictionary items are iterated in a random
order, because the hash function is randomized by default.
* Use byte strings for HTTP body
* On Python 3, decode serialized data to get Unicode
* Add a base TestCase class to factorize the _prepare_xml() method
* tests-py3.txt: add cinder.tests.unit.api.middleware.test_faults
Ryan McNair [Sat, 30 Jan 2016 16:24:32 +0000 (16:24 +0000)]
Split out NestedQuotas into a separate driver
Fixes the following issues with NestedQuotas:
* Requires conf setting change to use nested quota driver
* Enforces default child quota value with volume creation
* Disables the use of -1 to be set for child quotas
* Adds an admin only API command which can be used to validate
the current setup for nested quotas, and can update existing
allocated quotas in the DB which have been incorrectly set
by previous use of child limits with -1
There will be follow-up patches with the following improvements:
* make -1 limits functional for child projects
* cache the Keystone project heirarchies to improve efficiency
Note: ideally validation of nested quotas would occur in the setup
of the nested quota driver, but doing the validation requires a
view of ALL projects present in Keystone, so unless we require Keystone
change to allow "cinder" service user to be able to list/get projects,
we need the admin-only API for validation that should be called by
cloud-admin.
Gorka Eguileor [Tue, 16 Feb 2016 18:03:49 +0000 (19:03 +0100)]
Wait for periodic tasks to stop on exit
When a Service in Cinder exits it stops all operations and waits for
them to complete, and it also stops all periodica tasks created with
FixedIntervalLoop, but it doesn't wait until these are completed.
This patch will wait for all periodic tasks that received the stop
request without raising an exception before exiting.
Nate Potter [Mon, 4 Jan 2016 23:10:19 +0000 (23:10 +0000)]
Roll back reservations quota in RPC if necessary
Currently the latest version of the volume rpcapi passes along
old_reservations in retype whereas older versions do not. In the
case where old_reservations aren't passed, they will be checked again
in c-vol. This patch rolls back the quotas that were checked for
old_reservations if the client can't send the latest version in
the RPCAPI so that they aren't reserved twice.
This is an amendment to this recently merged patch:
Iba24edd8ad824837028353b52c90742df55c9173
LisaLi [Wed, 30 Dec 2015 07:03:10 +0000 (15:03 +0800)]
Scaling backup service
Currently the cinder backup service is tightly coupled to the cinder
volume service in ways that prevent scaling out backup services
horizontally across multiple physical nodes.
This patch is to loosen this coupling to enable backup processes
to run on multiple nodes without having to be colocated with volume
services.
The following works are not included in this patch:
1. Remote attach snapshot.
2. Vendor specific work.
3. Remove current backup_volume in driver.
4. Rolling upgrades.
DocImpact
Change-Id: I743e676372703e74178c79683dd622d530981e04
Partial-Implements: bp scalable-backup-service Co-Authored-By: Tom Barron <tpb@dyncloud.net>
Code Review / openstack-build / cinder-build.git / log