Victor Stinner [Tue, 1 Dec 2015 10:00:03 +0000 (11:00 +0100)]
xio: fix regression in authentication
The change I75a5dee2fd1f8cf45dbf416b0353736a43efd8e8 replaced
base64.encodestring(auth_key)[:-1] with
oslo_serialization.base64.encode_as_text(auth_key)[:-1]. It's wrong
because base64.encodestring() adds '\n' to the output, wheras
oslo_serialization.base64.encode_as_text() doesn't add '\n'.
This change removes '[:-1]' to strip the newline, it's no more
needed.
Gorka Eguileor [Mon, 30 Nov 2015 11:04:58 +0000 (12:04 +0100)]
Take into consideration races in XtremIOClient3
When working with FC and xtremio is using XtremIOClient3 on FC's
terminate_connection we can get VolumeBackendAPIException saying a
resource could not be found when we shouldn't.
The cause is that Client3 is open to race conditions on 2 methods,
find_lunmap and num_of_mapped_volumes, where a list of mappings is first
retrieved and then we iterate this list to retrieved additional info on
the mappings.
The race would happen if one of the mappings is removed from the backend
in the time it takes to retrieve the additional info after we have
retrieved the list.
This patch fixes this issue by ignoring any mappings that have been
removed and are now NotFound when retrieving additional information for
a mapping in those 2 methods.
This patch also fixes this kind of race problems on volume creation
since it uses find_lumap method.
Sean McGinnis [Fri, 27 Nov 2015 20:12:06 +0000 (14:12 -0600)]
Cleanup orphaned code in sqlalchemy API
There were several methods in the sqlalchemy code that were no longer
being used anywhere. While some of these have potential for being useful,
they are not in use and should not be sitting around the codebase waiting
for someone to need them.
Mitsuhiro Tanino [Fri, 13 Nov 2015 20:01:32 +0000 (15:01 -0500)]
Fix quotas issue during volume transfer
When admin transfers a volume with volume type from
one tenant to another, it correctly updates the quota
for the default values, but does not update for the
volume type values.
This patch fixes the problem to update quotas for
the volume type values.
wuyuting [Wed, 25 Nov 2015 02:00:44 +0000 (10:00 +0800)]
Declare multiattach is True in Dell Eqlx driver
An Eqlx volume can be attached to multi instances, however,
this feature is not declared in eqlx driver. As a result,
if a user create a multi-attached volume using eqlx as
backend, they will be failed.
In this commit, we declared multiattach is True in eqlx
driver.
Dirk Mueller [Thu, 29 Jan 2015 13:11:35 +0000 (14:11 +0100)]
RBD: Make snapshot_delete more robust
Since there is no equivalent of a force-snapshot-delete,
the normal snapshot-delete should be more robust to
also handle backend errors. In case the backend does not have
the image snapshot anymore, log an info message and succeed the
operation.
Kendall Nelson [Tue, 17 Nov 2015 18:39:58 +0000 (12:39 -0600)]
Hacking Checks for assertTrue/IsNone()
This patch adds a hacking check to make sure that assertEquals isn't
comparing the result value to True or None. When developers
use assertEquals(None, return_value) the check will catch it when
pep8 runs and suggest using assertIsNone(return_value) instead.
Similar situations will occur when trying to use assertTrue(True,
return_value).
This patch also makes the necessary changes that get caught by the
new hacking check.
Gorka Eguileor [Fri, 21 Aug 2015 17:06:07 +0000 (19:06 +0200)]
Improve metadata update operations
Currently our metadata operations in the DB and the API are less than
optimal, main issues are:
- To update metadata first we get all metadata in the API and add
requested update metadata, then on the DB we retrieve each of the
metadata keys and update one by one contents.
- When deletion of existing keys is requested in the DB we retrieve all
metadata contents and for those that are not in the new metadata we
retrieve the contents from the DB (again) and delete the entry.
This patch changes this so we no longer retrieve metadata in the API,
deletion does not retrieve any metadata from the DB and just makes 1
deletion request to the DB, and for changes we retrieve metadata once
and then issue a bulk change to create new entries and update existing
ones.
To allow atomic state changes across Cinder services we need to
implement a way to easily do compare-and-swap.
This patch adds methods to allow compare-and-swap on DB models and on
Cinder Versioned Objects as well.
Conditions for the compare part of the update can consist of:
- Inclusion: status == 'available'
- Exclusion: status != 'in-use'
- Multi-inclusion: status in ('available', 'error')
- Multi-exclusion: status not in ('attaching', 'in-use')
- Sqlalchemy filters
A complete example of usage would be the compare-and-swap used in volume
delete requests that has to take in consideration not only the status
but the attach and migration status as well as the volume not having
snapshots:
now = timeutils.utcnow()
expected = {'attach_status': db.Not('attached'),
'migration_status': None,
'consistencygroup_id': None}
good_status = ('available', 'error', 'error_restoring',
'error_extending')
if not force:
expected.update(status=good_status)
# Volume cannot have snapshots if we want to delete it
filters = [~sql.exists().where(models.Volume.id ==
models.Snapshot.volume_id)]
It can also be specified whether to save already dirtied fields from the
objects or not and by default (if no expected_values argument is
provided) it will make sure that the entry in the DB has the same values
as the objects we are saving.
We can select values based on conditions using Case objects in the
'values' argument. For example:
Exclusion and multi-exclusion will handle, by default, NULL values like
Python does instead of like SQL does, so NULL values will be considered
different than any non NULL values. That way if we search for something
not equal to 1 we will also get NULL values.
WARNING: SQLAlchemy does not allow selecting order of SET clauses, so
for now we cannot do things like
{'previous_status': model.status, 'status': 'retyping'}
because it will result in both previous_status and status being set to
'retyping'. Issue has been reported [1] and a patch to fix it [2] has
been submitted.
[1]: https://bitbucket.org/zzzeek/sqlalchemy/issues/3541
[2]: https://github.com/zzzeek/sqlalchemy/pull/200
Nicolas Trangez [Wed, 25 Nov 2015 13:45:09 +0000 (14:45 +0100)]
Revert "Add Scality SRB driver"
The Scality SRB kernel-driver is being re-designed to optimize for
specific workloads, which are (initially) not compatible with VM-style
block device access. Not to confuse users, we believe it's advisable to
remove the Cinder SRB driver in the meantime.
John Griffith [Tue, 24 Nov 2015 21:46:19 +0000 (14:46 -0700)]
Update list_replication_targets
We were planning to let the list targets call be a
free for all, and just be a generic dict of info.
This probably isn't such a great idea, inparticular since
we have a required identifier field for each replication
device it makes more sense to just use that in the response
and leave the rest of the info internal.
In the future if there's a need we can always implement a
show command that displays all the dirty details.
This patch changes the identifier name to 'target_device_id'
to be more self describing, and updates the docstrings and
devref docs to reflect the changes and agreed upon changes.
Michał Dulko [Tue, 24 Nov 2015 14:42:17 +0000 (15:42 +0100)]
Move oslo-incubator's scheduler module to cinder
oslo-incubator is ending its life and we should move remaining
dependencies from there to cinder namespace. This commit does so with
openstack.common.scheduler. Apart from that tests from oslo-incubator
repository are added.
vCenter supports different types of datastores such as
VMFS, NFS etc. Currently the VMDK driver supports only
vSAN, VMFS and NFS datastores for storing volume vmdks.
This patch adds a check to skip unsupported datastore
types during volume creation.
Victor Stinner [Sun, 8 Nov 2015 21:36:01 +0000 (22:36 +0100)]
Port IBM storewize_svc driver to Python 3
Change default preferred node in
StorwizeSVCDriver.initialize_connection(). For fiber channel without
multipath, pick the first node from the list of sorted nodes, to have
a determinist preferred node. The list of nodes is created indirectly
using list(set(nodes)) which doesn't have a determinist order. On
Python 3, the hash function is randomized, and so list(set()) has an
unknown order.
A similar change was done for volume throttling in the change
Icf7141f772397c7ac08f0f1e21ad74cb86a06351 to port the code
to Python 3.
Other changes:
* Use assertSetEqual() in test_storwize_svc to compare
initiator_target_map, because conn_wwpns has a random order on
Python 3.
* StorwizeSVCDriver._check_volume_copy_ops(): replace dict.items()
with list(dict.items()) to iterate on items. On Python 3,
dict.items() now returns a view instead of a copy. The loop
modifies the dictionary and a dict must not be modified while
iterating on it, we really need a copy of items.
* StorwizeHelpers: replace a/b with a//b to use integer division on
Python 3.
* tests-py3.txt: add cinder.tests.unit.test_storwize_svc
wuyuting [Tue, 24 Nov 2015 09:01:44 +0000 (17:01 +0800)]
Declare multiattach is True in RBD driver
An RBD volume can be attached to multi instances, however,
this feature is not declared in RBD driver. As a result,
if a user create a multi-attached volume using RBD as
backend, they will be failed.
In this commit, we declared multiattach is True in RBD
driver.
Shay Halsband [Sun, 15 Nov 2015 10:31:00 +0000 (12:31 +0200)]
XtremIO fix attach readonly bug
* Remove the access_mode from connection_info and let the manager decide
on the access mode.
* add a test to make sure access_mode won't be added in the future
The create method of consistency group API requires a list of
volume type ids; volumes belonging to only these types can be added
to the consistency group. The get method of the consistency group API
does not, however, return the volume type id information.
Horizon needs this value for its "Add volumes to consistency group"
panel so that it can filter the "available volumes" list to only
show volumes that match the volume type associated with the consistency
group.
This was fixed by propagating the volume type id information in
the api layer (this information is stored in the db, it just wasn't
being propagated).
APIImpact
The get method of the consistency group API now returns the volume
type ids information.
DocImpact
The screenshots in the admin guide (see
http://docs.openstack.org/admin-guide-cloud/blockstorage-consistency-groups.html)
should be updated to show this new field.
zhangsong [Mon, 9 Nov 2015 15:00:08 +0000 (23:00 +0800)]
Fix the bug of OSError when convert image
When I try to convert a image use image_utils.convert_image() method,
an error occurred like this :
'''
OSError: [Errno 2] No such file or directory:
'sheepdog:10.133.17.61:7000:volume-a0a70f9b-a50e-4369-885f-c41a894c9fe5'
'''
The reason is that in some cluster storage systems, like
ceph/sheepdog, QEMU can access an image directly via their private
protocol, and there’s no need to map an image as a block device on
the host. In this case, the qemu-img convert command may like:
#qemu-img convert -O raw sheepdog:Ip:port:image_name temp_file
#qemu-img convert -O raw rbd:pool_name/image_name temp_file
The source path may be 'sheepdog:Ip:port:image_name' or
'rbd:pool_name/image_name', it doesn't exist in OS. So, when it runs
the os.stat(source) in image_utils.convert_image(source,dest,out_format)
method, an OSError would be raised.
We can use qemu_img_info method instead to resolve this problem, because
the 'qemu-img info' command can always get the image size info which has
support qemu-img tool. Here we capture a ValueError just in case, but it
only need to give a warning message, because the image has been successfully
converted.
Michał Dulko [Mon, 23 Nov 2015 13:13:28 +0000 (14:13 +0100)]
Add guidelines for release notes to devref
Starting from this cycle (and stable Liberty) we're using reno for
release notes. This commit adds guidelines on when a release note is
required to be added.
Michał Dulko [Fri, 20 Nov 2015 11:36:22 +0000 (12:36 +0100)]
Eager load columns in volume_get_active_by_window
All other volume-related methods in db.sqlalchemy.api are eager loading
volume_metadata, volume_admin_metadata (if admin), volume_type,
volume_attachment and consistencygroup. volume_get_active_by_window
wasn't, causing fails because of SQLAlchemy Session being closed when
trying to lazy load volume_metadata. This commit adds missing
options(joinedload()) to this query.
Alex O'Rourke [Fri, 20 Nov 2015 19:25:19 +0000 (11:25 -0800)]
LeftHand: Remove self.db assignment
DB access was removed in a previous patch when it was no longer
needed at the driver level. The self.db assignment was overlooked
and not removed, even though it is no longer being used.
Ryan McNair [Thu, 12 Nov 2015 15:06:32 +0000 (15:06 +0000)]
Deprecate *_multipath_enabled flag for IBM drivers
Deprecate the *_multipath_enabled from the FlashSystem and
Storwize/SVC driver. When enabled, this flag was intended to create
a single mapping between the compute node and the storage device for
FC connections. However, all available storage WWNNs were still being
mapped to the compute node, and instead enabling this flag was
causing a terminate_connection to not cleanup all LUN mappings
(because Nova did not know of all the target_wwns that were actually
mapped). This can be seen by enabling the flag, connecting a volume,
disconnecting the volume and observing the stray LUN mappings under
'/dev/disk/by-path'.
Currently we do not have a way to limit the storage WWNNs which are
mapped during a FC connection, so we will ignore the config to fix
this broken behavior, and then remove the config next release.
Gorka Eguileor [Thu, 27 Aug 2015 23:14:02 +0000 (01:14 +0200)]
Move get_by_id to CinderObject
Currently each Versioned Object needs to implement its own get_by_id,
with this patch they don't need anymore, since it will be included in
the base class CinderObject and it will work for all the objects.
This will help for other things like having a refresh method or
conditional updates in the objects.
Code Review / openstack-build / cinder-build.git / log