With the current code, it's possible to end up with a zero-size
persistence file (or even corruption of the contents) if the
process gets killed or the system takes a power outage.
Switch to a "write to temp file and rename" model for writing the
persistence file. This will make it more robust against unclean
process termination or unclean system shutdown.
Eric Harney [Tue, 1 Dec 2015 23:14:17 +0000 (18:14 -0500)]
LIO: Handle initiator IQNs as case insensitive
If there is a mismatch in case between the stored
initiator IQN in the LIO ACL vs. the IQN provided
in a later call, rtstool would fail to find the
existing ACL.
Compare IQNs in a case insensitive manner to ensure
they will always match as intended. (Refer to RFC
3720 section 3.2.6.1 for details on this.)
VMware: Unit test refactoring (image to vol - 1/2)
There are cases where a single test tests multiple methods. This
patch refactors the unit tests for the following methods in the
vmdk module to fix this issue:
* copy_image_to_volume
* _create_volume_from_non_stream_optimized_image
There will be follow-up patches to fix the remaining unit tests.
LisaLi [Thu, 12 Nov 2015 06:08:11 +0000 (06:08 +0000)]
Delete unused codes in rbd.retype
Before rbd.retype is called, encryption and host have
been checked so that encryptions are different and
host are same.
Meanwhile, as extra_spec is not used during the
rbd driver. As a result, even if they are different
rbd.retype can still return True which means no need
to do further migration.
Victor Stinner [Tue, 1 Dec 2015 10:00:03 +0000 (11:00 +0100)]
xio: fix regression in authentication
The change I75a5dee2fd1f8cf45dbf416b0353736a43efd8e8 replaced
base64.encodestring(auth_key)[:-1] with
oslo_serialization.base64.encode_as_text(auth_key)[:-1]. It's wrong
because base64.encodestring() adds '\n' to the output, wheras
oslo_serialization.base64.encode_as_text() doesn't add '\n'.
This change removes '[:-1]' to strip the newline, it's no more
needed.
Volume object was missing fields that were defined on
the SQLAlchemy model as relationships or backrefs such as
consistencygroup, glance_metadata and snapshots.
This commit adds fields representing these relationships.
LisaLi [Fri, 13 Nov 2015 02:09:03 +0000 (02:09 +0000)]
Add retype in lvm driver
During retype operation, it makes sure that volume
types have same encryption, and same backend qos
before moving on.
In lvm driver, no extra_spec and qos are used. As
a result, if retype a volume to same host, it just
needs to return True, and no needs to migrate the
volume.
Gorka Eguileor [Mon, 30 Nov 2015 11:04:58 +0000 (12:04 +0100)]
Take into consideration races in XtremIOClient3
When working with FC and xtremio is using XtremIOClient3 on FC's
terminate_connection we can get VolumeBackendAPIException saying a
resource could not be found when we shouldn't.
The cause is that Client3 is open to race conditions on 2 methods,
find_lunmap and num_of_mapped_volumes, where a list of mappings is first
retrieved and then we iterate this list to retrieved additional info on
the mappings.
The race would happen if one of the mappings is removed from the backend
in the time it takes to retrieve the additional info after we have
retrieved the list.
This patch fixes this issue by ignoring any mappings that have been
removed and are now NotFound when retrieving additional information for
a mapping in those 2 methods.
This patch also fixes this kind of race problems on volume creation
since it uses find_lumap method.
zhangsong [Fri, 13 Nov 2015 16:44:53 +0000 (00:44 +0800)]
Fix the bug of can't get the desired image info
When we use 'qemu-img info' command to get image info, some kinds
of image may show an 'unavailable' disk size. For example:
$qemu-img info sheepdog:volume-1af1afa0-7820-4063-9b90-4fa6161a74a9
image: sheepdog:volume-1af1afa0-7820-4063-9b90-4fa6161a74a9
file format: raw
virtual size: 10G (10737418240 bytes)
disk size: unavailable
$qemu-img info -f rbd rbd:data/foo
image: rbd:data/foo
file format: rbd
virtual size: 10G (10737418240 bytes)
disk size: unavailable
cluster_size: 4194304
The current code didn't consider 'unavailable' size, and it will
raise a ValueError in this case. This patch fixed this bug by
adding the judgment of 'unavailable' size.
Sean McGinnis [Fri, 27 Nov 2015 20:12:06 +0000 (14:12 -0600)]
Cleanup orphaned code in sqlalchemy API
There were several methods in the sqlalchemy code that were no longer
being used anywhere. While some of these have potential for being useful,
they are not in use and should not be sitting around the codebase waiting
for someone to need them.
Mitsuhiro Tanino [Fri, 13 Nov 2015 20:01:32 +0000 (15:01 -0500)]
Fix quotas issue during volume transfer
When admin transfers a volume with volume type from
one tenant to another, it correctly updates the quota
for the default values, but does not update for the
volume type values.
This patch fixes the problem to update quotas for
the volume type values.
wuyuting [Wed, 25 Nov 2015 02:00:44 +0000 (10:00 +0800)]
Declare multiattach is True in Dell Eqlx driver
An Eqlx volume can be attached to multi instances, however,
this feature is not declared in eqlx driver. As a result,
if a user create a multi-attached volume using eqlx as
backend, they will be failed.
In this commit, we declared multiattach is True in eqlx
driver.
Dirk Mueller [Thu, 29 Jan 2015 13:11:35 +0000 (14:11 +0100)]
RBD: Make snapshot_delete more robust
Since there is no equivalent of a force-snapshot-delete,
the normal snapshot-delete should be more robust to
also handle backend errors. In case the backend does not have
the image snapshot anymore, log an info message and succeed the
operation.
Kendall Nelson [Tue, 17 Nov 2015 18:39:58 +0000 (12:39 -0600)]
Hacking Checks for assertTrue/IsNone()
This patch adds a hacking check to make sure that assertEquals isn't
comparing the result value to True or None. When developers
use assertEquals(None, return_value) the check will catch it when
pep8 runs and suggest using assertIsNone(return_value) instead.
Similar situations will occur when trying to use assertTrue(True,
return_value).
This patch also makes the necessary changes that get caught by the
new hacking check.
Gorka Eguileor [Fri, 21 Aug 2015 17:06:07 +0000 (19:06 +0200)]
Improve metadata update operations
Currently our metadata operations in the DB and the API are less than
optimal, main issues are:
- To update metadata first we get all metadata in the API and add
requested update metadata, then on the DB we retrieve each of the
metadata keys and update one by one contents.
- When deletion of existing keys is requested in the DB we retrieve all
metadata contents and for those that are not in the new metadata we
retrieve the contents from the DB (again) and delete the entry.
This patch changes this so we no longer retrieve metadata in the API,
deletion does not retrieve any metadata from the DB and just makes 1
deletion request to the DB, and for changes we retrieve metadata once
and then issue a bulk change to create new entries and update existing
ones.
To allow atomic state changes across Cinder services we need to
implement a way to easily do compare-and-swap.
This patch adds methods to allow compare-and-swap on DB models and on
Cinder Versioned Objects as well.
Conditions for the compare part of the update can consist of:
- Inclusion: status == 'available'
- Exclusion: status != 'in-use'
- Multi-inclusion: status in ('available', 'error')
- Multi-exclusion: status not in ('attaching', 'in-use')
- Sqlalchemy filters
A complete example of usage would be the compare-and-swap used in volume
delete requests that has to take in consideration not only the status
but the attach and migration status as well as the volume not having
snapshots:
now = timeutils.utcnow()
expected = {'attach_status': db.Not('attached'),
'migration_status': None,
'consistencygroup_id': None}
good_status = ('available', 'error', 'error_restoring',
'error_extending')
if not force:
expected.update(status=good_status)
# Volume cannot have snapshots if we want to delete it
filters = [~sql.exists().where(models.Volume.id ==
models.Snapshot.volume_id)]
It can also be specified whether to save already dirtied fields from the
objects or not and by default (if no expected_values argument is
provided) it will make sure that the entry in the DB has the same values
as the objects we are saving.
We can select values based on conditions using Case objects in the
'values' argument. For example:
Exclusion and multi-exclusion will handle, by default, NULL values like
Python does instead of like SQL does, so NULL values will be considered
different than any non NULL values. That way if we search for something
not equal to 1 we will also get NULL values.
WARNING: SQLAlchemy does not allow selecting order of SET clauses, so
for now we cannot do things like
{'previous_status': model.status, 'status': 'retyping'}
because it will result in both previous_status and status being set to
'retyping'. Issue has been reported [1] and a patch to fix it [2] has
been submitted.
[1]: https://bitbucket.org/zzzeek/sqlalchemy/issues/3541
[2]: https://github.com/zzzeek/sqlalchemy/pull/200
Nicolas Trangez [Wed, 25 Nov 2015 13:45:09 +0000 (14:45 +0100)]
Revert "Add Scality SRB driver"
The Scality SRB kernel-driver is being re-designed to optimize for
specific workloads, which are (initially) not compatible with VM-style
block device access. Not to confuse users, we believe it's advisable to
remove the Cinder SRB driver in the meantime.
John Griffith [Tue, 24 Nov 2015 21:46:19 +0000 (14:46 -0700)]
Update list_replication_targets
We were planning to let the list targets call be a
free for all, and just be a generic dict of info.
This probably isn't such a great idea, inparticular since
we have a required identifier field for each replication
device it makes more sense to just use that in the response
and leave the rest of the info internal.
In the future if there's a need we can always implement a
show command that displays all the dirty details.
This patch changes the identifier name to 'target_device_id'
to be more self describing, and updates the docstrings and
devref docs to reflect the changes and agreed upon changes.
Michał Dulko [Tue, 24 Nov 2015 14:42:17 +0000 (15:42 +0100)]
Move oslo-incubator's scheduler module to cinder
oslo-incubator is ending its life and we should move remaining
dependencies from there to cinder namespace. This commit does so with
openstack.common.scheduler. Apart from that tests from oslo-incubator
repository are added.
vCenter supports different types of datastores such as
VMFS, NFS etc. Currently the VMDK driver supports only
vSAN, VMFS and NFS datastores for storing volume vmdks.
This patch adds a check to skip unsupported datastore
types during volume creation.
Victor Stinner [Sun, 8 Nov 2015 21:36:01 +0000 (22:36 +0100)]
Port IBM storewize_svc driver to Python 3
Change default preferred node in
StorwizeSVCDriver.initialize_connection(). For fiber channel without
multipath, pick the first node from the list of sorted nodes, to have
a determinist preferred node. The list of nodes is created indirectly
using list(set(nodes)) which doesn't have a determinist order. On
Python 3, the hash function is randomized, and so list(set()) has an
unknown order.
A similar change was done for volume throttling in the change
Icf7141f772397c7ac08f0f1e21ad74cb86a06351 to port the code
to Python 3.
Other changes:
* Use assertSetEqual() in test_storwize_svc to compare
initiator_target_map, because conn_wwpns has a random order on
Python 3.
* StorwizeSVCDriver._check_volume_copy_ops(): replace dict.items()
with list(dict.items()) to iterate on items. On Python 3,
dict.items() now returns a view instead of a copy. The loop
modifies the dictionary and a dict must not be modified while
iterating on it, we really need a copy of items.
* StorwizeHelpers: replace a/b with a//b to use integer division on
Python 3.
* tests-py3.txt: add cinder.tests.unit.test_storwize_svc
wuyuting [Tue, 24 Nov 2015 09:01:44 +0000 (17:01 +0800)]
Declare multiattach is True in RBD driver
An RBD volume can be attached to multi instances, however,
this feature is not declared in RBD driver. As a result,
if a user create a multi-attached volume using RBD as
backend, they will be failed.
In this commit, we declared multiattach is True in RBD
driver.
Shay Halsband [Sun, 15 Nov 2015 10:31:00 +0000 (12:31 +0200)]
XtremIO fix attach readonly bug
* Remove the access_mode from connection_info and let the manager decide
on the access mode.
* add a test to make sure access_mode won't be added in the future
The create method of consistency group API requires a list of
volume type ids; volumes belonging to only these types can be added
to the consistency group. The get method of the consistency group API
does not, however, return the volume type id information.
Horizon needs this value for its "Add volumes to consistency group"
panel so that it can filter the "available volumes" list to only
show volumes that match the volume type associated with the consistency
group.
This was fixed by propagating the volume type id information in
the api layer (this information is stored in the db, it just wasn't
being propagated).
APIImpact
The get method of the consistency group API now returns the volume
type ids information.
DocImpact
The screenshots in the admin guide (see
http://docs.openstack.org/admin-guide-cloud/blockstorage-consistency-groups.html)
should be updated to show this new field.
Helen Walsh [Wed, 18 Nov 2015 16:11:30 +0000 (16:11 +0000)]
EMC VMAX - Change naming convention for MV and SG for FAST
Up until now we assumed a one to one relationship between FAST policy
and Pool. This does not have to be the case. A FAST policy can span
multiple pools. To ensure uniqueness we will change the SG nand MV
name to contain the FAST policy and not the pool name.
zhangsong [Mon, 9 Nov 2015 15:00:08 +0000 (23:00 +0800)]
Fix the bug of OSError when convert image
When I try to convert a image use image_utils.convert_image() method,
an error occurred like this :
'''
OSError: [Errno 2] No such file or directory:
'sheepdog:10.133.17.61:7000:volume-a0a70f9b-a50e-4369-885f-c41a894c9fe5'
'''
The reason is that in some cluster storage systems, like
ceph/sheepdog, QEMU can access an image directly via their private
protocol, and there’s no need to map an image as a block device on
the host. In this case, the qemu-img convert command may like:
#qemu-img convert -O raw sheepdog:Ip:port:image_name temp_file
#qemu-img convert -O raw rbd:pool_name/image_name temp_file
The source path may be 'sheepdog:Ip:port:image_name' or
'rbd:pool_name/image_name', it doesn't exist in OS. So, when it runs
the os.stat(source) in image_utils.convert_image(source,dest,out_format)
method, an OSError would be raised.
We can use qemu_img_info method instead to resolve this problem, because
the 'qemu-img info' command can always get the image size info which has
support qemu-img tool. Here we capture a ValueError just in case, but it
only need to give a warning message, because the image has been successfully
converted.
Code Review / openstack-build / cinder-build.git / log