Up till now we only had copy-on-write for cloning from snapshot. This
change optionally allows clone from volume to use copy-on-write
instead of a doing a full copy each time. This should increase speed
and reduce nearterm storage consumtion but could introduce some new
risks e.g. excessively long clone chains and flatten storms. To avoid
this, a new config option has been providedons are provided -
rbd_max_clone_depth - which allows the user to limit the depth of a
chain of clones i.e.
a->b->c->d as opposed to a->b
->c
->d
This will avoid flatten storms by breaking chains as they are formed
and at an early, predefined stage.
A second option - rbd_clone_from_volume_force_copy - allows the user
to use a full copy as before i.e. disable COW for volume clones.
VMDK copy_image_to_volume and copy_volume_to_image
Implemented copy_image_to_volume that creates a new volume backing (vm) and
copies over the vmdk file from the glance image. Only glance images of disk
format 'vmdk' can be used to create a volume using this driver.
Also implemented copy_volume_to_image that creates a new glance image using
the volume's vmdk file. The steps involved are to take a snapshot of volume
vmdk, take a copy of this read only file and upload to glance.
Cleaned up docstrings as pointed out in earlier reviews.
Added timeout to avoid blocking wait on read/write threads and also fixed a
bug to upload image from copy of vmdk instead of from original vmdk during
upload to glance.
We need to let folks know where brick is
currently being maintained for the Havana
release and where to file defects.
Once this simple patch lands, we'll pull this into
the Nova patch.
(fix hacking violation in commit message for the 2nd
time on this patch. To prevent it from blocking the
merge queue on feature freeze day.)
PLEASE DON'T +A PATCHES UNTIL THEY PASS JENKINS CHECK
Running pep8 tests in Nova gives a violation
in lvm.py for not importing a module. This
is a simple fix to import the module itself
instead of only importing a function in the
module.
Eric Harney [Mon, 19 Aug 2013 04:21:54 +0000 (00:21 -0400)]
QEMU-assisted-snapshots for GlusterFS volumes
Coordinate with Nova to create and delete snaphots for
GlusterFS volumes that are attached to VMs.
Cinder is responsible for creating a QCOW2 file which Nova
will activate in the VM's snapshot chain when a snapshot is
created.
When a snapshot is deleted, Cinder will request for Nova to
perform a block commit/rebase operation to logically delete
the snapshot from the QCOW2 chain.
- move to Zadara APIs 13.07
- added support for extend volume
- added support for create/delete snapshot
- added support for create clones from volumes and snaps
- added support for multi-backend
- added volume stats
- added tests
Set vg_thin_pool to pool name instead of pool_path
create_thin_pool is setting vg_thin_pool to the pool path instead of the
pool_name. This makes volumes creation fail when the create_thin_pool
method is called. This happens because create_volume builds the pool
path itself as create_thin_pool does.
Keeping the pool name in vg_thin_pool instead of the path makes more
sense and allows it to be used in other places in the brick. Also, most
commands return both vg_name and pool_name separated.
The Windows service fails due to missing non-blocking IO features
in eventlet. This fix adds a conditional path on Windows to execute
the service accordingly.
Michael Kerrin [Tue, 3 Sep 2013 12:53:34 +0000 (12:53 +0000)]
Fix errors in volume usage audit script
The snapshot_get_active_by_window DB query was returning snapshots
where we couldn't access the related volume object. This was due to
a) the volume not been loaded and the join failing due to detached
session object.
and b) the related volume was deleted so it didn't show return from
the join
Eric Harney [Fri, 19 Jul 2013 14:02:51 +0000 (10:02 -0400)]
New update_snapshot_status API
Adds new snapshot_actions module
Update_snapshot_status: Allows updating of 'state' and
'progress' fields of a snapshot. This is used by Nova
to inform Cinder of its outcome when performing snapshot
operations for attached volumes. Updates are restricted
to a subset of possible start and finish states.
Huawei OceanStor HVS-series enterprise storage system is an optimum
storage platform for next-generation data centers that feature
virtualization, hybrid cloud, simplified IT, and low carbon footprints.
This patch add an iSCSI driver and a FC driver for Huawei HVS storage
system, using REST. We define a common module for both iSCSI driver and
FC driver. The drivers support volume type, QoS.
zhangchao010 [Sat, 31 Aug 2013 02:24:43 +0000 (10:24 +0800)]
Add Fibre Channel drivers for Huawei storage systems
This is the third patch, changes as follows:
1.Add Fibre Channel drivers for huawei OceanStor T series and Dorado
series arrays. Dorado FC driver inherits codes from FC driver of T.
The FC drivers call module ssh_common which has been defined in
the preview patch: https://review.openstack.org/#/c/41721/
2.Add unit test for the changes.
zhangchao010 [Sat, 31 Aug 2013 02:23:35 +0000 (10:23 +0800)]
Refactor huawei Dorado array iSCSI driver
This is the second patch, changes as follows:
1.Add ISCSIDriver for Dorado arrays. The ISCSIDriver inherit from T.
1.Add a common class DoradoCommon for both FC and iSCSI drivers. The common class
inherit from T common for they have many common functions.
2.Add unit test for Dorado drivers.
zhangchao010 [Thu, 29 Aug 2013 02:33:52 +0000 (10:33 +0800)]
Refactor Huawei iSCSI driver
We plan to refactor Huawei iSCSI drivers and add Huawei FC
drivers. For that's a huge change, we break the codes into
three patches:
1.Refactor T iSCSI driver
2.Refactor Dorado iSCSI driver
3.Add FC drivers for both T and Dorado arrays.
This is the first patch, changes as follows:
1.Define a common class for both FC and iSCSI drivers, and
also provide a unified class HuaweiVolumeDriver for users.
The unified driver will call HuaweiTISCSIDriver according to
users' configuration. The HuaweiTISCSIDriver is a subclass of
driver.ISCSIDriver, so it could get good inheritance.
2.Support volume type.
3.Refactor unit test to make it more logic clear and add more
test cases to get higher coverage rate.
Mathieu Gagné [Sat, 31 Aug 2013 01:59:59 +0000 (21:59 -0400)]
Clone volume with right size with SolidFire
It is possible to clone a volume with a size greater than
the source volume. When using the SolidFire driver, the volume size
was not passed in the call made to the SolidFire API when
cloning a volume.
It resulted in a cloned volume having the same size as the source
volume but the correct size in the Cinder database.
This changes makes sure the cloned volume has the correct size
by explicitly passing the requested size when calling
the SolidFire API and cloning the volume.
Kaitlin Farr [Fri, 30 Aug 2013 19:38:38 +0000 (15:38 -0400)]
Fixes bug to allow for encrypted volume deletion
delete_key in cinder/volume/api.py was called with incorrect parameters.
The error prevented encrypted volumes from being deleted, but the fix
has been applied and unittests written to prove the functionality.
Zhiteng Huang [Fri, 30 Aug 2013 18:59:47 +0000 (02:59 +0800)]
Move comment back to right place
The comment about skipping schedule for create volume from snapshot
in cast_create_volume() was mis-placed when this code was ported to
taskflow. This patch moves the comment back to where it belongs.
Zhongyue Luo [Fri, 30 Aug 2013 05:33:22 +0000 (13:33 +0800)]
Utilizes assertIn and assertNotIn
Using assertTrue and the 'in' operator to test if an element is in a sequence
is too python2.4. Our unit testing framework 'testtools' support assertIn and
assertNotIn, included from python 2.7, which were created for these types of
tests. Let's use assertIn or assertNotIn for these tests.
- Implemented APIs: create_volume, delete_volume, initialize_connection,
create_snapthot, delete_snapshot, create_volume_from_snapshot,
create_cloned_volume
- Modified etc/cinder/cinder.conf.sample adding details for the driver
- Added suds dependency in requirements.txt
- create_volume: does no work. Volume's backing is created lazily
- delete_volume: delete backing if present
- initialize_connection: if backing, not present, create backing, else
provide backing details to nova to perform attach
Nova BP/vmware-nova-cinder-support
- create_snapshot: Creates the snapshot of the backing
- delete_snapshot: Deletes the snapshot of the backing
- create_volume_from_snapshot: Creates a full/linked clone from the snapshot
point in VC. In ESX, copies the VM backing files, registers and reverts to
the appropriate snapshot point.
- create_cloned_volume: Creates a full/linked clone in VC. In ESX, copies the
VM backing files and registers as a new backing.
- Written appropriate unit tests
- Work item in BP/vmware-vmdk-cinder-driver
Julia Varlamova [Thu, 29 Aug 2013 07:56:17 +0000 (11:56 +0400)]
Remove _create_volume function from several tests
Remove _create_volume function from
- db/test_transfers.py
- api/contrib/test_backups.py
- test_gpfs.py
- test_volume_transfer.py
- test_volume.py
And use create_volume from tests/utils.py instead
Kui Shi [Fri, 30 Aug 2013 07:18:10 +0000 (15:18 +0800)]
Don't need to init testr explicitly
In run_tests.sh, function init_testr will initialize testr if the
directory .testrepository is not existed. Actually, testr will do
the check before run the test:
In Python package testrepository, setuptools_command.py:Testr.run
68 def run(self):
69 """Set up testr repo, then run testr"""
70 if not os.path.isdir(".testrepository"):
71 self._run_testr("init")
Kurt Martin [Wed, 28 Aug 2013 21:08:49 +0000 (14:08 -0700)]
Add missing LH SAN driver features for Havana
The LeftHand driver did not support snapshot or extend volume.
The following features are part of the minimum driver features
that should be implemented in every cinder driver. This patch
adds the following missing features;
create snapshot
delete snapshot
create volume from snapshot
extend volume
It also fixes the DocStrings in the unit test so they pass the
checks.
Julia Varlamova [Thu, 29 Aug 2013 07:41:34 +0000 (11:41 +0400)]
Add kwargs to create_volume in tests/utils.py
Add availability_zone parameter and kwargs to make create_volume
function more flexible in order to use this function in
unit tests.
The goal is to remove inner _create_volume function in:
- db/test_transfers.py
- api/contrib/test_backups.py
- test_gpfs.py
- test_volume_transfer.py
- test_volume.py
and to use create_volume from tests/utils instead.
Zhi Yan Liu [Fri, 19 Jul 2013 22:18:24 +0000 (06:18 +0800)]
Adding Read-Only volume attaching support to Cinder
1. Adding an API extension to allow clients set volume Read-Only flag on
demand.
2. Require client to provide and be aware of volume attaching mode when
they call 'os-attach' API.
3. Adding a 'access_mode' field to connection info which
'os-initialize_connection' API returned. This field should be used by
client such as Nova to use correct mode accessing attached volume.
Currently access mode can be 'rw' or 'ro'.
4. In future, the driver within Cinder need to ensure the volume be
exposed under the correct access mode which connection info described,
for example backend should set volume to readonly mode when connection
info ask client using 'ro' access mode consume attached volume. That
means Read-Only is not only a attaching mode but also a status for a
volume.
blueprint read-only-volumes
Change-Id: I4c84614d6541d5f7c358abadb957da7b8c3d9c48 Signed-off-by: Zhi Yan Liu <zhiyanl@cn.ibm.com>
Ben Swartzlander [Wed, 28 Aug 2013 04:43:20 +0000 (21:43 -0700)]
Add NFS/GlusterFS support to brick library
Refactor NFS driver to use the brick library to perform mounts
rather than duplicating the mount code. Some of the NFS unit
test code was rewritten to test the brick implementation because
the old mount code was moved.
The Gluster driver was slightly affected in the refactoring, but
does not yet use brick to perform attaches.
Joshua Harlow [Wed, 21 Aug 2013 01:06:32 +0000 (18:06 -0700)]
Move create_volume flow to a subfolder
In order to start splitting up the create_volume flow into
smaller files we will start by moving the create_volume.py
file to a folder with just __init__.py and this will then
act as the base for create_volume task modules.
Also add in a utils file that contains to start the commonly
used debug listener attachment mechanism.
Zhongyue Luo [Tue, 27 Aug 2013 02:04:47 +0000 (10:04 +0800)]
Import order cleanup
According to
https://github.com/openstack-dev/hacking/blob/master/HACKING.rst#imports
imports must be ordered by the full module path by the below template
{{stdlib imports in human alphabetical order}}
\n
{{third-party lib imports in human alphabetical order}}
\n
{{project imports in human alphabetical order}}
\n
\n
{{begin your code}}
Michael Kerrin [Tue, 27 Aug 2013 10:48:45 +0000 (10:48 +0000)]
Migrate manage script needs import of db session module
This import looks like it is not used but importing this module as
the side effect of defining the olso configuration. Without it the
script will not work.
Avishay Traeger [Mon, 19 Aug 2013 10:45:38 +0000 (13:45 +0300)]
Migration for attached volumes
Enable migration for attached volumes by calling Nova to have the
hypervisor copy the data (Nova's swap_volume feature). A new API
function is added for Nova to call once it is done copying to finish
the Cinder side of the migration.
The overall 'generic' migration flow (i.e., not performed by a driver)
is as follows:
1. Creates a new volume
2a. If the source volume's state is 'available', attach both volumes
and run 'dd' to copy, then detach both. When finished, call the
migrate_volume_completion function.
2b. If the source volume's state is 'in-use', call Nova to perform the
copy. Nova will attach the new volume, copy the data from the
original volume to the new one, and detach the original volume.
When the copy completes, Nova will call Cinder's new
migrate_volume_completion function.
3. The migrate_volume_completion function deletes the original volume,
and calls the database API's finish_volume_migration function. This
copies all of the new volume's information to the original, and
deletes the new volume's row, and thus we can keep using the
original volume_id (the user sees no change).
We also don't change the original volume's status, and instead add a
migration_status column which only selected users can see (e.g., admin).
The migration status is None when no migration is in progress, whether
it succeeded or failed. The admin should check the volume's current
host to determine success or failure. This is meant to simplify
operations. The user will only be aware of a migration if they try to
change the volume's state during the course of a migration.
As mentioned, we change the volume while keeping the original volume
ID. Because a volume's name depends on its ID, the new volume will have
a different name than the original. This is the purpose of the name_id
field in the database - the name is now based on name_id. So although
we keep the original volume's ID, we use the new volume's ID as the
name_id. Thus we can remove the rename_volume function - it is no
longer necessary because the name_id field in the database already
allows for the volume's name on the backend to not rely on its ID.
The user than can see the migration_status can also see the
name_id, in case they need to find it on the backend.
There were a few other places throughout the code that relied on
constructing a volume's name from its ID, and those were fixed.
Add an optimized migration path for when the source and destination are
managed by the same storage. If the source and destination pools have
the same extent_size, it will use the optimal migratevdisk command.
Otherwise, it will use addvdiskcopy and rmvdiskcopy, which must be
watched until the copies are sync'ed.
Code Review / openstack-build / cinder-build.git / log