Alan Jiang [Thu, 3 Oct 2013 22:03:09 +0000 (17:03 -0500)]
long flashcopy operation may block volume service
Storwize family uses flashcopy for snapshot or volume clone. The
volume delete has to wait until flashcopy finishes or errors out.
The _delete_vdisk() will poll volume FlashCopy status in a loop.
This may block volume serivce heartheat since it is in the same
. The solution is to use openstack FixedIntervalLoopingCall
to run the FlashCopy status poll in a timer thread.
The cinder volume mananger will resume delete operation for those
volumes that are in the deleting state during volume service startup.
Since Storwize volume delete may wait for a long time, this can cause
volume service to have long delay before it becomes available.
A greenpool is used to offload those volume delete operations.
zhangchao010 [Wed, 9 Oct 2013 07:37:49 +0000 (15:37 +0800)]
Support Huawei driver upgrade from grizzly to havana
To make the driver upgrade from grizzly to higher versions, the patch
checks the old host name firstly when creating and deleting host in
Huawei storage system.
If the old host name exists:
*get and use it directly when adding host.
*delete it directly when deleting host.
The current implementation perform snapshot of attached volume.
This is invalid because when we take snapshot of the backing, the instance
continues pointing to the readonly delta and tries writing to it.
This patch will disallow snapshot creation of attached volume.
We have similar issue with delete of snapshot, when deleted the instance will
point to an invalid disk file, since that will get merged with its parent.
Hence any writes by the instance will fail with disk not found error.
This patch will ignore delete snapshot of attached volume.
Since we disallow snapshot of attached volume, we disallow linked clone of
attached source volume and also upload of attached volume.
Zhongyue Luo [Tue, 3 Sep 2013 05:07:21 +0000 (13:07 +0800)]
Clean up comparison assertions
Using assertTrue and the comparison operators to test
if an element is greater, equal, or less is too python2.4.
Our unit testing framework 'testtools' supports
- assertEqual
- and testtools Matcher classes
which were created for these types of tests.
Re-implemented assertGreater and assertGreaterEqual
with compatibility for python < 2.7.
Zhongyue Luo [Wed, 9 Oct 2013 01:54:30 +0000 (09:54 +0800)]
Utilizes assertIsNone and assertIsNotNone
Using assertEqual/assertNotEqual to test if an element
is or is not None is too python2.4.
Our unit testing framework supports assertIsNone and assertIsNotNone
which were created for these types of tests.
KIYOHIRO ADACHI [Tue, 8 Oct 2013 07:28:25 +0000 (16:28 +0900)]
remove unused methods in driver.Scheduler
cinder.scheduler.host_manager.HostManager doesn't have get_host_list()
and get_service_capabilities(). But we don't catch the AttributeError
by them. Therefore, get_host_list() and get_service_capabilities() in
cinder.scheduler.driver.Scheduler have not been used.
Also, get_host_list() and get_service_capabilities() in
cinder.scheduler.manager.SchedulerManager have not been used.
Dan Prince [Mon, 7 Oct 2013 18:09:34 +0000 (14:09 -0400)]
Ignore H803 from Hacking.
H803 checks for a period in commit messages, something the community
clearly rejected as a sensible check: periods in commit messages
should neither be rejected or required.
Dan Prince [Mon, 7 Oct 2013 16:41:28 +0000 (12:41 -0400)]
Drop conf_key_mgr warning message!
By default ConfKeyManager logs tons of WARNING message stating
that it isn't production ready...
Given that it is currently the only Cinder key manager option
which can be used/selected I don't think repeatedly logging
warnings is helpful. Lets just drop the warning message
for now and when a good "production ready" cinder key manager
implementation is implemented perhaps we can re-add a warning to
this class (hopefully making the production ready impl the default).
Ollie Leahy [Fri, 4 Oct 2013 14:56:26 +0000 (14:56 +0000)]
Provide user with more information on quota fail
Provide the user with useful information when a snapshot or volume
create fails because it would cause the user to exceed available quota.
Specifically report the user's current gigabyte usage and quota
allocations.
John Griffith [Thu, 3 Oct 2013 21:54:20 +0000 (15:54 -0600)]
Fix broken solidfire create-snapshot
A previous change to enable secifying size for cloned volumes
created introduced a bug that breaks the ability to create-snapshots
when using the SolidFire driver
(CID: I5628c7fa922780d6b0601e2daa79310a61085edc).
The problem is that we use a shared method in the SF driver for both
create-clone and create-snapshot, the added change only considers the
clone case and does a get ref['size'] but in the case of snapshots
this needs to be ref['volume_size'].
Joel Coffman [Tue, 24 Sep 2013 23:10:09 +0000 (19:10 -0400)]
Add key manager implementation with static key
Per feedback received on other patch sets, an example key manager
driver is required to support ephemeral storage encryption and
Cinder volume encryption -- see
* https://blueprints.launchpad.net/nova/+spec/encrypt-cinder-volumes
* https://blueprints.launchpad.net/nova/+spec/encrypt-ephemeral-storage
The ConfKeyManager class reads its key from the project's
configuration file and provides this key for *all* requests. As
such, this key manager is insecure but allows the aforementioned
encryption features to be used without further integration effort.
To clarify the above statements, the configuration-based key
manager uses a single, fixed key. When used to encrypt data (e.g.,
by the Cinder volume encryption feature), the encryption provides
limited protection for the confidentiality of data. For example,
data cannot be read from a lost or stolen disk, and a volume's
contents cannot be reconstructed if an attacker intercepts the iSCSI
traffic between the compute and storage host. If the key is ever
compromised, then any data encrypted with the key can be decrypted.
This commit copies the ConfKeyManager class from Nova as well as
synchronizing changes with the key manager interface in Nova.
John Griffith [Thu, 26 Sep 2013 21:23:01 +0000 (15:23 -0600)]
Remove need for CONF acces in brick iscsi
At some point we'd like brick to be a standalone lib,
and as such we don't want to have a requirement for
CONF files and having duplicate conf entries across
projects.
The better approach would be to let the projects decide
what they want to use, and how they want defaults to be set
and then pass those settings in via __init__ or when calling
the methods that need them.
Alan Jiang [Tue, 1 Oct 2013 05:38:03 +0000 (00:38 -0500)]
Storwize SVC driver hostname can't start with number
Storwize family storage does not allow host identifier start with
a numeric number. Otherwise cinder initialize_connection() API will
throw exception due to such invalid host name in the connector. The fix
is to add a '_' symbol as prefix if the host identifier in connector
starts with non-alphabet letters.
Joshua Harlow [Tue, 1 Oct 2013 22:53:30 +0000 (15:53 -0700)]
After commiting quota we should avoid certain reverts
After we commit the quota successfully we do not want to
set the database volume to destroyed or attempt further
rollback of the quota itself to reflect what the code
previously did.
This is part 1 of the work needed to
remove CONF from the brick subproject.
This patch removes the CONF usage
completely from the initiator portion of brick.
Doug Hellmann [Tue, 1 Oct 2013 22:01:59 +0000 (18:01 -0400)]
Disable lazy translation
Late in the Havana cycle bug 1225099 was found in the lazy
translation code, and to be safe it was decided to disable lazy
translation for Havana. This change does that.
John Griffith [Tue, 1 Oct 2013 18:05:14 +0000 (12:05 -0600)]
Check for backing lun on iscsi target create
Check to verify the backing lun was actually created and not just
the controller lun. If it was NOT created, attempt to issue
tgtadm --op new to see if we can recover.
If this fails, then we want to actually fail in Cinder rather than
pretending that everything went well, so we'll log the error and raise.
For some reason, using non-rbd volume driver whilst using
Ceph backup causes what looks to be a cyclic import error.
Ensuring that the backup.drivers.ceph module is imported
into volume.drivers.rbd ONLY if it is needed i.e. if both
drivers are in use, seems to remedy this.
Bill Owen [Fri, 27 Sep 2013 23:22:41 +0000 (16:22 -0700)]
v2 api - return bootable attr value on volume list
Add bootable to the list of values returned by display_list. This
was returned in v1 api, and is still in v2 header, but was missed
in the new implementation.
The backup service now keeps a dictionary of managers
for each type of volume backend. If multi backend is
not in use (and volume_type is None) the current
volume_driver is used.
John Griffith [Mon, 30 Sep 2013 14:45:52 +0000 (08:45 -0600)]
Dont retry if target creation succeeds
The target creation retry loop was not breaking when
the target was succesfully created. This is "sometimes" ok,
however in other cases this will actually cause a failure in
the second create and the backing Lun will be deleted and not
created again succesfully due most likely to a busy status on the
target.
The driver was earlier performing create from a source volume and create from
a snapshot point by copying the source volume container folder.
This is not valid in case of vSAN datastore where the volume data is not a
'file' in the container folder unlike VMFS/NFS datastores. So copying folder
will only copy descriptor file and hence the source and cloned volume will
end up writing to the same data location, eventually corrupting it.
The fix is to copy only the VMDK file (descriptor + data) and not the entire
container folder. The implementation does the following:
1. Create a volume backing
2. Delete the current VMDK
3. Make a copy of source volume's VMDK file to the destination using virtual
disk manager. In case of create from snapshot, we copy the VMDK file
corresponding to the snapshot point.
VMware: Usng RetrvProprtisEx & does multi ESX scan
The existing code uses deprecated API RetrieveProperties for querying through
property collector. The public doc asks to use RetrievePropertiesEx instead.
RetrievePropertiesEx works by retrieving results in serveral batches rather than
retrieval in a single API call. This works well in environments containing
large number of VMs or ESX hosts.
There are three steps involved 1. Get first batch 2. Continue retrieving using
a token 3. Cancel retrieval when not needed
Batch size is user configurable, default is 100 results per batch.
The review also fixes a careless programming error leading to scanning only
1 ESX host under the vCenter server for datastores.
This patch fixes XML serializer for QoS Specs exension calls like
list(), show(), associations(). Note that using XML as input for
create(), update() doesn't support yet.
Code Review / openstack-build / cinder-build.git / log