John Griffith [Wed, 16 Oct 2013 01:13:45 +0000 (19:13 -0600)]
Fix lvm.extend_volume to pass Gig suffix
The extend function in the lvm driver was not converting
the cinder size value to Gigabytes before passing the call
to the vg module. The result was that we would attempt to
extend a volume to "new size in Megabytes" which of course
is less than the current size since we do a Gigabyte string
conversion on create and everywhere else.
This change makes sure we pass the integer change through
the sizestr method to get the G suffix needed to work properly.
John Griffith [Tue, 15 Oct 2013 19:57:48 +0000 (13:57 -0600)]
FK lookup failures during migration
There are a couple of cases where migrations have failed
upgrading from Grizzly to Havana, this seems to be isolated
to a couple of migrations so far and the error message is a
failure to lookup/associate the volume-id FK dependency.
It appears that this is caused by not setting the charset
in the migration, so the result is that the initial db setup
uses utf8 and the migrations are using the default latin1.
This patch goes through all of the migrations in Havana that
specify InnoDB and explicitly sets the charset to utf8 to match
the volumes table (and the other original tables).
Giulio Fidente [Thu, 10 Oct 2013 12:59:44 +0000 (14:59 +0200)]
improves lvm version parsing for customised builds
supports_thin_provisioning now uses a regexp to ensure parsing of
lvm version succeeds when the build is customised; also adds a test
for a customised string parsing
Alan Jiang [Thu, 3 Oct 2013 22:03:09 +0000 (17:03 -0500)]
long flashcopy operation may block volume service
Storwize family uses flashcopy for snapshot or volume clone. The
volume delete has to wait until flashcopy finishes or errors out.
The _delete_vdisk() will poll volume FlashCopy status in a loop.
This may block volume serivce heartheat since it is in the same
. The solution is to use openstack FixedIntervalLoopingCall
to run the FlashCopy status poll in a timer thread.
The cinder volume mananger will resume delete operation for those
volumes that are in the deleting state during volume service startup.
Since Storwize volume delete may wait for a long time, this can cause
volume service to have long delay before it becomes available.
A greenpool is used to offload those volume delete operations.
zhangchao010 [Wed, 9 Oct 2013 07:37:49 +0000 (15:37 +0800)]
Support Huawei driver upgrade from grizzly to havana
To make the driver upgrade from grizzly to higher versions, the patch
checks the old host name firstly when creating and deleting host in
Huawei storage system.
If the old host name exists:
*get and use it directly when adding host.
*delete it directly when deleting host.
The current implementation perform snapshot of attached volume.
This is invalid because when we take snapshot of the backing, the instance
continues pointing to the readonly delta and tries writing to it.
This patch will disallow snapshot creation of attached volume.
We have similar issue with delete of snapshot, when deleted the instance will
point to an invalid disk file, since that will get merged with its parent.
Hence any writes by the instance will fail with disk not found error.
This patch will ignore delete snapshot of attached volume.
Since we disallow snapshot of attached volume, we disallow linked clone of
attached source volume and also upload of attached volume.
John Griffith [Thu, 3 Oct 2013 21:54:20 +0000 (15:54 -0600)]
Fix broken solidfire create-snapshot
A previous change to enable secifying size for cloned volumes
created introduced a bug that breaks the ability to create-snapshots
when using the SolidFire driver
(CID: I5628c7fa922780d6b0601e2daa79310a61085edc).
The problem is that we use a shared method in the SF driver for both
create-clone and create-snapshot, the added change only considers the
clone case and does a get ref['size'] but in the case of snapshots
this needs to be ref['volume_size'].
Joel Coffman [Tue, 24 Sep 2013 23:10:09 +0000 (19:10 -0400)]
Add key manager implementation with static key
Per feedback received on other patch sets, an example key manager
driver is required to support ephemeral storage encryption and
Cinder volume encryption -- see
* https://blueprints.launchpad.net/nova/+spec/encrypt-cinder-volumes
* https://blueprints.launchpad.net/nova/+spec/encrypt-ephemeral-storage
The ConfKeyManager class reads its key from the project's
configuration file and provides this key for *all* requests. As
such, this key manager is insecure but allows the aforementioned
encryption features to be used without further integration effort.
To clarify the above statements, the configuration-based key
manager uses a single, fixed key. When used to encrypt data (e.g.,
by the Cinder volume encryption feature), the encryption provides
limited protection for the confidentiality of data. For example,
data cannot be read from a lost or stolen disk, and a volume's
contents cannot be reconstructed if an attacker intercepts the iSCSI
traffic between the compute and storage host. If the key is ever
compromised, then any data encrypted with the key can be decrypted.
This commit copies the ConfKeyManager class from Nova as well as
synchronizing changes with the key manager interface in Nova.
John Griffith [Thu, 26 Sep 2013 21:23:01 +0000 (15:23 -0600)]
Remove need for CONF acces in brick iscsi
At some point we'd like brick to be a standalone lib,
and as such we don't want to have a requirement for
CONF files and having duplicate conf entries across
projects.
The better approach would be to let the projects decide
what they want to use, and how they want defaults to be set
and then pass those settings in via __init__ or when calling
the methods that need them.
Alan Jiang [Tue, 1 Oct 2013 05:38:03 +0000 (00:38 -0500)]
Storwize SVC driver hostname can't start with number
Storwize family storage does not allow host identifier start with
a numeric number. Otherwise cinder initialize_connection() API will
throw exception due to such invalid host name in the connector. The fix
is to add a '_' symbol as prefix if the host identifier in connector
starts with non-alphabet letters.
Joshua Harlow [Tue, 1 Oct 2013 22:53:30 +0000 (15:53 -0700)]
After commiting quota we should avoid certain reverts
After we commit the quota successfully we do not want to
set the database volume to destroyed or attempt further
rollback of the quota itself to reflect what the code
previously did.
This is part 1 of the work needed to
remove CONF from the brick subproject.
This patch removes the CONF usage
completely from the initiator portion of brick.
Doug Hellmann [Tue, 1 Oct 2013 22:01:59 +0000 (18:01 -0400)]
Disable lazy translation
Late in the Havana cycle bug 1225099 was found in the lazy
translation code, and to be safe it was decided to disable lazy
translation for Havana. This change does that.
John Griffith [Tue, 1 Oct 2013 18:05:14 +0000 (12:05 -0600)]
Check for backing lun on iscsi target create
Check to verify the backing lun was actually created and not just
the controller lun. If it was NOT created, attempt to issue
tgtadm --op new to see if we can recover.
If this fails, then we want to actually fail in Cinder rather than
pretending that everything went well, so we'll log the error and raise.
For some reason, using non-rbd volume driver whilst using
Ceph backup causes what looks to be a cyclic import error.
Ensuring that the backup.drivers.ceph module is imported
into volume.drivers.rbd ONLY if it is needed i.e. if both
drivers are in use, seems to remedy this.
Bill Owen [Fri, 27 Sep 2013 23:22:41 +0000 (16:22 -0700)]
v2 api - return bootable attr value on volume list
Add bootable to the list of values returned by display_list. This
was returned in v1 api, and is still in v2 header, but was missed
in the new implementation.
The backup service now keeps a dictionary of managers
for each type of volume backend. If multi backend is
not in use (and volume_type is None) the current
volume_driver is used.
John Griffith [Mon, 30 Sep 2013 14:45:52 +0000 (08:45 -0600)]
Dont retry if target creation succeeds
The target creation retry loop was not breaking when
the target was succesfully created. This is "sometimes" ok,
however in other cases this will actually cause a failure in
the second create and the backing Lun will be deleted and not
created again succesfully due most likely to a busy status on the
target.
The driver was earlier performing create from a source volume and create from
a snapshot point by copying the source volume container folder.
This is not valid in case of vSAN datastore where the volume data is not a
'file' in the container folder unlike VMFS/NFS datastores. So copying folder
will only copy descriptor file and hence the source and cloned volume will
end up writing to the same data location, eventually corrupting it.
The fix is to copy only the VMDK file (descriptor + data) and not the entire
container folder. The implementation does the following:
1. Create a volume backing
2. Delete the current VMDK
3. Make a copy of source volume's VMDK file to the destination using virtual
disk manager. In case of create from snapshot, we copy the VMDK file
corresponding to the snapshot point.
VMware: Usng RetrvProprtisEx & does multi ESX scan
The existing code uses deprecated API RetrieveProperties for querying through
property collector. The public doc asks to use RetrievePropertiesEx instead.
RetrievePropertiesEx works by retrieving results in serveral batches rather than
retrieval in a single API call. This works well in environments containing
large number of VMs or ESX hosts.
There are three steps involved 1. Get first batch 2. Continue retrieving using
a token 3. Cancel retrieval when not needed
Batch size is user configurable, default is 100 results per batch.
The review also fixes a careless programming error leading to scanning only
1 ESX host under the vCenter server for datastores.
This patch fixes XML serializer for QoS Specs exension calls like
list(), show(), associations(). Note that using XML as input for
create(), update() doesn't support yet.
John Griffith [Sun, 29 Sep 2013 16:01:58 +0000 (10:01 -0600)]
Add debug logging for targets
Seems we may be failing target create silently,
the situation appears to be that the actual device
lun is not created. This has been appearing in the
gates on create from snapshot.
This change adds some logging that will help us determine
if it's the actual create, or if it's the attempted update
while the dd process is possibly holding the volume in a busy
state causing the target update to add lun 1 to fail.
QuotaSetsController's _get_quotas already supports querying for
the usage. This patch allows sending a parameter "usage" to the
REST API when asking for quotas
Code Review / openstack-build / cinder-build.git / log