John Griffith [Thu, 3 Oct 2013 21:54:20 +0000 (15:54 -0600)]
Fix broken solidfire create-snapshot
A previous change to enable secifying size for cloned volumes
created introduced a bug that breaks the ability to create-snapshots
when using the SolidFire driver
(CID: I5628c7fa922780d6b0601e2daa79310a61085edc).
The problem is that we use a shared method in the SF driver for both
create-clone and create-snapshot, the added change only considers the
clone case and does a get ref['size'] but in the case of snapshots
this needs to be ref['volume_size'].
Joel Coffman [Tue, 24 Sep 2013 23:10:09 +0000 (19:10 -0400)]
Add key manager implementation with static key
Per feedback received on other patch sets, an example key manager
driver is required to support ephemeral storage encryption and
Cinder volume encryption -- see
* https://blueprints.launchpad.net/nova/+spec/encrypt-cinder-volumes
* https://blueprints.launchpad.net/nova/+spec/encrypt-ephemeral-storage
The ConfKeyManager class reads its key from the project's
configuration file and provides this key for *all* requests. As
such, this key manager is insecure but allows the aforementioned
encryption features to be used without further integration effort.
To clarify the above statements, the configuration-based key
manager uses a single, fixed key. When used to encrypt data (e.g.,
by the Cinder volume encryption feature), the encryption provides
limited protection for the confidentiality of data. For example,
data cannot be read from a lost or stolen disk, and a volume's
contents cannot be reconstructed if an attacker intercepts the iSCSI
traffic between the compute and storage host. If the key is ever
compromised, then any data encrypted with the key can be decrypted.
This commit copies the ConfKeyManager class from Nova as well as
synchronizing changes with the key manager interface in Nova.
John Griffith [Thu, 26 Sep 2013 21:23:01 +0000 (15:23 -0600)]
Remove need for CONF acces in brick iscsi
At some point we'd like brick to be a standalone lib,
and as such we don't want to have a requirement for
CONF files and having duplicate conf entries across
projects.
The better approach would be to let the projects decide
what they want to use, and how they want defaults to be set
and then pass those settings in via __init__ or when calling
the methods that need them.
Alan Jiang [Tue, 1 Oct 2013 05:38:03 +0000 (00:38 -0500)]
Storwize SVC driver hostname can't start with number
Storwize family storage does not allow host identifier start with
a numeric number. Otherwise cinder initialize_connection() API will
throw exception due to such invalid host name in the connector. The fix
is to add a '_' symbol as prefix if the host identifier in connector
starts with non-alphabet letters.
Joshua Harlow [Tue, 1 Oct 2013 22:53:30 +0000 (15:53 -0700)]
After commiting quota we should avoid certain reverts
After we commit the quota successfully we do not want to
set the database volume to destroyed or attempt further
rollback of the quota itself to reflect what the code
previously did.
This is part 1 of the work needed to
remove CONF from the brick subproject.
This patch removes the CONF usage
completely from the initiator portion of brick.
Doug Hellmann [Tue, 1 Oct 2013 22:01:59 +0000 (18:01 -0400)]
Disable lazy translation
Late in the Havana cycle bug 1225099 was found in the lazy
translation code, and to be safe it was decided to disable lazy
translation for Havana. This change does that.
John Griffith [Tue, 1 Oct 2013 18:05:14 +0000 (12:05 -0600)]
Check for backing lun on iscsi target create
Check to verify the backing lun was actually created and not just
the controller lun. If it was NOT created, attempt to issue
tgtadm --op new to see if we can recover.
If this fails, then we want to actually fail in Cinder rather than
pretending that everything went well, so we'll log the error and raise.
For some reason, using non-rbd volume driver whilst using
Ceph backup causes what looks to be a cyclic import error.
Ensuring that the backup.drivers.ceph module is imported
into volume.drivers.rbd ONLY if it is needed i.e. if both
drivers are in use, seems to remedy this.
Bill Owen [Fri, 27 Sep 2013 23:22:41 +0000 (16:22 -0700)]
v2 api - return bootable attr value on volume list
Add bootable to the list of values returned by display_list. This
was returned in v1 api, and is still in v2 header, but was missed
in the new implementation.
The backup service now keeps a dictionary of managers
for each type of volume backend. If multi backend is
not in use (and volume_type is None) the current
volume_driver is used.
John Griffith [Mon, 30 Sep 2013 14:45:52 +0000 (08:45 -0600)]
Dont retry if target creation succeeds
The target creation retry loop was not breaking when
the target was succesfully created. This is "sometimes" ok,
however in other cases this will actually cause a failure in
the second create and the backing Lun will be deleted and not
created again succesfully due most likely to a busy status on the
target.
The driver was earlier performing create from a source volume and create from
a snapshot point by copying the source volume container folder.
This is not valid in case of vSAN datastore where the volume data is not a
'file' in the container folder unlike VMFS/NFS datastores. So copying folder
will only copy descriptor file and hence the source and cloned volume will
end up writing to the same data location, eventually corrupting it.
The fix is to copy only the VMDK file (descriptor + data) and not the entire
container folder. The implementation does the following:
1. Create a volume backing
2. Delete the current VMDK
3. Make a copy of source volume's VMDK file to the destination using virtual
disk manager. In case of create from snapshot, we copy the VMDK file
corresponding to the snapshot point.
VMware: Usng RetrvProprtisEx & does multi ESX scan
The existing code uses deprecated API RetrieveProperties for querying through
property collector. The public doc asks to use RetrievePropertiesEx instead.
RetrievePropertiesEx works by retrieving results in serveral batches rather than
retrieval in a single API call. This works well in environments containing
large number of VMs or ESX hosts.
There are three steps involved 1. Get first batch 2. Continue retrieving using
a token 3. Cancel retrieval when not needed
Batch size is user configurable, default is 100 results per batch.
The review also fixes a careless programming error leading to scanning only
1 ESX host under the vCenter server for datastores.
This patch fixes XML serializer for QoS Specs exension calls like
list(), show(), associations(). Note that using XML as input for
create(), update() doesn't support yet.
John Griffith [Sun, 29 Sep 2013 16:01:58 +0000 (10:01 -0600)]
Add debug logging for targets
Seems we may be failing target create silently,
the situation appears to be that the actual device
lun is not created. This has been appearing in the
gates on create from snapshot.
This change adds some logging that will help us determine
if it's the actual create, or if it's the attempted update
while the dd process is possibly holding the volume in a busy
state causing the target update to add lun 1 to fail.
QuotaSetsController's _get_quotas already supports querying for
the usage. This patch allows sending a parameter "usage" to the
REST API when asking for quotas
Bill Owen [Fri, 27 Sep 2013 18:22:57 +0000 (11:22 -0700)]
Set vol driver initialized before deleting volumes
Move the call to set_initialized before calling delete_volume in
init_host. The delete_volume method has a precondition that the
driver be initialized.
Log some errors to avoid losing the exception messages.
HVS driver do repeated judgements for JSON results, here add error
logs and replace these judgement codes with a assertion function.
Luis A. Garcia [Tue, 24 Sep 2013 22:07:55 +0000 (22:07 +0000)]
Fix translation of CinderExceptions in REST API
When creating a Fault from a CinderException wrapped in an
HTTPException, we were converting the inner explanation to unicode
before it was able to reach the Fault's call() method which is where
translation occurs, and unicode objects can't be translated.
This patch preserves the CinderException's Message object and puts it in
another Message object as the explanation to the HTTPException so it can
be translated.
Eric Harney [Wed, 25 Sep 2013 19:43:32 +0000 (15:43 -0400)]
Allow upgrade from Grizzly with ThinLVMVolumeDriver
In Grizzly, LVM thin provisioning was used by setting
volume_driver=...ThinLVMVolumeDriver. In Havana this
functionality has moved under the standard LVM driver
with the option "lvm_type=thin" used to enable it.
We should account for the old volume_driver class path
so that the volume service behaves as expected for users
upgrading from Grizzly ThinLVM configurations.
Calls to create_volume (both volume_rpcapi and scheduler_rpcapi) pass
mandatory args as optional and reverse. This patch fixes the calls to be
according to the function definition.
Also, fix the call to create_volume in the migration code where
allow_reschedule defaults to True, but we don't pass request_spec so it
prints a warning and disables it.
Mike Perez [Wed, 25 Sep 2013 03:55:04 +0000 (20:55 -0700)]
Pass correct args to vol_rpc create_volume calls
In the chance and simple scheduler, create volume was originally using
snapshot_id and image_id for request_spec and filter_properties. This
corrects that by passing the correct arguments and keyword arguments to
create_volume.
Code Review / openstack-build / cinder-build.git / log