Gary W. Smith [Mon, 15 Dec 2014 22:38:26 +0000 (14:38 -0800)]
Uncouple scheduler stats from volume creation
Capture and report scheduler stats independently from volume creation.
Without this scheduler stats were not reported until the first volume
creation and were subsequently only updated each time a volume was
created.
David Pineau [Mon, 1 Dec 2014 11:21:35 +0000 (12:21 +0100)]
Add Scality SRB driver
This patch implements the `srb-driver` blueprint. It uses a similar
approach as Ceph's RBD in the way it exposes a block device to the
system.
The driver controls the SRB Linux kernel driver through a sysfs
interface and leverages LVM for snapshot management. The native block
driver talks to Scality's storage system (or any other vendor exposing
a compatible CDMI interface) through a HTTP-based RESTful protocol.
Bob-OpenStack [Mon, 24 Nov 2014 02:11:22 +0000 (18:11 -0800)]
Update volume driver for Huawei storage system
This driver is similar to the previous one, but contains
the following differences:
1. Remove smart tier support because that our product does not
support smart tier any more. 18000 does not support smart
tier, so this has no problem with 18000 series.
2. Update qos support implementation.
3. Add synchronization because that our tests show that when we
create volumes or snapshots in batch in a shell script with cli
command, our array may occur error.
Certification test result for Huawei storage drivers:
https://bugs.launchpad.net/cinder/+bug/1399038
zhangni [Thu, 4 Dec 2014 08:38:18 +0000 (16:38 +0800)]
Implement Huawei SDSHypervisor connector
Huawei SDSHypervisor uses a private key-value data protocal,
so we add a new connector inherited from InitiatorConnector
and implement connect_volume and disconnect_volume.
The connector uses sds_cli cmd to implement attach/detach/querydev vol,
sds_cli will be put to a specific dir and the path is registered as a
system environment variable when sds is installed.
Gloria Gu [Fri, 5 Dec 2014 01:59:10 +0000 (17:59 -0800)]
Added volume type description for volume type API
- Added the following APIs and tests for volume type
* update volume type
PUT http://<openstackhost>:8776/v2/${tenant_id}/types/${vol_type_id}
body
{
"volume_type": {
"description":"updated_desc"
}
}
** user can update description.
** if update description, descripiton can be empty spaces.
** description can not be None
** only admin can access this API
*get default volume type
GET http://<openstackhost>:8776/v2/${tenant_id}/types/default
** if default_volume_type is specified in cinder.conf and is valid,
the default volume type will be returned.
** if default_volume_type is not specified in cinder.conf or is not
valid, it will return 404 with a message saying default volume type
can not be found.
- Updated the following APIs and tests for volume type
* create volume type should take description as an option.
* list volume types or get one volume type will include description for
volume type if the description is not None.
- Upgraded the database cinder on table volume_types to include
the description. database upgrade/downgrade scripts and tests
are added.
- update API should send a notification to the message bus when
updating succeeds or fails.
- as of 12/5/2014, had to rebase with master which has volume type
access change, I also fixed the tests in that area in order to get
the unit tests pass.
Kurt Martin [Mon, 15 Dec 2014 23:14:42 +0000 (15:14 -0800)]
Fix 3PAR driver hang on SSH calls
In removing the 3PAR driver local file locks, part of the fix included
making a new SSH connection for each call in the hp3parclient as
oppose to leaving a connection open. Older hp3parclients(3.1.1 and
later) will result in the SSH calls hanging at driver initilization time.
This patch will now check against the correct minimum hp3parclient version
and remove the need for checking of the SSH args version since the minimum
hp3parclient is now greater than 3.1.1
Closes-Bug: #1402115
Anton Arefiev [Mon, 15 Dec 2014 10:33:38 +0000 (12:33 +0200)]
Delete default volume size 100M in drivers
There is check: if volume size == 0 we set default value to 100M,
it uses only in tests and it's little bit confusing.Use code only
for tests in drivers is bad practice.
Also the create volume flow already won't let user to create
a volume with size 0. So it was decided to remove this check.
Peter Pentchev [Wed, 3 Dec 2014 12:49:23 +0000 (14:49 +0200)]
Add the StorPool block storage driver.
StorPool is distributed data storage software running on standard x86
servers. StorPool aggregates the performance and capacity of all drives
into a shared pool of storage distributed among the servers. Within
this storage pool the user creates thin-provisioned volumes that are
exposed to the clients as block devices. StorPool consists of two parts
wrapped in one package - a server and a client. The StorPool server
allows a hypervisor to act as a storage node, while the StorPool client
allows a hypervisor node to access the storage pool and act as a compute
node. In OpenStack terms the StorPool solution allows each hypervisor
node to be both a storage and a compute node simultaneously.
To make full use of StorPool's native network communication protocol,
the Nova compute nodes will need to use the StorPool libvirt volume
attachment driver, nova.virt.libvirt.storpool.LibvirtStorPoolVolumeDriver.
Erlon R. Cruz [Thu, 11 Dec 2014 10:46:31 +0000 (08:46 -0200)]
Fix HNAS driver confusing error message
The error message shown when the parser finds a parser error
says, 'file not found' which causes confusion on the user when
he/she needs to debug the real cause o the problem. This patch fix
this by testing first if the file exist and then throwing a
proper error message.
John Griffith [Sun, 14 Dec 2014 19:02:30 +0000 (12:02 -0700)]
Remove iscsi_helper calls from base iscsi driver
There are a number of drivers that inherit from the base
iscsi driver, using the base implementation of initialize_connection
and terminate_connection. These drivers also don't have iscsi_helper
defined.
We've kept all iscsi_helper specific code in the tgt helpers
themselves, or in the various versions of the LVM driver
which is where they belong.
Over time however there have been some iscsi_helper specific
calls that have crept in to the base volume.driver:ISCSIDriver
class. These were LIO specific calls and as a result when a
configuration is set with multi-backend to use lioadm as the
target helper, or even if it's not used but the default iscsi_helper
is set to lioadm the initialize_connection and terminate_connection
calls will try and access the LIO specific calls and raise because
the drivers inheriting from this class don't have target_helper
members.
This went unnoticed mostly because there's no significant LIO testing
in place and there were no distros setting LIO as the default iscsi_helper,
it appears that this has changed however.
This patch moves the LIO specific calls back where they belong
and keeps the base ISCSIDriver generic. I've also added a test
case for this in the SolidFire driver because it's impacted by
this.
Tom Swanson [Wed, 3 Dec 2014 19:24:39 +0000 (13:24 -0600)]
Add Support for Dell Storage Center
This driver implements cinder FC and iSCSI volume drivers.
The file dell_storagecenter_api.py is called to interface with the
Dell Storage Center backend via a REST interface.
Dell_storagecenter_common.py is the base class implementation with
dell_storagecenter_fc.py and dell_storagecenter_iscsi.py being providing
fc and iscsi specific support respectivly.
Rushil Chugh [Tue, 18 Nov 2014 19:37:17 +0000 (14:37 -0500)]
Ensure that lun_id is an int for NetApp Drivers
Various NetApp drivers were treating the lun_id as a string
rather than an int. This was causing attempts to mount NetApp
volumes to Hyper-V nodes to fail as they were checking an integer
type against a unicode type which would fail.
This change casts result_lun to an integer after the value has
gone through the ssh injection attack check. This way Hyper-V
is able to verify if the found LUN is the target LUN, enabling
mounting of NetApp volumes to Hyper-V.
This change also refactors initialize_connection into some helper
methods so as to enable simpler unit testing of the patchset.
Ivan Kolodyazhny [Wed, 29 Oct 2014 13:52:54 +0000 (15:52 +0200)]
DB migration tests
Refactored migration tests to use OpportunisticTestCase, removed
unused code and ``test_migrations.conf`` file.
The main feature of this approach is to create a new database with
random name for each migration test. This will avoid migration tests of
race conditions and reduce tests intersection. After this change, database
``openstack_citest`` will be used only for initial connection to the database.
``test_migrations.conf`` file not required anymore, because we create test
database for migration test, so we no longer need to keep database credentials.
Trung Trinh [Mon, 15 Dec 2014 06:23:34 +0000 (20:23 +1400)]
Correct default service_name for nova_catalog*_info config option
The default service_name in config option (nova_catalog_info and
nova_catalog_admin_info) was mistakenly put as 'nova'. Consequently,
it is impossible to invoke novaclient APIs from Cinder code without
changing the default value of those config options.
The correct service_name defined in Keystone's config file
"/etc/keystone/default_catalog.templates" is:
catalog.RegionOne.compute.name = Compute Service.
This change replaces 'nova' with 'Compute Service' for default value
of 'nova_catalog*_info' config options.
The change is being reverted as it broke the Brocade FC SAN lookup
functionality. The change uses configuration options from
ssh_utils that are not initialized when the Brocade driver is
run causing an exception to be thrown complaining that
CONF.ssh_hosts_key_file is used before it is initialized.
The right solution is to change the Brocade driver to use ssh_utils to
make SSH connections.
Adam Gandelman [Thu, 4 Dec 2014 20:22:25 +0000 (12:22 -0800)]
Remove check_uptodate.sh check from tox.ini
This removes the check_uptodate.sh test from the py2* tox envs and
run_tests.sh. It also syncs the sample config one last time to document
some new options available to users running with the newer
oslo.messaging versions.
The backing VM corresponding to a volume is cloned during create volume
from another volume, snapshot or image. The backing VM is also cloned
during retype and backup restore. Currently, the target ESX host is
unset while calling vCenter CloneVM_Task API and hence the source ESX
host is used as the target. If the destination datastore returned by
the datastore selection logic is not accessible to the source host,
clone fails. This patch fixes the problem by setting the target ESX
host (returned by datastore selection logic) while invoking clone.
Vincent Hou [Fri, 10 Oct 2014 07:46:36 +0000 (15:46 +0800)]
IBM Storwize driver: Add local variable assignment to "ctxt"
* The method get_vdisk_params in helpers.py is missing a local variable
assignment for "ctxt", causing "UnboundLocalError: local variable
'ctxt' referenced before assignment. Adding the assignment should
resolve this issue.
* Add the unit tests coverage for get_vdisk_params.
Lucian Petrut [Fri, 14 Nov 2014 16:22:04 +0000 (18:22 +0200)]
Change CHAP secret default length
Some of the iSCSI initiators have a limit regarding the maximum
CHAP secret length. For example, the MS iSCSI Initiator
does not allow CHAP secrets longer than 16 characters, smaller
than the actual default 20 characters length.
This patch simply changes the default length to 16 characters,
value which is already used by default by some of the volume
drivers. In fact, the iSCSI specs state that: "Implementations
MUST support use of up to 128 bit random CHAP secrets".
Tomoki Sekiyama [Mon, 20 Oct 2014 18:32:55 +0000 (14:32 -0400)]
TgtAdm: Don't change CHAP username/password on live migration
As tgtd doesn't update CHAP username/password while the initiator is
connected, CHAP username/password must not be changed while a Nova
instance are performing live-migration; otherwise the compute node
which the instance migrates to cannot login to the volume and the
migration process is aborted.
This fixes TgtAdm implementation not to regenerate random
username/password every time initialize_connection is called.
Also, it enables CHAP auth in unit tests of TargetAdmin helpers.
Brianna Poulos [Thu, 6 Nov 2014 21:07:16 +0000 (13:07 -0800)]
Fix wrapper to work with barbicanclient 3.0.1
Due to the updates to the python-barbicanclient in version 3.0.1, the cinder
barbican wrapper no longer functions. This patch updates the cinder barbican
wrapper and test cases to work with barbicanclient 3.0.1.
Jay S. Bryant [Mon, 17 Nov 2014 19:11:03 +0000 (13:11 -0600)]
Update cinder.conf.sample to fix max db conn retries
Another external library change (oslo.db) has broken the gate
because cinder.conf.sample appears to be out of date. The change
was just to a message string:
Maximum number of database connection retries during
startup. Set to -1 to specify an infinite retry count. (integer value)
->
Maximum db connection retries during startup. Set to -1 to
specify an infinite retry count. (integer value)
This patch updates cinder.conf.sample to get beyond this failure. The
Cinder team will be talking about how to avoid these issues in the future.
Matthew Edmonds [Tue, 11 Nov 2014 21:03:23 +0000 (16:03 -0500)]
Fix Brocade FC SAN lookup MITM vulnerability
Modify the Brocade FC SAN lookup service implementation to use the
same SSH key config properties used elsewhere rather than relying on
arguments which are non-standard and never passed by the base lookup
service.
John Griffith [Tue, 4 Nov 2014 22:34:28 +0000 (23:34 +0100)]
Remove test_barbican from keymgr tests
Unfortunately it seems we have some very poorly
written keymgr unit tests, specifically
keymgr/test_barbican.py does this:
from barbicanclient.common import auth
First problem is that from a unit test perspective
that pretty much sucks, second problem is that
barbicanclient as of version 3.0.0.0 no longer
has an "auth" module, as a result Cinderunit tests
now fail.
The test_barbican.py unit tests need to be rewritten
to mock out the client components and actually test
Cinder components where needed without relying on
the barbicanclient.
For now in order to free up the gate, remove test_barbican.py
abhishekkekane [Tue, 21 Oct 2014 09:31:15 +0000 (02:31 -0700)]
Eventlet green threads not released back to pool
Presently, the wsgi server allows persist connections hence even after
the response is sent to the client, it doesn't close the client socket
connection.
Because of this problem, the green thread is not released back to the pool.
In order to close the client socket connection explicitly after the
response is sent and read successfully by the client, you simply have to
set keepalive to False when you create a wsgi server.
DocImpact:
Added wsgi_keep_alive option (default=True).
In order to maintain the backward compatibility, setting wsgi_keep_alive
as True by default. Recommended is set it to False.
Xing Yang [Thu, 9 Oct 2014 05:26:28 +0000 (01:26 -0400)]
Use look up service for auto zoning
The VMAX FC driver didn't use the look up service for auto zoning.
Instead it built initiator target map itself. However, that
requires the initiator to log into the fabric before zoning
in order to find out target WWNs.
This patch is to use the look up service to find out valid initiator
target WWNS and use that to build initiator target map. With this fix,
the initiator is no longer required to log into the fabric ahead of time.
During create_backup failure handling, backup_update fails with
DataError ("Data too long for column") if the fail_reason is
greater than 255 characters. As a result, backup status is stuck
in 'creating' state. This patch avoids the problem by truncating
fail_reason to 255 characters before update.
The WSDL URL of storage policy service is determined and a session is
created using it in do_setup(). This session is later used to initialize
the datastore selector property (ds_sel), which uses the session for all
storage policy related API calls.
After commit a8fa3ceb1e72bac2ab67f569a2ca009f995f59fd (Integrate
OSprofiler and Cinder), the properties defined in vmdk module are called
before do_setup(). As a result, the ds_sel (datastore selector) property
is initialized with a session instance containing a 'None' PBM (storage
policy service) WSDL URL. This results in failures of all storage policy
related APIs invoked using datastore selector. This patch fixes the
problem by re-initializing the property in do_setup().
Navneet Singh [Fri, 23 May 2014 04:57:33 +0000 (10:27 +0530)]
NetApp fix for default host type in eseries
This fixes the issue where the default host type
provided in mapping should be high performing LnxALUA
type for eseries. It also makes it configurable in case
users want to configure a different host type.
Clinton Knight [Thu, 9 Oct 2014 14:56:25 +0000 (10:56 -0400)]
FibreChannel drivers for NetApp Data ONTAP storage controllers
This patch adds FibreChannel support to NetApp's Cinder drivers
for Data ONTAP (7-mode and Cluster-mode). The drivers make full
use of Cinder's FibreChannel zone manager.
Silvan Kaiser [Wed, 26 Nov 2014 16:29:01 +0000 (17:29 +0100)]
First version of Cinder driver for Quobyte USP
Supported Operations are:
- Create Volume
- Delete Volume
- Attach Volume
- Detach Volume
- Extend Volume
- Create Snapshot
- Delete Snapshot
- List Snapshots
- Create Volume from Snapshot
- Create Volume from Image
- Create Volume from Volume (Clone)
- Create Image from Volume
The driver uses a file-based interface to access the configured
Quobyte volume. Therefore, the driver is similar to the existing
drivers NFS and GlusterFS.
Due to the similarities, I reused the snapshot code from the GlusterFS
driver. Gluster, thanks for that! I've kept the "Red Hat" copyright in
the header to credit you properly.
All driver functions are covered by unit tests. Snapshot tests were
taken over from test_glusterfs.py. New tests are written using "Mock"
instead of "mox".
John Griffith [Wed, 3 Dec 2014 17:44:49 +0000 (10:44 -0700)]
Fix use of invalid variable in tgt exists check
I added a check for tgt already exists here:
https://review.openstack.org/#/c/138173/
But, that was wrong, the err value is never going to be
set because of the exception. This should have inspected
the exception object and checked against it.
So this patch makes it do what it was supposed to do.
Gloria Gu [Wed, 10 Dec 2014 01:56:09 +0000 (17:56 -0800)]
Report better capacity info for a limitless 3par cpg
This change has the following improvement:
1. Uses getCPGAvailableSpace from hp3parclient to report
free_capapcity for a limitless cpg.
2. Uses cpg's SDUsage.usedMiB + UsrUsage.usedMiB + free_capacity
to calculate the total_capacity for a limitless cpg. This is the
best we can do for a limitless cpg.
Currently, the select_datastore method is called with an empty list of
ESX hosts. This causes the method to retrieve all the hosts in vCenter
as candidates for selecting a datastore. The reference to a retrieved
host is stored in attribute 'obj'. Some of the future bug fixes will
need to invoke this method with reference to a single ESX host. Such
invocations will fail with attribute error: 'val instance has no
attribute 'obj''. This patch fixes the attribute error.
Brianna Poulos [Fri, 29 Aug 2014 21:19:29 +0000 (17:19 -0400)]
Add support for backup encryption metadata
This modification allows the encryption key UUID field, which
has been added to the volume table, to remain valid when encrypted
volumes are backed up and then restored, which enables the restored
volume to be accessible and encrypted. This is related to patch
https://review.openstack.org/#/c/39573/, except that it uses the
backup metadata support recently added in patch
https://review.openstack.org/#/c/51900/ rather than modifying the
backup api.
Mike Mason [Thu, 4 Dec 2014 09:17:57 +0000 (09:17 +0000)]
Implementing the use of _L’x’/i18n markers
Placing the _Lx markers back into the code. No other cleaner solution has
has been implemented. Patches will be submitted in a series of sub
directories and in a fashion that is manageable.
eighth commit of this kind
This is the last run through to pick up the ones that were missed
Tom Barron [Wed, 26 Nov 2014 21:01:14 +0000 (16:01 -0500)]
Fixes intermittent NFS driver mount failure
During cinder volume driver initialization, NFS drivers often
fail to mount the NFS share backing their volumes, complaining
that the share in question is 'busy or already mounted'.
This commit introduces a retry loop around the ensure_mounted()
call inside set_nas_security_options() so that if there is contention
between volume process and backup process mounting the same share
the driver will not be stopped from loading.
Thang Pham [Fri, 20 Jun 2014 04:30:36 +0000 (00:30 -0400)]
Use pbr entry_points to setup the cinder scripts
The following patch changes how cinder scripts are
installed and unit tested. This patch moves all the
cinder scripts from bin into cinder/cmd and creates
entry_points for those scripts in setup.cfg. When
cinder is installed, these scripts will be installed
under /usr/local/bin by pbr.
DocImpact: 'host' config option for multiple-storage backends
in cinder.conf is renamed to 'backend_host' in order to avoid
a naming conflict with the 'host' to locate redis
Change-Id: If82cf0c58d765bf79dbf721ca95c10a468940cab
Implements: blueprint unit-test-cases-for-cinder-scripts
Code Review / openstack-build / cinder-build.git / log