Tom Swanson [Tue, 2 Jun 2015 19:02:33 +0000 (14:02 -0500)]
Dell SC: Expanded comments and update var names
Additional comments have been added to help explain what is happening
in the driver. This revolves mostly around dell_storagecenter_api.py
which is not an obvious file for a reviewer to review. Hopefully
additional comments will aid this.
Updated single character variable names except for r which is the
REST calls return throughout the code by convention.
Reworked create_volume for clarity. No functional changes.
Changed two obvious error messages from LOG.debug to LOG.error.
Tom Swanson [Wed, 13 May 2015 17:00:54 +0000 (12:00 -0500)]
Dell SC: Added support for alternate iscsi portals
Added support for returning alternate target portals,
luns and iqns in a non multipath setup. This was done
per https://review.openstack.org/#/c/140877/ Tests
were modified to support this change.
The iscsi initialize_connection and find_iscsi_properties
functions took the brunt of this. The full list of portals,
luns and iqns are returned along with the preferred portal,
lun and iqn rather than a single iqn, lun and portal.
Some minor log message changes for clarity. (Removed the
one character vars.)
Tom Swanson [Tue, 12 May 2015 17:37:43 +0000 (12:37 -0500)]
Dell: Added verify cert option for REST calls
If one goes through the trouble to create a cert and
install it on the Dell Data Collector and on the cinder
node then we can enable certificate verification on the
request calls to the Dell REST API.
This simply makes dell_sc_verify_cert a cinder.conf option
for a Dell backend and directly uses that as the verify
option on the REST requests. Default is False.
The update_migrated_volume function arrived via a rebase
and had a merge issue that was fixed. Specifically the
dell_sc_ssn value is no longer part of the api.find_sc
call.
Thang Pham [Thu, 12 Mar 2015 17:18:05 +0000 (13:18 -0400)]
Complete switch to snapshot objects
The following patch switches the remainder of cinder internals
to use snapshot object instead of direct db calls. Note that
db calls within cinder drivers were not switched over. This
is left to driver maintainers to do themselves and to
properly test out the changes on their hardware.
Also, note that there are three occurrences of
db.snapshot_update and one of db.snapshot_destroy left in
cinder/volume/manager.py. This is intentional because
driver.create_cgsnapshot and driver.delete_cgsnapshot returns
a list of snapshot dicts. Each driver needs to switched over
occurences of db.snapshot_get_all_for_cgsnapshot() to
SnapshotList.get_all_for_cgsnapshot() in create_cgsnapshot
and delete_cgsnapshot. Once each driver has done so, a follow
up patch can be created to remove db.snapshot_update and
db.snapshot_destroy in cinder/volume/manager.py. There are
bugs filed for these to be fixed -
https://bugs.launchpad.net/cinder/+bugs?field.tag=
cgsnapshot-objects.
Zhiteng Huang [Thu, 4 Jun 2015 18:23:58 +0000 (11:23 -0700)]
DriverFilter: don't check volume_backend_name
Currently DriverFilter checks if host_state.volume_backend_name
matches type.extra_specs.volume_backend_name. The same check has
already been done in CapabilitiesFilter and done more thoroughly and
gracefully, for example, volume_backend_name can be a set of names
not limited to one to one match. DriverFilter should just do one
thing, that is evaluating filter_function supplied by backend.
This change removes the volume_backend_name check from DriverFilter
without touching its core function, thus has no impact on existing
documentation because the checking of volume_backend_name is an
undocumented side-effect of this filter. This change also reduces
one warning log to debug log when no filter_function found in
host_state, for the reason it would generate a lot of warning log
for those backends don't supply filter_function.
UpgradeImpact:
DriverFilter had an undocumented side-effect before it evaluates
filter_function - it checked if volume_backend_name in host_state
reported by backend matches volume_backend_name in volume type
extra specs, and if they don't, the backend would fail to pass
the filter. Now that this side-effect has been removed, it
would impact users only under following circumstance: 1) user
enabled DriverFilter but not CapabilitiesFilter; 2) user relied
on DriverFilter to filter backends based on exact match of
'volume_backend_name' in type extra spec. If unfortunately this
is the case for user, enabling CapabilitiesFilter by adding
it to 'scheduler_default_filters' configure option and restart
cinder scheduler service then everything should work just fine.
CapabilitiesFilter actually does a much better job checking
volume_backend_name.
Daniel Wilson [Tue, 12 May 2015 17:47:35 +0000 (10:47 -0700)]
Add Pure Storage FibreChannel driver
This commit adds a FibreChannel driver for Pure Storage. The driver shares
the majority of functionality with base driver and only has the code
necessary to support connection to FibreChannel.
Tom Barron [Thu, 19 Mar 2015 23:10:54 +0000 (19:10 -0400)]
Fix remaining memory issues with nfs backup unit tests
Here we fix another place in the nfs backup unit tests where
the configured file size was larger than the chunk size in
the corresponding Swift backup tests. I missed this one in
https://review.openstack.org/#/c/165394
and even though code with that fix ran fine in a bare 1GB VM,
there are still memory issues when run in CI with a VM of that
size.
Michal Dulko [Thu, 4 Jun 2015 15:25:24 +0000 (17:25 +0200)]
Don't send heartbeats if Manager reports a problem
This commit adds is_working method to base Manager class that can be
used to indicate that service has some problems and isn't running
correctly. This is used to block refreshing Service heartbeats if
Manager will return False from is_working.
The method is implemented in c-vol Manager to return initialization
state of the underlying volume driver.
Kurt Martin [Tue, 2 Jun 2015 22:30:00 +0000 (15:30 -0700)]
Deprecate the HPLeftHandISCSIDriver CLIQ driver
This patch adds a deprecation message to the HPLeftHandISCSIDriver CLIQ
driver with the plan of removing it in the 2016.1 release. The
HPLeftHandISCSIDriver REST based driver should be used instead.
Xing Yang [Wed, 3 Jun 2015 14:40:26 +0000 (10:40 -0400)]
Allow provisioning to reach max oversubscription
Currently if provisioned ratio is equal to max_over_subscription_ratio,
the capacity filter does not allow any more thin provisioning. This patch
makes a change so that thin provisioning is allowed if provisioned ratio
is equal to max_over_subscription_ratio; it will only stop provisioning
if provisioned ratio is greater than max_over_subscription_ratio.
Sean McGinnis [Wed, 3 Jun 2015 04:29:01 +0000 (23:29 -0500)]
Clean up unused exceptions
Over the last cycle several drivers were removed. There
have also been changes over time in the code. As a
result, there were quite a few exception class declarations
that were no longer being used anywhere in the code base.
This removes Exception classes that were not referenced.
Tom Swanson [Tue, 2 Jun 2015 19:20:55 +0000 (14:20 -0500)]
Dell SC: update_volume_stats could use uninitialized vars
In _update_volume_stats in dell_storagecenter_common.py totalcapacitygb
and freespacegb are not necessarily initialized in the LOG.debug call.
Changed to data['total_capacity_gb'] and data['free_capacity_gb'].
Also initialized to total_capacity_gb and free_capacity_gb dict entries
to 'unavailable' which allowed the removal of two conditionals in the
same function.
Patrick East [Tue, 14 Apr 2015 20:45:22 +0000 (13:45 -0700)]
Refactor PureISCSIDriver into base and iSCSI classes.
This adds in a new class called PureBaseVolumeDriver that contains
all of the generic shared methods that utilize the management REST API.
The PureISCSIDriver now inherits from the base driver and implements
the functionality required for iSCSI hosts/initiators.
As part of this we’ve cleaned up some of the logging where it would
previously have hard coded the drivers class name in it, and moved some
static non-class helper methods into the base class.
These changes will ease future work to add additional drivers or split
them into separate files.
wanghao [Sun, 15 Feb 2015 07:40:53 +0000 (15:40 +0800)]
Non-admin user to query volume filter by az
Now cinder limits non-admin user to query volume just filtered by 'name',
'status', and 'metadata', those are hard coded in cinder-api. So improve
this using config option and allow non-admin to query volume filtered by
'availability zone'.
Gorka Eguileor [Mon, 1 Jun 2015 16:30:01 +0000 (18:30 +0200)]
Fix cinder concurrency issues on rtstool
Since there's no synchronized access to configfs in rtslib it can happen
that rtstool or rtslib access an element that no longer exists because
it has been removed just in the middle of a loop by another Cinder
request.
This results in quite a different number of exceptions:
- .dump()
- KeyError
- IOError
- RTSLibError on storage_object
This patch synchronizes access to all rtstool calls that access or
modify configfs using utils.synchronized decorator.
John Griffith [Fri, 29 May 2015 23:41:25 +0000 (23:41 +0000)]
Use SolidFire snapshots for Cinder snapshots
This patch just replaces the use of clones on the SolidFire
backend for SolidFire Snapshot objects. This isn't necessarily
an issue, but it is important necessary to do things like
consistency group snaps.
Note, that we have to keep compatability for things like
create_from_snapshot where the system may have clones that
were used for snapshots prior to updating the driver.
While we're here, we also removed the logging calls in the test
which shouldn't really be there, as well as the overly chatty
debug messages like "entering/leaving". More cleanup work
can be done to logs but should be in their own patch.
wanghao [Wed, 6 May 2015 04:08:00 +0000 (12:08 +0800)]
Notification with volume and snapshot metadata
Now notify message to ceilometer do not contain volume
metadata/glance_metadata and snapshot metadata, which are
useful to bill and enrich resource information in ceilometer.
This feature will add the metadata/glance_metadata of volume
and metadata of snapshot to notification usage info.
John Griffith [Fri, 29 May 2015 18:57:59 +0000 (12:57 -0600)]
Add volume ID to fake volumes in Gluster tests
This adds the volume ID field to the fake volume objects
in the GlusterFS tests. Turning up concurrency exposes some
issues where access of this field is attempted but doesn't
exist.
Xing Yang [Wed, 27 May 2015 16:58:04 +0000 (12:58 -0400)]
Fix capacity filter to allow oversubscription
Currently the capacity filter does not allow oversubscrition
if the free physical capacity is less than the new volume size.
This was based on the assumption that a new thinly provisioned
volume will be consumed right away. This assumption does not
allow us to take full advantage of thin provisioning.
The fix is to allow oversubscription if thin provisioning is
supported and max_over_subscription_ratio is greater than or
equal to 1. The free physical capacity will not be used to
compare with the new volume size for a backend that supports
oversubscription in thin provisioning. The oversubscription
ratio and virtual free capacity will still be used to decide
whether a backend can be chosen for thin provisioning.
yogeshprasad [Fri, 8 May 2015 10:37:22 +0000 (16:07 +0530)]
Add chap support to CloudByte cinder driver
CloudByte iSCSI cinder driver currently does not support
iSCSI CHAP security setting. This opens a security vulnerability
whereby any host machine can mount a CloudByte iSCSI volumes
created via OpenStack.
This patch adds CHAP authentication support for the CloudByte iSCSI
cinder driver.
Admin has to manually add the auth group and chap username/password in
CloudByte storage. Driver uses the first user found in auth group.
Tina [Mon, 18 May 2015 01:12:10 +0000 (21:12 -0400)]
Multiple pools support enhancement in VNX cinder driver
This introduced a new option "storage_vnx_pool_names" which
replaced the existing option "storage_vnx_pool_name". The new option
allows user specify more than one pools to be managed by a vnx
cinder back end.
DocImpact
This change will introduce a new configuration option, so that the
documentation need to be updated.
wanghao [Thu, 28 May 2015 03:33:34 +0000 (11:33 +0800)]
Remove un-used import at test_volume_transfer.py
There is an un-used import 'cinder.volume' at
ciner/tests/unit/api/contrib/test_volume_transfer.py.
Remove it, meanwhile there is using 'cinder.transfer.API'
at line #548 and line #581, so make the import consistent
by changed 'from cinder import transfer' to 'import cinder.transfer'.
Clinton Knight [Wed, 15 Apr 2015 14:50:49 +0000 (10:50 -0400)]
NetApp FC driver shims missing manage/unmanage
The NetApp FC drivers' passthrough shim layers are missing the
manage/unmanage methods, causing the manage/unmanage functions
to fail. The necessary functionality is already present in
the underlying library layer, so the trivial fix is to add the
functions to the passthrough layers. This patch also adds a
unit test to ensure the four block storage driver interfaces
remain in sync.
Daniel Allegood [Wed, 27 May 2015 17:27:06 +0000 (10:27 -0700)]
Updating cmd/manage.py get_arg_string() argument parser and adding unit test
Looks like get_arg_string() would improperly chop off the first two characters
of an argument that was passed in with one dash. For example,
passing '-option foo' would return 'tion foo' instead of 'option foo'.
Tom Barron [Mon, 23 Feb 2015 13:51:13 +0000 (08:51 -0500)]
Add standard QoS spec support to cDOT drivers
This commit adds support for standard cinder QoS specs to
NetApp cDOT drivers, alongside our pre-existing support for
externally provisioned QoS policy groups via qualified
extra specs.
Previously, the logic to calculate the LUN ID only considered a single attached
host. Because multiple hosts can be attached to a LUN, the logic needs to
consider multiple hosts. This patchset proposes to choose non-conflicting
LUN IDs across the whole array. This patch first checks for an empty LUN ID
across the array. If it doesn't find one, then it looks for the least used
LUN ID across the whole array.
The value of MAX_LUNS_PER_HOST has been changed here as the range function
iterates from 0 to n-1 LUN IDs. With Eseries backend, a maximum of 256 LUN_IDS
are supported, thus a value of 256 MAX_LUNS_PER_HOST is required.
Eric Brown [Tue, 5 May 2015 01:42:48 +0000 (18:42 -0700)]
VMware: insecure option should be exposed
Commit Ida730db66b154a4d445f7a91bccb9ca5b5a26f5e introduced
certificate verification but did not expose the option to turn
on/off verification using the insecure boolean of oslo.vmware.
Gorka Eguileor [Wed, 4 Mar 2015 13:24:25 +0000 (14:24 +0100)]
Create iSCSI lio portals with right IPs and port
Currently lioadm helper creates portals with fixed IP addresses 0.0.0.0
and ::0 and also uses fixed standard port 3260 instead of using store
configuration (iscsi_ip_address, iscsi_secondary_ip_addresses,
iscsi_port) from the configuration as it should.
This could lead to problems if a different port was defined in the
configuration as the target would be available only on 3260 and other
services would look for it at configured port.
With this fix we now create the right portals with the right port.
Since now we pass list of ips and port to helpers cxt helper has also
been updated to use these parameters instead of going directly to the
configuration.
Tina [Thu, 12 Mar 2015 00:34:36 +0000 (20:34 -0400)]
Create consistgroup from cgsnapshot support in VNX driver
Changed VNX Cinder driver to support creating a consistency group from
a consistency group snapshot and refactored the migration lun code for
reusing the code in create cg from cgsnapshot feature.
wanghao [Mon, 9 Mar 2015 10:49:31 +0000 (18:49 +0800)]
Fix response when querying host detail by host name
When querying host detail by host name like
'wanghao-devstack-snapshot@lvmdriver-1'
returned by host list command, cinder returns
incorrect result in 'total' block.
"resource": {
"volume_count": "0",
"total_volume_gb": "0",
"total_snapshot_gb": "0",
"project": "(total)",
"host": "wanghao-devstack-snapshot@lvmdriver-1",
"snapshot_count": "0"
}
This issue is caused by 'volume_data_get_for_host'
in sqlalchemy. This has nothing to do with multibackend,
this is due to the introduction of pool support for single
backend.
Fix this by querying volume filter by host or host name
like 'host#%' in volume_data_get_for_host.
Code Review / openstack-build / cinder-build.git / log