John Griffith [Thu, 19 Feb 2015 23:07:17 +0000 (16:07 -0700)]
Remove warnings for long vgs and lvs calls
We put some warning statements in a while back for
lvs and vgs commands that lasted more than 60 seconds
so we could easily query issues in the gate.
Since then we've made a number of changes to improve our
LVM interactions including the use of an LVM conf file with
a filter for only the VG's Cinder uses.
There's one issue remaining however where during the instance
rescue test in Nova where LVM get's stuck trying to open
devices that no longer exist. Part of this I think is due
to some manipulation that Nova/KVM does on the attached volume
when the test is run, but also the rescue process in Nova
uses LVM volumes as well and does some moving around and mounting
of devices.
The result is that LVM calls can take quite a while
(greater than 100 seconds) to complete during the rescue process.
This doesn't result in any failures, just that the rescue process
slows things down dramatically and is a pretty heavy process.
Also, the newer versions of process utils include completion
timing in the logs so we don't really need this any longer eve
to query timings and get info.
This patch removes those log statements, and proposes we
document somewhere that rescue operations on a setup that
includes c-vol service and nova-compute service on the same
node will temporarily impact LVM performance.
Steven Kaufer [Wed, 18 Feb 2015 21:45:57 +0000 (21:45 +0000)]
Implement refresh as kwargs in get_volume_stats
When implementing the base driver "get_volume_stats" function, not all
drivers implemented 'refresh' as a kwarg. This can cause failures for any
code that simply invokes the following (assuming that refresh will default
to False -- which is does for 50+ drivers):
driver.get_volume_stats()
This patch set updates the 7 drivers that did not implement refresh as a
kwarg so that the various driver implementations are consistent.
Tom Swanson [Tue, 17 Feb 2015 20:00:04 +0000 (14:00 -0600)]
Dell sc driver iscsi multipath enhancement
Added support for the connector's multipath boolean. If this is
set the initialize_connection function returns arrays of all the
IQNs, LUNs and portals associated with the mapping of the volume
to the server. If this is not set the standard return of a lone
IQN, LUN and portal is implemented.
Minor change to find_volume. If a volume name or instance isn't
set then it will not be sent as a payload filter to the Dell
Storage Center array.
Jeegn Chen [Tue, 17 Feb 2015 05:05:26 +0000 (13:05 +0800)]
Fix return value inconsistency in VNX Driver
When iscsi_initiators option for EMC Cinder Driver has no value,
a dict object is returned by find_available_iscsi_targets() in
EMC Cinder Driver while a list is expected. Thus an
initialize_connection() will fail due to the inconsistency.
This patch is to fix this inconsistency in the driver.
Jeegn Chen [Thu, 15 Jan 2015 01:35:09 +0000 (20:35 -0500)]
Pool-aware scheduler support in EMC VNX Cinder driver
Changed EMC VNX driver to report pools information in
update_volume_stats and removed the original pool selection
logic in VNX driver.
Besides, deprecated the VNX driver defined extra spec key
"storagetype:pool", since "pool_name" key is introduced by
the pool-aware scheduler feature to specify which pool to
place the volume.
Jay S. Bryant [Mon, 16 Feb 2015 23:05:01 +0000 (17:05 -0600)]
Sync policy module from oslo-incubator
The policy module has not had a sync since back in
November. There have been a number of changes that
should be pulled into Kilo.
Current HEAD in OSLO:
---------------------
commit 9bf01f9d98487cb13e3d95ad2a580fe8fc6f2479
Date: Fri Feb 13 14:18:58 2015 -0500
Switch from oslo.config to oslo_config
Changes merged with this patch:
--------------------- 2aacb111 Change oslo.config to oslo_config 2fbf5065 Remove oslo.log code and clean up versionutils API 262279b1 switch to oslo_serialization 07e9b32a Improving docstrings for policy API e67f5cd0 Merge "Don't log missing policy.d as a warning" 99d991ce Merge "Fixed a problem with neutron http policy check" b19af080 Don't log missing policy.d as a warning 2324c775 Add rule overwrite flag to Enforcer class 6166a960 Fixed a problem with neutron http policy check
Sean McGinnis [Thu, 12 Feb 2015 20:52:56 +0000 (14:52 -0600)]
Dell Storage Center: Add retries to API calls
In heavily loaded networks we have seen some cases
of temporary ConnectionErrors when making REST API
calls. There are usually successful calls just prior
and immediately after these failures, so it appears
to be a transient condition.
This patch utilizes the recently merged retry decorator
to add some retry handling to the REST API calls when
this condition is encountered.
With a test script looping through repeatedly running
CI against this first patch I was able to validate
that it addresses the issue. Out of 20 runs, a full
2/3 of the tests passed showing that the retry was
used. These test runs would have failed without the
retry.
The output from these test runs can be viewed here:
http://oslogs.compellent.com/?C=N;O=D
Pertinent results are dell-sc-iscsi-1554792015-12*
Once merged we should be able to enable full third
party CI testing with some expectation of reliable
results.
Anish Bhatt [Fri, 16 Jan 2015 05:31:58 +0000 (21:31 -0800)]
Add support for chiscsi iscsi helper
The chiscsi target works as a drop in replacement for IET/TGT with
minor configuration differences. This patch implements support for
this as the 'cxtadm' iscsi_helper.
Certification results : https://bugs.launchpad.net/cinder/+bug/1417499
PranaliDeore [Tue, 27 Jan 2015 12:46:58 +0000 (04:46 -0800)]
Snapshot of bootable volume goes in error state
If volume is not created from image, no glance
metadata would be available for that volume/snapshot
in volume glance metadata table. So simply catch
GlanceMetadataNotFound and do nothing.
Jay S. Bryant [Sun, 15 Feb 2015 02:24:56 +0000 (20:24 -0600)]
Sync periodic_task module from oslo-incubator
We have a need in Cinder to now be able to dynamically
add periodic tasks. This sync brings in new functionality
that supports that.
Current HEAD in OSLO:
---------------------
commit 03143669d2c996f7c20d2d75f6c67827d2db216a
Date: Tue Feb 10 11:59:45 2015 -0500
Update policy.py status in MAINTAINERS
Changes merged with this patch:
--------------------- 2aacb111 Change oslo.config to oslo_config 2fbf5065 Remove oslo.log code and clean up versionutils API fbd77a7b Allow overriding name for periodic tasks 9896e0e1 Separate add_periodic_task from the metaclass __init__
This test invokes Cinder WSGIService.Service() and asserts
against values that are only manipulated in wsgi.Server(),
which WSGIService is based on.
We also have TestWSGIServer.test_reset_pool_size_to_default
which does the same thing for wsgi.Server().
Remove this test since it duplicates the TestWSGIServer test
and tests code which is an additional layer of abstraction
(cinder.WSGIService) away from the class that the code belongs
to (wsgi.Server).
Mitsuhiro Tanino [Sat, 17 Jan 2015 03:04:17 +0000 (22:04 -0500)]
Refactoring for export functions in Target object
Currently, export functions such as create_export() are implemented
in individual Target code, but most of them are same and these are
common features in each target.
This patch moves these methods to parent ISCSITarget class to
commonalize and each Target simply inherit these methods from parent
class. As a result of this change, LioAdm can inherit ISCSITarget
class directly instead of inheriting TgtAdm class.
This simplifies dependency of targets and improves maintainability.
By refactoring these methods, this patch also fixes following issues.
(a) Fix bug #1410566
After transitioning to the new driver and target model, iscsi_targets
is not added to the table during create_export() phase.
However, remove_export() in LIO Target is still reffering empty
iscsi_targets table. This causes NotFound exception and remove_export()
skips to do remove_iscsi_target().
As a result, iscsi target is not removed and the target continues to
grab the volume(logical volume) as an in-use status.
This patch fix the problem.
(b) Re-export a volume with CHAP
Current Tgt Target recreate iscsi target without CHAP during
ensure_export() even if the volume is exported with CHAP previously.
This patch changes this bahaviour to recreate iscsi target using
previous CHAP which is stored in volume file on state_path dir.
Xing Yang [Mon, 9 Feb 2015 02:10:49 +0000 (21:10 -0500)]
EMC VMAX driver Kilo update
This patch updated the VMAX driver for the Kilo release.
Main changes in this patch include the following:
* Support for consistency group.
* Support for VMAX V3.
JordanP [Mon, 2 Feb 2015 13:36:52 +0000 (13:36 +0000)]
Fix Scality SRB driver security concerns
LP #1414531 raised 2 issues :
1)A potential arbitrary code execution if the Cinder Linux user
has write access to /etc/cinder/cinder.conf
2)An overall concern/question about the usage of the command
'sudo sh -c' throughout the srb driver
This patch fixes 1) with proper configuration validation and
2) with usage of cinder-rootwrap.
Xi Yang [Wed, 11 Feb 2015 07:06:50 +0000 (02:06 -0500)]
EMC VNX Cinder Driver iSCSI multipath enhancement
This commit is to be consistent with the iSCSI multipath
ehancement in Cinder and Nova:
* Return multiple portals and iqns when multipath=True is
specified in the connector info
* Return one portal and iqn when multipath=False is specified
in the connector info
Kurt Martin [Fri, 30 Jan 2015 21:59:56 +0000 (13:59 -0800)]
Add dedup provisioning to 3PAR drivers
3PAR now supports thin duplication provisioning. This review is for
adding the dedup provisioning support to the 3PAR provisioning extra
specs type as well as supporting volume retype between provisioning
types (thin<->dedup, full<->dedup).
The OpenStack Configuration Reference Guide will need to updated
to include dedup as a valid hp3par:provisioning value. A note
should also be added that this feature requires SSD disks and
3PAR firmware version 3.2.1 MU1 or greater.
Joshua Harlow [Wed, 11 Feb 2015 22:14:51 +0000 (14:14 -0800)]
Provided requirements are meant to be immutable
Since provided values are not meant to change (and
this statement is useless) we should not try to
perform an update() on the provided values (the update
is a no-op anyway, so might as well get rid of it).
John Griffith [Mon, 9 Feb 2015 21:45:40 +0000 (14:45 -0700)]
Don't fail target_delete if target doesn't exist
There are cases seen in the Gate where a target delete is
called and an exception is raised because the target does
not exist. In the cinder target driver code we raise this
as an ISCSITargetRemoveFailed exception, but if we're asking
to delete the target and the target doesn't exist we can
probably safely move along.
This patch adds a check for this specific case and logs a warning
and continues rather than failing. We also add a unit test to
check this case.
Marc Koderer [Tue, 10 Feb 2015 07:30:09 +0000 (08:30 +0100)]
Split volume driver into ABC classes
Use ABCmeta classes to define feature sets of drivers.
As interim solution the VolumeDriver class stays mainly unchanged to
ensure that all drivers work and can be migrated stepwise by the
driver maintainer.
Co-Authored-By: Mike Perez <thingee@gmail.com>
Implements: blueprint abc-volume-drivers
Change-Id: I558ded19368aaa7ce4be029fa16f63e6e188097a
Alex Meade [Mon, 2 Feb 2015 21:18:15 +0000 (16:18 -0500)]
Limit ram and disk used by ceph backup tests
The test_discard_bytes test in
cinder/tests/test_backup_ceph.py uses a lot of
ram and disk space since it tests the ceph driver with
the default chunk size of 128Mi. This patch lowers it
to use 1024 bytes instead.
Xing Yang [Thu, 5 Feb 2015 17:09:34 +0000 (12:09 -0500)]
Fix detach volume from host problem in VMAX driver
The VMAX driver unmaps a volume from a host without checking
the host info in the connector, resulting the wrong host to be
detached. This patch looks up the host info before detach and
fixes the problem.
VMware: Delay string interpolation in log messages
As per OpenStack developer guidelines, string interpolation
should be delayed to be handled by the logging code. This
patch fixes violations of this guideline in the VMDK driver.
The common code between various VMware drivers was moved to
oslo.vmware library during Icehouse release. The VMDK driver
should be updated to use this library. Changes are mostly
replacing import statements and removing redundant modules
and corresponding test cases.
Summary of changes:
* Replace api with oslo.vmware.api
* Replace vim with oslo.vmware.vim
* Replace pbm with oslo.vmware.pbm
* Replace io_util with oslo.vmware.image_transfer
* Replace vmware_images with oslo.vmware.image_transfer
* Replace read_write_util with oslo.vmware.rw_handles
* Remove error_util and use oslo.vmware.exceptions
* Move VMDK driver specific exceptions to a new module
'exceptions'
* Replace PBM WSDL files with the corresponding files in
oslo.vmware
* Replace PBM related methods in volumeops with the
corresponding ones in oslo.vmware.pbm
Tomoki Sekiyama [Thu, 11 Dec 2014 23:23:18 +0000 (18:23 -0500)]
Enhance iSCSI multipath support
Currently, nova-compute and brick support multipath for iSCSI volume
data path. It depends on response to targets discovery of the main
iSCSI portal, expecting multiple portal addresses to be contained.
However, some arrays only respond to discovery with a single portal
address, even if secondary portals are available. In this case,
the connector cannot know secondary portals and corresponding
iSCSI target IQN, so cannot establish multiple sessions for
the target(s). To enable the connector to login to secondary portals,
cinder should tell all the portal addresses and corresponding
target iqns/luns.
With this patch initialize_connection API will return connection_info
with multiple portal addresses/iqns/luns when multipath=True is
specified in the connector info. For example:
Tom Swanson [Thu, 5 Feb 2015 19:33:03 +0000 (13:33 -0600)]
Dell Storage Center Unit Test Updates for Kilo
Expanded our unit test coverage: test_dellscapi.py.
This tests our api module. We've also added some
minor driver fixes found by unit and other testing.
Some logging statements have been updated.
All changes are in the dell storage center driver
and unit test files.
ChangBo Guo(gcb) [Wed, 26 Nov 2014 03:30:40 +0000 (11:30 +0800)]
Add extra library oslo.concurrency to oslo.config.generator.rc
We used extra library oslo.concurrency in Cinder, and oslo.concurrency
provides configuration options, we need include these options in sample
config file. This commit handles that.
Change-Id: I534539b7e87a3f5dc36722395cbe241a10b2d75e
Xing Yang [Fri, 16 Jan 2015 21:27:23 +0000 (16:27 -0500)]
Support over subscription in thin provisioning
This patch adds support for over subscription in thin provisioning.
The following changes are proposed:
* A configuration option "max_over_subscription_ratio" will be
introduced.
* Driver reports the following capacities and ratios:
* provisioned_capacity
* max_over_subscription_ratio
* Driver can use the newly added configuration option to report
this ratio or it can decide what ratio to report itself.
The value of this ratio is depending on driver implementation
and will be reported together with other capabilities and
capacities by the driver.
* reserved_percentage
* Note: This is an existing parameter reported by the driver.
* Currently it is measured against the free capacity. In this
patch, it will be changed to measure against the total
capacity in the filter scheduler.
* Driver also reports the following capabilities:
* thin_provisioning_support (True or False)
* thick_provisioning_support (True or False)
* Scheduler will use the above new capabilities reported by the
driver to make decisions when choosing a backend.
For more details, please see Cinder spec:
https://review.openstack.org/#/c/129342/12/specs/kilo/
over-subscription-in-thin-provisioning.rst
Kurt Martin [Thu, 5 Feb 2015 00:09:44 +0000 (16:09 -0800)]
Lefthand driver fails to attach a cloned volume
The provider location was not being populated for cloned volumes.
This patch is updating the provider location for cloned volumes
resulting in successful volume attachments.
Abel Lopez [Tue, 13 Jan 2015 02:50:00 +0000 (18:50 -0800)]
Purge deleted rows
Adds the ability to clean up rows that are already marked as
deleted of a certain specified age. Age is calculated as timedelta
from now() in days, which are given at command line
Patrick East [Tue, 3 Feb 2015 20:09:58 +0000 (12:09 -0800)]
Make PureISCSIDriver iSCSI port discovery more flexible
While searching for reachable iSCSI ports on the target flash array it
might fail the discovery command. If this happens on driver setup it
will throw an exception from attempting to use fields on an
uninitialized class variable. This fixes the log message and adds a
retry to the method to make this more flexible with network timeouts.
Code Review / openstack-build / cinder-build.git / log