Erlon Cruz [Thu, 12 Jun 2014 10:55:38 +0000 (07:55 -0300)]
Adds SSH communication to HNAS drivers
This commit adds SSH communication to HNAS iSCSI and NFS drivers. The user now
has the option to export the server public keys to HNAS and configure the
authentication via public keys. This avoid exposing HNAS passwords in the plain text
config file and exposing the private HNAS internal network.
Tom Swanson [Wed, 4 Mar 2015 19:56:13 +0000 (13:56 -0600)]
Dell SC API change fails snapshot creation
The Dell Storage Center driver is failing snapshot creation on
later versions of the Dell REST API.
The doNotExpire option was removed in API 15.3.1.110. We were
setting this so that the replays are not expired and removed when
Cinder expects them to still be there. This has been replaced by
setting expiretime to 0. We were doing this already so the fix
was to simply remove doNotExpire from the REST request payload.
This will fail on every future release of the Dell Storage Center
so this is a critical fix.
Jeegn Chen [Wed, 4 Mar 2015 13:10:34 +0000 (21:10 +0800)]
More validation logic for VNX CG creation
In VNX array, compressed LUNs cannot be added into a Consistency Group.
VNX Cinder Driver should check the volume types.
If storagetype:provisioning=compressed is specified,
VNX Cinder Driver should reject the CG creation operation.
When parsing group['volume_type_id'], the trailing comma should be
considered.
Jay S. Bryant [Tue, 3 Mar 2015 03:31:46 +0000 (21:31 -0600)]
Move oslo.messaging to the oslo_messaging namespace
This is the sixth in a series of changes to move to using
the new oslo_<library> namespace that is being used for
oslo libraries.
There is currently a shim in place that is allowing the old
oslo.<library> imports to work, but we need to be prepared for
when the shims go away. Thus we need patches like this one to
move to the new namespace.
This patch also updates our hacking check to ensure that no
instances of oslo.messaging sneak back in.
Xing Yang [Tue, 24 Feb 2015 02:38:14 +0000 (21:38 -0500)]
Create Consistency Group from CG Snapshot API
This patch addressed the following:
* Added a new create Consistency Group from CG Snapshot API.
- Note this is separate from the Create Consistency Group
API which requires volume types as the input.
* Added a corresponding driver API.
Xing Yang [Wed, 31 Dec 2014 02:27:56 +0000 (21:27 -0500)]
Modify Consistency Group API
This patch addressed the following:
* Modify Consistency Group
* Added an API that supports adding existing volumes to CG and removing
volumes from CG after it is created. It also allows the name and the
description to be modified.
* Added a volume driver API accordingly.
Guan Qiang [Mon, 9 Feb 2015 06:18:35 +0000 (14:18 +0800)]
Remove useless storage_availability_zone import
Config 'storage_availability_zone' have been removed from cinder.volume.manager
long time ago, but Still been imported by cinder.volume.api and
cinder.consistencygroup.api.
Tomoki Sekiyama [Tue, 9 Dec 2014 22:09:43 +0000 (17:09 -0500)]
Failover to alternative iSCSI portals on login failure
When the main iSCSI portal is unreachable by network failure etc.,
volume attach will fail even if the array has the other portal
addresses alive. To enable nova-compute and brick (with corresponding
patches) to fall-back to alternative portals, this patch makes Cinder
to contains the multiple portal addresses and corresponding IQNs and
LUNs in connection_info returned by initialize_connection API.
The main portal information is also returned in target_portal,
target_iqn, target_lun for backward compatibility. For example:
This patch makes LVM iSCSI backend return multiple portals,
iqns, and luns when iscsi_secondary_ip_addresses is specified in
cinder.conf.
It also make brick/initiator/connector.py to failover to alternative
paths, if specified, when it fails to establish the main path.
In addition, to unify the data structure, target_portal, target_iqn,
target_lun will also be included even if 'multipath=True' is specified
in connector information.
Thang Pham [Mon, 10 Nov 2014 22:38:42 +0000 (17:38 -0500)]
Snapshot and volume objects
Abstract volumes and volume snapshots into objects.
Get, create, and delete snapshot APIs were changed to
use new snapshot objects. A skeleton volume object was
created, but cinder internals were not changed to use
the volume object, although volume is referenced and
used by the snapshot object. The internals will be
changed to use volume object in a subsequent patch.
Thang Pham [Mon, 10 Nov 2014 22:02:02 +0000 (17:02 -0500)]
Cinder objects base
This patch is to insert the objects code base, currently based
on nova.objects. It will be later based on
oslo_versionedobjects once it is available. This is done
to make progress on changing cinder internals to use objects,
since nova.objects and oslo.versionedobjects are similar.
Inserted objects code base into cinder and modified cinder services
to use object-aware serializer.
Tom Swanson [Fri, 27 Feb 2015 17:25:38 +0000 (11:25 -0600)]
Dell Storage Center Driver API bug
In later versions of the Dell Storage Center REST API
the index attribute is removed from the ScVolume object.
Referencing this caused an exception that failed volume
creation. Replaced with instanceId.
Also _init_volume was mapping/unmapping to every server
that was currently up on the Storage Center rather than
a single server. While this would certainly initialize
the volume it would create a lot of extra work on a busy
system. We now break after the first map/unmap.
Changed copyright to 2015 to match when it will actually
be released.
Clear migration_status from a destination volume if migration fails
When some error happens during volume migration, cinder-volume
keeps remaining a destination volume because the volume copy may be
in the completing phase and a source volume may be deleted already.
In this case, the migration_status of the destination volume in
data base should be cleared. As a result, the volume can be handled
properly even if volume migration is failed. If the migration is
in the migrating phase, the volume should be deleted and an entry
of database is also destroyed because the source volume is remaining.
Also there is an another problem that a source volume is remaining
as migrating status if create_export raises exception. In this case,
source volume can't be deleted even if it will be unnecessary.
Jon Bernard [Tue, 24 Feb 2015 21:16:26 +0000 (16:16 -0500)]
RBD: Query volume features from ceph.conf
This patch enabled RADOSClient to query the local ceph.conf for rbd
features configuration, falling back to layering if nothing is found.
This allows users to disable default features or to opt in to upcoming
features that have not yet become default.
Xing Yang [Fri, 27 Feb 2015 05:45:09 +0000 (00:45 -0500)]
Correct a few changes in the VMAX driver
The following patch made a few wrong changes in the VMAX driver.
https://review.openstack.org/#/c/145780/
The following two files contain the wrong changes:
https://review.openstack.org/#/c/145780/15/cinder/tests/
test_emc_vmax.py line 604
https://review.openstack.org/#/c/145780/15/cinder/volume/drivers/
emc/emc_vmax_https.py
Erlon R. Cruz [Thu, 27 Nov 2014 18:44:23 +0000 (16:44 -0200)]
Fix HNAS driver parsing errors
Depending on some configurations of HNAS array, the driver returns an error
during volume creation or driver initialization. This patch adds some checks
to fix those parsing errors.
Jeegn Chen [Fri, 27 Feb 2015 06:59:48 +0000 (14:59 +0800)]
More validation logic for VNX CG creation
In VNX array, compressed LUNs cannot be added into a Consistency Group.
VNX Cinder Driver should check the volume types.
If storagetype:provisioning=compressed is specified,
VNX Cinder Driver should reject the CG creation operation.
Mike Perez [Thu, 26 Feb 2015 20:07:22 +0000 (12:07 -0800)]
Update v1 deprecation warnings
As discussed on IRC and mailing lists, we're going to give more time for
deprecation of the v1 API. Since we introduced the deprecation of the v1
API in Juno, we can remove it after EOL of Juno, which will be the
release of Liberty.
Review 145780 broke the lefthand driver. The author and reviewers
did not even bother to look at the 3rd party CI results that clearly
showed that this patch broke the hplefthand driver, due to changing
the imports. The change removed an import to one of the hplefthandclient
modules, which was then later accessed.
Reviewers, please check 3rd party CI when a cinder patch touches a driver.
Authors of patches, if you touch a driver, you need to check the 3rd party
CI failures and make sure your patch changes aren't causing the failures.
Steven Kaufer [Tue, 10 Feb 2015 21:12:16 +0000 (21:12 +0000)]
Replication status periodic task optimization
A periodic task exists to update the replication status for all volumes.
Currently, this task executes for all drivers and always retrieves all
volumes for the current host from the DB.
This patch set:
* Ensures that the periodic task is only activated if the driver actually
supports replication
* Only retrieves volumes from the DB if 'replication_status'!='disabled' in
the periodic task
Also, the driver documentation in cinder.volume.driver.VolumeDriver.
get_volume_stats() is updated to reflect that the 'replication' key indicates
that the driver supports replication; this is the key that was actually
implemented in the drivers that support replication.
nikeshmahalka [Wed, 25 Feb 2015 15:14:13 +0000 (20:44 +0530)]
Use Unique SCST Group names in SCST cinder helper driver.
Currently, while setting up GROUPS under targets in scst.conf,
we are using the Initiator IQN as the group name. When we have
multiple targets and we want to connect LUN's from these multiple
targets to the same initiator, we are using the same GROUP name in
all the Target sections.
It would be good to provide a unique group name by prepending the
GROUP name with the target name. So the GROUP name will be
"Target name+Initiator IQN".
nikeshmahalka [Thu, 19 Feb 2015 12:33:01 +0000 (18:03 +0530)]
Add CHAP persistence to SCST target helper
The SCST target helper currently does not have a mechanism to
persist CHAP records. Added this support so that LUNs will
get re-exported if Cinder service restarts.
Long shot, but only related change in Cinder since this started was this one. It's possible that the config specification on the cmd line does something "different". Let's find out.
Eric Brown [Tue, 24 Feb 2015 07:02:50 +0000 (23:02 -0800)]
Use oslo_config choices support
The oslo_config library added support for a choices keyword argument in
version 1.2.0a3. This commit leverages the use of choices for StrOpts in
Cinder's configuration.
Angela Smith [Tue, 24 Feb 2015 21:04:02 +0000 (13:04 -0800)]
Custom zone name prefix not being used by brcd driver
Config file setting for zone name prefix and activate flag were
not being picked up by the brocade zone driver. Modified the
way the config options are used for each fabric for these config
options.
Steven Kaufer [Mon, 15 Dec 2014 20:18:42 +0000 (20:18 +0000)]
GET volumes API sorting REST/volume/DB updates
This change is to support updating the v2 /volumes and /volumes/detail
APIs to support multiple sort keys and sort directions.
Contains:
* Updates to the v2 REST API to pass multiple sort keys and directions
to the volume API
* Volume API updates to accept and pass the sort information the DB layer
* API signature updates on the DB layer to accept a list of sort keys and
directions
Note that the defaulting of the sort keys and directions is done in
the dependent patch set in the new 'process_sort_params' function
(invoked in db.sqlalchemy.api); by default, the sort keys are
'created_at' and 'id' in the 'desc' direction.
Steven Kaufer [Mon, 15 Dec 2014 19:10:25 +0000 (19:10 +0000)]
GET volumes API sorting enhancements common utilities
This change is to support updating the v2 /volumes and /volumes/detail
APIs to support multiple sort keys and sort directions using the single
'sort' parameter, see API working group guidelines for syntax and
examples:
https://github.com/openstack/api-wg/blob/master/guidelines/
pagination_filter_sort.rst
Note that the existing 'sort_key' and 'sort_dir' parameters are being
deprecated and they cannot be used with the new 'sort' parameter.
This patch set contains utility functions to:
* Process the input sorting parameters on the request
* Process the sort keys and directions to include the default keys
The main complexity in this change deals with defaulting the sort keys
and directions so that volume order does not change for existing
paths. A new 'process_sort_params' function is created to consistently
process the sort parameters that will be passed from the REST API and
down to the DB API. This utility function will be invoked when the
sort params need to be processed (and defaulted) in the DB sqlalchemy
layer (prior to invoking the common paginate_query function).
These functions will be consumed in the subsequent change set:
https://review.openstack.org/#/c/141915/
The zone manager and the lookup service had some cruft code that
was incorrectly reading the cinder.conf sections it needed.
When cinder.conf was configured to use the Cisco driver, the zone
manager would never see it, because it was reading from the wrong
section of cinder.conf.
Jay S. Bryant [Mon, 23 Feb 2015 23:14:57 +0000 (17:14 -0600)]
Sync scheduler.filters module from oslo-incubator
The scheduler.filters module hasn't been updated since
January of 2014. As a result there are a number of changes
that we should pull into Cinder's copy.
Current HEAD in OSLO:
---------------------
commit e589dde0721a0a67e4030813e582afec6e70d042
Date: Wed Feb 18 03:08:12 2015 +0000
Merge "Have a little fun with release notes"
Changes merged with this patch by file:
__init__.py
- be81d6ba - Cleanup unused log related code
availability_zone_filter.py
- 4a47188e - Allow filters to only run once per request if their data is static
capabilities_filter.py
- 2fbf5065 - Remove oslo.log code and clean up versionutils API
- dacc065a - Merge "Update oslo log messages with translation domains"
- 86707cd5 - Remove None for dict.get()
- fcf517d7 - Update oslo log messages with translation domains
extra_specs_ops.py
- ac17de97 - Use oslo_utils instead of deprecated oslo.utils
- 6ff6b4b4 - Switch oslo-incubator to use oslo.utils and remove old modules
ignore_attempted_hosts_filter.py
- 2fbf5065 - Remove oslo.log code and clean up versionutils API
- 39625e18 - Set pbr 'warnerrors' option for doc build
- dacc065a - Merge "Update oslo log messages with translation domains"
- 86707cd5 - Remove None for dict.get()
- fcf517d7 - Update oslo log messages with translation domains
Tomoki Sekiyama [Mon, 19 Jan 2015 21:34:18 +0000 (16:34 -0500)]
Limit volume copy bandwidth per backend
Currently, by setting volume_copy_bps_limit in cinder.conf,
volume/image copy can be throttled to mitigate slow down of
instances' volume access during the copy. However, this is a global
setting and applied to all the backends.
This change enables admins to configure volume_copy_bps_limit to
different values among backends by specifying it in each backend
section.
In addition, with this change, the bps limit will be divided when
multiple volume copy operations run concurrently on the same backend.
For example, if volume_copy_bps_limit is set to 100MB/s, and 2 copies
are running on a backend, each copy can use up to 50MB/s.
This behavior will be useful for QoS which ensures a certain amount
of bandwidth is kept for instances' volume access.
This introduces Throttle classes to count volume copies in a backend
and setup bandwidth limit.
Steven Kaufer [Thu, 12 Feb 2015 14:45:18 +0000 (14:45 +0000)]
Generic filter support for volume queries
DB functions exist to get all volumes, to get all volumes in a particular
project, to get all volumes in a particular group, and to get all volumes
hosted on a particular host. See the following functions in the DB API:
Only the queries that get all volumes and that get all volumes by project
support additional filtering.
The purpose of this patch set is to make the filtering support consistent
across these APIs, adding it to the volume_get_all_by_group and the
volume_get_all_by_host APIs.
John Griffith [Thu, 19 Feb 2015 23:07:17 +0000 (16:07 -0700)]
Remove warnings for long vgs and lvs calls
We put some warning statements in a while back for
lvs and vgs commands that lasted more than 60 seconds
so we could easily query issues in the gate.
Since then we've made a number of changes to improve our
LVM interactions including the use of an LVM conf file with
a filter for only the VG's Cinder uses.
There's one issue remaining however where during the instance
rescue test in Nova where LVM get's stuck trying to open
devices that no longer exist. Part of this I think is due
to some manipulation that Nova/KVM does on the attached volume
when the test is run, but also the rescue process in Nova
uses LVM volumes as well and does some moving around and mounting
of devices.
The result is that LVM calls can take quite a while
(greater than 100 seconds) to complete during the rescue process.
This doesn't result in any failures, just that the rescue process
slows things down dramatically and is a pretty heavy process.
Also, the newer versions of process utils include completion
timing in the logs so we don't really need this any longer eve
to query timings and get info.
This patch removes those log statements, and proposes we
document somewhere that rescue operations on a setup that
includes c-vol service and nova-compute service on the same
node will temporarily impact LVM performance.
Matthew Treinish [Sat, 21 Feb 2015 01:35:01 +0000 (20:35 -0500)]
Use subunit-trace to enable output during unit test runs
This commit adds the pretty_tox.sh script and the usage of
subunit-trace to the cinder unit test jobs. This also updates the
run_tests.sh script to use subunit-trace so it'll have consistent
output formatting with the tox job.
Code Review / openstack-build / cinder-build.git / log