Steven Kaufer [Thu, 6 Mar 2014 17:20:50 +0000 (17:20 +0000)]
Include next link when default limit is reached
The /volumes and /volumes/details APIs support pagination and a
"next" link should be included when more data is available. When
the default "osapi_max" limit is reached then the "next" link is
not included in the API reply. In this case, the caller cannot
determine if there are any more volumes and has no marker value
such that they can retrieve the rest of the volumes.
The fix for this is to include the "next" link when the number of
volumes being returned is the maximum limit, even if the "limit"
parameter is not supplied.
Jay S. Bryant [Mon, 24 Feb 2014 01:51:41 +0000 (19:51 -0600)]
Sync latest Oslo config code for i18n
This sync pulls the latest Oslo config code over
to Cinder. This sync is needed to include a fix
to config generator which is needed as part of
the work to enabled i18n messages. Config generator
was failing when lazy message translation was enabled.
The following changes are included with this patch. Each
patch includes the file(s) the patch touches:
-> c178e56 Add basic Python 3 tests
--> cinder/openstack/common/__init__.py
-> 12bcdb7 Remove vim header
--> cinder/openstack/common/__init__.py
-> 547ab34 Fix Copyright Headers - Rename LLC to Foundation
--> cinder/openstack/common/__init__.py
-> 44b6ea3 Import oslo.config configuration file generator
--> cinder/openstack/common/config/__init__.py
-> dd9aa2b Remove unused variables
--> cinder/openstack/common/config/generator.py
-> 5dce17b Use entry points to discover options in libraries
--> tools/config/generate_sample.sh
--> cinder/openstack/common/config/generator.py
-> e8e636c generator: add an EXTRA_LIBRARIES env variable
--> cinder/openstack/common/config/generator.py
-> e3dddd7 generator: use EXTRA_* env vars in the bash script
--> tools/config/generate_sample.sh
--> cinder/openstack/common/config/generator.py
-> 6da13e8 generator: rename EXTRA_MODULES_FILE to RC_FILE
--> tools/config/generate_sample.sh
-> 763eedf Fix DictOpt support in config sample generator
--> cinder/openstack/common/config/generator.py
-> e839886 Config generator fails with lazy messages
--> cinder/openstack/common/config/generator.py
-> 343686b Add check_uptodate to tools/config
--> tools/config/check_uptodate.sh
Oslo version:
-> 0f24d82 Fix migration.db_version when no tables
-> Date: Sat, 22 Feb 2014 00:32:18 +0000
Jim Branen [Wed, 5 Mar 2014 00:38:41 +0000 (16:38 -0800)]
Fix HP LeftHand Performance issue with AO
Setting AdaptiveOptimization (AO) to ‘true’, the default value,
at volume create time significantly slows down the operation on
the LeftHand array. If at create time, AO is set to ‘true’, it
will result in an update operation following the create operation
to set this value. Therefore, it is best to not specify the value,
when ‘true’, and let it default to ‘true’.
Navneet Singh [Wed, 12 Feb 2014 21:02:08 +0000 (02:32 +0530)]
NetApp implementation for copy offload in clustered nfs driver
The Copy offload binary is a special RPC implemented in Ontap that
allows NFS clients to ask the server to copy data between
volumes in the same cluster efficiently. The special
binary will be developed by NetApp and distributed to its customers.
It will address two copy cases after efficient image clone failed.
First when image cache file is present in a different share than
the one holding volume. Second when glance is backed by nfs share
which is on the same cluster as nfs driver backend. Instead
of regular http download the copy offload workflow will be used to copy
image to share and volume. Resubmitting it as there was a problem
with build in previous submission.
Jay S. Bryant [Wed, 26 Feb 2014 20:40:38 +0000 (14:40 -0600)]
Remove str() from LOG.* and exceptions
This commit removes the use of str() from LOG.* messages
and exceptions.
The reason we need to remove str() is because str() will fail if
it gets a Unicode string that has something that doesn't translate
to ASCII in it. If such a situation is encountered you will lose
the message string in question. In most cases, the use of str() is
unnecessary for LOG.* and exception messages. Using %s is smart
enough to figure out what to do with what it is passed. It first
tries to str() it, if this fails it falls back to using unicode. Either
way, the result will then be something that gettextutils can
handle and translate.
Geraint North [Thu, 13 Feb 2014 14:17:27 +0000 (09:17 -0500)]
Volume manage/unmanage support
Implements: blueprint add-export-import-volumes
Volume manage/unmanage support.
This change adds two new API extensions:
volume_unmanage.py:
Adds an "os-unmanage" action on an existing volume, which causes a
delete operation to flow through the stack, with a flag that indicates
that a different method ("unmanage") should be called on the driver
instead of delete_volume. A default, empty, implementation of unmanage
is provided.
volume_manage.py
Adds a new "os-manage-volume" API. A POST to this URI is very similar to
volume creation, except that the images, snapshots and existing volumes
cannot be specified. Instead the following must be specified:
host: Cinder host on which the existing storage resides
ref: Driver-specific reference to the existing storage object
name, description, volume_type, metadata and availability_zone are
supported as per a normal volume creation.
In order to support some re-use between volume_manage and the regular
volume creation, add_visible_admin_metadata has been factored out into
the cinder utils.py module.
The rest of the changes are just the implications of the host/ref
specification moving through the api, rpcapi, manager and flow (API and
Manager) layers. Management of an existing volume causes the
manage_existing_get_size() and manage_existing() methods to be called on
the driver, and a reference LVM implementation is provided.
brick/local_dev/lvm.py now includes a method to rename an LV.
Ollie Leahy [Thu, 9 Jan 2014 15:14:11 +0000 (15:14 +0000)]
Add user defined extra capabilities
Provide a mechanism that will allow sys admins who are managing cinder
installations with multiple backends to assign key/value pairs to
backends. The key/value pairs can be used by the capabilities scheduler
to select between backends when requests specify volume types.
For example a sysadmin could specify the key 'service_level' with the
values 'high', 'medium' and 'low' for different backends. The sys admin
specifies the 'service_level' for each backend in cinder.conf as a json
string, for example:
Jim Branen [Mon, 3 Mar 2014 18:53:17 +0000 (10:53 -0800)]
Fix HP LeftHand migration with snapshots
Check for snapshots associated with volumes during assisted
migration, and fail if a snapshot is found.
When a volume is cloned on the HP LeftHand array, a snapshot
is created and the cloned volume is created from the snapshot.
The snapshot remains associated with the newly created volume.
Therefore, backend assisted migration should check for volumes
with snapshots and not use native APIs to migrate any volume
that has a snapshot.
Xing Yang [Fri, 14 Feb 2014 16:47:33 +0000 (11:47 -0500)]
Add EMC VNX Direct Driver in Cinder
This patch implements a driver based on the Cinder iSCSIDrver.
It performs volume operations on VNX using the NaviSecCLI command
line tool. It supports all required driver features.
Xing Yang [Sat, 1 Mar 2014 12:28:02 +0000 (07:28 -0500)]
Check if snapshot is deleted cleanly.
This patch added a check in delete_snapshot to make sure the
snapshot is deleted cleanly. Without this check, if someone
tries to delete the source volume immediately after the snapshot
deletion, it will fail with error that the volume still has
snapshot.
Daniel Gollub [Wed, 19 Feb 2014 06:41:24 +0000 (07:41 +0100)]
Restrict rootwrap find filter for IBM NAS and GPFS
Additional make the name of the filter unique, so it does not override
any other rule. Like the find rule of the NetAppNFS driver.
Rootwrap is making use of plain python ConfigParser which handles INI files
with key=value pair like fashion. Where the key is unique.
Eric Harney [Thu, 20 Feb 2014 16:48:41 +0000 (11:48 -0500)]
GlusterFS: Set permissions on qcow2 snapshot files
The GlusterFS driver sets specific permissions on the
volume file when volumes are created. Setting them on the
snapshot files at snapshot creation time is also required,
otherwise volume attaches will fail for volumes with snapshots
when using QEMU's libgfapi functionality.
This does not weaken the current security model in any
significant way, as it only lines up the permissions used
for snapshot files with those already used for volume files.
This will eventually be addressed by functionality in libvirt:
https://bugzilla.redhat.com/show_bug.cgi?id=1052014
We don't want to block the volume manager from servicing
requests. So this patch changes our use of time.sleep
to use loopingcall wait, which uses eventlet greenthread
sleep.
Jim Branen [Thu, 27 Feb 2014 18:41:34 +0000 (10:41 -0800)]
Fix volume stats with multiple LeftHand clusters
When more than one cluster exists in a LeftHand management
group, get_volume_stats may return the stats for the wrong
cluster. This patch fixes the problem by listing the cluster
name in the getClusterInfo query.
Ronen Kat [Thu, 23 Jan 2014 09:58:58 +0000 (11:58 +0200)]
Export and import backup service metadata
Add new admin API for backup-export and backup-import.
The new commands export the backup details (not actual backup) to
a string that can be imported again in another OpenStack cloud or
if the backup database was corrupted.
The code includes a default backup driver implementation.
Backup test code converted to use mock.
blueprint cinder-backup-recover-api
DocImpact new admin API calls backup-import and backup-export
Zhiteng Huang [Thu, 27 Feb 2014 01:58:17 +0000 (09:58 +0800)]
Don't clear host_state_map when scheduling
host_state_map was added to scheduler for the purpose of caching latest
host_state in memory for scheduler. With this cache, scheduler has the
latest host_state (e.g. free_capacity, allocated_capacity, etc) of hosts
even hosts haven't reported their updated status to scheduler.
Unfortunately, this cache is flushed when scheduling pulling all available
volume services from DB in current implementation, which is a bug.
This change remove the host_state_map.clear() so that scheduler is able to
maintain an up-to-date (well, mostly) view of all volume services in memory.
Also, added code to remove non-active host from the cache every time when
scheduler handles a new request. Multi-line docstrings in cinder/scheduler/
host_manager.py are also fixed.
Duncan Thomas [Wed, 19 Feb 2014 19:03:24 +0000 (19:03 +0000)]
Add optional ionice to volume clearing process
Allow the volume clearing process to have an ionice priority set to
reduce the performance impact of volume zeroing. Note that this may
cause volume clearing processes to get backed up on a busy system, use
with care.
Daniel Gollub [Wed, 19 Feb 2014 06:37:20 +0000 (07:37 +0100)]
Restrict rootwrap find filter for NetAppNFS driver
Additional make the name of the filter unique, so it does not override
any other rule. Like the find rule of the GPFS driver.
Rootwrap is making use of plain python ConfigParser which handles INI files
with key=value pair like fashion. Where the key is unique.
Eric Harney [Wed, 26 Feb 2014 16:38:29 +0000 (11:38 -0500)]
GlusterFS: Increase snapshot delete job timeout to two hours
Increase the timeout for Nova snapshot delete operations from ten
minutes to two hours. This helps prevent Cinder from terminating
operations prematurely that are still being processed by Nova.
It is not uncommon for snapshot delete jobs to run for longer than
ten minutes depending on the size of the snapshot and speed of the
storage backend.
This will be followed up with a more robust mechanism to keep track
of snapshot job progress as a later effort.
This patch changes the way the config options are
layed out in the cinder.conf to follow the
standards for grouping settings together.
Instead of using dynamic config names to group
settings together by fabric name, we look for
fabric groups based upon the fc_fabric_names.
We also create a new fc-zone-manager group
for all of the global fczm settings.
Ben Swartzlander [Tue, 25 Feb 2014 19:44:14 +0000 (11:44 -0800)]
Change warning message in NetApp driver for vsadmin creds
The new warning messages explicitly lists the extra_specs that
can't be supported depending on the access level fo the creds
that were provided. Also corrects a spelling error for the
netapp_nodedup extra_spec.
Bill Owen [Thu, 13 Feb 2014 21:55:47 +0000 (14:55 -0700)]
TSM backup driver changes to support file backup
Extend TSM backup driver to support backup of file based volumes,
as well as restore and delete of the backups. This in addition
to the current block based volume support. This involves determining
the type of volume this is requested to be backed up, and modifying
the dsmc command options accordingly. When a volume is backed
up, the backup mode and backup path are saved in the backup
service_metadata for use on restore and delete operations.
This commit includes some further refactoring based on pylint
feedback and system testing.
Code Review / openstack-build / cinder-build.git / log