Joe Gordon [Mon, 10 Mar 2014 18:02:13 +0000 (11:02 -0700)]
Use debug level logging during unit tests
When unit tests fail we need to debug, so we should have debug level logging on.
We use fixtures FakeLogger during testing, to enable debug logging pass
in log level as a parameter.
john-griffith [Wed, 12 Mar 2014 16:59:45 +0000 (10:59 -0600)]
Sync log.py from oslo-incubator
This pulls in the latest version of log.py from oslo-incubator
which elminates the constant user_identity key error.
Note that this sync does a full sync, in other words it uses
the oslo update script and pulls in the log.py file and all
of the associated dependendencies for that module.
Current HEAD in OSLO:
--------------------
Merge: d9ea4f0fd33d1e
Date: Wed Mar 12 17:00:13 2014 +0000
Merge "Fix gettextutil.Message handling of deep copy failures"
--------------------
This patch is a follow up to a review that merged
included and had some issues raised after it was merged.
This patch includes some hacking cleanup and some
unit test cleanup for the Fibre Channel Zone Manager.
Lucian Petrut [Fri, 7 Mar 2014 15:09:33 +0000 (17:09 +0200)]
Fixes cinder-volume service startup on Windows
The Windows service fails due to missing non-blocking IO features
in eventlet. This fix adds a conditional path on Windows to execute
the service accordingly.
This bug had already been fixed by this this commit:
https://review.openstack.org/#/c/44744/ but it had been reverted
here, causing the Windows service to fail again
https://review.openstack.org/67031
Nova bypasses the cinder checks for a volume
being available, when it tries to attach a volume
to a new host during live migration. The assumption
in cinder to this point has been that volumes can only
be attached to one host. The 3PAR driver worked under
that assumption. This assumption fell apart during detach
time as the driver was only looking for a VLUN on the
entire 3PAR, since it assumed it could only exist on one host.
This patch ensures that the driver looks for the VLUN on the
hostname it expects.
Luis A. Garcia [Tue, 11 Mar 2014 15:34:02 +0000 (09:34 -0600)]
Adding domain to context
Add support for upcoming "domain" concept in Keystone V3 API in
the context.
This fix ensures there is a user_identity in the context for the
corresponding new attribute in the log format string added in oslo. The
oslo change has the same change-id as this one.
Vincent Hou [Wed, 4 Dec 2013 08:51:20 +0000 (03:51 -0500)]
VMware: Take the volume size from the user input
When we create a volume from an image or a snapshot, we need to
take the size of the volume from the user input and validate whether
the size is appropriate instead of taking the image size as the
volume size directly.
The VMware ESX driver in Nova is now marked as deprecated and will be
removed in Juno release. This change marks the VMware ESX vmdk driver
as deprecated since it is intended for use with the ESX driver in Nova.
The plan is to remove the ESX vmdk driver in Juno release.
Flavio Percoco [Tue, 11 Mar 2014 09:33:45 +0000 (10:33 +0100)]
Generate config samples for oslo.messaging
This patch adds oslo.messaging to the
CINDER_CONFIG_GENERATOR_EXTRA_LIBRARIES env variable, which adds
oslo.messaging config options to cinder.conf.sample
Flavio Percoco [Fri, 7 Feb 2014 11:20:44 +0000 (12:20 +0100)]
Port to oslo.messaging
The oslo.messaging library takes the existing RPC code from oslo and
wraps it in a sane API with well defined semantics around which we can
make a commitment to retain compatibility in future.
The patch is large, but the changes can be summarized as:
* oslo.messaging>=1.3.0a4 is required; a proper 1.3.0 release will be
pushed before the icehouse release candidates.
* The new rpc module has init() and cleanup() methods which manage the
global oslo.messaging transport state. The TRANSPORT and NOTIFIER
globals are conceptually similar to the current RPCIMPL global,
except we're free to create and use alternate Transport objects
in e.g. the cells code.
* The rpc.get_{client,server,notifier}() methods are just helpers
which wrap the global messaging state, specifiy serializers and
specify the use of the eventlet executor.
* In oslo.messaging, a request context is expected to be a dict so
we add a RequestContextSerializer which can serialize to and from
dicts using RequestContext.{to,from}_dict()
* The allowed_rpc_exception_modules configuration option is replaced
by an allowed_remote_exmods get_transport() parameter. This is not
something that users ever need to configure, but it is something
each project using oslo.messaging needs to be able to customize.
* We maintain a global NOTIFIER object and create specializations of
it with specific publisher IDs in order to avoid notification driver
loading overhead.
* rpc.py contains transport aliases for backwards compatibility
purposes. setup.cfg also contains notification driver aliases for
backwards compat.
* The messaging options are moved about in cinder.conf.sample because
the options are advertised via a oslo.config.opts entry point and
picked up by the generator.
* We use messaging.ConfFixture in tests to override oslo.messaging
config options, rather than making assumptions about the options
registered by the library.
Steven Kaufer [Fri, 7 Mar 2014 04:54:50 +0000 (04:54 +0000)]
get volumes API does not handle limit=0
This bug fixes a regression from bug 1288429, where the "next"
link is added when the number of volumes returned is the
maximum limit (even if the "limit" param is not specified).
The regression is hit when a "limit" of 0 is specified; in this
case the logic to create the "next" link is still executed and an
exception is thrown. The fix is to add back in the "if" check that
implictly checks if the max_items is non-0.
Test code was also created to verify that a limit of 0 is handled
correctly.
Xing Yang [Fri, 7 Mar 2014 18:43:22 +0000 (13:43 -0500)]
EMC SMI-S delete snapshot unit test takes too long
EMC SMI-S unit test for deleting snapshot takes too long.
This patch sets Timout to 0 in cinder_emc_config.xml
so that it will return immediately. Without this fix,
it will wait for 10 seconds by default.
Steven Kaufer [Thu, 6 Mar 2014 17:20:50 +0000 (17:20 +0000)]
Include next link when default limit is reached
The /volumes and /volumes/details APIs support pagination and a
"next" link should be included when more data is available. When
the default "osapi_max" limit is reached then the "next" link is
not included in the API reply. In this case, the caller cannot
determine if there are any more volumes and has no marker value
such that they can retrieve the rest of the volumes.
The fix for this is to include the "next" link when the number of
volumes being returned is the maximum limit, even if the "limit"
parameter is not supplied.
Jay S. Bryant [Fri, 31 Jan 2014 22:46:29 +0000 (16:46 -0600)]
Re-enable lazy translation
This patch re-enables lazy translation to enable REST API
responses to be translated to the requested locale (I.E. setting
'Accept-Language: zh-CN'.)
This patch also removes the unnecessary manual import of
'cinder.openstack.common.gettextutils _' in Cinder code. Since _ is
still being imported in the cinder.openstack.common files from Oslo
we still need to set gettexttutils.enable_lazy(). enable_lazy will
be able to be removed once the manual imports of _ have been removed
from Oslo.
Jay S. Bryant [Mon, 24 Feb 2014 01:51:41 +0000 (19:51 -0600)]
Sync latest Oslo config code for i18n
This sync pulls the latest Oslo config code over
to Cinder. This sync is needed to include a fix
to config generator which is needed as part of
the work to enabled i18n messages. Config generator
was failing when lazy message translation was enabled.
The following changes are included with this patch. Each
patch includes the file(s) the patch touches:
-> c178e56 Add basic Python 3 tests
--> cinder/openstack/common/__init__.py
-> 12bcdb7 Remove vim header
--> cinder/openstack/common/__init__.py
-> 547ab34 Fix Copyright Headers - Rename LLC to Foundation
--> cinder/openstack/common/__init__.py
-> 44b6ea3 Import oslo.config configuration file generator
--> cinder/openstack/common/config/__init__.py
-> dd9aa2b Remove unused variables
--> cinder/openstack/common/config/generator.py
-> 5dce17b Use entry points to discover options in libraries
--> tools/config/generate_sample.sh
--> cinder/openstack/common/config/generator.py
-> e8e636c generator: add an EXTRA_LIBRARIES env variable
--> cinder/openstack/common/config/generator.py
-> e3dddd7 generator: use EXTRA_* env vars in the bash script
--> tools/config/generate_sample.sh
--> cinder/openstack/common/config/generator.py
-> 6da13e8 generator: rename EXTRA_MODULES_FILE to RC_FILE
--> tools/config/generate_sample.sh
-> 763eedf Fix DictOpt support in config sample generator
--> cinder/openstack/common/config/generator.py
-> e839886 Config generator fails with lazy messages
--> cinder/openstack/common/config/generator.py
-> 343686b Add check_uptodate to tools/config
--> tools/config/check_uptodate.sh
Oslo version:
-> 0f24d82 Fix migration.db_version when no tables
-> Date: Sat, 22 Feb 2014 00:32:18 +0000
Jim Branen [Wed, 5 Mar 2014 00:38:41 +0000 (16:38 -0800)]
Fix HP LeftHand Performance issue with AO
Setting AdaptiveOptimization (AO) to ‘true’, the default value,
at volume create time significantly slows down the operation on
the LeftHand array. If at create time, AO is set to ‘true’, it
will result in an update operation following the create operation
to set this value. Therefore, it is best to not specify the value,
when ‘true’, and let it default to ‘true’.
Navneet Singh [Wed, 12 Feb 2014 21:02:08 +0000 (02:32 +0530)]
NetApp implementation for copy offload in clustered nfs driver
The Copy offload binary is a special RPC implemented in Ontap that
allows NFS clients to ask the server to copy data between
volumes in the same cluster efficiently. The special
binary will be developed by NetApp and distributed to its customers.
It will address two copy cases after efficient image clone failed.
First when image cache file is present in a different share than
the one holding volume. Second when glance is backed by nfs share
which is on the same cluster as nfs driver backend. Instead
of regular http download the copy offload workflow will be used to copy
image to share and volume. Resubmitting it as there was a problem
with build in previous submission.
Jay S. Bryant [Wed, 26 Feb 2014 20:40:38 +0000 (14:40 -0600)]
Remove str() from LOG.* and exceptions
This commit removes the use of str() from LOG.* messages
and exceptions.
The reason we need to remove str() is because str() will fail if
it gets a Unicode string that has something that doesn't translate
to ASCII in it. If such a situation is encountered you will lose
the message string in question. In most cases, the use of str() is
unnecessary for LOG.* and exception messages. Using %s is smart
enough to figure out what to do with what it is passed. It first
tries to str() it, if this fails it falls back to using unicode. Either
way, the result will then be something that gettextutils can
handle and translate.
Geraint North [Thu, 13 Feb 2014 14:17:27 +0000 (09:17 -0500)]
Volume manage/unmanage support
Implements: blueprint add-export-import-volumes
Volume manage/unmanage support.
This change adds two new API extensions:
volume_unmanage.py:
Adds an "os-unmanage" action on an existing volume, which causes a
delete operation to flow through the stack, with a flag that indicates
that a different method ("unmanage") should be called on the driver
instead of delete_volume. A default, empty, implementation of unmanage
is provided.
volume_manage.py
Adds a new "os-manage-volume" API. A POST to this URI is very similar to
volume creation, except that the images, snapshots and existing volumes
cannot be specified. Instead the following must be specified:
host: Cinder host on which the existing storage resides
ref: Driver-specific reference to the existing storage object
name, description, volume_type, metadata and availability_zone are
supported as per a normal volume creation.
In order to support some re-use between volume_manage and the regular
volume creation, add_visible_admin_metadata has been factored out into
the cinder utils.py module.
The rest of the changes are just the implications of the host/ref
specification moving through the api, rpcapi, manager and flow (API and
Manager) layers. Management of an existing volume causes the
manage_existing_get_size() and manage_existing() methods to be called on
the driver, and a reference LVM implementation is provided.
brick/local_dev/lvm.py now includes a method to rename an LV.
Ollie Leahy [Thu, 9 Jan 2014 15:14:11 +0000 (15:14 +0000)]
Add user defined extra capabilities
Provide a mechanism that will allow sys admins who are managing cinder
installations with multiple backends to assign key/value pairs to
backends. The key/value pairs can be used by the capabilities scheduler
to select between backends when requests specify volume types.
For example a sysadmin could specify the key 'service_level' with the
values 'high', 'medium' and 'low' for different backends. The sys admin
specifies the 'service_level' for each backend in cinder.conf as a json
string, for example:
Jim Branen [Mon, 3 Mar 2014 18:53:17 +0000 (10:53 -0800)]
Fix HP LeftHand migration with snapshots
Check for snapshots associated with volumes during assisted
migration, and fail if a snapshot is found.
When a volume is cloned on the HP LeftHand array, a snapshot
is created and the cloned volume is created from the snapshot.
The snapshot remains associated with the newly created volume.
Therefore, backend assisted migration should check for volumes
with snapshots and not use native APIs to migrate any volume
that has a snapshot.
Currently the PBM wsdl files have to be downloaded to the local file
system and needs to be configured in 'pbm_wsdl_location'. If it is
manually configured then PBM feature is enabled.
This patch does away with the manual configuration of
'pbm_wsdl_location'. PBM wsdl files are now shipped with the driver.
VC version is fetched from the backend server and the corresponding
wsdl file is used. If the backend is a VC version that is less than
5.5 then PBM feature is turned off.
Adding support for a global pbm policy configuration for the vmdk
driver. Setting the 'pbm_default_policy' in cinder.conf will be
used as the default storage profile name to be used when creating
a volume without associated vmware:storage_profile extra spec.
Also renaming 'vmware-pbm-wsdl' to 'pbm-wsdl-location' to use
nova driver's naming convention.
vmware: check datastore availability during create
The vmdk driver does a lazy creation of the volume's backing vmdk
only at attach time. This is done to save on the huge copy cost
if the vmdk needs to be moved 'closer' to the VM instance at
attach time.
Still during a create volume call the driver needs to verify
availability of a suitable datastore that matches the given
storage profile as well as capacity to accommodate the given
volume size. This check needs to happen at three places - create
volume, create volume from snapshot and create volume from another
source volume.
This change adds a new feature to vmdk cinder driver. Users will be
able to specify a vSphere storage policy in a volume-type. Creating
a volume with this volume-type associates the volume with the
storage policy in vSphere. The driver places this volume only on a
datastore that satisfies this storage policy.
Code Review / openstack-build / cinder-build.git / log