There was a race condition between the two Event() objects being
created within loopingcall and Retry in api module. Did away with
the Event object in Retry and used loopingcall the right way.
John Griffith [Fri, 31 Jan 2014 03:53:37 +0000 (03:53 +0000)]
Remove create_export from volume create
Currently the LVM driver creates iscsi targets on
volume creation, even though it's only used at
attach time. This isn't necessary and in fact
causes issues if a volume is extended because the
target information isn't currently updated after the
extend. In addition a number of other drivers inclduing
the LIO iscsi driver require the target be created with
initiator info, so the tgt that's created initially
again isn't valid.
This change removes the create_export call from the
volume create process and makes it part of the
managers intialize_connection routine which is
more appropriate.
This change also removes the target on detach since
it's not used any longer and again as the volume may
be modified or extended.
Time unit is missing in the description of vmdk driver option
"vmware_task_poll_interval". Due to this, the vmdk driver config
reference document doesn't have the timeunit in its description
column. This fix adds the missing time unit (seconds) in the
driver option.
Avishay Traeger [Sun, 26 Jan 2014 10:39:27 +0000 (12:39 +0200)]
Allow spaces in host names in the storwize driver
Storwize/SVC naming rules allow host objects to have spaces in their
names. The SSH injection filter will currently reject such names, and so
we surround the names with quotes. Please note that this doesn't allow
to have an arbitrary character in there except a space (anything else
will be cleaned up by the driver or rejected by the SSH injection filter
up the call chain).
Bertrand Lallau [Sat, 1 Feb 2014 13:33:33 +0000 (14:33 +0100)]
Remove a catching exception during delete_volume
GlanceMetadataNotFound Exception was catching during
a call to volume_glance_metadata_delete_by_volume().
This exception is never thrown by the implementation.
This catching has been removed.
Ryan McNair [Mon, 27 Jan 2014 21:48:51 +0000 (21:48 +0000)]
Add support for special char in volume metadata
Using special characters such as "&" in volume metadata caused SAX
parser to split text into multiple child nodes, as stated in the
specs for SAX parsers. In this case extract_text would return
'None' because text was return if there was exactly
one child node. Concatenating the values of child nodes accounts
for SAX parsers splitting contigous characters into
multiple chunks.
Brianna Poulos [Wed, 29 Jan 2014 20:49:20 +0000 (15:49 -0500)]
Stop volume_type_encryption creation when in use
This bug fix addresses bug #1274252. It requires a volume
type to have no volumes with the type before allowing it
to be made encrypted. This prevents a situation where
non-encrypted volumes could have an encrypted volume type.
This fix also adds a unit test to confirm functionality.
john-griffith [Wed, 29 Jan 2014 04:30:59 +0000 (21:30 -0700)]
Revert initialize_connection changes
A change was merged that reusulted in interittent
gate failures. It seems that in certain cases the
intitialize is doing an update targets and is
deleting the existing targets and not recreating them.
The idea of this patch was to fix up the iscis targets
for cases where volumes are extended.
A better solution is in progress that will remove the
target creation from the create_volume process altogether.
Currently we seem to have create_export, ensure_export and
now initialize that all are designed to do very similar
things.
A bug has been filed to address this and will attempt
to collapse these functions to be done at attach time.
Florian Haas [Mon, 20 Jan 2014 21:42:11 +0000 (22:42 +0100)]
VolumeManager: initialize even if a volume can't be found
If a previously volume cannot be exported (for example, because
an iSCSI target's backing LV is no longer present), VolumeManager
would previously bail out, leaving the volume service uninitialized.
This left volumes dangling in limbo: their state would be reported
as available (clearly not a reflection of reality), and they could
not be deleted (because the volume service that would be responsible
for deletion was unavailable).
Instead, catch an exception raised by ensure_export() separately,
log it, and set the volume to the error state.
For the tgt backend, this will also remove the volume_path file.
Previously this would cause an ISCSITargetRemoveFailed exception
when the volume would be removed. Instead, simply log a warning
and return, the way other backends (example: RBD) already do.
Also, add an info message that reflects the actual path and
contents of the volume_path file.
Finally, fix up the tgtadm unit test so that it tests for
identical IQNs,
Eric Harney [Mon, 27 Jan 2014 23:46:26 +0000 (18:46 -0500)]
Add create_iscsi_target stub to TargetAdmin
This should be defined in TargetAdmin, as it is assumed to be
present in all subclasses by callers.
IetAdm and LioAdm may need additional work to behave as desired
in all circumstances (such as volume extend), but this should
at least stabilize things in the meantime.
This patch validates the keys of the extra specs
before setting them.
This will make possible to remove those keys.
It allows alphanumeric characters, underscores,
periods, colons and hyphens.
Zhiteng Huang [Thu, 9 Jan 2014 06:54:22 +0000 (14:54 +0800)]
Make sure report_interval is less than service_down_time
Services that inherit service.py/Service class would register
themselves to DB and then update stats periodically (every
report_interval second). The consumer of this kind of information,
like scheduler or 'os-service' API extension, will consider a service
is 'up' (active) if last update from that service is not longer than
'service_down_time' ago.
The problem is if 'report_interval' was configured/provided greater
than 'service_down_time' by mistake, services would then be always
considered in 'down' state, which can result in unsuccesful placement
of volume create request for example. This is what Bug #1255685 is
about.
In previous fix: https://review.openstack.org/#/c/60760/, a
configuration check helper function basic_config_check() was added
*wrongly* to WSGIService class instead of Service class. This patch
moves the configuration check helper function and the check to the
right place to make sure 'report_interval' is less then
'service_down_time'.
john-griffith [Sat, 25 Jan 2014 02:12:01 +0000 (19:12 -0700)]
Fix up the way we do iqn variable in unit test
A recent bug was reported when a developer was trying to work
with the iscsi target unit tests. There were two problems discovered,
one of which has been addressed by the dependent patch of this commit
(https://review.openstack.org/#/c/68223/) which fixed the inconsistencies
in the string we used for iqn's in our test.
This patch addresses another issue that's somewhat related and was
exposed when the developer tried to use the "self.target_name" variable
instead of hard-coding the iqn's in the tests.
The problem is that the target_name variable uses an invalid format
for the iqn based on cinder-defaults. It neglected to use the
volume_name prefix (in this case: "volume-").
This patch modifies the self.target_name variable to be more accurate
with what we would see in real usage, it also modifies the test to just
use this variable rather than hard coding it and running the risk of
having mismatches again in the future.
John Griffith [Fri, 24 Jan 2014 21:51:21 +0000 (14:51 -0700)]
Catch new iscsi exception
A while back we fixed up a target create/update error
that was identified in the gating tests:
https://review.openstack.org/#/c/47513/
Since then a new patch was introduced to to the update and separate
it so that it could easily be used in other places.
https://review.openstack.org/#/c/58599/
The problem is this added a new exception "TargetUpdate" and
didn't add this to the except block that we have.
This patch changes the catch block to handle the new exception and
do the retry, thereby putting the original fix back in place.
ling-yun [Mon, 20 Jan 2014 10:48:43 +0000 (18:48 +0800)]
Delete volume transfer in volume_destroy function
Doing volume force delete operation on volume-A will change volume-A's
status to 'deleted', so all related data about volume-A should be deleted.
Now current version's volume_destroy function doesn't delete related data
of volume transfer. If there is volume transfer about volume-A, first do
force delete operation on volume A, and thus the volume-A's transfer data
would be residual. So remove the related data of volume transfer in
volume_destroy function.
Zhiteng Huang [Wed, 8 Jan 2014 06:50:29 +0000 (14:50 +0800)]
Enable multi-process for API service
Due to the limit of Python interpreter, API service of Cinder can't
really utilize underlying multi-core architecture even libraries
like eventlet has been used. To make API service much more scalable,
we'd adopt multi-process (worker) mode that has been used for long
in Glance/Swift/Nova.
The default behavior isn't changed with this patch, Cinder API
service will still run in one process (default value of
osapi_volume_workers is None).
Implementation wise, a good portion of cinder/service.py has been
removed because those content has been merged in Oslo version of
service module. cinder/wsgi.py is also updated to adopt the change
for multiple WSGI servers running in separate processes.
Implement bp: multi-process-api-service
DocImpact: 'New config option osapi_volume_workers is used to specify
number of API service workers (OS processes) to launch for Cinder
API service. Setting this config option to a proper value (e.g.
osapi_volume_workers = # of CPU cores/threads of the machine) can
greatly improve the total throughput of API service [# of API
requests can be handled per second].'
Also removed out-dated comments in bin/cinder-api due to the fact
that this bug [1] has been fixed in eventlet 0.9.13
Eric Harney [Thu, 16 Jan 2014 18:45:05 +0000 (13:45 -0500)]
Sync RPC module from Oslo
22e971a safe_log Sanitize Passwords in List of Dicts 8b2b0b7 Use hacking import_exceptions for gettextutils._ 23f6029 Use six.seraise() instead of `raise exc, val, tb` 6d0a6c3 Correct invalid docstrings 7cac1ac Fix mis-spellings ef406a2* Create a shared queue for QPID topic consumers e6494c2 Use six.iteritems to make dict work on Python2/3 e227c0e* Properly reconnect subscribing clients when QPID broker restarts 16fb43b Replace data structures' attribute with six module 27b21bc Unify different names between Python2/3 with six.moves 12bcdb7 Remove vim header 3970d46 Fix typos in oslo 1771a77 Adjust import order according to PEP8 imports rule 0717d1d matchmaker_redis compatibility with redis-py 2.4 f88d59a Drop RPC securemessage.py 39f6589 Use localisation 8a3996a Fix missing space in help text 4bfb7a2 Apply six for metaclass d7d74a7 Add `versionutils` for version compatibility checks 3cdd157 Add third element to RPC versions for backports 76972e2* Support a new qpid topology 284b13a Raise timeout in fake RPC if no consumers found 9721129 exception: remove 7b0cb37 Don't eat callback exceptions 69abf38 requeue instead of reject 28395d9 Fixes files with wrong bitmode bec54ac Fix case error in qpid exchange type "direct" 61c4cde Ensure context type is handled when using to_dict 223f9e1 Clarify precedence of secret_key_file a035f95 Don't shadow cfg import in securemessage 0f88575 Remove redundant global keyword in securemessage 848c4d5 Some nitpicky securemessage cleanups 5c71c25 Allow non-use of cfg.CONF in securemessage 9157286 RPC: Add MessageSecurity implementation 2031e60 Refactors boolean returns a047a35 Make ZeroMQ based RPC consumer threads more robust 34a6842 On reconnecting a FanoutConsumer, don't grow the topic name f52446c Add serializer param to RPC service 5ff534d Add config for amqp durable/auto_delete queues f9f1b4f Revert "Add support for heartbeating in the kombu RPC driver" 1a2df89 Enable H302 hacking check 7bfd443 Avoid shadowing Exception 'message' attribute 99b7c35 Convert kombu SSL version string into integer c37f6aa Add support for heartbeating in the kombu RPC driver 3006787 Sanitize passwords in _safe_log dea334a Replace sys.exit by a RPCException 3969355 Fix exception arg typo 22ec8ff Make AMQP based RPC consumer threads more robust 13650b1 rpc: remove some unused serialization code e204885 Optionally reject messages on exception. 688832f Remove unused zmq relay functionality 719eba4 Don't reconnect to exclusive fanout consumers. 980fe5f Allow exceptions to hop up cells 80476f0 Add can_send_version() to RpcProxy. 7119e29 Enable hacking H404 test. 32e2a25 zmq: remove someone unused code from ZmqClient 6d42ced Remove the amqp_rpc_single_reply_queue option from Havana 7ce5441 Fix problem with long messages in Qpid e3545f8 Enable hacking H402 test 15d8d69 Silence exceptions from qpid connection.close() 484a1df Enable hacking H403 test 35660da Enable hacking H401 test 1a9a744 MatchMakerStub - make it work & add test 35aad91 Use zero-copy in ZeroMQ proxy (zmq-rpc-receiver) b677b13 Remove rootwrap from IPC directory creation 93ee6e3 Add generic serialization support fe2f108 Enable hacking H702 localization test 0a14e1d Enable hacking H703: Multiple positional placeholders df7ea83 Allow RPC_API_NAMESPACE on RpcProxy objects aa89d8b Support capping message versions in the client. 1d7920a Extract matchmaker_ring to own module 3e33692 Merge "Removes len() on empty sequence evaluation" 44c79fc Merge "python3: use 'as' syntax for exception assignment" 0c54b72 python3: use 'as' syntax for exception assignment a514693 Removes len() on empty sequence evaluation fde1e15 Convert unicode for python3 portability 0c9047c* Ensure that qpid connection is closed. 120ddef Improve Python 3.x compatibility
(* after hash indicates this was already ported to Cinder)
Oslo version: 7a51572 Merge "Implement cache abstraction layer"
Date: Wed Jan 15 19:31:16 2014 +0000
Eric Harney [Thu, 16 Jan 2014 19:08:53 +0000 (14:08 -0500)]
Sync common modules from Oslo
Identified as dependencies for RPC sync by update.py
Contains code from: 12bcdb7 Remove vim header e6494c2 Use six.iteritems to make dict work on Python2/3 6d0a6c3 Correct invalid docstrings fe18bfa Remove keystone from default_log_levels default 0228b81 Logging excepthook: print exception info if debug=True 8b2b0b7 Use hacking import_exceptions for gettextutils._ 6d0a6c3 Correct invalid docstrings fe18bfa Remove keystone from default_log_levels default 13eb01c Adding domain to context and log 27b21bc Unify different names between Python2/3 with six.moves a6f40ac Don't log to stdout when log_dir is set 4c22556 Use py3kcompat urlutils functions instead of urlparse 0bf03b7 Add network_utils.urlsplit e456727 Remove useless logging in networks_utils 8b2b0b7 Use hacking import_exceptions for gettextutils._ 37e46bb disable SIGHUP restart behavior in foreground f08f6ba service: replace eventlet event by threading 32cd4c0 Allow configurable ProcessLauncher liveness check 25ff65e Make wait & stop methods work on all threads 7013471 Fix typo in parameter documentation (timeutils) ae931a9 Avoid TypeError in is_older_than, is_newer_than ad17a69 Fix filter() usage due to python 3 compability
Oslo version: 7a51572 Merge "Implement cache abstraction layer"
Date: Wed Jan 15 19:31:16 2014 +0000
Eric Harney [Thu, 16 Jan 2014 18:25:33 +0000 (13:25 -0500)]
Sync py3kcompat, sslutils, versionutils from Oslo
Dependencies identified by update.py for RPC update.
py3kcompat: 12d3bbc Add method quote_plus in module py3kcompat.urlutils 8575d87 Removed copyright from empty files 0d8f18b Use urlutils functions instead of urllib/urllib2 12bcdb7 Remove vim header 4c22556 Use py3kcompat urlutils functions instead of urlparse 71743d9 Add urlopen/URLError/pathname2url in urlutils 0f2906f py3kcompat: Add unquote eaec23b py3kcompat: Add urljoin compatibility 35c317b Fix capitalization, it's OpenStack 4d12ad1 python3: Import translation layer
sslutils: 8b2b0b7 Use hacking import_exceptions for gettextutils._ 12bcdb7 Remove vim header 1a2df89 Enable H302 hacking check 99b7c35 Convert kombu SSL version string into integer b0b37ce Fix IBM copyright strings f63ea05 Use oslo-config-2013.1b3 e50b68c Support for SSL in wsgi.Service
versionutils: 8b2b0b7 Use hacking import_exceptions for gettextutils._ a5ae087 fixed typos 45658e2 Fix violations of H302:import only modules 37ea814 Adds decorator to deprecate functions and methods 12bcdb7 Remove vim header d7d74a7 Add `versionutils` for version compatibility checks
Oslo version: 7a51572 Merge "Implement cache abstraction layer"
Date: Wed Jan 15 19:31:16 2014 +0000
Eric Harney [Thu, 16 Jan 2014 18:09:05 +0000 (13:09 -0500)]
Sync gettextutils from Oslo
Update gettextutils as a dependency for RPC updates
Requires changes within Cinder as well for
"get_localized_message" -> "translate" method rename.
221c37d Allow the Message class to have non-English default locales 0e1af5b Implementation of translation log handler f5686b1 Merge "Translation Message improvements" 8b2b0b7 Use hacking import_exceptions for gettextutils._ 2cfc1a7 Translation Message improvements 45658e2 Fix violations of H302:import only modules 12bcdb7 Remove vim header
Oslo version: 7a51572 Merge "Implement cache abstraction layer"
Date: Wed Jan 15 19:31:16 2014 +0000
Avishay Traeger [Wed, 22 Jan 2014 12:27:39 +0000 (14:27 +0200)]
Storwize driver cleanup
1. Moved Storwize driver into 'ibm' directory and refactored into
separate files to improve readability and debug-ability. Other IBM
drivers will be moved into this directory in a separate patch.
2. Errors coming from storage now raise VolumeBackendAPIException,
and those coming from the driver raise VolumeDriverException.
3. Fixed many pylint errors.
4. Removed any usage of mox from unit tests in favor of mock.
Li Min Liu [Thu, 9 Jan 2014 06:54:09 +0000 (14:54 +0800)]
Add support for retype in Storwize/SVC driver
In Storwize/SVC, retype can be handled by a few cases:
1. The change may just require setting an option on the volume
2. The change may require moving the data in the same pool (e.g.,
compression)
3. The scheduler may have chosen a different pool virtualized by the
same Storwize or SVC, in which case we move the data as in #2
Code Review / openstack-build / cinder-build.git / log