Eric Harney [Thu, 16 Jan 2014 18:45:05 +0000 (13:45 -0500)]
Sync RPC module from Oslo
22e971a safe_log Sanitize Passwords in List of Dicts 8b2b0b7 Use hacking import_exceptions for gettextutils._ 23f6029 Use six.seraise() instead of `raise exc, val, tb` 6d0a6c3 Correct invalid docstrings 7cac1ac Fix mis-spellings ef406a2* Create a shared queue for QPID topic consumers e6494c2 Use six.iteritems to make dict work on Python2/3 e227c0e* Properly reconnect subscribing clients when QPID broker restarts 16fb43b Replace data structures' attribute with six module 27b21bc Unify different names between Python2/3 with six.moves 12bcdb7 Remove vim header 3970d46 Fix typos in oslo 1771a77 Adjust import order according to PEP8 imports rule 0717d1d matchmaker_redis compatibility with redis-py 2.4 f88d59a Drop RPC securemessage.py 39f6589 Use localisation 8a3996a Fix missing space in help text 4bfb7a2 Apply six for metaclass d7d74a7 Add `versionutils` for version compatibility checks 3cdd157 Add third element to RPC versions for backports 76972e2* Support a new qpid topology 284b13a Raise timeout in fake RPC if no consumers found 9721129 exception: remove 7b0cb37 Don't eat callback exceptions 69abf38 requeue instead of reject 28395d9 Fixes files with wrong bitmode bec54ac Fix case error in qpid exchange type "direct" 61c4cde Ensure context type is handled when using to_dict 223f9e1 Clarify precedence of secret_key_file a035f95 Don't shadow cfg import in securemessage 0f88575 Remove redundant global keyword in securemessage 848c4d5 Some nitpicky securemessage cleanups 5c71c25 Allow non-use of cfg.CONF in securemessage 9157286 RPC: Add MessageSecurity implementation 2031e60 Refactors boolean returns a047a35 Make ZeroMQ based RPC consumer threads more robust 34a6842 On reconnecting a FanoutConsumer, don't grow the topic name f52446c Add serializer param to RPC service 5ff534d Add config for amqp durable/auto_delete queues f9f1b4f Revert "Add support for heartbeating in the kombu RPC driver" 1a2df89 Enable H302 hacking check 7bfd443 Avoid shadowing Exception 'message' attribute 99b7c35 Convert kombu SSL version string into integer c37f6aa Add support for heartbeating in the kombu RPC driver 3006787 Sanitize passwords in _safe_log dea334a Replace sys.exit by a RPCException 3969355 Fix exception arg typo 22ec8ff Make AMQP based RPC consumer threads more robust 13650b1 rpc: remove some unused serialization code e204885 Optionally reject messages on exception. 688832f Remove unused zmq relay functionality 719eba4 Don't reconnect to exclusive fanout consumers. 980fe5f Allow exceptions to hop up cells 80476f0 Add can_send_version() to RpcProxy. 7119e29 Enable hacking H404 test. 32e2a25 zmq: remove someone unused code from ZmqClient 6d42ced Remove the amqp_rpc_single_reply_queue option from Havana 7ce5441 Fix problem with long messages in Qpid e3545f8 Enable hacking H402 test 15d8d69 Silence exceptions from qpid connection.close() 484a1df Enable hacking H403 test 35660da Enable hacking H401 test 1a9a744 MatchMakerStub - make it work & add test 35aad91 Use zero-copy in ZeroMQ proxy (zmq-rpc-receiver) b677b13 Remove rootwrap from IPC directory creation 93ee6e3 Add generic serialization support fe2f108 Enable hacking H702 localization test 0a14e1d Enable hacking H703: Multiple positional placeholders df7ea83 Allow RPC_API_NAMESPACE on RpcProxy objects aa89d8b Support capping message versions in the client. 1d7920a Extract matchmaker_ring to own module 3e33692 Merge "Removes len() on empty sequence evaluation" 44c79fc Merge "python3: use 'as' syntax for exception assignment" 0c54b72 python3: use 'as' syntax for exception assignment a514693 Removes len() on empty sequence evaluation fde1e15 Convert unicode for python3 portability 0c9047c* Ensure that qpid connection is closed. 120ddef Improve Python 3.x compatibility
(* after hash indicates this was already ported to Cinder)
Oslo version: 7a51572 Merge "Implement cache abstraction layer"
Date: Wed Jan 15 19:31:16 2014 +0000
Eric Harney [Thu, 16 Jan 2014 19:08:53 +0000 (14:08 -0500)]
Sync common modules from Oslo
Identified as dependencies for RPC sync by update.py
Contains code from: 12bcdb7 Remove vim header e6494c2 Use six.iteritems to make dict work on Python2/3 6d0a6c3 Correct invalid docstrings fe18bfa Remove keystone from default_log_levels default 0228b81 Logging excepthook: print exception info if debug=True 8b2b0b7 Use hacking import_exceptions for gettextutils._ 6d0a6c3 Correct invalid docstrings fe18bfa Remove keystone from default_log_levels default 13eb01c Adding domain to context and log 27b21bc Unify different names between Python2/3 with six.moves a6f40ac Don't log to stdout when log_dir is set 4c22556 Use py3kcompat urlutils functions instead of urlparse 0bf03b7 Add network_utils.urlsplit e456727 Remove useless logging in networks_utils 8b2b0b7 Use hacking import_exceptions for gettextutils._ 37e46bb disable SIGHUP restart behavior in foreground f08f6ba service: replace eventlet event by threading 32cd4c0 Allow configurable ProcessLauncher liveness check 25ff65e Make wait & stop methods work on all threads 7013471 Fix typo in parameter documentation (timeutils) ae931a9 Avoid TypeError in is_older_than, is_newer_than ad17a69 Fix filter() usage due to python 3 compability
Oslo version: 7a51572 Merge "Implement cache abstraction layer"
Date: Wed Jan 15 19:31:16 2014 +0000
Eric Harney [Thu, 16 Jan 2014 18:25:33 +0000 (13:25 -0500)]
Sync py3kcompat, sslutils, versionutils from Oslo
Dependencies identified by update.py for RPC update.
py3kcompat: 12d3bbc Add method quote_plus in module py3kcompat.urlutils 8575d87 Removed copyright from empty files 0d8f18b Use urlutils functions instead of urllib/urllib2 12bcdb7 Remove vim header 4c22556 Use py3kcompat urlutils functions instead of urlparse 71743d9 Add urlopen/URLError/pathname2url in urlutils 0f2906f py3kcompat: Add unquote eaec23b py3kcompat: Add urljoin compatibility 35c317b Fix capitalization, it's OpenStack 4d12ad1 python3: Import translation layer
sslutils: 8b2b0b7 Use hacking import_exceptions for gettextutils._ 12bcdb7 Remove vim header 1a2df89 Enable H302 hacking check 99b7c35 Convert kombu SSL version string into integer b0b37ce Fix IBM copyright strings f63ea05 Use oslo-config-2013.1b3 e50b68c Support for SSL in wsgi.Service
versionutils: 8b2b0b7 Use hacking import_exceptions for gettextutils._ a5ae087 fixed typos 45658e2 Fix violations of H302:import only modules 37ea814 Adds decorator to deprecate functions and methods 12bcdb7 Remove vim header d7d74a7 Add `versionutils` for version compatibility checks
Oslo version: 7a51572 Merge "Implement cache abstraction layer"
Date: Wed Jan 15 19:31:16 2014 +0000
Eric Harney [Thu, 16 Jan 2014 18:09:05 +0000 (13:09 -0500)]
Sync gettextutils from Oslo
Update gettextutils as a dependency for RPC updates
Requires changes within Cinder as well for
"get_localized_message" -> "translate" method rename.
221c37d Allow the Message class to have non-English default locales 0e1af5b Implementation of translation log handler f5686b1 Merge "Translation Message improvements" 8b2b0b7 Use hacking import_exceptions for gettextutils._ 2cfc1a7 Translation Message improvements 45658e2 Fix violations of H302:import only modules 12bcdb7 Remove vim header
Oslo version: 7a51572 Merge "Implement cache abstraction layer"
Date: Wed Jan 15 19:31:16 2014 +0000
Dirk Mueller [Thu, 16 Jan 2014 21:59:41 +0000 (22:59 +0100)]
Remove env from rootwrap filter
Allowing 'env' as a CommandFilter is similar to
allowing '/bin/bash', which makes all of rootwrap pointless.
Use EnvFilter instead. Change corresponding commands
that use env for setting C locale and adjust rootwrap
filters accordingly.
Several commands that output information that is
parsed as input by cinder change their behavior
based on the environment locale, which is depending
on local system settings. The code is however only
able to parse in C locale, so enforce that one.
Ollie Leahy [Thu, 12 Dec 2013 11:01:34 +0000 (11:01 +0000)]
Allow user to specify audit period.
The script 'cinder-volume-usage-audit' is generally run by cron to
generate volume and snapshot exists audit notifications that can be
consumed by ceilometer or other auditing systems. In this mode it sends
notifications for volumes that existed in the 'last completed audit
period' as calculated by cinder.utils.last_completed_audit_period()
However, it may sometimes be necessary to be able to send notifications
for audit periods other than than the last completed audit period. For
example if the host running the audit script was down for a period.
This patch provides three optional command line parameters to the script
cinder-volume-usage-audit. If no parameters are provided then the
operation of the script is unchanged. If parameters are provided then
they have the following effects:
--start_time specifies a date and time that is used in place of the
start of the last completed audit period.
--end_time specifies a date and time that is used in place of the end of
the last completed audit period.
--send_actions As well as sending exits notifications the script will
send any create or delete notifications that would have been sent during
the specified audit period.
Joel Friedly [Fri, 17 Jan 2014 01:19:26 +0000 (17:19 -0800)]
Use a mirrored mirror log
By default, LVM will sync the two mirrors of a logical volume on
creation. This is I/O intensive and it takes a long time though, so we
tell it not to with --nosync. The next default behavior is to create a
mirror log which says what portions of the mirrors are in sync. But
then if a disk backing a mirror log fails, we have to deal with
resyncing the mirrors at each reboot.
This commit tells LVM to use a mirrored mirror log, which will use
approximately 4MB of extra space, but means we'll never have to
sync mirrors, ever.
Catch OSError(s) from os.getlogin() and fall back to looking at
environment variables to figure out the user's name. The OSError
is thrown where there is no tty for the python process when
os.getlogin() is called.
Swapnil Kulkarni [Thu, 16 Jan 2014 07:58:31 +0000 (13:28 +0530)]
Updates for version list to show correct references
Updated views to consider version while creating href.
Updated fakes to consider version while creating fake href.
Updated some test cases in v2 api which were using v1 as
expected output
Zhiteng Huang [Thu, 2 Jan 2014 06:16:18 +0000 (14:16 +0800)]
Pull Bug #1263122 fix for service module from Oslo
Get latest service module from Oslo mainly to avoid the impact of
bug #1263122 when multi-process API service is enabled.
Follow commits are included in this change:
* 8b2b0b7 2013-12-20 | Use hacking import_exceptions for gettextutils._
* 37e46bb 2013-12-20 | disable SIGHUP restart behavior in foreground
* 12bcdb7 2013-10-11 | Remove vim header
Zhiteng Huang [Thu, 2 Jan 2014 01:31:24 +0000 (09:31 +0800)]
Pull latest scheduler change from Oslo
The RetryFilter is now part of Oslo, although it's been renamed to
IgnoreAttemptedFilter. Thanks to entry_point, we are able to maintain
backwards compatibility after pulling that commit from Oslo.
Commits in this change:
* 66fe978 2013-12-10 | Change IgnoreAttemptedHostFilter to expect 'retry' key (attempt-retry)
* 135dd00 2013-12-10 | Remove start index 0 in range()
* 45658e2 2013-12-09 | Fix violations of H302:import only modules
* 70004c6 2013-12-04 | Add IgnoreAttemptedHostsFilter to oslo
* 880acf7 2013-11-14 | Change capabilities filters to use resource type (capfilter_message)
* 06e9d98 2013-11-10 | Add some log messages to capabilities_filter.py
* 3970d46 2013-11-02 | Fix typos in oslo
* 8718763 2013-08-19 | Replace list with dict in AvailabilityZoneFilter.host_passes
* c0d052a 2013-07-12 | python3: Add basic compatibility support.
* e3545f8 2013-06-02 | Enable hacking H402 test
* 484a1df 2013-05-30 | Enable hacking H403 test
* 35660da 2013-05-30 | Enable hacking H401 test
* 5dcc43b 2013-05-05 | Break out common functionality for filters and weights
* 1f2aba5 2013-05-03 | Renames filter to base_filter and weight to base_weight
Eric Harney [Fri, 10 Jan 2014 19:20:41 +0000 (14:20 -0500)]
GlusterFS: Synchronize additional operations
create_cloned_volume() and copy_volume_to_image() should also be
locked against other driver operations to prevent collisions
from concurrent updates of snapshot metadata or volume file data.
Flavio Percoco [Tue, 10 Dec 2013 11:31:50 +0000 (12:31 +0100)]
Move driver initialization check into the method
Volumes and backups managers' methods are decorated with
`require_initialized_driver` which checks whether the driver has been
initialized or not. The decorator fails with a `DriverNotInitialized`
exception if the driver hasn't been initialized.
This early failure leaves volumes and backups in a wrong status which is
not just confusing for the user but it also makes it difficult to do
anything with the resources after they've been left in a 'bogus' status.
For example, when a volume creation is requested, the volume is first
created in the database and its status is set to 'creating'. Then the
scheduler will pick an available volume node and send the task to it. If
the driver has not been initialized, the volume status will be left as
'creating' instead of 'error'.
This patch fixes that issue by moving the driver initialization check
into the various manager's methods. In some cases this check is done at
the very beginning of the method, in some others - either to avoid code
duplication or because the lines above the check made sense to be
executed first - this check is done later in the method.
Avishay Traeger [Sun, 12 Jan 2014 13:43:34 +0000 (15:43 +0200)]
Print driver exception on retype
If the driver raises an exception, make sure it is printed. Also change
the logging level from info to error because an exception should not be
thrown under normal conditions.
Zhiteng Huang [Thu, 2 Jan 2014 07:44:30 +0000 (15:44 +0800)]
Drop Chance/SimpleScheduler Implementation
This patch removes the implementation of ChanceScheduler and SimpleScheduler
as previous changes have made sure they are internally replaced by
FilterScheduler.
The "max_gigabytes" config option is deprecated and will leave it like that
for one more release before we can remove it.
DocImpact: "ChanceScheduler and SimpleScheduler have been deprecated and
their implementation have been removed from Cinder."
Ollie Leahy [Thu, 19 Dec 2013 16:11:29 +0000 (16:11 +0000)]
Fix sqlalchemy bug in transfer_get_all_by_project
Because of incorrect use of the sqlalchemy methods joinedload and filter,
incorrect lists of volume transfers were being returned by
transfer_get_all_by_project().
Eric Harney [Mon, 9 Dec 2013 20:34:39 +0000 (15:34 -0500)]
NFS/GlusterFS: Skip incorrectly formatted shares
Shares should always be of the form address:/volume. If they are
not (i.e., are missing the '/'), then skip them and issue a warning
message. This will prevent us from sending incorrect connection_info
dicts to Nova.
Lynxzh [Mon, 6 Jan 2014 03:45:33 +0000 (11:45 +0800)]
Remove unused message from iogrp_data setup
The iogrp_data is initialized during do_setup loaded.
The vdisk_count is originally used but later removed.
In ValueException raised, the variable is no longer required.
Remove this will give a clearer message when
ValueException raised.
Zhiteng Huang [Wed, 8 Jan 2014 03:22:25 +0000 (11:22 +0800)]
Remove legacy config option 'connection_type'
'connection_type' is a legacy config option originated from Nova.
Nova used this option to specify underlying virtualization type for
compute service and had deprecated it (with 'compute_driver') in
Folsom and had it removed in later release. In Cinder Folsom release,
the XenSMDriver in Cinder used this config option to verify the target
hypervisor but in Grizzly XenSMDriver has been renamed and refactored
and no longer used this config. Therefore, this option is dead for
a long time, and should be removed.
john-griffith [Tue, 31 Dec 2013 16:04:05 +0000 (09:04 -0700)]
Modify default prefix for solidfire account
The SolidFire driver creates a unique account at volume
creation composed of <cinderhost-tenantID>.This is fine
until one tries to implement HA Cinder or moves their
Cinder node to another machine.
This patch changes the default setting to be an empty string
(ie no prefix), but still allows backward compatability by setting
the config option to 'hostname'.
DocImpact: Changes default for account-prefix, for backwards compatability
use the non-default 'hostname'
Ramy Asselin [Mon, 6 Jan 2014 23:26:07 +0000 (15:26 -0800)]
3PAR: Raise Ex when del snapshot with depend vol
Deleting a 3PAR snapshot can result in an HTTP Conflict error message,
such as when a volume is dependent on it.
Previously, this exception was not caught, and the state was set
to "Error_Deleting" with no recovery path.
The code was updated to catch this exception and raise a
SnapShotIsBusy exception with an appropriate error message.
The state reverts back to 'Available' allowing the user to try again
after removing the dependency.
Jay S. Bryant [Mon, 6 Jan 2014 20:45:34 +0000 (14:45 -0600)]
Add bool_type dictionary to test_migrations
A number of MySQL migration tests fail because the test case
is verifying that an sqlalchemy.types.BOOLEAN column was created.
When the column type is checked, however, mysql reports the type as
sqlalchemy.dialects.mysql.TINYINT. The mismatch causes numerous
test cases to fail for MySQL.
This creates a dictionary for each engine being tested to handle the
different boolean types.
postgresql and DB2 encounter similar issues. Subsequent patches
will make the appropriate changes for those databases.
Xing Yang [Mon, 6 Jan 2014 22:16:05 +0000 (17:16 -0500)]
Fixed a problem in iSCSI multipath
Multipathing during copy image to volume and copy volume to image
operations doesn't work properly if there are different targets
associated with different portals for a mulitpath device.
This patch adds the volume retype operation, which allows a user to
change a given volume's type.
When retyping, the scheduler checks if the volume's current host can
accept the new type (i.e., it checks if the host passes the filters when
using the new type). If the current host is suitable, its manager is
called which calls upon the driver to change the volume's type.
There are two cases where a retype operation may require migrating the
volume:
1. The volume's current host cannot accept the new type
2. The volume's driver cannot perform the retype operation
In case of a migration, a volume with the new type is created, and the
data is migrated to it.
Volume migrations resulting from retype can be controlled by passing a
policy, which can be either:
1. 'never': Never migrate (the retype fails if migration is required)
(default)
2. 'on-demand': Migrate when necessary
This version will cause retype operations to fail if the current and
new volume types have different:
1. QoS settings that are enforced by the front-end for in-use volumes.
2. encryption settings.
Jay S. Bryant [Mon, 6 Jan 2014 18:44:26 +0000 (12:44 -0600)]
Fix downgrade in 002_quota_class.py for MySQL
Downgrade in the script 002_quota_class.py is failing for MySQL
because there is an attempt to delete the quota_classes table
while the reservations table still has a foreign key defined.
The foreign key causes the delete of the quota_classes table to fail.
Kurt Martin [Mon, 6 Jan 2014 17:06:18 +0000 (09:06 -0800)]
Removed deprecated config option hp3par_domain
The hp3par_domain config option was deprecated in the Havana release.
This just removes it from the code, test and config sample file
for the Icehouse release.
Update V2 API to return detailed volume information on create
Current implementation returns only summary information, so
cinderclient requires additional GET call to get details.
Updated the api to return the details by default so the GET in
cinderclient can be removed.
ling-yun [Fri, 3 Jan 2014 08:51:50 +0000 (16:51 +0800)]
removed unused context in check_*** methods
Input parameter 'context' of Check_*** methods(check_attach,
check_detach, _check_metadata_properties, _check_volume_availability)
in cinder/volume/api.py are not being used, so remove this 'context'
parameter.
Code Review / openstack-build / cinder-build.git / log