Dirk Mueller [Thu, 16 Jan 2014 21:59:41 +0000 (22:59 +0100)]
Remove env from rootwrap filter
Allowing 'env' as a CommandFilter is similar to
allowing '/bin/bash', which makes all of rootwrap pointless.
Use EnvFilter instead. Change corresponding commands
that use env for setting C locale and adjust rootwrap
filters accordingly.
Several commands that output information that is
parsed as input by cinder change their behavior
based on the environment locale, which is depending
on local system settings. The code is however only
able to parse in C locale, so enforce that one.
Catch OSError(s) from os.getlogin() and fall back to looking at
environment variables to figure out the user's name. The OSError
is thrown where there is no tty for the python process when
os.getlogin() is called.
Swapnil Kulkarni [Thu, 16 Jan 2014 07:58:31 +0000 (13:28 +0530)]
Updates for version list to show correct references
Updated views to consider version while creating href.
Updated fakes to consider version while creating fake href.
Updated some test cases in v2 api which were using v1 as
expected output
Zhiteng Huang [Thu, 2 Jan 2014 01:31:24 +0000 (09:31 +0800)]
Pull latest scheduler change from Oslo
The RetryFilter is now part of Oslo, although it's been renamed to
IgnoreAttemptedFilter. Thanks to entry_point, we are able to maintain
backwards compatibility after pulling that commit from Oslo.
Commits in this change:
* 66fe978 2013-12-10 | Change IgnoreAttemptedHostFilter to expect 'retry' key (attempt-retry)
* 135dd00 2013-12-10 | Remove start index 0 in range()
* 45658e2 2013-12-09 | Fix violations of H302:import only modules
* 70004c6 2013-12-04 | Add IgnoreAttemptedHostsFilter to oslo
* 880acf7 2013-11-14 | Change capabilities filters to use resource type (capfilter_message)
* 06e9d98 2013-11-10 | Add some log messages to capabilities_filter.py
* 3970d46 2013-11-02 | Fix typos in oslo
* 8718763 2013-08-19 | Replace list with dict in AvailabilityZoneFilter.host_passes
* c0d052a 2013-07-12 | python3: Add basic compatibility support.
* e3545f8 2013-06-02 | Enable hacking H402 test
* 484a1df 2013-05-30 | Enable hacking H403 test
* 35660da 2013-05-30 | Enable hacking H401 test
* 5dcc43b 2013-05-05 | Break out common functionality for filters and weights
* 1f2aba5 2013-05-03 | Renames filter to base_filter and weight to base_weight
Eric Harney [Fri, 10 Jan 2014 19:20:41 +0000 (14:20 -0500)]
GlusterFS: Synchronize additional operations
create_cloned_volume() and copy_volume_to_image() should also be
locked against other driver operations to prevent collisions
from concurrent updates of snapshot metadata or volume file data.
Flavio Percoco [Tue, 10 Dec 2013 11:31:50 +0000 (12:31 +0100)]
Move driver initialization check into the method
Volumes and backups managers' methods are decorated with
`require_initialized_driver` which checks whether the driver has been
initialized or not. The decorator fails with a `DriverNotInitialized`
exception if the driver hasn't been initialized.
This early failure leaves volumes and backups in a wrong status which is
not just confusing for the user but it also makes it difficult to do
anything with the resources after they've been left in a 'bogus' status.
For example, when a volume creation is requested, the volume is first
created in the database and its status is set to 'creating'. Then the
scheduler will pick an available volume node and send the task to it. If
the driver has not been initialized, the volume status will be left as
'creating' instead of 'error'.
This patch fixes that issue by moving the driver initialization check
into the various manager's methods. In some cases this check is done at
the very beginning of the method, in some others - either to avoid code
duplication or because the lines above the check made sense to be
executed first - this check is done later in the method.
Avishay Traeger [Sun, 12 Jan 2014 13:43:34 +0000 (15:43 +0200)]
Print driver exception on retype
If the driver raises an exception, make sure it is printed. Also change
the logging level from info to error because an exception should not be
thrown under normal conditions.
Zhiteng Huang [Thu, 2 Jan 2014 07:44:30 +0000 (15:44 +0800)]
Drop Chance/SimpleScheduler Implementation
This patch removes the implementation of ChanceScheduler and SimpleScheduler
as previous changes have made sure they are internally replaced by
FilterScheduler.
The "max_gigabytes" config option is deprecated and will leave it like that
for one more release before we can remove it.
DocImpact: "ChanceScheduler and SimpleScheduler have been deprecated and
their implementation have been removed from Cinder."
Ollie Leahy [Thu, 19 Dec 2013 16:11:29 +0000 (16:11 +0000)]
Fix sqlalchemy bug in transfer_get_all_by_project
Because of incorrect use of the sqlalchemy methods joinedload and filter,
incorrect lists of volume transfers were being returned by
transfer_get_all_by_project().
Eric Harney [Mon, 9 Dec 2013 20:34:39 +0000 (15:34 -0500)]
NFS/GlusterFS: Skip incorrectly formatted shares
Shares should always be of the form address:/volume. If they are
not (i.e., are missing the '/'), then skip them and issue a warning
message. This will prevent us from sending incorrect connection_info
dicts to Nova.
Lynxzh [Mon, 6 Jan 2014 03:45:33 +0000 (11:45 +0800)]
Remove unused message from iogrp_data setup
The iogrp_data is initialized during do_setup loaded.
The vdisk_count is originally used but later removed.
In ValueException raised, the variable is no longer required.
Remove this will give a clearer message when
ValueException raised.
Zhiteng Huang [Wed, 8 Jan 2014 03:22:25 +0000 (11:22 +0800)]
Remove legacy config option 'connection_type'
'connection_type' is a legacy config option originated from Nova.
Nova used this option to specify underlying virtualization type for
compute service and had deprecated it (with 'compute_driver') in
Folsom and had it removed in later release. In Cinder Folsom release,
the XenSMDriver in Cinder used this config option to verify the target
hypervisor but in Grizzly XenSMDriver has been renamed and refactored
and no longer used this config. Therefore, this option is dead for
a long time, and should be removed.
john-griffith [Tue, 31 Dec 2013 16:04:05 +0000 (09:04 -0700)]
Modify default prefix for solidfire account
The SolidFire driver creates a unique account at volume
creation composed of <cinderhost-tenantID>.This is fine
until one tries to implement HA Cinder or moves their
Cinder node to another machine.
This patch changes the default setting to be an empty string
(ie no prefix), but still allows backward compatability by setting
the config option to 'hostname'.
DocImpact: Changes default for account-prefix, for backwards compatability
use the non-default 'hostname'
Ramy Asselin [Mon, 6 Jan 2014 23:26:07 +0000 (15:26 -0800)]
3PAR: Raise Ex when del snapshot with depend vol
Deleting a 3PAR snapshot can result in an HTTP Conflict error message,
such as when a volume is dependent on it.
Previously, this exception was not caught, and the state was set
to "Error_Deleting" with no recovery path.
The code was updated to catch this exception and raise a
SnapShotIsBusy exception with an appropriate error message.
The state reverts back to 'Available' allowing the user to try again
after removing the dependency.
Jay S. Bryant [Mon, 6 Jan 2014 20:45:34 +0000 (14:45 -0600)]
Add bool_type dictionary to test_migrations
A number of MySQL migration tests fail because the test case
is verifying that an sqlalchemy.types.BOOLEAN column was created.
When the column type is checked, however, mysql reports the type as
sqlalchemy.dialects.mysql.TINYINT. The mismatch causes numerous
test cases to fail for MySQL.
This creates a dictionary for each engine being tested to handle the
different boolean types.
postgresql and DB2 encounter similar issues. Subsequent patches
will make the appropriate changes for those databases.
Xing Yang [Mon, 6 Jan 2014 22:16:05 +0000 (17:16 -0500)]
Fixed a problem in iSCSI multipath
Multipathing during copy image to volume and copy volume to image
operations doesn't work properly if there are different targets
associated with different portals for a mulitpath device.
This patch adds the volume retype operation, which allows a user to
change a given volume's type.
When retyping, the scheduler checks if the volume's current host can
accept the new type (i.e., it checks if the host passes the filters when
using the new type). If the current host is suitable, its manager is
called which calls upon the driver to change the volume's type.
There are two cases where a retype operation may require migrating the
volume:
1. The volume's current host cannot accept the new type
2. The volume's driver cannot perform the retype operation
In case of a migration, a volume with the new type is created, and the
data is migrated to it.
Volume migrations resulting from retype can be controlled by passing a
policy, which can be either:
1. 'never': Never migrate (the retype fails if migration is required)
(default)
2. 'on-demand': Migrate when necessary
This version will cause retype operations to fail if the current and
new volume types have different:
1. QoS settings that are enforced by the front-end for in-use volumes.
2. encryption settings.
Jay S. Bryant [Mon, 6 Jan 2014 18:44:26 +0000 (12:44 -0600)]
Fix downgrade in 002_quota_class.py for MySQL
Downgrade in the script 002_quota_class.py is failing for MySQL
because there is an attempt to delete the quota_classes table
while the reservations table still has a foreign key defined.
The foreign key causes the delete of the quota_classes table to fail.
Kurt Martin [Mon, 6 Jan 2014 17:06:18 +0000 (09:06 -0800)]
Removed deprecated config option hp3par_domain
The hp3par_domain config option was deprecated in the Havana release.
This just removes it from the code, test and config sample file
for the Icehouse release.
Update V2 API to return detailed volume information on create
Current implementation returns only summary information, so
cinderclient requires additional GET call to get details.
Updated the api to return the details by default so the GET in
cinderclient can be removed.
ling-yun [Fri, 3 Jan 2014 08:51:50 +0000 (16:51 +0800)]
removed unused context in check_*** methods
Input parameter 'context' of Check_*** methods(check_attach,
check_detach, _check_metadata_properties, _check_volume_availability)
in cinder/volume/api.py are not being used, so remove this 'context'
parameter.
ling-yun [Fri, 3 Jan 2014 07:56:05 +0000 (15:56 +0800)]
add 'force' verification in _volume_upload_image
_volume_upload_image API should only support to upload 'in-use' volume to
image when assign 'force' parameter as true value. Now
_volume_upload_image API does not do verification for 'force' parameter,
so we can upload 'in-use' volume to image when assign 'force' parameter
with string value 'false' or 'false111'. This patch add verification for
'force' parameter.
Shao Kai Li [Thu, 2 Jan 2014 05:51:41 +0000 (00:51 -0500)]
Raise max header size to accommodate large tokens.
The max header is exceeded in the following scenario
- Auth tokens built with a keystone v3 API catalog
- A catalog with approximately 8 or more endpoints defined
An equivalent fix will be required for all projects which use
eventlet.wsgi.
Copied from https://review.openstack.org/#/c/33362/
Jon Bernard [Thu, 2 Jan 2014 21:28:59 +0000 (16:28 -0500)]
LVM: update iscsi target on volume attach
This patch updates the existing iSCSI target for a LVM volume during
attach. This is necessary in the event that a user extended the volume
after creation so that the correct size is visible to the host.
This adds support only for tgtadm. Other changes may be needed for
ietadm/lioadm to provide the same functionality.
Eric Harney [Fri, 13 Dec 2013 15:39:09 +0000 (10:39 -0500)]
LVM: Activate Thin Pool LV upon initialization
If the LVM thin pool is not active, space calculation fails,
as data_percent is not known. Activate the pool upon initialization,
since we intend to use it anyway.
Slightly refactors _get_thin_pool_free_space so that it is clear
which missing/unexpected value is breaking things if a similar error
occurs for some other reason.
Eric Harney [Thu, 19 Dec 2013 00:11:44 +0000 (19:11 -0500)]
GlusterFS: Use correct base argument when deleting attached snaps
When deleting the most recent snapshot, the 'file_to_merge' field
which translates into the base= field for libvirt's blockRebase
call in Nova must be set depending on whether other snapshots exist.
If there are no other snapshots, base = None, which results in
libvirt clearing the qcow2 backing file pointer for the active
disk image.
If there are other snapshots, pass the parent of the file being
deleted as the new base file. The snapshot info pointer for the
prior base file must also be updated in this case.
Matt Fischer [Thu, 2 Jan 2014 02:31:20 +0000 (19:31 -0700)]
Removed copyright from empty files
According to policy change in HACKING:
http://docs.openstack.org/developer/hacking/#openstack-licensing
empty files should no longer contain copyright notices.
Zhiteng Huang [Thu, 2 Jan 2014 01:17:32 +0000 (09:17 +0800)]
Remove unused fake_flags
fake_flags.py should have been removed in change
I3424d0a401b3ef7a3254d3e913263554361a52ff but somehow it's left
untouched in that change. This module has been replaced by
cinder/tests/conf_fixture.py, and shouldn't be around anymore.
Zhiteng Huang [Tue, 31 Dec 2013 07:00:52 +0000 (15:00 +0800)]
Replace Simple/Chance Scheduler with FilterScheduler
This patch replaces SimpleScheduler and ChanceScheduler with
FilterScheduler. Scheduler manager is changed so that when it
detects 'scheduler_driver' is set to SimpleScheduler or
ChanceScheduler, it changes underlying scheduler driver to
FilterScheduler, and then host manager will use a pre-defined
combination of filters & weighers to keep scheduler behavior *MOSTLY*
the same as its old counterparts.
To simulate ChanceScheduler/SimpleScheduler, 'AvailabilityZoneFilter',
'CapacityFilter' and 'CapabilitiesFilter' are chosen to filter
out hosts that isn't in the target AZ, doesn't have sufficient
capacity and doesn't support needed capabilities. And newly
added 'ChanceWeigher' will randomly pick a host from hosts
that passes above filters - just like ChanceScheduler does;
for SimpleScheduler, 'AllocatedCapacityWeigher' will sort hosts
with their 'allocated_capacity' - the one allocates the least
is the winner by default.
So the two new weigher, 'ChanceWeigher' and 'AllocatedCapacityWeigher'
are the key FilterScheduler to act identical when choosing
back-ends to serve requests. The 3 filters on the other hand,
are essential to make sure this 'new' Simple/Chance Scheduler can
support volume types, encryption and QoS.
Zhiteng Huang [Wed, 11 Dec 2013 13:46:38 +0000 (21:46 +0800)]
Add AllocatedCapacityWeigher
AllocatedCapacityWeigher is a weigher that weigh hosts by their
allocated capacity. The main purpose of this weigher is to simulate
the SimpleScheduler's behavior, which sorts hosts by the size of
all volumes on them. So by allocated capacity, it equals to the
sum of size of all volumes on target host.
In order to keep track of 'allocated' capacity, host state is updated
to add a 'allocated_capacity_gb' attribute to record the value, which
means each back-end must report one extra stats to scheduler.
Fortunately, the 'allocated' capacity we are interested in here is
pure Cinder level capacity, the volume manager can take all the burden
to calculate this value without having to query back-ends. The volume
manager does the initial calculation in init_host() by the time when it
has to query all existing volumes from DB for ensure_export(). After
initial calculation, volume manager/scheduler will keep track of every
new request that changes 'allocated_capacity' and make sure this value
is up to date.
!DriverImpact! Cinder driver developers, please read on:
This patch contains a change that might IMPACT volume drivers: volume
manager now uses 'stats' attribute to save 'allocated_capacity_gb'.
And this information will be merged with those stats drivers provide
as a whole for scheduler to consume. If you plan to report any form
of allocated space other than the apparent Cinder level value, (e.g.
actual capacity allocated), Please choose a key name other than
'allocated_capacity_gb', otherwise it will *OVERWRITE* the value volume
manager has calculated and confuse scheduler.
ling-yun [Tue, 24 Dec 2013 03:45:16 +0000 (11:45 +0800)]
Handle terminate_connection() exception in volume manager
Due to the fact that we sometimes need to manually terminate a volume's
connection through volume api, it's possile that these backend drivers
throw exceptions while doing that. Currently exceptions are bubbled up to
volume API not being handled. This patch logs exception in volume manager
and then raises VolumeBackendAPIException to caller.
Code Review / openstack-build / cinder-build.git / log