Cory Stone [Wed, 26 Aug 2015 00:05:36 +0000 (19:05 -0500)]
Dont eager load volume type specs on volume list
Some drivers need extra specs loaded on create and the session could be
closed by the time they access them. However for a volume list, eager
loading these attributes causes a severe performance penalty.
Move the eager load to the single volume get case.
John Griffith [Wed, 19 Aug 2015 17:42:40 +0000 (17:42 +0000)]
Filter out extra-specs from type get for non-admin
Currently when a get_volume type call is made to the
db api, we don't check context and auto-fill the
extra-specs info for that type.
Extra specs are intended to be admin only info, but anybody
that calls the API directly to list or get volume-types is
also given this info which is not intended.
This patch just adds a context check to the db api's
private extra-specs builder method. In the case of
non-admin, we just skip adding the extra-specs.
Yuriy Nesenenko [Tue, 25 Aug 2015 10:55:56 +0000 (13:55 +0300)]
Parameter osapi_max_limit is always used by default
Default value max_limit is initialized before a value is read from
the config file. Default parameter values are always evaluated when
the “def” statement they belong to is executed. So parameter
osapi_max_limit is always used by default even it's set in the
configuration file.
Tomoki Sekiyama [Fri, 14 Aug 2015 18:13:10 +0000 (14:13 -0400)]
TemporaryImages to inspect image before conversion
This intruduces TemporaryImages utility class which enables
download images to temporary file for inspection before they are
used for format conversion. Using this utility, we can get image
information such as virtual size, which is required to implement
image cache, before calling fetch_to_raw() etc., without
downloading the image twice nor changing APIs of image_utils.
Tomoki Sekiyama [Fri, 8 May 2015 23:52:19 +0000 (19:52 -0400)]
Efficient image transfer for Glance cinder store
This adds an implementation of clone_image which offloads
the image copy when the image is stored in Glance Cinder store.
It uses "create_clone_volume" method to create a new volume
from an image that is placed on a cinder volume.
To enable this feature, glance_api_version in cinder.conf must be
set to 2, and allowed_direct_url_schemes must contain 'cinder'.
In glance-api.conf, show_multiple_locations must be set to True.
Also the Cinder store must be enabled.
In addition, if image_upload_use_cinder_backend is set to True,
upload-to-image in raw format will create a cloned volume and
register its location to the image service.
If image_upload_use_internal_tenant is set to True, the image
volume is stored in the internal tenant. Otherwise it is placed
in the current context's tenant.
This also changes LVM driver implementation to enable
image upload. Especially for thin LVM, it modifies
create_cloned_volume method to create thin snapshot LV.
Note that the thin snapshot LV can be treated as the normal
LVs; it can be read/written and can survive after the parent
volume is deleted.
Only raw format image is supported. Otherwise, normal image
upload/download method is used.
Currently Glance Cinder store does not support upload and
download, so the image created by this feature cannot be used
by other projects than Cinder. Patches to provide access to
images stored on Cinder are proposed here:
glance-specs: https://review.openstack.org/183363
glance patch(adding rootwrap): https://review.openstack.org/186201
glance_store patch: https://review.openstack.org/166414
Note that this change works even without Glance patches.
Gorka Eguileor [Tue, 18 Aug 2015 09:29:15 +0000 (11:29 +0200)]
Fix backup list all_tenants=0 filtering for admin
When a user with admin role lists backups specifying all_tenants=0 in
the request, it will not behave as expected, it will return backups for
all tenants just as if all_tenants=1 had been specified.
This patch fixes this issue and will only return all tenants in backup
listings if all_tenants is set to a value meaning True (1, true, yes,
y).
Anton Arefiev [Fri, 10 Jul 2015 13:44:59 +0000 (16:44 +0300)]
Add Cinder API wsgi application
This change adds cinder API application, so cinder API can be run
under wsgi server(Apache, Nginx, etc). Eventlet will still be used
by default.
Cinder API with eventlet as a webserver and wsgi application
managing has some cons:
* Cinder API is deployed in other way as a common web application.
Apache/Nginx is generally used web servers for REST API application.
* Cinder API is run as a separate service. It means that cloud
operators need to configure some software to monitor that the API
is running.
* Apache/Nginx works better under the real heavy load than eventlet.
To use c-api deployment under Apache in devstack assign
devstack var CINDER_USE_MOD_WSGI="True".
Related changes in devstack:
https://review.openstack.org/#/c/204643/
Cinder documentation on how deploy cinder API under Apache:
https://review.openstack.org/#/c/207020/
Patch for Infra to make it tested:
https://review.openstack.org/#/c/208498/
John Griffith [Thu, 9 Jul 2015 21:11:54 +0000 (21:11 +0000)]
Cinder replication V2
This adds a scaled back replication implementation
that leaves the bulk of the work up to the driver.
We just provide basic admin API methods to do things
like enable/disable and fail-over. Set up and
specification of replication targets for a specific
back end are now intended to be part of the cinder.conf
in the driver section itself.
Replication targets are configured via the cinder.conf
file in their associated driver section. See the devref
doc included in this commit for details on the format.
The next step in configuration is to create a volume-type
with replication info in the extra-specs.
extra-specs = replication=enable, volume_backend_name=foo
This instructs the driver to utilize replication, default is
up to the driver but would suggest single way rep and in the case
of multiple targets, driver could choose or have a default.
If the back end doesn't report replication=enabled in it's stats
updates the scheduler will fail to place the volume due to invalid
host, or no hosts available.
Vendors can easily modify extra-specs or their own config settings
to modify this behavior, any vendor-unique adaptation can be
provided through the use of scoped keys. Suggested examples
will be published in docs.
See doc/source/devref/replication.rst for more info
kedar-vidvans [Fri, 7 Aug 2015 21:23:46 +0000 (17:23 -0400)]
Assisted volume migration for Oracle ZFSSA drivers
The Oracle ZFSSA drivers will migrate the volume directly from the
source host to the destination host if both hosts are configured
with Oracle ZFSSA. This type of volume migration will bypass the
cinder host and will directly transfer the volume to the destination
backend.
The ZFSSA iSCSI driver will use zfssa's remote replication service
to move the the volume to the destination zfssa.
nikeshm [Fri, 21 Aug 2015 20:39:51 +0000 (02:09 +0530)]
Add https options and minor code changes
Adds https options x_verify_certificate and x_verify_certificate_path
in DotHill and rebranded drivers.
Minor cosmetic changes like replacing 'realstor' with 'virtual'
and 'wbi' with 'api'. Adding default value for 'x_backend_type'
and 'x_api_protocol' options. Minor code optimization for
finding array iSCSI addresses.
Here 'x' stands for dothill or hpmsa or lenovo.
DocImpact
Change-Id: I7a5924bbd56053aee6d652842b48e913c54ae7dd
Closes-Bug: #1486160 Co-Authored-By: Lakshman <lakshminarayanat@vedams.com> Co-Authored-By: Chris Maio <chris.maio@dothill.com>
Tom Swanson [Fri, 21 Aug 2015 22:00:23 +0000 (17:00 -0500)]
Dell SC: Added logging for the find_wwns functions
Added additional LOG.debug() statements to find_wwns and associated
functions. Right now there aren't enough bread crumbs to easily triage
a problem with the return for the FC intitialize_connection function.
Kendall Nelson [Fri, 21 Aug 2015 18:15:25 +0000 (13:15 -0500)]
Cleaning up CONF.register_opts() in compute/__init__.py
This patch makes the import of oslo_config and registration
of _compute_opts consistent with all other imports and
registration statements in other files. In a future patch
that dynamically generates Cinder config options, it will now
be possible to get the options being registered in this file.
John Griffith [Thu, 20 Aug 2015 19:09:57 +0000 (19:09 +0000)]
Update provider_id column on SolidFire init
This patch just implements an init routine to update
the cinder db with provider_id mappings to known/active
volumes on the SolidFire cluster.
Now that we have added a mapping feature in the DB, we need
to go back and populate these columns on an upgrade. It's
also helpful to have this mechanism for internal mappings on
init, that's why it's here as an init routine instead of part
of an upgrade migration or cinder-manager script.
John Griffith [Thu, 20 Aug 2015 16:45:03 +0000 (10:45 -0600)]
Add ability to update provider_id during init
During the Liberty release we added the ability to store
a provider_id for a volume in the Cinder database. This
simplifies a number of things for back ends when accessing
volumes.
One missing piece here however is that we have no way to
populate this data for existing deployments if/when they
upgrade.
This patch provides a simple solution that issues an update
call to the drivers on host init. It works by fetching a
list of volumes for the specific host, and passes those in
to the driver. The driver can do nothing, or it can go
through and sync up the Cinder ID's with it's internal
ID's and return the provider update info.
Cinder currently doesn't check the existing resource
when user lowers the quota sets. It should have an optional
action that doesn't allow to update the quota of resources
lower than current usage.
This patch adds code to validate the update value of quota and
ensure the value can't be lower than existing resource.
APIImpact
Add an optional boolean argument 'skip_validation' with default value True.
{
"quota_set": {xxxx},
"skip_validation": 'True'/'False'
}
If skip_validation=False and the update value is lower than existing resource,
API will return 400 Bad Request.
John Griffith [Thu, 20 Aug 2015 21:49:59 +0000 (15:49 -0600)]
Add SolidFire svip to config options
The SolidFire driver uses a simple method of gathering
the Storage Virtual IP, it just queries the Cluster and
asks for the SVIP. This is great almost all the time,
BUT it doesn't take into consideration that the deployer
may be using VLAN's and accessing the cluster from multiple
clouds, or multiple networks.
This patch just adds a config option to specify an SVIP
if the admin has configured VLANs in their environment and
on their cluster. The default behavior is to still use the
global SVIP.
Tina [Wed, 5 Aug 2015 03:15:03 +0000 (23:15 -0400)]
Return multiple iSCSI portals in VNX Cinder driver
Return multiple iSCSI portals (using keys target_portals, target_iqns
and target_luns) in VNX Cinder driver whether the multipath option
is used or not. A single portal is continuously being returned in
target_portal, target_iqn and target_lun for drivers other than
libvirt driver.
fileutils is graduated in the oslo.utils library.
Use the open function from builtins instead of fileutils.file_open.
Change the test_backup_volume (test_scality module) and
test_restore_backup (test_scality, test_volume) functions to use
mock library instead of mox.
John Griffith [Wed, 19 Aug 2015 21:07:49 +0000 (21:07 +0000)]
Parse out SolidFire account from api response
In the SolidFire create-cache account routine we broke the response
check and are trying to use the entire api response as if it were an
account object.
This patch just pulls the actual account object out of the response
instead of using the entire API response.
Tom Swanson [Tue, 18 Aug 2015 19:36:38 +0000 (14:36 -0500)]
Dell SC: Better exception handling in init_conn
Now re-raising VolumeBackendAPIExceptions in initialize_connection.
And logging any other exceptions as VolumeBackendAPIException. This
preserves the actual failure in the logs.
Adriano Rosso [Mon, 10 Aug 2015 19:19:31 +0000 (16:19 -0300)]
Adds the random option to cinder retry function
The current retry functionality in Cinder, always uses fixed
numbers to define the time to wait before performing the next
attempts. This behaviour can make the retries of multiple
requisitions to collide in each attempt as they will wait the
same amount of time and try to use the same resource together
again.
This patch changes the behaviour of cinder.utils.retry() to
receive a parameter that allows the function to implement
randomly generated wait intervals.
Kurt Martin [Tue, 18 Aug 2015 23:15:28 +0000 (16:15 -0700)]
LeftHand driver is ignoring reserved_percentage
The LH driver was hard coding the reserved_percentage parameter to 0.
The LeftHand driver should be reading this value from the cinder
configuration file as it's used in the capacity filter for calculating
capacities.
John Griffith [Tue, 18 Aug 2015 23:08:13 +0000 (17:08 -0600)]
Update devref unit tests doc
Simple update for our painfully out of date devref docs.
Just a short refresh on unit_tests, there's a lot that could
and probably should be added there, but this change just gets
rid of the obsolete references to Nose and adds info regarding
tox and how to use it.
Eric Harney [Tue, 18 Aug 2015 22:10:58 +0000 (18:10 -0400)]
Tests: Fix zfssa TestRestClientURL.test_request
This test can currently fail with mismatched parameters
to urlopen. Just ensure that urlopen is called so
that this works across all environments/versions.
Eric Harney [Tue, 18 Aug 2015 21:09:50 +0000 (17:09 -0400)]
Test whether sample config generation works
Adds --checkonly to tools/config/check_uptodate.sh.
This allows verifying that a configuration file was
generated.
We have had a number of issues introduced which
have caused config file generation to fail since we
removed the sample config file check in the gate.
This does not add back checks to ensure that the file
has been updated, it only checks to ensure that it is
still possible to update the sample file.
Adding optional dependency tracking would be nice,
but Cinder is not yet in a place to do this.
- Cinder supports tox 1.6 (minversion in tox.ini)
- This functionality does not work on 1.6, requires 1.7. Tox
fails to build a test environment if using tox 1.6.
- We have decided we can't move to tox 1.7 for now.
( https://review.openstack.org/#/c/211614/ )
Jay S Bryant [Tue, 18 Aug 2015 19:24:52 +0000 (14:24 -0500)]
Register the volume_opts config options in remotefs.py
With commit bc23e0bf415defb1956c9864ac33390bb4cc2017 a new
volume_opts configuration option list was added into
cinder/volume/drivers/remotefs.py. This new list, however,
was never registered. When an options list is not registered
genconfig cannot figure out the right place to put the config
options and will fail to generate the sample file.
To resolve this, the patch simply registers the new options
list in remotefs.py. Once the list is registered we are,
once again, able to generate the sample file.
Sean McGinnis [Tue, 18 Aug 2015 13:08:18 +0000 (08:08 -0500)]
Dell SC: Fix error causing missed log message
Extremely trivial fix. An extra comma at the end of a
logging line actually made it into a statement that
doesn't perform logging. This patch removes the extra
comma so that the message gets logged as expected.
Capacity filter uses free_virtual capacity to filter out a host if
thin provisioning is enabled and over-subscription ratio is less
than or equal to max_oversubscription_ratio. The current formula
for free_virtual is not calculating the real free virtual capacity.
Rather it computes "adjusted" free virtual capacity which is the
current free capacity (after taking into account of reserved space)
we can over-subscribe. This patch adds necessary documentation and
renames free_virtual to adjusted_free_virtual to clarify this.
Code Review / openstack-build / cinder-build.git / log