Clinton Knight [Fri, 26 Sep 2014 16:07:44 +0000 (12:07 -0400)]
Deprecate / obsolete NetApp volume extra specs
The NetApp Data ONTAP (Cluster-mode) NFS & iSCSI drivers for Juno support
the Cinder pools feature, but the drivers are reporting two qualified
extra specs that must be converted to unqualified extra specs in order to
be used by the Cinder scheduler's capability filter. Furthermore, there
are four extra specs that must be deprecated due to having the pools
feature. Warnings will be logged during volume creation if any of the
obsolete or deprecated extra specs are seen in the volume type.
Ian Govett [Tue, 16 Sep 2014 11:32:49 +0000 (07:32 -0400)]
Fix a problem with 'volume list' when 'all_tenants=0'
This change checks the value of 'all_tenants' and returns a list of volumes
for all tenants when 'all_tenants' is 1 (or True). A tenant-specific volume
list is returned when 'all_tenants' is 0 (or False).
An InvalidInput exception is thrown if the 'all_tenants' value is not 0, 1,
True, or False (case insensitive).
Provided new unit tests for the get_all api which check the all_tenants, and
limits parameters.
Fixed pep8 compliance test fails with code being improperly indented, and
changed calls using str() to use six.text_type()
Nilesh Bhosale [Sun, 24 Aug 2014 07:36:29 +0000 (13:06 +0530)]
IBMNAS: Remove call to set r/w permissions to all
During cinder volume create operation from a volume snapshot or
from an existing volume (volume clone operation), the ibmnas
driver sets 'rw' permissions to all, which is unnecessary and
also poses security concerns.
Fixing this issue, removing the calls to set rw permissions to
all during these operations and adding a call to set 'rw'
permissions only to the owner to make sure even if umask is set
at the filesystem level, which might deny 'rw' access to the owner
we explicitely set the required permissions on the volume file.
Xing Yang [Sun, 14 Sep 2014 23:50:01 +0000 (19:50 -0400)]
Getting iscsi_ip_address from cinder.conf
In the current version (2.0) of the VMAX driver, we retrieve iSCSI IP
addresses dynamically from SMI-S. However, we ran into situations where
we can't get them reliably during testing. The fix is to get this
information from cinder.conf, just like in version 1.0.
Xing Yang [Sat, 6 Sep 2014 17:32:02 +0000 (13:32 -0400)]
Volume types need to be specified when creating CG
When creating a consistency group, the scheduler will find a backend that
supports all input volume types. If volume types are not provided, the
default_volume_type in cinder.conf will be used, however, this could cause
inconsistent behavior in a user environment where the default_volume_type
is defined in some places but not in others. This fix removed the use of
default_volume_type for CG creation, added a check to verify that volume
types are provided when creating a CG.
When creating a volume and adding it to a CG, we need to make sure a
volume type is provided as well.
Andreas Jaeger [Fri, 12 Sep 2014 20:40:23 +0000 (22:40 +0200)]
Stop using intersphinx
Remove intersphinx from the docs build as it triggers network calls that
occasionally fail, and we don't really use intersphinx (links other
sphinx documents out on the internet)
This also removes the requirement for internet access during docs build.
This causes docs jobs to fail because we error out on warnings.
Clinton Knight [Fri, 15 Aug 2014 15:12:14 +0000 (11:12 -0400)]
Netapp drivers support for pool-aware scheduling
Adds pools support for all NetApp drivers: eseries, 7mode (iscsi and nfs), and
cmode (iscsi and nfs). With 7mode and cmode drivers, a pool is one-to-one with a
Ontap flexvol. With eseries, a pool is one-to-one with a dynamic disk pool.
ling-yun [Tue, 26 Aug 2014 10:39:30 +0000 (18:39 +0800)]
Add reset-state function for backups
Since there are volume reset-state function and snapshot reset-state
function, backup also needs reset-state as well.
When creating or restoring backup, it may leave the backup stuck in
creating or restoring status due to database down or rabbitmq down, etc.
Currently we could only solve these problems by restarting cinder-backup
service.
This BP is to provide another means for administrators to solve these
problems by calling backup reset state API, instead of directly
restarting cinder-backup service.
Mike Perez [Thu, 4 Sep 2014 04:05:53 +0000 (21:05 -0700)]
Destroy Datera export target after detach
Since there is only one lun associated with each target export object in
the Datera backend, we will destroy the target export object after
detaching the volume so they don't build up. This patch also fixes some
missing values for formatted strings.
The copy_volume_to_image requires in some cases an intermediary
conversion, which takes place in the configured image conversion
dir. If this folder is not created, the operation will fail.
This patch ensures that method will create the folder if it does
not exist.
Jay S. Bryant [Wed, 10 Sep 2014 15:18:22 +0000 (10:18 -0500)]
Sync service.py and its dependencies to Cinder
Bug 1362455 made note that our sevice.py was out of sync
from what is currently in oslo-incubator. We have been syncing
things piece meal. It would be good to get this code synced up.
This commit pulls in the following changes listed by file,
ordered from newest to oldest:
cinder/openstack/common/eventlet_backdoor.py
------------------------------------------------ 90ae24bf Remove redundant default=None for config options fcf517d7 Update oslo log messages with translation domains
cinder/openstack/common/jsonutils.py
------------------------------------------------ d9060f69 Check for namedtuple_as_object support before using it ad248f66 Specify namedtuple_as_object=False when using simplejson
cinder/openstack/common/log.py
------------------------------------------------ 037dee00 Set stevedore log level to WARN by default
cinder/openstack/common/loopingcall.py
------------------------------------------------ e3773930 Changes calcuation of variable delay ab5d5f1c Use timestamp in loopingcall bc48099a Log the function name of looping call fb4e863c Remove deprecated LoopingCall fcf517d7 Update oslo log messages with translation domains
cinder/openstack/common/service.py
------------------------------------------------ 5f12bba4 Implement stop method in ProcessLauncher 8c00b361 Merge "Remove duplicated "caught" message" f718e404 Remove duplicated "caught" message 356ecd4d Move notification point to a better place e2634a75 Add missing _LI for LOG.info in service module 0150ad75 Merge "Reap child processes gracefully if greenlet thread gets killed" 53e1214c notify calling process we are ready to serve a6fec438 Reap child processes gracefully if greenlet thread gets killed fcf517d7 Update oslo log messages with translation domains 1e700782 Revert "service: replace eventlet event by threading" 0644073e Simplify launch method 6b4d255e Merge "service: replace eventlet event by threading" a4f145e4 Merge "Allow configurable ProcessLauncher liveness check"
cinder/openstack/common/threadgroup.py
------------------------------------------------ 5a1a0166 Make stop_timers() method public fdc88831 Add graceful stop function to ThreadGroup.stop 5f8ace05 Merge "threadgroup: use threading rather than greenthread" 2d06d6ca Simple typo correction 4d18b57a threadgroup: use threading rather than greenthread
cinder/openstack/common/systemd.py
------------------------------------------------- 17c4e21e Fix docstring indentation in systemd 667d1bac Fixed spelling error - occured to occurred 53e1214c notify calling process we are ready to serve
Mark Sturdevant [Tue, 2 Sep 2014 23:25:47 +0000 (16:25 -0700)]
HP 3PAR configurable ssh-host-key-policy
Cinder added configurable SSH strict key policy (RejectPolicy)
and configurable known_hosts_file. This patch configures the
hp3parclient (used by the 3PAR drivers) using those settings.
Xing Yang [Fri, 5 Sep 2014 21:48:00 +0000 (17:48 -0400)]
Delete consistency group failed
Delete CG failed because it compares host@backend with host@backend#pool
(in delete_consistencygroup in volume/manager.py). The fix is to call
extract_host before doing the comparison.
Another issue is about deleting a CG with no host. This will throw exception
when extract_host(group['host']) is called in delete_consistencygroup in
volume/rpcapi.py. The solution is to check the host field in consistencygroup/
api.py and delete it from db there.
Introduces a volume driver which makes use of SMB shares for hosting
volumes as disk images, having a similar workflow with the other NFS
like drivers.
This driver is based on the SMB Volume driver proposed for Linux,
overriding the platform specific methods.
It includes all the features required by the Juno release.
Various commands were failing when running multiple copies of
cinder-volume. This is due to storing an in memory copy of the luns
which may get updated in the backend by another cinder-volume. Fix
it by attempting to refresh the list of luns if one is not found
during a request.
Introduces a volume driver which makes use of SMB shares for
hosting volumes as disk images, having a similar workflow with the
other NFS like drivers.
The driver uses Samba in order to access the SMB shares. It takes
SMB shares information such as share path, credentials and other
mount flags from a config file of which location can be configured
in the cinder config file.
It includes all the features required by the Juno release.
Given the non-blocking nature of RPC cast, what happens in volume
manager and volume driver can happen in parallel with the DB update.
If volume driver relies on original DB record to do things, then
DB record shouldn't be updated until volume driver finishes its job.
So this patch change volume RPC API accept_transfer() from cast
to call to make sure the workflow is in serialized manner. Also
elevated the context when volume manager tries to update the DB
record when driver has done accept_transfer().
This patch changes the glance client version unit tests
to mock the Client object completely. Previously the
tests were ensuring the right version of the client was
returned, but that required too much knowledge of glance's
implementation and ended up breaking the tests when glance
changed the implementation details of the Client class.
The new code tests if cinder is calling the integration
point correctly for the version, rather than if glance
is correctly returning the right client; that should be
a glance test, not a cinder test.
Since the test is `test_create_missing_specs_name`, we should define
the the body like `body = {'qos_specs': {'a': 'b'}}`. And add a test
to cover the invalid body case.
Zhi Kun Liu [Fri, 5 Sep 2014 08:51:30 +0000 (16:51 +0800)]
remove object in wsgi LOG.info
Method __init__ in Server class records log for wsgi server name, host
and port using its __dict__ which includes a socket object. i18n message
will deep copy each item's value in __dict__. In python2.6, deep copy
the socket object will raise "Exception RuntimeError" and can not be
caught. This makes cinder-api run into a hang loop. This patch uses the
related properties instead of __dict__ object to fix the problem.
Andrew Kerr [Fri, 5 Sep 2014 15:28:06 +0000 (10:28 -0500)]
Don't clear _mounted_shares list in remoteFS while updating
This fix makes the updating of the _mounted_shares list in
remoteFS more of an atomic operation. Previously this list
would be cleared, then rebuilt. That allowed a race
condition where operations that took place during this
update would have a list of 0 shares to work with.
liyingjun [Wed, 6 Aug 2014 21:58:17 +0000 (05:58 +0800)]
Add filter to volume transfer REST api
The transfer api supports filter param,
https://github.com/openstack/cinder/blob/master/cinder/transfer/api.py#L72
But the transfer REST api didn't pass the filter params when calling get_all.
The configuration option gpfs_storage_pool is assigned with
default value of None. And it gets initialized with None
instead of system, where as the intention is to set the default
value as system.
This change sets default value of gpfs_storage_pool to system
instead of None.
DocImpact
Needs an update in gpfs driver documentation, as this patch
proposes a change in default value of configuration option
(gpfs_storage_pool)
The method copy_volume_to_image from windows.py specifies the
wrong volume format, namely 'vpc'. In this case, the upload_volume
method from image_utils will attempt to convert the volume to vhd
which will result in an error from qemu as it does not recognize
the format.
This patch fixes this issue by using 'vhd' as a format when
uploading volumes
Extend virtual disk API is called during volume creation from image if
the image size is less than volume size. In the case of streamOptimized
and sparse vmdk images, the size of the virtual disk created from the
image can be greater than image size and equal to the volume size. For
example, streamOptimized image created from a new 1GB volume has size
69120 bytes and the size of the virtual disk created from this image
is 1GB. Therefore, relying on image size to invoke extend API might
result in VIM API fault, if the virtual disk size is same as the target
size (volume size). The fix is to read the current virtual disk size to
decide whether extend disk needs to be invoked or not.
Currently, backup-create and backup-restore operations are not
supported for volumes created by the VMDK driver. This change
implements backup_volume and restore_backup APIs for VMDK driver.
DocImpact
Added a new config option 'vmware_tmp_dir' which specifies
the directory in Cinder host to use for storing temporary
virtual disk files during backup/restore.
Kurt Martin [Tue, 2 Sep 2014 23:56:53 +0000 (16:56 -0700)]
Update the HP 3PAR default persona
The HP 3PAR drivers was setting the default persona to Generic.
The default should actually be Generic-ALUA for Linux based host.
DocImpact
This will require an update in the OpenStack Configuration Reference
Guide under the supported operations page for the HP 3PAR drivers.
The new default for the hp3par:persona should be '2 - Generic-ALUA'.
ling-yun [Sat, 2 Aug 2014 06:43:35 +0000 (14:43 +0800)]
Support Volume Backup Quota
Since quota take volumes, snapshots and gigabytes into account, it also
needs to take backup into account.
1. Backup create API is not admin interface, users of projects could
create any number of backups.
2. If some evil users create many more big backups to exhaust the free
space of backup storage-backend, it would cause cinder-backup in the state
of rejecting service.
Based on these two points, I think it is necessary to support volume
backup quota.
Code Review / openstack-build / cinder-build.git / log