Mark Sturdevant [Tue, 23 Sep 2014 05:45:14 +0000 (22:45 -0700)]
Fix ssh_host_key_file default in help and config.sample.conf
The commit message and the actual default say the default value for
ssh_host_key_file is $state_path/ssh_known_hosts, but the
config.conf.sample and the config opts help say it is
"$state_path/known_hosts".
Fix the help and config.conf.sample to match the actual default.
With pool support added to Cinder, now we are kind of in an awkward
situation where we require admin to input exact location for volumes
to-be managed (imported) or migrated, which must have pool info, but
there is no way to find out what pools are there for backends except
looking at the scheduler log. That causes bad user experience, and
thus is a bug from UX POV.
This change simply adds a new admin-api extension to allow admin to
fetch all the pool information from scheduler cache (memory), which
closes the gap for end users.
This extension provides two level of pool information: names only or
detailed information:
Pool name only:
GET http://CINDER_API_ENDPOINT/v2/TENANT_ID/scheduler-stats/get_pools
Detailed Pool info:
GET http://CINDER_API_ENDPOINT/v2/TENANT_ID/scheduler-stats/get_pools
\?detail\=True
The latest 3PAR firmware supports a hostname of 31 characters.
This patch increases the hostname on the 3PAR from 23 characters to
31.
The driver currently has a fallback mechanism in place for detecting
existing hosts. This handles the case where upgrading from
Icehouse to Juno where Icehouse hosts have a limit of 23 characters.
Mark Sturdevant [Sun, 14 Sep 2014 00:04:27 +0000 (17:04 -0700)]
HP 3PAR drivers should not claim to have 'infinite' space
The HP 3PAR drivers report 'infinite' space when there is not a limit
set on the CPG. In this case, it would be better to at least use the
free space estimate of the array instead of 'infinite'.
Xing Yang [Thu, 18 Sep 2014 20:50:58 +0000 (16:50 -0400)]
Add tests for consistency groups DB migration
The consistency group patch added 2 new DB migration versions:
025 and 026. However, there were no unit tests to support them in
cinder/tests/test_migrations.py. This patch added missing tests.
John Griffith [Thu, 18 Sep 2014 03:16:18 +0000 (21:16 -0600)]
Verify requested size in volume.api create
Currently we're not checking that the input value requested
for size on volume create is valid, but taskflow portion of
the code expects it to be and as a result when it receives invalid
input we litter the logs with Trace messages.
This patch adds a check that verifies that if we pass in a
size to create_volume in the volume.api that it is in fact
an int or a string representation of an int.
Mark Sturdevant [Fri, 12 Sep 2014 21:03:28 +0000 (14:03 -0700)]
HP 3PAR: Allow retype when the old snapshot CPG (3PAR pool) is None
A common provisioning group (CPG) is a virtual pool of logical disks.
A volume is stored in a CPG. The volume's snapshots may be stored in
the same CPG or in a separate CPG. In 3PAR this is the "snapCPG".
Before the 3PAR driver supported "manage", the snapCPG was never None in
OpenStack because when we create volumes, we explicitly default snapCPG
to match the volume CPG unless otherwise specified. So, the original
retype pre-checks raised an exception if this unexpected case occurred
and a unit test was provided for that exception.
Now that we support manage_existing(), it should be a valid use case
to manage an existing volume created outside of OpenStack with a snapCPG
set to None. Unfortunately, when we applied volume-type settings to this
volume we would hit that old exception.
That exception has been removed.
Removing it required the following changes:
1. When retype pre-checks validate the domain of the new snapCPG, it
will not use the optional old snapCPG. Instead it will compare the
new snapCPG domain with the old volume CPG domain. This satisfies
the requirement that domains cannot be mixed and avoids the need
to have an old snapCPG setting.
2. Remove the no longer used old_snap_cpg parameter and remove the
code that raised an exception "if not old_snap_cpg".
3. Remove the unit test that was just to test that obsolete exception.
4. Adjust the exiting test_retype_across_snap_cpg_domains to
verify that a snapCPG domain that does not match the volume
CPG domain will still raise Invalid3PARDomain
Ian Govett [Tue, 16 Sep 2014 11:32:49 +0000 (07:32 -0400)]
Fix a problem with 'volume list' when 'all_tenants=0'
This change checks the value of 'all_tenants' and returns a list of volumes
for all tenants when 'all_tenants' is 1 (or True). A tenant-specific volume
list is returned when 'all_tenants' is 0 (or False).
An InvalidInput exception is thrown if the 'all_tenants' value is not 0, 1,
True, or False (case insensitive).
Provided new unit tests for the get_all api which check the all_tenants, and
limits parameters.
Fixed pep8 compliance test fails with code being improperly indented, and
changed calls using str() to use six.text_type()
Nilesh Bhosale [Sun, 24 Aug 2014 07:36:29 +0000 (13:06 +0530)]
IBMNAS: Remove call to set r/w permissions to all
During cinder volume create operation from a volume snapshot or
from an existing volume (volume clone operation), the ibmnas
driver sets 'rw' permissions to all, which is unnecessary and
also poses security concerns.
Fixing this issue, removing the calls to set rw permissions to
all during these operations and adding a call to set 'rw'
permissions only to the owner to make sure even if umask is set
at the filesystem level, which might deny 'rw' access to the owner
we explicitely set the required permissions on the volume file.
Xing Yang [Sun, 14 Sep 2014 23:50:01 +0000 (19:50 -0400)]
Getting iscsi_ip_address from cinder.conf
In the current version (2.0) of the VMAX driver, we retrieve iSCSI IP
addresses dynamically from SMI-S. However, we ran into situations where
we can't get them reliably during testing. The fix is to get this
information from cinder.conf, just like in version 1.0.
Xing Yang [Sat, 6 Sep 2014 17:32:02 +0000 (13:32 -0400)]
Volume types need to be specified when creating CG
When creating a consistency group, the scheduler will find a backend that
supports all input volume types. If volume types are not provided, the
default_volume_type in cinder.conf will be used, however, this could cause
inconsistent behavior in a user environment where the default_volume_type
is defined in some places but not in others. This fix removed the use of
default_volume_type for CG creation, added a check to verify that volume
types are provided when creating a CG.
When creating a volume and adding it to a CG, we need to make sure a
volume type is provided as well.
Andreas Jaeger [Fri, 12 Sep 2014 20:40:23 +0000 (22:40 +0200)]
Stop using intersphinx
Remove intersphinx from the docs build as it triggers network calls that
occasionally fail, and we don't really use intersphinx (links other
sphinx documents out on the internet)
This also removes the requirement for internet access during docs build.
This causes docs jobs to fail because we error out on warnings.
Clinton Knight [Fri, 15 Aug 2014 15:12:14 +0000 (11:12 -0400)]
Netapp drivers support for pool-aware scheduling
Adds pools support for all NetApp drivers: eseries, 7mode (iscsi and nfs), and
cmode (iscsi and nfs). With 7mode and cmode drivers, a pool is one-to-one with a
Ontap flexvol. With eseries, a pool is one-to-one with a dynamic disk pool.
ling-yun [Tue, 26 Aug 2014 10:39:30 +0000 (18:39 +0800)]
Add reset-state function for backups
Since there are volume reset-state function and snapshot reset-state
function, backup also needs reset-state as well.
When creating or restoring backup, it may leave the backup stuck in
creating or restoring status due to database down or rabbitmq down, etc.
Currently we could only solve these problems by restarting cinder-backup
service.
This BP is to provide another means for administrators to solve these
problems by calling backup reset state API, instead of directly
restarting cinder-backup service.
Mike Perez [Thu, 4 Sep 2014 04:05:53 +0000 (21:05 -0700)]
Destroy Datera export target after detach
Since there is only one lun associated with each target export object in
the Datera backend, we will destroy the target export object after
detaching the volume so they don't build up. This patch also fixes some
missing values for formatted strings.
The copy_volume_to_image requires in some cases an intermediary
conversion, which takes place in the configured image conversion
dir. If this folder is not created, the operation will fail.
This patch ensures that method will create the folder if it does
not exist.
Jay S. Bryant [Wed, 10 Sep 2014 15:18:22 +0000 (10:18 -0500)]
Sync service.py and its dependencies to Cinder
Bug 1362455 made note that our sevice.py was out of sync
from what is currently in oslo-incubator. We have been syncing
things piece meal. It would be good to get this code synced up.
This commit pulls in the following changes listed by file,
ordered from newest to oldest:
cinder/openstack/common/eventlet_backdoor.py
------------------------------------------------ 90ae24bf Remove redundant default=None for config options fcf517d7 Update oslo log messages with translation domains
cinder/openstack/common/jsonutils.py
------------------------------------------------ d9060f69 Check for namedtuple_as_object support before using it ad248f66 Specify namedtuple_as_object=False when using simplejson
cinder/openstack/common/log.py
------------------------------------------------ 037dee00 Set stevedore log level to WARN by default
cinder/openstack/common/loopingcall.py
------------------------------------------------ e3773930 Changes calcuation of variable delay ab5d5f1c Use timestamp in loopingcall bc48099a Log the function name of looping call fb4e863c Remove deprecated LoopingCall fcf517d7 Update oslo log messages with translation domains
cinder/openstack/common/service.py
------------------------------------------------ 5f12bba4 Implement stop method in ProcessLauncher 8c00b361 Merge "Remove duplicated "caught" message" f718e404 Remove duplicated "caught" message 356ecd4d Move notification point to a better place e2634a75 Add missing _LI for LOG.info in service module 0150ad75 Merge "Reap child processes gracefully if greenlet thread gets killed" 53e1214c notify calling process we are ready to serve a6fec438 Reap child processes gracefully if greenlet thread gets killed fcf517d7 Update oslo log messages with translation domains 1e700782 Revert "service: replace eventlet event by threading" 0644073e Simplify launch method 6b4d255e Merge "service: replace eventlet event by threading" a4f145e4 Merge "Allow configurable ProcessLauncher liveness check"
cinder/openstack/common/threadgroup.py
------------------------------------------------ 5a1a0166 Make stop_timers() method public fdc88831 Add graceful stop function to ThreadGroup.stop 5f8ace05 Merge "threadgroup: use threading rather than greenthread" 2d06d6ca Simple typo correction 4d18b57a threadgroup: use threading rather than greenthread
cinder/openstack/common/systemd.py
------------------------------------------------- 17c4e21e Fix docstring indentation in systemd 667d1bac Fixed spelling error - occured to occurred 53e1214c notify calling process we are ready to serve
Mark Sturdevant [Tue, 2 Sep 2014 23:25:47 +0000 (16:25 -0700)]
HP 3PAR configurable ssh-host-key-policy
Cinder added configurable SSH strict key policy (RejectPolicy)
and configurable known_hosts_file. This patch configures the
hp3parclient (used by the 3PAR drivers) using those settings.
Xing Yang [Fri, 5 Sep 2014 21:48:00 +0000 (17:48 -0400)]
Delete consistency group failed
Delete CG failed because it compares host@backend with host@backend#pool
(in delete_consistencygroup in volume/manager.py). The fix is to call
extract_host before doing the comparison.
Another issue is about deleting a CG with no host. This will throw exception
when extract_host(group['host']) is called in delete_consistencygroup in
volume/rpcapi.py. The solution is to check the host field in consistencygroup/
api.py and delete it from db there.
Introduces a volume driver which makes use of SMB shares for hosting
volumes as disk images, having a similar workflow with the other NFS
like drivers.
This driver is based on the SMB Volume driver proposed for Linux,
overriding the platform specific methods.
It includes all the features required by the Juno release.
Various commands were failing when running multiple copies of
cinder-volume. This is due to storing an in memory copy of the luns
which may get updated in the backend by another cinder-volume. Fix
it by attempting to refresh the list of luns if one is not found
during a request.
Introduces a volume driver which makes use of SMB shares for
hosting volumes as disk images, having a similar workflow with the
other NFS like drivers.
The driver uses Samba in order to access the SMB shares. It takes
SMB shares information such as share path, credentials and other
mount flags from a config file of which location can be configured
in the cinder config file.
It includes all the features required by the Juno release.
Given the non-blocking nature of RPC cast, what happens in volume
manager and volume driver can happen in parallel with the DB update.
If volume driver relies on original DB record to do things, then
DB record shouldn't be updated until volume driver finishes its job.
So this patch change volume RPC API accept_transfer() from cast
to call to make sure the workflow is in serialized manner. Also
elevated the context when volume manager tries to update the DB
record when driver has done accept_transfer().
This patch changes the glance client version unit tests
to mock the Client object completely. Previously the
tests were ensuring the right version of the client was
returned, but that required too much knowledge of glance's
implementation and ended up breaking the tests when glance
changed the implementation details of the Client class.
The new code tests if cinder is calling the integration
point correctly for the version, rather than if glance
is correctly returning the right client; that should be
a glance test, not a cinder test.
Since the test is `test_create_missing_specs_name`, we should define
the the body like `body = {'qos_specs': {'a': 'b'}}`. And add a test
to cover the invalid body case.
Zhi Kun Liu [Fri, 5 Sep 2014 08:51:30 +0000 (16:51 +0800)]
remove object in wsgi LOG.info
Method __init__ in Server class records log for wsgi server name, host
and port using its __dict__ which includes a socket object. i18n message
will deep copy each item's value in __dict__. In python2.6, deep copy
the socket object will raise "Exception RuntimeError" and can not be
caught. This makes cinder-api run into a hang loop. This patch uses the
related properties instead of __dict__ object to fix the problem.
Andrew Kerr [Fri, 5 Sep 2014 15:28:06 +0000 (10:28 -0500)]
Don't clear _mounted_shares list in remoteFS while updating
This fix makes the updating of the _mounted_shares list in
remoteFS more of an atomic operation. Previously this list
would be cleared, then rebuilt. That allowed a race
condition where operations that took place during this
update would have a list of 0 shares to work with.
liyingjun [Wed, 6 Aug 2014 21:58:17 +0000 (05:58 +0800)]
Add filter to volume transfer REST api
The transfer api supports filter param,
https://github.com/openstack/cinder/blob/master/cinder/transfer/api.py#L72
But the transfer REST api didn't pass the filter params when calling get_all.
Code Review / openstack-build / cinder-build.git / log