Jay S. Bryant [Fri, 19 Sep 2014 17:46:21 +0000 (12:46 -0500)]
Fix unnecessary WSGI worker warning at API startup
There was a bug in WSGIService in the way that it was
checking the osapi_volume_workers option. It was using
getattr() to see if the option was set, if not it was supposed
to set the value to processutils.get_worker_count(). This,
however, never happened because getattr interpreted the default
'None' value to be a value. So, on any system with no value set
the self.workers < 1 check would be hit and a warning would be
output.
Nova had changed their approach to this option to avoid this
problem. This patch pulls Nova's approach into Cinder for
consistency. Cinder will now use processutils.get_worker_count()
if no option is set in /etc/cinder/cinder.conf and when the user sets
osapi_volume_workers to 0. A negative value will cause an
InvalidInput exception to be thrown.
Mark Sturdevant [Tue, 23 Sep 2014 05:45:14 +0000 (22:45 -0700)]
Fix ssh_host_key_file default in help and config.sample.conf
The commit message and the actual default say the default value for
ssh_host_key_file is $state_path/ssh_known_hosts, but the
config.conf.sample and the config opts help say it is
"$state_path/known_hosts".
Fix the help and config.conf.sample to match the actual default.
Downgrade 'infinite' and 'unknown' capacity in weigher
When FilterScheduler was first introduced into Cinder, drivers were
required for the first time to report capacity. Some drivers preferred
to report 'infinite' or 'unknown' capacity because they were doing
thin-provisioning or the total capacity kept increasing. Now that we
have better support for thin-provisioning and we do find unrealistic
capacity couldn't do us any good in making optimal scheduling decision,
because 'infinite' and 'unknown' would always have the highest weight
when the weight multiplier is positive, which in most cases it is.
Drivers are expected to avoid sending 'infinite' 'unknown' capacity
anymore, instead, should report an actual real number for total/free
capacity.
This fix doesn't fix the driver, instead a small tweak is added to
CapacityWeigher in order to downgrade those drivers who report
'infinite' or 'unknown' as free capacity. In particular, those who
report 'infinite'/'unknown' free capacity will be adjusted to be the
one has lowest weight, no matter in 'spreading' (weight multiplier>0)
or 'stacking' (weight multiplier<0) mode.
Xing Yang [Sat, 20 Sep 2014 22:23:11 +0000 (18:23 -0400)]
DB migration 25->24 failed when dropping column
"cinder-manage db sync 24" failed when dropping column cgsnapshot_id
from the snapshots table.
The reason that drop column failed was because of the foreign key
constraint. MySQL cannot drop column until the foreign key constraint
is removed. So the solution is to remove the foreign key first, and
then drop the column. This affects the cgsnapshot_id column in the
snapshots table and the consistencygroup_id column in the volumes table.
With pool support added to Cinder, now we are kind of in an awkward
situation where we require admin to input exact location for volumes
to-be managed (imported) or migrated, which must have pool info, but
there is no way to find out what pools are there for backends except
looking at the scheduler log. That causes bad user experience, and
thus is a bug from UX POV.
This change simply adds a new admin-api extension to allow admin to
fetch all the pool information from scheduler cache (memory), which
closes the gap for end users.
This extension provides two level of pool information: names only or
detailed information:
Pool name only:
GET http://CINDER_API_ENDPOINT/v2/TENANT_ID/scheduler-stats/get_pools
Detailed Pool info:
GET http://CINDER_API_ENDPOINT/v2/TENANT_ID/scheduler-stats/get_pools
\?detail\=True
The latest 3PAR firmware supports a hostname of 31 characters.
This patch increases the hostname on the 3PAR from 23 characters to
31.
The driver currently has a fallback mechanism in place for detecting
existing hosts. This handles the case where upgrading from
Icehouse to Juno where Icehouse hosts have a limit of 23 characters.
Mark Sturdevant [Sun, 14 Sep 2014 00:04:27 +0000 (17:04 -0700)]
HP 3PAR drivers should not claim to have 'infinite' space
The HP 3PAR drivers report 'infinite' space when there is not a limit
set on the CPG. In this case, it would be better to at least use the
free space estimate of the array instead of 'infinite'.
Xing Yang [Thu, 18 Sep 2014 20:50:58 +0000 (16:50 -0400)]
Add tests for consistency groups DB migration
The consistency group patch added 2 new DB migration versions:
025 and 026. However, there were no unit tests to support them in
cinder/tests/test_migrations.py. This patch added missing tests.
John Griffith [Thu, 18 Sep 2014 03:16:18 +0000 (21:16 -0600)]
Verify requested size in volume.api create
Currently we're not checking that the input value requested
for size on volume create is valid, but taskflow portion of
the code expects it to be and as a result when it receives invalid
input we litter the logs with Trace messages.
This patch adds a check that verifies that if we pass in a
size to create_volume in the volume.api that it is in fact
an int or a string representation of an int.
Currently socket options, socket.SO_REUSEADDR and socket.SO_KEEPALIVE
are set only if SSL is enabled.
The above socket options should be set no matter SSL is enabled or not.
Mark Sturdevant [Fri, 12 Sep 2014 21:03:28 +0000 (14:03 -0700)]
HP 3PAR: Allow retype when the old snapshot CPG (3PAR pool) is None
A common provisioning group (CPG) is a virtual pool of logical disks.
A volume is stored in a CPG. The volume's snapshots may be stored in
the same CPG or in a separate CPG. In 3PAR this is the "snapCPG".
Before the 3PAR driver supported "manage", the snapCPG was never None in
OpenStack because when we create volumes, we explicitly default snapCPG
to match the volume CPG unless otherwise specified. So, the original
retype pre-checks raised an exception if this unexpected case occurred
and a unit test was provided for that exception.
Now that we support manage_existing(), it should be a valid use case
to manage an existing volume created outside of OpenStack with a snapCPG
set to None. Unfortunately, when we applied volume-type settings to this
volume we would hit that old exception.
That exception has been removed.
Removing it required the following changes:
1. When retype pre-checks validate the domain of the new snapCPG, it
will not use the optional old snapCPG. Instead it will compare the
new snapCPG domain with the old volume CPG domain. This satisfies
the requirement that domains cannot be mixed and avoids the need
to have an old snapCPG setting.
2. Remove the no longer used old_snap_cpg parameter and remove the
code that raised an exception "if not old_snap_cpg".
3. Remove the unit test that was just to test that obsolete exception.
4. Adjust the exiting test_retype_across_snap_cpg_domains to
verify that a snapCPG domain that does not match the volume
CPG domain will still raise Invalid3PARDomain
Ian Govett [Tue, 16 Sep 2014 11:32:49 +0000 (07:32 -0400)]
Fix a problem with 'volume list' when 'all_tenants=0'
This change checks the value of 'all_tenants' and returns a list of volumes
for all tenants when 'all_tenants' is 1 (or True). A tenant-specific volume
list is returned when 'all_tenants' is 0 (or False).
An InvalidInput exception is thrown if the 'all_tenants' value is not 0, 1,
True, or False (case insensitive).
Provided new unit tests for the get_all api which check the all_tenants, and
limits parameters.
Fixed pep8 compliance test fails with code being improperly indented, and
changed calls using str() to use six.text_type()
Nilesh Bhosale [Sun, 24 Aug 2014 07:36:29 +0000 (13:06 +0530)]
IBMNAS: Remove call to set r/w permissions to all
During cinder volume create operation from a volume snapshot or
from an existing volume (volume clone operation), the ibmnas
driver sets 'rw' permissions to all, which is unnecessary and
also poses security concerns.
Fixing this issue, removing the calls to set rw permissions to
all during these operations and adding a call to set 'rw'
permissions only to the owner to make sure even if umask is set
at the filesystem level, which might deny 'rw' access to the owner
we explicitely set the required permissions on the volume file.
Xing Yang [Sun, 14 Sep 2014 23:50:01 +0000 (19:50 -0400)]
Getting iscsi_ip_address from cinder.conf
In the current version (2.0) of the VMAX driver, we retrieve iSCSI IP
addresses dynamically from SMI-S. However, we ran into situations where
we can't get them reliably during testing. The fix is to get this
information from cinder.conf, just like in version 1.0.
Xing Yang [Sat, 6 Sep 2014 17:32:02 +0000 (13:32 -0400)]
Volume types need to be specified when creating CG
When creating a consistency group, the scheduler will find a backend that
supports all input volume types. If volume types are not provided, the
default_volume_type in cinder.conf will be used, however, this could cause
inconsistent behavior in a user environment where the default_volume_type
is defined in some places but not in others. This fix removed the use of
default_volume_type for CG creation, added a check to verify that volume
types are provided when creating a CG.
When creating a volume and adding it to a CG, we need to make sure a
volume type is provided as well.
Andreas Jaeger [Fri, 12 Sep 2014 20:40:23 +0000 (22:40 +0200)]
Stop using intersphinx
Remove intersphinx from the docs build as it triggers network calls that
occasionally fail, and we don't really use intersphinx (links other
sphinx documents out on the internet)
This also removes the requirement for internet access during docs build.
This causes docs jobs to fail because we error out on warnings.
Clinton Knight [Fri, 15 Aug 2014 15:12:14 +0000 (11:12 -0400)]
Netapp drivers support for pool-aware scheduling
Adds pools support for all NetApp drivers: eseries, 7mode (iscsi and nfs), and
cmode (iscsi and nfs). With 7mode and cmode drivers, a pool is one-to-one with a
Ontap flexvol. With eseries, a pool is one-to-one with a dynamic disk pool.
ling-yun [Tue, 26 Aug 2014 10:39:30 +0000 (18:39 +0800)]
Add reset-state function for backups
Since there are volume reset-state function and snapshot reset-state
function, backup also needs reset-state as well.
When creating or restoring backup, it may leave the backup stuck in
creating or restoring status due to database down or rabbitmq down, etc.
Currently we could only solve these problems by restarting cinder-backup
service.
This BP is to provide another means for administrators to solve these
problems by calling backup reset state API, instead of directly
restarting cinder-backup service.
Mike Perez [Thu, 4 Sep 2014 04:05:53 +0000 (21:05 -0700)]
Destroy Datera export target after detach
Since there is only one lun associated with each target export object in
the Datera backend, we will destroy the target export object after
detaching the volume so they don't build up. This patch also fixes some
missing values for formatted strings.
The copy_volume_to_image requires in some cases an intermediary
conversion, which takes place in the configured image conversion
dir. If this folder is not created, the operation will fail.
This patch ensures that method will create the folder if it does
not exist.
Jay S. Bryant [Wed, 10 Sep 2014 15:18:22 +0000 (10:18 -0500)]
Sync service.py and its dependencies to Cinder
Bug 1362455 made note that our sevice.py was out of sync
from what is currently in oslo-incubator. We have been syncing
things piece meal. It would be good to get this code synced up.
This commit pulls in the following changes listed by file,
ordered from newest to oldest:
cinder/openstack/common/eventlet_backdoor.py
------------------------------------------------ 90ae24bf Remove redundant default=None for config options fcf517d7 Update oslo log messages with translation domains
cinder/openstack/common/jsonutils.py
------------------------------------------------ d9060f69 Check for namedtuple_as_object support before using it ad248f66 Specify namedtuple_as_object=False when using simplejson
cinder/openstack/common/log.py
------------------------------------------------ 037dee00 Set stevedore log level to WARN by default
cinder/openstack/common/loopingcall.py
------------------------------------------------ e3773930 Changes calcuation of variable delay ab5d5f1c Use timestamp in loopingcall bc48099a Log the function name of looping call fb4e863c Remove deprecated LoopingCall fcf517d7 Update oslo log messages with translation domains
cinder/openstack/common/service.py
------------------------------------------------ 5f12bba4 Implement stop method in ProcessLauncher 8c00b361 Merge "Remove duplicated "caught" message" f718e404 Remove duplicated "caught" message 356ecd4d Move notification point to a better place e2634a75 Add missing _LI for LOG.info in service module 0150ad75 Merge "Reap child processes gracefully if greenlet thread gets killed" 53e1214c notify calling process we are ready to serve a6fec438 Reap child processes gracefully if greenlet thread gets killed fcf517d7 Update oslo log messages with translation domains 1e700782 Revert "service: replace eventlet event by threading" 0644073e Simplify launch method 6b4d255e Merge "service: replace eventlet event by threading" a4f145e4 Merge "Allow configurable ProcessLauncher liveness check"
cinder/openstack/common/threadgroup.py
------------------------------------------------ 5a1a0166 Make stop_timers() method public fdc88831 Add graceful stop function to ThreadGroup.stop 5f8ace05 Merge "threadgroup: use threading rather than greenthread" 2d06d6ca Simple typo correction 4d18b57a threadgroup: use threading rather than greenthread
cinder/openstack/common/systemd.py
------------------------------------------------- 17c4e21e Fix docstring indentation in systemd 667d1bac Fixed spelling error - occured to occurred 53e1214c notify calling process we are ready to serve
Mark Sturdevant [Tue, 2 Sep 2014 23:25:47 +0000 (16:25 -0700)]
HP 3PAR configurable ssh-host-key-policy
Cinder added configurable SSH strict key policy (RejectPolicy)
and configurable known_hosts_file. This patch configures the
hp3parclient (used by the 3PAR drivers) using those settings.
Xing Yang [Fri, 5 Sep 2014 21:48:00 +0000 (17:48 -0400)]
Delete consistency group failed
Delete CG failed because it compares host@backend with host@backend#pool
(in delete_consistencygroup in volume/manager.py). The fix is to call
extract_host before doing the comparison.
Another issue is about deleting a CG with no host. This will throw exception
when extract_host(group['host']) is called in delete_consistencygroup in
volume/rpcapi.py. The solution is to check the host field in consistencygroup/
api.py and delete it from db there.
Introduces a volume driver which makes use of SMB shares for hosting
volumes as disk images, having a similar workflow with the other NFS
like drivers.
This driver is based on the SMB Volume driver proposed for Linux,
overriding the platform specific methods.
It includes all the features required by the Juno release.
Code Review / openstack-build / cinder-build.git / log