Joshua Harlow [Mon, 9 Jun 2014 22:32:17 +0000 (15:32 -0700)]
Handle the case where az is disabled/removed
Instead of retaining a local in-memory list of
availability zones that is never purged and therefore
never updates itself if availability zones are removed
we should instead cache those zones for a given period
of time (defaulting to one hour) and on cache expiry
refetch the enabled availability zones.
DocImpact: adds a new configuration that defines the
default availability zone cache expiry time in seconds.
This will enable the engine that runs the create volume
request to in a future change remove the need to have
a functor which proxies the validation logic around
availability zones.
abhishekkekane [Tue, 10 Jun 2014 06:29:20 +0000 (06:29 +0000)]
Made provision for providing optional arguments
The 'quota_committed' attribute of 'RequestContext' object is
a transient property, so it will not be saved in the taskflow
persistent storage. The updated value of 'quota_committed'
attribute will not be available while resuming/reverting the
flow, if cinder api-service is down/stopped after committing
the quota.
Since this 'quota_committed' attribute is not used anywhere
in cinder project other than in create-volume taskflow api, so
removed 'quota_committed' from RequestContext and made
provision to pass it as an optional argument which will be
passed to api-flow via create_what dictionary, in order to
make it persistent and use it as and when needed.
The reset-state API call only sets that main status
column on the volume object. There's also a method
to set attach status, however it turns out that due
to the implementation of the status validation method
in the parent class, this was impossible to use because
it only checked and accepted "status" NOT "attach_status".
This patch fixes that attach_status problem, it also implements
the ability to update the migration status. A cinderclient change
will be needed to expose these calls as well.
PranaliDeore [Tue, 6 May 2014 04:04:45 +0000 (04:04 +0000)]
Copy custom properties to image from volume
Presently after copying an image to volume, all properties
of the image are getting copied properly but while creating
image back from volume, it doesn't copy custom properties
to the image.
At present in volume-glance-metadata table all the properties
of volume are stored as key and value. Because of this it is
difficult to differentiate between core and custom properties.
To overcome this, I have added a new option
'glance_core_properties' in cinder.conf. This option defines
all core properties of an image. This way, it's easy to separate
core and custom properties from the glance_volume_metadata and add
custom property to the newly created image.
Fix Brocade FC driver's use of the fc_fabric_names
The Brocade Fibre Channel Zone Manager driver wasn't
stripping out whitespace in the fc_fabric_names
prior to using them. This would cause breakage later on
in the driver.
Eric Harney [Tue, 10 Jun 2014 15:25:01 +0000 (11:25 -0400)]
LIO: Don't add ACL for local initiator name
Currently cinder-rtstool requires /etc/iscsi/initiatorname.iscsi
to be present in order to create a LUN/ACL/portal successfully.
However, this is not required since Cinder will create the
required ACLs dynamically at attach time as well.
Removing this path means that iscsi-initiator-utils no longer
needs to be installed locally when using a remote Nova compute
node.
On upload-to-image failure before initiating the data transfer or during
data transfer, the source volume status is restored properly whereas the
image created remains in queued or saving state. This change deletes the
image during such failures.
This patch flushes IO for each SCSI device that
brick removes from the system. The patch also
flushes a multipath device prior to removing
each underlying LUN associated with the
multipath device. We were seeing kernel
IO errors in the syslog without calling flush
on the block device, during a detach for a
multipath FibreChannel copy image to volume
action.
Rahul Verma [Tue, 3 Jun 2014 22:26:48 +0000 (15:26 -0700)]
Remove Quota Exception Stack Traces from Cinder Logs
Added an if clause to the Cinder API FaultWrapper so that
a warning is printed rather than an error and stack trace
if an exception of type QuotaError is thrown.
Nikola Dipanov [Fri, 25 Apr 2014 11:38:31 +0000 (13:38 +0200)]
Make begin_detaching fail if volume not "in-use"
Like it's counterpart from Nova's volume-attach functionality standpoint
- reserve_volume, begin_detaching should fail if the volume is not in
the correct state to be detached. This will prevent nova from
attempting to check and then detach, which is inherently racy.
Mark Sturdevant [Tue, 27 May 2014 21:49:34 +0000 (14:49 -0700)]
3PAR volume detach with host in a host set
When deleting the last VLUN for a host, try to
delete the host. If the host is in a host set
this will fail. When that happens, log a nice
message including the reason, but don't fail
or hang the delete VLUN. Leave hosts in host
sets.
John Griffith [Fri, 30 May 2014 15:59:52 +0000 (15:59 +0000)]
Update cinder.conf
This is mostly just a sanity check, we're revmoving
this whole check from the gate, but we've had
conflicting versions of the truth here and I'm trying
to figure out why.
This patch is a fresh run of generate_sample, which differs
from https://review.openstack.org/#/c/96784/
John Griffith [Thu, 29 May 2014 18:18:12 +0000 (18:18 +0000)]
Remove second get call to list/show volumes
Recently it was observed that with large numbers of volumes
that things like "cinder list" could take extremely long to
return.
For example, running cinder list on a system with 1000 volumes took
greater than 30 seconds to return. It turns out that the cause of
this is the addition of visible admin_metadata.
There's two problems with this:
1. The original patch probably shouldn't have gone in
data is either admin data or it's not, selectively picking
pieces of admin data out to provide to the user just creates
complications and introduces confusion.
2. The REAL issue here is that since the standar gets are made
with the standard user context, the add_visible_admin_metadata
would go through and do an elevated context get on every single
volume individually. This is what caused the horrible performance
issue on cinder list with large numbers of volumes.
Running as admin, or removing the second call drops this down to about 3
seconds for the same 1000 volume list.
This patch removes the secondary admin context get_call. Instead where
we expect to do display the visible admin_meta, we pass in a flag
requesting that the volume object we're getting has the appropriate
metadata. This way we can elevate the context if needed and avoid
iterating through the gets again.
This patch also cleans up the get_visible_admin_meta methods, and
consolidates both V1 and V2 to use the utils method.
Juan Manuel Olle [Thu, 22 May 2014 13:24:36 +0000 (10:24 -0300)]
Fix a message format error in migration cleanup
In some situations, the volume migration failed. After the cleanup
begin, code was trying to format a string value as an int,
this caused "copy_error = True" not being invoked, resulted
in UnboundLocalError exception.
Yi Chun, Huang [Thu, 29 May 2014 10:10:50 +0000 (06:10 -0400)]
Add support for z/VM driver.
Add 'FCP' into connection_info in initialize_connection
in order to support IBM z/VM hypervisors. This change only
because z/VM need FCP(fibre channel port) info while other
platform don't need it. It will not have side effect
to other platforms since they can safely ignore it.
Mike Perez [Thu, 22 May 2014 21:29:06 +0000 (14:29 -0700)]
Handle volumes no longer existing in resume delete
init_host resumes deletes for volumes with a status of 'deleting'. It
does these sequentially, and sometimes a volume could be deleted under
it, causing a bad stacktrace. This will gracefully handle those
situations and move on. Added coverage for resume volume delete being
successful as well.
Eric Harney [Tue, 27 May 2014 14:11:53 +0000 (10:11 -0400)]
Fix ISER scan retry option
The 'num_iscsi_scan_tries' option was renamed to
'num_volume_device_scan_tries'.
Remove it from the test where the deprecated name is
referenced since it is not used there anyway.
Fix the ISER driver to set the new config field rather than the
old one. Presumably this made the num_iser_scan_tries option
not work since only the new field is passed to the brick
connector.
1. Currently cinder doesn't check whether the image is active
when create volume with an image id, we can even create volume
with a deleted/saving/queued image.
2. For deleted image, it only happened when using admin credentials.
When using non-admin credentials, it returned an error that image
is not found by using 'glance image-show' on a deleted image, which
is expected. Also when creating a volume from this deleted image,
it returned a 404 error upfront. This again is the expected result.
When using admin credentials, we can view the details of deleted image
with 'glance image-show'. If create a volume out of an image ID (which
is now deleted), the operation succeeds, but the volume is left in
'error' state.
3. This accounts for a bug(he volume shouldn't be created) and should be fixed.
This allows you to specify host=XXX in your backend configuration
to override the host value that is specified in the DEFAULT section,
allowing for some backends to share a hostname while others can
have unique names.
Andreas Jaeger [Thu, 22 May 2014 16:33:09 +0000 (18:33 +0200)]
Remove all mostly untranslated PO files
We only import PO files that are at least 75 % translated,
so we can delete now all other PO files. The patch only
removes the mostly untranslated files. Once a file becomes mostly
translated, the bot will import it again.
Joshua Harlow [Thu, 22 May 2014 02:26:15 +0000 (19:26 -0700)]
Remove create_from* functor jump table
To make it easier to debug during runtime which method was
called remove the dictionary type -> functor mapping and
use a set of if statements instead. This makes it so that
if one of these methods fails we can gather more information
from the traceback than what was previously possible.
Lynxzh [Mon, 19 May 2014 10:47:16 +0000 (18:47 +0800)]
SSHPool in utils should allow customized host key missing policy
The cinder/utils SSHPool should allow missing key policy and host key
file being customized so that any caller can determine by their own
scenario if the host key file can be customized, or if an 'AutoAdd' is
appropriate, or just reject the key when mismatch. This will give more
flexible customization and also prevent any security issue as a middle
man.
vitas.yuzhou [Mon, 12 May 2014 18:57:48 +0000 (02:57 +0800)]
Check whether O_DIRECT is supported to iflag and oflag separately
The problem is when this is used for volume clear and the clear method
specifies use of the 'zero' option (/dev/zero) setting iflag direct is
not a valid option for that input file, but oflag direct is a valid
option for output file, so we should check iflag and oflag separately.
Mike Perez [Sat, 17 May 2014 19:18:59 +0000 (15:18 -0400)]
Set volume usage audit period to not NoneType
The setting of NoneType causes cinder-volume-usage-audit to not start up
and throw a stacktrace. This sets the config default back to something
it can use.
Lynxzh [Mon, 19 May 2014 09:49:00 +0000 (17:49 +0800)]
BrcdFCSanLookupService should allow customize host key and policy
In BrcdFCSanLookupService, the initialization should allow the
customization of the known_hosts_file and missing_key_policy so that the
hosts key and missing policy can be customized according to the
different scenario and customer aspect. This will not change the default
behavior when no argument is given, but more flexible to allow the
caller to give more options according to different requirements.
Mark Sturdevant [Mon, 12 May 2014 21:27:48 +0000 (14:27 -0700)]
eliminate the need for hplefthandclient in tests
In order to eliminate the need to have the hplefthandclient in the
global-requirements project, we need to remove the hplefthandclient
from being imported in all HP LeftHand driver unit tests in cinder.
Scott Devoid [Mon, 5 May 2014 21:44:44 +0000 (16:44 -0500)]
Fix solaris_execute in SolarisISCSIDriver.
A previous refactor [1] of SolarisISCSIDriver and SanDriver renamed
the `_execute` function to `san_execute`. However, SolarisISCSIDriver
still called the _execute function, which no longer exists in it's
parent classes.
This change calls san_execute instead of _execute in
super(SolarisISCSIDriver, san).
Mark Sturdevant [Mon, 5 May 2014 21:13:48 +0000 (14:13 -0700)]
eliminate the need for hp3parclient in tests
In order to eliminate the need to have the hp3parclient in the
global-requirements project, we need to remove the hp3parclient
from being imported in all 3par driver unit tests in cinder.
Inhye Park [Tue, 15 Apr 2014 09:48:44 +0000 (18:48 +0900)]
Limit formatting routes when adding resources
By default, routes.mapper.Mapper.resource adds a bunch of formatted
routes that accept anything after a '.'. Our spec says only .xml
and .json are accepted so limit the formatting to those valuse.
This allows identifiers with a '.' in urls. A few tests were added
to the extensions test to prove that .xml and .json are stripped
but other values are not.
pran1990 [Mon, 5 May 2014 23:07:13 +0000 (16:07 -0700)]
Use oslo network utils function to set tcp_keepalive
Earlier, tcp_keepidle was set to 600s and other parameters left to default
values. A new function was added to oslo-incubator for setting tcp keepalive
settings, and cinder should use this to set any values we want.
In addition to the existing cfg option for tcp_keepidle, new values were added
for tcp_keepalive(to turn on/off this feature, tcp_keepintvl(to specify gap
between successive probes) and tcp_keepcnt(to specify number of probes). To
mimic existing behavior, tcp_keepalive defaults to True, and tcp_keepidle to
600(seconds). The other two default to None, and the oslo code will ignore
them.
Ran update.py in oslo-incubator to copy changed modules into cinder.
Changes are:
network_utils.py (which contains the function we need, others are deps) edd73c9 Merge "Improve help strings" fe3389e Improve help strings 8a0f567 Remove str() from LOG.* and exceptions f61a4e7 Fix python26 compatibility for RFCSysLogHandler da6d713 Revert setting oslo-incubator logs to INFO 0d18381 Set default log levels for oslo.messaging and oslo-incubator
gettextutils.py 9912e5d Add API for creating translation functions 6cc96d0 Fix test_gettextutils on Python 3
jsonutils.py 9e5a393 jsonutils.load() accepts file pointer, not string a6b2aec On Python <= 2.6, use simplejson if available e3a1d9c Use six.moves.xmlrpc_client instead of xmlrpclib 3a31bba Python3 support for xmlrpclib e6494c2 Use six.iteritems to make dict work on Python2/3
log.py edd73c9 Merge "Improve help strings" fe3389e Improve help strings 8a0f567 Remove str() from LOG.* and exceptions f61a4e7 Fix python26 compatibility for RFCSysLogHandler da6d713 Revert setting oslo-incubator logs to INFO 0d18381 Set default log levels for oslo.messaging and oslo-incubator
Current HEAD in OSLO:
----------------------------------------------
Merge: 2640847e433899
Date: Mon May 5 19:41:28 2014 +0000
Merge "Enable configuring tcp keepalive"
----------------------------------------------
Ran generate_sample to generate cinder.conf.sample
Code Review / openstack-build / cinder-build.git / log