Pascal Wehrle [Mon, 17 Nov 2014 22:20:17 +0000 (23:20 +0100)]
allow image_id for imageRef in create volume API v2
The create volume request for the V2 API has a property named
"imageRef" with the ID of the image to be used when creating the
volume.
The name of this property is inconsistent as all multi word
properties use snake case and the property in the reponse is named
"image_id".
This adds support for aliasing the imageRef key with image_id when
creating a volume in v2 of the Cinder API in a backward-compatible
way.
Jay S. Bryant [Mon, 17 Nov 2014 19:11:03 +0000 (13:11 -0600)]
Update cinder.conf.sample to fix max db conn retries
Another external library change (oslo.db) has broken the gate
because cinder.conf.sample appears to be out of date. The change
was just to a message string:
Maximum number of database connection retries during
startup. Set to -1 to specify an infinite retry count. (integer value)
->
Maximum db connection retries during startup. Set to -1 to
specify an infinite retry count. (integer value)
This patch updates cinder.conf.sample to get beyond this failure. The
Cinder team will be talking about how to avoid these issues in the future.
Sean McGinnis [Sat, 15 Nov 2014 01:39:54 +0000 (19:39 -0600)]
Add i18n _LX tags for relevant log levels in EQLX driver.
The decision has been made to follow the translation
convention used in other projects of marking all INFO
and above log strings with _LI, _LW, or _LE. This patch
adds these markers for all log strings in the EQLX
driver that are not being used in an exception or where
array output is being logged.
John Griffith [Fri, 14 Nov 2014 18:26:07 +0000 (11:26 -0700)]
Bump Req timeout to 30 seconds in SolidFire Driver
Local SolidFire CI system seems to have various networking
disconnects during runs. This appears to be a problem with
latencies in our lab.
Currently have resources working on this, but in the meantime
bumping the timeout value in requests to see if we can address
it that way and get CI running again. We'll either get to root
cause in our network or add this as a formal config option later
that will be used only for SolidFire CI system.
git-harry [Fri, 14 Nov 2014 17:10:58 +0000 (17:10 +0000)]
Remove cinder/compute/aggregate_states.py
This file is from the initial fork from nova. It defines the states a
host aggregate can take. It is not used anywhere in the cinder code
and is therefore unnecessary.
git-harry [Fri, 14 Nov 2014 11:22:39 +0000 (11:22 +0000)]
Remove deprecation warnings relating to api-paste
Remove deprecation warnings and backwards compatibility introduced by
commit c355d68a. This commit was introduce in havana and backported to
grizzly. It relates to config options in api-paste.ini.
This commit removes the following deprecated items:
* cinder.api.auth:CinderKeystoneContext
* cinder.api.auth:pipeline_factory
* cinder.api.openstack:FaultWrapper
* cinder.api.openstack.urlmap:urlmap_factory
* cinder.api.openstack.volume:APIRouter
* cinder.api.openstack.volume.versions.Versions
* cinder.api.sizelimit:RequestBodySizeLimiter
git-harry [Fri, 14 Nov 2014 09:09:55 +0000 (09:09 +0000)]
Mock isfile in test_ssh_missing_hosts_key_file
test_ssh_missing_hosts_key_file tests that when the ssh_hosts_key_file
is missing an InvalidInput exception is raised. It does this by
setting CONF.ssh_hosts_key_file to, what is assumed to be, the
location of a non-existant file. If the file, /tmp/blah, exists the
test fails. This test should not be dependent on what is on the
filesystem.
This commit adds a mock for os.path.isfile so that when
ssh_utils.SSHPool checks for existance of CONF.ssh_hosts_key_file it
always gets False regardless of the state of the filesystem.
Mike Mason [Tue, 11 Nov 2014 14:05:24 +0000 (14:05 +0000)]
Implementing the use of _L’x’/i18n markers
Placing the _Lx markers back into the code. No other cleaner solution has
has been implemented. Patches will be submitted in a series of sub
directories and in a fashion that is manageable.
This is the fifth commit of this kind
JordanP [Thu, 13 Nov 2014 16:14:00 +0000 (17:14 +0100)]
Scality driver:use self.configuration instead of CONF
In order for the Filter scheduler to be used to it's full advantage
Cinder drivers need to move away from using FLAGS directly and switch
to appending their specific options to self.configuration.
This patch converts the driver and updates tests.
Commit message stolen from Id1a7c583894ac368bdcc61facc6f72300db320c7
Closes-Bug: #1392633
Change-Id: I61b0b6a21628d5629b5f80768e73ee44db05d1ce
John Griffith [Thu, 13 Nov 2014 14:43:13 +0000 (07:43 -0700)]
Explicitly close requests obj in SolidFire Driver
Currently we're not explicitly returning requests
connection objects back to the ConnectionPool when
we're done with them. For most cases this isn't a
problem, however if we start adding higher and higher
concurrent actions the internal Connection Pool management
doesn't keep up.
This patch just adds an explicit close of the requests
connection after we're done with it to keep the ConnectionPool
as full as possible and avoid any resource issues that could
occur during heavy concurrent usage.
Manojkiran [Wed, 29 Oct 2014 02:36:24 +0000 (08:06 +0530)]
Add volume attribute support to volume created using clone, snapshot
The following patch adds support for volume attributes (setting of
storage pool, replicas, dio, write affinity etc) in case of volume
creation triggered using snapshot, clone.
Patrick East [Fri, 22 Aug 2014 23:43:20 +0000 (16:43 -0700)]
Add automatic creation and deletion of Purity hosts for PureISCSIDriver
The driver will now be responsible to manage hosts for new initiators
which do not already have a host created for them. This will allow for
backwards compatibility with the previous version that relied on hosts
being pre-configured for use by Cinder.
Matthew Edmonds [Tue, 11 Nov 2014 21:03:23 +0000 (16:03 -0500)]
Fix Brocade FC SAN lookup MITM vulnerability
Modify the Brocade FC SAN lookup service implementation to use the
same SSH key config properties used elsewhere rather than relying on
arguments which are non-standard and never passed by the base lookup
service.
Mike Mason [Thu, 6 Nov 2014 13:37:37 +0000 (13:37 +0000)]
Implementing the use of _L’x’/i18n markers
Placing the _Lx markers back into the code. No other cleaner solution has
has been implemented. Patches will be submitted in a series of sub
directories and in a fashion that is manageable.
This is the third commit of this kind
David Pineau [Thu, 6 Nov 2014 14:01:13 +0000 (15:01 +0100)]
Fix the LV NotFound situation for thin-type LVM
If the logical volume is not found, LVM displays on the error output
that the volumes could not be found. So here, we filter on this very
specific situation, and let all the other cases go through the stack.
Added a test for this new code path, which raises an exception of the
proper type to be caught by the new code.
Brianna Poulos [Thu, 6 Nov 2014 21:07:16 +0000 (13:07 -0800)]
Fix wrapper to work with barbicanclient 3.0.1
Due to the updates to the python-barbicanclient in version 3.0.1, the cinder
barbican wrapper no longer functions. This patch updates the cinder barbican
wrapper and test cases to work with barbicanclient 3.0.1.
Matt Riedemann [Fri, 7 Nov 2014 14:23:09 +0000 (06:23 -0800)]
Retry remove iscsi target
There is a race in the gate when removing an iscsi
target in the remove export flow where the target
is still active. It's about a 75% failure rate so
simply adding a configurable retry on the delete
call should clean that up.
John Griffith [Tue, 4 Nov 2014 22:34:28 +0000 (23:34 +0100)]
Remove test_barbican from keymgr tests
Unfortunately it seems we have some very poorly
written keymgr unit tests, specifically
keymgr/test_barbican.py does this:
from barbicanclient.common import auth
First problem is that from a unit test perspective
that pretty much sucks, second problem is that
barbicanclient as of version 3.0.0.0 no longer
has an "auth" module, as a result Cinderunit tests
now fail.
The test_barbican.py unit tests need to be rewritten
to mock out the client components and actually test
Cinder components where needed without relying on
the barbicanclient.
For now in order to free up the gate, remove test_barbican.py
Mike Mason [Thu, 23 Oct 2014 13:16:42 +0000 (13:16 +0000)]
Implementing the use of _L’x’/i18n markers
Placing the _Lx markers back into the code. No other cleaner solution has
has been implemented. Patches will be submitted in a series of sub
directories and in a fashion that is manageable.
ZhiQiang Fan [Sat, 1 Nov 2014 18:18:35 +0000 (02:18 +0800)]
Disable python-barbicanclient 3.0.0 version
python-barbicanclient 3.0.0 has introduced cliff module, but pins it
to 1.6.1, while global-requirements set it to >=1.7.0, now the whole
OpenStack projects depends on higher version of cliff, but
python-barbicanclient is needed by cinder, then grenade test will
fail because: pkg_resources.DistributionNotFound: cliff==1.6.1
As long as python-barbicanclient maintainers don't provide a patch
for 3.0.0 (which seems not happen FMPOV), we need to disbable this
version.
Adrien Vergé [Tue, 28 Oct 2014 21:00:09 +0000 (22:00 +0100)]
Cleanly override config in tests
CONF.set_override() is often called in tests but CONF.clear_override()
is never. Create a override_config() method in the base TestCase class
that restores previous conf value after test, as it is done in other
OpenStack projects.
Xing Yang [Thu, 9 Oct 2014 05:26:28 +0000 (01:26 -0400)]
Use look up service for auto zoning
The VMAX FC driver didn't use the look up service for auto zoning.
Instead it built initiator target map itself. However, that
requires the initiator to log into the fabric before zoning
in order to find out target WWNs.
This patch is to use the look up service to find out valid initiator
target WWNS and use that to build initiator target map. With this fix,
the initiator is no longer required to log into the fabric ahead of time.
Xing Yang [Sun, 26 Oct 2014 21:29:26 +0000 (17:29 -0400)]
CiscoFCSanLookupSerive uses extra argument in init
This patch fixed two issues with the __init__ routine in
CiscoFCSanLookupService:
1. There's an extra argument in super(CiscoFCSanLookupService,
self).__init__(self, **kwargs). It should be changed to
super(CiscoFCSanLookupService, self).__init__(**kwargs).
2. The last line 'self.fabric_configs = ""' should be removed.
self.fabric_configs was created in self.create_configuration() in the
middle of the __init__ routine. It shouldn't be cleared out at the end
of the __init__ routine.
John Griffith [Thu, 23 Oct 2014 16:37:11 +0000 (16:37 +0000)]
Fix SolidFire inaccurate model on migrated vols
The general migration impl in Cinder works
by creating a new volume, transfering the data
from the original volume to the new volume, and
then deleting the original and flipping the ID
of the new volume.
Turns out we missed the fact that this creates a
mismatch between the volume Cinder will later ask
for and what the volumes identity is on the backend
device.
This change adds a check on create_volume at the drivers
level to see if it's part of a migration and is infact
going to get renamed. If so, just use the new name
and avoid all the headaches that come later with updating
provider auth and location.
The model info won't change in this case and is accessible
independent of the ID field in the Cinder base and the
crazy change that's going to take place on that value
in the Cinder DB.
abhishekkekane [Tue, 21 Oct 2014 09:31:15 +0000 (02:31 -0700)]
Eventlet green threads not released back to pool
Presently, the wsgi server allows persist connections hence even after
the response is sent to the client, it doesn't close the client socket
connection.
Because of this problem, the green thread is not released back to the pool.
In order to close the client socket connection explicitly after the
response is sent and read successfully by the client, you simply have to
set keepalive to False when you create a wsgi server.
DocImpact:
Added wsgi_keep_alive option (default=True).
In order to maintain the backward compatibility, setting wsgi_keep_alive
as True by default. Recommended is set it to False.
John Griffith [Tue, 21 Oct 2014 23:19:22 +0000 (23:19 +0000)]
Add ability to update migration info on backend
The current migration process creates a new volume,
xfr's it's contents, then deletes the original and
modifies the new volume to have the previous ID.
All in all this is kinda troublesome, but regardless
the bigger problem is that the generic impl doesn't
provide any method to tell the backend devices that
their device names/id's have changed.
This patch provides a method to inform backends
that a migration operation has been completed on
their target volume.
It shouldn't be necessary to do anything with the originating
or source volume because it's deleted as part of the process.
John Griffith [Fri, 24 Oct 2014 14:19:19 +0000 (08:19 -0600)]
Reserve 5 migrations for backports
Reserve 5 migrations incase the need arises to backport any
fixes that require a db migration in stable/juno.
We've never set this up in the past and we did run into a case
last cycle where we had to hack some things around to make it work
without the place holder.
Why 5? Why not? For as little as we touch the DB historically
this number should be more than sufficient.
Tomoki Sekiyama [Wed, 22 Oct 2014 22:30:06 +0000 (18:30 -0400)]
LioAdm: Delete initiator from targets on terminate_connection
In current LioAdm implementation, initiators are remained even if
terminate_connection is called. This keeps volumes exported to hosts
after instances attaching the volumes are live-migrated to another
host, which is not good for security. It also causes an error on the
migration back to the original host, because cinder-rtstool doesn't
update CHAP authentication if the initiator already exists.
With this patch, initiators are deleted on terminate_conection.
'initiator-delete' operation is added to cinder-rtstool.
It makes the following live-migration succeed.
Also, this adds unit tests for initialize_connection and
terminate_connection methods in LioAdm.
This patch allows an OpenStack environment to run as a secure NAS
environment from the client and server perspective, including having
root squash enabled and not running file operations as the 'root'
user. This also sets Cinder file permissions as 660: removing
other/world file access.
The "nas_secure_file_permissions" option controls the setting of file
permissions when Cinder volumes are created. The option defaults to
"auto" to gracefully handle upgrade scenarios. When set to "auto",
a check is done during Cinder startup to determine if there are
existing Cinder volumes: no volumes will set the option to 'true',
and use secure file permissions. The detection of existing volumes will
set the option to 'false', and use the current insecure method of
handling file permissions.
The "nas_secure_file_operations" option controls whether file
operations are run as the 'root' user or the current OpenStack
'process' user. The option defaults to "auto" to gracefully handle
upgrade scenarios. When set to "auto", a check is done during Cinder
startup to determine if there are existing Cinder volumes: no volumes
will set the option to 'true', be secure and do NOT run as the 'root'
user. The detection of existing volumes will set the option to 'false',
and use the current method of running operations as the 'root' user.
For new installations, a 'marker file' is written so that subsequent
restarts of Cinder will know what the original determination had been.
This patch enables this functionality only for the NFS driver.
Other similar drivers can use this code to enable the same
functionality with the same config options.
Tomoki Sekiyama [Mon, 20 Oct 2014 18:32:55 +0000 (14:32 -0400)]
TgtAdm: Don't change CHAP username/password on live migration
As tgtd doesn't update CHAP username/password while the initiator is
connected, CHAP username/password must not be changed while a Nova
instance are performing live-migration; otherwise the compute node
which the instance migrates to cannot login to the volume and the
migration process is aborted.
This fixes TgtAdm implementation not to regenerate random
username/password every time initialize_connection is called.
Also, it enables CHAP auth in unit tests of TargetAdmin helpers.
John Griffith [Fri, 17 Oct 2014 04:43:20 +0000 (22:43 -0600)]
Turn on Flake-8 Complexity Checking
Flake8 provides the ability to measure code complexity. There are
a lot of modules in Cinder that are considered "too complex", the
worst being "cinder/tests/test_huawei_hvs.py:110:1:" with a complexity
ranking of 59.
There's some outlyers at the higher end here, but the majority of the
code checks in at under 30, so let's make that our threshold and ignore
the two offenders that are above that for now.
Granted this may or may not be valuable, but it doesn't hurt to try it
and if we all hate it or find there's no value but it makes life difficult
we can always turn it back off.
See flake8.readthedocs for more info on flake8 and McCabe complexity
checking.
Matt Riedemann [Thu, 16 Oct 2014 15:39:07 +0000 (08:39 -0700)]
Log a warning when getting lvs and vgs takes longer than 60 seconds
We know something is causing lvs/vgs commands to block while deleting a
volume and this is causing Tempest to timeout while waiting for the
volume to be deleted. What we don't have right now is very good
(specific) logging when this happens, unless we get messages in syslog
for lvm tasks taking more than 120 seconds, but that doesn't always
happen when we see the volume delete timeout in Tempest.
This patch adds a check for when getting logical volumes and volume
groups takes longer than 60 seconds and logs a warning if that happens.
This is helpful in production also because the default interval for
periodic tasks is 60 seconds so having these take longer than that time
could cause periodic tasks to block up on each other and you'll get
warnings from the FixedIntervalLoopingCall in oslo which is controlling
the task runs.
Code Review / openstack-build / cinder-build.git / log