John Griffith [Thu, 30 May 2013 22:05:45 +0000 (16:05 -0600)]
Fix config registration in cinder volume drivers.
The config documentation relies on options being registered
on a modules import. Our need to move the drivers to using
self.configuration for multi-backend support means that options
wouldn't be loaded until object initialization which breaks
documentation.
This patch puts a dummy CONF init/load back in the drivers. While putting
this change together I came across a number of drivers still using FLAGS,
and even worse a number of drivers using a mixture of FLAGS and CONF and
self.configuraiton. So most of those are cleaned up here as well.
Note there are two drivers that were not updated at all here:
1. windows.py
2. zadara.py
The zadara folks have indicated that they're in the process of updating and
releasing a new version of their driver so I left that as is.
The windows driver needs a bit of work to switch over.
John Griffith [Tue, 4 Jun 2013 18:59:20 +0000 (12:59 -0600)]
Elevate acceptors context on accept reserve udpate
When accepting a transfer we need to update the
quota data for both the donor and the acceptor's.
The acceptor is easy enough (as that's the context
we currently are using) however to update the donor's
quota data we need an elevated context.
John Griffith [Tue, 4 Jun 2013 19:02:37 +0000 (13:02 -0600)]
Add call to vol driver when accepting a transfer
Some back-ends have the notion of tenancy on volumes
and set ownership in the driver. The update of the DB
alone when doing volume transfers isn't enough, we need
to be able to propogate the update info all the way
down to the driver level.
This patch introduces the volume api/manger/rpc calls
and implements a stub in the base driver that can
be over-ridden for those that need to take some action.
Sergey Vilgelm [Mon, 3 Jun 2013 09:12:32 +0000 (13:12 +0400)]
Replace FLAGS with cfg.CONF in db
Replace all the FLAGS with cfg.CONF in cinder/db
Large commit "https://review.openstack.org/31172" was split into several parts
Use the common pattern: CONF = cfg.CONF
Jay S. Bryant [Mon, 3 Jun 2013 15:01:58 +0000 (10:01 -0500)]
Re-write of the cinder-manage man page.
The cinder-manage man page didn't appear to have been re-written
since splitting from Nova. There were numerous options that
no longer exist documented and references to Nova instead of
Cinder.
This version has been re-written based on the current code in
cinder-manage.
Sergey Vilgelm [Mon, 3 Jun 2013 13:36:42 +0000 (17:36 +0400)]
Replace FLAGS with cfg.CONF in scheduler
Replace all the FLAGS with cfg.CONF in cinder/scheduler
Large commit "https://review.openstack.org/31172" was split into several parts
Use the common pattern: CONF = cfg.CONF
Dirk Mueller [Sat, 1 Jun 2013 12:25:37 +0000 (14:25 +0200)]
Handle IPv6 specifid glance servers gracefully
IPv6 netlocs can for valid reasons contain ':', so
splitting by ':' is dangerous. Instead of splitting
by host and port in order to only reassemble it into
a netloc again later, simply always pass netloc, which
avoids the bug.
Add extra textcases to verify the new behavior.
Fix Flake8 warnings.
Lakhinder Walia [Tue, 7 May 2013 23:05:46 +0000 (16:05 -0700)]
HDS Cinder Driver. Rev #1
blueprint hds-hus-iscsi-cinder-driver
This is the first rev of Hitachi Data Systems Cinder iSCSI driver.
This driver works with HUS (df850) array.
This driver contains all the base-line features specified for Havana release.
Amended into this submission are changes from code-reviews.
Anton Frolov [Fri, 31 May 2013 10:27:49 +0000 (14:27 +0400)]
Add error reporting to generate_sample.sh on import failure.
Currently tools/conf/generate_sample.sh collect options from project
modules by importing each module and looking at it's global variables
for instances of classes derived from cfg.Opt (or lists of such
instances). Sometimes this import can fail due to various reasons,
cfg.DuplicateOptError for example. And if import fails no options
from imported module written to sample config file. Error reporting
will at least warn user that some of options didn't get into sample
config.
Mike Perez [Fri, 31 May 2013 07:11:24 +0000 (00:11 -0700)]
Prevent force delete if the volume is attached
Force deletes were eventually failing on the volume manager layer due to
being in an attached state. This will check that up front to inform the
user that they need to detach first.
Zhenguo Niu [Wed, 29 May 2013 08:50:03 +0000 (16:50 +0800)]
Rename requires files to standard names.
Rename tools/pip-requires to requirements.txt and tools/test-requires
to test-requirements.txt. These are standard files, and tools in the
general world are growing intelligence about them.
Josh Durgin [Wed, 22 May 2013 00:49:02 +0000 (17:49 -0700)]
rbd: simplify configuration and use librbd and librados
Add an rbd_ceph_conf options to mirror glance configuration, and use
the existing rbd_user option to choose how to connect to the cluster
instead of relying on an environment variable. Use these settings
when running command line programs and when connecting via librados.
Use absolute imports so that importing the python librbd bindings
via 'import rbd' does not try to import cinder.drivers.rbd again.
Create some convenience wrappers to simplify librbd and librados
error handling and cleanup. Using these everywhere also simplifies
testing. Mock out all the librados and librbd calls in the tests
so these libraries don't need to be installed.
Remove the local_path() method since it's never used. It was
left over from nova-volume.
There are only three things still relying on the command line:
- importing an image
- exporting to an image
- getting monitor addresses
Importing and exporting on the command line include zero-detection
that would be little benefit to replicate here. librados and librbd
don't have a simple interface to obtain the monitor addresses, so
leave that to a command line tool as well.
The 3PAR REST API server has a limited number of active sessions.
This change to the driver removes the ability of keeping the
REST API session alive for the life of the driver. Now on every
driver entry point, which is synchronized, we login to the 3PAR
and when the work is complete, we log out of the 3PAR. This has
the side affect of more overhead to the 3PAR but helps eliminate
failed commands due to maximum sessions reached to the 3PAR.
This patch also changes the locking to be the same lock for all
driver entry points and adds locking around the volume stats.
This patch also migrates all of the client code access to the common
class, which simplifies the drivers.
If, when using a Ceph as a backend for Cinder and Glance, you
create a volume from a Glance image, the request is actioned
in one of two ways; (1) if Glance does not supply sufficient
information to Cinder, a full copy is made of the Glance
image or (2) Ceph uses it's copy-on-write capability to do a
clone. If a full copy is done, the volume is marked as
Bootable=True but if a clone is done it is not.
This does not appear to break nova boot from volume at present
but the status displayed by the cinder client is incorrect.
Anton Frolov [Tue, 28 May 2013 15:34:41 +0000 (19:34 +0400)]
Support for NFS shares with spaces in path.
Unescape share address string listed in nfs_shares_config so that
'x.y.z.w:/foo\040bar' would become 'x.y.z.w:/foo bar' to support
mounting shares with spaces in name.
Use stat command to get available and total space on device instead
of df since it's output is easier to parse.
Jim Branen [Fri, 24 May 2013 16:30:10 +0000 (09:30 -0700)]
Fixes 3PAR Host already exists error.
Fixed “Host already exists” error by,
* Screen scraping error message and extracting the host name
used by the 3PAR backend.
* Cache the host name used by the 3PAR backend for subsequent
methods using host name.
* After a restart, and 3PAR host name it no longer available,
retrieve 3PAR host name using wwn/iqn, if a terminate_connection
fails.
Jay S. Bryant [Mon, 20 May 2013 20:02:11 +0000 (15:02 -0500)]
Ensure that pbr>=0.5.10 is installed
A bug in PBR levels less than 0.5.10 causes the
/usr/lib/python2.X/site-packages/cinder/db/sqlalchemy/migrate_repo/migrate.cfg
file to not be properly installed. This causes 'cinder-manage db sync'
to fail due to the missing file.
This change sets the required PBR level to >=0.5.10,<0.6 to avoid the problem.
In the case that user already has a 0.5.X level of PBR installed that is less
than 0.5.10 it should be updated rather than encountering issues as our
test system did.
Avishay Traeger [Fri, 24 May 2013 08:19:51 +0000 (11:19 +0300)]
Remove execute permissions from test files.
Remove execute permissions from test_volumes.py and
test_storwize_svc.py, which caused the tests defined in them
to be skipped.
Fixing the permissions caused test_create_volume_with_volume_type to
execute, which fails. The cause of the failure is that
(1) LoggingVolumeDriver doesn't implement get_volume_stats(), and
(2) The fake RPC doesn't pass the statistics to the scheduler for
some reason.
Because this test doesn't test the scheduler, just change to use the
simple scheduler and work around the issue.
Roman Podolyaka [Thu, 23 May 2013 14:55:10 +0000 (17:55 +0300)]
Migrate to Oslo DB code
- engine and session handling code is reused
- integrity errors caused by duplicating entries
are handled properly
- the base class for models is mostly reused
(the only exception is SoftDeleteMixin, which
requires changes to existing DB schema; this
will be implemented in BP db-unique-keys)
- use common exceptions handling code
John Griffith [Mon, 27 May 2013 20:04:56 +0000 (20:04 +0000)]
Catch and report errors from copy image to volume.
The copy image-to-volume errors weren't being handled
properly and the result was the lvcreate being retried
even though the lvcreate itself succeeded.
The result of this was misleading errors stating that
the volume couldn't be created because it already existed
(which it did, becuase the create itself was succesful).
Adam Gandelman [Tue, 28 May 2013 21:34:30 +0000 (14:34 -0700)]
test_glance.py: Stub out _get_member_model as well.
glanceclient's v2 client needs to also have its _get_member_model()
stubbed out as well, to avoid attempts to reach 'fake_host' during
client instantiation.
Update: Only stub when _get_member_model() when required.
Josh Durgin [Sat, 25 May 2013 01:18:30 +0000 (18:18 -0700)]
rbd: send ceph monitor addresses with connection info
Previously we relied on a ceph configuration file on the compute host
for this information. Sending the info directly from cinder makes more
complex setups with multiple ceph clusters talking to the same compute
hosts possible.
Refresh the monitor addresses for each initialize_connection() call,
since monitors may be added or removed while cinder-volume is
running.
Xing Yang [Tue, 28 May 2013 04:28:37 +0000 (00:28 -0400)]
Remove cinder_emc_config.xml.sample
Remove this sample config file from emc folder based on discussions
at the IRC meeting. Parameters specified in this sample file are
already documented in Cinder Admin Guide.
John Griffith [Sat, 25 May 2013 23:52:30 +0000 (17:52 -0600)]
Update cloned volumes QoS settings.
The SolidFire device inherits all settings (including QoS) from
the parent on a clone. This meant that although we were expecting
QoS to be updated when a clone was issued with a different Volume
Type than the parent volume it was never acted upon.
This change fixes that bug by simply adding the *new* qos settings
to the ModifyVolume call after cloning.
A warning message appears in Cinder logs which state that
inheritance-based rules are deprecated and use default
Brain instead of HttpBrain. It looks like HttpBrain class
doesnt have any difference from Brain class, so changing
all the instances of HttpBrain to Brain shouldn't affect
anything and removes this warning from logs.
Digging further, I found out that the policy engine has
gone through a major rework in Oslo and Nova, and I think
at some point of time in future, we will also be moving
to use the latest policy engine code. In that way of
thinking, I guess this fix is acceptable for now.
Reference: https://review.openstack.org/#/c/14122
Changed the brain from HttpBrain -> Brain, which makes the
warnings in the log disapper. A more comprehensive work will be
to pull the latest policy change code from Oslo.
Mukul Patel [Sun, 26 May 2013 18:12:06 +0000 (23:42 +0530)]
Added '%' before snapshot variable
In function create_snapshot for cinder volume, '%' was missed
before variable snapshot from message defination. It was causing
an error during translation of debug message. Included '%'
for getting proper debug log.
John Griffith [Fri, 24 May 2013 19:46:28 +0000 (13:46 -0600)]
Hack run_tests.sh to work with single tests again.
Even though we're moving to testr and this will eventually
go away, we should maintain our ability to run and debug
individual tests until the broader change is released.
This just adds some text around the nosetests call
to get us back where we were.
Eric Harney [Fri, 24 May 2013 15:44:13 +0000 (11:44 -0400)]
Don't throw ValueError for invalid volume id
If a user runs something like "cinder-manage volume delete a1234",
a ValueError is thrown because it fails to cast to int.
Catch this and treat the parameter as an id which will result in a
later VolumeNotFound error rather than breaking this way.
John Griffith [Fri, 24 May 2013 04:57:57 +0000 (22:57 -0600)]
ModifyVolume attributes on Clone.
Clones copy over the attributes from the source volume,
we want to keep some admin data like source-volid and if
it's a clone or not. To do that we just need to call modifyvolume
after the clone operation and update this info.
Code Review / openstack-build / cinder-build.git / log