john-griffith [Tue, 4 Dec 2012 04:02:19 +0000 (21:02 -0700)]
Make copy_to_volume a bit more useful.
The current implementation of cinder.driver.copy_to_volume
does not take in to account image type and is only valid
for raw images. This doesn't do much good for the majority
of cases, and rather than require folks to have a raw version
uploaded to Glance, let's just do a convert if needed on the fly.
This simply creates a temp file, does the conversion and then
writes the temp file directly to the volume.
John Griffith [Thu, 6 Dec 2012 20:36:23 +0000 (13:36 -0700)]
Delete type call in api needs update to use ID.
https://github.com/openstack/cinder/commit/51a438c8f3f0d13c6779602264d72cde82feab9c
Introduced UUID's for volume_type ID and also changed delete to use ID
instead of name.
That changed neglected to update the call in contrib/types_manage,
this patch fixes that.
john-griffith [Sun, 2 Dec 2012 07:01:49 +0000 (00:01 -0700)]
Convert volume_type id from int to uuid.
This converts the volume_type id from int to uuid. In
addition, this also corrects an issue of deleting volume_types
by name. Even though the client/api is taking the id, it was
converting to the name of the volume_type for the db access.
We also want to continue enforcing that the name is unique on creation
so as to avoid any confusion and for clarity in Horizon. The api
has also been enhanced to allow you to specify the type by name
or by uuid.
This does NOT modify the things like the volume details display field
(currently shows the type 'name' not 'uuid'), these types of changes
will go in to V2 of the API.
Russell Bryant [Fri, 30 Nov 2012 14:19:05 +0000 (09:19 -0500)]
Ensure request_spec can be serialized.
request_spec includes some objects that must be converted to primitive
types to ensure they can be serialized by any of the rpc drivers. In
this case, it's the volume type. It's a sqlalchemy model.
to_primitive() will ensure that it gets converted to a dict so that it
can be serialized for messaging.
I saw this error in a smokestack log that was running tempest with qpid.
You won't see this error if you're using rabbit with the kombu driver.
We have some magic with the kombu driver that will fix these problems
automatically if they are missed.
Mike Perez [Thu, 29 Nov 2012 11:18:08 +0000 (03:18 -0800)]
attaching volumes will set instance_uuid instantly
This change allows the instance_uuid field to be set instantly when
beginning an attach action. The user will no longer have to wait for the
attachment to be complete to know what it is attaching to. If attaching
fails on the driver layer, we got error state and keep the instance_uuid
set for troubleshooting purposes.
John Griffith [Wed, 28 Nov 2012 19:19:14 +0000 (12:19 -0700)]
Revert changes to monkey_patch.
As a work-around to deal with bug 1075838, we modified
monkey_patch in the Cinder binaries. The root cause of the
issue is/was actually a bug in eventlet but this was a quick
work-around.
Since the change Change-Id: I69ba75136e32e95c6fdf108f0c4fe21a5e3bdbe6
to do this landed, the problem has also been addressed in Keystone.
So, let's revert this patch to get back the greenthread performance
that we may have lost with this change. For now, everything should
be addressed as long as we don't implement multi api worker in Cinder,
if we do and the evenlet fix hasn't been released we'll need to turn
the modification back on for bin/cinder-api.
Adam Gandelman [Wed, 28 Nov 2012 01:51:26 +0000 (17:51 -0800)]
Improve provider_location cleanup code for RBD.
The RBD driver does not make use of the 'provider_location' field
but the current cleanup code assumes it does. Ensure the field
is in use before testing whether or not it needs fixing.
Matthew Treinish [Tue, 27 Nov 2012 15:41:25 +0000 (10:41 -0500)]
Fix None being passed into as_int().
Currently when a create is received and the size is of NoneType
a stack trace is dumped because of the TypeError. This will
correct it so the functionality won't change, but the TypeError
is will be caught now.
Yaguang Tang [Tue, 27 Nov 2012 09:06:25 +0000 (17:06 +0800)]
Use auth_token middleware from keystoneclient.
auth_token middleware has been move from keystone to python-keystonecient,
so add python-keystoneclient dependence instead of keystone.Also adds
missing signing_dir option.
John Griffith [Tue, 27 Nov 2012 05:46:58 +0000 (05:46 +0000)]
Revert "use O_DIRECT when copying from /dev/zero too"
It turns out running oflag=direct against the LVM snapshot fails for an IO error every time. Works fine against the regular volume, but fails on snapshot.
John Griffith [Sun, 25 Nov 2012 03:17:32 +0000 (20:17 -0700)]
Make pep8 checks a bit stricter.
Along with moving to pep8 1.3.3, we also want to standardize
on what we ignore. This patch get's us most of the way there
by setting the ignore list to:
N4,E125, E126, E711,E712.
Almost all changes made here are white-space/indentation changes.
The removal of Hacking N4 errors from the ignore list will
be handled in a seperate patch.
Pádraig Brady [Fri, 23 Nov 2012 11:24:44 +0000 (11:24 +0000)]
use O_DIRECT when copying from /dev/zero too
We need to avoid trying O_DIRECT with virtual
devices like /dev/zero and /dev/urandom etc.
as it's not supported there, which because of
our later check, would cause O_DIRECT to be
not applied to the output device either.
Related to bug 937694
Possibly related to bug 1023755
Change-Id: I034d8345c3d00689c1f1fffcc2c110735c49ee01
john-griffith [Thu, 22 Nov 2012 16:44:53 +0000 (09:44 -0700)]
Add CONTRIBUTING file.
If the CONTRIBUTING[.md] file exists, Github will show a link to it to
anyone who files an issue on Github or opens a pull request on
Github. This files points folks to the OpenStack wiki page which
provides the proper instructions on how to contribute to OpenStack.
See also <https://github.com/blog/1184-contributing-guidelines>.
Mike Perez [Sun, 4 Nov 2012 08:34:27 +0000 (01:34 -0700)]
Moving common api code into cinder.api
This is progress for beginning to have separate versions of the
cinder api. Common code that would be shared by v1, v2 etc will use
these modules, so we'll just move them to a common place. Test directory
structure has also been updated.
Moved all the remaining drivers under
cinder.volume.drivers. Updated the tests with
the new module paths and add backwards compatibilty
for the existing paths.
Mike Perez [Mon, 19 Nov 2012 05:27:39 +0000 (21:27 -0800)]
Add volume bootable information to api response
If there is any glance metadata associated with a volume, we'll consider
the volume bootable. This will return the information for both /volumes
and /detail calls.
Mate Lakat [Mon, 5 Nov 2012 09:57:52 +0000 (09:57 +0000)]
Add XenAPINFSDriver
Related to blueprint xenapi-storage-manager-nfs
As xensm was not pulled to Cinder, this change shows an implementation
example for using an NFS export for volumes. This change contains the
volume create and delete functions, as a minimal set of functionality.
The connection information structure is compatible with the previous
xensm implementation.
Activate the driver with the following configuration entries in
cinder.conf:
Ollie Leahy [Thu, 15 Nov 2012 11:48:27 +0000 (11:48 +0000)]
Add db table for Glance Metadata
This commit implements the blueprint
https://blueprints.launchpad.net/cinder/+spec/retain-glance-metadata-for-billing
It creates the new table volume_glance_metadata in the cinder
database, provides the CRUD methods for it, and populates the table
when a volume or snapshot is created from a Glance image.
Patch set 2: remove superflous line
Patch set 3: Fix incorrect column types in sqlalchemy/models.py
Patch set 4: Define exception class GlanceMetadataExists
Zhiteng Huang [Tue, 20 Nov 2012 17:02:37 +0000 (01:02 +0800)]
Remove redundant db.volume_update() in volume manager create_volume()
Since scheduler has already updated db to set the 'host' field and
recorded 'scheduled_at' time, it's not necessary to do db.volume_update()
again in volume manager create_volume() method. Also let volume api
to update db when creating volume from snapshot while snapshot_same_host
is true, thus the behavior is consistent (updating db when placement
decision is made).
John Griffith [Tue, 20 Nov 2012 00:50:27 +0000 (00:50 +0000)]
Pass in correct volume_ref to create_from_snapshot.
The new rpc versioning change introduced a bug in the
create_from_snapshot code. The originating volume_ref
was being passed in instead of the newly created volume_ref
as expected.
John Griffith [Tue, 13 Nov 2012 19:03:50 +0000 (19:03 +0000)]
Add hosts extension to Cinder.
This adds the hosts extension to Cinder which is similar to
the extension by the same name in Nova.
This extension allows an admin user to check the status of
Cinder services on different hosts. In addition it also
provides the capability to get host and project specific information
regarding volume count and space allocated for volumes on a specified
host.
Dan Prince [Thu, 15 Nov 2012 16:27:06 +0000 (11:27 -0500)]
Make tox.ini run pep8/hacking checks on bin.
This updates the pep8 checks in our tox.ini file so that
we are also scanning the bin directory.
Additionally, it updates the main pep8 check so that it scans
cinder properly as well. Previously there were a bunch of files
getting skipped due to some of the pep8 pattern matching.
Zhiteng Huang [Fri, 9 Nov 2012 06:37:21 +0000 (14:37 +0800)]
Remove unused volume API method - remove_from_compute()
remove_from_compute() method in cinder.volume.api is never used
by others and it is also broken since the interface in Nova has
been changed to remove_volume_connection(), thus it should be
removed from Cinder.
Zhiteng Huang [Sat, 6 Oct 2012 17:11:41 +0000 (01:11 +0800)]
Scheduler API clean up and refactor
Unlike Nova scheduler, which has to consider serving compute and volume
scheduling, Cinder scheduler only serves volume scheduling, so there's no
need to keep generic interface 'schedule'. Instead, 'schedule_create_volume'
is added (if missing) to manager/driver class and chance/simple scheduler
driver implementation.
Also this patch changes the interface between API service and scheduler to
allow more information about volume is passed to scheduler for advanced
scheduling while maintained backward compatibility. And this change bumps
scheduler RPC API to version 1.2.
John Griffith [Tue, 6 Nov 2012 18:08:58 +0000 (11:08 -0700)]
Remove dm_setup(remove) call in volume_delete.
While it makes sense to perform the dm_setup(remove) before
performing the lvm remove, it seems to cause some buffer IO
errors as evident by kern.log.
I'm also a bit suspicous in that it seems the delete timeouts
we see seems to have increased in frequency with this addition.
I propose this step is removed until we completely understand the
impact of the kernel error on Precise kernels.
john-griffith [Tue, 6 Nov 2012 06:26:07 +0000 (23:26 -0700)]
Add ability to disable secure volume delete.
Due to gating issues caused by secure delete on LVM volumes,
add the ability to disable via flag.secure_delete.
Default will be set to True, but we add a way for the gate tests
to disable this until we can come up with a fix for
https://bugs.launchpad.net/cinder/+bug/1023755 OR implement an alternative
secure delete method that doesn' suck.
Rushi Agrawal [Sat, 3 Nov 2012 06:51:28 +0000 (12:21 +0530)]
Fix run_tests.sh ambiguous usage msg and behaviour for -x option
In commit
https://github.com/openstack/cinder/commit/ 50792c2ff41fdc7e73b3e3c51f4510366debfd14,
the committer wanted to add an option in run_tests.sh for generating XML
coverage report, but ended up accidentally overriding the default nose
option -x. But, the usage message still showed that using option -x will
'Stop running tests after the first error or failure'. So, I changed the
option to generate XML report to -X (capitalized 'x'). So now both -x and
-X options are available, with different purposes.
clayg [Thu, 25 Oct 2012 09:55:13 +0000 (02:55 -0700)]
Add admin only action for force detach
This action calls the same methods nova would after it successfully
detaches a volume. By exposing it to the administrator it makes it's
easier to repair un-syncronized state between services. Generally when
the host is no longer attached, but the volume state is wrong.
Future work: The Iscsi based drivers don't seem to use
initialize_connection and terminate_connection to create the export for
the volume. This would be more useful with drivers that do that.
I added the force parameter to terminate_connection for drivers that may
want to differintiate between a normal terminate and the force detach.
Future Nova work: Nova will want an admin action to update the bdm
tables - today it's a bit of nova-manage shell work.
The openssl popen call to verify the token returns an error code
of 2 when a file doesn't exist. Normally this is caught and the
relevant file is downloaded, but monkeypatching os causes this
error code to be eaten and cinder to return a 401. Note that this
is likely a race condition that depends on exactly when greenthreads
are switching that is causing the return code from Popen to get
overwritten.
It is possible that this is exposing an underlying bug in eventlet
but the workaround is simply to make sure that we don't monkeypatch
os.
Moved RBDDriver, SheepdogDriver into their own respective driver
files under cinder/volume/drivers/*. Moved FakeISCSIDriver and
LoggingVolumeDriver into a fake driver file under the tests,
since that's the only place where those are used.
Minor optimization in create_volume in HpSanISCSIDriver
Removing extra call to getClusterInfo in HpSANISCSIDriver
on every create_volume call. The cluster info is now
being lazily stored in a local variable.
Clay Gerrard [Wed, 31 Oct 2012 20:25:55 +0000 (15:25 -0500)]
Return volume type name on volume create
Update openstack api controller for volumes to fully inflate the
returned volume model on create to populate the volume_type fields so
the response serializer will return the name of the volume_type instead
of the id on volume create requests. Returning the name on create is
consistent with the show and index commands.
Sean Dague [Wed, 31 Oct 2012 14:10:29 +0000 (10:10 -0400)]
pin sqlalchemy to the 0.7 series
sqlalchemy 0.8 beta has hit mirrors, and changes dependencies, thus
breaking jenkins. This pins it to 0.7 series until all the projects
agree to move forward to 0.8.
Brian Waldon [Tue, 30 Oct 2012 22:06:12 +0000 (15:06 -0700)]
Add VolumeTenantAttribute API extension
Expose the ID of the project to which a volume belongs through a
new API extension. This is only presented to Admins by default.
Visibility can be controlled with the 'volume_tenant_attribute'
policy rule.
The inability to deduce an /action request from the logs
makes diagnosing integration with nova difficult. This change
logs the action method and arguments
Code Review / openstack-build / cinder-build.git / log