Michał Dulko [Mon, 5 Oct 2015 15:03:39 +0000 (17:03 +0200)]
Block subtractive operations in DB migrations
To achieve rolling upgrades we need to make non-backward-compatible DB schema
migrations in a very specific manner that spans through 3 releases. In
particular we need to be very careful when dropping or altering columns
and tables.
This commit adds a test that blocks all the ALTER and DROP operations
from DB migrations. It allows to specify exceptions from this rule for two
purposes:
* Some DROP/ALTER migrations aren't subtractive (e.g. dropping a
constraint).
* When following the process we've designed for non-backward-compatible
migrations, we should be able to drop first unused columns or tables
in O release.
The test is based on similar one implemented in Nova.
note: Servers must be prepared to deal with multiple
OpenStack-API-Version headers. This could happen when a client
designed to address multiple services always sends the headers it
thinks it needs. Most Python frameworks will handle this by setting
the value of the header to the values of all matching headers,
joined by a ',' (comma). For example ``compute 2.11,identity
2.114``.
Gorka Eguileor [Thu, 3 Mar 2016 13:37:29 +0000 (14:37 +0100)]
Readd iscsi_target table
Rolling upgrades was broken when iscsi_target table was dropped on
https://review.openstack.org/268320
We cannot stop using a table and drop it in the same release for rolling
upgrades to work, we have to stop using it in one release and then drop
it in the next or in the post rolling upgrade mechanism (which is still
not in place).
So this patch fixes this by removing the dropping and adding another
migration that ensure that the table is really there. That way we can
be sure that anyone using M will have the table, which then will get
dropped in N.
Patrick East [Fri, 4 Mar 2016 05:47:22 +0000 (21:47 -0800)]
Fix issue with Pure drivers delete_snapshot exception handling
We were checking for only a single possible error that can occur when
the snapshot was missing. We now check for both which helps prevent
any snapshots or volumes deleted out from underneath Cinder to get
things stuck in an error state.
This also adjusts the warning message to be a little more descriptive.
Patrick East [Fri, 4 Mar 2016 00:15:49 +0000 (16:15 -0800)]
Add backend id to Pure Volume Driver trace logs
With multi-backend deployments it was very hard to follow which backend
was making which calls. When there was an error it was often unhelpful
to look at the tracing to know which backend had the problem.
This will now print out the active backend id with the tracing log
messages.
Kurt Martin [Wed, 2 Mar 2016 22:57:06 +0000 (14:57 -0800)]
Don't fail on clearing 3PAR object volume key
The 3PARs drivers write a key value pair on the 3PAR backend volumes
to track the instance that the volume is exported to. However, in
certain cases the key is not present and we would throw an
exception and not allow the detach to continue. If the key is not
present then we do not need to clear it. This patch will just
log a warning that it wasn't present and continue with the detaching
the volume.
Tom Barron [Wed, 2 Mar 2016 17:29:47 +0000 (12:29 -0500)]
Fix invalid uuid warnings in test_volume.py
Currently 'tox -epy27 -- -n cinder.tests.unit.test_volume'
emits 36 FutureWarnings from oslo.versionedobjects about invalid
uuids as documented here [1].
This commit changes this module to use valid uuids so that when its
tests are run these warnings are no longer emitted.
apoorvad [Wed, 23 Dec 2015 23:07:30 +0000 (15:07 -0800)]
Tintri image direct clone
Fix for the bug 1400966 prevents user from specifying image nfs
share location as location value for an image.
This broke Tintri's image direct clone feature.
This addresses the issue by using provider_location of image
metadata to specify image nfs share location.
When multipath is enabled, and the 3PAR driver is configured
to export a VLUN over N iSCSI IP(ports), we now use the same
LUN ID for each iSCSI IP export. Previously, when we would
export a volume over N iSCSI ports, we would get a new LUN ID
for each iSCSI port.
Gorka Eguileor [Thu, 3 Mar 2016 13:00:34 +0000 (14:00 +0100)]
Fix oslo.service config generation
Cinder generation configuration sample does not contain any of
oslo.service configuration options.
This is because we are not using the right namespace when calling
oslo-config-generator since oslo.service is not a valid entrypoint as
mentioned in the documentation:
Patrick East [Wed, 17 Feb 2016 04:38:22 +0000 (20:38 -0800)]
Allow for Pure drivers to verify HTTPS requests
This pipes in the config options (driver_ssl_cert_verify and
driver_ssl_cert_path) for verifying https requests to the
purestorage python module. This will allow for the underlying https
management API requests to the array to be verified.
To use this feature a newer (>1.4.0) version of the pure storage python
module will be required.
DocImpact: Need to update Pure Storage Volume Driver config reference
to mention that it can use the new config options and the python module
version requirements that go along with it.
Cinder v2 API allows creating a volume with a quoted (single or double
quotes or even unbalanced number of quotes) display name. But when we
try to get info for such volume, we end up getting an error message
saying that no volume with such a name or ID exists. This error is due
to the inadvertent stripping of quotes from the filter in the api layer.
The api call eventually comes to check_volume_filters() in
cinder/volume/api.py. The invocation of ast.literal_eval() inside this
method strips the quotes for certain quoted strings leading to this
incorrect filtering. ast.literal_eval() is used to convert string
representations into python objects which are then used to frame the
SQL queries in the db layer. For example, the string "[1,2]" for a
filter (not the display name filter) gets converted to a list object
and results in an "IN" operation being emitted in the SQL query as
opposed to an exact match.
When display_name does not contain any quotes or contains an unbalanced
number of quotes, then ast.literal_eval() throws (just like the Python
interpreter would throw for an unquoted string literal or one with
unbalanced number of quotes). We handle this by ignoring the exception
and using the raw input value as the filter and moving on. For string
containing balanced number of quotes, such as, '"foo"',
ast.literal_eval() succeeds and returns the input with the surrounding
quotes stripped (just like how the python interpreter strips quotes
from a string literal to initialize a string var's value in memory).
To always use the raw user input string as the filter value, we can
either not pass string inputs to ast.literal_eval() or encode the
string using encode("string-escape") so that we get the original string
back after passing through ast.literal_eval(). We choose the former as
the latter buys us nothing.
Tom Barron [Wed, 2 Mar 2016 02:43:17 +0000 (21:43 -0500)]
Don't run test_volume.VolumeTestCase twice
Currently the test_volume.VolumeMigrationTestCase class inherits
from test_volume.VolumeTestCase and therefore runs all the unit tests
in the former class in addition to the 29 tests defined in its
own class. This provides no gain in coverage and is costly since
the tests in VolumeTestCase are among the most expensive we have.
This commit refactors cinder/tests/unit/test_volume.py such that
VolumeMigrationTestCase inherits directly from BaseVolumeTestCase,
thereby reducing the number of tests run from test_volume from
557 to 412, and total execution time (on my system) from 335s to
202s. Only duplicate tests are removed.
Gorka Eguileor [Wed, 2 Mar 2016 13:21:31 +0000 (14:21 +0100)]
Fix test_create_volume_flow test issue
One of the tests in
cinder/tests/unit/volume/flows/test_create_volume_flow.py incorrectly
mocks time.time function which means that when there's a call to
time.time() it will raise an error because real time function doesn't
expect any arguments but the mock method does expect self as an
argument.
For some reason time.time function does not get called on every test
run, but when it does get called it will fail.
This patch fixes this by binding the mock method to the current self
instance.
Gorka Eguileor [Wed, 2 Mar 2016 12:23:31 +0000 (13:23 +0100)]
Fix test isolation issues related to versions
In cinder.rpc.RPCAPI methods _determine_rpc_version_cap and
_determine_obj_version_cap use global variables LAST_RPC_VERSIONS and
LAST_OBJ_VERSIONS to cache version values so we need to clear these
values between tests so one test values don't get passed to the next
test.
This was problematic for example when running some
cinder.tests.unit.api.contrib tests in a specific order:
Tom Barron [Sun, 21 Feb 2016 12:00:54 +0000 (07:00 -0500)]
Convert huawei ISCSIDriver unit tests to RFC5737 addrs
In the course an earlier review [1] it was suggested that
it would be good to change the live IPv4 addresses in these
tests to RFC5737 addresses [2], a range of addresses dedicated
for documentation, unit tests, and the like.
This commit makes these address substitutions without impacting
the function of the unit tests themselves.
Wilson Liu [Wed, 17 Feb 2016 04:48:17 +0000 (12:48 +0800)]
Huawei: Manage volume fails due to lower array version
When we try to import an existing volume to cinder, we try
to find whether the volume already exists in a hypermetro.
But if the array version is lower than V3R3, the request
to find the hypermetro info will fail because array version
lower than V3R3 does not support hypermetro.
LisaLi [Mon, 29 Feb 2016 07:12:18 +0000 (15:12 +0800)]
Fix exception during service update
The function objects.Service.get_by_args raises HostBinaryNotFound
exception, but in service update function, it catches ServiceNotFound
exception by mistake.
This patch is to remove HostBinaryNotFound and use ServiceNotFound to make
things simple and consistent.
Helen Walsh [Tue, 26 Jan 2016 23:12:37 +0000 (23:12 +0000)]
EMC VMAX - Limit SG and MV to 64 characters
VMAX restricts storage group and masking view names to a maximum
length of 64 characters. This patch only adds extra unit tests to
prove that both MV and SG names are below the 64 chars in all
scenarios.
Helen Walsh [Sat, 13 Feb 2016 23:26:47 +0000 (23:26 +0000)]
EMC VMAX - Recreating SG when it has been deleted
The default storage group for VMAX3 can be deleted when
the last volume has been removed from it. Certain
scenarios require a volume to be returned to the default
storage group.
Helen Walsh [Wed, 11 Nov 2015 21:52:18 +0000 (21:52 +0000)]
VMAX-Replacing deprecated API EMCGetTargetEndpoints
8.x version of Solution Enabler no longer supports
EMCGetTargetEndpoints InvokeMethod. Replacing with
an AssociatorNames call from protocol controller
to EMC_FCSCSIProtocolEndpoint. It will retrieve
the same information.
Alex O'Rourke [Wed, 3 Feb 2016 16:11:58 +0000 (16:11 +0000)]
3PAR: Update replication to v2.1
This patch updates replication to match the v2.1 spec. This makes it
so an entire backend can be replicated, and upon failover, all
replicated volumes will be failed over together.
Both sync and periodic replication modes are supported. Each
replication_device entry should have a replication_mode value set
to sync|periodic.
A volume type extra_spec value of replication:mode
should also be set. If replication:mode is periodic,
replication:sync_period should be set as well. Which replication_device
entry(s) are used is determined by the value of replication:mode set for
each volume type. NOTE: If no extra_specs are provided, periodic mode is
defaulted with a replication period of 900 seconds.
cinder.conf should have the replication config group:
Alex O'Rourke [Tue, 1 Mar 2016 00:42:16 +0000 (16:42 -0800)]
LeftHand: Update replication to v2.1
This patch updates replication to match the v2.1 spec. This makes it
so an entire backend can be replicated, and upon failover, all
replicated volumes will be failed over together.
cinder.conf should have the replication config group:
Patrick East [Fri, 5 Feb 2016 23:52:31 +0000 (15:52 -0800)]
Update Pure replication to cheesecake
Migration to cheesecake replication. This basically
modifies any of the existing replication code to work on the whole
backend instead of the specified volume.
It also completes the transition to allow swapping the underlying array
being managed by cinder upon failover. In the v2 implementation it was
conditional on some operations where things *could* be on the other
array. Now it is all driver operations.
This also switches the target_device_id to backend_id in the config for
replication devices.
Code Review / openstack-build / cinder-build.git / log