Zane Bitter [Mon, 21 Jan 2013 15:36:21 +0000 (16:36 +0100)]
ReST API: Return 400 for malformed JSON input
Previously we had an unhandled exception that resulted in a 500 (Internal
Server Error) result and a stack trace when an invalid JSON file was
submitted in a POST/PUT request. Change to catching the exception and
returning a 400 (Bad Request) result.
Steven Hardy [Thu, 17 Jan 2013 17:32:25 +0000 (17:32 +0000)]
heat engine : watchrule don't run rule for every SampleCount
We should not be triggering a run_rule (which is an alarm action)
for every datapoint when the SampleCount statistic is specified,
otherwise e.g IHA instances get replaced every heartbeat signal!
run_rule should only be called in response to a rule evaluate(),
or an explicit override via set_watch_state()
Steven Hardy [Thu, 17 Jan 2013 11:22:09 +0000 (11:22 +0000)]
heat templates : fix IHA HeartbeatFailureAlarm during instance build
The HeartbeatFailureAlarm triggers prematurely while the instance is
building, because cfn-init doesn't create the files (hence the
heartbeat cron-job) until after the packges are all installed.
If we make the HeartbeatFailureAlarm depend on the WaitCondition, it
won't be created until the instance is fully built.
Steven Hardy [Thu, 17 Jan 2013 11:19:05 +0000 (11:19 +0000)]
heat templates : IHA HeartbeatFailureAlarm should be Ref
cfn-push-stats reference to HeartbeatFailureAlarm should be a
Ref not a hard-coded string, or the engine fails to find the
watch because the name lacks the stack prefix
Zane Bitter [Thu, 17 Jan 2013 10:10:14 +0000 (11:10 +0100)]
Prepare the groundwork for more exception types
We have been abusing AttributeError and ValueError rather than raising
exceptions that actually describe the problem. This opens the way for
changes that will allow us to handle a wide variety of specific exceptions.
Zane Bitter [Tue, 15 Jan 2013 12:42:57 +0000 (13:42 +0100)]
Don't inherit from NestedStack
The NestedStack class is intended only to be the implementation of the
AWS::CloudFormation::Stack resource type. Other resources whose internal
implementation uses a nested stack should subclass the abstract
StackResource class.
Zane Bitter [Tue, 15 Jan 2013 12:42:56 +0000 (13:42 +0100)]
Move abstract nested stack class to separate file
The class heat.engine.resources.stack.Stack is intended to be an abstract
class which resources that are implemented as nested stacks may inherit
from. Rename it to StackResource and move it to the
heat.engine.stack_resource module, so that other resource modules
(including plugins) may include it without having to also import the
NestedStack class (which implements the AWS::CloudFormation::Stack resource
type).
Steven Dake [Tue, 15 Jan 2013 20:38:21 +0000 (13:38 -0700)]
Remove instance in ERROR state after failed create
If a VM was created but resulted in a nova error, the VM would enter
an error state and not be deleteable via heat delete. Instead
the VM would persist after the delete operation.
Steven Hardy [Mon, 14 Jan 2013 16:59:03 +0000 (16:59 +0000)]
heat engine : Implement Count property for WaitCondition
The Count property of WaitCondition is currently ignored, so
regardless of the number specified the WaitCondition resource will
always go to CREATE_COMPLETE on the first WaitConditionHandle signal.
This patch fixes this so the Count property is handled correctly.
Steven Hardy [Fri, 11 Jan 2013 13:43:52 +0000 (13:43 +0000)]
heat engine : Add metadata validation to WaitConditionHandle
Validate metadata before updating the WaitConditionHandle resource
metadata, if we raise a ValueError in response to incorrectly
formatted metadata then the CFN API will map this to a
HeatInvalidParameterValueError, resulting in a descriptive error
response to the malformed request
Change-Id: I6fde27066e9e0eaee97b59a7fbbace09ca620b48 Signed-off-by: Steven Hardy <shardy@redhat.com>
Abstract updates to resource metadata so they are implemented per-resource,
with a generic implementation which is a no-op, since currently the only
resource which needs metadata update via the RPC API is WaitConditionHandle.
This allows resource-specific metadata validation and update logic to be more
easily added.
Tomas Sedovic [Wed, 2 Jan 2013 16:34:23 +0000 (17:34 +0100)]
Add auth middleware for custom cloud backend
Custom backends will want to provide their own authentication mechanisms
instead of using the Keystone token or EC2-like systems we have in
place.
This adds a new middleware and paste pipeline for the custom backend
that will skip the normal authentication and queries the backend's
`authenticated(context)` method instead.
Since the backend is connected to the Engine whereas the auth middleware
is run in the API service (which may sit on a separate box and have no
access to the engine config or the custom backend itself), we add a new
RPC call that lets API verify the passed credentials.
Change-Id: I2fc4a19564b1e410adb79bd9266f6b6da07dd6c9 Signed-off-by: Tomas Sedovic <tomas@sedovic.cz>
Steven Hardy [Tue, 8 Jan 2013 19:21:56 +0000 (19:21 +0000)]
heat engine : DBInstance don't pass credentials to cfn-init
Don't create user/accesskey and pass them to cfn-init as resource
metadata for internal nested stacks is not available via the cfn API
so passing them causes an error in the engine logs (and creates some
unnecessary keystone credentials)
Ensure LoadBalancer resource deletes the underlying nested
stack when the owning stack is deleted - needs to inherit
from stack.NestedStack as this implements handle_delete()
Steven Hardy [Tue, 8 Jan 2013 18:10:22 +0000 (18:10 +0000)]
heat engine : DBInstance fix so nested stack is deleted
Currently the RDS DBInstance resource inherits from stack.Stack but it
should be stack.NestedStack, which implements handle_delete whereas
stack.Stack does not, meaning the DBInstance nested stack is never
deleted when the owning stack gets deleted.
Steven Hardy [Tue, 8 Jan 2013 15:47:55 +0000 (15:47 +0000)]
heat engine : allow NestedStack template validation to work
Nested stack templates (e.g WordPress_Composed_Instances.template)
will currently fail validation because FnGetAtt attempts to get
outputs for a stack which doesn't yet exist. So add a fallback which
returns a string "None" for Outputs.* which will allow template
validation to succeed, and will still return the actual output when
the nested stack has actually been created.
Steven Hardy [Tue, 8 Jan 2013 14:37:23 +0000 (14:37 +0000)]
heat engine : ensure create thread exits on stack delete
Detect GreenletExit exception when creating resources - this may
happen if the stack gets deleted while the stack is still in
CREATE_IN_PROGRESS state. If we don't detect this then the stack
continues trying to create resources during/after the stack has been
deleted, with predictably nasty results..
Jeff Peeler [Mon, 7 Jan 2013 17:36:20 +0000 (12:36 -0500)]
Handle different cloud-init versions gracefully
Change loguserdata script to python to allow easy detection of which
version of cloud-init installed. Some logging was added to
part-handler.
Took out injecting the command to touch provision-finished in the user
data. This is now handled in loguserdata.py.
Note that up until cloud-init version 0.6.0, the user data is not
passed to part-handler. This behavior is why it's not possible to log
the provisioning process with older versions. (Technically could rely
on the redirection support added post 0.6.0, but having a separate
file just for provisioning seems beneficial.)
The DBInstance nested stack template defines a DBSecurityGroups list parameter,
which should be "CommaDelimitedList" type, curently it's a "List" type which
breaks when the template is parsed. To make this work we also have to mangle
the property (passed in the top level template) format to match the parameter
format, because in AWS Lists are not represented in the same way for Properties and
Parameters (sigh..)
NoEcho is now implemented in the engine for parameters, so we
need to change the expected ParameterValues to '******' to match
the API response for the NoEcho==True parameters
Change-Id: I4f50633dc5358e8779bfbaf03dda2f7fe1b7d452 Signed-off-by: Steven Hardy <shardy@redhat.com>
Steven Hardy [Wed, 2 Jan 2013 17:19:17 +0000 (17:19 +0000)]
heat tests : convert functional tests to folsom glanceclient
Convert functional tests to work with folsom glanceclient, also
remove the conditional essex/folsom module import since this will
no longer work on essex (this was bogus anyway since this code can
never have worked on folsom)
Note : this breaks compatibility with essex for the functional tests, so
any platform running them must be running >= Folsom from now on.
(The essex/folsom glance client interfaces are too different to make it worth
supporting both for the functional tests, ref heat-jeos..)
Change-Id: I4463bfab7f8acf93b2eafa4bb702bf86d8c78652 Signed-off-by: Steven Hardy <shardy@redhat.com>
Zane Bitter [Fri, 21 Dec 2012 16:47:10 +0000 (17:47 +0100)]
CFN API: Fix DescribeStackResources with physical ID
DescribeStackResources failed when called with a physical resource ID
instead of a stack name. Also add a unit test to alert us to any future
breakage.
Zane Bitter [Fri, 21 Dec 2012 16:25:25 +0000 (17:25 +0100)]
Handle duplicate physical resources IDs
Because the unit tests create multiple stacks in different tenants with the
same physical resource ID, the database lookup code must change to
accomodate this.
This enables us to add a unit test for the describe_stack_resources RPC
API call for the case where we pass a physical resource ID to lookup.
Steven Hardy [Thu, 20 Dec 2012 15:37:29 +0000 (15:37 +0000)]
run_tests.sh fixup venv/novenv logic
The section which prompts y/n for creating a new venv can never be
reached with any combination of CLI options (since always_venv=0 and
never_venv=0 can never happen simultaneously)
This actually makes the --no-virtual-env option pointless, so we may
as well remove it and explicitly set it as the default (it was anyway
previously but by virtue of the never_venv option being undefined)
Change-Id: I3c803533fda55a6814b7536b5c1328312acd3d57 Signed-off-by: Steven Hardy <shardy@redhat.com>
Steven Hardy [Wed, 19 Dec 2012 17:27:49 +0000 (17:27 +0000)]
update tox.ini to pep8 latest (1.3.4)
Most of the core projects are currently 1.3.3, but since 1.3.4 is
the current latest it seemed best to align with the latest.A
Also add a check to run_tests.sh to ensure peoples local tests align
with the new gate version
Zane Bitter [Wed, 19 Dec 2012 18:31:32 +0000 (19:31 +0100)]
Split nested stack implementation into abstract and concrete
Split the Stack resource class into an abstract part from which other
resource types inherit and a concrete implementation of the
AWS::CloudFormation::Stack resource type.
Steven Hardy [Wed, 19 Dec 2012 18:01:15 +0000 (18:01 +0000)]
Avoid logging.getChild for python2.6 compatibility
Tweaking common.config setup_logging to avoid using the getChild()
convenience function means we can run on python 2.6, as getChild was
new in python 2.7
Change-Id: I03698ad09eb16d4f1004512d27b049f22b41c22c Signed-off-by: Steven Hardy <shardy@redhat.com>
Steve Baker [Fri, 14 Dec 2012 01:13:05 +0000 (14:13 +1300)]
A new documention structure, ready for contributions
Here are some skeletons for the documentation which will need to be written.
doc/source
A Sphinx based document which will be aimed at heat developers.
This also contains .rst files which generate the man files
doc/docbkx/api-ref
Docbook and WADL for the REST API, with the intent of moving this
to api-site to publish to api.openstack.org
doc/docbkx/heat-admin
Docbook manual targeted at Heat admins, with the intent of moving this
to openstack-manuals to publish to docs.openstack.org
doc/docbkx/heat-cli
Docbook manual targeted at users of the Heat CLI, with the intent of
moving this to openstack-manuals to publish to docs.openstack.org
Dude, wheres my man pages?
docs/man
is deleted, now generated into doc/build/man
Packaging will need to be updated to generate the man pages
Zane Bitter [Mon, 17 Dec 2012 18:56:23 +0000 (19:56 +0100)]
GettingStarted: Install python-pip package
pip-python is not installed by default, so we need to install it in order
to be able to use it. (An alternative would be to use easy_install, which
is packaged as part of setuputils and therefore already present, but it
lacks any of pip-python's package management capabilities.)
Zane Bitter [Mon, 17 Dec 2012 18:56:22 +0000 (19:56 +0100)]
Fix issues with deleting a WaitCondition
Previously we were ignoring the reference to the WaitConditionHandle stored
in the database by always setting it to None on load. This meant that we
had to try to recreate it on delete, which would fail if e.g. the
WaitConditionHandle had already been deleted.
Code Review / openstack-build / heat-build.git / log