Zane Bitter [Fri, 30 Nov 2012 10:05:41 +0000 (11:05 +0100)]
Create a new heat.rpc package
The definition of the RPC API is common between the engine (server) and the
clients, so move the client and common definitions out of the heat.engine
package.
Steven Hardy [Fri, 23 Nov 2012 10:41:03 +0000 (10:41 +0000)]
heat engine : subclass keystone client to encapsulate common code
Encapsulate the keystone client in a heat-specific wrapper subclass
so we can put heat-specific implementation related to keystone in
one place. This will allow easier reuse of common code between
resources which need to manipulate stack users and ec2 keys
blueprint metsrv-remove
Change-Id: I3d9751023c52cb75ab5e1f62415b1db4e4361dec Signed-off-by: Steven Hardy <shardy@redhat.com>
Zane Bitter [Thu, 29 Nov 2012 19:51:01 +0000 (20:51 +0100)]
Process engine config before initialisation
Don't import the heat-engine service module until the configuration options
have been initialised. This will allow us to read the configuration during
initialisation of the engine.
Steven Hardy [Thu, 22 Nov 2012 17:11:15 +0000 (17:11 +0000)]
heat api : add waitcondition to cfn api
Add a new path to the CFN api, which implements CFN style waitcondition
notification - this means we can move away from the separate heat-metadata
service for this purpose, instead using the authenticated CFN API
blueprint metsrv-remove
Change-Id: I62cff7cb4c7a009fea2c8f62ea446d8d758f5429 Signed-off-by: Steven Hardy <shardy@redhat.com>
Zane Bitter [Tue, 27 Nov 2012 14:13:55 +0000 (15:13 +0100)]
Lookup the class for an Instance in Autoscaling
Instead of hardcoding the class for handling instances in autoscaling,
look up the class for the AWS::EC2::Instance resource type using the same
mechanism used for creating resources normally.
Zane Bitter [Tue, 27 Nov 2012 14:38:44 +0000 (15:38 +0100)]
Distribute resource mapping to individual modules
Define the mapping from resource names to classes locally in each module
and then aggregate them. This moves the mappings near the definitions, and
provides the format for an eventual plug-in resource architecture.
Zane Bitter [Wed, 28 Nov 2012 13:43:40 +0000 (14:43 +0100)]
Move the resource module out of the resources package
Leave only the actual resource implementations in the heat.engine.resources
package. This is the first step to implementing a plugin-style architecture
for resources.
Steven Hardy [Mon, 26 Nov 2012 13:51:03 +0000 (13:51 +0000)]
pip-requires fix boto version
Boto version added to pip-requires is wrong - we need a version containing
the commit discussed under https://github.com/boto/boto/pull/742, which is
only included in boto>=2.4.0
Change-Id: Iaaa61f306d4035e63ca3f7a0eafe2d5715585f16 Signed-off-by: Steven Hardy <shardy@redhat.com>
Steve Baker [Sun, 25 Nov 2012 23:52:47 +0000 (12:52 +1300)]
Module for converting JSON to YAML, and parsing both
convert_json_to_yaml is a utility function used for tests and file conversion
utilities.
parse_to_template will take any string, infer the format, and parse to a
python structure. Currently it assumes the file is JSON if it starts with '{'
otherwise it attempts to parse it as YAML.
Steve Baker [Mon, 26 Nov 2012 01:34:38 +0000 (14:34 +1300)]
Add packages and versions in response to grizzly-1
- Paste is now PasteDeploy
- boto is now an explicit requirement
- python-glanceclient replaces removed glance client
- added PyYAML for new YAML format
Zane Bitter [Thu, 22 Nov 2012 10:06:56 +0000 (11:06 +0100)]
Improve the identifier class for Resources
Enable the resource name to be retrieved from an object. Also allow a
ResourceIdentifier to be constructed from only the data passed through the
RPC API. This means we can use a single format for identifiers, but still
be able to extract important information such as the stack identifier and
resource name.
Steve Baker [Wed, 21 Nov 2012 19:15:05 +0000 (08:15 +1300)]
Get db session from the context.
The aim is to use a single sqlalchemy session for an RPC request.
The context object passed to EngineAPI methods is actually an RpcContext
which contains the same data as the RequestContext. The @request_context
decorator turns this back into a RequestContext which can now have other
behaviours added to it.
RequestContext now has a lazy loaded session attribute.
Save calls on created entities need to be passed the shared session.
Steven Hardy [Wed, 21 Nov 2012 11:34:51 +0000 (11:34 +0000)]
heat engine retrieve credentials every periodic task interval
Avoid saving a session context, instead retrieve the DB stored
credentials for every periodic task trigger, and create a new
session context for each periodic event
Change-Id: I24ca70b478f06be26ca6cc74385b37d1a0cf098d Signed-off-by: Steven Hardy <shardy@redhat.com>
Steven Hardy [Wed, 21 Nov 2012 11:31:14 +0000 (11:31 +0000)]
heat engine Add admin flag to dbapi stack_get
Add a flag to the stack_get dbapi call, defaulted to False,
which allows us to specify that the admin context is being used
hence we don't want tenant-scoping condition applied.
This is needed to allow the admin context to retrieve stored
credentials per-stack (e.g for the periodic per-stack tasks)
Change-Id: I55e307b7940f7da13bd169271744e80d95ea0bd9 Signed-off-by: Steven Hardy <shardy@redhat.com>
Steven Hardy [Mon, 19 Nov 2012 22:23:24 +0000 (22:23 +0000)]
heat engine allow WatchRule load() from DB object
Allow WatchRule.load to initialize an object based on an existing
sqlalchemy DB object, similar to parser.Stack, avoids doing two
DB lookups when running the periodic watch tasks
Change-Id: I280942b81e5fdee17acf5582e3d319045afa7914 Signed-off-by: Steven Hardy <shardy@redhat.com>
Steven Hardy [Mon, 19 Nov 2012 16:55:23 +0000 (16:55 +0000)]
heat engine move to per-stack periodic watch threads
Use the stack thread groups, so a separate watch thread is started
for each stack - this avoids some of the context scoping problems
previously encountered since we can pass the correct context to the
periodic task when starting it
Jeff Peeler [Mon, 19 Nov 2012 17:10:12 +0000 (12:10 -0500)]
Provide more information with template URL error
Previously a generic read error was given. Inform the user with proper
exceptions for: the case when a resource is present but non-responsive
and when a resource is present but the requested data could not be
found.
Zane Bitter [Wed, 14 Nov 2012 16:51:26 +0000 (17:51 +0100)]
ReST API: Add a convenience redirect for resources
Since the path stacks/{stack_name}/{stack_id}/resources exists for each
valid stack we can safely redirect to it from stacks/{stack_name}/resources
without having to perform any further checks in the engine, so we may as
well allow it.
Zane Bitter [Fri, 16 Nov 2012 11:10:21 +0000 (12:10 +0100)]
RPC API: Include less detail in resource list
Since the stack_list_resources command could be returning data for a large
number of resources, avoid returning the metadata and description (which
are not needed) to save space.
Steven Hardy [Thu, 15 Nov 2012 10:03:46 +0000 (10:03 +0000)]
heat workaround for HA/Autoscaling regression
Interim workaround for HA/Autoscaling regression, caused by change of
scope for stack_get_by_name to be per-tenant, meaning we need to do a
brute-force lookup to find the stack entry when using the stored admin
context.
A better, more comprehensive (but more intrusive) fix is in-progress
but this at least returns us to the same functionality we had before
Note this means that all stacks using WatchRule still need to have names
unique accross all tenants.
Zane Bitter [Mon, 12 Nov 2012 16:42:36 +0000 (17:42 +0100)]
Actually validate properties of resources
Properties.validate() returns an error on failure; it does not raise an
exception (and never has). So we should look at the return value when we
call it.
Steven Hardy [Tue, 13 Nov 2012 09:48:49 +0000 (09:48 +0000)]
heat engine : create boto config via instance userdata
Create boto config via instance userdata, which the instance
cfntools can read instead of /etc/boto.cfg, avoids messy post
install sed-mangling in the templates
Change-Id: Ic94bfc2e362a3a3f93b9bdeaad6a5f988371346e Signed-off-by: Steven Hardy <shardy@redhat.com>
Steven Hardy [Mon, 12 Nov 2012 18:31:16 +0000 (18:31 +0000)]
heat engine : append watch server url to instance userdata
Append url for watch server to instance userdata, which avoids
post-install sed mangling in the template, and will make it easier
to transparently switch to a different metric service
Change-Id: I59b9b7efcd75d44e88ebe0a116a9ce1e3ef20c14 Signed-off-by: Steven Hardy <shardy@redhat.com>
Zane Bitter [Tue, 13 Nov 2012 17:19:12 +0000 (18:19 +0100)]
Don't use OpenStack plugin in tox
This plugin provides syntax colouring of the tox output, at the cost of no
longer reporting errors in test setup (instead it just reports no tests run
-> success).
Steven Hardy [Mon, 12 Nov 2012 19:59:58 +0000 (19:59 +0000)]
heat engine : Don't wait() for killed greenthreads
We don't care about the return value of any killed threads on
delete_stack, and the greenthreads will all raise a GreenletExit
exception after being kill()'d anyway.
Also see related openstack-common fix : https://review.openstack.org/15906
Steve Baker [Fri, 9 Nov 2012 02:45:06 +0000 (15:45 +1300)]
Switch to UUID for the Stack primary key.
For the migrate upgrade, existing integer keys remain and any subsequent stack
will be assigned a uuid.
For a migrate downgrade, it should work fine if there have been no stacks added
since ugrade. Added stacks will have their IDs 'truncated' back to an integer
which will require some manual fixup after downgrade.
Steve Baker [Thu, 8 Nov 2012 19:43:23 +0000 (08:43 +1300)]
Add a list_stacks RPC call.
This differs from the old show_stack(None) by not resolving stack data.
heat list does not display any of the resolved stack data. Resolving outputs
can be especially slow since FnGetAtt may block while it calls out
to another service (such as with the quantum resources).
Without this change, running heat list on a single Quantum.template takes 1.8s
vs 0.7s.
In the future, list_stacks can be made to support paging of results.
Steve Baker [Thu, 8 Nov 2012 02:49:19 +0000 (15:49 +1300)]
Move client connection out of resources.py.
Client connection references are currently cached at the resource instance
level, which doesn't seem very useful.
This change moves the client connection and caching to clients.py, then puts a
Clients instance in a Stack. This means that (for example) all requests to nova
in one stack will come from the same client instance.
Code Review / openstack-build / heat-build.git / log