5 [DDL]: /mcollective/reference/plugins/ddl.html
6 [DiscoveryPlugins]: /mcollective/reference/plugins/discovery.html
9 Up to MCollective 2.0 the discovery system could only discover against
10 installed agents, configuration management classes or facts and the node
11 identities. We're extending this to support discovery against many
12 sources through a simple plugin system.
14 *NOTE:* This feature is available since version 2.1.0
16 The basic idea is that you could do discovery statements like the ones
19 {% highlight console %}
20 % mco find -S "fstat('/etc/rsyslog.conf').md5=/4edff591f6e38/"
23 {% highlight console %}
24 % mco find -S "sysctl('net.ipv4.conf.all.forwarding').value=1"
27 {% highlight console %}
28 % mco find -S "sysctl('net.ipv4.conf.all.forwarding').value=1 and % location=dc1"
31 You could also use these data sources in your own agents or other
36 reply[:value] = Data.sysctl(request[:sysctl_name]).value
40 *NOTE:* As opposed to the [DiscoveryPlugins] which are used by the client
41 to communicate to the nodes using direct addressing, data plugins on the other
42 hand refer to data that the nodes can provide, and hence this uses the normal
43 broadcast discovery paradigm.
45 These new data sources are plugins so you can provide via the plugin
46 system and they require DDL documents. The DDL will be used on both the
47 client and the server to provide strict validation and configuration.
49 The DDL for these plugins will affect the client libraries in the
52 * You will get errors if you try to discover using unknown functions
53 * Your input argument values will be validated against the DDL
54 * You will only be able to use output properties that are known in the DDL
55 * If a plugin DDL says it needs 5 seconds to run your discovery and maximum run times will increase by 5 seconds automatically
57 On the servers the DDL will:
59 * be used to validate known plugins
60 * be used to validate input arguments
61 * be used to validate requests for known output values
63 ## Viewing or retrieving results from a data plugin
65 You can view the output from a data plugin using the *rpcutil* agent:
67 {% highlight console %}
68 % mco rpc rpcutil get_data source=fstat query=/etc/hosts
72 atime: 2012-06-14 21:41:54
74 atime_seconds: 1339706514
75 ctime: 2012-01-18 20:28:34
77 ctime_seconds: 1326918514
79 md5: 54fb6627dbaa37721048e4549db3224d
81 mtime: 2010-01-12 13:28:22
83 mtime_seconds: 1263302902
92 The same action can be used to retrieve data programatically.
94 ## Writing a data plugin
96 ### The Ruby logic for the plugin
97 The data plugins should not change the system in anyway, you should take
98 care to create plugins that only reads the state of the system. If you
99 want to affect the status of the system you should write Agents.
101 These plugins are kept simple as they will be typed on the command line
102 so the following restrictions are present:
104 * They can only take 1 input argument
105 * They can only return simple String, Numeric or Booleans no Hashes or complex data types
106 * They should be fast as these will impact discovery times and agent run times.
108 Writing data plugins is easy and mimic the basics of writing agents,
109 below we have a simple *sysctl* plugin that was used in the examples
112 {% highlight ruby linenos %}
115 class Sysctl_data<Base
116 activate_when { File.executable?("/sbin/sysctl") && Facter["kernel"] == "Linux" }
119 shell = Shell.new("/sbin/sysctl %s" % sysctl)
122 if shell.status.exitstatus == 0
123 value = shell.stdout.chomp.split(/\s*=\s*/)[1]
126 value = Integer(value) if value =~ /^\d+$/
127 value = Float(value) if value =~ /^\d+\.\d+$/
130 result[:value] = value
138 The class names have to be *Something_data* and they must inherit from
139 *Base* as in the example here. The file would be saved in the *libdir*
140 as *data/sysctl_data.rb* and *data/sysctl_data.ddl*.
142 This plugin will only be activated if the file */sbin/sysctl* exist, is
143 executable and if the system is a Linux server. This allow us to install
144 it on a Windows machine where it will just be disabled and those
145 machines will never be discovered using this function.
147 We then create a block that would be the main body of the query. We use
148 the *MCollective::Shell* class to run sysctl, parse the output and save
149 it into the *result* hash.
151 The result hash is the only way to return values from these plugins. You
152 can only save simple strings, numbers or booleans in the result.
154 ### The DDL for the plugin
155 As mentioned every data plugin requires a DDL. These DDL files mimic
156 those of the [SimpleRPC Agents][DDL].
158 Below you'll find a DDL for the above sysctl data plugin:
160 {% highlight ruby linenos %}
161 metadata :name => "Sysctl values",
162 :description => "Retrieve values for a given sysctl",
163 :author => "R.I.Pienaar <rip@devco.net>",
164 :license => "ASL 2.0",
166 :url => "http://marionette-collective.org/",
169 dataquery :description => "Sysctl values" do
171 :prompt => "Variable Name",
172 :description => "Valid Variable Name",
174 :validation => /^[\w\-\.]+$/,
178 :description => "Kernel Parameter Value",
179 :display_as => "Value"
183 The *timeout* must be set correctly, if your data source is slow you
184 need to reflect that in the timeout here. The timeout will be used on
185 the clients to decide how long to wait for discovery responses from the
186 network so getting this wrong will result in nodes not being discovered.
188 Each data plugin can only have one *dataquery* block with exactly 1
189 *input* block but could have multiple *output* blocks.
191 It's important to get the validation correct, here we only accept the
192 characters we know are legal in sysctl variables on Linux. We will
193 specifically never allow backticks to be used in arguments to avoid
194 accidental shell exploits.
196 Note the correlation between output names and the use in discovery and
197 agents here we create an output called *value* this means we would use
200 {% highlight console %}
201 % mco find -S "sysctl('net.ipv4.conf.all.forwarding').value=1"
204 And we would output the result from our plugin code as:
206 {% highlight ruby linenos %}
207 result[:value] = value
210 And in any agent where we might use the data source:
212 {% highlight ruby linenos %}
213 something = Data.sysctl('net.ipv4.conf.all.forwarding').value
216 These have to match everywhere, you cannot reference undeclared data and
217 you cannot use input that does not validate against the DDL declared
220 Refer to the full [DDL] documentation for details on all possible values
221 of the *metadata*, *input* and *output* blocks.
223 ## Auto generated documentation
224 As with agents the DDL can be used to generate documentation, if you
225 wanted to know what the input and output values are for a specific
226 plugin you can use *mco plugin doc* to see generated documentation.
228 {% highlight console %}
229 % mco plugin doc sysctl
233 Retrieve values for a given sysctl
235 Author: R.I.Pienaar <rip@devco.net>
239 Home Page: http://marionette-collective.org/
241 QUERY FUNCTION INPUT:
243 Description: Valid Variable Name
244 Prompt: Variable Name
246 Validation: (?-mix:^[\w\-\.]+$)
249 QUERY FUNCTION OUTPUT:
252 Description: Kernel Parameter Value
257 ## Available plugins for a node You can use the *mco inventory*
258 application to see remotely what plugins a node has available:
260 {% highlight console %}
261 % mco inventory your.node
262 Inventory for your.node: