Automation seems to be the mantra of DevOps. One of the challenges that comes along with Automation, is the safe collection of data, storing it somewhere useful, and later acting on it. Sometimes this data is not something commonly collected, or relates to proprietary software that requires some custom integration.
Chef is automation software. It is written in Ruby, and does everything with a culinary related twist. For example, you write recipes that the server will interpret, to put itself in the state you specified(eg gcc installed and latest version). A collection of those recipes is a cookbook. For more information on Chef check out.
Resolvers are something we all have, but tracking on them is rather poor. You would think, the control mechanism we use for DNS resolution would get a bit more love. How can we be better admins, and at least start collecting this data across such a vast fleet? Lets add it to our chef ohai data!
What is ohai data, you ask? When a chef node connects back to the Chef server, it sends a ton of metadata about the node. Disk sizes, RAM, CPU, ip addresses, etc. So, what we are going to do, is add the resolvers of each node to this data. This will enable us to query the Chef server, rather than each individual node to find the resolvers of each host.
I assume that you already have a functional chef server, and node connected. In this example, I will be using the free tier hosted Chef.
By installing the Chef Development Kit, we get access to some binaries we can use. For example, If you call “ohai”, it will print out the metadata for the machine you are on. This gives us a good place to develop our plugin.
/etc/chef/ohai/plugins/resolver.rb
Ohai.plugin(:Resolvers) do
provides 'resolvers'
def get_resolvers
if File.exist?("/etc/resolv.conf") then
nameserver = {}
position = 0
File.open('/etc/resolv.conf').each do |line|
if line =~ /^nameserver .*$/ then
nsip = line.split(' ')[1]
nameserver[position] = { 'nameserver' => nsip }
position = position + 1
end
end
return nameserver
end
end
collect_data(:default) do
resolvers Mash.new
resolvers['nameservers'] = get_resolvers
end
end
Our get_resolvers function gets our resolvers, formats our input, and feeds them back to the Mash data structure in the collect_data function. For all intents and purposes consider collect_data to be main().
Don’t let it intimidate you, if you look closely, there is one function doing work, and another just formatting the data for us. Nothing scary.
How can we test this to make sure it actually works?
On your workstation, with chefdk, put this code in a directory, and then run ohai -d /path/to/our/code.
root@DESKTOP:~# mkdir tmp
root@DESKTOP:~# mv resolvers.rb tmp
root@DESKTOP:~# ohai -d /root/tmp | jq .resolvers
{
"nameservers": {
"0": {
"nameserver": "8.8.8.8"
},
"1": {
"nameserver": "8.8.6.6"
}
}
}
root@DESKTOP:~#
If you aren’t already using jq you should look into it.
As you can see, we now return our nameservers in our ohai test. Okay, so now that we know the code works, and we have it on our workstation. Lets get it deployed!
how do we get it to our nodes?
For this post to be a reasonable size, I can’t go into great detail on generating the cookbook, and basic chef usage. So I assume a certain amount of previous chef knowledge.
To make this a completely chef solution, we need to take the above ohai plugin, and dump it in /etc/chef/ohai/plugins when chef runs. We can do that by using the template directive in the main chef default.rb. To get started lets use the following:
cookbooks/test-stuff/recipes/default.rb
template '/etc/chef/ohai/plugins/resolvers.rb' do
source 'resolvers.erb'
owner 'root'
group 'root'
mode '0755'
end
This tells chef, to look in cookbooks/test-stuff/templates/default/$source, and write it to the location defined after template.
Now, lets move our plugin where chef is going to look for it. We previously had it in /root/tmp for testing. One thing to note is, we are changing the extension from .rb to .erb. This is specific to ruby templating. Also note, your paths are likely different from mine.
root@DESKTOP:~# mkdir -p ~/chef-starter/chef-repo/cookbooks/test-stuff/templates/default/
root@DESKTOP:~# mv /root/tmp/resolvers.rb ~/chef-starter/chef-repo/cookbooks/test-stuff/templates/default/resolvers.erb
Lets recap, which files have we modified:
root@DESKTOP:~/chef-starter/chef-repo/cookbooks/test-stuff# find ./ | grep -v git | grep -v .deliver
./
./.kitchen.yml
./Berksfile
./CHANGELOG.md
./LICENSE
./README.md
./chefignore
./metadata.rb
./recipes
./recipes/default.rb <---- Our file where we write our ohai plugin as a template.
./spec
./spec/spec_helper.rb
./spec/unit
./spec/unit/recipes
./spec/unit/recipes/default_spec.rb
./templates
./templates/default
./templates/default/resolvers.erb <---- Our resolvers.rb file, with erb extension.
./test
./test/integration
./test/integration/default
./test/integration/default/default_test.rb
root@DESKTOP:~/chef-starter/chef-repo/cookbooks/test-stuff#
Everything with the exception of the two files noted above has been generated using chef generate cookbook…
Lets bump our metadata file, and then upload our cookbook, I bumped mine to 0.1.2:
root@DESKTOP:~/chef-starter/chef-repo# grep ^version cookbooks/test-stuff/metadata.rb
version '0.1.2'
root@DESKTOP:~/chef-starter/chef-repo#
Now we upload the cookbook to our chef server:
root@DESKTOP:~/chef-starter/chef-repo# knife cookbook upload test-stuff
Uploading test-stuff [0.1.2]
Uploaded 1 cookbook.
root@DESKTOP:~/chef-starter/chef-repo#
At this point, we have uploaded our cookbook, and our template file(resolvers.erb) to our repo. When chef runs, it will call cookbooks/test-stuff/recipes/default.rb first. That file tells chef to grab the template we named resolvers.erb and write it in place. Then the ohai plugin runs, and we should get the new metadata for any nodes running the test-stuff cookbook.
Now, you need to make sure that your cookbook, in my case “test-stuff” is in the chef runlist for the node you expect to get this data from. We can check that doing the following:
root@DESKTOP:~/chef-starter/chef-repo# knife node list
prewikka2
root@DESKTOP:~/chef-starter/chef-repo# knife node show prewikka2
Node Name: prewikka2
Environment: berks-test-env
FQDN: prewikka2.demobox.org
IP: 172.16.0.120
Run List: recipe[test-stuff]
Roles:
Recipes: test-stuff, test-stuff::default, berks-testing, berks-testing::default
Platform: centos 7.5.1804
Tags:
root@DESKTOP:~/chef-starter/chef-repo#
As you can see, the test-stuff recipe is in runlist for this node. Now we could wait and let the node do its thing automatically, or we can call chef-client on the node and get to watch as it updates.
[root@prewikka ~]# chef-client
Starting Chef Client, version 14.1.12
resolving cookbooks for run list: ["test-stuff"]
Synchronizing Cookbooks:
- test-stuff (0.1.2)
Installing Cookbook Gems:
Compiling Cookbooks...
Converging 1 resources
Recipe: test-stuff::default
* template[/etc/chef/ohai/plugins/resolvers.rb] action create
- create new file /etc/chef/ohai/plugins/resolvers.rb
- update content in file /etc/chef/ohai/plugins/resolvers.rb from none to 46965f
--- /etc/chef/ohai/plugins/resolvers.rb 2019-02-03 02:08:39.963922597 -0500
+++ /etc/chef/ohai/plugins/.chef-resolvers20190203-17325-cef0zi.rb 2019-02-03 02:08:39.962922597 -0500
@@ -1 +1,26 @@
+Ohai.plugin(:Resolvers) do
+ provides 'resolvers'
+
+ def get_resolvers
+ if File.exist?("/etc/resolv.conf") then
+ nameserver = {}
+ position = 0
+ File.open('/etc/resolv.conf').each do |line|
+ if line =~ /^nameserver .*$/ then
+ nsip = line.split(' ')[1]
+ nameserver[position] = { 'nameserver' => nsip }
+ position = position + 1
+ end
+ end
+ return nameserver
+ end
+ end
+
+
+ collect_data(:default) do
+ resolvers Mash.new
+ resolvers['nameservers'] = get_resolvers
+ end
+
+end
- change mode from '' to '0755'
- change owner from '' to 'root'
- change group from '' to 'root'
Running handlers:
Running handlers complete
Chef Client finished, 1/1 resources updated in 07 seconds
[root@prewikka ~]#
The chef run wrote our plugin to /etc/chef/ohai/plugins like it was supposed to. Now, lets check the metadata for those resolvers on the node itself:
[root@prewikka ~]# ohai | jq .resolvers
{
"nameservers": {
"0": {
"nameserver": "8.8.8.8"
}
}
}
[root@prewikka ~]#
Verify we have some nameserver data in the backend.
Lets double check that our chef server actually picked up the ohai from the node. From our workstation, lets ask the chef server for the node data.
root@DESKTOP:~/chef-starter/chef-repo/cookbooks/test-stuff/attributes# knife node show prewikka2 -a resolvers.nameservers
prewikka2:
resolvers.nameservers:
0:
nameserver: 8.8.8.8
root@DESKTOP:~/chef-starter/chef-repo/cookbooks/test-stuff/attributes# knife node show prewikka2 -a resolvers.nameservers.0.nameserver
prewikka2:
resolvers.nameservers.0.nameserver: 8.8.8.8
root@DESKTOP:~/chef-starter/chef-repo/cookbooks/test-stuff/attributes#
So the trick here is, what you name your Mash in the template(we used resolvers), will be the first level name of our json. So, since we used resolvers->nameservers->index#->nameserver:$value… We pass in the parameter “-a resolvers.nameservers”. Periods can be used as delimiters to traverse the tree. eg resolvers.nameservers.0.nameserver
Conclusion
As you can see our data is now being harvested by our ohai plugin. In the above example, we were able to use our workstation(root@DESKTOP), to query our chef server(hosted chef), and ask for data about prewikka2(root@prewikka|prewikka2). The chef server then gave us the value of 8.8.8.8 which…lets just double check…
[root@prewikka ~]# cat /etc/resolv.conf | grep nameserver
nameserver 8.8.8.8
[root@prewikka ~]#
SUCCESS! Thanks for reading!