Chef is a configuration management tool that is written in Ruby and Erlang. It’s capable of managing both your on premise and cloud instances with ease.
As your infrastructure requirements grow, managing each server by hand becomes an increasingly difficult task. Configuration management solutions are designed to turn your infrastructure administration into a code base. Instead of performing individual tasks on a number of machines, these tools allow you to commit your requirements to a central location where each component can connect, pull down their configuration, and apply it.
You can easily manage up to 10000 nodes using chef. Replicating the infrastructure components is easy once we have them automated via chef.
Requirements:
- You have an account and are logged into console.scaleway.com
- You have an Ubuntu Xenial server
- You have configured your SSH Key
1 . Connect to your server using SSH
ssh root@SERVER_IP
If you do not know your server IP, you can list your existing servers using scw ps
(Scaleway CLI).
Type yes
when prompted to confirm that we wish to proceed. If the Scaleway CLI is not installed, you have several options:
brew install scw
Install the latest stable release on Mac OS X manually:
# prepare for first install and upgrade
mkdir -p /usr/local/bin
mv /usr/local/bin/scw /tmp/scw.old
# get latest release
wget "https://github.com/scaleway/scaleway-cli/releases/download/v1.19/scw-darwin-amd64" -O /usr/local/bin/scw
# test
scw version
Install the latest release on Linux:
# get latest release
export ARCH=amd64 # can be 'i386', 'amd64' or 'armhf'
wget "https://github.com/scaleway/scaleway-cli/releases/download/v1.19/scw_1.16_${ARCH}.deb" -O /tmp/scw.deb
dpkg -i /tmp/scw.deb && rm -f /tmp/scw.deb
# test
scw version
Prior to installing Chef, ensure that the that the hostname of the server is a resolvable fully qualified domain name or IP address.
2 . On the server where you plan to install Chef, launch:
hostname -f
3 . If the command does not return an address where the server can be reached, edit the configuration file to add a domain name or IP address where the server can be reached:
sudo nano /etc/hosts
4 . Edit the configuration file to like the example below:
127.0.1.1 fqdn_or_IP_address host_alias
127.0.0.1 localhost
IP_address fqdn_or_IP_address host_alias
5 . Check that the value was set correctly:
hostname -f
The command returns the value that you can use to reach your Chef server from anywhere in your infrastructure.
Note: If you use the root user, you can remove the
sudo
before each command.
1 . Update Ubuntu packet manager:
sudo apt-get update
2 . Upgrade the Ubuntu packages already installed:
sudo apt-get upgrade
3 . Go on the downloads page and copy the URL of the Chef Server for Ubuntu
4 . Launch the installation command with the URL retrieved from the website:
wget https://packages.chef.io/files/stable/chef-server/12.17.33/ubuntu/16.04/chef-server-core_12.17.33-1_amd64.deb
5 . Once the download is complete, install the package:
sudo dpkg -i chef-server*.deb
6 . Call the reconfigure
command, which configures the components that make up the server to work together in your specific environment:
sudo chef-server-ctl reconfigure
Creating an admin user enables a specific user to make changes to the infrastructure components in the organization. An Organization is a grouping of infrastructure and configuration within Chef.
1 . Launch the chef-server-ctl
with the user-create
subcommand:
chef-server-ctl user-create USERNAME FIRST_NAME LAST_NAME EMAIL PASSWORD
For our example, we will create a user with the following information:
We will also add -f
option in order to specify a filename in which to output our new user’s private RSA key. We will need this in order to authenticate using the knife
management command later. In the end, the command translates to:
chef-server-ctl user-create testuser test user testuser@example.com pwdexample -f testuser.pem
2 . Launch the chef-server-ctl
with the org-create
subcommand:
chef-server-ctl org-create SHORTNAME LONGNAME --association_user USERNAME
For our example, we will create an organization with the following information:
Again, we will add the -f
flag to specify the private key file location. The key that will be created is used to validate new clients as part of the organization until they can get their own unique client key. In the end, the command translates to:
chef-server-ctl org-create scaleway "Scaleway, Inc." --association_user testuser -f scaleway-file.pem
The Chef server installation is now complete.
The actual infrastructure coordination and configuration does not take place on the Chef server. This work is done on a workstation which then uploads the data to the server to influence the Chef environment.
The Chef Development Kit, a suite of software designed for Chef workstations.
1 . On another server, download the CDK which is available here
wget https://packages.chef.io/files/stable/chefdk/3.0.36/ubuntu/16.04/chefdk_3.0.36-1_amd64.deb
2 . Install the package:
sudo dpkg -i chefdk_*.deb
3 . After the installation, you can verify that all of the components are available in their expected location through the new chef command:
chef verify
Note: If your workstation is primarily used to manage Chef for your infrastructure, you will likely want to default to the version of Ruby installed with Chef. You can do this by modifying your .bash_profile so that Chef’s Ruby takes precedence:
echo 'eval "$(chef shell-init bash)"' >> ~/.bash_profile
Afterwards, you can source your .bash_profile file to set the correct environmental variables for the current session:
source ~/.bash_profile
The Chef configuration for your infrastructure is maintained in a hierarchical file structure known as a Chef repo. As the Chef GitHub repository is deprecated, we recommend using the chef generate repo
command that comes with ChefDK.
chef generate repo chef_repo
At this point, your workstation has all of the software needed to interact with a Chef server and compose infrastructure configurations. However, it is not yet configured to interact with your Chef server and your environment.
Create the hidden directory where you can store the user key and the organization validator key that we created on the Chef server.
mkdir ~/chef_repo/.chef
The method that you use to connect to the Chef server will determine how to download the keys.
1 . Leave your workstation
exit
2 . Once you are back on your local computer, you need to add the SSH keys you use to connect to the Chef server to an SSH agent.
eval $(ssh-agent)
which should return something similar to
Agent pid 2893
3 . Once the agent is started, add your SSH key to it:
ssh-add
4 . This will keep your SSH key stored in memory. Now, you can forward the stored key to your workstation using the -A
option with ssh. It allows you to connect to any computer from your workstation as if you were connecting from your local computer:
ssh -A username@workstation_domain_or_IP
Now, you can connect to your Chef server without needing a password using the forwarded SSH credentials.
If you log into the Chef server using the root user account, your commands will look something like this.
scp root@server_domain_or_IP:/root/testuser.pem ~/chef_repo/.chef
scp root@server_domain_or_IP:/root/scaleway-file.pem ~/chef_repo/.chef
If you connect to your Chef server using a non-root user, the commands will look more like this:
scp username@server_domain_or_IP:/home/username/testuser.pem ~/chef_repo/.chef
scp username@server_domain_or_IP:/home/username/scaleway-file.pem ~/chef_repo/.chef
Note: By default, Scaleway deactivates server connection with password. For more information, refer to the dedicated Ubuntu Forum
On your workstation:
~/chef_repo/.chef
in our case).If you log into the Chef server using the root user account, your commands will look something like this.
scp root@server_domain_or_IP:/root/testuser.pem ~/chef_repo/.chef
scp root@server_domain_or_IP:/root/scaleway-file.pem ~/chef_repo/.chef
If you connect to your Chef server using a non-root user, the commands will look more like this:
scp username@server_domain_or_IP:/home/username/testuser.pem ~/chef_repo/.chef
scp username@server_domain_or_IP:/home/username/scaleway-file.pem ~/chef_repo/
Now that you have your Chef credentials available on your workstation, we can configure the knife
command with the information it needs to connect to and control your Chef infrastructure.
This is done through a knife.rb
file that we will place in the ~/chef_repo/.chef
directory along with our keys.
1 . Open up a file called knife.rb
in that directory in your text editor:
nano ~/chef_repo/.chef/knife.rb
2 . Paste the following information replacing the information with your infrastructure information:
current_dir = File.dirname(__FILE__)
log_level :info
log_location STDOUT
node_name "name_for_workstation"
client_key "#{current_dir}/name_of_user_key"
validation_client_name "organization_validator_name"
validation_key "#{current_dir}/organization_validator_key"
chef_server_url "https://server_domain_or_public_IP/organizations/organization_name"
syntax_check_cache_path "#{ENV['HOME']}/.chef/syntaxcache"
cookbook_path ["#{current_dir}/../cookbooks"]
#{current_dir}
snippet to fill in the path if the key is in the same directory as the knife.rb
file.-validator
.client_key
, this includes the name and path to the validation key you copied from the Chef server. Again, you can use the #{current_dir}
.https://
, followed by your Chef server’s domain name or Public IP address. Afterwards, the path to your organization should be specified by appending /organizations/your_organization_name.For our tutorial, the file looks this:
current_dir = File.dirname(__FILE__)
log_level :info
log_location STDOUT
node_name "testuser"
client_key "#{current_dir}/testuser.pem"
validation_client_name "scaleway-validator"
validation_key "#{current_dir}/scaleway-file.pem"
chef_server_url "https://domaine_name_or_public_IP/organizations/scaleway"
syntax_check_cache_path "#{ENV['HOME']}/.chef/syntaxcache"
cookbook_path ["#{current_dir}/../cookbooks"]
3 . Save and close the knife.rb
file.
4 . Test the configuration file. We need to be in our ~/chef-repo
directory for our configuration file to be read correctly:
cd ~/chef_repo
knife client list
which returns an error similar to:
ERROR: SSL Validation failure connecting to host: 51.15.240.56 - SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate)
ERROR: Could not establish a secure connection to the server.
Use `knife ssl check` to troubleshoot your SSL configuration.
If your Chef Server uses a self-signed certificate, you can use
`knife ssl fetch` to make knife trust the server's certificates.
Original Exception: OpenSSL::SSL::SSLError: SSL Error connecting to https://51.15.240.56/organizations/scaleway/clients - SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate)
It occurs because we do not have our Chef server’s SSL certificate on our workstation.
5 . Launch
knife ssl fetch
which returns
WARNING: Certificates from public_IP will be fetched and placed in your trusted_cert
directory (/root/chef_repo/.chef/trusted_certs).
Knife has no means to verify these are the correct certificates. You should
verify the authenticity of these certificates after downloading.
Adding certificate for public_IP in /root/chef_repo/.chef/trusted_certs/public_IP.crt
After the SSL certificate has been fetched, the previous command should now work.
6 . Launch
knife client list
which returns
scaleway-validator
If the above command correctly returns, your workstation is now set up to control your Chef environment.