Scaleway Documentationtutorials
configure chef ubuntu xenial

Jump toUpdate content

How to use and configure Chef on Ubuntu

Reviewed on 02 December 2021 • Published on 05 July 2018
  • Instances
  • Chef
  • Ubuntu-Xenial
  • Ubuntu-Focal-Fossa

Chef Overview

Chef is a configuration management tool that is written in Ruby and Erlang. It’s capable of managing both your on-premise machines and your Scaleway resources with ease.

As your infrastructure requirements grow, managing each server individually becomes an increasingly difficult task. Configuration management solutions are designed to turn your infrastructure administration into a code base. Instead of performing individual tasks on a number of machines, these tools allow you to commit your requirements to a central location where each component can connect, pull down their configuration, and apply it.

You can easily manage up to 10,000 nodes using Chef. Chef also makes it easy to automate the replication of infrastructure components.

Requirements:

Core Concepts

  • Chef Server: This central server holds all configuration data that the nodes will use for configuration.
  • Workstation: This machine holds all the configuration data that can later be pushed to the central Chef server. Several Chef command line utilities are available in this system, which can be used to interact with nodes, update configurations etc.
  • Node: This is a client-server/system that will be registered to the central Chef server, from which it can pull configuration data to apply.

Configuring a Chef Server

Accessing the Server by Hostname

  1. Connect to your remote machine via SSH. Follow the instructions depending on your remote machine type: Instance, Elastic Metal or Dedibox.

    Note:

    This tutorial assumes that you connect to your machine as root. If this is not the case, you may need to add sudo to the commands shown.

    To install Chef, the hostname of the machine must be a resolvable fully qualified domain name or IP address. Check that in the following steps:

  2. On the machine where you plan to install Chef, launch:

    hostname -f

    If the command does not return an address where the server can be reached, follow steps 3-5 to resolve this. Otherwise, proceed to the next section of this tutorial.

  3. Open the etc/hosts configuration file in a text editor:

    nano /etc/hosts
  4. Edit the configuration file to add a domain name or IP address where the server can be reached, following the example below:

    127.0.1.1 fqdn_or_IP_address host_alias
    127.0.0.1 localhost
    IP_address fqdn_or_IP_address host_alias
    Tip:

    You can find the IP address of your Instance or Elastic Metal server in its Overview tab in the Scaleway console. For an Instance with host alias scw-flamboyant-boyd, IP address 192.158.1.38 and no fully qualified domain name, the configuration file once edited might look like this:

    127.0.1.1 192.158.1.38 scw-flamboyant-boyd
    127.0.0.1 localhost
    192.158.1.38 192.158.1.38 scw-flamboyant-boyd

    # The following lines are desirable for IPv6 capable hosts
    ::1 ip6-localhost ip6-loopback
    fe00::0 ip6-localnet
    ff00::0 ip6-mcastprefix
    ff02::1 ip6-allnodes
    ff02::2 ip6-allrouters
    ff02::3 ip6-allhosts
    127.0.0.1 scw-flamboyant-boyd
  5. Check that the value was set correctly:

    hostname -f

    The command returns the value that you can use to reach your Chef server from anywhere in your infrastructure. This could be an IP address or fully-qualified domain name.

Installing Chef Server

  1. Update the Ubuntu package manager:

    apt-get update
  2. Upgrade the Ubuntu packages already installed:

    apt-get upgrade
  3. Go to the Chef Infra Server Download Page, from your local machine’s browser, and click Download next to the version of Ubuntu you are running.

  4. Complete your information in the form that displays and click Download.

  5. Right-click Download Chef Infra Server directly and copy the link.

  6. Launch the installation using wget + the link you copied, from the terminal of your remote machine. Make sure to use your own link and not the one shown below as an example:

    wget https://packages.chef.io/files/stable/chef-server/14.10.23/ubuntu/20.04/chef-server-core_14.10.23-1_amd64.deb?_ga=2.xxxxxxxx.xxxxxx.xxxxxxxxx-xxxxxxxx.xxxxxx
  7. Install the package you downloaded:

    dpkg -i chef-server*
  8. Call the reconfigure command, which configures the components that make up the server to work together in your specific environment:

    chef-server-ctl reconfigure

    You are given a link to the license agreement, and prompted to accept its terms.

    The configuration may take some minutes to complete.

Creating an Admin User and Organization

Creating an admin user enables a specific user to make changes to the infrastructure components in the organization. An Organization is a grouping of infrastructure and configuration within Chef.

  1. Launch the chef-server-ctlwith the user-create subcommand:

    chef-server-ctl user-create USERNAME FIRST_NAME LAST_NAME EMAIL PASSWORD

    For our example, we will create a user with the following information:

    • Username: testuser
    • First Name: test
    • Last Name: user
    • Email: testuser@example.com
    • Password: pwdexample
    • Filename: testuser.pem

    We will also add -f option in order to specify a filename in which to output our new user’s private RSA key. We will need this in order to authenticate using the knife management command later. In the end, the command translates to:

    chef-server-ctl user-create testuser test user testuser@example.com pwdexample -f testuser.pem
  2. Launch the chef-server-ctl with the org-create subcommand:

    chef-server-ctl org-create SHORTNAME LONGNAME --association_user USERNAME

    For our example, we will create an organization with the following information:

    • Short Name (refers to the organization within Chef): scaleway
    • Long Name (name of the organization): Scaleway, Inc.
    • Association User (username that has access to administer the organization): testuser
    • Filename: scaleway-file.pem

    Again, we will add the -f flag to specify the private key file location. The key that will be created is used to validate new clients as part of the organization until they can get their own unique client key. In the end, the command translates to:

    chef-server-ctl org-create scaleway "Scaleway, Inc." --association_user testuser -f scaleway-file.pem

The Chef server installation is now complete.

Configuring a Chef Workstation

The actual infrastructure coordination and configuration does not take place on the Chef server. This work is done on a workstation which then uploads the data to the server to influence the Chef environment. This workstation can be your local machine or a remote one.

Installing the Chef Development Kit (CDK)

The Chef Development Kit, a suite of software designed for Chef workstations.

  1. Go to the CDK Ubuntu Linux Download Page, and click Download next to the version of Ubuntu you are running.

  2. Complete your information in the form that displays and click Download.

  3. Right-click Download ChefDK directly and copy the link.

  4. Launch the installation using wget + the link you copied on the machine you want to install ChefDK. Make sure to use your own link and not the one shown below as an example:

    wget https://packages.chef.io/files/stable/chefdk/4.13.3/ubuntu/20.04/chefdk_4.13.3-1_amd64.deb?_ga=2.xxxxxxxx.xxxxxxxxxxx.xxxxxxxxxxx-xxxxxxxxxxx.xxxxxxxxxxx
  5. Install the package you downloaded:

    dpkg -i chefdk_*
  6. Verify that all of the components are available in their expected location through the new chef command:

    chef verify
  7. Enter yes, when you are prompted to accept the license agreement, to agree.

Note: If your workstation is primarily used to manage Chef for your infrastructure, you will likely want to default to the version of Ruby installed with Chef. You can do this by modifying your .bash_profile so that Chef's Ruby takes precedence:
echo 'eval "$(chef shell-init bash)"' >> ~/.bash_profile

Afterwards, you can source your .bash_profile file to set the correct environmental variables for the current session:

source ~/.bash_profile

Cloning the Chef Repository

The Chef configuration for your infrastructure is maintained in a hierarchical file structure known as a Chef repo. As the Chef GitHub repository is deprecated, we recommend using the chef generate repo command that comes with ChefDK.

chef generate repo chef_repo

Downloading the Authentication Keys to the Workstation

At this point, your workstation has all of the software needed to interact with a Chef server and compose infrastructure configurations. However, it is not yet configured to interact with your Chef server and your environment.

Create the hidden directory where you can store the user key and the organization validator key that we created on the Chef server.

mkdir ~/chef_repo/.chef

The method that you use to connect to the Chef server will determine how to download the keys.

Option 1: (Recommended) Downloading Keys when Connecting to a Chef Server Using SSH Keys

  1. Leave your workstation

    exit

    The next steps should be carried out from the machine that holds the private SSH key needed to connect to your Chef server.

  2. Add the SSH key you use to connect to the Chef server to an SSH agent:

    eval $(ssh-agent)

    which should start the agent and return something similar to

    Agent pid 2893
  3. Add your SSH key to the agent:

    ssh-add

    This keeps your SSH key stored in memory.

  4. Forward the stored key to your workstation using the -A option with SSH. This also connects you to your workstation:

    ssh -A username@workstation_domain_or_IP

    Now you can connect to your Chef server from your workstation without needing a password, using the forwarded SSH credentials.

  5. Copy the testuser.pem and scaleway-file.pem files from the Chef server to the folder you created in your Chef workstation with the following commands:

    scp root@chef_server_domain_or_IP:/root/testuser.pem ~/chef_repo/.chef
    scp root@chef_server_domain_or_IP:/root/scaleway-file.pem ~/chef_repo/.chef

    If you connect to your Chef server using a non-root user, the commands will look more like this:

    scp username@chef_server_domain_or_IP:/home/username/testuser.pem ~/chef_repo/.chef
    scp username@chef_server_domain_or_IP:/home/username/scaleway-file.pem ~/chef_repo/.chef

Option 2: Downloading Keys when Connecting to a Chef Server with Passwords

Note:

By default, you cannot connect to a Scaleway Instance or Elastic Metal server with a password. For more information, refer to the dedicated Ubuntu Forum

In this case we will construct the scp commands directly. When executing the scp commands, you will be asked for the password to connect to your Chef server.

From your Chef workstation, construct the scp commands as follows:

  1. Specify the username and domain name or IP address used to connect to the Chef server.

  2. Follow this immediately with a colon (:) and the path to the file you wish to download.

  3. After adding a space, indicate the directory on the local computer where you wish the download the files to be placed (~/chef_repo/.chef in our case).

If you log into the Chef server using the root user account, your commands will look something like this.

scp root@chef_server_domain_or_IP:/root/testuser.pem ~/chef_repo/.chef
scp root@chef_server_domain_or_IP:/root/scaleway-file.pem ~/chef_repo/.chef

If you connect to your Chef server using a non-root user, the commands will look more like this:

scp username@schef_server_domain_or_IP:/home/username/testuser.pem ~/chef_repo/.chef
scp username@chef_server_domain_or_IP:/home/username/scaleway-file.pem ~/chef_repo/

Configuring Knife to Manage your Chef Environment

knife is a command-line tool that provides an interface between a local Chef repo and the Chef server.

Now that you have your Chef credentials available on your workstation, we can configure the knife command with the information it needs to connect to and control your Chef infrastructure.

This is done through a knife.rb file that we will place in the ~/chef_repo/.chef directory along with our keys.

  1. Create and open a file called knife.rb in that directory in your text editor:

    nano ~/chef_repo/.chef/knife.rb
  2. Paste the following information replacing the information with your infrastructure information:

    current_dir = File.dirname(__FILE__)
    log_level :info
    log_location STDOUT
    node_name "name_for_workstation"
    client_key "#{current_dir}/name_of_user_key"
    validation_client_name "organization_validator_name"
    validation_key "#{current_dir}/organization_validator_key"
    chef_server_url "https://server_domain_or_public_IP/organizations/organization_name"
    syntax_check_cache_path "#{ENV['HOME']}/.chef/syntaxcache"
    cookbook_path ["#{current_dir}/../cookbooks"]
    • node_name: Specifies the name that knife will use to connect to your Chef server. This should match your username.
    • client_key: Name and path to the user key that you copied over from the Chef server. We can use the #{current_dir} snippet to fill in the path if the key is in the same directory as the knife.rb file.
    • validation_client_name: Name of the validation client that knife will use to bootstrap new nodes. This will take the form of your organization short name, followed by -validator.
    • validation_key: Like the client_key, this includes the name and path to the validation key you copied from the Chef server. Again, you can use the #{current_dir}.
    • chef_server_url: This is the URL where the Chef server can be reached. It should begin with https://, followed by your Chef server’s domain name or Public IP address. Afterwards, the path to your organization should be specified by appending /organizations/your_organization_name.

    For our tutorial, the file looks this:

    current_dir = File.dirname(__FILE__)
    log_level :info
    log_location STDOUT
    node_name "testuser"
    client_key "#{current_dir}/testuser.pem"
    validation_client_name "scaleway-validator"
    validation_key "#{current_dir}/scaleway-file.pem"
    chef_server_url "https://chef_server_domainname_or_IP/organizations/scaleway"
    syntax_check_cache_path "#{ENV['HOME']}/.chef/syntaxcache"
    cookbook_path ["#{current_dir}/../cookbooks"]
  3. Save and close the knife.rb file.

  4. Test the configuration file. We need to be in our ~/chef-repo directory for our configuration file to be read correctly:

    cd ~/chef_repo
    knife client list

    which returns an error similar to:

    ERROR: SSL Validation failure connecting to host: 51.15.240.56 - SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate)
    ERROR: Could not establish a secure connection to the server.
    Use `knife ssl check` to troubleshoot your SSL configuration.
    If your Chef Server uses a self-signed certificate, you can use
    `knife ssl fetch` to make knife trust the server's certificates.

    Original Exception: OpenSSL::SSL::SSLError: SSL Error connecting to https://51.15.240.56/organizations/scaleway/clients - SSL_connect returned=1 errno=0 state=error: certificate verify failed (self signed certificate)

    This occurs because we do not have our Chef server’s SSL certificate on our workstation.

  5. Launch the following command:

    knife ssl fetch

    which returns

    WARNING: Certificates from public_IP will be fetched and placed in your trusted_cert
    directory (/root/chef_repo/.chef/trusted_certs).

    Knife has no means to verify these are the correct certificates. You should
    verify the authenticity of these certificates after downloading.

    Adding certificate for public_IP in /root/chef_repo/.chef/trusted_certs/public_IP.crt

    After the SSL certificate has been fetched, the previous command should now work.

  6. Launch the following command:

    knife client list

    which returns

    scaleway-validator

    If the above command correctly returns, your workstation is now set up to control your Chef environment.