July 30, 2020
12 minutes
2097 words

In this article, I'm going to walk through using Pulumi to provision a couple of bare metal servers on Packet , using the user data to provision and configure a working SaltStack setup with a single master and single minion node.
NB: Packet is now known as Equinix Metal and Pulumi templates and commands have been updated with the new name. Remember to translate as you work through this.
Why Packet?
I joined Packet on July 27th, 2020 (3 days ago, at the time of writing) and I wanted a quick project that would allow me to try out the platform and see what fun I could have. Packet is a hosting provider that has commoditised bare metal compute over an API. Nice, right? 💯
Why SaltStack?
I've been using and advocating SaltStack since around 2015, it's my goto tool for configuration management and provides some very cool features that aren't available in alternatives, such as beacons , the reactor , and salt-cloud .
When you're running bare metal over an API, there's a few convenience factors that aren't available to you that you'd get for
vendor-lockin
free on other clouds, such as ASGs; so by providing a SaltStack backbone to our compute, we can begin to put these pieces together for ourselves. The old adage of
Cattle vs Pets
is still aplicable in a bare metal environment, but with different applications. I'll dive into this in a future post.
Why Pulumi?
Most people reach for Terraform. Hell, I used to. Terraform is fantastic. That being said ... while HashiCorp are making good progress on evolving HCL to support more code-like features with 0.12, why not use a programming language straight up? That's what Pulumi is.
Pulumi is a Infrastructure as Code (IaaC) tool that allows you to describe your infrastructure, much like Terraform . Unlike Terraform, Pulumi doesn't impose a specific DSL, HCL, on you; and instead, you can use your programming language of choice ... provided it's supported.
At the time of writing, Pulumi supports:
- C#
- F#
- Go
- JavaScript
- Python
- TypeScript
- VisualBasic
I know, I know. I'm disappinted Rust isn't there either. Maybe one day .
Walkthrough
Creating the Stack
So to create our stack, we need to generate a new Pulumi project. For this example, we'll use the Pulumi local login, which stores the statefile on our local disk. The statefile is very similar to Terraform state. It is needed to build an execution plan for our
apply
commands. Using local will suffice today, but you should investigate using
alternative options
for production deployments.
pulumi login --local
We're going to use the TypeScript template to get started. Unfortunately, there isn't a template for all supported languages, but templates do exist for the following:
packet-go # A minimal Packet.net Go Pulumi program
packet-javascript # A minimal Packet.net JavaScript Pulumi program
packet-python # A minimal Packet.net Python Pulumi program
packet-typescript # A minimal Packet.net TypeScript Pulumi program
If you want to use one of the dotNet languages, you can use a generic template; then add the Packet provider manually. Generic templates for dotNet are called:
csharp # A minimal C# Pulumi program
fsharp # A minimal F# Pulumi program
visualbasic # A minimal VB.NET Pulumi program
To create our project from the TypeScript template, let's run:
pulumi new packet-typescript
You'll be walked through a couple of questions to create your stack, after which you'll have a directory that looks like:
drwxr-xr-x - rawkode 30 Jul 18:07 node_modules
.rw------- 286 rawkode 30 Jul 18:06 index.ts
.rw-r--r-- 28k rawkode 30 Jul 18:07 package-lock.json
.rw------- 201 rawkode 30 Jul 18:06 package.json
.rw-r--r-- 85 rawkode 30 Jul 18:07 Pulumi.dev.yaml
.rw------- 100 rawkode 30 Jul 18:06 Pulumi.yaml
.rw------- 438 rawkode 30 Jul 18:06 tsconfig.json
If we take a look inside of
index.ts
, we'll see:
import * as pulumi from "@pulumi/pulumi";
import * as packet from "@pulumi/packet";
// Create a Packet resource (Project)
const project = new packet.Project("my-test-project", {
name: "My Test Project",
});
// Export the name of the project
export const projectName = project.name;
This TypeScript uses the Pulumi SDKs to provide a nice wrapper around the Packet API. Hopefully it's pretty self-explanatory; you can see that it creates a new project and exports its name.
Exports are similar to Terraform outputs. We use an export when we want to make some attribute from our stack available outside of Pulumi. Pulumi provides the
pulumi stack output
command, which displays these exports.
We'll use these later.
Cleaning Up the Stack
This step is completely subjective, but I don't like my code just chilling in the top level directory with all the other stuff. What is this, Go? 😏
Fortunately, we can append
main: src/index.ts
to the
Pulumi.yaml
file, which tells Pulumi our entry point for this stack lives somewhere else. I'm going to use a
src
directory.
mkdir src
mv index.ts src/
echo "main: src/index.ts" >> Pulumi.yaml
Creating a "Platform"
I like to create a
Platform
object / type / class that can be used to pass around the Pulumi configuration and some other common types that my Pulumi projects often need. This saves my function signatures getting too gnarly as we add new components to our stacks.
The
Platform
object I'm using for this is pretty trivial. It loads the Pulumi configuration and stores our Packet Project, which means we can pass around
Platform
to other functions and it's a single argument, rather than many.
// ./src/platform.ts
import { Config } from "@pulumi/pulumi";
import { Project } from "@pulumi/packet";
export type Platform = {
project: Project;
config: Config;
};
export const getPlatform = (project: Project): Platform => {
return {
project,
config: new Config(),
};
};
Now we can update our
./src/index.ts
to look like so:
import * as packet from "@pulumi/packet";
import { getPlatform } from "./platform";
const project = new packet.Project("pulumi-saltstack-example", {
name: "pulumi-saltstack-example",
});
const platform = getPlatform(project);
export const projectName = platform.project.name;
Creating the SaltMaster
Now we want to create the Salt Master server. For this, I create a new directory with an
index.ts
that exports a function called
createSaltMaster
; which I can consume in our
./src/index.ts
, much like we did with our
Platform
.
Lets start with the complete file, then we'll break it down; sound good? Good 😀
// ./src/salt-master/index.ts
import {
Device,
IpAddressTypes,
OperatingSystems,
Plans,
Facilities,
BillingCycles,
} from "@pulumi/packet";
import { Platform } from "../platform";
import * as fs from "fs";
import * as path from "path";
import * as mustache from "mustache";
export type SaltMaster = {
device: Device;
};
export const createSaltMaster = (
platform: Platform,
name: string
): SaltMaster => {
// While we're not interpolating anything in this script atm,
// might as well leave this code in for the time being; as
// we probably will shortly.
const bootstrapString = fs
.readFileSync(path.join(__dirname, "./user-data.sh"))
.toString();
const bootstrapScript = mustache.render(bootstrapString, {});
const saltMaster = new Device(`master-${name}`, {
hostname: name,
plan: Plans.C1LargeARM,
facilities: [Facilities.AMS1],
operatingSystem: OperatingSystems.Debian9,
billingCycle: BillingCycles.Hourly,
ipAddresses: [
{ type: IpAddressTypes.PrivateIPv4, cidr: 31 },
{
type: IpAddressTypes.PublicIPv4,
},
],
projectId: platform.project.id,
userData: bootstrapScript,
});
return {
device: saltMaster,
};
};
SaltMaster Return Type
Because this is TypeScript, we want to be very explicit about the return types within our code. This allows us to catch errors before we ever run our Pulumi stack. As we're using
createSaltMaster
function to create our SaltMaster, we want that function to return a type with the resources we create.
export type SaltMaster = {
device: Device;
};
While our function only returns a
Device
, it's still nice to encapsulate that in a named type that allows for our function to evolve over time.
This is the function signature for
createSaltMaster
. You can see our return type is the type we just created,
SaltMaster
.
import { Platform } from "../platform";
export const createSaltMaster = (
platform: Platform,
name: string
): SaltMaster =>
Our function also takes a couple of parameters, namely
platform
and
name
. The platform is our
Platform
object with our Pulumi configuration and the Packet Project, so we also need to import those too. The
name
allows us to give our SaltMaster a name when we create the device on Packet. We could hard code this inside the function as
salt-master
, but then we can't use
createSaltMaster
more than once for a highly available set up in a later tutorial. Always be thinking ahead, right? 😉
Provisioning with User Data
import * as fs from "fs";
import * as path from "path";
import * as mustache from "mustache";
const bootstrapString = fs
.readFileSync(path.join(__dirname, "./user-data.sh"))
.toString();
const bootstrapScript = mustache.render(bootstrapString, {});
I know ... I know what you're thinking ... but trust me, this will make sense.
As Pulumi allows us to use a feature complete programming language to describe our infrastructure, we also have access to that programming languages ENTIRE eco-system of libraries (npm). As such, if I want to template some user data for a server ... say, to provision and install SaltStack ... I can use a popular template tool, such as mustache, from npm 😉 This is useful because we're not only using a programming language we're familiar with to provision our server, but we can use the same libraries we're familiar with too. It's nice when a plan comes together.
My user data for the Salt Master looks like so: The user data for installing the Salt master is pretty simple and we're not actually using the mustache interpolation yet; however, it's good to bake this in early instead of using TypeScript/JavaScript literals for the user-data. Why? Because we can keep the file saved as
user-data.sh
, which means we get syntax highlighting in our IDE of choice as we make changes.
# ./src/salt-master/user-data.sh
#!/usr/bin/env sh
apt update
DEBIAN_FRONTEND=noninteractive apt install -y python-zmq python-systemd python-tornado salt-common salt-master
LOCAL_IPv4=$(ip addr | grep -E -o '10\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}')
cat <<EOF >/etc/salt/master.d/listen-interface.conf
interface: ${LOCAL_IPv4}
EOF
systemctl restart salt-master
Creating the Server
Lastly, we need to create the device with the Packet API. We use the Pulumi SDK to do so. I'm explicitly importing the required types that I need to use as much of the type system as I can.
Instead of hard coding strings like
c1-large-arm
, I can rely on enums provided by the SDK to verify that my string is correct. We do this for
BillingCycles
,
OperatingSystems
, and
IpAddressTypes
too. Funky! 🦩
import {
Device,
IpAddressTypes,
OperatingSystems,
Plans,
Facilities,
BillingCycles,
} from "@pulumi/packet";
const saltMaster = new Device(`master-${name}`, {
hostname: name,
plan: Plans.C1LargeARM,
facilities: [Facilities.AMS1],
operatingSystem: OperatingSystems.Debian9,
billingCycle: BillingCycles.Hourly,
ipAddresses: [
{ type: IpAddressTypes.PrivateIPv4, cidr: 31 },
{
type: IpAddressTypes.PublicIPv4,
},
],
projectId: platform.project.id,
userData: bootstrapScript,
});
Consuming our Function
Next up, we can call our
createSaltMaster
function from
./src/index.ts
and we'll have a server with the correct user data for running our
salt-master
.
const saltMaster = createSaltMaster(platform, "master-1");
export const saltMasterPublicIp = saltMaster.device.accessPublicIpv4;
We're going to export it's public IPv4 address; so that we can access it easily later and SSH into the machine later.
Creating the SaltMinion
// ./src/salt-minion/index.ts
import { interpolate } from "@pulumi/pulumi";
import {
BillingCycles,
Device,
Facilities,
OperatingSystems,
Plans,
} from "@pulumi/packet";
import { Platform } from "../platform";
import { SaltMaster } from "../salt-master";
import * as fs from "fs";
import * as path from "path";
import * as mustache from "mustache";
type SaltMinion = {
device: Device;
};
export const createSaltMinion = (
platform: Platform,
name: string,
master: SaltMaster
): SaltMinion => {
const bootstrapString = fs
.readFileSync(path.join(__dirname, "./user-data.sh"))
.toString();
const saltMinion = new Device(`minion-${name}`, {
hostname: name,
plan: Plans.C1LargeARM,
facilities: [Facilities.AMS1],
operatingSystem: OperatingSystems.Debian9,
billingCycle: BillingCycles.Hourly,
projectId: platform.project.id,
userData: master.device.accessPrivateIpv4.apply((ipv4) => {
return mustache.render(bootstrapString, { master_ip: ipv4 });
}),
});
return {
device: saltMinion,
};
};
I'm not going to go over the
createSaltMinion
code like I did the master, as they're rather similar. Lets cover the major difference.
Mustache interpolation
Unlike our Salt master user-data, we need to use the mustache interpolation for the minion. This is because our minions need to know where the master is, and this requires accessing some state from the Pulumi provisioning; namely the IPv4 address of our master. For this, we use mustache template syntax:
cat <<EOF >/etc/salt/minion.d/salt-master.conf
master: {{ master_ip }}
EOF
Instead of passing
{}
to the mustache render function, we now need to provide some variables. I know this looks weird, so lets explain.
userData: master.device.accessPrivateIpv4.apply((ipv4) => {
return mustache.render(bootstrapString, { master_ip: ipv4 });
}),
The
device.accessPrivateIpv4
variable is a Pulumi
Output
. That means that the variable isn't available until the server has been provisioned. So we can't access that variable directly, because its an promise that the value will exist later. As such, we need to use the
apply
function that's provided which defers the execution of the calling code until the server has been provisioned, thus the IP address is known.
This is where most people, including me, trip up with Pulumi at the beginning. Definitely read the programming model guide, which really helps understand what is going on; as well, it provides some good tips for working with outputs and inputs.
Other than this, our code is pretty much the same as the master. Our user data is a little different, but I'm going to assume that's self-explanatory too. Here's the user data, for completeness.
# ./src/salt-minion/user-data.sh
#!/usr/bin/env sh
apt update
DEBIAN_FRONTEND=noninteractive apt install -y python-zmq python-systemd python-tornado salt-common salt-minion
cat <<EOF >/etc/salt/minion.d/salt-master.conf
master: {{ master_ip }}
EOF
systemctl restart salt-minion
Spinning up the Stack
Before we spin this up, we need to provide our Packet API key as we create the stack.
pulumi stack init production
This will prompt you for a password which is used to encrypt your secrets, namely your Packet API key. This is because we're using a local provider for state. When you do this for your real production deployments, use the Pulumi managed service or Cloud KMS .
pulumi config set --secret packet:authToken
pulumi up
It'll show you a "plan" which you can now confirm and run if you're happy.
Previewing update (production):
Type Name Plan
+ pulumi:pulumi:Stack pulumi-saltstack-production create
+ ├─ packet:index:Project pulumi-saltstack-example create
+ ├─ packet:index:Device master-master-1 create
+ └─ packet:index:Device minion-minion-1 create
Resources:
+ 4 to create
Do you want to perform this update? [Use arrows to move, enter to select, type to filter]
yes
no
details
That's it! Accept the plan, and you'll have a Salt Master and Salt Minion running on bare metal on Packet cloud.
Confirming the Awesome
Of course, I don't expect you to take my word for it that this "just works" (it does).
Lets use the
pulumi stack
command to get the public IPv4 address of our Salt master.
pulumi stack output
Which gives us:
Current stack outputs (2):
OUTPUT VALUE
projectName pulumi-saltstack-example2
saltMasterPublicIp 147.75.81.106
Now we can SSH into our Salt master and confirm the setup.
ssh -l root 147.75.81.106
We can check that our Salt minion is configured correctly by checking the key approval was requested.
salt-key -L
Accepted Keys:
Denied Keys:
Unaccepted Keys:
minion-1
Rejected Keys:
We have one unaccepted key! Awesome. We can automate the approval of that in another session. Lets accept the key and test out our Salt magic.
salt-key -A minion-1
The following keys are going to be accepted:
Unaccepted Keys:
minion-1
Proceed? [n/Y] y
Key for minion minion-1 accepted.
Lets try a
test.ping
, a Salt command that makes sure the minions can be contacted by the master.
salt minion-1 test.ping
minion-1:
True
Finally, lets actually have our minion run a command and show us the output:
salt minion-1 cmd.run "ip addr"
minion-1:
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enP2p1s0f2: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
link/ether 70:10:6f:b9:eb:49 brd ff:ff:ff:ff:ff:ff
3: enP2p1s0f1: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc mq master bond0 state UP group default qlen 1000
link/ether 70:10:6f:b9:eb:49 brd ff:ff:ff:ff:ff:ff
4: bond0: <BROADCAST,MULTICAST,MASTER,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 70:10:6f:b9:eb:49 brd ff:ff:ff:ff:ff:ff
inet 147.75.81.34/30 brd 147.75.81.35 scope global bond0
valid_lft forever preferred_lft forever
inet 10.80.41.131/31 brd 255.255.255.255 scope global bond0:0
valid_lft forever preferred_lft forever
inet6 2604:1380:2000:5500::1/127 scope global
valid_lft forever preferred_lft forever
inet6 fe80::7210:6fff:feb9:eb49/64 scope link
valid_lft forever preferred_lft forever
Until next time 😀