Managing PostgreSQL instance on RDS using Pulumi and Go
New year, new tool!
I have been dabbling with Pulumi since a couple of months, which is an Infrastructure as Code tool. I would not call myself an expert at this stage, but having closely worked with IaC it was easy for me to learn Pulumi. This is the first ever post I have written on Pulumi, and in this post I discuss how easy it is to spin up a PostgreSQL database instance in RDS.
Provisioning
Creating a relational database in RDS is easy. But the database experts would disagree with this statement. Data is important, and thus how we configure the database for security and efficiency matters a lot. Spinning a basic PostgreSQL instance is a matter of few seconds, whether you do it with web console or any API calls. But to configure the same, there are hundreds of parameters available.
The reason I tried to create and configure RDS database using Pulumi is that I really wanted to experience for myself - how exactly can we configure the database instance parameters. It was as expected - straightforward.
The code below provisions a basic PostgreSQL instance. I have used Pulumi’s Golang SDK, and the same is achievable using TS, Java, Python, C#, and YAML.
package main
import (
"github.com/pulumi/pulumi-aws/sdk/v6/go/aws/rds"
"github.com/pulumi/pulumi/sdk/v3/go/pulumi"
)
func main() {
pulumi.Run(func(ctx *pulumi.Context) error {
// Create the database instance
db, err := rds.NewInstance(ctx, "myrds", &rds.InstanceArgs{
AllocatedStorage: pulumi.Int(10),
DbName: pulumi.String("mypgdb"),
Engine: pulumi.String("postgres"),
EngineVersion: pulumi.String("15.4"),
InstanceClass: pulumi.String("db.t3.micro"),
Password: pulumi.String("pass1234"),
SkipFinalSnapshot: pulumi.Bool(true),
Username: pulumi.String("bob"),
})
if err != nil {
return err
}
})
}
Since this post focuses on managing PG Database using Pulumi, I will not go through the details of Pulumi here. However, the code above is fairly readable. The IaC code in Golang to create RDS PostgreSQL instance is wrapped in the pulumi.Run() function.
rds.NewInstance() function defines several parameters like allocated storage, database name, engine and it’s version, etc. Of course, you can specify more arguments, and here is the link to Pulumi docs for reference. The parameters specified here are basic - enough to create a database instance.
Let’s try to provision this database using Pulumi’s up command, and observe the output below.
pulumi up
Previewing update (aws-rds)
View in Browser (Ctrl+O): <URL to Pulumi dashboard>
Type Name Plan
+ pulumi:pulumi:Stack aws-rds-aws-rds create
+ └─ aws:rds:Instance myrds create
Resources:
+ 2 to create
Do you want to perform this update? yes
Updating (aws-rds)
View in Browser (Ctrl+O): <URL to Pulumi dashboard>
Type Name Status
+ pulumi:pulumi:Stack aws-rds-aws-rds created (210s)
+ └─ aws:rds:Instance myrds created (205s)
Resources:
+ 2 created
Duration: 3m33sNote: Since I am using Pulumi’s backend to manage state and stack information, I have removed the URL information from above output.
Running pulumi up command has confirmed the creation of RDS instance myrds within a stack named aws-rds-aws-rds. Pulumi creates a stack to manage resources being provisioned in the current directory/project. It counts the stack as one resource, due to which we see that 2 resources are created in the duration of 3m33s.
Fetching the database info after provisioning
If we check the AWS console, we can find the newly created database. It is also possible to configure output variables to print certain information of the newly created. Some information like the database endpoint is only available after the provisioning process. It is convenient to have relevant information available in the console output for various purposes.
Let us update the code above by adding ctx.Export function as shown below. Place this function before the return of Run() function.
//Get database connection string
ctx.Export("DB endpoint", db.Endpoint)Here, we are accessing the database endpoint output of the Pulumi run. The Export() function saves this output in the state managed on Pulumi’s dashboard, as well as outputs the same in console. The screenshot below confirms the same.
Also observe the console output. You will find the same information there.
...
Outputs:
DB endpoint: "myrdsf487fca.cosay9dfuboe.eu-central-1.rds.amazonaws.com:5432"
Resources:
+ 2 created
Duration: 3m33sConfiguring PostgreSQL database parameters
As discussed previously, various types of relational database have their specific set of configuration parameters. These configuration parameters help database administrators to apply more settings as per their needs. In AWS RDS, this is achieved by creating a Parameter Group and associating the same with relevant database instance.
The arguments we used in the NewInstance() functions apply to all kinds of databases to be provisioned in AWS RDS. However, since we have provisioned Postgres database and would like to further configure the same, we need to use another Pulumi resource to create Parameter Groups. In the code below, the function rds.NewParameterGroup() provisions a parameter group. Here we have specified a couple of parameters - autovaccum and deadlock_timeout - which are specific to Postgres.
Note: Refer to this link to find reference to more parameters.
If you are following along, place this function before rds.NewInstance() function.
//Parameter group for PostgreSQL
pgParameterGroup, err := rds.NewParameterGroup(ctx, "pg-param-group", &rds.ParameterGroupArgs{
Family: pulumi.String("postgres15"),
Parameters: rds.ParameterGroupParameterArray{
&rds.ParameterGroupParameterArgs{
ApplyMethod: pulumi.String("immediate"),
Name: pulumi.String("autovacuum"),
Value: pulumi.String("1"),
},
&rds.ParameterGroupParameterArgs{
ApplyMethod: pulumi.String("immediate"),
Name: pulumi.String("deadlock_timeout"),
Value: pulumi.String("10"),
},
// ... more Parameters
},
})
if err != nil {
return err
}The code above will independently provision a parameter group with above values, but will not associate the same to with our Postgres database instance. To do the same, add one argument named “ParameterGroupName” to the NewInstance() function we previously wrote and associate it with the parameter group (pgParameterGroup) we created in above function, and as shown below.
// Create the database instance
db, err := rds.NewInstance(ctx, "myrds", &rds.InstanceArgs{
AllocatedStorage: pulumi.Int(10),
DbName: pulumi.String("mypgdb"),
Engine: pulumi.String("postgres"),
EngineVersion: pulumi.String("15.4"),
InstanceClass: pulumi.String("db.t3.micro"),
Password: pulumi.String("pass1234"),
SkipFinalSnapshot: pulumi.Bool(true),
Username: pulumi.String("bob"),
ParameterGroupName: pgParameterGroup.Name,
})
if err != nil {
return err
}This time when we run “pulumi up” the output confirms that we are creating an additional resource - Parameter Group.
pulumi up
Previewing update (aws-rds)
View in Browser (Ctrl+O): <URL to Pulumi dashboard>
Type Name Plan
+ pulumi:pulumi:Stack aws-rds-aws-rds create
+ ├─ aws:rds:ParameterGroup pg-param-group create
+ └─ aws:rds:Instance myrds create
Outputs:
DB endpoint: output<string>
Resources:
+ 3 to create
Do you want to perform this update? yes
Updating (aws-rds)
View in Browser (Ctrl+O): <URL to Pulumi dashboard>
Type Name Status
+ pulumi:pulumi:Stack aws-rds-aws-rds created (307s)
+ ├─ aws:rds:ParameterGroup pg-param-group created (2s)
+ └─ aws:rds:Instance myrds created (299s)
Outputs:
DB endpoint: "myrds26be326.cosay9dfuboe.eu-central-1.rds.amazonaws.com:5432"
Resources:
+ 3 created
Duration: 5m10sAfter a successful Pulumi execution, login to AWS console and verify if the parameter group is associated with the Postgres database instance.
Conclusion
This was a small demonstration of how can we provision and configure RDS Postgres database using Pulumi. If you would like me to write more similar articles, feel free to subscribe and send me a note.