Building Microservices in Go : Part 4 (Environment Variables, Validators, CRUD)

Hi everyone !!. Welcome to Part 4 in our series of building Micro-services using Go. I’ll list the previous part here just in case you guys missed them

1. Part 1 (Basic Structure Setup)
2. Part 2 (Live Reload using Air)
3. Part 3 (Database, Models, Migrations)

We’ve come a long way since we all started this journey together so let’s take a moment to appreciate our progress

In this part we are going to configure environment variables, setup validators and define CRUD endpoints for the author model that we created in the last part.

Let’s start by talking about environment variables and why we need them. There can be multiple reasons behind using env vars but the biggest use case that I use them for is to hide and manage secrets like passwords, credentials, access and secret keys, basically anything that I don’t want to include in my git repository.

So now that we know why we want to use env variables let’s talk about the how? We’ll start by adding in the required dependency

cd services/web_blog
go get github.com/joho/godotenv

Now let’s start our service by running

docker-compose up

Now we are going to create two files .env and .env.example. Our directory structure should look like this :

We will add the first file to our .gitignore so it doesn’t get included in the working tree and we are going to maintain the second file so that if any other developer comes to work on the project, they can know which vars they have to include to get the project working. For now we will only be storing our database credentials in the .env file.

.gitignore:

.env.example :

We add dummy values to the example files so that other devs can get a gist of what they need to add in here. We are going to populate the .env files with actual values now

.env :

Now let’s use these vars in our code. Let’s go to the main.go file and first add in our dependency:

import (
  ....
 "github.com/joho/godotenv"
 "os"
  ....
)

And then right above the app initialization let’s add this snippet

if err := godotenv.Load(); err != nil {
  fmt.Println(err)
 }

We called the load function which by default looks for a .env file in the root directory. We then checked for errors and if all goes well we shouldn’t see anything printed on the terminal. Let’s verify our env vars by printing one of the them

 fmt.Println(os.Getenv("DB_HOST"))

We should be able to see db printed in the terminal. Let’s modify our database.go file so that it reads from env vars

 var dbUrl = "postgresql://postgres:postgres@db:5432/web_microservice?sslmode=disable"

Replace this line of code with :

dbUrl := fmt.Sprintf("postgresql://%s:%s@%s:%s/%s?sslmode=disable",
  os.Getenv("DB_USER"), os.Getenv("DB_PASSWORD"), os.Getenv("DB_HOST"), os.Getenv("DB_PORT"), os.Getenv("DB_NAME"))

This is pretty basic stuff. We just substituted our env vars into our connection string. So now that we’ve configured our env vars we are going to create CRUD endpoints for the author model. We are going to start by creating a route directory in our web_blog folder and then we will create a file author_routes.go inside that directory. So your directory should look like this :

Now we are going to add in our functions here :

We’ve just defined a few functions and all of them are returning a string for now. Let’s open our main.go file and specify the signatures of our endpoints. I am going to remove the test endpoint that we made in the last part and replace it with a function which will hold the signatures for all our endpoints.

The endpoints are self explanatory. We defined five endpoints which can add a new author, get all authors, get a single author, delete and update an author. We can go and test these endpoints on postman

Now before we start implementing our endpoints we need to add validation for our post bodies. Why do we want to this ? Well let’s imagine a scenario where you’re adding a movie record to the database. There are multiple fields some of which are optional and some of those are required so one way to validate this would be to write if checks on each parameter for instance

if len(movie_title) > 0 and len(movie_plot) > 0 and len(movie_excerpt) > 0 :
  //process information 

This approach is very tedious and you end up writing hundreds of lines of code to implement something as basic as validations. Good thing for us is that go has an amazing validation package which we will use in order to write clean, optimized and readable code. We can specify a lot of validations like the min, max characters, if the field should be valid email or if its optional etc.

First we are going to add in our dependency by running :

go get github.com/go-playground/validator/v10

Let’s create a new folder “validators” and inside it we will create a file “validators.go”. Your directory structure should look like this :

I’ll break down the validators.go into three parts

1. Define a struct that would handle errors for us
2. Create a validator object
3. Create a function that validates post body and return error if there are any

Let’s see how to do this in code. We’ll start by declaring our package and adding imports

package validators

import "github.com/go-playground/validator/v10"

Now according to our check list we need to specify a struct to handle errors

type ErrorResponse struct {
 FailedField string
 Tag         string
}

Failed field will show us the field that failed the validation and tag will show us which validation failed. Next we are going to create a validator object

var validate = validator.New()

Seems simple enough. Now we are going to define our function

func ValidateStruct(postBody interface{}) []*ErrorResponse {
 var errors []*ErrorResponse
 err := validate.Struct(postBody)
 if err != nil {
  for _, err := range err.(validator.ValidationErrors) {
   var element ErrorResponse
   element.FailedField = err.StructNamespace()
   element.Tag = err.Tag()
   errors = append(errors, &element)
  }
 }
 return errors
}

This function takes in an interface which will be our post body and returns list of ErrorResponse type which we defined above. It basically checks if there are any validation errors and returns them. Let’s define our first validator struct which we will use when adding in a new author. So in the validators.go let’s add another struct

type AuthorAddPostBody struct {
 Title string `json:"title" validate:"required"`
}

We define a single field title and set its validation to required. You can add in a lot of parameters which you can find here. Now before we start implementing our endpoints, I want to create a separate file where I can store my utility functions. So let’s create a directory “utilities” and then a file inside it “utilities.go”. Your directory structure should look like this :

I am going to add a simple function here that returns us a dictionary with two fields, status and message, which we will use as our base response object. utilities.go should look like this

package utilities

func GetBaseResponseObject() map[string]interface{} {
 response := make(map[string]interface{})
 response["status"] = "fail"
 response["message"] = "something went wrong"
 return response
}

We also need to update our database.go to give us access to the ORM layer by making the following changes. First we add in the ORM field in our DBInstance struct like this :

type DBInstance struct {
 Db  *sql.DB
 Orm orm.Ormer
}

And then right before we initialize our DBInstance variable in ConnectDB() function, in the last else block, we are going to create an instance of ORM layer and pass it to our DBInstance so we can use it

   orm.Debug = true
   o := orm.NewOrm()
   Database = DBInstance{Db: db, Orm: o}

We set the debug field to true so we can see the queries in our terminal. Now that we are done with the configuration part (finally), we are going to start implementing our endpoints starting with AddAuthor so let’s open the author_routes.go file.

So let me break down what we are going to do.

1. Get a base response object
2. Create a validator object
3. Verify if the post body is in proper JSON format. Check errors if there are none move forward.
4. Validate the post body. Check errors if there are none move forward.
5. And if everything goes well we are going to insert the record into our database

Let’s see what the code looks like for AddAuthor

func AddAuthor(c *fiber.Ctx) error {
 response := utilities.GetBaseResponseObject()
 postBody := &validators.AuthorAddPostBody{}

 if err := c.BodyParser(postBody); err != nil {
  response["error"] = err.Error()
  return c.Status(fiber.StatusInternalServerError).JSON(response)
 } else {
  if err := validators.ValidateStruct(postBody); err != nil {
   return c.Status(fiber.StatusInternalServerError).JSON(err)
  } else {
   author := models.Authors{Title: postBody.Title}
   if _, err := database.Database.Orm.Insert(&author); err != nil {
    response["error"] = err.Error()
    return c.Status(fiber.StatusInternalServerError).JSON(response)
   } else {
    response["message"] = "Author successfully added"
    response["status"] = "pass"
    return c.Status(fiber.StatusCreated).JSON(response)
   }
  }
 }
}

So this is exactly what we did in the code: we get the base response object, create a validator object, check JSON structure, validate post body and then finally use our ORM layer to insert the record. Let’s try running this in postman. We are going to check all three cases: without JSON, with proper JSON but missing fields and then with proper fields

Without Proper JSON

With Proper JSON but missing fields

If I don’t add in the title field then the validation kicks in and sends this response back to me. Now lets check by passing the proper field

We are able to get the successful message. Let’s see if the record also got added into our Database

PGAdmin

And voila we got our endpoint working .

I had no idea that this would turn out to be so lengthy when I started writing so this seems like a good point to end the article. In the next one we are going to continue implementing the rest of the endpoints, integrate swagger and deploy our service to cloud run. You can find the code for this part here.