CRUD with DynamoDB Using Serverless and NodeJS

an introduction

In this post, we will see how to create a CRUD application using DynamoDB, AWS Serverless and NodeJS, we will cover all CRUD operations like DynamoDB GetItem, PutItem, UpdateItem, DeleteItem and list all items in a table. Everything will be done using the AWS Serverless framework and on NodeJS, this is the first part of this series, in part 2 we will add authentication to this application, for now, let’s get started.

Project preparation

Our project folder structure will look like this

Project folder structure

Let’s discuss what each of these volumes does

configuration – This folder will contain all configuration related files, in our case it keeps one file that creates DynamoDB AWS SDK instance to use everywhere in our project, so instead of importing DynamoDB instance in each file we just import it in one file and then export the instance from this File and import anywhere else.

mission – This is for keeping all files related to any utility functionality.

Mail – This is the main folder that will contain all the lambda functions for our CRUD operations.

Serverless.yml . file

This file is the soul and heart of every serverless project, let’s try to see in parts what this file looks like to us

service: dynamodb-crud-api

provider:
name: aws
runtime: nodejs12.x
environment:
DYNAMO_TABLE_NAME: PostsTable
iamRoleStatements:
- Effect: Allow
Action:
- dynamodb:Query
- dynamodb:Scan
- dynamodb:GetItem
- dynamodb:PutItem
- dynamodb:UpdateItem
- dynamodb:DeleteItem
Resource: "arn:aws:dynamodb:${opt:region, self:provider.region}:*:table/${self:provider.environment.DYNAMO_TABLE_NAME}"

Here we define one environment variable which will store our DynamoDB table name and we also add different permissions that our lambda functions will need to do different operations like dynamodb:GetItem To get the data element from the table, dynamodb:PutItem To insert a new entry into the table, etc.

lambda functions

Now we will define all lambda functions with their own configuration

functions:
listPosts:
handler: post/list.listPosts
events:
- http:
path: posts/list
method: get
cors: true

createPost:
handler: post/create.createPost
events:
- http:
path: post/create
method: post
cors: true

getPost:
handler: post/get.getPost
events:
- http:
path: post/{id}
method: get
cors: true

updatePost:
handler: post/update.updatePost
events:
- http:
path: post/update
method: patch
cors: true

deletePost:
handler: post/delete.deletePost
events:
- http:
path: post/delete
method: delete
          cors: true

We are now defining all of our lambda functions that will be called when we send requests to our API gateway URLs, and an HTTP event is attached to each lambda function so it can be called through the API gateway.

road – This is the endpoint relative path that we want to use, for example, if the API Gateway URL is https://abc.com and then getPost The lambda function will be called with this endpoint https://abc.com/post/{id}.

method – This is only API request type, POST, GET, DELETE, etc.

Before using DynamoDB, be sure to check out AWS DynamoDB Pricing so you only spend what you want.

Define a DynamoDB table

Finally, we need to define and configure our DynamoDB table

resources:
Resources:
UsersDynamoDbTable:
Type: AWS::DynamoDB::Table
DeletionPolicy: Retain
Properties:
AttributeDefinitions:
- AttributeName: id
AttributeType: S
KeySchema:
- AttributeName: id
KeyType: HASH
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
        TableName: ${self:provider.environment.DYNAMO_TABLE_NAME}

Attribute definitions – Here we define all the main fields of our table and our indicators.

KeySchema – Here we set any field we defined in AttributeDefinitions as a key field, either the sort key or the partition key.

Gross Output – Here we specify the number of read and write capacity units for our DynamoDB table.

Entire serverless.yml file

service: dynamodb-crud-api

provider:
name: aws
runtime: nodejs12.x
environment:
DYNAMO_TABLE_NAME: PostsTable
iamRoleStatements:
- Effect: Allow
Action:
- dynamodb:Query
- dynamodb:Scan
- dynamodb:GetItem
- dynamodb:PutItem
- dynamodb:UpdateItem
- dynamodb:DeleteItem
Resource: "arn:aws:dynamodb:${opt:region, self:provider.region}:*:table/${self:provider.environment.DYNAMO_TABLE_NAME}"

functions:
listPosts:
handler: post/list.listPosts
events:
- http:
path: posts/list
method: get
cors: true

createPost:
handler: post/create.createPost
events:
- http:
path: post/create
method: post
cors: true

getPost:
handler: post/get.getPost
events:
- http:
path: post/{id}
method: get
cors: true

updatePost:
handler: post/update.updatePost
events:
- http:
path: post/update
method: patch
cors: true

deletePost:
handler: post/delete.deletePost
events:
- http:
path: post/delete
method: delete
cors: true

resources:
Resources:
UsersDynamoDbTable:
Type: AWS::DynamoDB::Table
DeletionPolicy: Retain
Properties:
AttributeDefinitions:
- AttributeName: id
AttributeType: S
KeySchema:
- AttributeName: id
KeyType: HASH
ProvisionedThroughput:
ReadCapacityUnits: 1
WriteCapacityUnits: 1
        TableName: ${self:provider.environment.DYNAMO_TABLE_NAME}

configuration file

This config file will be inside a folder named config in our project as shown above in the project structure image, and this file will contain the code that will export the DynamoDB AWS SDK instance so we can call the DynamoDB APIs anywhere we want in other parts of the code.

const AWS = require("aws-sdk");

const dynamo = new AWS.DynamoDB.DocumentClient();

module.exports = dynamo;

job profile

In this project we are using a single file which contains all the auxiliary/common functions that we will be using many times in our project.

const sendResponse = (statusCode, body) => {
const response = {
statusCode: statusCode,
body: JSON.stringify(body),
headers: {
'Content-Type': 'application/json',
'Access-Control-Allow-Origin': '*',
'Access-Control-Allow-Credentials': true
}
}
return response
}

module.exports = {
sendResponse
};

We will call this sendResponse A function from several places, actually from all of our lambda functions to return the response to a request, this will return the JSON response back to the user, it has two arguments, one is an HTTP status code and one is the body of JSON that we will pass whenever we will call this function, we also pass some headers Required with response dealing with the most common cors issues “Access not allowed”.

lambda functions

Now its time to start working on our lambda function which will hold all our logic, we will split four lambda functions for four different CRUD operations.

DynamoDB PutItem (insert)

lambda function is defined inside a file create.js file , in this lambda function we will do our first operation which is to insert a new data element into the table, let’s break it into parts.

const dynamoDb = require("../config/dynamoDb");
const { sendResponse } = require("../functions/index");
const uuidv1 = require("uuid/v1");

We need to import our DynamoDB instance from the config file we created earlier, we have sendReponse The function we use NPM called uuid Which is used to generate a random identifier, this identifier will be used as a partition key for each post.

 const body = JSON.parse(event.body);
try {
const { postTitle, postBody, imgUrl, tags } = body;
const id = uuidv1();
const TableName = process.env.DYNAMO_TABLE_NAME;
const params = {
TableName,
Item: {
id,
postTitle,
postBody,
imgUrl,
tags
},
ConditionExpression: "attribute_not_exists(id)"
    };

Here we get different properties from the request payload which we are going to include in our post table, after that we generate a random identifier by calling a function provided by uuid Library.

Attribute_none – By default, DynamoDB PutItem will overwrite the content of any element if we are trying to enter data with the same partition key, but we don’t want it to only enter data if partition key is not found, we use this conditional expression.

await dynamoDb.put(params).promise();
return sendResponse(200, { message: 'Post created successfully' })

We pass the parameters we created in the previous step into the DynamoDB put API call and send the 200 status code with the relevant message.

The entire create.js file
"use strict";

const dynamoDb = require("../config/dynamoDb");
const { sendResponse } = require("../functions/index");
const uuidv1 = require("uuid/v1");

module.exports.createPost = async event => {
const body = JSON.parse(event.body);
try {
const { postTitle, postBody, imgUrl, tags } = body;
const id = uuidv1();
const TableName = process.env.DYNAMO_TABLE_NAME;
const params = {
TableName,
Item: {
id,
postTitle,
postBody,
imgUrl,
tags
},
ConditionExpression: "attribute_not_exists(id)"
};
await dynamoDb.put(params).promise();
return sendResponse(200, { message: 'Post created successfully' })
} catch (e) {
return sendResponse(500, { message: 'Could not create the post' });
}
};

DynamoDB GetItem (read)

The lambda function is defined inside get.js File, this will perform the read operation, which means getting the data from DynamoDB using the partition key.

const { id } = event.pathParameters;
const params = {
TableName: process.env.DYNAMO_TABLE_NAME,
KeyConditionExpression: "id = :id",
ExpressionAttributeValues: {
":id": id
},
Select: "ALL_ATTRIBUTES"
 };

We get the id from the request parameters, then match that to the partition key in our table and select all the fields from the table.

const data = await dynamoDb.query(params).promise();
if (data.Count > 0) {
return sendResponse(200, { item: data.Items });
} else {
return sendResponse(404, { message: "Post not found" });
}

Now we query the table with parameters and check if there are any items returned or not If any items were found then we return the array of items otherwise we return an appropriate message.

Entire get.js file
"use strict";

const { sendResponse } = require("../functions/index");
const dynamoDb = require("../config/dynamoDb");

module.exports.getPost = async event => {
try {
const { id } = event.pathParameters;
const params = {
TableName: process.env.DYNAMO_TABLE_NAME,
KeyConditionExpression: "id = :id",
ExpressionAttributeValues: {
":id": id
},
Select: "ALL_ATTRIBUTES"
};

const data = await dynamoDb.query(params).promise();
if (data.Count > 0) {
return sendResponse(200, { item: data.Items });
} else {
return sendResponse(404, { message: "Post not found" });
}
} catch (e) {
return sendResponse(500, { message: "Could not get the post" });
}
};

DynamoDB UpdateItem

The lambda is defined inside update.js file, in this lambda function we will do the update process which will update the data inside the DynamoDB table.

const body = JSON.parse(event.body);

const { postTitle, postBody, imgUrl, tags, id } = body
const params = {
TableName: process.env.DYNAMO_TABLE_NAME,
Key: {
id
},
ExpressionAttributeValues: {
":postTitle": postTitle,
":postBody": postBody,
":imgUrl": imgUrl,
":tags": tags
},
UpdateExpression:
"SET postTitle = :postTitle, postBody = :postBody, imgUrl = :imgUrl, tags = :tags",
ReturnValues: "ALL_NEW"
    };

We get the data from the request payload, and there is an additional property that we need to send with the request which is id of the item we want to update.

ExpressionAttributeValues – DynamoDB has many reserved keywords, so there may be a case where our table field name matches that reserved keyword, in which case this update will throw an error. To avoid this, DynamoDB has a system to set the original field name with some alias temporarily just for this purpose, so we set all the field values ​​in this object.

to update – To update any element in DynamoDB we need to pass the field name along with its update expression.

return values ​​- This just indicates that we need the updated fields data in the response when we run our update process.

const data = await dynamoDb.update(params).promise();
if (data.Attributes) {
return sendResponse(200, data.Attributes);
} else {
return sendResponse(404, {
message: "Updated post data not found"
});
}

Now we just need to call update API with parameters, we also check if the updated attributes data is returned or not, if yes then we will return this data otherwise we return 404 status code with message.

Entire update.js file
"use strict";

const { sendResponse } = require("../functions/index");
const dynamoDb = require("../config/dynamoDb");

module.exports.updatePost = async event => {
try {
const body = JSON.parse(event.body);

const { postTitle, postBody, imgUrl, tags, id } = body
const params = {
TableName: process.env.DYNAMO_TABLE_NAME,
Key: {
id
},
ExpressionAttributeValues: {
":postTitle": postTitle,
":postBody": postBody,
":imgUrl": imgUrl,
":tags": tags
},
UpdateExpression:
"SET postTitle = :postTitle, postBody = :postBody, imgUrl = :imgUrl, tags = :tags",
ReturnValues: "ALL_NEW"
};

const data = await dynamoDb.update(params).promise();
if (data.Attributes) {
return sendResponse(200, data.Attributes);
} else {
return sendResponse(404, { message: "Updated post data not found" });
}
} catch (e) {
return sendResponse(500, { message: "Could not update this post" });
}
};

DynamoDB DeleteItem

The lambda function will be in delete.js file, in this lambda function we will delete an element from the table.

"use strict";

const { sendResponse } = require("../functions/index");
const dynamoDb = require("../config/dynamoDb");

module.exports.deletePost = async event => {
try {
const body = JSON.parse(event.body);
const { id } = body;
const params = {
TableName: process.env.DYNAMO_TABLE_NAME,
Key: {
id
}
};
await dynamoDb.delete(params).promise();
return sendResponse(200, { message: "Post deleted successfully" });
} catch (e) {
return sendResponse(500, { message: "Could not delete the post" });
}
};

This lambda function is self-explanatory, we just get it id From the element we want to remove in the request and we pass that as a parameter in the DynamoDB delete API.

We are now done with all our 4 create/read/update/delete but still missing something, we don’t have any lambda function to list all posts, let’s look at how to do it.

DynamoDB Check

We will use a DynamoDB scan to get all the items from the table, and scans can be expensive while using DynamoDB so we need to be careful with it and try to avoid using it as much as possible, and even if we have to use it we should only get the data we need and not do unnecessary scans of the items .

"use strict";

const dynamoDb = require("../config/dynamoDb");
const { sendResponse } = require("../functions/index");

module.exports.listPosts = async event => {
try {
const params = {
TableName: process.env.DYNAMO_TABLE_NAME,
}
const posts = await dynamoDb.scan(params).promise();
return sendResponse(200, { items: posts.Items });
} catch (e) {
return sendResponse(500, { message: "Could not get the posts list" });
}
};

This job will be in list.js file, we do a very simple DynamoDB scan here and return the data.

conclusion

This was a long post, but if you were able to get to this point, congratulations, because you now have a full CRUD API built with DynamoDB, AWS Serverless and Nodejs, we can always improve and improve this app, here are some ideas –

  • Add authentication to the application.
  • Add posting on a user basis.
  • Add validations.
  • Which makes it more cost effective
  • Added DynamoDB Global Secondary indicator for better performance and cost reduction.

If you have any other suggestions, feel free to add them in the comments below, there will be more parts for this app, so stay tuned.

Get this code

Source code on github

.

Leave a Comment