Press "Enter" to skip to content

Using K2 with NoSQL (MongoDB + dynamic REST API)

0

Such scenarios are not common, that you need to use K2 with NoSQL databases. However, I would still like to show the possibility of a K2 platform to connect to NoSQL databases to ‘CRUD’ data. Moreover, the REST API I am trying to show is kind of dynamic and is also suitable for a K2 Cloud platform.

The key features of using K2 with NoSQL with the help of the below described REST API are the following ones:

  • K2 should not be extended with a ServiceBroker, i.e. K2 Cloud can also use that;
  • The collection you are working with can be specified in the url, which makes it dynamic;
  • Schema of the document depends on the swagger file definition, which also makes it dynamic.

Since K2 cannot directly connect to the MongoDB service, we need an intermediary layer for that. I decided to use Node.js with Express.js, a framework for quick and easy creation of REST API.

Architecture overview

1. Get MongoDB instance

I do not want to describe in detail, how you need to install MongoDB server. There is a lot of information, available on the internet. Instead, I will use Database-as-a-Service from mLab Service Provider. It is very easy to register and get your MongoDB for 500 Mb free. After you configure everything and create DB users, you will have the following url for MongoDB connection, which will contain some confidential data. We will use this connection string in our Express.js API.

mongodb://<dbuser>:<dbpassword>@ds211865.mlab.com:11865/<dbname>

2. Create Express project

Before we start, make sure you have installed Node.js on your machine. Otherwise, the commands below will not work for you. We will use an express-generator tool, which will create the skeleton for our project. Run the following command in your command line or bash to install the express-generator tool.

npm install express-generator -g

Next run the following command to create the project:

express --no-view k2-express-mongodb

You will see the following list of files, created inside your project:

Generated express app.

The project structure was installed, however, all the dependent node modules are missing. You need to install them. Otherwise, you will not be able to run and test your API. For that, navigate to your project folder and run the following command:

npm install

We are almost ready . But before we start writing any code, please, navigate to the newly created Express project and install the mongodb driver with the following command:

 npm install mongodb --save

To test your project, simply, run the command below and you should see the Welcome message in the browser, if you navigate to http://localhost:3000.

npm start

And now we are ready to code.

3. Add REST endpoints for MongoDB integration

I want to show an example of a dynamic REST API. My idea is that you can manage from a swagger file, with which collection you would like to work. So the endpoint should look like this:

  • GET /:collection – returns all documents from the collection (e.g. /request)
  • GET /:collection/:id – returns 1 document from a collection (e.g. /request/5)
  • POST /:collection – creates a new document in a collection (e.g. /request)
  • PATCH /:collection/:id – updates the document in a collection (e.g. /request/5)
  • DELETE /request/:id – deletes the request in a collection (e.g. /request/5)

Add mongo.js file into the routes folder. It will contain the code for all the basic methods to work with MongoDB. The most important points are the following ones:

a) middleware to handle authentication

In the very beginning I have added the Express.js middleware to handle authentication. To get access to MongoDB I need a login and a password. I decided not to store it inside the REST API, but to use Basic authentication options of the REST Broker and store it inside the Service Instance configuration.

By default, REST Broker does not send the Authorization Header (Basic …) in the very beginning. It will send the header only after it receives 401 Unauthorized response The response should also contain the headers, which tells the REST Broker the supported methods of authentication.

...
//Set specific Header for K2 Rest Broker
res.set('WWW-Authenticate', 'Basic realm="401"');
res.status(401).send('Authentication required.');
...

b) extract a login and a password for the endpoints

After I made sure the Basic authentication is present in the call, I am extracting it and decoding the string to use it in the MongoDB connection string. I attach the string to the request object so that the subsequent endpoints can use it inside.

...
//Getting credentials and use them for MongoDB authentication.
const base64Credentials = req.headers.authorization.split(' ')[1];
const credentials = Buffer.from(base64Credentials, 'base64').toString('ascii');
req.mongoUrl = mongodb://${credentials}@ds211865.mlab.com:11865/db-test;
next();
...

c) add all methods to handle the calls

Afterwards there are all the methods to handle the calls.

//Create - POST /request - Creates 1 item
router.post('/:collection', function(req, res, next){
const collection = req.params.collection;
const dataForInsert = req.body;
MongoClient.connect(req.mongoUrl, function(err, client){
const db = client.db();
db.collection(collection).insertOne(dataForInsert,
function(err, r){
res.send({_id:r.insertedId});
client.close();
}
);
})
});
//GET /request/:id - Returns 1 document
router.get('/:collection/:id', function(req, res, next){
const collection = req.params.collection;
const id = req.params.id;
MongoClient.connect(req.mongoUrl, function(err, client){
const db = client.db();
db.collection(collection).findOne({_id: new ObjectId(id)},
function(err, doc){
res.send(doc);
client.close();
}
);
})
});
//GET /request - returns all Documents
router.get('/:collection', function(req, res, next){
const collection = req.params.collection;
MongoClient.connect(req.mongoUrl, function(err, client){
const db = client.db();
db.collection(collection).find({},
async function(err, docs){
const results = await docs.toArray();
res.send(results);
client.close();
}
);
})
});
//PATCH /request/:id - Updates a document
router.patch('/:collection/:id', function(req, res, next){
const collection = req.params.collection;
const id = req.params.id;
const item = req.body;
MongoClient.connect(req.mongoUrl, function(err, client){
const db = client.db();
db.collection(collection).updateOne(
{id: new ObjectId(id)},
  {$set: .omitBy(item, _.isNil)},
function(err, doc){
res.send(doc);
client.close();
}
)
})
});
//DELETE /request/:id - Deletes a document
router.delete('/:collection/:id', function(req, res, next){
const collection = req.params.collection;
const id = req.params.id;
MongoClient.connect(req.mongoUrl, function(err, client){
const db = client.db();
db.collection(collection).deleteOne(
{_id: new ObjectId(id)},
function(err, doc){
res.send(doc);
client.close();
}
)
})
});

d) add the new router to the express app

In the app.js file it is necessary to add 1 line, to make the new router available for the calls.

...
app.use('/mongo', mongoRouter);
...

The full code for the REST API application can be found here:

4. Create swagger for REST Broker

In the swagger file you can define the paths, specifying the name of the collection and also the structure of the objects you would like to work with. I am using ‘request‘ as a collection name and the following url:

http://localhost:3000/mongo/request

The swagger file can be found here:

5. Create K2 REST Service Instance, SMO…

After you created a Service Instance on the basis of the swagger file, you can create SmartObjects and test.

ServiceTester

The code for the Web Application can be found on my GitHub.

The code is provided mostly for the demo purposes, to show how to use K2 with NoSQL. Before it can be used for production, some changes/modifications might be necessary. Below I can think about the following ones:

  • Error Handling – currently the errors are not handled in a correct way and are simply ignored.
  • Message format – I would recommend to change the format, how the data is sent, adding information, if the call was successful.
  • Optimization or adding correct data handling for MongoDB – the methods for getting a list of documents from MongoDB is quite resource consuming, when big amounts of data are involved. It is better to add filters or additional parameters.
  • Additional parameters – currently the mongoDB host, port and the database name are hard coded in the REST API code. It also makes sense to add those as additional parameters and send them from the REST ServiceBroker.

Read my next post, in which you can find out how to Deploy your REST API to Azure App Service.

If you want to know about other ways to integrate K2 with Azure, you can also read my article about Integration of K2 with Azure functions.

As you see, using K2 with NoSQL can be also dynamic to certain extent. If you have any questions or need help, feel free to contact me.

Leave a Reply

Your email address will not be published. Required fields are marked *

five + sixteen =