It is really easy to setup language agnostic open, staggered client-server interactions in a bidirectional, streaming way using the gRPC framework

This is done due to the fact that gRPC runs on HTTP2, so resource multiplexing comes into the picture and helps client and server both keep pushing data and listen and respond accordingly

Resources

I have committed all the protofiles in detailed branches at this github repository for your perusal

This is the file structure that I have followed while designing this API.

root_folder/
|-- client/
|   |-- client.js
|-- node_modules  
|-- protos/
|   |-- max.proto
|-- server/
     |-- index.js
     |-- protos/
        |-- max_pb.js
        |-- max_grpc_pb.js

0. Sample Problem

In this exercise, we will be building an API to return the maximum of numbers streamed from client (client streaming) every couple of seconds (server streaming)

The function takes a stream of Request messages that have one integer each, and returns a stream of Responses that represents the maximum of all the nummbers streamed to it upto that point.

This interaction is :

1. Bidirectional - as client and server both keep streaming data to each other.
2. Staggered - as client and server streams have different speeds

1. Writing the protofile

This is pretty basic and will involve

syntax="proto3";

package max;

service MaximumService{
    rpc CalciMax(stream MaxRequest) returns (stream MaxResponse ){}
}

message MaxResponse{
    int32 result=1;
}

message MaxRequest{
    int32 num=1;
}

Here, the stream keyword next to AvgRequest and AvgResponse is of key importance as it tells gRPC to generate code accordingly such that the function called from the stream of client request will return stream of responses from server.

2. Generate the protofiles

Packages required : protoc, grpc-tools

1. grpc-tools

npm i grpc-tools

We get MaxServiceService and MaxServiceClient generated from the "grpc-tools" module that will act on our proto to create the client and service (server)

Output file ⇒ PACKAGENAME_grpc_pb.js

2. protoc

Whereas, the protoc will simply generate the Request and Response objects and give us getters and setters on them

Output filename ⇒ PACKAGENAME_pb.js

sudo protoc -I=. ./protos/max.proto \                                        
  --js_out=import_style=commonjs,binary:./server \  
  --grpc_out=./server \
  --plugin=protoc-gen-grpc=`which grpc_tools_node_protoc_plugin`

Here the inside the main project directory, we have a client, server and protos subdirectory. Currently, we are generating the API from proto file in the protos directory into the protos subdirectory inside the server directory

SIDE NOTE

I shall be using the chalk library here to setup beautiful text outputs in our console so we can clearly see how each request and response are communicated

npm i chalk

3. Setup a gRPC server

npm i grpc google-protobuf

Import the necessary dependencies. Here,

1. max_pb ⇒  Contains all the details about the request and response object.
2. max_grpc_pb ⇒  Will help the grpc server create an API from the proto generated code.

const grpc = require("grpc")
const max = require('../server/protos/max_pb')
const maxService = require('../server/protos/max_grpc_pb')

// HELPER FUNCTION
const sleep = async (interval) =>
{
    return new Promise(resolve =>
    {
        setTimeout(()=>resolve(),interval)
    })
}

// MAIN FUNCTION
const calciMax = async(call, callback) =>
{
    const noOfTimesYouWannaComputeMaximum = 15
    const numArray = []
    call.on('data', request =>
    {
        const num = request.getNum()
        console.log(chalk.green(`Client just pushed this number to me : ${num}`))
        numArray.push(num)
    })

    call.on('status', status => console.log(chalk.magenta(status)))

    call.on('end', ()=>{console.log(chalk.cyanBright('Client has stopped streaming from the server'))})
    

    for (let i = 0; i < noOfTimesYouWannaComputeMaximum; i++)
    {
        const maxResponse = new max.MaxResponse()
        const maximum = _.max(numArray)
        console.log(maximum)
        maxResponse.setResult(maximum)
        call.write(maxResponse)
        await sleep(3000)
    }
    call.end()
}

Define the function that will handle the request and return a response. This function is what we defined in the proto file and this is also the function that will be callled from the client

Here the call.on() methods allow us to basically monitor the stream sent from the client for proper response and error handling.

Also, by setting up different sleep intervals for client and server we can establish a staggered communication between them.

4. Setup the Client Side

Import the necessary dependencies.

1. max_grpc_pb ⇒ Contains proto generated code that will help us setup the client and link it to the server functions we have defined
2. max_pb ⇒ Contains code to help us access and work with the request object

const grpc = require('grpc');
const max = require('../server/protos/max_pb')
const maxService = require('../server/protos/max_grpc_pb')

Setup a client using the proto generated code

const URL_ENDPOINT = `localhost:50051` 

//* Client setup
    const maxClient = new maxService.MaximumServiceClient(
        URL_ENDPOINT,
        grpc.credentials.createInsecure()
    )

//* Setup the response handling
    const maxReq = new max.MaxRequest()
    const call = maxClient.calciMax(maxReq, () => { })
    call.on('data', (response) =>
    {
        const max = response.getResult()
        console.log(chalk.yellow(`Server says that uptil now, this is the max num : ${max}`))
    })
    
    call.on('error', err => console.log(chalk.yellow(err)))
    call.on('status', status => console.log(chalk.magenta(status)))
    call.on('end', ()=>{console.log(chalk.cyanBright('Server has stopped streaming from the client'))})

const numArray = [1, 5, 3, 6, 2, 20, 1,23, 54, 78, 34, 545,23,54, 4,544,5,23,12,12,,35,4,55,6345214,11231231]
    for (let i = 0; i < numArray.length; i++)
    {
        maxReq.setNum(numArray[i])
        call.write(maxReq)
        await sleep(1000)
    }
    call.end()

Add service to server and start it

//* Setup a server
const server = new grpc.Server()

server.addService(maxService.MaximumServiceService, { calciMax })

//? START THE SERVER
const URL_ENDPOINT = "127.0.0.1:50051"
server.bind(URL_ENDPOINT, grpc.ServerCredentials.createInsecure())
server.start()

console.log(chalk.green(`Server running on ${URL_ENDPOINT}`))

Conclusion

Congratulations ! You are now equipped with the futuristic skills of bidirectional streaming type API designing. Thanks a lot for reading this blog. If you like more content like this, subscribe to my mailing list