An Introduction to Multikey Indexes with Examples


After understanding single-field and compound indexes, now is the time to learn about multikey indexes. When a field has an array value, an index key is generated for each of its array elements. These indexes are referred as multikey indexes. They can be used with arrays that have scalar values as well as those with nested documents.

To generate a multikey index, you have to use the standard db.collection.createIndex() method. Such indexes are automatically generated by MongoDB whenever it senses an indexed field specified as an array. Hence, there is no need for explicit definition of a multikey index.

Examples

While working with a standard array, let’s suppose we have a collection “student”.

{ _id: 21, name: “Adam”, marks: [ 80, 50, 90 ] }

To build an index on the “marks” field, write the following query.

db.student.createIndex( { marks: 1 } )

As the marks field is an array, thus this index is an example of a multikey index. All of the keys (80, 50, and 90) in its elements point to the same document.

To create a multikey index for array fields having embedded documents, let’s suppose we have a collection “products”.

{

_id: 5,

name: “tshirt”,

details: [

{ size: “large”, type: “polo”, stock:50 },

{ size: “small”, type: “crew neck”, stock:40 },

{ size: “medium”, type: “v neck”, stock: 60 }

]

}

{

_id: 6,

name: “pants”,

details: [

{ size: “large”, type: “cargo”, stock: 35 },

{ size: “small”, type: “jeans”, stock: 65 },

{ size: “medium”, type: “harem”, stock: 10 },

{ size: “large”, type: “cotton”, stock: 10 }

]

}

{

_id: 7,

name: “jacket”,

details: [

{ size: “large”, type: “bomber”, stock: 35 },

{ size: “medium”, type: “leather”, stock: 25 },

{ size: “medium”, type: “parka”, stock: 45 }

]

}

We can build a multikey index with the details.size and details.stock fields.

db.products.createIndex( { “details.size”: 1, “details.stock”: 1 } )

This index is now good to go against queries which have only a single field of “details.size” as well as queries having the both of the indexed fields. As such, these types of queries would benefit from the index.

db.products.find( { “details.size”: “medium” } )

db.products.find( { “details.size”: “small”, “details.stock”: { $gt: 10 } } )

Bounds in Multikey Index

Bounds represent the limits of an index .i.e. how much it needs to scan for searching a query’s results. If there are more than a single predicate with an index, then MongoDB integrates them through compounding or intersection. So what do we mean by intersection and compounding?

Intersection

Bounds intersection point towards the presence of “AND” (logical conjunction) for bounds. For example, if there are two bounds [ 4, Infinity]  and [ – Infinity, 8 ], then intersection bounds process [[4, 8 ]] When $elemMatch operation is used, then MongoDB applies intersection on multikey index bounds.

 

What Is $elemMatch?

Before moving forward, let’s understand the use of $elemMatch first.

$elemMatch is used for matching documents in array field where at the bare minimum, atleast one of the element is matched with the query. Bear in mind, that the operator does not work with $text and $where operators. For a basic example, consider a “student” collection.

{ _id: 8, marks: [ 72, 75, 78 ] }

{ _id: 9, marks: [ 65, 78, 79 ] }

The following query only processes a match with documents in which the “marks” array has at least a single element which is less than 75 and greater or equal to 70.

db.student.find(

{ marks: { $elemMatch: { $gte: 70, $lt: 75 } } }

In response, the result set is comprised of the following output.

{ “_id” : 8, “marks” : [72, 75, 78 ] }

Despite the fact that both 75 and 78 do not conform to the conditions but because 72 had a matched, hence the $elemMatch selected it.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

For instance, we have a collection student which has a field “name” and an array field “marks”.

{ _id: 4, name: “ABC”, marks: [ 1, 10 ] }

{ _id: 5, name: “XYZ”, marks: [ 5, 4 ] }

To build a multikey index with the “marks” array.

db.student.createIndex( { marks: 1 } )

Now, the following query makes use of $elemMatch which means that the array must have at least one element which fulfils the condition of both predicates.

db.student.find( { marks : { $elemMatch: { $gte: 4, $lte: 8 } } } )

Computing the predicates one by one:

  • For the first predicate, the bounds are equal to or greater than 4 [ [ 4, Infinity ] ].
  • For the second predicate, the bounds are equal to or less than 8 [ [ – Infinity, 8 ] ].

Since $elemMatch is used here, therefore MongoDB can apply intersection on the bounds and integrate it like the following.

marks: [ [ 4, 8 ] ]

On the other hand, if the $elemMatch is not used, then MongoDB applies intersection on the multikey index bounds. For instance, check the following query.

db.student.find( { marks : { $gte: 4, $lte: 8 } } )

The query processes the marks array for atleast a single element which is equal to or greater than 4 “AND” atleast a single element which is equal to or less than 8. However, it is not necessary for a single element to conform to the requirements of both predicates, hence MongoDB does not apply intersection on the bounds and uses either [[4, Infinity]] or [[-Infinity, 8]].

Compounding

Compounding bounds means the use of bounds with compound index. For example, if there a compound index { x: 1, y:1 } which has a bound on the x field  [[4, Infinity] ] and a bound on the field y [[-Infinity, 8]]. By applying compounding,

{ x: [ [ 4, Infinity ] ], y: [ [ -Infinity, 8 ] ] }

Sometimes, MongoDB is unable to apply compounding on the given bounds. For such scenarios, it uses the bound on the leading field which in our example is x: [ [4, Infinity] ].

Suppose in our example indexing is applied on multiple fields, where one of the fields is an array. For instance, we have the collection student which stores the “name” and “marks” field.

{ _id: 10, name: “Adam”, marks: [ 1, 10 ] }

{ _id: 11, name: “William”, marks: [ 5, 4 ] }

Build a compound index with the “name” and the “marks” field.

db.student.createIndex( { name: 1, marks: 1 } )

In the following query, there is a condition which applies on both of the indexed keys.

db.student.find( { name: “William”, marks: { $gte: 4 } } )

Computing these predicates step-by-step.

  • For the “name” field, the bounds for the “William” predicate are the following [ [ “William”, “William” ] ].
  • For the “marks” field, the bounds for { $gte: 4 } predicate are [ [ 4, Infinity ] ].

MongoDB can apply compounding on both of these bounds.

{ name: [ [ “William”, “William” ] ], marks: [ [ 4, Infinity ] ] }

 

 

 

 

 

 

 

Advertisement

Design Patterns for Functional Programming


In software circles, a design pattern is a methodology and documented approach to a problem and its solution which is bound to be found repeatedly in several projects as a tumbling block. Software engineers customize these patterns according to their problem and form a solution for their respective applications. Patterns follow a formal structure to explain a problem and then go over a proposed answer as well as key points which are related to either the problem or the solution. A good pattern is one which is well known in the industry and used by the IT masses. For functional programming, there are several popular design patterns. Let’s go over some of these.

Monad

Monad is a design pattern which takes several functions and integrates them as a single function. It can be seen as a type of combinatory and is a core component of functional programming. In monad, a value is wrapped in a box which is then unwrapped and a function is passed to use the wrapped value.

To go into more technicalities, a monad can be classified into running on three basic principles.

·         A parameterized type M<T>

According to this rule, T can possess any type like String, Integer, hence it is optional.

·         A unit function T -> M<T>

According to this rule, there can be a function taking a type and its processing may return “Optional”. For instance, Optional.of(String) returns Optional<String>.

·         A bind operation: M<T> bind T -> M <U> = M<U>

According to this rule which is also known as showed operator due to the symbol >>==. For the monad, the bind operator is called. For instance, Optional<Integer>. Now this takes a lambda or function as an argument for instance like (Integer -> Optional<String> and returns and processes a Monad which has a different type.

Persistent Data Structures

In computer science, there is a concept known as a persistent data structure. Persistent data structure at their essence work like normal data structure but they preserve their older versions after modification. This means that these data structures are inherently immutable because apparently, the operations performed in such structures do not modify the structure in place. Persistent data structures are divided into three types:

  • When all the versions of a data structure can be accessed and only the latest version can be changed, then it is a partially persistent data structure.
  • When all the versions of a data structure can be accessed as well as changed, then it is a fully persistent data structure.
  • Sometimes due to a merge operation, a new version can be generated from two prior versions; such type of data structure is known as confluently persistent.

For data structure which does not show any persistence, the term “ephemeral” is used.

As you may have figured out by now, since persistent data structures enforce immutability, they are used heavily in functional programming. You can find persistent data structure implementations in all major functional programming language. For instance, in JavaScript Immutable.js is a library which is used for implementing persistent data structures. For example,

import { MapD } from ‘immutable’;

let employee = Map({

employeeName: ‘Brad’,

age: 27

});

employee.employeeName; // -> undefined

employee.get(’employeeName’); // -> ‘Brad’

Functors

In programming, containers are used to store data without assigning any method or properties to them. We just put a value inside a container which is then passed with the help of functional programming. A container only has to safely store the value and provide it to the developer in need. However, the values inside them cannot be modified. In functional programming, these containers provide a good advantage because they help with forming the foundation of functional construct and assist with asynchronous actions and pure functional error handling.

So why are we talking about containers? Because functors are a unique type of container. Functors are those containers which are coded with “map” function.

Among the simplest type of containers, we have arrays. Let’s see the following line in JavaScript.

const a1 = [10, 20,30, 40, 50];

Now to see a value of it, we can write.

Const x=y[1];

In functional, the array cannot be changed like.

a1.push(90)

However, new arrays can be created from an existing array. An array is theoretically a function. Technically, whenever a unary function is mapped with a container, then it is a functor. Here ‘mapped’ means that the container is used with a special function which is then applied to a unary function. For arrays, the map function is the special function. A map function processes the contents of an array and performs a special function for all the elements of the element step-by-step after which it responds with another array.

Zipper

A zipper is a design pattern which is used for the representation of an aggregate data structure. Such a pattern is good for codes where arbitrarily traversal is common and the contents can be modified, therefore it is usually used in purely functional programming environments. The concept of Zipper dates back to 1997 where Gérard Huet introduced a “gap buffer” strategy.

Zipper is a general concept and can be customized according to data structures like trees and lists. It is especially convenient for data structures which used recursion. When used with zipper, these data structure are known as “a list with zipper” or “a tree with zipper” for making it apparent that their implementation makes use of zipper pattern.

In simple terms, zipper with data structure has a hole. They are used for the manipulation and traversal in data structures where the hole indicates the present focus for the traversal. Zipper facilitates developers to easily move within the data structure.

Dos and Don’ts of Functional Programming


While working with functional programming in Java, there are several do’s and don’ts. Let’s go over the most common ones. By making use of these tips, you can improve your functional programming code-base with good results.

Standard Functional Interfaces

The Functional interfaces in “java.util.function” are good enough to fulfill requirements for method references and lambda’s target types. As explained earlier, these interfaces are abstract, thereby making it easier to customize them freely into a lambda expression. Hence, before writing new functional interfaces, makes sure to check this package which may satisfy your requirements.

Suppose you have in interface FuncInt

@FunctionalInterface

public interface FuncInt {

String f1(String txt);

}

Now you have a class for using the interface with the name useFuncInt which has a method for sum.

public String sum(String txt, FuncInt) {

return FuncInt f1(txt);

}

For execution of it, you would have to add the following:

FuncInt fi = txt -> txt + ” coming from lambda”;

String output = useFuncInt.add(“Message “, useFuncInt);

Now, if you observe closely, you will realize that FuncInt is a function which takes one argument and returns with an answer. With the release of Java 8, this functionality is enabled through the interface in “Function<T,R> which is entailed in the package of “java.util.function”. Our FuncInt can be fully eliminated and our code can be modified to.

public String sum(String txt, Function<String, String> f) {

return f.apply(txt);

}

For execution, write the following:

Function<String, String> f =

parameter -> parameter + ” coming from lambda”;

String output = useFuncInt.sum(“Message “, f);

Annotation Matters

Make the use of annotation by adding functional interfaces with the help of @FunctionalInterface. If you are unfamiliar with annotations, then put it with a tag which is used to represent the metadata; this could be related to an interface, method, field, or class about any useful information that can prove to be helpful for the JVM (Java Virtual Machine) or the complier.

The @FunctionalInterface may not seem logical at the first glance because without adding it, the interface would work fine as a functional one as long it contains only one abstract method.

However, in large-scale projects where the increasing number of interfaces may overwhelm you, manual control is tricky and exhaustive; it becomes a nightmare to keep track of it. Sometimes, an interface which was created as a functional interface may get additional abstract methods, therefore ceasing to exist as a functional interface. This is where the @FunctionalInterface annotation works like a charm. By using it the complier would display an error whenever there is any intrusion by a programmer to change the structure of a functional interface and disrupt its “purity”. Likewise, while working in teams, it can be helpful for other coders to understand your code and identify functional interfaces easily.

Hence, always use something like this,

@FunctionalInterface

public interface FuncInt{

String printSomething();

}

Instead of writing something like this:

public interface FuncInt{

String printSomething();

}

Even if you are working on beginner projects, make it a habit to always add the annotation or you may face the repercussions in future.

Lambda Expressions Cannot Be Used Like Inner Classes

When an inner class is used, “scope” is generated. This means that we can add variables which are local with the enclosing scope after instantiating local variables having identical names. The keyword “this” can also be used in our class for referencing to its own instance.

With lambda expressions, we work by using enclosing scope. This means that variables cannot be overwritten in the body of the lambda expression from the enclosing scope. We can use “this” for referencing.

For instance, for our class UseTest, we have “test” as our instance variable:

private String txt = “Encircling the scope”

Now, you can use another method of our class with the following code and run it like this:

public String example()

{

Test test = new Test();

String mssg = “inner value of the class”

@Override

Public String returnSomething(String txt)

Return this.mssg;

}

};

String output = test.returnSomething(“”);

Test testLambda = parameter -> {

String mssg = “Lambda Message”;

return this.mssg;

};

String outputLambda= testLambda.returnSomething(“”);

return “Output: output = “ + output + “outputLambda = “ + outputLambda;

When you will run the “example” method, you may get the following: “Output: output =  inner value of the class, outputLambda = Encircling the scope

This means that you can access and use a local variable by accessing its instance. By using “this”, you might have used the variable “test” of the UseTest class but you were unable to get the “test” value which in entailed in the body of the lambda.

Lines of Code in Lambdas

Ideally, lambda expressions should comprise of a single line of code because this makes it a self-explanatory concrete implementation where the clear execution of an action at some data is represented.

If you require more lines for your functionality, then write something like this:

Test test = parameter -> generateText(txt);

private String generateText(String txt) {

String output = “Some Text” + txt;

// several lines of code

return output;

}

Refrain from writing something like this:

Test test = txt -> { String output= “Some Text” + txt;

// several lines of code

return output;

};

Though, it is important to note that sometimes, it is ok to have more than one line of lambda expressions where the use of another method may prove to be counter-productive.

Type of Parameters

The compilers are powerful enough to ascertain the types of parameters in lambda expressions with type inference. Hence, there is no need for adding the parameter type explicitly.

Don’t write something like this.

(String x, String y) -> x.toUpperCase() + y.toUpperCase();

Instead, write something like this,

(x, y) -> x.toUpperCase() + y.toUpperCase();

Java Lambdas


So far we have talked a lot about functional programming. We discussed the basics and even experimented with some coding of functional interfaces. Now is the right time to touch one of the most popular features of Java for functional paradigm, known as lambda expressions or simply lambdas.

What Is a Lambda Expression?

A lambda expression provides functionality for one or more functional interface’s instances with “concrete implementations”. Lambdas do not require the use of a class for their use. Importantly, these expressions can be viewed and worked by coding them as objects. This means that, like objects, it is possible to pass or run a lambda expression. The basic style for writing a lambda expression requires the use of an “arrow”. See below:

parameter à the expression body

On the left side, we have a “parameter”. We can write single or multiple parameters for our program. Likewise, it is not mandatory to specify the parameter type because compilers already ascertain the parameter type. If you are using a single parameter, then you may or may not add a round bracket.

However, if you intend to add multiple parameters, then make sure to use round brackets (). Sometimes, there is no need of parameters in a lambda expression. For such cases, it is possible to signify them by simple adding an unfilled round bracket. To avoid error, use round brackets for parameter whether you are using a 0, 1, or more parameters.

On the right side of the lambda expression, we can have an expression. This expression is entailed in curly brackets. Like parameters, you do not require brackets for a single expression while multiple expressions require one. However, unlike parameter, the return type of a function can be signified by the body expression.

Without Lambdas

To understand lambdas, check this simple example.

package fp;

public class withoutLambdas {

public static void main(String[] args) {

withoutLambdas wl = new withoutLambdas(); // generating instance for our object

String lText2 = “Working without lambda expressions”; // here we assign a string for the object’s method as a parameter

wl.printing(lText2);

}

public void printing(String lText) { // initializing a string

System.out.println(lText);              // creating a method to print the String

}

}

 

The output of the program is “Working without lambda expressions”. Now if you are familiar with OOP, then you can understand how the caller was unaware of the method’s implementation i.e. it was hidden from it. What is happening here is that the caller gets a variable which is then used by the “printing” method. This means we are dealing with a side effect here—a concept we explained in our previous posts.

Now let’s see another program in which we go one step ahead, from a variable to a behavior.

package fp;

public class withoutLambdas2 {

interface printingInfo {

void letsPrint(String someText);  //a functional interface

}

public void printingInfo2(String lText, printingInfo pi) {

pi.letsPrint(lText);

 

}

 

public static void main(String[] args) {

withoutLambdas2 wl2 = new withoutLambdas2(); // initializing instance

String lText = “So this is what a lambda expression is”; // Setting a value for the variable

printingInfo pi = new printingInfo() {

@Override // annotation for overriding and introducing new behavior for our interface method

public void letsPrint(String someText) {

System.out.println(someText);

}

};

wl2.printingInfo2 (lText, pi);

}

 

 

}

In this example the actual work to print the text was completed by the interface. We basically formulated and designed the code for our interface’s implementation. Now let’s use Lambdas to see how they provide an advantage.

 

package fp;

public class firstLambda {

 

interface printingInfo {

void letsPrint(String someText);    //a functional interface

}

public void printingInfo2(String lText, printingInfo pi) {

pi.letsPrint(lText);

}

 

public static void main(String[] args) {

firstLambda fl = new firstLambda();

String lText = “This is what Lambda expressions are”;

printingInfo pi = (String letsPrint)->{System.out.println(letsPrint);};

fl.printingInfo2(lText, pi);

}

See how we improved the code by integrating a line of lambda expression. As a result, we are able to remove the side effect too. What the expression did was use the parameter and processed it to generate a response. The expression after the arrow is what we call as a “concrete implementation”.

Core Concepts of Functional Programming


Now that you have learned about the paradigm shift to functional programming, let’s go into the depths of the fundamentals concepts that power functional programming. The comprehension of these basic concepts is important to create high-quality functional programming applications. You may be tempted to directly begin coding but these concepts can help you become a better coder.

1- Pure Functions

In functional programming, everything is seen as a function. Each function has to be “pure”. Pure here refers to two basic capabilities:

No Side Effects

A function can never be pure if it carries even a single side effect. A side effect is a property when a function’s states are modified by other functions. By states, we mean the data like variables or data structures. Pure functions do not carry any side effects; hence their memory or I/O operations can’t be affected. Now, you might wonder that why exactly does their presence considered bad. Well, because they make functions “unpredictable” where a function has to rely on its system’s state.

On the contrary, if a function’s state cannot be changed, then the same output is generated for the given input. A side effect of a function can also mean to write any operation which has been applied to the disk or turning on/off a control of your front-end UI’s function.

Same Result with Multiple Calls

Whenever a function is called without any modifications in its arguments, the same results are generated. Consider an example where you have designed a simple function “multiply (4, 5). Now, this function is expected to generate the same results .i.e. for each invocation. However, if you were programming in other paradigms then random functions or global variables may not allow your result to remain the same.

Pure functions also offer “memoisation”. Memeoisation refers to a technique in which pure functions’ output (always same result) is saved in the cache memory. Now, whenever such functions are invoked, caching helps to enhance the performance and speed of the application.

2- Higher Order Function

The concept of a function which is higher order is known in mathematics as well as computer science. Generally, they possess two fundamental characteristics.

Return Type

The return type of a higher-order function has to be a function. For example, review the following code in Java.

package fp;

 

public class higherOrderfunction {

public static void main(String args[]) {

 

System.out.println(A(4));

 

}

static int marks() {

int a=5;

return a;

}

static int A(int total) {

total =4;

return total+marks();

}

}

To simplify things, we have constructed a simple function marks. This function holds an integer value “a” which is returned. Now, we have a function A. A takes an argument for an integer total and assigns it a value. Now, comes the actual part. See how in return, we have used marks method as a return type. Since we have processed our function by returning another function; hence this function is a higher-order function.

Arguments

Another characteristic of a higher-order function can be its use of functions as input parameters. For instance, see this see pseudo-code:

Public areaRect (lb) // Here areaRect represents a function that takes arguments from another function lb to calculate length  and breadth

{

int area;

area =l*b;

return lb; //

}

This function is a higher-order function because it processes itself using another function as a parameter.

3- First Class Functions

After higher-order functions, we have first-class functions. These are not too dissimilar to higher-order functions. It is important to note that a first-class function always adheres to the terms and conditions of a higher-order function. So what this means is that a first-class function has to return another function as well as contain a parameter in the form of function. Hence, by default, all first-class functions are higher-order functions. So what exactly is the difference between them? Well, context matters!

By referring a language to have support for first-class functions, we generally mean that uses its functions as values that can be easily passed around.

On the other hand, the term higher-order is more associated with the mathematics outlook when pure problem-solving requires a more theoretical and general perspective of the problem.

4- Evaluation

Some languages support strict evaluation while others offer non-strict evaluation support. This evaluation is targeted at the language’s processing of an expression while considering the function parameters or arguments. To familiarize yourself better with the concept, check this simple example:

Print length([4-3,4+6,1/0,7*7,3-8])

When a programming language uses non-strict evaluation to process this expression, then it simply returns back a value of 5 .i.e. the total number of elements. Such evaluation does not concern itself with the depth of values.

On the other hand, the same expression returns an error with strict evaluation because it found the third element “1/0” to be incorrect. Hence, this means that strict evaluation is more stringent and processes an expression more deeply.

In a few scenarios, non-strict evaluation has to enforce processing of strict evaluation when a function needs to be evaluated on a “stricter” basis due to an invocation.

5- Referential Transparency

In functional programming, the usual assignment of values is not offered. Variables are immutable; a value defined once is the final one which is not possible to be changed in future. It is the property which makes them without any side-effects. To understand further, consider the following example, where the value of x is changed after each evaluation.

x=x*15

In the start of the program, x was assigned a value of 1. The first evaluation made it 15.

Now, in the second evaluation, the value of x changes to 225.

Now, this function is not referentially transparent because the value of x is continuously changing.

So now, if we go by the functional concepts, then our example can be altered into this pseudocode:

int sum(int x)

{

x=x+2;

}

This type of function ensures that the value of x remains constant and it cannot be altered implicitly.

What Is Functional Programming? Why the Paradigm Changed the Game?


The Path to Functional Programming

Before understanding functional programming, ask yourself how much do you know about another programming “styles”? When programming initially emerged to solve the major problems of the world through a few lines of code, Computer Scientists realized that they required a standard format or style which could help them to program effectively and efficiently. This style is commonly known as “programming paradigm”.

Soon developers began coding in C by using the procedural paradigm. Procedural programming mainly deals with coding with a step-to-step design similar to kitchen recipes; where a set of instructions is followed sequentially. At that time, the paradigm was indeed excellent at solving problems. However, as technologies evolved and programming became much more complex—websites were built and businesses began to adopt IT—the flaws of procedural programming bugged developers.

Enter Object-Oriented Programming, the next popular paradigm. The vision behind OOP was simple; it modeled programming on the basis of real-world examples. For example, a car could be seen as an object which possessed certain behavior (methods in programming) and states (members like variables in programming). OOP succeeded in decreasing global codebases. OOP concepts like encapsulation and inheritance were fundamental to attain an unexpected degree of productivity.

However, soon developers realized that OOP was not up to the task for a number of things. Hence, to address certain issues, functional programming came into the scene. Popular languages like PHP, Python, etc., are examples of languages which support the functional paradigm. Remember, not each language is built to support all paradigms. For example, while C can support procedural programming, it does not offer support for object-oriented programming.

However, most modern languages support procedural, OOP, and functional programming. There are some like Haskell, who received recognition due to features for functional programming. Java was initially not supportive of the paradigm, but since the last few years, Java releases have introduced features like Lambda Expressions to support functional programming. However, the question is, what exactly is functional programming?

What Is Functional Programming

The name suggests that it is linked to “functions”. Now, if you think that this function relates to the programming of methods, then your assumption is flawed. Functional programming refers to functions which incorporate a certain piece of code in the form of a feature or operation to an application. This function facilitates programmers to avoid changing the other parts of the application.

Functional programming highly borrows from mathematics. These functions are highly interlinked with those “mathematical functions” that you may have studied in your college courses.  In mathematics, a specific question is solved with a method without going much in the theoretical complexities of that method; similarly, functional programming is there to gain a higher degree of abstraction in applications.

One key aspect of functional programming is that it avoids change in the mutability and states of data. Another thing to note is that unlike “statements”, which are generally used in the OOP landscape, “expressions” power functional codebases.

Paradigm Shift to Functional Programming

Previously business applications used C++, a phenomenon that can prove to be a nightmare for modern developers. In those days, software engineers focused a lot on low-level aspects where issues related to memory management did not allow them to achieve much productivity. Then, Java came and provided more abstractions and managed these tasks through new features, thus resulting in saving developers from de-coding low-level complexities.

Today, languages like Scala (JVM) and F# (.NET) are promising a similar level of convenience to developers. Even outside of JVM and .NET ecosystems, you can clearly see Apple going with Swift and Facebook with React JS. Many of today’s “cool” languages are known for their functional paradigms. So why exactly has it become famous? Maybe, the following factors may have something to do with it.

Parallelism

One of the major reasons behind the positive reception of functional programming is its support for parallel computing. Traditionally, basic computing requirements like data storage and processing were not too intensive .i.e. there was no need of running several things at once. However, this changed as several technologies came and evolved one after another, expanding the world of software development and transforming it to power as a backbone of global operations.

Today, parallel computing is a highly valued domain of computer science where multiple applications and data require processing at the same time. Here, functional programming has made its impact due to its natural features which support independence components to run and support modern software infrastructure like microservices. This means if you try to add a specific functionality or modify an existing one, then “functions” ensure that you do not affect the other components of your system.

Data Streaming

Another reason which can be associated with the ‘functional leap’ is the advent of “streaming services”. a few years ago, entertainment was mainly associated with TV—the only medium for watching shows and movies.

However, today streaming services like Netflix have changed the game. Entertainment has shifted online. People prefer to take advantage of the luxury of watching football matches on mobile apps while commuting, rather than hurrying their way back to home for their television sets. Since functional programming works well with streaming due to its natural concepts; hence its growth makes a lot of sense.

AI

AI is perhaps the most exciting branch of computer science. Over the past few decades, extensive research has been carried out about AI while its applications and sub-branches like machine learning, deep learning, speech recognition, etc have become their own field of studies. AI is seen with a bright hope that it can modernize the world and take it to the next level through robots and intelligent machines. However, all this AI implementation requires coding where a paradigm has to be ultimately chosen.

Since AI is closely related to mathematics, logic, statistics, and theoretical computer science, hence you can certainly understand why functional paradigm—a paradigm founded on mathematics—is gaining so much prominence among the AI community.

What’s in it for the Developer?

Well, so far you might have understood how the elevation of some technologies and necessities of some requirements raised the need for functional programming. However, why should you as a developer working with “objects” (OOP) think about learning something new? Is the hassle worth it? Well, to be honest, functional programming is not a magic potion that can generate great results for all OF YOUR applications.

However, there are several cases where the use of functional programming can assist you to achieve better results in your projects where the time spent on learning the paradigm may result in saving you a great deal of resources. Following are a few of the reasons to learn it.

Decreasing LOC

What was one of the most well-liked thing aspects about OOP? The programs that used to require 5,000 line of codes via procedural programming, were reduced to less than 1,000 lines of codes by adopting OOP’s fundamental concepts like inheritance.

Similarly, functional programming also promises the same .i.e. it provides shorter code-bases while maintaining the performance of the application. As a result, productivity is increased where fewer lines of codes do more while maintenance is also relatively easier.

Testing and Debugging

When everything is a function; testing is less challenging. You just have to test each function where one’s output does not affect the result of the other function.

The states of functions cannot be modified from outside of its scope. Hence, its input can be tracked effectively, resulting in an efficient management of its output too. As a result, debugging is easy because it is easy to check what went wrong.

Career Growth

There are no negatives in learning a paradigm which goes well with all the hottest modern technologies. With a chance to work on IoT, AI, and, other futuristic fields, functional programming can prove pivotal to your success as a computer scientist. Learning and adapting are the hallmarks of every successful computer science. Also, if you are tired of doing the same old programming, then it can provide a new challenge that can motivate you to work contentedly.

Now that you have learned about some basic definition, importance, and advantages of functional programming, now is the right time to understand its basics.

What is RabbitMQ?


What is RabbitMQ?

The concept of messaging in the software environment is similar to the daily life processes. For example, you went for a morning coffee. After taking your order, the manager inputs it into the system. If there is no rush in the coffee shop, your order does not require to be added in a queue.  However, if there are previous orders, the system puts it behind other orders. Thus, your order becomes part of the queue.

However, what if there are countless orders and the server is unable to manage all those due to a hardware issue? What to do now? In such cases, a service like RabbitMQ can prove to be the game changer. RabbitMQ will take all the orders and only forward them to the server when it can manage the workload.

Before understanding RabbitMQ, it is essential to equip yourself with the knowledge of a message broker. A message broker is an intermediary program that works on the translation of the contents of a message with the messaging protocols of both the receiver and the sender. Message brokers are used as a middleware solution for a variety of software applications.

RabbitMQ is a message broker software that is used for the queuing of messages. There are three main actors in the RabbitMQ lifecycle. First, we have a ‘publisher’ or ‘producer’. A publisher is the one who creates a message and sends it. Second, we have an ‘exchange’. Exchange receives the message with a routing key from the producer. The exchange will then save the message and store it in a queue. Third, we have a consumer. A consumer is a party for which the message was intended. A consumer can either be a third party or the publisher itself who consumes the message after getting it from the queue of the broker.

For the above example, we have used a single queue, but in real-world applications, there would be multiple queues. An exchange is connected to a queue through a binding key. The exchange will use the routing key and binding key to confirm the consumer of a message. However, it is important to note that sometimes an exchange will link a routing key with the name of a queue instead of using a binding key. There are mainly four types of exchanges: direct, topic, headers and fanout.

Whenever a message goes to a consumer, RabbitMQ makes it certain that it is received in the correct order. The queues do not let a message get lost.

RabbitMQ comes with a protocol known as AMQP (Advanced Message Queuing Protocol). AMQP helps to define three major components.

  • Where should the message go?
  • How will it get delivered?
  • What goes in must also come out.

AMQP does not require a learning curve and can be easily programmed due to its flexibility. Thus, if a developer works with the HTTP and TCP requests and responses, they will easily adapt its protocol.

RabbitMQ supports development support for all the popular programming languages including Java, .NET, Python, PHP, JavaScript, etc.

Example

For a practical explanation, we will write a simple application in Java with RabbitMQ. The application will consist of a producer, which will send a message, as well a consumer, which will receive that message. For sending, we have a file named Send.java. You will require the following import.

import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.Channel;

 Now, setup the class.

public class Send { 
private final static String QUEUE_NAME = “hello”;
public static void main(String[] argv)      throws java.io.IOException {      …  }}

Now we will have to link our class with the server.

ConnectionFactory factory = new ConnectionFactory();
factory.setHost(“localhost”);
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();

 

This code helps in the abstraction of the socket connection. Now, the next step is the creation of a channel. For this purpose, you will have to define a queue.

channel.queueDeclare(QUEUE_NAME, false, false, false, null);
String message = “Hello World!”;
channel.basicPublish(“”, QUEUE_NAME, null, message.getBytes());
System.out.println(” [x] Sent ‘” + message + “‘”);

Lastly, we close the channel and the connection;

channel.close();
connection.close();

This ends the code for the sender.

Here is complete send java class.

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

public class Send {

private final static String QUEUE_NAME = “hello”;
public static void main(String[] argv) throws Exception {         ConnectionFactory factory = new ConnectionFactory();    factory.setHost(“localhost”);
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();    channel.queueDeclare(QUEUE_NAME, false, false, false, null);    String message = “Hello World!”;
channel.basicPublish(“”, QUEUE_NAME, null, message.getBytes(“UTF-8”));
 System.out.println(” [x] Sent ‘” + message + “‘”);
 channel.close();
connection.close();

  }

}

Now you will have to write the code for the consumer. For this purpose, create a Recv.java class. Use the following import.

import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Consumer;
import com.rabbitmq.client.DefaultConsumer;

 

Now we will open a connection here too.

public class Recv { 

private final static String QUEUE_NAME = “hello”; 
public static void main(String[] argv)      throws java.io.IOException,             java.lang.InterruptedException {     ConnectionFactory factory = new ConnectionFactory();    factory.setHost(“localhost”); 
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();     channel.queueDeclare(QUEUE_NAME, false, false, false, null);    System.out.println(” [*] Waiting for messages. To exit press CTRL+C”);

    …    }

}

 

Now you will have to notify the server so it can fetch the messages that are accumulating in the queue.

Consumer consumer = new DefaultConsumer(channel) {  @Override  public void handleDelivery(String consumerTag, Envelope envelope,   AMQP.BasicProperties properties, byte[] body)      throws IOException {    String message = new String(body, “UTF-8”); 
  System.out.println(” [x] Received ‘” + message + “‘”); 
}};
channel.basicConsume(QUEUE_NAME, true, consumer);

Now run both the consumer and the producer, and you will have your RabbitMQ hello world application.

Complete Recv.java

import com.rabbitmq.client.*;
import java.io.IOException;

public class Recv {

private final static String QUEUE_NAME = “hello”;
public static void main(String[] argv) throws Exception {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost(“localhost”);
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
System.out.println(” [*] Waiting for messages. To exit press CTRL+C”);
Consumer consumer = new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body)
throws IOException {
String message = new String(body, “UTF-8”);
System.out.println(” [x] Received ‘” + message + “‘”);
}
};
channel.basicConsume(QUEUE_NAME, true, consumer);
}
}

Final Thoughts

RabbitMQ has been adopted in thousands of deployment environments. It provides a significant boost in the scalability and loose coupling of applications. Today, it is by far the most popular message broker. Moreover, it also provides convenience with the cloud and also supports various message protocols, making it a desirable option for your development toolbox.

Serverless Computing: An Introduction to Amazon Lambda


AWS Lambda is a service provided by AWS that relates to the compute service. It assists in the running of application code where a physical server is not required. Lambda processes code per user requirements and raises the scalability bar when the need arises.

Whether your application deals with a few user requests or you need to handle thousands of requests, Lambda can be handy. Lambda utilizes Amazon’s highly powerful IT cloud infrastructure for running its compute services where the hardware and OS intricacies are delegated.

Considerations for Writing Lambda Functions

Irrespective of your choice of programming language, the understanding, and usage of the following components are important for the creation of a function.

Handler

Handler is used by Lambda for the execution of your function. Handler is adjusted after a function is created. Whenever a function has to be invoked, execution initiates with the help of the handler.

Context Object

A handler also receives a context object from the AWS Lambda, which can be marked as the second parameter. Context object provides communication between AWS Lambda and your written code.

Logging

Lambda functions keep a track of logging statements.

Exceptions

A result of your Lambda function’s execution is necessary to be conveyed to the AWS Lambda. This can be done through various strategies so a request’s lifecycle can come to an end. Likewise, the occurrence of an error can also be notified to the AWS Lambda. AWS Lambda passes the function execution result to the client if a function is invoked through synchronous means.

Writing a Simple AWS Lambda Function

Let’s see an example of our traditional “hello world” where we can run code of an AWS Lambda function without the use of a server— purely on the cloud!

1-     Lambda Console

Open the AWS Management Console. Check for the option of Lambda that appears under the Compute button. Click it so that the Lambda Console can be opened.

2-     Choosing a Blueprint

Now, you have to choose a blueprint. Blueprints have pre-existing code to speed up the processing. Blueprints can handle events from different sources. In the console, click the button of “Create a Function”. Now, click the “Blueprint” option. Find the filter box and enter the following details.

“hello-user-python”.

Choose the associated blueprint.

Now, press the Configure option.

3-Configuring the Function

Lambda functions store lines of code that are written by the users while they also manage dependencies and configuration. Configuration details can include the allocation of the resources for compute like memory, timeout of execution, etc. Lambda takes these details as input and does the required processing in return.

Now, you have to enter a description for your function. This description includes the following.

  • Name – Select a name for your function. For this article, we can use the “hello-user-python”.
  • Role – An execution role can be created which carries certain authorizations. Lambda uses the role for the invocation of a function. Choose the option of “Create a new role from template”.
  • Role Name – Select a role name for your function. For this example, we can use lambda_simple_procsessing.
  • Policy Templates – Before the generation of a function, a role is assigned after selecting an appropriate template.

A sample code is provided under the label of Lambda Function Code. Go to the lower part of your screen now and choose the option of “Create Function”. The coding for Lambda function is supported in all the popular programming languages like C#, Node.js, Java, Python. By default, Python is used for the runtime.

For managing the code, a handler method can be defined in the code. Lambda passes data related to events to the handler after which processing of the event is initiated. By scrolling down the options, any configuration for the execution time or memory can be configured, though we will not modify it in this example.

Invoking the Function and Checking the Results

The Lambda function of the hello-user-python appears on the console. This function can be tested where you can review the results and view the logs. Find a dropdown menu with the name of the “Select a test event” and click on the “Configure Test Event”.

You have a textbox for the testing of a function through an event. Go through the template list of sample events and select the “HelloWorld”. You can now assign any name to your event like “HelloUser”. The field values for the text in JSON can be modified. However, the event structure should not be changed. Modify the “value1” field with the “hello user”.  Now, finish by clicking on the “Create Button” and click the “Test Button”.

If everything goes alright, you can view your results from the console. These results are classified into the following:

  • Execution – Confirms if the execution was successful or not.
  • Summary – Presents the crucial details from the log input.
  • Log Output – Views all the logs that are produced due to the execution of your Lambda function.

Metrics

Amazon CloudWatch engages in the supervision and reporting of the Lambda function’s metrics. For effective management of the code for its execution, Lambda keeps a record of the following and publishes them:

  • Count of requests.
  • A request’s latency.
  • Requests concluding with an error.

Click on the “Test” button a few times so the metrics can be produced and displayed. Now, choose the option “Monitoring” for the displaying of results. As you scroll down, you can find various Lambda function metrics.

Since Lambda supports the “pay-as-you-go” model, users have to pay according to the request numbers of the Lambda functions. To be specific, pricing is based on two invocation factors: Duration and count.

Removing the Function

A Lambda function does not incur any charges. It can be deleted through the console. Go the “Actions” button and choose the “Delete Function”. A pop-up will appear now for confirmation; choose “Delete”.

Well, now you have successfully created, managed and deleted a simple AWS Lambda Function.

Micro Service part 3 with an example


Problem

Before going into the details about microservices, it is important to understand the background of another architecture known as a monolithic architecture. A monolith application is generally a large one that has tightly coupled components. The challenges of monolithic application include the following:

  • It is hard to scale a monolithic application. Since all the components are tightly coupled, larger modifications are needed.
  • If the business aims to adopt a different technology then it is not viable to adopt it, due to the presence of certain constraints.
  • Automation is a hard nut to crack with monolithic applications.
  • Modern-day coding conventions and solutions are also difficult to implement.

What Are Microservices?

One of the easiest definitions of microservices is explained by Sam Newman.

Small autonomous services that work together

Microservices can be seen as a self-contained solution that helps in the provision of distinct business functionalities for applications. Various microservices may appear as separate but their combination as a whole runs the entire application smoothly.

For instance, suppose there is an e-commerce website. For simplicity purposes, we will divide its business processes into two modules. Firstly, we have the order module that helps customers to order a product by selecting customized options. Secondly, we have the processing module that will communicate with the back-end and verify the banking and other relevant details of the customer. In a monolithic application, a change in the order module means a change in the processing module too.

However, if we are talking about microservices, then essentially we separate these mini processes. In microservices architecture, we have an order microservice and a processing microservice. These microservices can exchange information through a protocol or interface like REST. Generally, this communication is stateless which means that there is no dependence on the state of a component. Additionally, each of the microservice is independent and manages its own data.

Why Use Microservices?

Work on the Immediate Problem

In the case of a monolithic application, an upgrade, repair or modification means tinkering with the entire codebase of the application. This dilemma is solved through the emergence of the microservices. Microservices help to focus and modify only the relevant component of the example. For example, if the above-mentioned order microservice needs a change in its business logic, then only its microservice needs to be worked upon. Restructuring or recoding might not seem like a difficult problem for small applications but in the case of enterprise applications, they consume a great deal of time and resources.

Organization of Teams

Often IT managers are unable to properly utilize their developers as they are unsure about how to divide the tasks of different modules. Subsequently, developers from different teams struggle in the debugging and modification of the code. With microservices, each service can be allocated a small team. Due to its loose coupling, developers are empowered to focus on their own services. Consequently, they are also saved from consulting with other teams for the updating of single business functionality.

Different Languages

Often a problem in an enterprise application is the selection of a programming language and framework. Sometimes, PHP is good for certain business functionality while sometimes Java’s security is the need of the hour. Luckily, the microservices architecture allows the writing of code for each service in the language of the developer’s choice. Since all the services communicate with each other through standardized protocols, hence microservices provide flexibility.

How Microservices Improve on the Previous Architectures?

There is a misunderstanding regarding the nature of microservices architecture. Some people believe that the microservices divide an application’s web, business and data models. This approach is not dissimilar to the vision behind the previous out-dated architectures. However, this is a faulty analysis.

Instead, each microservice manages its own data model. Hence, only the team of a specific microservice can change its behavior. Another feature that separates microservices from others is stateless communication. Stateless communication helps in the scalability of the application as each pair of the request and response is handled independently.

Example

For a practical implementation, let’s take a look at an example of Hello World application using Microservices in Java. We will create a HelloWorldService class.

class HelloWorldService {

public String greet() {

return “Hello, World!”;

}

The above-mentioned code can be written in different Java environments. For example, for our console application, we can write the following.

class Starter {

  HelloWorldService helloWorldService = new HelloWorldService();

  public static void main(String[] args) {

    String message = helloWorldService.greet();

    System.out.println(message);

  }

}

For java servlets, we can write the following lines of code.

class HelloWorldServlet extends HttpServlet {

  HelloWorldService helloWorldService = new HelloWorldService();

  public void doPost(HttpServletRequest request,

    HttpServletResponse response) throws ServletException, IOException {

    String message = helloWorldService.greet();

    response.getWriter().println(message);

  }

}

For coding the controllers of Spring MVC applications, we will have to write the following piece of code.

@Controller

class HelloWorldController {

  HelloWorldService helloWorldService = new HelloWorldService();

  @RequestMapping(“/helloWorld”)

  public String greet() {

    String message = helloWorldService.greet();

    return message;

  }

}

Now, we have to solve the service issues that can be either related to the entire service’s unavailability or its ineffectiveness in returning an appropriate response. Service unavailability is mainly the client’s headache to deal with. With code written with the help of frameworks like unirest.io, clients are able to manage the service unavailability better.

Future<HttpResponse<JsonNode>>future= Unirest.post(“HTTP://helloworld.myservices.local/greet”).header(“accept”,”application/json”).asJsonAsync(new Callback<JsonNode>() {

public void failed(UnirestException e) {  //tell them UI folks that the request went south

    }

public void completed(HttpResponse<JsonNode> response) { //extract data from response and fulfill it’s destiny }

    public void cancelled() {//shot a note to UI dept that the request got cancelled   } } );

In the case the service fails, you can use the following code for the response.

{

  “status”:”ok”,

  ”message”:”Hello, World!”

}

The status attributes help to show the correct nature of a response.

Final Thoughts

Since its emergence, microservices have reinvented several enterprise applications and helped save developers from a great deal of complexities. Now, developers do not need to reduplicate their codebases. The working of entire projects has been improved vastly this way.

Microservice through my lens (Part2)


Have you finally taken a leap of faith and resolved to adopt the microservices architecture? Hopefully, it is for the best. The other out-dated solution architecture might have been compromising the performance, security, and customer satisfaction of your applications, website or mobile app.

However, as you will delve deeper in the microservices world, you will face certain hiccups. Thankfully, a number of design patterns and best practices have been introduced in the software community that can help you to tackle these challenges and move forward with your application.

API Gateway

While using the micro-services architecture, often the client-side has to face a certain issue. The problem stems from the client’s inability to gain access to different services. Through applying the API gateway pattern, client-side is provided with a single access point.

This point acts as a center from which it can interact with other services of the application. Sometimes, the API will send a request from the client to its suitable receiver while other times, more than a single service will receive the request. As a result, the client-side does not need to fall into the complexities about how different microservices have been split up by the application.

Service Registry

Often microservices in an application struggle to locate the free instances of the services. In this case, the service registry can be helpful. Service registry can be basically considered as a DB for all of your services.

It holds the instances of the services which are registered and de-registered on each startup and shutdown. Hence, clients can search for any available or free instance through the service registry. However, there are few problems that are related to the pattern.

One of them is that it needs constant configuration and management. In case a service registry falters, crucial data can be lost. Hence, it has to be ensured that the service registry is always available for use by the application.

Circuit Breaker

In a microservices application, services often have to communicate and execute tasks together. A service can receive a request for which it has to call another service for action. However, the summoned service can sometimes be unavailable due to an issue. In such a predicament, valuable resources can be wasted and the actual service that was called will not be able to process other incoming requests. As a result, one service’s failure incurs a significant loss to the entire application’s resources.

In such events, ‘circuit breaker’ is the need of the hour. It is a remote service that listens to the communication between two services. If a service fails to respond after a certain limit, then the circuit breaker will trip. The summoned service cannot communicate any further during the timeout interval. After the interval, few requests can be accepted by the circuit breaker to see if the service has regained its working. In case the service is working, the communication will be restored. However, a timeout interval will follow if the called service is still unavailable.

DB per Service

Let’s suppose you are working with an e-commerce application. For the order of products, you have an ‘Order’ service while for payment, there is a ‘payment’ service. Since the majority of the services require data to be stored, how will you manage your data storage of services? Since it is a microservices application, you cannot use a single DB. Instead, you can link each microservices’ API with its own DB.

As a result, each service will be able to keep its data confidential. You can also try to use separate tables, schemas and DB servers for your services. With every service having a dedicated DB, one of the primary requirements for microservices architecture, loose coupling, is achieved. Additionally, each service will be able to use a DB according to its needs.

Health Check API

Often, a service instance is running fine but it is unable to process requests. This can happen if the DB connections are not available. For this scenario, a monitoring mechanism is required that can serve as a warning tool. Hence, in order to alert about a running service that is facing difficulty in processing requests, we can use health check API. As the name suggests, it is used to examine and alert about the health of a microservice. The API will examine things like application-specific logic, disk space, connection status etc. However, it must be noted that a service instance can still falter in the middle of a health check and thus the pattern should not be considered to deliver 100% success.

Messaging

For handling and processing the requests from the clients as well as working together with other microservices, a standard communication mechanism is required that can enhance the performance of the application through effective communication. For this purpose, the asynchronous message can be the solution to your problems. It helps in inter-service communication through which microservices are able to pass all types of messages to each other. Kafka and RabbitMQ are one of the most popular messaging tools available. Due to the message broker mechanism, requests are handled better and do not get lost. However, the message broker has to be available 24/7 while the client will also need to know the message broker’s address.

Log Aggregation

While dealing with a large application that is built on microservices architecture, you will have to deal with a number of instances spawning from each service. There would be a continuous stream of requests that have to be handled by all the instances. These instances produce information about their workings to a log file.

The log file will entail debug, warnings, errors and other information. Hence, in order to increase the understanding of an application’s complete behavior, a logging service can be used that is based on the centralized model. The service will help to accumulate the log data from all the instances of services. As a result, IT professionals can find and understand these logs and apply configuration for alerts so any important message can be displayed on a priority basis.