Evolution of Event-Driven Architecture


As I explained in my previous post (here and here) Event Driven Architecture is key to digital transformation. here I will be talking about how it evolved.

Nowadays, the trend in event-driven architectures is one in which messaging is complex and quite different from a basic pipe which establishes connections between systems. Today, event-driven architectures combine elements from distributed database systems and streaming systems through where users can join, aggregate, modify and store data. The modern day implementations of these practices have been classified into four general patterns.

1.   Global Event Streaming Platform

This pattern bears similarity to the traditional enterprise messaging architectures. By using this event-driven approach, a company uses core datasets. These datasets share data from application modules like customers, payments, accounts, orders, and trades while an event streaming platform (for .e.g. Apache Kafka) is used.

This approach is a replacement for point-to-point communication technologies which are used in legacy systems. In such systems, applications are connected over multiple separate locations in real time.

For example, a business which runs a legacy system in New York can also operate through international branches in Stockholm and Berlin while connected through microservices with Amazon Web Services, all at the back of the same event-driven approach. A more complex use can include the connection of various shops across regional waters.

Some renowned companies who have adopted this approach include Audi, Netflix, Salesforce, ING, and others.

2.   Central Event Store

Events can be cached by streaming platforms for a specific or undefined time period to generate an event store, a type of organizational ledger.

This approach is used by companies for retrospective analysis. For instance, it can be used to train models in ML (machine learning) for detecting frauds of an e-commerce website.

This approach facilitates developers to create new applications without needing the republishing of previous events by the source systems. As a result, it becomes easier to replay datasets from their actual source which can be a legacy, external, or mainframe system.

Some companies store their complete data in Apache Kafka. This approach comes under event sourcing, forward event cache, or event streaming.

Event storage is necessary for stream processing with states to generate self-sufficient and enriched events through several distinct data sources. For example, it can be used to enrich orders from a customer module.

FaaS implementations or microservices are easily able to consume enriched events as these events carry all the required data for the service. Additionally, they are also used for the provision of a database’s denormalized input. These enrichments are executed by stream processors which require event storage to store data which works with tabular operations.

3.   Event-First and Event Streaming Applications

Usually, in conventional setups, applications gather data from several locations and import datasets in the database after which it can be filtered, joined, aggregated, and cleaned. This is a particularly effective strategy for applications which require creating dashboards, reports or operate via online services. However, in business processing, efficiency can be attained by ignoring the DB step and instead of sending real-time events into a serverless function or microservices.

For such approaches, stream processors like KSQL and Kafka Streams execute the operations like joining, filtering, and aggregating event streams to manipulate data.

For instance, suppose there is a service for limit checking which joins payments and orders through KSQL or a similar stream processing. It extracts the required fields and records and then sends it to a FaaS (function as a service) or a microservice where a check is executed for the limit while no DB is used.

This event-driven approach offers a greater degree of responsiveness to systems because they can be quickly and conveniently built due to the lesser data and infrastructural requirements.

4.   Automated Data Provisioning

This approach mixes the above-mentioned practices while combining serverless/PaaS implementations with the aim of making data provisioning as self-service.

In automate data provisioning, a user specifies the type of data which they require including its form or where it should land like a microservice, FaaS, database, or a distributed cache.

The infrastructure is provisioned by the system which also pre-populates it accordingly and supervises the flow of events. The buffering and manipulation of data streams is done by stream processing. What this means is that for instance, a retail business may have to join payments, customer, and real-time orders after which they send it into a FaaS or a microservice.

The migration of businesses to both private and public clouds has overseen an adoption of this pattern. It helps to initiate new project environments. Since datasets are stored or cached in the messaging system; therefore users only have to use data which they require at a specific time. Meanwhile, earlier in traditional messaging, it was a common practice to hoard and consume complete datasets so they can be used later.

 

Shedding Light on the Evolution

Over the last few years, there has been considerable evolution in event-driven architectures. In the beginning, they were only used to pass messages—use of state transfer and notification for standard messaging systems.

Afterward, these architectures were enhanced with the help of improved centralized control and better out-of-the-box connectivity. However, centralized control was a tricky part; due to standardization, teams found it difficult to progress. In recent memory, storage patterns like CQRS and event sourcing have gained popularity. The above-mentioned approaches are based on it.

Nowadays modern event streaming systems have taken the next step to unify storage, processing, and events into the same platform. This amalgamation is necessary as it shows that these systems are different from databases which store data in one place. Moreover, they do not even fall into the category of messaging systems in which data is transitory. Thus, they are a combination of both databases and messaging systems.

The correct use of these standard categories has allowed organizations to target several regions and clouds for global connectivity and data has become one of their most prized commodities used in provisioned as a service. This means that it can be pushed into a cache, database, machine learning model, serverless function, or a microservice.

 

 

 

 

 

Advertisement

An Introduction to Multikey Indexes with Examples


After understanding single-field and compound indexes, now is the time to learn about multikey indexes. When a field has an array value, an index key is generated for each of its array elements. These indexes are referred as multikey indexes. They can be used with arrays that have scalar values as well as those with nested documents.

To generate a multikey index, you have to use the standard db.collection.createIndex() method. Such indexes are automatically generated by MongoDB whenever it senses an indexed field specified as an array. Hence, there is no need for explicit definition of a multikey index.

Examples

While working with a standard array, let’s suppose we have a collection “student”.

{ _id: 21, name: “Adam”, marks: [ 80, 50, 90 ] }

To build an index on the “marks” field, write the following query.

db.student.createIndex( { marks: 1 } )

As the marks field is an array, thus this index is an example of a multikey index. All of the keys (80, 50, and 90) in its elements point to the same document.

To create a multikey index for array fields having embedded documents, let’s suppose we have a collection “products”.

{

_id: 5,

name: “tshirt”,

details: [

{ size: “large”, type: “polo”, stock:50 },

{ size: “small”, type: “crew neck”, stock:40 },

{ size: “medium”, type: “v neck”, stock: 60 }

]

}

{

_id: 6,

name: “pants”,

details: [

{ size: “large”, type: “cargo”, stock: 35 },

{ size: “small”, type: “jeans”, stock: 65 },

{ size: “medium”, type: “harem”, stock: 10 },

{ size: “large”, type: “cotton”, stock: 10 }

]

}

{

_id: 7,

name: “jacket”,

details: [

{ size: “large”, type: “bomber”, stock: 35 },

{ size: “medium”, type: “leather”, stock: 25 },

{ size: “medium”, type: “parka”, stock: 45 }

]

}

We can build a multikey index with the details.size and details.stock fields.

db.products.createIndex( { “details.size”: 1, “details.stock”: 1 } )

This index is now good to go against queries which have only a single field of “details.size” as well as queries having the both of the indexed fields. As such, these types of queries would benefit from the index.

db.products.find( { “details.size”: “medium” } )

db.products.find( { “details.size”: “small”, “details.stock”: { $gt: 10 } } )

Bounds in Multikey Index

Bounds represent the limits of an index .i.e. how much it needs to scan for searching a query’s results. If there are more than a single predicate with an index, then MongoDB integrates them through compounding or intersection. So what do we mean by intersection and compounding?

Intersection

Bounds intersection point towards the presence of “AND” (logical conjunction) for bounds. For example, if there are two bounds [ 4, Infinity]  and [ – Infinity, 8 ], then intersection bounds process [[4, 8 ]] When $elemMatch operation is used, then MongoDB applies intersection on multikey index bounds.

 

What Is $elemMatch?

Before moving forward, let’s understand the use of $elemMatch first.

$elemMatch is used for matching documents in array field where at the bare minimum, atleast one of the element is matched with the query. Bear in mind, that the operator does not work with $text and $where operators. For a basic example, consider a “student” collection.

{ _id: 8, marks: [ 72, 75, 78 ] }

{ _id: 9, marks: [ 65, 78, 79 ] }

The following query only processes a match with documents in which the “marks” array has at least a single element which is less than 75 and greater or equal to 70.

db.student.find(

{ marks: { $elemMatch: { $gte: 70, $lt: 75 } } }

In response, the result set is comprised of the following output.

{ “_id” : 8, “marks” : [72, 75, 78 ] }

Despite the fact that both 75 and 78 do not conform to the conditions but because 72 had a matched, hence the $elemMatch selected it.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

For instance, we have a collection student which has a field “name” and an array field “marks”.

{ _id: 4, name: “ABC”, marks: [ 1, 10 ] }

{ _id: 5, name: “XYZ”, marks: [ 5, 4 ] }

To build a multikey index with the “marks” array.

db.student.createIndex( { marks: 1 } )

Now, the following query makes use of $elemMatch which means that the array must have at least one element which fulfils the condition of both predicates.

db.student.find( { marks : { $elemMatch: { $gte: 4, $lte: 8 } } } )

Computing the predicates one by one:

  • For the first predicate, the bounds are equal to or greater than 4 [ [ 4, Infinity ] ].
  • For the second predicate, the bounds are equal to or less than 8 [ [ – Infinity, 8 ] ].

Since $elemMatch is used here, therefore MongoDB can apply intersection on the bounds and integrate it like the following.

marks: [ [ 4, 8 ] ]

On the other hand, if the $elemMatch is not used, then MongoDB applies intersection on the multikey index bounds. For instance, check the following query.

db.student.find( { marks : { $gte: 4, $lte: 8 } } )

The query processes the marks array for atleast a single element which is equal to or greater than 4 “AND” atleast a single element which is equal to or less than 8. However, it is not necessary for a single element to conform to the requirements of both predicates, hence MongoDB does not apply intersection on the bounds and uses either [[4, Infinity]] or [[-Infinity, 8]].

Compounding

Compounding bounds means the use of bounds with compound index. For example, if there a compound index { x: 1, y:1 } which has a bound on the x field  [[4, Infinity] ] and a bound on the field y [[-Infinity, 8]]. By applying compounding,

{ x: [ [ 4, Infinity ] ], y: [ [ -Infinity, 8 ] ] }

Sometimes, MongoDB is unable to apply compounding on the given bounds. For such scenarios, it uses the bound on the leading field which in our example is x: [ [4, Infinity] ].

Suppose in our example indexing is applied on multiple fields, where one of the fields is an array. For instance, we have the collection student which stores the “name” and “marks” field.

{ _id: 10, name: “Adam”, marks: [ 1, 10 ] }

{ _id: 11, name: “William”, marks: [ 5, 4 ] }

Build a compound index with the “name” and the “marks” field.

db.student.createIndex( { name: 1, marks: 1 } )

In the following query, there is a condition which applies on both of the indexed keys.

db.student.find( { name: “William”, marks: { $gte: 4 } } )

Computing these predicates step-by-step.

  • For the “name” field, the bounds for the “William” predicate are the following [ [ “William”, “William” ] ].
  • For the “marks” field, the bounds for { $gte: 4 } predicate are [ [ 4, Infinity ] ].

MongoDB can apply compounding on both of these bounds.

{ name: [ [ “William”, “William” ] ], marks: [ [ 4, Infinity ] ] }

 

 

 

 

 

 

 

Design Patterns for Functional Programming


In software circles, a design pattern is a methodology and documented approach to a problem and its solution which is bound to be found repeatedly in several projects as a tumbling block. Software engineers customize these patterns according to their problem and form a solution for their respective applications. Patterns follow a formal structure to explain a problem and then go over a proposed answer as well as key points which are related to either the problem or the solution. A good pattern is one which is well known in the industry and used by the IT masses. For functional programming, there are several popular design patterns. Let’s go over some of these.

Monad

Monad is a design pattern which takes several functions and integrates them as a single function. It can be seen as a type of combinatory and is a core component of functional programming. In monad, a value is wrapped in a box which is then unwrapped and a function is passed to use the wrapped value.

To go into more technicalities, a monad can be classified into running on three basic principles.

·         A parameterized type M<T>

According to this rule, T can possess any type like String, Integer, hence it is optional.

·         A unit function T -> M<T>

According to this rule, there can be a function taking a type and its processing may return “Optional”. For instance, Optional.of(String) returns Optional<String>.

·         A bind operation: M<T> bind T -> M <U> = M<U>

According to this rule which is also known as showed operator due to the symbol >>==. For the monad, the bind operator is called. For instance, Optional<Integer>. Now this takes a lambda or function as an argument for instance like (Integer -> Optional<String> and returns and processes a Monad which has a different type.

Persistent Data Structures

In computer science, there is a concept known as a persistent data structure. Persistent data structure at their essence work like normal data structure but they preserve their older versions after modification. This means that these data structures are inherently immutable because apparently, the operations performed in such structures do not modify the structure in place. Persistent data structures are divided into three types:

  • When all the versions of a data structure can be accessed and only the latest version can be changed, then it is a partially persistent data structure.
  • When all the versions of a data structure can be accessed as well as changed, then it is a fully persistent data structure.
  • Sometimes due to a merge operation, a new version can be generated from two prior versions; such type of data structure is known as confluently persistent.

For data structure which does not show any persistence, the term “ephemeral” is used.

As you may have figured out by now, since persistent data structures enforce immutability, they are used heavily in functional programming. You can find persistent data structure implementations in all major functional programming language. For instance, in JavaScript Immutable.js is a library which is used for implementing persistent data structures. For example,

import { MapD } from ‘immutable’;

let employee = Map({

employeeName: ‘Brad’,

age: 27

});

employee.employeeName; // -> undefined

employee.get(’employeeName’); // -> ‘Brad’

Functors

In programming, containers are used to store data without assigning any method or properties to them. We just put a value inside a container which is then passed with the help of functional programming. A container only has to safely store the value and provide it to the developer in need. However, the values inside them cannot be modified. In functional programming, these containers provide a good advantage because they help with forming the foundation of functional construct and assist with asynchronous actions and pure functional error handling.

So why are we talking about containers? Because functors are a unique type of container. Functors are those containers which are coded with “map” function.

Among the simplest type of containers, we have arrays. Let’s see the following line in JavaScript.

const a1 = [10, 20,30, 40, 50];

Now to see a value of it, we can write.

Const x=y[1];

In functional, the array cannot be changed like.

a1.push(90)

However, new arrays can be created from an existing array. An array is theoretically a function. Technically, whenever a unary function is mapped with a container, then it is a functor. Here ‘mapped’ means that the container is used with a special function which is then applied to a unary function. For arrays, the map function is the special function. A map function processes the contents of an array and performs a special function for all the elements of the element step-by-step after which it responds with another array.

Zipper

A zipper is a design pattern which is used for the representation of an aggregate data structure. Such a pattern is good for codes where arbitrarily traversal is common and the contents can be modified, therefore it is usually used in purely functional programming environments. The concept of Zipper dates back to 1997 where Gérard Huet introduced a “gap buffer” strategy.

Zipper is a general concept and can be customized according to data structures like trees and lists. It is especially convenient for data structures which used recursion. When used with zipper, these data structure are known as “a list with zipper” or “a tree with zipper” for making it apparent that their implementation makes use of zipper pattern.

In simple terms, zipper with data structure has a hole. They are used for the manipulation and traversal in data structures where the hole indicates the present focus for the traversal. Zipper facilitates developers to easily move within the data structure.

Best Functional Programming Practices – When to Use Functional?


For any paradigm, the global developer community experiences several common issues in their development of projects. To counter the recurring issues, they begin exercising certain practices for getting the most out of a paradigm. For functional programming in Java too, there have been a number of practices which have been deemed as useful and valuable for programmers. Let’s go over some of them.

Default Methods

Functional interfaces remain “functional” even if default methods are added. Though, if more than one abstract method is added, then they are no longer a functional interface.

 

 

 

 

@FunctionalInterface

public interface Test {

String A();

default void defaultA() {}

}

As long as the abstract methods of functional interface retain identical signatures, they can be extended by other functional interfaces. For instance,

 

 

 

 

 

 

 

 

 

 

 

 

@FunctionalInterface

public interface TestExtended extends B, C {}

@FunctionalInterface

public interface B {

String A();

default void defaultA() {}

}

 

@FunctionalInterface

public interface C {

String A();

default void defaultB () {}

}

 

 

When usually interfaces are extended, they encounter certain issues. They are recurrent with functional interfaces too when they run with default methods. For instance, if the interfaces B and C have a default method known as defaultD()  then you may get the following error.

interface Test inherits unrelated defaults for defaultD() from types C and D…

To solve this error, the defaultD() method can be overridden with the Test interface.

It should be noted that from the software architecture perspective, the use of a lot of default methods in an interface is detrimental and discouraged. This is something which should be used for updating older interfaces while escaping any backward compatibility issue.

Method References

Many times, methods which were implemented before are called out by lambdas. Hence, for such cases, it is good to make use of method references, a new feature in Java 8.

For example, if we have the following lambda expression.

x -> x.toUpperCase();

Then it can be replaced by:

String::toUpperCase;

Now, this type of code does not only reduce line of codes but it is also quite readable.

Effectively Final

Whenever a variable is accessed which is not “final” and resides in lambda expression, then an error is likely to be caused. This is where “effectively final” comes into play. When a variable is only assigned once then the compiler thinks of it as a final variable. There is nothing wrong in using these types of variables in lambda expressions as their state is managed by the compiler and it can notify for an error as soon as their state is meddled with. For instance, the following code cannot work.

public void A(){

String lVar = “localvariable”;

Test test = parameter -> {

String lVar = parameter;

return lVar;

};

}

In return, the compiler may notify you that “lVar” does not need to be defined because it already has been in the scope.

No Mutation for Object Variables

Lambda expressions are predominantly used in parallelism or parallel computing because of their protection for threads. The paradigm “effectively final” can help at times but sometimes it is not good enough. An object’s value cannot be changed from the closing scope by lambdas. On the other hand, with mutable object variables, it is possible for a state in lambda expression to be modified. For instance check the following.

int[] n = new int[1];

Runnable rn = () -> n[0]++;

rn.run();

Now the above code is perfectly legal because the “n” variable stays “effectively final”. However, it has referenced an object and the state of that object can change. Hence, use this example to remember not writing code which may give rise to mutations.

When to Use Functional?

Before learning functional programming, you must be curious about its actual advantage over other paradigms. When you have a task at hand where you are dabbling with parallelism and concurrency, in such cases functional programming can be a good choice. In real life scenarios, for this purpose Erlang was used a lot in Erricson for its telecommunication work. Likewise, Whatsapp has always been involved in a similar use. Other success stories include the reputable Lucent.

For any individual who has dialed a number in the past three decades in US, there are strong possibilities of their use of devices which have code in a language known as Pdiff. Pdiff itself was created from a functional programming language, Standard ML.

Pdiff’s example can be used to recall functional programming’s brilliance with DSLs (domain specific languages). Sometimes, common programming languages like C++, C#, and Java struggle to create a solution for certain issues where DSLs were the life-savers. While DSLs are not used to design entire systems but they can prove invaluable to code one or two modules. Industry experts consider functional programming as an excellent option to write DSL.

Moreover, functional programming is quite good at solving algorithms, particularly those filled with mathematics. Mathematical problems can be solved well in functional, perhaps due to its closeness in theoretical foundations with mathematics.

When to Not Use Functional?

So when to not use functional programming? It is said that functional programming does not work well with the general “library glue code”. It is a disaster for recipe with the general building of structure classes which are used in mainstream development. This means that in case your code-base is filled with classes working like structures, and if the properties of your object are changing continuously, then functional is probably not the best idea.

Likewise, functional is also not good for GUIs (graphical user interface). The reason is that GUIs have always been deemed more suitable for OOP because of the reusability factor. In GUI applications, modules are derived with little changes from other modules. There is also the “state” factor as GUIs are stateful (at least in the view).

 

Dos and Don’ts of Functional Programming


While working with functional programming in Java, there are several do’s and don’ts. Let’s go over the most common ones. By making use of these tips, you can improve your functional programming code-base with good results.

Standard Functional Interfaces

The Functional interfaces in “java.util.function” are good enough to fulfill requirements for method references and lambda’s target types. As explained earlier, these interfaces are abstract, thereby making it easier to customize them freely into a lambda expression. Hence, before writing new functional interfaces, makes sure to check this package which may satisfy your requirements.

Suppose you have in interface FuncInt

@FunctionalInterface

public interface FuncInt {

String f1(String txt);

}

Now you have a class for using the interface with the name useFuncInt which has a method for sum.

public String sum(String txt, FuncInt) {

return FuncInt f1(txt);

}

For execution of it, you would have to add the following:

FuncInt fi = txt -> txt + ” coming from lambda”;

String output = useFuncInt.add(“Message “, useFuncInt);

Now, if you observe closely, you will realize that FuncInt is a function which takes one argument and returns with an answer. With the release of Java 8, this functionality is enabled through the interface in “Function<T,R> which is entailed in the package of “java.util.function”. Our FuncInt can be fully eliminated and our code can be modified to.

public String sum(String txt, Function<String, String> f) {

return f.apply(txt);

}

For execution, write the following:

Function<String, String> f =

parameter -> parameter + ” coming from lambda”;

String output = useFuncInt.sum(“Message “, f);

Annotation Matters

Make the use of annotation by adding functional interfaces with the help of @FunctionalInterface. If you are unfamiliar with annotations, then put it with a tag which is used to represent the metadata; this could be related to an interface, method, field, or class about any useful information that can prove to be helpful for the JVM (Java Virtual Machine) or the complier.

The @FunctionalInterface may not seem logical at the first glance because without adding it, the interface would work fine as a functional one as long it contains only one abstract method.

However, in large-scale projects where the increasing number of interfaces may overwhelm you, manual control is tricky and exhaustive; it becomes a nightmare to keep track of it. Sometimes, an interface which was created as a functional interface may get additional abstract methods, therefore ceasing to exist as a functional interface. This is where the @FunctionalInterface annotation works like a charm. By using it the complier would display an error whenever there is any intrusion by a programmer to change the structure of a functional interface and disrupt its “purity”. Likewise, while working in teams, it can be helpful for other coders to understand your code and identify functional interfaces easily.

Hence, always use something like this,

@FunctionalInterface

public interface FuncInt{

String printSomething();

}

Instead of writing something like this:

public interface FuncInt{

String printSomething();

}

Even if you are working on beginner projects, make it a habit to always add the annotation or you may face the repercussions in future.

Lambda Expressions Cannot Be Used Like Inner Classes

When an inner class is used, “scope” is generated. This means that we can add variables which are local with the enclosing scope after instantiating local variables having identical names. The keyword “this” can also be used in our class for referencing to its own instance.

With lambda expressions, we work by using enclosing scope. This means that variables cannot be overwritten in the body of the lambda expression from the enclosing scope. We can use “this” for referencing.

For instance, for our class UseTest, we have “test” as our instance variable:

private String txt = “Encircling the scope”

Now, you can use another method of our class with the following code and run it like this:

public String example()

{

Test test = new Test();

String mssg = “inner value of the class”

@Override

Public String returnSomething(String txt)

Return this.mssg;

}

};

String output = test.returnSomething(“”);

Test testLambda = parameter -> {

String mssg = “Lambda Message”;

return this.mssg;

};

String outputLambda= testLambda.returnSomething(“”);

return “Output: output = “ + output + “outputLambda = “ + outputLambda;

When you will run the “example” method, you may get the following: “Output: output =  inner value of the class, outputLambda = Encircling the scope

This means that you can access and use a local variable by accessing its instance. By using “this”, you might have used the variable “test” of the UseTest class but you were unable to get the “test” value which in entailed in the body of the lambda.

Lines of Code in Lambdas

Ideally, lambda expressions should comprise of a single line of code because this makes it a self-explanatory concrete implementation where the clear execution of an action at some data is represented.

If you require more lines for your functionality, then write something like this:

Test test = parameter -> generateText(txt);

private String generateText(String txt) {

String output = “Some Text” + txt;

// several lines of code

return output;

}

Refrain from writing something like this:

Test test = txt -> { String output= “Some Text” + txt;

// several lines of code

return output;

};

Though, it is important to note that sometimes, it is ok to have more than one line of lambda expressions where the use of another method may prove to be counter-productive.

Type of Parameters

The compilers are powerful enough to ascertain the types of parameters in lambda expressions with type inference. Hence, there is no need for adding the parameter type explicitly.

Don’t write something like this.

(String x, String y) -> x.toUpperCase() + y.toUpperCase();

Instead, write something like this,

(x, y) -> x.toUpperCase() + y.toUpperCase();

Java Lambdas


So far we have talked a lot about functional programming. We discussed the basics and even experimented with some coding of functional interfaces. Now is the right time to touch one of the most popular features of Java for functional paradigm, known as lambda expressions or simply lambdas.

What Is a Lambda Expression?

A lambda expression provides functionality for one or more functional interface’s instances with “concrete implementations”. Lambdas do not require the use of a class for their use. Importantly, these expressions can be viewed and worked by coding them as objects. This means that, like objects, it is possible to pass or run a lambda expression. The basic style for writing a lambda expression requires the use of an “arrow”. See below:

parameter à the expression body

On the left side, we have a “parameter”. We can write single or multiple parameters for our program. Likewise, it is not mandatory to specify the parameter type because compilers already ascertain the parameter type. If you are using a single parameter, then you may or may not add a round bracket.

However, if you intend to add multiple parameters, then make sure to use round brackets (). Sometimes, there is no need of parameters in a lambda expression. For such cases, it is possible to signify them by simple adding an unfilled round bracket. To avoid error, use round brackets for parameter whether you are using a 0, 1, or more parameters.

On the right side of the lambda expression, we can have an expression. This expression is entailed in curly brackets. Like parameters, you do not require brackets for a single expression while multiple expressions require one. However, unlike parameter, the return type of a function can be signified by the body expression.

Without Lambdas

To understand lambdas, check this simple example.

package fp;

public class withoutLambdas {

public static void main(String[] args) {

withoutLambdas wl = new withoutLambdas(); // generating instance for our object

String lText2 = “Working without lambda expressions”; // here we assign a string for the object’s method as a parameter

wl.printing(lText2);

}

public void printing(String lText) { // initializing a string

System.out.println(lText);              // creating a method to print the String

}

}

 

The output of the program is “Working without lambda expressions”. Now if you are familiar with OOP, then you can understand how the caller was unaware of the method’s implementation i.e. it was hidden from it. What is happening here is that the caller gets a variable which is then used by the “printing” method. This means we are dealing with a side effect here—a concept we explained in our previous posts.

Now let’s see another program in which we go one step ahead, from a variable to a behavior.

package fp;

public class withoutLambdas2 {

interface printingInfo {

void letsPrint(String someText);  //a functional interface

}

public void printingInfo2(String lText, printingInfo pi) {

pi.letsPrint(lText);

 

}

 

public static void main(String[] args) {

withoutLambdas2 wl2 = new withoutLambdas2(); // initializing instance

String lText = “So this is what a lambda expression is”; // Setting a value for the variable

printingInfo pi = new printingInfo() {

@Override // annotation for overriding and introducing new behavior for our interface method

public void letsPrint(String someText) {

System.out.println(someText);

}

};

wl2.printingInfo2 (lText, pi);

}

 

 

}

In this example the actual work to print the text was completed by the interface. We basically formulated and designed the code for our interface’s implementation. Now let’s use Lambdas to see how they provide an advantage.

 

package fp;

public class firstLambda {

 

interface printingInfo {

void letsPrint(String someText);    //a functional interface

}

public void printingInfo2(String lText, printingInfo pi) {

pi.letsPrint(lText);

}

 

public static void main(String[] args) {

firstLambda fl = new firstLambda();

String lText = “This is what Lambda expressions are”;

printingInfo pi = (String letsPrint)->{System.out.println(letsPrint);};

fl.printingInfo2(lText, pi);

}

See how we improved the code by integrating a line of lambda expression. As a result, we are able to remove the side effect too. What the expression did was use the parameter and processed it to generate a response. The expression after the arrow is what we call as a “concrete implementation”.

Core Concepts of Functional Programming


Now that you have learned about the paradigm shift to functional programming, let’s go into the depths of the fundamentals concepts that power functional programming. The comprehension of these basic concepts is important to create high-quality functional programming applications. You may be tempted to directly begin coding but these concepts can help you become a better coder.

1- Pure Functions

In functional programming, everything is seen as a function. Each function has to be “pure”. Pure here refers to two basic capabilities:

No Side Effects

A function can never be pure if it carries even a single side effect. A side effect is a property when a function’s states are modified by other functions. By states, we mean the data like variables or data structures. Pure functions do not carry any side effects; hence their memory or I/O operations can’t be affected. Now, you might wonder that why exactly does their presence considered bad. Well, because they make functions “unpredictable” where a function has to rely on its system’s state.

On the contrary, if a function’s state cannot be changed, then the same output is generated for the given input. A side effect of a function can also mean to write any operation which has been applied to the disk or turning on/off a control of your front-end UI’s function.

Same Result with Multiple Calls

Whenever a function is called without any modifications in its arguments, the same results are generated. Consider an example where you have designed a simple function “multiply (4, 5). Now, this function is expected to generate the same results .i.e. for each invocation. However, if you were programming in other paradigms then random functions or global variables may not allow your result to remain the same.

Pure functions also offer “memoisation”. Memeoisation refers to a technique in which pure functions’ output (always same result) is saved in the cache memory. Now, whenever such functions are invoked, caching helps to enhance the performance and speed of the application.

2- Higher Order Function

The concept of a function which is higher order is known in mathematics as well as computer science. Generally, they possess two fundamental characteristics.

Return Type

The return type of a higher-order function has to be a function. For example, review the following code in Java.

package fp;

 

public class higherOrderfunction {

public static void main(String args[]) {

 

System.out.println(A(4));

 

}

static int marks() {

int a=5;

return a;

}

static int A(int total) {

total =4;

return total+marks();

}

}

To simplify things, we have constructed a simple function marks. This function holds an integer value “a” which is returned. Now, we have a function A. A takes an argument for an integer total and assigns it a value. Now, comes the actual part. See how in return, we have used marks method as a return type. Since we have processed our function by returning another function; hence this function is a higher-order function.

Arguments

Another characteristic of a higher-order function can be its use of functions as input parameters. For instance, see this see pseudo-code:

Public areaRect (lb) // Here areaRect represents a function that takes arguments from another function lb to calculate length  and breadth

{

int area;

area =l*b;

return lb; //

}

This function is a higher-order function because it processes itself using another function as a parameter.

3- First Class Functions

After higher-order functions, we have first-class functions. These are not too dissimilar to higher-order functions. It is important to note that a first-class function always adheres to the terms and conditions of a higher-order function. So what this means is that a first-class function has to return another function as well as contain a parameter in the form of function. Hence, by default, all first-class functions are higher-order functions. So what exactly is the difference between them? Well, context matters!

By referring a language to have support for first-class functions, we generally mean that uses its functions as values that can be easily passed around.

On the other hand, the term higher-order is more associated with the mathematics outlook when pure problem-solving requires a more theoretical and general perspective of the problem.

4- Evaluation

Some languages support strict evaluation while others offer non-strict evaluation support. This evaluation is targeted at the language’s processing of an expression while considering the function parameters or arguments. To familiarize yourself better with the concept, check this simple example:

Print length([4-3,4+6,1/0,7*7,3-8])

When a programming language uses non-strict evaluation to process this expression, then it simply returns back a value of 5 .i.e. the total number of elements. Such evaluation does not concern itself with the depth of values.

On the other hand, the same expression returns an error with strict evaluation because it found the third element “1/0” to be incorrect. Hence, this means that strict evaluation is more stringent and processes an expression more deeply.

In a few scenarios, non-strict evaluation has to enforce processing of strict evaluation when a function needs to be evaluated on a “stricter” basis due to an invocation.

5- Referential Transparency

In functional programming, the usual assignment of values is not offered. Variables are immutable; a value defined once is the final one which is not possible to be changed in future. It is the property which makes them without any side-effects. To understand further, consider the following example, where the value of x is changed after each evaluation.

x=x*15

In the start of the program, x was assigned a value of 1. The first evaluation made it 15.

Now, in the second evaluation, the value of x changes to 225.

Now, this function is not referentially transparent because the value of x is continuously changing.

So now, if we go by the functional concepts, then our example can be altered into this pseudocode:

int sum(int x)

{

x=x+2;

}

This type of function ensures that the value of x remains constant and it cannot be altered implicitly.

What Is Functional Programming? Why the Paradigm Changed the Game?


The Path to Functional Programming

Before understanding functional programming, ask yourself how much do you know about another programming “styles”? When programming initially emerged to solve the major problems of the world through a few lines of code, Computer Scientists realized that they required a standard format or style which could help them to program effectively and efficiently. This style is commonly known as “programming paradigm”.

Soon developers began coding in C by using the procedural paradigm. Procedural programming mainly deals with coding with a step-to-step design similar to kitchen recipes; where a set of instructions is followed sequentially. At that time, the paradigm was indeed excellent at solving problems. However, as technologies evolved and programming became much more complex—websites were built and businesses began to adopt IT—the flaws of procedural programming bugged developers.

Enter Object-Oriented Programming, the next popular paradigm. The vision behind OOP was simple; it modeled programming on the basis of real-world examples. For example, a car could be seen as an object which possessed certain behavior (methods in programming) and states (members like variables in programming). OOP succeeded in decreasing global codebases. OOP concepts like encapsulation and inheritance were fundamental to attain an unexpected degree of productivity.

However, soon developers realized that OOP was not up to the task for a number of things. Hence, to address certain issues, functional programming came into the scene. Popular languages like PHP, Python, etc., are examples of languages which support the functional paradigm. Remember, not each language is built to support all paradigms. For example, while C can support procedural programming, it does not offer support for object-oriented programming.

However, most modern languages support procedural, OOP, and functional programming. There are some like Haskell, who received recognition due to features for functional programming. Java was initially not supportive of the paradigm, but since the last few years, Java releases have introduced features like Lambda Expressions to support functional programming. However, the question is, what exactly is functional programming?

What Is Functional Programming

The name suggests that it is linked to “functions”. Now, if you think that this function relates to the programming of methods, then your assumption is flawed. Functional programming refers to functions which incorporate a certain piece of code in the form of a feature or operation to an application. This function facilitates programmers to avoid changing the other parts of the application.

Functional programming highly borrows from mathematics. These functions are highly interlinked with those “mathematical functions” that you may have studied in your college courses.  In mathematics, a specific question is solved with a method without going much in the theoretical complexities of that method; similarly, functional programming is there to gain a higher degree of abstraction in applications.

One key aspect of functional programming is that it avoids change in the mutability and states of data. Another thing to note is that unlike “statements”, which are generally used in the OOP landscape, “expressions” power functional codebases.

Paradigm Shift to Functional Programming

Previously business applications used C++, a phenomenon that can prove to be a nightmare for modern developers. In those days, software engineers focused a lot on low-level aspects where issues related to memory management did not allow them to achieve much productivity. Then, Java came and provided more abstractions and managed these tasks through new features, thus resulting in saving developers from de-coding low-level complexities.

Today, languages like Scala (JVM) and F# (.NET) are promising a similar level of convenience to developers. Even outside of JVM and .NET ecosystems, you can clearly see Apple going with Swift and Facebook with React JS. Many of today’s “cool” languages are known for their functional paradigms. So why exactly has it become famous? Maybe, the following factors may have something to do with it.

Parallelism

One of the major reasons behind the positive reception of functional programming is its support for parallel computing. Traditionally, basic computing requirements like data storage and processing were not too intensive .i.e. there was no need of running several things at once. However, this changed as several technologies came and evolved one after another, expanding the world of software development and transforming it to power as a backbone of global operations.

Today, parallel computing is a highly valued domain of computer science where multiple applications and data require processing at the same time. Here, functional programming has made its impact due to its natural features which support independence components to run and support modern software infrastructure like microservices. This means if you try to add a specific functionality or modify an existing one, then “functions” ensure that you do not affect the other components of your system.

Data Streaming

Another reason which can be associated with the ‘functional leap’ is the advent of “streaming services”. a few years ago, entertainment was mainly associated with TV—the only medium for watching shows and movies.

However, today streaming services like Netflix have changed the game. Entertainment has shifted online. People prefer to take advantage of the luxury of watching football matches on mobile apps while commuting, rather than hurrying their way back to home for their television sets. Since functional programming works well with streaming due to its natural concepts; hence its growth makes a lot of sense.

AI

AI is perhaps the most exciting branch of computer science. Over the past few decades, extensive research has been carried out about AI while its applications and sub-branches like machine learning, deep learning, speech recognition, etc have become their own field of studies. AI is seen with a bright hope that it can modernize the world and take it to the next level through robots and intelligent machines. However, all this AI implementation requires coding where a paradigm has to be ultimately chosen.

Since AI is closely related to mathematics, logic, statistics, and theoretical computer science, hence you can certainly understand why functional paradigm—a paradigm founded on mathematics—is gaining so much prominence among the AI community.

What’s in it for the Developer?

Well, so far you might have understood how the elevation of some technologies and necessities of some requirements raised the need for functional programming. However, why should you as a developer working with “objects” (OOP) think about learning something new? Is the hassle worth it? Well, to be honest, functional programming is not a magic potion that can generate great results for all OF YOUR applications.

However, there are several cases where the use of functional programming can assist you to achieve better results in your projects where the time spent on learning the paradigm may result in saving you a great deal of resources. Following are a few of the reasons to learn it.

Decreasing LOC

What was one of the most well-liked thing aspects about OOP? The programs that used to require 5,000 line of codes via procedural programming, were reduced to less than 1,000 lines of codes by adopting OOP’s fundamental concepts like inheritance.

Similarly, functional programming also promises the same .i.e. it provides shorter code-bases while maintaining the performance of the application. As a result, productivity is increased where fewer lines of codes do more while maintenance is also relatively easier.

Testing and Debugging

When everything is a function; testing is less challenging. You just have to test each function where one’s output does not affect the result of the other function.

The states of functions cannot be modified from outside of its scope. Hence, its input can be tracked effectively, resulting in an efficient management of its output too. As a result, debugging is easy because it is easy to check what went wrong.

Career Growth

There are no negatives in learning a paradigm which goes well with all the hottest modern technologies. With a chance to work on IoT, AI, and, other futuristic fields, functional programming can prove pivotal to your success as a computer scientist. Learning and adapting are the hallmarks of every successful computer science. Also, if you are tired of doing the same old programming, then it can provide a new challenge that can motivate you to work contentedly.

Now that you have learned about some basic definition, importance, and advantages of functional programming, now is the right time to understand its basics.

What is RabbitMQ?


What is RabbitMQ?

The concept of messaging in the software environment is similar to the daily life processes. For example, you went for a morning coffee. After taking your order, the manager inputs it into the system. If there is no rush in the coffee shop, your order does not require to be added in a queue.  However, if there are previous orders, the system puts it behind other orders. Thus, your order becomes part of the queue.

However, what if there are countless orders and the server is unable to manage all those due to a hardware issue? What to do now? In such cases, a service like RabbitMQ can prove to be the game changer. RabbitMQ will take all the orders and only forward them to the server when it can manage the workload.

Before understanding RabbitMQ, it is essential to equip yourself with the knowledge of a message broker. A message broker is an intermediary program that works on the translation of the contents of a message with the messaging protocols of both the receiver and the sender. Message brokers are used as a middleware solution for a variety of software applications.

RabbitMQ is a message broker software that is used for the queuing of messages. There are three main actors in the RabbitMQ lifecycle. First, we have a ‘publisher’ or ‘producer’. A publisher is the one who creates a message and sends it. Second, we have an ‘exchange’. Exchange receives the message with a routing key from the producer. The exchange will then save the message and store it in a queue. Third, we have a consumer. A consumer is a party for which the message was intended. A consumer can either be a third party or the publisher itself who consumes the message after getting it from the queue of the broker.

For the above example, we have used a single queue, but in real-world applications, there would be multiple queues. An exchange is connected to a queue through a binding key. The exchange will use the routing key and binding key to confirm the consumer of a message. However, it is important to note that sometimes an exchange will link a routing key with the name of a queue instead of using a binding key. There are mainly four types of exchanges: direct, topic, headers and fanout.

Whenever a message goes to a consumer, RabbitMQ makes it certain that it is received in the correct order. The queues do not let a message get lost.

RabbitMQ comes with a protocol known as AMQP (Advanced Message Queuing Protocol). AMQP helps to define three major components.

  • Where should the message go?
  • How will it get delivered?
  • What goes in must also come out.

AMQP does not require a learning curve and can be easily programmed due to its flexibility. Thus, if a developer works with the HTTP and TCP requests and responses, they will easily adapt its protocol.

RabbitMQ supports development support for all the popular programming languages including Java, .NET, Python, PHP, JavaScript, etc.

Example

For a practical explanation, we will write a simple application in Java with RabbitMQ. The application will consist of a producer, which will send a message, as well a consumer, which will receive that message. For sending, we have a file named Send.java. You will require the following import.

import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.Channel;

 Now, setup the class.

public class Send { 
private final static String QUEUE_NAME = “hello”;
public static void main(String[] argv)      throws java.io.IOException {      …  }}

Now we will have to link our class with the server.

ConnectionFactory factory = new ConnectionFactory();
factory.setHost(“localhost”);
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();

 

This code helps in the abstraction of the socket connection. Now, the next step is the creation of a channel. For this purpose, you will have to define a queue.

channel.queueDeclare(QUEUE_NAME, false, false, false, null);
String message = “Hello World!”;
channel.basicPublish(“”, QUEUE_NAME, null, message.getBytes());
System.out.println(” [x] Sent ‘” + message + “‘”);

Lastly, we close the channel and the connection;

channel.close();
connection.close();

This ends the code for the sender.

Here is complete send java class.

import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.ConnectionFactory;

public class Send {

private final static String QUEUE_NAME = “hello”;
public static void main(String[] argv) throws Exception {         ConnectionFactory factory = new ConnectionFactory();    factory.setHost(“localhost”);
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();    channel.queueDeclare(QUEUE_NAME, false, false, false, null);    String message = “Hello World!”;
channel.basicPublish(“”, QUEUE_NAME, null, message.getBytes(“UTF-8”));
 System.out.println(” [x] Sent ‘” + message + “‘”);
 channel.close();
connection.close();

  }

}

Now you will have to write the code for the consumer. For this purpose, create a Recv.java class. Use the following import.

import com.rabbitmq.client.ConnectionFactory;
import com.rabbitmq.client.Connection;
import com.rabbitmq.client.Channel;
import com.rabbitmq.client.Consumer;
import com.rabbitmq.client.DefaultConsumer;

 

Now we will open a connection here too.

public class Recv { 

private final static String QUEUE_NAME = “hello”; 
public static void main(String[] argv)      throws java.io.IOException,             java.lang.InterruptedException {     ConnectionFactory factory = new ConnectionFactory();    factory.setHost(“localhost”); 
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();     channel.queueDeclare(QUEUE_NAME, false, false, false, null);    System.out.println(” [*] Waiting for messages. To exit press CTRL+C”);

    …    }

}

 

Now you will have to notify the server so it can fetch the messages that are accumulating in the queue.

Consumer consumer = new DefaultConsumer(channel) {  @Override  public void handleDelivery(String consumerTag, Envelope envelope,   AMQP.BasicProperties properties, byte[] body)      throws IOException {    String message = new String(body, “UTF-8”); 
  System.out.println(” [x] Received ‘” + message + “‘”); 
}};
channel.basicConsume(QUEUE_NAME, true, consumer);

Now run both the consumer and the producer, and you will have your RabbitMQ hello world application.

Complete Recv.java

import com.rabbitmq.client.*;
import java.io.IOException;

public class Recv {

private final static String QUEUE_NAME = “hello”;
public static void main(String[] argv) throws Exception {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost(“localhost”);
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueDeclare(QUEUE_NAME, false, false, false, null);
System.out.println(” [*] Waiting for messages. To exit press CTRL+C”);
Consumer consumer = new DefaultConsumer(channel) {
@Override
public void handleDelivery(String consumerTag, Envelope envelope, AMQP.BasicProperties properties, byte[] body)
throws IOException {
String message = new String(body, “UTF-8”);
System.out.println(” [x] Received ‘” + message + “‘”);
}
};
channel.basicConsume(QUEUE_NAME, true, consumer);
}
}

Final Thoughts

RabbitMQ has been adopted in thousands of deployment environments. It provides a significant boost in the scalability and loose coupling of applications. Today, it is by far the most popular message broker. Moreover, it also provides convenience with the cloud and also supports various message protocols, making it a desirable option for your development toolbox.

Serverless Computing: An Introduction to Amazon Lambda


AWS Lambda is a service provided by AWS that relates to the compute service. It assists in the running of application code where a physical server is not required. Lambda processes code per user requirements and raises the scalability bar when the need arises.

Whether your application deals with a few user requests or you need to handle thousands of requests, Lambda can be handy. Lambda utilizes Amazon’s highly powerful IT cloud infrastructure for running its compute services where the hardware and OS intricacies are delegated.

Considerations for Writing Lambda Functions

Irrespective of your choice of programming language, the understanding, and usage of the following components are important for the creation of a function.

Handler

Handler is used by Lambda for the execution of your function. Handler is adjusted after a function is created. Whenever a function has to be invoked, execution initiates with the help of the handler.

Context Object

A handler also receives a context object from the AWS Lambda, which can be marked as the second parameter. Context object provides communication between AWS Lambda and your written code.

Logging

Lambda functions keep a track of logging statements.

Exceptions

A result of your Lambda function’s execution is necessary to be conveyed to the AWS Lambda. This can be done through various strategies so a request’s lifecycle can come to an end. Likewise, the occurrence of an error can also be notified to the AWS Lambda. AWS Lambda passes the function execution result to the client if a function is invoked through synchronous means.

Writing a Simple AWS Lambda Function

Let’s see an example of our traditional “hello world” where we can run code of an AWS Lambda function without the use of a server— purely on the cloud!

1-     Lambda Console

Open the AWS Management Console. Check for the option of Lambda that appears under the Compute button. Click it so that the Lambda Console can be opened.

2-     Choosing a Blueprint

Now, you have to choose a blueprint. Blueprints have pre-existing code to speed up the processing. Blueprints can handle events from different sources. In the console, click the button of “Create a Function”. Now, click the “Blueprint” option. Find the filter box and enter the following details.

“hello-user-python”.

Choose the associated blueprint.

Now, press the Configure option.

3-Configuring the Function

Lambda functions store lines of code that are written by the users while they also manage dependencies and configuration. Configuration details can include the allocation of the resources for compute like memory, timeout of execution, etc. Lambda takes these details as input and does the required processing in return.

Now, you have to enter a description for your function. This description includes the following.

  • Name – Select a name for your function. For this article, we can use the “hello-user-python”.
  • Role – An execution role can be created which carries certain authorizations. Lambda uses the role for the invocation of a function. Choose the option of “Create a new role from template”.
  • Role Name – Select a role name for your function. For this example, we can use lambda_simple_procsessing.
  • Policy Templates – Before the generation of a function, a role is assigned after selecting an appropriate template.

A sample code is provided under the label of Lambda Function Code. Go to the lower part of your screen now and choose the option of “Create Function”. The coding for Lambda function is supported in all the popular programming languages like C#, Node.js, Java, Python. By default, Python is used for the runtime.

For managing the code, a handler method can be defined in the code. Lambda passes data related to events to the handler after which processing of the event is initiated. By scrolling down the options, any configuration for the execution time or memory can be configured, though we will not modify it in this example.

Invoking the Function and Checking the Results

The Lambda function of the hello-user-python appears on the console. This function can be tested where you can review the results and view the logs. Find a dropdown menu with the name of the “Select a test event” and click on the “Configure Test Event”.

You have a textbox for the testing of a function through an event. Go through the template list of sample events and select the “HelloWorld”. You can now assign any name to your event like “HelloUser”. The field values for the text in JSON can be modified. However, the event structure should not be changed. Modify the “value1” field with the “hello user”.  Now, finish by clicking on the “Create Button” and click the “Test Button”.

If everything goes alright, you can view your results from the console. These results are classified into the following:

  • Execution – Confirms if the execution was successful or not.
  • Summary – Presents the crucial details from the log input.
  • Log Output – Views all the logs that are produced due to the execution of your Lambda function.

Metrics

Amazon CloudWatch engages in the supervision and reporting of the Lambda function’s metrics. For effective management of the code for its execution, Lambda keeps a record of the following and publishes them:

  • Count of requests.
  • A request’s latency.
  • Requests concluding with an error.

Click on the “Test” button a few times so the metrics can be produced and displayed. Now, choose the option “Monitoring” for the displaying of results. As you scroll down, you can find various Lambda function metrics.

Since Lambda supports the “pay-as-you-go” model, users have to pay according to the request numbers of the Lambda functions. To be specific, pricing is based on two invocation factors: Duration and count.

Removing the Function

A Lambda function does not incur any charges. It can be deleted through the console. Go the “Actions” button and choose the “Delete Function”. A pop-up will appear now for confirmation; choose “Delete”.

Well, now you have successfully created, managed and deleted a simple AWS Lambda Function.