Data Centric Architecture

Data Centric architecture

The value proposition of global systems integrators (GSIs) has changed remarkably in the last 10 years. By 2010, it was the waning days of the so-called “your mess for less” (YMFL) business model. GSIs would essentially purchase and run a company’s IT shop and deliver value through right-shoring (moving labor to low cost places), leveraging supply chain economies of scale and, to a lesser degree, automation.

This model had been delivering value to the industry since the ‘90s but was nearing its asymptotic conclusion. To continue achieving the cost savings and value improvements that customers were demanding, GSIs had to add to their repertoire. They had to define, understand, engage and deliver in the digital transformation business. Today, I am focusing on the value GSIs offer by concentrating on their client’s data, rather than being fixated on the boxes or cloud where data resides.

In the YMFL business, the GSIs could zero in on the cheapest, performance compliant disk or cloud to house sets of applications, logs, analytics and backup data. The data sets were created and used by and for their corresponding purpose. Often, they were tenuously managed by sophisticated middleware and applications for other purposes, like decision support or analytics.

Getting a centralized view of the customer was difficult, if not impossible. First, it was due to the stove piping of the relevant data in an application-centric architecture. In tandem, data islands were created for analytics repositories.

Now enters the “Data Centric Architecture.” Transformation to a data-centric view is a new opportunity for GSIs to remain relevant and add value to customer’s infrastructures. It is a layer deeper than moving to cloud or migrating to the latest, faster, smaller boxes.

A great way to help jump start this transformation is by rolling out Data as a Service offerings. Rather than taking the more traditional Storage as a Service or Backup as a Service approach, Data as a Service anticipates and provides the underlying architecture to support a data-centric strategy.

It is first and foremost a repository for collected and aggregated data that is independent of application sources. From this repository, you can draw correlations, statistics, visualizations and advanced analytical insights that are impossible when dealing with islands of data managed independently.

It is more than the repository of the algorithmically derived data lake. A Data as a Service approach provides cost effective accessibility, performance, security and resilience – aimed at addressing the largest source of both complexity and cost in the landscape.

Data as a Service helps achieve these goals by minimizing, simplifying and reducing the data and its movement within and outside of the enterprise and cloud environments. This is achieved around four primary use cases, which range from enterprise storage to backup and long-term retention:

 

 

Each of the cases illustrates the underlying capabilities necessary to cost effectively support the move to a data-centric architecture. Combined with a “never migrate or refresh again” evergreen approach, GSIs can focus on maximizing value in the stack of offerings. This approach is revolutionary.  In past, there was merely a focus on the refresh of aging boxes, or the specifications of a particular cloud service, or the infrastructure supporting a particular application. Today, GSIs can focus on the treasured asset in their customer’s IT — their data

Microservices vs Monolithic Architecture

Microservices vs. Monolith Architecture - DEV Community

Microservices architecture has become a hot topic in the software backend development world. The ecosystem carries a profound impact on not just the enterprises’ IT function but also in the digital transformation of an entire app business.

The debate of Microservices vs monolithic architecture defines a revolutionary shift in how an IT  team approaches their software development cycle: Whether they go with the approach that brands like Google, Amazon, and Netflix chose or do they go with the simplicity quotient that a startup which is at the development stage demands.

In this article, we are going to get startups an answer to which backend architecture they should choose when they are starting their journey to become a startup.

Table Of Content:

  1. What are Microservices Architecture?
  2. What is Monolithic Architecture?
  3. Microservices vs Monolithic Architecture: Advantages and Disadvantages
  4. How to Choose Between Monolithic and Microservice Architecture?
  5. Migrating from a Monolithic Architecture to a Microservice Ecosystem
  6. Conclusion

What are Microservices Architecture?

Microservice Architecture pattern

Microservices architecture contains a mix of small and autonomous services where every service is self-contained and must be implemented as a single business ability. It is a distinct approach used for development of software systems which focus on developing several single-function modules with clearly-defined operations and interfaces. The approach has become a popular trend in the past several years as more and more Enterprises are looking to become Agile and make a shift towards DevOps.

Components of Microservices architecture that makes it one of the best enterprise architecture:

  • The services are independent, small, and loosely coupled
  • Encapsulates a business or customer scenario
  • Every service is different codebase
  • Services can be independently deployed
  • Services interact with each other using APIs

With the question of what are microservices architecture now answered, let us move on to look into what is monolithic architecture.

What is Monolithic Architecture? 

Introduction to Monolithic Architecture and MicroServices Architecture | by Siraj ul Haq | KoderLabs | Medium

Monolithic application has a single codebase having multiple modules. The modules, in turn, are divided into either technical features or business features. The architecture comes with a single build system that helps build complete application. It also comes with a single deployable or executable binary.

Now that we have looked into what is monolithic architecture and microservices architecture, let us look into the disadvantages and benefits that both the backend system offers to get an understanding of what separates them from each other.

Microservices vs Monolithic Architecture: Advantages and Disadvantages

Advantages of Monolithic Architecture

A. Zero Deployment Dependencies

Dependencies Handling in Python

An organized and well-documented Monolith architecture makes it possible for Backend developers to not worry about which version would be compatible with which service, how to find which services are present and what they do, etc.

B. Error Tracing

Poor man's performance and error tracing (ft. HAProxy) | Sysbee

One of the biggest benefits of monolithic is that all the transactions are logged into one place, making error tracing task a breeze.

C. No Silos

no-silos – TeamStrength, Inc.

The one factor that works in the favour of monolithic in the microservices vs monolithic architecture debate is absence of silos. It becomes very easy for the developers to work on multiple parts of the app for they are all structured similarly, using the same tools, which makes it okay to have no prior distributed computing knowledge.

D. Cross-cutting concerns:

Cross cutting concern example - Stack Overflow

Spending time in defining the services which do not bleed in each other’s time is the time that you can actually spend in developing things that help the customers.

E. Shared Code:

The Problem with Shared Code. This is the first of a 4-part series on… | by Jeff Whelpley | Medium

No shared libraries where the complete scope needed for services to operate is sent along each request.

Limitations of Monolithic Architecture

A. Lack of Flexibility:

Lack of flexibility in the workplace kills productivity, says report - Employee Benefits

Monolithic architectures are not flexible. You cannot use different technologies when you have incorporated Monolithic. The technology stack which have been decided at the beginning have to be followed throughout the project, making upgrades a next to impossible task.

B. Development Speed:

How to Decrease Time for Software Development Projects - BIT Studios

Microservices speed development process is famous when you compare microservices architecture vs monolithic architecture. Development is very slow in monolithic architecture. It can be very difficult for team members to understand and then modify the code of large monolithic applications. Additionally, as the size of codebase increases, the IDE gets overloaded and gets slower. All of this results in a slowed down app development speed.

C. Difficult Scalability:

Scalability Complex Like A Puzzle - Pictured As Word Scalability On A Puzzle Pieces To Show That Scalability Can Be Difficult And Stock Illustration - Illustration of idea, analogy: 164219546

Scaling monolithic applications becomes difficult when the apps becomes large. While developers can develop new instances of monolith and load balancer to distribute the traffic to new instances, monolithic architecture cannot scale with the increasing load.

Benefits of Microservices Architecture

All You Need to Know about Microservices Architecture in 2019

  1. The biggest factor in favour of microservices in the difference between microservices and monolithic architecture is that it handles complexity issues by decomposing the app into manageable service set that are faster to develop and easier to maintain and understand.
  2. It enables independent service development through a team which is focused on the particular service, which makes the ideal choice of businesses that work with an Agile development approach.
  3. It lowers the barrier of adopting newer technologies as the developers have the freedom to choose whatever technology that makes sense to their project.
  4. It makes it possible for every microservice to be deployed individually. The result of which is that continuous deployment of complex application becomes possible.

Drawbacks of Microservices Architecture

Microservice Architecture: 5 Major Benefits - Skelia

  1. Microservices add a complexity to project simply by the fact that the microservices application is distributed system. To solve the complexities, developers have to select and implement inter-process communication that is based on either RPC or messaging.
  2. They work with partitioned database architecture. The business transactions which update multiple business entities inside the microservices application also have to update different databases that are owned by multiple services.
  3. It is a lot more difficult to implement changes which span across multiple services. While in case of Monolithic architecture, an app development agency only have to change the corresponding modules, integrate all the changes, and then deploy them all in one go.
  4. Deployment of a microservice application is very complex. It consists of a number of services, which individually have multiple runtime instances. In contrast, a monolithic application is deployed on set of identical servers behind load balancer.

The benefits and limitations are prevalent in both monolithic and microservices architecture. This makes it extremely difficult for a startup to gauge which backend architecture to incorporate in their journey.

Let us help you.

How to Choose Between Monolithic and Microservice Architecture? 

The fact that both the approaches come with their own set of pros and cons are a sign that there is no one size fits all methodology when it comes to choosing a backend architecture. But there are a few questions that can help you decide which is the right direction to head into.

Are You Working in a Familiar Sector?

Akash - Aligarh,Uttar Pradesh : Take a step to be familiar with computer and start your career in IT sector.

When you work in an industry where you know the veins of the sector and you know the demands and the needs of the customers, it becomes easier to enter into the system with a definite structure. The same, however, is not possible with a business that is very new in the industry, for the amount of looming doubts are much greater.

So, the use of microservice architecture in app development is best suited in cases where you know the industry inside out. If that is not the case, go with monolithic approach to develop your app.

How Prepared is Your Team?

Who is on the Team?

Is your team aware with the best practices for implementing microservices? Or are they more comfortable with working around the simplicity of monolithic? Will your team and your business offering expand in the coming time? You will have to find answers to all these questions to gauge whether the people who have to work on a project are even ready to migrate.

What is Your Infrastructure Like?

The Big Three: Comparing AWS, Azure and Google Cloud for Computing - The Sunshower.io Blog

Everything from the development to the deployment of a monolithic web application would require a cloud-based infrastructure. You will have to make use of Amazon AWS and Google Cloud for deploying even tiny elements. While the cloud technologies make the process easier, The idea of setting up database server for every other microservice and then scaling out is something that startup entrepreneur might not be comfortable with.

Have you Evaluated the Business Risk?

The art of predicting business risks: Why non-experts do it better | Fortune

More often than not, businesses take microservices’ side in the Microservices vs Monolithic Architecture thinking it is the right thing for their business. What they forget to factor in is the chance that their application might not become as scalable as they are optimistically expecting and they might have to suffer the risks of adding a highly scalable system in their process.

Here is a short list of pointers that would help you make the decision of choosing to opt for software development processes with microservices vs monolithic architecture:

When to Choose Monolithic Architecture?

  • When your team is at a founding stage
  • When you are developing a proof of concept
  • When you have no experience in microservices
  • When you have experience in the development on solid frameworks, like the Ruby on Rails, Laravel, etc.

When to Choose Microservices Architecture?

  • You need independent, quick delivery service
  • You need to extend your team
  • Your platform need to be extremely efficient
  • You don’t have a tight deadline to work with

Migrating from a Monolithic Architecture to a Microservice Ecosystem

Migrate From Monolithic Applications to a Microservice Architecture: Part 2

The right approach for migrating a monolithic architecture to a microservice ecosystem is to divide the monolith processes and turn them into microservices. The result of this is a two-factor plan:

  1. Identification of existing monolithic elements which can get decoupled
  2. A validation that the new functionality can be developed as microservice

One of the main challenges that can emerge when initiating the migration from a monolithic architecture to a microservice architecture is to design and create an integration between existing system and a new microservice. A solution for this can be to add a glue code which allows them to connect later, something like an API.

API gateway can also help in combining multiple individual service calls in one coarse-grained service, and this in turn would help reduce the integration cost with monolithic system.

Conclusion

When you compare microservices architecture vs monolithic architecture, you will find the former being a hot trend. Every entrepreneur wants to say that their app is based on this architecture. But the temptation to focus only on the problems of monolithic architecture and abandon the architecture should be measured against the actual value of microservice architecture.

The right approach would be to develop new apps using a monolithic approach and move to microservices only when the justification of the move is backed by proper metrics like performance monitoring.

For established businesses, microservices tend to be avenues for continuous deployment, team based development, and an agility to shift to new technologies. But for startups, or companies that are just starting, adopting microservices can impact the software project success very negatively.

Hadoop based Data Lakes: The Beast is Born

What happened to Hadoop. Hadoop was the next big thing in… | by Derrick Harris | ARCHITECHT

1997 was the year of consumable digital revolution – the year when cost of computation and storage decreased drastically resulting in conversion from paper-based to digital storage. The very next year the problem of Big Data emerged. As the digitalization of documents far surpassed the estimates, Hadoop was the step forward towards low cost storage. It slowly became synonymous and inter-changeable with the term big data. With explosion of ecommerce, social chatter and connected things, data has exploded into new realms. It’s not just the volume anymore.

What is Big Data? Let's answer this question! | by Ilija Mihajlovic | Towards Data Science

In part 1 of this blog, I had set the premise that the market is already moving from a PPTware to dashboard and robust machine learning platforms to make the most of the “new oil”.

Today, we are constantly inundated with terms like Data Lake and Data Reservoirs. What do these really mean? Why should we care about these buzz words? How does it improve our daily lives?

I have spoken with a number of people – over the years – and have come to realize that for most part they are enamoured with the term, not realizing the value or the complexity behind it. Even when they do realize, the variety of software components and the velocity with which they change are simply incomprehensible.

The big question here would be, how do we quantify Big Data? One aspect to pivot is that it is no longer about the volume of data you collect, rather the insight through analysis that is important. Data when used for the purpose beyond its original intent can generate latent value. Making the most of this latent value will require practitioners to envision the 4V’s in tandem – Volume, Variety Velocity, and Veracity.

Translating this into reality will require a system that is:

  • Low cost
  • Capable of handling the volume load
  • Not constrained by the variety (structured, unstructured or semi-structured formats)
  • Capable of handling the velocity (streaming) and
  • Endowed with tools to perform the required data discovery, through light or dark data (veracity)

Hadoop — now a household term — had its beginnings aimed towards web search. Rather than making it proprietary, the developers at Yahoo made a life-altering decision to release this as open-source; deriving their requisite inspiration from another open source project called Nutch, which had a component with the same name.

Over the last decade, Hadoop with Apache Software Foundation as its surrogate mother and with active collaboration between thousands of open-source contributors, has evolved into the beast that it is.

Hadoop is endowed with the following components –

  • HDFS (Highly Distributed File System) — which provides centralized storage spread over number of different physical systems and ensures enough redundancy of data for high availability.
  • MapReduce — The process of distributed computing on available data using Mappers and Reducers. Mappers work on data and reduce it to tuples and can include transformation while reducers take data from different mappers and combines them.
  • YARN / MESOS – The resource managers that control availability of hardware and software processes along with scheduling and job management with two distinct components – Namely ResourceManager and NodeManager.
  • Commons – Common set of libraries and utilities that support other Hadoop components.

While the above forms the foundation, what really drives data processing and analysis are frameworks such as Pig, Hive and Spark for data processing along with other widely used utilities for cluster, meta-data and security management. Now that you know what the beast is made of (at its core) – we will cover the dressings in the next parts of this series. Au Revoir!

Android Architecture Components: Exploring

At Google I/O 2017, Google introduced new architecture components for Android. It is a new library which will help developers to maintain their activities or fragments lifecycle very easily.

This new library by Google provides some relief to the Android Developers by providing a complete solution to problems like memory leaks, data persistence during configuration changes and also helps in reducing some boilerplate code.

Exploring the new Android Architecture Components (Part 1) | Humble Bits

There are 4 Android Architecture Components :

  1. Lifecycle
  2. LiveData
  3. ViewModel
  4. Room

LiveData Clean Code using MVVM and Android Architecture Components | by Rohit Singh | AndroidPub | Medium

Lifecycle

Lifecycle class helps us in building lifecycle aware components. It is a class which holds all the information about the states of an activity or a fragment. It allows other objects to observe lifecycle states like a resume, pause etc.

There are 2 main methods in the Lifecycle class:

  1. addObserver() – Using this method, we can add a new instance of a “LifecyleObserver” class which will be notified whenever our “LifecycleOwner” changes state. For example: if an activity or a fragment is in RESUMED state then our “LifecycleObserver” will receive the ON_RESUME event.
  2. removeObserver() – This method is used for removing active “LifecycleObserver” from the observer’s list.

LifecycleObserver

With the help of this interface, we can create our Observer class which will observe the states of an activity or a fragment. We need to simply implement this “LifecycleObserver” interface.

LifecycleOwner

It is an interface with a single method called “getLifecycle()”. This method must be implemented by all the classes which are implementing this interface. This interface denotes that this component (an activity or a fragment) has a lifecycle.

Any class whose states we want to listen must implement the “LifecycleRegistryOwner” interface. And any class who wants to listen must implement the “LifecycleObserver” interface.

There are several events which we can listen with the help of the “LifecycleObserver” interface:

  • ON_CREATE – Will be called after onCreate() method of the “LifecycleOwner”.
  • ON_START – Will be called after onStart() method of the “LifecycleOwner”.
  • ON_RESUME -Will be called after onResume() method of the “LifecycleOwner”.
  • ON_PAUSE – Will be triggered upon the “LifecycleOwner” being paused (before onPause() method).
  • ON_STOP – Will be triggered upon the “LifecycleOwner” being stopped (before onStop() method).
  • ON_DESTROY – Will be triggered by the “LifecycleOwner” being destroyed (before onDestroy() method).

All these events are enough for managing the lifecycle of our views. With the help of these  “LifecycleObserver” and “LifecycleOwner” classes, there is no need to write methods like onResume(), onPause() etc in our activities or fragments. We can handle these methods in our observer class.

We can also get the current state of the “Lifecycle Owner” with the help of getCurrentState() method.

LiveData

LiveData is a data holder class. It provides us the ability to hold a value and we can observe this value across lifecycle changes.

Yeah, you heard that right!

LiveData handles the lifecycle of app components automatically. We only need to observe this LiveData class in our components. That’s it and we are done.

There are some important methods of LiveData:-

  • onActive() – Called when there is an active observer. An observer is called active observer when its lifecycle state is either STARTED or RESUMED and the number of active observers changes from 0 to 1.
  • onInactive() – Called when there are no active observers. An observer is called inactive observer when its lifecycle state is neither STARTED nor RESUMED (like an activity in the back stack) and the number of active observers changes from 1 to 0.
  • setValue() – Used to dispatch the results to the active observers. This method must be called from the main thread.

In above method observe() we pass the Lifecycle Owner as an argument, this argument denotes that LocationHelperLiveData class should be bound to the Lifecycle of the MyActivity class. This bounding relationship of helper class with our Activity denotes that:-

  • Even if the value changes, the observer will not be called if the Lifecycle is not in an active state (STARTED or RESUMED).
  • The observer will be removed automatically if the Lifecycle is destroyed.

Whenever MyActivity is in either PAUSE or STOP state it will not receive location data. It will start receiving Location Data again once MyActivity is in either RESUME or START state.

There can be multiple Activities or Fragments which can observe LocationHelperLiveData instance and our LiveData class will manage their Lifecycle automatically.

There is no need to remove observer in onDestroy() method, it will be removed automatically once LifecycleOwner is destroyed.

error: Content is protected !!