Saturday, November 10, 2018

Overview of Azure Storage


Azure storage provide cloud storage that is highly available, secure, durable, scalable and redundant. It includes,
  • Azure Blobs (objects)
  • Azue Data Lake Storage Gen 2,
  • Azure files
  • Azure queue and
  • Azue tables

Blobs - Scalable object storage in Azure

Blob storage is a massively scalable object storage for unstructured data such as images, videos, audio, documents etc. These objects can be stored in hot, cool or archived tiers depending on frequency of access. 


  • Hot storage: For frequent access
  • Cool storage: Infrequent access
  • Archive for rarely accessed data
  • Premium tier (new) : Low latency accessed data

Storage account

All access to data objects in Azure Storage happens through a storage account. It contains all your Blobs, files, queues, tables and disks. There are few options like General purpose V1, V2 and Blob storage. Azure recommends V2 for most scenarios. You can always upgrade from V1 without any downtime.

All data in storage account is encrypted on server side using 256-bit AES encryption. Encryption does not affect Azure Storage performance. By default access is allowed only to the account owner. You can control this with Azure AD, shared key authentication etc. 

You can define CORS urls in Azure for storage accounts. 

Container


A container organizes a set of blobs, similar to a folder in a file system. All blobs reside within a container. A storage account can include an unlimited number of containers, and a container can store an unlimited number of blobs.

Blob

  • Block blobs store text and binary data
  • Append blobs are made up of blocks like block blobs, but are optimized for append operations. Append blobs are ideal for scenarios such as logging data from virtual machines
  • Page blobs store random access files up to 8 TB in size. Page blobs store the VHD files that back VMs
There may be times where large data-sets and network constraints make uploading data to Blob storage over the wire unrealistic. You can use Azure Data Box Disk to request solid-state disks (SSDs) from Microsoft. You can then copy your data to those disks and ship them back to Microsoft to be uploaded into Blob storage. (Covered below)

To ensure your data is durable azure storage replicates multiple copies of data. You can select from following options for this,
  • Locally redundant storage (LRS)
  • Zone redundant 
  • Geo-redundant
  • Read access Geo redundant

Getting started is very easy, as mentioned in Microsoft docs tutorial, Clone the git repo. Set the connection string of your storage account and run. It will do,
  • Create Blob container under selected account
    • Get a reference to blob container 
    • Set access to public
  • Create a file in local environment
  • Get a reference to block Blob 
  • Upload created file to block Blob
  • List all block Blobs
  • Download uploaded file

Little bit more on Amazon S3, 
There are many many large companies which uses Amazon S3, one of them is Netflix (Why Netflix migrated to AWS). Netflix even have geographically redundancy of there servers.  There are other popular clients like Airbnb which uses Amazon S3 for storage.

Azure Files


Offers file shares in cloud accessible via standard Server Message Block protocol. Azure files can be used to,

  • Replace on-premise file severs
  • Lift and Shift (strategy for moving an application or operation from one environment to another – without redesigning the app) applications
More reads,

Azure Data Lake




Data lake is a storage repository, usually in Hadoop, that holds a vast amount of raw data in its native format until it is needed (Why use a data lake). It is a central repository allows you to store all your structured/unstructured data at any scale. You can run analytics from dashboards, visualize for big data processing and machine learning for better decisions. (Best Practices, Azure Data Lake Storage Gen1 vs Azure Blob Storage, Azure vs AWS)




Azure Queue Storage


Provides asynchronous cloud messaging between app components. Single queue message can be up to 64KB in size and a queue can contain millions of messages. 

Common usage include
  • Creating a backlog of work to process asynchronously
  • Passing messages from an Azure web role to an Azure worker role

Another alternative to queue storage is to use Service bus (Comparison) which is more enterprise. There is no direct alternative to Service bus in Amazon but Amazon got Simple Queue Service (SQS) for the same purpose. Google cloud got Cloud Pub/Sub.

Azure CosmosDB (Alternative to Table Storage)


Table storage is a NoSQL key-value store for development using datasets. It uses JSON to serialize data and can perform OData-based queries. But as an alternative Azure Cosmos DB can be used to achieved for same purpose. It offers throughput-optimized tables, global distribution and automatic secondary indexes. (Azure table storage vs CosmosDB Table API). Azure CosmosDB is a very secure database.

Alternative to CosmosDB in AWS is DynamoDB (See a comparison). 

Azure Data Box


Is a on-premise offline physical box which can be used to transfer TBs of data between your offline premise to azure cloud more easily when busy networks aren't an option.

Data box accommodates both offline and online (Creates a link between your site and azure) scenarios.

Case Study

Cloud Storage options in AWS

AWS also has many cloud storage options. It has many options which is highlighted in the following article. Cloud Storage options in AWS. 
https://aws.amazon.com/products/storage/

Cloud Storage AWS vs Azure

It's natural for us to compare what other options are there compared to Azure. While going through the options available in both AWS and Azure. Here's a comparison of both the platforms. 

Further reads,


Thursday, November 8, 2018

Monday, November 5, 2018

Working with Containers

What is a container?


What is a container?

Container is a unit of software that packages up code and all of its dependencies. Container images become containers at run time. Containerized software will always run the same regardless of the the infrastructure.


Docker.com

What is a container runtime?

A container runtime is software that executes containers and manages container images on a node. Docker engine is one such container runtime.

Docker.com

Containers and Virtual Machines. The Difference?

Docker.com

What is Container Orchestration and Kubernetes?

Managing few containers and a fewer applications is not a difficult task. But if you have 1000s of containers and services managing them is difficult. Container orchestration is about managing lifecycle of contaienrs in large environments.

Container orchestration achieves following

  • provision and deploy containers
  • scaling up or removing containers to spread application load evenly
  • Moving containers between hosts 
  • Resource allocation
  • Load balancing
  • Health monitoring

Kubernetes and Docker swarm are examples of container orchestration tools. 




https://markheath.net/post/containers-on-azure

Sunday, October 28, 2018

Understanding Angular Directives


Directives in angular can be used to implement life-cycle hooks. Directives are declared with @Directive attribute. In Angular there are 3 types of directives,
  1. Component 
  2. Structural
  3. Attribute
Component directives is a directive which comes with a template. Angular components are subset of directives. A component must belong to a NgModule in order for it to available to another component. 


Structural directives (link)
Used to deal with manipulating DOM elements. Examples of structural directives are *ngIf, *ngSwitch and *ngFor. 

NgIf directive
ngIf doesn't hide elements with CSS. It adds or removes them physically. Internally angular translates the *ngIf attribute to a <ng-template> element, wrapped around the host element. (What is a ng-template)

You can also create custom structural directives.

Attribute directives (link)
Changes the appearance or behavior of a DOM element.  You create a class, decorate it with @Directive decorator. After that you can pass in the ElementRef in constructor and change the behavior of the element.

You can make this more dynamic by adding HostListener and listening to DOM events.

Even more you can pass input parameters to the attribute directives. 

Monday, September 17, 2018

Observables and Subjects in Angular

Simply put ReactiveX (Reactive Extensions) is an API for asynchronous programming with observable streams. It mainly follows Observer pattern as well as the Iterator pattern with functional programming. ReactiveX has few implementations for main programming languages such as 

  • RxJava for Java
  • RxJS for JavaScript
  • Rx.NET for .NET
Here we'll focus on RxJS...

RxJS is a Reactive Extensions Library for JavaScript. It makes it easier to write asynchronous or callback based calls.

Observable: 
Invokable collection of future values or events. Subscribe to it to get the values. of returns an Observable. (check link for more details)



Observer is isolated from the logic. It just reacts to the details provided. Here you need to subscribe to it to get a value. You need to understand that Observables is not a angular specific feature. It's a standard for managing async data proposed to release with ES7.

Observables by default are “Cold” meaning they are lazy and won’t run any code until there is a subscriber.

Here there is no way to update the productIdList and run the subscription. You have to recreate the Observable. Still previous subscriptions won't get fired. 

Observer: Collection of callbacks that knows how to listen to values of Observables

Subscription: Execution of an Observable (see above example)

Operators: Pure functions that enables a functional programming style for dealing with observables. Above of is an operator which emit variable amount of values in a sequence and then emits a complete notification.

Subject: Same as EventEmitter. Way of multicasting a value or event to multiple Observables. Subject is another Observable type in rxjs. It can emit multiple event values.

Subject implements Observable. Unlike observables, anyone can listen and trigger events of Subject. 

Subject can emit events before subscriptions.


Outputs

An observable can be created from both Subject and BehaviorSubject using subject.asObservable().

Here you'll notice 1 is not printed because subscription started after value 1 is emitted. You can use ReplaySubject to emit previous values as well


BehaviorSubject is same as Subject but you can give it a default value. 



Notice that rack sensor values. Since we have not emitted any value initially. It'll print 1000 the default value. 

Scheduler: Centralized dispatchers to control concurrency. Using schedulers you can control when a notification from a rxjs method needs to be sent out. We don't use schedulers much in Angular

RxJS can be think of as Lodash for events.

 Check following articles for more information

 https://stackoverflow.com/questions/39494058/behaviorsubject-vs-observable
https://stackoverflow.com/search?q=%5Bangular%5D+observable

Tuesday, September 4, 2018

Angular 6 - Routing How to do it


While developing Angular apps, I thought of compiling a list of things we need to know on each of the areas. Here I cover routing.

Basics
  • A routed Angular application has one singleton instance of the Router service
  • The router uses a first-match wins strategy
    • The wildcard route comes last because it matches every URL and should be selected only if no other routes are matched first
    • Use path: '**' to match invalid routes (any other route that is not defined) for 404 redirections
  • Use enableTracing: true for debugging purposes by passing as 2nd parameter to forRoot
  • The router uses browsers history.pushState for navigation
Router Link
  • Use RouterLinkActive directive helps identify active links. The router adds active css class to the a tag
  • After the end of each successful navigation life cycle, the router builds a tree of ActivatedRoute objects that make up the current state of the router. You can access the current RouterState from anywhere in the application using the Router service and the routerState property.

  • The router adds the <router-outlet> element to the DOM and subsequently inserts the navigated view element immediately after the <router-outlet>.
  • You can add redirectRoute for redirections (defined in router module)
  • Only call RouterModule.forRoot in the root AppRoutingModule (or the AppModule if that's where you register top level application routes). In any other module, you must call the RouterModule.forChild method to register additional routes. (article)
  • Always put app routing as last in main module
Authentication
  • Use CanActivate guard to manage navigation business rules
  • You can create component-less routes to manage authentication easily
  • You can create a service which extends CanActivate which coul contain the security logic
    • Use canActivate attribute with that service
  • You can use CanActivateChild guard to protect child routes. It runs before any child route is activated

Sunday, August 19, 2018

Domain Driven Design - What is it and why

What is DDD?

In very simple terms DDD is about making your software as a model of a real world system or a process. When practicing DDD you'll work with a domain expert to build a ubiquitous language (UL). UL is a conceptual description of the system. UL is written down in such a way that domain expert should be able to understand and verify the UL is correct.

UL will include words that is used in actual domain. UL forms the basis of the OO design. DDD will have  following object categories
  • Value objects
  • Entities
  • Aggregate roots 

Recommended patterns for DDD
  • Repository 
  • Factory
  • Service

When not to use DDD (airbrake)

DDD is not suitable for projects where requirements and domain expertise is less. DDD sometimes goes in as an iteration which may not be suited for some projects. It may not be suited for projects which involves high technical complexity as opposed to domain complexity as well. 





Notes

https://stackoverflow.com/questions/1222392/can-someone-explain-domain-driven-design-ddd-in-plain-english-please/1222488#1222488

https://docs.microsoft.com/en-us/dotnet/standard/microservices-architecture/microservice-ddd-cqrs-patterns/ddd-oriented-microservice

https://docs.microsoft.com/en-us/dotnet/standard/microservices-architecture/microservice-ddd-cqrs-patterns/net-core-microservice-domain-model



Sunday, August 12, 2018

All you need to know about Angular

Angular Main Concepts

Directives

Directive is a function which gets executed whenever Angular compiler find one in the DOM. There are component, structural and attribute directives.

Modules

NgModules consolidate components, directives, and pipes into blocks of functionality.

Components and Life cycle hooks

Dynamic components, Component interaction, Attribute and Structural directives

Observable and Subject


Routing


Services


Forms

Template driven forms, Reactive forms, Form validation and Dynamic forms


Angular Advanced Topics


Server side rendering

Service worker and PWA

Security

Compiling


Wednesday, July 4, 2018

Integrating Repository Layer with Service Layer

Nowadays it is very common to use repository pattern to improve readability and maintainability of projects. But things gets complex when you have multiple repositories and services interacting with each other. 

When you have many of these it is important to find ways to generalize things so original requirement of readability and maintainability is achieved. 


Resources

Tuesday, July 3, 2018

Organizing Resources in Azure with Resource Manager

When working with large scale projects proper organization of resources is a necessary. Azure has resource groups for grouping resources as required.

Resource Manager in Azure

You can use resource manager to organize resources within a subscription. Use it to control access, audit and tagging resources. Actions you take in azure are handles through resource managers. 


Following is how azure structure can be seen from scope prospective 


Template deployment in Azure Resource Manager

In large scale deployments you can use azure templates for automate tasks. Templates are simply JSON files containing metadata about the resources.


Integrating Azure Key Vault in Azure Resource Manager template deployment

You can pre-store the password in an Azure key vault and then customize the template to retrieve the password from the key vault during the deployment. Check this tutorial.

Resource Access Management in Azure


Friday, June 15, 2018

Internet of Things

IoT, Internet of Things is a system of interrelated computing devices which communicates over a network and can be remotely monitored.

Image Credits: Embitel India 
Some of the key goals of IoT is to,

  • Improving business processes
  • Improving customer experience
  • Reducing costs
  • Increasing competitiveness
Some of the challenges in IoT are,
  • Data/network security
  • Integrating with business processes
  • Expected ongoing costs
  • Upfront capital investment requirements
  • Integrating with existing ICT infrastructure

Edge Computing

Edge computing is the practice of processing data near the edge of your network, where the data is being generated, instead of in a centralised data-processing warehouse. (Microsoft Azure enables a new wave of edge computing. Here’s how.)



Tuesday, June 12, 2018

Angular Components



In general a component controls a part of the screen or a view. Angular creates, updates, and destroys components as the user moves through the application. Your app can take actions on component using life cycle hooks suck as ngOnInit().

@Component decorator identifies a class as a component. You can keep metadata here. A component usually accommodated by a view.

Component Interaction

  • Using @Input parameters
  • Event listeners with EventEmitter
  • Parents call a @ViewChild()

Friday, March 30, 2018

Artificial Intelligence and Beyond

Artificial Intelligence is not a buzzword nowadays. Instead what we hear mostly today is concepts like machine learning, deep learning, natural language processing (NLP) etc. But one should understand that. This all comes under Artificial Intelligence.

Image Credits: Quora

Artificial Intelligence 

Is the intelligence performed by the machines instead of by humans or other animals by natural intelligence. AI would perceive information it gets, process it gives a better meaning than it previously had. 

Machine Learning

Is how a machine learns to perform a task and gain more information (learn) by looking at the patterns it gets. 

Image Credits: Medium
Deep Learning: is a concept where machine learning creates artificial neural network that can take intelligent decisions on its own. 



Natural Language Processing (NLP)

Is how a program process and analyzes large amount of data.

IBM Watson is a question-answering system capable of answering questions in natural language. 

Sentence segmentation is a technique used to identify similar sentences. 


Tuesday, March 13, 2018

Serverless Computing - Azure Functions

In Serverless computing cloud provider provides backend as a service. The architecture is inherently scalable, can be deployed quickly. But comes with a cost such as testing and debugging. 

Azure Functions is the implementation in Azure. Azure functions can be integrated with Cosmos DB, Service bus and Storage.

Serverless APIs with Azure Function Proxies

Azure functions can be used to host an API. Not very often you host APIs in Functions. But this s a good way for scalability, costs or ease of implementation.

Cloud Messaging in Azure

You can enhance the behavior in Azure by using following

Event Grid

Fully managed event routing services by connecting serverless logic to events.



Service Bus

Messaging infrastructure in Azure.


Logic Apps

Helps you to provide workflows allowing developers to integrate data with their apps.

Analytics

Azure Stream Analytics allows you to query data in SQL like language

Serverless computing in AWS




Resources

https://azure.microsoft.com/en-us/services/functions/
Subscribe to: Posts (Atom)
Powered by Blogger.


Software Architect at Surge Global/ Certified Scrum Master

Experienced in Product Design, Software Engineering, Team management and Practicing Agile methodologies.

Search This Blog

Facebook