Category Archives: Cost Control

1. Cloud Defects:How the Cloud Professionals can Create the costly defects and the reasons ?

With reference to the booming Cloud Market [refer to some of my videos/blogs] and the Conversions to Cloud by different business organizations; Globally millions of IT professionals were mandated to learn the Cloud technologies to sustain their jobs in IT.

When we talk about these converted IT Professionals, they were forced to move into Cloud due to the sustainability of their job.

At the same time their employers are also enforced by their management to accelerate the cost savings for IT infra.

So how they do the planning for Cloud ?

On top there can be a CTO/CIO plans it to convert into Cloud. And they have the complete knowledge of the infrastructure with reference to their technology being used for business systems.

There will be Cloud Architects, they are responsible to create the Cloud implementation design.

Then the plan and the design is released to the Cloud engineers by allocating the tasks.

In between due to lack of [turn around] time, the Cloud engineers are pressured to create the environments. Here everybody can find one point, the management did not plan the knowledge transfer perfectly to the low level people. Then they can create many costly defects as per their knowledge and experience. So there is no ‘uniformity was followed’ in this case. It need not be a cost effective solution the Cloud Architects or Cloud Engineers have done. And there was no review also done. Once the monthly Cloud bill is arrived they will realize on it. Then they can realize the costly burning/bill happened by comparing to their planned cost saving on Infra. So this way they are enforced to track the resources who created these costly defects.

The following picture has got this kind of analysis. And what are the basic reasons the Architects and Engineers are not able to understand the domain conversion scenarios, when the management plans and release it for implementation.

Here we need to remember, every IT Professional need to realize that one fine day the current size of the resources will not be there in any project due to Cloud and the automation implementation is happening.

So the self matured professional need to realize where they are lacking and step down to learn those from experts, on war footing basis to prove and safeguard their jobs in IT.

If you want to learn the details, I have discussed in the below video with the relevant coaching on AWS-SAA course also.

https://business.facebook.com/vskumarcloud/videos/642868796242922/

Also go through the below blog to know the Cloud and DevOps Roles:

https://vskumar.blog/2018/12/30/how-a-cloud-architect-is-different-from-devops-role/

To know the real practices lacking and the root causes towards creating these costly defects, visit the below video based blog. It can show the ways to improve your Cloud projects ROI.

https://vskumar.blog/2019/10/30/4-cloud-defectswhat-could-be-the-reasons-for-costly-defects/

Also visit my video on How to Initiate Cloud Conversion with some best practices:
https://www.youtube.com/watch?v=98zboBaXgoU&t=379s

1. Cloud architect: How to build your Infrastructure planning practice ?

If you are a Cloud Architect, you might do project initiation for Cloud migration projects. During that time you need to have a plan to get series of activities and to make a project schedule. You might need to see this discussion Video also along with your planning. It will add value for your future efforts savings or can reduce repeat activities. Please send your feedback by e-mail [mentioned in it], which can encourage us to make such Consulting/discussion videos sharing on Social.

Build Cloud architects-FB promotion

With reference to my previous blog on the role of Cloud architect, in this blog I would like to present on:

  • What is Traditional Infrastructure planning and building analysis ?

  • How to setup a new Infrastructure for an E-commerce [simple site] in Traditional manner ?

  • What are the Activities we might do ?

  • How to compare them in high level with a Cloud Architecting ?

  • If the Cloud architect apply these practices in his/her area, lot of time for roll back/back out tasks can be reduced during migration.

The following One hour Video has the entire elaboration  for your clarity with a Consulting/Training discussion:

You can also join for similar discussions:

https://www.facebook.com/groups/1911594275816833/about/

If you are looking for coaching on your role Cloud performance, please contact me on my FB with your Linkedin URL.

For details on my coaching visit:

https://vskumar.blog/2018/11/13/coaching-mentoring-on-aws-solution-architect-associate-exam/

 

If you are interested to know the Cloud initiation activities, visit my video:

 

 

11. DevOps: How to Launch a container as a daemon ?

Docker-logo

In continuation of my previous blog on “10. DevOps: How to Build images from Docker containers?”, I am continuing my lab exercises. In this session we can see ”

How to Launch a container as a daemon ?:

Note: If you want to recollect the docker commands to be used during your current lab practice, visit my blog link:

https://vskumarblogs.wordpress.com/2017/12/13/some-useful-docker-commands-for-handling-images-and-containers/

 

Let us recap the past exercises; So far we have experimented with an interactive container, tracked the changes that were made to the containers., created images from the containers, and then gained insights in the containerization scenarios.

Now, let us see the container usage in a detached mode.

When we run the container in a detached mode it runs under a daemon process.

I want to use the “ubuntu” image and run detached mode command.

First, let me check my current docker images:

==================>

vskumar@ubuntu:~$

vskumar@ubuntu:~$ sudo docker images

[sudo] password for vskumar:

REPOSITORY TAG IMAGE ID CREATED SIZE

docker-exercise/ubuntu-wgetinstall latest e34304119838 7 days ago 169MB

<none> <none> fc7e4564eb92 7 days ago 169MB

hello-world latest f2a91732366c 12 days ago 1.85kB

ubuntu 16.04 20c44cd7596f 2 weeks ago 123MB

ubuntu latest 20c44cd7596f 2 weeks ago 123MB

busybox latest 6ad733544a63 4 weeks ago 1.13MB

busybox 1.24 47bcc53f74dc 20 months ago 1.11MB

vskumar@ubuntu:~$

===================>

You can see my previous image with ‘docker-exercise/ubuntu-wgetinstall ‘. This was created in the previous exercise.

As per our plan in this session I am using the below commands to run the ubuntu image as below:

sudo docker run -d ubuntu \

    /bin/bash -c "while true; do date; sleep 5; done";

========== Output ======>
vskumar@ubuntu:~$  sudo docker run -d ubuntu \
>     /bin/bash -c "while true; do date; sleep 5; done";
0fe495fc93edee3aaadc7fc0fbf21997f0ca3cde4d7e563aa8c61352a43957dd
vskumar@ubuntu:~$ $ 
=======================>

Now, to view the docker logs I want to run the docker logs subcommand on image id: ‘ 0fe495fc93edee3aaadc7fc0fbf21997f0ca3cde4d7e563aa8c61352a43957dd’

$ sudo docker logs 0fe495fc93edee3aaadc7fc0fbf21997f0ca3cde4d7e563aa8c61352a43957dd;

=====See the output of the Daemon process running with the ubuntu image ===============>

vskumar@ubuntu:~$ sudo docker logs 0fe495fc93edee3aaadc7fc0fbf21997f0ca3cde4d7e563aa8c61352a43957dd;

Sun Dec 3 05:11:57 UTC 2017

Sun Dec 3 05:12:02 UTC 2017

Sun Dec 3 05:12:07 UTC 2017

Sun Dec 3 05:12:12 UTC 2017

Sun Dec 3 05:12:17 UTC 2017

Sun Dec 3 05:12:22 UTC 2017

Sun Dec 3 05:12:27 UTC 2017

Sun Dec 3 05:12:32 UTC 2017

Sun Dec 3 05:12:37 UTC 2017

Sun Dec 3 05:12:42 UTC 2017

Sun Dec 3 05:12:48 UTC 2017

Sun Dec 3 05:12:53 UTC 2017

Sun Dec 3 05:12:58 UTC 2017

Sun Dec 3 05:13:03 UTC 2017

Sun Dec 3 05:13:08 UTC 2017

Sun Dec 3 05:13:13 UTC 2017

Sun Dec 3 05:13:18 UTC 2017

Sun Dec 3 05:13:23 UTC 2017

Sun Dec 3 05:13:28 UTC 2017

Sun Dec 3 05:13:33 UTC 2017

Sun Dec 3 05:13:38 UTC 2017

Sun Dec 3 05:13:43 UTC 2017

Sun Dec 3 05:13:48 UTC 2017

Sun Dec 3 05:13:53 UTC 2017

Sun Dec 3 05:13:58 UTC 2017

Sun Dec 3 05:14:03 UTC 2017

Sun Dec 3 05:14:08 UTC 2017

Sun Dec 3 05:14:13 UTC 2017

Sun Dec 3 05:14:18 UTC 2017

Sun Dec 3 05:14:23 UTC 2017

Sun Dec 3 05:14:28 UTC 2017

Sun Dec 3 05:14:33 UTC 2017

Sun Dec 3 05:14:38 UTC 2017

Sun Dec 3 05:14:43 UTC 2017

Sun Dec 3 05:14:48 UTC 2017

Sun Dec 3 05:14:53 UTC 2017

Sun Dec 3 05:14:58 UTC 2017

Sun Dec 3 05:15:03 UTC 2017

Sun Dec 3 05:15:08 UTC 2017

Sun Dec 3 05:15:13 UTC 2017

Sun Dec 3 05:15:18 UTC 2017

Sun Dec 3 05:15:23 UTC 2017

vskumar@ubuntu:~$

=================You can see the output for every few seconds listed =======>

It means the container is running as a daemon.

Now, let us use ps -eaf command to check the processed running in linux by using :

$ ps -eaf | grep ‘daemon’

========= See the output of daemon processes ==========>

vskumar@ubuntu:~$

vskumar@ubuntu:~$ ps -eaf | grep ‘daemon’

message+ 837 1 0 20:26 ? 00:00:05 /usr/bin/dbus-daemon –system –address=systemd: –nofork –nopidfile –systemd-activation

root 871 1 0 20:26 ? 00:00:03 /usr/sbin/NetworkManager –no-daemon

avahi 873 1 0 20:26 ? 00:00:00 avahi-daemon: running [ubuntu.local]

root 876 1 0 20:26 ? 00:00:01 /usr/lib/accountsservice/accounts-daemon

avahi 893 873 0 20:26 ? 00:00:00 avahi-daemon: chroot helper

rtkit 1370 1 0 20:28 ? 00:00:00 /usr/lib/rtkit/rtkit-daemon

vskumar 2426 1 0 20:55 ? 00:00:00 /usr/bin/gnome-keyring-daemon –daemonize –login

vskumar 2508 2428 0 20:55 ? 00:00:00 upstart-udev-bridge –daemon –user

vskumar 2515 2428 0 20:55 ? 00:00:04 dbus-daemon –fork –session –address=unix:abstract=/tmp/dbus-nPaV5rWlQc

vskumar 2570 2428 0 20:55 ? 00:00:03 /usr/lib/x86_64-linux-gnu/bamf/bamfdaemon

vskumar 2572 2428 0 20:55 ? 00:00:04 /usr/bin/ibus-daemon –daemonize –xim –address unix:tmpdir=/tmp/ibus

vskumar 2575 2428 0 20:55 ? 00:00:00 upstart-file-bridge –daemon –user

vskumar 2579 2428 0 20:55 ? 00:00:00 upstart-dbus-bridge –daemon –system –user –bus-name system

vskumar 2582 2428 0 20:55 ? 00:00:00 upstart-dbus-bridge –daemon –session –user –bus-name session

vskumar 2605 2428 0 20:55 ? 00:00:00 /usr/lib/ibus/ibus-x11 –kill-daemon

vskumar 2630 2428 0 20:56 ? 00:00:00 gpg-agent –homedir /home/vskumar/.gnupg –use-standard-socket –daemon

vskumar 2645 2428 0 20:56 ? 00:00:02 /usr/lib/unity-settings-daemon/unity-settings-daemon

vskumar 2664 2653 0 20:56 ? 00:00:00 /usr/bin/dbus-daemon –config-file=/etc/at-spi2/accessibility.conf –nofork –print-address 3

vskumar 2851 2654 0 20:56 ? 00:00:01 /usr/lib/unity-settings-daemon/unity-fallback-mount-helper

vskumar 2914 2428 0 20:57 ? 00:00:00 /bin/sh -c /usr/lib/x86_64-linux-gnu/zeitgeist/zeitgeist-maybe-vacuum; /usr/bin/zeitgeist-daemon

vskumar 2920 2914 0 20:57 ? 00:00:00 /usr/bin/zeitgeist-daemon

vskumar 3094 2428 0 21:00 ? 00:00:01 /usr/lib/x86_64-linux-gnu/unity-lens-files/unity-files-daemon

root 4148 1253 0 21:11 ? 00:00:00 docker-containerd-shim –namespace moby –workdir /var/lib/docker/containerd/daemon/io.containerd.runtime.v1.linux/moby/0fe495fc93edee3aaadc7fc0fbf21997f0ca3cde4d7e563aa8c61352a43957dd –address /var/run/docker/containerd/docker-containerd.sock –runtime-root /var/run/docker/runtime-runc

vskumar 4480 3206 0 21:19 pts/19 00:00:00 grep –color=auto daemon

vskumar@ubuntu:~$

======== You can see the list of processes running currently ========>

So we are successful! to run a container in a detached mode [not in an interactive mode!] using the command: ‘ sudo docker run -d ubuntu’

You can think in an application architecture having multiple servers or SOA running with different services.

You can simulate the same services using the docker containers, by setting up as images by configuring the required services and connect them to the architecture.

This way the advantages of containers can be utilized well. Where different companies are using and implementing their applications into containers architecture by saving lot of infrastructure cost. No hardware or physical servers are required. Lot of space also can be saved. The microservices architecture leads to the same way.

At this point, I would like to stop this session and in the next blog we will see other exercises.

Vcard-Shanthi Kumar V-v3

 

 

Protected: 8. DevOps:How to control and operate docker containers

This content is password protected. To view it please enter your password below:

5. DevOps: How to work with Docker Images

Docker-logo

In  continuation of my previous blog on “4. DevOps: How to work with Docker Containers”, in this blog I would like to give some lab practice on Docker Images.

How to pull the docker public images ?:
Docker portal will have numerous images available  under public.
Now, there is a need for us to know the usage of docker pull command, which is the defacto command to download Docker images.

Now, in this section, we will use the busybox image, one of the smallest but a very handy Docker image, to dive deep into Docker image handling:
$sudo docker pull busybox
============= Output ============>
vskumar@ubuntu:/var/tmp$ sudo docker pull busybox
Using default tag: latest
latest: Pulling from library/busybox
0ffadd58f2a6: Pull complete
Digest: sha256:bbc3a03235220b170ba48a157dd097dd1379299370e1ed99ce976df0355d24f0
Status: Downloaded newer image for busybox:latest
vskumar@ubuntu:/var/tmp$
=================>
Sometimes it might reject this request. We need to keep on trying to get it. I tried 4 times at different timings to connect to it.
Please note now, we have three images as below through all of our so far exercises:
===================>
vskumar@ubuntu:/var/log$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
hello-world latest f2a91732366c 4 days ago 1.85kB
ubuntu latest 20c44cd7596f 7 days ago 123MB
busybox latest 6ad733544a63 3 weeks ago 1.13MB
vskumar@ubuntu:/var/log$
===============>
Now, let us stop the docker services and check the status as below:
=================>
vskumar@ubuntu:/var/log$ sudo docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
hello-world latest f2a91732366c 4 days ago 1.85kB
ubuntu latest 20c44cd7596f 7 days ago 123MB
busybox latest 6ad733544a63 3 weeks ago 1.13MB
vskumar@ubuntu:/var/log$ sudo service docker stop
vskumar@ubuntu:/var/log$ sudo service docker status
● docker.service – Docker Application Container Engine
Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: e
Active: inactive (dead) since Sat 2017-11-25 02:52:25 PST; 8s ago
Docs: https://docs.docker.com
Process: 1224 ExecStart=/usr/bin/dockerd -H fd:// (code=exited, status=0/SUCCE
Main PID: 1224 (code=exited, status=0/SUCCESS)
Nov 25 02:21:42 ubuntu dockerd[1224]: time=”2017-11-25T02:21:42.863518710-08:00″
Nov 25 02:21:43 ubuntu dockerd[1224]: time=”2017-11-25T02:21:43-08:00″ level=inf
Nov 25 02:27:08 ubuntu dockerd[1224]: time=”2017-11-25T02:27:08.010096274-08:00″
Nov 25 02:27:08 ubuntu dockerd[1224]: time=”2017-11-25T02:27:08-08:00″ level=inf
Nov 25 02:27:08 ubuntu dockerd[1224]: time=”2017-11-25T02:27:08.199685599-08:00″
Nov 25 02:52:25 ubuntu dockerd[1224]: time=”2017-11-25T02:52:25.010875880-08:00″
Nov 25 02:52:25 ubuntu systemd[1]: Stopping Docker Application Container Engine.
Nov 25 02:52:25 ubuntu dockerd[1224]: time=”2017-11-25T02:52:25.081714537-08:00″
Nov 25 02:52:25 ubuntu systemd[1]: Stopped Docker Application Container Engine.
Nov 25 02:52:25 ubuntu systemd[1]: Stopped Docker Application Container Engine.
vskumar@ubuntu:/var/log$
====================>
You can see the inactive status of docker.
In such cases, restart the Docker service, as shown here:

$ sudo service docker restart
You can see the output as below:
==================>
vskumar@ubuntu:/var/log$ sudo service docker restart
vskumar@ubuntu:/var/log$ sudo service docker status 
docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: e
   Active: active (running) since Sat 2017-11-25 02:54:42 PST; 6s ago
     Docs: https://docs.docker.com
 Main PID: 3769 (dockerd)
    Tasks: 18
   Memory: 24.6M
      CPU: 989ms
   CGroup: /system.slice/docker.service
           ├─3769 /usr/bin/dockerd -H fd://
           └─3778 docker-containerd --config /var/run/docker/containerd/containe

Nov 25 02:54:41 ubuntu dockerd[3769]: time="2017-11-25T02:54:41.159062708-08:00"
Nov 25 02:54:41 ubuntu dockerd[3769]: time="2017-11-25T02:54:41.159806997-08:00"
Nov 25 02:54:41 ubuntu dockerd[3769]: time="2017-11-25T02:54:41.163503112-08:00"
Nov 25 02:54:41 ubuntu dockerd[3769]: time="2017-11-25T02:54:41.743276580-08:00"
Nov 25 02:54:41 ubuntu dockerd[3769]: time="2017-11-25T02:54:41.955217284-08:00"
Nov 25 02:54:41 ubuntu dockerd[3769]: time="2017-11-25T02:54:41.975961283-08:00"
Nov 25 02:54:42 ubuntu dockerd[3769]: time="2017-11-25T02:54:42.092220161-08:00"
Nov 25 02:54:42 ubuntu dockerd[3769]: time="2017-11-25T02:54:42.094334663-08:00"
Nov 25 02:54:42 ubuntu systemd[1]: Started Docker Application Container Engine.
Nov 25 02:54:42 ubuntu dockerd[3769]: time="2017-11-25T02:54:42.190194886-08:00"

vskumar@ubuntu:/var/log$ 

========================>

Now, let us reconfirm the existing docker images as below:
================>

vskumar@ubuntu:/var/log$ ^C
vskumar@ubuntu:/var/log$ sudo docker images
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
hello-world         latest              f2a91732366c        4 days ago          1.85kB
ubuntu              latest              20c44cd7596f        7 days ago          123MB
busybox             latest              6ad733544a63        3 weeks ago         1.13MB
vskumar@ubuntu:/var/log$ 
=======================>

By default, Docker always uses the image that is tagged as latest.

Each image variant can be directly identified by qualifying it with an appropriate tag.

An image can be tag-qualified by adding a colon (:) between the tag and the repository name (<repository>:<tag>). For demonstration, we will pull the 1.24 tagged version of busybox as shown here:

Now, For lab demonstration, we will pull the 1.24 tagged version of busybox as shown here:

$ sudo docker pull busybox:1.24

Now. You can see the total output before and after executing the above command:

==============================>

vskumar@ubuntu:/var/log$ ^C

vskumar@ubuntu:/var/log$ sudo docker images

REPOSITORY TAG IMAGE ID CREATED SIZE

hello-world latest f2a91732366c 4 days ago 1.85kB

ubuntu latest 20c44cd7596f 7 days ago 123MB

busybox latest 6ad733544a63 3 weeks ago 1.13MB

vskumar@ubuntu:/var/log$ ^C

vskumar@ubuntu:/var/log$

vskumar@ubuntu:/var/log$

vskumar@ubuntu:/var/log$ sudo docker pull busybox:1.24

1.24: Pulling from library/busybox

385e281300cc: Pull complete

a3ed95caeb02: Pull complete

Digest: sha256:8ea3273d79b47a8b6d018be398c17590a4b5ec604515f416c5b797db9dde3ad8

Status: Downloaded newer image for busybox:1.24

vskumar@ubuntu:/var/log$ sudo docker images

REPOSITORY TAG IMAGE ID CREATED SIZE

hello-world latest f2a91732366c 4 days ago 1.85kB

ubuntu latest 20c44cd7596f 7 days ago 123MB

busybox latest 6ad733544a63 3 weeks ago 1.13MB

busybox 1.24 47bcc53f74dc 20 months ago 1.11MB

vskumar@ubuntu:/var/log$

==============================>

There are two busybox containers with different versions.

So, on the basis of TAG values the containers are being pulled.

How to Search Docker images:

So far we have pulled the known images from the docker-hub.

Let us identify some docker images by using a search option as below. We can search for Docker images in the Docker Hub Registry using the docker search subcommand, as shown in this example:

$ sudo docker search mysql

You can see the displayed output of mysql images from the Docker Hub Registry:

=============>

vskumar@ubuntu:/var/log$ sudo docker search mysql

NAME DESCRIPTION STARS OFFICIAL AUTOMATED

mysql MySQL is a widely used, open-source relation… 5278 [OK]

mariadb MariaDB is a community-developed fork of MyS… 1634 [OK]

mysql/mysql-server Optimized MySQL Server Docker images. Create… 368 [OK]

percona Percona Server is a fork of the MySQL relati… 303 [OK]

hypriot/rpi-mysql RPi-compatible Docker Image with Mysql 74

zabbix/zabbix-server-mysql Zabbix Server with MySQL database support 64 [OK]

centurylink/mysql Image containing mysql. Optimized to be link… 53 [OK]

sameersbn/mysql 48 [OK]

zabbix/zabbix-web-nginx-mysql Zabbix frontend based on Nginx web-server wi… 38 [OK]

tutum/mysql Base docker image to run a MySQL database se… 29

1and1internet/ubuntu-16-nginx-php-phpmyadmin-mysql-5 ubuntu-16-nginx-php-phpmyadmin-mysql-5 17 [OK]

schickling/mysql-backup-s3 Backup MySQL to S3 (supports periodic backup… 16 [OK]

centos/mysql-57-centos7 MySQL 5.7 SQL database server 15

linuxserver/mysql A Mysql container, brought to you by LinuxSe… 12

centos/mysql-56-centos7 MySQL 5.6 SQL database server 6

openshift/mysql-55-centos7 DEPRECATED: A Centos7 based MySQL v5.5 image… 6

frodenas/mysql A Docker Image for MySQL 3 [OK]

dsteinkopf/backup-all-mysql backup all DBs in a mysql server 3 [OK]

circleci/mysql MySQL is a widely used, open-source relation… 2

cloudfoundry/cf-mysql-ci Image used in CI of cf-mysql-release 0

cloudposse/mysql Improved `mysql` service with support for `m… 0 [OK]

ansibleplaybookbundle/rhscl-mysql-apb An APB which deploys RHSCL MySQL 0 [OK]

astronomerio/mysql-sink MySQL sink 0 [OK]

inferlink/landmark-mysql landmark-mysql 0 [OK]

astronomerio/mysql-source MySQL source 0 [OK]

vskumar@ubuntu:/var/log$ ^C

vskumar@ubuntu:/var/log$

====================>

You can get the top 5 images by suing head -5 linux command.

$sudo docker search mysql | head -5

=============>

vskumar@ubuntu:/var/log$ sudo docker search mysql | head -5

NAME DESCRIPTION STARS OFFICIAL AUTOMATED

mysql MySQL is a widely used, open-source relation… 5278 [OK]

mariadb MariaDB is a community-developed fork of MyS… 1634 [OK]

mysql/mysql-server Optimized MySQL Server Docker images. Create… 368 [OK]

percona Percona Server is a fork of the MySQL relati… 303 [OK]

vskumar@ubuntu:/var/log$

===============>

If you see the above list, The mysql image curated and hosted by Docker Inc has a 5278 star rating, which is indicated as this is the most popular mysql image and aslo as Official image to use it. For security reasons we should use the official and highly rated images only.

As we planned, in this blog we have worked with the Docker images.

At this point we can stop this session and in the next blog we can see on “How to work with interactive containers”.

Vcard-Shanthi Kumar V-v3

Feel free to Contact me :