Load Balancer

Load balancers are used to increase capacity (concurrent users) and reliability of applications. They improve the overall performance of applications by decreasing the burden on servers associated with managing and maintaining application and network sessions, as well as by performing application-specific tasks.

Basically, a load balancer is a device that acts as a reverse proxy and distributes network or application traffic across a number of servers.

Load balancers are generally grouped into two categories:

1. Layer 4 and

2. Layer 7.

Layer 4 load balancers act upon data found in network and transport layer protocols (IP, TCP, FTP, UDP).

Layer 7 load balancers distribute requests based upon data found in application layer protocols such as HTTP.

Requests are received by both types of load balancers and they are distributed to a particular server based on a configured algorithm. Some industry standard algorithms are:

  • Round robin
  • Weighted round robin
  • Least connections
  • Least response time

Layer 7 load balancers can further distribute requests based on application specific data such as HTTP headers, cookies, or data within the application message itself, such as the value of a specific parameter.

Load balancers ensure reliability and availability by monitoring the “health” of applications and only sending requests to servers and applications that can respond in a timely manner.

 

Open Source Tools

Ultra Monkey

Ultra Monkey is a project to create load balanced and highly available network services. For example a cluster of web servers that appear as a single web server to end-users. The service may be for end-users across the world connected via the internet, or for enterprise users connected via an intranet.

More Details: http://www.ultramonkey.org/

Red Hat Cluster Suite

It is a high availability cluster software implementation from Linux leader Red Hat. It provide two type services:

  1. Application / Service Failover – Create n-node server clusters for failover of key applications and services
  2. IP Load Balancing – Load balance incoming IP network requests across a farm of servers

More Details:- http://www.redhat.com/cluster_suite/

Linux Virtual Server (LVS)

LVS is ultimate open source Linux load sharing and balancing software. You can easily build a high-performance and highly available server for Linux using this software.

More Details:- http://www.linux-vs.org/

The High Availability Linux Project

Linux-HA provides sophisticated high-availability (failover) capabilities on a wide range of platforms, supporting several tens of thousands of mission critical sites.

More Details: http://www.linux-ha.org/.

The cron command

The crontab command shown below will activate the cron tasks automatically on the hour:
0 * * * * wget -O – -q -t 1 http://www.jit.co.in/cron.php

In the above sample, the 0 * * * * represents when the task should happen. The first figure represents minutes – in this case, on the “zero” minute, or top of the hour. (If the number were, say, 10, then the action would take place at 10 minutes past the hour.) The other figures represent, respectively, hour, day, month and day of the week. A * is a wildcard, meaning “every time.”

The rest of the line basically tells the server to “ping” the url http://www.jit.co.in/cron.php.
Here is a diagram of the general crontab syntax, for illustration:

# +—————- minute (0 – 59)
# | +————- hour (0 – 23)
# | | +———- day of month (1 – 31)
# | | | +——- month (1 – 12)
# | | | | +—- day of week (0 – 7) (Sunday=0 or 7)
# | | | | | * * * * * command to be executed

Add ONE of the following lines:

45 * * * * /usr/bin/lynx -source http://jit.co.in/cron.php
45 * * * * /usr/bin/wget -O – -q -t 1 http://www.jit.co.in/cron.php
45 * * * * curl –silent –compressed http://jit.co.in/cron.php.

Fedora 14: Wireless network is not working

Wireless Network is not working in Fedora 14 by default, here is the solution
su –
yum install b43-fwcutter wget

Next, download the Windows driver:
wget downloads.openwrt.org/sources/broadcom-wl-4.150.10.5.tar.bz2

Next decompress, extract and copy firmware to /lib/firmware directory.

tar xjf broadcom-wl-4.150.10.5.tar.bz2
cd broadcom-wl-4.150.10.5/driver
b43-fwcutter -w /lib/firmware/ wl_apsta_mimo.o .

File Operation & Sudo Operations

Add the Linux groups to your server:

[root@jit.co.in tmp]# groupadd parents
[root@jit.co.in tmp]# groupadd children
[root@jit.co.in tmp]# groupadd soho

Add the Linux users and assign them to their respective groups

root@jit.co.in tmp]# useradd -g parents ram
[root@jit.co.in tmp]# useradd -g parents jagmohan
[root@jit.co.in tmp]# useradd -g children ramesh
[root@jit.co.in tmp]# useradd -g children hema
[root@jit.co.in tmp]# useradd -g soho accounts
[root@jit.co.in tmp]# useradd -g soho sales

User root changing the password for user ram

[root@jit.co.in root]# passwd ram
Changing password for user ram.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.
[root@jit.co.in root]#

Delete Users

userdel ram

There is also an optional -r switch that additionally removes all the contents of the user’s home directory.

# userdel -r ram

How to Tell the Groups to Which a User Belongs

[root@jit.co.in root]# groups ram
ram : parents
[root@jit.co.in root]#

How to Change the Ownership of a File

[root@jit.co.in tmp]# ll test.txt
-rw-r–r–  1 root root 0 Nov 17 22:14 test.txt
[root@jit.co.in tmp]# chown testuser:users test.txt

[root@jit.co.in tmp]# ll test.txt
-rw-r–r–  1 testuser users 0 Nov 17 22:14 test.txt

[root@jit.co.in tmp]#

SUDO Operation

Temporarily Gaining root Privileges

[bob@jit.co.in bob]$ more /etc/sudoers
/etc/sudoers: Permission denied

[bob@jit.co.in bob]$

Bob tries again using sudo and his regular user password and is successful:

[bob@jit.co.in bob]$ sudo more /etc/sudoers
Password:


[bob@jit.co.in bob]$

Becoming root for a Complete Login Session

The su command allows a regular user to become the system’s root user if they know the root password. A user with sudo rights to use the su command can become root, but they only need to know their own password, not that of root as seen here.

someuser@u-jit.co.in:~$ sudo su –
Password:
root@u-jit.co.in:~#

The /etc/sudoers file contains all the configuration and permission parameters needed for sudo to work.

Granting All Access to Specific Users

You can grant users bob and bunny full access to all privileged commands, with this sudoers entry.

bob, bunny  ALL=(ALL) ALL

This is generally not a good idea because this allows bob and bunny to use the su command to grant themselves permanent root privileges thereby bypassing the command logging features of sudo. The example on using aliases in the sudoers file shows how to eliminate this prob

Granting Access To Specific Users To Specific Files

This entry allows user peter and all the members of the group operator to gain access to all the program files in the /sbin and /usr/sbin directories, plus the privilege of running the command /usr/local/apps/check.pl. Notice how the trailing slash (/) is required to specify a directory location:

peter, %operator ALL= /sbin/, /usr/sbin, /usr/local/apps/check.pl

Granting Access to Specific Files as Another User

The sudo -u entry allows allows you to execute a command as if you were another user, but first you have to be granted this privilege in the sudoers file.

This feature can be convenient for programmers who sometimes need to kill processes related to projects they are working on. For example, programmer peter is on the team developing a financial package that runs a program called monthend as user accounts. From time to time the application fails, requiring “peter” to stop it with the /bin/kill, /usr/bin/kill or /usr/bin/pkill commands but only as user “accounts”. The sudoers entry would look like this:

peter ALL=(accounts) /bin/kill, /usr/bin/kill, /usr/bin/pkill
User peter is allowed to stop the monthend process with this command:

[peter@jit.co.in peter]# sudo -u accounts pkill monthend

Granting Access Without Needing Passwords

This example allows all users in the group operator to execute all the commands in the /sbin directory without the need for entering a password. This has the added advantage of being more convenient to the user:

%operator ALL= NOPASSWD: /sbin/

To create a Tar file
tar -czf archive.tar.gz *.txt

To list files in a compressed Tar file
tar -tzf archive.tar.gz.

MySQL Performance

MySQL Performance: Replication, Partition and Memcached

3 main step to increase mysql performance:

  1. Split Read and Write
  2. Horizontally partition
  3. Implement Memchache
  4. Setup Replication
  • You can spread your reads with replication, and that helps a lot, but you can’t spread writes (they have to process on all machines) and they’ll eventually consume all your resources. You’ll find yourself adding replicated slaves at an ever-increasing rate to make up for the diminishing returns each additional slave provides.
  • The next logical step is to horizontally partition your dataset onto different master/slave clusters so you can spread your writes, and then teach your application to connect to the correct cluster depending on the data it needs.
  • With memcached you can reduce your database reads to a mere fraction, leaving the databases to mainly do infrequent writes, and end up getting much more bang for your buck, since your databases won’t be blocking themselves doing ACID bookkeeping or waiting on writing threads.

How to setup Memcached server

Few Imp Links for detailed Study: http://www.danga.com/memcached/

Installing memcached


1
2
yum install memcached
yum install php5-memcache

Configure and Setting MemCache

  • After installing these two packages next step is to configure memcached.conf file. ( /etc/memcached.conf )
  • After that edit the memecached.ini file. ( /etc/httpd/conf.d/memcache.ini )

A sample memcache.ini file


1
2
3
4
5
6
7
8
9
10
11
; uncomment the next line to enable the module
extension=memcache.so
[memcache]
memcache.dbpath="/var/lib/memcache"
memcache.maxreclevel=0
memcache.maxfiles=0
memcache.archivememlim=0
memcache.maxfilesize=0
memcache.maxratio=0
session.save_handler = memcache
session.save_path = "tcp://192.168.1.1:11211?weight=1,tcp://192.168.1.2:11211"

Camand Line Memcache


1
2
memcached -d   ==> Start memcache as a background process
memcached -vv ==> Start memcached as a foreground process

Another method for checking whether memcached was enamble or not is to check the phpinfo() function

php MemChache Conf

php MemChache Conf

.

MySQL Step-by-Step Installations

If you are on Fedora or Red Hat Box

1
$ setenforce 0 #disbles selinux temporarily

$ yum install mysql

If you have a rpm then

1
$ rpm -i <strong><strong><code>/mysqlSRC_RPM_Folder/MySQL*



You can download rpm’s from http://www.mysql.org

1
2
$ restorecon -R /var/lib/mysql/ #restore selinux labels
$ setenforce 1 #enable selinux again

Now, I successfully installed MySQL. Now I wanted to change the location of where the actual data would be stored, I tried configuring the my.cnf file and using symlinks. I found using the symlinks to be easier and more compatible.

My data directory is /home/mysql.

The following is what I did to change the data directory after using RPM to install.

1
2
3
4
5
1. stop mysql server
2. mkdir /home/mysql
3. mv /var/lib/mysql/* /home/mysql/
4. rmdir /var/lib/mysql
5. ln -s /home/mysql /var/lib/mysql

Instead of using symlinks,we can just modify the my.cnf file (/etc/my.cnf)  by including “datadir = /home/mysql”

1
2
6. chown -R mysql:mysql /home/mysql
7. start mysql server again

.

LAMP

LAMP is the combination of Linux + Apache + MySQL + PHP (PHP/PERL/Python).

Linux is a free Unix-type operating system originally created by Linus Torvalds with the assistance of developers around the world. Developed under the GNU General Public License , the source code for Linux is freely available to everyone.

Apache is the most popular web server on the net. It is very secure, fast, and reliable.
MySQL is the fastest RDBMS in the world. It is very good for web based applications as well.

PHP is an open source project of the Apache Software Foundation. It is a powerful server-side scripting language for creating dynamic and interactive websites. PHP is the widely-used, free, and efficient alternative to competitors such as Microsoft’s ASP. PHP is used mainly in server-side scripting, but can be used from a command line interface or in standalone graphical applications. Textual User Interfaces can also be created using ncurses. PHP is a recursive initialism for PHP: Hypertext Preprocessor. PHP is perfectly suited for Web development and can be embedded directly into the HTML code. The PHP syntax is very similar to Perl and C. PHP is often used together with Apache (web server) on various operating systems. It also supports ISAPI and can be used with Microsoft’s IIS on Windows.

Ref.

.

Installation of FFmpeg, FFmpeg-PHP, Lame, Libogg, Libvorbis, FLVtool2, Mplayer, Mencoder, AMR Installation

Introduction

The following HOWTO will show you exactly how to install the following packages on a Debian Etch or Ubuntu 7.06 system:

  • FFmpeg
  • FFmpeg-PHP
  • Mplayer + Mencoder
  • flv2tool
  • LAME MP3 Encoder
  • AMR (for 3gp file conversions)
  • Libogg
  • Libvorbis

Do some prep-work

apt-get update
apt-get upgrade
apt-get install libjpeg-progs libjpeg62 libjpeg62-dev libsdl1.2-dev php5-dev build-essential unzip

Download all the files needed

cd /usr/local/src
wget http://www3.mplayerhq.hu/MPlayer/releases/codecs/essential-20071007.tar.bz2
wget http://rubyforge.org/frs/download.php/17497/flvtool2-1.0.6.tgz

lame
Project Location:
wget http://easynews.dl.sourceforge.net/sourceforge/lame/lame-3.97.tar.gz

ffmpeg-php
Project Location: http://sourceforge.net/projects/ffmpeg-php/
wget http://jaist.dl.sourceforge.net/sourceforge/ffmpeg-php/ffmpeg-php-0.6.0.tbz2

X———————————————————————————————
— Not Working —
wget http://superb-west.dl.sourceforge.net/sourceforge/ffmpeg-php/ffmpeg-php-0.5.0.tbz2
— Do not use it —
———————————————————————————————–X

wget http://downloads.xiph.org/releases/ogg/libogg-1.1.3.tar.gz
wget http://downloads.xiph.org/releases/vorbis/libvorbis-1.2.0.tar.gz
wget http://ftp.penguin.cz/pub/users/utx/amr/amrnb-7.0.0.2.tar.bz2
wget http://ftp.penguin.cz/pub/users/utx/amr/amrwb-7.0.0.3.tar.bz2
wget http://downloads.xiph.org/releases/theora/libtheora-1.0beta3.tar.gz

Extract all the files

tar zxvf lame-3.97.tar.gz
tar zxvf libogg-1.1.3.tar.gz
tar zxvf libvorbis-1.2.0.tar.gz
tar zxvf flvtool2-1.0.6.tgz
tar jxvf essential-20071007.tar.bz2
tar jxvf ffmpeg-php-0.5.0.tbz2
bzip2 -cd amrnb-7.0.0.2.tar.bz2 | tar xvf –
bzip2 -cd amrwb-7.0.0.3.tar.bz2 | tar xvf –
tar zxvf libtheora-1.0beta3.tar.gz
We need a codec directory
mkdir /usr/local/lib/codecs/

Install Ruby on Rails, subversion & ncurses

apt-get install subversion ruby libcurses-ruby

Run some SVN queries

svn checkout svn://svn.mplayerhq.hu/ffmpeg/trunk ffmpeg
svn checkout svn://svn.mplayerhq.hu/mplayer/trunk mplayer
cd /usr/local/src/mplayer
svn update

Copy Codecs for mplayer

mkdir /usr/local/lib/codecs
mv /usr/local/src/essential-20071007/* /usr/local/lib/codecs/
chmod -R 755 /usr/local/lib/codecs/
We also need to secure the tmp directory
mkdir /usr/local/src/tmp
chmod 777 /usr/local/src/tmp
export TMPDIR=/usr/local/src/tmp

Install lame

cd /usr/local/src/lame-3.97
./configure
make && make install

Install libogg

cd /usr/local/src/libogg-1.1.3
./configure && make && make install

Install libvorbis

cd /usr/local/src/libvorbis-1.2.0
./configure && make && make install

Install flvtool2

cd /usr/local/src/flvtool2-1.0.6/
ruby setup.rb config
ruby setup.rb setup
ruby setup.rb install

Install mplayer & mencoder

cd /usr/local/src/mplayer
./configure –enable-jpeg
make && make install

Install AMR (for 3gp conversion)

cd /usr/local/src/amrnb-7.0.0.2
./configure
make && make install
cd /usr/local/src/amrwb-7.0.0.3
./configure
make && make install

Install libtheora (for ogg video encoding)

cd /usr/local/src/libtheora-1.0beta3
./configure
make && make install

Install ffmpeg

cd /usr/local/src/ffmpeg/
./configure –enable-libmp3lame –enable-libogg –enable-libvorbis –disable-mmx –enable-shared –enable-amr-nb –enable-libtheora
Ubuntu 7.06 users please use the following configure command:
./configure –enable-libmp3lame –enable-libogg –enable-libvorbis –disable-mmx –enable-shared –enable-libamr-nb –enable-libtheora
Ubuntu 8.04 users please use the following configure command:
./configure –enable-libmp3lame –enable-libvorbis –disable-mmx –enable-shared –enable-libamr-nb –enable-libamr-wb –enable-nonfree –enable-libtheora
For Fedora
./configure –enable-libmp3lame –enable-libvorbis –disable-mmx –enable-shared –enable-libamr-nb –enable-libamr-wb –enable-nonfree –enable-libtheora

make
make install
ln -s /usr/local/lib/libavdevice.so.52 /usr/lib/libavdevice.so.52
ln -s /usr/local/lib/libavformat.so.52 /usr/lib/libavformat.so.52
ln -s /usr/local/lib/libavcodec.so.51 /usr/lib/libavcodec.so.51
ln -s /usr/local/lib/libavutil.so.49 /usr/lib/libavutil.so.49
ln -s /usr/local/lib/libmp3lame.so.0 /usr/lib/libmp3lame.so.0
ln -s /usr/local/lib/libavformat.so.51 /usr/lib/libavformat.so.51
ln -s /usr/local/lib/libamrnb.so.2 /usr/lib/libamrnb.so.2

Install ffmpeg-php

cd /usr/local/src/ffmpeg-php-0.5.0/
phpize

X—————————————————————————
if phpize will not work and shows command not found, then we need to install
php-devel for the php version.
—————————————————————————-X

./configure
make
make install
You now need to add the new ffmpeg-php module to the php.ini file
vim /etc/php5/apache2/php.ini
extension=ffmpeg.so (add this line to the end of the file)

Restart & done

/etc/init.d/apache2 force-reload

Trouble Shooting

“# ffmpeg
ffmpeg: error while loading shared libraries: libavcodec.so.52: cannot open shared object file: No such file or directory”

ln -s /usr/local/lib/libamrwb.so.3 /usr/lib/libamrwb.so.3
ln -s /usr/local/lib/libamrnb.so.3 /usr/lib/libamrnb.so.3
ln -s /usr/local/lib/libavcodec.so.52 /usr/lib/libavcodec.so.52

Test Your application:.