Home » Linux
Category Archives: Linux
NFS Server
Network File System (NFS)
- A Network File System (NFS) allows remote hosts to mount file systems over a network and interact with those file systems as though they are mounted locally. This enables system administrators to consolidate resources onto centralized servers on the network.
- All versions of NFS can use Transmission Control Protocol (TCP) running over an IP network, with NFSv4 requiring it. NFSv2 and NFSv3 can use the User Datagram Protocol (UDP) running over an IP network to provide a stateless network connection between the client and server.
- The NFS server sends the client a file handle after the client is authorized to access the shared volume. This file handle is an opaque object stored on the server’s side and is passed along with RPC requests from the client. The NFS server can be restarted without affecting the clients and the cookie remains intact. However, because UDP is stateless, if the server goes down unexpectedly, UDP clients continue to saturate the network with requests for the server. For this reason, TCP is the preferred protocol when connecting to an NFS server.
- After the client is granted access by TCP wrappers, the NFS server refers to its configuration file,/etc/exports, to determine whether the client is allowed to access any of the exported file systems. Once access is granted, all file and directory operations are available to the user.
Required Services
- nfs — (/sbin/service nfs start) starts the NFS server and the appropriate RPC processes to service requests for shared NFS file systems.
- nfslock — (/sbin/service nfslock start) is a mandatory service that starts the appropriate RPC processes to allow NFS clients to lock files on the server.
- portmap — accepts port reservations from local RPC services. These ports are then made available (or advertised) so the corresponding remote RPC services access them. portmap responds to requests for RPC services and sets up connections to the requested RPC service. This is not used with NFSv4.
The general syntax for the line in /etc/fstab is as follows:
server:/usr/local/pub /pub nfs rsize=8192,wsize=8192,timeo=14,intr
The mount point /pub must exist on the client machine before this command can be executed. After adding this line to /etc/fstab on the client system, type the command mount /pub at a shell prompt, and the mount point /pub is mounted from the server.
The /etc/fstab file is referenced by the netfs service at boot time, so lines referencing NFS shares have the same effect as manually typing the mount command during the boot process.
A sample /etc/fstab line to mount an NFS export looks like the following example:
<server>:</remote/export> </local/directory> <nfs-type> <options> 0 0
Email Server
The Real E-mail System
For the vast majority of people right now, the real e-mail system consists of two different servers running on a server machine. One is called the SMTP server, where SMTP stands for Simple Mail Transfer Protocol. The SMTP server handles outgoing mail. The other is either a POP3 server or an IMAP server, both of which handle incoming mail. POP stands for Post Office Protocol, and IMAP stands for Internet Mail Access Protocol. A typical e-mail server looks like this:
The SMTP server listens on well-known port number 25, POP3 listens on port 110 and IMAP uses port 143.
The SMTP Server
Whenever you send a piece of e-mail, your e-mail client interacts with the SMTP server to handle the sending. The SMTP server on your host may have conversations with other SMTP servers to deliver the e-mail.
Let’s assume that I want to send a piece of e-mail. My e-mail ID is brain, and I have my account on howstuffworks.com. I want to send e-mail to jsmith@mindspring.com. I am using a stand-alone e-mail client like Outlook Express.
When I set up my account at arpittak.com, I told Outlook Express the name of the mail server — mail.arpittak.com. When I compose a message and press the Send button, here’s what happens:
- Outlook Express connects to the SMTP server atmail.arpittak.com using port 25.
- Outlook Express has a conversation with the SMTP server, telling the SMTP server the address of the sender and the address of the recipient, as well as the body of the message.
- The SMTP server takes the “to” address (jsmith@mindspring.com) and breaks it into two parts: the recipient name (jsmith) and the domain name (mindspring.com). If the “to” address had been another user at arpittak.com., the SMTP server would simply hand the message to the POP3 server for arpittak.com (using a little program called the delivery agent). Since the recipient is at another domain, SMTP needs to communicate with that domain.
- The SMTP server has a conversation with a Domain Name Server, or DNS. It says, “Can you give me the IP address of the SMTP server for mindspring.com?” The DNS replies with the one or more IP addresses for the SMTP server(s) that Mindspring operates.
- The SMTP server at arpittak.com connects with the SMTP server at Mindspring using port 25. It has the same simple text conversation that my e-mail client had with the SMTP server for arpittak, and gives the message to the Mindspring server. The Mindspring server recognizes that the domain name for jsmith is at Mindspring, so it hands the message to Mindspring’s POP3 server, which puts the message in jsmith’s mailbox.
If, for some reason, the SMTP server at arpittakcannot connect with the SMTP server at Mindspring, then the message goes into a queue. The SMTP server on most machines uses a program called sendmail to do the actual sending, so this queue is called the sendmail queue. Sendmail will periodically try to resend the messages in its queue. For example, it might retry every 15 minutes. After four hours, it will usually send you a piece of mail that tells you there is some sort of problem. After five days, most sendmail configurations give up and return the mail to you undelivered.
The SMTP server understands very simple text commands like HELO, MAIL, RCPT and DATA. The most common commands are:
- HELO – introduce yourself
- EHLO – introduce yourself and request extended mode
- MAIL FROM: – specify the sender
- RCPT TO: – specify the recipient
The POP3 and IMAP Servers
When you check your e-mail, your e-mail client connects to the POP3 server using port 110. The POP3 server requires an account name and a password. Once you’ve logged in, the POP3 server opens your text file and allows you to access it. Like the SMTP server, the POP3 server understands a very simple set of text commands. Here are the most common commands:
- USER – enter your user ID
- PASS – enter your password
- QUIT – quit the POP3 server
- LIST – list the messages and their size
The IMAP Server
As you can see, the POP3 protocol is very simple. It allows you to have a collection of messages stored in a text file on the server. Your e-mail client (e.g. Outlook Express) can connect to your POP3 e-mail server and download the messages from the POP3 text file onto your PC. That is about all that you can do with POP3.
Many users want to do far more than that with their e-mail, and they want their e-mail to remain on the server. The main reason for keeping your e-mail on the server is to allow users to connect from a variety of machines. With POP3, once you download your e-mail it’s stuck on the machine to which you downloaded it. If you want to read your e-mail both on your desktop machine and your laptop (depending on whether you’re working in the office or on the road), POP3 makes life difficult.
IMAP (Internet Mail Access Protocol) is a more advanced protocol that solves these problems. With IMAP, your mail stays on the e-mail server. You can organize your mail into folders, and all the folders live on the server as well. When you search your e-mail, the search occurs on the server machine, rather than on your machine. This approach makes it extremely easy for you to access your e-mail from any machine, and regardless of which machine you use, you have access to all of your mail in all of your folders.
Web Server
The Basic Process
Your browser formed a connection to a Web server, requested a page and received it.
The browser broke the URL into three parts:
- The protocol (“http”)
- The server name (“www.howstuffworks.com”)
- The file name (“web-server.htm”)
The browser communicated with a name server to translate the server name “www.arpittak.com” into an IP Address, which it uses to connect to the server machine. The browser then formed a connection to the server at that IP address on port 80.
Following the HTTP protocol, the browser sent a GET request to the server, asking for the file “http://www.arpittak.com/index.htm.” (Note that cookies may be sent from browser to server with the GET request.)
The server then sent the HTML text for the Web page to the browser. (Cookies may also be sent from server to browser in the header for the page.) The browser read the HTML tags and formatted the page onto your screen.
Clients and Servers
In general, all of the machines on the Internet can be categorized as two types: servers and clients. Those machines that provide services (like Web servers or FTP servers) to other machines are servers. And the machines that are used to connect to those services are clients. When you connect to Yahoo! at http://www.yahoo.com to read a page, Yahoo! is providing a machine (probably a cluster of very large machines), for use on the Internet, to service your request. Yahoo! is providing a server. Your machine, on the other hand, is probably providing no services to anyone else on the Internet. Therefore, it is a user machine, also known as a client. It is possible and common for a machine to be both a server and a client, but for our purposes here you can think of most machines as one or the other.
A server machine may provide one or more services on the Internet. For example, a server machine might have software running on it that allows it to act as a Web server, an e-mail server and an FTP server. Clients that come to a server machine do so with a specific intent, so clients direct their requests to a specific software server running on the overall server machine. For example, if you are running a Web browser on your machine, it will most likely want to talk to the Web server on the server machine. Your Telnet application will want to talk to the Telnet server, your e-mail application will talk to the e-mail server, and so on…
IP Addresses
To keep all of these machines straight, each machine on the Internet is assigned a unique address called an IP address. IP stands for Internet protocol, and these addresses are 32-bit numbers, normally expressed as four “octets” in a “dotted decimal number.” A typical IP address looks like this: 216.27.61.137
The four numbers in an IP address are called octets because they can have values between 0 and 255, which is 28 possibilities per octet.
Every machine on the Internet has a unique IP address. A server has a static IP address that does not change very often. A home machine that is dialing up through a modem often has an IP address that is assigned by the ISP when the machine dials in. That IP address is unique for that session — it may be different the next time the machine dials in. This way, an ISP only needs one IP address for each modem it supports, rather than for each customer.
Domain Names
Because most people have trouble remembering the strings of numbers that make up IP addresses, and because IP addresses sometimes need to change, all servers on the Internet also have human-readable names, called domain names. For example, www.arppittak.com is a permanent, human-readable name.
The name http://www.arpittak.com actually has three parts:
- The host name (“www”)
- The domain name (“arpittak”)
- The top-level domain name (“com”)
Name Servers
A set of servers called domain name servers (DNS) maps the human-readable names to the IP addresses. These servers are simple databases that map names to IP addresses, and they are distributed all over the Internet. Most individual companies, ISPs and universities maintain small name servers to map host names to IP addresses. There are also central name servers that use data supplied by VeriSign to map domain names to IP addresses.
Ports
Any server machine makes its services available to the Internet using numbered ports, one for each service that is available on the server. For example, if a server machine is running a Web server and an FTP server, the Web server would typically be available on port 80, and the FTP server would be available on port 21. Clients connect to a service at a specific IP address and on a specific port.
Each of the most well-known services is available at a well-known port number. Here are some common port numbers:
- echo 7
- daytime 13
- qotd 17 (Quote of the Day)
- ftp 21
- telnet 23
- smtp 25 (Simple Mail Transfer, meaning e-mail)
- time 37
- nameserver 53
Protocols
Once a client has connected to a service on a particular port, it accesses the service using a specific protocol. The protocol is the pre-defined way that someone who wants to use a service talks with that service. The “someone” could be a person, but more often it is a computer program like a Web browser. Protocols are often text, and simply describe how the client and server will have their conversation.
Perhaps the simplest protocol is the daytime protocol. If you connect to port 13 on a machine that supports a daytime server, the server will send you its impression of the current date and time and then close the connection. The protocol is, “If you connect to me, I will send you the date and time and then disconnect.”
Awk Tutorials
Awk Introduction and Printing Operations
Awk is a programming language which allows easy manipulation of structured data and the generation of formatted reports. Awk stands for the names of its authors “Aho, Weinberger, and Kernighan”
The Awk is mostly used for pattern scanning and processing. It searches one or more files to see if they contain lines that matches with the specified patterns and then perform associated actions.
Some of the key features of Awk are:
- Awk views a text file as records and fields.
- Like common programming language, Awk has variables, conditionals and loops
- Awk has arithmetic and string operators.
- Awk can generate formatted reports
Awk reads from a file or from its standard input, and outputs to its standard output. Awk does not get along with non-text files.
Syntax:
awk '/search pattern1/ {Actions}
/search pattern2/ {Actions}' file
In the above awk syntax:
- search pattern is a regular expression.
- Actions – statement(s) to be performed.
- several patterns and actions are possible in Awk.
- file – Input file.
- Single quotes around program is to avoid shell not to interpret any of its special characters.
Awk Working Methodology
- Awk reads the input files one line at a time.
- For each line, it matches with given pattern in the given order, if matches performs the corresponding action.
- If no pattern matches, no action will be performed.
- In the above syntax, either search pattern or action are optional, But not both.
- If the search pattern is not given, then Awk performs the given actions for each line of the input.
- If the action is not given, print all that lines that matches with the given patterns which is the default action.
- Empty braces with out any action does nothing. It wont perform default printing operation.
- Each statement in Actions should be delimited by semicolon.
Let us create employee.txt file which has the following content, which will be used in the
examples mentioned below.
$cat employee.txt 100 Thomas Manager Sales $5,000 200 Jason Developer Technology $5,500 300 Sanjay Sysadmin Technology $7,000 400 Nisha Manager Marketing $9,500 500 Randy DBA Technology $6,000
Awk Example 1. Default behavior of Awk
By default Awk prints every line from the file.
$ awk '{print;}' employee.txt
100 Thomas Manager Sales $5,000
200 Jason Developer Technology $5,500
300 Sanjay Sysadmin Technology $7,000
400 Nisha Manager Marketing $9,500
500 Randy DBA Technology $6,000
In the above example pattern is not given. So the actions are applicable to all the lines.
Action print with out any argument prints the whole line by default. So it prints all the
lines of the file with out fail. Actions has to be enclosed with in the braces.
Awk Example 2. Print the lines which matches with the pattern.
$ awk '/Thomas/ > /Nisha/' employee.txt 100 Thomas Manager Sales $5,000 400 Nisha Manager Marketing $9,500
In the above example it prints all the line which matches with the ‘Thomas’ or ‘Nisha’. It has two patterns. Awk accepts any number of patterns, but each set (patterns and its corresponding actions) has to be separated by newline.
Awk Example 3. Print only specific field.
Awk has number of built in variables. For each record i.e line, it splits the record delimited by whitespace character by default and stores it in the $n variables. If the line has 4 words, it will be stored in $1, $2, $3 and $4. $0 represents whole line. NF is a built in variable which represents total number of fields in a record.
$ awk '{print $2,$5;}' employee.txt
Thomas $5,000
Jason $5,500
Sanjay $7,000
Nisha $9,500
Randy $6,000
$ awk '{print $2,$NF;}' employee.txt
Thomas $5,000
Jason $5,500
Sanjay $7,000
Nisha $9,500
Randy $6,000
In the above example $2 and $5 represents Name and Salary respectively. We can get the Salary using $NF also, where $NF represents last field. In the print statement ‘,’ is a concatenator.
Awk Example 4. Initialization and Final Action
Awk has two important patterns which are specified by the keyword called BEGIN and END.
Syntax:
BEGIN { Actions}
{ACTION} # Action for everyline in a file
END { Actions }
# is for comments in Awk
Actions specified in the BEGIN section will be executed before starts reading the lines from the input.
END actions will be performed after completing the reading and processing the lines from the input.
$ awk 'BEGIN {print "NametDesignationtDepartmenttSalary";}
> {print $2,"t",$3,"t",$4,"t",$NF;}
> END{print "Report Generatedn--------------";
> }' employee.txt
Name Designation Department Salary
Thomas Manager Sales $5,000
Jason Developer Technology $5,500
Sanjay Sysadmin Technology $7,000
Nisha Manager Marketing $9,500
Randy DBA Technology $6,000
Report Generated
--------------
In the above example, it prints headline and last file for the reports.
Awk Example 5. Find the employees who has employee id greater than 200
$ awk '$1 >200' employee.txt 300 Sanjay Sysadmin Technology $7,000 400 Nisha Manager Marketing $9,500 500 Randy DBA Technology $6,000
In the above example, first field ($1) is employee id. So if $1 is greater than 200, then just do the default print action to print the whole line.
Awk Example 6. Print the list of employees in Technology department
Now department name is available as a fourth field, so need to check if $4 matches with the string “Technology”, if yes print the line.
$ awk '$4 ~/Technology/' employee.txt 200 Jason Developer Technology $5,500 300 Sanjay Sysadmin Technology $7,000 500 Randy DBA Technology $6,000
Operator ~ is for comparing with the regular expressions. If it matches the default action i.e print whole line will be performed.
Awk Example 7. Print number of employees in Technology department
The below example, checks if the department is Technology, if it is yes, in the Action, just increment the count variable, which was initialized with zero in the BEGIN section.
$ awk 'BEGIN { count=0;}
$4 ~ /Technology/ { count++; }
END { print "Number of employees in Technology Dept =",count;}' employee.txt
Number of employees in Tehcnology Dept = 3
Linux Boot Process
6 Stages of Linux Boot Process (Startup Sequence)
Press the power button on your system, and after few moments you see the Linux login prompt.
Have you ever wondered what happens behind the scenes from the time you press the power button until the Linux login prompt appears?
The following are the 6 high level stages of a typical Linux boot process.
1. BIOS
- BIOS stands for Basic Input/Output System
- Performs some system integrity checks
- Searches, loads, and executes the boot loader program.
- It looks for boot loader in floppy, cd-rom, or hard drive. You can press a key (typically F12 of F2, but it depends on your system) during the BIOS startup to change the boot sequence.
- Once the boot loader program is detected and loaded into the memory, BIOS gives the control to it.
- So, in simple terms BIOS loads and executes the MBR boot loader.
2. MBR
- MBR stands for Master Boot Record.
- It is located in the 1st sector of the bootable disk. Typically /dev/hda, or /dev/sda
- MBR is less than 512 bytes in size. This has three components 1) primary boot loader info in 1st 446 bytes 2) partition table info in next 64 bytes 3) mbr validation check in last 2 bytes.
- It contains information about GRUB (or LILO in old systems).
- So, in simple terms MBR loads and executes the GRUB boot loader.
3. GRUB
- GRUB stands for Grand Unified Bootloader.
- If you have multiple kernel images installed on your system, you can choose which one to be executed.
- GRUB displays a splash screen, waits for few seconds, if you don’t enter anything, it loads the default kernel image as specified in the grub configuration file.
- GRUB has the knowledge of the filesystem (the older Linux loader LILO didn’t understand filesystem).
- Grub configuration file is /boot/grub/grub.conf (/etc/grub.conf is a link to this). The following is sample grub.conf of CentOS.
#boot=/dev/sda
default=0
timeout=5
splashimage=(hd0,0)/boot/grub/splash.xpm.gz
hiddenmenu
title CentOS (2.6.18-194.el5PAE)
root (hd0,0)
kernel /boot/vmlinuz-2.6.18-194.el5PAE ro root=LABEL=/
initrd /boot/initrd-2.6.18-194.el5PAE.img
- As you notice from the above info, it contains kernel and initrd image.
- So, in simple terms GRUB just loads and executes Kernel and initrd images.
4. Kernel
- Mounts the root file system as specified in the “root=” in grub.conf
- Kernel executes the /sbin/init program
- Since init was the 1st program to be executed by Linux Kernel, it has the process id (PID) of 1. Do a ‘ps -ef | grep init’ and check the pid.
- initrd stands for Initial RAM Disk.
- initrd is used by kernel as temporary root file system until kernel is booted and the real root file system is mounted. It also contains necessary drivers compiled inside, which helps it to access the hard drive partitions, and other hardware.
5. Init
- Looks at the /etc/inittab file to decide the Linux run level.
- Following are the available run levels
- 0 – halt
- 1 – Single user mode
- 2 – Multiuser, without NFS
- 3 – Full multiuser mode
- 4 – unused
- 5 – X11
- 6 – reboot
- Init identifies the default initlevel from /etc/inittab and uses that to load all appropriate program.
- Execute ‘grep initdefault /etc/inittab’ on your system to identify the default run level
- If you want to get into trouble, you can set the default run level to 0 or 6. Since you know what 0 and 6 means, probably you might not do that.
- Typically you would set the default run level to either 3 or 5.
6. Runlevel programs
- When the Linux system is booting up, you might see various services getting started. For example, it might say “starting sendmail …. OK”. Those are the runlevel programs, executed from the run level directory as defined by your run level.
- Depending on your default init level setting, the system will execute the programs from one of the following directories.
- Run level 0 – /etc/rc.d/rc0.d/
- Run level 1 – /etc/rc.d/rc1.d/
- Run level 2 – /etc/rc.d/rc2.d/
- Run level 3 – /etc/rc.d/rc3.d/
- Run level 4 – /etc/rc.d/rc4.d/
- Run level 5 – /etc/rc.d/rc5.d/
- Run level 6 – /etc/rc.d/rc6.d/
- Please note that there are also symbolic links available for these directory under /etc directly. So, /etc/rc0.d is linked to /etc/rc.d/rc0.d.
- Under the /etc/rc.d/rc*.d/ directories, you would see programs that start with S and K.
- Programs starts with S are used during startup. S for startup.
- Programs starts with K are used during shutdown. K for kill.
- There are numbers right next to S and K in the program names. Those are the sequence number in which the programs should be started or killed.
- For example, S12syslog is to start the syslog deamon, which has the sequence number of 12. S80sendmail is to start the sendmail daemon, which has the sequence number of 80. So, syslog program will be started before sendmail.
There you have it. That is what happens during the Linux boot process.
Raid Tutorials
RAID stands for Redundant Array of Inexpensive (Independent) Disks.
On most situations you will be using one of the following four levels of RAIDs.
- RAID 0
- RAID 1
- RAID 5
- RAID 10 (also known as RAID 1+0)
This article explains the main difference between these raid levels along with an easy to understand diagram.
In all the diagrams mentioned below:
- A, B, C, D, E and F – represents blocks
- p1, p2, and p3 – represents parity
RAID LEVEL 0
Following are the key points to remember for RAID level 0.
- Minimum 2 disks.
- Excellent performance ( as blocks are striped ).
- No redundancy ( no mirror, no parity ).
- Don’t use this for any critical system.
RAID LEVEL 1
Following are the key points to remember for RAID level 1.
- Minimum 2 disks.
- Good performance ( no striping. no parity ).
- Excellent redundancy ( as blocks are mirrored ).
RAID LEVEL 5
Following are the key points to remember for RAID level 5.
- Minimum 3 disks.
- Good performance ( as blocks are striped ).
- Good redundancy ( distributed parity ).
- Best cost effective option providing both performance and redundancy. Use this for DB that is heavily read oriented. Write operations will be slow.
RAID LEVEL 10
Following are the key points to remember for RAID level 10.
- Minimum 4 disks.
- This is also called as “stripe of mirrors”
- Excellent redundancy ( as blocks are mirrored )
- Excellent performance ( as blocks are striped )
- If you can afford the dollar, this is the BEST option for any mission critical applications (especially databases).
Linux Commands
1. tar command examples
Create a new tar archive.
$ tar cvf archive_name.tar dirname/
Extract from an existing tar archive.
$ tar xvf archive_name.tar
View an existing tar archive.
$ tar tvf archive_name.tar
2. grep command examples
Search for a given string in a file (case in-sensitive search).
$ grep -i "the" demo_file
Print the matched line, along with the 3 lines after it.
$ grep -A 3 -i "example" demo_text
Search for a given string in all files recursively
$ grep -r "ramesh" *
3. find command examples
Find files using file-name ( case in-sensitve find)
# find -iname "MyCProgram.c"
Execute commands on files found by the find command
$ find -iname "MyCProgram.c" -exec md5sum {} ;
Find all empty files in home directory
# find ~ -empty
4. ssh command examples
Login to remote host
ssh -l jsmith remotehost.example.com
Debug ssh client
ssh -v -l jsmith remotehost.example.com
Display ssh client version
$ ssh -V OpenSSH_3.9p1, OpenSSL 0.9.7a Feb 19 2003
5. sed command examples
When you copy a DOS file to Unix, you could find rn in the end of each line. This example converts the DOS file format to Unix file format using sed command.
$sed 's/.$//' filename
Print file content in reverse order
$ sed -n '1!G;h;$p' thegeekstuff.txt
Add line number for all non-empty-lines in a file
$ sed '/./=' thegeekstuff.txt | sed 'N; s/n/ /'
6. awk command examples
Remove duplicate lines using awk
$ awk '!($0 in array) { array[$0]; print }' temp
Print all lines from /etc/passwd that has the same uid and gid
$awk -F ':' '$3==$4' passwd.txt
Print only specific field from a file.
$ awk '{print $2,$5;}' employee.txt
7. vim command examples
Go to the 143rd line of file
$ vim +143 filename.txt
Go to the first match of the specified
$ vim +/search-term filename.txt
Open the file in read only mode.
$ vim -R /etc/passwd
8. diff command examples
Ignore white space while comparing.
# diff -w name_list.txt name_list_new.txt 2c2,3 < John Doe --- > John M Doe > Jason Bourne
9. sort command examples
Sort a file in ascending order
$ sort names.txt
Sort a file in descending order
$ sort -r names.txt
Sort passwd file by 3rd field.
$ sort -t: -k 3n /etc/passwd | more
10. export command examples
To view oracle related environment variables.
$ export | grep ORACLE declare -x ORACLE_BASE="/u01/app/oracle" declare -x ORACLE_HOME="/u01/app/oracle/product/10.2.0" declare -x ORACLE_SID="med" declare -x ORACLE_TERM="xterm"
To export an environment variable:
$ export ORACLE_HOME=/u01/app/oracle/product/10.2.0
11. xargs command examples
Copy all images to external hard-drive
# ls *.jpg | xargs -n1 -i cp {} /external-hard-drive/directory
Search all jpg images in the system and archive it.
# find / -name *.jpg -type f -print | xargs tar -cvzf images.tar.gz
Download all the URLs mentioned in the url-list.txt file
# cat url-list.txt | xargs wget –c
12. ls command examples
Display filesize in human readable format (e.g. KB, MB etc.,)
$ ls -lh -rw-r----- 1 ramesh team-dev 8.9M Jun 12 15:27 arch-linux.txt.gz
Order Files Based on Last Modified Time (In Reverse Order) Using ls -ltr
$ ls -ltr
Visual Classification of Files With Special Characters Using ls -F
$ ls -F
13. pwd command
pwd is Print working directory. What else can be said about the good old pwd who has been printing the current directory name for ages.
14. cd command examples
Use “cd -” to toggle between the last two directories
Use “shopt -s cdspell” to automatically correct mistyped directory names on cd
15. gzip command examples
To create a *.gz compressed file:
$ gzip test.txt
To uncompress a *.gz file:
$ gzip -d test.txt.gz
Display compression ratio of the compressed file using gzip -l
$ gzip -l *.gz
compressed uncompressed ratio uncompressed_name
23709 97975 75.8% asp-patch-rpms.txt
16. bzip2 command examples
To create a *.bz2 compressed file:
$ bzip2 test.txt
To uncompress a *.bz2 file:
bzip2 -d test.txt.bz2
17. unzip command examples
To extract a *.zip compressed file:
$ unzip test.zip
View the contents of *.zip file (Without unzipping it):
$ unzip -l jasper.zip
Archive: jasper.zip
Length Date Time Name
-------- ---- ---- ----
40995 11-30-98 23:50 META-INF/MANIFEST.MF
32169 08-25-98 21:07 classes_
15964 08-25-98 21:07 classes_names
10542 08-25-98 21:07 classes_ncomp
18. shutdown command examples
Shutdown the system and turn the power off immediately.
# shutdown -h now
Shutdown the system after 10 minutes.
# shutdown -h +10
Reboot the system using shutdown command.
# shutdown -r now
Force the filesystem check during reboot.
# shutdown -Fr now
19. ftp command examples
Both ftp and secure ftp (sftp) has similar commands. To connect to a remote server and download multiple files, do the following.
$ ftp IP/hostname ftp> mget *.html
To view the file names located on the remote server before downloading, mls ftp command as shown below.
ftp> mls *.html - /ftptest/features.html /ftptest/index.html /ftptest/othertools.html /ftptest/samplereport.html /ftptest/usage.html
20. crontab command examples
View crontab entry for a specific user
# crontab -u john -l
Schedule a cron job every 10 minutes.
*/10 * * * * /home/ramesh/check-disk-space
21. service command examples
Service command is used to run the system V init scripts. i.e Instead of calling the scripts located in the /etc/init.d/ directory with their full path, you can use the service command.
Check the status of a service:
# service ssh status
Check the steatus of all the services.
service --status-all
Restart a service.
# service ssh restart
22. ps command examples
ps command is used to display information about the processes that are running in the system.
While there are lot of arguments that could be passed to a ps command, following are some of the common ones.
To view current running processes.
$ ps -ef | more
To view current running processes in a tree structure. H option stands for process hierarchy.
$ ps -efH | more
23. free command examples
This command is used to display the free, used, swap memory available in the system.
Typical free command output. The output is displayed in bytes.
$ free
total used free shared buffers cached
Mem: 3566408 1580220 1986188 0 203988 902960
-/+ buffers/cache: 473272 3093136
Swap: 4000176 0 4000176
If you want to quickly check how many GB of RAM your system has use the -g option. -b option displays in bytes, -k in kilo bytes, -m in mega bytes.
$ free -g
total used free shared buffers cached
Mem: 3 1 1 0 0 0
-/+ buffers/cache: 0 2
Swap: 3 0 3
If you want to see a total memory ( including the swap), use the -t switch, which will display a total line as shown below.
ramesh@ramesh-laptop:~$ free -t
total used free shared buffers cached
Mem: 3566408 1592148 1974260 0 204260 912556
-/+ buffers/cache: 475332 3091076
Swap: 4000176 0 4000176
Total: 7566584 1592148 5974436
24. top command examples
top command displays the top processes in the system ( by default sorted by cpu usage ). To sort top output by any column, Press O (upper-case O) , which will display all the possible columns that you can sort by as shown below.
Current Sort Field: P for window 1:Def Select sort field via field letter, type any other key to return a: PID = Process Id v: nDRT = Dirty Pages count d: UID = User Id y: WCHAN = Sleeping in Function e: USER = User Name z: Flags = Task Flags ........
To displays only the processes that belong to a particular user use -u option. The following will show only the top processes that belongs to oracle user.
$ top -u oracle
25. df command examples
Displays the file system disk space usage. By default df -k displays output in bytes.
$ df -k Filesystem 1K-blocks Used Available Use% Mounted on /dev/sda1 29530400 3233104 24797232 12% / /dev/sda2 120367992 50171596 64082060 44% /home
df -h displays output in human readable form. i.e size will be displayed in GB’s.
ramesh@ramesh-laptop:~$ df -h Filesystem Size Used Avail Use% Mounted on /dev/sda1 29G 3.1G 24G 12% / /dev/sda2 115G 48G 62G 44% /home
Use -T option to display what type of file system.
ramesh@ramesh-laptop:~$ df -T Filesystem Type 1K-blocks Used Available Use% Mounted on /dev/sda1 ext4 29530400 3233120 24797216 12% / /dev/sda2 ext4 120367992 50171596 64082060 44% /home
26. kill command examples
Use kill command to terminate a process. First get the process id using ps -ef command, then use kill -9 to kill the running Linux process as shown below. You can also use killall, pkill, xkill to terminate a unix process.
$ ps -ef | grep vim ramesh 7243 7222 9 22:43 pts/2 00:00:00 vim $ kill -9 7243
27. rm command examples
Get confirmation before removing the file.
$ rm -i filename.txt
It is very useful while giving shell metacharacters in the file name argument.
Print the filename and get confirmation before removing the file.
$ rm -i file*
Following example recursively removes all files and directories under the example directory. This also removes the example directory itself.
$ rm -r example
28. cp command examples
Copy file1 to file2 preserving the mode, ownership and timestamp.
$ cp -p file1 file2
Copy file1 to file2. if file2 exists prompt for confirmation before overwritting it.
$ cp -i file1 file2
29. mv command examples
Rename file1 to file2. if file2 exists prompt for confirmation before overwritting it.
$ mv -i file1 file2
Note: mv -f is just the opposite, which will overwrite file2 without prompting.
mv -v will print what is happening during file rename, which is useful while specifying shell metacharacters in the file name argument.
$ mv -v file1 file2
30. cat command examples
You can view multiple files at the same time. Following example prints the content of file1 followed by file2 to stdout.
$ cat file1 file2
While displaying the file, following cat -n command will prepend the line number to each line of the output.
$ cat -n /etc/logrotate.conf
1 /var/log/btmp {
2 missingok
3 monthly
4 create 0660 root utmp
5 rotate 1
6 }
31. mount command examples
To mount a file system, you should first create a directory and mount it as shown below.
# mkdir /u01 # mount /dev/sdb1 /u01
You can also add this to the fstab for automatic mounting. i.e Anytime system is restarted, the filesystem will be mounted.
/dev/sdb1 /u01 ext2 defaults 0 2
32. chmod command examples
chmod command is used to change the permissions for a file or directory.
Give full access to user and group (i.e read, write and execute ) on a specific file.
$ chmod ug+rwx file.txt
Revoke all access for the group (i.e read, write and execute ) on a specific file.
$ chmod g-rwx file.txt
Apply the file permissions recursively to all the files in the sub-directories.
$ chmod -R ug+rwx file.txt
33. chown command examples
chown command is used to change the owner and group of a file.
To change owner to oracle and group to db on a file. i.e Change both owner and group at the same time.
$ chown oracle:dba dbora.sh
Use -R to change the ownership recursively.
$ chown -R oracle:dba /home/oracle
34. passwd command examples
Change your password from command line using passwd. This will prompt for the old password followed by the new password.
$ passwd
Super user can use passwd command to reset others password. This will not prompt for current password of the user.
# passwd USERNAME
Remove password for a specific user. Root user can disable password for a specific user. Once the password is disabled, the user can login without entering the password.
# passwd -d USERNAME
35. mkdir command examples
Following example creates a directory called temp under your home directory.
$ mkdir ~/temp
Create nested directories using one mkdir command. If any of these directories exist already, it will not display any error. If any of these directories doesn’t exist, it will create them.
$ mkdir -p dir1/dir2/dir3/dir4/
36. ifconfig command examples
Use ifconfig command to view or configure a network interface on the Linux system.
View all the interfaces along with status.
$ ifconfig -a
Start or stop a specific interface using up and down command as shown below.
$ ifconfig eth0 up $ ifconfig eth0 down
37. uname command examples
Uname command displays important information about the system such as — Kernel name, Host name, Kernel release number,
Processor type, etc.,
Sample uname output from a Ubuntu laptop is shown below.
$ uname -a Linux john-laptop 2.6.32-24-generic #41-Ubuntu SMP Thu Aug 19 01:12:52 UTC 2010 i686 GNU/Linux
38. whereis command examples
When you want to find out where a specific Unix command exists (for example, where does ls command exists?), you can execute the following command.
$ whereis ls ls: /bin/ls /usr/share/man/man1/ls.1.gz /usr/share/man/man1p/ls.1p.gz
When you want to search an executable from a path other than the whereis default path, you can use -B option and give path as argument to it. This searches for the executable lsmk in the /tmp directory, and displays it, if it is available.
$ whereis -u -B /tmp -f lsmk lsmk: /tmp/lsmk
39. whatis command examples
Whatis command displays a single line description about a command.
$ whatis ls ls (1) - list directory contents $ whatis ifconfig ifconfig (8) - configure a network interface
40. locate command examples
Using locate command you can quickly search for the location of a specific file (or group of files). Locate command uses the database created by updatedb.
The example below shows all files in the system that contains the word crontab in it.
$ locate crontab /etc/anacrontab /etc/crontab /usr/bin/crontab /usr/share/doc/cron/examples/crontab2english.pl.gz /usr/share/man/man1/crontab.1.gz /usr/share/man/man5/anacrontab.5.gz /usr/share/man/man5/crontab.5.gz /usr/share/vim/vim72/syntax/crontab.vim
41. man command examples
Display the man page of a specific command.
$ man crontab
When a man page for a command is located under more than one section, you can view the man page for that command from a specific section as shown below.
$ man SECTION-NUMBER commandname
Following 8 sections are available in the man page.
- General commands
- System calls
- C library functions
- Special files (usually devices, those found in /dev) and drivers
- File formats and conventions
- Games and screensavers
- Miscellaneous
- System administration commands and daemons
For example, when you do whatis crontab, you’ll notice that crontab has two man pages (section 1 and section 5). To view section 5 of crontab man page, do the following.
$ whatis crontab crontab (1) - maintain crontab files for individual users (V3) crontab (5) - tables for driving cron $ man 5 crontab
42. tail command examples
Print the last 10 lines of a file by default.
$ tail filename.txt
Print N number of lines from the file named filename.txt
$ tail -n N filename.txt
View the content of the file in real time using tail -f. This is useful to view the log files, that keeps growing. The command can be terminated using CTRL-C.
$ tail -f log-file
43. less command examples
less is very efficient while viewing huge log files, as it doesn’t need to load the full file while opening.
$ less huge-log-file.log
One you open a file using less command, following two keys are very helpful.
CTRL+F – forward one window CTRL+B – backward one window
44. su command examples
Switch to a different user account using su command. Super user can switch to any other user without entering their password.
$ su - USERNAME
Execute a single command from a different account name. In the following example, john can execute the ls command as raj username. Once the command is executed, it will come back to john’s account.
[john@dev-server]$ su - raj -c 'ls' [john@dev-server]$
Login to a specified user account, and execute the specified shell instead of the default shell.
$ su -s 'SHELLNAME' USERNAME
45. mysql command examples
mysql is probably the most widely used open source database on Linux. Even if you don’t run a mysql database on your server, you might end-up using the mysql command ( client ) to connect to a mysql database running on the remote server.
To connect to a remote mysql database. This will prompt for a password.
$ mysql -u root -p -h 192.168.1.2
To connect to a local mysql database.
$ mysql -u root -p
If you want to specify the mysql root password in the command line itself, enter it immediately after -p (without any space).
46. yum command examples
To install apache using yum.
$ yum install httpd
To upgrade apache using yum.
$ yum update httpd
To uninstall/remove apache using yum.
$ yum remove httpd
47. rpm command examples
To install apache using rpm.
# rpm -ivh httpd-2.2.3-22.0.1.el5.i386.rpm
To upgrade apache using rpm.
# rpm -uvh httpd-2.2.3-22.0.1.el5.i386.rpm
To uninstall/remove apache using rpm.
# rpm -ev httpd
48. ping command examples
Ping a remote host by sending only 5 packets.
$ ping -c 5 gmail.com
49. date command examples
Set the system date:
# date -s "01/31/2010 23:59:53"
Once you’ve changed the system date, you should syncronize the hardware clock with the system date as shown below.
# hwclock –systohc # hwclock --systohc –utc
50. wget command examples
The quick and effective method to download software, music, video from internet is using wget command.
$ wget http://prdownloads.sourceforge.net/sourceforge/nagios/nagios-3.2.1.tar.gz
Download and store it with a different name.
$ wget -O taglist.zip http://www.vim.org/scripts/download_script.php?src_id=7701
