linux – Page 3 – Piyush Gupta Blog

Find Public IP using Linux Command

Author : Piyush Gupta

Public IP is used for communication between computers over the Internet. A computer running with public IP is accessible all over the world using the Internet. So we can say that it is the identity of the computer on the internet. Now the question is how do we know our public IP?. For computers having GUI can easily get there IP using web tools but how to get public IP of the computers having terminal access only. The solution is here – use one of the following commands to find public IP of your system using Linux terminal. These are also useful to use in a shell script.

Find Public IP using Linux Command

Command 1 –

Use dig command to find your public IP address. The dig command is a DNS lookup utility for Linux systems to look up your public IP address by connecting to the OpenDNS servers.

dig +short myip.opendns.com @resolver1.opendns.com

Command 2 –

Use wget command to get your Public IP address as below example.

wget http://ipecho.net/plain -O - -q ; echo

Command 3,4,5 –

Use curl command to get your Public address.

curl ipecho.net/plain; echo

curl icanhazip.com

curl ifconfig.me

Get Public IP in Shell Script

We can simply use following commands in our shell script get our computers public IP and store them in a variable to use anywhere in a shell script.

#!/bin/bash

PUBLIC_IP=`wget http://ipecho.net/plain -O - -q ; echo`
echo $PUBLIC_IP

How to Remove Empty Lines from File in Linux

Author : Piyush Gupta

Some time we need to remove empty lines from a file. Its can be done manually if file have few lines but if file have thousands of line this is hard to be done manually. Use one of following method to remove empty lines from a file.

Method 1 – Using sed

Sed is an stream editor. We can easily remove all blank lines using sed command. Use one of following sed command to remove blank lines from file. For example main.txt is your original file from which you need to remove blank lines.

Below command will remove all blank line and save content in seconf file out.txt. It will not affect the original file.

# sed '/^$/d' main.txt > out.txt

Now if you want to make changes in original file using -i switch sed command.

# sed -i '/^$/d' main.txt

-i ( edit files in place ) Used for make changes in same file.

Method 2 – Using perl

Instead of sed, you can also use perl (a programming languege) to remove blank lines. Use the below example command to remove blank lines from main.txt file.

# perl -i -n -e "print if /S/" main.txt

Method 3 – Using awk

Also you can use AWK command line tool to remove blank lines from a file. For example use below command.

# awk 'NF > 0' main.txt > out.txt

How to use tar command in Linux

Author : Piyush Gupta

GNU TAR (Tape Archive) combines multiple files together into a single tape or disk archive, and can restore individual files from the archive. Here are some Linux tar command with Useful practical examples.

Some useful command line switches are given below, which are used in this article.

-c => create a new archive file
-v => show the detailed output (or progress) of command
-x => extract an archive file
-f => specify file name of an archive
-z => Use archive through gzip
-j => Use archive through bzip2
-J => Use archive through xz
-t => viewing content of archive
-O => Display file content on stdout
-r => append file to existing archive
-C => Use of define destination directory
-W => Verify a archive file

Creating Archive File

Use the following examples to create new archive file in different formats like .tar (a simple archive file), .tar.gz (gzip archive), .tar.bz2 (bzip2 archive file), and .tar.xz (xz archive file).

1. Create .tar archive file – Compress all content of /var/www directory to a archive.tar file including all subdirectories.

tar -cvf archive.tar /var/www

2. Create .tar.gz archive file – Compress all content of /var/www directory to a archive.tar.gz file including all subdirectories. This will create higher compressed file that above.

tar -zcvf archive.tar.gz /var/www

3. Create .tar.bz2 archive file – Compress all content of /var/www directory to a archive.tar.bz2 file including all subdirectories. This takes more time to compress than others and provides the highest compressed file that above.

tar -jcvf archive.tar.gz /var/www

4. Create .tar.xz archive file – Compress all content of /var/www directory to a archive.tar.xz file including all subdirectories. This takes more time to compress than others and provides the highest compressed file that above.

tar -Jcvf archive.tar.xz /var/www

Extract Archive File

Use the following commands example to extract archive files. In this section each example has two commands, the First command will extract content in the current directory and the second command will extract file content in the specified directory with -C option.

5. Extract .tar archive file –

tar -xvf archive.tar
tar -xvf archive.tar -C /tmp/

6. Extract .tar.gz archive file –

tar -zxvf archive.tar.gz
tar -zxvf archive.tar.gz -C /tmp/

7. Extract .tar.bz2 archive file –

tar -jxvf archive.tar.bz2
tar -jxvf archive.tar.bz2 -C /tmp/

8. Extract .tar.xz archive file –

tar -Jxvf archive.tar.xz
tar -Jxvf archive.tar.xz -C /tmp/

List Archive File Content

You can list all the content inside a tar (Tape ARchive) file without extracting it. It helps the user to check available files in an archive and save users time.

9. List .tar archive file content –

tar -tvf archive.tar

10. List .tar.gz archive file content-

tar ztvf archive.tar.gz

11. List .tar.bz2 archive file content-

tar jtvf archive.tar.bz2

12. List .tar.xz archive file content-

tar Jtvf archive.tar.xz

Update Archive File

You can use -u option to simply update archive file. Using this option, it will only append files newer than the copy in the archive file.

13. Update .tar archive file –

tar -uvf archive.tar /var/www

14. Update .tar.gz archive file –

tar -zuvf archive.tar.gz /var/www

15. Update .tar.bz2 archive file –

tar -juvf archive.tar.bz2 /var/www

16. Update .tar.xz archive file –

tar -Juvf archive.tar.xz /var/www

Other Useful Archive File Commands

Below are some more useful options available for handling tape archive files. Below examples are shown in the simple .tar file. You can use them with .tar.gz (-z option), .tar.bz2 (-j option) and .tar.xzf (-J option).

17. Display file content – use -O followed by filename, this will display content of specified file.

tar -uvf archive.tar -O backup/index.html

18. Adding file to archive – use -r followed by filename to add more files to existing archive file.

tar -rvf archive.tar add_new_file.txt

Wget Linux Command to Download Files

Author : Piyush Gupta

wget is Linux command line utility. wget is widely used for downloading files from Linux command line. There are many options available to download a file from remote server. wget works same as open url in browser window.

1: Download File using wget

Below example will download file from server to current local directory.

$ wget https://example.com/file.zip

2: Download File & Save to Specific Location

Below command will download zip file in /opt folder with name file.zip. -O is used for specify destination folder

# wget https://example.com/file.zip -O /opt/file.zip

3: Download File from FTP

Some time you required to download file from ftp server, so wget can easily download file from ftp url like below.

# wget ftp://ftp.example.com/file.zip

4: Download File from Password Protected URLs

Sometimes we required to specify username and password to download a file. While using browser its easy but using command line it doesn’t prompt for login credentials. Below examples will show to how to use username,password while downloading files from password protected sources.

4.1: Download file from Password protected ftp server.

$ wget --ftp-user=username --ftp-password=secretpassword ftp://ftp.example.com/file.zip

$ wget ftp://username:secretpassword@ftp.example.com/file.zip

4.2: Download file from password protected http server.

# wget --http-user=username --http-password=secretpassword https://example.com/file.zip

# wget --user=username --password=secretpassword https://example.com/file.zip

4.3: Download file behind password protected proxy server.

$ wget –proxy-user=username –proxy-password=secretpassword https://example.com/file.zip

5: Download File from Untrusted Secure URL.

If any download url is using untrusted ssl certificate, wget will not download that file. But we can download it by using –no-check-certificate parameter in url.

$ wget  https://example.com/file.zip  --no-check-certificate

How to Use zip Command in Linux

Author : Piyush Gupta

The zip command is used for compression and file packaging under Linux/Unix operating systems. unzip is used for decompress an archive. See the below examples of some typical uses of zip and unzip.

1 – Zip All Files in Directory

This command will create zip of all files in /backup directory. I will not create zip recursively.

$ zip backup.zip /backup/*

Sample Output:

adding: backup/anaconda.ifcfg.log (deflated 47%)
adding: backup/anaconda.log (deflated 78%)
adding: backup/anaconda.program.log (deflated 84%)
adding: backup/anaconda.storage.log (deflated 90%)
adding: backup/boot.log (deflated 72%)
adding: backup/dracut.log (deflated 92%)
adding: backup/httpd/ (stored 0%)
adding: backup/kadmind.log (deflated 74%)
adding: backup/krb5kdc.log (deflated 71%)
adding: backup/mysqld.log (deflated 82%)

2 – Zip files with Wildcard

Use Linux wildcards to archive files of specific extensions only. Like backup only .log extension files in a directory.

$ zip backup.zip /backup/*.log

Sample Output:

adding: backup/anaconda.ifcfg.log (deflated 47%)
adding: backup/anaconda.log (deflated 78%)
adding: backup/anaconda.program.log (deflated 84%)
adding: backup/anaconda.storage.log (deflated 90%)
adding: backup/boot.log (deflated 72%)
adding: backup/dracut.log (deflated 92%)
adding: backup/kadmind.log (deflated 74%)
adding: backup/krb5kdc.log (deflated 71%)
adding: backup/mysqld.log (deflated 82%)
adding: backup/pm-powersave.log (deflated 15%)
adding: backup/wpa_supplicant.log (stored 0%)
adding: backup/Xorg.0.log (deflated 83%)
adding: backup/Xorg.9.log (deflated 83%)
adding: backup/yum.log (deflated 77%)

3 – Zip files Recursively

Below command will create an archive recursively with files in sub directories also.

$ zip -r backup.zip /backup

4 – Create Password Protected Zip

Some times we need to create password protected archive. Use -p to make an archive password protected.

$ zip -P backup.zip /backup/*.log

5 – Zip with Compression Levels

Zip command provides 10 levels of compression ( 0-9 ).

-6 is used as default compression level.

-0 is used for lowest level compression.

-9 is used for hightest level comression

$ zip -9 high-compressed-file.zip /backup/*
$ zip -0 lowest-compressed-file.zip /backup/*

Check differences between compressed file

$ ls -lh lowest-compressed-file.zip high-compressed-file.zip

-rw-r--r--. 1 root root 50K Apr 11 14:14 high-compressed-file.zip
-rw-r--r--. 1 root root 447K Apr 11 14:14 lowest-compressed-file.zip

You can see the difference between between both file sizes.

6 – List content of zip File

Using -l switch with unzip command to list only files inside a zip archive without decompressing it.

$ unzip -l backup.zip

Sample Output:

Archive: backup.zip
Length Date Time Name
--------- ---------- ----- ----
140 04-11-2013 14:07 backup/anaconda.ifcfg.log
11153 04-11-2013 14:07 backup/anaconda.log
15446 04-11-2013 14:07 backup/anaconda.program.log
136167 04-11-2013 14:07 backup/anaconda.storage.log
2722 04-11-2013 14:07 backup/boot.log
211614 04-11-2013 14:07 backup/dracut.log
0 04-11-2013 14:08 backup/httpd/
1382 04-11-2013 14:07 backup/kadmind.log
1248 04-11-2013 14:07 backup/krb5kdc.log
6485 04-11-2013 14:07 backup/mysqld.log
87 04-11-2013 14:07 backup/pm-powersave.log
0 04-11-2013 14:07 backup/wpa_supplicant.log
30186 04-11-2013 14:07 backup/Xorg.0.log
31094 04-11-2013 14:07 backup/Xorg.9.log
6739 04-11-2013 14:07 backup/yum.log
--------- -------
454463 15 files

7 – Extract a Zip File.

unzip command is used to extract a zip file. Use below command to simply extract a zip file.

$ unzip backup.zip

8 – Check an archive file

Use -t to check and archive files. This option extracts each specified file in memory and compares the CRC (cyclic redundancy check, an enhanced checksum) .

$ unzip -t backup.zip

Sample Output:

 Archive: backup-11Apr2013.zip
testing: backup/anaconda.ifcfg.log OK
testing: backup/anaconda.log OK
testing: backup/anaconda.program.log OK
testing: backup/anaconda.storage.log OK
testing: backup/boot.log OK
testing: backup/dracut.log OK
testing: backup/httpd/ OK
testing: backup/kadmind.log OK
testing: backup/krb5kdc.log OK
testing: backup/mysqld.log OK
testing: backup/pm-powersave.log OK
testing: backup/wpa_supplicant.log OK
testing: backup/Xorg.0.log OK
testing: backup/Xorg.9.log OK
testing: backup/yum.log OK
No errors detected in compressed data of backup.zip.

Linux Find Command Examples

Author : Piyush Gupta

find is a Linux command line tool to search files and directories in the file system. The find command works much fast than any other command. It provides a large number of options for more specific search. It also supports wildcard characters.

Every Linux user must read this article and understand the uses of the find command. This command is very useful in your daily tasks. This article will help you to understand find command and its uses in Linux system.

Syntax: To search a file or directory under specified filesystem.

find /search/in/dir -name filename

Explanation:
find => command line tool
/search/in/dir => Directory name where to start search
-name => Switch to specify filename
filename => File or Directory name

Find files by Name

Use -name option to find a file with name “hello.txt” under root (/) file system.

find / -name hello.txt

Find files by Type

Search for the file (not directory) named “backup.zip” in entire file system. Use -type f to specify search for files and ignore direcories.

find / -type f -name backup.zip

Search directory only

Search for the directory (not file) named “backup” in entire file system. Use -type d to specify search for directory and ignore files.

find / -type d -name backup

Find files by Size

Search all files systemwide, which are greater than or equal 10MB with find command

find / -type f -size +10M

And this command will search all files in system which are less than 10MB.

find / -type f -size -10M

-size: switch are used for searching file on bais of there size. Plus ( + ) is used to greater than size and minus ( – ) sign is used for less than size.
like: +100MB, -50KB, +1GB etc…

Find files by Time

Search all files which modification time is more than 30 days.

find / -type f -mtime +30

Search all files which modification time is less than 30 days.

find / -type f -mtime -30

Find files by User/Group

Find command also provides search based on user and group ownership. like:

Search all .txt files own by user bob.

find  / -user bob -name "*.txt"

Search all .txt files with group ownership of root.

find  / -group root -name "*.txt"

You can combine both to more specific search to files with owner bob and group ownership of the root.

find  / -user bob -group root -name "*.txt"

Find files by Permissions

Search all files which are having 777 permissions under /var/www directory tree. This can be helpful for the security audit.

find . -perm 777

Find files by Inode Number

This command is used to search files on basis of their inode number. -inum is used for this search.

find / -inum 1532

If you want check inode number of a file using below command. The first field of output is an inode number

ls -li piyush.txt

30878 -rw-r--r--. 1 root root 0 Mar 22 17:20 piyush.txt

Find Empty files

This command is very useful to search empty files and directories. It is useful to the cleanup system with empty files and directories.

$ find / -empty

Find files by File Types

Search all block special files available under / filesystem.

find / -type b

Other file type options are as below:

b – block (buffered) special
c – character (unbuffered) special
d – directory
p – named pipe (FIFO)
f – regular file
s – socket
l – symbolic link; this is never true if the -L option or the -follow option is in effect unless the symbolic link is broken. If you want to search for symbolic links when -L is in effect, use -xtype.

One Time Task Scheduling using at Command in Linux

Author : Piyush Gupta

While working on Linux systems we preferred crontab for scheduling jobs generally. There are another utility at command is very useful for scheduling one time tasks. It reads commands from standard input or script/file which can be executed later once. But we can’t use at command for any recurring tasks. For recurring tasks use Linux crontab.

At command can be useful for shutdown system at the specified time, Taking a one-time backup, sending email as a reminder at the specified time etc. This article will help you to understand the working of at command with useful examples.

Commands used with at:

at : execute commands at specified time.
atq : lists the pending jobs of users.
atrm : delete jobs by their job number.

1. Schedule first job using at command

Below example will schedule “sh backup.sh” command to be executed on next 9:00 AM once.

at 9:00 AM
at> sh backup.sh
at> ^d
job 3 at 2019-03-23 09:00

Use ^d to exit from at prompt.

You can also use the following option to schedule a job. The below command will run “sh backup.sh” at 9:00 in the morning.

echo "sh backup.sh" | at 9:00 AM

2. List the scheduled jobs using atq

When we list jobs by root account using atq, it shows all users jobs in the result. But if we execute it from a non-root account, it will show only that users jobs.

atq
3       2019-03-23 09:00 a root
5       2019-03-23 10:00 a piyush
1       2019-03-23 12:00 a root

Fields description:

First filed: job id
Second filed: Job execution date
third filed: Job execution time
Last field: User name, under which job is scheduled.

3. Remove scheduled job using atrm

You can remove any at job using atrm with their job id.

atrm 3
atq
5       2019-03-23 10:00 a piyush
1       2019-03-23 12:00 a root

4. Check the content of scheduled at job

atq command only shows the list of jobs but if you want to check what script/commands are scheduled with that task, below example will help you.

at -c 5

In the above example, 5 is the job id.

Examples of at Command:

Example 1: Schedule task at coming 10:00 AM.

# at 10:00 AM

Example 2: Schedule task at 10:00 AM on coming Sunday.

at 10:00 AM Sun

Example 3: Schedule task at 10:00 AM on coming 25’th July.

at 10:00 AM July 25

Example 4: Schedule task at 10:00 AM on coming 22’nd June 2015.

at 10:00 AM 6/22/2015at 10:00 AM 6.22.2015

Example 5: Schedule task at 10:00 AM on the same date at next month.

at 10:00 AM next month

Example 6: Schedule task at 10:00 AM tomorrow.

at 10:00 AM tomorrow

Example 7: Schedule task at 10:00 AM tomorrow.

at 10:00 AM tomorrow

Example 8: Schedule task to execute just after 1 hour.

at now + 1 hour

Example 9: Schedule task to execute just after 30 minutes.

at now + 30 minutes

Example 10: Schedule task to execute just after 1 and 2 weeks.

at now + 1 weekat now + 2 weeks

Example 11: Schedule task to execute just after 1 and 2 years.

at now + 1 yearat now + 2 years

Example 12: Schedule task to execute at midnight.

at midnight

The above job will execute on next 12:00 AM

Thanks for reading this article, We hope you will understand how to use ‘at’ command in Linux.

How to Delete Files Older than 30 days in Linux

Author : Piyush Gupta

This is the best practice to remove old unused files from your server. For example, if we are running daily/hourly backup of files or database on the server then there will be much junk created on the server. So clean it regularly. To do it you can find older files from the backup directory and clean them. This article will help you to find files older than 30 days.

1. Delete Files Older Than 30 Days

This command will delete all files older than 30 days in system /opt/backup directory.

find /opt/backup -type f -mtime +30 -exec rm -f {} \;

2. Delete Files Older Than 30 Days with .log Extension

If you want to delete only specified extension files, you can use the following command.

find /var/log -name "*.log" -type f -mtime +30 -exec rm -f {} \;

Above command will delete only files having .log extension.

Basic Linux Commands for daily usages

Author : Piyush Gupta

This blog will explore the basic Linux commands and how to use them.

ls: How would we know what a folder contains? With a graphical interface, you’d do this by opening a folder and inspecting its contents. From the command line, you use the command ls instead to list a folder’s contents.
By default, ls will use a very compact output format. Many terminals show the files and subdirectories in different colors that represent different file types. Regular files don’t have special coloring applied to their names. Some file types, like JPEG or PNG images, or tar and ZIP files, are usually colored differently, and the same is true for programs that you can run and for directories. Try ls for yourself and compare the icons and emblems your graphical file manager uses with the colors that ls applies on the command line. If the output isn’t coloured, you can call ls with the option –color:
```
$ ls --color
```

man: You can learn about options and arguments to be used in any command in Linux. man(man is short for manual) is used to give description of any linux command like this:
```
$ man ls
```
Here, man is being asked to bring up the manual page for ls. You can use the arrow keys to scroll up and down in the screen that appears and you can close it using the q key (for quit).

info: An alternative to obtain a comprehensive user documentation for a given program is to invoke info instead of man:
```
$ info ls
```
This is particularly effective to learn how to use complex GNU programs. You can also browse the info documentation inside the editor Emacs, which greatly improves its readability. But you should be ready to take your first step into the larger world of Emacs. You may do so by invoking:
```
$ emacs -f info-standalone 
```
that should display the Info main menu inside Emacs (if this does not work, try invoking emacs without arguments and then type Alt + x info, i.e. by pressing the Alt key, then pressing the x key, then releasing both keys and finally typing info followed by the Return or Enter key). If you type then m ls, the interactive Info documentation for ls will be loaded inside Emacs. In the standalone mode, the q key will quit the documentation, as usual with man and info.

apropos: If you don’t know what something is or how to use it, the first place to look is its manual and information pages. If you don’t know the name of what you want to do, the apropos command can help. Let’s say you want to rename files but you don’t know what command does that. Try apropos with some word that is related to what you want, like this:
```
$ apropos rename
...
mv (1)               - move (rename) files
prename (1)          - renames multiple files 
rename (2)           - change the name or location of a file
...
```
Here, apropos searches the manual pages that man knows about and prints commands it thinks are related to renaming. On your computer this command might (and probably will) display more information but it’s very likely to include the entries shown.

mv: The mv command is used to move or rename files.
```
$ mv oldname newname
```
Depending on your system configuration, you may not be warned when renaming a file will overwrite an existing file whose name happens to be newname. So, as a safe-guard, always use `-i’ option when issuing mv like this:
```
$ mv -i oldname newname
```
If the last argument happens to be an existing directory, mv will move the file to that directory instead of renaming it. Because of this, you can provide mv more than two arguments:
```
$ mv first_file second_file third_file ~/stuff 
```
If ~/stuff exists, then mv will move the files there. If it doesn’t exist, it will produce an error message, like this:
```
$ mv first_file second_file third_file ~/stuff 
mv: target 'stuff' is not a directory
```

mkdir: mkdir command is used to create a subdirectory in your current working directory type.
```
$ mkdir practice
```
To see the directory practice you have just created, type ls. If you wish to create a subdirectory (say the directory bar) inside another directory (say the directory foo) but you are not sure whether this one exists or not, you can ensure to create the subdirectory and (if needed) its parent directory without raising errors by typing:
```
$ mkdir -p ~/foo/bar  
```
This will work even for nested sub-sub-…-directories.

cd: The command cd directory means change the current working directory to ‘directory’. The current working directory may be thought of as the directory you are in, i.e. your current position in the file-system tree.
To change to the directory you have just made, type:
```
$ cd practice
```
Now, if you go back to your home directory, type
```
$ cd ..
```
NOTE: there is a space between cd and the dot

rmdir: Now, that you are in the home directory and try to remove the directory called practice, rmdir will produce an error message:
```
$ cd ..
$ rmdir practice
rmdir: failed to remove 'practice': Directory not empty
```
If the directory you wish to remove is not empty, rmdir will produce an error message and will not remove it. If you want to remove a directory that contains files, you have to empty it.

rm: rm removes each specified file like this:
```
$ rm practice/fstab practice/hosts practice/issue practice/mod
```
And now you can try removing the directory again:
```
$ rmdir practice
```
And now it works, without showing any output.
But what happens if your directories have directories inside that also have files, you could be there for weeks making sure each folder is empty! The rm command solves this problem through the amazing option -R, which as usual stands for “recursive”. In the following example, the command fails because foo is not a plain file:
```
$ rm ~/foo/ 
rm: cannot remove `~/foo/`: Is a directory
```
So maybe you try rmdir, but that fails because foo has something else under it:
```
$ rmdir ~/foo
rmdir: ~/foo: Directory not empty
```
So you use rm -R, which succeeds and does not produce a message.
```
$ rm -R ~/foo/
```
So when you have a big directory, you don’t have to go and empty every subdirectory.
But be warned that -R is a very powerful argument and you may lose data you wanted to keep!

cat: You don’t need an editor to view the contents of a file. What you need is just to display it. The cat program fits the bill here:
```
$ cat myspeech.txt
Friends, Coders, Linux-lovers! This is an article in GeeksForGeeks.
```

less: Here, cat just opens the file myspeech.txt and prints the entire file to your screen, as fast as it can. However if the file is really long, the contents will go by very quickly, and when cat is done, all you will see are the last few lines of the file. To just view the contents of a long file (or any text file) you can use the less program:
```
$ less myspeech.txt
```
Just as with using man, use the arrow keys to navigate, and press q to quit.

Note: we took references from linuxcommand.org