Kavoir » Linux Server Administration Tips

Linux: How to delete / remove hidden files with ‘rm’ command?

Yang Yang — Sat, 12 Feb 2011 09:41:35 +0000

To delete all content in any directory, including all sub-directories and files, I’ve been using this:

rm -rf somedir/*

If it is to delete all content of the current directory:

rm -rf *

However, it turns out ‘rm -rf’ doesn’t remove hidden files such as .htaccess (Files with a name starting with a dot are hidden in Linux). To delete all the hidden files as well, I have to run a 2nd command:

rm -rf .??*

Linux: How to ‘find’ and search ONLY text files?

Yang Yang — Sat, 22 Jan 2011 13:30:25 +0000

The ‘find’ command in Linux systems searches through a directory and return files that satisfy certain criteria. For instance, to find a file that contains the string ‘needle text’ in the ‘mydocs’ directory:

find mydocs -type f -exec grep -l "needle text" {} \;

The problem of this approach is that it would search through ALL files in this directory including the binary ones such as images, executables and zip packages. Sensibly, we would only want to search through text files for a specific string. If there are far too many of binary files in present, it’d be a significant waste of CPU usage and time to get what you want, because it’s totally unnecessary to go through the binary files.

To achieve this, use this version of the above command:

find mydocs -type f -exec grep -l "needle text" {} \; -exec file {} \; | grep text | cut -d ':' -f1

I asked the question at stackoverflow.com and peoro came up with this solution. It works great.

Basically, the bold part checks each file’s mime type information and only searches the files that have ‘text’ in its mime type description. According to the Linux ‘file’ command manual, we can be fairly sure that files with ‘text’ in its mime type string are text files AND all text files have ‘text’ in its mime type description string.

Use stat command to display file system meta information of any file or directory under Linux

Yang Yang — Fri, 19 Feb 2010 16:34:41 +0000

PHP has a stat() function that returns an array containing the meta information of a file such as owner, size, time of last access, last modification or last change. It’s basically the stat command under Linux that returns and shows the file system meta information of any file or directory:

stat myfile.txt

Which returns:

  File: `myfile.txt'
  Size: 1707            Blocks: 8          IO Block: 4096   regular file
Device: 811h/2065d      Inode: 96909802    Links: 1
Access: (0644/-rw-r--r--)  Uid: (1354144/    voir)   Gid: (255747/pg940032)
Access: 2010-02-16 08:00:00.000000000 -0800
Modify: 2010-02-18 04:16:51.000000000 -0800
Change: 2010-02-18 04:16:51.000000000 -0800

To get the meta information of the current working directory:

stat .

Which returns:

  File: `.'
  Size: 4096            Blocks: 8          IO Block: 4096   directory
Device: 811h/2065d      Inode: 96904945    Links: 4
Access: (0755/drwxr-xr-x)  Uid: (1354144/    voir)   Gid: (255747/pg940032)
Access: 2009-08-31 17:07:16.000000000 -0700
Modify: 2009-12-20 05:18:57.000000000 -0800
Change: 2009-12-20 05:18:57.000000000 -0800

Linux: How to open and extract an RAR zipped file and unrar the archive?

Yang Yang — Sat, 19 Sep 2009 03:29:32 +0000

Funny I should use “zipped” for an RAR compressed package. Anyway, you can easily zip or unzip a zip file or tar compress a package, but how does one do it with an RAR file? WinRAR is well distributed across all Windows systems. But on Linux, you have to first install the command package rar.

However, if your host has been around for quite some time such as DreamHost, you will not need to install it yourself as it’s come with the system. Just fire up this command to unrar any RAR archives:

rar x myfiles.rar

Which will then extract all the data from myfiles.rar into the current working directory.

There are other commands you can rely on to achieve the same task though, depending on your host and the server distribution. For example, you may have unrar instead of rar. Other than these, you may also find RAR related installation packages on Debian and Ubuntu by:

aptitude search unrar

It will search and show you related available packages:

p   unrar-free                           - Unarchiver for .rar files

Which is another utility to unrar any RAR files on Linux. Just install it by aptitude install unrar-free and use to unpackage the compressed RAR.

scp, rsync: Safely and Quickly Transfer Files between Your Hosts / Hosting Accounts / Servers on SSH tunnel

Yang Yang — Fri, 18 Sep 2009 08:37:34 +0000

Chances are you have a bunch of different hosts that are housing your website files, for the sake of data safety (never put all eggs in a single basket) and possibly some SEO advantage. If that is the case, you will infrequently come to the need to move some files from one host server to another. How does one do that?

Well the straight answers include downloading the files from the source host and then uploading it to destination one via FTP. It’s not much of a time-waster with small number of files, especially those small in size. However, if it’s an impressively large chunk of package, say, 4GB, or thousands of files, this’d be quite a daunting job that may very well take the better part of your day or even a few days.

The shortcut is to transfer those files directly from the original host to the other, via SSH. That is of course, if you have both hosts enabled with SSH.

scp Command

Log into the destination host via SSH and try the following command:

scp -r remoteuser@remote.host.com:/home/remoteuser/dir-to-be-transferred/. /home/localuser/backup

Wherein remote.host.com is the address of the source host and remoteuser is the SSH user (shell user) account that can read the remote directory to be transferred, namely /home/remoteuser/dir-to-be-transferred. The last argument is the local path that’s receiving the incoming files / directory.

The dot at the end of dir-to-be-transferred makes sure that all hidden files such as .htaccess are copied as well. Without the current directory sign (dot), hidden files are NOT copied by default.

You can also transfer a specific file:

scp remoteuser@remote.host.com:/home/remoteuser/mybackup.tar.gz /home/localuser/backup

As a matter of fact, scp works the exactly same way as an ordinary cp command except it’s able to copy files back and forth remote hosts. The “s” of “scp” stands for safe, because all the data transferred is encrypted on SSH.

It’s a great way to back up your valuable website data across multiple different hosts that are physically far away from each other. With the help of crontab jobs that do the regular backups automatically, this is even better than some of the commercial backup services.

rsync Command

The command of rsync is a more preferable option to scp for synchronizing stuff across different hosts because it compares differences and works incrementally, thus saving bandwidth, especially with large backups. For examples,

rsync -av --progress remoteuser@remote.host.com:/home/remoteuser/dir-to-be-transferred /home/localuser/backup

This would copy and transfer the directory dir-to-be-transferred with all its content into backup so that dir-to-be-transferred is a sub-directory of backup.

rsync -av --progress remoteuser@remote.host.com:/home/remoteuser/dir-to-be-transferred/. /home/localuser/backup

With an extra /. at the end of the source directory, only the content of the directory dir-to-be-transferred are copied and transferred into backup. Thus all the content of the directory dir-to-be-transferred are now immediate children of backup.

To specify the SSH port, such as 8023, just add:

 --rsh='ssh -p8023'

rsync automatically takes care of all hidden files, so there’s no need to add a dot at the end of the source directory.

Linux: Find files changed or modified within 1 day or older than 1 day

Yang Yang — Wed, 09 Sep 2009 09:08:10 +0000

One of the utility commands of Linux that you should know in the first day of your Linux learning seminar is find.

To search recursively in the directory somedir for files changed / created / modified within 1 day:

find somedir -ctime -1

To search recursively in the directory somedir for files changed changed / created / modified more than 1 days ago:

find somedir -ctime +1

To search recursively in the directory somedir for files modified within 1 day:

find somedir -mtime -1

To search recursively in the directory somedir for files modified more than 1 days ago:

find somedir -mtime +1

Linux: Change Directory or CD to the Previous Directory / Last Path

Yang Yang — Sat, 05 Sep 2009 01:54:11 +0000

cd is the command in Linux to change the current working directory. While you can change to your home directory by cd ~, you can change to the previous directory or last directory you were in by:

cd -

Which would come very handy when you are working across multiple directories back and forth. To change to the second last directory in the path history, simply add a slash:

cd --

Linux: Check how much disk storage each directory takes up (Disk Usage command – du)

Yang Yang — Wed, 02 Sep 2009 10:00:56 +0000

The Linux command du stands for disk usage which is used to check the amount of disk storage any particular directory or file is using. By default, the simple command:

du

Would return the disk usage in God-knows-what-unit of each of the directories in the current working directory and those beneath them – in a recursive manner. If you happen to have lots of them, the returned stats would be scrolling down crazily which barely makes it any useful.

Even if you have specified a specific directory such as "somedir":

du somedir

It still works in this uncomfortable way.

The solution is to use the -sh switch, the one switch a beginner will ever need:

du -sh

Which simply returns the amount of disk space the current directory and all those stuff in it are using as a whole, something like:

2.4G

Much much more intuitive and readable.

By:

du -sh somedir

You can find out how much disk storage directory "somedir" is using:

101M    somedir

To get all the subsequent / child directories disk usage from the current directory, simply use the asterisk:

du -sh *

It will then list the disk usage of all of them (but not recursively) one by one in a very readable manner:

8.0K    dir1
1.4G    dir2
135M    dir3

Linux: How to find all the files containing a particular text string?

Yang Yang — Thu, 27 Aug 2009 09:20:50 +0000

At Linux command line, to find a particular text string in all the files from the current directory recursively (that is, including all those files from the child or grandchild directories), use something like this via SSH:

find . -exec grep -l "needle" {} \;

This command searches through all directories from the current directory recursively for the files that contain the string “needle”.

To search only .php files:

find *.php -exec grep -l "needle" {} \;

To search a specific directory:

find dirname -exec grep -l "needle" {} \;

To find all .php files that contain “needle” in all the directories under “somedir”:

find somedir -name *.php -exec grep -l "needle" {} \;

Linux wget Command to Download and Mirror a Website in Static Local Copy

Yang Yang — Sun, 02 Aug 2009 08:46:57 +0000

wget is such a wonderful command in Linux you can ever get. Consider blessed to have it in your SSH arsenal. Now, not only does it allow you to download something neatly from the command line to the current working directory – that’s why WordPress always puts the latest version of the blog script at http://www.wordpress.org/latest.zip, that way you don’t even have to visit the site before using wget to get it in no hassle:

wget http://www.wordpress.org/latest.zip

And the latest version of WordPress will be in your local working directory in no time – but it can also help you to create a local mirrored copy of a remote website for backup or for local browsing.

How to Make Mirrors from a Site with wget command?

wget comes equipped with so many useful switches and features and one of them is to mirror stuff off the Internet, from online to offline. For example, you have a blogspot blog and you want a local copy of the entire blog files / web pages, including all those CSS style sheets, images and scripts. No problem, just go with

wget -mk http://yours.blogspot.com

Don’t worry, wget will create a new directory named by the domain, in this case, "yours.blogspot.com", to store all those files. After screens and screens of real-time logging, when it’s done, the local mirror should be ready to use.

The –m switch is short for –mirror and the –k switch translates all absolute URLs in the HTML into relative ones for totally offline browsing.

The problem is that wget works much better with sites using sanitized / clean URLs such as a WordPress blog. Though a blogspot blog is basically in static HTML, it contains quite some complicated URLs here and there that’s filled with parameters. The result doesn’t look too pretty.

However, after trying wget to recursively download and mirror a WordPress blog, it looks fantastic. Try it with a permalink enabled WordPress blog and it will surely amaze you.

It’s best used to generate mirrors of websites completely built in static files or sites with simple URL schemes.

How to Recover or Reset MySQL root Password after You Forgot and Lost It

Yang Yang — Sat, 20 Jun 2009 15:16:50 +0000

MySQL has come with a safe mode wherein access privileges are not checked, which essentially enables you to log in anonymously to change anything in any database.

And we are going to get in this safe mode to reset the lost MySQL root password.

First, you need to stop the current MySQL daemon by:
/etc/init.d/mysql stop
Then you can start the safe mode of MySQL and skip privileges check by:
mysqld_safe --skip-grant-tables &
The & in the end gets it to run in the background so we can continue to reset the root password.
Now you can log into the MySQL server anonymously (well not anonymously but without privilege check):
mysql --user=root mysql

And reset the root password by updating the user table of the mysql database:

UPDATE user SET Password=PASSWORD('newrootpwd') WHERE user='root';
FLUSH PRIVILEGES;

Get out of the safe mode and restart the normal MySQL daemon:
/etc/init.d/mysql restart
At last, you should be able to connect to the database server by your new root password.

As always, you will have to log in as root of the entire server or you won’t be able to stop the MySQL daemon and start the safe mode in the first place.

Note that I’m recovering the root password of MySQL in Ubuntu 9.04 Jaunty with MySQL installed by aptitude. Specific procedures may vary distribution by distribution but I’m sure the fundamental logic remains the same: getting in the safe mode to reset the root password.

Linux, PHP: Differences between File Modification Time: filemtime() and File Change Time: filectime()

Yang Yang — Tue, 02 Jun 2009 14:45:58 +0000

In most Unix file systems, the modification time and the change time of a file may not necessarily be the same, they are actually 2 very distinct concepts to deal with:

File modification time represents when the data blocks or content are changed or modified, not including that of meta data such as ownership or ownergroup.
File change time represents the time when the meta data or inode data of a file is altered, such as the change of permissions, ownership or group.

Accordingly, PHP function filemtime() returns the time the content is modified or updated while function filectime() returns the time when the meta data is updated or changed.

When you need to display a web page last modification time, for example, you should use filemtime() because you want to show the time when the page content is updated.

Where is php.ini located?

Yang Yang — Mon, 01 Jun 2009 14:30:01 +0000

Well it depends on the Linux distribution you are using, the version of php and the way you install it with Apache web server. Php.ini may be here:

/etc/php.ini

Or here:

/etc/php/php.ini /etc/php5/php.ini

Or here:

/usr/bin/php5/bin/php.ini

Anyway, you can always find any file named php.ini in this manner

find / -name php.ini

The simplest yet most powerful usage of the renowned find command. By its help I was able to locate the php.ini on my Ubuntu 9.04 Apache2 and PHP5:

/etc/php5/apache2/php.ini

Use phpinfo() to find out

Write a simple PHP file with the following content:


And upload it to your server and access it in your web browser to get a page like this:

	
	PHP configuration file php.ini path


Related Posts:

Linux: How to open and extract an RAR zipped file and unrar the archive?
PHP: open_basedir in php.ini to Restrict and Limit PHP File Accesses to a Certain Directory
CSS: How to write CSS rules to detect or target Chrome, Safari or Opera browsers only?
How to execute / run PHP scripts in the background?
What is this Linux command?



Accidental Ctrl+S Locks and Freezes Linux Terminal / SSH / Telnet
Yang Yang — Tue, 26 May 2009 02:54:40 +0000
Ctrl+S happens to be a rather handy and popular combination as it’s used in Windows applications to save your current working data. I accidentally used it several times in Vim and it keeps locking the screen up and halting the interactivity – basically, after Ctrl+S I can’t do anything to the terminal (window). I use PuTTY in Windows to access my Linux hosting via SSH.

A few lookups in Linux manuals reveal the secrets, the Ctrl+S key combination temporarily freezes the screen but the input is recorded though not dealt with. Once you press Ctrl+Q, it is unlocked and all the flood of input you have provided to the terminal takes immediate effect.
So, don’t panic when you accidentally pressed Ctrl+S and seemingly freezes everything, just press Ctrl+Q to exit it.
Related Posts:

JavaScript: Confirmation / Warning before Leaving or Navigating Away from a Page
4 Tips for Zend Studio 8 (Eclipse)
What is this Linux command?
A Simple PHP Contact Form Script
MySQL: How to export a database / table to XML?





How to change Vim syntax highlighting colors?
Yang Yang — Tue, 26 May 2009 02:35:38 +0000
Unbuntu has auto-configured Vim to use syntax highlighting for text (mostly, programs and configuration files of course) editing, the problem however, is that some of the colors appear to be darker than wanted on SSH console and it’s a little hard to recognize comfortably.

So how can we change the default colors of the syntax highlighting?

Locate Vim configuration file which is located at /etc/vim/vimrc, add a line after ‘syntax on’: colorscheme desert 
Now that we are prescribing Vim to use the syntax color scheme desert, we will want to find the scheme file to change the colors that we don’t feel comfortable with. The desert color scheme file is located at: /usr/share/vim/vimcurrent/colors/desert.vim 
Proceed to open it with vi and replace the color values with new ones. For example, the default comment color may be a little too dark on a black screen, let’s make it lighter by changing      
 
hi Comment      ctermfg=darkcyan
to 
hi Comment      ctermfg=blue

Reload the vimrc configuration file by performing ‘source /etc/vim/vimrc’ at shell. 

Related Posts:

Colorful Linux Bash Console by .alias and .bash_profile
HTML: Change Text and Font Colors
How to bring down / optimize memory usage in your unmanaged Linux VPS box and avoid OOM (Out Of Memory) errors?
Customize WordPress Post Editor CSS Styles
MySQL: Change Default Character Set or Default Collation in phpMyAdmin