Posted on

Linux How to Use du Command to List Files, Directories, and their total sizes

We know we can use ls command to list out all the files and directories under a given directory.

But it doesn’t give us size information. Sometimes, size information is important, for example we are examining a code repository, and need to find out and clean up large files and directories.

Linux List Files, Directories, and Their Total Sizes

du command, short for disk usage, can be useful in case like this. It is used to estimate file space usage.
To list all files, directories, and their total sizes, first cd to the directory, and run the command below.

It will give us an output like below:

List Files, Directories, and their total sizes

Linux List Total Size of Current Directory

To simply list the size of current directory, first cd to the directory, and run the command below.

It will calculate the current directory size:

Linux List Total Size of Current Directory

Linux List Files, Directories, and Their Total Sizes in Descending Order

We’ve already known how to list the size for all files and directories in a certain directory, but can we take one more step further to sort the results.

Then answer is YES! See the command below:

We will need to drop the -h argument, since it will give you human readable output, e.g. 10K, 2.5MB, and they cannot be sorted when we pipe the output to sort command.

Linux List Files, Directories, and Their Total Sizes in Descending Order

Linux Find the Top 10 Largest Files or Directories in Current Directory

We can pass the output from previous command to a head command to accomplish this.

Linux List the Top 10 Largest Files or Directories

Posted on

How to Find and Remove Hanging Cron Jobs in Magento

Recently, one of our Magento client reported that their website stopped sending out order confirmation. However, invoice and shipment emails seemed to work with no issue. After an investigation, we discovered their site has caused a hanging Cron Jobs.

How to Find and Remove Hanging Cron Jobs in Magento

Starting from version 1.9.1, order confirmation emails are not sent directly during checkout, instead, they are queued in Magento, and then send out later by execution of Cron Jobs. A hanging Cron Job will cause email send stopped.

Check if Magento Cron Jobs is Hanging

Log into the server and run the following command:

As the screenshot below, you can see a couple lines of outputs

Magento Hanging Cron Jobs

The first line is the Cron daemon process running on this server.

The second line shows the command we just ran.

The third and fourth lines are Magento Cron Jobs that are currently being executed.

Furthermore, take a look at the second column from the right, that is the execution time of the process. Magento Cron Jobs usually runs real quick. So if it is caught by our ps command, and showing a long execution time, I would say if it is longer than 30 mins,  then it is possibly hanging.

Kill Hanging Cron Job

Now find the pid(second column) of the hanging process ending with ‘-mdefault’, in our case above, which is 10231, kill the process with the command below.

Make sure replace  10231 with the pid showing on your console, and you may run the ps aufx | grep cron command again to confirm the hanging Cron Job is killed.  Your Magento Cron Job should resume momentarily.

Usually, following the procedure above will resolve the hanging Cron Job for Magent, but if the Cron Job is still not working normally, or becomes hanging again. I suggest:, First check if you have your Magento Cron Job setup correctly. Check this post if you are not sure how to configure Magento Cron Job.

You can also try doing a truncate (delete all records) for table cron_schedule, and then run the commands again, see if new Cron Job tasks are populated in the cron_schedule table. Additionally, identifying the Cron Job task that is hanging with RUNNING status in the cron_schedule table may help you locate the module which is causing the hanging Cron Job.

 

 

 

Posted on

List All Failed SSH Login and Group by IP Address on Centos

Surprised by the huge number of fail login attempts when you log into your Centos server? Want to find out who was the bad guy trying to crack your server login?

This post will walk you through how to exam SSH login log on your Centos server, find the IP addresses that initiated a large number of login attempts. and block these IP from further brute force attacks.

Follow the instruction below you can get a list all failed SSH logins and group them by the IP addresses.

Locate the Log Files

Fist, use CD command to locate the log file directory on your Centos server.

You will see a lot log files. The SSH login log files are those starting with secure, e.g. secure-20171224. Now we want to combine all SSH login files before we go over them.

Combine all SSL Log Files

Run the command below to combine them together to a new file.

Analyze Log

We have combined all SSH login log to a single file named merged-file. We can run some analysis now.

Run the command above, you will generate another file(named result) in which each row is an IP address that attempted to login and failed, along with how many times he tried, for example you will see results like below:

18 101.95.26.126
1 102.250.51.207
1 102.251.192.26
31 103.10.168.138
37 103.10.230.126
2 103.11.135.132
36 103.12.196.125
3 103.15.74.82
7 103.16.168.138
385 103.192.253.37
6 103.192.61.9
1 103.192.76.100

SSH brute force

Download the file, convert it to a spreadsheet, so you can do sorting by the number of attempts. Then you can use iptable or access.denied to block access from those IPs on your Centos server.