Using Shebang in Linux

Imagine you're working on a Linux computer and have a recipe with instructions written in different languages. But your computer only understands English! That's where the shebang comes in. It's like a tiny label at the beginning of the recipe that tells your computer which "translator" to use for each language. This magic symbol, #!, followed by the interpreter's path, is the key to understanding and running all sorts of scripts on your system.

Evolution and Historical Significance of Shebang

The shebang has a cool history. It started way back in an old operating system called Multics, but it really took off with Unix. There, it became the go-to way for computers to figure out which program to use for different scripts. Now, it's not just for shell scripts anymore! The shebang works for all sorts of scripting languages, making it a cornerstone of how Linux and its friends run these handy tools

Practical Application of the Shebang

In some Linux programs that are written in scripting languages like Python, Perl, and Shell-script, you will see a line right at the top that starts with #!/ like this one:

Executables in Linux

If you are familiar with Windows, then you know that there is a type of file that we call Executable. It uses .exe as the file format, and you can run such a file by double-clicking on it. You can also run such a file by typing its name inside the command line and hitting Enter.

When you enter the world of Linux, you may realize that there is no fixed executable format. Long-time Linux users know that in Linux, you can run any kind of file as an executable so long as its executable flag has been set. When you run the ls command in a folder, the files which can be executed appear green, and you can type dot, slash, and then the name of that file and hit Enter to run that file, as seen in this picture:

When the green file 'greet ' in the above picture was run, it executed some Linux shell commands. In fact, the file greet is a text file containing one line of code:

echo Hello, World!

By default, any text file with lines of Linux shell commands inside it can be turned into an executable file by setting its Executable Flag, which is done with the command:

chmod +x <filename>

But sometimes, you want to run a text file which has commands in Python, Perl, Ruby, Javascript, etc. languages instead of Linux Shellscript. That is where the Shebang comes into the picture.

A Python Example

Suppose we wish to make an executable in Python. We can write a Python script which will input a number and return its square.We want to run this executable. If we have Python installed, we can run the file simply by using the following command:

 python <filename>

But suppose we want to run this program without using the Python command? In other words, can we run the Python script square.py only using ./square.py and without using the python command?

Yes, you can, and that's where the Shebang comes into play.

Using a Shebang in Linux

You can use a shebang to tell Linux which interpreter is needed to run the script. For instance, in the case of a Python file, you will need the python executable. Usually, this is located in /usr/bin. You will need the complete path to the executable.

To know the path to the executable, you can use the following command in Linux.:

which

To use a shebang, you need to know where the interpreter is located in your computer. For the Python interpreter, you will need the location of the Python executable. Once you know this, the Shebang command is simple :

#!<path-to-interpreter>

For instance, if your Python interpreter is located in /usr/bin/python like ours is, then your shebang is:

#!/usr/bin/python

The next thing to remember is to write your Shebang in the first line of the Python script, even before any other line. So the overall Python code for the earlier file square.py becomes:

#!/usr/bin/python
number = int(input())
print(number * number)

The last step is to set the executable flag for the Python script. To do this, we need to run the command

chmod +x square.py

And we are ready to run square.py as an executable.

Note: The Shebang does not make your file executable by itself. It simply tells Bash (the shell) where to find the interpreter to run it. You will still need to have the interpreter installed.

Making Shebangs Portable - Using env

Of course, if you write a script with a Shebang on your machine, it may not work on other machines. For instance, another person may not have Python installed. Also, another person may not have Python installed in the same location. So how do we make the Shebang portable?

First, remember to use specific executable names. For instance, instead of using /usr/bin/python, consider using /usr/bin/python3 in the Shebang. Some Ubuntu versions have Python 3 installed, but the interpreter is named python3 instead of python.

Another important tip is to use the env command. The env command is a part of the Linux System Base, which is a list of programs that have to be present in a system for it to be called a Linux OS. Ubuntu, Fedora, Manjaro, CentOS, RHEL, etc. machines all carry the env command.

Used on its own, env simply lists all available commands. But if you give env an argument which is the name of an installed interpreter (such as python3), it will identify the correct location of the interpreter on its own. Plus, env itself is always located in the same place on all Linux machines:

/usr/bin/env

So you can use env to make your shebangs portable:

For Python:
#!/usr/bin/env python3

For Node-JS (if the script is in Javascript):
#!/usr/bin/env node

For Perl:
#!/usr/bin/env perl

Executable Path and Shell Commands

When you use an executable script in Linux with a shebang, you can make it become a shell command! To do this, you need to place it in a PATH location. A PATH location is a special folder in your Linux filesystem, and all executables inside a PATH location can be used like commands in Linux.

There are two ways to do this:

1. You can look up the existing PATH locations, find one that is to your liking, and place your executable script file there. Remember to set the executable flag with chmod +x <filename> after you move the file there.
2. You can add the folder in which your script file is located to the PATH variable, and make the file a command.

Below, we will discuss both of these methods.

Adding Your File to a PATH Location

First, look up the existing PATH locations using this command:

echo $PATH | sed "s/:/\\n/g"

This will list all the PATH locations in your terminal:

Notice that not all of the folders listed here are accessible to you without sudo, since they do not belong inside your Home directory. The most preferred location is

/home/user/.local/bin

If this folder doesn't exist, you can create it and add it to the list by following the instructions under the next subheading. Place the file inside that location and make sure that the executable flag is set. The file then becomes a command.

Adding a Folder to the PATH List

In general, it is a good idea to make yourself a new directory, add all your executable script files there, and then add that directory to the PATH locations. This is especially useful if you do not already have a folder named bin inside ~/.local directory.

First, make the folder which you need to add to PATH.

• If the folder already exists, then you can skip this step.
• If it doesn't exist, then you can make it - either using the terminal command mkdir, or using the GUI.

Second, not the path address of the folder. For instance, if the folder is named mypath and it is inside my home directory, then the location will be

/home/user/mypath

Third, open the .bashrc file in a preferred editor. This is a hidden file inside the home folder. You can see it in your GUI by using Ctrl+H to view hidden files, and then double-click on it. Alternatively, inside a terminal, you can use nano, vi, or vim to open the file with a command:

nano ~/.bashrc

Remember, this is a very important system file. Do not mess with this file unless you know what you are doing! Simply add one line at the end of the file, with this content:

export PATH="$PATH:<your-directory-path>"

For instance, if the directory is /home/user/mypath, then we can use the command:

export PATH="$PATH:/home/user/mypath"

The last step is to close and reopen your terminal. All terminals you open from this point on will treat executable files inside mypath as commands. If you do not want to close and reopen your terminal, simply run the command

. ~/.bashrc

and it will update itself and your executable file with a Shebang will be available as a command.

Remember: Your script must have a correct Shebang for it to become a command!

Some Other Uses

Shebangs allow you to choose the interpreter which runs a script, so that the user does not have to select the interpreter at run time. This can be used to turn a script into a command, but it also has some other uses.We are going to go over some of them.

1. Ensuring Compatibility

Different distros have different shells, and sometimes users override their shells. For instance, some users prefer to use Zsh, the shell which comes with Apple's MacOS. This can lead to some problems when for the people who write shell-scripts. For instance, if we write a script which behaves differently on Bash and Zsh, the script will run differently on the machines of people who use Zsh.

To avoid this, we can specify the shell we wish to run the script on. For instance, we can write the script to be run from The Bourne Shell (sh), which is the old UNIX shell. The Bourne Shell is present on Linux, MacOS and all other UNIX-based systems. Hence, we can ensure that our script runs identically on all of these systems by writing a script for the Bourne Shell using this shebang:

#!/bin/sh

2. Specifying Interpreter Version

There exist differences between different versions of the same shell. For instance, the Bourne Again SHell (Bash) has different versions. When you go through the list of CVE (Common Vulnerabilities and Exceptions) list associated with versions of Bash, you will find that security vulnerabilities exist in certain older versions of Bash which have been patched in later versions.

Unfortunately, enterprise system users (which are usually large corporate bodies) cannot simply upgrade their enterprise Linux systems. They need to pay money for licensed Linux systems from Red Hat, Oracle, SUSE, etc. So they use old Linux machines which have older versions of Bash. To avoid the vulnerabilities, they can install the latest version of Bash. But to set this version as the default for a shell script to run with, they override the default shell using a shebang. For instance, to ensure that the script runs with Bash 5.2, they can use:

#!/bin/bash-5.2

One possible downside is that, when a newer version of Bash is installed (say Bash 5.3), all the scripts used by the sysadmins and devops personnel will have to have their shebang line updated to Bash 5.3; to get around this problem, the sysadmin creates a symlink (which is essentially a shortcut to a file) to the latest version of Bash, and gives it a name like bash-latest. Then the shebang simply points to this symlink:

#!/bin/bash-latest

When a later version of Bash is installed, the symlink is deleted and a new one is created at the same location, with the same name, but pointing to the newer version of Bash.

3. Preventing Direct Execution

Sometimes you don't want a shell-script file to be executed. Maybe you only want it to be invoked internally, using commands like source or using the os.popen function in Python. If you make a software with such a script, and you want to prevent a user from unexpectedly running the script, you can deliberately use a shebang which points to an executable that cannot interpret the script. A common choice of such an executable is false, which returns a non-zero value to the shell (to indicate an error), and does nothing.

To disable a script from being run in the shell by a user who didn't read your README file, just use this shebang:

#!/usr/bin/false

You can still run a script with such a shebang using the source keyword, but it cannot be run directly as an executable.

Possible Malicious Use

Its always a good idea to be warned of all the ways in which something can be used for evil. Shebang can be used for evil. A shebang only specifies the path to an executable, and the OS assumes that the executable is an interpreter to run the script on. But we can run any executable using a shebang.

For instance, we can create a very innocent-looking and seemingly harmless shellscript file, which performs nothing dangerous, but then give it a shebang which invokes an executable of my choice. This can include a malicious executable.

Before running a script which has its executable flag enabled using chmod +x, it is a good idea to quickly go over the script if you don't know where its from. Lots of people ignore comments in the script while going over it, but it is important to check the shebang. The shebang starts with a '#' and most editors give it the same syntax highlighting as any other comment, but always make sure to quickly glance at the shebang before running a script.

Conclusion

In this article we discussed Linux shebang which is a special code, marked by #!, at the start of a script that tells a computer which language the script is written in. Originating from Multics and gaining popularity with Unix, the shebang is crucial for running scripts in various languages on Linux systems. This article explored its history, practical applications, and versatility, demonstrating its use beyond shell scripts for languages like Python. It explained how shebangs enable the conversion of text files into executable ones, using commands like 'chmod +x.' The article also covered making shebangs portable with specific executable names and the 'env' command. Lastly, it highlighted additional uses, such as ensuring compatibility across different shells, specifying interpreter versions, and preventing direct execution, while urging caution against potential malicious uses by reviewing shebangs before running scripts.