Learn how to automate repetitive tasks using Makefiles.

2 years ago
Sophia Anderson
20 minutes

In the beginning of the paper,

If you have ever installed software from source code on your Linux server, you must have encountered the make utility. This tool is mainly utilized for compiling and constructing programs, enabling the source code developer to clearly define the necessary steps to build that particular project.

Even though make was initially designed to automate software compilation, it has been engineered in such a versatile way that it can be employed to automate almost any task that can be executed through the command line. This guide will explain how you can utilize make to automate sequential repetitive tasks.

Requirements

You can use any Linux operating system for this tutorial. Installation instructions are available for both Ubuntu/Debian Linux and Red Hat/Rocky Linux.

Setting up Make

Many Linux distributions provide the option to install a compiler with a single command, but they do not include one as a default feature.

You can install the build-essential package on Ubuntu to get all the necessary packages for a modern and well-supported compiler environment. Simply update your package sources and use apt to install the package.

  1. sudo apt update
  2. sudo apt install build-essential

 

To obtain the same compiler functionality on Rocky Linux or other Red Hat derivatives, you have the option to install a set of packages known as Development Tools. Using dnf, you can easily install these packages.

  1. dnf groups mark install “Development Tools”
  2. dnf groupinstall “Development Tools”

 

To determine if a compiler is present on your system, you can confirm its availability by checking if the make command exists. This can be done using the which command.

  1. which make

 

Output

/usr/bin/make

Now you possess the necessary tools to utilize and benefit from make in its typical capability as well.

Comprehending Makefiles

The Makefile is the main source of instructions for the make command.

Makefiles are specific to directories, which means that the make command will look for them in the directory where it was invoked. To ensure smooth execution and proper calling of our scripts, it is recommended to place the Makefile in the main directory relevant to the task at hand or where it logically belongs.

In the Makefile, we adhere to a precise structure where targets, sources, and commands are utilized by Make in the subsequent manner:

Create a file named “Makefile”
target: source
    command

The proper alignment and formatting play a crucial role in this. In this discussion, we will delve into the significance and arrangement of each of these elements.

Provide a single alternative phrasing for the word “Target” in a native manner.

The target represents a name provided by the user to refer to a collection of commands, comparable to a programming function.

The target, which runs continuously without any spaces, is positioned at the left-hand column and concludes with a colon (:).

When we invoke the make command, we have the option to provide a specific target by typing:

  1. make target_name

 

Afterwards, Make will verify the Makefile and carry out the instructions linked to the specific target.

Please provide the source you would like paraphrased.

Sources are citations to documents or alternative aims. They serve as requirements or interdependencies for the particular aim they are affiliated with.

For example, you may have a specific part in your Makefile that resembles the following:

Create a file named Makefile.
target1: target2
    target1_command

target2:
    target2_command

In this instance, we have the ability to label target1 in the following manner:

  1. make target1

 

Afterwards, Make will proceed to the Makefile and examine the target1 target. Subsequently, it will verify whether any sources have been indicated.

Temporarily jumping to the target, it would identify the source dependency for target2.

After that, it would verify whether target2 has any sources specified. Since it doesn’t, it will then proceed to carry out target2_command. Once the target2 command list is completed, make would return control to the target1 target. Subsequently, it would execute target1_command and terminate.

Files or targets can serve as sources. The “Make” tool uses file timestamps to check if a file has been modified since its last use. If a source file has been changed, the corresponding target is rerun. If there haven’t been any changes, the dependency is considered satisfied, and the process moves on to the next source or executes the commands if it was the only source.

The main concept is that through the inclusion of sources, we have the ability to create a series of dependencies that need to be performed before the current target. You have the option to indicate multiple sources, separated by spaces, following any target. This allows you to envision the creation of intricate task sequences.

Give me a command.

The make command is highly flexible due to the fact that the command portion of the syntax allows for endless possibilities. You have the ability to designate any command to execute for the target and can include multiple commands as required.

After declaring the target, commands are listed on the next line. For proper formatting, they should be indented with a single tab character. While certain versions of make may allow variations in indentation, it is generally advised to stick with a single tab to ensure that make correctly interprets your instructions.

Make treats each indented line beneath the specified definition as an individual command, considering them separately. There is no limit to the number of indented lines and commands you can include. Make will execute them sequentially.

We can prefix certain elements before commands to instruct make on how to handle them in various ways.

  • -: A dash before a command tells make to not abort if an error is encountered. For instance, this could be useful if you want to execute a command on a file if it is present, and do nothing if it is not.
  • @: If you lead a command with the @ symbol, the command call itself will not be printed to standard output. This is mainly used just to clean up the output that make produces.

Extra capabilities/attributes

In your Makefile, certain added functionalities can aid in the creation of intricate rule chains.

Variables can be paraphrased as “unknown quantities” or “values that can change.”

Variables (or macros) in Make are placeholders that can be substituted in your makefile. It is recommended to declare them at the beginning of your file.

Every variable’s name is written in all caps. After the name, we use an equal sign to assign a value on the right side. For example, if we want to set the installation directory to /usr/bin, we can write INSTALLDIR=/usr/bin at the beginning of the file.

Afterwards in the document, we can indicate this place by employing the $(INSTALLDIR) syntax.

Avoid line breaks

One other valuable thing we can do is enable multiline commands.

Within the command section, we have the flexibility to utilize any command or shell functionality, which also involves escaping newline characters by concluding the line with \.

Create a file called “Makefile.”
target: source
    command1 arg1 arg2 arg3 arg4 \
    arg5 arg6

If you utilize some of the shell’s programmable features, such as if-then statements, this becomes increasingly crucial.

Rewrite the Makefile in a native manner.
target: source
    if [ "condition_1" == "condition_2" ];\
    then\
        command to execute;\
        another command;\
    else\
        alternative command;\
    fi

Executing this block of code is similar to executing a single command. Although we could have written it as one line, breaking it down like this significantly enhances readability.

Ensure that when escaping end of line characters, there are no additional spaces or tabs following the \. Otherwise, an error will occur.

Rules for File Suffix

One more function that you can utilize for file processing is file suffixes. These are universal guidelines that offer a method of handling files according to their extension.

If you desire to convert all the .jpg files in a folder into .png files using the ImageMagick suite, the following Makefile code can accomplish that task.

A Makefile is required.
.SUFFIXES: .jpg .png

.jpg.png:
    @echo converting $< to $@
    convert $< $@

There are a couple of factors we should examine in this situation.

The initial segment is the declaration of .SUFFIXES. It informs make about the various suffixes we will be utilizing in file suffixes. Certain frequently used suffixes in source code compilation, such as .c and .o files, are automatically included and do not require labeling in this declaration.

The forthcoming section encompasses the announcement of the specific suffix rule, which is formatted as original_extension.target_extension:.

This target isn’t a specific one, yet it will correspond to any request for files having the second extension and assemble them from the file having the first extension.

If a file named file.jpg exists in our directory, we can simply use the make command to create a file called file.png for our situation.

  1. make file.png

 

Upon encountering the png file in the .SUFFIXES declaration, make will acknowledge the rule for generating .png files. Subsequently, it will search for the target file in the directory, wherein .png has been replaced with .jpg. Finally, it will carry out the ensuing commands.

The suffix rules incorporate certain variables that have yet to be introduced. These variables aid in the substitution of varying information depending on the current stage of the process.

  • $?: This variable contains the list of dependencies for the current target that are more recent than the target. These would be the targets that must be re-done before executing the commands under this target.
  • $@: This variable is the name of the current target. This allows us to reference the file you are trying to make, even though this rule was matched through a pattern.
  • $<: This is the name of the current dependency. In the case of suffix rules, this is the name of the file that is used to create the target. In our example, this would contain file.jpg
  • $*: This file is the name of the current dependency with the matched extension stripped off. Consider this an intermediate stage between the target and source files.

Make a Makefile for creating a conversion.

We are going to generate a Makefile, which will carry out certain image manipulations and subsequently transfer the files to our file server. This way, our website can showcase the modified images effectively.

Before you start, make sure you have the ImageMagick packages installed if you want to follow along. These packages consist of command line tools used for image manipulation, which we will use in our script.

To update your package sources and install using apt, follow these steps on Ubuntu or Debian.

  1. sudo apt-get update
  2. sudo apt-get install imagemagick

 

To access additional packages such as this one on Red Hat or Rocky, it is necessary to include the epel-release repo. Once added, you can install the package using dnf.

  1. dnf install epel-release
  2. dnf install ImageMagick

 

In the directory you are currently in, create a file named Makefile.

  1. nano Makefile

 

In this document, we will begin executing our conversion objectives.

Please change the format of every JPEG file to PNG.

Our server is specifically configured to handle only .png images. Therefore, prior to uploading, all .jpg files must be converted to .png format.

As mentioned earlier, a suffix rule is a highly effective method for achieving this. Let’s commence by utilizing the .SUFFIX directive to specify the file formats we intend to convert between, which are .jpg and .png.

Later on, we have the option to establish a rule that will transform .jpg files to .png files. We can achieve this using the convert command provided by the ImageMagick suite. The convert command follows the syntax convert from_file to_file.

In order to execute this command, we require the suffix rule that defines the initial and final formats we are working with.

Create a file named Makefile.
.SUFFIXES: .jpg .png

.jpg.png:           ## This is the suffix rule declaration

Now that we have established the applicable rule, we must proceed with carrying out the necessary conversion.

Since the exact matched filename is unknown, we must utilize the variables we have learned about. Specifically, we should use $< to refer to the original file and $@ to indicate the file we are converting to. By incorporating this knowledge with the convert command, we obtain the following rule.

Create a file that allows for the compilation and execution of the program.
.SUFFIXES: .jpg .png

.jpg.png:
    convert $< $@

To enhance our program’s functionality, we can introduce a feature that provides clear information on the execution of an echo statement. By prefixing the new command and the existing command with the @ symbol, we can prevent the command from being displayed when it is executed.

A file called Makefile is needed.
.SUFFIXES: .jpg .png

.jpg.png:
    @echo converting $< to $@ using ImageMagick...
    @convert $< $@
    @echo conversion to $@ successful!

At this juncture, let’s save and close the file in order to conduct testing.

Retrieve a jpg file and place it in the present directory. In case you do not possess a file, you can acquire one from the Silicon Cloud webpage utilizing wget.

  1. wget https://opensource.nyc3.cdn.digitaloceanspaces.com/attribution/assets/PoweredByDO/DO_Powered_by_Badge_blue.png
  2. mv DO_Powered_by_Badge_blue.png badge.jpg

 

To check the functionality of your make file, you can verify its progress by instructing it to generate a badge.png file.

  1. make badge.png

 

Output

converting badge.jpg to badge.png using ImageMagick… conversion to badge.png successful!

When Make searches in the Makefile, it will locate the .png file declared in the .SUFFIXES section. Once found, it will proceed to execute the corresponding suffix rule and follow the listed commands.

Produce a list of files.

At this stage, make has the capability to generate a .png file only when we clearly specify our desire for it.

Creating a list of .jpg files in the current directory and subsequently converting them would be an improved approach. For accomplishing this, we can create a variable that stores all the files intended for conversion.

One option for paraphrasing the sentence natively could be:
To accomplish this, the most effective approach would be to use the wildcard directive, such as utilizing JPG_FILES=$(wildcard *.jpg).

One option for native paraphrasing could be:
Instead of using a bash wildcard like JPG_FILES=*.jpg to specify a target, there is a drawback to this approach. When there are no .jpg files present, the conversion commands will actually be attempted on a file named *.jpg, resulting in failure.

If there are no .jpg files in the current directory, the wildcard syntax mentioned earlier will not assign any value to the variable.

As we proceed with this task, we should also address a minor difference commonly observed in .jpg files. It is frequently noticed that these image files are labeled with the .jpeg extension instead of .jpg. To automatically manage this, we can modify their names within our program to .jpg files.

In place of the aforementioned sentences, we will utilize the following pair.

Create a file called Makefile.
JPEG=$(wildcard *.jpg *.jpeg)     ## Has .jpeg and .jpg files
JPG=$(JPEG:.jpeg=.jpg)            ## Only has .jpg files

The initial command creates a collection comprising of .jpg and .jpeg files within the present folder and saves them as a variable named JPEG.

The second line concerns this variable and performs a name conversion by replacing the names in the JPEG variable that have the .jpeg extension with names that have the .jpg extension. This operation is achieved using the $(VARNAME:.convert_from=.convert_to) syntax.

After completing these two lines, a new variable named JPG will be created, which will only include filenames with the extension .jpg. It is possible that some of these files may not exist in reality on the system as they are actually saved as .jpeg files, and no actual renaming has occurred. However, this is not an issue because we are utilizing this list solely to generate a new list of .png files that we intend to create.

One possible paraphrase for “Makefile” could be:

– “File for creating and managing tasks”

JPEG=$(wildcard *.jpg *.jpeg)
JPG=$(JPEG:.jpeg=.jpg)
PNG=$(JPG:.jpg=.png)

Currently, we possess a group of files that we intend to ask for called PNG. This compilation solely includes filenames with the .png extension, as we have transformed the names. Consequently, any file that used to be .jpg or .jpeg in this folder has been utilized to generate a catalog of desired .png files.

We must update the .SUFFIXES declaration and the suffixes rule to incorporate the fact that we are currently managing .jpeg files.

Create a file named “Makefile.”
JPEG=$(wildcard *.jpg *.jpeg)
JPG=$(JPEG:.jpeg=.jpg)
PNG=$(JPG:.jpg=.png)
.SUFFIXES: .jpg .jpeg .png

.jpeg.png .jpg.png:
    @echo converting $< to $@ using ImageMagick...
    @convert $< $@
    @echo conversion to $@ successful!

As you can observe, we’ve incorporated the .jpeg into the list of suffixes, along with an additional suffix match for our rule.

Set specific goals

Currently, our Makefile contains a substantial amount of information, but it lacks any regular targets. To address this issue, we should rectify it by enabling the passing of our PNG list to the suffix rule.

Create a file named Makefile.
JPEG=$(wildcard *.jpg *.jpeg)
JPG=$(JPEG:.jpeg=.jpg)
PNG=$(JPG:.jpg=.png)
.SUFFIXES: .jpg .jpeg .png

convert: $(PNG)

.jpeg.png .jpg.png:
    @echo converting $< to $@ using ImageMagick...
    @convert $< $@
    @echo conversion to $@ successful!

The sole purpose of this new target is to display a list of the filenames that we collected as a necessary condition. Then, Make examines if there is a possibility to obtain the .png files and utilizes the suffix rule to accomplish this.

We can now utilize this command to transform all our .jpg and .jpeg files into .png files.

  1. make convert

 

Why don’t we include an additional goal? Another usual step while uploading images to a server is to change their size. By ensuring the images are correctly sized, we can spare our users the hassle of resizing them when they access the images.

We can use a command called mogrify in ImageMagick to resize images according to our requirements. Suppose the display area for our images on the website is 500px in width. We can achieve this conversion using the following command:

  1. mogrify -resize 500\> file.png

 

We desire to resize images larger than 500px wide to fit this area, while leaving smaller images untouched. To achieve this, we can incorporate the following rule:

Create a file named Makefile.
resize: $(PNG)
    @echo resizing file...
    @mogrify -resize 648\> $(PNG)
    @echo resizing is complete!

We have the option of including this in our file in the following manner:

A file used for enhancing the compilation and execution process of a program.
JPEG=$(wildcard *.jpg *.jpeg)
JPG=$(JPEG:.jpeg=.jpg)
PNG=$(JPG:.jpg=.png)
.SUFFIXES: .jpg .jpeg .png

convert: $(PNG)

resize: $(PNG)
    @echo resizing file...
    @mogrify -resize 648\> $(PNG)
    @echo resizing is complete!

.jpeg.png .jpg.png:
    @echo converting $< to $@ using ImageMagick...
    @convert $< $@
    @echo conversion to $@ successful!

Now we can connect these two targets together as dependencies for another target.

A program file for building and managing a software project.
JPEG=$(wildcard *.jpg *.jpeg)
JPG=$(JPEG:.jpeg=.jpg)
PNG=$(JPG:.jpg=.png)
.SUFFIXES: .jpg .jpeg .png

webify: convert resize

convert: $(PNG)

resize: $(PNG)
    @echo resizing file...
    @mogrify -resize 648\> $(PNG)
    @echo resizing is complete!

.jpeg.png .jpg.png:
    @echo converting $< to $@ using ImageMagick...
    @convert $< $@
    @echo conversion to $@ successful!

You might observe that resize will execute the same commands as convert. However, we will mention both just in case they don’t always function similarly. In the future, convert might include more extensive processing.

The current objective of webify is to transform and adjust the size of images.

Transfer files to a remote server.

Once our web-ready images are prepared, we can establish a destination to transfer them to the static images directory on our server. This can be achieved by utilizing scp and passing our converted files list.

The representation of our objective will resemble this.

Create a file named Makefile.
upload: webify
    scp $(PNG) root@ip_address:/path/to/static/images

All of our files will be uploaded to the remote server, resulting in a change in the appearance of our file.

A file of instructions for building a program.
JPEG=$(wildcard *.jpg *.jpeg)
JPG=$(JPEG:.jpeg=.jpg)
PNG=$(JPG:.jpg=.png)
.SUFFIXES: .jpg .jpeg .png

upload: webify
    scp $(PNG) root@ip_address:/path/to/static/images

webify: convert resize

convert: $(PNG)

resize: $(PNG)
    @echo resizing file...
    @mogrify -resize 648\> $(PNG)
    @echo resizing is complete!

.jpeg.png .jpg.png:
    @echo converting $< to $@ using ImageMagick...
    @convert $< $@
    @echo conversion to $@ successful!

Tidy up

After the .png files have been transferred to the remote server, we should incorporate a cleanup feature to remove all the local files.

Create a file called Makefile.
clean:
    rm *.png

We can include an additional target at the beginning that will invoke this one once we have uploaded our files to the remote server. This target will be the most comprehensive and the default one we desire.

To make it clear, we will designate it as the initial option. It will serve as the default and will be commonly referred to as “all.”

Can you provide more context or details about the term “Makefile”? It is a configuration file used in software development to compile and build programs.
JPEG=$(wildcard *.jpg *.jpeg)
JPG=$(JPEG:.jpeg=.jpg)
PNG=$(JPG:.jpg=.png)
.SUFFIXES: .jpg .jpeg .png

all: upload clean

upload: webify
    scp $(PNG) root@ip_address:/path/to/static/images

webify: convert resize

convert: $(PNG)

resize: $(PNG)
    @echo resizing file...
    @mogrify -resize 648\> $(PNG)
    @echo resizing is complete!

clean:
    rm *.png

.jpeg.png .jpg.png:
    @echo converting $< to $@ using ImageMagick...
    @convert $< $@
    @echo conversion to $@ successful!

Once you have made these final adjustments, simply navigate to the directory that contains the Makefile and .jpg or .jpeg files. By running the make command without any input, your files will be processed, sent to the server, and the uploaded .png files will be automatically removed.

  1. make

 

You can observe that it is achievable to connect tasks and select a specific process until a certain stage. For example, if you solely wish to convert your files and require them to be hosted on a separate server, you can simply utilize the webify target.

In summary, to conclude

By now, you should have a solid understanding of the general usage of Makefiles. Specifically, you should be knowledgeable about utilizing make as an effective tool for automating a wide range of procedures.

Although in certain situations it might be more effective to create a script, Makefiles provide a means of establishing a structured and hierarchical connection among processes. Acquiring proficiency in utilizing this tool can assist in simplifying repetitive tasks.

 

More tutorials

Converting a Python string to an integer and vice versa.(Opens in a new browser tab)

convert string to character array in Java.(Opens in a new browser tab)

Python 3 installing on Rocky Linux 9(Opens in a new browser tab)

permissions in PostgreSQL(Opens in a new browser tab)

Spring Boot CLI(Opens in a new browser tab)

 

Leave a Reply 0

Your email address will not be published. Required fields are marked *