How to test website speeds with curl

Have you noticed a slight slowdown in your websites lately, but timing them with a stopwatch is doing you no favors? If you find this is happening, you know you’ll have to report it to management, but you don’t want to go in without actionable information.

After all, a slight reduction in speed now could lead to a much larger reduction later. And with the competition constantly improving and evolving, your company cannot take a chance that those slowdowns will turn into a much bigger problem.

SEE: Hiring kit: Back-end Developer (TechRepublic Premium)

To that end, you should keep a record of your website speeds, so you have the information you need to go to those in power and say, “We have a problem.”

With the help of the curl command, you can piece together a simple bash script that can collect those speeds in a handy, date-stamped file. I’m going to show you exactly how to do that.

What you’ll need to test website speeds

The only things you’ll need for this are a running instance of Linux and a user with sudo privilege. The version of Linux can be a desktop or a server and can be any distribution you like. I’ll demonstrate with Ubuntu Server 22.04, so if you’re using a different distribution, you might have to alter the curl installation command.

That’s it. Let’s get scripting.

How to install curl

Curl should already be installed on your operating system. If not, it’s easy to do. Log in to your Linux distribution, open a terminal window and issue the command:

sudo apt-get install curl -y

Remember, if you’re using a different distribution, the installation command will vary, such as sudo dnf install curl -y for RHEL-based distributions.

With curl installed, it’s time to create our script.

How to create the web speed test bash script

The script we’ll use is actually simple to create using the command:

nano ~/webtest

In that, paste the following:

#!/bin/bash

curl -s -w 'Testing Website Response Time for :%{url_effective}nnLookup Time:tt%{time_namelookup}nConnect Time:tt%{time_connect}nPre-transfer Time:t%{time_pretransfer}nStart-transfer Time:t%{time_starttransfer}nnTotal Time:tt%{time_total}n' -o /dev/null URL > webspeedtest_"$(date)"

Where URL is the address of the site to be tested.

Here’s the breakdown of the script:

• Lookup time shows the amount of time for a request to receive a response.
• Connect time records the time until a TCP connection to a remote server is completed.
• AppCon time records the time needed for an SSL communication to process.
• Redirect time records the total time for redirection requests.
• Pre-transfer time records the total time before file transfer starts.
• Start-transfer time records the amount of time before the first byte is transmitted to a remote server.
• Total time records the time it takes to complete a response.

At the end of the script, we direct the output to the file webspeedtest that appends the date and time to the end of the filename. By doing this, we not only have a file to view the test results of that instance, but we have a file for every instance the test is run, so you can compare results.

Save and close the file.

Give the script executable results with the command:

chmod u+x ~/webtest

How to run the test script

To run the test, issue the command:

./webtest

The script should run fairly quickly and will result in a file named webspeedtest_DATE, where DATE is the date and time stamp for the run. The output of the script will look something like this:

Testing Website Response Time for: URL

Lookup Time:            0.120128
Connect Time:           0.177519
Pre-transfer Time:      0.177644
Start-transfer Time:    0.240367

Total Time:             0.240540

Where URL is the address you configured in the script.

How to automate the test

Instead of having to remember to run the test daily, let’s use cron to automate it. Edit your crontab file with the command:

crontab -e

At the bottom of your crontab file, add the following to automatically run the test at 8 a.m. every morning:

0 8 * * * /home/USER/webtest

Where USER is your Linux username. If you’ve saved the script in a different directory, make sure to change that path.

Save and close the crontab file with Ctrl + X.

You can also set that test to run hourly if need be with the line:

0 * * * * /home/USER/webtest

The one thing to remember is that you’ll need to prune those test result files, as they will add up.

A simple test for an important metric

And there you have it — an automated speed test script to test an important metric for your websites. If you need to test more than one site, create a script for each, and add a crontab entry for every test.

Although this doesn’t give you every metric you need, it will certainly keep you apprised of one particular test that can help you understand if something is going on with your websites. If you find they continue to slow down, it’s time to start troubleshooting.

Subscribe to TechRepublic’s How To Make Tech Work on YouTube for all the latest tech advice for business pros from Jack Wallen.