Curl Command Without Using Cache

Introduction

cURL (Client URL) is a command-line tool. It allows data to be transferred to or from a server without requiring user interaction by utilising the supported libcurl library. cURL can also be used to troubleshoot network connections.

In some cases, we may need to send requests that bypass the cache and generate a new response from the server. Caching can occur on the client side (browser cache) or the server side.

When using the cURL command, remember that it is only an HTTP client and does not cache any requests on the client side. As a result, any caching that occurs while using this command occurs on the server. To avoid the server-side cache, we can modify the HTTP request we're sending. However, depending on how the server is configured for caching, these may or may not work. These changes will be discussed further below. We can use curl command without using cache by ?

• Cache-Control HTTP Header Inclusion

• Pragma HTTP Header Inclusion

• Changing the URL

1. Cache-Control HTTP Header Inclusion

We can use curl command with the Cache-Control header and include the following directives ?

$ curl -H 'Cache-Control: no-cache'  http://www.tutorialspoint.com

The Cache-Control header may or may not be respected by the server. As a result, whether or not this method works is dependent on the server or website to which we are sending the HTTP request.

2. Pragma HTTP Header Inclusion

We can make use of the Pragma header ?

$ curl -H 'Pragma: no-cache'  http://www.tutorialspoint.com

The server may or may not consider this directive to serve a new request, as with the previous method, but it's worth a shot.

Changing the URL

The -H 'Cache-Control: no-cache' argument is not guaranteed to work because it can be ignored by the remote server or any proxy layers in between. If that fails, you can do it the old-fashioned way by including a unique querystring parameter. Typically, servers/proxies will consider it a unique URL and will not use the cache.

$ curl http://www.tutorialspoint.com?$RANDOM

You must, however, use a different querystring value each time. Otherwise, the server/proxies will again match the cache. To generate a different querystring parameter every time, use date +%s, which returns the seconds since the epoch.

$ curl http://www.tutorialspoint.com?$(date +%s)

Conclusion

To begin, we must note that the cURL command does not perform any client-side caching, and any caching that occurs while using this command occurs on the server side. To bypass the cache on the server side, we can use HTTP headers such as CacheControl and Pragma with applicable directives, or we can change the URL being accessed by modifying the query parameters of the URL.

These methods may or may not work depending on the server-side configuration. Tweaking query parameters also runs the risk of changing the intended response.

As a result, if we have control over the server-side code and deployment, we should always approach this problem from the server end. Otherwise, we can refer to the documentation provided or contact the support team of the service we're using. This is the only way to be certain of how caching works for the system in question.