Video instructions
In my experience, about 80% of websites have this problem.
Due to incorrect settings of the Last-Modified and Not-Modified headers:
- the server gets overloaded,
- website loading in the browser suffers,
- indexing and crawling of the site by search bots slows down.
Look, your website isn't some monolith — it's made up of many different files. Their number depends on the size of the site, but even taking just ONE page, it consists of several dozen different files. These are scripts, fonts, styles, images, and so on. The number of such files on each page depends on the design and markup.
None of these files change for years, or ever. But even if a bot scanned another page of your site a couple of seconds ago, where the exact same logo, the exact same scripts, and the exact same styles are used, due to incorrect server settings the bot thinks the files have changed in that time and downloads and processes them again...
I have a separate video on how to find out how many static resources a single page of your website consists of. Watch it later, but for now just take my word that it's not a small number.
If everything is set up correctly, this significantly speeds up the process and saves bandwidth. Then, even if a bot returns a year later, it will only download the files that have changed since then.
How to check if there is an error
I foresee that your site has this problem too... You can easily check this yourself.
If you don't want to check it yourself, just write to me on Telegram and I'll check it for free. If you want to do it yourself, read on...
To check, go to the resource last-modified.com. For the test, don't enter a link to the website, but a link to some static file of the site. For example, right-click on an image and select "Copy image URL," or open the page source, press Ctrl+F, and search for any file by its extension. Right-click and select "Copy link address." Paste it into the resource you opened earlier and check.
If you see an error message like this, it needs to be fixed.
If the image has an error, there's a 90% chance the styles and scripts have errors too. And if there's no error, I'd give 40% odds that you likely forgot to specify the woff2 font, the webp image format, or something else. So for reliability, it's better to check images of different formats — SVG, PNG, WebP, AVIF — as well as styles, scripts, various font formats, and PDF.
When there's no error, the test will look like this.
If there is an error and you need help fixing it, read on.
How to assign the task to a specialist
If you'd like to handle this without me, here are instructions on how to assign the task to a system administrator or hosting support.
Thanks to Demi Murych for the method, since I'm not very knowledgeable about server settings myself. He pointed out that the error is mostly in Nginx server settings, and to this day I haven't come across another cause. You need to describe the error to the system administrator or hosting support and ask them to change the value if_modified_since exact; to if_modified_since before; in the Nginx configuration file, or simply add if_modified_since before; for the static data.
Or you can use this text to contact support
Hello! My domain: [domain link] The site is not returning the Last-Modified and/or Not-Modified headers correctly I'm checking this file: [full link to image/style/script] Using this resource: https://last-modified.com/ When requesting static content (.jpg, .png, .css, etc.) using the If-Modified-Since or If-None-Match header, a 304 Not Modified code should be returned if the file has not changed since the specified time. Most likely, the current Nginx configuration is set up incorrectly, and by default Nginx works with the setting: if_modified_since exact; As a result, it will only return the correct 304 Not Modified header if the dates from the If-Modified-Since header exactly match the date from the Last-Modified header. If the last modification (Last-Modified) was the day before yesterday, and the If-Modified-Since header contains yesterday's date, the correct server response (304 Not Modified) should still be returned, and it should not contain the page body. To fix this, the setting "if_modified_since exact;" needs to be changed to "if_modified_since before;", or "if_modified_since before;" should be specified in the configuration file for the static files location. Please fix this!
On Beget hosting, they're happy to discuss the issue and easily make the changes. Though sometimes they ask for a full list of static files:
jpg|jpeg|gif|swf|png|avif|webp|ico|mp3|css|zip|tgz|gz|rar|bz2|doc|docx|xls|xlsx|exe|pdf|avi|ppt|txt|tar|mid|midi|wav|bmp|rtf|wmv|mpeg|mpg|mp4|m4a|spx|ogx|ogv|oga|webm|weba|ogg|tbz|js|7z|woff|woff2|ttf|svg|iso
On reg-ru hosting, they refuse to help, but there you can switch hosting management to manual mode and set it up yourself.
In many other places, they don't want to make the changes. They start arguing that it's supposed to be that way and look for other excuses not to configure it. Sometimes the correspondence drags on for a long time, but I've never had a case where I couldn't reach an agreement.
If you need help with the setup or communicating with support, feel free to reach out.
May your site be found, understood, and loved not only by visitors, but by search engines too.
Comments
You can leave a comment, ask a question, or share your opinion. Feedback helps us improve these materials.
