Save images and video files to Amazon S3.
This file is made public, and only those who want to show it can view it by providing the URL.
Looking at various information, I saw information that if you make it public, it will be indexed by crawlers and it will be a hit by some kind of search.
I want to keep it public and not display it in my browser's search results.
I know that placing the following files in the root domain can avoid crawler patrols, but if I put them at the top of the bucket, will the image files in my bucket not be indexed?
robots.txt
User-agent:*
Disallow :/
Instead of retrieving a domain and putting an image file in it, it's just saving the file in the S3 bucket.
Thank you for your cooperation.
aws amazon-s3
As a general rule, the public URL of the object is public and should not be indexed unless it is found as a crawler, but it is probably assumed that the URL is shared again.
As far as I'm concerned, I think most cases will be fine by listing them on robots.txt.However, there are possible exceptions.
If you do not simply want to be indexed or should not be displayed to anyone other than yourself, there are several possible measures.
X-Robots-Tag:noindex.However, this seems to be a specification that is not explicitly stated in the document.
This file is made public, and only those who want to show it can view it by providing the URL.
If you make it public, it will be indexed by crawlers and it will be a hit with some kind of search
Crawlers, as well, are indexed and when a third party makes the download, the bucket owner is charged for the transfer of S3.
For this reason, public disclosure is not recommended in S3See also Amazon S3 security best practices.
As mentioned by hinaloe, it is recommended that the bucket be private and then share objects using the signed URL.
You can also set an expiration date for this signed URL.Even if the URL is leaked, it will become invalid after the expiration date, preventing third parties from downloading it.
© 2024 OneMinuteCode. All rights reserved.