Squarespace has enhanced the privacy controls for website owners with a new feature that allows them to hide their sites from both search engines and AI crawlers, while still keeping the site accessible to visitors. This option is particularly significant for artists and content creators who are concerned about their work being used to train AI models without their consent.

A web crawler is a bot used by search engines to scan website content, which then appears in search results. AI companies also use web crawlers to collect data for training large language models (LLMs), aiming to improve their AI's accuracy and capabilities. The fear among artists is that such data scraping could use their content without permission, potentially impacting their rights and the reach of their work.

Squarespace users can now opt-out by adjusting their settings, disabling search engine crawlers, and specifically turning off Artificial Intelligence crawlers. This modifies the site's robots.txt file to block bots from companies like Anthropic AI and even those associated with OpenAI's ChatGPT. However, it's important to note that this measure doesn't retroactively remove content already scraped by AI models, nor does it allow hiding specific pages from AI crawlers​​.

The new Squarespace feature to opt-out of AI crawlers is particularly relevant for artists who are concerned about the use of their work. With the rise of AI technologies capable of learning from vast amounts of data available online, there is a growing worry that these models might utilize artists' creations without fair compensation or even consent. This feature empowers artists with more control over their digital content, ensuring that their work is not unwittingly used to train AI systems. It underlines a commitment to protecting creative rights and recognizes the need for artists to manage the distribution and use of their work in the digital realm.