Prevent user-generated spam on your site and platform

Spammers often take advantage of open comment forms and other user generated content inputs and generate spammy content on an unsuspecting victim site. Hosting platforms and tools that let users publish shared pages, generated content, or web apps may be similarly open to abuse; spammers may create a large number of pages or sites that violate our spam policies and add little or no value to the web.

Preventing abuse on your platform or site is usually not hard. Even simple deterrents such as an unusual challenge users have to complete before interacting with your property may discourage spammers.

Tell users that you don't allow spam on your service

Publish a clear abuse policy and communicate it to your users, for example during the sign-up process. Furthermore, allow trusted users to report content on your property that they consider spammy.

Identify spammy accounts

Keep a record of signups and other user interactions with your platform, and try to identify typical spam patterns, such as:

  • Form completion time
  • Number of requests sent from the same IP address range
  • User agents used during signup
  • User names or other form-submitted values chosen during signup

These signals may help you create a user reputation system, which can not only help you engage users, but it can also help identify spammers. Since many comment spammers want their content in search engines, consider adding the noindex robots meta tag on posts that come from new users that don't have any reputation on your platform. Then, after some time, when the user gains reputation, you can allow their content to be indexed. This will greatly demotivate spammers from interacting with your platform.

Since oftentimes spammers are motivated by leaving a link to their site, consider adding a nofollow or ugc rel attribute to all links in untrusted content.

Only index content meant for Search

If your service lets users create shareable pages, generated outputs, or interactive tools, consider whether that content is meant to be found through search engines or only accessed directly by people with a link.

When pages are created for direct sharing, collaboration, or utility rather than public search discovery, apply the noindex rule (using a robots meta tag or X-Robots-Tag HTTP response header) or block crawling in robots.txt by default.

Use manual approval for suspicious user interactions

Manual approval (or moderation) for certain user interactions can decrease spam on your platform considerably by preventing spammers to instantly create content that may be spam. Moderation adds overhead to your daily workflows, however it's a very effective way of fighting spam. Its efficacy is why, for example, comment moderation is a built-in feature in most CMSes.

Use a blocklist to prevent repetitive spamming attempts

Once you find a single spammy profile, make it simple to remove any others. For example, if you see several spammy profiles coming from the same IP address, you can add that IP address to a permanent ban list. For CMSes (for example, WordPress), there are plugins like Akismet that can help, but adding the IP address to your firewall's deny list can be very effective also.

Block automated account creation

In your sign-up form, consider using reCAPTCHAs or similar verification tools to only allow human submissions and prevent automated scripts from generating a lot of sites on your hosting service.

Monitor your service for abuse

  • Verify your site or subdomain in Search Console so you receive email alerts if Google issues a manual action against your site or detects security issues. Also check the Page Indexing report and Search performance report regularly for unexpected spikes in indexed pages, impressions, or queries that don't match your platform's purpose.
  • Track platform-level creation and indexing metrics, and set up alerts for anomalies such as spikes in newly published public pages or a small number of users generating a large share of indexable URLs.
  • Monitor your property for spam signals such as redirects, large numbers of ad sections, certain spammy keywords, and large sections of encoded JavaScript code. The site: search operator or Google Alerts can help detect problems.
  • Keep an eye on your webserver log files for sudden traffic spikes.
  • Monitor your property for phishing and malware-infected pages. For example, you can use the Google Safe Browsing API to regularly test URLs from your service.
  • Come up with a few confidence checks. For example, if you're mainly targeting users in Japan, what are the odds of thousands of user interactions from an Italian IP overnight on your property? A number of tools are available to detect the language of newly created sites—for example language detection libraries or the Google Translate API v2.