What is vebidoobot?
We indexes public content from the web. To do this we use a web-crawler which identifies itself as ‚vebidoobot‘.
Load Spreading Technique
We utilize a load spreading technique which reduces the load on the indexed domains‘ web-servers. The effect of this is that the crawler will not put any noticeable strain on the servers.
How to Avoid Being Indexed
If you do not wish us to index your website content please add ‚vebidoobot‘ to your robots.txt file according to the The Robot Exclusion Standard (example provided below).
- robots.txt example:
Include the following in your robots.txt file:User-agent: vebidoobot
- Robots META tag example:
Include the following in the head section of your web-page:<meta name=“robots“ content=“noindex,nofollow“>
Although vebidoobot should not put any noticeable strain on a webserver we apologize for any inconvenience it may cause and would ask you to notify us about any problems at vebidoobot (at) vebidoo.de in order for us to rectify them as quickly as possible.