Robots.txt File

Spend £140 or more and receive 500 Silk or Matt Laminated business cards FREE of charge.

Click 'More' link below for the Offer Code to enter with your order

What is a 'robots.txt' file

Basically, the robots.txt file is a text file containing instructions to the search engine spiders about which sections and pages to crawl or specifically NOT to crawl.

How do I create a robots.txt file?

1. Create a text file and call it robots.txt

2. Type the following in the first line
  user-agent: *

3. Then for each folder that you DON'T want to be spidered/crawled type lines like...
  disallow: /foldername/

4. For specific pages or files, type this...
  disallow: /filename.html   or   disallow: /filename.php  etc.

Follow the examples below to create your own robots.txt file...

User-agent: *        alerts ALL robots
Disallow: /*?        disallows search by bots of all dynamically created pages
Disallow: /foldername*/        disallows all folders within folder name
Disallow: /*.php$        disallows search by bots of all pages with php ext. Swap for html to html$
Disallow: /foldername        disallows search by bots specified folders
Disallow: /foldername/file.html        disallows search by bots of specific pages

Use the example below to Allow robots to ALL Sections and Pages of your website

User-agent: *

Use this example to Restrict Access to the Listed Folders & Files and to permit ALL other areas

User-agent: *
Disallow: /cgi
Disallow: /otherfolder
Disallow: /filename.php
Disallow: /filename.html 

To inform the spiders of your Sitemap file add this:

Upload the completed robots.txt file to the root directory of your website.

You only need ONE. Do not add any other robot.txt files to any other directories, they'll just be ignored.

By creating a robots.txt file, the respectable search engine spiders will obey the instructions. There are some spiders out there that will ignore it altogether.

Reasons to use Artwork Network

Easy to work with

We'll come and talk to you in your place of business. We will bring samples of our work to illustrate just how we can work with you.

Awesome designs

Clients are always impressed with our work. We will go to great lengths to create design examples that are brand oriented with the clients business.

Our Experience

By combining our in depth knowledge with our passion for digital design, we can ensure that your designs are perfect and fit for purpose.

About Artwork Network

Based in Middlesbrough, Cleveland UK... Artwork Network are a graphic, print & website design company.

Providing Professionally designed and printed media to promote your business online and offline.

Our Mission

With considerable resources at our disposal, ArtWork NetWork will Plan, Design & Implement projects of any size.
Step by step communication with you is maintained throughout the project from conception to delivery.

A passion & dedication to listening to the clients needs and delivering the desired result. Whether a box of business cards to a full company branding, we will attend to every detail.