Skip URLs tab

This tab allows you to enter URLs which DeepTrawl should never try to download. Full or part URLs should be entered one per line.

This is useful, for example, for instructing DeepTrawl to never check specific links or images, or if you are using a Form Action to login to your website and you don't want DeepTrawl to follow a logout URL.


URL List

Enter full or part urls which should never be downloaded,


This stops DeepTrawl from downloading the two individual resources.



This blocks downloading of the first two resources and also blocks anything in the myDir directory.