This is a forked repository. Have a look at
a-priori’s spider wiki
for more information.

Requirements:

• ICU 3.8 installed

  icu-config --version

• autoconf version 2.59c or higher

  autoconf --version</pre>

Getting Started:

0. Get a copy of the repository on your local machine.

1. run autogen.sh

2. run ./configure and make

3. run spider_run.sh to run the application

4. You’ll see some continuous output indicating the server is running. Type:

task_master:insert_task(“http://www.example.com”).

… where ‘example’ can be any domain, but preferably something small and out of the way, perhaps. At this point, you should see some ‘processing http://www.example.com/example.html’ messages.

5. Type:

application:stop(spider).

… or C-c to make the madness end.