This is a nodejs module that allows you to create crawlers of websites.
It basically manages multiple requests, and concatenate the chunked data from node before passing to the provided parser. The parser can add more tasks based on the data parsed. The number of concurrent requests is limited to avoid overload.
This module contains a single class called Spider.
tasks:
Array of {uri: string|object, parse, body}
parse:
function(Buffer data, http.IncomingMessage response,
http.ClientRequest request)
max: Maximum number of concurrent requests
If uri is a string, then it parsed with
url.parse. The object is then passed to
http.request. A default http.Agent and headers
are used if not present in object. The body is passed to
http.ClientRequest.end method, this can be useful for POST.
The parse function receives the concatenated data from multiple data
events. It can add more tasks based on the data parsed. You can use the
method toString to convert the data to a string.
npm install rfc-spiderThe source code is licensed under GPL v3. License is available here.
Source codeLanguage: Ruby
This script asks the user questions in an order that helps him or her memorize information.
Questions answered correctly are less likely to be asked again and wrong answers are more likely to be asked again. The order is taken from a queue of questions, a correct answer moves a question down the queue in powers of two.
Questions are taken from a .txt file. Each line
contains a question followed by a question mark and space, followed by
the answer. The user needs to type the answer as it is written in the
file to get a right answer. An example file with the multiplication
table is provided with the script.
Your progress is saved after you quit the program. The save file can be replaced in order to use a different set of questions without losing your progress.
DownloadLanguage: AutoIt
Platform: Windows
This script takes screenshots after every key press and saves them on the desktop.
It can be used to record the steps used to perform a task on the computer.
DownloadLanguage: Bash
Platform: GNU/Linux
This script creates a function called mls. This function
allows you to navigate in the file system by typing the number of the
folder to open it, dot to go to the parent folder, and q to
quit.
To install, simply add this script to your ~/.bashrc file.
Language: Python 3
This script moves all files of all folders of the current directory to the current directory and removes all those folders recursively. A number can be passed as command line argument to restrict the level of recursion.
DownloadLanguage: Ruby
This script moves all files of all folders of the current directory to the current directory and removes all those folders recursively. A number can be passed as a command line argument to restrict the level of recursion.
Download