« July 2005 | Main | September 2005 »

August 10, 2005

Ooh, this is new..

O'Reilly CodeZoo

Posted by pj at 11:31 AM

August 09, 2005

mysql_robot2.py site indexing script

I've uploaded a copy of my mysql_robot2.py site indexing script for safe keeping.

And here's the DB structure:

mysql_robot.sql

Posted by pj at 10:40 AM

Table for mysql_robot2.py

The ltsn01_ng.web_site_text table:

CREATE TABLE `web_site_text` (
  `link_id` int(10) NOT NULL auto_increment,
  `link_title` text NOT NULL,
  `link_base_href` text NOT NULL,
  `link_url` text NOT NULL,
  `link_scheme` varchar(255) NOT NULL default '',
  `link_host` varchar(255) NOT NULL default '',
  `link_path` text NOT NULL,
  `link_query` text NOT NULL,
  `link_params` text NOT NULL,
  `link_fragment` varchar(255) NOT NULL default '',
  `link_content_type` varchar(255) NOT NULL default '',
  `link_html` text NOT NULL,
  `link_xhtml` text NOT NULL,
  `link_text` text NOT NULL,
  `response_code` varchar(255) NOT NULL default '',
  `link_date_stamp` timestamp(14) NOT NULL,
  `link_type` enum('internal','external','broken_internal','mailto') NOT NULL default 'internal',
  PRIMARY KEY  (`link_id`),
  KEY `response_code` (`response_code`)
) TYPE=MyISAM
CREATE TABLE `referer_refered_link` (
  `rrl_id` int(10) NOT NULL auto_increment,
  `referer_url` text NOT NULL,
  `rrl_link_url` text NOT NULL,
  PRIMARY KEY  (`rrl_id`)
) TYPE=MyISAM

Posted by pj at 10:37 AM