Discovering knowledge using web structure mining

1.1 Problems With Web
 Difficulty in finding

relevant information
 Personalization of

information
 Learning about

consumers or individual
users

2.Objectives
i.

To Survey the area of
web mining.

ii.

Introduction to Link
Mining.

iii.

Review of HITS and
Page Rank algorithm.

3. Web Mining: Definition
 Process of discovering
 potentially useful &
 previously unknown

information or knowledge
from the web data.

3.1 Web Mining: Subtasks
 Resource finding

 Information selection

and pre-processing
 Generalization
 Analysis

3.1 Web Mining Categories
Web Mining

Web Content
Mining

Web Structure
Mining

Text and
Multimedia
Documents

Hyperlink
Structure

Web Usage
Mining

Web Log
Records

3.1.1 Web Content Mining
 Scanning data of a Web page to determine content
relevance with respect to search query.
Web Content
Mining

Agent Based
Approach

Database
Approach

3.1.2 Web Structure Mining
 Identifies relationships

between Web pages.
 Focuses on following

problems
 Reducing irrelevant search

results.
 Helps indexing
information on the web.

3.1.3 Web Usage Mining
 Focuses on techniques that predict user behavior while

interacting with the WWW.
 Web log records analyzed to discover user access pattern.
 The challenges could be

divided into three phases:
 Pre-processing
 Pattern discovery

 Pattern Analysis

4. Link Mining
 It is located at the intersection of the work in





Link analysis
Hypertext and web mining
Relational learning and inductive logic programming
Graph mining.

 Some tasks of link mining applicable in web structure

mining are:






Linked-based classification
Linked-based cluster analysis
Link Type
Link Strength
Link Cardinality

(i) Link-based Classification
 Predicts category of a web

page, based on
 words that occur on the page

 Links between pages
 anchor text
 HTML tags
 and other possible attributes

on web page.

 Eg: Predicting the category

of a paper, based on its
citations and the co-citations.

(ii) Link-based Cluster Analysis
 Goal : Finding naturally occurring subclasses.
 Data is segmented into groups
 similar objects - grouped together
 dissimilar objects - different groups.
 Helps in discovering hidden patterns.
 Eg: Finding diseases with similar transmission pattern.

(iii) Link Type
 Predicting link type

between two entities.
 Predicting purpose of

a link.
 Eg. Navigational or

Advertising

(iv) Link Strength
 Links could be associated with weights.
 Strong links - higher weight
 Weak links – lower weight

(v) Link Cardinality
 Refers to the number

of inbound links to a
web site.
 Link popularity :
 combination of
factors that weigh the
importance of each
incoming link.

5. Hyperlink-Induced Topic Search
(HITS)
 Link analysis algorithm that

rates pages.
 Identifies two kinds of pages

from Web hyperlink structure:

Web
Pages

With
Links
To

Web
Pages

With

 Authorities: Contains valuable

information on the subject.
 Hubs: Contains useful links
towards the authoritative
pages.

Other
Pages

Hubs

Content

Authority

HITS Contd…
 Two step process:
 Sampling step: Set of
relevant pages collected
 Iterative step: Hubs and
authorities are found
using output of above step

HITS Contd…
 Sampling Step:
 Query submitted to search engine yields a root set
 From root set we expand to base set

Expanding the root set into base set

HITS Contd…
 Iterative step:
 Associate non-negative authority weight x<p> and nonnegative hub weight y<p>.

Computing Authority Weight

Computing Hub Weight

Problems With HITS Algorithm
 Some problems with the HITS algorithm are:
 Mutually reinforced relationships between hosts
 Automatically generated links
 Non-relevant nodes
 Hubs and authorities
 Topic drift
 Efficiency

6. PageRank Model
 It is a link analysis algorithm.
 Numeric value to know the

importance of a web page
 Computes importance by no.

of incoming links

PageRank Contd…
 Rank of a page is divided evenly among its out-links to

contribute to the ranks of the pages they point to.

 Page Ranks form a probability distribution over web

pages, so the sum of all pages’ Page Ranks will be one.

PageRank Contd…
 PageRank can be calculated by:
PR(A)= (1-d) + d (PR (T1)/C (T1) +…+ PR (Tn)/C (Tn))
 T1..Tn are the pages that point to page A.
 C(A) is defined as the number of links going out of page A.
 d is the dampening factor which is usually set to 0.85

 The dampening factor is the probability at each page a

random surfer will get bored and will request another
random page.

Applications
 HITS was used in Clever search engine by IBM.
 PageRank is used by Google.

References
 Knowledge Discovery and Retrieval on World Wide Web Using Web Structure









Mining: Sekhar Babu Boddu, V.P Krishna Anne, Rajesekhara Rao Kurra and
Durgesh Kumar Mishra, 2010, In proceedings of Fourth Asia International
Conference on Mathematical/Analytical Modelling and Computer Simulation
(AMS), IEEE.
Link Mining: A New Data Mining Challenge by Lise Getoor, 2003, SIGKDD
Explorations, Volume 4, Issue 2
Authoritative Sources in a Hyperlinked Environment by Jon M. Kleinberg, 1998, In
proceedings of ACM-SIAM Symposium on Discrete Algorithms
The PageRank Citation Ranking: Bringing Order to the Web by L. Page, S. Brin and
T. Winograd, 1998, Technical report, Stanford University
wikipedia.org
web-datamining.net
maya.cs.depaul.edu

Discovering knowledge using web structure mining

More Related Content

What's hot (20)

Viewers also liked (16)

Similar to Discovering knowledge using web structure mining (20)

Recently uploaded (20)

Discovering knowledge using web structure mining