information retrieval summer school padova

We inform you that on 25 may       2009 ore 16.44.47

You were registered at our conference on EUROPEAN SUMMER SCHOOL IN INFORMATION RETRIEVAL 2009.

Web Information Retrieval: Spam Detection and Named Entity Recognition

Spam detection

Linguistic features

Adds: web spam

Full search engines: for ranking it-self

Challenge:

Complexity

Scale

Co-adaptation.

 

 

 

Blog spam: blog of hide links

Attractive keywords

Linguistic analysis

Light-weight linguistic analysis

Air web – workshop

Attributes for ML.

Lexical diversity

Syntactical entropy

Labels

-          Hosts

-          Documents

 

String distance metrics

 

Name variations complicate the t…

Permutations, abbreviations, speling mistakes, declensions

 

Edit distance metrics:

Levenshtein

Bag distance

Needleman-wunsh

Smith-watermar

Smith-waterman with affine gaps.

Common character-level n-grams

q-grams, positional, q-grams, skip-grams

 

longest common substring LCS

 

string distance

jaro

jaro – winkler

jwm 

Web Information Retrieval: Spam Detection and Named Entity Recognition

2 hours seminar

by Marcin Sydow

Query decomposition

 

Topical query descomposition

 

Query in red

Do blue set

 

Quality of decomposition

Cost: low topical scattered

Red faction low `(small quantity of documents)

Inter-query overlap (well descompose)

Uncovered (try to cover all the query).

 

Annealing function

Solutions are neighbors if they differed by 1 query.  

 

Properties:

Quasi-transitive (temperature preference)

Utility function

Social preferences

Majority voting: is not transitive

Single peakedness

Arrows social welfare function

 

We like:

Arrow’s impossibility theorem.

TCD

2 PART

machine learning

Machine learning

A tentative table of contents:

• Bayesian Decision Theory
• minimum error rate classification
• discriminant functions and decision surfaces
• Parametric models and parameter estimation
• Non-parametric techniques
• K-Nearest neighbors classifier
• Decision trees
• Linear models
• Perceptron
• Logistic regression (Maxent)
• Large margin and kernel methods
• Generative versus discriminative modeling
• Sequence labeling and structure prediction
• Hidden Markov and MaxEnt Markov Models
• Sequence perceptron
• Conditional Random Fields
• Learning + Inference models

Unsupervised and Constrained Dirichlet Process Mixture Models for Verb 

Speaker: Andreas Vlachos Computer Laboratory Cambridge 

University Abstract:  

In this work we apply Dirichlet Process Mixture Models (DPMMs) to a learning task in natural language processing (NLP): lexical-semantic verb clustering. Furthermore, we propose a novel method of guiding the DPMM towards a particular clustering solution using pairwise constraints. The quantitative and qualitative evaluation performed highlights the benefits of both standard and constrained DPMMs compared to previously used approaches

Intelectual Propety Management

by Steve Gotz

Patents

trade marks

designs

copyright

know-how

requeriments:

novel

inventive

industrial aplication

Important to get a Lab Notebook

the pattens do not need to be perfect

Tutorial on Grammar induction

key words:

Bayesian learning

signal '--> concept

making up a grammar:

we start with a grammar where each rule describe one of the sentence of the corpus: so there are the same number of non terminal nodes/symbols as a terminal nodes or words at the corpus.

2 operations:

Merge - > reduce the number of nodes.

Chinking - > increase the number of nodes.

Selecting operations the grammar become more and more open.

The objective is to compare trees analysis, so the accuracy is measure as how near get the tree analysis of the emerge parser compared with the tree of a stablish parser.

(??? not very clear point)

for artificial grammars it give a very good results.

For real data sets not very good.

Full information:

www.webexperiment.nl

http://turing.science.uva.nl/~jzuidema/teaching/dublin09/

cngl

meting at dcu

cngl meeting

cngl meeting