Debugging your search

Sometimes it is not quite clear why a query matches a given search string. To debug your query, PostgreSQL offers the ts_debug function. From a user's point of view it can be used just like to_tsvector. It reveals a lot about the inner workings of the FTS infrastructure:

test=# \x 
Expanded display is on. 
test=# SELECT * FROM ts_debug('english', 'go to www.postgresql-support.de'); 
-[ RECORD 1 ]+---------------------------- 
alias        | asciiword 
description  | Word, all ASCII 
token        | go 
dictionaries | {english_stem} 
dictionary   | english_stem 
lexemes      | {go} 
-[ RECORD 2 ]+---------------------------- 
alias        | blank 
description  | Space symbols 
token        |   
dictionaries | {} 
dictionary   |  
lexemes      |  
-[ RECORD 3 ]+---------------------------- 
alias        | asciiword 
description  | Word, all ASCII 
token        | to 
dictionaries | {english_stem} 
dictionary   | english_stem 
lexemes      | {} 
-[ RECORD 4 ]+---------------------------- 
alias        | blank 
description  | Space symbols 
token        |   
dictionaries | {} 
dictionary   |  
lexemes      |  
-[ RECORD 5 ]+---------------------------- 
alias        | host 
description  | Host 
token        | www.postgresql-support.de 
dictionaries | {simple} 
dictionary   | simple 
lexemes      | {www.postgresql-support.de} 

ts_debug will list every token found and display information about the token. You will see which token was found by the parser, the dictionary used, as well as the type of object. In my example, blanks, words, and hosts have been found. You might also see numbers, e-mail addresses, and a lot more. Depending on the type of string, PostgreSQL will handle things differently. For example, it makes absolutely no sense to stem hostnames and e-mail addresses.