last day (17 days later) » 

4:06 AM
0
Q: Pass "-" in Elastic Search Query

code_blueWhen we are passing a query containing special characters, Elastic Search is splitting the text. E.g. If we pass "test-test" in query how can we make Elastic Search treat this as a single word and not split it up. Analyzer used on the field we are searching: "text_search_filter": { "typ...

 
what is your use case? what is your mapping?because there are different ways to achieve this
 
Updated with Analyzer used and Query for search. We can see a token has been created "test-test" using the analyzer.
 
what is the output of curl -XGET 'localhost:9200/your_index_name/_analyze?analyzer=test_search_analyzer' -d 'test-test'
Do you see a token "test-test" in the output?
 
Updated post with Analyze API output
 
are you sure all fields you are searching against have test_search_analyzer applied? because test-test is one of the tokens and it should match. you are not using different search_analyzer right?
 
4:06 AM
Yes the field we are searching on has the analyzer as mentioned. The issue is when we pass "test-test" in query Elastic search is not treating it as a single text rather splitting it up. How can we make Elastic Search treat it as a single word.
 
I created index with same settings and it worked for me
 
We have documents which contain various combinations of the word. Like "test" "test-123","test test" "test-test-123" "te". Now when we are searching with "test-test" only the document matching the pattern ("test-test-123") should come up ideally. But all documents show up containing "test".
 
so test-test should match test-test-123 and not test, test test? am I right
 
yes
similar to like search in sql query
 
ok. let me create some sample documents and test
 
4:10 AM
ok great
 
I think you could use match_phrase_prefix rather than cross_fields
could you please try that and let me know
sorry it should be "type": "phrase_prefix" although it is called match_phrase_prefix
 
"multi_match": {
"query": "test-test",
"type": "phrase_prefix",
"fields": [
"FIELD_NAME^5"
],
}
are you suggesting this?
not working same result
 
I think I got it
 
This is due to edge_ngram filter
 
4:16 AM
ok..what do i need to change
 
test-test will even match document like "we t r" because it will generate token "t"
first why are you using edge_ngram, what is the use case?
 
yes correct
i need to generate a token "test-test"
using analyzers like standard, simple or keyword it doesn't generate the token
 
but you are using whitespace tokenizer so test-test will remain the same
and if you you use "phrase_prefix" query it will match test-test-123
 
phase_prefix or cross_fields does return test-test-123 but also returns other documents containing test, test test test 123
standard tokenizer instead of whitespace returns the same result
 
As I said, this is due to ngram, could you please remove that filter and try again?
 
4:24 AM
"type": "custom",
"tokenizer": "whitespace",
"filter": [
"lowercase",
"asciifolding" ]
 
is this what you are suggesting
i believe this is same as standard analyzer
standard analyzer is not generating token test-test from test-test-123
using only a whitespace analyzer the only token generated is test-test-123 which means user has to do an exact search
 
This is not standard analyzer and you are right it wont generate token test-test from test-test-123
but we can use phrase_prefix query to match test-test-123
could you give me 1 minute? I am testing it
 
ok sure
 
I tested
{
"query": {
"multi_match": {
"query": "test-tes",
"fields": ["context"],
"type": "phrase_prefix"
}
}
}
this query gave me documents which had "test-test" and "test-test-123", also it did not give me documents which had test, tes etc
could you try that on your machine?
 
4:33 AM
what is the analyzer you have used?
 
nothing, same one as yours, just removed edge_ngram filter
"analyzer": {
"syn_analyzer": {
"tokenizer": "whitespace",
"filter": [
"asciifolding",
"lowercase"
]
}
},
 
ok let me try and get back to you
 
4:45 AM
when you use the analyzer what tokens are generated for "test-test-123"
 
only test-test-123
but if you use phrase_prefix query on test-test it will be equivalent to sql like 'test-test%' and will match test-test-123
 
well i just ran the query it seems to be working..but i can run it with cross_fields instead of phrase_prefix
phrase_prefix is giving me a search exception
but cross_fields seems to be working
let me run a few more test cases and confirm
 
ok, what is the search_exception?
 
ElasticsearchParseException[No type found for value: phase_prefix];
by the way thanks a lot for your time and patience...you guys make stackoverflow awesome :)
 
there is a typo, it seems. should be phrase_prefix
and Your Welcome, always ready to help whenever I can
:)
 
4:57 AM
got it
:P
my bad
i will run few of my use cases and will confirm you
thanks again
:)
one question whats the difference between cross_fields & phrase_prefix
 
ES documentation will give lot of info on that, btw cross_fields does not give test-test-123, it only gives test-test, it is an exact match query while prefix query will be like sql as I said, pls again check with cross field query
I am going to sleep now, it is late here. Let me know if you have any doubts or something is not working. :)
 
 
2 hours later…
7:13 AM
using phrase_prefix when i search with "test-test-123" no document is being returned. "test-test" however works
 
 
7 hours later…
2:15 PM
test-test-123 will match test-test-123, test-test-123456, test-test-123-test etc but wont match test-test
do you want to match it?
 

  last day (17 days later) »