last day (14 days later) » 

05:23
0
A: Find 10 latest record for each BUYER_ID basis on time

Frank Computer SELECT FIRST 10 * FROM TestingTable1 WHERE buyer_id = 34512201 ORDER BY created_time DESC;

Thanks Frank for commenting out. I am not specific to that buyer_id. It can be many. And I don't need First 10. I need latest for that particular day. By latest means last 10 time for that particular BUYER_ID in desc order
Since you gave that example, I used it for my example :) .. well, by ORDERing created_time in DESCending, the FIRST 10 would be the latest 10 rows? ..ahh, why is created_time not defined as a DATETIME datatype?.. and if you want a previous days results, you can add a qualifier to the WHERE clause.
I have no idea why they didn't defined as DATETIME datatype. I am working on someone else design. They have defined that as a string. So I have to use that I guess. I have also posted sample output.
then you can cast that string to a DATETIME. The STRING is in proper DATETIME YEAR TO SECOND format!
And how can I enable this to all the BUYER_ID as currently it has enabled for only one. Can you provide me the example which can work in my scenario. it will be of great help to me. And FIRST 10.* will not work with Hive, as I am working in Hive and Hive supports SQL like syntax.
05:23
I'm not familiar with Hive, this example works for Informix, however not much different for other DB's.. remove buyer_id qualifier from the WHERE clause and ORDER BY buyer_id, created_time DESC.
I tried using that and I am working with Hive and FIRST 10.* doesn't working in that. Is there any other way apart from that?
not sure.. there must be some equivalent for obtaining the FIRST/LAST 10 rows of the result set in hive?.. perhaps some other ORM, like Hibernate, provides examples of how to obtain FIRST 10 rows?
I just did the searching, but couldn't found out anything.
I think using TOP 10 will do the trick! ragrawal.wordpress.com/2011/11/18/…
I tried using this SELECT TOP 10 FROM TestingTable1 WHERE ORDER BY buyer_id, created_time DESC; and it doesn't works.
LIMIT will restrict the whole query to 10 but I need TOP 10 for each BUYER_ID. That's is causing more problem.
Hey..
sorry for bothering you so much...
I dunno, then you might have to use several SELECT INTO TEMP queries with LIMIT = 10, one for each buyer_id, or some other workaround?.. In looking at HQL syntax, it appears to have considerable limitations!
Yes, Hive has lot of limitations as compared to SQL.
Yes indeed!.. Are you limited to using Hive or could you use an ORM like Hibernate?
I need to work with HiveQL only. As we have lot of data in Hadoop HDFS... So that is the reason we are using Hive to access the data from HDFS>
05:32
understood, is their only one flavor of Hive or are there others which perhaps might have more functionality?
I don't think so we have any flavor of Hive. They have different Hive versions and currently we are using Hive 0.6.
OK, I was thinkin perhaps using ROWID or COUNT in some form or another to achieve your goals..
I found one solution on SO
related to Hive. But not sure how to refactor this to work around for my problem.
Any idea how I can refactor this for my solution.
which is?.. maybe doing several SELECT INTO TEMP statements with LIMIT 10 and passing the buyer_id into $buyerid variable, or whatever?
0
Q: Hive getting top n records in group by query

TopCoderI have following table in hive user-id, user-name, user-address,clicks,impressions,page-id,page-name I need to find out top 5 users[user-id,user-name,user-address] by clicks for each page [page-id,page-name] I understand that we need to first group by [page-id,page-name] and within each group ...

see the above link.
05:39
yes!.. RANK() was what I saw in karmasphere.com/20101004185/Karmasphere-Analyst/…
I tried using RANK too. But I guess the version I have for Hive doesn't support RANK function.
I get error as- FAILED: Error in semantic analysis: line 3:30 Invalid Function rank
for this query-SELECT buyer_id, item_id, created_time
FROM (
SELECT buyer_id, item_id, rank(buyer_id) as rank, created_time
FROM testingtable1
DISTRIBUTE BY buyer_id, item_id
SORT BY buyer_id, item_id, created_time desc
) a
WHERE rank < 10
ORDER BY buyer_id, rank
sorry, wrong link.. twas the other link I had previously commented on
I think you have to create a user-defined func for rank?
ohhh I see..
package com.example.hive.udf;
02 import org.apache.hadoop.hive.ql.exec.UDF;
03
04 public final class Rank extends UDF{
05 private int counter;
06 private String last_key;
07 public int evaluate(final String key){
08 if ( !key.equalsIgnoreCase(this.last_key) ) {
09 this.counter = 0;
10 this.last_key = key;
11 }
12 return this.counter++;
13 }
14 }
I just saw that.. I thought RANK itself is a function in HiveQL..
let me try that then..
05:44
nope, its a UDF
yeah..
kay, good luck!
yup.. Thanks..
yw
BTW, rank is a native-SQL func in many data warehouse DB's like Informix, Oracle, etc. :)

last day (14 days later) »