Question

如果mytable中的值相同，我想在col1中删除重复项。

WITH CTE AS
(
    SELECT 
        *, ROW_NUMBER() OVER (PARTITION BY col1 ORDER BY col1) AS RN
    FROM
        mytable
)
DELETE FROM CTE 
WHERE RN <> 1

我遇到错误：

在语句中无法识别“ DELETE”，“ FROM”，“ CTE”附近的输入

Answer 1

我认为Hive不支持DELETE的语法。试试这个：

DELETE FROM mytable t
    WHERE t.id > (SELECT MIN(t2.id)  -- some sort of unique id
                  FROM t t2
                  WHERE t2.id = t.id
                 );

如果您有完整的副本，则以上内容将无效。在最新版本的Hive中，您可以使用MERGE。在旧版本中：

create table temp_t as
     select distinct t.*
     from t;

truncate table t;

insert into t
    select * from temp_t;

当然，请先备份表！

Answer 2

另一种方式：假设您具有“唯一ID列”。

  Delete from MyTable where ID in 
    (SELECT ID FROM (SELECT *, ROW_NUMBER() OVER (PARTITION BY col1 ORDER BY col1) AS RN
        FROM mytable) a where RN <> 1)

无法在语句中识别“ DELETE”，“ FROM”，“ CTE”附近的输入

2 个答案: