将CSV数据上传到Hive表时出现问题

时间:2018-10-10 10:42:44

标签: hive

用于创建配置单元表的格式serde property

我的csv数据csv file demo

csv数据的某些字段具有换行符,因此会引起问题,例如,当我们从表中选择一列时,具有换行符“ \ n”的字段会分成多个行。

csv文件的猫one row data of csv file

“ 2018年5月27日”,77266157-8b26-46bb-93f0-a1ef20931a,'2124272900300,OD212427213119003000,62029200,已交付,NON_FBF,BKPE7CPDZ2,ECHO RED1,黑色和红色07171202,FABOK019000001,NA,5,NA,05 / 27 / 18,739,674,65,1,739,Jishad,Jishad,“ NASSCO瓷砖POYILTHODI TOWER近税支票过帐后忠清寺”,喀拉拉邦KOZHIKODE,673631,“ 2018年5月27日15:59:29”,“ 2018年5月29日” 10:00:00“ ,, FM324875856,10,8,6,0.3,没有引起问题的行。 NASSCO标题和POYILTHODI之间有换行符

预期结果Expected result of the query

实际结果Getting this result

帮助我将CSV文件数据准确地加载到配置单元表中,并获得所需的结果

1 个答案:

答案 0 :(得分:0)

使用Perl将多行记录合并为一行的解决方法。请检查是否可行

<ScrollViewer VerticalScrollBarVisibility="Auto">
    <ItemsControl ItemsSource="{Binding Images}">
        <ItemsControl.ItemTemplate>
            <DataTemplate>
                <!-- Bind IsExpanded to MouseOver to expand it automatically -->
                <Expander Width="100" IsExpanded="{Binding Path=IsMouseOver, RelativeSource={RelativeSource Self}, Mode=OneWay}">
                    <!-- Normal View -->
                    <Expander.Header>
                        <StackPanel>
                            <Image Height="70" Width="70"/>
                            <Label Content="Title"/>
                        </StackPanel>
                    </Expander.Header>
                    <!-- Details displayed underneath, with same width as the preview -->
                    <StackPanel Height="80" Background="LightGray" Width="{Binding Path=Width, RelativeSource={RelativeSource AncestorType={x:Type Expander}}}">
                        <Label Content="DETAILS"/>
                        <TextBlock Text="Lorem ipsum dolor sit amet" TextWrapping="Wrap"/>
                    </StackPanel>
                    <!-- Set negative Margin to render Details outside of the Boundaries -->
                    <Expander.Style>
                        <Style TargetType="{x:Type Expander}">
                            <Style.Triggers>
                                <Trigger Property="IsMouseOver" Value="True">
                                    <!-- Note: The Margin must be the same as the Detail Height -->
                                    <Setter Property="Margin" Value="0,0,0,-80"/>
                                </Trigger>
                            </Style.Triggers>
                        </Style>
                    </Expander.Style>
                </Expander>                      
            </DataTemplate>
        </ItemsControl.ItemTemplate>

        <!-- Bring Item to front when Mouse is over it -->
        <ItemsControl.ItemContainerStyle>
            <Style TargetType="{x:Type ContentPresenter}">
                <Style.Triggers>
                    <Trigger Property="IsMouseOver" Value="True">
                        <Setter Property="Panel.ZIndex" Value="1" />
                    </Trigger>
                </Style.Triggers>
            </Style>
        </ItemsControl.ItemContainerStyle>

        <ItemsControl.ItemsPanel>
            <ItemsPanelTemplate>
                <WrapPanel/>
            </ItemsPanelTemplate>
        </ItemsControl.ItemsPanel>
    </ItemsControl>
</ScrollViewer>

更新:

> nl kislay_stack.dat
     1  "May 27, 2018",77266157-8b26-46bb-93f0-a1ef29f0931a,'21242721311900300,OD212427213119003000,62029200,Delivered,NON_FBF,BKPE7CPYUWYFVD
     2  Z2,PUMA ECHO RED1,Puma Echo Plus 27 L Medium Backpack Black And Red 07171202,FABOK01900002001,NA,5,NA,05/27/18,739,674,65,1,739,Jisha
     3  d ,Jishad ,"NASSCO TILES
     4  POYILTHODI TOWER
     5  NEAR TAX CHECK POST FAROKE CHUNGAM",,KOZHIKODE,Kerala,673631,"May 27, 2018 15:59:29","May 29, 2018 10:00:00",,FMPC0324875856,10,8,6,0.3,NO
     6  "May 28, 2018",77266157-8b26-46bb-93f0-a1ef29f0931a,'21242721311900300,OD212427213119003000,62029200,Delivered,NON_FBF,BKPE7CPYUWYFVD
     7  Z2,PUMA ECHO RED1,Puma Echo Plus 27 L Medium Backpack Black And Red 07171202,FABOK01900002001,NA,5,NA,05/27/18,739,674,65,1,739,Jisha
     8  d ,Jishad ,"NASSCO TILES2
     9  POYILTHODI TOWER
    10  NEAR TAX CHECK POST FAROKE CHUNGAM",,KOZHIKODE,Kerala,673631,"May 27, 2018 15:59:29","May 29, 2018 10:00:00",,FMPC0324875856,10,8,6,0.3,NO
    11  "May 29, 2018",77266157-8b26-46bb-93f0-a1ef29f0931a,'21242721311900300,OD212427213119003000,62029200,Delivered,NON_FBF,BKPE7CPYUWYFVD
    12  Z2,PUMA ECHO RED1,Puma Echo Plus 27 L Medium Backpack Black And Red 07171202,FABOK01900002001,NA,5,NA,05/27/18,739,674,65,1,739,Jisha
    13  d ,Jishad ,"NASSCO TILES3
    14  POYILTHODI TOWER
    15  NEAR TAX CHECK POST FAROKE CHUNGAM",,KOZHIKODE,Kerala,673631,"May 27, 2018 15:59:29","May 29, 2018 10:00:00",,FMPC0324875856,10,8,6,0.3,NO
    16  "May 30, 2018",77266157-8b26-46bb-93f0-a1ef29f0931a,'21242721311900300,OD212427213119003000,62029200,Delivered,NON_FBF,BKPE7CPYUWYFVD
    17  Z2,PUMA ECHO RED1,Puma Echo Plus 27 L Medium Backpack Black And Red 07171202,FABOK01900002001,NA,5,NA,05/27/18,739,674,65,1,739,Jisha
    18  d ,Jishad ,"NASSCO TILES4
    19  POYILTHODI TOWER
    20  NEAR TAX CHECK POST FAROKE CHUNGAM",,KOZHIKODE,Kerala,673631,"May 27, 2018 15:59:29","May 29, 2018 10:00:00",,FMPC0324875856,10,8,6,0.3,NO
> perl -ne ' if ( /^\"\S+ \d+, \d{4}\"/ && $y++) { $x=~s/\n//g;print "$x\n";$x=$_ } else { $x.=$_ } END { $x=~s/\n//g; print "$x\n";} ' kislay_stack.dat | nl
     1  "May 27, 2018",77266157-8b26-46bb-93f0-a1ef29f0931a,'21242721311900300,OD212427213119003000,62029200,Delivered,NON_FBF,BKPE7CPYUWYFVDZ2,PUMA ECHO RED1,Puma Echo Plus 27 L Medium Backpack Black And Red 07171202,FABOK01900002001,NA,5,NA,05/27/18,739,674,65,1,739,Jishad ,Jishad ,"NASSCO TILES POYILTHODI TOWER NEAR TAX CHECK POST FAROKE CHUNGAM",,KOZHIKODE,Kerala,673631,"May 27, 2018 15:59:29","May 29, 2018 10:00:00",,FMPC0324875856,10,8,6,0.3,NO
     2  "May 28, 2018",77266157-8b26-46bb-93f0-a1ef29f0931a,'21242721311900300,OD212427213119003000,62029200,Delivered,NON_FBF,BKPE7CPYUWYFVDZ2,PUMA ECHO RED1,Puma Echo Plus 27 L Medium Backpack Black And Red 07171202,FABOK01900002001,NA,5,NA,05/27/18,739,674,65,1,739,Jishad ,Jishad ,"NASSCO TILES2 POYILTHODI TOWER NEAR TAX CHECK POST FAROKE CHUNGAM",,KOZHIKODE,Kerala,673631,"May 27, 2018 15:59:29","May 29, 2018 10:00:00",,FMPC0324875856,10,8,6,0.3,NO
     3  "May 29, 2018",77266157-8b26-46bb-93f0-a1ef29f0931a,'21242721311900300,OD212427213119003000,62029200,Delivered,NON_FBF,BKPE7CPYUWYFVDZ2,PUMA ECHO RED1,Puma Echo Plus 27 L Medium Backpack Black And Red 07171202,FABOK01900002001,NA,5,NA,05/27/18,739,674,65,1,739,Jishad ,Jishad ,"NASSCO TILES3 POYILTHODI TOWER NEAR TAX CHECK POST FAROKE CHUNGAM",,KOZHIKODE,Kerala,673631,"May 27, 2018 15:59:29","May 29, 2018 10:00:00",,FMPC0324875856,10,8,6,0.3,NO
     4  "May 30, 2018",77266157-8b26-46bb-93f0-a1ef29f0931a,'21242721311900300,OD212427213119003000,62029200,Delivered,NON_FBF,BKPE7CPYUWYFVDZ2,PUMA ECHO RED1,Puma Echo Plus 27 L Medium Backpack Black And Red 07171202,FABOK01900002001,NA,5,NA,05/27/18,739,674,65,1,739,Jishad ,Jishad ,"NASSCO TILES4 POYILTHODI TOWER NEAR TAX CHECK POST FAROKE CHUNGAM",,KOZHIKODE,Kerala,673631,"May 27, 2018 15:59:29","May 29, 2018 10:00:00",,FMPC0324875856,10,8,6,0.3,NO
>