如何拆分大文件？

Question

我有一个大型CSV文件（7.3GB; 16,300,000行），如何将此文件拆分为两个文件？

Answer 1

您是否看过split命令？有关详细信息，请参阅this man 页面。

此page包含此命令的示例用法。

除了：

如果您不太确定具体命令是什么，man -k命令对于查找unix / linux命令非常有用。使用man -k命令指定关键字，系统将提取相关命令。如，

% man -k split

将产生：

csplit (1)           - split a file into sections determined by context lines
dirsplit (1)         - splits directory into multiple with equal size
dpkg-split (1)       - Debian package archive split/join tool
gpgsplit (1)         - Split an OpenPGP message into packets
pnmsplit (1)         - split a multi-image portable anymap into multiple single-image files
ppmtoyuvsplit (1)    - convert a portable pixmap into 3 subsampled raw YUV files
split (1)            - split a file into pieces
splitdiff (1)        - separate out incremental patches
splitfont (1)        - extract characters from an ISO-type font.
URI::Split (3pm)     - Parse and compose URI strings
wcstok (3)           - split wide-character string into tokens
yuvsplittoppm (1)    - convert a Y- and a U- and a V-file into a portable pixmap
zipsplit (1)         - split a zipfile into smaller zipfiles

Answer 2

split -d -n l / N filename.csv tempfile.part。

将文件拆分为N个文件而不拆分行。如上面的注释中所述，标题不会在每个文件中重复。