Question

我在一个目录中有几个.csv个文件。我希望迭代这些文件合并/在一定条件下将它们组合成单个.csv文件。

每个文件使用相同的命名约定：

Date       Name   City   Supervisor

2015-01-01_Steve__Boston_Steven.csv
2015-10-03_Michael_Dallas_Thomas.csv
2015-02-10_John_NewYork_Michael.csv

每个文件只包含一个长度不一的列：

2015-01-01_Steve__Boston_Steven.csv

Sales
100
20
3
100
200

或

2015-10-03_Michael_Dallas_Thomas.csv

Sales
1
2

2015-02-10_John_NewYork_Michael.csv

或

Sales
1
2
3

因为标题＆＃34;销售＆＃34;在每个文件中可能以不同的方式命名我想跳过第一行并始终与第二行一起开始。

我想得到一个包含以下信息的决赛表：

Sales Name     City    Supervisor
100   Steve    Boston  Steven
20    Steve    Boston  Steven
30    Steve    Boston  Steven
3     Steve    Boston  Steven
100   Steve    Boston  Steven
200   Steve    Boston  Steven
1     Michael  Dallas  Thomas
2     Michael  Dallas  Thomas
1     John     NewYork Michael
2     John     NewYork Michael
3     John     NewYork Michael

我是python的新手，对此给您带来的不便深表歉意。

我尝试过：

import pandas as pd
from os import listdir

source_path, dst_path = '/oldpath', '/newpath'

files = [f for f in listdir(source_path) if f.endswith('.csv')]

def combining_data(files):
    df_list = []
    for filename in files:
        df_list.append(pd.read_csv(filename))

combining_data(files)

但不幸的是，这并不会产生所需的输出

Answer 1

这需要多个步骤。首先，我将解析CSV名称以获取名称，城市和主管。从它的外观来看，您可以使用名称上的split来获取这些值。然后，您必须读取文件并将其附加到新CSV。使用熊猫也有点矫枉过正。您可以使用csv模块。

import csv
import os

files = [f for f in os.listdir(source_path) if f.endswith('.csv')]

with open(os.path.join(source_path, 'new_csv.csv'), 'wb') as new:
    writer = csv.writer(new)
    writer.writerow(['Sales','Name','City','Supervisor'])  # write the header for the new csv
    for f in files:
        split = f[:-4].split('_')  # split the filename on _, while removing the .csv
        name = split[1]  # extract the name
        city = split[2]  # extract the city
        supervisor = split[3]  # extract the supervisor
        with open(os.path.join(source_path, f), 'rb') as readfile:
            reader = csv.reader(readfile)
            reader.next()  # Skip the header from the file you're reading
            for row in reader:
                writer.writerow([row[0], name, city, supervisor])  # write to the new csv

Answer 2

用熊猫：

import pandas as pd
import os

df=pd.DataFrame(columns=['Sales','Name','City','Supervisor'])
files = [f for f in os.listdir('.') if f.startswith('2015')]

for a in files:
    df1 = pd.read_csv(a, header=None, skiprows=1, names=['Sales'])
    len1 = len(df1.index)
    f = [b for b in a.split('_') if b]
    l2, l3 = [f[1], f[2], f[3][:-4]], ['Name','City','Supervisor']
    for b,c in zip(l2,l3):
        ser = pd.Series(data=[b for _ in range(len1)],index=range(len1))
        df1[c]=ser
    df = pd.concat([df,df1],axis=0)
df.index = range(len(df.index))
df.to_csv('new_csv.csv', index=None)
df

输出：

CPU times: user 16 ms, sys: 0 ns, total: 16 ms
Wall time: 22.6 ms

如何在某些条件下合并.csv文件？

2 个答案: