Question

我目前正在从firebase将数据库导出为JSON，并希望将其上传到Bigquery。但是，数据库中的某些字段名称具有嵌套信息，而Bigquery不会以这种方式接受它。如何删除＆＃39; Peripherals＆＃39;来自我的JSON中存在的每个数据集。但它并不存在于每个数据集中。我提供了一个JSON代码如下所示的示例。谢谢你的帮助！

   {"AppName": "DataWorks", "foundedPeripheralCount": 1, "version": "1.6.1(8056)", "deviceType": "iPhone 6", "createdAt": "2017-04-05T07:05:30.408Z", "updatedAt": "2017-04-05T07:08:49.569Z", "Peripherals": {"1CA726ED-32B1-43B4-9071-B58BBACE20A8": "Arduino"}, "connectedPeripheralCount": 1, "iOSVersion": "10.2.1"}
{"objectId": "20H5Hg2INB", "foundedPeripheralCount": 0, "DeviceVendorID": "5B7F085E-B3B6-4270-97DC-F42903CDEAC1", "version": "1.3.5(5801)", "deviceType": "iPhone 6", "createdAt": "2015-11-10T06:16:45.459Z", "updatedAt": "2015-11-10T06:16:45.459Z", "connectedPeripheralCount": 0, "iOSVersion": "9.1"}
{"AppName": "DataWorks", "foundedPeripheralCount": 2, "version": "1.6.2(8069)", "deviceType": "iPhone 6s", "createdAt": "2017-04-12T10:05:05.937Z", "updatedAt": "2017-07-06T07:33:02.006Z", "Peripherals": {"060EBAFD-3120-4AAD-8B0A-EC14A323FA25": "28902               ", "identifierInternalSensors": "Internal Sensors", "0521A273-FAA5-462E-B9EC-FBB3D60F5E99": "28895               "}, "connectedPeripheralCount": 8, "iOSVersion": "10.2.1"}

我试过这个

import json
with open('firetobq_peripheral.json') as out_file:
    out = json.load(out_file)
    for element in out:
      del element['Peripherals']

print(out)

但我收到此错误

Traceback (most recent call last):
  File "editjson.py", line 3, in <module>
    out = json.load(out_file)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 290, in load
    **kw)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/__init__.py", line 338, in loads
    return _default_decoder.decode(s)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 369, in decode
    raise ValueError(errmsg("Extra data", s, end, len(s)))
ValueError: Extra data: line 2 column 1 - line 629 column 1 (char 311 - 203056)

Answer 1

看起来'firetobq_peripheral.json'中的数据无效json。如果每行都在新行上，您可以使用以下代码：

with open('firetobq_peripheral.json', 'r') as in_file:
    dicts = []
    for line in in_file.readlines() : 
        d = json.loads(line.strip())
        if d.get('Peripherals'): 
            del d['Peripherals']
        dicts += [d]

with open('firetobq_peripheral.json', 'w') as out_file:
    out_file.write('[\n')
    for i,v in enumerate(dicts):
        out_file.write(json.dumps(v)+('\n' if i == len(dicts)-1 else ',\n'))
    out_file.write(']')

<小时/> 将此代码用于格式正确的json数据：

with open('firetobq_peripheral.json', 'r') as in_file:
    dicts = json.load(in_file)
    for d in dicts: 
        if d.get('Peripherals'): 
            del d['Peripherals']

with open('firetobq_peripheral.json', 'w') as out_file:
    out_file.write(json.dumps(dicts, indent=2))

使用Python

1 个答案: