如何使用map和split来解析python的文本文件?

时间:2018-02-20 01:58:32

标签: python pyspark

我有一个RDD,其数据如下所示:

[‘ID: <166.1079>\nDate: 7 Dec 2010 16:06:42\nOwner: Mary Jones\nLab-Techs: Bob Chandra\nHarry Alley\n Julie Stevens\nNotes: West Position\nTemp: 1.0 C\nX-FileName:  temp-g\nX-Storage:  3544-ABC
X-Location: Building-A’]

它是一个实验室记录,一个文本文件。我有这个python代码无法正常工作。它不是正确地拆分字段然后将每个字段放在他们尊重的RDD中。这是代码:

lab_data = sc.textFile('data_file')
fields = lab_data.map(lambda x: x.split('Notes:')(0).split('\n')
ID_field = fields(0).map(lambda x: x.split(':')(1).trim())
date_field = fields(1).map(lambda x: x.split(':')(1).trim())
owner_field = fields(2).map(lambda x: x.split(':')(1).trim()) 

基本上,我试图抓住ID,日期和所有者等前三个字段,但忽略了所有其他字段。因此,字段RDD将具有如下值:

ID field = <166.1079>
Date field =  7 Dec 2010 16:06:42
Owner field =  Marry Jones

你能帮我纠正一下代码吗?

1 个答案:

答案 0 :(得分:1)

要解析列表中的实验室记录的字符串,您可以执行以下操作:

代码:

java.lang.RuntimeException: Unable to start activity ComponentInfo{com.example.kim.deptenv2/com.example.kim.deptenv2.UserAreaActivity}: android.view.InflateException: Binary XML file line #77: Error inflating class android.support.design.widget.NavigationView
                                                                              at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2348)
                                                                              at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2410)
                                                                              at android.app.ActivityThread.access$800(ActivityThread.java:151)
                                                                              at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1313)
                                                                              at android.os.Handler.dispatchMessage(Handler.java:102)
                                                                              at android.os.Looper.loop(Looper.java:135)
                                                                              at android.app.ActivityThread.main(ActivityThread.java:5345)
                                                                              at java.lang.reflect.Method.invoke(Native Method)
                                                                              at java.lang.reflect.Method.invoke(Method.java:372)
                                                                              at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:947)
                                                                              at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:742)
                                                                              at de.robv.android.xposed.XposedBridge.main(XposedBridge.java:102)
                                                                           Caused by: android.view.InflateException: Binary XML file line #77: Error inflating class android.support.design.widget.NavigationView
                                                                              at android.view.LayoutInflater.createView(LayoutInflater.java:633)
                                                                              at android.view.LayoutInflater.createViewFromTag(LayoutInflater.java:743)
                                                                              at android.view.LayoutInflater.rInflate(LayoutInflater.java:806)
                                                                              at android.view.LayoutInflater.inflate(LayoutInflater.java:504)
                                                                              at de.robv.android.xposed.XposedBridge.invokeOriginalMethodNative(Native Method)
                                                                              at de.robv.android.xposed.XposedBridge.handleHookedMethod(XposedBridge.java:334)
                                                                              at android.view.LayoutInflater.inflate(<Xposed>)
                                                                              at android.view.LayoutInflater.inflate(LayoutInflater.java:414)
                                                                              at android.view.LayoutInflater.inflate(LayoutInflater.java:365)
                                                                              at android.support.v7.app.AppCompatDelegateImplV9.setContentView(AppCompatDelegateImplV9.java:292)
                                                                              at android.support.v7.app.AppCompatActivity.setContentView(AppCompatActivity.java:140)
                                                                              at com.example.kim.deptenv2.UserAreaActivity.onCreate(UserAreaActivity.java:37)
                                                                              at android.app.Activity.performCreate(Activity.java:6012)
                                                                              at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1106)
                                                                              at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2292)
                                                                              at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2410) 
                                                                              at android.app.ActivityThread.access$800(ActivityThread.java:151) 
                                                                              at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1313) 
                                                                              at android.os.Handler.dispatchMessage(Handler.java:102) 
                                                                              at android.os.Looper.loop(Looper.java:135) 
                                                                              at android.app.ActivityThread.main(ActivityThread.java:5345) 
                                                                              at java.lang.reflect.Method.invoke(Native Method) 
                                                                              at java.lang.reflect.Method.invoke(Method.java:372) 
                                                                              at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:947) 
                                                                              at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:742) 
                                                                              at de.robv.android.xposed.XposedBridge.main(XposedBridge.java:102) 
                                                                           Caused by: java.lang.reflect.InvocationTargetException
                                                                              at java.lang.reflect.Constructor.newInstance(Native Method)
                                                                              at java.lang.reflect.Constructor.newInstance(Constructor.java:288)
                                                                              at android.view.LayoutInflater.createView(LayoutInflater.java:607)
                                                                              at android.view.LayoutInflater.createViewFromTag(LayoutInflater.java:743) 
                                                                              at android.view.LayoutInflater.rInflate(LayoutInflater.java:806) 
                                                                              at android.view.LayoutInflater.inflate(LayoutInflater.java:504) 
                                                                              at de.robv.android.xposed.XposedBridge.invokeOriginalMethodNative(Native Method) 
                                                                              at de.robv.android.xposed.XposedBridge.handleHookedMethod(XposedBridge.java:334) 
                                                                              at android.view.LayoutInflater.inflate(<Xposed>) 
                                                                              at android.view.LayoutInflater.inflate(LayoutInflater.java:414) 
                                                                              at android.view.LayoutInflater.inflate(LayoutInflater.java:365) 
                                                                              at android.support.v7.app.AppCompatDelegateImplV9.setContentView(AppCompatDelegateImplV9.java:292) 
                                                                              at android.support.v7.app.AppCompatActivity.setContentView(AppCompatActivity.java:140) 
                                                                              at com.example.kim.deptenv2.UserAreaActivity.onCreate(UserAreaActivity.java:37) 
                                                                              at android.app.Activity.performCreate(Activity.java:6012) 
                                                                              at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1106) 
                                                                              at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2292) 
                                                                              at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2410) 
                                                                              at android.app.ActivityThread.access$800(ActivityThread.java:151) 
                                                                              at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1313) 
                                                                              at android.os.Handler.dispatchMessage(Handler.java:102) 
                                                                              at android.os.Looper.loop(Looper.java:135) 
                                                                              at android.app.ActivityThread.main(ActivityThread.java:5345) 
                                                                              at java.lang.reflect.Method.invoke(Native Method) 
                                                                              at java.lang.reflect.Method.invoke(Method.java:372) 
                                                                              at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:947) 
                                                                              at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:742) 
                                                                              at de.robv.android.xposed.XposedBridge.main(XposedBridge.java:102) 
                                                                           Caused by: android.view.InflateException: Binary XML file line #2: Error inflating class <unknown>
                                                                              at android.view.LayoutInflater.createView(LayoutInflater.java:633)
                                                                              at com.android.internal.policy.impl.PhoneLayoutInflater.onCreateView(PhoneLayoutInflater.java:55)
                                                                              at android.view.LayoutInflater.onCreateView(LayoutInflater.java:682)
                                                                              at android.view.LayoutInflater.createViewFromTag(LayoutInflater.java:741)
                                                                              at android.view.LayoutInflater.inflate(LayoutInflater.java:482)
                                                                              at de.robv.android.xposed.XposedBridge.invokeOriginalMethodNative(Native Method)
                                                                              at de.robv.android.xposed.XposedBridge.handleHookedMethod(XposedBridge.java:334)
                                                                              at android.view.LayoutInflater.inflate(<Xposed>)
                                                                              at android.view.LayoutInflater.inflate(LayoutInflater.java:414)
                                                                              at android.support.design.internal.NavigationMenuPresenter.inflateHeaderView(NavigationMenuPresenter.java:211)
                                                                              at android.support.design.widget.NavigationView.inflateHeaderView(NavigationView.java:269)
                                                                              at android.support.design.widget.NavigationView.<init>(NavigationView.java:178)
                                                                              at android.support.design.widget.NavigationView.<init>(NavigationView.java:100)
                                                                              at java.lang.reflect.Constructor.newInstance(Native Method) 
                                                                              at java.lang.reflect.Constructor.newInstance(Constructor.java:288) 
                                                                              at android.view.LayoutInflater.createView(LayoutInflater.java:607) 
                                                                              at android.view.LayoutInflater.createViewFromTag(LayoutInflater.java:743) 
                                                                              at android.view.LayoutInflater.rInflate(LayoutInflater.java:806) 
                                                                              at android.view.LayoutInflater.inflate(LayoutInflater.java:504) 
                                                                              at de.robv.android.xposed.XposedBridge.invokeOriginalMethodNative(Native Method) 
                                                                              at de.robv.android.xposed.XposedBridge.handleHookedMethod(XposedBridge.java:334) 
                                                                              at android.view.LayoutInflater.inflate(<Xposed>) 
                                                                              at android.view.LayoutInflater.inflate(LayoutInflater.java:414) 
                                                                              at android.view.LayoutInflater.inflate(LayoutInflater.java:365) 
                                                                              at android.support.v7.app.AppCompatDelegateImplV9.setContentView(AppCompatDelegateImplV9.java:292) 
                                                                              at android.support.v7.app.AppCompatActivity.setContentView(AppCompatActivity.java:140) 
                                                                              at com.example.kim.deptenv2.UserAreaActivity.onCreate(UserAreaActivity.java:37) 
                                                                              at android.app.Activity.performCreate(Activity.java:6012) 
                                                                              at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1106) 
                                                                              at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2292) 
                                                                              at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2410) 
                                                                              at android.app.ActivityThread.access$800(ActivityThread.java:151) 
                                                                              at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1313) 
                                                                              at android.os.Handler.dispatchMessage(Handler.java:102) 
                                                                              at android.os.Looper.loop(Looper.java:135) 
                                                                              at android.app.ActivityThread.main(ActivityThread.java:5345) 
                                                                              at java.lang.reflect.Method.invoke(Native Method) 
                                                                              at java.lang.reflect.Method.invoke(Method.java:372) 
                                                                              at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:947) 
                                                                              at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:742) 
                                                                              at de.robv.android.xposed.XposedBridge.main(XposedBridge.java:102) 
                                                                           Caused by: java.lang.reflect.InvocationTargetException
                                                                              at java.lang.reflect.Constructor.newInstance(Native Method)
                                                                              at java.lang.reflect.Constructor.newInstance(Constructor.java:288)
                                                                              at android.view.LayoutInflater.createView(LayoutInflater.java:607)
                                                                              at com.android.internal.policy.impl.PhoneLayoutInflater.onCreateView(PhoneLayoutInflater.java:55) 
                                                                              at android.view.LayoutInflater.onCreateView(LayoutInflater.java:682) 
                                                                              at android.view.LayoutInflater.createViewFromTag(LayoutInflater.java:741) 
                                                                              at android.view.LayoutInflater.inflate(LayoutInflater.java:482) 
                                                                              at de.robv.android.xposed.XposedBridge.invokeOriginalMethodNative(Native Method) 
                                                                              at de.robv.android.xposed.XposedBridge.handleHookedMethod(XposedBridge.java:334) 
                                                                              at android.view.LayoutInflater.inflate(<Xposed>) 
                                                                              at android.view.LayoutInflater.inflate(LayoutInflater.java:414) 
                                                                              at android.support.design.internal.NavigationMenuPresenter.inflateHeaderView(NavigationMenuPresenter.java:211) 
                                                                              at android.support.design.widget.NavigationView.inflateHeaderView(NavigationView.java:269) 
                                                                              at android.support.design.widget.NavigationView.<init>(NavigationView.java:178) 
                                                                              at android.support.design.widget.NavigationView.<init>(NavigationView.java:100) 
                                                                              at java.lang.reflect.Constructor.newInstance(Native Method) 
                                                                              at java.lang.reflect.Constructor.newInstance(Constructor.java:288) 
                                                                              at android.view.LayoutInflater.createView(LayoutInflater.java:607) 
                                                                              at android.view.LayoutInflater.createViewFromTag(LayoutInflater.java:743) 
                                                                              at android.view.LayoutInflater.rInflate(LayoutInflater.java:806) 
                                                                              at android.view.LayoutInflater.inflate(LayoutInflater.java:504) 
                                                                              at de.robv.android.xposed.XposedBridge.invokeOriginalMethodNative(Native Method) 
                                                                              at de.robv.android.xposed.XposedBridge.handleHookedMethod(XposedBridge.java:334) 
                                                                              at android.view.LayoutInflater.inflate(<Xposed>) 
                                                                              at android.view.LayoutInflater.inflate(LayoutInflater.java:414) 
                                                                              at android.view.LayoutInflater.inflate(LayoutInflater.java:365) 
                                                                              at android.support.v7.app.AppCompatDelegateImplV9.setContentView(AppCompatDelegateImplV9.java:292) 
                                                                              at android.support.v7.app.AppCompatActivity.setContentView(AppCompatActivity.java:140) 
                                                                              at com.example.kim.deptenv2.UserAreaActivity.onCreate(UserAreaActivity.java:37) 
                                                                              at android.app.Activity.performCreate(Activity.java:6012) 
                                                                              at android.app.Instrumentation.callActivityOnCreate(Instrumentation.java:1106) 
                                                                              at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2292) 
                                                                              at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2410) 
                                                                              at android.app.ActivityThread.access$800(ActivityThread.java:151) 
                                                                              at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1313) 
                                                                              at android.os.Handler.dispatchMessage(Handler.java:102) 
                                                                              at android.os.Looper.loop(Looper.java:135) 

测试代码:

data = dict(tuple(x.strip() for x in line.split(':', 1))
            for line in record.split('\n')[:3])

文件1:

with open('file1', 'rU') as f:
    data = dict(tuple(x.strip() for x in line.split(':', 1))
                for line in f.readlines()[:3])
print(data)

结果:

ID: <166.1079>
Date: 7 Dec 2010 16:06:42
Owner: Mary Jones
Lab-Techs: Bob Chandra
Harry Alley
 Julie Stevens
Notes: West Position
Temp: 1.0 C
X-FileName:  temp-g
X-Storage:  3544-ABC
X-Location: Building-A