Question

我们有一个csv：

targetID , cpd_number , assay_id , alt_assay_id , type_desc , operator ,   result_value, unit_value , experiment_date , discipline, activity_flag 
51, cpd-7788990 ,9999,0,  IC50  ,,10, uM ,  2006-07-01 00:00:00 , Biochemical ,
51, cpd-7788990 ,4444,5555, Ki , > ,5, uM ,  2006-08-01 00:00:00  ,  Enzyme ,
51, cpd-7788990 ,1212,2323,  IC50  ,,100, uM ,  2006-09-01 00:00:00  , Cell ,

我们的最终目标是获得：如果“cpd_number”（$ 2）相同但“纪律”（$ 10）不是“Cell”，则合并“纪律”（$ 10）的行是“Cell”而不是“细胞”在一起。（“学科”只有3个选项：生物化学，细胞，酶。）以下是理想的输出（注）新的“result_value”（$ 7）=“result_value”（$ 7），其中“纪律”（$ 10）是“Cell”除以“reason_value”（$ 7）的“纪律”（$ 10）是“生物化学”或“酶”。

targetID , cpd_number , Cell_assay_id , Cell_alt_assay_id , type_desc , assay_id , alt_assay_id , type_desc ,Operator,   result_value, unit_value ,Cell_experiment_date,experiment_date, Cell_discipline , discipline 
51,cpd-7788990,1212,2323, IC50 ,9999,0,IC50,,10,uM, 2006-09-01 00:00:00 , 2006-07-01 00:00:00 ,Cell,Biochemical
51,cpd-7788990,1212,2323, IC50 ,4444,5555,Ki,>,20,uM, 2006-09-01 00:00:00 , 2006-08-01 00:00:00 ,Cell,Enzyme

一次做这件事看起来很复杂。因此，我试图首先整合整行：如果“cpd_number”（$ 2）相同但“纪律”（$ 10）是“不同”，则合并“纪律”（$ 10）的行是“Cell”在一起线条“纪律”（10美元）不是“细胞”。在此合并之后，我们可以使用awk进一步清理/重命名标题。任何大师都可以提供一些如何写这个单行的想法吗？这只是一个玩具的例子。实际的csv文件非常庞大，所以从/ ^ 51 /开始可能并不理想。谢谢！

targetID , cpd_number , assay_id , alt_assay_id , type_desc , operator ,   result_value, unit_value , experiment_date , discipline, activity_flag, targetID , cpd_number , assay_id , alt_assay_id , type_desc , operator ,   result_value, unit_value , experiment_date , discipline, activity_flag 
51, cpd-7788990 ,9999,0,  IC50  ,,10, uM ,  2006-07-01 00:00:00 , Biochemical , 51, cpd-7788990 ,1212,2323,  IC50  ,,100, uM ,  2006-09-01 00:00:00  , Cell ,
51, cpd-7788990 ,4444,5555, Ki , > ,5, uM ,  2006-08-01 00:00:00  ,  Enzyme , 51, cpd-7788990 ,1212,2323,  IC50  ,,100, uM ,  2006-09-01 00:00:00  , Cell ,

额外的例子：

targetID , cpd_number , Cell_assay_id , Cell_alt_assay_id , type_desc , assay_id , alt_assay_id , type_desc ,Operator,   result_value, unit_value ,Cell_experiment_date,experiment_date, Cell_discipline , discipline 
51, cpd-7788990 ,9999,0,  IC50  ,,10, uM ,  2006-07-01 00:00:00 , Biochemical ,
51, cpd-7788990 ,4444,5555, Ki , > ,5, uM ,  2006-08-01 00:00:00  ,  Enzyme ,
51, cpd-7788990 ,1212,2323,  IC50  ,,100, uM ,  2006-09-01 00:00:00  , Cell ,
51, cpd-7788990 ,8888,9999,  IC50  ,,200, uM ,  2006-09-01 00:00:00  , Cell ,

输出：

targetID , cpd_number , Cell_assay_id , Cell_alt_assay_id , type_desc , assay_id , alt_assay_id , type_desc ,Operator,   result_value, unit_value ,Cell_experiment_date,experiment_date, Cell_discipline , discipline 
51,cpd-7788990,1212,2323, IC50 ,9999,0,IC50,,10,uM, 2006-09-01 00:00:00 , 2006-07-01 00:00:00 ,Cell,Biochemical
51,cpd-7788990,1212,2323, IC50 ,4444,5555,Ki,>,20,uM, 2006-09-01 00:00:00 , 2006-08-01 00:00:00 ,Cell,Enzyme
51,cpd-7788990,8888,9999, IC50 ,9999,0,IC50,,20,uM, 2006-09-01 00:00:00 , 2006-07-01 00:00:00 ,Cell,Biochemical
51,cpd-7788990,8888,9999, IC50 ,4444,5555,Ki,>,40,uM, 2006-09-01 00:00:00 , 2006-08-01 00:00:00 ,Cell,Enzyme

Answer 1

这是一个根据您的示例输入和最终所需输出一起入侵的awk脚本。随意调整它以满足您的需求。它应该足以让你开始。它需要两次传递给你的csv文件。在第一遍中，它构建了一个基于第二列的数组，其中规则为单元格，在第二遍中，它将这些行格式化在一起。由于您尚未说明如何处理没有Cell规则的行，以下解决方案将忽略它们。

script.awk的内容

BEGIN { 
    FS  = " *, *"                             # Set input field sep to this regex
    OFS = ","                                 # Set output field sep to comma
}

NR==FNR {                                     # In the first pass to the file
    if ($(NF-1) == "Cell") {                  # If the second last field is Cell
        flds[$2,$3,$4] = $3 OFS $4 OFS $5;    # Create an array to store col 3,4 and 5 separated by comma
        date[$2,$3,$4] = $9                   # Store date
        result[$2,$3,$4] = $7                 # Store col 7
    }
    next                                      # Move to the next record
} 

{                                             # For the second pass to the file
    for (fld in flds) {                       # For every entry in our array
        split (fld, tmp, SUBSEP);             # Split the composite key
        if ($(NF-1) != "Cell" && tmp[1] == $2) {   # If the last field is not Cell and first piece of key is same as col 2
            line = $0                         # Save the current line in a variable
            $3 = flds[fld] OFS $3             # modify col3 to put the value from array in front of col3
            $7 = result[fld] / $7             # Calculate the new result value
            $9 = date[fld] OFS $9             # Add the date
            $(NF-1) = "Cell" OFS $(NF-1)      # Place the Cell text
            NF--                              # Remove the last field
            print                             # print the line
            $0 = line                         # Swap the modified line back
        }
    }
}

$(NF-1) == "Cell" {                           # If the last field is Cell don't print it 
    next
}

运行方式如下：

$ awk -f script.awk file file
51,cpd-7788990,1212,2323,IC50,9999,0,IC50,,10,uM,2006-09-01 00:00:00,2006-07-01 00:00:00,Cell,Biochemical
51,cpd-7788990,8888,9999,IC50,9999,0,IC50,,20,uM,2006-09-01 00:00:00,2006-07-01 00:00:00,Cell,Biochemical
51,cpd-7788990,1212,2323,IC50,4444,5555,Ki,>,20,uM,2006-09-01 00:00:00,2006-08-01 00:00:00,Cell,Enzyme
51,cpd-7788990,8888,9999,IC50,4444,5555,Ki,>,40,uM,2006-09-01 00:00:00,2006-08-01 00:00:00,Cell,Enzyme

您可以在BEGIN块中包含header语句的打印。

awk基于多个字段匹配/不匹配合并行

1 个答案: