Question

我有一个JavaDStream，它从外部源获取数据。我正在尝试集成Spark Streaming和SparkSQL。众所周知，JavaDStream是由JavaRDD组成的。我只能在拥有JavaRDD时应用函数applySchema（）。请帮我转换为JavaRDD。我知道scala中有函数，它更容易。但是用Java帮助我。

Answer 1

您无法将DStream转换为RDD。如您所述，DStream包含RDD。访问RDD的方法是使用foreachRDD将函数应用于DStream的每个RDD。请参阅文档：https://spark.apache.org/docs/1.1.0/api/java/org/apache/spark/streaming/api/java/JavaDStreamLike.html#foreachRDD(org.apache.spark.api.java.function.Function2)

Answer 2

您必须首先使用forEachRDD访问DStream中的所有RDD：

public class MyFragmentLayout extends Fragment {

View view;

SubFragment1 subFragment1 = new SubFragment1();
SubFragment2 subFragment2 = new SubFragment2();

@Override
public View onCreateView(LayoutInflater inflater, ViewGroup container, Bundle savedInstanceState) {
    super.onCreate(savedInstanceState);

    view = inflater.inflate(R.layout.fragment_layout, container, false);

    FragmentTransaction fragmentTransaction = getFragmentManager().beginTransaction();
    fragmentTransaction.replace(R.id.fragmentContainer, subFragment1);
    fragmentTransaction.commit();

    return view;

}

public void showSubFragment2() {
    FragmentTransaction fragmentTransaction = getFragmentManager().beginTransaction();
    fragmentTransaction.setCustomAnimations(R.animator.slide_in_left, R.animator.slide_out_right);
    fragmentTransaction.replace(R.id.fragmentContainer, subFragment2);
    fragmentTransaction.commit();
}

public void showSubFragment1() {
    FragmentTransaction fragmentTransaction = getFragmentManager().beginTransaction();
    fragmentTransaction.setCustomAnimations(R.animator.slide_in_right, R.animator.slide_out_left);
    fragmentTransaction.replace(R.id.fragmentContainer, subFragment1);
    fragmentTransaction.commit();
}
}

Answer 3

我希望这有助于将JavaDstream转换为JavaRDD！

fos_rest:
    format_listener:
        rules:
            - { path: '^/api', fallback_format: json }
            - { path: '^/', fallback_format: json }

将JavaDStream <string>转换为JavaRDD <string> </string> </string>

3 个答案: