Question

我有一个分布式地图，我想找到最低或最高的密钥（实现可比较的对象）。获得这些密钥的最有效方法是什么？我的意思是每个节点提供最低密钥，最后最低密钥是每个节点的最低密钥。

所以我想：

MyObj max = Collections.max(map.keySet());

不是最有效的方式。如果我想使用

new DistributedTask<>(new Max(input), key);

我现在需要密钥，因此可以通过线路获取所有密钥。我想在那种情况下我可以做Collections.max（map.keySet（））;同样。

嗯......有什么想法吗？

Answer 1

您可以使用EntryProcessor.executeOnEntries - 使用有状态EntryProcessor - 然后让它为您完成所有工作;如果它们是最小值和最大值，则将每个键映射到标记MIN和MAX枚举。

如果你对边界有所了解，你也可以附加一个过滤器Predicate来加快速度。

Answer 2

这个地图缩减解决方案似乎有很多开销，但这是我完成工作的最佳方式。任何更好的想法仍然是受欢迎的。

public static void main(String[] args) throws ExecutionException, InterruptedException {
    IMap<String, Integer> map = instance.getMap("test");
    JobTracker jobTracker = instance.getJobTracker( "default" );
    KeyValueSource<String, Integer> source = KeyValueSource.fromMap( map );
    Job<String, Integer> job = jobTracker.newJob(source);

    JobCompletableFuture<Map<String, String>> future = job
            .mapper(new MaxMapper())
            .reducer(new MaxReducerFactory())
            .submit();

    System.out.println("mr max: " + future.get());
}


public static class MaxMapper implements Mapper<String, Integer, String, String> {
    private volatile String max = null;

    @Override
    public void map(String s, Integer integer, Context<String, String> ctx) {
        if (max == null || s.compareTo(max)>0) {
            max = s;
            ctx.emit("max", max);
        }
    }
}

public static class MaxReducerFactory implements ReducerFactory<String,String,String> {

    @Override
    public Reducer<String, String> newReducer(String s) {
        return new MaxReducer();
    }

    private class MaxReducer extends Reducer<String, String> {
        private volatile String max = null;

        @Override
        public void reduce(String s) {
            if (max == null || s.compareTo(max)>0) max = s;
        }

        @Override
        public String finalizeReduce() {
            return max; // == null ? "" : max;
        }
    }
}

Answer 3

映射器：

import com.hazelcast.mapreduce.Context;
import com.hazelcast.mapreduce.Mapper;
import stock.Stock;

public class MinMaxMapper implements Mapper<String, Stock, String, Double> {

    static final String MIN = "min";
    static final String MAX = "max";

    @Override
    public void map(String key, Stock value, Context<String, Double> context) {
        context.emit(MIN, value.getPrice());
        context.emit(MAX, value.getPrice());
    }
}

合

import com.hazelcast.mapreduce.Combiner;
import com.hazelcast.mapreduce.CombinerFactory;

public class MinMaxCombinerFactory implements CombinerFactory<String, Double, Double> {

    @Override
    public Combiner<Double, Double> newCombiner(String key) {
        return new MinMaxCombiner(MinMaxMapper.MAX.equals(key) ? true : false);
    }

    private static class MinMaxCombiner extends Combiner<Double, Double> {

        private final boolean maxCombiner;

        private double value;

        private MinMaxCombiner(boolean maxCombiner) {
            this.maxCombiner = maxCombiner;
            this.value = maxCombiner ? -Double.MAX_VALUE : Double.MAX_VALUE;
        }

        @Override
        public void combine(Double value) {
            if (maxCombiner) {
                this.value = Math.max(value, this.value);
            } else {
                this.value = Math.min(value, this.value);
            }
        }

        @Override
        public Double finalizeChunk() {
            return value;
        }

        @Override
        public void reset() {
            this.value = maxCombiner ? -Double.MAX_VALUE : Double.MAX_VALUE;
        }
    }
}

减速机：

import com.hazelcast.mapreduce.Reducer;
import com.hazelcast.mapreduce.ReducerFactory;

public class MinMaxReducerFactory implements ReducerFactory<String, Double, Double> {

    @Override
    public Reducer<Double, Double> newReducer(String key) {
        return new MinMaxReducer(MinMaxMapper.MAX.equals(key) ? true : false);
    }

    private static class MinMaxReducer extends Reducer<Double, Double> {

        private final boolean maxReducer;

        private volatile double value;

        private MinMaxReducer(boolean maxReducer) {
            this.maxReducer = maxReducer;
            this.value = maxReducer ? -Double.MAX_VALUE : Double.MAX_VALUE;
        }

        @Override
        public void reduce(Double value) {
            if (maxReducer) {
                this.value = Math.max(value, this.value);
            } else {
                this.value = Math.min(value, this.value);
            }
        }

        @Override
        public Double finalizeReduce() {
            return value;
        }
    }
}

返回带有min和max的两个元素map：

ICompletableFuture<Map<String, Double>> future =
        job.mapper(new MinMaxMapper())
        .combiner(new MinMaxCombinerFactory())
        .reducer(new MinMaxReducerFactory())
        .submit();

Map<String, Double> result = future.get();

Answer 4

为什么不创建有序索引？虽然我不太确定目前是否可以使用谓词找到最大值，但一旦找到，就中止对谓词的评估。

查找分布式地图的最低/最高键

4 个答案: