如何有效地存储大型Java地图?

时间:2016-11-24 21:30:31

标签: java sql large-data

我蛮力一场比赛,我需要存储所有位置和结果的数据。数据可能会达到数百Gb。我考虑过SQL,但我担心在紧密循环中查找会破坏性能。如果已知,程序将迭代可能的位置并返回获胜的移动,如果已知所有移动都已丢失则返回最长的丢失序列并检查未知移动的结果。

存储大型Map<Long,Long[]> positionIdToBestMoves的最佳方法是什么?我正在考虑SQL或数据序列化。

我希望通过强制解决微小的检查器来强制Java中的所有可行动作。头寸的上限约为100亿。它们中的大多数都不合理(即比游戏开始时存在的碎片多)。大约100亿是一个合理的估计。每个Map<Long, Long[]> positionLong positionID映射到Long whiteToMoveLong blackToMove。正值表示位置获胜,应选择导致存储在值中的位置的移动。负值-n表示位置最多会丢失n次。

搜索本身会有这样的递归:

//this is a stub

private Map<Long, Long[]> boardBook =...

//assuming that all winning positions are known
public Long nextMove(Long currentPos, int whiteOrBlack){
Set<Long> validMoves = calculateValidMoves(currentPos, whiteOrBlack);
boolean hasWinner = checkIfValidMoveIsKnownToWin(validMoves, whiteOrBlack);

if(hasWinner){  //there is a winning move - play it
    Long winningMove = getWinningMove(validMoves, whiteOrBlack);
    boardBook.get(currentPos)[whiteOrBlack] = winningMove ;    
    return winningMove ;
    }
boolean areAllPositionsKnown = checkIfAllPositionsKnown(validMoves, whiteOrBlack);
if(areAllPositionsKnown){  //all moves are losing.. choose longest struggle
    Long longestSequenceToDefeat = findPositionToLongestSequenceToDefeat(validMoves, whiteOrBlack);
    int numberOfStepsTodefeat = boardBook.get(longestSequenceToDefeat)[whiteOrBlack];
    boardBook.get(currentPos)[whiteOrBlack] = longestSequenceToDefeat ;
    return longestSequenceToDefeat;
    }

Set<Long> movesToCheck = getUntestedMoves(validMoves, whiteOrBlack);
Long longeststruggle;
int maxNumberOfMovesToDefeat =-1;
for(Long moveTocheck : movesToCheck){
    Long result = nextMove(moveToCheck, whiteOrBlack);
    if(result>0){ //just discovered a winning move
            boardBook.get(currentPos)[whiteOrBlack] = winningMove ;    
            return winningMove ;
        }else {
            int numOfMovesToDefeat = -1*boardBook.get(moveTocheck)[whiteOrBlack];
            if( numOfMovesToDefeat >maxNumberOfMovesToDefeat ){
                 maxNumberOfMovesToDefeat =numOfMovesToDefeat ; 
                 longeststruggle = moveTocheck;
                  }
         }
      }
boardBook.get(currentPos)[whiteOrBlack] = -1*maxNumberOfMovesToDefeat;
return  longeststruggle;
}

1 个答案:

答案 0 :(得分:3)

您可能需要查看Chronicle。它是高度优化的键值存储,它应该适合您的目的。

或者你可以自己编写存储空间,但是你仍然会在地图和内存映射文件之下做一些事情。

相关问题