在python

时间:2015-11-06 01:59:53

标签: python bloom-filter

考虑bloom filter的以下Python实现:

class BloomFilter(object):
    """A simple bloom filter for lots of int/str"""

    def __init__(self, array_size=(1 * 1024), hashes=13):
        """Initializes a BloomFilter() object:
            Expects:
                array_size (in bytes): 4 * 1024 for a 4KB filter
                hashes (int): for the number of hashes to perform"""

        self.filter = bytearray(array_size)     # The filter itself
        self.bitcount = array_size * 8          # Bits in the filter
        self.hashes = hashes                    # The number of hashes to use

    def _hash(self, value):
        """Creates a hash of an int and yields a generator of hash functions
        Expects:
            value: int()
        Yields:
            generator of ints()"""

        # Build an int() around the sha256 digest of int() -> value
        value = value.__str__() # Comment out line if you're filtering strings()
        digest = int(sha256(value).hexdigest(), 16)
        for _ in range(self.hashes):
            yield digest & (self.bitcount - 1)
            digest >>= (256 / self.hashes)

    def add(self, value):
        """Bitwise OR to add value(s) into the self.filter
        Expects:
            value: generator of digest ints()
        """
        for digest in self._hash(value):
            self.filter[(digest / 8)] |= (2 ** (digest % 8))


    def query(self, value):
        """Bitwise AND to query values in self.filter
        Expects:
            value: value to check filter against (assumed int())"""
        return all(self.filter[(digest / 8)] & (2 ** (digest % 8)) 
            for digest in self._hash(value))

这个布隆过滤器的一个简单用法是:

bf = BloomFilter()

bf.add(30000)
bf.add(1230213)
bf.add(1)

print("Filter size {0} bytes").format(bf.filter.__sizeof__())
print bf.query(1) # True
print bf.query(1230213) # True
print bf.query(12) # False

您能详细解释 如何使用哈希函数?例如在行中:

self.filter[(digest / 8)] |= (2 ** (digest % 8))

似乎代码使用ORAND操作使用self._hash创建的散列函数写入过滤器但是我无法理解这种按位操作在添加和查询过滤器的值。

  • 为何选择2**(digest %8)
  • 为什么使用OR写入过滤器并AND进行查询?

0 个答案:

没有答案