Question

我正在阅读论文：＆{34; Affinity Measures＆＃34;} Ferrari et al.部分。我明白法拉利等人。试图通过以下方式获得亲和力：

位置亲和力 - 使用两个检测之间的交叉结合区域
外观亲和力 - 使用直方图之间的欧几里德距离
KLT点亲和力度量

但是，我有两个主要问题：

我无法理解2次检测之间的交叉结合实际意味着什么以及如何计算
我尝试了略微不同的外观亲和力测量。我将RGB检测转换为HSV ..将Hue和Saturation连接成1个向量，并用它与其他检测进行比较。然而，使用这种技术失败了，因为检测到的包具有比检测到同一个人头部（具有不同方向）更好的相似性得分。

上述问题的任何建议或解决方案？谢谢你，非常感谢你的帮助。

Answer 1

尝试交叉联盟

Intersection over Union是一个评估指标，用于衡量特定数据集上对象检测器的准确度。

更正式地说，为了应用Intersection over Union来评估我们需要的（任意）物体探测器：

地面实况边界框（即，测试集中标记为边界框的手，指定图像中我们的对象所在的位置）。
我们模型中预测的边界框。

下面我列出了一个地面实况边界框与预测边界框的可视化示例：

预测的边界框用红色绘制，而地面实况（即手工标记）边界框用绿色绘制。

在上图中，我们可以看到我们的物体探测器已检测到图像中存在停车标志。

因此可以通过以下方式确定计算联盟的交叉点：

只要我们有这两组边界框，我们就可以在Union上应用Intersection。

这是Python代码

# import the necessary packages
from collections import namedtuple
import numpy as np
import cv2

# define the `Detection` object
Detection = namedtuple("Detection", ["image_path", "gt", "pred"])

def bb_intersection_over_union(boxA, boxB):
    # determine the (x, y)-coordinates of the intersection rectangle
    xA = max(boxA[0], boxB[0])
    yA = max(boxA[1], boxB[1])
    xB = min(boxA[2], boxB[2])
    yB = min(boxA[3], boxB[3])

    # compute the area of intersection rectangle
    interArea = (xB - xA) * (yB - yA)

    # compute the area of both the prediction and ground-truth
    # rectangles
    boxAArea = (boxA[2] - boxA[0]) * (boxA[3] - boxA[1])
    boxBArea = (boxB[2] - boxB[0]) * (boxB[3] - boxB[1])

    # compute the intersection over union by taking the intersection
    # area and dividing it by the sum of prediction + ground-truth
    # areas - the interesection area
    iou = interArea / float(boxAArea + boxBArea - interArea)

    # return the intersection over union value
    return iou

gt和pred

gt：真实的边界框。
pred：我们模型中预测的边界框。

有关详情，请点击this post

Answer 2

1）您有两个重叠的边界框。您计算框的交集，这是重叠的区域。您计算重叠框的并集，即整个框的面积减去重叠面积的总和。然后用联合划分交叉点。计算机视觉系统工具箱中有一个名为bboxOverlapRatio的功能。

2）通常，您不希望连接颜色通道。您想要的是3D直方图，其中尺寸为H，S和V.

Answer 3

当前答案已经清楚地说明了问题。因此，在这里，我提供了使用Python的IoU更好的版本，当两个边界框不相交时，它不会中断。

import numpy as np

def IoU(box1: np.ndarray, box2: np.ndarray):
    """
    calculate intersection over union cover percent
    :param box1: box1 with shape (N,4) or (N,2,2) or (2,2) or (4,). first shape is preferred
    :param box2: box2 with shape (N,4) or (N,2,2) or (2,2) or (4,). first shape is preferred
    :return: IoU ratio if intersect, else 0
    """
    # first unify all boxes to shape (N,4)
    if box1.shape[-1] == 2 or len(box1.shape) == 1:
        box1 = box1.reshape(1, 4) if len(box1.shape) <= 2 else box1.reshape(box1.shape[0], 4)
    if box2.shape[-1] == 2 or len(box2.shape) == 1:
        box2 = box2.reshape(1, 4) if len(box2.shape) <= 2 else box2.reshape(box2.shape[0], 4)
    point_num = max(box1.shape[0], box2.shape[0])
    b1p1, b1p2, b2p1, b2p2 = box1[:, :2], box1[:, 2:], box2[:, :2], box2[:, 2:]

    # mask that eliminates non-intersecting matrices
    base_mat = np.ones(shape=(point_num,))
    base_mat *= np.all(np.greater(b1p2 - b2p1, 0), axis=1)
    base_mat *= np.all(np.greater(b2p2 - b1p1, 0), axis=1)

    # I area
    intersect_area = np.prod(np.minimum(b2p2, b1p2) - np.maximum(b1p1, b2p1), axis=1)
    # U area
    union_area = np.prod(b1p2 - b1p1, axis=1) + np.prod(b2p2 - b2p1, axis=1) - intersect_area
    # IoU
    intersect_ratio = intersect_area / union_area

    return base_mat * intersect_ratio

Answer 4

这是我实施的另一个对我有用的解决方案。

大量借用PyImageSearch

import numpy as np

def bbox_intersects(bbox_a, bbox_b):
    if bbox_b['x0'] >= bbox_a['x0'] and bbox_b['x0'] <= bbox_a['x1'] and \
        bbox_b['y0'] >= bbox_a['y0'] and bbox_b['y0'] <= bbox_a['y1']:
        # top-left of b within a
        return True
    elif bbox_b['x1'] >= bbox_a['x0'] and bbox_b['x1'] <= bbox_a['x1'] and \
        bbox_b['y1'] >= bbox_a['y0'] and bbox_b['y1'] <= bbox_a['y1']:
        # bottom-right of b within a
        return True
    elif bbox_a['x0'] >= bbox_b['x0'] and bbox_a['x0'] <= bbox_b['x1'] and \
        bbox_a['y0'] >= bbox_b['y0'] and bbox_a['y0'] <= bbox_b['y1']:
        # top-left of a within b
        return True
    elif bbox_a['x1'] >= bbox_b['x0'] and bbox_a['x1'] <= bbox_b['x1'] and \
        bbox_a['y1'] >= bbox_b['y0'] and bbox_a['y1'] <= bbox_b['y1']:
        # bottom-right of a within b
        return True
    return False

def bbox_area(x0, y0, x1, y1):
    return (x1-x0) * (y1-y0)

def get_bbox_iou(bbox_a, bbox_b):
    if bbox_intersects(bbox_a, bbox_b):
        x_left = max(bbox_a['x0'], bbox_b['x0'])
        x_right = min(bbox_a['x1'], bbox_b['x1'])
        y_top = max(bbox_a['y0'], bbox_b['y0'])
        y_bottom = min(bbox_a['y1'], bbox_b['y1'])

        inter_area = bbox_area(x0 = x_left, x1 = x_right, y0 = y_top , y1 = y_bottom)
        bbox_a_area = bbox_area(**bbox_a)
        bbox_b_area = bbox_area(**bbox_b)

        return inter_area / float(bbox_a_area + bbox_b_area - inter_area)
    else:
        return 0

两次检测之间的交叉联合

4 个答案: