Question

我使用opencv的预训练dnn模型和dlib的hog模型运行了实时流人脸检测代码。我从几台摄像机获得了检测结果，并且代码为opencv和dlib都打印出了边界框坐标。我期待相同的结果，但结果却截然不同。有没有办法将opencv坐标转换为dlib的坐标？

我试图找到一个数学模型（线性模型）来将两者联系起来，但这没用。

import numpy as np
import argparse
import imutils
import pickle
import time
import cv2
import os
import align
import dlib
import time
import datetime

face_detector = dlib.get_frontal_face_detector()
predictor_model = "shape_predictor_68_face_landmarks.dat"
face_aligner = align.AlignDlib(predictor_model)

ap = argparse.ArgumentParser()
ap.add_argument("-d", "--detector", required=True,
    help="path to OpenCV's deep learning face detector")
ap.add_argument("-m", "--embedding-model", required=True,
    help="path to OpenCV's deep learning face embedding model")
ap.add_argument("-r", "--recognizer", required=True,
    help="path to model trained to recognize faces")
ap.add_argument("-l", "--le", required=True,
    help="path to label encoder")
ap.add_argument("-c", "--confidence", type=float, default=0.8,
    help="minimum probability to filter weak detections")
args = vars(ap.parse_args())

print("[INFO] loading face detector...")
protoPath = os.path.sep.join([args["detector"], "deploy.prototxt"])
modelPath = os.path.sep.join([args["detector"],
    "res10_300x300_ssd_iter_140000.caffemodel"])
detector = cv2.dnn.readNetFromCaffe(protoPath, modelPath)
print("[INFO] starting video stream...")
vs = cv2.VideoCapture(0)
time.sleep(2.0)

while True:
    ret, frame = vs.read()
    frame = imutils.resize(frame, width=600)
    (h, w) = frame.shape[:2]
    imageBlob = cv2.dnn.blobFromImage(
        cv2.resize(frame, (300, 300)), 1.0, (300, 300),
        (104.0, 177.0, 123.0), swapRB=False, crop=False)
    detector.setInput(imageBlob)
    detections = detector.forward()
    for i in range(0, detections.shape[2]):
        confidence = detections[0, 0, i, 2]
        if confidence > args["confidence"]:
            box = detections[0, 0, i, 3:7] * np.array([w, h, w, h])
            (startX, startY, endX, endY) = box.astype("int")
            face = frame[startY:endY, startX:endX]
            (fH, fW) = face.shape[:2]
            if fW < 20 or fH < 20:
                continue
            rgb = cv2.cvtColor(face, cv2.COLOR_BGR2RGB)
            detected_faces_dlib = face_detector(rgb, 1)
            detected_faces = dlib.rectangle(left=startX, top=startY, right=endX, bottom=endY)
            print(detected_faces)
            print(detected_faces_dlib)

以下是结果：

[(333, 191) (490, 414)]
rectangles[[(-22, 47) (150, 202)]]
[(333, 190) (490, 413)]
rectangles[[(-22, 47) (150, 202)]]
[(333, 190) (491, 414)]
rectangles[[(-22, 47) (150, 202)]]
[(334, 191) (491, 416)]
rectangles[[(-22, 47) (150, 202)]]
[(334, 196) (493, 416)]
rectangles[[(-22, 47) (150, 202)]]

Answer 1

我只是花了很多时间来解决这个问题，如果您的目标是要检测dnn检测器检测到的脸上的面部标志，那么最好的选择是使用dnn中的矩形重新训练shape_predictor_68_face_landmarks.dat探测器。

以this article为指导，我编写了一个python脚本，该脚本通过了ibug300训练集，重新检测了面部的边界框，重写了训练集的xml文件，然后运行了train_shape_predictor脚本以获得新的.dat文件。

与尝试重塑“ dnn rect”以逼近“猪箱”相比，结果非常好。

在进行再培训之前，有一个技巧：dnn人脸检测会返回矩形，并且矩形的宽度和高度会有很大不同。这对于形状预测器训练效果不佳。最好使用侧面为~1.35 * dnn_rect.width.的正方形，看起来像一个魔术数字，但这是dnn人脸检测矩形的高宽平均比。

Answer 2

# take a bounding predicted by opencv and convert it
# to the dlib (top, right, bottom, left) 
def bb_to_rect(bb):
    top=bb[1]
    left=bb[0]
    right=bb[0]+bb[2]
    bottom=bb[1]+bb[3]
    return np.array([top, right, bottom, left]) 


# take a bounding predicted by dlib and convert it
# to the format (x, y, w, h) as we would normally do
# with OpenCV
def rect_to_bb(rect):

    x = rect.left()
    y = rect.top()
    w = rect.right() - x
    h = rect.bottom() - y

    # return a tuple of (x, y, w, h)
    return (x, y, w, h)

如何将opencv的人脸检测到的边界框坐标转换为dlib的人脸检测到的边界框坐标？

2 个答案: