通常，keras层的输出维的形式为(None, c, h, w)或(None, h, w, c)，具体取决于channels_first或channels_last的配置。
我正在尝试使用具有两个输入的自定义keras图层。当我打印模型摘要时，它不显示None维度。
如何对自定义图层进行编程以包括此None维度？
我认为这可能是我收到错误的原因
No data provided for "crfrnn". Need data for each key in: ['crfrnn']
crfrnn是我的自定义图层的名称
很不幸，我尝试用call()方法重塑输出，但无济于事。
我确保输入数据的形状和准备正确。请注意，在我尝试在plant_output层之上添加此层之前，我的代码训练得很好（请参见下面的摘要）
下面是自定义层的代码，它是从this github repository开始采用的：
class CrfRnnLayer(tf.keras.layers.Layer):
""" Implements the CRF-RNN layer described in:
Conditional Random Fields as Recurrent Neural Networks,
S. Zheng, S. Jayasumana, B. Romera-Paredes, V. Vineet, Z. Su, D. Du, C. Huang and P. Torr,
ICCV 2015
"""
def __init__(self, image_dims, num_classes,
theta_alpha, theta_beta, theta_gamma,
num_iterations, NCHW, **kwargs):
self.image_dims = image_dims
self.num_classes = num_classes
self.theta_alpha = theta_alpha
self.theta_beta = theta_beta
self.theta_gamma = theta_gamma
self.num_iterations = num_iterations
self.NCHW = NCHW
self.spatial_ker_weights = None
self.bilateral_ker_weights = None
self.compatibility_matrix = None
super(CrfRnnLayer, self).__init__(**kwargs)
def build(self, input_shape):
if not self.NCHW:
channel_axis = -1
if input_shape[channel_axis] is None:
raise ValueError('The channel dimension of the inputs '
'should be defined. Found `None`.')
input_dim = input_shape[0][channel_axis]
self.input_spec = [tf.keras.layers.InputSpec(shape=(None, input_shape[0][1], input_shape[0][2], input_shape[0][3])), tf.keras.layers.InputSpec(shape=(None, input_shape[1][1], input_shape[1][2], input_shape[1][3]))]
else:
channel_axis = 1
if input_shape[channel_axis] is None:
raise ValueError('The channel dimension of the inputs '
'should be defined. Found `None`.')
input_dim = input_shape[0][channel_axis]
self.input_spec = [tf.keras.layers.InputSpec(shape=(None, input_shape[0][1], input_shape[0][2], input_shape[0][3])), tf.keras.layers.InputSpec(shape=(None, input_shape[1][1], input_shape[1][2], input_shape[1][3]))]
# Weights of the spatial kernel
self.spatial_ker_weights = self.add_weight(name='spatial_ker_weights',
shape=(self.num_classes, self.num_classes),
initializer=_diagonal_initializer,
trainable=True)
# Weights of the bilateral kernel
self.bilateral_ker_weights = self.add_weight(name='bilateral_ker_weights',
shape=(self.num_classes, self.num_classes),
initializer=_diagonal_initializer,
trainable=True)
# Compatibility matrix
self.compatibility_matrix = self.add_weight(name='compatibility_matrix',
shape=(self.num_classes, self.num_classes),
initializer=_potts_model_initializer,
trainable=True)
super(CrfRnnLayer, self).build(input_shape)
def call(self, inputs):
unaries = tf.transpose(inputs[0][0, :, :, :], perm=(2, 0, 1))
rgb = tf.transpose(inputs[1][0, :, :, :], perm=(2, 0, 1))
#input is channels first
c, h, w = self.num_classes, self.image_dims[0], self.image_dims[1]
all_ones = np.ones((c, h, w), dtype=np.float32)
# Prepare filter normalization coefficients
spatial_norm_vals = custom_module.high_dim_filter(all_ones, rgb, bilateral=False,
theta_gamma=self.theta_gamma)
bilateral_norm_vals = custom_module.high_dim_filter(all_ones, rgb, bilateral=True,
theta_alpha=self.theta_alpha,
theta_beta=self.theta_beta)
q_values = unaries
for i in range(self.num_iterations):
softmax_out = tf.nn.softmax(q_values, 0)
# Spatial filtering
spatial_out = custom_module.high_dim_filter(softmax_out, rgb, bilateral=False,
theta_gamma=self.theta_gamma)
spatial_out = spatial_out / spatial_norm_vals
# Bilateral filtering
bilateral_out = custom_module.high_dim_filter(softmax_out, rgb, bilateral=True,
theta_alpha=self.theta_alpha,
theta_beta=self.theta_beta)
bilateral_out = bilateral_out / bilateral_norm_vals
# Weighting filter outputs
message_passing = (tf.matmul(self.spatial_ker_weights,
tf.reshape(spatial_out, (c, -1))) +
tf.matmul(self.bilateral_ker_weights,
tf.reshape(bilateral_out, (c, -1))))
# Compatibility transform
pairwise = tf.matmul(self.compatibility_matrix, message_passing)
# Adding unary potentials
pairwise = tf.reshape(pairwise, (c, h, w))
q_values = unaries - pairwise
#output is channels last
return tf.transpose(tf.reshape(q_values, (1, c, h, w)), perm=(0, 2, 3, 1))
def compute_output_shape(self, input_shape):
return input_shape
此层有两个输入，每个输入的大小为(None, 384, 512, 3)。我希望输出结果是一样的，但是当我编译模型时，生成的摘要如下（请注意，我没有在中间显示很多层，因为我违反了此平台上的字符限制，最重要的层要查看的是input_node，plant_output和自定义crfrnn）：
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_node (InputLayer) (None, 384, 512, 3) 0
__________________________________________________________________________________________________
encoder_conv1 (Conv2D) (None, 384, 512, 32) 128 input_node[0][0]
__________________________________________________________________________________________________
bneck1_dense_encoder1 (Conv2D) (None, 384, 512, 8) 264 encoder_conv1[0][0]
__________________________________________________________________________________________________
batch_normalization (BatchNorma (None, 384, 512, 8) 32 bneck1_dense_encoder1[0][0]
__________________________________________________________________________________________________
leaky_re_lu (LeakyReLU) (None, 384, 512, 8) 0 batch_normalization[0][0]
__________________________________________________________________________________________________
conv1_dense_encoder1 (Conv2D) (None, 384, 512, 4) 292 leaky_re_lu[0][0]
__________________________________________________________________________________________________
dropout (Dropout) (None, 384, 512, 4) 0 conv1_dense_encoder1[0][0]
__________________________________________________________________________________________________
concatenate (Concatenate) (None, 384, 512, 36) 0 encoder_conv1[0][0]
dropout[0][0]
__________________________________________________________________________________________________
bneck2_dense_encoder1 (Conv2D) (None, 384, 512, 8) 296 concatenate[0][0]
__________________________________________________________________________________________________
batch_normalization_1 (BatchNor (None, 384, 512, 8) 32 bneck2_dense_encoder1[0][0]
__________________________________________________________________________________________________
leaky_re_lu_1 (LeakyReLU) (None, 384, 512, 8) 0 batch_normalization_1[0][0]
__________________________________________________________________________________________________
conv2_dense_encoder1 (Conv2D) (None, 384, 512, 4) 292 leaky_re_lu_1[0][0]
__________________________________________________________________________________________________
dropout_1 (Dropout) (None, 384, 512, 4) 0 conv2_dense_encoder1[0][0]
__________________________________________________________________________________________________
concatenate_1 (Concatenate) (None, 384, 512, 40) 0 concatenate[0][0]
dropout_1[0][0]
__________________________________________________________________________________________________
bneck3_dense_encoder1 (Conv2D) (None, 384, 512, 8) 328 concatenate_1[0][0]
__________________________________________________________________________________________________
batch_normalization_2 (BatchNor (None, 384, 512, 8) 32 bneck3_dense_encoder1[0][0]
__________________________________________________________________________________________________
leaky_re_lu_2 (LeakyReLU) (None, 384, 512, 8) 0 batch_normalization_2[0][0]
__________________________________________________________________________________________________
conv3_dense_encoder1 (Conv2D) (None, 384, 512, 4) 292 leaky_re_lu_2[0][0]
__________________________________________________________________________________________________
dropout_2 (Dropout) (None, 384, 512, 4) 0 conv3_dense_encoder1[0][0]
__________________________________________________________________________________________________
concatenate_2 (Concatenate) (None, 384, 512, 12) 0 dropout[0][0]
dropout_1[0][0]
dropout_2[0][0]
__________________________________________________________________________________________________
encoder_concat1 (Concatenate) (None, 384, 512, 44) 0 encoder_conv1[0][0]
concatenate_2[0][0]
__________________________________________________________________________________________________
encoder_bneck1 (Conv2D) (None, 384, 512, 22) 990 encoder_concat1[0][0]
__________________________________________________________________________________________________
encoder_downsample1 (Conv2D) (None, 192, 256, 22) 12122 encoder_bneck1[0][0]
__________________________________________________________________________________________________
bneck1_dense_encoder2 (Conv2D) (None, 192, 256, 8) 184 encoder_downsample1[0][0]
__________________________________________________________________________________________________
batch_normalization_3 (BatchNor (None, 192, 256, 8) 32 bneck1_dense_encoder2[0][0]
__________________________________________________________________________________________________
leaky_re_lu_3 (LeakyReLU) (None, 192, 256, 8) 0 batch_normalization_3[0][0]
__________________________________________________________________________________________________
conv1_dense_encoder2 (Conv2D) (None, 192, 256, 4) 292 leaky_re_lu_3[0][0]
__________________________________________________________________________________________________
dropout_3 (Dropout) (None, 192, 256, 4) 0 conv1_dense_encoder2[0][0]
__________________________________________________________________________________________________
concatenate_3 (Concatenate) (None, 192, 256, 26) 0 encoder_downsample1[0][0]
dropout_3[0][0]
__________________________________________________________________________________________________
bneck2_dense_encoder2 (Conv2D) (None, 192, 256, 8) 216 concatenate_3[0][0]
__________________________________________________________________________________________________
batch_normalization_4 (BatchNor (None, 192, 256, 8) 32 bneck2_dense_encoder2[0][0]
__________________________________________________________________________________________________
leaky_re_lu_4 (LeakyReLU) (None, 192, 256, 8) 0 batch_normalization_4[0][0]
__________________________________________________________________________________________________
conv2_dense_encoder2 (Conv2D) (None, 192, 256, 4) 292 leaky_re_lu_4[0][0]
__________________________________________________________________________________________________
dropout_4 (Dropout) (None, 192, 256, 4) 0 conv2_dense_encoder2[0][0]
__________________________________________________________________________________________________
concatenate_4 (Concatenate) (None, 192, 256, 30) 0 concatenate_3[0][0]
dropout_4[0][0]
__________________________________________________________________________________________________
bneck3_dense_encoder2 (Conv2D) (None, 192, 256, 8) 248 concatenate_4[0][0]
__________________________________________________________________________________________________
batch_normalization_5 (BatchNor (None, 192, 256, 8) 32 bneck3_dense_encoder2[0][0]
__________________________________________________________________________________________________
leaky_re_lu_5 (LeakyReLU) (None, 192, 256, 8) 0 batch_normalization_5[0][0]
__________________________________________________________________________________________________
conv3_dense_encoder2 (Conv2D) (None, 192, 256, 4) 292 leaky_re_lu_5[0][0]
__________________________________________________________________________________________________
dropout_5 (Dropout) (None, 192, 256, 4) 0 conv3_dense_encoder2[0][0]
__________________________________________________________________________________________________
concatenate_5 (Concatenate) (None, 192, 256, 12) 0 dropout_3[0][0]
dropout_4[0][0]
dropout_5[0][0]
__________________________________________________________________________________________________
encoder_concat2 (Concatenate) (None, 192, 256, 34) 0 encoder_downsample1[0][0]
concatenate_5[0][0]
__________________________________________________________________________________________________
encoder_bneck2 (Conv2D) (None, 192, 256, 17) 595 encoder_concat2[0][0]
__________________________________________________________________________________________________
encoder_downsample2 (Conv2D) (None, 96, 128, 17) 7242 encoder_bneck2[0][0]
__________________________________________________________________________________________________
bneck1_dense_encoder3 (Conv2D) (None, 96, 128, 8) 144 encoder_downsample2[0][0]
__________________________________________________________________________________________________
batch_normalization_6 (BatchNor (None, 96, 128, 8) 32 bneck1_dense_encoder3[0][0]
__________________________________________________________________________________________________
leaky_re_lu_6 (LeakyReLU) (None, 96, 128, 8) 0 batch_normalization_6[0][0]
__________________________________________________________________________________________________
conv1_dense_encoder3 (Conv2D) (None, 96, 128, 4) 292 leaky_re_lu_6[0][0]
__________________________________________________________________________________________________
dropout_6 (Dropout) (None, 96, 128, 4) 0 conv1_dense_encoder3[0][0]
__________________________________________________________________________________________________
concatenate_6 (Concatenate) (None, 96, 128, 21) 0 encoder_downsample2[0][0]
dropout_6[0][0]
__________________________________________________________________________________________________
bneck2_dense_encoder3 (Conv2D) (None, 96, 128, 8) 176 concatenate_6[0][0]
__________________________________________________________________________________________________
batch_normalization_7 (BatchNor (None, 96, 128, 8) 32 bneck2_dense_encoder3[0][0]
__________________________________________________________________________________________________
leaky_re_lu_7 (LeakyReLU) (None, 96, 128, 8) 0 batch_normalization_7[0][0]
__________________________________________________________________________________________________
conv2_dense_encoder3 (Conv2D) (None, 96, 128, 4) 292 leaky_re_lu_7[0][0]
__________________________________________________________________________________________________
dropout_7 (Dropout) (None, 96, 128, 4) 0 conv2_dense_encoder3[0][0]
__________________________________________________________________________________________________
concatenate_7 (Concatenate) (None, 96, 128, 25) 0 concatenate_6[0][0]
dropout_7[0][0]
__________________________________________________________________________________________________
bneck3_dense_encoder3 (Conv2D) (None, 96, 128, 8) 208 concatenate_7[0][0]
__________________________________________________________________________________________________
batch_normalization_8 (BatchNor (None, 96, 128, 8) 32 bneck3_dense_encoder3[0][0]
__________________________________________________________________________________________________
leaky_re_lu_8 (LeakyReLU) (None, 96, 128, 8) 0 batch_normalization_8[0][0]
__________________________________________________________________________________________________
conv3_dense_encoder3 (Conv2D) (None, 96, 128, 4) 292 leaky_re_lu_8[0][0]
__________________________________________________________________________________________________
dropout_8 (Dropout) (None, 96, 128, 4) 0 conv3_dense_encoder3[0][0]
__________________________________________________________________________________________________
concatenate_8 (Concatenate) (None, 96, 128, 12) 0 dropout_6[0][0]
dropout_7[0][0]
dropout_8[0][0]
__________________________________________________________________________________________________
__________________________________________________________________________________________________
plant_conv (Conv2D) (None, 384, 512, 32) 416 concatenate_20[0][0]
__________________________________________________________________________________________________
plant_output (Conv2D) (None, 384, 512, 3) 99 plant_conv[0][0]
__________________________________________________________________________________________________
crfrnn (CrfRnnLayer) (1, 384, 512, 3) 27 plant_output[0][0]
input_node[0][0]
==================================================================================================
Total params: 41,833
Trainable params: 41,497
Non-trainable params: 336
__________________________________________________________________________________________________
请注意crfrnn层的形状（1、384、512、3）。我相信这会导致程序无法训练，并引发错误：
Epoch 1/2000
No data provided for "crfrnn". Need data for each key in: ['crfrnn']
答案 0 :(得分：0)
问题有两个，第一个更重要。
在上面的代码中，在call（）函数中，您看到：
unaries = tf.transpose(inputs[0][0, :, :, :], perm=(2, 0, 1))
rgb = tf.transpose(inputs[1][0, :, :, :], perm=(2, 0, 1))
我删除了它，而是写道：
unaries = inputs[0]
rgb = inputs[1]
这假定通道首先被输入为通道，因此不需要转置。而且，但是像这样阅读它们，keras能够推断形状，并且摘要现在具有（None，c，h，w）作为该层的形状。
错误No data provided for "crfrnn". Need data for each key in: ['crfrnn']实际上对我来说是一个愚蠢的错误，我正在输入的标签字典的密钥错误（应该是crfrnn）。我将此内容发布在crfrnn存储库的原始github上，因为其他人很好奇如何将这一层应用到他们自己的网络中，而且我已经这样做了。