Question

总结问题 我有一个来自传感器的原始信号，该信号的长度为76000个数据点。我想要使用CNN处理这些数据。为此，我想我可以使用Lambda层从原始信号（如

）形成短时傅立叶变换

x = Lambda(lambda v: tf.abs(tf.signal.stft(v,frame_length=frame_length,frame_step=frame_step)))(x)

完全有效。但是我想更进一步，提前处理Raw数据。希望Convolution1D层充当过滤器，以使某些频率通过并阻止其他频率。

我尝试过的事情 我确实有两个单独的程序（用于原始数据处理的Conv1D示例和我在其中处理STFT“图像”的Conv2D示例）并正在运行。但是我想将它们结合起来。

Conv1D 输入为：input =输入（形状=（76000，））

  x = Lambda(lambda v: tf.expand_dims(v,-1))(input)
  x = layers.Conv1D(filters =10,kernel_size=100,activation = 'relu')(x)
  x = Flatten()(x)
  output = Model(input, x)

Model: "model"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_1 (InputLayer)         [(None, 76000)]           0         
_________________________________________________________________
lambda_2 (Lambda)            (None, 76000, 1)          0         
_________________________________________________________________
conv1d (Conv1D)              (None, 75901, 10)         1010      
________________________________________________________________

Conv2D 相同的输入

  x = Lambda(lambda v:tf.expand_dims(tf.abs(tf.signal.stft(v,frame_length=frame_length,frame_step=frame_step)),-1))(input)
  x = BatchNormalization()(x)
Model: "model_4"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
input_6 (InputLayer)         [(None, 76000)]           0         
_________________________________________________________________
lambda_8 (Lambda)            (None, 751, 513, 1)       0         
_________________________________________________________________
batch_normalization_3 (Batch (None, 751, 513, 1)       4         
_________________________________________________________________
. . .
. . . 
flatten_4 (Flatten)          (None, 1360)              0         
_________________________________________________________________
dropout_2 (Dropout)          (None, 1360)              0         
_________________________________________________________________
dense_2 (Dense)              (None, 1)                 1361

我正在寻找一种方法来组合从“ conv1d”到“ lambda_8”层的开始。如果我把它们放在一起，我会得到的：

  x = Lambda(lambda v: tf.expand_dims(v,-1))(input)
  x = layers.Conv1D(filters =10,kernel_size=100,activation = 'relu')(x)
  #x = Flatten()(x)
  x = Lambda(lambda v:tf.expand_dims(tf.abs(tf.signal.stft(v,frame_length=frame_length,frame_step=frame_step)),-1))(x)
Layer (type)                 Output Shape              Param #   
=================================================================
input_6 (InputLayer)         [(None, 76000)]           0         
_________________________________________________________________
lambda_17 (Lambda)           (None, 76000, 1)          0         
_________________________________________________________________
conv1d_6 (Conv1D)            (None, 75901, 10)         1010      
_________________________________________________________________
lambda_18 (Lambda)           (None, 75901, 0, 513, 1)  0         <-- Wrong
=================================================================

这不是我想要的。它看起来应该更像（None，751,513,10,1）。到目前为止，我找不到合适的解决方案。有人可以帮我吗？

谢谢！

Answer 1

在the documentation中，似乎stft仅接受(..., length)输入，而不接受(..., length, channels)。

因此，第一个建议是首先将通道移至另一个维度，以将长度保持在最后一个索引处并使该函数起作用。

现在，您当然需要匹配的长度，您无法将76000与75901匹配。因此，第二个建议是在1D卷积中使用padding='same'来保持长度相等。

最后，由于stft的结果中已经有10个通道，因此您不需要在最后一个lambda中扩展暗淡。

总结：

一维零件

inputs = Input((76000,)) #(batch, 76000)

c1Out = Lambda(lambda x: K.expand_dims(x, axis=-1))(inputs) #(batch, 76000, 1)
c1Out = Conv1D(10, 100, activation = 'relu', padding='same')(c1Out) #(batch, 76000, 10)

#permute for putting length last, apply stft, put the channels back to their position
c1Stft = Permute((2,1))(c1Out) #(batch, 10, 76000)
c1Stft = x = Lambda(lambda v: tf.abs(tf.signal.stft(v,
                                                    frame_length=frame_length,
                                                    frame_step=frame_step)
                                     )
                    )(c1Stft) #(batch, 10, probably 751, probably 513)
c1Stft = Permute((2,3,1))(c1Stft) #(batch, 751, 513, 10)

2D零件，您的代码似乎还可以：

c2Out = Lambda(lambda v: tf.expand_dims(tf.abs(tf.signal.stft(v,
                                                              frame_length=frame_length,
                                                              frame_step=frame_step)
                                               ),
                                        -1))(inputs) #(batch, 751, 513, 1)

现在，所有内容都具有兼容的尺寸

#maybe
#c2Out = Conv2D(10, ..., padding='same')(c2Out) 

joined = Concatenate()([c1Stft, c2Out]) #(batch, 751, 513, 11) #maybe (batch, 751, 513, 20)

further = BatchNormalization()(joined)
further = Conv2D(...)(further)

警告：我不知道它们是否使stft与众不同，Conv1D部分仅在定义了渐变的情况下才起作用。

从卷积1D到卷积2D

1 个答案: