Decoder

rnn_decoder

RNN Decoder modules

class mindnlp.modules.decoder.rnn_decoder.RNNDecoder(embedding, rnns, dropout_in=0, dropout_out=0, attention=True, encoder_output_units=512, mode='RNN')

Bases: DecoderBase

Stacked Elman RNN Decoder.

Parameters

• embedding (Cell) – The embedding layer.

• rnns (list) – The list of RNN cells.

• dropout_in (Union[float, int]) – If not 0, append Dropout layer on the inputs of each RNN layer. Default 0. The range of dropout is [0.0, 1.0).

• dropout_out (Union[float, int]) – If not 0, append Dropout layer on the outputs of each RNN layer except the last layer. Default 0. The range of dropout is [0.0, 1.0).

• attention (bool) – Whether to use attention. Default: True.

• encoder_output_units (int) – Number of features of encoder output. Default: 512.

Examples

>>> vocab_size = 1000
>>> embedding_size = 32
>>> hidden_size = 16
>>> num_layers = 2
>>> dropout_in = 0.1
>>> dropout_out = 0.1
>>> encoder_output_units = 16
>>> embedding = nn.Embedding(vocab_size, embedding_size)
>>> input_feed_size = 0 if encoder_output_units == 0 else hidden_size
>>> rnns = [
...     nn.RNNCell(
...         input_size=embedding_size + input_feed_size
...         if layer == 0
...             else hidden_size,
...         hidden_size=hidden_size
...         )
...         for layer in range(num_layers)
... ]
>>> rnn_decoder = RNNDecoder(embedding, rnns, dropout_in=dropout_in, dropout_out=dropout_out,
...                          attention=True, encoder_output_units=encoder_output_units, mode="RNN")
>>> tgt_tokens = Tensor(np.ones([8, 16]), mindspore.int32)
>>> encoder_output = Tensor(np.ones([8, 16, 16]), mindspore.float32)
>>> hiddens_n = Tensor(np.ones([2, 8, 16]), mindspore.float32)
>>> mask = Tensor(np.ones([8, 16]), mindspore.int32)
>>> output, attn_scores = rnn_decoder(tgt_tokens, (encoder_output, hiddens_n, mask))
>>> print(output.shape)
>>> print(attn_scores.shape)
(8, 16, 1000)
(8, 16, 16)

construct(prev_output_tokens, encoder_out=None)

Construct method.

Parameters

• prev_output_tokens (Tensor) – Output tokens for teacher forcing with shape [batch, tgt_len].

• encoder_out (Tensor) – Output of encoder. Default: None.

Returns

Tuple, a tuple contains (output, attn_scores).

• output (Tensor): Tensor of shape (batch, tgt_len, vocab_size).

• attn_scores (Tensor): Tensor of shape (batch, tgt_len, src_len) if attention=True otherwise None.

extract_features(prev_output_tokens, encoder_out=None)

Extract features of encoder output.

Parameters

• prev_output_tokens (Tensor) – Output tokens for teacher forcing with shape [batch, tgt_len].

• encoder_out (Tensor) – Output of encoder. Default: None.

Returns

Tuple, a tuple contains (output, attn_scores).

• output (Tensor): The extracted feature Tensor of shape (batch, tgt_len, hidden_size).

• attn_scores (Tensor): Tensor of shape (batch, tgt_len, src_len) if attention=True otherwise None.

output_layer(features)

Project features to the vocabulary size.

Parameters

features (Tensor) – The extracted feature Tensor.

Returns

Tensor, the output of decoder.

transformer_decoder

Decoder class