Models

bert

MindNLP bert model

class mindnlp.models.bert.BertAttention(config)

Bases: Cell

Bert Attention

construct(input_tensor, attention_mask=None, head_mask=None)

Defines the computation to be performed. This method must be overridden by all subclasses.

Note

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

Parameters

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

Returns

Tensor, returns the computed result.

class mindnlp.models.bert.BertConfig(vocab_size=30522, hidden_size=768, num_hidden_layers=12, num_attention_heads=12, intermediate_size=3072, hidden_act='gelu', hidden_dropout_prob=0.1, attention_probs_dropout_prob=0.1, max_position_embeddings=512, type_vocab_size=2, initializer_range=0.02, layer_norm_eps=1e-12, **kwargs)

Bases: PretrainedConfig

Configuration for BERT-base

class mindnlp.models.bert.BertEmbeddings(config)

Bases: Cell

Embeddings for BERT, include word, position and token_type

construct(input_ids, token_type_ids=None, position_ids=None)

Defines the computation to be performed. This method must be overridden by all subclasses.

Note

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

Parameters

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

Returns

Tensor, returns the computed result.

class mindnlp.models.bert.BertEncoder(config)

Bases: Cell

Bert Encoder

construct(hidden_states, attention_mask=None, head_mask=None)

Defines the computation to be performed. This method must be overridden by all subclasses.

Note

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

Parameters

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

Returns

Tensor, returns the computed result.

class mindnlp.models.bert.BertForPretraining(config, *args, **kwargs)

Bases: Cell

Bert For Pretraining

construct(input_ids, attention_mask=None, token_type_ids=None, position_ids=None, head_mask=None, masked_lm_positions=None)

Defines the computation to be performed. This method must be overridden by all subclasses.

Note

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

Parameters

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

Returns

Tensor, returns the computed result.

class mindnlp.models.bert.BertIntermediate(config)

Bases: Cell

Bert Intermediate

construct(hidden_states)

Defines the computation to be performed. This method must be overridden by all subclasses.

Note

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

Parameters

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

Returns

Tensor, returns the computed result.

class mindnlp.models.bert.BertLMPredictionHead(config)

Bases: Cell

Bert LM Prediction Head

construct(hidden_states, masked_lm_positions)

Defines the computation to be performed. This method must be overridden by all subclasses.

Note

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

Parameters

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

Returns

Tensor, returns the computed result.

class mindnlp.models.bert.BertLayer(config)

Bases: Cell

Bert Layer

construct(hidden_states, attention_mask=None, head_mask=None)

Defines the computation to be performed. This method must be overridden by all subclasses.

Note

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

Parameters

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

Returns

Tensor, returns the computed result.

class mindnlp.models.bert.BertModel(config)

Bases: Cell

Bert Model

construct(input_ids, attention_mask=None, token_type_ids=None, position_ids=None, head_mask=None)

Defines the computation to be performed. This method must be overridden by all subclasses.

Note

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

Parameters

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

Returns

Tensor, returns the computed result.

class mindnlp.models.bert.BertOutput(config)

Bases: Cell

Bert Output

construct(hidden_states, input_tensor)

Defines the computation to be performed. This method must be overridden by all subclasses.

Note

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

Parameters

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

Returns

Tensor, returns the computed result.

class mindnlp.models.bert.BertPooler(config)

Bases: Cell

Bert Pooler

construct(hidden_states)

Defines the computation to be performed. This method must be overridden by all subclasses.

Note

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

Parameters

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

Returns

Tensor, returns the computed result.

class mindnlp.models.bert.BertPreTrainingHeads(config)

Bases: Cell

Bert PreTraining Heads

construct(sequence_output, pooled_output, masked_lm_positions)

Defines the computation to be performed. This method must be overridden by all subclasses.

Note

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

Parameters

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

Returns

Tensor, returns the computed result.

class mindnlp.models.bert.BertPredictionHeadTransform(config)

Bases: Cell

Bert Prediction Head Transform

construct(hidden_states)

Defines the computation to be performed. This method must be overridden by all subclasses.

Note

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

Parameters

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

Returns

Tensor, returns the computed result.

class mindnlp.models.bert.BertSelfAttention(config)

Bases: Cell

Self attention layer for BERT.

construct(hidden_states, attention_mask=None, head_mask=None)

Defines the computation to be performed. This method must be overridden by all subclasses.

Note

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

Parameters

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

Returns

Tensor, returns the computed result.

transpose_for_scores(input_x)

transpose for scores

class mindnlp.models.bert.BertSelfOutput(config)

Bases: Cell

Bert Self Output

construct(hidden_states, input_tensor)

Defines the computation to be performed. This method must be overridden by all subclasses.

Note

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

Parameters

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

Returns

Tensor, returns the computed result.

class mindnlp.models.bert.Matmul(auto_prefix=True, flags=None)

Bases: Cell

Matmul Operation

construct(a, b)

Defines the computation to be performed. This method must be overridden by all subclasses.

Note

It is not supported currently that inputs contain both tuple and non-tuple types at same time.

Parameters

• inputs (tuple) – Tuple of variable parameters.

• kwargs (dict) – Dictionary of variable keyword parameters.

Returns

Tensor, returns the computed result.

class mindnlp.models.bert.PretrainedConfig(**kwargs)

Bases: object

Pretrained Config.