huggingface笔记：LLama 2

1 前提tip

1.1 使用什么数据类型训练模型？

• Llama2模型是使用bfloat16训练的
  • 上传到Hub的检查点使用torch_dtype = 'float16'，这将通过AutoModel API将检查点从torch.float32转换为torch.float16。
  • 在线权重的数据类型通常无关紧要，这是因为模型将首先下载（使用在线检查点的数据类型），然后转换为torch的默认数据类型（变为torch.float32），最后，如果配置中提供了torch_dtype，则会使用它。
• 不建议在float16中训练模型，因为已知会产生nan；因此，应该在bfloat16中训练模型

1.2 Llama2 的tokenizer

• LlaMA的tokenizer是基于sentencepiece的BPE模型。
• sentencepiece的一个特点是，在解码序列时，如果第一个令牌是词的开头（例如“Banana”），令牌器不会在字符串前添加前缀空格。 

2  transformers.LlamaConfig

根据指定的参数实例化LLaMA模型，定义模型架构。使用默认值实例化配置将产生与LLaMA-7B相似的配置

2.1 参数介绍

2.2 举例

from transformers import LlamaModel, LlamaConfig


configuration = LlamaConfig()
# 默认是Llama-7B的配置
configuration
'''
LlamaConfig {
  "attention_bias": false,
  "attention_dropout": 0.0,
  "bos_token_id": 1,
  "eos_token_id": 2,
  "hidden_act": "silu",
  "hidden_size": 4096,
  "initializer_range": 0.02,
  "intermediate_size": 11008,
  "max_position_embeddings": 2048,
  "mlp_bias": false,
  "model_type": "llama",
  "num_attention_heads": 32,
  "num_hidden_layers": 32,
  "num_key_value_heads": 32,
  "pretraining_tp": 1,
  "rms_norm_eps": 1e-06,
  "rope_scaling": null,
  "rope_theta": 10000.0,
  "tie_word_embeddings": false,
  "transformers_version": "4.41.0",
  "use_cache": true,
  "vocab_size": 32000
}
'''

3 transformers.LlamaTokenizer 

构建一个Llama令牌器。基于字节级Byte-Pair-Encoding。

默认的填充令牌未设置，因为原始模型中没有填充令牌。

3.1 参数介绍

4  transformers.LlamaTokenizerFast

4.1 参数介绍

 4.2 和 LlamaTokenizer的对比

调用from_pretrained从huggingface获取已有的tokenizer时，可以使用AutoTokenizer和LlamaTokenizerFast，不能使用LlamaTokenizer

from transformers import AutoTokenizer

tokenizer1=AutoTokenizer.from_pretrained('meta-llama/Meta-Llama-3-8B')
tokenizer1.encode('Hello world! This is a test file for Llama tokenizer')
#[128000, 9906, 1917, 0, 1115, 374, 264, 1296, 1052, 369, 445, 81101, 47058]


from transformers import LlamaTokenizerFast

tokenizer2=LlamaTokenizerFast.from_pretrained('meta-llama/Meta-Llama-3-8B')
tokenizer2.encode('Hello world! This is a test file for Llama tokenizer')
#[128000, 9906, 1917, 0, 1115, 374, 264, 1296, 1052, 369, 445, 81101, 47058]

from transformers import LlamaTokenizer

tokenizer3=LlamaTokenizer.from_pretrained('meta-llama/Meta-Llama-3-8B')
tokenizer3.encode('Hello world! This is a test file for Llama tokenizer')

5 LlamaModel

5.1 参数介绍

• config
  • 模型配置类，包含模型的所有参数
  • 使用配置文件初始化不会加载与模型相关的权重，只加载配置
  • 使用from_pretrained() 方法以加载模型权重

5.2 介绍

• LLaMA 模型的基本形式，输出原始隐藏状态，顶部没有任何特定的头部。
  • 这个模型继承自 PreTrainedModel
  • 此模型也是 PyTorch torch.nn.Module 的子类
• 解码器包括 config.num_hidden_layers 层。每层是一个 LlamaDecoderLayer

使用前面的LlamaConfig：

from transformers import LlamaModel, LlamaConfig


configuration = LlamaConfig()
# 默认是Llama-7B的配置

model=LlamaModel(configuration)
model
'''
LlamaModel(
  (embed_tokens): Embedding(32000, 4096)
  (layers): ModuleList(
    (0-31): 32 x LlamaDecoderLayer(
      (self_attn): LlamaSdpaAttention(
        (q_proj): Linear(in_features=4096, out_features=4096, bias=False)
        (k_proj): Linear(in_features=4096, out_features=4096, bias=False)
        (v_proj): Linear(in_features=4096, out_features=4096, bias=False)
        (o_proj): Linear(in_features=4096, out_features=4096, bias=False)
        (rotary_emb): LlamaRotaryEmbedding()
      )
      (mlp): LlamaMLP(
        (gate_proj): Linear(in_features=4096, out_features=11008, bias=False)
        (up_proj): Linear(in_features=4096, out_features=11008, bias=False)
        (down_proj): Linear(in_features=11008, out_features=4096, bias=False)
        (act_fn): SiLU()
      )
      (input_layernorm): LlamaRMSNorm()
      (post_attention_layernorm): LlamaRMSNorm()
    )
  )
  (norm): LlamaRMSNorm()
)
'''

model.from_pretrained('meta-llama/Meta-Llama-3-8B')
model
'''
LlamaModel(
  (embed_tokens): Embedding(32000, 4096)
  (layers): ModuleList(
    (0-31): 32 x LlamaDecoderLayer(
      (self_attn): LlamaSdpaAttention(
        (q_proj): Linear(in_features=4096, out_features=4096, bias=False)
        (k_proj): Linear(in_features=4096, out_features=4096, bias=False)
        (v_proj): Linear(in_features=4096, out_features=4096, bias=False)
        (o_proj): Linear(in_features=4096, out_features=4096, bias=False)
        (rotary_emb): LlamaRotaryEmbedding()
      )
      (mlp): LlamaMLP(
        (gate_proj): Linear(in_features=4096, out_features=11008, bias=False)
        (up_proj): Linear(in_features=4096, out_features=11008, bias=False)
        (down_proj): Linear(in_features=11008, out_features=4096, bias=False)
        (act_fn): SiLU()
      )
      (input_layernorm): LlamaRMSNorm()
      (post_attention_layernorm): LlamaRMSNorm()
    )
  )
  (norm): LlamaRMSNorm()
)
'''

5.3 forward方法

参数：

5.3.1 举例

prompt = "Hey, are you conscious? Can you talk to me?"
inputs = tokenizer(prompt, return_tensors="pt")


a=m.forward(inputs.input_ids)

a.keys()
#odict_keys(['last_hidden_state', 'past_key_values'])


a.last_hidden_state
'''
tensor([[[ 4.0064, -0.4994, -1.9927,  ..., -3.7454,  0.8413,  2.6989],
         [-1.5624,  0.5211,  0.1731,  ..., -1.5174, -2.2977, -0.3990],
         [-0.7521, -0.4335,  1.0871,  ..., -0.7031, -1.8011,  2.0173],
         ...,
         [-3.5611, -0.2674,  1.7693,  ..., -1.3848, -0.4413, -1.6342],
         [-1.2451,  1.5639,  1.5049,  ...,  0.5092, -1.2059, -2.3104],
         [-3.2812, -2.2462,  1.8884,  ...,  3.7066,  1.2010,  0.2117]]],
       grad_fn=<MulBackward0>)
'''

a.last_hidden_state.shape
#torch.Size([1, 13, 4096])


len(a.past_key_values)
#32


a.past_key_values[1][0].shape
#torch.Size([1, 8, 13, 128])

6 LlamaForCausalLM 

用于对话系统

6.1 forward方法

参数：

6.2  举例

from transformers import LlamaForCausalLM

m1=LlamaForCausalLM.from_pretrained('meta-llama/Meta-Llama-3-8B')
m1
'''
LlamaForCausalLM(
  (model): LlamaModel(
    (embed_tokens): Embedding(128256, 4096)
    (layers): ModuleList(
      (0-31): 32 x LlamaDecoderLayer(
        (self_attn): LlamaSdpaAttention(
          (q_proj): Linear(in_features=4096, out_features=4096, bias=False)
          (k_proj): Linear(in_features=4096, out_features=1024, bias=False)
          (v_proj): Linear(in_features=4096, out_features=1024, bias=False)
          (o_proj): Linear(in_features=4096, out_features=4096, bias=False)
          (rotary_emb): LlamaRotaryEmbedding()
        )
        (mlp): LlamaMLP(
          (gate_proj): Linear(in_features=4096, out_features=14336, bias=False)
          (up_proj): Linear(in_features=4096, out_features=14336, bias=False)
          (down_proj): Linear(in_features=14336, out_features=4096, bias=False)
          (act_fn): SiLU()
        )
        (input_layernorm): LlamaRMSNorm()
        (post_attention_layernorm): LlamaRMSNorm()
      )
    )
    (norm): LlamaRMSNorm()
  )
  (lm_head): Linear(in_features=4096, out_features=128256, bias=False)
)
'''

从结构上来说，多了一个lm_head

prompt = "Hey, are you conscious? Can you talk to me?"
inputs = tokenizer(prompt, return_tensors="pt")

tokenizer.batch_decode(m1.generate(inputs.input_ids, max_length=30))
'''
['<|begin_of_text|>Hey, are you conscious? Can you talk to me? Can you hear me? Are you sure you can hear me? Can you understand what']
'''


m1.generate(inputs.input_ids, max_length=30)
'''
tensor([[128000,  19182,     11,    527,    499,  17371,     30,   3053,    499,
           3137,    311,    757,     30,   3053,    499,   6865,    757,     30,
           8886,    499,   2103,  27027,     30,   3053,    499,   1518,    757,
             30,   3053,    499]])
'''