？？？ ？？？？？ ？？？

LLM ? ? Fine Tuning? . ? ? ?. ? Fine Tuning? , ? LLM ? ? ?.

Fine Tuning? ? ? Pre-trained Model? . https://jins-sw.tistory.com/48 Foundation Model? ? LLM ? ? ? ? ?.

? ? ? ? . ? ? ? .

• ?
• ? ? ?
• ? ? ? Machine Learning (Deep Learning? ?)?

, ? ? ?. ? ? ? ? .

• ? ? . ? ? ? ?.
• ? ? ? ? ? ? ? .

? ? ? ? ?.

• ? ? ? ? ?
• ? ? ? ? ? ? ?
• ? ? ? ?
• ? ?
  1. ( ?) ? ? ? ? ? ? ?
  2. ? ? ? ? ? ? ? ? ? ?

? ? ? ? ? ? . ? 1 ? ? ? ? , ? ? ? ? .

? ? ? ? ? ? ?.

1? ? Task? ? ? Pre-training( )?, ? ? Pre-trained Model( )? ?.

2? Pre-trained Model? ? ? ? Task? ?. ? 2? Fine Tuning( )? ?. ? ? (Fine) ? ? .

? Pre-training? Fine Tuning ? ? ? .

• (Pre-training ) ? ? ? ? . ? ? ? Pre-trained Model? ? ?.
• Pre-trained Model? ? ? ? ? ? ?. Pre-trained Model? ? Fine Tuning Task ? ? ? ? ?.

? Pre-training? Fine Tuning ? ? ? ? . Pre-training? , , ? . Pre-trained Model? ? ? ? ? (? ? ? ? ?), ? ? ? ? ?. ? ? , ? ? ? ?.

? ? Pre-trained Model? ? ? ?. ? ? ? ? , ? ? .

, , ? ? Pre-trained Model? ? ? . ? ? ( ) Pre-trained Model? Task? ? Fine Tuning? ?. Fine Tuning? ? .

Pre-training? Fine-tuning? Deep Learning ? . ? ? Neural Net? ? ? ? . Weights? ? ?. ? ? ?. Pre-training? Fine Tuning? ?

Pre-training? Neural Net? Weight? 0 ? ?. ?. ? ? ? Neural Net? Weights? ? . ? ? ? ? ? ?.

Fine Tuning? . ? , ? ? ? ? Weights? ?. Pre-training? 0 Weight ?, Pre-training? Weight ? .

? ? ? .

• ? ? ? ? . ? ? ?. Pre-training? ? ?. ? ? -> ? , ? ? ? -> ? ?.
• ? . ? ? Fine Tuning ? ? ? ? ? . ?, ? ? . ?, ? -> ? ? ? ? .

Pre-trained Model? ? Fine Tuning? ? ? ? Fine Tuning ? Fine Tuning ? ? ? ?.

? ? ? ? . Pre-trained Model? ? Fine Tuning ? .

Fine Tuning? Pre-training ? ? ? ? ? ? ?. Pre-training? 70? ? , Fine Tuning? 70? ? . Pre-training 70? ? ? ?, Fine Tuning? ? ? ? ?, ? ? ? . ? ? -> ? . -> ? ? .

Pre-training ? ? ? Pre-trained Model ? Fine Tuning? , , ? ? ?.

? ? . Fine Tuning ? ? ? ? Pre-trained Model? ? Fine Tuning ? ? ? ? , ? ? ?. 70? ? Pre-trained Model? ? , , ? ? 70? * 3 Fine Tuning Model? ?.

? Fine Tuning ? ? ? ?. ? ? ? Pre-trained Model? . ? ? ? ?.

? ? ? ? ?. ? ? ? ? ?. LoRA? Parameter Efficient Fine Tuning ?.

Parameter Efficient Fine Tuning (PEFT)? ? Fine Tuning? ( ? ) Parameter Efficient ?. ? "Fine Tuning ? ? ? Pre-trained Model? ? " ?. ? ? ? ? ? Pre-trained Model ? ? ? ? ? .

? ? ? PEFT . ? PEFT ? ? ? LoRA?.

LoRA? ? ? ? ?. PEFT ? LoRA? ? ?.

• ? ?
  • Fine Tuning ? ? Pre-trained Model? Weights? ?. Weight? Freeze? ?.
  • ? Pre-trained Model? Weights ? Weights? ? ? Weights? . ? Weight? LoRA Weight .
• (Inference)? ?
  • ? Pre-trained Model? Weight? LoRA Weight? . ? Pre-trained Weight? LoRA Weight? ? ? (Weighted Sum) ?.

? ? Freeze ? Pre-trained Model? Weight, ? ? LoRA Weight?.

Fine Tuning Pre-trained Model ? ? ? LoRA Weight? ? ? ?. ? ?

LoRA ? ? GPT-2 Large Full Fine Tuning(774M ) ? 0.1%? Weight(0.77M )? Full Fine Tuning? ? ? ? ?. GPT-3? Full Fine Tuning(175B) 0.0026%? Weight(4.7M)? ? ? ? ? ? ?.

? Fine Tuning? ? ? LoRA ? ? ? ?. Pre-trained Model ? ? Full Fine Tuning? ? ? ?, Full Fine Tuning? LoRA? ? ? Full Fine Tuning? ? ? ?.

? ? ?.

LoRA? Low-Rank Adapation? . Adpation? Pre-trained Model ? ? ? (Adapt) ? ?. ? ? ? ?(?)? ?.

Low-Rank Rank Decomposition ? ? ? ?. ? ? ? W? ? ? ? A? B? ? ? ? (W=A X B). A? B? W Rank? . Rank? ? ? column? ? ?, ? ? ? ? ?.

LoRA Weight? ? ? r? ? . Pre-trained Weight W? ? 500x500? . ? 250,000 ?()? . r? 4? ? ?. A? ? 500x4 (2,000? ), B? ? 4x500(2,000? )? ?. A, B? ? 4,000? ?. r=4 ? ? LoRA Weight? Pre-trained Weight ? 1.6% (4,000 / 250,000) ? ?.

r? 2? ? ? ? ?. r? Pre-trained Weight? ? ? ? ? , r? ? ?. r? LoRA Weight? . ? LoRA r? ? ? ?.

? :)

LoRA? ?

? Full Fine Tuning ? , Full Fine Tuning? ? ? ? ? ? ? .

? Fine Tuning ? Task Weight? ? Pre-trained Model? ? LoRA Weight? ? ? . Pre-trained Model? LoRA ? ? ? Fine Tuning? ? . ? Pre-trained Model? ? ? ? LoRA Weights? ? ?. ? ? ? ? ? LoRA Weight? ? ?.

? ? ? ? ? ? ? ? . GPT-4? LLM? ? . ? ? LLM? ? ? Task? ? Fine Tuning? ?. LLM? ? LoRA? ? Fine Tuning ? ? ?. ? Fine Tuning ? ? , Fine Tuning ? . Full Fine Tuning ? ? GPT-4 Fine Tuning ? ?, ? ?. ? LoRA ? ? LLM? ? ?, Task? LoRA Weights? ?.

LoRA? ? .

Full Fine Tuning ? Weights? ? ? ? (LoRA Weight? ? Pre-trained Weight? ? ?) ? ? ? ? ? . ? ? ? LoRA Weight? Pre-trained Weight ? ? ? ? ? .

? ? LoRA Weight? Pre-trained Weight? ?(Adaptation) ? Pre-trained Model? Weight? ? LoRA Weight? ? . LoRA? Fine Tuned Model LoRA Weight ? .

? Fine Tuning? Prompt Engineering? ? ? ? ? ?. ? ? https://jins-sw.tistory.com/51 ? ? ?.

Fine Tuning? Prompt Engineering? ? ? Pre-trained Model? ? Task ? ? ? ?. Prompt Engineering? Fine Tuning? ? Model? Weight? .

Prompt Engineering? ? ? ? In-Context Learning? . Pre-trained Weight LoRA Weight ? ? ?. ? In-Context Learning? Pre-trained Model(LLM) ? ? (Context)? (Instruct)? ? ? ?. Prompt Engineering? ? https://jins-sw.tistory.com/51 ? ?.

? Prompt Engineering ? LoRA PEFT ? ?. PEFT? ? ? ? ? ? ? ? ? , ? ? ?. ? Prompt Engineering? ? (?)? LLM? ? ? ? ?, ? ? (Context, Instruct)? ? ? ? .

Full Fine tuning? , LoRA PEFT? , Prompt Engineering? ? ? , ? ? ?.

• ChatGPT, GPT-4, PaLM LLM? ? ( 70B ),
  • Prompt Engineering? ? . . zero-shot? ?, ? ? few-shot? .
  • ? ? ? ? LoRA PEFT? .
  • ? ? ? ? Full Fine Tuning ?.
• ? ? ? (7B ~ 30B?)
  • Zero-shot? ?.
  • Task? ? ? few-shot? ? . ? ? ? ? ?.
  • LoRA? ? . 7B Full Fine Tuing? ? ?.
• ? (2B )
  • few-shot? ? ? ?.
  • Pre-trained Model? ? ? ? Fine Tuning? ?.
  • ? ? GPU? LoRA? Full Fine Tuning? ? .
  • LoRA? ? Full Fine Tuning ? ? .

? Pre-trained Model, Task, ? ? ? ?.

? Fine Tuning? LoRA? LLM . ? ? ? ? ? ? ? ?. ? LLM? ? ? ? ? ?.