Abstract: In recent years, the advent of large language models (LLMs) has revolutionized the field of artificial intelligence, particularly in natural language processing (NLP). A pivotal aspect of LLM training is parameter fine-tuning, a technique that enables the enhancement of model capabilities under limited resources. This paper offers an exhaustive review of the primary fine-tuning approaches for LLMs from 2019 to 2024, examining over 50 methods. It categorizes and elucidates these methods into four distinct types: full-parameter tuning, partial-parameter tuning, additional-parameter tuning, and parameter-free tuning. Each approach is discussed in terms of its underlying principles, the specific parameters adjusted, and its unique characteristics. Further, the paper delves into a comparison of the parameter volume, memory usage, and computational demands of these methods. Concluding with insights drawn from the fine-tuning research and practical applications, the paper presents recommendations for optimizing LLM fine-tuning.