Use of large language models in medicine
TL/DR
The implementation of Language Models in the medical field represents a major evolution. These models promise to revolutionise the way in which diagnoses, treatments and interaction with patients are approached. From pre-training to fine-tuning and prompting, these artificial intelligence techniques are setting new standards in medical care. Still, the application of these complex systems comes with significant challenges, particularly in terms of biases in training data, privacy and security of patient data, and the need to respect regulations and ethical standards. It is therefore crucial that as LLMs improve and become even more effective tools, it is important to improve the standards that govern them. These standards must ensure that the models work well in hospitals and clinics, while protecting patients’ private information. In addition, it is necessary that everyone understands how these systems work and that they are used fairly. In this way, we can maximise the benefits of these models in medicine, while ensuring that the technology is used safely and for the benefit of all patients.
Use of large language models in medicine
As we have been seeing in this blog, in recent years, Artificial Intelligence has transformed countless fields, offering innovative solutions to complex problems and significantly improving efficiency and production in many institutions and industries. One of the most notable advances is in the area of Natural Language Processing through language models. These computer algorithms, capable of understanding, generating and translating human language, have revolutionised the field of Artificial Intelligence, especially in text generation, opening new frontiers in fields such as medicine. In this sector, advanced language models, such as the Generative Pre-trained Transformer (GPT) and Bidirectional Encoder Representations from Transformers (BERT) families, among others, have led to remarkable advances that were unthinkable just a few years ago. They are used for a wide variety of tasks, from understanding and automatically generating medical reports to creating online medical consultation systems, enabling more personalised treatments and diagnoses tailored to the needs and problems of each patient, improving customer service, optimising processes and facilitating medical discoveries.
Principles of Language Models in Medicine
The integration of language models in medicine has significantly improved the way clinical and research problems are addressed. Thanks to these advanced Artificial Intelligence techniques, it has been possible to understand and generate medical language much more effectively. The principles behind these techniques consist of pre-training, fine-tuning and prompting.
Pre-training
The first step, pre-training, involves training the models with large datasets. Two BERT-based models used in this area are PubMedBERT and ClinicalBERT, both trained on PubMed and MIMIC-III, respectively. These data allow them to absorb extensive medical knowledge from health records to specialised knowledge.
Finetuning
Second, finetuning adapts these general models to more specific medical needs. For this, methods such as Supervised Fine Tuning (SFT) or Low Range Adaptation (LoRA), a training technique that significantly reduces the number of trainable parameters, are used.
Prompting
Finally, prompting helps to find the desired response. Techniques such as the Chain of Thought (CoT) and the use of models such as MedPaLM for prompt tuning help to optimise the model’s ability to perform specific instructions. MedPaLM, specifically, applies prompting methods to improve its performance on specific medical tasks. This allows the general PaLM model to specialise in complex medical tasks without the need for additional training. MedPaLM, which is based on the general PaLM model, is significantly enhanced by providing you with concrete examples, such as medical question and answer pairs, allowing the model to learn from these examples and respond accurately to medical queries.
Clinical Applications of Language Models in Medicine
The incorporation of language models in clinical applications helps to improve many areas, from medical diagnostics to mental health support. Below are some relevant applications of models in different medical domains:
Medical diagnosis: many language models use graph-based architectures to improve diagnostic accuracy. However, they do not have the ability to analyse medical images directly. Therefore, vision models such as ChatCAD are used as a complement to interpret images and translate them into text, providing more accurate diagnoses.
ICD Code Coding: Language models are able to automate the assignment of ICD (International Classification of Diseases) codes, which are international codes for the classification of diseases. Models such as PLM-ICD are an example showing how LLMs are able to automate this process.
Medical Robotics: Applications in medical robotics, such as in path planning for assistive robots, show how LLMs can improve human-computer interaction in clinical settings.
Mental Health Support: Language models can also be used as chatbots. Acting as conversational assistants, they can help patients, offering an accessible and low-cost alternative for mental health support.
Ethical Considerations of Language Models in Medicine
The application of language models in medicine raises important ethical considerations. Biases in the training sets with which the models are trained are a major concern, especially in fields such as medicine where sensitive information is handled. These biases can lead to erroneous clinical decisions, negatively affecting patients. Moreover, with such sensitive data, the privacy and security of patient data is crucial. That is why strict controls are needed to protect sensitive data. Also, applying these models to healthcare involves following regulations and ethical principles to ensure patient safety.
