Web Science and Digital Libraries Research Group

As Natural Language Processing (NLP) evolves, Large Language Models (LLMs) are capable of interpreting text and extracting meaningful information. However, during my recent experimentation with Named Entity Recognition (NER) tasks, I found that different LLMs often produce varying outputs, much like humans offering diverse answers based on their unique perspectives. This leads to an interesting challenge: how can we unify the inconsistent entity list generated by multiple LLMs into a consistent and reliable output?  If you want to replicate these experiments, you can access the GitHub repository . Figure 1. Differing Entity Extraction Results from Two LLMs for the Same Text Input. The Challenge LLMs Capture Entities Differently For this experiment, I used Ollama , an open-source project that serves as a powerful and user-friendly platform for running LLMs on a local machine. Specifically, I employed Llama 3.1 8B (Meta), Phi 3.5 8B (Microsoft), and Gemma 2 9B (Google). During ...