Selecting your enterprise LLM: Moving beyond the hype to make the right choice
Over the past year, I’ve been asked regularly what the best way of selecting a Large Language Model (LLM) is. With over 146 LLMs listed in Ollama’s model library alone, selecting the right model has become increasingly complex. While ChatGPT dominates headlines, businesses must look beyond hype to match the right model to their specific needs. Choosing the right LLM isn’t just a technical decision—it directly impacts your organisation’s ability to drive innovation and achieve its strategic goals, making collaboration between technical and non-technical teams essential to ensure that both business needs and operational constraints are fully understood.
Do you actually need generative AI?
Before we get into LLM selection, there is one question to ask first: Is your use-case a good fit for generative AI?. I’ve seen numerous cases where organisations, caught up in the AI hype cycle, attempt to force-fit generative AI and LLMs into problems that could be more appropriately solved using more ’traditional’ AI or even no AI at all.
For example, a retail company aiming to automate the categorisation of customer support tickets might consider generative AI, attracted by its flexibility. However, if ticket categorisation follows fixed criteria, a straightforward rules engine could achieve faster and more accurate results with minimal computational overhead. Or a financial services company wanting to use LLMs to detect potential credit card fraud in real-time - a task much better suited to traditional machine learning models trained on historical transaction patterns.
So, assuming you’re happy that your use-case requires generative AI, the next step is determining how to implement it effectively. Drawing from my experience guiding organisations through this process, I’ve observed that the most successful implementations start by addressing four fundamental business considerations: strategic value, risk management, organisational readiness, and operational sustainability.
Understanding Strategic Value
Strategic value in LLM implementation isn’t just about automation or cost reduction - it’s about identifying where these models can create genuine business advantage. This advantage often emerges in areas where the work requires understanding context, handling ambiguous requests, or generating creative content. For instance, a healthcare provider might use an LLM to summarise patient records, allowing doctors to focus on diagnosis, while an e-commerce platform could deploy one to generate personalised marketing content.
The key is to look for use cases where LLMs can augment rather than replace existing capabilities. Rather than viewing LLMs as a replacement for human expertise, successful organisations typically deploy them to handle routine cognitive tasks, freeing up their people to focus on higher-value work that requires complex judgment, emotional intelligence, or creative problem-solving.
Consider where your organisation spends significant time on tasks that require understanding and generating human language. Content creation, document analysis, code development, and customer interaction are common areas where LLMs can provide strategic value. However, the value isn’t just in the immediate task automation - it often comes from the ability to scale operations without proportionally scaling costs, or from enabling new services that weren’t previously feasible.
Once you’ve identified where LLMs can drive strategic advantage, it’s crucial to consider the risks associated with their deployment.
Managing Risk
Risk assessment for LLM deployment requires a broader view than traditional technology implementations. Beyond the usual considerations of data security and system reliability, organisations need to evaluate risks specific to generative AI technology.
Model provenance and licensing present fundamental risks that must be addressed early. Many LLMs describe themselves as ‘open source’, but this can be misleading - there’s an important distinction between models that are truly Open Source (meeting the criteria of the Open Source Initiative) and those that merely make their weights publicly available under restricted licenses. Organisations need to understand how their chosen LLM was trained, what data it was trained on, and whether its use could infringe on intellectual property rights. The licensing terms of many models can be complex and may change over time, potentially affecting both deployment options and long-term costs.
Bias, explainability, and hallucinations represent another critical risk category. All LLMs can exhibit biases present in their training data, potentially leading to unfair or discriminatory outputs. The opaque nature of these models can make it difficult to explain their decision-making process - a particular concern in regulated industries. This is why implementing ‘human in the loop’ processes is crucial - having human experts review and validate model outputs, particularly for high-stakes decisions. Enterprise platforms like Amazon Bedrock provide built-in guardrails and evaluation tools that can help organisations detect and mitigate these risks, but they don’t eliminate the need for human oversight. Remember that generative AI is fundamentally making mathematical predictions about which text should come next - it can confidently present incorrect information through hallucinations, making this combination of technical safeguards and human validation essential for responsible deployment.
Data privacy stands at the forefront of operational risks. Organisations must understand how their chosen LLM handles input data, whether that data could be used for model training, and what controls exist to protect sensitive information. This becomes particularly crucial when dealing with customer data, intellectual property, or regulated information.
Vendor stability and model continuity present another key risk area. As LLM technology evolves rapidly, organisations need assurance regarding the longevity of their chosen solutions and clarity on dependencies related to specific providers or models.
Ongoing monitoring and regular model evaluations are essential to ensure that performance remains aligned with organisational goals and mitigates evolving risks.
Organisational Readiness
Your organisation’s current capabilities should significantly influence your LLM selection. Different models demand different levels of technical expertise, infrastructure, and organisational support.
Some LLMs require substantial in-house expertise in prompt engineering and model behaviour to use effectively. Others come with more user-friendly interfaces and built-in guardrails. Managed services might be more appropriate for organisations early in their AI journey, while those with mature AI capabilities might benefit from more flexible, self-hosted options, on-premises, and hybrid solutions.
Consider also your organisation’s ability to handle model outputs. Some LLMs provide more consistent, predictable responses but might be less powerful. Others offer greater capabilities but require more sophisticated validation processes. Your choice should align with your team’s ability to effectively verify and utilise the model’s outputs.
Operational Sustainability
Selecting an LLM isn’t just about its current capabilities - it’s about ensuring sustainable, efficient operations over time. The model you choose will have long-term implications for both resource consumption and operational costs.
Cost structures vary significantly between LLMs. While some models might appear cheaper based on token pricing, you need to consider the total cost of ownership. This includes not just the direct costs of model usage, but also the computational resources required, the number of iterations typically needed to get good results, and any associated infrastructure costs.
Environmental impact is another key consideration. Different models have varying computational requirements and energy footprints. Smaller, more efficient models might be more appropriate for routine tasks, while larger models could be reserved for complex operations where their additional capabilities justify the increased resource consumption.
Tracking metrics such as energy usage, inference latency, and iteration costs can provide insights into the sustainability of your chosen model and demonstrate strong oversight of the environmental impact of the use of AI in your business.
As sustainability becomes a priority, AI’s environmental footprint is under increasing scrutiny. Smaller, efficient models reduce costs, align with corporate social responsibility goals, and help organisations adapt to rising regulatory pressures.
Making Your Choice
Selecting the right LLM is a nuanced decision that requires balancing multiple factors. While it’s tempting to choose the most powerful or popular model, success lies in finding the LLM that best matches your specific needs across all four considerations: strategic value, risk management, organisational readiness, and operational sustainability.
As the LLM landscape continues to evolve, maintaining flexibility in your approach becomes crucial. The LLM landscape is dynamic, and what fits today may require reassessment tomorrow. Plan for adaptability to harness advancements without being locked into suboptimal choices. The key is to make an informed choice based on your current requirements while keeping an eye on future scalability and adaptation.
Remember that different use cases might benefit from different models. Just as organisations learned to use different compute instance types for different workloads, the future of LLM implementation will likely involve multiple models, each chosen to address specific needs while maintaining clear alignment with business objectives.
As generative AI continues to redefine industries, staying informed and agile in your LLM strategy will be vital. By carefully weighing strategic value, risk, readiness, and sustainability, organisations can unlock the transformative potential of LLMs while ensuring responsible and scalable deployments.
Generative AI’s transformative potential is within reach for organisations willing to balance ambition with thoughtful planning. Begin by aligning your AI strategy with your business goals and build a foundation for scalable and responsible innovation.
About the Author
Mario Thomas is a transformational business leader with nearly three decades of experience driving operational excellence and revenue growth across global enterprises. As Head of Global Training and Press Spokesperson at [Amazon Web Services](https://aws.amazon.com) (AWS), he leads worldwide enablement delivery and operations for one of technology's largest sales forces during a pivotal era of AI innovation. A Chartered Director and Fellow of the [Institute of Directors](https://www.iod.com), Mario partners with Boards and C-suite leaders to deliver measurable business outcomes through strategic transformation. His frameworks and methodologies have generated over two-billion dollars in enterprise value through the effective adoption of AI, data, and cloud technologies.