Running local models
When building LLM applications with LangChain, you need to decide where your models will run.
- Advantages of local models:
- Complete data control and privacy
- No API costs or usage limits
- No internet dependency
- Control over model parameters and fine-tuning
- Advantages of cloud models:
- No hardware requirements or setup complexity
- Access to the most powerful, state-of-the-art models
- Elastic scaling without infrastructure management
- Continuous model improvements without manual updates
- When to choose local models:
- Applications with strict data privacy requirements
- Development and testing environments
- Edge or offline deployment scenarios
- Cost-sensitive applications with predictable, high-volume usage
Let’s start with one of the most developer-friendly options for running local models.
Getting started with Ollama
Ollama provides a developer-friendly way to run powerful open-source models locally. It provides a simple interface for...
