Modelo de Inteligencia Artificial en el borde basado en Ollama para inferencia local en dispositivos de Internet de las cosas

The Internet of Things (IoT) increasingly relies on cloud computing for data processing and decision-making, introducing latency, connectivity, and privacy risks in sensitive domains such as assisted living and smart homes. This paper presents the Ollama-Integrated Edge Architecture (OLLiE), which enables local reasoning on embedded IoT devices using quantized large language models (LLMs). OLLiE was implemented in an assisted smart home scenario using Raspberry Pi 5, ThingsBoard, Redis, Ollama, and local and cloud-based LLMs from the Gemma, Phi, and Qwen families. The evaluation considered functional metrics, including precision, recall, and F1-score, and non-functional metrics, such as latency, RAM usage, CPU load, temperature, and estimated energy consumption. Results reveal a trade-off between inference speed, privacy, and computational sustainability. Cloud models achieved lower latency (2.5–3.1 s), whereas local models preserved edge processing with higher latency (76–115 s). Among them, phi-3.5-mini (Q2_K) achieved the best local inference quality.