A Django-Powered System for Translating and Enriching RSS Feeds with Local LLM Intelligence
This project is a Django-based system that automatically imports, translates, and enriches RSS feeds using a local Large Language Model.
As the volume of online content continues to grow, the ability to automatically collect, translate, and enrich information becomes increasingly valuable. This project delivers a fully automated RSS feed translation and processing platform built on top of modern Python tooling, Django 5, and a locally hosted LLM stack. Designed for reliability, reproducibility, and extensibility, the system transforms raw RSS feeds into structured, multilingual content—complete with full-text extraction, translation, enrichment, and long-term lifecycle management.
Modern Python Foundation with Django 5
The platform is built on Python 3.13 and Django 5.x, leveraging Django’s ORM, admin panel, and management commands as the backbone of the application. Feeds, feed items, translation settings, and processing metadata are fully managed through Django models. The built-in Django Admin provides a powerful control panel where users can monitor import status, inspect translation outputs, adjust retention policies, and modify automation settings without writing a single line of code.
Custom management commands extend Django’s tooling to handle tasks such as browser-based article extraction, bulk imports, and scheduled maintenance operations. This keeps the system modular, scriptable, and easy to automate during development and deployment.
Automated Full-Text Extraction with Headless Browsers
While RSS feeds often include only summaries, this system retrieves the full article body using configurable headless Chrome or Firefox sessions. These browsers simulate real user visits, ensuring accurate retrieval of dynamically generated article content.
Extracted HTML is then processed with trafilatura, a specialized library that isolates clean article text from boilerplate elements. The system also employs langid to verify detected languages and ensure translation logic applies correctly.
Local LLM Pipeline with LangChain, Ollama, and tiktoken
At the core of the translation workflow is a local Large Language Model served through Ollama, orchestrated with LangChain. This gives the system LLM performance without relying on external APIs, keeping translations private, fast, and cost-controlled.
Large article bodies are intelligently broken down using tiktoken, which tokenizes HTML content into LLM-safe chunks. This ensures translations respect model context window limits while preserving structural continuity across sections.
LangChain handles prompt management, chain logic, retry strategies, and streaming responses, enabling the system to deliver robust translations even with large or inconsistent source material.
Celery-Powered Background Processing and Scheduling
To keep the system responsive and scalable, heavy operations—feed refreshing, content extraction, translation, summarization, and cleanup—run in the background via Celery. Celery Beat schedules recurring tasks at configurable intervals, ensuring feeds stay fresh and translations stay up-to-date.
Redis serves as the Celery broker, result backend, and Django cache layer, providing fast, in-memory coordination for distributed tasks. Logging is handled through rotating file logs, giving developers actionable audit trails and simplifying debugging.
Retention Policies and Automated Cleanup
The platform includes a lifecycle management system that automatically deletes old feed items based on configurable retention settings. This prevents uncontrolled database growth and keeps admin views clean while ensuring relevant content remains accessible.
During development, the application defaults to SQLite for simplicity, but can easily be upgraded to PostgreSQL or another production-grade database.
Flexible, Extensible, and Built for Automation
Because the entire system is driven by Django, Celery, and Python’s rich ecosystem, it is easy to extend with:
- additional LLM models
- new translation pipelines
- enrichment logic (summaries, keywords, sentiment)
- multi-feed batching strategies
- export endpoints or APIs
Optional PowerShell scripts streamline setup and development tasks on Windows environments, making onboarding fast and repeatable.
Conclusion
This project provides a complete solution for automatically importing, translating, and enriching RSS feeds using modern Django architecture and local LLM technology. With robust background processing, browser-based full-text extraction, HTML-safe chunking, and an intuitive admin interface, it delivers a capable and extensible platform for multilingual content processing—fully automated and built for real-world deployment.