Project Update: Linux AI Backend POC

Over the past few months, I’ve been developing an on-premises AI solution to help creative professionals leverage powerful AI tools—without compromising performance while running heavy creative applications. Today, I’m excited to share a major milestone: a full rebuild of our AI backend system, Samaritan.

The Challenge

Creative work on Windows workstations can suffer when AI tools compete for the same resources. Running text generation models alongside Adobe Creative Suite, video editors, or 3D rendering software often results in frustrating slowdowns—where neither side performs optimally.

The Solution: A Dedicated AI Backend

The new approach separates these concerns by offloading AI workloads to a dedicated Linux server. This “Samaritan” AI backend now handles:

• Text generation via Ollama (models like TinyLlama)
• Image generation through ComfyUI
• Unified access via a centralized web dashboard

This modular system gives creative professionals full access to AI capabilities over the network—while preserving peak performance on their primary workstations.

Technical Architecture

The rebuilt stack is clean, scalable, and containerized using Docker. Core components include:

• Base OS: Ubuntu Server with configured networking, security, and LVM storage
• Ollama Container: Hosts language models with API-ready endpoints
• Web UI: A browser-based control center for interacting with AI services
• Shared Storage: Networked access to models, logs, and output across devices

Lessons Learned

This rebuild reflects growth from our earlier proof-of-concept. Key insights include:

• Streamlined container orchestration for AI workflows
• Proxy/API configurations to solve CORS and routing issues
• Smarter model/data management in low-resource environments
• Clean separation of services for easier updates and debugging

What’s Next

With the foundation rebuilt, we’re now focused on next-level enhancements:

• Persona switching and multi-user support
• Advanced model lifecycle management
• System logging, analytics, and monitoring
• Additional AI tools (audio, voice cloning, video, etc.)

Conclusion

Rebuilding Samaritan from the ground up has resulted in a smarter, more maintainable backend that allows for growth without sacrificing creative performance. This architecture keeps your creative tools running at full speed, while AI tasks run where they belong—on a dedicated backend built for the job.

I’m excited to keep pushing forward and share more soon. If you’re a creative pro exploring how to run AI locally or streamline your workflows, feel free to reach out—I’d love to share ideas.