The first version of Eliza I ever played with ran on my Commodore 64. I loaded it up and started answering its questions. It kept asking more. For about ten minutes I was genuinely impressed. What can I say, I was a slow learner. This thing was prescient. It understood me. It was following my train of thought.
Then it wasn't. It was really just pattern matching. Eliza scanned what I typed for keywords and reflected them back as questions. She wasn't following my train of thought. She was running a script. Faster than a human. Better vocabulary. Same trick, every single time.
I think about Eliza a lot when I watch people use RAG-based AI systems these days.
Retrieve. Synthesize. Discard. That's the loop every RAG system runs when you ask it a question. Every query starts fresh. The system has no memory of what it figured out yesterday. It doesn't build judgment about your domain. It doesn't know it worked through a complicated architecture question last Tuesday and reached something useful. It just retrieves and synthesizes again. From scratch. Every time.
We've spent two years building products on top of this loop and calling it knowledge management. It isn't. It's fast lookup with a language layer on top. Yes, it's useful. But it's not compounding. Like Eliza, it doesn't get better with time.
Compounding knowledge looks different. The system keeps what it figured out, not just where it looked. It builds a model of your domain over time. It knows what it worked through last week and uses that as a starting point today. That's architecturally different from RAG. It's not a better RAG, it's a different system altogether.
I know this because I built it. My personal AI assistant gets a little smarter about my world every day. That's not magic. That's a design decision.
So the question worth asking isn't "how do I make my RAG smarter?" It's "does my use case actually need a system that gets smarter over time, and if so, am I building the right thing?"
Eliza asked good questions. The problem was that she couldn't really learn from the answers, even though it seemed like she could sometimes.
Originally posted on LinkedIn: "I Learned About AI from a 40-Year-Old Chatbot"