Pick up a gallon of milk and if they have eggs get a dozen

"The trouble with computers, of course, is that they're very sophisticated idiots." - The Fourth Doctor (and countless others)

Jack Gangi

20 Dec 2024 • 3 min read

✋

It's recommended that you first read https://blog.jackserv.io/i-built-my-own-in-house-ai-model-and-hes-kind-of-a-dick-2/ for a little bit of context. Although I can't promise you any clarity.

Within Open WebUI, there's a setup area called "Knowledge." The idea is that the user can upload documents, files, or data that the model can reference during conversations.

While this has a wide variety of uses I want to narrow in on a certain area. More of a personal AI type of set up. I downloaded all of my prescription records from my local pharmacy for the last year and uploaded them into Zoltar. The idea being I could ask questions about my last year of prescribed medicine and get answers.

There are models designed for this very thing but I am still using a basic Llama model because I don't plan to go too far into the weeds with these inquiries.

Once you go to the knowledge section and create an area. I'll call this one test. You then upload your media. I chose to convert a PDF to text for this process.

⚠️

Once upon a time not all computers had the same fonts and font substitutions was a black art. Even printers came preinstalled with certain font sets. The point of a PDF was that the document looked the same on the page as its did on the screen. Uploading a PDF will work 99% of the time but I'm beginning to think it's good practice to convert the document to text before uploading. Or don't, I'm not a cop.

From there, you create a Knowledge area to have it upload. The AI won't just search everything you upload with every inquiry (that would add some extra time), so you have to tell it what you're looking for, kind of.

It's best to keep what you want in categories. Here I've created a few dummy categories as an example. I could upload all of my owner's manuals and then double-check how to do a hard reset on a smart bulb I own. But truth be told, it's probably quicker to Google at this point. It's more useful for information that isn't floating around on the internet, like prescription records, family recipes, and that sort of thing. And it's also not just about calling up information—it's about doing stuff with it.

When you ask the AI a question that you want it to use your uploaded documents to answer you just preference it with the # symbol.

Of course I substituted a real medication with Flintstone vitamins for privacy reasons. If my insurance plan covered prescriptions for Flintstone vitamins I would be unstoppable.

The AI came back and told me I had been prescribed Flintstone vitamins 90 times in the last year. Untrue.

Upon further examination, I found six prescriptions for the year with a quantity of 90 in the bottle ("Qty: 90"). So, it definitely looked at and pulled the information; it just got a little confused parsing it. Logical mistake and easily fixed with some massaging.

But I still wanted proof of concept. Which is just a fancy way of saying I wasn't done playing around.

I wrote the most basic of documents. Probably my best writing work to date.

And voila!

Concept proven.

While no earth shattering results were found. t’s a great starting point for exploring the potential of a more advanced personal AI system.