[AI Search Engine] Bitcointalk

[Vod]

I have not settled on a model yet, could be donation-based or a limited free tier with optional premium access if demand justifies it, It all depends on how useful people find it here and whether it gains traction.

If you don't mind me asking (and feel free to not answer): what kind of running cost are we looking at here?

Wow!  With 256GB of RAM, you should be able to process queries almost instantly, depending on your CPU strength.  I tried running a Deepseek clone on xeon processors with only 32GB of RAM, and it took 17 mins to process a query.  But you only have a small subset of what it was trained on, so I think you are probably running on a third party LLM service, would I be right?

Regarding indexes, I'm not sure how they work in AI queries, or even if they are needed when you can store the entire database in memory, but I think that's where you can find the best improvement potential.  AI query analyzers must exists, run one on your dataset and it can tell you how you should be storing your data if not by simply one record per post.

Finally keep in mind that the big AI teams are releasing new models every week, which are followed quickly by many smaller companies providing these "one plan fits all" type AI offerings.   

[ABCbits]

I have not settled on a model yet, could be donation-based or a limited free tier with optional premium access if demand justifies it, It all depends on how useful people find it here and whether it gains traction.

If you don't mind me asking (and feel free to not answer): what kind of running cost are we looking at here?

I thought AI these days use GPU or even specialized hardware for AI. Is there any reason you're not using either?

[u0u]

I have not settled on a model yet, could be donation-based or a limited free tier with optional premium access if demand justifies it, It all depends on how useful people find it here and whether it gains traction.

If you don't mind me asking (and feel free to not answer): what kind of running cost are we looking at here?

https://i.ibb.co/pjMhhwKG/image.png

Wow!  With 256GB of RAM, you should be able to process queries almost instantly, depending on your CPU strength.  I tried running a Deepseek clone on xeon processors with only 32GB of RAM, and it took 17 mins to process a query.  But you only have a small subset of what it was trained on, so I think you are probably running on a third party LLM service, would I be right?

Regarding indexes, I'm not sure how they work in AI queries, or even if they are needed when you can store the entire database in memory, but I think that's where you can find the best improvement potential.  AI query analyzers must exists, run one on your dataset and it can tell you how you should be storing your data if not by simply one record per post.

Finally keep in mind that the big AI teams are releasing new models every week, which are followed quickly by many smaller companies providing these "one plan fits all" type AI offerings.   

I have not settled on a model yet, could be donation-based or a limited free tier with optional premium access if demand justifies it, It all depends on how useful people find it here and whether it gains traction.

If you don't mind me asking (and feel free to not answer): what kind of running cost are we looking at here?

https://i.ibb.co/pjMhhwKG/image.png

I thought AI these days use GPU or even specialized hardware for AI. Is there any reason you're not using either?

That setup was more of a sneak peek and not the whole thing, It is more about the architecture and how the system is optimized than just raw GPU power, Im afraid that you are comparing apples and oranges, RAM helps a lot with caching and avoiding disk IO, but that does not mean "instant" unless you are showing a cached answer for the same query, There are also tools and best practices, but the idea that a magical analyzer will tell you how to structure everything perfectly is oversimplified.

[Vod]

Wow!  With 256GB of RAM, you should be able to process queries almost instantly, depending on your CPU strength.  I tried running a Deepseek clone on xeon processors with only 32GB of RAM, and it took 17 mins to process a query.  But you only have a small subset of what it was trained on, so I think you are probably running on a third party LLM service, would I be right?

Regarding indexes, I'm not sure how they work in AI queries, or even if they are needed when you can store the entire database in memory, but I think that's where you can find the best improvement potential.  AI query analyzers must exists, run one on your dataset and it can tell you how you should be storing your data if not by simply one record per post.

Finally keep in mind that the big AI teams are releasing new models every week, which are followed quickly by many smaller companies providing these "one plan fits all" type AI offerings.   
[/quote]

I think I was wrong here - I was comparing old CPU/GPU technology to your AI setup.  I see Nvidia now has AI chips with TB of RAM and several thousand CPUs!
https://www.nvidia.com/en-us/data-center/gb200-nvl72/

[Asiska02]

This is great, I just tried it out now and I was so amazed by the result given to me on what I asked about the forum. I used a mobile to asked a question, I didn’t get a full screen capture that can be reposted here but it really gave me more than enough information on the question I asked about the forum.

The only thing I noticed which if improved will be worthwhile is the time taken by the AI search tool to present results just instantly. In your description, you mentioned that it is a demo that you’re trying out, really good one. I suggest making an adjustment on that for faster result when asking a question.

This will be a great tool to use around in the forum. How do you intend to incorporate this in the forum?

[satscraper]

I was so amazed by the result given to me on what I asked about the forum.

Opposite experience here.

To test this tool, I asked about merit sources on the Russian local board, but the result I received was outdated and didn’t reflect the current situation. It might have been accurate for users active five or six years ago, but definitely not for now. So unfortunately, I can’t give this tool much credit, sorry.