Mentions not searching by partial strings

[KyleMaas]

mentions operator appears to be built to check when a term is included. However, it acts more like an equal operator - I can't search based on part of a mention. It has to be the whole thing for me to get results.

[arj03]

True, but for these prefix indexes it should be possible to do something akin to a LIKE 'XYS%' where XYS can then be the first 32 bits of the search term.

[staltz]

Could you give an example of input, output, and expected outcome?

[KyleMaas]

On the console, this: SSB.db.query(SSB.db.operators.and(SSB.db.operators.mentions(SSB.net.id)), SSB.db.operators.toCallback((err, msgs) => { console.log("err: ", err); console.log("msgs: ", msgs); }))

Yields this (not expanding the arrays, for brevity's sake):

err:  null
msgs:  (4) [{…}, {…}, {…}, {…}]

If I instead do this: SSB.db.query(SSB.db.operators.and(SSB.db.operators.mentions(SSB.net.id.substring(0, 4))), SSB.db.operators.toCallback((err, msgs) => { console.log("err: ", err); console.log("msgs: ", msgs); }))

I get this:

err:  null
msgs:  []

If I cut off the first byte of my ID: SSB.db.query(SSB.db.operators.and(SSB.db.operators.mentions(SSB.net.id.substring(1, 5))), SSB.db.operators.toCallback((err, msgs) => { console.log("err: ", err); console.log("msgs: ", msgs); }))

I get this:

err:  null
msgs:  []

[staltz]

Ok, I see. But why would you need to do id.substring(1,5)? What's the purpose?

[KyleMaas]

What I would ideally love to be able to do is search for a mention with a search term "ima" and get blobs with names "image.png" back, not just IDs. I'm not sure this is possible now, but while I was hunting through the code trying to figure out if that kind of thing was supported, I ran into this issue.

[KyleMaas]

@staltz To limit it to a 32-bit prefix search, as @arj03 mentioned, to try to cover both test cases.

[KyleMaas]

@staltz Oh, you mean why would you cut off the first byte? Just in case one it didn't like the initial "@" and was searching only on the first 32 bits of the ID itself.

[staltz]

What I would ideally love to be able to do is search for a mention with a search term "ima" and get blobs with names "image.png" back, not just IDs. I'm not sure this is possible now, but while I was hunting through the code trying to figure out if that kind of thing was supported, I ran into this issue.

Ok, I sort of see the purpose now, but I think these prefix indexes are not suited for full-text search, or search of any type. If you want to find all msgs that have msg.value.content.mentions[].name === 'image.png', that's possible with another prefix index for .name and you would pass in "image.png" as the input, not "image.png".substring(0,3). I still don't understand why would you want to pass a substring as the input.

[staltz]

Perhaps the term "INCLUDES" hinted towards some full-text search capabilities. That's maybe a misnomer. INCLUDES is merely an "EQUAL" over the items in an array, similar to JS Array.prototype.some.

[KyleMaas]

Ah, gotcha. So it is still a "full match" type search and does not support partial strings, and that is intentional. Good to know.

This issue was about not being able to search for partial IDs. I should probably file another one for the "what I was hoping it would do when I ran into this". But the feature I was looking for would have been for the Markdown editors we're use in ssb-browser-demo, which allow users to type "@" and have a list of user suggestions pop up which get winnowed down as you type. I was hoping to do the same thing with "&", per @arj03's feature request arj03/ssb-browser-demo#124 and I got it working in arj03/ssb-browser-demo#162 using a port of ssb-meme to db2 here: https://github.com/ssbc/ssb-meme

But it's...kludgy. I ended up having to pull the most recent posts and run them through ssb-meme's mention search. It would have been much nicer if I could do a prefix search for mention names. I'll go ahead and file that as another issue, though, to keep from cluttering up this one, since this one was about partial ID searches.

Whether "full match search" is the expected behavior or not, it would be awfully nice if that was documented. It tripped me up, and even @arj03 mentioned above that that should be possible.

[staltz]

Nice, this gives me much better context on what you were trying to do, and I agree we would should find a way to make db2 support allow users to type "@" and have a list of user suggestions pop up which get winnowed down as you type and I believe prefix indexes will be limited for that.

[KyleMaas]

Sounds good. We're using ssb-suggest right now for the "@" mentions. It's not perfect, and apparently the license of it is incompatible with a lot of other stuff (including ssb-browser-demo), but it mostly works.