Kevin Vance - Okay, I'm on vacation, so I shouldn't even be posting here. That…

Entries | Archive | Friends | Friends' Friends | User Info

11:19 pm

Wednesday, August 20th, 2003
Previous Entry Share Next Entry
Okay, I'm on vacation, so I shouldn't even be posting here. That said, sometimes I get ideas, and I need to write them down to make sure they're not totally stupid.

The other day, I was reading this article on implementing a vector space search engine in perl. I'd never heard of that type of search engine before, but I liked the idea. Today, I saw this LJ sitcom thing, which spits out some random stuff interspersed with names from your friendslist.

Well, it's not a huge leap from there. The executive enters a username or word/phrase with aliases, and a timeframe. We can break down the next part into a couple stages.

Document gathering. We spider all friends' entries, the target user's entries, his friends' entries (if there was a target user), over the given timeframe. Maybe even jump out to friendsfriends, assuming sick amounts of cpu and memory (cache everything!!!) Cull out anything that's friends-filtered.

Vectorize every entry, making sure to include specific phrases the executive has input.

Find the phrase. Find other common phrases near it. That was the easy part. Now we have to use something like Lingua::LinkParser to see if teferi drives the car, or the car drives teferi. Note whose entries provided this information.

In an easy-to-read format, report who said what about our target user or phrase. I don't suppose there's a perl module for this.

So, now that I know it's a stupid idea, I can get to sleep.
Link )Reply )

Comments
From: (Anonymous)
2003-08-20 09:22 pm (UTC)
That sounds incredibly useful and worth a few billion dollars if you can get it working.

I have a feeling Lingua won't be good enough, really.
-md
(Reply) (Thread)
From: zztzed
2003-08-20 10:18 pm (UTC)
Now we have to use something like Lingua::LinkParser to see if teferi drives the car, or the car drives teferi.

. o O ( Lingua::LinkParser::YakovSmirnoff )
(Reply) (Thread)
[User Picture]From: teferi
2003-08-21 06:55 am (UTC)
For what it's worth, the car does, in fact, drive me.
(Reply) (Thread)