Recommendation engine for the oncology community

Consider this tweet first:

I have issues with Twitter because there is an unseen algorithm which determines the visibility of the tweets algorithmically. These “associations” depend on the follows/retweets and the spawning complex of Twitter cookies. I abhor Twitter’s cookies following me around the web, so I usually restrict them to my sub-domain and they get deleted automatically. Besides, I use system wide ad blocking and avoid the mobile applications.

This tweet gathered little traction- I wanted this to get a wider view and despite mutliple tags and “re-tweets”, I could get seven recommended reads. I saved them up and published them all with their summaries and annotations (wherever I could). Some merit a deeper look though.

The bigger question was- What if I could eliminate the need for Twitter posts like above and get the recommendations in real time? What if there were a recommendation engine to suggest best articles from the community, for the community and in complete privacy?

My prayers were answered with Findka. Long time readers will know that I am old school tech enthusiast and strongly believe in privacy by design. Therefore, I consume an immense amount of RSS feeds using Inoreader. There was a hacker news discussion around the novel way the RSS feeds could be mixed and matched through the recommendation engine, and I found the perfect answer- Findka. The developer was kind enough to listen to my ramblings on Zoom and subsequently on Telegram.

First consider this:

Most RSS aggregators keep your feeds separate. Findka instead merges them into a single feed using a bandit algorithm. If you’ve subscribed to three feeds—A, B and C—Findka will start out picking articles from the feeds uniformly. 1/3 of the articles will come from feed A, etc. As time goes on, Findka will adjust the distribution based on your usage data. If you never click on articles from feed A and you always click on articles from feed B, then Findka will show you fewer articles from A and more articles from B.

This is a perfect recipe for the recommendation engine for oncology too! You get a target list of five daily links (depending on your schedule) that uses multiple signals on how you interact with the links. The beauty of this system is that it comes through the emails, you refine the product at your pace and it gets more meaningful throughout your product usage.

Here’s what I suggested on the zoom meet up and became part of the product roadmap too!

I had a Zoom call a few days ago with someone who suggested making a Telegram bot that would post articles and send back the emoji reactions. I had never thought of that; it sounds awesome. It might be an alternative to the “niche articles” idea above—if you want to make a community around a certain topic, just make a channel and include the bot. We’d need to figure out how exactly to source articles. I’m thinking we’d let the channel admin/channel participants submit articles and RSS feeds and pull just from those. Perhaps we’d pull from Findka’s main pool of articles if there are some that are similar to the ones the channel participants submit.

The algorithm is interesting:

Findka uses an off-the-shelf collaborative filtering algorithm (k-NN), which compares link click data between users to find new essays you might like. I supplement this with content-based filtering, which involves analyzing the text of each essay to guess which ones are similar. I also use a technique I’ve dubbed “popularity smoothing,” which ensures that popular essays don’t get recommended too often. And to prevent filter bubbles, a portion of the articles you get are chosen more-or-less randomly (exploration vs. exploitation).

In addition, I personally review every submitted article, and I give extra weight to the ones I think are best. I’ll continue this for as long as I can keep up.

I can upload the OPML file of the filtered RSS feed from Pubmed. Each of the links goes to the users who sign up for the same. Currently, I have done it for head and neck cancers and brain tumours. I can refine the criteria to include specific sub-topics or the broad range of articles that include the biological studies with clinical applications or clinical studies/trials. Your daily interaction with the emails makes it easier for the entire community while sharpening the focus for you.

It solves the biggest problem- literature recommendation. It respects your time, attention and privacy.

You can sign up here.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.