Project and Product Search Engine


There are now more open-source projects on the Internet than you can shake a stick at! My go-to search engine for them is still Google, but that is no longer a good option. So I think there should be a dedicated search engine for open-source software projects and similarly for software products, such as SaaS offerings and closed-source libraries.

There are two main challenges for this to work as a startup project. First, getting the data. A lot of projects are hosted on GitHub, and I’m not sure if they would allow a newcomer search to crawl their entire site. Projects not hosted on GitHub would have to be found similarly. Software products are usually hosted on their own top-level domain, so all domains would have to be crawled, making sure to distinguish between software products and other forms of content. This would require a clever algorithm of some sort.

The second challenge consists of making these projects and products easy to compare and discover. Simply finding them on a keyword basis is already on offer from incumbent searches. To go beyond this in terms of user experience, an algorithm would have to know which projects are associated with or are complementary to each other, to offer these up alongside a project in the search results. Discoverability is an important feature for websites these days, so a good feature would be to recommend projects to users, based on a few search queries in a session.

The commercial value of this idea as a startup would be tremendous. Selling software products to interested users based on their search queries has considerable potential. And there are several tens of millions of developers out there looking for such products!