Archive

Open Knowledge Maps

I have just applied for a Shuttleworth Fellowship.

The fellowships are issued by the Shuttleworth Foundation, which describes its vision as “We would like to live in an open knowledge society with limitless possibilities for all.” I very much share this vision – it is the main reason I founded Open Knowledge Maps. Our goal is to build a visual interface to the world’s scientific knowledge in order to dramatically increase the visibility of research findings for science and society alike.

We want to provide a solution to a challenge that’s almost paradoxical: on the one hand, more research is openly available than ever, and we see considerable interest in science and technology. On the other hand, we are faced with a serious crisis of trust in scientific research, with anti-vaccination movements and climate change deniers on the rise.

I believe that the root of this problem is that it is very hard to get an “in” on research. Access does not equal discoverability or even participation. People outside academia trying to understand a research field are therefore often lost. I want to empower these people by providing better gateways into scientific research. Think: policy makers attempting to optimize decision-making by using evidence from relevant research, educators striving to convey the state-of-the-art, fact checkers trying to verify statements, or patients who would like to learn about the newest findings on their illness.

To make this happen, I believe that we need to do two things: first, improving the discoverability of research findings. Second, turning discovery into a collaborative process – thus enabling participation, and allowing people to create pathways through science for each other. Take a rare disease as an example: wouldn’t it be great, if researchers, doctors and patients would collaboratively map the newest research on this disease – and then share the results of their efforts for the benefit of patients, who don’t have access to specialists?

Enter Open Knowledge Maps: we use knowledge maps, a powerful tool for exploration and discovery. Knowledge maps provide an instant overview of a field by showing the main areas of the field at a glance, and papers related to each area (see below). In addition, knowledge maps make it possible to easily identify useful, pertinent information by separating papers into meaningful clusters – and they are exposing important concepts in the field that you would often need weeks to find out.

Examplary knowledge map for research on heart diseases

During the fellowship year, I want to explore how we can create a space for participatory discovery around these maps. How can different communities interact on a level-playing field, so that they create pathways through science for each other?

A little backstory

If you are an avid reader of this blog, you may recall that I alreay applied for a Shuttleworth Fellowship a year ago. The fellowships first caught my eye, when I learned that amazing projects like ContentMine (Peter Murray-Rust), Hypothes.is (Dan Whaley), and Koruza (Luka Mustafa) had all been enabled by a Shuttleworth Fellowship.

Back in May 2016, Open Knowledge Maps was just starting out, with an enthusiastic group of volunteers, and a prototypical service that enabled users to create a knowledge map for a topic based on the PLOS library (160,000 articles).

Since then, a lot has happened in the project.

The team has grown: I am developing Open Knowledge Maps together with nine amazing volunteers. I have also found great advisors and strong partners from the open knowledge community. We put out two major updates of our search service – pushing our coverage to 100 million scientific articles from all disciplines thanks to BASE. We’ve considerably improved the user experience based on feedback from the community, and we’ve enabled features such as  collaborative annotation thanks to Hypothes.is. And we held numerous workshops and sessions at events such as OpenCon, MozFest and re:publica, with more than 300 people in attendance.

Knowledge map for digital education. Click on the map to get to the interactive version on Open Knowledge Maps.

I am very happy that our efforts resonated with the community. Open Knowledge Maps was featured on the frontpages of reddit and HackerNews. Our user base has quickly grown: in less than a year, we saw over 100,000 vists and more than 30,000 maps have been created to date. We’ve received hundreds of enthusiastic tweets, e-mails and blog posts, motivating us to proceed with our vision.

We’ve now reached the limits of what we can do as a pure volunteer project. In order to realize the full potential of the idea, we need support. This is why I decided it’s time to give it another go. I also believe that it is a critical time that we are creating this platform in. There are several closed solutions for providing visual overviews that are being developed right now. If we do not provide an open alternative in time, we risk being stuck with proprietary solutions and wasted public money for decades.

As usual, the proposal was developed in the open. Special thanks go to Maxi Schramm, Christopher Kittel, Florian Heigl, Rufus Pollock, Antica Culina and Daniel Mietchen for comments on the draft. But I’d like to thank the Open Knowledge Maps family – team, advisors, partners and users. I am very lucky to shape this vision together with you.

You can find the full proposal on Github.

baseintegration

We have now connected Open Knowledge Maps to one of the largest academic search engines in the world: BASE. This means, you are able to visualize a research topic from 100+ million documents. And for the first time, you can search within different types of resources, including datasets and software. I would like to thank our collaborators BASE and rOpenSci for their outstanding support in making this happen!

We have also spent a lot of time improving the naming of the sub-areas to make the concepts in a field more visible – which means that this update improves our existing PubMed integration too.

In addition, we have added much more information to the site about the project and our approach. Open Knowledge Maps follows the motto “open science, all the way”. From our roadmap to our source code and our data, we publish everything under an open license that is compatible to the Open Definition.

Try it out now and let me know of any feedback you may have!

Create a visualization based on 28 million articles

Today, I am very proud to announce a milestone for Open Knowledge Maps. Thanks to an outstanding team and continued support by our partners and advisors, we have added two major content sources: the Directory of Open Access Journals (DOAJ) with more than 2.3 million articles and PubMed with more than 26.5 million articles. Taking into account a certain overlap between the two sources, we can now credibly state that one can create maps based on 28 million papers. That’s a content pool that is 175 times larger than in the previous iteration using PLOS (about 160,000 articles).

We have also completely overhauled our design & overall presentation and improved the user experience considerably. In addition, we have included the open annotation software Hypothes.is in our PDF preview.

We believe that this is a major step towards revolutionizing the way we discover research. There are many new things to try out and explore – we are looking forward to your feedback and suggestions!

Try it out now!

okmaps_start

We officially announced the launch of Open Knowledge Maps, the visual interface to the world’s scientific knowledge, with a tweet and a message to our advisors late on May 11. We had soft-launched the site more than a week before that, and a bare-bones version of the PLOS visualization service had been online since Mozfest. The website was already getting some attention, and people were using the service on a daily basis. One of the highlights was a feature on Storybench in the very early days of the project. The idea behind the announcement was to get broader feedback on the search service and the overall idea behind Open Knowledge Maps. We had come a long way since the Mozfest days, and we thought that the website was ready for a wider audience.

What was to follow though went far beyond my highest expectations. Over the next 48 hours, we saw more than 350,000 hits on openknowledgemaps.org, generated by 12,000 visitors from all over the world.

What had happened? On the morning of the next day (May 12), I noticed that the tweet had gained a lot of traction, which had translated to acitivity on the site. Lots of people were using the search service, and a new map was created every few seconds. Much to our delight, the feedback was overwhelmingly positive. We started filing all the reactions as many of them contained useful pointers for future improvements. At this point, our server was still humming along fine. Granted, you had to wait a few extra seconds on the search here and there, but nothing out of the ordinary.

During the day, news about Open Knowledge Maps spread to other channels, and at some point around noon CEST, we hit the front page of Hacker News. I immediately noticed a spike in all our parameters. We went from a map every few seconds to multiple maps being created each second. Search time began to rise and we started receive complaints about failed or endless searches. Around 3:30 pm, our server finally gave in. Hundreds of searches were running at the same time, each of them taking minutes to be processed. It was time to take the search service offline and to post our version of the “Fail Whale”. You can still find a version of this screen here.

okmaps_fail

While we frantically rewrote the search service to handle a larger amount of requests (it was back a mere 60 minutes later), the stream of positive feedback continued to roll in. Up until today, Open Knowledge Maps was mentioned in over 100 tweets, with the announcement tweet creating more than 22,000 impressions alone. You can find a collection of my favourite tweets in this collection. But it was not just Twitter – the news was shared on Facebook, blogs, and in discussion forums.

At one point, we were called “the Wikipedia of scientific knowledge”. It is clear that we still have to go a long way to really deserve this tagline, but it is encouraging that people see the potential of the idea. Needless to say, the positive feedback also sparked the ambition of the Open Knowledge Maps team of volunteers. We are currently busy improving the site and the service; the first results will be available in just a few weeks.

It was a fascinating day in the eye of the storm. I’d like to thank my awesome team for their outstanding work and our great advisors for their help in shaping Open Knowledge Maps. And I’d like to thank all of you out there for the love that you have shown for this project. It means a lot to me – Open Knowledge Maps is a project that is very close to my heart. Please continue providing feedback via social media, e-mail, or on our Github repositories. You can also sign up to the newsletter to stay on top of everything #OKMaps.

It is time to change the way we discover research, and we are off to a good start!

%d bloggers like this: