Visualizing the evolution of a scientific conference with altmetrics

From September 3 to 5, I will be attending STI 2014, the 19th International Conference on Science and Technology Indicators. There, I will present a paper entitled “Altmetrics-based Visualizations Depicting the Evolution of a Knowledge Domain” that I co-authored with Philipp Weißensteiner and Peter Brusilovsky (download the PDF here). In this work-in-progress paper, we present an approach to visualizing the topical evolution of a scientific conference over time.

Below you can see the results: a topical overview of the 19th and 20th iteration of UMAP, representing the conference years of 2011 and 2012, as well as the evolution of the domain.An interactive prototype can be found on http://stellar.know-center.tugraz.at/umap/.

Data Source and Method

co-bookmarking

Relationship between two documents established by co-bookmarking

The data source for these visualization is quite unique: it is the conference scheduling system Conference Navigator 3 (CN3). CN3 allows conference attendees to create a personal schedule by bookmarking talks from the program that they intend to follow. And it is exactly this scheduling data that we have employed to create the above visualizations: we used co-bookmarking as a measure of subject similarity, meaning that two documents are related when they are bookmarked by the same user in the system (see example to the right). The more often two documents are bookmarked together, the more similar they are subject-wise.

On top of this co-bookmarking data, we performed the knowledge domain visualization process from the open source visualization Head Start to create individual representations of the field (please refer to the paper for details). This resulted in the first two visualizations pictured earlier on. The blue bubbles represent research areas. The size of an area is determined by number of bookmarks that the papers related to this area have received. Spatial closeness implies topical similarities. In 2011, “User modeling” is the area with most papers and most bookmarks. It is closely connected to several other larger areas, including “Recommender system”. A second cluster of areas can be found on the right hand side of the visualization, involving “Intelligent tutoring system”, “Adaptive system”, and “Problem solving”.

Overview of UMAP 2011

Overview of UMAP 2011

Timeline Visualization

The next question was how to visualize the evolution of the conference. As far as time series visualization goes, there are many types of visualizations, most prominently index charts and stacked graphs. In the case of knowledge domain visualizations, simple visualizations are unfortunately not able to convey all necessary dimensions of the data (in terms of ordination, size of research areas and closeness). One possibility would have been to use animation, as shown in the video below with Hans Rosling.



In the end, we did not choose to use animation. Why? The reason for that is a psychological phenomenon called change blindness (Simons and Rensink, 2005). It means that people are bad at recognizing change in an object or scence. In the next video, the phenomenon is explained and illustrated with an astonishing example.



Animation seems to be especially prone to change blindness; in the video below by Suchow and Alvarez (2011), the colored dots making up the ring are constantly changing. This changing of color seems to stop when the circle itself starts to move – except that it does not. If you concentrate on individual dots, you can see that they keep changing color.

Surely, this is an extreme example, but think about it: if Hans Rosling were not to be talking you through the video above, would you have recognized all the changes taking place and would you have been able to interpret them correctly? If you concentrate on one country specifically, could you remember the movement of the other countries as well? Chances are, you would have to watch the animation many times to come up with the same interpretation as Prof. Rosling.

All of these considerations led us to choose a different visualization concept popularized by Edward Tufte: small multiples. In small multiples, a graph is drawn for each of the steps in a time series. Then the graphs are positioned next to each other. This approach thus allows for direct visual comparison between different representations.

Evolution of UMAP

Evolution of UMAP

To aid the user in detecting changes between the representations, we introduced two visual helpers. First, a grid is drawn to help with comparing size and position of the research areas. Second, whenever users hover over an area, the corresponding area is highlighted in the other representation, and a line is drawn between the two entities. There are three areas that are present in both years: “User modelling”, “Recommender system” and “Intelligent tutoring system”. While the relative position of the areas to each other has not changed much, the area with the most papers and bookmarks is now “Recommender system”.

Future Work

As you can see from the examples above, this is just a first prototype, albeit a promising one. Using small multiples allows for a comparison of knowledge domain visualizations over various years.

Nevertheless, there are certain weaknesses in the current approach: first, the topology of the visualizations is not ideal, as many areas may overlap each other. Second, the usefulness of the method depends on the usage of the system by conference participants. Therefore, we are looking into supplementing bookmarking data with content-based measures when there was lower participation. Third, the continuity between the two years is very low. This could be improved by introducing moving time windows of two years. Finally, it will be important to evaluate the method and the interface.

Any comments on the issues mentioned above and the paper in general are of course very welcome!

ResearchBlogging.orgKraker, P., Weißensteiner, P., & Brusilovsky, P. (2014). Altmetrics-based Visualizations Depicting the Evolution of a Knowledge Domain 19th International Conference on Science and Technology Indicators (STI 2014), 330-333

Advertisements
2 comments

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: