web analytics

— urbantick

Tag "data handling"

The Institute for Science, Innovation and Society (InSIS) is organising a two-day conferenceVisualisation in the Age of Computerisation‘ on 25-26 March 2011 at Saïd Business School, University of Oxford.
“The theme of the conference is the permeation of science and research with computational seeing. How does computer mediated vision as a mode of engagement with information as well as with one another effect what we see (or think we see), and what we take ourselves to know?”

The event is structured along three main topics: Changing Notions of Cognition, Changing Notions of Objectivity and Changing ontologies of scientific vision.

Rain at musicfestivals
Image taken from onlinejournalism blog / for a viral-friendly piece of visualisation, it’s hard to beat this image of festival rainfall in the past 3 decades.

Speakers include: Peter Galison, Department of the History of Science, Harvard University, Michael Lynch, Department of Science and Technology Studies, Cornell University, Steve Woolgar, InSIS, Saïd Business School, University of Oxford and the summarising discussants are: Anne Beaulieu, Virtual Knowledge Studio, Paolo Quattrone, IE Business School and Fulbright New Century Scholar

I will be presenting a paper together with Tim Webmoore on ethics and visualisation of large scale dataset mined from the web, with a focus on twitter. We’ll be using the NCL mapping project for examples, to develop an illustrated argument for ethics in this field. However, the aim is to use ethics to support this kind of research, using ethics and a clear position as a framework. We believe that such structures are of additional value to the research and researchers and ensure in the long term academic research quality and standards.

Abstract: In this paper, we examine some of the implications of born-digital research environments by discussing the emergence of data mining and analysis of social media platforms. With the rise of individual online activity in chat rooms, social networking platforms and now micro-blogging services new repositories for social science research have become available in large quantities. The change in sample sizes, for instances, from 100 participants to 100,000 is a dramatic challenge in numerous ways, technically, politically, but also in terms of ethics and visualisation. Given the changes of scale that accompany such research, both in terms of data mining and communication of results, we term this type of research ‘massified research’. These challenges circle around how the scale of, and coordination work involved with, this digitally enabled research enacts different researcher-participant relationships. Consequently, much of the very innovative and creative research resulting from mining such open data sets operates on the boundaries of institutional guidelines for accountability. In this paper we argue that while the private and commercial processing of these new massive datasets is far from unproblematic, the use by academic practitioners poses particular challenges. These challenges are manifold by the augmentation of the capacity to distribute and access the results of such research, particularly in the form of web-based visualisations.
Specifically we are looking at the spatial and temporal implications of raw data and processed data. We consider the case study of using Twitter’s public API or application programming interface for research and visualisation. An important spatial consequence of such born-digital research is the embedding of geo-locative technology into many of these platforms. A temporal consequence has to do with the creation of ‘digital heritage’, or the archiving of online traces that would otherwise be erased. To unpack these implications we consider how a selection of tweets can be collected and turned into data sets amenable to content and spatial analysis. Finally, we step through how visualisation transforms such vast quantities of tabular data into a more comprehensible format through the presentation of several visualisations generated from Twitter’s API. These include what one of us has developed as ‘Tweetographies’ of urban landscapes, as well as examples of recent Twitter activity surrounding the disasters in Japan.
Such analysis raises issues of privacy and ethics in relation to academic ethical approval committees’ standards of informed consent and risk reduction to participants. Such massified research and its outputs operate in a grey area of undefined conduct with respect to these concerns. For instance, what are the shifting boundaries of public and private space when using Twitter and other platforms like it? Are Twitter and other social media platforms’ disclaimers as to privacy sufficient justification for academic and commercial use? Are the standards of social science research protocols applicable to research on and for ‘the masses’?
To conclude, we discuss propose some potential best practices or protocols to extend current procedures and guidelines for such massified research.

Mountains out of Molehills
Image taken from Nora Oberle’s blog / Another beautiful data visualisation. Even though in this case, the topic is not that hilarious- it’s about news coverage of scare stories. Remember tumours and cellphones or “killer wifi”?

Full conference programm to download HERE.

Read More

News media coverage on the Afganistan War, started in 2001 and the Iraq War started back in 2003 by the then President of america, George W. Bush has slowly but surely been pushed of the headlines. It was big back then and still is every now and then, but the general public has gotten used to it and doesn’t know where to go with additional information about similar events. In Afganistan the war is into its tenth year and in Iraq into the eighth.

Nevertheless people are still dying in this war, on all kinds of different sides. Long gone are the days where there was only two sides. Who is with who and who was with whom last time, or tomorrow? Nobody knows and probably nobody will ever know. The only hope is that progress can be made, even if slowly, steadily towards a stable but independent and local rooted sense of direction.

Wikileaks has in July and October last year released a massive amount of documents related to the war. These basically are reports on events and incidents, a War Diary essentially. Tse dat was picket up by the Guardian visualised via Google Fusion Tables. The Wiki Dum consist of some 391,000 records. The Guardian Data Journalists have been pondering over it for weeks and theyr summary is available on the Guardian Data Blog.

Iraq Death Map

Link taken from the Guardian Data Blog / The data on all reported deaths in Iraq is available on the Google Fusion Tables. Full details available directly from the Wikileaks page. For a full view of Iraq zoom out, it really covers the entire country!

It is important to look very closely at this data. The visualisation is extremely missleading and is is vastly unclear what it show. For example as jpsnodgrass in a comment on the Guardian Data Blog points out, there are a great number of unrelated incidents mapped in the same fashion implying they are contributing to the full picture. He notes “the same visual points represent vastly different categories of incidents involving injuries or deaths. For example, look north of Fallujah city where two red dots lie very close to each other. This is the information about one of the events:

Type: Friendly Action
Category: Attack
Region: MNF-W
Attack on: FRIEND
Enemies killed: 14

This means that Regimental Combat Team One (part of the 1st Marines) conducts Opposing Force (generic “The Enemy” for training purposes) rehearsal and feint. i.e. They were practicing the maneuver into Fallujah and pulled out.
The 14 “enemies” seem to be fictional entities hypothetically killed in a theatric rehearsal.

The dot next to this:
Type: Non-Combat Event
Category: Accident
Region: MNF-W
Attack on: NEUTRAL

Read: Tank accidentally rolls over (and kills) Coalition member in the vicinity of Fallujah.

Click on the other points on the map, especially the ones outside of the major cities. You will find that many of them are “criminal events” (i.e. civilian murder, or murderer unknown, or a body was found, cause of death unknown).”

Taking out the ‘Accidents’, the category ‘Others’ and turning the map from a dot map into a heat map changed the picture

Link taken from the Guardian Data Blog / Map is based on the Wikileaks information on the Iraq War dumped in October 2010. It is available on Google Fushion Tables. You can log in via your Google Account and play around with the data yourself.

The Wikileaks do not mix things up or are misleading in themselves, but it is just a very large pile of raw data. This includes everything, including information that doesn’t belong or is wrong, a dump really. Authoring a visualisation will still require a lot of attention and knowledge on the subject. Even though Google supplies this amazing Fusiontables tool, visualisation has not become easier. It has become more accessible.

By far these documents are the largest and most detailed public available documentation of any war do far. But is it really making a difference? Does this map convince you of the necessity of this operation? Are you now, after clicking a few of these incident reports believe in the strength of military forces and their ability to foster society culture and maybe democracy?

Guess the same critique applies to the Afganistan War Diary data released by Wikileaks around the same time. Here is also a Google Table available together with a crude mapping. The same category mixing is going on. It hoever includes many more, since it does not only focus on deaths.

Available on Google Fusion Tables or directly on Wikileaks, or also via the Guardian.

Afganistan War Diary Map
Data available on Google Fusion Tables via the Guardian Data Blog.

Link taken from the Guardian Data Blog / Map is based on the Wikileaks information on the Afganistan War dumped in July 2010. It is available on Google Fushion Tables.

War Reporting is not done from the desktop, not with Google and not with Wikileaks. As it is with ‘clean warfare’ and ‘smart bombs‘, putting the distance between operation and action, or directing devastation from the comfort of a remoteness location is not changing the reality on the ground. War kills and imprecise reporting does not help save lives either.

war reporter action figure
Image taken from dcrtv / THe war reporter as action figure.

Read More

The Feltron Annual Reports have become something like an institution. On one hand for the graphics they celebrate but on the other hand also for the content or topic each report focuses on.

In the current design climate of hyped data visualisation the Feltron Report is one of the leading publications. Even though the series started earlier in 2004, it became an icon with the 2007 report. It features for example in the great documentary ‘Journalism on the Age of Data‘.

Feltron 2006
Image taken from Feltron / Page 3. The 2010 Annual Report on personal activities.

How are these reports compiled? Feltron explains in the FAQ: “The first and most difficult step is to keep meticulous records of the year. On top of my own records, I rely on services like Last.fm, Netflix and Flickr to keep records of my music-listening, movie-renting and photo activities. For my offline activities, I make daily recordings in iCal which are later poured into spreadsheets to enable counting and comparisons within the data.”

The rising interest has lead to the creation of the logging platform Daytum. Here individuals can start collecting data about everyday live activities as a sort of log book. Further more the page offers the option to also visualise the data, of course very much in the style of the Feltron Annual Reports.

EMoore report
Image taken from asis&t / The personal logbook at EMoore.

Other Self Surveillance Services could be Moodstats or lifemetric. A more embedded approach takes grafitter. It monitors your twitter account and picks up on defines hashtags. This can then be graphed out. So you can tweet away your log book.

Image taken from farm4 / The settings and hashtag definition page at grafitter.

The lates Feltron report departs from the personal log book and maps out a third persons life line. In this case it is Feltron’s father. This relates very much to the Travel Pattern Over Generations report compiled by David Bradley. It is an interesting documentation, not so much in it self, but very much as a personal comparison. With the personal dimension of the data it is very easy to relate to it and the primary reading of it will be in relation to one’s own history.

Feltron 2010
Image taken from Feltron / Page 4. The 2010 Annual Report is an encapsulation of my father’s life, as communicated by the calendars, slides and other artifacts in my possession.

The Feltron Reports are available for purchase in the Feltron Shop.

Read More

This is sort of movie time at urbanTick. Before you press play here, get your popcorn ready and fill your bottle with whatever. The movie you are going to watch is a dramatic 53:57 long, but it is definitely worth it. The most comprehensive documentation at the moment about the visualisation trend we are currently already in the middle of. ‘Journalism in the Age of Data’ is produced by Geoff McGhee: “Journalists are coping with the rising information flood by borrowing data visualization techniques from computer scientists, researchers and artists. Some newsrooms are already beginning to retool their staffs and systems to prepare for a future in which data becomes a medium. But how do we communicate with data, how can traditional narratives be fused with sophisticated, interactive information displays?”

It covers everything, mainly graphics of course, but also technology, narratives, truth, journalism, documentation, colour, interaction and of course data. Data in many forms and shades. This ranging from free data to collected data, data gathering, data collection, data storage, data cleaning, data preparation, data, data, data, data…

Flight paths
Image by Aaron Koblin / Flight paths over the United States. The colours represent the plane model.

It is great how they get the producers of the visualisations to talk about their work, the movement and the critiques. This makes it a rather personal documentation. Of course you also get to see the best visualisations of the past two years. Of course some of them you have seen here on urbanTick before, including for example the ‘Movie Character Interaction Charts‘ by XKCD, or of course the US flight path maps by Aaron Koblin. But now, GO!

And a note here, you can watch a ‘fuller’ version directly at datajournalism.stanford.edu. It is shown together with additional material and comments as a sort of interactive version.

Read More

Mapping has become a big thing in our everyday lives. This goes hand in hand with stepped up efforts to collect but also release data. It has become a data war with big web players such as Google, but also traditional produces such as OS or the sensus data.
The bottleneck currently is more the traditional software tools to actually work with the data and produce meaningful output.
So it is great to get news about a new effort to open this up and make it more accessible and intuitive. Here we go INDIEMAPPER!
It is an online flash based mapping tool developed over the past two years by Axis Maps.

Image taken from Indiemapper / A map of Hurricane Gustav showing its path and wind speed, 25 August to 2 September 2008. The reference map data, including bathymetry, is from Natural Earth. The hurricane data is from Geocommons Finder.

It offers all you you can think of: Unlimited online storage, Secure data handling and storage, No software to install, Mac, PC or Linux ,Constant secure backups, Built-in collaboration tools, Premium customer support. A mapping tool to go really, work from were ever you have access, don’t worry about the data. All for $30 a month, you can get a 30 day free trial to test it. For academic use there is a reduced pricing.

Indiemapper also integrates with independent cartographic tools on the web like ColorBrewer, TypeBrewer and Natural Earth.

“Indiemapper is a Web-based app that loads geo-data, allows custom control over mapmaking, and exports static maps in vector and raster formats.
We’ve balanced indiemapper so you have the tools you need to make beautiful thematic maps without overwhelming you with hundreds of obscure GIS functions. Nothing is more than 2 clicks away. This keeps mapmaking simple, fast, and fun.” (Indimapper)
Find a detailed introduction on the Cartogrammer Blog or of course infos directly on the Indimapper page, this also features a blog with updates, also see the review on FlowingData.

Read More

A quick visualisation illustrating the location war between foursquare, brightkite, Gowalla, twitter, flickr, blockChalck and Bump. This is a weeks worth of data. Animation is created in processing.org The data was intially collected during the South by SouthWest Interactive Festival. The live datastream was available on http://austin.vicarious.ly It is a demonstration using SimpleGEO, the online geo database project.

Read More

I will be giving a talk today showing investigations on a city level using digital footprint data. I was invited to talk at the MSc for Adaptive Architecture and Computation at the Bartlett School of Architecture. ‘Digital Footprints / Tracing Bodies Through Narratives of the Everyday’ will be looking at temporal aspects of citizens trails, data collection and visualisation.
New material processed from our Twitter project will feature as an example.
The talk will focus on digital aspects of the data, but still I would like to draw the connections to the physical aspects, as well as very importantly to the people themselves.
After the talk the students will be presenting some of the work they are developing at the moment and have a discussion.

Imge by UrbanTick / Story board One.

Read More

While living in the era of knowledge the visualisation of content has become ever so important. At least this is what current trends suggest. At the same time incredible and powerful tools are available to do so and synthesis new knowledge as a result. The spiral is turning fast especially in the field of digital or web based knowledge. However there are a few people out there that produces very high quality syntheses with intriguing visualisations. One of my favorit is BLPRNT.
Only recently BLPRNT has put online the visual comparison between two speeches by President Obama on the same topic. One speech was given in July 2009 in Cairo and the second one in Tokyo, during Obama’s far east trip in November 2009. It is all produced using processing 1.0 an open source tool. The project featured in an article on cluster.
It works on the basis of word comparison. The word in the centre is shared by both texts, the size of each word shows how often it is used and text snippets show the context of words or word groups.

Image by BLPRNT taken from cluster, more can be found on BLPRNT’s flickr page.

BLPRNT has earlier developed the tool to compare two texts on a different subject. For this project a clip demonstrate how the software works.

Two Sides of the Same Story: Laskas & Gladwell on CTE & the NFL from blprnt on Vimeo.

Read More

Image by UrbanTick using Autodesk Map 3D

Following an earlier post about the UrbanDiary data in Autodesk Map 3D, I would like to talk about further development on this topic.
The plan was to generate the UrbanDiary maps using this software and with this move towards a more automated workflow from GPS data to map with maintaining the level of possible graphical intervention.
As described in the earlier post data from the database could be connected and be represented in the software including context data. Simple manipulations and representations of the data could be made easily in Map 3D and actually I grew a little bit fond of the software although I did not like AutoCAD when I last had to use it. So I was determent to stick to this and work it all through in this one application.
Unfortunately, too soon I seemed to exceed the capacity of the software.

Image by UrbanTick using Autodesk Map 3D

Following the processing of the UrbanDiary interviews the task is to develop a combination of interview/mental map data and the GPS/map data. The idea is to look at the work related spatial movement with a special focus on the mental map features.
It turned out that the number of recorded GPS points per participant combined with the building, street and land use information is too much for Map 3D to handle. It started to crash continuously; up to the state I was not possible to open a file.
The method I used was maybe not the most economic one but seem simple to me. From the GPS points I defined a buffer to establish a zone of “experience”, which I intersected with the base map to only be working with relevant information.

The issue in Map3D led to the move across to ArcGIS, which appeared to be comfortable with the data. It also turned out that the same steps of work are quite simple achievable, although a little less intuitive. This reaches down to the export for Illustrator. Map 3D translate much simpler into Illustrator with its native dxf format. In ArcGIS, I could not manage to produce a workable file that remained distinct in terms of features. So I had to rely on the map export using about 12000 dpi. Proper maps will follow as they are processed in Illustrator.

Image by UrbanTick using ArcGIS and exported to Illustrator

Read More

Visualisation is part of our daily lives and we re constantly reading, interpreting or producing this kind of communication. In today’s book visualisations are described, as “Visual metaphors are a powerful aid to human thinking.” It goes o with referring to the modern time and the newly found complexity with “As our experience of the world has become more complex and nuanced, the demands to our thinking aids have increased proportionally.”
Anyway, the book is about visualiing information and called Data Flow – Visualizing Information in Graphic Design. It is not new, new, but recent, published by Gestalten in late 2008. It brings together a great collection of recent graphic visualisations of information and data. Of course the book it self is highly designed and a real joy to look at.

Image from Data Flow – Cover

The book is structured into six topics, namely Datasphere, Datanet, Datascape, Dataoid, Datalogy, and Datablocks. This is a purely visual characterisation of the final products, but formally helpful and of course sexy. Actually sexy is pretty much everything in this book.
As these titles already tell you the represent circle, net, surfaces, blocks. Those are the simple ones, the hard ones are Dataoid and Datalogy, here some references are needed to explain what the editor means with the title. Datanoid is deriving from humanoid, meaning “having human characteristics or form”. It describes visualisations humans can easily relate to through different ways, by integrating people, photographs and actions. The Datalogy seems to derive from analogy and refers to “similar to”, combining comparison and experience.
To make things simper a complex structure is needed. The book certainly achieves this. They seem fairly formal groupings at first, but have some thinking behind.

Images from Data Flow – Chapter introduction

The introduction to each chapter stars with a quite poetic description of each topic. This makes reading the book fun but is little helpful if one is interested in details. It is followed by a summary of the chapter again in a fairly superficial manner, but comparing or introducing a few key examples. This gives a good impression of what follows. Each graphic is then described with a short text block of around 60 words. This is very brief an each creator could probably fill pages with contextual information, but the visualisation is designed to speak for it self so it might be a good compromise. This kind of defines the character of the book; it is more of a compendium than a reader, containing a collection and not a description.
Between all this there are a few interview with designers of some of the presented visualisations, they are, Lust, Jessica Hagy, Cybu Richli and Catalogtree. Some of their work also features over a number of pages, whereas normally through out the book, one page is one visual, with 256 pages this might be about 180 different visuals in full colour obviously.
The interviews are rather short, something around eight questions. The style of the interview is a rather school like question and answer game. A bit more of a flexible chat would probably make the discussion more interesting. In the end one gets the feeling that the questions generally have been rather implicit, which leaves little room for surprising answers. Anyway, I you have the patience to read through them there is interesting insight on who the designers approach projects and what they think about the topic of visualisation.

There are a number of diagrams we have seen published elsewhere before. One of Christian Nolde’s Emotion Maps, the San Francisco Emotion Map is published here. For his book Emotional Cartography see earlier post here. Funny enough this is in the chapter Datascape and not in Dataoid. Other projects are the cap spotting project that features with a graphic (see blog entry), or the “manual” visualisation of mobile phone activity by Nicolas Fischer, maybe something the MIT should be thinking about (upcoming blog post) or the plotting of the 90 minutes movement of footballer, taken from the Game England vs. Poland in the 2006 World Cup. Who won 2:1?

Images from Data Flow – Sample pages

Some of the other stuff, mainly the Dataspheres recently featured in the Computer Arts 2009 March edition.
To conclude on this review, the book is great and very sexy, as mentioned above. It is one of the sort of books that give you real inspiration and immediately makes you wana pimp all that recent stuff you have produced. And once more you find yourself saying, I knew it for long it is possible to actually produce great visuals! And for a very short moment you forget about all the crap and ugly stuff your are surrounded by, nice!

Image from Data Flow – Sample page, just because it is so nice.

The book:
R. Klanten, N. Bourquin, S. Ehmann, F. van Heerden, T. Tissot, 2008. Data Flow. Berlin: Gestalten

Some links to designer featuring in the book:

Read More