Sketching User Experiences part I — notes

I recently reread part I of Sketching User Experiences by Bill Buxton for UX Book Club Zuid-Holland and made notes during the proces. ‘Sketching’ is one of my favorite UX  books and well worth a reread which I found reveals different layers and gets you to reflect differently with the experience you have accumulated since the last time I read it.

I thought it may be worthwhile to share those notes here, so here goes.

Coming change to more digital behaviour embedded in the fabric of everyday life is going to force us to focus on context.

How do you design for context?

Does Buxton deliver on the promise he makes at the start of the book? He tells a nice story but where’s the sketching for software?

He talks about designing agents systems and complex behaviours. Doesn’t emergence play a large part? Don’t we need foundational guidelines more than anything?

p. 13 This is a start. It is a rough sketch.

Many participants of the bookclub thought the book fell short and was overly meandering. No it is not a howto guide to designing user experiences.
By Buxton’s own admission it is a sketch an initial concept for how a book like this should look. But still I don’t know any other book which provides such a broad view on the field of UX and such an in depth treatment of one of its foundational processes (sketching).

Physical devices can recast a problem in a new light.

p. 37 We must make our best efforts to understand the larger social and physical context within which it is intended to function.

p. 37 We ideally need to be able to experience our designs in the wild during the early stages of the process.

p. 38 Without informed design, technology is more likely to be bad than good.

p. 47 Why shouldn’t executives want to have their company create breakthrough products that generate great returns?

Great realistic analysis of the Apple design process for executives. In that light the piece “You can’t innovate like Apple” is also worth a read.

p. 53 Everyone is essential but no person or group is sufficient on his or her own.

What’s the relevance of (software) product design with its version iterations to website projects which are unfortunately mostly one off?

p. 71 My underlying approach in what follows will be to put forward a holistic approach to experience-based design. Along the way, I will show how the weaknesses of software product development can be complemented by the strengths of traditional product design, and likewise, how the weaknesses of traditional product design can be complemented by the very real strengths of software developers. But my strongest argument is for the need for an explicit and distinct design process, integrated into the larger organization, supported by appropriate executive leadership.

p. 78 Get the right design. Get the design right.

p. 80 It takes very strong and brave management to admit that we don’t know what we are doing at the start, and therefore need to accomodate that in our process.

My addition: The act of sketching constrains your freedom. Every stroke you make in a sketch finalizes something. That is the whole point and that what makes the process converge. Then if the result doesn’t please you start anew with a blank sheet.

p. 105 [a sketch is] a graphic means of technical exploration

p. 111-2 Sketches are:

  • Quick
  • Timely
  • Inexpensive
  • Disposable
  • Plentiful
  • Clear vocabulary
  • Distinct gesture
  • Minimal detail
  • Appropriate degree of refinement
  • Suggest and explore rather than confirm
  • Ambiguity

p. 117 By examining the externalizations, designers can spot problems they may not have anticipated.

Ambiguity but also the resolution of complexity.

p. 135 “Sketching Interaction”

How do you sketch tone of voice? For instance by employing product personas that act out the interactions your product has with users.

How do you sketch look and feel? By creating broad mockups supported by mood boards?

p. 139 “Sketches are not prototypes”

Prototypes may be less disposable but they can also be very agile and reusable. But admittedly prototyping would take place post-sketching.

p. 143 Arguing for the need for user involvement in a modern book on product design is as pointless as a discussion about the need to know the rules of arithmetic in an advanced mathematics textbook.

p. 147 A healthy team is made up of people who have the attitude that it is better to learn something new than to be right.

p. 151 “You make that sound like a negative thing”

Design rationale and strong criticism are essential to move forward but hard to find.

p. 154 “If someone made a sketch in the forest and nobody saw it”

A communal corkboard provides:

  • Shared awareness
  • Baking in
  • Collaboration
  • Communication
  • Juxtaposition
  • Critique

Enjoyment of Public Space

Jan Chipchase had asked this question recently in a post on this street scene which already had me thinking.

Enjoy Public Space

Yesterday I was present at a lecture by Dirk Overduin on some succesful interventions around the use of public space.

It’s hard to strike a balance between the use and abuse/overuse of public space. Mostly perceived ‘abuse’ of public space stems from other social problems (homelessness etc.) and regulating public space too rigidly impoverishes all of us.

Looking at the street scene from China, I’m disappointed that we do not have similar uses of public space here. To be able to play tavla on the sidewalk in the city while drinking çay…

Gecekondu on the island at ARCAM is a fun initiative and they have a full program for the next couple of days so be sure to visit them.

Social Drinking in Print

Things are a bit slow in Amsterdam, this being high summer. Having Foursquare (see previous post) and being able to drink some beers in the city now and then does make it a lot more relaxed.

Robert Gaal and I initiated the launch in Amsterdam and it looks like it is really taking off right now. Dutch daily wrote a two page spread on the site and quoted me and some others in it. Normally I don’t bother with papers, but I went to our neighborhood store and bought a copy:

Ben in Paradiso, kom ook man

It’s a nice balanced overview piece of the service and the trends towards more ubicomp in your nightlife experience. The big question is, these guys have delivered, will anybody else match them in the foreseeable future?

Local social network Hyves is bluffing they will introduce similar functionality but it begs the question why they didn’t already? I’ve seen this concept come by in briefings more times than I care to count and nobody has been able to get the buy-in and pull off the experience.
And when was the last update of the Hyves iPhone App? Or of the Facebook App for that matter?

In other news: There was some drama recently with people from outside of Amsterdam adding venues outside of Amsterdam and getting rebuked by some people within AMS.

Foursquare’s policy is that anybody who wants to play in whatever fashion should be allowed to do so. The data generated can always be filtered better later on using better algorithms and more insight. Just think twice about friending me if you plan on checking in outside of Amsterdam.

Urban Teleportation Device

I can’t say enough how pleased I am to be riding this bike through the city.

Urban Teleportation Device

I’ve given her a name but it functions mostly as a teleport to a random location in the city where t < 15m. Already being familiar with the topography of Amsterdam is a plus and tearing through the city, riding faster than most people without any effort is fantastic.

Now comes the more difficult task of keeping this bike in this city.

Four Square or can you smell the beer already?

This week returning from my new haunt in Amsterdam back to Delft, walking the streets with an appetite for beer I felt strangely disconnected.

Together with Robert we persuaded the kind people of Four Square to open up a branch in Amsterdam. Being one of the starters of the service here means I’m friends with tons of people. Knowing what is happening in the city, where the cool bars and restaurants are and immediately knowing who’s having a beer right now and where, is fantastic.

S&W say that being able to see through a city is a superpower. If so, Four Square is the superpower of being able to smell beer miles away.

Once you’re used to a service such as this and the peripheral vision it provides —and does ever so unintrusively— walking around a city which doesn’t have it feels like having blinders on.

Augmented reality in HTML5

One of the last bastions of Flash and native apps is the processing of video from outside sources such as webcams. It does not at all seem difficult to add this functionality to HTML5.

I don’t have much of any experience in designing these kind of specs (though I did request <audio> and <video> elements some two and a half years ago), but here are some design notes which seem to make some sense:

Get outside video

Create a specific src parameter for video, for instance src=”webcam” to get image data from a current installed webcam. The user-agent can mediate the presence of cameras and the routing of sources. This gives the user-agent a way to get device video into the web application.

Besides augmented reality this could be used for most webcam related applications on websites but for that some more facilities for retrieving and transmitting the video stream will be necessary.

Get at the frames

Now to get at the raw video data the addition to HTMLVideoElement of a method (like the canvas already has) would seem to fit:
ImageData getImageData();
that returns an ImageData object for the current frame of video. This would either work for the current frame when the video is paused or the current frame unpredictably when the video is playing (for applications to retrieve frames of video as fast as they can process them).
Alternatively register a callback function to the <video> element where every video frame is pushed to.

Update: Mark points out below this is already possible using the drawImage function of the canvas rendering context which accepts a video element and draws the current playback position.

If you can reliably extract all video frames and store them locally, you may even be able to build a non-linear video editing application.

Process and redraw

Processing the frames to create an augmented reality is left as an exercise for the reader.

Ideally each frame of video could also be rendered into a canvas where the client could draw other primitives on top of the video frames. This seems to be necessary for the augmented part of the augmented reality.

Update: This already seems to be possible by putting the image back into a canvas, but I don’t think that would sync up the audio properly.

So all that is needed is the addition of an extra source and an extra method to the <video> element. Doesn’t seem like that much, does it?


I had my Yashica D repaired and also recently acquired a lightmeter to make the process of shooting easier and now I’m slowly but surely shooting stuff with it.


Mony is helping me out scanner-wise and I must say I’m amazed at the quality, the vibrance and sharpness this ‘old piece of junk’ shoots and this archaic process returns.

Double Chocolate Muffin

And added to that I’m completely untrained and have hardly any experience with this stuff, so who knows how much improvement is still possible. Here’s to film in a digital age.


Last week I got Rebooted in Copenhagen again. Drove up with Peter and Iskander.


After your fourth time attending such a great and small conference as Reboot, it becomes familiar like a reunion. This year’s action theme meaned the two days were completely packed. There was so much to do and so much going on that the first day was over already before you knew it.

The one thing that struck me as being odd was the fact that many speakers on the large stage weren’t asked any questions (I had one or two for Richard Falkvinge). If anything, Reboot does not strike me as an event where one person can rain down the gospel unquestioned. So for a next time more interactive maybe even adversarial formats may be nice.


It was a great event and a lot of fun cruising around like a local on the Yahoo! bike and crashing at Mark’s place with Anne (his writeup). The opening talk by Matt was great (as always) and the Bruce Sterling’s closing talk was also fantastic. Ton has a great writeup.

But yes it was very awesome. I don’t know many tech conferences that can pull off a street party like this one:

Update: And the talk by Bruce Sterling just got posted:

Friendlier and more open government data

This is an English rewording of a post in Dutch earlier this week just in time for this Reboot session.

I’ve recently been involved with several government initiatives to make government data more accessible and to show what’s possible if that data is publicly available. The premise is that if government data is open, developers and other third parties kan reuse that data and make useful stuff with it. Stuff that can for instance serve as inspiration for our Dutch show us a better way contest: Dat Zou Handig Zijn.

Most recently we had a Hack the Government devcamp/unconference where people interested in this stuff could exchange ideas and build stuff.

Hack de Overheid
photograph by Anne Helmond

A while back I did a project on widgets which was mostly a supply side initiative triggered by a listing of readily available data.

Simultaneously Ton and James did a project on process recommendations for the public sector when it comes to releasing their data. As a part of that project I helped them sift through the currently available data sources from a technical point of view and to see which of those sources could be repackaged in an interesting and more user and developer friendly way.

That wasn’t very easy. Dutch government does publish a ton of data online but usually in rather unaccessible formats and interfaces and without clear descriptions on what it is. We picked two data sources and we managed to realize two new websites based on that data: Schoolvinder (‘School finder’) and Vervuilingsalarm (‘Pollution alert’).

School finder

A Dutch institution called the CFI already has a store with a lot of data on schools. We used their search interface and output (which though ugly and slow was remarkably reusable) and rebranded that into a simplest possible school search engine which is easily understood by parents looking for schools in their area.

Besides that we also create a canonical URL for every school in the Netherlands so other parties can refer to that and we can build stuff on top of those school pages.

The first problem this fixes is that the website is poorly usable and worded almost exclusively in jargon. Employees from the ministry of education told us later that the CFI data is not meant to be used by the public but we think this is still a fix.

Secondly it fixes the fact that this information is quite hard to find and to refer to. Our search engine is easy and open and school data is republished at unique URLs using microformats.

We would have liked to link our school pages to some numerical data from another database of the CFI but that was too hard to realize within the alotted time. Even linking to that other site proved to be too hard because those webpages were shielded in a needlessly complex ASP.NET environment.

Pollution alert

Pollution alert takes the predicted particulate values for a number of sensor stations and makes those accessible. I made a scraper to take the predicted data from the RIVM site and store it in Google App Engine. From our own store in Google App Engine, we show the geocoded stations, we push alerts out to Pachube and Twitter, graph the data and provide an API. We believe there is a lot of latent interest in the general public for this kind of data but that usable presentation forms have not been forthcoming.

RIVM to their credit does publish most of their data online, but definitely not in the most accessible formats nor in ways that enable normal people to audit their living environment.


The sites we built are very advanced prototypes which are completely functional and even highly scalable. During building we followed some principles which may be interesting to touch upon here.

Friendly design

Both sites have a pleasant and friendly design created by Buro Pony. This is important because people are more inclined to use sites that look nice and experience those sites as being more user friendly.

A nice design needs to be accompanied by clear and simple writing without jargon that connects with the way people think about the stuff you’re describing.

Most websites can be improved massively by just implementing these two points.

Playing well with others

Both websites also connect with a bunch of other sites that improve upon the concept. They’ve been built on Google App Engine a system which is easy to develop for and which is readily scalable. Maps are retrieved from Google Maps, graphs are provided by Google Charts and sensor data is pushed to Pachube and Twitter.

The experience on the main site is just a part of the whole. The data needs to be accessible from and easily pushable to where it’s needed most.

On the Hack the Government event somebody said something along these lines: ‘systems integration is difficult and complicated and if you’re good at it, you can make a lot of money with it’. This is a well known Enterprise IT mantra but if there’s one thing that the abundances of mashups proves, it’s that integrating systems on the open web is everything but complex.

On the open web we have usable and developer friendly API standards with tooling, besides that we have proper standards for identity and authentication.

If you don’t dig yourself into a hole, it really doesn’t have to be that difficult. And none of this is exactly new, this technology has been around for ages and it just builds on the strengths of the internet.

Standards based

Both sites are completely standards based. As an experiment I wrote both in a conservative form of HTML5 (and validated that) partly out of curiosity to see how it would turn out and partly because I think that our current Dutch industry standard XHTML is a dead end.

Added to that I have sprinkled in some microformats in places where it was obvious to do so (e.g. school addresses). The notion that it takes significant extra time to add microformats to a project is absurd and these days the advantages of adding them keep piling up.

Yes, it takes some effort to learn to use standards and microformats properly, but once learned I think it actually takes more effort not to use them.


Finally, both sites have been built in a couple of days over the course of about a week and a half. We wanted to show that when we’re talking about a quick project, it really can be quick and that building a non-trivial usable beautiful website does not need to cost a lot of time or money.

All of this can be improved. Let’s get at it.

I come not to save Flash

Adobe’s CEO said that Flash is safe from HTLM5. Just the fact that Adobe made a board level comment on this is telling enough, the answer itself is somewhat lacking.

It’s nice that they’re optimistic for improvements in HTML. Seeing what disasters both the XHTML and in a somewhat lesser extent SVG tracks of the W3C standardization process have been, we could definitely use them. They should admit though, that a full fledged HTML5 with rich CSS3+ is the death knell for Flash.

The more sites we see like the Holland Festival one the more it means that clients are choosing for an open and more semantic website with gradual improvements in JavaScript and CSS. It may take ten years, but we may see the first signs of a shift in this industry.

No mobile?

He’s right to say that it’s difficult to deliver a consistent HTML5 experience across browsers on the desktop. The fact that he conveniently ignores mobile browsers is more than a bit misleading.

Mobile internet usage is exploding and most of that usage is centered around Webkit based browsers. Mobile webkit is also pretty rapidly incorporating HTML5 features and things like CSS transitions. This gives developers who want to make a great mobile site for a big and important part of their audience a single very compelling target to develop for.

Like Gruber said, one of the most noteworthy non-announcements for iPhone 3.0 was no Flash. Who really needs it anyway on the iPhone? And why would Apple let somebody else in on their platform?

Is Flash dead? No. Is there for the first time a clear path forward to a world without Flash? Yes, finally!