ai artificialintelligence data development search

The UX of Big Data

Following on from my Dangers of Big Data talk at DunDDD, I’ve been thinking about what a good user experience for data analytics would look like, imagining the business user presented with useful, actionable information rather than notepad and a copy of the R or Python cookbook. I want something deceptively simple like the Google search box, rather than deceptively complex like Excel.

Excel, and R and Python, put a lot of tools at your disposal, and you could use any of them to construct an answer, but the secret to analytics relies on getting a valid, useful answer. The first is a matter of restricting the answer space to that which can be supported by the data (for example, disallowing multiplication of time-based input streams, or aggregating when there is no statistical basis for it), the second is a matter of allowing the user to explore the space so they can determine (and where appropriate, train the system to recognise) which factors are most important, how they affect the desired outcome, and how changes to the environment affect these factors.

Then the question becomes, how much should the software take over. Do we have a duty to protect users from themselves by preventing invalid analysis where we can detect it, or do we have to accept that the frustration that will cause leads to alienation and users will be less likely to respond well to further corrections. Even nudging had its possible, as anyone who had been frustrated by grammar checkers can attest. But at least nudging helps the user to understand, rather than putting up roadblocks. Nudging encourages learning, roadblocks encourage switching to another way.

How would you encourage users to handle analysis appropriately?

code development programming security

CodeCraftConf by @codecraftuk

Follow @codecraftuk on twitter
CodeCraft logo

The tickets for CodeCraftConf are now on sale. It’s based around the idea of full participation, where every attendee is there to discuss the topics. The format is of guided conversations, which has worked well for sessions in the past, and I’ve included a list of some of my previous sessions using a similar format below for reference. I’ve used the mindmap format for sessions I’ve done in the past so that there is a solid summary that attendees can refer to if they wish, but I have also been involved in sessions at other conferences that were deliberately transient in order to foster honesty and controlled venting, more like a support group for frustrated developers.

I am looking to guide a session at the conference and will be talking to the organisers about the topics,. If you look at the list below, and the conditions on the conference site, it should be clear that this will be a philosophy of code conference rather than a discussion of all the hot technology you don’t know about yet, but really should. (and number 5 will change your life forever).

I realise that holding the conference in a pub, even one as good as Drygate, will challenge the training budgets of some companies, but it looks like a great concept, and I hope the conference takes off. Many thanks to Joe and Gary for bringing this together, and I look forward to meeting everyone there.

What you missed at DDDs

My previous chaired discussion sessions, with mindmaps for reference.

code data programming ux

#dunddd Analyse This : The dangers of big data

Thanks to everyone who came to my DunDDD talk. Lots of interesting questions, although I’m not a lawyer so couldn’t answer them all.

If you want the slides, with references in the notes, you’ll find them here. All the images are creative commons, and you can use the sides yourself under CC by Attribution. Link to slides : Dunddd Analyse This – The Dangers Of Big Data (Google Drive)

If you missed the talk, the arguments I made and the references, apart from the privacy sections, are in this

Link to previous post

If you want the references for the Personal Data and anonymisation parts, have a look at these :

AOL searches are not private

IBM privacy-preserving data mining

development programming

#DunDDD recruitment follow-up discussion

There’s an updated MindMap for the follow up discussion, looking at employee’s expectations, and the difference a manager can make. Thanks to everyone who turned up, if you’re interested in more, keep an eye out in the new year.

Recruitment in the IT industry with employee expectations follow-up
code development programming

#DunDDD Cross Country Coding slides and notes #xcc

Cross Country Coding Prezi Slides

Thanks to everyone who attended my Cross Country Coding talk at DunDDD. Hope the technology failure didn’t put you off too much. I really enjoyed the Q&A session afterwards, but according to the feedback, the talk was too short. I’ve got a few ideas following the discussion about coding across time zones. In particular, looking at non-real-time communication options such as wikis, twitter/, blogs, and the use of multiple stand-ups to handover between time zones. If anyone’s got any other ideas, let me know and I’ll see what I can incorporate into the next iteration.

If you want to present this yourself, you can grab a copy of the prezi here, and you’ll find the speaker notes on Google Docs available to copy.

If you do decide to use them, or you have any other comments or suggestions, please feel free to add a comment below or give me a shout on the usual channels.


#DunDDD follow up recruitment discussion Thursday 29th November : What should companies offer?

Many thanks to everyone who attended the #DunDDD open discussion on recruitment.

I would like to follow up the discussion we had about what interviewees should do with an online discussion about what employers should do. This will be held on Google+ on Thursday 29th at 7pm. See here for details and to sign up :

For those who couldn’t make it, the midmap is available here:

Recruitment in the IT industry Mindmapmind map: Recruitment in the IT industryMind Mapping – MindMeister.
Create your own mind maps at MindMeister

code development programming

DunDDD : What do you think about recruitment in the IT industry?

I’ve been invited to run a group discussion on recruitment at DunDDD, and to kick off the thinking, I want to know what topics you want to discuss, either as a job hunter, or looking for staff. What do you think about interview techniques, probation periods and recruitment agencies? What, apart from salary, do you look for in your job? What do you think schools, universities and employers should be doing but aren’t?

Give me your thoughts below, or email me (via Gmail or a twitter pm) or bring your comments to Dundee at 9am Saturday. These sessions can be lively so make sure you have a coffee and get there early to get a seat.

If you can’t make it, I’ll put up some notes at some point next week.

Hope to see you all there.

code development programming

Slides and Mind Maps for DunDDD

DunDDDAs promised, here’s the slides and mindmaps for the sessions I was involved in at DunDDD 2011. The Mind Maps were generated using FreeMind.

The Philosophy of Code

This talk was an experiment on my part, given the knowledge I’ve picked up from reading books and articles from some of the smartest people in the business and beyond, and I wanted to share some of that. As Gary Park noted, it still needs some polish, but I think there’s a good idea in there, so I hope to get another chance to present it in the future. The presentation itself is licensed under creative commons, but please pay attention to the photo attributions if you want to use them in your own work. I’ve also included a link to the original mind map which contains many more great quotes.

Google Docs : The Philosophy of Code

Mind42 : The Philosophy Of Code

Download The Philosophy Of Code Mindmap (.mm format)

Software Requirements

The original presentation was given by Craig Murphy (on Twitter as @CAMurphy) and is available here : Open Discussion on Software Requirements

The mind map generated from the discussion is reproduced below.

There was a good discussion of how requirements can have different levels of detail and how the methodology can shape the process and the documentation, as well as the change process. A bit of waterfall vs. agile, but each has their place.

Mind42 : Software Requirements

Download the Software Requirements Mindmap (.mm format)

How The Web Was Lost

This talk drifted a little, since we agreed fairly quickly that with the demise of Flash and Silverlight, and the rise of the web-powered desktop in Windows 8, the web has in fact won. +1 for open standards. But where does that leave the behemoths like Apple and Microsoft who have benefited the most from the traditional role of the desktop. Can they keep developers and users on their platforms, or will they be lost to cross-platform development?

Mind42 : How The Web Was Lost

Download How The Web Was Lost (.mm format)

code development programming


Just a quick post on my way back from DunDDD (slides and mind maps to follow)

It’s the first time I’ve been to a developer event in Dundee and I was impressed with the enthusiasm and commitment of the attendees, so I hope that’s a good sign that there will be another one next year. Aside from the 3 sessions I was leading, I also attended Gary Short’s mining Twitter talk and Colin Gemmel’s Ruby talk. Lots of good info in both so thanks to them, all the other speakers and organisers for another great event.

If you missed out on this one and are interested in more, check out for more events. A couple in Edinburgh in 2011, followed by more events in Edinburgh, Dundee, Aberdeen and Glasgow next year. Well worth following to keep up with the geek scene in Scotland.


For more reviews please have a look at the following blogs:

Dundee DDD Conference – Chris Hillman (@chillax7)

A review of #DunDDD – Gary Park (@gep13)