2013 in review : agile, dynamic and dangerous data

The WordPress.com stats helper monkeys prepared a 2013 annual report for this blog.

Here’s an excerpt:

A San Francisco cable car holds 60 people. This blog was viewed about 1,600 times in 2013. If it were a cable car, it would take about 27 trips to carry that many people.

Click here to see the complete report.

Leave a Comment

#dunddd Analyse This : The dangers of big data

Thanks to everyone who came to my DunDDD talk. Lots of interesting questions, although I’m not a lawyer so couldn’t answer them all.

If you want the slides, with references in the notes, you’ll find them here. All the images are creative commons, and you can use the sides yourself under CC by Attribution. Link to slides : Dunddd Analyse This – The Dangers Of Big Data (Google Drive)

If you missed the talk, the arguments I made and the references, apart from the privacy sections, are in this

Link to previous post

If you want the references for the Personal Data and anonymisation parts, have a look at these :

AOL searches are not private

IBM privacy-preserving data mining

Leave a Comment

Announcing DunDDD 2013

craignicol:

I hope to make it this year, are there any of my blog posts you want to hear in a talk format?

Originally posted on Scottish Developers:

DDD ScotlandScottish Developers are pleased to announce that DunDDD 2013 will take place on Saturday 23rd November in the Queen Mother Building of the University of Dundee.

http://dun.dddscotland.com

This will be the 3rd DunDDD built on the popular foundation of the Developer! Developer! Developer! conference series which has spread to all corners of the UK and the international arena.  DDD conferences are community-run days where passionate and enthusiastic people come together to learn, share ideas, and to network within the many hubs of the development community. Best of all DDD events are free to everyone.

This year DunDDD will be featuring an entire track dedicated to Data Science.

Call for speakers

We are looking for sessions relating to all aspects of development from the code and technology level through to methodology and theory. DDD conferences are about sharing experience regardless of skill level so even if you have never spoken publicly…

View original 100 more words

Leave a Comment

User Experience Matters : Pecha Kucha at TechMeetup Glasgow

Thanks to all of you who came to TechMeetup Glasgow, hope to see you next time in Edinburgh or at Open Rights Group in Glasgow (also on Google+). The Pecha Kucha is an interesting challenge, and this was my first one, so any advice on how to improve is greatly appreciated. The slides (including the pptx download) are available on SlideShare, and I’ll add the video once it is available:

The book that I recommended in the talk is : Lean UX: Applying Lean Principles to Improve User Experience but I was also recommended Don’t Make Me Think!: A Common Sense Approach to Web Usability by Steve Krug and Rocket Surgery Made Easy: The Do-it-yourself Guide to Finding and Fixing Usability Problems (Voices That Matter) by Steve Krug which I’ve added to my reading list.

The Creative Commons graphics I used are :

Crayons 1

paperclip

Drafting

Machine Transcription

Heatmap

test

Clipboard and Pens

Mobile Usability Testing Demonstration

Leave a Comment

Open Letter To Jon Skeet – Semantics Matter, that’s why you should use dynamic languages

Dear Jon,

In your Semantics Matter talk, you expressed some doubt that dynamic languages allow you to provide meaning to code and therefore we should prefer languages like C# and Java because the compiler-time type checking provides a safety net. As someone who’s developed systems using C#, Java (1.4) and Python, I found myself feeling uneasy about the suggestion. I spoke to you briefly in the bar before I had to leave and spoke about APIs and you suggested types help guide less experienced developers. Unfortunately I didn’t have more time to speak to you, but these are the thoughts I had about it on the way home.

I realise my experience of Java isn’t as recent as yours, but I hope you take the examples provided in the spirit of examples rather than best practice.

I can see how types aid developers to understand an API, and I can see the benefits of strongly types rather than “stringly” typed interfaces. My problem with that is typing isn’t nearly enough, as you showed with a number of Data and string examples, and we know we have to test, and add contracts to ensure that the value passed not only matches the type, but isn’t null, that an ID key matches a known value, that a credit card number is the right length and passes the checksum, and a number of other validations that typing doesn’t capture, because the developer has to layer meaning onto the data.

Typing hides meaning. It beats semantics into a corner – UTF8-encoded text or ASCII-encoded text or object Identifiers can are all be strings, and their meaning is subsumed into their type.But the only time we can identify a type is in the run time context : because the user typed something in the Address field, rather than the Postcode field. It doesn’t have a semantic type and it doesn’t have meaning until we validate it. In this way, typing lulls you into a false sense of security. XML provides slightly better types – XML is strict and allows you to enforce regex, length and other constraints that make sense for that type, but it doesn’t solve the full problem – it still can’t tell you if an email or postcode are valid – they only have a semantic meaning as a locator once they’ve located you.

More than that, types get in the way of meaning. Consider Java 1.4 and the horrible pre-generics casting that stripped all meaning from objects in a list (which I admit has been mostly resolved now with generics and array covariance, but they have their own problems). This casting also removed any type information. Even with generics, we need to define IComparer before we can sort, which helps to understand what the sort algorithm does, but loses the clarity of “but I can compare that, why can’t I sort it”, because a type that can be compared may not inherit the right parent. This is one particular strength of Python’s duck typing because it removes the need to define an interface and relies on the language semantics, allowing the logic to be expressed independently of the framework you need to define. I remember this being a particular problem when I wanted to pass strongly-typed functions into a genetic algorithm library I was building in C++ : the amount of scaffolding required to define the callbacks hid the meaning of the underlying problem I was trying to solve, whereas the Python version, by removing the typing, and moving the validation into the callee, greatly simplified the interface.

The semantics of a type are at the wrong level to convey business meaning. I want a number to represent 1p, 1 Euro cent or the UK national debt. I shouldn’t have to care if that’s an Integer, a BigInteger or a Decimal, so long as it can store it, and not fail on overflow or division. Making me care about the hardware means I mix the semantics of the business logic and the implementation, driving me to hold 2 semantic models in my head, making the StackOverflow mistakes you highlighted more likely. In Python, the model fails under division but otherwise keeps out of my way. The dynamic typing of Python means that it can automatically change a number from integer to BigInteger if the operation would otherwise fail, isolating my code from the hardware implementation, and allowing the interpreter to do the job of using the most efficient representation, in the same way that HotSpot can compile certain objects onto the stack where it improves performance, removing implementation details out of the developer’s head and into the compiler, allowing the develop to focus on business meaning.

I’m not writing a compiler, I don’t want to care about the semantics of the hardware because I can’t guarantee them. If I write an Android app, do I care if my hardware is little endian or big endian? No. So why should I care if a number is 32 or 64 bit. In Python, I don’t need to care, in C# it matters. And that’s wrong.

The types that matter aren’t the types the compiler checks. A max function won’t tell you that you can’t compare two ids, or two parsed credit card numbers. Hungarian notation used to define purpose helps (screenWidth, screenX and viewportWidth, viewportX for example) but without a language like Ada or encapsulating types into non-castable forms, you cannot rely on the compiler to enforce it. You may as well be using defaults or string typing, and I know how much you love those.

Is it better to have “something” than “nothing”? Surely having static types is safer than dynamic types? You start with something, you know what it is, and you can deal with it. Only you don’t know what it is. Surely it’s cleaner and clearer if a variable of a less specific type is replaced by a more specific type once it has been validated? You can do it by creating new variables, but overwriting keeps the intent clear and prevents the use of the invalid form once the validation has been done. To replicate the behaviour in C# or Java, we’d have to encapsulate the target type in a class and use a constructor to validate, and use hungarian notation to distinguish between the raw and validated forms.

If you care about semantics, static typing can get in the way, but that’s just my view.

Cheers,
Craig

Comments (3)

So Long Hangouts, And Thanks for all the fish

I’ve been re-evaluating the time I spend on the developer hangouts, and despite the interest I get from the invites, the turnouts have not been high enough to sustain these meetings. As a result, I will not be hosting any more.

If there is interest, I may think about a podcast-like Hangout-On-Air, but I would need some speakers for that, and I think that outlet would be better served by the plans +TechUp Inverness has for streaming their talks, and I might see if I can persuade other Scottish developer talks to make more content available online for streaming and reviewing (although I know from experience at work, there is a significant extra effort involved in setting such a thing up).

It’s been an interesting experiment, but it’s time to try something else. In the short term, I will be using the time I spent on this to update my blog as I’ve got a backlog of posts about node.js that I want to complete.

Many thanks to everyone who attended any of the hangouts this year. I hope to catch up with you all again in person at other events.

Leave a Comment

Users or consumers?

I’ve just been reading this article about how capitalism has changed our language in the last 200 years, and the move from talking about people as users of products to consumers. “How capitalism has changed our language” http://feedly.com/k/18naQTK

There is an old computer joke that about programmers referring to their customers as users, but this article highlighted to me something deeper that feeds into something about how technology is shaping our world.

Software development, when it works best, happens with active users rather than passive consumers, people who have an active interest in the problem the software is designed to solve, and sometimes ideas on how to solve it.

The biggest red flag I have to indicate the success or failure of a software project is the lack of user engagement. If no-one on the project team is a user, or is listening to the users, the project will almost certainly fail.

Software customers are not consumers, and often the user is not a customer, and they will only use the software they are forced to use by management or that they are comfortable with, and will likely not use it if they know a better alternative. If we can’t write software that engages users and makes their life easier, they will not use it.

This may not be a problem unique to software development, but at least the language of software recognises that there is a problem to solve, and that the user experience is the difference between software that delights and software that frustrates.

Leave a Comment

Older Posts »
Stack trace ramblings

Just another WordPress.com weblog

The DIEM Project

Dynamic Images and Eye Movements

Voidspace - Cyberpunk, Spirituality, & Python

Just another WordPress.com weblog

the HabitForge blog

Simple Accountability. Positive Change.

James Wiseman

Mainly programming, with a bit of science thrown in

codeface

on making software

Don Charisma

because anything is possible with Charisma

Kendall F. Person, thepublicblogger

Where writing is a performance art and every post is a show.

THIS DAY, THIS SONG

Discover a new song every day

Scottish Games Network

Representing the video games industry in Scotland

UK Constitutional Law Association

affiliated to the IACL

Coding Nuggets

Musing of a .Net coder based in Glasgow, Scotland.

justanotherhumanoid

This WordPress.com site is the bee's knees

Free Scotland Now!

I believe in Scots independence and actively campaigning to make it happen.

Gary Short

There's no place like 127.0.0.1

The Daily Post

The Art and Craft of Blogging

Mainly programming, with a bit of science thrown in

You've Been Haacked

Mainly programming, with a bit of science thrown in

Follow

Get every new post delivered to your Inbox.

Join 1,289 other followers

%d bloggers like this: