I’d rather be proved wrong than miss the chance to improve

One of my favourite books is Nightfall by Isaac Asimov and Robert Silverberg. The setup for the sorry is an old astronomy professor who has won his planet’s equivalent of the Nobel Prize for proving that the six suns in his solar system would never set on his planet at the same time. And night couldn’t fall.

And then his students did some calculations and realized he was wrong, and for a time every few thousand years there would be night.

They didn’t want to tell him thinking it would ruin his life’s work and leave him feeling dejected and worthless.

He got angry with them because he didn’t want to miss the chance to learn something new, and the thrill of discovery had the opposite effect to what the students expected and renewed his enthusiasm in the subject.

Renewed enthusiasm

I’m not a Nobel winner by a long shot, but the professor’s attitude embodies the way I want to work. Doing the same thing and thinking there’s nothing left to learn would demotivate me.

I love mentoring because I love learning.

When I build a team, I want people who aren’t afraid to tell me I’m wrong, when I am, because that’s what makes the team stronger. I know not everyone is confident at this at first but if I can find people who are confident in their opinions, my job is to nurture that.

If I’m not being challenged, I’m not learning. I know my experience has helped me get where I am, but I need to know where me and my team are going.

Tasks for you

If you are a leader or a mentor, embrace the opportunity to learn that you get from being wrong. Be vulnerable, sometimes, so that everyone knows it’s ok to be wrong, if you acknowledge it and make amends. Keep learning.

If you’re not, embrace your mistakes. (Link – my mistake) Next time you find yourself in that position, you’ll call it experience. Learn by doing as well as by reading. The most interesting challenges you’ll face won’t have a manual. Keep learning.

Don’t let your career go dark.

Advertisement

Debugging Asp.Net Core 2 apps in Azure

I’ve been getting under the skin of Asp.Net Core 2 following a failed experiment with Asp.Net Core v1. I certainly found .Net standard and the associated framework support to be very welcome, and unlike my previous project we didn’t need to support GDI or SignalR libraries, so it’s much more in the Core sweet spot.

I’ll talk about the project and the technologies involved in some follow up posts, so if you’ve got any real-world questions on Asp.Net core, React/Redux with Typescript, or CosmosDb in C#. let me know and I’ll try and address them as I get to those technologies.

For now, though, I want to start with the basics. Asp.Net core on Azure works great, mostly, but if something goes wrong in Startup.cs, there’s no Application Insights, a generic 502.5 IIS error – which just means it can’t talk to Kestrel, and no web logs to help you. So before you deploy to Azure, do yourself a favour and add the following to web.config so you’ve got logs to help you.

<system.webServer>
<handlers>
<add name="aspNetCore" path="*" verb="*" modules="AspNetCoreModule" resourceType="Unspecified" />
</handlers>
<aspNetCore processPath="dotnet" arguments=".\web.api.dll" stdoutLogEnabled="true" stdoutLogFile=".\logs\stdout" forwardWindowsAuthToken="true" />
</system.webServer>

That way, if you do see a 502.5 error on your site, you can jump into Kudu and start reading the logs. They can grow quite quickly, depending on your web app lifecycle settings, so you may benefit from a regular cleanout of your logs folder.

If the logs still aren’t helping, or you don’t understand what you’re seeing, there’s a nice Asp.Net core 2 troubleshooting guide over at MSDN, but not all of it applies to Azure.

Ethics in technology

This is an extension of a twitter thread I wrote in response to this tweet, and thread about the Cambridge Analytica revelations.

Wow. This thread is powerful and important. And computer science is having that moment of reckoning now. Facebook and Cambridge Analytica is only the tip of the iceberg. https://t.co/6C0Kw1Dt74

— timoreilly (@timoreilly) March 19, 2018

One of the key modern problems is how easy it is to access these tools. You don’t need professional training to string these together.

It’s as dangerous as if someone invented a weapon that could kill 10s or 100s of people, light enough to carry anywhere, and available in any store, without training. And expecting owners to police themselves.

People are terrified of AI. We know we don’t need AI to disable hospitals. We don’t need AI to intercept Facebook logins (although FireSheep and the pineapple are less effective now). We don’t need AI to send a drone into a crowded market.

Make a website the only place for government applications, such as medicare or millennials railcards and it’s easy to remove access for all citizens.

But combine all that with data and you can fuck up someone’s life without trying. You can give 2 people the same national insurance number or other id. You can flag them on the no fly list.

You can encode prejudice into the algorithm and incarcerate someone because they grew up in a black neighborhood.

The algorithm is God. The algorithm is infallible. Trust the algorithm.

Even when it tells you someone is more capable than the humans says she is, and punishes them.

(unless you’re under GDPR where you have the right to question the algorithm)

But tell anyone that people will use data for purposes they hadn’t considered (like using RIPA anti-terror legislation to see if someone’s in the school catchment area) then you’re paranoid.

Be paranoid. People will always stick crowbars in the seams. Whatever your worst case scenario for your code is, you’re probably not even close.

---

You can see my original tweet, and the repies, here:

One of the key modern problems is how easy it is to access these tools. You don't need professional training to string these together. https://t.co/zhaA8Blz3L

— craignicol (@craignicol) March 19, 2018

The Guardian has a great interview on AI, existential threats and ethics on their podcast here.

New Job : Welcome to Screenmedia

Following my request across my network looking for a new job, I started at Screenmedia 3 weeks ago. For those who don’t know, it’s a digital design practice, which means I’m back to consulting, and I get to work with a lot of smart people, covering technology and design. I’m a Technical Architect in the integrations team, so that’s APIs, voice assistants, serverless, analytics, so should be a good wee adventure. I’ve got a few thoughts on the job hunt which I’m sure will come up at ddd.scot and future blog posts. But if you’re currently looking, we’re hiring. If you’ve got any burning career questions, the DDD Scotland panel survey is still open.

I’ve been working on lots of interesting projects already, so groking multiple domains, sometimes multiple for the same customer, Checking the checklists, and reviewing the onboarding process. Sometimes the best change is the one that lets you re-evaluate what you think you know.

 

 

Cloud thinking : storage as data structures

We’ve all experienced the performance implications of saving files. We use buffers, and we use them asynchronously.

Azure storage has Blobs. They look like files, and they fail like files if you write a line at a time rather than buffering. But they’re not files, they’re data structures, and you need to understand them as you need to understand O(N) when looking at Arrays and Linked Lists. Are you optimising for inserts, appends or reads, or cost?

I know it’s tempting to ignore the performance and just blame “the network” but you own your dependencies and you can do better. Understand your tools, and embrace data structures as persistent storage. After all, in the new serverless world, that’s all you’ve got.

---

Understanding Block Blobs, Append Blobs, and Page Blobs | Microsoft Docs

 

My 2017 in review

After the whirlwind of 2016, 2017 looked like a quieter year. Fewer talks, some interesting and rewarding challenges bringing a new product to market, and a chance to build and reflect on what it means to be a technical leader, to move jobs, and to be productively lazy. Although I notice there’s still a lot of interest in obscure bugs thanks to Chrome’s URL limit, and the User Experience when 2 factor authentication needs to be reset.

I’ve not had quite as many blog views as last year, but I’ve accepted I’m not here to be a blogging superstar. This is my scratchpad for the talks I want to give, and a place to share useful reminders and signposts for future me, and others. Thanks to all of you who have helped shape and refine these thoughts here, on twitter and via other channels.

I wasn’t planning a new job in 2017, but more on that next week (and many, many thanks to my twitter and LinkedIn connections on that front – I’ve been humbled), which means I have some more thoughts on the product life vs the consultancy life that I hope to share this year.

I got a few opportunities to think about applying Conway’s Law to build teams that make the right software, most notably in the Architecting Teams guided conversation I led for CodeCraft.

Looking to 2018

I’d love to keep up 2 blogs a week, playing with styles and topics, as I’ve started to do last year. I’ve got enough topics on my Trello board for a few years at that pace (including one describing the Trello board). I’ve got a new adventure, and some experiments in productivity that I’ll hopefully get more time to explore, as well as reflecting on design and the next generation.

I don’t do New Year resolutions. It’s always a bad day to start something. Always reflect, always refine. And if you leave it to New Year, you’re only giving yourself 70-odd chances to change. Why limit yourself?

Sláinte

User Experience : A quick introduction for developers

The following is an internal summary I wrote for a team that no longer exists, summarising a number of references from UXScotland, various book and blog posts. It extends the thoughts from my Pecha Kucha talk. For more details, please refer to the links throughout and the references at the bottom. The context here is consulting and long-term B2B projects, but some of these discussions are more widely relevant.

User Experience : Project considerations

User experience is about making sure we are solving the right problems in the right way. It is the intersection of design, users, and context. Context here is a combination of one or more of the device in use, the user’s location, any social cues such as nearby friends, and anything else that may be available from sensors or historical information.

In many cases, the requirements we have are assumptions (e.g. what users want is Facebook integration). Where the benefits of a requirement are unclear, we should treat it as an assumption to be tested. Embrace data, and analyse it.

At the requirements stage, we need to make sure we are solving the right problem (pretotyping : “building the right *it*”), and that our chosen design helps the user to solve the problem without frustration (i.e. prototyping the design, rather than the implementation, with wireframes/sketches).

In user testing, particularly in an agile development, we can refine those ideas by seeing how well the implementation solves the problem, by testing with users. We can also test deployed code by analysing heat maps and http logs to see what users are doing to inform further tests and the assumptions that feed into further design cycles.

In a Lean/Agile project, we need to be explicit about our assumptions about the user and test them at every stage of the development to ensure that we always meet user needs.

How does UX fit in our process?

Stakeholders

A system that supports user’s need effectively will need to understand that the user is a Stakeholder in the process. Whilst the users themselves may not be directly involved in the generation or review of design artefacts, there should be a user representative, either a super-user on the customer side, or a 3rd party researcher who has determined user needs, and has authority to verify any proposed solution and high-level requirements against those needs.

Functional Requirements

Personas / Typical Users

A persona is an abstraction of a system user. In a simple system, there may be only one type of user, but more sophisticated systems will typically have users and administrators, and may have multiple classes of each. A persona is defined to encapsulate the types of tasks a specific user may wish to perform, and any limitations that may be imposed (for example, administrators may be able to install specific browsers or client software, but members of the public using the system must be supported across multiple browsers at multiple screen sizes).

User Journeys

Each persona will have one or more tasks they wish to perform in the system. A User Journey describes the tasks as a series of steps that a specific persona will follow in order to achieve that task.

Consider the tasks that a user wants to perform. See also BDD – design from user in.

E.g.

User wants to process a case:

• User logs in to the system
• User selects case from their task list
• User reviews latest document
• User finds agent for case, and calls to discuss
• User adds comments to case
• User saves case and returns to their task list

This process may identify a new use case (“Display task list”), and specific actions that need to be defined within a use case (“Display latest document” and “Display agent contact details”)

The User Journeys provide the context between the Stakeholders (and User Types therein) and the Use Cases. Each User Journey will link to one or more Use Cases, but some Use Cases may not have an associated User Journey (nightly payment processing, for example).

User feedback

If the solution is replacing or improving an existing system, the best source of information on the current system are the users. The requirements capture process should take into account both the tasks that the users perform and gather feedback on any areas of frustration. The prioritization exercise should consider these improvements as well as new functionality.

Testing

As well as testing the Use Cases for functional acceptance, the FAT/UAT process should also test that the final system supports the User Journeys defined up front.

On-going support

Where projects have regular support meetings, the input of users has been valuable in identifying problems areas and possible changes. When on-going service delivery contracts are defined, SDMs should consider whether ongoing user feedback is appropriate as part of the planning and scoping of releases within that framework.

Questions to ask

• Have the requirements been tested on users? If not, why not? (Are these the right requirements?)
• Will users be given the opportunity to provide feedback on these through the development? (And if so, how, when and where?)
• What user outcomes are we trying to achieve with the release? These may not be requirements that we put a cost on, but an expectation that we can measure against to show improvement – we would need to communicate this appropriately.
  • E.g. minimise clicks to access the 5 main functions
  • E.g. reduce time-to-complete for function x, y and z by 10%
  • E.g. Align existing UI with iOS and Android norms
  • E.g. Increase usage of function z by 5%
  • E.g. 99% AAA compliance
• Who represents users on the project team?
  • How many user types do we need?
  • Can normal users and administrators share UX, or are their goals divergent? – different apps, different ASP Areas, different branding, …
• What platforms and form factors need to be supported/tested?
  • Does each platform need a native UX? If native app, probably yes, if web app, maybe.
  • If mobile, do we need to adapt to context : location/orientation/communication with nearby devices/…
  • If social, do we need to adapt to context : can I approve my own work?/who’s online/recommendations/who’s nearby/…
• Do we, as developers, have any input to the UI design? If not, why not?
• Have the designs been tested on users? If not, why not? (Does the UI fit user expectations?)
• Do we have appropriate guidelines for the appropriate platform, and are they listed in the requirements and estimates?
  • iOS : Mobile Human Interface Guidelines
  • Android : Design principles
  • Windows

Potentially useful resources

 

• Lean UX by Jeff Gothelf and Josh Seiden
• Steve Krug presentation : Usability – One more thing you don’t have time for
  • Also check out his books, including Don’t make me think
• UX Scotland
• UX Professionals communities
• UXPA-Scotland (mainly Edinburgh)
• Government Digital Service (Agile, UX and a few other real-world discussions)
• A List Apart
• .net magazine
• Ask Tog
• HTML5 Rocks
• Color Me Impressed: 3 Tools For Effortlessly Finding Beautiful Color Schemes For Any Project

Government insecurity agencies

Given the SSL attacks that could be traced back to classing secure encryption as weapons subject to export restrictions, it’s clear that government security agencies have a deep conflict of interest that has led to significantly reduced security protection for their own citizens.

It’s clear that the Ransomware (or Ransomware as diversion) attacks on UK and US hospitals and many other sites are directly due to the NSA backdoor toolkit that was stolen earlier this year. Because if the government has a back door into a system, or an encryption platform, everyone has a backdoor, even if they don’t have access to it yet.

Which is why it’s great to see the EU outlawing backdoors in order to protect us as patients, service users, and data subjects, and I completely expect this will apply, like GDPR, to any system holding EU citizens data. So when the UK puts on its “we need a back door” legislation, companies need to choose to trade with the UK and compromise their security, or trade with the much bigger EU and protect their customers.

Encryption is like a lock, but it isn’t. It’s like a safe door, but it isn’t. Abstractions help to frame the problem, but they can obscure the issues. They make lawmakers think that what applies to banks applies to data.

(note: bank processes are optimised to replace credit cards because security works best when you can throw away a channel and start again if it’s compromised; this includes reversing transactions – which is hard to do when it’s the release of your personal data that needs reverted, rather than a row in a ledger than can be corrected by an additional row).

Encryption isn’t the problem. The San Bernardino iPhone had no useful intel. All the recent attackers in the UK were known, reported, and could have been tracked if they were prioritised. Banning encryption will have about as much impact as banning white vans. Breaking encryption weakens our security, threatens international trade especially with the EU, and when security holes lead to attacks on our hospitals and other infrastructure, bad security threatens our lives.

But so long as we’re afraid of terrorism, it’s OK for the populous to suffer?

My .net journey

With the release of Visual Studio 2017 and .net core, I’ve seen a few folk talking about their story with the platform. This is mine.

I’ve never been the biggest Microsoft fan, ever since I grabbed a copy of Mandrake Linux and figured out how much more tinkering was available and how much more logical certain operations were than on Windows 95. But it was definitely still a tinkerers platform.

But I got an internship at Edinburgh University whilst I was a student there, funded by Microsoft. I got a laptop for the summer and a iPaq (remember that?) to keep. I also got a trip to Amsterdam to meet the other interns and some folk from Microsoft, back before they had much more than sales people in the UK. And they told me, no matter how much anyone hates Microsoft, they always hate Oracle more.

It meant that I was among the first to get the .net 1.0 CD, so I could legitimately claim later that yes, I did have 2 years of .net experience.

But from there, I stayed in Linux, learning the joys of Java Threading on Solaris (top tip : Sun really should have known what they were doing, that they didn’t means I can see some of why they failed – it was far easier working with threads on Red Hat or Windows).

And then I did my PhD, digging into device drivers, DirectX and MFC. I hated Microsoft’s Win32 GUI stuff, but the rest, in C++, was quite nice. I enjoyed being closer to the metal, and shedding the Java ceremony. I trained on templates and started to understand their power. And Java just wasn’t good enough.

I wrote research projects in C++ and data analysis engines in Python. It was good.

But Java came back, and I wrote some media playback for MacOS, and fought iTunes to listen to music. And I vowed never to buy from Apple because both were a right pain.

And I needed a new job. And I’d written bots in IronPython against C#, so I got a .Net job. And I missed the Java and Python communities, the open source chatter. And I wanted to write code in C# that was as beautiful and testable as C++. And I wanted to feel that Bulmer’s Developers! chant was a rallying call, not a lunch order from a corporate monster.

So I found alt.net and it was in Scotland, and I wrote a lot of code, and I learned that open source did exist in c#, and that there was a conference named after that chant and I met more like minded developers. I fought my nervousness and my stumbling voice and I found some confidence to present. And blog. And help write a package manager. And then everyone else learned Ruby.

And then the Scotts joined Microsoft and alt.net became .net. And then LINQ came and I remembered how clean functional programming is, and I started feeling like I was writing Python if I squinted hard, and ignored types. And then core came, and Microsoft had some growing pains. But it’s a sign that the company has completely shifted in the right direction, learning from the guys who left for Ruby. And Node.

I’m proud of what I’ve built in C#, and it’s a much better language than Java, now. It’s definitely the right choice for what I’ve built. The documentation is definitely better than Apple or Sun/Oracle produce, although MSDN and docs.microsoft.com are having some migration pains of its own.

And alt.net is making a comeback.

And I still use Python on hobby projects.

Windows resource limit in services

Here’s a little something that stumped us for a few days and might be worth posting to save others time.

Following a move to IIS8.5, we started seeing “Out of resource” errors on a server that did not appear to be bottlenecked by disk, CPU or RAM.

It turns out that since a previous version of IIS, the Application Pool service doesn’t grab GDI handles as it runs as a non-interactive service, so anything relying on that, such as a DLL with GDI dependencies, like an image resizing library, only gets the non-interactive desktop heap for graphical services. As soon as you get enough calls into that DLL, the heap fills and the program crashes with the “Out of resources” error.

So you recreate the issue in the debugger, attached to IIS Express, running in user space, with the full interactive desktop heap, and you can’t recreate the issue.

To fix the problem, you need to carefully adjust the heap limit in one of the ugliest registry values in Windows. Have a read here to find out what the Desktop Heap is and the registry key that controls it, then up the 3rd SharedSection value (the non-interactive heap) in small increments (lest you put in a value too high, break the interactive heap and lose the ability to log on).

And then find a way to rewrite the DLL.