Categories
development

CodeCraftConf 2019 : What is data anyway? (Answers)

Here’s my thoughts on data following my CodeCraftConf guided conversation. Here are the questions I asked during my guided conversation at CodeCraftConf 2019. They are also available on GitHub if you would like to fork and modify them for your own use.

Most developers are data driven, start with the data structure, not the algorithm. Either data driven design, or the Merise Methodology.

Data, whilst often divided by microservice, is often stored on the same server/cluster, creating a monolith behind the microservices.

Not all data access is secured and audited, although there does appear to be a trend to on-behalf-of flows through the microservice, allowing user-centered access control. Strict data access design is prevalent, although the efficacy was less clear, and strict design applies to all data, including publicly available data.

Keeping sight of data in distributed systems is hard. Jepsen was suggested as one resource to help, but I’m happy to hear of others.

As well as data that can be used to discriminate by collecting gender, name, postcode etc., we also discussed how missing data can be used to discriminate, such as when Glasgow accents aren’t included in voice training data, or when women aren’t used in medical trails.

There’s also the big and growing problem of data collected by people who do not consider the discrimination or privacy implications. For a biologist, DNA is a puzzle that helps them decode cancer, and more examples make the puzzle easier to solve. But for others, DNA is a tool to map insurance risk, to find criminals, and to track down family members whether or not they want to be found. How do we train everyone else to understand?

And the takeaway question : what questions aren’t you asking about your data?

Categories
data development

CodeCraftConf 2019 : What is data anyway? (Questions)

Here are the questions I asked during my guided conversation at CodeCraftConf 2019. They are also available on GitHub if you would like to fork and modify them for your own use. Thankyou to everyone who came to the discussion, I will post a follow-up to discuss some of the interesting answers.

What is data anyway?

Navigating SQL, NoSQL, JSON and how to work with data in a post-RDMS, big-data world

Questions

Data modelling

  1. When designing a system, do you start with the data or the code?
  2. Has the rise of cloud based or non relational data stores changed how we model our data?
  3. Do you need to update your data when the models in the code change? How do you do it?
  4. Does all your data have to have the same shape?
  5. Should the data you expose to the outside world broadly match the data at rest?

Data security

  1. How do you secure your data?
  2. In light of GDPR, How do you ensure you aren’t collecting too much data?
  3. Who has access to your data?
  • Do you know if anyone unauthorised has accessed it?
  1. How do you protect yourself against bad data and trojan data?
  • Bad data = data that is fake, or is used for real world attacks
  • Trojan data = data that can compromise your or your customer’s systems

Ethical data

  1. Can your data be used to discriminate?
  • Can you prove it?
  • Is your data biased?
  • Are you recording hidden correlations? (ZIP code suggests race)
  1. Who owns your data?
  2. What questions aren’t you asking?

Unused questions

  1. What makes data big?
  2. Are you collecting the right data?
  3. Is the data you’re collecting right?
  4. Where is your data?

Technology choices

  1. Do you still have a place for traditional RDBMS?
Categories
development leadership

CodeCraftConf follow-up : notes on growing a team

Whilst I have a few thoughts I’m still processing after the whirlwind of fantastic insights I got from CodeCraftConf, I wanted to capture some of the highlights from some of the answers to my questions on Growing a Team.

Many thanks to everyone who came to the session. Lots of thoughts coming out of the conference.

When is it time to grow your team?

  • You never grow a team, you create a new one.
  • There’s always too much work, you should grow when you have capacity, so you don’t put new starts off

How do you deal with resistance from existing team members?

  • Communicate clearly and address concerns (e.g, time to mentor)
  • Involve team early
  • Do you have a choice who joins?

Is it more important to find a culture fit or build a diverse team?

  • Do candidates know what they’re signing up for?
  • Introvert vs extrovert (see also hiring practice)
  • Interviews should be structured to filter out arseholes – do they have empathy (or have they been taught to suppress it)
  • Every new hire should bring the level up
  • Don’t just hire for technical skill
  • Diversify your interview pool

What is your biggest worry with your current team size, or with growing your current team?

  • Are you doing Health Checks? Survey staff regularly
  • Make sure the bigger team, outside your daily work, understand the culture – it only feeds from top down
  • Fear of churn
  • Loss of culture
  • Try to understand
  • Make the culture explicit

What practices do you use to ensure sustainable growth?

  • Pair a lot, reflect (e.g. retro)
  • face-to-face regularly, even if it’s video
  • Values workshop – does everyone share them?
  • Social convenor to ensure cohesion
  • Slack channel dedicated to positive feedback on living the values
  • Office/company changes should have their own backlog with an open grooming process so priorities are explicit

How long does it take to integrate someone new?

  • Be careful about language
  • Train everyone in personal skills
  • Listen even when you disagree
  • lack of ego
  • Culture changes people
  • What personalities do you want?
Power corrupts
Categories
development leadership

CodeCraftConf 2018 : Successfully Growing A Team

Thanks to everyone who came to my CodeCraftConf session today, and to the organisers for all their hard work. Here’s the questions I asked, and I’ll follow up with my thoughts from the discussion.

Successfully Growing A Team

  1. When is it time to grow your team?
  2. How do you deal with resistance from existing team members?
  3. Is it more important to find a culture fit or build a diverse team?
  4. What is your biggest worry with your current team size, or with growing your current team?
  5. How frequently can you add new people to your team?
  6. How long does it take to integrate someone new?
  7. What practices do you use to ensure sustainable growth?
  8. How do you know when a team is too big?
  9. How do you split a team that’s grown too big?
  10. How do you grow a team when the existing members are already overworked?
  11. How long is your recruitment pipeline in terms of short-term planning (getting people in the door) and long-term planning (having the right team in place for next year or 5 years?
  12. How do you recruit outside your specialty?
Categories
code leadership

CodeCraft – Architecting Teams Notes and follow-up

Thanks to everyone who came to the CodeCraft guided conversation last week. I’ve added some notes to the questions below to carry on the discussion, and there’s a few topics I want to revisit in this blog, and in DDD Scotland if my talk is accepted.

The code craft team did a great job of reviewing and improving the questions so thanks to them, but I think in the very busy session itself, some of the questions were too ambiguous. For example, when I was thinking about goals, I was thinking about project goals and strategy goals rather than just “Make more money”, but I suspect that’s as much to do with the ambiguity many people have about their company strategy and culture in general.

It was also interesting to me to hear how agile is interpreted. The agile manifesto is clear that people take priority over procedures, and software over documentation, but explicitly the authors “still value the things on the right”. So it’s not about no procedures, and no documentation (please tell me you have coding standards), it’s about the right ones to support the people, the delivery of working software, etc. rather than documentation in its own right. Indeed, there are a number of explicit rules that an agile team needs in order to function. Is there a framework? What is the lifecycle of a unit of work? What language are we using? Not all of it needs to be documented, but some of it absolutely does, especially if you want to understand and improve it, and remove the rules you no longer need.

Be wary of anything that interrupts transparency, whether it’s Chinese walls, secret tasks, hidden agendas or ninja developers who work in the shadows and surprise everyone when they’re least expecting it.

And it’s never just enough to say who you are and what you do. Promote, encourage, and challenge. Be proud to be a diverse employer, to be wheelchair accessible, to have staff with more than 2 years real world experience, and welcoming to fresh faces with new ideas.

Trust. Respect. Communicate.


As a general point, I’m thinking of a team as “the people you work regularly with, usually daily”.

  1. How do you keep track of your team and the company goals?
  2. How does your team manage risk to and changes to those goals?
    • Pre-mortem
    • Chaos monkey
    • Agile “threw away risk register and other important documents”
  3. What makes a great team?
    • People
    • Feel comfortable
    • Trust
    • Communication
    • Balance
    • Good Leader
    • Sustainable development
    • Transparency
    • Healthy Debate
    • Growth and change
    • Common Goals
    • Motivation tailored to each person
    • Rules
      • Raise concerns
      • Feedback should be timely and structured
      • Coding standards
      • Process rules (e.g. how do we report a story can’t be implemented, what’s the weekly schedule)
  4. What would you change about your current team?
    • Management
      • Decisions made outside the team, not taking input from the team
      • Coaching developers to talk to management
      • Wrong person promoted to lead
    • Too many rules
      • “Because we’ve always done that”
      • Needs the context of why that rule exists
    • Clear roles
    • Respect for decisions the team has made
    • No secret tasks
    • Co-located, or remote, not mixed.
    • No “them and us”
      • Manager vs developer
      • Techie vs non-techie
  5. What makes you feel unsafe in a team?
    • Lack of:
      • Support
      • Communication
      • Respect
      • Goals
    • Secret genius / hero developer who does their own thing
    • Decisions made without buy-in
    • Teams without source control
    • No tests
    • Secrets
    • Success at the expense of others
    • Developers are not “resources”
      • And managers are not “overhead”
    • Fear
    • Stagnation
  6. How well would your current team survive a conflict?
    • To survive, have Strong Opinions, Weakly Held
    • Have a “naughty step” project for someone who doesn’t follow the rules, or doesn’t respect the team
    • Avoid imbalance in workloads that lead to loading stress onto key individuals
  7. Should teams change when projects change?
    • Teams should be mixed to avoid stagnation, but not completely break them up
    • Good teams can inspire others
    • Every good team cares about the product they produce
  8. How does the culture of a team change as it grows?
    • Negatively. Easily leads to “Us vs Them” (e.g. backend vs frontend vs DBAs, developers vs testers)
    • Lines of communication fail, especially as distance increases.
    • Old vs new – “I prefered the company when it was smaller”
    • T-shaped individuals are better collaborators
    • Tribe structures can help by allowing multiple communication lines
  9. What kinds of diversity should we seek in the teams we work in?
    • Diverse teams build diverse products
  10. Should we recruit to enforce the current culture or diversify it?
    • Put accessibility positively on ad when true (wheelchair accessible, BSL-friendly)
    • Do you know the current culture?
    • What is “culture fit”?
    • Does anyone know your company values? (not just the 5 Apprentice team names on the posters around the office)
    • Get job adverts reviewed by as diverse a team as possible
    • Diversify where the jobs are advertised
    • Not every candidate has 20 hours for a coding exercise, or wants to give up 2 holidays to pair with you
    • Coding review should be blind
  11. What one thing can a leader do to make a team great?
    • Trust
    • Delegate authority and empower the team
    • Protect the team
    • “Reading the room” understand what’s not being said so you can investigate
    • Push everyone to improve (including yourself)
    • Empathy
    • Happy people
    • Communicate
  12. Are effective teams a democracy or a dictatorship?

codecraftuk-sessions/architecting-teams.md at master · craignicol/codecraftuk-sessions · GitHub

Categories
code leadership

CodeCraft – Architecting Teams

Thanks to everyone who came to the CodeCraft guided conversation yesterday. If you want to reconsider any of the questions yourself, you’ll find them below.


As a general point, I’m thinking of a team as “the people you work regularly with, usually daily”.

  1. How do you keep track of your team and the company goals?
  2. How does your team manage risk to and changes to those goals?
  3. What makes a great team?
  4. What would you change about your current team?
  5. What makes you feel unsafe in a team?
  6. How well would your current team survive a conflict?
  7. Should teams change when projects change?
  8. How does the culture of a team change as it grows?
  9. What kinds of diversity should we seek in the teams we work in?
  10. Should we recruit to enforce the current culture or diversify it?
  11. What one thing can a leader do to make a team great?
  12. Are effective teams a democracy or a dictatorship?

codecraftuk-sessions/architecting-teams.md at master · craignicol/codecraftuk-sessions · GitHub

Categories
development leadership

Everyone has a voice 

​One of the challenges that Technical Leads face that isn’t always easy to resolve is making sure the whole team is involved in decisions. 

There is a large overlap between people who can think in code and people who are shy around other people. For some, the enthusiasm of the former can overcome the latter, but for many, especially younger members of the team, they need encouragement. 

There are a few key ways to do it. If you’ve built the right team, pairing with a patient mentor, who asks questions, is a good way to build confidence. We all hate it, but asking someone their opinion in a meeting is important too, until they get confident enough to speak up themselves, because they know they’ll be heard. So pick your meetings wisely. 

There’s also non-verbal ways to increase interaction and confidence. I’m a big fan of asynchronous code reviews, because they help people focus on the code rather than the coder. I realise there is a risk that this can lead to an atmosphere where someone being reviewed feels under attack, but in my experience, with ego-less teams, and especially with teams who understand and fight technical debt, they see reviews as a chance to improve the code and their own understanding, and make everyone’s life easier next time they look at that feature. 

Lean Coffee meetings are also good for helping encourage people to suggest ideas, as they can see what others want to talk about, so they know they have something worth dating. 

Make sure the quiet people speak up

Categories
code development

Usable APIs @CodeCraftConf 

Thanks for those of you who came to the APIs conversation. If you want to continue the discussion, these are the questions I had on my cards. 

  1. What’s the worst thing an API has done to you?
  2. What’s the first thing you check when evaluating the use of a new API?
  3. What’s the first thing you do when developing a new API?
  4. Do you approach developing APIs internally differently to one for the public?
  5. How do you make an API foolproof?
  6. How much structure should an API have? Does it need a contract? (see SOAP vs REST)
  7. How should you change an API?
  8. How should you retire an API?
  9. Which stakeholders need input to API upgrades and changes, and how do you engage them?
  10. How do you keep an API consistent if you grow it via tests?
  11. Do bug fixes necessitate a new version?
  12. How do you isolate your application from an insecure API?
  13. How do you isolate your application from an unreliable API?
Categories
code development programming security

CodeCraftConf by @codecraftuk

Follow @codecraftuk on twitter
CodeCraft logo

The tickets for CodeCraftConf are now on sale. It’s based around the idea of full participation, where every attendee is there to discuss the topics. The format is of guided conversations, which has worked well for sessions in the past, and I’ve included a list of some of my previous sessions using a similar format below for reference. I’ve used the mindmap format for sessions I’ve done in the past so that there is a solid summary that attendees can refer to if they wish, but I have also been involved in sessions at other conferences that were deliberately transient in order to foster honesty and controlled venting, more like a support group for frustrated developers.

I am looking to guide a session at the conference and will be talking to the organisers about the topics,. If you look at the list below, and the conditions on the conference site, it should be clear that this will be a philosophy of code conference rather than a discussion of all the hot technology you don’t know about yet, but really should. (and number 5 will change your life forever).

I realise that holding the conference in a pub, even one as good as Drygate, will challenge the training budgets of some companies, but it looks like a great concept, and I hope the conference takes off. Many thanks to Joe and Gary for bringing this together, and I look forward to meeting everyone there.

What you missed at DDDs

My previous chaired discussion sessions, with mindmaps for reference.