cloud data development security

Cloud is ephemeral

The Cloud is just someone else’s servers, or a portion thereof. Use the cloud because you want to scale quickly, only pay for what you use, and put someone else, with a global team, on the hook for recovering from outages. You’d also like a safety net, somewhere out there with the data you cannot afford to lose. But whatever is important to you, don’t keep it exclusively somewhere out of your control. Don’t keep your one copy “out there”. Back it up, replicate it. Put your configuration and infrastructure in source control. Distributed. Cloud thinking is about not relying on a machine. Eliminate Single Points of Failure, where you can, although there’s little you can do about a single domain name.

Understand your provider. Don’t let bad UI or configuration lose your data : Slack lost 800,000 messages.

Your cloud provider is a dependency. That makes it your responsibility.¬†Each will give you features you can’t get on your own. They give you an ecosystem you can’t get from your desktop, and a platform to collaborate with others. They give you federated logins, global backups and recovery, content delivery networks, load balancing on a vast scale. But if the worst happens, know how to recover. “It’s in the cloud” is not a disaster recovery strategy, just ask the GitLab customers (although well played to them on their honesty so the rest of us can learn). Have your own backup. And remember, it’s not a backup unless you’ve verified you can restore.

It takes you 60 seconds to deploy to your current provider. How long does it take to deploy if that service goes dark?

cloud code development programming

12 reasons why I love software development

1 Automation means fixing it once.

Although please note that fixing it once doesn’t mean always fixed. I’ve got a 20 year old Mandelbrot generator that still works but I’ve had to tweak as the C compiler evolved. The algorithm is the same, but I had to modify the implementation slightly as GCC changed how it handled the standard library.

2 Working with smart people.

If you’re scared of learning, this isn’t the job for you. Experienced people will show you solutions you’d never thought of, everyone will ask you questions that cause you to re-evaluate your decisions, and you’ll find yourself saying “I don’t know” more often than not. If you think you know everything, you haven’t even started.

3 Variety

Once a problem is fixed you move on to new domains and new problems. You need to grok new domains, learn new technology, analyse how you work and adapt.

4 Alternative perspective

Software development reaches into many aspects, and covers a variety of topics, so allows healthy discussions across disciplines. (commit strip cartoon?) How you see a process or a program is likely not how the users see it. Human processes are messy and full of holes. Sometimes we can fill them, sometimes we just have to let the users work around them.

5 Meritocracy

(Disclaimer: I realise I’m talking from a position of privilege, and this may not reflect everyone’s experience)

All the teams I’ve worked in are judged on their ability to deliver good software, not colour, gender, age, disability or otherwise. People are praised on their ability to deliver and to work worth others, and criticised when they fall short. I realise I only see a small proportion though, and I did work as a STEM ambassador and a youth club volunteer to help broaden the pool. There’s a lot of work to be done, but good teams benefit from the diversity, and recognise that. If your team doesn’t, fix it. And help get out there to fix it for all teams.

6 Pride in seeing our work being used.

I’ve worked on projects to distribute EU funds to projects around the country, to track prisoner learning to make sure they could get qualifications, to model factories and save companies millions, to help people navigate their bankruptcy, to help nurses and managers find inefficiencies in hospitals. The software I’ve worked on its being used by people for whom it makes a real difference. There’s few jobs that allow you to impact so many people, in so many different ways.

7 Multi-disciplinary

Software development touches on many disciplines, and successful software projects provide opportunities to discuss ethics, psychology, politics, law and other areas. There’s a lot of specialism, but if you want to understand your users, and understand security, and understand the requirements, you often need to dig in and uncover answers from many disciplines to truly understand what you’re building.

8 Satisfaction

It’s clear when a problem is solved. It might not be clear when a project is finished, but each step on the way should be a clear, this is Done. If you don’t have that on your team, define Done. That’s your route markers so you know you’re making progress.

9 Mentoring

It’s great to see others catch on. See them awaken to the fact that there’s so much more to learn, that software development is a rich, diverse discipline, and then see them run in a direction to become the expert you seek advice from in their chosen area.

10 Challenge

Sometimes the satisfaction from development comes from tackling hard problems with no easy answer. Knowing that no-one has solved it before. Land on your own moon.

11 Communication

There’s always someone interested in what you’re doing, and there’s multiple international communities. There’s websites and forums, user groups, conferences. Lots of opportunities to engage with others and learn and teach outwith your company circle.

12 Changing the world

Software has become the driver for most of our daily lives. Without it, there would be no smartphones, no Internet, no social media, no video calling, no Uber. If you want to change the world, first you must understand it.

cloud development

Cloud thinking : efficiency as a requirement

In the old world, you bought as big a machine as you could afford, and threw some code at it. If it could fit in memory and the disk I/O wasn’t a bottleneck, everything was golden.

In the cloud, however, CPU cycles and disk storage cost real money. Optimisation is key, so long as it’s not premature. Monitor it.

In cloud thinking, it’s less about the O(N), it’s relatively easy to scale to the input, as long as you’re not exponential. In the cloud, it’s about O($) – how well does your code scale to the amount of money you throw at your servers (or inversely, what’s the code increase per user).

Fixed costs are vanishingly small in the cloud, but incremental costs can change quickly, depending on your base platform. Not the provider, as costs between them are racing to the bottom, but the platform of your architecture.

Quite simply, the more control you ask for, the more you pay in time, and the bigger the ramp-up steps. Get metal, and you’ll pay for everything you don’t use, get a VM and you can scale quicker to match demand, get a container, get an Electric Beanstalk or an Azure website and give up the OS, get a Lambda or a Function.

I can’t recommend to you how much you should abstract. I don’t know how big your ops team are, or how much computation you need to do for each user. I suspect you might not know either. Stop optimising for things you don’t care about. Optimise for user experience and optimise for cost per user, and measure everything.