Categories
ai artificialintelligence code development

Learning to think in code

After I posted my thinking in code post, I had an interesting discussion over on my Google+ page.

It’s a difficult mindset to cultivate. When you get it and are in the zone, it’s effortless in a way that is hard to explain. It basically feels no different than writing in proper English.
When you don’t have it, it’s like trying to speak in a different language. Which makes sense, but it isn’t quite exact.

I’ve been coding for so long, thinking in code is second nature, but I liken it to when I learned to drive. At first I was very bad, I made a lot of mistakes and it really knocked my confidence, but I stuck at it because I lived in the country and had no choice, I was still nervous after passing my test, and it took about another 2 years before I got to the point where I wasn’t consciously thinking about the mechanics of driving, the meaning of each marking and road sign, and how to react to different conditions.

A lot of people I’ve mentored get scared when they lose confidence, but that’s an important step, and it always takes a long time to get past the conscious thinking stage, where they’re switching between the customer domain and the coding domain rather than thinking in the customer domain and writing in the coding domain.

Switching to Haskell got me back to newb mode again. It’s less like programming and more like magic. I can see how it works, and it does things as expected, but intuition tells me it shouldn’t work.

That’s how I felt when I started with node.js – first time I’ve really dug into the guts of JavaScript. I’ve done first-order functions and callbacks in other languages, but the way JavaScript does it, compared to C# or Python, forces me to think about what I’m doing.

I know some of the problems are addressed in ES6, but prototypes rather than the OO I’ve used elsewhere is a massive shift for me, and I definitely don’t think in JavaScript yet.

https://plus.google.com/+CraigNicolGeek/posts/dPLRZFbHsn1

And then I read Code rant: Learn To Code, It’s Harder Than You Think, and it got me thinking about the AI Gods, and how unnatural it is to think in code. And that’s important. When you talk to users and customers, and when you talk to managers, they look at all the information saying coding is easy and then ask why it’s taking so long, or why you can’t just put that number there, and then switch off when you try to explain. Indeed, you can end up in a meeting like this one:

The expert : https://www.youtube.com/watch?v=BKorP55Aqvg

So it’s important for people to understand that code is hard, and it is a specialist skill, whilst still trying to encourage people to take up software, because we need a new generation of AI gods, and they need the time, and the inclination to grok coding.

I grew up in the 8-bit days, with the MSX, and coding was an integrated part of the experience. The command line was the programming language, and all the magazines has type-ins, so there was a lot of opportunity to practice typing, as well as coding, which is a valuable subsidiary skill. There’s a similar modern experience with the Raspberry Pi, but the industry as a whole has moved on so much, that the days of spending a few hours typing to get something that has the same surface quality as the games in the shops, are mainly behind us, at least as far as the big box games, rather than the casual field. And it will be about games, because who is going to write a tax auditing application for fun?

How do I learn?

So if you’re just starting out, and you want to write software, or you want to be the next AI God, what do you need?

Here’s what I think is most important:

  • Teach yourself. Attend classes if it helps, but if you’re not self-motivated to learn and keep up to date, you will quickly fall behind.
  • Crave simplicity. Challenge yourself to cut complexity, to streamline. The less code you have to write, the easier it is to work with. Always seek new ways to simplify what you do.
  • Practice the basics. Use the many programming exercises to capture branching, modularisation, testing. Find examples you like, either smaller exercises, or complex Mandelbrot calculations and keep reworking them to understand new languages, or to challenge yourself to understand that basics. Can you write a program without if statements, or can you throw away the tests and rewrite the code?
  • Practice often. Spend 20 hours writing code. Then reflect, understand what you know, and write more. The more you write, the more you’ll understand what you don’t know.
  • Read code. And think about it. Just because it exists, doesn’t necessarily mean it’s good, but think about how you would solve the same problem. Understand what decisions were made and why.
  • Understand principles deeply, technologies less so. Know what it means to be Object Oriented, or Event Driven, or Functional. Understand what they are and how to use them. It doesn’t matter whether you’re using C# or Python or VB or Java or Haskell or The Next Big Thing. If you understand the principles, you can apply them with whatever syntax your current language provides, and you can understand what to use when.
  • Think in code, but talk in human. If you can’t explain what you’re doing to your users or your manager, you don’t understand it enough. Crave simplicity in your communication as well as your code.
  • Talk publicly. Present, blog, chat. Find a local user group and connect. If you’re enthusiastic, you’ll be welcome. If you’re shy, everyone will understand. We’ve all been there.
  • Simplicity doesn’t mean brevity. Be as brief as possible, but don’t lose context, or nuance. Be clear.

For those of you with experience, what else would you add?

Categories
code development programming

Dynamic language and semantics

Given the opportunity to revisit my Genetic Algorithms library, I wanted to revisit my open letter to Jon Skeet, about semantics and dynamic languages, and compare how the code looks in C++ vs Python. I accept that C++ is an overly verbose language but the problem I want to discuss is similar in C# and Java. I’m just going to pick a few examples, but feel free to explore the code yourself, either as C++ templates or as Python examples. These examples all expect a certain knowledge of Genetic Algorithms, so I’d recommend reviewing my previous post first if you are unfamiliar with the concepts.

If you’re trying to compare them directly from the repositories, please note that I have removed some comments and logging, and changed the formatting in this post for reasons of clarity, and have yet to apply those changes back to the repository. I have also added some functionality to the python code that doesn’t currently exist in the repository to ensure the examples are comparable.

Fitness

In C++

  /***********************************************************************
   *            SAMPLE FITNESS FUNCTIONS                                 *
   ***********************************************************************/

  template<class _Ctype, int _Csize>
  struct fitness_base {
    virtual double operator()(_Ctype chrom[_Csize]) = 0;
  } ;

  template<class _Ctype, int _Csize>
  struct maxones : public fitness_base<_Ctype,_Csize>{
    double operator()(_Ctype chrom[_Csize]) {
      return std::count(chrom, chrom+_Csize, 1);    
    };
  } ;

The fitness function for a chromosome returns a number that is used to rank the chromosome such that fitter solutions are more likely to survive and breed. For the example below, note that by default, the chromosome is an array of type _Ctype (default: bool), of length _CSize (default: 32)

The fitness function must be passed as a parameter to the constructor so that different problems can be explored using the same representation. This requires a type for fitness functions fitness_base which is defined as virtual to ensure it can be overridden, and contains an () operator so it can be called. In C#, this would be a delegate type, and we would be able to use the function/action types from the library as the base class for maxones. The fitness function shown simply counts the number of ones in the chromosome.

In Python

    def fitness(self):
        return sum(self.genome) 

Functions are first-class objects in Python, so we don’t need anything special to pass them as arguments, or require overloading the () operator. However, Python also allows patching classes post-initialisation, so we can overwrite the fitness function, or we can use Python’s multiple inheritance mechanism to subclass 2 different versions of chromosome, one of which overrides the fitness function, and the other overriding the crossover function.

Note that the fitness function is different between the two fitness functions is simple, but I now prefer the cleanliness of sum to a conditional count, but the overall character of the code will not be altered by the 2-character difference.

Run one generation

In C++

    void run_once_replace() { // one generation = psize crossovers
      pop_t newpop;
      newpop.reserve(_population_size);

      for(int i = 0; i < _population_size; ++i) {
          chrom_t child = select_chromosome() + select_chromosome();
          newpop.push_back(child);
      }

      copy(newpop.begin(),newpop.end(),_population.begin());

    };

In Python

    def run_once(self):
        "Run one iteration of the genetic algorithm"
        # Use replacement algorithm
        new_pop = []
        for i in range(len(self.pop)):
            a, b = self.select_two_members()
            new_pop.append(a + b)
        self.pop = new_pop

In these examples, apart from some small syntactic sugar that allows us to return 2 values from a function, the C++ and Python code are just as clear as each other.

Conclusion

In contrast to my letter, there isn’t as big a difference as I believed when I originally wrote the letter, at least in terms of the resultant code, although I remember the pain trying to get the types lining up properly in the first place. Semantics still matter, and types definitely got in the way designing the chromosomes, but actually, outside some specific cases, with well structured code, static and dynamic languages can both make semantic meaning clear.

Categories
ai artificialintelligence c++ code development programming search

Updated slides on Genetic Algorithms

I had the opportunity at work to revisit my Generic Algorithm talk, to refactor it with a bit more time to hopefully make it clearer. I also ported the C++ template code to Python to make it easier to demo. I’ll be talking about the implementation differences in a future post but I’ve included the links for the talk below for public consumption.

Categories
ai artificialintelligence code programming

Why AI doesn’t scare me

Tripod Machine
The chances of anything coming from Mars are a million to one…

You may have heard about a letter going about where a lot of people say they’re scared of AI and autonomous weapons. There are some very interesting signatories on the list. As someone who studied AI, and knows a couple of people on that list from my course, the letter interested me greatly.

I’ll discuss the implications of autonomous weapons in a future post, but a lot of the reporting and comments about the letter talked about fear of AI itself. As if smarter machines in and of themselves are the greater threat to humanity, without considering whether or not these smarter machines are designed to kill. This fear appears to be unrelated to whether the machines are smarter than us, just about being smart enough to be a threat (although I want to discuss the singularity in a later post too).

Perhaps for a small minority, the fear comes from the same place as fear of other humans. The ignorance breeding fear that someone or something smarter or different from you will be threat to your way of life, or your life itself. There’s legitimate fears within that, linked back through history to the luddites whose jobs were at risk from automated machines.

It’s clear that jobs will replace humans with machines, and the better machines get, the further up the pay scale the threat will come, just as globalization pushed low paid jobs to other countries. It’s a real problem that needs to be addressed politically via training, diversification of the economy, and other tactics, but I don’t want this to be a political blog, so decide for yourself how jobs can be created.

If your fear comes from not being the smartest person in the room – get over yourself. Success builds on success, and humanity’s greatest achievements have come from those who stand on the shoulders of giants. These days, these may be iron giants. AI that is smarter than us already exists, machines are better than us at Chess, Jeopardy, code breaking, and calculating Sine tables.

AI is already all around us – it does fraud detection, voice recognition and a bunch of other things – Once it works, it’s not AI any more (as .Net Rocks reiterates), and it will get better.

If you want to know more, the following article and video are definitely worth a look.

<How safe is AI : http://www.bbc.co.uk/news/science-environment-34249500 >

<How smart is today’s AI : https://www.youtube.com/watch?v=poLZqn2_dv4 >

Categories
ai artificialintelligence development

AI and I

My degree was in Computer Science and Artificial Intelligence. Unfortunately, AI was a tough field to get a job in when I graduated, so I concentrated on the Computer Science part in the main. However, AI still fascinates me, and a lot of the “Big Data” movement is actually appropriating techniques from AI to filter and transform data, because of the fields in AI that study pattern recognition, feedback and learning, and application of structure onto data (think classifying pictures by their content, or detecting spam from a collection of email).

There’s a few news stories recently that I want to talk about over a series of posts, but first of all, I want to direct folk to this nice summary from Wired, which perfectly demonstrates an old truth in AI – that AI is the stuff we haven’t figured out yet. Once we understand it, it becomes Computer Science. The Three Breakthroughs That Have Finally Unleashed AI on the World | WIRED

The article, is some ways, is many years too late, fraud detection systems are almost as old as credit cards, we’ve had spam filters for years, most humans are easily defeated by chess computers. Many of the things we used to hold as either evidence for intelligence, or super hard problems for computers to solve, have been solved. Those thresholds have passed, and our world is building of smart, learning systems, adapting to you, learning more about each of us to provide more tailored experiences, either for our benefit, or for the benefit of marketers.

There’s still plenty to learn, and it’s still an exciting field, but we’re no longer in a world where AI is “just around the corner”. Computers are smarter than us in many ways. General intelligence may still be unsolved, but practical intelligence is now part of the fabric of our lives. AI is not something to be scared of. It’s here, and it’s helping us. But like any technology, the ethical concerns are still a matter to be discussed.

Categories
development

Google Code Migration : Genetic Algorithm Templates

With the closure of Google Code, I’ve moved some projects to github. All personal projects so far, but related to talks our blog posts from the past, so may still be of interest.

The first project I want to highlight again is written in C++ and implements genetic algorithms using mainly C++ templates, just to see how powerful they were. It taught me a lot about generic code, and how a poor type system can interfere with the clarity of your code. It also prepared me for one of my first talks, about Genetic Algorithms at a Beauty of Code techmeetup.

I’d like to look at a Python port, to see if my expectations of using dynamic typing would answer the concerns I have about code clarity. For now though, it’s available for reference. It’s not production tested, and there are parts that are embarrassing, but it might be interesting if you want to know what genetic algorithms might look like.

Categories
code programming

Genetic Algorithm Templates

I had a great time at the Tech MeetUp (@techmeetup) in Glasgow this week. If you want to meet up with other tech minded folk in Scotland, it’s definitely worth checking out. See the details at the end of this post.

At the meeting this week, I got talking to a few folk (including John Gallagher who gave a great talk) about AI and Genetic Algorithms. This inspired me to dig out my old experimental code for doing GAs using C++ Templates. The code’s very rough and ready, and it was written long before I discovered TDD or subversion, but it should be standards-compliant, and it compiles in the latest GCC (I can’t vouch for other compilers, if you try it, let me know). As it’s a source-code library, you’ll have to compile it yourself, but if you don’t know how to use a compiler, you’re probably reading the wrong blog anyway. I will be tidying up the code and creating a wish list as I get the chance, but I’m throwing it out there in case it’s useful to anyone.

If you’re interested, check it out at the link below and let me know what you think.

Genetic Algorithm Templates (edit : link moved to github)

If you’re interested in Tech MeetUp, you can see videos from previous talks at their website

The Tech MeetUp is the informal opportunity to meet other developers and tech companies, to showcase your hacks or projects, and to find out what’s happening around us. Help build the tech community – set up a profile and come along to a Tech MeetUp.

Home – Techmeetup

You can find out about upcoming meetings on their mailing list

Description: An easily accessible and friendly community of tech minds, skills and startups around Scotland and Northern UK.


Tech Meetup | Google Groups

Blogged with the Flock Browser
Categories
code programming search

Genetic Algorithm Templates

I had a great time at the Tech MeetUp (@techmeetup) in Glasgow this week. If you want to meet up with other tech minded folk in Scotland, it’s definitely worth checking out. See the details at the end of this post.

At the meeting this week, I got talking to a few folk (including John Gallagher who gave a great talk) about AI and Genetic Algorithms. This inspired me to dig out my old experimental code for doing GAs using C++ Templates. The code’s very rough and ready, and it was written long before I discovered TDD or subversion, but it should be standards-compliant, and it compiles in the latest GCC (I can’t vouch for other compilers, if you try it, let me know). As it’s a source-code library, you’ll have to compile it yourself, but if you don’t know how to use a compiler, you’re probably reading the wrong blog anyway. I will be tidying up the code and creating a wish list as I get the chance, but I’m throwing it out there in case it’s useful to anyone.

If you’re interested, check it out at the link below and let me know what you think.

Genetic Algorithm Templates (edit : updated link to github)

If you’re interested in Tech MeetUp, you can see videos from previous talks at their website

The Tech MeetUp is the informal opportunity to meet other developers and tech companies, to showcase your hacks or projects, and to find out what’s happening around us. Help build the tech community – set up a profile and come along to a Tech MeetUp.

Home – Techmeetup

You can find out about upcoming meetings on their mailing list

Description: An easily accessible and friendly community of tech minds, skills and startups around Scotland and Northern UK.


Tech Meetup | Google Groups

Blogged with the Flock Browser