2016-06-10

!echo Bye-bye Python 2 !

In early 2013, we decided to port our chatbot project Errbot to Python 3. It was for its version 2.0.0beta.

I still remember that vividly, it was crazy at that time. First, absolutely nobody would use Errbot under Python 3. Finding compatible libraries was a gigantic pain not only for the plugin designers but also for ourselves: Check out this extract from the change log...

- xmpp backend has been replaced by sleekxmpp
- flask has been replaced by bottle (sorry flask no py3 support, no future)
- now the IRC backend uses the simpler python/irc package

We had to remove flask for rocket a small alternative and switch over the other 2 main dependencies we had for the 2 chat systems we supported. We barely wiggled through this ... and of course we broke stuff all over (I still have PTSD from the unicode breakages :) ).

2.0.0 stayed in beta for 10 months while users stayed safely on 1.7.1, a version from 2012.

We managed pretty well the next 3 years: the code was developed in Python 3 and translated automatically at install time to Python 2 with a tool called 3to2. The project grew in popularity and the general quality of the code, unit tests and documentation with it.

But something changed progressively, maintaining the Python 2 compatibility became more and more time consuming for not much benefit: simply less users were stuck with it. We could not find any reason for keeping the backward compatibility: all the main distributions have Python 3, the ecosystem moved on (even flask !), most plugins have the dual compatibility.

So today we pulled the plug, we are removing the Python 2 compatibility on master. We will maintain the backward compatibility for the latest version (4.2.x) until the end of the year while the new version will be Python 3.3+ only.

Some takeways

- It is quite impressive that, despite 3to2 making a very good job at converting the source code, a lot of hacks crept in over the years.  Just by browsing the diff, it is easy to see the jump in code readability.

- 3to2 was retrospectively the good choice: It allowed us to switch to the new Python version 3 years ago and today in just one long but very simple PR remove the support for the old one.

- 3 years ago it was definitely a little early to support Python 3 but today it is definitely the right time to drop Python 2.

If you have to port your project today, you are in luck ! You can drop Python 2 right away without all the complexity of maintaining a dual compatibility. Polish your test coverage, 2to3 in-place, fix it until it is done and enjoy the cleanliness and the new features of Python 3 without ever thinking again about Python 2.

2015-10-04

Errbot recipes

I presented at the last SF chatops meetup an introduction to errbot. I realized that the snippets used during the presentation could be a nice set of "recipes" that could get you started to use a specific feature. So here they are all in one page, enjoy !

Note: I tried to not repeat the files too much so you can use the .plug file from the first example for all the other ones.


2014-12-15

Little things I would teach to my younger software engineer self - #6 The "debugger trance"

Looking back at the various stages of my career, I realized I could have learned some basic stuff way earlier than I did.
Even if I missed the original opportunity, all those little skills I acquired late served me tremendously well ever since.

[credits to my colleague Eamonn McManus at Google for reminding me this one and cointing the term!]


1999-me: We have a memory corruption in this LDAP server implemented in C++, I'll figure it out with the debugger, I am sure, it is just a matter of time!

2014-me: A debugger is a really powerful tool but you need to use it only when it makes sense.




It gives you a very thin slice of what is happening in excruciating details. 

Don't get lost in there, often a log statement correctly placed after some general reflection about where the problem might comes from saves you a ton of time stepping through thousands of unrelated lines of code. It also makes you a better programmer because you'll train yourself to stay on top of those pieces of code growing in complexity.

And BTW, you'll never find this memory corruption, it was corrupting the stack and render your debugger useless after stepping through thousands of irrelevant lines, few days later you'll give up. Instead of that, you should have tried to understand the code, reproduce the bug with a reduced concurrency, dichotomize to focus on the problematic area and figure it out for real.

Any tips you would give to your younger self programmer ? Feel free to comment below !

Little things I would teach to my younger software engineer self - #5 Taking unit testing seriously

Looking back at the various stages of my career, I realized I could have learned some basic stuff way earlier than I did.
Even if I missed the original opportunity, all those little skills I acquired late served me tremendously well ever since.





2000-me: Common, I just need to be productive here: I code, try it a little bit, it looks good. Let's ship it!

2014-me: You are headed to a huuuge pain. After a regression during a bug fix for an initial regression you were just trying to fix for another bug on some horrible code you don't even dare to try to improve, you'll finally see the light and read:

Refactoring: Improving the Design of Existing Code by Martin Fowler

I had a huge code quality bump when I settled and consistently affirm that "Untested code is equivalent to broken code".

Initially, unit tests look like a huge waste of time but very quickly it will pay off:

1. It will give you some assurance that your code is at least working as intended. This will save you some exponential time on manual testing as you add complexity in the code base.

2. It will save you a huge deal of time when you gonna refactor or improve the code base.  The unit tests will tell you instantly what might have been impacted by your last change.

3. It will force you to write single-purpose modules or classes or functions. Once you start to bloat your code, you'll notice it quickly as it will become very hard to come up with a test for it.

Oh, and another tip before you go, when you'll see the light you might become a little bit over-enthusiastic about it. You'll see 100% of code coverage as the holy grail. But no, not really, the last 10% gonna be sooo painful to get and will not add anything quality wise. Test only ~80%-90% of the critical code you know is interesting to test.

Any tips you would give to your younger self programmer ? Feel free to comment below !