I’m currently working on a Drupal project, and it did well to remind me as to why I’ve switched most of my development over to Django some time ago. I have a comprehensive comparison in the works between these two frameworks and a few others as well, but for now I’d like to focus on just one thing and what it means for development in Drupal and Django: coder happiness.
Configuration over coding
Most people who have a career in software development actually, surprisingly, really like coding a lot. We code after work, perhaps hacking together a widget for personal use or contributing to an open-source project. A lot of coders write about coding as well.
One of the more frustrating things about Drupal is that most of your time spent hacking together a project does not go towards actually coding, but is spent configuring and tweaking settings in a graphical user interface. Another big lump of your time will go towards searching through the gargantuan directory listing of Drupal modules that now contains about 5.000 of those, to see if any of these modules do something you need. And the coding that does happen is split 50/50 between actually writing code yourself and getting familiar with the code of the existing modules that you’ve just installed, because you’re not happy with how this or that works and need to patch things up.
There are tremendous benefits to the reuse of existing software. The Not Invented Here syndrome is an ailment that has crippled many a good developer. It led Steve Yelvington to coin the First Rule of Coding for Drupal, namely: “We do not write code for Drupal.”
Any code you or someone else on your team writes has to be documented and maintained, and can break in unexpected ways. Coding things yourself has subtle hidden costs. So Steve raises a valid point and it’s an issue that I hope to write about in the foreseeable future. But this post is about coder happiness. And, being a coder, having my work reduced to that of an administrative clerk, searching for and configuring modules, just sucks.
Yelvington is in the news biz, and he talks about how building news websites in Drupal actually goes about:
configuring and tailoring the platform to the needs of a news site is a huge creative challenge that involves fairly little in the way of heavy technology (i.e., developing new modules) but a lot of what I regard as “configuration” work, including interface design and theming.
The description of the work is accurate. But I disagree with his evaluation. There’s nothing creative about configuring a Drupal site at all. It offers very little satisfaction. It requires a lot of experience but hardly any skill.
Take, for example, how you access your database in Drupal. You use the Views module. It’s incredibly flexible. It also means that instead of writing a simple line of Django ORM code like
and processing that in your view with
you get to fill out a gigantic form that generates your view, after which you’ll have to fill out a similarly large form in the Panels module to get that list displayed where you want it to. And if want to customize the html that Views produces (because, being geared towards end-users, Views is both a query builder and an html generator) you’ll have to override a bunch of theme files as well.
Could you give me the latest comments?" — "Sure, just fill in this small form, file it in triplicate, and we'll get back to you."
Easier for them is harder for me
Configuration over coding sucks. I’ll take Django’s approach any day. It doesn’t work for non-technical users, true enough. You can expect a non-technical user to complete a form but you can’t expect him or her to handle an object-relational mapper. However, I’m not a non-technical user and I don’t want to be treated like one. Easier for them is harder for me.
Dries Buytaert, the project lead for Drupal, is adamant about lowering the barriers to participation on the web for non-techies. That’s really cool. But I can write code perfectly well, thank you, so that’s just not a selling point for me.
The developers behind Drupal understand that user experience (UX) and developer experience (DX) are two different things and that they can work against each other. But because they have to serve both communities, they have to compromise. Django is geared towards developers and developers only, so it can skip all these delicate discussions about how to balance the needs of developers and those of end-users and just do whatever makes the framework more productive or comfortable for coders. And it shows.
The Sharpened Knife
Clicking in menus isn’t a very satisfying way to spend your day, but that’s not the only reason why I dislike it. Coding-by-configuring takes you away from the skills that make you so valuable as a coder. You’re not advancing as a developer because you’re not learning any skills that remain valuable outside of the CMS.
I often felt that I was coding in Drupal, not in PHP. Drupal has modules and wrappers for just about anything. You can’t fault Drupal for providing so many conveniences, but it does mean that you never come into contact with e.g. the plain Google Maps API, which means that you can’t transfer that experience to other languages or frameworks.
After working with Drupal for more than three years, I started to feel that I was handicapping myself.
In Django, you’re mostly writing plain Python and using general Python libraries. Django does a lot of work for you, but it’s nonetheless fairly compact. The four-part introductory tutorial teaches about half there is to coding in Django, and can be completed in a few hours. That gives me a warm and fuzzy feeling. If I ever feel the inclination to try out, say, Ruby on Rails or Symfony, I know I have the basics down of MVC-style frameworks and can re-use my knowledge of external API’s, so I shouldn’t find it hard to adapt.
It goes deeper than just skills. The Python and Django communities talk about programming techniques, about continuous integration, about agile methodology. Contrast that with Newspapers on Drupal, a group where I used to spend a lot of time, where almost no best practices are being shared or discussions being held about the future of online news, but rather is filled with questions about what modules to use for this or that.
Building rather than modifying
Because Drupal is still at heart a CMS (although it’s closer to a framework than most other CMSes), it’s all-encompassing. In Django you build things, which is easy. In Drupal you modify how the existing code works, so you need a good working knowledge of what can be modified and how to modify it. There are hooks, theme layer overrides, helper functions, existing modules that you can leverage and so on. Getting familiar with Drupal is not easy.
Coding in Drupal, using hooks, feels somewhat similar to aspect-oriented programming, and the same benefits and caveats apply.
If you haven’t tried out aspect-oriented programming techniques before, you should. It’s very doable in Python with some metaprogramming, and it’s a fun exercise. If you have, you know that it’s wonderfully flexible and that you can modify the behavior of your code in any way that you could possibly want. Kudos to the Drupal devs for pulling that off in a crummy language like PHP.
The trouble, though: if you’re not careful with aspect-oriented coding, the flow of code becomes unbearably opaque. While it does not often come to that point in Drupal, it does make for code that is tougher to write and maintain than it would be if you could simply change how Drupal behaves e.g. by subclassing, as you would in Django.
Regardless of which is more productive, the ‘construction’ approach just feels better to me. It’s not that big of an issue when what you need is close to what Drupal provides out of the box or through modules, but the more custom development you have to do, the more cumbersome tweaking existing behavior becomes, and the more it starts to feel like you’re working around Drupal rather than with Drupal.
Frameworks should enable you to get to the cool stuff quicker, not after you’ve worked around a bunch of default behavior that is sensible for a plain CMS but not so much for the kind of websites people have come to expect in 2010.
Documentation
Because of the added complexity of steering an existing system in the direction you want versus just building something, you’d expect that Drupal would be very well documented. Some parts of it are. The docs for the basic API’s are pretty good. But those docs only work if you know full well what you want and what you’re doing.
Drupal lacks documentation on how all of its API’s fit together, which is what makes the learning curve so tough on developers new to Drupal. And, as you’d expect, the documentation for user-contributed modules (which do most of the work for you) is mostly non-existent.
Contrast that with Django, which has a simpler architecture to begin with — it’s a framework, not a CMS — and has really good documentation on top of that. Django’s documentation is made up of step-by-step tutorials, topical guides and low-level reference material (cf. what Jacob Kaplan-Moss has to say about that) so people new to Django can get up to speed quickly, and gradually delve deeper without facing a wall.
Good documentation is probably the biggest contributor to programmer happiness. If the docs are good, you spend more time coding and less time scouring the web. Good docs feel empowering. Bad docs aren’t just frustrating, they also make you feel like an idiot.
Drupal made me feel like an idiot, Django doesn’t. I’m happy to have made the switch.
Stay tuned for a (shorter) second part about coder happiness in Drupal and Django, which looks at the issue from a bigger perspective: how running a project in both systems feels like.
10 Comments
What amazes me most about stuff like this is how little some communities learn from each other. A very good case in point with this is that much of what you describe above has been covered by the Zope and Plone communities.
Zope has been around for over a decade now. When it came out it was pretty revolutionary. It was an entire web app stack (as opposed to just using CGI scripts) it had templating, persistence, an object request broker, etc long before these things became common vocabulary. You think NoSQL is something new and trendy? Zope has been using it since the late 90s.
In the early days of Zope, you developed most of your ‘applications’ through the web with a series of clicks and a bit of restricted python coding in an HTML textarea. This in itself was also revolutionary, as it meant that distributed teams could all work together via the web on an actual application. It meant that non-developers could pick it up and assemble applications themselves. They might not have been as elegant or refined as those developed by an actual developer, but they got the job done.
After a while though, it became clear that whilst this was great, developers wanted to use their own editors (not edit via a web browser), and they wanted to control the changes in a source control system. They needed a way to do repeatable deployments and run unit tests against code. This was all very difficult with all your code entered via clicks in a web browser, and thus started the tension between the UX and DX you refer to above.
A few years later, along came the CMF (content management framework) on top of Zope to provide certain common content management tools. And on top of that eventually came Plone. With Plone 1.0 most configuration and development was done in the same way as with Zope via clicks and entering code in the browser. With Plone 2.0 much of that started to move to the filesystem, and the bias between simple ‘customerizers’ or ‘integrators’ and ‘developers’ swung towards the latter. The barrier to entry was raised higher and in order to get started you needed to learn a whole lot more.
Fast forward to the current time, and with Plone 4.0 on the horizon the bias has been centering. The codebase has become a lot simpler, and tools such as buildout make repeatable deployments much easier. Dexterity the new content type system will finally realise the nirvana of allowing a ‘power user’ to create new content types via the web (ala Drupal’s CCK) yet serialise them to the filesystem such that ‘developers’ can take them on and developer further functionality without having to start from scratch at a lower abstraction layer. Whilst still retaining the ability for the ‘power user’ to still get at them via the web.
So these struggles have been going on in other communities for a while now. I just hope some other systems can learn from the mistakes that others have made along the way to getting where they are.
Good point, thanks for the insight. I often think about the advice given by Andrew Hunt and David Thomas in The Pragmatic Programmer: learn a new programming language every year. Their point is really that you should add as many tools to your belt as possible, but I think the bigger win comes from participating in different communities and seeing how they approach and solve problems. Even within a single language there are a lot of islands with very little cross-pollination, e.g. very few people in the Django crowd (including myself) know anything about the Zope/Plone-world and even less about what the TurboGears devs are doing.
With regards to Drupal: they’re doing something similar to Dexterity as you describe it. The effort is being spearheaded by some very bright fellows over at Development Seed. They call it Exportables and Features.
It’s a pretty neat system which solves the deployment/SCM-problem, but it is still not very convenient for coders — the serialized files are pretty opaque and not really meant to be edited directly by hand, so you’re still stuck with cumbersome GUIs and wizards. I’m past the point where I’m willing to make that compromise
To be clear, Plone has had serialisation of settings for a while now as “Generic Setup”, usually in rather cumbersome XML files. You can edit/create them by hand, but you really need to know what you’re doing. The main way people get them is by clicking around in the admin interface and hitting export.
The exciting new thing about dexterity is the transparent round-tripping. So, for example, a colleague could create a new content type through the web, and let me know some things need changing. I’d export that into my codebase, add some python to handle those change requests and commit. Then, when the changes get pushed back up-stream my Python code starts to get used.
The upshot of all this is that you as a developer treat the exported information about the content type as a superclass without having to worry about those generated files. The people making things through the web get extra options as you code new things in Python so they can combine them and modify things themselves.
There’ll always be edge cases when you need to modify an exported file but I think the vast majority of people will have no need to. I actually tried out a pre-alpha version of dexterity in production (no, I’m not crazy) the ability to pick and choose if I wanted to click around or write code for a particular thing was very liberating.
You make some excellent points and you’re right, not building for and with reusable code is more fun but really it depends who you are and what you are building.
What I’ve learnt about CMS vs lightweight frameworks (LWF) is:
1. Building for reuse is more indirect and less fun.
2. Integrating is harder than coding since it’s less likely to fit in your head.
3. Reuse is very productive, secure and maintainable long term
4. If you’re building an app it’s highly custom and one off so LWF is the way to go
5. If you build sites for a living then CMS is worth the learning hit to gaim reuse advantages. It won’t be as fun but you and your customers will be better off.
I choose Plone because I build sites for a living, not apps.
Question is, what combination of LWF and CMS give you the most bang for your learning buck? Can a framework make reuse fun?
If you can code in Plone using Five you can code in BlueBream, Grok and BFG without too much trouble but I’ve noticed that with every simplification (like BFG) you lose reusability.
Great article! While what you describe is obviously very specific to Drupal vs. Django, I think there is a related — and interesting — generalization to be made. I find that typically PHP code is much more about writing framework code than actual “PHP” code. My personal explanation is that PHP typically doesn’t provide the language features that developers need for large-scale projects, and so framework developers add in their own layers. There are certainly some very extreme examples where frameworks re-implement everything including arrays & strings into more palatable APIs, but I think even a modest framework has to take this upon itself. For example, the fact that PHP has no uniform error handling system means that you’ll need to tailor your code to the framework you’re using (e.g. returning PEAR_Error if you must … or throwing some sanctioned Exception subclasses, re-wrapping non-Exception errors, etc.). Similar challenges exist for namespacing/packages, DB access (helped by PDO), and a host of other things that other communities have solved much more thoroughly.
Certainly Python frameworks have their quirks too, but on the whole I would say that writing code for Pylons (which is what I tend to do) is really just Python code. Despite the fact that zope.interface does get used with some frequency, I do believe that Python really isn’t crying out to be supplemented.
@Matt Wilkes: yes, you did a far better job of explaining the round-tripping of Dexterity than I did
@Dylan: I think this is something that people in the python community need to be more aware of. I think most of the non-zope python community still see Zope as something archaic and alien. Its true that would be a fair assessment 6-8 years ago, back when there were no trails to follow, so Zope has to blaze its own. The fact I can switch between Plone and grok/BFG is great. I can choose the level of framework I need and use the same technology and concepts.
The Zope community has given a lot back to the general python community (Zope Corp was one of the two founding sponsors of the PSF for example) and now pretty much everything in Zope is usable outside of Zope.
Europython is coming up soon, and I really want to get more people in the python community aware of some of the great stuff that has come out of the Zope world that they can use e.g.:
Zope Page Templates
Zope Component Architecture
ZODB
zc.buildout (why does Django not use buildout by default?!)
Using just those first 3 above you along with a WSGI stack you can create something very powerful very quickly (hence success of BFG).
-Matt
Excellent notes about a very important software engineering subject. I think it can be also presented as using library versus using a framework. Drupal is a framework – it controls the whole flow and only gives you possibility to “call in” in places defined by an author of the framework. Django is a library – you use it as you like, you call pieces of it which are helpful to define your solution.
When I look back at my programming interests then I see fascination for frameworks (mainly Java) in a beginning and a middle of my career. After years I avoid frameworks where possible.
There is a danger here of missing the comparison:
- drupal is a cms that has become a framework
- django is a web framework that allows you to build a content management system
- plone is a content management system (and a lot more!) built on top of a web framework zope.
Each has it’s place, but the talent is in the choices, i.e. what technology for what domain of problem. For me, django fits my set of problems very well. The aspect to think about is the language issue: php vs python and the inherent programming differences. Python IMHO is a far nicer language than php, and thus increases my coder happiness.
@Dan: I think it’s a fair comparison though, because in the end they’re all tools to build websites, and it’s not always obvious which tool would serve which kind of project best — as you say, the talent is in the choices.
A few months ago there was a pretty good Drupal versus Django comparison over at Scot Hacker’s blog. He made the case for using WordPress for modest sites and Django for anything advanced. That’s the path I’ve taken myself (plus a bit of Werkzeug now and then). Every tool has its place, but Drupal sits somewhat uncomfortably between the CMS and the framework space, and because it doesn’t excel at either, Drupal unfortunately has a rather narrow band of use-cases where it really shines.
About Python versus PHP: it sure helps
By the way, read this: “It’s really all about Phase 1; and even if Phase 2 takes more effort and time, it’s still only the ‘i’s that we dot and the ‘t’s that we cross to give our beautiful code a life outside of our own computers. And the problem with modern software development is that it’s all Phase 2.”
One Trackback
Social comments and analytics for this post…
This post was mentioned on Twitter by emergetec: Interesting #drupal vs #django comparison. Guess most large systems would have the same issues #plone http://bit.ly/dcSckn...