eZ Shows Quality Codebase in Technical Debt Evaluation
Recently, SensioLabs released a graph showing the technical debt of some relevant PHP projects. The graph included eZ Publish as well as some of our peers in the CMS space: Wordpress, Drupal and Joomla. It has stirred up a lot of discussions on the internet, and for good reason, so I thought I'd share my not-so-technical point of view here!
eZ Publish ranks better than Drupal, Joomla!, and Wordpress in technical debt, according to SensioLabs.
What is Technical Debt and Why You Should Care?
The first thing we need to do if we want to discuss this graph seriously, is to understand what it's about (i.e. technical debt) and how the service from SensioLabsInsight measures it.
Technical debt is a new concept which is well explained by Wikipedia. The concept is rather simple. Flaws in your codebase create a "debt" that must be repaid. Over time your "debt" collects interest, which makes it more difficult and time consuming to repay. Improperly finished work in a codebase, as well as in an architecture, continue to add up as the software evolves and is maintained. This inevitably increases the maintenance work. And so, high technical debt equals high maintenance cost.
High technical debt can also create problems for the developers that use the software to install, integrate, or customize it -- even if they don't maintain it. For anyone implementing the system, a low technical debt is valuable because it means a cleaner and better codebase architecture.
From this, you can understand why technical debt is a primary concern for the people building and maintaining a product -- the eZ Systems team and the eZ community in our case.
For end users, technical debt is not as much of a concern because they rarely interact with the codebase.
eZ Publish's High Score and What It Means
In the graph created by SensioLabsInsight, eZ Publish 5 scores well. eZ Publish's technical debt, according to SensioLabsInsight is evaluated at one year. Others in our industry include: Drupal 8 with a technical debt of 3.7 years, Joomla with 7.1 years, and Wordpress with more than 20 years!
On top of this is the fact that technical debt should increase with the size of the codebase. This makes the technical debt evaluation even more positive for eZ Publish. Indeed, eZ Publish is a bit of a different beast, offering by far the widest scope with a much larger code base.
You can conclude that eZ Publish's intrinsic quality is way better than most other projects.
So what does this really mean?
Like any metric, you need to carefully read the hypothesis and understand what is actually being measured before drawing a real conclusion. In the sea of discussions that followed the publication of this graph, many quickly jumped to the wrong conclusions, including one which implies the graph is biased because it's based on a service provided by SensioLabsInsight. I truly don't see why SensioLabsInsight would favor eZ Systems against others like Drupal!
How Does SensioLabsInsight Measure Technical Debt?
To fully dive in here, you really need technical skills. I've probably forgotten too much of my developer classes to write about this in detail. But one good thing is that Sensio is extremely clear on how they've arrived at their results.
Sensio tests the codebase against a list of checkpoints ranked differently depending on importance. Each issue they encounter has an estimated "time to fix." Checkpoints range from important security issues such as, "the website should be protected against XSS vulnerability," to minor issues like, "default favicon should be changed."
Unfortunately, the audit details aren't totally open and you need to be a subscriber on a paid plan to have access to the details of each issues. If you want to know more, visit SensioLabInsights's website to find out exactly what they analyze.
Code Quality Vs Product Quality
We've covered the eZ Publish high score, and we've covered the meaning of technical debt and how it's measured by Sensio in this case. But does having a lot of technical debt as measured by SensioLabsInsight really equate to a poor product?
While it definitely is an indicator of the code quality from a technical perspective, in my opinion, it isn't a direct indicator of the overall product quality, even if it indirectly affects it. However, people shouldn't be misguided here.
The best written code can always produce the wrong behaviors which will result in mediocre product quality. On the other end, the most terrible code can be lucky and produce the right behavior for a system, resulting in good product quality, despite its difficulty to maintain for the software maker.
So, to really measure technical debt, I believe we should go a bit further than what SensioLabsInsight measures. Hazardous software behaviors, even if well coded, can accumulate debt too. These flaws can become exponentially more expensive as the user interfaces and APIs grow in complexity. Measurement of this part is of course very difficult, even with the recent advancements in areas such as Behavior Driven Development (BDD).
At the end of the day, the insights brought by this analysis are not much of a surprise and tells us something that most of eZ Publish developers already knew. Compared to most of the other PHP projects available, eZ Publish 3, 4 and now 5 have been developed with high standards and strong attention to code quality and architecture, closer to other enterprise-grade software developed on top of the Java or .NET platforms. This is a comment that I have heard hundreds of times from partners and customers.
It's nice to be reminded of this, and even better to see that this is improving with the eZ Publish Platform 5.x since the adoption of Symfony2 as our framework. Now, I'll let you decide what it means for you and your use of eZ Publish and if it makes it a better solution or not, it definitely makes it better for us!