Learning About SPL Can Be Fun – At Least in Japan

If you want to refresh both your Japanese and your knowledge about SPL, have a look at this nice movie. If you need more, check out the related web page. If your Japanese skills (like mine) are rather limited, there is also an English variant of this page and movie. The eXmotion guys behind this training have been working in the field of product line development for quite a while, they are using pure::variants , and have some nice toys to play with in this training. So even I would like to book this training if only my Japanese would be better .

Want To Talk About (Your) Product Line Experiences? Check out Practical Product Lines 2009!

After returning from SPLC 2009 (August 24-28,  in the exciting city of San Francisco) there is one more product line event to consider:  Practical Product Lines (October 20-21, in the equally interesting city of Amsterdam). While the former event is well know for bringing together researchers and practitioners to learn from each other, the later is more oriented towards exchange between industry practitioners and experts, similar to the very successful “Code Generation” series of events. At this point in time you can involve yourself by proposing talks, workshops etc. Different formats are possible, just have a look at the call for speakers.

Yes, this reads like an advertisement. I feel the time is right for this kind of events as companion to the SPLC. And I admit, that after being to the Code Generation events twice I (and presumably not only I) asked Mark of Software Acumen, if he would be interested in organizing something for the product line community. It seems, he is .

Knowing where you are: Product Relation Patterns

When the time comes and you feel adventurous, you may want to migrate from your current development approach to a (software) product line approach. There is a lot to reflect on before and while migrating. A few simple checks can tell you a lot in terms of where you stand and what your migration options are. This article will concentrate on discussing product relation patterns and their influence on product line migration approach decisions.

The Product Forest

Your products are developed mostly independent from each other but have a set of functionalities they share. The problem domain of these products is also very similar, for instance all products are intended to control air conditioning systems (in different ways).  In this case your products form what I call a Product Forest. Sharing of basic libraries like UI widgets or mathematical function collections (anything which does not contain shared application logic) might happen already as a basic form of reuse. This kind of pattern is often seen when a company runs several projects for different clients in a similar domain  independently and concurrently. In the figure below you see how the products (P₀ to P₃) appear over time and how they evolve due to changes or bug fixes (P₀’-P₁’’’).

Another example for a product forest are  earlier releases of Microsoft Office. The included applications Excel, Word, and PowerPoint shared some basic concepts and were basically sold as a set of closely related products, but came from very different code bases. You could see and feel in many places in these tools where the applications behaved similar but not identical. This (slightly) deviating behavior is one of the major problems you will run into with a product forest. Over time it gets harder and harder to ensure consistent behavior over the range of products. Which in turn increases cost of support and maintenance (since each product needs individual care and knowledge). But this could reduce customer satisfaction also, since they expect similar behavior since they got the product from the same company but don’t get it.

The Product Bush

Much stronger bonds between products can be found in a Product Bush. In a product bush your products have shared ancestors and are created via clone-and-own. A new product starts basically as a copy of an existing one and is developed (read “tweaked”) until it fits the new and changed requirements.

What is wrong with this pattern? As with the product forest, in some cases nothing at all. Since you just have to pick the right ancestor for your new product (the one which can be evolved into the new product with least cost / time / … ) this often sounds like a good plan and might even be one.  Doing a copy does not involve much extra cost.  The only tooling you need is called “copy” and that is for free. So “clone” is the cheap part here. However, while this approach might feel good in the beginning , later the “own” part might strike back at you. For instance when  a customer of P₀ wants a combination of features of the “older” P₀ and the “newer” P₁. If you ever tried to merge code of systems which both had some time of independent evolution on their back, you know what I mean. This is most often a non-trivial task up to the point that it’s cheaper to reimplement the desired combination of functionalities. If the product bush has been growing for long enough, integrating bug fixes, which affect several products, becomes a nightmare too.

I have seen bushes with several hundreds, even thousands of branches. Would you draw it on a map, it would have looked like random drawings of a child just learning to use a pen. How can one be able to understand and maintain this thing properly?

However, as I said before, it is not always a bad thing to have a product bush. If you sell applications without long-term maintenance and have a relatively serialized development of products (one product is developed mostly after finishing the previous one) or if new products always contain the superset of functionalities of the old ones, this pattern might be yours.

If this scenario is used in your organization and you wonder why it is called a bush, just draw the relationship between your projects/products on a sheet of paper …

The Product Gang

The next level of product relationship is the Product Gang. Your products are build from sets of reusable assets by way of configuration and customization. Reuse means that for a new product you take the reusable assets and put them together in a way that they can be used for developing the missing piece for the new product. Changes to the reusable assets (here called the “Platform” PF) do not directly affect a product until the application developer decides to pull in the new version of the PF. The products of the product gang are now sharing more / most assets and are technically derived from the platform assets. Hence products are now in fact instances of the platform, and  I start calling them Product Variants (PV₀-PV₃) instead of products as in the earlier figures. Read more about my definition of variants and version in this post.

Highly configurable application frameworks, products build on top of reusable components, you name it. Basically a lot of the current “reuse” based development approaches belong here. One might say, ‘this looks pretty good to me, what is the catch here?’ Well, while the Product Gang certainly is much better than the two previous ones when it comes to reuse, there is still something missing to make it a real Software Product Line. Just like in a real-world gang, there is a strong link between all the members. However, there are also secrets: No one is really disclosing information about himself/herself to the other members of the gang. And this is the fundamental challenge of this approach: because there is no or not much knowledge about the members of the gang, effects of changes to the shared assets are hard to estimate and also hard to coordinate. For example, if you have to change an interface method in a component, knowing that it is never actually called by any other application than yours would be nice. Since gang members evolve separately (in separate branches or even different version control repositories), there is no easy way to find out whether this is the case or not. Basically, when there is not enough knowledge about the usage of provided reusable elements in product gang members, the assumption must be very conservative. I. e. in this case you must assume that more or less every member might use it in its existing form. That in turn makes changes very complicated. Also when a change is introduced, it would be useful to verify that it works for all members which use it. But to do so before releasing a change one has to run tests on at least a selection of gang members (or make sure the change works in all possible cases by other means). However, this is often impossible due to slightly different build processes for the different products, the inability to access the other projects’ repositories, or simply lack of time. This reduces over time reuse in favor of product individual solutions, since for those the effects are local only.

The Product Family

At last we arrive at a pattern which is more or less the ultimate goal of software product line development: the Product Family. The main difference to the previous scenario, the product gang, is the availability of knowledge about members of the family (PV₀-PV₃) within the whole development organization. The product family approach also includes continuous care for all relevant products during the evolution of the product family assets. At all points in time all (currently as relevant categorized) members of the family are involved when making changes to the shared assets. This, of course, is not only a technical process but also involves communication between the individuals responsible for the assets and the product (variants) to discuss/prioritize/reject changes.

The bidirectional link between reusable core assets and the products which instantiate them is the key difference between Software Product Line reuse and “traditional” reuse approaches. Of course this does not come for free. Those of us who have a large family know what I mean…

From here we go

All of the afore mentioned patterns have their use case, none is “bad” as such. But when it comes to product lines there can be only one, the product family. The important question is: how to migrate from any of the other patterns to the Product Family. There are basically two scenarios. One I call the “Separate Product Merger”, because individual products, often fairly independently developed, are the starting point. What is shared among the existing products is the membership to a common problem space, but the solution spaces are either entirely independent or at least deviated to a degree so that merging the solution space would be to much effort. However, using one or more of the products’ solution assets as a base for the new product line should at least be considered. For instance pick the (two) best product(s), the most simple one, or the most complex one.

With a product bush or product gang, reuse is already happening. That’s why I call the other scenario simply “Reuse Improvement”. Since both problem space and solution assets are closely related a lot of the existing assets can be used in the new product line. Often one of the goals of the improvement scenario is to be able to create all existing products more or less unchanged from the new product line assets. In this scenario there is a choice what is analyzed first. The often simpler and quicker approach is to start with solution space variations (e.g. by comparing and merging on code level, more on this in a later article) and take the road to the problem space variation from there. The other approach  starts with the problem space going only in a second step to the solution space, much like a software product line development when it starts from the scratch. In this case it is more complicated to provide an easy mapping between problem space and solution space because problem space variations might not so easily map to the existing variations in solution space. However, this second approach is more or less mandatory for the “Separate Product Merger” scenario.

The  variation point analysis and the following documentation of problem space and solution space is not an easy activity and will be left for a later article in this blog.

SPLC 2008 Notes: Application Engineering – a Dying Discipline?

One goes to SPLC to learn new ways of seeing and doing things. This year’s conference confirmed to me at least that I see some things  different from some other participants.

Who needs Application Engineers?

Some of the presenters called for the end of the tedious application engineering. If you are not too familiar with Software Product Line language: Application Engineering are the activities concerned with defining, building and deploying actual applications based on reusable core assets produced by Domain Engineering. Surely one wants to reduce the effort to create new applications to a minimum. And yes, in a lot of cases well designed core assets and PL architectures can even remove the necessity to develop application specific artifacts. Does this make application engineering obsolete? Not at all I would say. If a product was developed based on customer specifications, who is discussing the resulting application with the customer? Who makes sure that there are tests that are based on customer requirements for this specific application? Who tries to map the customer requirements to a configuration of the core assets in the first place? That are clearly application engineer’s activities. Depending on the domain and the product line implementation, these activities might take only a few hours or even minutes but they normally don’t go away even in perfect product lines if you have customer specified products. And finally, the description of the instance from your product line (configuration file, model, what ever used in the product line approach) is for sure an application engineering artifact.

Urban Myths

Another urban SPLC myth you hear once in a while is that one is able to derive THE product line architecture “automatically” from feature models just by doing the feature model right.

I must admit, I have seen architectures derived from features. So yes, it is possible. Question is one supposed to do this? A feature model shall be an easy to understand representation of variability in the problem space. A product line architecture is a solution space artifact providing means to easily produce products based on this architecture. In my experience, often this requires architecture and feature model structure to be very different. How would one resemble a layered architecture often used in communication protocol stacks such as TCP/IP stacks in a feature model?  The basic IP protocol is the layer on top of which we build TCP and UDP. So TCP and UDP could be “or” features below “IP”. IP in turn uses a physical layer which again provides different physical protocols such as Ethernet or serial lines (PPP) to transmit its data. So were do we put the “or” choice for Ethernet or PPP? On top of the IP Layer? Below?

I don’t know exactly, because both models offer similar but not identical choices. In the first case, the physical layer can be selected without an IP layer present, in the second case IP will imply also the physical layer to be present. Both models make sense but at least I would implement both feature models using the same architecture in order to be able to reuse the same core assets in both cases.

Also the implementation techniques (e.g. C vs. C++ vs. Java, use of aspect-oriented concepts, domain specific languages or models, … ) have a strong influence on the decomposition of the architecture into building blocks. For instance in an aspect-oriented approach a cross-cutting concern can be implemented in a single artifact whereas in a simple Java approach it has to be spread out in every implementation class.

And many features simple cannot be decompose in any kind of direct architectural mapping or crosscutting influence, they related to both kinds of solution space elements no matter how you do things.

The longer I think, the more I feel uncomfortable with a quasi direct mapping of problem space variation to solutions space structure…

What else?

Yeah, what else brought SPLC 2008? I am not sure. While it was the best attended SPLC ever, I was disappointed by the lack of new companies coming to SPLC to learn about Software Product Line Development. Sure, there was a strong Swedish group there (some 16 I counted) and some from the UK (about 8). But Germany being very strong in research (24) but only 6 were from industry (tool vendor participants excluded), if we leave out the product line consultants from Siemens and SAP, we are down to 4.  I am happy to hear that next years SPLC will try to attract more industry for instance by running an Industry track. I hope the promising location (San Francisco, USA) will help as well.

The weather was perfect for the conference (intermittent periods of hard rain throughout the week), the conference itself very well organized by the local team. However, I am already looking forward to the next SPLC and hope to see some improvements then.