The project management discipline is in crisis

I ran into this publication, The underlying theory of project management is obsolete, a while ago and thought it was quite interesting. I finally got around to writing a digest of it. Most of the material is directly quoted from the paper.

The article starts by establishing a baseline of the current PM theory as known according to the PMBOK Guide.

Theory of Project is described by these three principles:

1. Project management is about managing work; this is the conceptualization.
2. The work can be managed by decomposing the total work effort into smaller chunks of work, which are called activities and tasks in the PMBOK Guide.
3. Implicit assumption associated with decomposition, namely that tasks are related if at all by sequential dependence.

Theory of Management consists of the following three sub-theories:

• Theory of Planning
• Theory of Executing
• Theory of Control

The planning processes dominate the scene in the PMBOK Guide. The emphasis is on planning, with little offered on executing especially, making the overall management theory “management-as-planning”.

The underlying theory of execution turns out to be similar to the concept of job dispatching in manufacturing where it provides the interface between plan and work.  The basic issue in dispatching is allocating or assignment of tasks or jobs to machines or work crews, usually by a central authority.

Then the article proceeds to examining the current PM theory and argues that is not the best available:

Theory of Project

The current model is basically represented by a non-cyclical network graph in which each activity connects directly into its immediate successors.  Such a model fails to account for uncertainty in production process, which affects time as well as interdependencies between tasks.

The alterative “flow view” based on the queueing theory where the focus is directed towards uncertainty and linkages models the reality much better. The flow view especially addresses the goal “unnecessary work is not done”.

Yet another theory exists,  the value generation view. The basic thrust here is to reach the best possible value from the point of the customer.  The major difference between the suggested PMBOK view and the value generation view is that the customer is included in the conceptualization of the latter. The value generation view admits that at the outset, customer requirements are not necessarily available or well understood, and that the allocation of requirements to different parts of the (project) product is a difficult problem.

Theory of Management

Theory of planning:  As for the management-as-planning approach:

• it has been held that it is not generally possible to maintain a complete and up-to-date representation of the current circumstances and the plan to change them.
• the absolute separation of management and execution is not seen to adequately correspond to organizational reality.
• the plans push tasks to execution without taking the status of the production system into account.

The two last aspects mean that this model “leaves the task of management essentially uncoupled from everyday activity” (Johnston & Brennan 1996). Also the model implies that the process and outputs of planning are not questioned.

There is another approach to management, called management-as-organizing, which has been presented as a counterpart to management-as-planning.  It models an organization as a set of interacting sub-units capable of sensing, planning and acting.  Communication is non-hierarchical, based on interaction between sub-units. In this approach, management involves design, co-ordination and enabling of otherwise autonomous activities.

Theory of Executing:  The main issue with the current dispatching model is that the unproblematic realization of tasks pushed by the plan to the execution is assumed.  It is very difficult to maintain an up-to-date plan, and thus the tasks pushed by the plan do not correspond to reality, i.e. their prerequisites in terms of predecessor tasks (or other inputs) do not necessarily exist. This leads to the situation that a major share of tasks to be commenced, when pushed by the plan, chronically lack one or more of their inputs.

Another erroneous assumption is that the task is fully understood, started and completed according to the plan once authorized. Whereas in reality the work in organizations is coordinated through making and keeping commitments. Thus action is coordinated by the commitments people make rather than by central control acting through commands.

The two basic shortcomings of the PBMOK suggested dispatching model are:

• there should be two-way communication between the controller and the executors.
• it is necessary to consider the commitment of the executor; a job will actually be started and completed only if the executor is committed to realize it.

Theory of Control: Another theory of control,  “scientific experiment”, that addresses learning and improvement  is suggested as a better approach to finding root causes for problems , and improving performance.  It reveals a fatal shortcoming of the PMBOK suggested model that does not address finding reasons for deviations, and eliminating those root causes.

The article goes on to present empirical evidence from one application domain, construction, that the existing PM theory is not valid.  The evidence is abundant, and you should read it yourself. I just want to include here one last quote:

The deficiencies of the theory of the project and of the theory of management reinforce each other and their detrimental effects propagate through the life cycle of a project. Typically, customer requirements are poorly investigated at the outset, and the process of requirement clarification and change leads disruption in the progress of the project. The actual progress starts to drift from the plan, the updating of which is too cumbersome to be done regularly. Without an up-to-date plan, the work authorization system transforms to an approach of informal management. Increasingly, tasks are commenced without all inputs and prerequisites at hand, leading to low efficiency or task interruption and increased variability downstream. Correspondingly, controlling by means of a performance baseline that is not based on the actual status becomes ineffective or simply counterproductive. All in all, systematic project management is transformed to a facade, behind which the job actually gets done, even if with reduced efficiency and lessened value to the customer.

A few points here that resonate with me personally when thinking of software development projects and project management processes used:

• Rework indeed seems to constitute the bulk of the software development projects. No wonder the non-cyclical network graph model (think MS project / GANTT chart model) mentioned earlier does not reflect the reality. The execution of the original plans never goes as planned.
• Most of the project planning sessions dont have a good way of incorporating customer feedback during requirement development and feature prioritization and so focus on product value is not always clear.
• Planning & management indeed seems to be removed from execution. Management thinks of execution as of an ideal machine. Uncertainties of the production and requirement understanding are poorly accounted for in the plans and.
• One way communication within command-and-control model. Such an important factor as commitments from your team members are usually overlooked although those are the most important elements enabling project success.

Cheers!

Are you managing your backlog well enough?

A very valid point made by the following post Is Agile Enough? (part 1)

Therein lies the key omission of Scrum (and most other agile methods): The backlog of user stories are taken as a starting point. They are not examined for their “quality”, “completeness”, “cluefulness”, budget impact, etc. The user stories are assumed “good”, and used as an “input” to the Scrum process.

(… good read …)

Does your request tracker allow you to have hierarchical requests so you could be breaking down original user stories and requirements into more detailed tasks suitable for iteration planning?

Does it track the duplicate requests so you could see the most popular requests and prioritize accordingly?

Does it allow you to specify dependencies between requests and respect those in iteration plans and project deadline estimations?

Does it allow your customers selectively (in “controlled by you” fashion) access the request records to give you more feedback regarding the content and priorities?

Check out a demo account at www.yoxel.com to see how Yoxel request tracking system can help you do all that within agile environment.

Release planning with YOXEL SW: Starting a new release

The next several posts I intend to dedicate to covering release planning and release/project tracking. In agile terms, planning and execution of an “iteration/sprint”. Expect me to talk about YOXEL SW (our solution for software product management) quite a bit although I hope the details of the described process will still be useful in general, no matter which tools you prefer.

Release, Incremental Release, Iteration, Sprint

Some methodologies and tools like to talk about releases (longer term plan) and multiple iterations (short incremental chunks of work) within each of the releases. My terminology is simplified. Essentially I will use the terms release, incremental release, iteration, and sprint interchangeably. You start an agile incremental release and work on it for a month (call it an iteration or a sprint). Or you start a waterfall release, plan it for months and work on it for another year (call it a production release or whatever). So I will be using term ‘release’ and ‘incremental release’ and it will mean ‘iteration’ and ‘sprint’ too 🙂 .

Backlog

What I mean by backlog may differ slightly from what you think of it. My definition is very practical, the backlog is all requests in your request tracking database. The request tracker for our users is either Yoxel’s native tracker or a combination of an external bug-tracker (Bugzilla/Mantis/GNATS/…) and Yoxel’s. We understand that many teams out there already have their flows built around their favorite bug-trackers so we designed YOXEL SW to be able to connect to your existing bug-tracker and synchronize with it as often as you desire. It is up to you whether you continue filing all requests in your Bugzilla, switch completely to Yoxel, or separate bugs and simple enhancement requests from other classes of requests (requirements, feature request, customer support request) to file them in different systems. Your choice! In the end, Yoxel database contains all your requests: synchronized copies from your bug-tracker plus native ones, filed directly.

You can probably tell by now that my religion is to keep all requests (bugs, enhancements, new development, features,…) in one centralized database, even requirements and support tickets from your customers. Once you have done that, it is all about defining the right work flows for each class of requests to make it equally convenient for developers, product managers, and other involved stakeholders. Developers, support team, product managers even some of your expert customers file their requests into this centralized database where later the requests can be refined and acted upon. Especially if your system has hierarchical capability you can always break down requirements into a number of meaningful enhancement sub-requests, fill each of those with functional details, and have them ready for actual implementation.

Prioritized Backlog

It is better if your backlog is prioritized! One could think of using some complex ROI based functions (derived from win/loss analysis, customer questionnaires, risk assessments, … ) that rank all your requests in the backlog but Yoxel SW is pretty simple in this regard at the moment. All we use is ‘customer urgency’. Forget about so widely used and IMHO quite often confusing ‘severity and priority P1,2,3’. Just urgency as expressed by your customer: critical, high, medium, low. [I actually think that having all the data to apply the ROI based functions is good but I believe that the functions should be dynamic, meaning, they should order your backlog items based on the current focus of your incremental release/iteration. Sometimes your release is focused on a particular customer, sometimes on robustness of the software, sometimes on adding new functionality; the same order in the backlog does not work for all the situations.] Anyway, for now we prefer for product managers and stakeholders involved in the release planning process (which I will discuss in my next post) to define the exact content of the release based on their experiences and knowledge, backlog priorities are only used to generate preliminary release plans, to kick off the actual planning session. BTW, requests with urgency ‘critical’ are handled in a completely different manner, see my previous post about it Urgent vs. Planned.

Preliminary release plan

Once you have a prioritized backlog you need to take a chunk of the most important requests and start working on those, right? Well, may be not that simple. I agree though in one, you need to take a chunk to begin with, and lets call that chunk our preliminary release plan. Then, I believe, you want to collaborate with your team of stakeholders, prioritize the list of requests in the plan further, refine some of the items, make sure you have not missed anything, estimate and do whole lot of other interesting things. I will continue this thought in my next post …

YOXEL SW actually tries to simplify the generation of the preliminary plan for you, to make it a meaningful start point. Usually it is one release owner/product manager/scrum master who starts the release planning session. He/She simply fills out a simple form that drives an automatic plan generator. In that form he/she specifies a list of release members who will be collaborating and shaping up the final plan, objectives and goals of the release to keep all the members on the same page, and a requests filter. Depending on your release goals you may not want to grab the top N items from the backlog, instead you may want to be more selective (enhancements only, or requests related to these few customers only, or some other subset of the requests). Also the release owner specifies perfect fix/test time per request (based on your previous experience), % of time your team usually spends on critical/unscheduled/unexpected work (kind of backward way of specifying team velocity), and your desired deadline (for agile teams would probably be 2weeks-1month away). It is not very important to be accurate with all these parameters at this point (you can always refine the plan later).

The generator then makes an “educated guess” of how many requests your team can handle (and it knows quite a bit about your team too from the assignees and product definitions in the request tracker), given the deadline, and grabs that number of the requests from your backlog, after having applied the filter. It also informs all the release members that a new release planning session has started, with the specified goals, inviting them to take part in the planning session.

Your preliminary release plan is ready. From here on you are in the release planning mode, collaborating with all the release members to form the final plan that can then be used to drive your development/sprint/iteration.

Compare it with what you are doing today to generate the preliminary plan.

1. Manually examine your bug-tracker for any important enhancement requests already filed
2. Create Excel table with those enhancements plus potential new features
3. Give it all preliminary priority
4. Oh, yes, do not forget to include some important bugs
5. And keep it all in Excel for now
6. …. more manual steps?

Think bigger, Think Agile Product Management

 

Agile business, Agile product, Agile team.

The topic of applying agile methodologies to your business processes seems to have become quite popular these days. The benefits of becoming agile are pretty attractive for software companies and yet many of them still do not clearly understand how they could make a smooth transition and whether this “agile” thing really works. I believe part of the confusion is owing to the fact that most of the information and discussions on this topic focus on software development aspect and forget that the companies are businesses and there is a little more to creating a great product than just code development. I prefer to look at the problem of becoming more agile and adaptive from a slightly higher level, product management (PM) level. PM for me is about product development and coordination of multiple teams and processes (development, customer support, marketing, management, sales). In my opinion agile product management (and product development of cause as its essential part) is what you should consider.

Re-visiting, re-fining and re-prioritizing product requirements, plus soliciting, and digesting customer feedback, for each development iteration, are the key ideas of the agile methodologies. In smaller companies your development team may be the one in charge of all of those functions but as your company grows you will most probably have more specialized roles and teams responsible for market requirement and customer feedback management (product managers, technical marketing, professional services, sales engineering …). Hence the agility of your whole business will depend on how well these different teams can interact with each other and especially with the development team to facilitate relevant development iterations and produce what your customers expect. This collaborative work of multiple teams is nothing else but your product management process. And in my opinion it is a very challenging task to make this process efficient and agile. Unfortunately just a white board and story cards may not be enough.

I strongly believe your company will benefit if it finds an appropriate tool to support its PM process. Ideally you will want to have a magic system that engages your development team, your product managers, your support team, and even customers to enable better collaboration, coordination, and reporting. Such a system should be as attractive to the development folks as to product managers, marketing, sales and management. Sure, it could be several systems but then they had better be all integrating and collaborating well with each other. As of today I do not really know of any standard open interfaces/frameworks allowing different vendors to create product management solutions that could smoothly interact and collaborate with each other (Eclipse may be?), so the choice is still usually a one vendor solution, or better (my own preference) an open source solution.

“Hybrid Agile” methodologies and the tools.

Most of the companies that are serious about their business are already agile. They have figured out a certain process that works well for them (with whatever tools they could find). Such companies may not follow strict SCRUM, or strict XP but they are agile. A hybrid agile methodology is often a real life situation.

For me it is always interesting to learn what kind of tools software companies use in support of their hybrid agile approaches. I like to talk to product managers of small/mid-size software companies and ask them about their current methods and tools: how they manage and prioritize product requirements and backlog of requests; how they plan, estimate and resource allocate/balance for each iteration; how they track project implementation and testing; how they gather customer feedback. In order to substantiate my statement that you should look for better tools let me give you a few examples here of what the companies use currently; you will see that the situation could be better.

Company A:
—————-
MS Word and Excel for requirements/prioritization
Wikis for collaborative documentation
Jira for project planning/tracking
Jira for bugs and tasks tracking

Company B:
—————-
MS Excel for requirements/prioritization
MS Excel and sketching tasks/stories on white boards for planning/tracking
Scrum plugin for MS Visual Studio Team System

Company C:
————-
MS Excel for requirements/prioritization
X-Planner and Wikis as the collaboration workspace
X-Planner for project planning/tracking
Bugzilla for backlog/bugs and task tracking

Company D:
————-
Shared Google spreadsheet for requirements/prioritization
Omniplan for resource allocation, and project planning/tracking
FogBugz for backlog/bugs and task tracking

Company E,F,G,H,…
———————-
MS Excel/salesforce.com for requirements/prioritization
MS Project/basecamphq.com/Excel for resource allocation, project planning/tracking
Bugzilla/Jira/Mantis/GNATS for task tracking

Amazingly enough there is abundance of agile companies that simply use Bugzilla/Mantis/GNATS and MS Excel. It is probably explained by popularity of the MS Excel and open-source bug-trackers (thousands of development teams have standardized their processes on those), lack of knowledge about better alternatives, and the fact that smaller companies tend to spend less money on tools in this area. My personal observations are that although these companies managed to stay agile their current processes could still be improved and accelerated. A number of new generation tools could help them in that:

1. Smaller teams tend to do fine with just Wikis but in other cases it seems like more than two or three different tools have to be used to support the process properly. A bug-tracker (from R&D side) seems to always be in the picture and a requirements management tool (from PM side, usually Excel) also. Plus various project management, resource allocation, time tracking tools. My problem with this is the lack of integration hence a lot of manual work to keep all the tools synchronized plus perhaps burden on IT to support all those tools and integrate them:
   • Spreadsheet tool (except Google’s) is not a collaborative environment.
   • Sooner or later PM has to manually transfer the data from his spreadsheet to a bug-tracker and also to a project manager/time tracker.
   • Every now and then PM has to update project status manually based on the status of individual tasks from the bug-tracker.
   • Real-time detailed status of projects is not readily available
2. The reporting capabilities are pretty poor in all these examples or achieved at the expense of significant manual work. Reporting can be extremely important for keeping your management and team members updated and informed. Good paper reports can actually be almost as effective as white boards and story cards during your agile team meetings.
3. A few companies that have been more flexible and also educated on the available new generation tools for agile methodologies have made their transition to X-planner, MS team SCRUM plugin, RallyDev or VersionOne. Again, that is mostly to support agile development process. This is great! Although I still have a few concerns: these tools enforce a certain agile methodology (SCRUM or XP), new terminology, and may require significant ramp-up/learning time from your development team whereas you may want to do an agile variation of your own that does not disturb your current team specifics so much. Just think how Mozilla, Apache, MySQL, Eclipse projects would react if you suggested to them to abandon their Bugzillas and Mantises and switch to another “agile” request-tracker. The same is true for almost any software company.
4. Customer feedback is usually managed by product managers, professional services and support organizations. Among the companies I talk to I do not see much adoption of customer feedback management tools although there are some interesting tools out there. IdeaScope seems to be one of them! Its advertised most valuable use seems to be mostly in combination with Featureplan product which may be an overkill for a smaller company, but still. (Featureplan, by the way, also enforces a certain pragmatic framework.)

Any recommendations?

There are some interesting new generation tools designed specifically to support agile methodologies. And yet the real challenge is for the companies that do not feel comfortable with radical changes of their current tooling or are reluctant to abandon their bug-trackers. There are fewer options for these companies (although I may be wrong).

• Smaller co-located teams probably will not need any specific tools. Wikis and any bug-tracker will be enough. Think about your future though, if you are growing, a day may come when your wiki/bug-tracker environment is not enough to accommodate your team/business dynamics.
• If your team is bigger, especially if it is distributed, and wants to do a strict SCRUM/XP and you can afford a significant change to your current processes, you should probably go with some of these new tools: RallyDev, VersionOne, X-Planner. They are great! Pick one that addresses your product managers’ and other non-developer’s needs also.
• The teams that want to transition to or improve the support for their existing agile processes but are not comfortable with radical changes, in my opinion should look for a tool that integrates with their Bugzilla, Mantis, GNATS, Retrospectiva, Jira, … and is designed to be a tool for product managers too. Make sure it has the release/iteration manager that allows you to plan and track your iterations and derive useful statistics.

Yoxel Systems

The project I am involved with, Yoxel Systems , should be a really good fit for the last category mentioned above where you really care about smooth transition or are looking to support your hybrid “agile” approach. We try to create a flexible solution for product managers, field support, and developers that can accommodate a variety of agile methodologies, so Yoxel is also an alternative to RallyDev, VersionOne, X-Planner. On top of the release/iteration management aspect Yoxel tries to be very customer focused providing customer support, product knowledge management, and lightweight CRM capabilities too, all integrated into one product management system.

(Please try our demo accounts at http://yoxel.com)

Enjoy!