Why do projects fail in large enterprises? - some thoughts

James McGovern in his blog, “Why do projects fail in large enterprises?”, makes a number of important statements

• “how come open source projects can deliver high quality on time when folks don't even talk to each other”
• outsourcing is “about taking otherwise junior IT folks and making them usable regardless of their actual individual competencies”
• outsourcing firms “have convinced others that process can become a substitute for competence”
• “think of open source as a methodology for software development”
• “in an open source project, the architect is king and folks follow him”
• “project managers with budgets and deadlines are handcuffed to delivery of perception management at the expense of high quality software”
• “it is still fundamentally easier for me to put crap code into production at work than it is for me to do the same with almost any open source project”

The key messages here are “open source communities tend to prefer competence over process and hence can develop higher quality software” and “with experience and a desire to participate in a community comes quality”.

I agree and disagree with James!

I agree that the balance between project manager and architect is often wrong – this is a well known problem summarised by the eternal triangle of project management. The project manager is responsible for the tangible dimensions of cost and time to benefit. A motley array of solution engineers including architects, configuration manages, release managers, quality champions, requirements managers, etc are responsible for fitness for purpose. While this area operates in this way, quality will never get on the steering group agenda.

I agree that many of the techniques of open source and agile can be deployed to improve large scale solution delivery. Test first, pair programming, frequent builds and smoke tests, prototyping and a range of “non-traditional” techniques can be applied to projects of any scale and geographic distribution with positive productivity and quality effects. We need to stop dressing these up as some great agile revolution – it dilutes the message, it puts people off, it sounds like a risk. These are just good practices that have been out there for a long time – I used all them at least 20 years ago.

I disagree that with the view that only experienced teams can deliver successfully. I have taken teams of trainees and delivered high quality complex software rapidly. We need to find a way of growing people. We need to find a way of delivering the less interesting parts of a solution. We need senior people who are interested and creative in growing junior people. Delivery teams must contain inexperienced staff for solution delivery to be sustainable.

Process is never a substitute for competence. Correctly applied process teaches, supports, and grows people. Process needs to be applied in a balanced pragmatic manner with a view of the competence of people, the technology risks, and the time and cost constraints. This way it improves productivity, minimises bureaucracy and improves quality. Poorly applied process does the opposite. To apply process correctly – you need managers who have a balance of delivery and software engineering skills.

There is an implicit and sometimes explicit view that software delivery is the most important aspect of a project – this is 100% wrong. It is the deployment of a new business capability and the consequent delivery of business benefit and sometimes “crap code” in production achieves more for a business than good code later. It may deliver more business benefit to have 100 staff fixing data errors on overtime than to wait a month for the code to do it. Benefits delivery is the measure not quality.

A great thing about the open source community is the sense of purpose and commitment to delivery. High performance teams share this high level of motivation. A commitment to developing high performance teams and a commitment to developing leaders is essential to improving delivery. The leader needs to have a mix of skills – project management, enterprise architecture, software engineering, teacher, and politician.

One area I do disagree with James is that Indian outsourcing teams think process is a substitute for experience. If you use an outsourcer then challenge them to show you how CMMi benefits your business. Challenge them to increase the benefits both in terms of quality and productivity. Get them to show how they have already done this. Ask them what you need to do to allow you to maximise benefits. Work collaboratively with your outsourcer – you made them part of your business.

What do we need to do?

• Develop hybrid solution delivery leaders with a balanced mix of programme management and software engineering skills
• Leverage experience in large teams with a mix of skill levels to create high performance teams
• Create and sustain high performance teams
• Unify the solution engineering team
• Leverage delivery best practice which include techniques and process ruthlessly to drive quality and productivity
• Focus on benefit delivery and make transparent and rational compromises over quality

Enterprise Architecture: Ten Reasons why Outsourcing tends to fail... – A Repost!

First, I must declare an interest – I work for a leading global IT and business process outsourcing services provider.

More importantly, I must counter or clarify the ten reasons that James Govern has cited for outsourcing failure. I am not going to say that outsourcing is always right or that it never fails – on the contrary, the reasons listed are in general correct but the industry and its customers are learning. James has written an outline for a supplier evaluation, if you choose an outsourcer then they should be able to satisfy you on all these points. There are answers…

Cost-reduction expectations
James is half right here! An outsourcing strategy should be more sophisticated than cost reduction. However, the assertion that the cost differential cannot be sustained is flawed. First, the US Dollar will stabilize against other currencies. Second, as the labour pool in India is further expanded beyond the tier 1 cities, wage inflation is likely to reduce. Third, the offshore labour force is being developed in many countries around the world which will further restrain wage inflation.

Data security/protection
Large organisations will typically have many sites that are sited in different countries. They are already communicating with their suppliers and customers electronically. Adding another business partner, the outsource supplier, poses no greater security challenge. 

Equally, the same organisations have contractors and dissatisfied employees who are just as capable of injecting malicious or insecure code. A mainstream outsource supplier will have invested in security technologies and processes that most large organisations can only dream of. They have to in order to be credible.

Process discipline (CMM)
James hints at three issues here. The first is poor implementation of new processes. As with all best practice processes, they require best practice implementation. Good process implementation will, in general, speed up activities, reduce risk and costs. Initial implementation will often have some flaws. Process integration between the client and supplier organisation relies on cooperation by staff at the client organisation. Over time these flaws will be eliminated because the mainstream suppliers use self optimising processes.

The second issue relates to how the experience client staff work with the new supplier. Well designed process and organisational integration between the client and supplier will leverage the value of the experienced, heavyweight folk that work in client organisations. This takes effort on both sides, the supplier will have gone through the transition many times and will be able to advise on how to make it work.

The third issue is the assertion that experienced folk don’t need heavyweight processes. This is correct in most cases because these guys will have discipline and judgement that means that looser guidelines will suffice. However, few organisations can sustain IT departments built entirely from highly experienced staff.

Loss of business knowledge
One reason for outsourcing is to secure organisational knowledge. In many organisations, critical and IT business knowledge is trapped in the heads of a few key individuals. This poses a major threat to those businesses. Mainstream outsourcing organisations have well developed methods for knowledge capture and dissemination. They have to be good at it because their businesses depend on it.

In the short term there is the potential that some key element of knowledge will not be captured and an issue that relies on this knowledge will emerge. Over time, this potential will be reduced and the overall threat to the organisation will be reduced. Having the knowledge formally captured ensures that the threat does not re-emerge.

Vendor failure to deliver
IT as a whole does not have a great record of successful delivery. I suspect that research will show that mainstream outsourcers have a better record than end user companies. The reason is that they are specialists at it and invest far more in getting it right than a typical end user organisation. That is why they are growing.

Scope creep
I wasn’t quite sure what James was getting here. From an SDLC and commercial point of view there different approaches to handling changes. Different outsource vendors have different philosophies and approaches to managing changes – you can get a mismatch if you don’t think this through when you let the contract. You need an outsourcer that manages change in the way that you want to manage change.

Government oversight/regulation
I agree organisations need to identify and protect their business critical IP. Any outsourcer that lost critical IP would lose its credibility in the market. Again it is an area where the mainstream suppliers have to ensure that they are secure.

Culture
Culture is a big deal. The key is to take a good look at the engagement model – a strong onsite/onshore presence will reduce the risks in this area, as will a strong vertical focus.

Turnover of key personnel
The assertion that outsourcing removes the possibility of working with good people is plain wrong. Just the opposite, mainstream outsourcers have exceptional talent.

But your role may change, and the transition will involve bringing the outsource staff up to speed and this may not be what you want to be doing. So, in some cases people will leave. But some good people will thrive on the challenge of working to a new model and will embrace the opportunity to learn that bringing in an outsourcer brings with it.

Knowledge transfer
As I outlined in my discussion of “business knowledge”, knowledge transfer is a core competence of outsourcers. Staff from an outsource supplier bring different experience that may have more or less value than that of the incumbent staff.

Ventoro Outsourcing Research Report

I have added a link to the Ventoro report Offshore Outsourcing 2005: Results and Conclusions by Phillip J. Hatch, President of the Ventoro Institute.  Ventoro polled over 5,000 executives around the world and spoke with over 300 offshore outsourcing vendors to identify the true secrets to outsourcing success. The report covers what it takes to:

1. Develop a successful outsourcing strategy
2. Implement a global sourcing model
3. Manage a global team

Strategy & Management Bookshelves

I have updated the small list of books that I have found useful for developing business and IT strategy. I have added another small list of general management books.

RACI considered harmful...

As Chief Architect I had a responsibility for defining the security architecture.  We had a virus attack on some remote devices due to a supplier failing to install anti-virus software and then carrying out an upgrade using contaminated discs. What was my Responsibility or Accountability for this incident?

Formally, according to the RACI chart, I was in the clear.  We had delivered the policy.  The supplier had the "R"  for implementation, the internal infrastructure team had the "A".

My reaction was that we, the architecture team, had dropped the ball. We had let the business down and the subsequent loss of trade was in part down to us.  I had failed personally to ensure that the architecture was delivered, and my team felt the same way.

There were two things that, in hindsight, we could have done better.  We should have made sure the implementation plans were complete. And we should have made sure they were carried out.

What did we do to fix things?  The reaction from the team was instantaneous.  They got out there and made sure the situation was fixed both short term and long term. Was this "architecture"?  No, it was closer to incident management, problem management, and to project management. Did we ask permission?  No. Did we step on other people's toes?  Yes. Did we violate the RACI?  Yes. Was it the right thing to do? Absolutely!

Was the RACI chart wrong? No. If we were to get the RACI to cover every situation then we would be working on it for ever. And we would tie ourselves up in bureaucracy trying to deliver it.

There is something for more important than the formal RACI statements. It is pride in a job done well.  It is the personal contract with yourself. It is the expectation that you have of yourself. A high performance team develops a shared expectation.

Why is RACI harmful? It devalues pride. It allows you to point the finger at someone else when in reality you could have done something. It allows you to abdicate your collective and corporate responsibility to do the right thing, to be helpful, to be a good citizen. It encourages the worst kind of institutional politicing.