Fact: risk assessment is the #1 determinant of project success

Everyone knows that technology projects are laden with risks. But, until now, has anyone ever proven that the anticipation and mitigation of risk is the #1 activity that is most highly correlated with outcomes? Well, it's true.

Donald Rumsfeld is famous for describing risks as the "unknown unknowns":

[T]here are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns—the ones we don't know we don't know. And [...] it is the latter category that tend to be the difficult ones.

There are all sorts of unknown unknowns that can derail a project, from unanticipated personnel challenges and budgetary issues, to market / competitive threats that alter the landscape, to the vexing technology complications that tend to arise out of nowhere. Anticipating as many of these liabilities early in the project as possible and identifying ways to mitigate them is a critical skill, lest project managers are left playing an expensive game of Whac-A-Mole with extremely high stakes.

Everyone can relate to projects fraught with challenges, which is why it's not hard to accept that risk anticipation and mitigation is an extremely important skillset. But, is it really the most directly correlated with outcomes among all project activities? Yes, it is. In our VOCalis voice of customer survey, we have asked over 1000 customers to rate their digital services partners' performance using a 6-point scale across 30+ activities at different stages of their technology implementations. We ask about everything from assigning roles and responsibilities, to developing measures of success, to managing expectations, to meeting milestones and time commitments, and so forth. And while correlation does not equal causation, we analyzed the final customer satisfaction scores of all the projects against the feedback of each of the activities provided over the course of the projects to determine whether or not any correlations exist. And we can confirm that there are no activities more closely tied to success or failure than "identifying potential project risk mitigations" early in a project. The activity has a +0.884 correlation coefficient for the data geeks among us (we use "geek" as a term of endearment). This strong positive correlation means that the more positively a customer rates their digital agency or systems integrator at risk mitigation planning early in the project, the higher the customer satisfaction rating the project receives at the end. And the inverse is also true: when a partner receives a poor risk mitigation score early in the project, there is an extremely high chance that the customer satisfaction score of the final project will be poor.

This insight is extremely relevant for all parties -- customers, digital partners, and technology vendors.

• For customers, they finally have a highly predictive way of determining the outcome of a project based on activities and planning that take place early in the project. In other words, customers should:

1. Assess the risk mitigation planning capabilities of prospective partners during the selection process, and use references and case studies as proof points.
2. Get good at risk mitigation internally, since many of the challenges and "unknown unknowns" are not fully within the partner's power to mitigate, or even anticipate (especially with internal politics, etc.). Some partners are extremely good technically, for example, but they don't possess great risk mitigation skills. There is no reason that risk mitigation cannot be done by the customer in consultation with the partner.
3. If a project gets beyond the planning and analysis stages without some formal risk assessment activities, discussions, and deliverables being produced, it's time to raise the red flag. Or at least the yellow one.

• For digital partners (agencies and integrators), they can plan resources and training accordingly to improve project outcomes. Partners should:

1. Train project managers (and all project personnel) in the practice of thorough risk mitigation and assessment.
2. Mandate and formalize risk assessment activities and deliverables on all projects, and directly involve customers throughout the process. Just because the partner may lead this activity and own the deliverable does not mean the partner is accountable for addressing each of the risks should they come fruition. Be sure to assign clear ownership during the process, even if it is to the customer.
3. Incorporate risk reviews at milestone points in a project, not only to review the results of the risk planning activities early on, but also to assess and evaluate the evolving likelihood of each as the project progresses.

• For technology vendors, they have a stake in this too, since risk mitigation planning also has the highest correlation to the technology vendor's Net Promoter Score (+0.496) versus any other planning activity -- a whopping 68% higher than any other activity that takes place in the planning phase of a project. This makes intuitive sense, since customers tend to be happier with a product if the implementation goes well. Vendors, therefore, should:

1. Pay closer attention to the capabilities of their partners. We have stated before that vendors aren't the best at making partner recommendations, and that's partly because they don't look under the hood to understand what happens once they turn over the keys. But our point is that vendors have a lot of skin in the game, aside from the "social contract" they have with their customers to help them succeed.
2. Advise partners of this best practice, as well as customers. And there's nothing wrong with offering training to either or both parties to help them get proficient in the skill.
3. Conduct in-flight check-ins on customer projects to ensure that the assessment has been completed and that the risks are mitigated as much as possible. This will be an actionable, early indicator of project success. Also, be ready and able to help address at least the technical challenges as they arise, and ensure the lines of communication with both the customer and partner are open to be able to do so.

In our analysis, some agencies are consistently rated highly at this skill. For example, Cognifide (a WPP company and the first VOCalis Award winner), Valtech, Slalom, POSSIBLE (also a WPP company), VML (yet a third WPP company), and ConnectiveDX (now a ProKarma company) are all very good at risk assessment according to their customers. As a result, we have yet to see a poorly rated technology implementation by any of them. On the other hand, there are partners who consistently under-achieve in their overall project performance, and their ability to assess risks are always poorly rated even when other skills tend to live up to customer expectations. While these partners may receive positive feedback about their on-time delivery or their technical fluency, poor risk assessment scores seemingly weigh more heavily on their customers' perception of the overall outcomes.

So, are we 100% certain that if partners improve at risk assessment and mitigation that all their projects would improve? Well, yes...they would certainly all improve. But, are we sure their projects would always live up customer expectations? Of course not, since there are so many other factors. But, we can all agree by now that it sure would be a good starting point. 

Have any thoughts or questions? We'd love to hear them in the comments below. And if you are interested in learning more about our VOCalis voice of customer program, please contact us. We work with buyers, partners, and technology vendors to help them all improve outcomes.