One of my favorite parts about leading product development organizations is having the privilege to work with incredibly gifted engineers who are capable of solving problems in ways that I never thought possible. As a website programmer who eventually studied software engineering, only to later pursue a career that involves working closely with customers, I have spent time working with teams that span the entire IQ and EQ spectrums. From meetings with people that sure are a lot of fun to be around but don’t necessarily resolve anything of significance, to working with some of the brightest minds in their field who will not leave the room until every last problem is solved –I have been fortunate to have experienced working with teams of all types. And unequivocally what I can say about all of them is this: making decisions together as a team is hard. Or at least, it used to be.

At Unanimous, we invented technology to help groups of people come together to form highly optimized decisions, evaluations, prioritizations and assessments. Our technology is called Swarm AI, and we’ve gained a lot of publicity for our track record of bringing people together and accurately predicting seemingly impossible events. Claims-to-fame include helping reducing error rates in pneumonia diagnoses, beating experts predictions in the Kentucky Derby and Oscars, predicting within a percentage point the anomalous 100-day approval rating of the new administration, Super Bowl scores, English Premier League results, Financial Markets, the Time Person of the Year, and so on. It’s not magic, though often times it does feel that way. Instead we bring together real human beings – humans who are full of knowledge, wisdom, insights and intuition – and combine them with our proprietary AI technology that is based off of the biological principles of Swarm Intelligence, to form artificial super experts that are capable of forming extraordinarily accurate results.

And we do so with a team that probably looks a lot like yours. We have talented engineers, UX designers, salespeople, marketing specialists, customer support representatives, executives -all of whom bring approach problems differently from one another, drawing upon years of experience and what they have learned to be the best way of doing things.  

But unlike your team, we’ve had a superweapon in our back pocket: the technology that we have spent the last 5 years inventing. We have been developing Swarm AI to help teams make optimized decisions. And one of the reasons why we have been so successful in doing so, is by using Swarm ourselves. Now that Swarm is available as a publicly available SaaS solution, we wanted to share how we use our own technology to make better decisions as a team.

In this article, I will be talking through some of the natural challenges associated with making decisions as a team, and how we use Swarm to overcome those challenges.

Now you should know something before you continue reading…. No one on our team has a natural, super-human strength for decision making. In many ways, our technology was created out of a curiosity for how to make decisions together in groups better than ways that we have personally experienced or learned about. As I write this, I can’t help but think about my own childhood engineering experiences. If you would have given me a new Lego set, I would have it built by the book in record breaking time. But put me in the Lego isle at the toy store and I would have been walking up and down the aisle for hours considering all of the pros and cons associated with each box.

What is “Good Decision Making” anyways?

We think about optimizing team decision making across multiple dimensions.

1. Accuracy
   Of course, hitting the bullseye should not involve bulldozing over other people or other options.
2. Speed
   We achieve our greatest ROI when important decisions can be made as quickly as possible.
3. Transparency
   When team members understand how a group decision was reached, they feel better about it, whether they agree or disagree. The important thing is knowing that their input, beliefs and expressions are heard, understood, and considered during the decision making process.
4. Equality
   Every participant should have equal capacity to influence the outcome when contributing to an optimized decision, forecast, or  prioritization (assuming that all participants have an equal understanding of the topic at hand). The strength of their impact should be modulated by their own confidence or conviction, not by corporate hierarchy or politics.

Engineers are people, too

As a melancholic introvert, I can honestly say that two of my greatest fears in life have been (1) making the wrong decisions, and (2) talking with people to find out that, they too, are perhaps afraid of making decisions. For as often as I have been called “robotic” and “mechanical” throughout my life, I’ve been surprised to find that my own experiences in making decisions on teams are a lot like what other engineers have experienced, too.

Why is it traditionally difficult for engineers to make decisions within a team or group?

1. Introverts and Extroverts inherently approach collaboration differently
   As an overgeneralization, we tend to see introverts being the deep thinkers – individuals who prefer to spend more time thinking than talking. And conversely, we tend to see extroverts being quick to speak, particularly when there is a lull and other people are being contemplative. As a consequence, sometimes our best thinkers get “crowded out” from being able to participate because of their personal communication preferences.
2. Fear of being wrong
   As professionals whose expertise and careers are focused on getting things right, the fear of being wrong can be crippling in discussions and conversations. As a result, some engineers may be apprehensive to vocalize their opinions or beliefs out of concern for being wrong. Some engineers believe it is safer to make no decision than to take a risk and make a wrong decision.
3. It’s common to pick popular choices instead of optimal choices
   The most common way to reach a group decision is by majority vote, but that merely reveals the most popular answer, not the optimal answer.  Research shows Swarm enables groups to converge on optimal solutions over 80% of the time, while the three most common forms of voting (Majority Voting, Ranked Voting, and Pairwise Comparisons) achieve optimal solutions less than 60% of the time. Read full paper here.
4. The loudest person in the room may dominate a conversation, resulting in a continual snowball effect
   As volume and tone escalate, it’s common for the original problem to lose focus and for something (or someone) to take its place: the person with the loudest voice in the room crowding out the conversation.
5. Organizational hierarchy drives distorted decisions
   How many team cultures would encourage you to adamantly continue disagreeing with your manager or more senior experts in front of everyone in a meeting, without feeling uncomfortable?
   

How Swarm helps us overcome challenges with team-based decision making

For starters, you should know that we haven’t always used Swarm to make better decisions as a team. In the early days when there was only one person wearing the product management hat and one software engineer, we didn’t have a large enough team to use Swarm (you need at least 3). Plus at the time, Swarm was primarily being used by business teams making long-term forecasts and predictions. Eventually our team grew but we continued to use the traditional methods we were accustomed to (a lot of meetings, discussions, having the most senior person decide what to do, etc.). It was only after an ineffective half-day prioritization workshop that we asked ourselves, “Can we Swarm on this?” We did. We reached consensus quickly. Our decisions ended up being highly favored by customers. And from there, using Swarm became the new norm.

Let’s talk about how Swarm works in the first place

When using Swarm, teams “think together” in real-time, converging on optimized combinations of their knowledge, wisdom and insights. As you can see in the image below, each team member influences the outcome by controlling an animated graphical magnet. Swarm AI algorithms process the real-time interactions and determine the ideal motion of the spinning puck (which represents the group’s consensus at any point in time). You would participate in real-time while your colleagues or team members are also connected and participating together in real time.

How we use Swarm to make product roadmap prioritization decisions

Our goal in making Swarm available as a self-service SaaS based solution is to help every team on the planet make as optimized decisions as possible. And like most organizations, our team members all have different interpretations of what our customers’ primary pain points are based upon their personal experiences and data that they have collected. Especially when making strategic decisions about which priorities we end up focusing on (or said differently, which problems we prioritize solving first), there are several factors that are important to us:

1. As a technology company, we need to make accurate decisions quickly.
2. We want our stakeholders to all feel like they were included in the decision making process.
3. We want everyone to have transparency into how decisions are made.

We used to use techniques that we all learned about and have used at one point or another: focus groups, customer satisfaction surveys and 1:1 customer interviews. These techniques have their pros and cons and can be effective in certain situations.

But now we use Swarm to arrive at truly optimized decisions. When we are deciding which projects to prioritize, we bring all of the necessary stakeholders into a live Swarm Session. As a distributed team, meeting together in a virtual “Swarm Room” works really well to compliment the live discussions that we are having over video chat.

From here, we make sure that everyone who is participating fully understands the descriptions and impact of the issues that we are prioritizing that we solve. For example, say we are deliberating between two issues: optimizing our UX for tablet users vs optimizing our UX for phone users. Both are important issues, but if we have 10x as many tablet users as phone users, the impact of the issues are not the same. Everyone needs to understand what the issues are, and their potential impact.

Although we will evaluate roadmap priorities internally, Swarm is particularly useful when we want to bring together our external users to have them weigh in on which initiatives we should prioritize. Here is an example of users coming together on Swarm to weigh in on various initiatives we are considering:

Three interesting insights come to light when reviewing the consensus that our user group came to within about 15 seconds. Although the group ultimately chose one feature “Plan question scripts ahead of time”, I can easily see that there is a fair amount of support going towards other choices as well. Secondly, some of the options that we presented to users were a direct result of features that they requested and voluntarily said they “cannot live without because of how critical they were”. And yet, when we look at their individual behavior, we see that their conviction about what was originally important to them was actually incredibly low, as demonstrating by them changing their mind nearly instantly after seeing the other options presented to them in real time.

Now when we make roadmap decisions, our customers and internal stakeholders have a much higher level of inclusion in the decision making process. Because Swarm creates a real-time feedback loop for participants, users have the opportunity to re-evaluate their original choices and ideas, and either stick with them or switch their choices.

How we use Swarm to size projects

At Unanimous AI, our project teams utilize both Scrum and Kanban practices. After projects are fully groomed and prioritized, we always size them so that we can track our budgeted time and team velocity.

We used to do what every team does: scrum poker. And, like most teams, we ran into a bunch of challenges:

1. This is difficult to do with a distributed team. Video conferencing helps slightly, but not a whole lot.
2. Team members frequently try to guess what the manager (or senior developer) will choose, instead of their gut reaction.
3. Finding the right card, placing it in the air, looking around the room, and then collecting any and all dissenting opinions, including dissensions against those opinions, can be a lengthy process.
4. Not everyone understood the requirements the same way.
5. Often times our deepest thinkers would be the least likely to want to disagree strongly with anyone or vocalize their opinion too loudly.
6. One person’s “8 points” does not mean the same as another person’s “8 points”. Two people may both say “8 points” while one person TRULY believes it, and another actually just doesn’t know, so is merely guessing (and may be willing to switch their choice as soon as they see what others are doing).

Swarm has, for us, significantly helped with all of these issues. We now use Swarm to size projects with incredibly higher accuracy, speed and team member buy-in than ever before. Here is an example of one of our teams coming together to use Swarm to size a recent project.

Sometimes our group is so aligned that our “Group Conviction” score (i.e. alignment, or unanimity) is above 90%. When it is, we move on to the next decision. But here we see that our Conviction is 56%, which means that we need to have a discussion. A low conviction score is also a strong indicator that there is a lot of risk within a project, and we may need to be more careful about who we assign the project to (i.e. based upon background knowledge) or take more time to flesh out the spec and implementation details of the issue.

Another example of how we use Swarm to size projects is when we are trying to determine the amount of energy that may be required in implementing various design concepts we are considering. Say that we have two designs we are evaluating between, and we need a high level estimate as to which concept is going to be faster to implement. In this scenario, we can now only arrive at an answer, but we can also evaluate confidence based upon team-members behaving within Swarm as well.

There are two interesting insights that jump out when we ask questions like this. The first is that had we initially taken a poll or a survey, we would have thought that Concept B would be faster to implement (Concept B (by a little) had the most support initially). But after participants re-evaluated their original decision based on their own conviction, we see the group converge on Concept A (by a little). The second insight is that the majority of the time was spent deliberating between Concept A (by a little) and Concept B (by a little). And if both projects are going to take nearly the same amount of time, then we should make the decision based on other dimensions and considerations instead (e.g. user preference, maintainability, cross-browser support, etc.)

Our team now uses Swarm for nearly all of our important decisions, including evaluating and hiring candidates

Anyone who has worked on a team before knows that just as important as the “what”, “how” or “when”, is the “who”. Who we work with and who we choose to bring onto our team is one of the single most important decisions we can make.

But traditional methods of collaborating on whom to interview and hire can be awkward or challenging. Especially when candidates are considered based on personal recommendations, we certainly don’t want to offend our colleagues. And when we’re hiring for critical roles, no one wants to make a “wrong” assessment or appear to be non-inclusive, which leads to being “OK with hiring any of the candidates”. Making team-based hiring decisions is hard; really hard. That’s why we use Swarm to help.

Here are a few examples of questions that we typically ask interviewers when they are evaluating candidates together after a round of interviews. The first is to evaluate how long said candidate is “most likely looking to stay in this role?”

Another example is a question that is of the utmost important for engineering teams: assessing the impact that the candidate will have on the culture of the team. In these scenarios, we keep our team member’s identities anonymous while swarming together and reviewing the results, to allow people to separate the emotional/interpersonal aspect of the business decision from the input most useful for hiring.

Swarm helps us, and business teams around the world, make the best decisions possible

At Unanimous AI, we’ve known for over 4 years that Swarm can lead groups of people to make highly optimized predictions, evaluations, assessments and prioritizations. We shared some of the ways that we use Swarm internally because we want to inspire teams like yours make as optimal decisions as possible, too. From teams using Swarm to make medical diagnoses to determining which biochemical research projects to prioritize, we continue to be amazed at the quality of decisions that teams are making by using our technology. We hope you’ll try it too, and let us know what kinds of decisions your team is making better together than with your previous methods. Give Swarm a try at https://unanimous.ai/swarm and let us know what you think!

Thanks for reading,

Alex McClure

Chief Product Officer, Unanimous AI

product@unanimous.ai