A competitive advantage comes with focusing on projects that your business can execute on and deliver the most impact with. In this article, we show you how to develop and apply idea evaluation best practices to move the best ideas forward and keep people in the loop.
You’ve launched your IdeaPlace program and are now getting amazing Ideas from previously untapped sources throughout the organization. Maybe you’ve even begun to use IdeaPlace as a better way to engage strategic customers and partners. Congratulations! Now you’re faced with a new challenge and are asking yourself, How can I manage all of these Ideas? You might even be thinking:
- The volume of Ideas is overwhelming;
- What evaluation criteria should I use?
- How can I evaluate duplicate or underdeveloped Ideas? and
- How will I say “no” to some Ideas?
You promoted change. You’ve expressed your desire to listen to your teams to find new areas of improvement, but now you actually have to do something with all these Ideas. Your community is waiting for feedback, waiting for you to make decisions, and, most importantly, waiting for recognition. But where do you begin? How do you make some sense from the insights you’ve received, sort through all the feedback, and begin evaluating?
The good news is you’re not alone. Many organizations struggle with this same challenge, which is why we’ve developed this article compiling approaches our customers have taken to help their teams tackle the challenge of managing all the Ideas in their innovation programs. Your goal should be to thoughtfully evaluate all Ideas while spending as little time and resources necessary in identifying the Ideas with the best potential. It is then those Ideas, those filled with the most potential to solve the problem at hand, that justify increased attention.
Determining Evaluation Criteria
Consider the evaluation criteria before you begin evaluating Ideas to determine which Ideas show the most promise and which Ideas can be easily put to the side. By investing up front in identifying these criteria, the evaluation process will be much simpler and more efficient. Think of the Ideas in your Challenge as your innovation pipeline and evaluation criteria as the funnel you put them through to filter.
You may also use different evaluation criteria at different points in an Idea’s life cycle. For instance, your first round of Idea evaluation may rely solely on popularity while your second or third may require you to take a closer look at the timeline or budget for implementation.
Typical questions evaluation criteria might address include:
- Does the innovation community support this Idea?
- Did it receive a lot of interest through active conversation, voting, or volunteering?
- Is this Idea in line with our strategic goals for this year?
- Is it in line with the goals of this Challenge?
- Do we have the budget to implement it?
- Do we have the necessary tools or team to implement it?
- Can it be implemented within the desired time frame?
- Is the executive committee on board with moving forward with it?
Your evaluation criteria will help you determine which Ideas are worth your investment of time and resources, and which are not a good fit and can thus be declined.
There are two aspects to the strategy for evaluating Ideas: your evaluation workflow and your evaluation timeline. Evaluation workflow encompasses the Stages your Idea will pass through on the way to a decision being made. Stages map to the evaluation criteria you’ll consider at each part of the evaluation process. Your evaluation timeline is the pace at which you’ll move Ideas from one stage to the next. Ideally, you’ve determined both your workflow and timeline prior to your Challenge launch, but if you haven’t, the following two exercises will help you determine a process most appropriate for your organization. See this additional article about Idea Stages.
Exercise: Define Your Stages
To determine the appropriate Stages, start by pinpointing what needs to happen to make the Idea a good fit. Ask yourself: What are my must-haves? What characteristics must this Idea have in order to move forward to the next stage? And what are my deal-breakers? Which attributes make this Idea a no-go? Go through the exercise of writing these down! These characteristics are your evaluation criteria and will help you define your Idea Stages for IdeaPlace.
For example, if all Ideas must pass through the legal review team, Legal Compliance is an evaluation criterion and should be a Stage. If all Ideas must cost less than $10,000, cost should also be part of your evaluation criteria with the potential to become a Stage such as Cost Analysis.
Examples of Stages used by IdeaPlace customers:
- Cost Analysis / Budget Analysis: An analysis of implementation cost is necessary to determine whether you can more forward with this Idea.
- Legal Compliance: An analysis by the legal team is necessary to determine whether you can move forward with this Idea.
- Executive Review / Committee Review: Ideas that you would like to move forward with, that require executive approval or approval by a review committee.
- Innovation Team Review / Under Evaluation / Expert Assessment: This Idea shows potential and is currently being evaluated or assessed to determine whether it can move forward.
- More Information Required / Needs Elaboration: Ideas that require more information before they can be evaluated.
- Duplicate: Another participant has already suggested a similar Idea.
- Mid- to Long-term / Some Day / Backlog / Parked: Ideas that should be reviewed at a later date but are currently paused.
- Declined / Not Progressed: Ideas you want to decline that don’t make sense for your organization at this time.
- Previously Attempted: Declined because it was previously attempted by your organization but was not successful.
- Approved: Ideas that you plan to move forward with.
- Prototyping: Approved Ideas that you are attempting to implement as test cases.
- Completed: Ideas that have made it through the implementation phase.
Exercise: Map out your Evaluation Timeline
Your evaluation timeline depends on a number of factors including the type of challenge you’re running, whether a limited number of Ideas can be approved or “win” the challenge, and whether your Moderators are empowered decision-makers. First identify whether you'll be using an event where Idea submission is ongoing with no set deadline, or a challenge where the timeline has fixed start and end dates. Idea evaluation can happen at specific points, such as during Voting and Expert Review.
Next, consider whether there will be a limited amount of Ideas that can be approved. If there can be only one or two winning Ideas, then you’ll need to wait until the end of the Challenge or until a designated time, when all Ideas have been submitted, to make your final decision. Finally, think about who will ultimately make the decision about Ideas submitted. If Ideas must be approved by a review committee (budget, legal compliance, executive), then your evaluation timing will need to take into account the frequency and timing of committee meetings. However, if your Moderators have full decision-making power and budget approval, they may have more flexibility with timing than a committee that meets at specific intervals.
Define your criteria and strategy for evaluation prior to Challenge launch and be sure to share them with all Moderators.
Share your evaluation criteria with innovators, too, in a IdeaPlace Email about your Challenge. This will provide added context to your Challenge and help to make sure that you get the most relevant Ideas in response.
Managing the Volume of Ideas
It’s important to remember that the purpose of evaluating Ideas is not to simply reduce the volume, but to try and identify, as soon as possible, and with minimal resources or distractions, which Ideas warrant further consideration.
IdeaPlace allows you to effectively evaluate your Ideas in stages. Easy to apply widgets also help you determine which Ideas to respond to first. For example, you might use built-in metrics like star votes, views, or comments, or time of last activity to determine which Ideas are in most need of response, which ones are potential winners, and which can easily be taken out of the mix. Ideas that have broad support from your participants through votes or comments show promise for implementation and are good candidates to move to more comprehensive stages of evaluation, while Ideas that linger and draw minimal engagement may be prime for declining or deferring for future consideration. Using IdeaPlace’s Pairwise feature, the crowd can make decisions on multiple ideas at once, drawing on the power of the crowd to help reduce the volume of Ideas.
By crowdsourcing the validation process; you rapidly identify the best Ideas for Expert Review. Then, after they’ve reduced the volume of Ideas, Experts can spend more time on an in-depth evaluation for those remaining Ideas.
Another approach to handling the volume of Ideas is by scheduled Moderator check-ins. These meetings can be a great way to spread the responsibility of managing a high volume of Ideas. Moderators can collaborate with each other, evaluate Ideas as a team, and discuss any challenges or questions they have about evaluation criteria. Designating a specific time for your Moderators to evaluate Ideas increases the likelihood that it will be accomplished and, by working together, your Moderators can move through Ideas more quickly and ensure a consensus about which Ideas make it to the next stage.
Avoiding Underdeveloped Ideas
The best way to preempt incomplete or underdeveloped Ideas is to be as specific as possible about what you’re looking for when starting a Challenge. Establish a strong context and use the Challenge description in the email to state what your goals are and what factors you’ll need to evaluate Ideas. By being transparent about your evaluation criteria, you can help ensure that participants propose Ideas relevant to the challenge and include all of the information you’re looking for.
If you’ve already launched your Challenge and you’re finding that you keep coming up against underdeveloped Ideas, you can share a IdeaPlace Email reminding participants of the evaluation criteria and expectations for Idea submission and highlight a well-developed Idea that better illustrates what information you’re looking for.
View Ideas with vague details as an opportunity to reach out to the submitter. Suggest a revision to the Idea where you outline the missing information, or send a set of questions you’d like the author to address. To engage the community, add a comment to the Idea eliciting contributions from other participants. With additional feedback from your participants, each with their unique experience and perspective, Ideas can evolve into more fully formed concepts.
Always send an email describing a new Challenge. Use this as an opportunity to catch people’s attention enough to get them to participate, and provide enough context so that people’s submissions best meet the goal of the Challenge.
Managing Duplicate Ideas
People are realistic and understand that not all Ideas can be approved. We hear regularly that people want their Ideas considered—they want to be listened to—but understand the constraints of the organization.
Saying “no” is a necessary and important part of the Idea evaluation process. Declining an idea is as much a decision as approving it. Close the loop on the conversation and steer clear of that void where submitters never learn whether their Ideas were even considered.
An Idea can be declined for any number of reasons: it’s not feasible, for example, or it doesn’t align well with established goals, or it has already been tried without success before. Since not all Ideas will be implemented, it’s important to design a strategy for declining Ideas that cannot move forward. An Idea that is deemed to be impractical but stays in its open state indefinitely appears no different to a participant than an Idea that has never been looked at. Participants need feedback, especially when their Ideas won’t be implemented.
To respond “no,” you may choose to use the built-in Closed - Unsuccessful state, or to create custom states that provide additional detail about why an Idea is being declined. Some examples are:
- Exceeded Budget Limit;
- Turned Down by the Executive Committee;
- Rejected by Client; and
- Failed End-user Testing.
Each decline action should always be accompanied by an explanatory message, even if it just shows that the Idea was considered. Some examples are:
- “This does not meet the budgetary limits for this Challenge but we’re open to reconsidering if you have ideas for cost reduction.”
- “This Idea isn’t projected to meet the minimum cost-saving requirements.”
- “We can’t purchase the software you’ve recommended at this time.”
- “This technique was attempted several years ago, but with little result.”
Being transparent about your reasoning helps mitigate disappointment about an Idea being declined. More than having every Idea approved, most users are concerned that their Ideas are heard and given due consideration. So, don't be afraid to say "no". Just do it transparently and thoughtfully.
Work with your Moderators to design a strategy for saying “no.” This will empower them to take action more quickly and consistently while providing ways to acknowledge the effort put in by the Idea author.
Every business is faced with constraints and most organizations don't have the luxury of allocating valuable resources to projects that don't stand a strong likelihood for success. A competitive advantage comes with focusing on projects that your business can execute on and deliver the most impact with. Through a thoughtful evaluation process, teams can eliminate wasted expenditure on Ideas that aren’t a great fit, and focus on those Ideas that best meet the issues at hand. Everyone is busy, including your Moderators, so create a process that maximizes people’s time while still providing feedback to everyone who took the time to contribute an Idea to your program.