Product Evaluation Preferences
Evaluating a product for purchase is tricky business. As the cost and complexity of the product increases, so does the risk; risk of failure, embarrassment. Your reputation on the line when making a product selection, and decisions have to be made. So Code Project asked its developer community “How do you prefer to evaluate a product?”
From a marketing perspective, there are a number of tactics you can use to promote a product for evaluation. In this survey, developers were presented with a list of common evaluation tactics and asked to rate them. Developers could choose more than one answer, so the percent response total does not add up to 100%.
Survey Results
| How do you prefer to evaluate a product? | Survey period June 1 - June 8, 2020 |
||
|---|---|---|---|
| Response | Votes | Response % (n=758) | % Error (95% confidence/1.96) |
| Just give me a login or download and let me loose | 337 | 44.46% | 3.56% |
| Give me the full docs so I can dig in at my own pace and depth | 321 | 42.35% | 3.54% |
| Let me do a lab with a sandbox system where I can try things out for myself | 308 | 40.63% | 3.52% |
| Cut to the chase: A video or animation that shows me what it does. | 300 | 39.58% | 3.50% |
| A guided walkthrough of the product showing me features on the way | 295 | 38.92% | 3.49% |
| I prefer just to read what others have said about the product | 235 | 31.00% | 3.31% |
| Other | 67 | 8.84% | 2.03% |
| Responses | 758 | ||
The structure of the responses correlate to a spectrum of support requirements directly related to the notoriously independent disposition of a developer.
- The majority of the respondents (44%), want a hands-on opportunity to play with the product to learn strengths and weaknesses firsthand.
- The next tier down (39-42%) wants some measure of integrated support in the form of documentation, a lab, guided tour or video.
- At the opposite end, we have a smaller percentage (31%), of developers who will rely on others for evaluation and guidance.
It’s significant to note that the spread between the fully independent majority and the armchair evaluators was 13%––not a very wide margin.
Additional Approaches and Preferences
While not statistically significant, it is both amusing and useful to take a look at the individual comments in response to a poll. Check out a few highlights below:
The Hybrid Approach
One commenter offered, “I don’t like being the first person in. I’m just not good at reading documents, downloading/installing things, getting them set up, etc.…I prefer to let somebody else do the evaluation, then get an overview from them on how stuff basically works, then take off with experimenting/pushing the boundaries from there.”
The Pragmatist With Due Diligence Approach
Another perspective was offered by a commenter who said, “I know very few who don’t use a multi-step evaluation process. Mine generally is to read advertisements and magazine articles, to determine what the product is supposed to do. I then read “professional” detailed reviews (e.g. sites that review products). What’s the consensus across multiple reviews? From there I read reviews from people who like to go first — do they think it does what it is supposed to do? Again, multiple reviews help form a better picture, good and bad. At this point, if the product is hardware, I make a buy/no-buy decision. If it’s software, I download a demo, limited version, or full version (if possible) and play with it.”
Avoid the Razzle-Dazzle Approach
In the words of this commenterm “If the video content were more than just the usual manic presentations with unreadable screen-shots, surfeited with unnecessary animations, video effects, distracting music tracks …I’d be voting for that.”From a marketing perspective, this comment also rings true for product videos and corporate-recruiting videos. Communication is more effective when we temper our desire to overwhelm and instead provide our audience with the content they need without the fireworks.
The Trust but Verify Approach
References matter, and they have to be real. As this commenter explains, “I do a decent online search/eval and will (by way of example) even check the evaluations where possible with sites like ReviewMeta.com or Amazon Review Checker. This is the same as price checking (even when it’s not my money).”
Some Evaluation Criteria Were…
…Not Captured In The Survey
While not necessarily revealed by the survey responses, the following evaluation criteria will be taken into account by developers:
- The initial perception of quality of the product and support including documentation.
- Benchmarking and the option of an alternative, such as “build it yourself.”
- The importance of references from the developer community
- Cost – is it both affordable and necessary
Give Developers What They Want
To help developers evaluate your product, provide the tools and information for them to understand the benefits the product brings. Consider a multi-tier approach to your advertising and marketing efforts. Offer access to full documentation, a sandbox to try things out, video and animation that shows what the product can do, and how-to’s/tutorials. And, don’t forget to highlight developer reviews. Ultimately, developers all want the same thing: Confidence that they are making the right selection based on a fair evaluation.
Resources
- Code Project https://www.codeproject.com/
- Code Project Survey https://www.codeproject.com/Surveys/2279/How-do-you-prefer-to-evaluate-a-pro
- Software Development Times – 5 Steps to Evaluating Software Tools https://sdtimes.com/evaluation/guest-view-five-steps-evaluating-selecting-software-tools/
