Here's tenet #9 from my whitepaper "Tenets of Identity Management". Part #1 of the "Tenets of Identity Management" podcast is posted on the Quest web site. Part #2 is also now available.
Beware of Vendor ROI, TCO, and Other Trappings—YMWV
Face it—vendors have one goal in mind: to separate you from your money. They will provide you with all kinds of ROI and total cost of ownership (TCO) spreadsheets that will promise extraordinary savings in a matter of months. Beware of these reports. Take all of this with many grains of salt. Ask the vendors if they will help you define and, more importantly, stand behind the metrics for the projects to prove those savings and watch them scurry for the exit. Can these reports be used as baselines where savings and ROI might come from? Absolutely. They may give you new ideas for where you can find savings, but don’t take these ideas to your manager or CFO unless you’re willing to put your butt on the line regarding what the vendors are representing. (And my advice to you is that doing so may be a CLM—career-limiting move).
Another area for caution concerns third-party promotional material. In many cases, companies have paid for product reviews or have ensured that the business scenarios in which their products are reviewed will illuminate their products in the best possible light. When industry analysts write papers about market segments such as identity management, they have their own prejudices, angles and chests to thump, notwithstanding the fact that every software company has a room full of people whose job it is to influence those analysts. Again, exercise caution and take everything with a grain of salt.
I have always advised both a top-down and bottom-up approach to choosing vendors. A top-down approach is gathering research from the Web, industry analysts and trade magazines (for example, eWeek, CIO Magazine, Network Computing, Network World). This will provide good information and general guidance, but keep in mind what I said earlier. A bottom-up approach involves reading the vendor’s Web site and product literature, and testing its product in your lab using your business scenarios and use cases.
Remember, your mileage will vary.
identity management, Quest Software