Evaluating Application Lifecycle Management Products

By application lifecycle management (ALM) I mean all the technical/business processes that go into managing software applications end-to-end and all the interactions along the way with other applications, infrastructures, business processes, customers, partners, and vendors. This includes infrastructure management, portfolio management, project management, resource management, demands/finances, processes, builds & automated testing, testing management, deployment management, configuration & version control.

It’s always good to ask the proponents & evaluators of ALM products/solutions the list of questions below in order to see whether they have a viable alternative to Microsoft Team Foundation Server (TFS). Question 21 in particular is vital because it shows the hidden costs (which are less than the costs of all integration being manual) on picking ALM solutions and how this process is primarily an exercise is looking for tight data integration between the ALM processes being built-in. It's a reminder to all of us that the whole term "Application Integration" is a misnomer. It's really "Data Integration" and its where most organizations seem to spend most of their IT dollars and more than they should because most people fail to understand this misnomer.

Questions 20/21 are the #1 reason why I'm a big fan of TFS. The foundation for all of that is all built-in. If you look at the power of Network Isolation in Visual Studio Lab Management and how it tightly integrates with all the other processes/data in TFS, you'll see what I mean. You'll never look at the millions annually that many IT organizations spend making up for theses integration difficulties the same ever again. And frankly I refuse to accept that TFS is any sort of holy grail. If a more effective alternative (or collection of alternatives) comes along with better ROI then I'm all ears.

Here's the questions:

1. How does the product/solution do project planning (including work breakdown, areas & iterations)?

2. How does the product/solution do applications' portfolio management?

3. How does the product/solution do Human Resources Planning for projects/teams? This could include (or at least be integrated easily) information on managing demand/priorities and the financials for the projects/teams.

4. How does the product/solution do infrastructure/machines planning/management?

5. How does the product/solution do requirements analysis?

6. How does the product/solution do testing management?

7. How does the product/solution do automated testing?

8. How does the product/solution do software architecture (i.e. UML, diagrams, etc.)?

9. How does the product/solution do design (i.e. wireframes, mockups, etc.)?

10. How does the product/solution do development/compiling software?

11. How does the product/solution do documentation management?

12. How does the product/solution do version control on project artifacts (i.e. code, documentation, etc.)?

13. How does the product/solution do bug tracking?

14. How does the product/solution do risk management?

15. How does the product/solution do task management?

16. How does the product/solution do environments provisioning/decommissioning?

17. How does the product/solution do environments' templates management?

18. How does the product/solution do auditing/compliance from end-to-end?

19. This is a 18-part question (i.e. on each of the 18 capabilities) - how do you customize the capability in case of a business need (i.e. changing vision, changing processes, reorganizations, mergers, acquisitions, etc.) and the need to integrate with processes not covered in the 17 capabilities listed?

20. This is a 153-part question (i.e. half of 17*18 on each of the first 18 questions) - how do each of these capabilities integrate with each other?

21. This is also a 153-part question - how can each of these integrations be customized if needed in case of a business need?

22. This is a 6+ part question - How does the product/solution integrate with the organization’s current key users' tools - i.e. Office Excel, Office Project, Clipboard, Outlook, Visual Studio, Eclipse, etc.?

23. This is a 5+ part question - How does the product/solution integrate with the organization’s current key server/infrastructure tools - i.e. VMWare, Hyper-V, Active Directory, SQL Server, SharePoint, etc.?

24. How does the product/solution integrate with other product/solutions that the organization is using, thinking of using, partnering with other organizations where they need to integrate, or unplanned future integration needs? For example, DOORS, Rational, Google Apps, etc. or other types of systems (i.e. Oracle (PeopleSoft, Sun, Siebel), SAP, SalesForce.com, etc.), and cloud solutions like Service-Now & Concur?

25. What is the backup/recovery process for the product/solution?

26. What does the potential migration path away from this product/solution to a future ALM product/solution look like?