For more than 50 years, Auerbach Publications has been printing cutting-edge books on all topics IT.

Read archived articles or become a new subscriber to IT Today, a free newsletter.

This free newsetter offers strategies and insight to managers and hackers alike. Become a new subscriber today.


Contact

Interested in submitting an article? Want to comment about an article?

Contact John Wyzalek editor of IT Performance Improvement.

 

Using Balanced Scorecard To Measure BYOD Effectiveness

Jessica Keyes

From an organizational perspective, the concepts of performance management are very much the base that supports the balanced scorecard framework. Indeed, the balanced scorecard approach becomes very understandable when one realizes that, instead of being a radical new approach to performance management and measurement, it merely brings together and organizes tried-and-true performance-enhancing "best practices" that companies have been practicing for decades.

Heralded by the Harvard Business Review as one of the most significant management ideas of the past 75 years, balanced scorecard, shown in Figure 1, has been implemented in companies to both measure and manage the IT effort—and by extension BYOD.

Figure 1. The balanced scorecard and its four perspectives.

Robert S. Kaplan and David P. Norton developed the balanced scorecard approach in the early 1990s to compensate for their perceived shortcomings of using only financial metrics to judge corporate performance. They recognized that in this new economy, it was also necessary to value intangible assets, such as the satisfaction an end user gains because he or she is permitted to use his or her own device. Because of this, they urged companies to measure such esoteric factors as quality and customer satisfaction.

By the mid-1990s, balanced scorecard became the hallmark of a well-run company. Kaplan and Norton often compare their approach for managing a company to that of pilots viewing assorted instrument panels in an airplane cockpit—both have a need to monitor multiple aspects of their working environment.

In the scorecard scenario, a company organizes its business goals into discrete, all-encompassing perspectives: financial, customer, internal process, and learning/growth. The company then determines cause–effect relationships, for example, satisfied customers buy more goods, which increases revenue. Next, the company lists measures for each goal, pinpoints targets, and identifies projects and other initiatives to help reach those targets.

Departments create scorecards tied to the company's targets, and employees and projects have scorecards tied to their department's targets. This cascading nature provides a line of sight between each individual, what they are working on, the unit they support, and how that impacts the strategy of the whole enterprise.

The balanced scorecard approach is more than just a way to identify and monitor metrics. It is also a way to manage change and increase a company's effectiveness, productivity, and competitive advantage. Essentially, a company that uses the scorecard to identify and then realize strategic goals is a strategy-focused organization.

For IT managers, the balanced scorecard is an invaluable tool that permits IT to link to the business side of the organization using a "cause-and-effect" approach. Some have likened the balanced scorecard to a new language, which enables IT and business line managers to think together about what IT can do to support business performance. A beneficial side effect of the use of the balanced scorecard is that, when all measures are reported, one can calculate the strength of relations between the various value drivers. For example, the relationship between BYOD usage and cost levels might infer that the usage of BYOD does not sufficiently contribute to results as expressed by the other (e.g., financial) performance measures. Table 1 demonstrates a balanced scorecard approach to measuring the usage of BYOD.

Table 1.Sample Balanced Scorecard Including BYOD Metrics

Martinsons et al. (1999) suggest that the four balanced scorecard perspectives might require some modification to be effective as an IT scorecard. The reason is that the IT department is typically an internal rather than external service supplier and projects are commonly carried out for the benefit of both the end users and the organization as a whole—rather than individual customers within a large market.

The suggested four perspectives include the following:

  1. User orientation (end-user view)
    1. Mission: Deliver value-adding products and services to end users
    2. Objectives: Establish and maintain a good image and reputation with end users, exploit IT opportunities such as BYOD, establish good relationships with the user community, satisfy end-user requirements, and be perceived as the preferred supplier of IT products and services
  2. Business value (management’s view)
    1. Mission: Contribute to the value of the business
    2. Objectives: Establish and maintain a good image and reputation with management, ensure that IT projects provide business value, control IT costs, sell appropriate IT products and services to third parties, and perform BYOD return on investment (ROI)
  3. Internal processes (operations-based view)
    1. Mission: Deliver IT products and services in an efficient and effective manner
    2. Objectives: Anticipate and influence requests from end users and management, be efficient in planning and developing IT applications, be efficient in operating and maintaining IT applications, be efficient in acquiring and testing new hardware and software, and provide cost-effective training and support that satisfies the end users
  4. Future readiness (innovation and learning view)
    1. Mission: Deliver continuous improvement and prepare for future challenges
    2. MissionObjectives: Anticipate and prepare for IT problems that could arise, continuously upgrade IT skills through training and development, regularly upgrade IT applications portfolio, regularly upgrade hardware and software, and conduct cost-effective research into emerging technologies and their suitability for the business (e.g., BYOD, cloud services)

Martinsons et al. (1999) drill down to provide IT-specific measures for each of these four perspectives. Most of the metrics have been derived from mainstream literature (Table 2). Many of them are quite applicable to BYOD.

Table 2. IT Scorecard Metrics

Martinsons et al. (1999) explain that the three key balanced scorecard principles are built into their IT scorecard:

  1. Cause-and-effect relationships
  2. Sufficient performance drivers
  3. Linkage to financial measures

They explain that cause-and-effect relationships can involve one or more of the four perspectives. For example, better staff skills (future readiness perspective) will reduce the frequency of bugs in an application (internal operations perspective).

Progressive scorecard practitioners track metrics in five key categories:

  1. Financial performance. Sample metrics include IT spending in the content of service levels, project progress, cost of data communications per seat and relative spending per portfolio category.
  2. Project performance. Sample metrics include percentage of new development investment resulting in new revenue streams and percentage of IT R&D investment leading to IT service improvements.
  3. Operational performance. Instead of concentrating measurement efforts on day-to-day measures, best-in-class practitioners seek to pro- vide an aggregate, customer-focused view of IT operations. Sample metrics include peak time availability, critical process uptime.
  4. Talent management. This category of metrics seeks to manage IT human capital. Measures include staff satisfaction and retention as well as attractiveness of the IT department to external job seekers. Metrics include retention of high-potential staff and external citations of IT achievement.
  5. User satisfaction. Sample metrics include focused executive feedbacks and user perspective.

The Working Council also found that best-of-breed practitioner included two additional metric categories:

  1. Information security. These metrics monitor remediation efforts for known vulnerabilities and track proactive policy and certification efforts. Sample metrics include percentage of staff receiving security training, percentage of external partners in compliance with security standards, and percentage of BYOD-related breaches.
  2. Enterprise initiatives. Best-of-breed practitioners also use the scorecard to highlight IT's contributions to initiatives of corporate strategic importance, including the introduction and support of novel technological practices such as BYOD. Sample metrics include percentage of acquired company systems integrated in merger and acquisition category and number of business process steps enabled by technology in process reengineering category.

Van Grembergen et al. (2003) did an intensive study of the methodology used by Canada-based Great-West Life to develop their IT balanced scorecard. Great-West Life is the result of a merger of three financial services companies, each with its own IT services department. Stakeholders were quite concerned that they would lose control of their IT groups after the merger, so the merged IT department decided to utilize the balanced scorecard to formalize the controls and measures required to ensure IT success.

The merged IT department consisted of seven units: career centers, management services, account management, application delivery, technology services, corporate technology, and eBusiness solutions center. At the time of the study, the IT department employed 812 full-time and parttime employees.

The organizational structure of the IT department is quite interesting. Application delivery was created as a stand-alone unit to focus on continuous improvement of delivery performance. Account management was created to ensure effective communications with the company's end users. This department takes great pains to educate end users on IT corporate agendas and translate the business needs into IT processes. As its name implies, the career center is focused on the professional development of IT staff. The corporate technology group utilizes a centralized approach to the development of a common enterprise architecture and technology policies. Finally, the management services group focuses on running IT as a business and provides for an effective financial reporting and adherence to the IT scorecard.

As you can see, the organizational structure of the IT department roughly parallels that of the four perspectives of the balanced scorecard:

  1. Financial perspective—Management services
  2. Customer perspective—Account management
  3. Internal perspective—Application delivery, technology services, corporate technology, eBusiness solutions
  4. Learning and growth perspective—Career centers

Senior management of the three companies questioned the benefits of large investments in IT and wanted IT to be better aligned with corporate strategy. Some of the concerns of the different stakeholder groups were as follows:

  1. Senior management
    1. Does IT support the achievement of business objectives?
    2. What value does the expenditure on IT deliver?
    3. Are IT costs being managed effectively?
    4. Are IT risks being identified and managed?
    5. Are targeted intercompany IT synergies being achieved?
  2. Business unit executives
    1. Are IT services delivered at a competitive cost?
    2. Does IT deliver on its service-level commitments?
    3. Do IT investments positively affect business productivity or the customer experience?
    4. Does IT contribute to the achievement of our business strategies?
  3. Corporate compliance internal audit
    1. Are the organization's assets and operations protected?
    2. Are the key business and technology risks being managed?
    3. Are proper processes, practices, and controls in place?
  4. IT organization
    1. Are we developing the professional competencies needed for successful service delivery?
    2. Are we creating a positive workplace environment?
    3. Do we effectively measure and reward individual and team performance?
    4. Do we capture organizational knowledge to continuously improve performance?
    5. Can we attract/retain the talent we need to support the business?

One of the most important initiatives the new Chief Information Officer (CIO) undertook was to migrate the new information services group to a strategic partner as opposed to an IT services provider. As articulated by Venkatraman (1999), and summarized in Table 3, there are some important differences.

Table 3. Moving IT from Service Provider to Strategic Partner

Great-West Life's IT scorecard, as described by Van Grembergen et al. (2003), encompasses the following four quadrants:

  1. Customer orientation. To be the supplier of choice for all information services, either directly or indirectly through supplier relationships
  2. Corporate contribution. To enable and contribute to the achievement of business objectives through effective delivery of value-added information services
  3. Operational excellence. To deliver timely and effective services at targeted service levels and costs
  4. Future orientation. To develop the internal capabilities to continuously improve performance through innovation, learning, and personal organization growth

The relationship between IT and business can be more explicitly expressed through a cascade of balanced scorecards, as shown in Figure 2.

Figure 2. Cascade of balanced scorecards.

Cascading was used effectively at Great-West Life, similarly to the example in Figure 2, with the addition of "governance services" scorecards. You will notice the use of the term "scorecards" (plural). Each set of scorecards is actually composed of one or more unit scorecards. For example, the IT operations scorecard also includes a scorecard for IT service desk. Great-West Life's four quadrant IT scorecard consists of objectives, measures, and benchmarks, as shown in Tables 4 through 7.

The measures of each of these unit scorecards are aggregated in the IT balanced scorecard. This, in turn, is fed into and evaluated against the business balanced scorecard.

Table 4. Corporate Contribution Scorecard Evaluates IT from the Perspective of Senior Management



Table 5. Customer Orientation Scorecard Evaluated the Performance of IT from the Perspective of Internal Business Users



Table 6. Operational Excellence Scorecard Views IT from the Perspective of IT Managers and Audit and Regulatory Bodies



Table 7. Future Orientation Perspective Shows IT Performance from the Perspective of the IT Department Itself: Process Owners, Practitioners, and Support Professionals


References

Martinsons, M., Davison, R., and Tse, D. (1999). The balanced scorecard: A foundation for the strategic management of information systems. Decision Support Systems. 25(1), 71-88.

Van Grembergen, W., Saull, R., and De Haes, S. (2003). Linking the IT balanced scorecard to the business objectives at a major Canadian Financial Group. Journal of Information Technology Cases and Application. 5(1), 23-50.

Venkatraman, N. (1999). Valuing the IS contribution to the business. Computer Sciences Corporation Report.


Read more IT Performance Improvement

This article is an excerpt from:

For many healthcare professionals BYOD is a necessity rather than a privilege to use their own devices. This book helps IT healthcare professionals manage the BYOD phenomenon and understand its strategic and tactical ramifications. The book covers a broad range of technical considerations such as selection, connectivity, training, support, and security.

About the Author

Jessica Keyes is president of New Art Technologies, Inc., a high-technology and management consultancy and development firm started in New York in 1989.