Journal Article Analysis Each student will select one of the key terms presented in the module and conduct a search of Campbellsville University’s online Library resources to find 1 recent peer-reviewed academic journal article (within the past 3 years) that closely relates to the concept. Your submission must include the following information in the following format: DEFINITION: a brief definition of the key term followed by the APA reference for the term; this does not count in the word requirement. SUMMARY: Summarize the article in your own words- this should be in the 150-200 word range. Be sure to note the article’s author, note their credentials and why we should put any weight behind his/her opinions, research or findings regarding the key term. DISCUSSION: Using 300-350 words, write a brief discussion, in your own words of how the article relates to the selected chapter Key Term. A discussion is not rehashing what was already stated in the article, but the opportunity for you to add value by sharing your experiences, thoughts and opinions. This is the most important part of the assignment. REFERENCES: All references must be listed at the bottom of the submission–in APA format. Be sure to use the headers in your submission to ensure that all aspects of the assignment are completed as required. Any form of plagiarism, including cutting and pasting, will result in zero points for the assignment .  

 The following terms for your research:   Balanced scorecard  

JANUARY-APRIL 2020 DOI NUMBER: 10.19255/JMPM02212

JMPM Issue #22 | Vol.07 Num.04

Measuring PMO Performance – Application of the

Balanced Scorecard in a Collaborative Research

Context

Simon P Philbin

London South Bank University

Rajneet Kaur

Imperial College London, United Kingdom

Abstract: This a

rt

icle provides a discussion and analysis of the successful deployment by a

university-based project management office (PMO) of the balanced scorecard as a performance

measurement tool. The research study builds on a supporting literature review on the balanced

scorecard along with background material on collaborative research projects. This is followed

by a case study investigation of the design and implementation of the scorecard for a university

PMO over a 4-year period. Various managerial insights have been generated that have value to

project management professionals engaged in the roll-out of a performance measurement

system to support the management of research projects. There is a need to carefully adapt

scorecard metrics to the academic requirements regarding the management of a portfolio of

research projects. Furthermore, although data collection is necessary for the sustained use of

the scorecard to support team operations, it is also essential to consider the people or social

dimensions when utilizing the scorecard approach. The article also includes specific details on

how scorecard’s key performance indicators have been derived through distilling strategic

objectives into operational requirements.

Keywords: Project Management Office, PMO, Balanced Scorecard, Performance

Measurement, University

JAN-APR 2020 JOURNALMODERNPM.COM

2 Measuring PMO Performance …

Introduction

The PMO (project management office) is an organizational unit that is established in order to

standardize how projects are managed and to secure efficiencies through generating best

practices from the delivery of a portfolio of projects (Desouza and Evaristo, 2006). The PMO

approach to project management has been deployed in different sectors, such as construction

information systems (Ward and Daniel, 2013), the public sector (Esquierro et al., 2013), and

research administration (Wedekind and Philbin, 2018). This is because many organizations

increasingly deliver activities according to projects; indeed, projects as temporary

organizations can be viewed as a production capability, as a unit for allocating resources to

change initiatives, and as a unit for managing uncertainty (Turner and Müller, 2003). We can

also consider that projects may be delivered for clients as part of funded contracts as well as

there being internally commissioned projects, such as those associated with business and digital

transformation. While projects and project management are highly prevalent in the modern

organization, some projects continue to encounter difficulties, for instance, in the case of IT

(information technology) projects (Nelson,

2007).

The project management office provides a mechanism to allow organizations to maximize the

likelihood of project success. Furthermore, the PMO is able to develop project management

best practices through the successive delivery of multiple projects. This knowledge can be

augmented by various tools, techniques, and standard operating procedures that a PMO

develops to support the project management community within the organization (do Valle et

al., 2008). In regard to the positioning of a PMO, the Project Management Body of Knowledge

(PMBOK®) has identified three different types of PMOs, which are supportive, controlling,

and directive (PMI, 2013). Each of these types has a different level or extent of involvement in

the delivery of project management within the organization, where supportive PMOs are more

advisory in nature, controlling PMOs provide governance for projects, and directing PMOs

provide both of these functions in addition to the provision of resourcing for the

project

management function. Moreover, Dai and Wells (2004) identified a set of standard features for

a PMO, which are as follows: Developing and maintaining project management standards and

methods; developing and maintaining project historical archives; providing project

administrative support, providing human resource/staffing assistance, providing project

management consulting and mentoring, and providing or arranging project management

training.

Projects are traditionally delivered according to the so-called ‘iron triangle’, i.e. delivery of the

project to meet the schedule, budget and specification criteria while achieving a required

quality level (Atkinson, 1999), but the successful functioning of a PMO will rest on many

factors. There is the ability of the PMO to secure new projects along with the various

financial

metrics associated with the development and subsequent delivery of a portfolio of projects;

there are factors associated with the PMO resourcing, such as resource deployment and

training; there are process and systems related factors, and there are yet further factors

associated with the expectations of both internal and external stakeholders of the PMO. The

adoption of an integrated performance measurement system should, therefore, be able to

JAN-APR 2020 JOURNALMODERNPM.COM

3 Measuring PMO Performance …

accommodate a range of different factors, and in this context, the balanced scorecard is well

suited to this application.

Collaborative research projects offer a number of benefits, such as enabling an exchange of

knowledge between collaborators, increasing the scope and scale of data generated by

research

studies, and allowing access to complementary research infrastructure (such as experimental

equipment and numerical models). However, there are also certain challenges associated with

research collaborations (Cummings and Kiesler, 2005), and these include coordination costs,

development of multidisciplinary teams, ensuring alignment of the collaborative partners, and

generating the required level of impact from the project. Moreover, if collaborative research

and technology projects are to generate potential solutions to address societal needs, across

areas such as improved healthcare solutions, mitigating the effects of climate change and new

forms of transportation, it is important that such projects can be managed to a high degree of

success. This level of performance can be viewed in terms of achievement of the project

milestones in the required timeframe and according to the quality, budget and schedule

requirements, but the performance is also predicated on the research including the necessary

creativity and scientific freedom to facilitate developments in the particular scientific or

engineering discipline. In this context, it is important to have the necessary systems and

processes to enable the effective performance measurement of research projects and especially

in the case where a portfolio of projects is managed by a PMO.

This article includes details of an investigation into the use of the balanced scorecard to support

performance measurement for a PMO focused on the delivery of a portfolio of collaborative

research projects. The structure of the article is as follows. After the introduction, there is a

literature review on the balanced scorecard, and this is followed by background material on

collaborative research projects. This is followed by the method section and the results from a

case study investigation of a university-based PMO, where the balanced scorecard has been

successfully implemented and used to support operations over a four-year period. The case

study includes a discussion of how the scorecard was designed from a strategic perspective and

how it was deployed from an operational perspective. This is accompanied by lessons learned

from the case, conclusions, and future work.

Literature review on the balanced scorecard

The balanced scorecard was originally developed to support the strategy development process

and help organizations to measure business unit performance (Kaplan and Norton, 1996). The

scorecard has four perspectives (namely financial, customer, internal business processes, and

learning and growth), which provide a balanced view of the operating performance of a

business unit along with the corresponding drivers contributing to future performance. A

scorecard is a tool that offers a number of organizational benefits, and these can be summarised

as follows: It provides a top-down perspective on an organization’s mission and overall

strategy; it captures current as well as future success and so is inherently forward-looking; it is

broad-based through integrating both internal and external perspectives; it allows organizations

JAN-APR 2020 JOURNALMODERNPM.COM

4 Measuring PMO Performance …

to focus only on the performance metrics that are the most important and have the highest

priority (Kaplan and Norton, 1998).

A core feature of the scorecard is clearly the balanced approach – it seeks to move beyond the

historical viewpoint of managing organizations through solely relying on tracking financial

performance (Kaplan and Norton, 2001). Finances are, of course, necessary to all organizations

– industrial companies need to make a profit from operations, and even charities need to

generate revenues to cover operational costs and ideally make a surplus for reinvestment into

the organization. The scorecard, therefore, links operational performance with both customer

and financial performance, while capturing metrics on innovation, employee capabilities,

technology, organizational learning, and customer success (Kaplan, 2009).

The balanced scorecard has been investigated and applied in many different organizational

situations and applications including, for instance, the healthcare sector (Zelman et al., 2003),

textile industry (Cebeci, 2009), the retail sector (Biggart et al., 2010), manufacturing

(Fernandes, 2006), banking sector (Wu, 2012) as well as government and non-profit agencies

(Niven, 2011). Despite such widespread investigation and usage of the tool to support a range

of different types of organizations, there have also been issues raised about the scorecard.

While Mooraj et al. (1999) have reported that the scorecard adds value through the provision

of both relevant and balanced information in a concise manner for management, they also

highlight that use of the scorecard is reliant on both formal and informal processes and they

suggest that further research is required to understand the cost-benefit outcome for adoption of

the scorecard.

Others argue that in the public healthcare sector while the scorecard can be adapted for strategic

performance management purposes, it does not give sufficient weight to the political aspects

and wider context for public sector organizations (Chang, 2007). It has also been pointed out

that there have been numerous iterations of the scorecard over the last 20 years and that there

is a need for an agreed taxonomy for such iterations, which would provide a comparison of the

research findings (Perkins et al., 2014). Nevertheless, the balanced scorecard continues to be

deployed in organizations to support performance measurement and advocated as a potential

enabler for strategy implementation (Atkinson, 2006).

In regard to the higher education sector, there are a number of research studies that have

examined the scope and application of the balanced scorecard (Lawrence and Sharma, 2002;

Karathanos and Karathanos, 2005). The scorecard has been implemented as a tool for

measuring the impact of industry-university collaborations (Al-Ashaab, 2011), where there is

a need to ensure adequate measurement of the outputs of research and development (R&D)

arising from collaborative research projects. The scorecard has been implemented to support

the operational management of a university institute that also included a need for management

of the university-industry interface in order to ensure the requirements of the institute funders

were accommodated (Philbin, 2011).

Pertaining to the strategic context for higher education institutions, there is a need for such

organizations to meet the stated missions by offering educational value to the students. This

often has to be achieved while management processes are improved along with adequately

JAN-APR 2020 JOURNALMODERNPM.COM

5 Measuring PMO Performance …

containing costs – moreover, universities need to effectively and efficiently use scarce

resources, including intellectual capital, funding from government and other sources as well as

people and the available time. The balanced scorecard has been described as a useful approach

to help higher education institutions to operate in such an arena (Beard, 2009).

The utility of the scorecard has been explored as a strategic management tool to support

performance evaluation to help the Taiwanese higher education sector to address the significant

challenge in regard to supply and demand (Chen et al. 2006). In India, the scorecard has been

found to provide higher education institutions with an opportunity to assemble a cascade of

measures to translate the institution’s mission of knowledge creation, sharing and utilization

into a single framework that is comprehensive, coherent, communicable and mobilizing for

institutional stakeholders (Umashankar and Dutta, 2007).

In terms of scorecard derivatives, it has been argued that the scorecard, which is a performance-

oriented approach can be combined with a process-oriented approach (such as ‘Hoshin Kanri’)

in order to create synergy and this has been examined in the context of an

engineering

management graduate programme at a higher education institution (Serdar Asan and Tanyaş,

2007). More broadly a strategy formulation framework for vocational education has been

investigating through integrating the balanced scorecard with SWOT (strengths, weaknesses,

opportunities, and threats) analysis, QFD (quality function deployment) methodology and

MBNQA (Malcolm Baldrige National Quality Award) education criteria (Lee et al., 2000).

These studies point to the extended utility of the scorecard and the potential for it to be

integrated with other management support tools and processes.

In regard to the research reported herein, we have sought to recognize the merit of applying the

scorecard as part of the management infrastructure to support the strategic orientation and

operational delivery for a university-based PMO. Indeed, Kutsch et al. (2015) have used the

scorecard as an analytical lens to focus on the contribution of the project management office

through leading to an increase in project efficiency, reduction in project costs as well as

improved success rates for project delivery.

Background on collaborative

research

projects

In terms of the broader context for collaborative research projects, there is an increasing trend

towards research being collaborative in nature. Indeed, bibliometric research by Wuchty et al.

(2007), which was based on an analysis of 19.9 million papers and 2.1 million patents over 5

decades, demonstrated that teams increasingly dominate solo authors in the production of

knowledge (see Table 1).

JAN-APR 2020 JOURNALMODERNPM.COM

6 Measuring PMO Performance …

Table 1: Knowledge production patterns by academic field (source of data: Wuchty et al.,

2007).

Fields Nfields Increasing

team size

RTI > 1

(with self-citations)

RTI > 1

(no self-citations)

Nfields % Nfields % Nfields %

Science &

engineering

171 170 99.4 167 97.7 159 92.4

Social

sciences

54 54 100.0 54 100.0 51 94.4

Arts &

humanities

27 24 88.9 23 85.2 18 66.7

Patents 36 36 100.0 32 88.9 – –

This large-scale study included ISI (Institute for Scientific Information) Web of Science data

covering research publications from science and engineering since 1955, social sciences since

1956, arts and humanities since 1975, and the US-registered patents since 1975. In this regard,

Table 2 highlights the number (N) and percentage (%) of the subfields that exhibit larger team

sizes in the last 5 years compared to the first five years. The data was based on the relative

team impact (RTI) for a given time period and field, where RTI is the mean number of citations

obtained by team-authored work divided by the mean number of citations obtained by solo-

authored work. The data highlights that in the case of no self-citations and for science and

engineering, 99.4% of the 171 subfields have experienced increased levels of collaborative

activity, with the corresponding figures for social sciences, arts & humanities, and patents

being 100.0%, 88.9%, and 100.0%. Therefore, this data indicates that there is an increasing

trend of scientific research studies being delivered by collaborative teams.

In regard to the trend towards international collaboration, we can consider data from the United

Nations Educational, Scientific, and Cultural Organization (UNESCO, 2015). Figure 1

provides data on the percentage (%) of publications with international co-authors from 2008-

2014. The data is based on total publications across all major fields of science and is provided

for the top 12 countries based on the total number of international publications. This highlights,

for instance, that 55.9% of scientific publications from the United Kingdom (UK) involve

international co-authors, while the figures for the USA and Japan are 34.8% and 27.1%,

respectively. Although there are variations between countries, it is evident that

international

research collaboration is a major feature associated with modern science and technology.

JAN-APR 2020 JOURNALMODERNPM.COM

7 Measuring PMO Performance …

Figure 1: Level of international co-authored publications per country (source of data:

UNESCO, 2015).

Focusing on the case for collaborative research projects, a number of benefits for the

researchers involved have been identified, and this includes researchers based at higher

education institutions as well as those at other organizations, such as government labs, research

institutes, and industrial companies. These benefits have been assembled based on the authors’

experience in managing collaborative research projects over the past 20 years and are

summarized in Table 2.

Table 2: Benefits for collaborative research (source: author’s experience).

Benefits for conducting collaborative research projects

• Enable researchers to work together in order to complement their skills and knowledge.

• Allow the exchange of information and knowledge between collaborators.

• Enable multidisciplinary research studies to address major technical challenges, e.g. societal,
industrial, or academic challenges.

• Provide access to specialized equipment, numerical models or other research infrastructure.

• Support the expansion of the field of data available for research studies.

• Provide scope to explore joint publishing opportunities.

• Support researcher mobility between institutions.

• Help secure research funding designated for supporting collaborative

projects.

While there are compelling reasons to undertake collaborative research projects, there are also

certain challenges associated with managing such projects. These challenges can be articulated

in terms of the technical, commercial, and people-based (or social) aspects of the projects.

Table 3 provides a summary of the challenges for managing collaborative research projects,

and these insights are based on the authors’ experience in this area. In the context of both the

benefits as well as challenges for managing collaborative research projects, it can be observed

that the PMO is an ideal supporting mechanism to enable the delivery of a portfolio of research

projects. Moreover, it is important to implement an effective performance measurement system

for the PMO, and the balanced scorecard is well suited to this application.

24.4

27.1

34.8

46.0

47.8

50.4

51.6

52.6

54.3

55.9

58.3

68.9

0.0 20.0 40.0 60.0 80.0

China

Japan

USA

Italy

Spain

Canada

Australia

Germany

France

United Kingdom

Netherland

s

Switzerland

Percentage (%) of Publications with International
Co-authors (2008-14)

Percentage (%)
of publications

JAN-APR 2020 JOURNALMODERNPM.COM

8 Measuring PMO Performance …

Table 3: Challenges for collaborative research projects (source: author’s experience).

Technical Commercial People (social)

• Availability of research
resources (staff, facilities,

materials).

• Maintaining research
quality while meeting

project requirements

(schedule, cost, quality).

• Generating sufficient
‘impact’ over the required

timeframe.

• Inadequate planning of
technical aspects of the

project.

• Ensuring alignment with
industry needs (in regard to

products and services).

• Availability of funding
(industry, government, or

other).

• Ensuring financial costing
captures all the project

costs.

• Lack of a robust

business

case to support the project.

• Flexibility in

commercial

arrangements, e.g. for IPR

(intellectual property

rights).

• Other legal matters,
especially for international

contracts.

• Managing contributions
from multiple

partners.

• Managing delivery across
international borders,

different languages, and

cultures.

• Insufficient leadership of
the project, either academic

or commercial.

• Establishing effective
multidisciplinary teams.

• Maintaining regular and
open communications with

partners.

Method

The method employed in the research study is based on a case study investigation (see Figure

2). The method involves three main stages, namely literature review and background, case

study investigation, and deductive reasoning. A process of reflective analysis (Schön, 1987)

was employed to ascertain how the scorecard was designed and implemented at a university-

based PMO in the United Kingdom. The method also involves a process of deductive reasoning

(Johnson-Laird et al., 2017) in order to derive the findings from the case study that are

structured according to three main areas, namely strategic perspective – design of the scorecard

(1), operational perspective – deployment of the scorecard (2), and lessons learned from the

case (3).

The authors were responsible for the direction and operational management of the PMO team,

and the scorecard was utilized over a 4-year period (2014 to 2018), and this involved production

of a scorecard report on a monthly basis and consequently 48 successive monthly reports were

generated. The scorecard reports were used to implement the strategic objectives for the PMO

and support the operational delivery of a substantial portfolio of collaborative research projects.

JAN-APR 2020 JOURNALMODERNPM.COM

9 Measuring PMO Performance …

Figure 2: Research methodology employed in the study.

Case study investigation

The PMO featured in the case study was established with a mission to provide high-quality

project management services to academic teams at the university engaged in

the delivery of

collaborative projects. The projects were funded from a range of different sources,

including

the European Commission’s Horizon 2020 programme (European Commission, 2019), as well

as other international organizations engaged in collaborative funding projects. The PMO team

provided a range of integrated services to support the academic units at the university, and this

included consortium management, project management, and commercial services as the

primary capabilities alongside a distinct focus on process and systems management. Figure 3

provides a schematic view of the core capabilities of the PMO.

Figure 3: Schematic view of the core capabilities of the PMO.

(1). Literature review
and background

• Supporting data
and information
on PMO and the
need for
performance
measuremen

t

• Literature review
carried out on the
balanced
scorecard

• Background
material on
collaborative
research projects

(2). Case study
investigation

• Reflective inquiry
of PMO case
study at a
university

• Managerial
insights captured
on PMO structure
and management
processes for
measuring the
performance of a
portfolio of
research projects

(3). Deductive
reasoning

• Findings from the
case study
organised in three
main areas

• Strategic
perspective –
Design of the
scorecard (1)

• Operational
perspective –
Deployment of
the scorecard (2)

• Lessons learnt
from the case (3)

C
o

n
so

rt
iu

m

m
a

n
a

g
e

m
e

n
t

P
ro

je
ct

m

a
n

a
g

e
m

e
n

t
C
o

m
m

e
rc

ia
l

se
rv

ic
e

s

Process and systems management

Academic
value generated

PMO level coordination

JAN-APR 2020 JOURNALMODERNPM.COM

10 Measuring PMO Performance …

The PMO provided both coordination and oversight of research projects that were ultimately

driven by the need to deliver academic value across the university. Furthermore, the PMO

adopted a series of management processes that were geared towards the needs of academic

teams engaged in the delivery of collaborative research projects. The structures and

processes

of the PMO team were designed according to recognized best practices for project management

to ensure that benefits were realized and project risks were properly managed (Office of

Government Commerce, 2002).

The PMO supported academic teams across the university to deliver two main types of

technical projects; these were European Commission-funded consortium projects and

academic-driven commercial projects. Both of these types of projects directly underpinned the

achievement of the university’s organizational strategy. Moreover, it was important that the

PMO’s scope of activities and projects are strongly aligned with the strategic direction of the

university. This alignment and positioning of the PMO, as well as the two main types of

projects supported, is conceptualized in the strategy diagram in Figure 4.

Figure 4: Strategic alignment of the PMO and projects supported.

The resources available to the PMO include staff and non-staff areas as part of a clearly defined

organizational structure. The PMO team included a team leader (PMO director), operations

manager, project managers, the back-office administrative team as well as other specialists,

such as contracts manager. The team worked together to enable support to be provided to the

two main types of projects and in accordance with the PMO level strategy that is closely aligned

with the university’s organizational level strategy. The team included the required levels of

experience and knowledge to ensure high-quality project management is available, and this

includes the necessary project management certification (namely the European PRINCE2

standard) held by team members. As mentioned previously, the PMO provided project

management for two types of projects, and it is useful to provide further details on these

projects (see Table 4).

University Strategy
• Strategy includes research, education and knowledge exchange activities geared to meet the strategic objectives of the univer sity
• PMO ensures close alignment of projects delivered to meet academic needs and according to overall strategy of university
• This alignment is maintained through an integrated set of governance and reporting arrangements

U
n

i

v
e

rs
it

y

st
ra

te
g

y

European Commission Funded Consortium Research Projects
• Leading research in strategically important areas, such as healthcare, security, sustainable energy

and integrated transport systems
• Ability to attract high quality PhD students as well as supporting researcher mobility

Academic-Driven Commercial Projects
• Research and technical services delivered for a range of partners
• Projects based on delivery of knowledge -based services across different sectors (

healthcare,

engineering, and other areas) for international partners

PMO
• Supporting European Commission funded consortium research projects and academic-driven commercial projects
• Professional services with significant experience of international projects as well as commercial services
• Process driven approach, standard operating procedures, and supporting systems to enable project delivery

A
ca

d
e

m
ic

p
ro

je
ct

s
d

e
li

v
e

re
d

b
y

u
n

iv
e

rs
it

y
P

M
O

S

u
p

p
o

rt

JAN-APR 2020 JOURNALMODERNPM.COM

11 Measuring PMO Performance …

Table 4: Features and details of project types supported by the PMO.

Project type Project features Sector/areas Project outputs PMO services

Consortium

research projects

Projects involve a

number of

consortium

partners

(universities,

research

institutes, and

companies)

working together,

typically over 4-5

years to achieve a

series of research

goals.

Projects across

various aspects of

healthcare,
including

pediatrics and

other areas (low

TRL). Also,

engineering
projects

involving the

development of

new technologies

(higher TRL).

Research outputs,

including PhDs,

graduated,

publications,

papers, etc.

The impact

generated across

societal aspects

(

improved

healthcare),

industrial

development and

economic

outputs.

Provision of

project

management for

the overall

consortium,

managing project

delivery and

periodic

reporting,

communication

and knowledge

dissemination

activities.

Academic-driven

commercial
projects

Projects involve

the delivery of

technical services

for partners based

on knowledge-

driven activities,

such as analysis,

advisory services,

and testing.

Projects are

across healthcare

and engineering

sectors, typically

at a high TRL

level in most

cases.

Knowledge-

exchange with

commercial

partners, capacity

building with

international

partners,

improved

technical capacity

and knowledge

provision.

Provision of
project

management,

including

tracking

deliverables and

milestones. Also,

commercial and

financial

administration

activities.

As can be observed, the two main types of projects have different features and characteristics.

The consortium projects were research-based, with much of the project management being

focused on supporting the delivery of consortium activities across the various partners. The

projects included fundamental research at a lower TRL (technology readiness level), such as

TRL1-2, as well as technology development projects with a mid-level TRL of 3-4. Conversely,

the academic-driven commercial projects were typically at the higher TRL level of 5-6. For

further information on the use of technology readiness levels to characterize research at the

different stages of development, see the work of Mankins (2009). Also, the commercial

projects involved the delivery of various knowledge-based services, such as technical analysis,

advice, and testing. Both types of projects were supported by project management delivered by

the PMO, although the emphasis of the management support varies and was tailored according

to the needs of the project and the academic team. In the case of the consortium projects, the

project management was focused on supporting the project’s PI (principal investigator) to

deliver the research objectives across the consortium of partners. Conversely, in the case of the

commercial projects, the project management included traditional management of deliverables

and milestones combined with commercial services, including contractual and financial

administration.

Delivery of the project portfolio was based on the provision of project management across the

full project lifecycle, including supporting pre-award (proposal development) and post-award

(project delivery) activities. The delivery of high-quality support to academic teams requires

JAN-APR 2020 JOURNALMODERNPM.COM

12 Measuring PMO Performance …

close working with other professional services teams at the university as well as providing

close support to the principal investigators. This work was supported by internal

communications (including in-reach

events

, workshops, and other activities) as well as external

communications (including the use of social media and websites). Adoption of an integrated

communication strategy, therefore, supports the development of new projects, maintaining the

project portfolio, and helping to underpin the financial sustainability of the

PMO team.

Moreover, the project portfolio was delivered through a series of tailored management

processes designed to ensure projects are properly controlled, and progress is monitored

through project and team level reports (including use of key performance indicators and the

balanced scorecard). The provision of high-quality project management (as evidenced by

academic and customer level feedback as well as achievement of project objectives) was also

dependent on access to the necessary management tools, standard operating procedures (SOPs),

and systems including appropriate ICT (information and communications technology)

infrastructure.

Discussion of the case

Strategic perspective – Design of the scorecard

The scorecard used by the PMO team was designed to enable the PMO to deliver its strategic

objective of providing high-quality proposal and project management support to academic

teams involved in the proposal writing and delivery of collaborative research projects across

the university. The PMO was responsible for ensuring that projects were delivered according

to the budget, schedule, and scope as well as quality requirements. This involved a series of

project management tasks focused on the planning, organizing, controlling and monitoring of

activities for collaborative research projects. Consequently, commercial project management

practices were required to support this process but the PMO was operating in a university (i.e.

non-profit) environment, and therefore care was needed in how commercial approaches were

adopted as a university should be focused on knowledge creation and sharing as a primary

driver and not the generation of commercial benefits.

In order to provide clarity on the strategy formulation process for the PMO and how this relates

to the four perspectives of the balanced scorecard, Figure 5 illustrates the cause-and-effect

links for the PMO team, adapted from Mooraj et al. (1999, p. 483). This diagram translates the

strategy and objectives of the PMO team into broad areas of requirements for the scorecard

according to the four specified perspectives. This approach also highlights the interconnectivity

between these broad areas and how the different perspectives are aligned to support the

achievement of strategy through the eventual achievement of financial performance by the

PMO. In order to give clarity on the scorecard metrics for the university-based PMO, Table 5

provides more detailed requirements along with definitions, according to Kaplan and Norton

(1996, p. 54).

JAN-APR 2020 JOURNALMODERNPM.COM

13 Measuring PMO Performance …

Figure 5: Cause-and-effect links for the PMO team according to the scorecard perspectives,

adapted from Mooraj et al. (1999, p. 483).

Perspective Definition Requirements for scorecard metrics

Financial “To succeed financially,

how should we appear

to our shareholders?”

• There is a need to monitor the performance of

the PMO team regarding the financial value of

proposals submitted and

awarded.

• Other financial areas to be considered include

outstanding debt levels on projects as well as

the value of charges against PMO managed

projects.

Customer

“To achieve our vision,

how should we appear

to our customers?”

• There is a need to monitor the level of

engagement of the PMO team with the ‘internal

customer base,’ which is the academic

community.

• Other areas include the number of new projects

that have been initiated with external customers

as well as communications and social media

activities with stakeholders.

Le
a

rn
in

g
a

n
d

g

ro
w

th
C

u
st

o
m

e
r

F
in

a
n

ci
a

l
In

te
rn

a
l b

u
si

n
e

ss

p
ro

ce
ss

e
s

Financial
performance

of projects

Skills
development

Project
management
capabilities

New
projects
awarded

Team
sustainability

Standard
operating

procedures

Social media
activities

ICT
systems

Project opportunities
identified

Academic
engagement

Communications
and dissemination

events

JAN-APR 2020 JOURNALMODERNPM.COM

14 Measuring PMO Performance …

Internal

business
processes

“To satisfy our

shareholders and

customers, what

business processes must

we excel at?”

• There is a need to monitor the performance of

the PMO team regarding the development of

new processes and procedures as part of

capturing best practices for project management

and other functional areas.

• Other areas include the use of new ICT systems

by the PMO team to support efficiency

improvements.

Learning and

growth

“To achieve our vision,

how will we sustain our

ability to change and

improve?”

• There is a need to monitor the performance of

the PMO team regarding the level of training

undertaken across the whole team.

• Other areas include the need to measure the

number of external presentations given by team

members as part of their professional

development.

Table 5: Requirements for scorecard metrics, with definitions according to Kaplan and

Norton (1996, p. 54).

These requirements can be further refined to provide a specific set of key performance

indicators that were part of the scorecard. Consequently, Table 6 provides a summary of the

balanced scorecard developed to support the university PMO and the 15 KPIs across the four

perspectives. This set of KPIs was carefully assembled to reflect the relative areas of

‘performance concentration’ according to the strategy and objectives of the PMO team. It can

be observed that the scorecard includes an uneven number of KPIs across the four perspectives,

which reflects there is a greater concentration of performance in the financial perspective (6

KPIs) and customer perspective (5 KPIs) when compared to the internal business processes

perspective (2 KPIs), and learning and growth perspective (2 KPIs). However, this does not

mean that monitoring the performance of the team in these latter two perspectives is a lower

priority than the former two perspectives, but rather it is a reflection of the concentration and

number of areas that need to be measured to properly monitor and control the performance of

the team. Conversely, implementation of the scorecard for another team in a different

organizational scenario may result in a different set of KPIs with a corresponding different

level of performance concentration across the four perspectives.

Perspective KPI Description Comments

Financial 01 Value of proposals submitted to

funding bodies.

• This is the financial value (£) of

proposals submitted to funding bodies

during the

month.

• The financial value (£) is tracked

against an annual target.

02 Value (consortium) of new

projects awarded.

• This is the financial value (£) of new

projects awarded (based on the

university and collaborative partner

allocations) during the month.

JAN-APR 2020 JOURNALMODERNPM.COM

15 Measuring PMO Performance …

• The financial value (£) is tracked
against an annual target.

03 Value (institutional) of new

projects awarded.
• This is the financial value (£) of new
projects awarded (based on the

allocation for the university only)

during the month.

• The financial value (£) is tracked
against an annual target.

04 Value (team) of new projects

awarded.
• This is the financial value (£) of new

projects awarded during the month.

• The financial value (£) is tracked
against an annual target.

05 Value of staff charges to PMO

managed projects.

• This is the financial value (£) of staff

charges (or bookings) made to PMO

managed

projects during the month.

• The financial value (£) is tracked
against an annual target.

06 Value of past due to receivables

on PMO managed projects.

• This is the financial value (£) of

outstanding debt from customers on

PMO managed projects during the

month.
• The financial value (£) is tracked
against an annual target.

Customer 07 The number of university

academics supported by the

PMO team on proposals (pre-

award).

• This is the number of academic

members of staff (principal

investigators and co-investigators)

that are engaged in the development

of proposals and negotiation of

contracts during the month.

• The number of academics is tracked

against an annual target.

08 The number of university

academics supported by the

PMO team on projects (post-

award).
• This is the number of academic
members of staff (principal
investigators and co-investigators)

that are engaged in the delivery of

projects during the month.
• The number of academics is tracked
against an annual target.

09 The number of new projects

initiated by the PMO team.

• This is the number of new projects

that have been initiated during the

month.

• The number of projects is tracked

against an annual target.

10 The number of events and

communications activities

delivered by the PMO team to

• This is the number of events, such as

workshops and other significant

communications activities that were

JAN-APR 2020 JOURNALMODERNPM.COM

16 Measuring PMO Performance …

grow project management

engagement.

delivered by the PMO team during the

month.

• The number of events and

communications activities is tracked

against an annual target.

11 Level of social media activity to

raise the profile of PMO both

internally and externally.

• This is the cumulative number of

followers on the PMO team’s social

media sites.

• No annual target is set for this KPI.

Internal
business
processes

12 The number of new PMO

processes captured,

documented, and disseminated.

• This is the number of new standard

operating procedures (SOPs) issued

by the PMO team during the month.

• The number of SOPs is tracked

against an annual target.

13 Level of ICT system used to

improve efficiency and

effectiveness across PMO

projects.

• This is a qualitative assessment in

regard to the adoption and

deployment of new ICT systems to

support PMO operations during the

month.
• No annual target is set for this KPI.

Learning

and growth

14 Level of training carried out by

team members to grow the

skills-base and intellect of the

PMO team.

• This is the number of days of training

undertaken by PMO team members

during the month.

• The number of training days is

tracked against an annual target.

15 The number of external

presentations delivered by PMO

team members at project

meetings, conferences, and

external events.

• This is the number of external

presentations delivered by PMO team

members at project meetings,

conferences, and external events

during the month.

• The number of external presentations

is tracked against an annual target.

Table 6: Summary of balanced scorecard and KPIs developed to support the university PMO.

Operational perspective – Deployment of the scorecard

In terms of the operational use of the scorecard, the data for the KPIs were gathered on a

monthly basis to summarise the performance of the team for the previous month. Data was

assembled every month through coordinating the contributions from project managers in the

PMO team and also through interrogation of a number of ICT (information and

communications technology) systems, including the university’s ERP system (enterprise

resource planning), CRM (customer relationship management) system, project management

system as well as local records held in spreadsheets. Consequently, there was a certain

administrative requirement associated with operating the scorecard. Initially, when the

JAN-APR 2020 JOURNALMODERNPM.COM

17 Measuring PMO Performance …

scorecard was adopted, an investigation was carried out over whether the scorecard KPIs could

be integrated into one of the existing corporate ICT systems with an automatic generation of

the required KPI data every month. While this would have understandably reduced the

administrative effort required, it was deemed to be excessively costly to undertake the required

systems integration work. Therefore, it was decided to operate the scorecard through a

spreadsheet template that was populated with data every month. This also had the benefit of

being readily accessible to all of the team members and still allowed all the KPIs and monthly

updates to be available on a single screenshot view.

Each month and once the scorecard had been assembled, it was forwarded to the divisional

director in order to ensure effective oversight and governance of the PMO team. The scorecard

and the performance of the team according to the specific KPIs were also displayed and

discussed at the monthly PMO team meeting. At the end of each academic year, the

performance of the team against the annual KPI targets was also identified, and this information

was shared across the team and with senior management. Additionally, the scorecard KPIs

were reviewed on an annual basis to ensure they were still aligned to the team strategy and

objectives; where required, adjustments were made both to the KPIs themselves and also to the

annual numerical targets.

Lessons learned from the case

The following lessons have been learned through considering the findings from the case study

investigation into how the balanced scorecard was deployed at a university-based PMO.

• The balanced nature of the scorecard provides an ideal mechanism to adopt a holistic

view of the work and functioning of a PMO and the projects that are delivered, and this

includes the project, process, and people considerations.

• The essential need for reporting financial metrics is highly consistent with the needs of

a PMO to gather and report on the performance of projects, but this is augmented

through also capturing KPIs from the other perspectives, i.e., customer, internal

business processes, and learning and growth.

• The utility and benefits gained from using the scorecard to guide operations and track

team performance need to be weighed against the administrative requirements for

servicing the scorecard and gathering the required data. In the case study, monthly

updates of the scorecard provided the optimal balance, but in other instances, the

frequency could be different, such as weekly or quarterly (i.e., every three months).

• In the ideal situation, it is suggested that the scorecard should integrate with the main

corporate IT systems that hold the data for the KPIs, such as the ERP, CRM, and project

management systems. This integration was not possible in the case, and hence the data

capture exercise did carry an administrative requirement. Conversely, where a ‘low

tech’ option, such as gathering scorecard KPI data in a document table or spreadsheet,

is required for a straightforward application, such an option can be readily adopted

without the need for extensive systems integration activities. In such a scenario, more

complex roll-outs of the scorecard can still be implemented in a phased approach as

JAN-APR 2020 JOURNALMODERNPM.COM

18 Measuring PMO Performance …

and when the organization has the resources available for a more complex system, or

when the circumstances of the team or organization have sufficiently changed.

• The team leader (or business unit leader) needs to be accountable for the performance

of the scorecard KPIs, although responsibility for the performance of specific KPIs or

groups of KPIs can be allocated (or delegated) to specific members of the team.

• Regular review of the scorecard and the performance of the KPIs with team members

helps to focus the attention of the team and socialize any issues associated with

underperformance in a particular area. This is important as it encourages discussion on

corrective action within the team, and any actions required to address such

underperformance can be shared across all the relevant team members. Team leaders

need to guide this process, but team members also need to properly engage with the

process and any follow-on activities. A further reflection is that the scorecard served as

a motivation tool and helped ignite an element of healthy competition between the team

members.

• Capturing data on learning and growth, such as training undertaken as well as

presentations given by team members, helps, in particular, to further motivate team

members. This is because an individual can clearly observe how their achievements and

efforts are being recognized; conversely, it can also spur other team members on to

undertake training or volunteer for an initiative or presentation that extends beyond

their current role.

• Adopting the balanced scorecard in a sustainable manner requires the commitment of

the team, and implicitly the team leader will need to drive forward both the development

and delivery of the scorecard – a continuing focus is then required if the scorecard is to

remain a part of the team’s core management processes. Where possible, the use of the

scorecard should be integrated with wider governance processes, such as those at the

divisional and corporate levels.

Conclusions

This article has reported on the findings from a case study investigation into how the balanced

scorecard was designed and deployed at a university PMO over a 4-year period. The application

of the scorecard was to enable performance measurement for a substantial portfolio of

collaborative research projects managed by a PMO team and working in conjunction with

academic teams across the university. This research study provides a supporting literature

review on the balanced scorecard along with accompanying background material on

collaborative research projects, which illustrates the context for the study and helps underpin

the need for the PMO in this project management application.

The balanced scorecard has been adopted widely over the last 20-25 years, and this has

extended across different types of organizations and applications. The scorecard continues to

be a powerful tool to help management track the performance of a team or business unit through

reporting on the progress of the team to senior management and controlling the allocation of

resources to ensure that progress is maintained. While there have been some authors that have

JAN-APR 2020 JOURNALMODERNPM.COM

19 Measuring PMO Performance …

raised certain issues to be addressed in regard to the scorecard, the continued use of the

scorecard as strategic development and operational management tool remains obvious. It is

important though to properly adapt the scorecard to the specific organizational circumstances

– this may result, for instance, is there is a higher concentration of KPIs in certain perspectives

of the scorecard, but this is an essential feature of aligning the scorecard perspectives to the

strategy for the business unit.

The design and operational use of the scorecard are very much associated with the collection

of data and information, and in the case of a PMO, there is a need for a significant amount of

data associated with project management. But in regard to the successful adoption of the

scorecard, while such data collection is, of course, necessary, it is the people or social

dimension that has a pivotal role to play in regard to the sustainable use of the scorecard over

the medium-to-long term. As a further parallel, the use of the tool to monitor the performance

of the team and subsequently control the allocation of resources across the team is a pre-

requisite use for the scorecard, but again from a ‘soft viewpoint,’ the scorecard can be equally

valuable in regard to helping to motivate team members to maintain and even increase

performance levels. The scorecard, therefore, provides benefits that extend beyond the initial

performance measurement aspects into more fundamentally important areas relevant to the

sustainability of the team or business unit.

In regard to future work, it is suggested that further longitudinal studies are undertaken on how

the balanced scorecard has been adopted in different organizational settings over extended

periods of time. This will provide greater insights into how the scorecard can be adapted for

different applications and inform future areas of research.

References

Al-Ashaab, A., Flores, M., Doultsinou, A., & Magyar, A. (2011). A balanced scorecard for measuring

the impact of industry–university collaboration. Production Planning and Control, 22(5-6), 554-570.

Atkinson, H. (2006). Strategy implementation: a role for the balanced scorecard?. Management

Decision, 44(10), 1441-1460.

Atkinson, R. (1999). Project management: cost, time and quality, two best guesses and a phenomenon,

its time to accept other success criteria. International Journal of Project Management, 17(6), 337-342.

Beard, D. F. (2009). Successful applications of the balanced scorecard in higher education. Journal of

Education for Business, 84(5), 275-282.

Biggart, T. B., Burney, L. L., Flanagan, R., & Harden, J. W. (2010). Is a balanced scorecard useful

in a competitive retail environment?. Management Accounting Quarterly, 12(1), 1.

Cebeci, U. (2009). Fuzzy AHP-based decision support system for selecting ERP systems in textile

industry by using balanced scorecard. Expert Systems with Applications, 36(5), 8900-8909.

Chang, L. C. (2007). The NHS performance assessment framework as a balanced scorecard

approach. International Journal of Public Sector Management, 20(2), 101-117.

JAN-APR 2020 JOURNALMODERNPM.COM

20 Measuring PMO Performance …

Chen, S. H., Yang, C. C., & Shiau, J. Y. (2006). The application of balanced scorecard in the

performance evaluation of higher education. The TQM Magazine, 18(2), 190-205.

Cummings, J. N., & Kiesler, S. (2005). Collaborative research across disciplinary and organizational

boundaries. Social Studies of Science, 35(5), 703-722.

Dai, C. X., & Wells, W. G. (2004). An exploration of project management office features and their

relationship to project performance. International Journal of Project Management, 22(7), 523-532.

Desouza, K. C., & Evaristo, J. R. (2006). Project management offices: A case of knowledge-based

archetypes. International Journal of Information Management, 26(5), 414-423.

do Valle, J. A. S., e Silvia, W. D. S., & Soares, C. A. P. (2008). Project management office (PMO)-

Principles in practice. AACE International Transactions, PM71.

Esquierro, J. C., Valle, A. B., Soares, C. A. P., & Vivas, D. C. (2014). Implementation of a project

management office in a public sector organization: A case study involving a sanitation

institution. International Review of Management and marketing, 4(1), 1-12.

European Commission (2019), Presentation on Horizon 2020 – The New EU Framework Programme

Research and Innovation, 2014 to 2020, website accessed on 5th January 2020:

https://ec.europa.eu/programmes/horizon2020/sites/horizon2020/files/281113_Horizon%202020%20s

tandard%20presentation .

Fernandes, K. J., Raja, V., & Whalley, A. (2006). Lessons from implementing the balanced scorecard

in a small and medium size manufacturing organization. Technovation, 26(5-6), 623-634.

Johnson-Laird, P. N., Goodwin, G. P., & Khemlani, S. S. (2017). Mental models and reasoning.

In International handbook of thinking and reasoning (pp. 346-365). Routledge.

Kaplan, R. S., & Norton, D. P. (1996). Linking the balanced scorecard to strategy. California

Management Review, 39(1), 53-79.

Kaplan, R. S., & Norton, D. P. (1998). Putting the balanced scorecard to work. The Economic Impact

of Knowledge, 27(4), 315-324.

Kaplan, R. S., & Norton, D. P. (2001). Transforming the balanced scorecard from performance

measurement to strategic management: Part I. Accounting horizons, 15(1), 87-104.

Kaplan, R. S. (2009). Conceptual foundations of the balanced scorecard. Handbooks of management

accounting research, 3, 1253-1269.

Karathanos, D., & Karathanos, P. (2005). Applying the balanced scorecard to education. Journal of

Education for Business, 80(4), 222-230.

Kutsch, E., Ward, J., Hall, M., & Algar, J. (2015). The contribution of the project management office:

A balanced scorecard perspective. Information Systems Management, 32(2), 105-118.

Lawrence, S., & Sharma, U. (2002). Commodification of education and academic labour—using the

balanced scorecard in a university setting. Critical Perspectives on Accounting, 13(5-6), 661-677.

Lee, S. F., Lo, K. K., Leung, R. F., & Ko, A. S. O. (2000). Strategy formulation framework for

vocational education: integrating SWOT analysis, balanced scorecard, QFD methodology and MBNQA

education criteria. Managerial Auditing Journal, 15(8), 407-423.

JAN-APR 2020 JOURNALMODERNPM.COM

21 Measuring PMO Performance …

Mankins, J. C. (2009). Technology readiness assessments: A retrospective. Acta Astronautica, 65(9),

1216-1223.

Mooraj, S., Oyon, D., & Hostettler, D. (1999). The balanced scorecard: a necessary good or an

unnecessary evil?. European Management Journal, 17(5), 481-491.

Nelson, R. R. (2007). IT project management: Infamous failures, classic mistakes, and best

practices. MIS Quarterly Executive, 6(2), 67-78.

Niven, P. R. (2011). Balanced scorecard: Step-by-step for government and nonprofit agencies, John

Wiley & Sons.

Office of Government Commerce (2002). Managing Successful Projects with PRINCE2, The United

Kingdom Stationery Office.

Perkins, M., Grey, A., & Remmers, H. (2014). What do we really mean by “Balanced

Scorecard”?. International Journal of Productivity and Performance Management, 63(2), 148-169.

Philbin, S. P. (2011). Design and implementation of the Balanced Scorecard at a university

institute. Measuring Business Excellence, 15(3), 34-45.

Project Management Institute (2013). A Guide to the Project Management Body of Knowledge

(PMBOK®), Fifth Edition, PMI.

Schön, D.A. (1987). Educating the reflective practitioner, San Francisco: Jossey-Bass.

Serdar Asan, Ş., & Tanyaş, M. (2007). Integrating Hoshin Kanri and the balanced scorecard for

strategic management: The case of higher education. Total Quality Management, 18(9), 999-1014.

Turner, J. R., & Müller, R. (2003). On the nature of the project as a temporary

organization. International Journal of Project Management, 21(1), 1-8.

Umashankar, V., & Dutta, K. (2007). Balanced scorecards in managing higher education institutions:

an Indian perspective. International Journal of Educational Management, 21(1), 54-67.

UNESCO (2015). UNESCO Science Report: Towards 2030 by United Nations UNESCO (United

Nations Educational, Scientific and Cultural Organization). Paris: UNESCO Publishing, 820 pp.

Ward, J., & Daniel, E. M. (2013). The role of project management offices (PMOs) in IS project

success and management satisfaction. Journal of Enterprise Information Management, 26(3), 316-336.

Wedekind, G. K., & Philbin, S. P. (2018). Research and Grant Management: The Role of the Project

Management Office (PMO) in a European Research Consortium Context. Journal of Research

Administration, 49(1), 43-62.

Wu, H. Y. (2012). Constructing a strategy map for banking institutions with key performance indicators

of the balanced scorecard. Evaluation and Program Planning, 35(3), 303-320.

Wuchty, S., Jones, B. F., & Uzzi, B. (2007). The increasing dominance of teams in production of

knowledge. Science, 316(5827), 1036-1039.

Zelman, W. N., Pink, G. H., & Matthias, C. B. (2003). Use of the balanced scorecard in health

care. Journal of Health Care Finance, 29(4), 1-16.

JAN-APR 2020 JOURNALMODERNPM.COM

22 Measuring PMO Performance …

About Authors

Simon P Philbin is Professor and Director of the Nathu Puri Institute for
Engineering and Enterprise at London South Bank University (LSBU) in the
United Kingdom. He joined LSBU in 2018 and previously worked at Imperial
College London. Prior to joining Imperial in 2003, he was at the UK Ministry of
Defence. He is published across several areas, including project management,
research & technology management, and chemistry. Previous academic roles
include Visiting Fellow at Imperial College Business School and Visiting Fellow at
Birkbeck, University of London. He holds a BSc and Ph.D. in chemistry as well as
an MBA.

Rajneet Kaur is an Associate Director at Imperial College London in the United
Kingdom, looking after the delivery of academic-driven commercial projects.
Before joining Imperial College London in 2014, Rajneet worked at one of the
Imperial’s subsidiary companies, Imperial Consultants, in the capacity of HR
and Office Manager from 2010. Prior to joining Imperial Consultants, Rajneet
has worked in other private and public sector organizations in various HR
roles. Rajneet holds a B Tech in Electronics and Communication Engineering
from India, an MSc in Engineering Management from Brunel University, and an
MBA from Imperial College London.

Copyright of Journal of Modern Project Management is the property of Editora Mundo and its
content may not be copied or emailed to multiple sites or posted to a listserv without the
copyright holder’s express written permission. However, users may print, download, or email
articles for individual use.

APRIL 2020 39INTERNAL AUDITOR

oard and management stakeholders want internal audit to
demonstrate greater business acumen. They want auditors
to have a broad understanding of the organization, as well as
anticipate how to help the organization achieve its objectives.

That means auditors need to see beyond their area of
expertise and responsibility. They must be agile to act proac-
tively and congruently with the organization’s way of doing
business. By recognizing these expectations, internal audit
can show that the department is an excellent place to develop
business acumen.

FIT BUSINESS NEEDS
Organizations expect all senior managers to have the busi-
ness acumen to lead their areas of responsibility and support
broader organizational success. Managers should be able to
anticipate and act on ways to add value to the organization
and its stakeholders.

Likewise, internal audit needs to identify the best ways
for the function to develop business acumen that fits the
organization’s needs. It can’t take a one-size-fits-all approach,

B

Internal audit can
incorporate elements of
the Balanced Scorecard
approach to build its ability
to anticipate and meet the
organization’s needs.

Audit With

Basil Orsini

Acumen

BUSINESS ACUMEN

-S
TR

IZ
H

–
/

SH
U

TT
ER

ST
O

CK
.C

O
M

AUDIT WITH ACUMEN

APRIL 2020

though, because business acumen will
vary by industry, type of business, and
the kind of service a business unit pro-
vides. For example, internal audit will
require different aspects of business
acumen than business lines, such as
sales and production, or support ser-
vices such as fi nance and security.

Moreover, internal audit’s assur-
ance role in relation to other assurance
roles within the organization impacts
the kind of business acumen it needs.
Developing business acumen can
enhance internal audit’s risk-based

coverage of the organization’s main
lines of business, as well as the fi rst two
lines of defense.

In developing business acumen,
internal audit should not be seen as
narrowly focused rule-followers who
avoid innovation and taking risks.
Chief audit executives (CAEs) should
ensure the audit staff understands the
capabilities of the organization’s fi rst
two lines of assurance, as well as the
business’ main products and services.

Their strategy for establishing business
acumen should involve human resource
activities, such as hiring, promotions,
and career planning, as well as profes-
sional development activities.

ENABLED BY THE STANDARDS
Internal audit’s use of business acumen
must reinforce, and not compromise,
auditors’ professional competence. The
International Standards for the Profes-
sional Practice of Internal Auditing place
great importance on risk-based plan-
ning — multiyear, annual, and engage-
ment — to ensure that services are
strategic and add value. Having busi-
ness acumen enables internal audit to
proactively plan and adapt all forms of
audit activity to anticipate the organiza-
tion’s assurance needs. This capability
goes far beyond simply repeating cycli-
cal coverage or responding to senior
management requests.

There is no trade-off between
demonstrating business acumen and
conforming to the Standards. On the
contrary, internal audit can build busi-
ness acumen on a sound understanding
and innovative implementation of the
Standards and associated guidance.

CAEs have used a variety of meth-
ods and approaches to attune their staff
to the business needs of their organiza-
tions. The examples in the boxes that
begin on page 41 demonstrate how
business acumen can work in internal

Internal audit can build business
acumen on a strong understanding and
implementation of the Standards.

40 INTERNAL AUDITOR

TO COMMENT on this article,
EMAIL the author at basil.orsini@theiia.org

GOVERNANCE

These examples can improve mutual understanding, enhance business capabilities, and strengthen relationships at the governance level of
the organization:
» Have the CAE actively participate in regular meetings of the audit com-

mittee operational governing body.
» Assign individual audit managers to each of the major lines of business

as account managers.
» Build the internal audit universe on top of the organization’s strate-

gic objectives.
» Conduct organizationwide internal audits in support of key corporate

activities such as internal communications.

audit. These examples are based on
four perspectives adapted from the Bal-
anced Scorecard strategic planning and
management tool: governance, client,
internal processes, and innovation and
learning. The boxes substitute gov-
ernance for the Balanced Scorecard’s
fi nance measure. CAEs should plan,
track, and report to the board and
management on initiatives in each of
these areas.

INTERNAL AUDIT’S ACUMEN
CAEs are likely undertaking some or
many of these initiatives, as well as some
others. To get the attention and mutual
understanding needed, annual internal

CLIENT

These examples can improve mutual understanding, improve business capabilities, and strengthen relationships with the organization’s busi-
ness units:
» Base multiyear, annual audit, and engagement plans on the organization’s

corporate and business risk profi les.
» Include strategic upside risks of opportunities and strengths in annual

internal audit plans to complement the traditional focus on key downside
risks of weaknesses and threats.

» Reinforce the role of other internal assurance functions (second line of
defense), such as risk management and fi nancial control, by auditing
their processes.

» Invite business units to link the timing of audit engagements to their busi-
ness information needs, such as in support of future fi nancial approval
submissions for major initiatives or new programs.

» Provide information on assessment criteria well in advance of an audit
engagement when there are known shortcomings, to enable managers to
take corrective action before the audit.

Business acumen is one of the top three skills CAEs focus on when recruiting, but 39%
say it is very diffi cult to recruit effectively, according to the 2018 North American Pulse of Internal Audit.

41INTERNAL AUDITORAPRIL 2020

APRIL 2020

INTERNAL PROCESSES

These internal audit processes can improve mutual understanding and business capabilities, as well as strengthen client relationships throughout the organization:
» Report more deeply on audit fi ndings by avoiding a narrow-minded approach to audit issues.

For example, reports should discuss the broader implications and possibilities of fi ndings,
such as their impact on broader business objectives. Internal audit also should show how
fi ndings link to implications for other business purposes and recommend reducing ineffi cient
internal controls.

» Submit periodic status reports on the internal audit plan’s implementation and adjust them dur-
ing the year to better address emerging business assurance needs.

» Issue periodic reports on signifi cant operational risks based on analyses of internal audit fi nd-
ings within the organization or across the industry.

» Offer to provide consulting and research services in conjunction with individual engagements.
» Invite internal audit team members to meet the audit committee and observe its discussion of

their individual engagements.

audit plans and year-end reports should
include a formal strategy on investments
in building staff capabilities to better
respond to the emerging needs of the
organization. This approach can foster
productive discussions and improved
understandings with management and
the audit committee.

BASIL ORSINI, CIA, CGAP, CRMA, CFE,

is a recently retired internal auditor from

the Government of Canada in Ottawa.

43INTERNAL AUDITOR

INNOVATION AND LEARNING

These examples of innovation and learning can improve mutual under-standing, enhance business capabilities, and strengthen relationships:
» Assign talented employees from other business units to short-term

engagements within internal audit. This practice can develop those
employees, as well as bring their insight to audit staff members.

» Send talented internal auditors on developmental, nonaudit assignments
within business units. This practice can help those auditors build business
acumen and pass their knowledge to the teams with whom they work.

» Bring internal auditors from fi eld offi ces to work at headquarters.
» Train new managers on internal audit’s role and areas of expertise such as

management control and risk management.
» Participate in professional associations other than internal audit, such as

risk management, IT, security, and fraud prevention. Such groups can help
auditors keep abreast of leading practices and share lessons learned with
audit colleagues.

Knowledge of the organization and its risks and industry-specifi c knowledge are among
the business acumen attributes spelled out in The IIA’s Global Internal Audit Competency Framework.

Copyright of Internal Auditor is the property of Internal Auditor and its content may not be
copied or emailed to multiple sites or posted to a listserv without the copyright holder’s
express written permission. However, users may print, download, or email articles for
individual use.