Organizational Change and Development:

ABSTRACT
This chapter first discusses the complexities of change in organizations and why so many OCD programs
fail and makes the case for change agents to become evidence-based in their change agency practice.
The author then offers a definition of evidence-based organizational change and development (EBOCD)
and outlines the types of “best evidence” that can be used to inform and shape the formulation and
implementation of OCD strategies and to critically evaluate the associated processes and change agency
practices. Various distinctive evidence-based initiatives for OCD are discussed and several case examples
from the United Kingdom are presented. The chapter closes with a discussion of the specific merits of
“design science,” “professional partnership” research, and “replication” research.
INTRODUCTION
Since the 1980s, organizations in all organizational sectors have had to react to huge environmental pressures
for change, the main drivers of which have been: Technology, particularly IT; Governments, which in
all parts of the globe have until recent times increasingly embraced notions of deregulation, privatization
and free trade; and Globalization, where private sector companies have had to compete more aggressively,
and public sector organizations have had to deliver more value for money services and products
(see Barkema, Baum, & Mannix, 2002; Champy & Nohria, 1996; Dess & Picken, 2000; Smith, Lewis
& Tushman, 2016; Yukl, 2006). These pressures have resulted in mergers, acquisitions, amalgamations,
decentralization, flatter structures, downsizing, multidimensional restructuring, increased flexible work
practices, drives on quality and value, greater emphasis on customer/client/consumer orientation and care,
and increasing stress levels at work (see, for example, Hamlin, 2001a; Gunnigle, Lavelle & Monaghan,
2013; Shook & Roth, 2011). Furthermore, since the early 2000s most organizations have been operating
Organizational Change
and Development:
The Case for Evidence-Based Practice
Robert G. Hamlin
University of Wolverhampton, UK
2
Organizational Change and Development

increasingly in VUCA (volatile, uncertain, complex, ambiguous) environments where: Volatile refers to
the pace of change which is rapid and unrelenting, and the associated challenges which are unexpected
or unstable; Uncertainty refers to the difficulty in getting clarity and certainty about what is going on
in business contexts through incomplete or insufficient information, which is complicated by opposing
views and opinions; Complexity refers to the many interconnected parts and variables that can have a
‘cause-effect’ impact in a given situation, and to the volume or nature of the available information that
can be overwhelming to process thus making diagnosis difficult; and Ambiguity refers to causal relationships
which are completely unclear because no precedents exist and leaders/managers are faced with
‘unknown unknowns’ (see Bennett & James Lemoine, 2014; Mack, Khare, Kramer & Burgatz, 2016).
In response to these trends of change, most executives in the 21st century recognize that their respective
organizations need to adapt continuously to constantly changing environments. However, they tend to
struggle with the transformational changes that are necessary for ensuring the survival of their respective
organizations, or they fail to raise their game to the higher levels of performance that are required
(Rogers, Shannon, & Gent, 2003). Therefore, when they initiate programs of organizational change and
development (OCD) they tend to rely on lower level managers to facilitate and implement the change
processes. Thus, in a very real sense, most managers in most organizations are agents of change (Axley,
2000). Furthermore, in some organizations where the modern-day conceptualization of human resource
development (HRD) is well embedded into the fabric of managerial thinking and management practice,
managers proactively use the services of HRD colleagues possessing strong change agency capabilities
to help them formulate and implement OCD plans, strategies and interventions. In so doing they treat
them as ‘strategic partners’, just as they do with external OD specialists and management consultants
whose change agency services they use.
However, as I have discussed elsewhere, “one of the major challenges facing contemporary managers
and HRD professionals is how best to help people through the transitions of change, and to survive
or thrive in working environments that are in a constant state of flux.” (Hamlin, 2016a, p.121). Hence,
this chapter is concerned with: (a) the challenges that confront managers and the various ‘strategic partners’
they turn to for help in bringing about effective and beneficial organizational change, and (b) the
practical contribution that management research and specific OCD-related research can make toward
improving the efficacy of their respective change agency endeavors. The specific purpose can be summarized
as follows:

1. To outline the extent to which organizational change programs fail and why they fail.
2. To argue that managers and their strategic partners should adopt an ‘evidence-based practice’ (EBP)
   approach to organizational change agency.
3. To highlight certain obstacles to ‘evidence-based’ OCD, and to describe and illustrate how these
   may be overcome through ‘professional partnership’ research and empirical generalization ‘replication’
   research respectively.
4. To highlight and discuss other ‘evidence-based’ initiatives for facilitating effective and beneficial
   OCD programs.
   3
   Organizational Change and Development
   
   BACKGROUND
   Although it may be widely recognized that most managers in most organizations are to a greater or lesser
   extent agents of change, this cannot be assumed to be the case for most HRD practitioners. However, as
   Stewart (2015) claims, modern-day HRD professional practitioners are change-agents skilled in advising
   and helping managers with the facilitation of OCD programs, either in their capacity as a colleague or as
   an external consultant. Indeed, Stewart and others have argued that HRD is of itself a strategic function
   which, when fully utilized, can have a significant impact on the survival and long-term business success
   of organizations (see Fredericks & Stewart, 1996; Stewart & McGoldrick, 1996). This view is reflected in
   an ‘all-embracing’, ‘catch-all’, ‘composite’ but ‘non-definitive’ statement of HRD offered by Hamlin and
   Stewart (2011) which asserts that, in essence, “HRD encompasses processes, activities or interventions”
   which “enhance organizational and individual learning, develop human potential, improve or maximize
   effectiveness and performance at either the individual, group/team and/or organizational level, and/or
   bring about effective, beneficial personal or organizational behaviour change and improvement” (p.
   213) within, across, and/or beyond the boundaries of private, public and third sector organizations, entities,
   and other types of host system. This understanding of HRD is consistent with Phillips and Shaw’s
   (1989) ‘consultancy approach for trainers’ which involves HRD practitioners increasingly operating not
   only as ‘training consultants’ and ‘learning consultants’ but also as ‘organizational change consultants’.
   Thus, both in theory and practice, the contribution of appropriate HRD consultancy type practices can
   be a major influence on the interplay of culture, leadership, and commitment of employees through:
   a) shaping organizational culture; b) developing current and future leaders; c) building commitment
   among organization members; and d) anticipating and managing responses to changed conditions (Gold,
   Holden, Iles, Stewart, & Beardwell, 2009). This view accords with McKenzie, Garavan and Carbery’s
   (2012) observation that “the shift from operational and tactical HRD to strategic HRD has witnessed
   a metamorphosis for HRD practitioners increasingly becoming partners in the business tasked with
   aligning people, strategy, and performance rather than simply promoting learning and development”
   (p. 354). It also chimes with Kohut and Roth’s (2015) view that “HRD practitioners and scholars need
   [increasingly] to enter the fray of the discussion on change management” (p. 231).
   As happened during the last three decades of the 20th century, the need for managers to initiate and
   facilitate ‘organizational change’ (OC) and ‘organization development’ (OD) programs effectively and
   beneficially in the 21st century is increasing in frequency, pace, and complexity (see Hamlin, Keep
   & Ash, 2001). In this context, a major challenge facing modern day managers and HRD professional
   practitioners is how best to help staff cope effectively within working environments that are in a state of
   constant flux and with the transitions of major change programs. Unfortunately, many OCD programs
   fail because managers and their HRD colleagues, as well as many external professional change-agents
   (e.g. OD specialists, management consultants, executive coaches, etc.) whose services are used in support
   roles, find themselves unable to rise to the challenge. However, for those organizations that do facilitate
   OCD programs effectively and beneficially, change initiatives become welcomed as opportunities for
   increasing efficiency and for building new organizational success.
   4
   Organizational Change and Development
   
   MAIN FOCUS OF THE CHAPTER
   This section explores the reasons for the failure of so many OCD programs, and then discusses the case
   for EBP approaches to OCD change agency.
   Past and Current Failure Rates of OCD Programs
   It is of crucial importance to all those who have a stake in the success of an organization that OCD processes
   are managed effectively and beneficially. Sadly, the success rates of planned OC strategies have
   been poor with over two thirds or more of OC and OD programs having failed to achieve their intended
   aims (Beer & Nohria, 2000; Burnes, 2004; Choi & Ruona, 2011; Stanford, 2016; Szabla, 2007). As I
   have claimed elsewhere, based on a comprehensive review and synthesis of OCD context and practice
   that I conducted in 2001, “the majority of ‘downsizing’ and ‘delayering’ initiatives were unsuccessful,
   with few ever reaching the aimed for goals of increased competitiveness and profitability, and with many
   ending up with lower profit margins and poorer returns on assets and equity than achieved by equivalent
   firms that had not downsized” (Hamlin, 2016a, p.122). Furthermore, based on a review of more
   recent literature “downsizing (now more frequently referred to as ‘right sizing’) brings with it largely
   negative outcomes that adversely affect the entire workforce in a profound manner” which include, for
   example, “the destruction of morale and organizational commitment; decreased levels of productivity,
   efficiency, job performance, innovation, employee effort and quality of work, and increased levels of
   staff absenteeism, staff turnover, sabotage, fraud, embezzlement and theft (see Fong & Kleiner, 2004;
   Gandolfi, 2006; Nutt, 2004; Thornhill, Lewis, Millmore, & Saunders, 2000)” (In Hamlin, 2016a, p. 122).
   Similarly, during the 1980s and 1990s, over 50% of Business Process Re-engineering (BPR) and Total
   Quality Management (TQM) programs were reported as having failed, with a further 20% or more failing
   to produce the full benefits (Hamlin, 2001a). As will be discussed later, there is little contemporary
   evidence to indicate there has been any substantial increase in the success rate of OCD programs during
   the first two decades of the 21st century. However, although the historical success rate of IT-related
   change programs has been dismal with less than 30% of new project investments succeeding in meeting
   their performance goals (see Hamlin, 2001a; Sirkin, Keenan, & Jackson, 2005), in the USA the success
   rates have increased somewhat over time-from 27% in 1996 to 39% in 2012 (Standish Group, 2013).
   According to Pfeffer and Sutton (2006), most organizational mergers fail to deliver the expected
   benefits; instead, the ways they are facilitated tend toward a destruction of the combined economic value
   of the merged businesses. Walker and Price (2000) had earlier suggested that such failures are caused
   not so much by financial issues but rather by the incompatibility of the merging cultures, the clashing
   of management styles, and the lack of skill of those managers implementing the change. Furthermore,
   they argue that these problematic issues are not addressed early enough or effectively enough in the
   merger process.
   Recent literature also reports that overall 70% or more of rightsizing, mergers acquisitions, and other
   types of organizational change programs, either fail or are just partially successful, and that the workplace
   challenges posed by OCD initiatives typically have a negative impact on employees (Carnall & Todnem
   By, 2014; Shook & Roth, 2011; ten Have, ten Have, Huijsmans, & Otto, 2017). However, the volume
   of empirical evidence supporting such reports has been questioned by Hughes (2011) who claims that
   the 70% OC failure rate rhetoric has been informed more by magazine articles and practitioner books
   5
   Organizational Change and Development
   
   than by academic studies. Indeed, as Candido and Santos (2015) revealed from their comprehensive
   review of the literature, recorded failure rates of OC programs have spanned from as low as 7% to one
   as high as 90%, with some rates having been derived from outdated or fragmentary empirical evidence
   lacking in scientific rigor, or based solely on subjective opinion. Nevertheless, they argue that strategy
   implementation failure is a persistent problem which remains an important and ongoing concern for
   researchers and practitioners.
   However, as Stanford (2016) observes, although leaders, talent management professionals and others
   within organizations may realize that change is needed, they often fail to grasp the complexity of what
   change management entails. Furthermore, they tend not to recognize that their expectations are not
   aligned with what is feasible within the organization, particularly when the change leaders fail to take
   the time necessary to understand the scope of the change, the involved stakeholders, the environmental
   complexity, and the impact of the change from a systems perspective. Unfortunately, many OCD programs
   fail badly with unintended and damaging consequences which increasingly take a psychological toll on
   both the ‘victims’ and the ‘survivors’, not least the surviving managers (see Hamlin, 2001a; Hareli &
   Tzafrir, 2004; Worrall & Cooper, 1997-2001, 2007).
   From the foregoing it is unarguably the case that too many organizational change initiatives fail and/
   or lead to outcomes that are largely negative. As I have claimed elsewhere, the “weight of evidence
   suggests that the process issues associated with formulating and implementing organizational change
   initiatives are far more complex and difficult than is often supposed, and that both managers and HRD
   practitioners [and those other OCD-related strategic partners] are generally insufficiently skilled in the
   practice of organizational change agency” (Hamlin, 2016a, p.123).
   Reasons for OCD Program Failure
   Bearing in mind the plethora of books on ‘organizational change management’ authored by management
   consultants and high-profile academics who offer ‘how to’ advice and guidance, it is surprising that so
   many OCD programs fail. Some of the books can be criticized for being too pragmatic, prescriptive,
   or simplistic, whilst others for being too theoretical and philosophical. All can be of some help when
   change agents are thinking through the issues that need to be considered, and/or when they are developing
   and implementing an OCD intervention strategy. Many of these books describe ‘planned change
   management’ approaches based on Lewin’s ‘unfreeze-move-refreeze’ model and incorporate a processual
   ‘stage by stage’ procedure for managing the change. Drawing upon the various models offered in
   the 1990s, Hamlin (2001a) created a composite ‘generic model for managing planned organizational
   change’ comprised of six phases, adapted versions of which are as follows:
   Phase 1: Diagnose/explore the current organizational state and define what the future organizational
   state should be
   Phase 2: Create a strategic vision that gives focus to the intended direction of travel
   Phase 3: Plan the change management strategy
   Phase 4: Secure ownership, commitment and involvement from all stakeholders affected by the change,
   and ensure there is support from top management
   Phase 5: Project manage the implementation of the strategy and sustain the momentum
   Phase 6: Stabilize, integrate and consolidate to prevent regression and perpetuate the change
   6
   Organizational Change and Development
   
   The implicit assumption when managers and their strategic partners utilize any change management
   model is that they have the knowledge and expertise to apply it effectively. However, the rates of
   OCD program failure discussed in the previous section suggest they do not, and that most managers
   are perhaps deficient in the requisite change agency skills. Although most if not all offered models are
   basically sound with high face validity, in many cases their simplified diagrammatic/summary formats
   can appear overly simplistic and just plain common sense. Regrettably, this can lead to important phases
   being ‘skipped’ or to a lax application of the model; and this can be made worse by the manager’s and
   his or her strategic partner’s lack of change agency expertise. Based on a review of organizational change
   management studies from various researchers in the USA, the UK, and from various other European
   countries I have identified six root ‘failings’ of managers and HRD practitioners which help explain why
   OCD programs fail (Hamlin, 2001a). Five of these ‘failings’ relate to managers and the sixth to HRD
   practitioners, as follows:
   Failing 1: Managers not knowing the fundamental principles of change management.
   Managerial ‘complacency’ and ‘ignorance’ are two of the most significant factors which can contribute
   to the failure of OCD programs; yet this should not be the case when professionally qualified managers
   are acting in the role of a change agent. A significant reason for their failings can be attributed to insufficient
   attention having been given during their professional education and training to the ‘soft stuff’
   of management, to the ‘behavioral aspects’ of change management, to the training and development of
   their people, and to the interpersonal communication skills required for managing change effectively. As
   various writers have claimed, much of what is taught and delivered in university business schools is not
   perceived to be part of the manager’s real world, or of practical relevance (Bones, 2007; Mintzberg, 2004).
   Failing 2: Managers succumbing to the temptations of the ‘quick fix’ or ‘simple solution’.
   Going through all six phases of the change process normally requires a substantial amount of time
   if the change strategy is to be implemented successfully. However, driven by ‘short termism’, many
   managers are tempted to adopt ‘simple’ or ‘quick-fix’ solutions that skip through the phases. Such approaches
   rarely deliver the results required, but instead lead to failure. As Kotter (1996) argues, although
   the skipping of phases can create the illusion of speed it never produces satisfying results; and omissions
   or critical mistakes in any of the phases can devastatingly impede or even negate further progress.
   Failing 3: Managers not fully appreciating the significance of the leadership and cultural aspects of
   change.
   The three factors that many expert commentators regard as most important in a fully functioning
   organization are ‘the leadership’, ‘the culture’ and ‘the management of change’. In the absence of
   strong strategic/organizational leadership, and managers who lead effectively using the right style of
   supervisory leadership, OCD related initiatives and interventions designed to change and improve the
   organization will underachieve the desired outcomes and will most likely make things worse. To bring
   about significant organizational change requires the commitment of top management; but this needs to
   be clearly in evidence, consistent, and sustained throughout the change process. Additionally, managers
   need to recognize that if they pay insufficient attention to the cultural issues of change, the organization
   7
   Organizational Change and Development
   
   could risk suffering from ‘cultural lag’- a term coined by Bate (1996) to describe the condition when a
   culture becomes ‘out-of-fit’ with the needs of the changing organization and thus impedes the change
   process. For OCD programs to succeed, adequate attention must be given to the ‘cultural’ issues and to
   the style and quality of the ‘leadership’ manifested by managers.
   Failing 4: Managers not appreciating sufficiently the significance of the people issues.
   In many OCD programs insufficient attention to the associated people issues is given by those
   who develop the change strategy and those who implement it. These issues include the psychological
   processes that employees experience in dealing with change and in coping with the change transition.
   Employees need to be enabled: a) to let go of the past ways of working and their memories of the old
   culture (endings) whilst holding on to certain still relevant aspects; b) to engage with the required new
   ways of working (in-between time), and c) to fully embrace the new direction and organizational culture
   (new beginning). This means managers need to understand the distinction between ‘change’ and ‘transition’
   and recognize the critical importance of paying sufficient attention to the ‘soft’ human aspects in
   the change management process.
   Failing 5: Managers not knowing the critical contribution that the HRD function can make to the management
   of change.
   When managers strive to address the ‘soft’ human aspects of change management they tend not to
   use to best effect the services of HRD practitioners. This is despite the acknowledged critical importance
   of preparing employees adequately for change by giving sufficient time to training them (Cornell HR
   Review, 2013). The reason for the omission is because too many managers and HR professionals view
   HRD as one of many sub-sets of administrative activities that constitute the HR function, with the training
   specialism responsive only to immediate knowledge and skill deficits. Indeed, for many if not most
   organizational leaders, HRD does not register very much on their radar screens and is very much a third
   or fourth order consideration (Gold, Rogers, & Smith, 2003). Furthermore, this is exacerbated by the
   fact that although HR has long been claimed to be strategic in focus, evidence suggests it remains more
   rhetoric than reality (Hamlin, 2001a; Power, 2011). As Power observes, it is hard to find leaders of the
   HR function who are active in helping their organization improve the way it works, and this is because:
   a) top management often view HR as an expense with a transaction focus rather than an adding value
   contribution with a strategic focus; b) historically HR has been mainly engaged in personnel, compliance,
   and transactions, and c) HR professionals without operational experience have less credibility and
   are not comfortable giving operational advice.
   However, as discussed briefly above, and in considerable depth elsewhere (Hamlin, 2001a, 2002),
   it has long been the case that various HRD professional practitioners have practiced at a strategic level,
   have made pivotal contributions towards the achievement of organizational effectiveness and success,
   and have helped create the critical capabilities and competencies required for the organization to change
   and develop. Unfortunately, “many managers have ‘blind spots’ regarding HRD, seeing it only in terms
   of high-cost external training courses, or long-term development/qualification programmes for young
   and new employees” (Hamlin, 2001a, p. 27). Furthermore, managers remain largely ignorant of modern
   day conceptualizations of strategic HRD. Yet there are HRD implications implicit in all organizational
   change whether at the individual, group, or organization level. As Hamlin also argues:
   8
   Organizational Change and Development
   
   For every change, both large and small, either ‘new’ knowledge, attitudes, skills and habits (KASH)
   have to be acquired, as in the case, for example, when new products, services, technologies, structures
   or systems are introduced; or alternatively ‘existing’ knowledge, attitudes, skills and habits must be redistributed,
   as in the case of downsizing or when mergers or acquisitions take place. Unless the KASH
   gaps flowing inevitably from organizational change initiatives are bridged efficiently and effectively,
   whether at the organizational, group or individual level, the organization will not develop the critical
   capabilities [and competencies] required to make a successful transition from an [undesired] present
   state to the new [desired] future state. (Hamlin, 2001a, p. 27)
   The extent to which appropriate HRD effort is incorporated into OCD programs will critically determine
   whether a planned change program succeeds or fails. Hence, for managers to be in control of OCD
   initiatives they must be ‘learning focused’, which means being in control of the KASH issues associated
   with change itself. Thus, sufficient time and attention needs to be given to the ‘soft’ HRD aspects of
   managing the change process (Hamlin, 2001a, 2002; Ning & Jing, 2012; Thornhill et al., 2000). For
   managers to manage change effectively and beneficially they need consciously to incorporate HRD into
   the very ‘fabric’ of their everyday management practice, and to use it as a tool for managing change.
   However, if they have not already done so, they need to stop thinking of HRD merely as traditional
   ‘training and development’, and instead recognize the 21st century understanding of the whole HRD
   domain of study and practice, as reflected in the following definition:
   Contemporary HRD is: the study or practice concerned with the diagnosis of performance-related behaviour
   change requirements at the individual, group and organizational level within any host entity, and
   the design, delivery and evaluation of formal and/or informal learning activities to meet the identified
   needs. (Copyright © r.g.hamlin, 2017)