by Rod Dilnutt
University of Melbourne
[email protected]
1. Introduction
Higher education purports to provide graduates with the knowledge and experience as pathways to professional careers, while in industry, there has been a recent trend towards professional certification of skills and competencies offered by various vendor and standards bodies to uplift employee capability. However, criticisms of graduate job-readiness are common with employers often reporting the need to invest on in-job training to raise the level of graduate competence to productive levels.
This discussion references recent experience of The University of Melbourne (UniMelb) in gaining accreditation for its Enterprise Architecture (EA) subject for graduating students to achieve TOGAF® certification as an outcome of their Master of information Systems studies. Meeting the learning objectives applicable in the higher education and industry certification curricula required balancing often competing pedagogical paradigms. This case scenario narrates this dilemma, discusses the approach taken, and overviews the solution deployed with the ultimate aim of equipping graduates with the skills and knowledge required to ‘hit the ground running’ when entering the workforce.
2. Background
Higher education is generally accepted as formal learning beyond secondary school level and typically offers coursework channeling students into particular disciplines, i.e. Arts, Science, Medicine, Engineering and Information and Communications Technologies (ICT) at universities, with trade and vocational schools providing higher learning opportunities for traditional blue-collar occupations.
The Australian Resources 2030 Taskforce, (2018) recommends that industry should ‘re-skill, upskill and better support current and prospective resources workers, promoting a culture of continuous learning’ and this sentiment is reflected internationally. In the ICT industry, various certifications in practice areas have become increasingly popular and are common features of professional development. Consequently, there has been a recent trend towards Higher Education providers offering industry focused Micro Certifications, Credentials and MOOC’s (Massive Open Online Courses) that supplement and compete with numerous vendor/product specific certifications in technology products, and independent certifications of vendor neutral capabilities including PRINCE2, COBIT, ITIL, Cyber-security and TOGAF®.
The Open Group TOGAF® Standard, is described as “a proven EA development methodology and framework used by the world’s leading organizations to improve business efficiency”. The Open Group Architecture Forum evolves the standards to keep pace with the currency of industry practice and claims TOGAF® certification has been achieved by over 103,000 individuals worldwide. (Open Group 2018).
Given feedback from the employer community regarding graduate readiness and the aim to provide job-ready graduates, UniMelb explored the option to provide students with both formal qualification and industry certification as an outcome of their studies in EA within the Master of Information Systems degree programme (MIS). The value proposition for aligning academic and industry educational outcomes is presented at Figure 1. It shows the dichotomy that needs balancing between formal qualifications that are theory based and assumed not to be practical, with the need for on the job skilling on entering the workforce and subsequent certifications that are based in applied experience providing hands on job ready skills.
Figure 1 – Value Proposition Formal qualification and industry Certification alignment
3. Course Pedagogy
Enterprise Architecture is a core subject with the MIS degree programme and its focus provides an opportunity to align the subject content with the TOGAF® body of knowledge to award both industry certification and formal qualification. Accreditation provides an authoritative and independent assurance of the quality and relevance of TOGAF® training courses.
TOGAF® certification is typically offered as two two-day courses for level 1 and level 2 certification. This involves 30 study hours with assessment examination of 60 minutes and 90 minutes for levels 1 and 2 respectively. UniMelb’s EA subject is taught over a 12-week semester with weekly three hour lectures and tutorials, 36 hours in total. Assessment is by way of individual participation, Group Assignment and final two-hour examination. The classroom learning time is similar to TOGAF certification training however, assessment of the EA curriculum judged that 75 – 80% of content conformed to TOGAF® standards. To address this gap, a one-day bridging course was developed and accredited by Open Group.
The starting point for the pedagogical design that would render students eligible to sit the TOGAF® level 1 and 2 examinations is the alignment of learning outcomes of the respective courses. To calibrate this alignment, we adopt the de-facto standard for comparing learning outcomes from Bloom’s Taxonomy (1956, Revised 2001) which provides a hierarchy of goals of the learning process as follows’
- Level 1 Remember – Recognizing, Recalling
- Level 2 Understand – Interpreting, Classifying, Summarizing, Inferring, Comparing, Explaining
- Level 3 Apply – Executing, Implementing
- Level 4 Analyze – Differentiating, Organizing, Attributing
- Level 5 Evaluate – Checking, Critiquing,
- Level 6 Create – Generating, Planning, Producing
The EA subject has been designed and accredited to Australian Quality Framework standards at AQF9 (Masters) level and accredited to the ACS Core Body of Knowledge (2015) which complements the role of Australia’s Tertiary Education Quality and Standards Agency (TEQSA), in turn aligned with international standards i.e. IFIP IP3 and Seoul Accord.(2008) affording mutual international recognition for undergraduate and postgraduate (master’s level).
TOGAF® training content is developed to conform to The Open Group Body of Knowledge Standards. Certification is available at two levels: Level 1: Knowledge of the fundamentals of TOGAF 9 sufficient to be able to contribute to an architecture effort or to work with the results, and Level 2: Knowledge, comprehension, and ability to analyze and apply TOGAF 9 concepts. These are proximate to Bloom’s taxonomy standards ‘Level 2 – Understand’ and Level 4 – Analyse’ respectively (Figure 2). However, masters tertiary standards at level 5 are applicable which require critical evaluation.
Figure 2 – Blooms’ Learning Outcome Requirement
4. The Alignment Challenge
At a practical level, the development of course content to comply with learning outcomes set to different standards and levels requires balancing of the learning content. This is not a straightforward task and figure 3 presents the respective courses and identifies the learning outcome gaps relative to Bloom’s Taxonomy.
Figure 3 – TOGAF And EA Course Mapping to Bloom’s Taxonomy
Learning Outcome Gap A is the gap between TOGAF Level 1 and Level 2 courses. This gap does not present issues as standard practice for TOGAF Certification courseware assumes each course is delivered to the respective learning outcomes i.e. separate courses. However, for the EA course to achieve TOGAF compliance and AQF9 Masters standards Learning Outcomes Gap B must be addressed i.e. the course must provide for higher order learning outcomes required to meet ‘Evaluate’ standards but must also meet the lower level needs to Remember, Understand, Apply, and Analyse. Presentation of this content requires different pedagogical teaching and assessment methods. Herein lies the challenge when attempting to satisfy both needs in one place.
Levels 1 and 2 require a student to learn and understand concepts and assessment will measure successful recall as a proxy for understanding. Level 3 and 4 require a student to develop an understanding of a concepts and assessment provides a scenario requiring a correct response and application in the context of TOGAF concepts. However, to achieve Level 5 Evaluate as required in the EA course, students are required to reflect critically and demonstrate the technical and creative skills to synthesize complex problems. An assessment question would be open ended requiring interpretation and justification of a response with critical analysis drawing on a wide body of knowledge. Questions would typically include verbs like, critically assess or evaluate.
As an example, learning content requiring evaluative discussion could be prompted by general open statements assuming understanding of the topic and encouraging open and informed response (Figure 4a). Conversely, learning content aimed at foundational understanding needs to be prescriptive requiring the learner to recall structured facts (Figure 4b).
 |  |
| Figure 4b Level 1 -4 learning content example | Figure 4b Level 5 learning content example |
Integration of the learning outcomes produces somewhat uncomfortable fit between learning content presentation styles which are predominantly slide based. To cater to level 2 and 3 requires incorporation of detailed and often lengthy text explanations whereas higher level content presentation benefits from broader learning prompts that point to the learner in a general direction where responses are likely to be context dependent and subjectively diverse. The Figure 4a presentation slides can appear an uninspiring ‘sea of text’, whereas Figure 4b can be adjudged to be lacking in detail. Both observations are valid depending on the perspective of the learning objective. Interestingly, iterative submissions of learning material for accreditation assessment were frequently, returned requiring ‘more detail’ and/or precise reflection of the standard which tended to add more text. This results in a much larger volume of detailed slides than would be the case for master’s level. To overcome this dilemma a one-day Bridging Course was developed for specific focus on the TOGAF standards. Unfortunately, this also leads to a level of duplication between the respective courses as the context needs to be set for the detailed information presented.
5. Conclusion
In time, the effectiveness of this combined pedagogical approach can be validated as graduates enter the workforce. In the meantime, our assumption is that alignment of industry certifications within formal education programmes provide significant potential value to graduates and employers alike. However, striking a balance between the differing learning outcome foci can be difficult to achieve and requires a trade-off in the presentation style of learning content. The successful achievement of both industry and formal education qualifications present a potential resolution to reconcile employer demands for ‘job-ready’ graduates with the theoretical learning outcomes expected of higher education.
As graduates with both industry certifications and formal qualifications join the workforce, employer feedback will inform the effectiveness of this approach.
About the Author
Rod has over 25 years’ experience as a management consultant. He is the Director of William Bethwey & Associates and is an internationally recognised thought leader in Strategy & Architecture and Managing Change. He is a sought-after expert in ‘Disruptive Technologies and Change’.
He is an Industry Fellow at The University of Melbourne where he lectures in Higher Degree programmes including Organisational Change, Enterprise Architecture, and IS Strategy. Rod has authored over 30 publications and a book on Organisational Change.
Rod is a Fellow of the ACS, ACS Management Committee member, and Chair-ACS Victoria . He is also a Fellow of the British Computer Society and a BCS Certified Assessor.
