{"id":9448,"date":"2011-11-23T07:00:00","date_gmt":"2011-11-23T13:00:00","guid":{"rendered":"https:\/\/trainingmag.com\/last-word-remember-when\/"},"modified":"2020-12-17T14:50:12","modified_gmt":"2020-12-17T20:50:12","slug":"last-word-remember-when","status":"publish","type":"post","link":"https:\/\/trainingmag.com\/last-word-remember-when\/","title":{"rendered":"Last Word: Remember When"},"content":{"rendered":"<p>\n\tBy Don Kirkpatrick<\/p>\n<p>\n\t<em>Training<\/em>magazine graciously asked me to write the final Last Word column for 2011 in view of my<br \/>\n\t50-plus years as a training professional, my creation of the Four Levels of Evaluation, and my recent<br \/>\n\tretirement. Much has happened through these last 50 years.<\/p>\n<p>\n\tThe Four Levels of Evaluation were born when I went to work for the Management Institute of the University of Wisconsin, where I conducted workshops on soft skills including communication, leadership, coaching, managing change, and managing time. While I was doing this, I decided to work for my Ph.D., and I chose \u201cEvaluating a Human Relations Program for Supervisors\u201d as my dissertation. I received the degree in 1954 and continued to evaluate programs. In 1959, Bob Craig, editor of ASTD\u2019s journal, called and asked me to write an article on evaluation. I told him I would write a series of four articles\u2014one on each of the Four Levels: Reaction, Learning, Behavior, and Results\u2014and he agreed. Thirty years later, I published my first book called \u201cEvaluating Training Programs: The Four Levels.\u201d To date, I\u2019ve\u00a0 authored seven management books.<\/p>\n<p>\n\tIn industry, I served as Training director for International Minerals and Chemical Corp., where I developed a Performance Appraisal Program. Later, I served as Human Resources manager of Bendix Products Aerospace Division. I was elected president of the American Society for Training and Development in 1975 and received ASTD\u2019s Gordon Bliss and Lifetime Achievement in Workplace Learning and Performance awards.<\/p>\n<p>\n\tOn a personal note, my wife, Fern, has always been in the picture. While I was traveling around the world presenting programs, she was taking care of our four children. During our marriage, we moved eight times, and she often had to try to sell our homes while I was gone. She showed our house in Park Ridge, IL, more than 100 times before it finally sold.<\/p>\n<p>\n\tBut she always supported me in my work. The latest highlight was a family reunion at Wisconsin Dells, where we celebrated our 60<sup>th<\/sup> wedding anniversary. All four of our children and their five spouses, our three grandchildren, and our two great-grandchildren helped us celebrate.<\/p>\n<p>\n\tMy son, Jim, and his wife, Wendy, are carrying the training torch by conducting certification programs and workshops; communicating through www.kirkpatrickpartners.com; and writing books containing upgrades to the Four Levels and others, including \u201cTraining On Trial.\u201d<\/p>\n<p>\n\t<strong>Tips for Today\u2019s Trainers<\/strong><\/p>\n<ol>\n<li>\n\t\tOne of my transparencies is \u201c10 Requirements for an Effective Training Program.\u201d Be sure you have <em>planned<\/em>an effective program so the evaluation will prove you have <em>done<\/em>it.<\/li>\n<li>\n\t\tDon\u2019t underestimate the importance of<br \/>\n\t\tevaluating Reaction, a measure of customer satisfaction. Learners had better go back to their jobs saying good things about the program, because their words probably will filter up to higher management.<\/li>\n<li>\n\t\tRegarding the evaluation of Learning, there will be no change in behavior unless the learners have some specific ideas to take back to the job.<\/li>\n<li>\n\t\tOne of the possible reasons change in behavior does not occur is a job climate where the manager prevents any change. So be sure the learner\u2019s manager will provide the support and accountability to encourage the change.<\/li>\n<li>\n\t\tBefore you even develop a training program, find out what top management\u2014or the \u201cJury\u201d\u2014expects the program to accomplish in terms of Results. The program then is developed around these expectations instead of the curriculum design team creating the objectives. In the evaluation of the program, the actual results can be compared to what the Jury expected. We call this measuring Return on Expectations (ROE) instead of ROI.<\/li>\n<\/ol>\n<p>\n\t<em>Training legend, author, speaker, and consultant Don Kirkpatrick is the father of the Four Levels of Evaluation. <\/em><em>Now retired, he plans to pursue his hobbies: fishing, tennis, golf, big band and classical music, and directing church choirs. To contact him or request an autographed transparency, e-mail him at<\/em><strong><em><a href=\"mailto:dleekirk1@aol.com\">mailto:dleekirk1@aol.com<\/a><\/em><\/strong><em>.<\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>By Don Kirkpatrick<\/p>\n<p>\tTrainingmagazine graciously asked me to write the final Last Word column for 2011 in view of my<br \/>\n\t50-plus years as a training professional, my creation of the Four Levels of Evaluation, and my recent<br \/>\n\tretirement. Much has happened through these last 50 years.<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"cybocfi_hide_featured_image":"","footnotes":""},"categories":[12],"tags":[24],"class_list":{"0":"post-9448","1":"post","2":"type-post","3":"status-publish","4":"format-standard","6":"category-magazine","7":"tag-last-word","8":"magazine_issues-november-2011"},"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v25.2 (Yoast SEO v25.2) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Last Word: Remember When<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/trainingmag.com\/last-word-remember-when\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Last Word: Remember When\" \/>\n<meta property=\"og:description\" content=\"By Don Kirkpatrick  Trainingmagazine graciously asked me to write the final Last Word column for 2011 in view of my 50-plus years as a training professional, my creation of the Four Levels of Evaluation, and my recent retirement. Much has happened through these last 50 years.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/trainingmag.com\/last-word-remember-when\/\" \/>\n<meta property=\"og:site_name\" content=\"Training\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/TrainingMagazine\" \/>\n<meta property=\"article:published_time\" content=\"2011-11-23T13:00:00+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2020-12-17T20:50:12+00:00\" \/>\n<meta name=\"author\" content=\"Lorri Freifeld\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@TrainingMagUs\" \/>\n<meta name=\"twitter:site\" content=\"@TrainingMagUs\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Lorri Freifeld\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"3 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/trainingmag.com\/last-word-remember-when\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/trainingmag.com\/last-word-remember-when\/\"},\"author\":{\"name\":\"Lorri Freifeld\",\"@id\":\"https:\/\/trainingmag.com\/#\/schema\/person\/cd5c8bf4e4da92690ae292cbcfcc6849\"},\"headline\":\"Last Word: Remember When\",\"datePublished\":\"2011-11-23T13:00:00+00:00\",\"dateModified\":\"2020-12-17T20:50:12+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/trainingmag.com\/last-word-remember-when\/\"},\"wordCount\":648,\"publisher\":{\"@id\":\"https:\/\/trainingmag.com\/#organization\"},\"keywords\":[\"Last Word\"],\"articleSection\":[\"Magazine Articles\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/trainingmag.com\/last-word-remember-when\/\",\"url\":\"https:\/\/trainingmag.com\/last-word-remember-when\/\",\"name\":\"Last Word: Remember When\",\"isPartOf\":{\"@id\":\"https:\/\/trainingmag.com\/#website\"},\"datePublished\":\"2011-11-23T13:00:00+00:00\",\"dateModified\":\"2020-12-17T20:50:12+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/trainingmag.com\/last-word-remember-when\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/trainingmag.com\/last-word-remember-when\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/trainingmag.com\/last-word-remember-when\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/trainingmag.com\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Last Word: Remember When\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/trainingmag.com\/#website\",\"url\":\"https:\/\/trainingmag.com\/\",\"name\":\"Training\",\"description\":\"\",\"publisher\":{\"@id\":\"https:\/\/trainingmag.com\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/trainingmag.com\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/trainingmag.com\/#organization\",\"name\":\"Training Magazine\",\"url\":\"https:\/\/trainingmag.com\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/trainingmag.com\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/trainingmag.com\/wp\/wp-content\/uploads\/2020\/12\/training-logo-544.png\",\"contentUrl\":\"https:\/\/trainingmag.com\/wp\/wp-content\/uploads\/2020\/12\/training-logo-544.png\",\"width\":554,\"height\":150,\"caption\":\"Training Magazine\"},\"image\":{\"@id\":\"https:\/\/trainingmag.com\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/TrainingMagazine\",\"https:\/\/x.com\/TrainingMagUs\",\"https:\/\/www.instagram.com\/training_magazine\/\",\"https:\/\/www.linkedin.com\/company\/lakewood-media-group-llc\",\"https:\/\/www.youtube.com\/channel\/UCBRVPGzg9czIhhjPcViYh-w\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/trainingmag.com\/#\/schema\/person\/cd5c8bf4e4da92690ae292cbcfcc6849\",\"name\":\"Lorri Freifeld\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/trainingmag.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/ec060ebbbf717a62b45723490c570a40c5a6eee721a0e07c4a2ee09b3822eff9?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/ec060ebbbf717a62b45723490c570a40c5a6eee721a0e07c4a2ee09b3822eff9?s=96&d=mm&r=g\",\"caption\":\"Lorri Freifeld\"},\"description\":\"Lorri Freifeld is the editor\/publisher of Training magazine. She writes on a number of topics, including talent management, training technology, and leadership development. She spearheads two awards programs: the Training MVP Awards and Emerging Training Leaders. A writer\/editor for the last 30-plus years, she has held editing positions at a variety of publications and holds a Master\u2019s degree in journalism from New York University.\",\"sameAs\":[\"http:\/\/www.trainingmag.com\"],\"url\":\"https:\/\/trainingmag.com\/author\/lorri-freifeld\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Last Word: Remember When","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/trainingmag.com\/last-word-remember-when\/","og_locale":"en_US","og_type":"article","og_title":"Last Word: Remember When","og_description":"By Don Kirkpatrick  Trainingmagazine graciously asked me to write the final Last Word column for 2011 in view of my 50-plus years as a training professional, my creation of the Four Levels of Evaluation, and my recent retirement. Much has happened through these last 50 years.","og_url":"https:\/\/trainingmag.com\/last-word-remember-when\/","og_site_name":"Training","article_publisher":"https:\/\/www.facebook.com\/TrainingMagazine","article_published_time":"2011-11-23T13:00:00+00:00","article_modified_time":"2020-12-17T20:50:12+00:00","author":"Lorri Freifeld","twitter_card":"summary_large_image","twitter_creator":"@TrainingMagUs","twitter_site":"@TrainingMagUs","twitter_misc":{"Written by":"Lorri Freifeld","Est. reading time":"3 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/trainingmag.com\/last-word-remember-when\/#article","isPartOf":{"@id":"https:\/\/trainingmag.com\/last-word-remember-when\/"},"author":{"name":"Lorri Freifeld","@id":"https:\/\/trainingmag.com\/#\/schema\/person\/cd5c8bf4e4da92690ae292cbcfcc6849"},"headline":"Last Word: Remember When","datePublished":"2011-11-23T13:00:00+00:00","dateModified":"2020-12-17T20:50:12+00:00","mainEntityOfPage":{"@id":"https:\/\/trainingmag.com\/last-word-remember-when\/"},"wordCount":648,"publisher":{"@id":"https:\/\/trainingmag.com\/#organization"},"keywords":["Last Word"],"articleSection":["Magazine Articles"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/trainingmag.com\/last-word-remember-when\/","url":"https:\/\/trainingmag.com\/last-word-remember-when\/","name":"Last Word: Remember When","isPartOf":{"@id":"https:\/\/trainingmag.com\/#website"},"datePublished":"2011-11-23T13:00:00+00:00","dateModified":"2020-12-17T20:50:12+00:00","breadcrumb":{"@id":"https:\/\/trainingmag.com\/last-word-remember-when\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/trainingmag.com\/last-word-remember-when\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/trainingmag.com\/last-word-remember-when\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/trainingmag.com\/"},{"@type":"ListItem","position":2,"name":"Last Word: Remember When"}]},{"@type":"WebSite","@id":"https:\/\/trainingmag.com\/#website","url":"https:\/\/trainingmag.com\/","name":"Training","description":"","publisher":{"@id":"https:\/\/trainingmag.com\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/trainingmag.com\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/trainingmag.com\/#organization","name":"Training Magazine","url":"https:\/\/trainingmag.com\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/trainingmag.com\/#\/schema\/logo\/image\/","url":"https:\/\/trainingmag.com\/wp\/wp-content\/uploads\/2020\/12\/training-logo-544.png","contentUrl":"https:\/\/trainingmag.com\/wp\/wp-content\/uploads\/2020\/12\/training-logo-544.png","width":554,"height":150,"caption":"Training Magazine"},"image":{"@id":"https:\/\/trainingmag.com\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/TrainingMagazine","https:\/\/x.com\/TrainingMagUs","https:\/\/www.instagram.com\/training_magazine\/","https:\/\/www.linkedin.com\/company\/lakewood-media-group-llc","https:\/\/www.youtube.com\/channel\/UCBRVPGzg9czIhhjPcViYh-w"]},{"@type":"Person","@id":"https:\/\/trainingmag.com\/#\/schema\/person\/cd5c8bf4e4da92690ae292cbcfcc6849","name":"Lorri Freifeld","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/trainingmag.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/ec060ebbbf717a62b45723490c570a40c5a6eee721a0e07c4a2ee09b3822eff9?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/ec060ebbbf717a62b45723490c570a40c5a6eee721a0e07c4a2ee09b3822eff9?s=96&d=mm&r=g","caption":"Lorri Freifeld"},"description":"Lorri Freifeld is the editor\/publisher of Training magazine. She writes on a number of topics, including talent management, training technology, and leadership development. She spearheads two awards programs: the Training MVP Awards and Emerging Training Leaders. A writer\/editor for the last 30-plus years, she has held editing positions at a variety of publications and holds a Master\u2019s degree in journalism from New York University.","sameAs":["http:\/\/www.trainingmag.com"],"url":"https:\/\/trainingmag.com\/author\/lorri-freifeld\/"}]}},"_links":{"self":[{"href":"https:\/\/trainingmag.com\/wp-json\/wp\/v2\/posts\/9448","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/trainingmag.com\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/trainingmag.com\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/trainingmag.com\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/trainingmag.com\/wp-json\/wp\/v2\/comments?post=9448"}],"version-history":[{"count":0,"href":"https:\/\/trainingmag.com\/wp-json\/wp\/v2\/posts\/9448\/revisions"}],"wp:attachment":[{"href":"https:\/\/trainingmag.com\/wp-json\/wp\/v2\/media?parent=9448"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/trainingmag.com\/wp-json\/wp\/v2\/categories?post=9448"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/trainingmag.com\/wp-json\/wp\/v2\/tags?post=9448"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}