Area of Expertise (AOE) – Measuring and Evaluating

by

This area of expertise is about gathering data to answer questions regarding the value and impact of learning solutions. Not only does it focus on individual programs or courses, but it also encompasses the overall measure of system effectiveness. It takes advantage of findings to increase effectiveness and provide further recommendations for change.

You can expect to see about 12% of the CPLP exam cover this AOE.

Some of the skills used in this AOE include:

  • Statistical theory and methods
  • Designing research
  • Analysis methods
  • Interpreting and reporting data
  • Data Evaluation 


You use statistics to document current performance at the individual, group, or organization level, measure the impact of the program, and provide reliable feedback.

As a professional that measures and evaluates the impact of training, you must have a broad understanding of how data falls into distributions and how the different sets of data relates to the other data. You also need to have an understanding of hypothesis testing (effect sizes and confidence intervals). Because it is easy to misuse or misread your statistics, this skillset requires that you apply statistics correctly, represent data accurately, and draw correct conclusions.

Some theories and methods you should be familiar with include:

  • Frequency distributions
  • Measurement scales and statistical implications
  • Measures of central tendency
  • Measures of variance
  • Distributions of standard scores
  • Correlation
  • Statistical inference
  • Hypothesis testing
  • Effect sizes
  • Confidence intervals

You need to take knowledge about the theories and methods and apply it to designing research models that allow you to implement measurement and evaluation and make recommendations on how to implement measurement and evaluation. You must know how to select the appropriate tools, define populations, collect data, and find supplemental references when you need them.

You need to be skilled at the process of discovering and interpreting meaningful relationships within the data and summarizing the results. Specific analysis techniques often summarize the impact of solutions, but specialized expertise is widely applied to the workplace. Knowledge of the workplace allows you to explore data comprehensively to inform decision-making and to balance strictness and accuracy with feasibility and usability. It also requires ROI, cost-benefit analysis, and forcasting.

Your target audience must be able to interpret and understand your compiled data. This starts with you selecting the data and results to report. The translation format should be a clear and accurate graphic format and provide supporting text.

Some evaluation theories and types you should be familiar with include:

  • Assessment development issues
  • Measurement processes
  • Tyler’s goal attainment models
  • Scriven’s output models
  • Kirkpatrick’s four levels of evaluation
  • Phillip’s fifth level of evaluation for measuring ROI
  • Six Sigma methodology (at a high level)
  • Balanced scorecard approach
  • Stolovitch’s handbook on Human Performance Technology (HPT)
  • Meta-evaluation methods

If you want to share your thoughts or enhance this AOE with your experiences, please feel free to do so as a comment.

Advertisements

Tags: , , , , , , ,

One Response to “Area of Expertise (AOE) – Measuring and Evaluating”

  1. ASTD Competency Model « ASTD – Central Florida Chapter Blog Says:

    […] Measuring and evaluating […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


%d bloggers like this: