Validity Of Fce Measurements example essay topic
A shift from Prediction (PSYCHOMETRIC APPROACH) to an ability training orientation that fosters the remediation of skill deficiencies and development of skills needed to sustain employment. ON SITE ASSESSMENT EMPHASIZES IMPORTANCE OF DIRECT ASSESSMENT OF ACTUAL SKILLS AND BEHAVIORS OVER A PERIOD OF TIME AND IN THE ACTUAL WORK ENVIRONMENT. Not merely an event yielding data -- -highly dynamic process charged with meaning for both client and evaluator GOALS FOR A SUPPORTED EMPLOYMENT ENVIRONMENT ID clients employment goals, interests or preferences (some may have difficulty articulated their goals) ID clients entering behaviors in relationship to those exiting job ID or resources (people, place, things, activities that client needs to be successful in work environment Appraisal of client's work performance (ESPECIALLY Problem behaviors) (Work adjustment behaviors, productivity behavior, interpersonal skills, occupational job skills) Formulation from assessment information that proceeds on a step by step basis toward specified behavioral objectives derived from actual job demands Exploration of clients satisfaction w / job in supported environment Id social factors as the attitudes of co-workers, the extent of supervisory support or encouragement, degree of work stress or pressure, understanding of physical setting (lighting, temperature, task variety, awareness of financial rewards and ability and training requirements. Transportation to and from work site should also be considered. TO ACHIEVE ASSESSMENT GOALS KEEP IT SIMPLE! EVALUATING PROGRESS PERFORMED WITH CLIENTS Client input is important - preferences and perceived capabilities and limitations are important to solicit from clients.
IDEALLY the job coach is the person who handles the evaluation. Observations should be accomplished in short periods of time during clients work day. Information should be recorded daily. Particular attention should be given to critical behaviors. A Critical Review of Functional Capacity Evaluations The role of functional capacity evaluations (FCEs) appears to be increasing as employers and insurers rely more heavily on them for decision making. To meet credibility requirements, and the American Physical Therapy Association's standards for measurement and documentation, all FCE suppliers need to validate and refine their systems.
This article provides information that can be used to make informed decisions in the selection of an FCE and in functional assessment practices. Features of well-designed FCEs are discussed. Ten well-known FCE systems are analyzed according to these features and other common characteristics. Current issues such as the qualifications of the evaluators, reliability and validity, length of assessments, projection of endurance to 8 hour workdays, standards of practice, safety protocols, and behavioral assessment and management strategies are discussed. [King PM, Tuck well N, Barrett TE. A critical review of functional capacity evaluations.
Phys Ther. 1998; 78: 852-866.1 Functional capacity evaluations (FCEs) have become part of practice in work injury prevention and rehabilitation. These tools are supposed to define an individual's functional abilities or limitations in the context of safe, productive work tasks. A series of test activities is usually administered to measure whether an individual has the ability to meet the required job demands. Ideally, FCEs are used following an injury or illness to assist with cost-effective vocational rehabilitation. A multitude of factors are currently shaping growth and evolution of FCEs.
To contain health care costs, some efforts are underway to reduce the amount of time and money spent on administration of FCEs. Regulatory agencies such as the occupational safety and health administration are promoting the use of FCEs in an effort to ensure employee health and safety in job placement. The Americans with Disabilities Act (ADA) places emphasis on identifying an individual's physical abilities and limitation for employment and accommodation considerations. In litigation cases, FCEs have become critical for the determination of whether a claimant has wage-earning potential based on physical abilities testing. State welfare reforms and the Social Security Administration are opening new markets for FCE providers as they seek a streamlined process for disability determinations. The FCE has become a widely used tool for determining a person's readiness to return to work after injury, for performing pre-employment and post-offer screening, for making disability determinations, for setting goals and planning treatment for industrial rehabilitation, for monitoring progress throughout industrial rehabilitation, and for determining case closure.
Insurance companies and the legal profession appear to rely heavily on data acquired from FCEs to make important reimbursement and return to work decisions. The purpose of this article is to provide information that can be used to make an informed decision in the selection of an FCE system. Based on survey responses obtained from the designers of 10 well known FCE systems, comparisons are made according to common characteristics. Current issues surrounding this area of practice also are discussed. Differences Among FCEs A great number of FCEs currently are available and in use. Table 1 identifies 10 of these FCEs and compares their general characteristics.
Differences among the various approaches to FCEs include variations in the number of measurements obtained, degree of standardization, clarity of concepts and underlying theories, variety in choice of measuring instruments, adequacy of measurement for certain injury groups (eg, lifting assessments used with individuals with low back pain, use and availability of normative data, and ability to predict return to work or recurrence of injury. Tramposh attempted to classify FCEs into 2 categories: the controlled FCE and the uncontrolled FCE. Table 2 details the 2 categories, and table 3 describes some of the advantages and disadvantages of both categories. the comparison of uncontrolled versus controlled FCEs deals with issues such as type of training for FCE administration, degree of work simulation, ability to alter the test design, and generic versus job-specific testing. The classification into controlled and uncontrolled categories focuses on different aspects of these issues, but the classification is arbitrary and overlapping.
A new classification system is needed, one that further defines the methodological difference among the various types of FCEs. The commercially available FCEs and those developed within individual clinics all share the common goal of attempting to measure work-related functional performance objectively. Whether they accomplish this objective can be answered only with research. the differences in FCEs revolve around the way they assess cooperation and sincerity of effort and safety, determination of end points for stopping clients during performance of manual material-handling tests, use of isometric testing, training processes, degree of work simulation, ability to alter the test design, generic versus job-specific testing, expense of equipment, use of algorithms for scoring, methods of projecting endurance to an 8-hour workday, degree of standardization, evidence of reliability and validity, and so on. Choosing an FCE Although the practice of administering FCEs for the determination of an individual's physical capacities has been around for decades, research to justify the use of FCEs is lacking. Little is known regarding the reliability and validity of data obtained with FCEs.
In addition to not being standardized, many of the testing systems lack comprehensiveness and objectivity in data collection. Given the lack of scientific evidence to support the use of the various FCEs, the considerable financial investment when purchasing an FCE, and the magnitude of medical, legal, and financial implications arising from the administration of FCEs, the decision of which FCE to choose becomes crucial to the provider's practice. A Well-Designed FCE In our view, functional capacity testing requires the evaluator to use tests that are most appropriate for a given client. We believe there is no single most appropriate test for any one client or for any one assessment situation, because no one assessment can provide 100% of all the answers concerning work injury and return to work. A clear understanding from the referral source regarding the purpose of the FCE is essential in choosing an FCE. In evaluations where return to work is the major focus, a job analysis should be performed to determine the tasks required for the job.
The results of the FCE can then be compared with the job's physical requirements. In cases where an individual is being placed in a new job, we believe that a more comprehensive and generic assessment is needed. A range of physical demands must be tested to yield as much information as possible in order to consider a variety of job possibilities. We do not have data, however, to suggest how much information is necessary to infer form an FCE whether a person can safely function on the job. In our opinion, if a disability assessment is the purpose of the FCE, then the evaluation often can be more limited in scope and correspond to the information requested by whoever is determining the level of disability. Although the application of FCE testing is dependent on the purpose and setting, we contend that every attempt should be made to follow standardized procedures when available.
Jobs may differ from one FCE to another, but consistency should be adhered to in the methods for designing and conducting the assessment. We lack data, however, to demonstrate that such standardization leads to reliability. The National Institute of Occupational Safety and Health (NIOSH) Work Practices Guide to Lifting cited by both Miller and Matheson and the American Physical Therapy Association's (APTA) Standards for Tests and Measurements in Physical Therapy Practice indicate that therapists need to consider the following questions: Is the test safe to administer? Does the test give reliable results? Is the test valid specific to job requirements for predicting a safe level of work? Is the test valid specific to work-related abilities?
Is the test practical to administer? Does the test predict the risk of future injury or illness? For FCEs, the principles of scientific measurement should be considered, as they are for any other test. Functional capacity evaluations, therefore, should yield reliable and valid measurements. Standardization is one way FCE developers attempt to enhance reliability.
Additionally, an FCE should be comprehensive, feasible, safe to administer, and have flexibility for job-specific testing. Standardization Standardization refers to the development of a clear set of procedures for administering and scoring tests. These procedures should be written in an easy-to-use instruction manual that describes the general approach and philosophy for the development and administration of the test and the specific methods for administering all items in the evaluation. The instructions, task demonstrations, subject placement, and data collection and analysis should be documented and followed each time the evaluation is administered and should not change, regardless of the individual administering the assessment. We believe that each task should be defined and described according to the equipment needed and the procedures to be followed. Verbal instructions for administration of the test, in our opinion, are essential to minimize examiner bias.
All terminology should be defined to avoid misinterpretation of meaning. Specifications for scoring should be clear and easy to interpret to promote accuracy in recording of results. If the examiner needs to extrapolate or project data, the manual should provide instructions for accomplishing this task. Some assessments, such as manual dexterity tests, have standardized procedures and equipment by which normative data have been established. For example, the Minnesota Rate of Manipulation Test measures the ability to make skillful, controlled arm-hand manipulations of larger objects. Norms bayed on older, unemployed adults and on young adults are available in percentile tables.
Caution, however, should be used when comparing an individual's performance with normative data, because the ADA, for example, prohibits using this method to make decisions regarding return to work. Decisions need to be made based on the functional abilities and limitations of each individual as they relate to his or her job situation. Denying a job to an individual with a disability based on data that compares his or her functioning with that of the general population is illegal under the ADA. There are procedure manuals for all 10 FCEs reviewed for this article. The degree, however, to which they incorporate the components described varies. Objectivity The term 'objective' is used to indicate a measurement that has a degree of reliability and is relatively free from examiner bias.
Objectivity in testing can be promoted when the procedures, variables for observation, and scoring system are operationally defined. Visual observations can be objective if operational definitions and scoring criteria are applied. ' Reliability The importance of reliability and validity of FCE measurements cannot be overstated. If an FCE measurement does not have established reliability, test results could be different with each administration.
Without validity testing, there is no way of knowing whether the results are accurate. Reliability refers to consistency in measurement. Interrater and test-retest reliability have been purported to be the 2 most important forms of reliability in FCE testing. Interrater reliability refers to the ability to achieve similar scores on an evaluation when administered by different evaluators. In industrial rehabilitation programs, one therapist may administer an FCE to a client prior to treatment, whereas another therapist may administer an FCE following the rehabilitation program. Interrater reliability helps to ensure that the differences between the 2 FCE scores were not due to the differences in the therapists' test administration and scoring.
Interrater reliability is tested by having 2 or more administrators give a test independently to the same group of subjects. The administrators's cores are then compared by calculating correlation coefficients to determine how closely the scores are related. Test-retest reliability or intra rater reliability refers to the stability of a score derived from one administration of an FCE to another when administered by the same rater. Variables potentially affecting the results of this type of measurement include the time between the 2 evaluations, the stability of the client's physical condition, and the treatment received by the client between the FCEs. For the FCE, it is important to determine a time interval that is long enough to minimize a learning effect from the first test for both the client and the examiner but short enough so that the client's medical condition will not have changed substantially between tests.
Most of the reliability studies of FCEs were performed on subjects without disabilities. The reliability FCE measurements on persons seen in clinical practice, therefore, is unknown. Caution should be exercised when applying these results to various disabilities, because conclusions based on such extrapolations may be highly misleading. From the systems reviewed for this article, only the Physical Work Performance Evaluation (PWPE) and the WEST-EPIC (lifting-capacity section only) FCEs have been examined for intra rater and inter rater reliability with results published in peer-reviewed journals. Some components of the Blankenship FCE are based on methods developed and studied by other investigators. Some of these tests, however, were developed for persons without injuries.
No published peer-reviewed research documenting the reliability and validity of measurements obtained with the Blankenship FCE is currently available. This can also be said for the WorkAbility Mark Isernhagen Work System, A RCON Key Method, Work Hab, Access Ability and ERGO S systems. Validity A score is considered valid if it measures the properties that it purports to measure and can be used to make inferences. In FCE testing, this means that the score predicts real-world function. The interpretation of the test results should predict or reflect the client's performance in a target work setting or predict an overall level of work if there is no target work setting.
' Validity is an essential requirement for all measurements. Establishing the validity of a physical work performance measure is more difficult than establishing its reliability due to the need for a gold standard for comparison with predictions. There are various ways to evaluate the validity of FCE measurements. The most common, content validity, asks whether a representative sample of the components of the task in question are incorporated into the assessment.
14's 4 Wor example, content validity of FCE measurements can be implied based on the clinician's professional judgment as to the degree to which the test measures the job's demands, but this approach is considered relatively weak and provides no more than a logical argument for validity. We believe a thorough and systematic job analysis is necessary to accurately determine job demands. If an FCE is not job-specific, content validity can be established by covering all 20 physical demands of work described by the Dictionary of Occupational Titles Because content validity deals with theoretical arguments, it should not be used as the sole basis for suggesting that an FCE is valid. From an examination of the existing FCEs, it appears that several FCEs have good content validity when judged by the US Department of Labor's 20 demands. Criterion-related validity deals with whether there is evidence that a measure can be used to make an inference.
In the case of FCEs, the most common inference is whether the measure predicts an individual's performance in specified activities. Concurrent validity deals with the correctness of an inference at the time of the measurement and may be used to determine whether a client needs therapy. If FCE scores distinguish between those clients who are currently unable to perform at a certain level of physical ability and those clients who can perform at this level, then the FCE is said to have good concurrent validity. Predictive validity refers to a measure's ability to predict future performance. In the case of an FCE, if a client's scores predict a certain level of performance and the client successfully returns to work at that level, the FCE is said to have good predictive validity. Defining and measuring the criterion by which clients are compared becomes the greatest challenge to this test.
External factors such as the work environment, the work pace, and the work schedule may affect the client's level of performance anti ability to return to work. In addition, for individuals with low back pain, we believe that ability or disability at any age should be evaluated relative to their current aspirations for a 'normal' life. This, individual, cultural, and economic factors must be taken into account. Responses obtained from the 10 FCE designers chosen for this article indicate publication of a validity study in a refereed scientific journal exists only for the PWPE. Lack of peer-reviewed publications for the FCEs reporting the completion of validity studies leaves open the question of whether the FCEs are acceptable. With the exception of the PWPE, the FCEs reviewed for this article do not provide the validity studies that are seen as the prerequisite for demonstrating that a measure is credible.
Components of FCEs Data Gathering Table 4 lists the major components of FCEs. Most methods begin with collecting a client's medical, social, and vocational history. This information assists with determining a client's perceptions of his or her own abilities. Data may be obtained through record reviews, a client interview, or a self-administered client questionnaire. A medical history can identify conditions that may indicate that an FCE is contraindicated or that require precautions to be taken during testing. We believe it is important to determine how healing from an injury has progressed and how the client is affected by his or her condition.
Gathering information regarding exercise programs, home and recreational activities, and level of functioning in activities of daily living is helpful as part of the FCE in order to establish a baseline on the client and reduce the risk of re injury. Some FCEs incorporate psychological screening and pre-vocational testing into the initial data-gathering phase. A vocational history is particularly helpful if job exploration is necessary. Physical Examination Some FCEs include a musculo skeletal examination, whereas other FCEs incorporate the musculo skeletal component into the evaluation. A third approach is to perform a musculo skeletal screening only when 'red flags' (eg, high blood pressure, elevated heart rate, recent surgery) are raised in the intake history or review of the medical record. ' ' Regardless when it is performed, the musculo skeletal examination, along with the questionnaire and interview information, are taken into account for comparison during the performance of the physiological and functional assessment portions of the FCE.
The primary reason for performing a physical assessment is to identify clinical signs related to conditions that are contraindications for testing or that should be monitored closely during testing. Physiological Measurement Measures include items such as muscular endurance and cardiovascular endurance. Muscular endurance is usually measured by describing the demands of the activity performed and the duration of muscle performance prior to fatigue. Sub-maximal protocols, which have predetermined termination points based on a percentage of the client's estimated maximum heart rate, are a method of assessing cardiovascular endurance. A variety of approaches are used to predict the weights that can be lifted occasionally and frequently. The Isernhagen Work Systems FCE notes changes in body mechanics, accessory muscle use, heart rate increase, and other physical signs to determine whether the effort expended to lift a load is light, moderate, or maximum.
Blankenship calculates the weight lifted frequently from the weight lifted occasionally, using 1 of 3 formulas. This system also provides for direct measurement of lifting performance by requiring the client to perform one lift every 5 seconds, with additional weight being added every 4 lifts. The Key Method calculates the weight lifted frequently as 50% to 60% of the weight that can be lifted occasionally. None of these protocols have been published in peer-reviewed journals.
Furthermore, a recent study attempted to predict weight lifted frequently by regression analysis and included factors such as subject's age, time since injury, and weight that could be lifted occasionally. A large amount of error in the estimates led the researchers to conclude that the use of formulas in predicting the weight that can be lifted frequently is questionable. Grading of exercise intensity and the accuracy achieved in predicting maximal oxygen uptake provided by treadmill and bicycle ergo meter testing make these tests more popular for use in clinical and research environments. There is little reference in the literature to the inclusion of fitness evaluations in FCES. It appears that these assessments are usually conducted as 'stand-alone' fitness evaluations rather than as part of an FCE. Functional Performance Most comprehensive FCEs include the physical demands of work as specified by the US Department of Labor in the DOT.
The job factors listed in the DOT express both the physical requirements of jobs and the physical capacity a worker must have to meet those demands. Developing a method of measuring physical capacity in terms of job factors should allow a direct translation of which jobs a client can perform, but only research can demonstrate whether this is true. These 20 physical demands are often used by rehabilitation specialists and vocational counselors to classify jobs. The DOT is a valuable resource in initially attempting to identify potential work opportunities. Just because a task is covered by a test, however, does not mean that it has been evaluated thoroughly or that the data produced are objective specific and quantified.
An may not have a standard protocol or an objective measurement tool. Examiners need to determine not only whether a skill is evaluated in a particular FCE but also how it is evaluated. Testing should mimic the tasks and equipment as they occur in industry, and validity studies are the means of determining whether this has been done successfully. The FCEs that depend on very sophisticated technologies tend to cover fewer than the 20 physical demands of work. Whether the data generated by this equipment are more reliable or valid is yet to be determined. Critics of the DOT suggest it is not comprehensive enough in defining all types of jobs.
Despite these deficiencies, the DOT remains the most comprehensive set of occupational characteristics currently available. Physical capacity as it relates to lifting abilities is most often defined using the DOT classifications of physical work (Tab. 5) In our opinion, broadly classifying the client as having sufficient strength to perform at the various levels of physical demands usually is not sufficient information for safe, productive job placement. We believe, therefore, that it is important to test the client's lifting capacity under a variety of conditions and with the client in different postures. Many lifting procedures related to functional testing have been identified.
Historically, 3 types of force assessments have been used: isometric, iso kinetic, and iso inertial. Isometric testing requires exertion of a force against an object that does not move. This type of testing is relatively simple, quick, inexpensive to administer, and easy to control. With iso kinetic testing, movement is restricted to a specific speed. This type of testing is usually limited to muscle testing rather than to assessment of task performances because tasks are almost never performed at constant speeds. Isoinertial testing is described as lifting progressively heavier weights at a set frequency over a specific vertical range.
This type of testing has been adapted to test adults with medical problems, and this version relies on established criteria such as cardiovascular response and the patient's requests for stopping the test. Isoinertial testing has been reported to be reliable for the amount of weight lifted. The client self-report method, sometimes called 'psychophysical method' of testing, requires a subject to randomly adjust a load during a period of 20 to 25 minutes until an acceptable load is found that the subject estimates can he lifted repetitively during a regular 8-hour work shift. This method relies on the client to determine capacity. Researchers have proven, however, that subjects tend to overestimate the maximum acceptable weight of lifting. Critics of' this method suggest that clients may have reasons to be fearful and, as a result, that measurements may not be reliable.
Another approach to evaluating lifting capacity is to observe movement to determine maximal effort. This method of testing is described as lifting progressively heavier weights until the examiner makes a decision to stop the test. The examiner relies on observational cues and cardio-respiratory signs to determine the degree of safety of the lifting tasks. The results of isometric or iso kinetic tests of individual muscle or whole-body torque correlate poorly with performance of functional activities, and these tests do not appear to be valid for predicting function. These tests, therefore, may be more useful in measuring impairments. Direct tests of material-handling capabilities also have the practical advantages of simplicity, low cost, and portability.
Although reliability has been demonstrated in lifting studies, these free-lifting protocols have been criticized for inadequate anatomic stabilization, subjectivity of the psychophysical end point, and lack of control for speed and acceleration variables. To ensure safe and consistent documentation of maximum functional levels, logical and ethical considerations should guide the development of lifting-capacity testing. Most clinicians initially subject the client to low levels of weight at a given frequency and gradually increase the load while monitoring the client's response. The clinician should encourage the client to lift slowly, smoothly, and continuously to avoid exacerbation of the injury. Report Writing Several authors have emphasized the importance of logical and clear reporting of observations, and that reports be in an easy-to-read format and free of jargon. The reader should get a clear picture of the client's physical capabilities and limitations as they relate to critical demands of the job.
Optimally, we contend, this comparison should be made using the results of a job analysis, identifying both the essential physical demands and the necessary qualifications for the targeted job. Obtaining a job analysis, however, is not yet a standard in many workplaces. Sometimes, comparisons of the worker's abilities with the job demands need to be based on the clinician's estimates and on reports from the employer as well as from the employee. If the results of a job analysis are not available for comparison with the client's abilities and the employer will not cooperate with a job analysis, then job demands can be determined through client and employer self-report.
The source of information on job demands should always be documented in the report. This documentation is particularly important if the client is involved in litigation. Client cooperation, consistency of the client's performance during the assessment, and safety of movements and body mechanics should be noted as well. As part of an FCE report, there should be clearly written descriptions of proposed job modifications and recommendations for any further modifications. Such descriptions are important because the results of FCEs are utilized by a wide range of professionals. A logical and clear reporting style is essential when assessing clients for Workers' Compensation purposes.
All recommendations, therefore, should be substantiated by the data obtained during the FCE. Where appropriate, the final report of an FCE should compare an injured worker's status with the requirements of the job to which he or she is returning. Issues in FCE Development Qualifications of the Evaluator There is some debate surrounding the issue of who is best qualified to administer FCEs. Occupational therapists and physical therapists appear to have been the first health care professionals to perform FCEs. These therapists have traditionally evaluated function, performed task analyses, and have an educational background that provides an understanding of pathology and the musculo skeletal system, including muscle function and movement patterns.
Other health care professionals currently involved in administering FCEs include athletic trainers, chiropractors, vocational evaluators, nurses, physical therapist assistants, certified occupational therapy assistants, exercise physiologists, psychologists, kinesiologist's, and physicians. These individuals often claim to have acquired skills and knowledge in task and movement analysis from clinical practice and continuing education courses. The issue of who is qualified to administer FCEs is being addressed by determining the competencies required to perform a safe, reliable, and valid FCE. Once determined, these competencies guide mandatory training sessions in which clinicians eventually demonstrate competence through written and practical skills as well as testing procedures. Training Given the difference in skill levels among the disciplines, evaluator training should, in our view, become an important means of ensuring quality and consistency in evaluation, scoring, and report writing related to FCEs. All FCEs on the market provide for training; however, there is variance in how this training is delivered.
Some FCE designers use the 'train-the-trainer' approach, whereas other FCE designers insist on certifying each individual desiring to administer their FCE. Regardless of the type of approach used, we believe that training is essential, but research has yet to provide data as to how much training is needed to obtain valid and reliable measurements. Training ensures that administrators of FCEs are knowledgeable and competent in carrying out the established procedures. Length of Assessment The length of time to administer an FCE varies among designers. Some FCEs require less than 2 hours to administer, whereas other FCEs require 3 to 4 hours to administer. One FCE is administered over a 2 day period.
Le chner et al and Isernhagen reported that the shorter FCEs (1-2 hours) seem to provide less reliable data and appear to be less comprehensive, but they did not provide data to support their contention. The shorter FCEs are less likely to include all 20 physical demand items identified by the DOT. Miller and Isernhagen recommended that the standard FCE be 4 to 6 hours in length and assess general work demands such as lifting, carrying, reaching, sitting, standing, and walking, as well as hand strength and coordination. Tramposh indicated that, to meet consumer demand, an FCE should be conducted in one session. Isernhagen, however, stated that the most reliable format for conducting an FCE is over a 2-day period, with the most critical tasks being repeated on the second clay. The 2-day format allows for retesting for accuracy and for evaluating the effect of the first day's assessment on the client, but Isernhagen did not provide data to support this assertion.
There are no data to support the selection of a specific length of time for FCE testing. Functional capacity evaluations that do not take long to administer may compromise content validity because addressing.