Marvin Conn; K Mark Derby; T F McLaughlin

Case Report

Austin J Psychiatry Behav Sci. 2014;1(2): 1010.

Review of a Behavioral Assessment Process from Descriptive Assessment to Experimental Analysis: A Summary of 118 Cases

Marvin Conn, K Mark Derby and T F McLaughlin*

Department of Special Education, Gonzaga University, USA

*Corresponding author: TF McLaughlin, Department of Special Education, Gonzaga University, Spokane, WA 99258-0025, USA

Received: Fabruary 10, 2014; Accepted: March 10, 2014; Published: March 21, 2014

Abstract

Functional analysis is one of the more robust advances in the assessment and treatment of severe behavior. Using technologies to identify the function of aberrant behavior and adjunctive reinforces to develop treatment packages is crucial to developing interventions designed to reduce aberrant behaviors while increasing alternative replacement responses. Descriptive assessments, preference assessments, reinforce assessments, and experimental functional analyses are tools that have been empirically validated to accomplish this goal. These methods have been shown to be effective in inpatient settings, homes, schools, and outpatient clinics. This study examines the use of these procedures for 118 clients evaluated in either an outpatient clinic and in the natural setting (i.e., schools, homes, and daycares). All procedures were conducted by graduate students enrolled a BCBA approved program for Masters Level practitioners. In addition, all evaluations were supervised by a doctoral level behavior analyst. We present a descriptive summary of the effectiveness of (a) the effectiveness of using the Motivation Assessment Scale at identifying maintaining functions compared to experimental analysis outcomes, (b) the preference assessment to identify reinforcing stimuli, and (c) the effectiveness of treatment packages developed. The efficacy of the outpatient model to serve clients with severe problem behaviors is discussed.

Keywords: Functional behavioral assessment; Preference assessments; University–based clinic; Undergraduate and graduate students; Aberrant behavior.

Introduction

The advent of functional assessment technology by Iwata, Dorsey, Slifer, Bauman, and Richman [1], may be one of the more robust advances in the assessment and treatment of severe behavior displayed by persons with developmental disabilities [2]. Its’ development has lead to a proactive rather than reactive treatment approach for severe problem behavior [3]. As described by Iwata et al. the procedure itself consists of the systematic replication and withdrawal of analogue conditions until a pattern of responding is shown. Typically, these analogue conditions are designed to evaluate the role of positive, negative, and sensory reinforcement functions. To date, this technology has been shown to be robust across diagnostic groups, behaviors and settings [4–6,1,3,7].

Despite the robust success of both functional analysis and descriptive assessment procedures, most published clinical evaluations have been conducted in extremely controlled environments [7,8,5]. Since its inception, a general concern has been the level of expertise and precision needed to conduct a functional analysis [8]. Initially, there were concerns of not being able to establish control over the environment in outpatient settings similar to the level that can be accomplished in a clinical setting [9–10]. As such, early replications of the procedure focused on its application across settings [8,11]. However, there is strong evidence that these procedures can be used in less clinical settings. For example, Derby et al. [4]. Evaluated 79 cases in an outpatient setting and the utility of the procedures were established. In its simplest form, treatment based on functional analysis results consists of contingencies to increase the learning of alternative behaviors while systematically decreasing problem behaviors targeted for reduction [3].

Studies have shown that the use of functional analysis based treatment in combination with adjunctive reinforces can reduce the time needed to decrease problem behaviors and increase alternative or adaptive behaviors [12]. One method to identify adjunctive reinforces that has been shown to have fidelity within treatment is the forcedchoice hierarchal procedure described by Fisher, Piazza, Bowman, Hagopian, Owens, and Slevin [13]. With this technique, the evaluator presents stimuli in a concurrent operant paradigm that more closely resembles a natural environment when choice options are present.

The purpose of this study is to examine the outcomes of the assessment, functional analysis, and reinforcement assessment procedures [4,7,11], completed at a university based clinic over a ten–year period (1999 – 2009) [14]. During this period, 118 clients were assessed. The data represented are the findings from those cases. Questions to be answered through this study were: For what percentage of clients did the preferred items identified during the preference assessment match the outcomes obtained during the reinforcement assessment? What percentage of clients did the controlling function identified through the indirect assessment [15] (Alter, Conroy, Mancil & Haydon, 2008) match the function identified during the experimental functional analysis? How did the functions identified during this study compare to similar epidemiological studies [1,4]? Lastly, how effective was the developed treatment for the reduction of aberrant behaviors?

Method

The investigation was conducted across three phases. Phase 1 consisted of the initial data gathering and visual inspection of available case files. In phase 2, outcomes of the assessments were compared. The preference assessment (PA) outcomes were compared with the outcomes of the reinforce assessment (RA). The maintaining condition identified through the Motivational Assessment Scale (MAS) [15] was compared against the maintaining condition identified during experimental analysis. During phase 3, we evaluated the treatment effectiveness by comparing the reduction of aberrant behavior observed across functional analysis baseline and treatment sessions.

Participants and settings

Participants were served in a university–based clinic located at Gonzaga University in Spokane Washington [14]. Client’s evaluated through the clinic ranged in age from 2 years to 24 years old. All of the clients were referred for engaging in severe problem behaviors. Problem behaviors ranged from physical and verbal aggression to self–injurious behavior. Clients assessed were previously diagnosed with a multitude of problems including mild to profound intellectual delays, autism, mental illness, physical disabilities, and traumatic brain injury.

Assessments and treatment sessions were conducted in clinic, school, or home settings. The same procedures were used across all settings. Assessments conducted in the clinic setting were completed in a room designed to serve as a integrated preschool and clinic context. The room had items found in typical preschool classrooms (i.e., toys, tables, chairs). The room was equipped with a one–way mirror that permitted unobtrusive observation. Community based assessment and treatment sessions were conducted in locations specific to each client (e.g., school classrooms, homes, or daycare settings).

Assessment and treatment conditions were completed by graduate students enrolled in a BCBA approved Masters in Special Education Functional Analysis program [16]. All cases were supervised by two clinic directors, who were trained behavioral analysts. Evaluations were conducted across varying time periods ranging from 3 weeks to 4 months, which mirrored the universities semester and summer schedule.

General clinic procedures

Assessments

The assessments completed included a functional assessment interview adopted from O’Niel, Horner, Albin, Sprague, Steney and Newton [12] that included the Motivation Assessment Scale [15]. Direct assessments consisted of preference [13], and reinforce assessments [12] to determine possible tangible items and the relative reinforce strength of across items. Next, a brief functional analysis [1,4,7,11,17] was used to identify maintaining conditions for problem behavior. Lastly, a treatment package was developed based on the outcomes of the both assessments.

Indirect assessment: Prior to being evaluated, the clients’ care provider was asked to complete an informational packet that included the functional assessment interview and the MAS [15]. Definitions and topographies of targeted aberrant behaviors were identified via the assessment questionnaire and a follow–up phone interview. A graduate student clinician conducted a reinforce assessment interview [18] to identify potential adjunctive reinforces (i.e. eight to ten items that could be used during the preference assessment). The interview consisted of questions about items that provide visual, olfactory, edible, auditory, tactile, and social stimuli to the client. A list of items that could serve as a reinforce was created based in this parent report.

Motivation assessment scale: The MAS [15] is a questionnaire that consists of 16 questions. Care providers are instructed to answer how frequently the target behavior is likely to occur in specific situations (e.g., when presented with a difficult task). Questions are specific to one of four functions, escape, attention, tangible, and sensory. Each function has four questions directly pertaining to the function. Respondents answer each question using a Likert scale from 0 to 6 (0 behavior never occurs, 6 behaviors always occurs). Scores for each function were totaled then divided by 4 to obtain a mean score. The scores were rated relative to each other, the highest mean score was then rated as a 1 the lowest mean score was rated as a 4. A 1 rating was considered to be the most likely maintaining function of the behavior. Using this information, clinicians developed a hypothesis about function of the behavior.

Direct assessments

Motivation Assessment Scale Using a forced–choice format [13,19] each item was paired against all other items. The client was verbally prompted (i.e., which one do you want ________ or ________”) to pick between two presented items; thereby, gaining access to that choice item for a brief period of time. The non–selected item was simultaneously removed with the delivery of chosen item to client. Data collectors recorded the item selected for each pairing. Attempts to gain access to both choice items were redirected by the therapist. Forced–choice selections were continued until all items had been paired with each other at least once. If there were any ties (i.e., an item was chosen an equal amount of times) a tie breaker presentation was conducted (i.e., the two items presented until one item was selected).

Preference assessment outcomes were graphed using a bar graph format. To aid in the visual interpretation of the results the item chosen most often (i.e., highly preferred) was placed closest to the abscissa followed by the remaining items in order from most to least selected. When tie–breaker presentations were conducted, the more preferred item chosen was identified by a (*) symbol being placed above that items bar. A typical graph is shown in Figure 1. This client’s preferred items were: balloon, duck, frog, weeble, hat, seal, Nemo, keys, and square. The graph depicts these items from most to least preferred respectively. For this client, tie breaker presentations were conducted for the frog versus weeble and seal versus Nemo choices.

Figure 1 :Typical graph for preference assessment Highly, Middle, and Low preferred items are identified. Seal was chosen as middle preferred based on the hypothesis of attention maintained behaviors. The hat required the therapist to get close to the client so that the client could remove the hat from the therapists head, potentially influencing the choice made. Tie breakers are identified by stars above the item that was chosen as more preferred.



Figure 1 :Typical graph for preference assessment Highly, Middle, and Low preferred items are identified. Seal was chosen as middle preferred based on the hypothesis of attention maintained behaviors. The hat required the therapist to get close to the client so that the client could remove the hat from the therapists head, potentially influencing the choice made. Tie breakers are identified by stars above the item that was chosen as more preferred.

Reinforce assessment: Following the preference assessment a reinforcement assessment [12] was conducted using the high preferred (HP), middle preferred (MP), and low preferred (LP) items identified via the preference assessment. The items were paired against each other in a concurrent operant format (i.e., HP versus MP, HP versus LP, and MP versus LP). An identical task, determined by the abilities of the client, was placed in front of each item. A therapist verbally prompted the client to complete the task by saying “On this side is the ________ and on this side is the ___________; If you want the ___________ then ___________, if you want the ________ then __________”. After the client completed the task, access was provided to the item and the other item was removed. Data collectors recorded completion of the task associated with each item to determine the reinforce efficacy of each item. Each session consisted of at least five presentations of the two items. Items were presented in a counterbalanced manner to prevent side preference. Sessions were conducted until a clear preference could be made or it was determined that no clear reinforce could be identified. The clinician graphed the outcomes of the assessment in a line graph format. Figure 2, shows a typical reinforce assessment line graph using the same client as Figure 1. In this example the HP was chosen more often than both LP and MP items, and MP was chosen more often than LP.

Figure 2: Example of typical reinforcement assessment graph.



Figure 2: Example of typical reinforcement assessment graph.

Experimental analysis

Functional analysis: Clinicians used brief functional analysis procedures [11,7,4] to identify potential maintaining functions for aberrant behavior. Utilizing a multi–element design (Kazdin, 2011), analogue conditions were evaluated to determine the role of escape, tangible, attention, and sensory functions. Sessions continued until a pattern of behavior across conditions were identified. For each client, analogue condition selection and the order of conditions conducted were determined based on hypothesis developed from information obtained during the indirect assessment. Sessions typically lasted 5 minutes in duration. For escape conditions, the client was presented with a task demand. If the client engaged in aberrant behavior the task demand was immediately removed for approximately 30–s. The task was represented after 30–s or after the cessation of problem behavior, whichever occurred first. Tangible analogue conditions were completed by first allowing the client to engage with preferred stimuli for 2 minutes. After two minutes, the therapist took the item from the client simultaneously with the verbal prompt “My turn”. If the client engaged in aberrant behavior the item was returned to the client for 30–s. This sequence of events was repeated every 30–s throughout the session. During the attention condition, no adult attention was provided unless aberrant behavior occurred. If the client engaged in aberrant behavior, the clinician would deliver a verbal reprimand. Following the reprimand, adult attention was removed until another aberrant behavior occurred. Alone conditions were conducted by placing the client in the clinic room with no other person in the room. Each functional analysis utilized the free play context as a control condition. During the free play condition, the client was given access to all items and non–contingent attention was provided by the caregiver and the therapist. No demands or requests were made of the client during the free play condition. An example of a functional analysis is provided in Figure 3. As shown, the client displayed a higher rate of aberrant behavior during escape conditions when compared to all other conditions. Thus, an escape function was identified for aberrant behavior.

Figure 3: Typical functional analysis line graph. Graph shows a maintaining condition of tangible.



Figure 3: Typical functional analysis line graph. Graph shows a maintaining condition of tangible.

Choice assessment: For FA’s that were inconclusive (i.e. no function identified) or no target behaviors were observed during sessions, a choice assessment [19,20] was conducted. First, tape was placed on the floor dividing the room in half creating two choice areas. Each side of the room was then paired with differing levels of adult attention, task demands, or preferred tangibles. For example, to evaluate the role of adult attention versus tangible reinforcement, one side of the room was associated with the high preferred item with no attention and the other side of the room was associated with the low preferred item but continuous attention was provided. Thus, if the client moved to the side of the room associated with the high preferred item a tangible reinforcement function was hypothesized. To assess the relative effects of attention versus escape as a possible function, one side of the room was associated with high demands with continuous attention and the other side of the room was associated with low demands and low levels of attention. If the client moved to the side of the room associated with attention and demands an attention function was hypothesized. Data were recorded on which side of the room the child walked toward. Clients were given access to the chosen side for 30–s then returned to the center of the room and instructed to walk to the side of the room they wanted to go to. If no choice was made, the therapist would take the client to the center of the room and verbally prompt them to choose a side and condition [21]. Attempts to take the stimuli to the other side or leave the area resulted in therapist returning the stimuli to the appropriate choice area, and client was taken back to the tape and told to walk to the side of the room they wanted to be.

Experimental analysis graphing: Data were plotted using line graphs to provide a visual representation of outcomes for each session. Typical graphs were completed using percentage or rates per minute of aberrant behavior observed during sessions. Points were plotted on line graphs with sessions and session type clearly indicated on the graphs (Figure 4). Clinicians visually inspected graphs to determine if a specific analogue condition resulted in higher rates of behavior when compared across all conditions

Figure 4: Shows line graph of treatment using Functional Communication Training. Percent of aberrant behavior and number of mands shown.



Figure 4: Shows line graph of treatment using Functional Communication Training. Percent of aberrant behavior and number of mands shown.

Treatment packages: Clinicians developed treatment packages based on the maintaining function identified during experimental analysis. Figure 3 shows a client who exhibited an escape function during the experimental analysis. The treatment selected for this client was Functional Communication Training (FCT). The client was taught to mind for a break, pressing a micro switch that activated a recording of “break please”. Figure 4 represents the graph of treatment sessions conducted in teaching the use of the mind. Along the left abscissa the percentage of aberrant behaviors across sessions are shown, the right abscissa shows the number of minds observed during sessions. As sessions progressed, aberrant behaviors decreased while a concomitant increase in minds was observed.

Treatment packages when experimental analysis was inconclusive: In cases in which no behavior was observed in either the FA or CA, treatment packages were developed based on the hypothesized function derived from the indirect assessments (i.e., interview, MAS). Conditions were developed to determine if a functional reinforce could be identified for alternative behaviors, typically a mind. Thus, a contingency reversal was conducted as described by Derby et al. [4]. If the functional reinforce was identified, care providers were trained to implement the treatment contingencies.

Data collection and interobserver reliability

Data collection

Independent observers collected data for direct assessments and treatment sessions. Data were collected by using paper and pencil. In addition to the data collectors, sessions were videotaped for further analysis if necessary (e.g., if a new behavior was identified during functional analysis or when inter–observer reliability was below 80%). For evaluations conducted in the clinic setting, independent trained observers documented the presence or absence of defined targeted behaviors within each session using a partial interval collection system. The primary data collectors were graduate level students in the functional analysis program and reliability was conducted by undergraduate students. Prior to collecting any data, training was provided on the data collection methods. Prior to each session, the clinician would teach the data collectors the definitions of the target behaviors and tell them the planned order of conditions. A second independent data collector was present for data reliability for a minimum of 30% of all sessions completed for each client. Data collectors independently recorded observations manually on a data collection form designed for each client’s evaluation.

Interobserver reliability

To obtain reliability data, each 6–s interval was compared between the primary and reliability data sheets. If both observers score matched for the segment an agreement was tallied. A disagreement was scored if the observer’s scores did not match. For each client, inter–observer reliability was required to be 80% or better across 30% of the sessions. If the minimum reliability levels were not obtained sessions were rescored by observers watching the videotape of each session. Rescoring of data was continued until agreement of 80% was reached.

Phase 1: Chart Review Procedures

The first author reviewed all case files available and coded all assessment outcomes on a data collection form designed for the study, (Attachment A). Each assessment was coded independently of all others (i.e., preference assessments were coded separately from reinforce assessments), providing a method to compare and contrast assessment procedures.

Inclusion and exclusion criteria

Inclusion: To be included in the study the client case file needed to include one or all of: (a) completed MAS and experimental analysis graph, (b) graphs for experimental analysis and treatment sessions, (c) graphs for preference assessment and reinforce assessment.

Case file information

Client information: Prior to inspecting assessment data, basic client information was collected. This information included the name, age, gender, clinic dates, date of birth, client’s diagnosis and current medications.

Target behavior identification: To identify behaviors of interest, the client’s final report was reviewed for definition of the behavior. The report included the specific behaviors identified and examined during the assessment process.

Indirect assessment: Each completed MAS assessment was visually reviewed. Prior to recording the function rankings the behavior of concern identified on the MAS was compared against target behavior identified in the target behavior identification section in the client report. If more than one MAS was completed, rankings for each completed MAS and target behavior was recorded. The rankings of each function were then documented on the study data sheet.

Incomplete or no MAS: Client files that did not have completed MAS were recorded as incomplete.

Multi–function behavior: In cases two or more functions tied, (e.g., suggesting behavior may have multiple functions) the rankings were recorded on the data sheet as identified (e.g., both escape and tangible functions obtained mean scores of 4). Thus, both functions were given the same ranking within a hierarchy of potential maintaining functions of problem behavior (e.g. Tangible 1, Escape 1, Attention 3, and Sensory 4).

Experimental analysis: Each case file was visually inspected and functional assessment or choice assessment graphs analyzed. Each graph was reviewed to determine if a maintaining function could be identified. To identify a maintaining condition the graph had to show a higher level of aberrant behavior when compared across all conditions. The condition was recorded on the data sheet as the maintaining condition. A typical graph with one maintaining condition is shown in Figure 3. For this client the escape condition results in higher observed levels of aberrant behavior was observed when compared to all other conditions.

Multiple functions: If two conditions showed higher levels of aberrant behavior across all conditions, both were recorded as a maintaining function. The condition with higher levels of behaviors (i.e., estimation of approximate percentage) was determined to be the primary and the other as secondary functions. If high levels of aberrant behavior were observed across all conditions a maintaining function of automatic reinforcement was scored.

Aberrant behavior not observed: For clients in which no or low frequency of aberrant behavior was observed, the assessments were scored as inconclusive.

Estimation of percentage of aberrant behavior across conditions: Through visual inspection, the researcher estimated the percentage of behavior observed across sessions identified as a maintaining condition. In Figure 3, the maintaining function was identified as escape with the approximation percentage of aberrant behavior of 38%.

Preference assessments: Prior to interpreting the preference assessment graph, all of the items used during the assessment were recorded on the data sheet. Preference assessment bar graphs were then visually inspected. The item chosen most often were coded as HP, item chosen closest to the mean was coded as MP, and item chosen least often as LP. Choices were recorded on the data collection sheet. In Figure 1, the HP, MP and LP items were recorded as, balloon, seal, and square.

Incomplete or no preference assessment graph: An incomplete preference assessment was scored when no clear preference was obtained or a graph was not available for visual inspection.

Incomplete or no MAS: Line graphs for reinforce assessments for each client was visually inspected. Each pairing of items was viewed as three distinct assessments (i.e., HP versus MP, HP versus LP, and MP versus LP). For data interpretation high, middle, or low was recorded for the item that was hypothesized to be more reinforcing. Graphs were interpreted by reviewing the number of times reinforce was chosen over the paired item. The item chosen more often wasdetermined to be more reinforcing than the other choice item. Figure 2 shows a typical line graph used for a reinforce assessment. This example was score as the HP being more reinforcing then the LP, HP over MP, and MP over LP, respectively.

Inconclusive and incomplete reinforce assessments: Assessments that no clear choice was determined by visual inspection, incomplete number of sessions, or assessment not attempted were recorded as incomplete. For example, when HP versus LP sessions was conducted and the two items were chosen an equal amount of times or the graph did not clearly show a preferred choice.

Treatment packages: Each client chart was reviewed for treatment assessment graphs. If no graph was available, data were scored as no graph. Visual interpretation of treatment effectiveness was completed by visually approximating the frequency of aberrant behavior during treatment sessions. To allow for extinction bursts and acquiring new skills, the last three data points were used to approximate percentages of aberrant behavior following treatment implementation. Thus, the data points for session 5, 6, and 7 were used in Figure 4 to estimate percentage of aberrant behavior following treatment for this client. The client here was estimated to have approximately 3% of aberrant behaviors across these three treatment sessions.

For some clients, the last three sessions of treatment occurred within a generalization phase; that is changes may have occurred for therapist, settings, an increased delay for reinforce delivery may have been initiated. For these clients, the last three data points prior to the change in treatment parameters were used for estimating percentage of aberrant behavior.

Phase 2: Assessment comparisons

MAS and experimental analysis comparison

Inclusion and exclusion criteria: For inclusion in this comparison, the MAS and experimental analysis had to be completed, and behaviors topography for each assessment the same. For example; a MAS was completed for physical aggression and aggression was assessed the experimental analysis. If one of the assessments was not completed or the definitions did not match, the client was not included in the comparison.

The function that scored 1 in the MAS was compared against the function identified during the experimental analysis. If the condition identified in the MAS matched the function identified during the experimental analysis, an agreement was scored. If the outcomes did not match, a disagreement was scored. To obtain the percentage of clients included in the study the total number of completed MAS’s was divided by 118 and multiplied by 100. To obtain the percentage, of agreements the total number of agreements was divided by the total number of cases reviewed and multiplied by 100.

Multiple functions: If multiple functions were identified, two conditions receiving ratings of 1 via the MAS and the FA or CA, both functions would be compared. To be considered an agreement, one or both of the identified functions in the MAS had to match an identified function in the experimental analysis. Only one agreement or disagreement was scored for each case file reviewed.

Preference assessment and reinforce assessment match

Inclusion and exclusion criteria: To be included in the preference versus reinforcement comparison: a) both assessments needed to be completed and b) identical items needed to be used for both assessments. For example, if the preference assessment identified balloon as the HP, seal as the MP, and square toy as the LP the reinforce assessment needed to use balloon as HP, seal as MP, and square toy as LP. Exclusion criteria from the study were: a) the preference assessment or reinforce assessment was not completed, b) if the preference items identified did not match the items used in the reinforce assessment and c) if the outcome of the reinforce assessment was scored as incomplete or inconclusive.

Using the data sheets completed during Phase 1, the reviewer compared preference assessment outcomes and reinforce assessment outcomes. To obtain percentage of cases included in the study the total number of cases compared was divided by 118 and multiplied by 100.

The outcomes of each of the comparisons (i.e., HP versus MP, HP versus LP, and MP versus LP), were compared against the outcomes of the preference assessment. For an agreement to be scored the HP had to be chosen more often in the reinforce assessment. Items identified as HP, MP, and LP was compared against items that were chosen most often during the reinforce assessment. Each pairing for the RA was compared against the outcomes of the PA. This provided for three separate data percentages for matching HP versus MP, HP versus LP, and MP versus LP.

Treatment Effectiveness

Inclusion and exclusion criteria: Reviewers excluded any cases that did not have both experimental data and treatment data. Using the data recorded during Phase 1, the reviewer compared percentages of aberrant behaviors the FA condition with the highest percentage of problem behavior to the percentage of problem behaviors observed during treatment, (TX). To determine percentage of change, the TX percent interval was subtracted from percent interval of FA baseline and divided by FA percentage; This outcome was then divided by the FA baseline percent interval. This outcome was then multiplied by 100. This resulted in the following equation [(FA–TX)⁄FA) x100)]. A highly effective outcome was defined as a decrease in problem behavior greater than 80%. Moderately effective outcome was defined as a 50% to 79% decrease in aberrant behavior. Not effective was defined as less than 50% decrease in problem behavior. For example, the client in Figure 3. Percent interval aberrant behavior was estimated at 38% within the FA. Conversely, the estimated percent interval of aberrant behavior in treatment was 0%. The calculation to determine percent effectiveness was; (38%–0%) ÷ 38% = 100%) or 100% reduction in problem behavior. or this client the treatment was highly effective.

Interobserver agreement

A second investigator familiar with the assessments and visual inspection of graphs, reviewed a random sampling of 36% of the clients reviewed (n = 43). For an agreement to be scored both the primary and second investigator had to agree on the outcome. An agreement was also scored if both researchers determined that the assessment was not completed. Agreement scores were calculated by dividing agreements by agreements plus disagreements and multiplying by 100%.

For the treatment effectiveness portion of the study a range was developed, which would provide a slight margin as each investigator may estimate the behavior levels observed during experimental analysis and treatment to be at slightly different percentages. For example: the graph in Figure 3 shows an escape function, which the primary researcher estimated the aberrant behavior to be 38%, and reliability researcher estimated aberrant behavior to be 31%. Both primary and reliability researchers estimated the aberrant behavior in treatment to be 0% (Figure 4). Researchers agreed that the treatment was highly effective at reducing aberrant behaviors. Agreement scores were calculated by dividing agreements by agreements plus disagreements and multiplying by 100%.

Results

There were 118 cases reviewed for the study diagnosis the most typical diagnosis conditions from least to most are as follows: no previous diagnosis 13% (n=15), developmental disability 14% (n=16), ADD⁄ADHD 21% (n=25), and autism 37% (n=44).

MAS and functional assessment match

Of the cases reviewed, 81% (n=96) met the inclusion criteria for the study. The MAS identified the same function for 42% (n=40) of the comparisons made. Percentage was obtained by dividing the total number of matches by the total number of MAS’s completed. Table 1 shows the number of times each function was identified through the MAS; and the number of times the function identified through experimental analysis matched the MAS. The MAS and experimental analysis agreed across functions, from most to least; tangible 26%, escape 11%, sensory 4%, and attention 0%. Inter–observer reliability was 95% (n=43).

Table 1: MAS and experimental analysis by function.



  
    Function
    MAS
    FA Match
    % Matched
  
  
    Escape
    18
    11
    11%
  
  
    Tangible
    54
    25
    26%
  
  
    Sensory
    18
    4
    4%
  
  
    Attention
    6
    0
    0%



Table 1: MAS and experimental analysis by function.

Preference assessment – reinforcement assessment match

Highly preferred item was chosen 81% (n=85) more often than the MP and LP combined. Percentage obtained by dividing the total number of cases for each comparison (i.e., HP versus. MP, and HP versus. LP) into the total number of times the HP was chosen, when HP was an option. Inconclusive or incomplete assessments were not used in the number of cases reviewed.

Highly preferred versus middle preferred

The HP versus MP comparisons were made for 69% (n=82) of the cases reviewed (Table 2). HP was chosen 78% of the time when paired with the MP item and MP was chosen 22% of the time (Figure 5). Figure 5 depicts that the HP was chosen more often than MP. Not included in the percentages were any inconclusive, incomplete, or assessments that did not use the same items identified during preference assessment. Inter–observer reliability was 91% (n=43).

Table 2: Reinforce assessment outcomes by times chosen.



  
    Pairing
    HP
    MP
    LP
    Inconclusive/Incomplete
  
  
    HP versus MP
    64
    18
     
    36
  
  
    HP versus LP
    72
     
    13
    33
  
  
    MP versus LP
     
    69
    1
    38



Table 2:  Reinforce assessment outcomes by times chosen.

Figure 5: Percentage of times highly preferred items was chosen over the middle preferred items during the reinforcement assessment (n=82).



Figure 5: Percentage of times highly preferred items was chosen over the middle preferred items during the reinforcement assessment (n=82).

Highly preferred versus low preferred

Outcomes were reviewed for 72% (n=85) of cases (Table 2) (Figure 6). When HP and LP were paired the HP item was chosen 85% of the time and LP was chosen 15% of the time. An inter–observer agreement was 88% (n=43).

Figure 6: Percentage of times highly preferred items were chosen over low preferred items during reinforcement assessments (n=85).



Figure 6: Percentage of times highly preferred items were chosen over low preferred items during reinforcement assessments (n=85).

Middle preferred versus low preferred

There were 80 (67%) files reviewed for MP versus LP pairings. The MP was chosen 86% of the time and LP was chosen 14% of the time (Table 2) (Figure 7). Inter–observer reliability was 88% (n=43).

Figure 7: Percentage of times middle preferred chosen over low preferred items during reinforcement assessments (n-80).



Figure 7: Percentage of times middle preferred chosen over low preferred items during reinforcement assessments (n-80).

Treatment effectiveness

Treatment was completed in 74% (n=87) of the clients. Of those clients, the effectiveness of the treatment package at reducing identified behaviors was: 74% (n=64) highly effective, 18% (n=16) minimally effective, and 8% (n =7) not effective. For 26% (n =31) of cases no treatment was attempted. Table 4 shows the number of cases for each level of effectiveness. Percentages were obtained by dividing the number of cases for each level by the total number of cases that treatment was completed. Inter–observer reliability was 86%.

Functions of behaviors

Experimental analysis was conducted for 97% (n=102) of the cases. Table 3 shows that secondary functions were identified in 48% (n=49) of the cases. No maintaining function could be identified in 15% (n=15) of cases. Maintaining primary functions were from most to least: escape 47%, tangible 36%, attention 8%, and sensory 4%. Secondary functions were identified from most to least: tangible 47% escape 37%, attention 14%, and sensory 2%. Overall identified functions were from most to least: escape 84%, tangible 47%, attention 22%, sensory 6%, and no function identified 15%. Table 3 shows the number of times each function was identified as a maintaining condition.

Table 3: Primary and secondary functions identified through experimental analysis.



  
    Function
    Primary
    Secondary
    Total
  
  
    Escape
    56 (47%)
    23 (37%)
    79 (84%)
  
  
    Tangible
    36 (30%)
    18 (47%
    54 (77%)
  
  
    Attention
    8 (8%)
    7 (14%)
    15 (22%)
  
  
    Sensory
    4 (4%)
    1(1%)
    5 (5%)
  
  
    None Identified
    15 (15%)
     
    15 (15%)



Table 3:  Primary and secondary functions identified through experimental analysis.

Table 4: Treatment efficacy by reduction of aberrant behavior.



  
    Efficacy
    Percentage of Cases
  
  
    Highly
    74% (n =64)
  
  
    Moderately
    18% (n=16)
  
  
    Not
    8% (n=7)
  
  
    No Treatment*
    26% (n=31)



Table 4:  Treatment efficacy by reduction of aberrant behavior.

Reliability

Reliability of measurement was completed for 36% of the cases inspected (n=43). RA reliability for HP versus MP was 91%, HP versus LP 88%, MP versus LP 88%. For MAS agreement reliability was 100%, Experimental analysis reliability was 100% for primary functions and 95% for secondary functions. Treatment efficacy agreement was 86%.

Discussion

The study examined 118 client files evaluated over a ten–year period (1999–2009). We set out to answer specific questions about the efficacy of each processes used to identify and treat problem behaviors. Specifically we wanted to know if the MAS would identify the same function as the experimental functional analysis. We found that similar outcomes were obtained across the MAS and the FA for only 42% of the clients. The second question we sought to answer was if the preference assessment would identify items that would serve as reinforces. The data shows that of the pairings made between the HP, MP, and LP; the HP was chosen 81% of the time. The final question that we wanted to answer was how effective was the treatment at reducing aberrant behaviors. We found that for 81% of the cases aberrant behaviors were reduced 50% or more when compared to baseline conditions. Treatment was found to be highly effective, resulting in the reduction of more than 80% of aberrant behavior (n=80) for 74% (n=64). Of the clients that received treatment.

This study provides an epidemiological analysis of PA, RA, and FA procedures for a large sample size of clients across a broad range of diagnostic conditions and aberrant behaviors. The largest diagnostic group was autism, with 37% of clients evaluated. There was a small group of clients, 13%, which had no diagnosis at the time of their evaluation.

The study replicated findings of other outpatient clinics [4] that use brief functional analysis procedures to develop function based treatment programs. That is, a function was identified for a similar number of the clients evaluated. The study provides documentation on the efficacy of the FA, PA, RA, and TX conducted in an outpatient setting for a relatively large sample size.

A limitation of the study was that the client data were not separated between clinic and natural settings. This delineation would have provided additional data on the efficacy of the procedures in the natural setting compared to clinic–based evaluations. Some questions raised in the investigation are: was there a better percentage rate of MAS and FA identifying the same function when the FA was conducted in the natural setting? What was the percentage of attention maintained behaviors in the natural setting compared to the clinical setting? Another limitation was not all clients seen at the clinic were included in the study; there were clients that did not attend all scheduled sessions and other assessment procedures were utilized (i.e. structural analysis, antecedent analysis). Lastly, the use of visual inspection may have lead to a slight margin of error in data interpretation, although reliability was obtained. To decrease the margin of error, raw data could have been examined instead of using visual inspection of completed graphs.

We were surprised to see such a small number of clients with attention maintained behavior. This may have been due to several factors. First, the clients were primarily referred through the school system and, in most cases, had not been successfully treated in the community through typical community resources. It is possible that previous interventions had utilized a non–function based timeout procedure; thus, teaching the client to use problem behavior to escape non–preferred activities. Conversely, the use of time–out for caregiver attention within previous treatment attempts would have reduced the occurrence of problem behavior prior to being seen in the clinic. These two possibilities would result in: a) an increased level of problem behavior in the escape condition in the clinic, and b) a reduced number of children being referred whose behavior is maintained by attention. Alternatively, the novelty of the clinic processes in itself may have provided attention to the client. Thus, abolishing the establishing operation for attention maintained behavior.

Future research may further expand the study by conducting maintenance probes after the clinic sessions have ended. Asking care givers about the acceptability of the treatment packages developed (e.g., what is the likelihood of the care giver continuing the treatment package after leaving the clinic) could also increase the utility of future investigations. We found the procedures used in the clinic to be effective at: (a) identifying items to serve as reinforces, (b) identification of social conditions that maintain problem behaviors and (c) that the function based treatment packages reduced problem behaviors.

References

Download PDF

Citation: Conn M, Derby KM, McLaughlin TF. Review of a Behavioral Assessment Process from Descriptive Assessment to Experimental Analysis: A Summary of 118 Cases. Austin J Psychiatry Behav Sci. 2014;1(2): 1010. ISSN: 2381-9006.

Instruction for Authors

Submit Your Article

Function	MAS	FA Match	% Matched
Escape	18	11	11%
Tangible	54	25	26%
Sensory	18	4	4%
Attention	6	0	0%

Pairing	HP	MP	LP	Inconclusive/Incomplete
HP versus MP	64	18		36
HP versus LP	72		13	33
MP versus LP		69	1	38