All event-based IOA algorithms have in common the analysis of concordance on frequency counts and event records. These measures consist of (a) total number, b) partial concordance in intervals, c) exact match, and (d) DOA trial by trial algorithms. After a brief overview of each event-based algorithm, Table 1 summarizes the strengths of the four event-based algorithms for behavioral reliability analysis considerations. Suppose a research team collects frequency data for a target response of 15 observations of 1 m (see Figure 1). This technical report provides detailed information about the reasons for using a common computer calculation program (Microsoft Excel®) to calculate different forms of interobserver agreement for continuous and discontinuous data series. We also offer a brief tutorial for using an Excel table to automatically calculate the traditional total number, partial match in intervals, exact match, sample for trial version, interval, point interval, non-disassembly interval, total duration and average duration per interval of Interobserver tuning algorithms. We conclude with a discussion on how practitioners can integrate this tool into their clinical work. IOA at the point interval. One approach to improving the accuracy of two observers` compliance with interval recording is simply to limit compliance testing to cases where at least one of the observers recorded a target response at an interval. Intervals during which none of the observers reported a target response are excluded from the calculation in order to obtain stricter concordance statistics. Cooper et al. (2007) suggest that the IOA is most advantageous in the point range (also referred to as the “presence agreement” in the research literature) when target responses are low. In the data examples in Figure 2, the second, third and fourth intervals are ignored for calculation purposes because none of the controls responded at these intervals.
Therefore, IOA statistics are calculated only from the first, fifth, sixth and seventh intervals. As there was only one agreement on half of the intervals (the fifth and sixth intervals), the concordance is 50% (2/4). The idea that practicing behavioral analysts should collect and report reliability or interobserver compliance (IOA) in behavioral assessments is illustrated by the Behavior Analyst Certification Board`s (BACB) assertion that behavioral analysts are proficient in using “different methods of evaluating the results of measurement methods, such as.B. compliance between observers, accuracy and reliability” (BACB, 2005). In addition, Vollmer, Sloman and St. Peter Pipkin (2008) argue that the exclusion of these data significantly limits the interpretation of the effectiveness of a behaviour modification procedure. Therefore, the inclusion of reliability data for validity claims in any study that includes behavioural assessment should be the inclusion of reliability data (Friman, 2009). Given these considerations, it is not surprising that a recent review of journaled journals in Applied Behavior Analysis (JABA) from 1995 to 2005 (Mudford, Taylor & Martin, 2009) found that 100% of articles reporting continuously recorded dependent variables contained IOA calculations. These data, along with previously published reports on JABA`s reliability procedures (Kelly, 1977), indicate that the inclusion of IOA is in fact a trademark – if not a standard – of behavioural assessment. Three broad categories of reliability metrics will be discussed in the remaining part of this article: a) event-based, b) interval-based, and (c) time-based. Event-based measurements can be considered as any form of IOA, based on data collected using event records or frequency censuses during observations. Interval-based measurements come from data collected as part of interval records (e.g.B partial or full readings) or as part of a time sampling method.
Finally, duration-based algorithms are used when data is derived from timings (e.g. .B. latencies, duration, inter-response periods). . . .