Inter-coder agreement

Jason Czarnezki (Empirical Legal Studies) has started an interesting discussion on Inter-coder Reliability, the degree to which multiple analysts (or the same analyst over time - i.e., intra-coder reliability) characterize data similarly.

The problem is not as simple as merely calculating the ratio of agreement within overlapping observations. I'm slogging through a lot of this myself these days, so am keenly interested in the responses ELS generates on the subject.

No comments: