The Northern Ireland group held a meeting at 3pm on Tuesday the 3rd of December, 2013 in the Peter Froggatt Centre in the Queen’s University of Belfast with speaker Professor Robin Henderson of the University of Newcastle, UK. This was another look at explained variation in survival analysis.
Robin noted that Applications of Statistics to Medicine appeared in the top 11 contributions to Medicine over the millennium (NEJM, 2001) and survival analysis had been a major component of this success. However, despite this progress, explained variation remained, as he was to show, an open question.
The first part of the talk introduced Exponential, Weibull and Cox’s Proportional Hazards survival regression models and emphasised the notion of prediction based on a given set of covariates. He contrasted individual prediction with group prediction and compared the predictions of doctors with those made on the basis of statistical models. He gave several examples which showed how poor individual predictions could be even when group prediction (based on Kaplan-Meier curves) was reasonable. He challenged the audience to state their preferences for: (a) doctors’ predictions or (b) those based on a statistical model. In the examples studied there was hardly any difference - both methods of prediction, as measured by Parkes’ error proportion, were poor: the median proportion for the hospice data set was 63% and for the lung cancer data set was 53%.
The need for measures of explained variation was clear and there was no shortage of these; Robin showed a slide with 40 such measures. He traced the history of Schemper’s measures which Henderson had claimed were most promising in a 1995 paper (SIM, 1995). However, Graf and Schumacher showed that the censoring was not handled properly (RSS, Series D, 1995) and it was not until some years later (Graf, et al 1999, Schemper and Henderson, 2000) that this problem was solved by using inverse probability weights. Once again the percentages of explained variation were not high in the various examples shown.
Robin argued that the problem was fundamental to survival analysis. He showed a sequence of graphs in which relatively large hazard ratios implying high statistical significance of the hypothetical covariate involved led to quite modest values of the percentage of variation explained. The situation was analogous to risk factor evaluation in Epidemiology where it is well known that high relative risks do not lead to accurate individual point predictions of the risk of developing a disease. The difficulties were compounded in survival analysis due to the shapes of the distributions involved.
He went on to review uses of the C-index and Nagelkerke’s R2, illustrating a weakness inherent in the latter statistic, which was the most popular measure in use as measured by citation count. Next he proposed a new measure based on ranks, namely, RE and illustrated its computation. The new statistic repaired the problem inherent in Nagelkerke’s R2. Thereafter, Robin illustrated its use in an event history context and suggested some avenues for additional work in this area.
This challenging presentation led to several questions and an interesting discussion.