Jonathan Stray @jonathanstray.com · 3d In this paper we suggest holdouts as an important transparency measure. We specifically discuss the possibility of changes that would be unethical to withhold. But we still want to know if proposed changes really work, or are just safety theatre! nyaspubs.onlinelibrary.wiley.com/doi/abs/10.1... 3/
0 likes 1 replies
?
Replies
TikTok has since reduced holdout size (10%->1%) and duration (6 months->7 days). But this doesn't solve the core problem. One potential solution: statistical protocols from medical research where you terminate one arm of the protocol when it becomes obvious that the other one is much better. /fin