Listening to the most recent episode of ATP, I learned a surprising fact about how Apple developers see their user statistics: the number of people who opt out of sharing is not available, and the only statistics developers get is about the users who opted in. This may make naïve sense — hey, they don’t want you to know about them so we’ll erase them from existence Soviet-style — but is in fact statistical malpractice. The numbers Apple does share are only a sample of the total user base, which is fine as long as you know or can estimate the size of the population you are sampling. Without a way to estimate the denominator, the numbers are meaningless.
The bare minimum that Apple could and should do, without breaking any privacy rules, is to share the number of people who opted out. With this number in hand, a developer would be able to project worst-case scenarios and likely ranges for each statistic of interest. If Apple dedicated just a tiny bit more resources to developer relations, they could automate this step and build in worst-case numbers into the interface. A more sophisticated company that cares about the developer ecosystem could even create complex predictions models that would give both worst-case numbers and the 95% confidence intervals for each statistic in question.
Why should Apple care, other than doing a solid to their developer ecosystem with a minimal investment? Well, by making their default user statistics useless they are in fact incentivizing developers large and small to track their users by other means. Large developers may do that anyway since they are likely to have a could component to their app. But an indie developer may turn to third party services: if I am a teenager working on my first big app I will probably turn to the fastest and cheapest way to track, and not knowing anything about anything I am going to guess some of the offerings in that part of the price-value spectrum are going to be less than scrupulous.
So this is how Apple’s privacy-minded view combined with thriftiness and lack of care towards developer experience can lead directly to worse overall privacy for their users. What a surprise.