Talk:Censorship Monitoring Project Functional Spec

Minimum viable product

Immediate functional aims of the project from ORG's point of view (as discussed on the org-tech-vols mailing list

On Sun, Dec 29, 2013 at 07:02:45PM +0000, Jim Killock wrote:

Our immediate aim was to allow anyone to submit a 'check' request, and then for us to make the check on as many networks as possible. Then, over time, to build a list of domains or URLs that had been mis-categorised. Some we think are prone to regular mis-categorisation. This takes [1] on a step - at the moment people can report errors and problems, but we have been manually verifying them. The process is currently:

  1. We receive a report
  2. We manually check O2 / EE / Vodafone etc using PAYG mobile phones
  3. That's it, no further checking as the data isn't in a DB

It could be:

  1. We receive a report
  2. We use probes to check O2 / EE / Vodafone / fixed line ISPs
  3. We report the results to the person who asked;
  4. We report problems to the affected ISPs or mobile networks
  5. We publish the results
  6. We recheck on a regular basis

On Sat, Jan 04, 2014 at 12:47:03PM +0100, Richard (talk) wrote:

The project has grown many interesting tentacles that could perform step (2) of this process automatically, but for an MVP, perhaps all we really need is a few features to wrap this process for "users" (i.e. blocked.org.uk visitors):

  • A list of URLs waiting to be checked
  • A way of recording who asked for a URL to be checked so that we can send them the result (we'd keep this private).
  • A mechanism for sending results to people who have registered interest in a particular URL.
  • A published list of checked URLs, the results on each network, and when they were last checked.

As we improve the system, the solution for checking whether a URL is blocked might move from a manual process, through a mechanical turk system and manually configured probes, to a fully-automated botnet and who knows what else. We'll still need these other parts of the process to work, though, and if we build them first we can start delivering more value to blocked.org.uk users sooner (IMO).