ORG-tech-vols IRC meeting log 2014 03 13

(19:28:52) graphiclunarkid: Evening all - how's it going?
(19:29:01) Allll: hi
(19:29:24) Dantheta: hi! doing well, thanks
(19:29:39) graphiclunarkid: Good good :)
(19:30:17) korikisulda [59f1938d@gateway/web/freenode/ip.89.241.147.141] entered the room.
(19:30:50) vasilis: Hi everyone, good.
(19:31:05) graphiclunarkid: Hi vasilis, korikisulda
(19:31:16) korikisulda: Hello
(19:31:42) graphiclunarkid: Well, shall we get started?
(19:32:00) graphiclunarkid: I wrote a thing... http://blocked.org.uk/help
(19:32:08) vasilis: Let's start.
(19:32:28) graphiclunarkid: You may have noticed ORG drawing lots of attention to our project this week.
(19:32:48) korikisulda: I did not
(19:33:03) vasilis: graphiclunarkid: I just had a quick look, we should set a proper SSL certificate.
(19:33:05) graphiclunarkid: That will continue next week too as I'm giving a talk in Manchester and another in Edinburgh to introduce the project at those local groups
(19:34:17) graphiclunarkid: korikisulda: There was a blog post and a couple of emails to various lists. e.g. https://www.openrightsgroup.org/blog/2014/support-orgs-censorship-monitoring-project
(19:35:00) graphiclunarkid: vasilis: Yes, we should sort out SSL in time for the revised site to go live.
(19:35:27) graphiclunarkid: I wrote http://blocked.org.uk/help because I wanted to make it easy for new people to join the project in the light of all the publicity.
(19:35:44) graphiclunarkid: If anyone has any comments or suggestions feel free to let me know :)
(19:36:00) vasilis: graphiclunarkid: Good work! ;)
(19:36:08) graphiclunarkid: vasilis: I will talk to Lee about ordering a certificate from wherever they usually get them.
(19:36:10) Allll: looks good
(19:36:30) graphiclunarkid: vasilis, Allll: thanks :)
(19:37:39) vasilis: We can even get a free SSL certificate from startssl.
(19:37:39) graphiclunarkid: Over this week and next I'll mainly be focusing on writing my presentation - thanks again for the slides vasilis!
(19:37:56) vasilis: I hope that were helpful.
(19:38:20) graphiclunarkid: vasilis: I know ORG have bought certificates in the past - https://www.dontspyonus.org.uk/org has a proper one, for example.
(19:38:49) graphiclunarkid: It will probably be easiest for them to acquire a certificate for blocked.org.uk from the same source to keep management simple.
(19:39:07) vasilis: OK
(19:39:33) vasilis: Regarding funding you can have a look at http://webwewant.org/
(19:39:54) vasilis: They offer some mini-grants.
(19:40:52) graphiclunarkid: vasilis: The slides are helpful, thanks. I'll probably use a few of the OONI ones.
(19:42:43) graphiclunarkid: vasilis: Interesting on the mini-grants, thanks. Could be enough to cover purchase of the hardware needed to do mobile monitoring (probably not the network subscriptions too though). I will pass that on to ORG-central and see if they think it's worth pursuing.
(19:44:38) Dantheta: just catching up, sorry ...
(19:44:41) graphiclunarkid: Allll: I prompted the ORG staff to review your website template and I think Ruth raised a few issues.
(19:45:04) Allll: She did, I should have them all done by the end of the weekend
(19:45:15) graphiclunarkid: Coolio
(19:45:19) Allll: do you know if that was THE list or might there be more to come?
(19:45:38) graphiclunarkid: Allll: Nobody else has commented to me. I can prompt them again but I'm not sure how much longer you want to wait for comments before we do the MODx implementation bit.
(19:46:20) Allll: I'll start ModX next week, I'm hoping to put a lot more time in next week, and I doubt there'll be any changes that are too tricky to implement in ModX anyway
(19:46:30) graphiclunarkid: I would be tempted to roll it out but allow for the possibility we'll have to make changes subsequently.
(19:46:37) graphiclunarkid: Heh, yeah, what you said.
(19:46:42) Allll: cool
(19:47:40) graphiclunarkid: The advantage of having the initial version up on the test server is that it'll then be possible to start linking the site to the middleware and running some integration tests.
(19:48:03) Allll: yeah, makes sense
(19:48:25) graphiclunarkid: We can also start asking people to contribute content via the MODx interface.
(19:48:49) Allll: cool, I'll make that a priority
(19:49:03) graphiclunarkid: Awesome :)
(19:49:54) graphiclunarkid: Dantheta, korikisulda, how close are we to having the latest API ready to start firing requests at? Would it be possible to massage it into place by the end of next week so we can start linking the two together the week after, do you think?
(19:51:01) Dantheta: possible. I'm working on getting the queuing to work with multiple isps atm.
(19:51:18) graphiclunarkid: Interesting!
(19:51:34) Dantheta: I haven't heard back about access to the org servers though.
(19:51:55) graphiclunarkid: Really? Right - I'll sort that out tomorrow. Lee is working so I should be able to pester him.
(19:52:21) Dantheta: that would be cool.
(19:52:59) graphiclunarkid: I'd be interested to hear how the queueing is going to work. Is there a wiki page I should read or are you still figuring it out yourself? :)
(19:53:00) Dantheta: many thanks!
(19:54:10) Dantheta: Still figuring it out, although it requires the api clients to lookup the isp that they're connected to (using an endpoint we provide)
(19:55:05) Dantheta: it's possible to do a medium-scalable but good-enough-for-now implementation just using the db.
(19:55:06) graphiclunarkid: Hmm. Are we going for a "test once per ISP regardless of the number of probes" approach or will it be more like "each probe tests each URL once regardless of how many probes we have connected to the same ISP?"
(19:55:56) Dantheta: i was thinking the former, but we can cycle round the list if we have probe capacity to spare on a given isp.
(19:57:26) Dantheta: by the time we have >50 probes we should probably look at message queues to distribute requests.
(19:57:46) Dantheta: but in the meantime mysql managed queues should be fine.
(19:58:24) graphiclunarkid: How late can we decide which approach is best before it would require awkward changes to switch between them?
(19:59:07) graphiclunarkid: I'll be honest, I've no idea which will give us better data, but I'm a bit nervous about relying on a single test to draw any conclusions. Especially from volunteer-run probes.
(19:59:22) Dantheta: I don't think there will be awkwardness. the api stays the same but the implementation changes
(19:59:58) graphiclunarkid: OK - it's basically the bit of the system that manages the queue we're talking about, right?
(20:00:04) Dantheta: we can also change the queue strategy without changing the api endpoints.
(20:00:14) Dantheta: yep, that's right.
(20:00:21) graphiclunarkid: OK cool.
(20:02:12) Dantheta: I think i'm almost done with things that break backwards compat with the db and api-so-far
(20:02:37) Dantheta: korikisulda: did you see the /status/ip endpoint that got added?
(20:02:51) korikisulda: Yeah, sorry
(20:02:55) graphiclunarkid: (Just to finish the last point - A discussion on queueing strategy will be more meaningful when we have some actual data anyway - so let's go get it :) )
(20:02:59) korikisulda: I've been having.. Other.. problems
(20:03:19) Dantheta: oh dear!
(20:05:11) graphiclunarkid: korikisulda: Anything we can help with - or is it a Java thing? ;-)
(20:05:31) korikisulda: No, it's not technical at all, or in any way related to this
(20:06:26) graphiclunarkid: korikisulda: OK - well, life happens. Hope you manage to sort it :)
(20:06:58) korikisulda: Unlikely *sigh* Oh well. When I stop being lazy, I'll pull the latest version of the middleware and play around with it
(20:07:34) Dantheta: if there's anything I can help with, give me a shout.
(20:08:10) korikisulda: Partly, I just keep putting it off because it's annoying to have to keep cloning the repo ^.^
(20:08:24) korikisulda: The sooner we have it set up properly, the better..
(20:09:03) graphiclunarkid: korikisulda: well if I can get an account on the server for Dantheta tomorrow hopefully we can get that bit working...
(20:09:08) Dantheta: i am putting stuff into the main repo now, rather than the fork on my personal github
(20:09:13) korikisulda: That'd be good
(20:09:22) korikisulda: Yeah, I didn't notice that for a day or two ^.^
(20:10:05) graphiclunarkid: vasilis: Did you see we might have another volunteer prepared to run a Raspberry Pi probe for you? I CC'd you into my reply to them - hope that was OK?
(20:10:11) Dantheta: although I've been using git for a while, i'm still quite new to github
(20:10:36) vasilis: graphiclunarkid: Yes that's nice, thank you!
(20:12:15) graphiclunarkid: vasilis: I think it'd be handy if we could get a generic image available for download so you don't have to keep setting up individual downloads for people. Is that what the github repo is for? If so, should I be pointing people at that rather than sending them to you?
(20:13:08) vasilis: graphiclunarkid: This is doable but we would need to figure out how we can manage these nodes.
(20:13:54) Dantheta: sorry I haven't had a chance to plug in the pi image yet, but will do this weekend
(20:14:06) vasilis: Dantheta: Thanks!
(20:14:58) graphiclunarkid: vasilis: What kind of management would they need?
(20:15:27) vasilis: Package upgrades, a way to feed them with tests, etc..
(20:17:47) graphiclunarkid: Package upgrades will be interesting - but we're talking about a very small number of probes to start with. Maybe it's OK to say people have to pull updated images from github rather than having an automated process for now?
(20:18:38) graphiclunarkid: I was under the impression the OONI back-end stuff handled feeding the probes with tests - and that the middleware would in turn feed the OONI queue. Is that true?
(20:18:44) Dantheta: i'm a sysadmin by day - we can run a puppet/chef install for those if you need infrastructure
(20:19:13) graphiclunarkid: Dantheta: Ooh, that's a good idea.
(20:20:34) vasilis: atm we need a different img per probe, since the private key for the tor hidden service is unique.
(20:20:50) Dantheta: i don't know how much of that stuff exists in the ooni project so far, but just thought it was an idea
(20:21:35) vasilis: Up to now is the best solution I 've since we don't need to deal with firewalls, NAT and we can be assured at least that we talk to the "correct" probe.
(20:22:16) vasilis: Dantheta: puppet/chef could be an option as well but still doesn't solve this issue.
(20:23:28) graphiclunarkid: Won't all the volunteer probes be receiving essentially the same list of URLs to test? If so, why do we need to identify each probe uniquely?
(20:24:03) Dantheta: sure - give me a bit more time to read docs. Once I'm on the org server I'll see what we've got running there
(20:24:44) vasilis: graphiclunarkid: By uniquely you mean a unique private key?
(20:24:56) graphiclunarkid: Using the hidden service is nice in the event that our API endpoint gets censored, of course, but is there a way we can configure it to go directly in the first instance? We can then figure out how to get the system to generate its hidden service URL and let us know what it is automatically, but later.
(20:24:56) Dantheta: unique probe identification is to be able to disable bad probes, and for analytical purposes
(20:26:22) graphiclunarkid: vasilis: No - you said we need to make sure we're talking to "the correct probe" and I was wondering why we care which particular probe we're talking about. Dantheta's point is good - but it also means we have a way of identifying probes uniquely by their API credentials so we don't need to care about pushing data to a specific hidden service.
(20:27:09) graphiclunarkid: Dantheta: Is there a field in the probe registration service that allows probes to report their TOR hidden service URL?
(20:27:15) graphiclunarkid: And is that something we should be considering?
(20:27:46) vasilis: Well if we don't have such a unique identifier then we can't really monitor probes for failures.
(20:28:17) Dantheta: not at the moment - the ooni stuff was largely unspecified and unknown from v1.1
(20:28:32) Dantheta: i'll be able to find out more on the server
(20:29:56) vasilis: In general if something goes wrong we can "fix" it with access to the probes.
(20:30:10) graphiclunarkid: I think it might be useful if the two of you started a thread on the mailing list to discuss how we manage the OONI software itself over the API. I suggest the mailing list because there are probably other people who need to know / can help out.
(20:30:44) Dantheta: Ok, sounds good.
(20:30:47) vasilis: True but we can make pre-discussion about this.
(20:30:51) graphiclunarkid: It might be as simple as a bit of code that reports the OONI hidden service URL to an API endpoint to start with, just so we can get it without having to build the images manually.
(20:31:28) graphiclunarkid: Yeah, not suggesting we stop discussing it now, just that it should probably end up on the list at some point :)
(20:31:33) vasilis: But still do we trust the probes to report their hidden service hostname?
(20:31:46) graphiclunarkid: We're trusting them to report everything else.
(20:32:18) vasilis: At the moment this is being done manually.,,
(20:32:34) vasilis: Running the tests
(20:33:02) vasilis: Ideas?
(20:33:22) Dantheta: does ooni backend push urls to the probes, or do the probes poll?
(20:34:06) vasilis: The ooni backend can push URLs to the probes as well.
(20:36:33) vasilis: A very likely scenario: Someone plugs the img into the Pi, the probe (Pi) runs out of space and doesn't submit any results. If we can't access the probe then we should rely only to the person that have the probe to fix the issue and get the probe (Pi) back and running.
(20:36:33) Dantheta: Hmmm ... will read some docs.
(20:36:51) vasilis: and this 'll happen...
(20:38:22) graphiclunarkid: vasilis: Are you suggesting we'd reach inside their network, log on to the probe manually, and erase some stuff to make more space in that circumstance?
(20:38:24) vasilis: I agree that we can release a generic image that there would be no "remote help" in case something goes wrong.. but at the moment we don't so many requests for images.
(20:39:06) vasilis: graphiclunarkid: Yes.
(20:39:26) Dantheta: I've got some ideas. I don't think we should ever count on having access to a device thats's running the image, once we claim to be production ready.
(20:39:29) graphiclunarkid: vasilis: We don't at the moment, but if we want to publish this thing and prompt ORG supporters to install it in large numbers, I don't think it will scale.
(20:40:09) vasilis: You are both right...
(20:40:24) Dantheta: true - so we need a generic image, which auto-configures and can run for <n>weeks unnattended
(20:40:44) graphiclunarkid: vasilis: I'm also not sure people would be comfortable with us reaching inside their networks and tinkering with their machines manually. Maybe reporting problems via the API so we can contact the owner and tell them to go fix things themselves, or ask us for help, would be better.
(20:40:46) vasilis: but you should consider that a Pi probe is like a server running tests.
(20:41:37) vasilis: This is definetely for people that would like to have a managed probe.
(20:42:26) korikisulda left the room (quit: Quit: Page closed).
(20:43:42) graphiclunarkid: OK - well we need both types, as we'll obviously be managing the probes on the AAISP VMs ourselves, however I think we should start putting some thought into how we make the unmanaged probes work via the API.
(20:43:58) Dantheta: Sure.
(20:46:03) Dantheta: Time for me to read some docs!
(20:46:42) vasilis: OK I 'll work on the generic image.
(20:47:06) graphiclunarkid: Cool - let's start a thread on the list about how this might work - would be good to get some other people's opinions.
(20:47:19) vasilis: Definetely.
(20:47:23) graphiclunarkid: :)
(20:50:31) Dantheta: Ok, cool. I'm going to need to head off soon. If it's possible to get access to the org server, that'd be great.
(20:50:42) graphiclunarkid: Dantheta: Yep - I'll chase that up tomorrow.
(20:51:05) graphiclunarkid: Good chatting to you all - anyone have anything else for me or should we call it a night?
(20:52:01) Dantheta: I'm all good, I think!
(20:52:19) vasilis: ah
(20:53:25) vasilis: graphiclunarkid: Do you need any blocked reports for your mini-tour campaign?
(20:54:06) graphiclunarkid: vasilis: I don't need any - but if you have some I'll gladly show them off
(20:54:31) vasilis: OK
(20:55:41) graphiclunarkid: Cool, thanks :)
(20:55:42) vasilis: need to go, thank you all for attending.
(20:55:54) graphiclunarkid: Likewise - thanks everyone. See you next time.
(20:56:02) vasilis: Bye Dantheta, graphiclunarkid
(20:56:08) Allll: cya
(20:56:21) graphiclunarkid: Cheers Allll, Dantheta, vasilis
(20:56:29) vasilis left the room (quit: Quit: quit).
(20:56:34) Dantheta: cheers! have a good evening all!
(20:56:49) Dantheta left the room.
(20:59:21) Allll left the room (quit: ).