Delivery-Date: Tue, 05 Aug 2014 15:09:42 -0400
Return-Path: <tor-talk-bounces@lists.torproject.org>
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on moria.seul.org
X-Spam-Level: 
X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED,
	RP_MATCHES_RCVD autolearn=ham version=3.3.1
X-Original-To: archiver@seul.org
Delivered-To: archiver@seul.org
Received: from eugeni.torproject.org (eugeni.torproject.org [38.229.72.13])
	(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))
	(No client certificate requested)
	by khazad-dum.seul.org (Postfix) with ESMTPS id A98F51E0CC0;
	Tue,  5 Aug 2014 15:09:40 -0400 (EDT)
Received: from eugeni.torproject.org (localhost [127.0.0.1])
	by eugeni.torproject.org (Postfix) with ESMTP id 959012F6FC;
	Tue,  5 Aug 2014 19:09:37 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by eugeni.torproject.org (Postfix) with ESMTP id 23B7D2F6EA
 for <tor-talk@lists.torproject.org>; Tue,  5 Aug 2014 19:09:34 +0000 (UTC)
X-Virus-Scanned: Debian amavisd-new at eugeni.torproject.org
Received: from eugeni.torproject.org ([127.0.0.1])
 by localhost (eugeni.torproject.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id V7Pn4Tvbr0rJ for <tor-talk@lists.torproject.org>;
 Tue,  5 Aug 2014 19:09:34 +0000 (UTC)
Received: from necrid.teamhugs.is (teamhugs.is [93.95.226.146])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client CN "*.teamhugs.is", Issuer "*.teamhugs.is" (not verified))
 by eugeni.torproject.org (Postfix) with ESMTPS id E72722F642
 for <tor-talk@lists.torproject.org>; Tue,  5 Aug 2014 19:09:33 +0000 (UTC)
Received: from kilik (teamhugs.is [93.95.226.146])
 by necrid.teamhugs.is (Postfix) with ESMTPA id 2ECC41EC345
 for <tor-talk@lists.torproject.org>; Tue,  5 Aug 2014 19:09:30 +0000 (UTC)
Received: from [IPv6:::1] (localhost [IPv6:::1])
 by kilik (Postfix) with ESMTPS id A427DCC106C
 for <tor-talk@lists.torproject.org>; Tue,  5 Aug 2014 15:09:29 -0400 (EDT)
Message-ID: <53E12BE9.5020207@torproject.is>
Date: Tue, 05 Aug 2014 15:09:29 -0400
From: Andrew Lewman <andrew@torproject.is>
Organization: The Tor Project
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:24.0) Gecko/20100101 Icedove/24.7.0
MIME-Version: 1.0
To: tor-talk@lists.torproject.org
References: <20140805143135.GM1414@ix.home> <20140805175837.GN1414@ix.home>
In-Reply-To: <20140805175837.GN1414@ix.home>
X-Enigmail-Version: 1.6
OpenPGP: url=http://wiki.lewman.is/files/0x6B4D6475.asc
Subject: Re: [tor-talk] dutch police crawling hidden servers
X-BeenThere: tor-talk@lists.torproject.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: tor-talk@lists.torproject.org
List-Id: "all discussion about theory, design,
 and development of Onion Routing" <tor-talk.lists.torproject.org>
List-Unsubscribe: <https://lists.torproject.org/cgi-bin/mailman/options/tor-talk>, 
 <mailto:tor-talk-request@lists.torproject.org?subject=unsubscribe>
List-Archive: <http://lists.torproject.org/pipermail/tor-talk/>
List-Post: <mailto:tor-talk@lists.torproject.org>
List-Help: <mailto:tor-talk-request@lists.torproject.org?subject=help>
List-Subscribe: <https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk>, 
 <mailto:tor-talk-request@lists.torproject.org?subject=subscribe>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: tor-talk-bounces@lists.torproject.org
Sender: "tor-talk" <tor-talk-bounces@lists.torproject.org>

On 08/05/2014 01:58 PM, Rejo Zenger wrote:
> How would they have done that? Of course, there are wiki's listing 
> hidden services, but they are most likely far from complete and I 
> wouldn't expect websites with hardcore child abuse to be "advertised" 
> there. So, what do you think this crawler did?

Unfortunately, the hardcore sites are/were advertised on one of the 20+
hidden wikis which exist at any time. The press and police only seem to
find these sites and assume everything is just this set. There are
seriously 20+ hidden wikis, each one claiming to be the
original/canonical wiki.

Given the resources of a national police force, it seems probable they
can create a crawler to simply crawl every permutation of hidden service
addresses on port 80 alone.

Of course, it's easier to crawl the 20+ hidden wikis and go from there.

-- 
Andrew
pgp 0x6B4D6475
https://www.torproject.org/
+1-781-948-1982
-- 
tor-talk mailing list - tor-talk@lists.torproject.org
To unsubscribe or change other settings go to
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk

