Delivery-Date: Sat, 14 Jun 2014 03:56:16 -0400
Return-Path: <tor-talk-bounces@lists.torproject.org>
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on moria.seul.org
X-Spam-Level: 
X-Spam-Status: No, score=-4.8 required=5.0 tests=BAYES_00,RCVD_IN_DNSWL_MED,
	RP_MATCHES_RCVD,UNPARSEABLE_RELAY autolearn=ham version=3.3.1
X-Original-To: archiver@seul.org
Delivered-To: archiver@seul.org
Received: from eugeni.torproject.org (eugeni.torproject.org [38.229.72.13])
	(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))
	(No client certificate requested)
	by moria.seul.org (Postfix) with ESMTPS id 114251E0A2D
	for <archiver@seul.org>; Sat, 14 Jun 2014 03:56:15 -0400 (EDT)
Received: from eugeni.torproject.org (localhost [127.0.0.1])
	by eugeni.torproject.org (Postfix) with ESMTP id 1D74C2FC7B;
	Sat, 14 Jun 2014 07:56:14 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by eugeni.torproject.org (Postfix) with ESMTP id 7B9962FC66
 for <tor-talk@lists.torproject.org>; Sat, 14 Jun 2014 07:44:52 +0000 (UTC)
X-Virus-Scanned: Debian amavisd-new at eugeni.torproject.org
Received: from eugeni.torproject.org ([127.0.0.1])
 by localhost (eugeni.torproject.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id Ew5ZZ0-FdtOQ for <tor-talk@lists.torproject.org>;
 Sat, 14 Jun 2014 07:44:52 +0000 (UTC)
Received: from mx1.riseup.net (mx1.riseup.net [198.252.153.129])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (Client CN "*.riseup.net", Issuer "Gandi Standard SSL CA" (not verified))
 by eugeni.torproject.org (Postfix) with ESMTPS id 4A42E2E9A7
 for <tor-talk@lists.torproject.org>; Sat, 14 Jun 2014 07:44:52 +0000 (UTC)
Received: from fulvetta.riseup.net (fulvetta-pn.riseup.net [10.0.1.75])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (Client CN "*.riseup.net", Issuer "Gandi Standard SSL CA" (not verified))
 by mx1.riseup.net (Postfix) with ESMTPS id 37E374F089
 for <tor-talk@lists.torproject.org>; Sat, 14 Jun 2014 00:44:49 -0700 (PDT)
Received: from [127.0.0.1] (localhost [127.0.0.1])
 (Authenticated sender: mirimir@fulvetta.riseup.net)
 with ESMTPSA id 38DB61F1
Message-ID: <539BFD6B.7040104@riseup.net>
Date: Sat, 14 Jun 2014 01:44:43 -0600
From: Mirimir <mirimir@riseup.net>
User-Agent: Mozilla/5.0 (X11; Linux x86_64;
 rv:24.0) Gecko/20100101 Thunderbird/24.5.0
MIME-Version: 1.0
To: tor-talk@lists.torproject.org
References: <CAD2Ti29i-Rh=80edAs2KQaufj-+5HoKp+mzCxKNsWkY2fRo7KQ@mail.gmail.com>
 <53913B05.7000709@riseup.net>
 <CAD2Ti2-u-pKLOgvXhxrZL3P-0_Exxq5ahC4O5VQ+VnfzpBuqDQ@mail.gmail.com>
 <539A7E01.3090803@riseup.net> <539A83EF.4040602@riseup.net>
 <CAD2Ti2-LygGjWtJM01u0QgmNbQQGkp81YNPKdQrZu5poZ4XeCg@mail.gmail.com>
 <539BD1DE.3080008@riseup.net>
 <CAD2Ti2_8e3VieUw_zVdRGP+cBCZ0tWvGDKRr1aq1=JMao3jZow@mail.gmail.com>
In-Reply-To: <CAD2Ti2_8e3VieUw_zVdRGP+cBCZ0tWvGDKRr1aq1=JMao3jZow@mail.gmail.com>
X-Enigmail-Version: 1.6
X-Virus-Scanned: clamav-milter 0.98.1 at mx1
X-Virus-Status: Clean
Subject: Re: [tor-talk] Craigslist now giving Tor the slows, lol
X-BeenThere: tor-talk@lists.torproject.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: tor-talk@lists.torproject.org
List-Id: "all discussion about theory, design,
 and development of Onion Routing" <tor-talk.lists.torproject.org>
List-Unsubscribe: <https://lists.torproject.org/cgi-bin/mailman/options/tor-talk>, 
 <mailto:tor-talk-request@lists.torproject.org?subject=unsubscribe>
List-Archive: <http://lists.torproject.org/pipermail/tor-talk/>
List-Post: <mailto:tor-talk@lists.torproject.org>
List-Help: <mailto:tor-talk-request@lists.torproject.org?subject=help>
List-Subscribe: <https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk>, 
 <mailto:tor-talk-request@lists.torproject.org?subject=subscribe>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: tor-talk-bounces@lists.torproject.org
Sender: "tor-talk" <tor-talk-bounces@lists.torproject.org>

On 06/13/2014 11:51 PM, grarpamp wrote:
> On Sat, Jun 14, 2014 at 12:38 AM, Mirimir <mirimir@riseup.net> wrote:
>>> No more variance = tor issue.
>>> Still variance = IP <--> CL stack/path issue, or CL issue alone.
>> As I understand it, you're just getting the HTML. I'm getting the entire
> 
> It was the first time I saw any site serving slow to some tor exits.
> So I removed all variables and went for a single url fetch to confirm...
> no recursion, redirects, embedded elements, robots.txt, or anything else.
> I'm waiting for a slow affected exit operator to get back to me about
> test to eliminate unlikely possibility of tor software itself.

That makes sense. I'll add that to the test mix. I gather that you're
using something like liburi-fetch-perl, yes? A little reading tells me
that sites more often reject curl and wget, compared with fetch and
lynx. But I'll use whatever you're using for basic HTML.

>> page, or at least whatever Midori grabs while pretending to be Firefox.
>> For example, I get http://xvideos.com/ with numerous (X-rated) images ;)
>>
>> Also, I was hitting sites at 1-2 minute intervals
> 
> This may actually be far less than overall fetch rate from tor users
> to the top50, and certainly insignificant to the sites daily hit count.
> Someone needs to research overall exit traffic sometime too.

Sorry, I wasn't clear. I meant that I might have been overloading my Tor
client with too many simultaneous circuits.

>> craigslist, the greatest loading time was about 500 seconds. So perhaps
> 
> If other sites are loading similarly slow it may be possible to find out
> why or what is being used to do it. CL never replies to support queries.

Fundamentally, CL doesn't care what anyone else thinks ;)

>> 30-60 minutes to 20-40 minutes. That may reduce page-size variance.
> 
> A lot of the top50 use dynamic 'content' so it is expected on those,
> unless fetching single elements.

Again, I'm talking about effects on my client and the VM it's in, not on
Tor relays or websites.

>> There's also wkhtmltopdf. Maybe it does a better job, being lighter even
>> than Midori. But I worry that it also may look less like a browser than
>> command-line Midori.
> 
> I'm not too worried about emulation/hiding unless it affects the results
> being studied. ie: content/blocking differences depending on supplied
> User-agent.

Right.

>> Once I work out kinks, and collect enough data, I'll write this up
>> somewhere with results for all 50 top sites.
> 
> Good, we are doing some generic things it seems. And should not
> use this CL specific thread subject anymore for it :)

Agreed. But I would like a response about fetch (liburi-fetch-perl?).
-- 
tor-talk mailing list - tor-talk@lists.torproject.org
To unsubscribe or change other settings go to
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk

