Delivery-Date: Sat, 14 Jun 2014 01:56:42 -0400
Return-Path: <tor-talk-bounces@lists.torproject.org>
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on moria.seul.org
X-Spam-Level: 
X-Spam-Status: No, score=-4.7 required=5.0 tests=BAYES_00,DKIM_ADSP_CUSTOM_MED,
	DKIM_SIGNED,FREEMAIL_FROM,RCVD_IN_DNSWL_MED,RP_MATCHES_RCVD,T_DKIM_INVALID
	autolearn=ham version=3.3.1
X-Original-To: archiver@seul.org
Delivered-To: archiver@seul.org
Received: from eugeni.torproject.org (eugeni.torproject.org [38.229.72.13])
	(using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits))
	(No client certificate requested)
	by moria.seul.org (Postfix) with ESMTPS id 78F491E09F5
	for <archiver@seul.org>; Sat, 14 Jun 2014 01:56:40 -0400 (EDT)
Received: from eugeni.torproject.org (localhost [127.0.0.1])
	by eugeni.torproject.org (Postfix) with ESMTP id F38F62FC14;
	Sat, 14 Jun 2014 05:56:37 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
 by eugeni.torproject.org (Postfix) with ESMTP id E9BA92EDF5
 for <tor-talk@lists.torproject.org>; Sat, 14 Jun 2014 05:51:06 +0000 (UTC)
X-Virus-Scanned: Debian amavisd-new at eugeni.torproject.org
Received: from eugeni.torproject.org ([127.0.0.1])
 by localhost (eugeni.torproject.org [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id PtxeUEWyiJMT for <tor-talk@lists.torproject.org>;
 Sat, 14 Jun 2014 05:51:06 +0000 (UTC)
Received: from mail-ve0-x22a.google.com (mail-ve0-x22a.google.com
 [IPv6:2607:f8b0:400c:c01::22a])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (Client CN "smtp.gmail.com",
 Issuer "Google Internet Authority G2" (not verified))
 by eugeni.torproject.org (Postfix) with ESMTPS id CA02D2ED16
 for <tor-talk@lists.torproject.org>; Sat, 14 Jun 2014 05:51:06 +0000 (UTC)
Received: by mail-ve0-f170.google.com with SMTP id i13so694267veh.29
 for <tor-talk@lists.torproject.org>; Fri, 13 Jun 2014 22:51:04 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:in-reply-to:references:date:message-id:subject:from:to
 :content-type; bh=a62URKTmZeRgi5YfRl+gw5QGO8NbYcbI0B6+GaytWYs=;
 b=A5T7OsGPF3+uxDN8kUWm/sWC+oXorxqszB0STvOc/MiPBZS6ukNz8YgXEXHREBWPq6
 WsSfrn7w3QnwIK7s/pB1GnqN7wM9XMj8BZoTpS8e7mr4BUL8XEfWSM+wWzAdRTrSC0q7
 9P+GWPM0UnAD93hEE0N3N7J+omM0JPq3sSCjb9XWcXkO+Uayx5HCOwbARAzJkFW451ux
 4cpESUANwJP3AsJrvSA69j+AlNp70KIe0IhWrsw4tKGgoqEqnr1tnQR1EVEqg53lncm9
 M1HfAB7tQPGqqcz2ZDi1AGP6tzPcXkizJTKwnVc/ScMmBiTRyW6JeI305hRqAw1X+60s
 2LoQ==
MIME-Version: 1.0
X-Received: by 10.52.179.38 with SMTP id dd6mr4561224vdc.21.1402725064270;
 Fri, 13 Jun 2014 22:51:04 -0700 (PDT)
Received: by 10.221.65.198 with HTTP; Fri, 13 Jun 2014 22:51:04 -0700 (PDT)
In-Reply-To: <539BD1DE.3080008@riseup.net>
References: <CAD2Ti29i-Rh=80edAs2KQaufj-+5HoKp+mzCxKNsWkY2fRo7KQ@mail.gmail.com>
 <53913B05.7000709@riseup.net>
 <CAD2Ti2-u-pKLOgvXhxrZL3P-0_Exxq5ahC4O5VQ+VnfzpBuqDQ@mail.gmail.com>
 <539A7E01.3090803@riseup.net> <539A83EF.4040602@riseup.net>
 <CAD2Ti2-LygGjWtJM01u0QgmNbQQGkp81YNPKdQrZu5poZ4XeCg@mail.gmail.com>
 <539BD1DE.3080008@riseup.net>
Date: Sat, 14 Jun 2014 01:51:04 -0400
Message-ID: <CAD2Ti2_8e3VieUw_zVdRGP+cBCZ0tWvGDKRr1aq1=JMao3jZow@mail.gmail.com>
From: grarpamp <grarpamp@gmail.com>
To: tor-talk@lists.torproject.org
Subject: Re: [tor-talk] Craigslist now giving Tor the slows, lol
X-BeenThere: tor-talk@lists.torproject.org
X-Mailman-Version: 2.1.15
Precedence: list
Reply-To: tor-talk@lists.torproject.org
List-Id: "all discussion about theory, design,
 and development of Onion Routing" <tor-talk.lists.torproject.org>
List-Unsubscribe: <https://lists.torproject.org/cgi-bin/mailman/options/tor-talk>, 
 <mailto:tor-talk-request@lists.torproject.org?subject=unsubscribe>
List-Archive: <http://lists.torproject.org/pipermail/tor-talk/>
List-Post: <mailto:tor-talk@lists.torproject.org>
List-Help: <mailto:tor-talk-request@lists.torproject.org?subject=help>
List-Subscribe: <https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk>, 
 <mailto:tor-talk-request@lists.torproject.org?subject=subscribe>
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: 7bit
Errors-To: tor-talk-bounces@lists.torproject.org
Sender: "tor-talk" <tor-talk-bounces@lists.torproject.org>

On Sat, Jun 14, 2014 at 12:38 AM, Mirimir <mirimir@riseup.net> wrote:
>> No more variance = tor issue.
>> Still variance = IP <--> CL stack/path issue, or CL issue alone.
> As I understand it, you're just getting the HTML. I'm getting the entire

It was the first time I saw any site serving slow to some tor exits.
So I removed all variables and went for a single url fetch to confirm...
no recursion, redirects, embedded elements, robots.txt, or anything else.
I'm waiting for a slow affected exit operator to get back to me about
test to eliminate unlikely possibility of tor software itself.

> page, or at least whatever Midori grabs while pretending to be Firefox.
> For example, I get http://xvideos.com/ with numerous (X-rated) images ;)
>
> Also, I was hitting sites at 1-2 minute intervals

This may actually be far less than overall fetch rate from tor users
to the top50, and certainly insignificant to the sites daily hit count.
Someone needs to research overall exit traffic sometime too.

> craigslist, the greatest loading time was about 500 seconds. So perhaps

If other sites are loading similarly slow it may be possible to find out
why or what is being used to do it. CL never replies to support queries.

> 30-60 minutes to 20-40 minutes. That may reduce page-size variance.

A lot of the top50 use dynamic 'content' so it is expected on those,
unless fetching single elements.

> There's also wkhtmltopdf. Maybe it does a better job, being lighter even
> than Midori. But I worry that it also may look less like a browser than
> command-line Midori.

I'm not too worried about emulation/hiding unless it affects the results
being studied. ie: content/blocking differences depending on supplied
User-agent.

> Once I work out kinks, and collect enough data, I'll write this up
> somewhere with results for all 50 top sites.

Good, we are doing some generic things it seems. And should not
use this CL specific thread subject anymore for it :)
-- 
tor-talk mailing list - tor-talk@lists.torproject.org
To unsubscribe or change other settings go to
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk

