话不多说,先上第一版的代码:

use Time::HiRes qw/time/;
use AnyEvent::HTTP;
use AnyEvent;
use Coro;

my %code;
my $count = $ARGV[0];
my $url = "https://10.10.10.10/";
my $begin = time;
my @coro = map {
    async {
        my $cv = AnyEvent::condvar;
        $cv->begin;
        my $header_time;
        http_request GET => "$url",
            sub {
                my (undef, $hdr) = @_;
                $code{$hdr->{'Status'}}++;
                $cv->end;
            }
        ;
        $cv->recv;
    }
} (1 .. $count);
$_->join for @coro;
print $cpus*$ARGV[0]/(time-$begin);

上面这段脚本,作用是在每个进程中运行事件驱动的协程,以达到尽可能大的并发请求。

初步的测试,在单核Coro的情况下可以每秒发送1000+的https请求。

注意:如果使用的是AnyEvent::HTTP::LWP::UserAgent模块,虽然POD里写它用的其实就是AnyEvent::HTTP的代码套LWP的API格式,但实际只能用到30%的CPU,单核情况下的qps也就不到350的样子。

注意:本例测试的是HTTPS,AnyEvent::HTTP在TLS模式(即https请求)下,无法开启persistent连接。如果是普通http请求,开启persistent参数的qps应该会更高!

然后上第二版的代码,改用了EV循环,性能比Coro协程提高了大概5%的样子。使用了fork多进程,并且绑定到不同的CPU核上。

#!/usr/bin/perl
use strict;
use warnings;
use autodie;
use Parallel::ForkManager;
use Time::HiRes qw/time/;
use Sys::CpuAffinity;
use AnyEvent::HTTP;
use AE;
use EV;

my $cpus = Sys::CpuAffinity::getNumCpus();
my $pm = new Parallel::ForkManager($cpus, "/tmp/");
my @urls = qw(https://10.11.19.35/ https://10.11.21.121/);
my $res;

$pm->run_on_finish( sub {
    my $data = $_[5];
    $res->{'code'}->{$_} += $data->{'code'}->{$_} for keys %{$data->{'code'}};
    $res->{'size'} += $data->{'size'};
    $res->{'time'} += $data->{'time'};
});

my $begin = time;
foreach my $cpu (1 .. $cpus) {
    my $pid = $pm->start and next;
    Sys::CpuAffinity::setAffinity($$, 2 ** ($cpu-1));
    my $data = ae_get($ARGV[0], \@urls);
    $pm->finish(0, $data);
}
$pm->wait_all_children;
my $use = time - $begin;

printf "%d fetches, %d max processes, in %.03f seconds\n", $ARGV[0] * $cpus, $cpus, $use;
printf "%.03f fetches/sec, %.03f bytes/sec\n", $ARGV[0] * $cpus / $use, $res->{'size'} / $res->{'time'};
printf "HTTP response codes:\n";
printf "       %d - %d\n", $_, $res->{'code'}->{$_} for sort keys %{$res->{'code'}};

sub ae_get {
    my ($count, $urls) = @_;
    my $data;
    my $tmptime;
    my $cv = AE::cv;
    for (1 .. $count) {
        my $hdr_time;
        my $url = $urls->[int(rand($#{$urls}+1))];
        $cv->begin;
        http_request
            GET       => "$url", 
            on_header => sub {
                $hdr_time = time;
            },
            sub {
                my (undef, $hdr) = @_;
                $data->{'code'}->{$hdr->{'Status'}}++;
                $data->{'size'} += $hdr->{'content-length'};
                $data->{'time'} += ( time - $hdr_time );
                $cv->end;
            }
        ;
    } 
    $cv->recv;
    return $data;
}

增加了简单的统计功能,包括每秒请求数、平均下载速度,状态码汇总等。因为不好计算header的长度,所以只计算body部分的下载速度。 增加了url列表功能,每次请求会随机的抽取其中的一个url。