Python vs Perl vs Lua - speed comparison

I have written the base64 encoding algorithm in Perl, Python and Lua to compare the performance in basic arithmetic and file-io operations. Perl implementation was written a long time ago and it supposed to be a one liner. That's why the script looks so obfuscated:

alex@thinkpad:~$ cat bin/base64.pl
#!/usr/bin/env perl

$/=\1;
@a=();$b=0;

sub c2 {p(@_[0]>>2);p(((@_[0]&3)<<4)+((@_[1]&240)>>4))}
sub c3 {c2(@_);p(((@_[1]&15)<<2)+((@_[2]&192)>>6))}
sub c {c3(@_);p(@_[2]&63)}
sub p {
  if($_[0]<26){$n=ord("A")+$_[0]}
  elsif($_[0]<52){$n=ord("a")+$_[0]-26}
  elsif($_[0]<62){$n=ord("0")+$_[0]-52}
  elsif($_[0]==62){$n=ord("+")}
  else{$n=ord("/")}
  print chr($n);
  $b+=1;
  if($b%76==0){print "\n"}
}

while(<>) {
  push(@a,ord($_));
  if(3==@a) {
    c(@a);
    @a=()
  }
}

if(1==@a) {
  c2(@a[0],0);
  print "=="
}
elsif(2==@a) {
  c3(@a[0],@a[1],0);
  print "="
}
if($b>0){print "\n"}
 

Python and Lua implementations are just rewrites of the Perl version. The difference is that they both can only read from the stdin whereas the Perl version can read from the stdin and also from files specified on the command line.

Python version:

alex@thinkpad:~$ cat bin/base64.py
#!/usr/bin/env python

import sys
a=[]
b=0

def p(a):
  global b
  if a<26:
    n=ord("A")+a
  elif a<52:
    n=ord("a")+a-26
  elif a<62:
    n=ord("0")+a-52
  elif a==62:
    n=ord("+")
  else:
    n=ord("/")
  sys.stdout.write(chr(n));
  b+=1;
  if b%76 is 0:
    print
  return
def c2(a):
  p(a[0]>>2);
  p(((a[0]&3)<<4)+((a[1]&240)>>4));
  return
def c3(a):
  c2(a);
  p(((a[1]&15)<<2)+((a[2]&192)>>6));
  return
def c(a):
  c3(a);
  p(a[2]&63);
  return

try:
  if sys.version_info < (3,0):
    stream=sys.stdin
  else:
    stream=sys.stdin.detach()

  while True:
    s = stream.read(1)
    if not s:
      break; # eof condition
    a.append(ord(s))
    if len(a)==3:
      c(a);
      a=[]

  if 1==len(a):
    c2([a[0],0,0]);
    sys.stdout.write("==");
  elif 2==len(a):
    c3([a[0],a[1],0]);
    sys.stdout.write("=");
  if b is not 0:
    print
except KeyboardInterrupt:
  sys.stdout.flush()
  pass
 

and the Lua version:

alex@thinkpad:~$ cat bin/base64.lua
#!/usr/bin/env lua

function p(a)
  if a<26 then
    n=string.byte("A")+a
  elseif a<52 then
    n=string.byte("a")+a-26
  elseif a<62 then
    n=string.byte("0")+a-52
  elseif a==62 then
    n=string.byte("+")
  else
    n=string.byte("/")
  end
  io.write(string.char(n))
  b=b+1
  if (b%76)==0 then
    io.write("\n")
  end
end

function c2(a)
  p(bit32.rshift(a[1],2))
  p(bit32.lshift(bit32.band(a[1],3),4)+bit32.rshift(bit32.band(a[2],240),4))
end

function c3(a)
  c2(a)
  p(bit32.lshift(bit32.band(a[2],15),2)+bit32.rshift(bit32.band(a[3],192),6))
end

function c(a)
  c3(a)
  p(bit32.band(a[3],63))
end

a={}
b=0

while true do
  local s = io.read(1)
  if s == nil then break end 
  table.insert(a,string.byte(s))
  if 3==#a then
    c(a) 
    a={}
  end
end

if 1==#a then
  c2({a[1],0,0});
  io.write("==")
elseif 2==#a then
  c3({a[1],a[2],0});
  io.write("=")
end
if b > 0 then
  io.write("\n")
end
 

First of all tests to check if implementations are correct:

alex@thinkpad:~$ dd if=/dev/urandom bs=1M count=10 of=/tmp/10M
 
alex@thinkpad:~$ diff -s <(base64 /tmp/10M) <(bin/base64.pl /tmp/10M)
Files /dev/fd/63 and /dev/fd/62 are identical
 
alex@thinkpad:~$ diff -s <(base64 /tmp/10M) <(cat /tmp/10M|bin/base64.py)
Files /dev/fd/63 and /dev/fd/62 are identical
 
alex@thinkpad:~$ diff -s <(base64 /tmp/10M) <(cat /tmp/10M|bin/base64.lua)
Files /dev/fd/63 and /dev/fd/62 are identical

I should note that Python script haven't passed the test with the Python3 interpreter and that's why I skipped its results. There must be some problem with detaching the stream and this is the only way I know how to solve unicode problem with the stdin.read() function.

And here are the results of the measuring:

alex@thinkpad:~$ dd if=/dev/urandom bs=1M count=10 | bin/base64.pl > /dev/null
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 13.9454 s, 752 kB/s
 
alex@thinkpad:~$ dd if=/dev/urandom bs=1M count=10 | bin/base64.py > /dev/null
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 19.1276 s, 548 kB/s
 
alex@thinkpad:~$ dd if=/dev/urandom bs=1M count=10 | bin/base64.lua > /dev/null
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 20.941 s, 501 kB/s
 
alex@thinkpad:~$ dd if=/dev/urandom bs=1M count=10 | base64 > /dev/null
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.808636 s, 13.0 MB/s
 
alex@thinkpad:~$ dd if=/dev/urandom bs=1M count=10 | openssl base64 > /dev/null
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 0.765342 s, 13.7 MB/s

Conclusion: Perl version is the fastest one, Python2 version is second and the Lua version is the slowest one! Still Perl version is more than 10 times slower than compiled C version

Versions of interpreters:

alex@thinkpad:~$ python
Python 2.7.4 (default, Apr 19 2013, 18:28:01) 
[GCC 4.7.3] on linux2
 
alex@thinkpad:~$ perl --version
This is perl 5, version 14, subversion 2 (v5.14.2) built for x86_64-linux-gnu-thread-multi
 
alex@thinkpad:~$ lua -v
Lua 5.2.1  Copyright (C) 1994-2012 Lua.org, PUC-Rio

Continue to the Part 2
Continue to the Part 3