Saturday, April 29, 2017

How to Connect the rtl_433 to a remote rtl_tcp server

One of the issues with the rtl_433 application is that it is not possible to connect it to a remote rtl_tcp server using syntax supported by other SDR application like gqrx. One of proposed solutions is to use the TCP enabled version of the librtlsdr which is a bit complicated. There's actually a much simpler solution.

The idea is to connect a simple client to a remote rtl_tcp server and redirect its standard output stream to the standard input of locally started rtl_433. I've implemented a simple rtl_tcp client in bash:
#!/bin/sh
# A simple bash rtl_tcp client

SET_FREQUENCY=1
SET_SAMPLE_RATE=2
SET_GAIN_MODE=3
SET_GAIN=4
SET_FREQUENCY_CORRECTION=5
SET_AGC_MODE=8

show_error_exit()
{
  echo "$1" >&2
  echo "For help: $0 -h"
  exit 2
}

show_usage()
{
cat <<HEREDOC
Usage: $1 [options]

Options:
  -h,             show this help message and exit
  -f FREQUENCY,   Frequency in Hz to tune to
  -a ADDRESS,     Address of the server to connect to (default: localhost)
  -p PORT,        Port of the server to connect to (default:1234)
  -s SAMPLERATE,  Sample rate to use (default: 2400000)
  -g GAIN,        Gain to use (default: 0 for auto)
  -P PPM,         PPM error (default: 0)
HEREDOC
}

byte()
{
  printf \\$(printf "%03o" $1)
}
int2bytes()
{
  byte $(($1>>24))
  byte $(($1>>16&255))
  byte $(($1>>8&255))
  byte $(($1&255))
}

set_frequency()
{
  byte $SET_FREQUENCY
  int2bytes $1
}
set_sample_rate()
{
  byte $SET_SAMPLE_RATE
  int2bytes $1
}
set_gain()
{
  if [ $1 -eq 0 ]; then
    # automatic gain control
    byte $SET_GAIN_MODE
    int2bytes 0
    byte $SET_AGC_MODE
    int2bytes 1
  else
    byte $SET_GAIN_MODE
    int2bytes 1
    byte $SET_AGC_MODE
    int2bytes 0
    byte $SET_GAIN
    int2bytes 0
    byte $SET_GAIN
    int2bytes $1
  fi
}

set_ppm()
{
  byte $SET_FREQUENCY_CORRECTION
  int2bytes $1
}

address='localhost'
port=1234
frequency=0
samplerate=0
gain=0
ppm=0

OPTIND=1 #reset index
while getopts "ha:p:f:s:g:P:" opt; do
  case $opt in
     h)  show_usage $(basename $0); exit 0; ;;
     a)  address="$OPTARG" ;;
     p)  port="$OPTARG" ;;
     f)  frequency="$OPTARG" ;;
     s)  samplerate="$OPTARG" ;;
     g)  gain="$OPTARG" ;;
     P)  ppm="$OPTARG" ;;
     \?) exit 1 ;;
     :)  echo "Option -$OPTARG requires an argument" >&2;exit 1 ;;
  esac
done
shift $((OPTIND-1)) 
 
[ ! "$frequency" -eq 0 ] || show_error_exit "Wrong frequency"
[ ! "$samplerate" -eq 0 ] || show_error_exit "Wrong sample rate"

(set_frequency $frequency;
 set_sample_rate $samplerate;
 set_gain $gain;
 set_ppm $ppm) | nc $address $port 

Usage examples

 

1. Using default rtl_433 parameters for frequency, sample rate and automatic gain for it to work:
$ rtlclient.sh -f 433920000 -a 192.168.1.153 -P 46 -s 250000 -g 0 | rtl_433 -G -r /dev/stdin
Here I'm connecting to the rtl_tcp server at address 192.168.1.153 and enable all rtl_433 decoders. My dongle ppm is 46.


2. Now I want to show the temperature in the terminal window and also on a graph in real time:
$ rtlclient.sh -f 433920000 -a 192.168.1.153 -P 46 -s 250000 -g 0 | \
  rtl_433 -G -r /dev/stdin -F json | tee /dev/stderr | grep --line-buffered ^{ | \
  jq --unbuffered -c 'select(.temperature_C!=null)|.temperature_C' | \
  bin/gp/gnuplotwindow.sh 200 "0:20" "Outside temperature"
Figure 1. Last 200 temperature samples
There are couple of problems here on the graph:
  1. There are single wrong values produces by sensors sometimes 
  2. The X axis shows number of processed samples and there's no relation to the time, i.e. samples appear asynchronously but I want to plot one sample per minute
The first problem could be solved by adding a simple median filter. The second problem could be solved by using the syncpipe.pl script that I described here. Additionally there are problems with the rtl_433 itself. First of all the program crashes with a segmentation fault so I put it inside of an endless loop. Second despite the announced -U flag it doesn't output an absolute time stamp for each decoded sample. I didn't even bother to look up the code to figure out why does it still output time stamps relative to the time of the program start. A simple jq filter solves this problem too.

The resulting command:
$ bin/rtlclient.sh -f 433920000 -a 192.168.1.153 -P 46 -s 250000 -g 0 | \
  (while true; do rtl_433 -G -r /dev/stdin -F json; done) | tee /dev/stderr | \
  grep --line-buffered ^{ | jq --unbuffered -c '.time |= (now|gmtime|mktime)' | \
  tee -a sensors.log | jq --unbuffered -c 'select(.temperature_C!=null)|.temperature_C'| \
  awk -v n=5 'BEGIN{m=int(n/2)+1}{if(1==i){for(j=1;j<n;++j){a[j]=a[j+1]}a[n]=$1}else{for(j=1;j<=n;++j){a[j]=$1};i=1}asort(a,b);print(b[m]);fflush()}' | \
  bin/gp/syncpipe.pl 60 0 | bin/gp/gnuplotwindow.sh "$((24*60))" ":" "Outside temperature"
Figure 2. Temperature samples for the last 24 hours

3. The command above also stores decoded samples in the sensors.log file. Using this file I can always recreate the graph for a specific sensor and for example add an integrating filter:
$ stdbuf -oL tail -n+1 -F sensors.log | grep --line-buffered Acurite |\
  jq --unbuffered -r 'select(.temperature_C!=null)|[.temperature_C,.time]|@csv' |\
  awk -v n=5 'BEGIN{m=int(n/2)+1}{if(1==i){for(j=1;j<n;++j){a[j]=a[j+1]}a[n]=$0}else{for(j=1;j<=n;++j){a[j]=$0};i=1}asort(a,b);print(b[m]);fflush()}' |\
  awk -F, '{if(firstTime==""){firstTime=$2;temp=$1};while($2-firstTime>0){firstTime+=60;print temp}temp=$1;fflush()}' |\
  bin/gp/addaverage.sh 15 1 |bin/gp/gnuplotwindow.sh "$((24*60))" ":" "Acurite Sensor" "Acurite Sensor averaged"
Figure 3. Acurite sensor temperature samples for the last 24 hours

All mentioned scripts are available here

4 comments:

Craig said...

Do you have any suggestions on how to get the rtlclient.sh to exit when using an execution time limit within rtl_433. The time limit is set using the -T option

example: rtlclient.sh -f 433920000 -a 127.0.0.1 -p 1234 -s 250000 | rtl_433 -R 40 -F json -T 15 -U -r /dev/stdin

alex said...

will this do?

timeout 15 rtlclient.sh -f 433920000 -a 127.0.0.1 -p 1234 -s 250000 | rtl_433 -R 40 -F json -U -r /dev/stdin

Anonymous said...

Thanks a lot! Works great.

Brendan Taylor said...

rtl_433 has rtl_tcp support built in now: https://github.com/merbanan/rtl_433/pull/894