Sunday, August 30, 2020

YAAC is not Yak


YAAC is not Yak but Gnu, you really ought to k-now wa-who's wa-who. YAAC is - Yet Another APRS Client written in Java under GNU license. YAAC, I'm quoting here - "can be used as a stand-alone APRS client, an APRS RF-Internet gateway (I-Gate), or as a AX.25 digipeater". It doesn't matter what does this even mean but this app can actually be used to display multiple weather sondes at the same time on a map.

As I previously mentioned one could use auto-rx to display multiple sondes on a map. I noticed though that it taxes CPU a lot which isn't actually a problem if I would receive sondes on one computer and display them on another one. Another advantage of using YAAC is that I can also display additional information if I want to.

Installation and Usage


As YAAC is simply a jar file, no installation is required - just download and start it with 'java -jar YAAC.jar'.

First of all map tiles should be downloaded. There are menu entries under 'File' to do that but I find that it's much easier to let it download them automatically. For that there's a well hidden option under File -> Configure -> Expert Mode -> Behavior :

When map tiles for the needed region are downloaded this option can be deactivated.

Feeding YAAC with data

 

Now the questions is how do I feed YAAC with my json sondes messages? Well there are several options that YAAC understands but the easiest one I found was to fake an APRS-IS server. An APRS-IS server is used normally to commit APRS messages received locally over the radio and also to receive such messages that other users are committing at the same time. There's a connection protocol in plain text form which makes faking such a server a bit of a challenge. First of all after connection is established server sends its header back to the client and then client sends its credentials which are parsed for correctness and a reply is sent back to the client. Then server should send its header periodically to its clients to show that its alive. Otherwise YAAC shows a very scary warning dialog that the server is not active any longer.

At first I implemented a really simple shell script for that but because of the 'tail -f' I couldn't manage to kill it cleanly. That darn tail simply won't die. Because of that I have rewritten the script in perl:

#!/usr/bin/perl

use strict;
use warnings;
use sigtrap 'handler' => \&sig_handler, qw(INT TERM KILL QUIT);
use Socket;

my ($soc_listen,$soc_broadcast);
my $paddr;
my @pids;

sub sig_handler {
  print "Signal handler is called, exiting...\n";
  kill HUP => $pids[0];
  kill HUP => $pids[1];
  exit(0);
}

# flush after every write
$| = 1;

my $port = '30448'; # TODO make the port number configurable
my $remote = '0.0.0.0';
my $server_ping="# aprsfakeserver (c) 0v2\n";

# create sockets to listen and broadcast
socket($soc_listen, AF_INET, SOCK_DGRAM, getprotobyname('udp')) or die "Can't open socket $!\n";
socket($soc_broadcast, AF_INET, SOCK_DGRAM, getprotobyname('udp')) or die "Can't open socket $!\n";
 
setsockopt($soc_listen, SOL_SOCKET, SO_REUSEADDR, 1) or die "Can't set socket option to SO_REUSEADDR $!\n";
setsockopt($soc_broadcast, SOL_SOCKET, SO_BROADCAST, 1) or die "Can't set socket option to  SO_BROADCAST $!\n";
 
my $iaddr = inet_aton($remote) or die "Unable to resolve hostname : $remote";
$paddr = sockaddr_in($port, $iaddr); #socket address structure

my $child = 0;

for ( my $i = 0; $i<2; $i++)
{
  my $pid = fork();
  die "Error in fork: $!" unless defined $pid;
  $pids[$i] = $pid;

  if (not $pid) {
    if (not $child) {
      # Code executed by the child process 0
      bind($soc_listen, $paddr) or die "Connect failed : $!";
      print "# Connected to $remote on port $port\n";
      local $SIG{HUP} = sub { close($soc_listen);exit(0); };

      while(1) {
        my $result = '';
        my $datastring = '';
        my $hispaddr;
        $hispaddr = recv($soc_listen, $datastring, 400, 0); # blocking recv MSG_WAITALL
        if (!defined($hispaddr)) {
          print ("# recv failed: $!\n");
          last;
        }
        $datastring =~ s/[\r\n]+$//;
        if ($datastring =~ /^$/) {
          next;
        }
        if ($datastring =~ /^#.*$/) {
          $result = $datastring; # just print comment (ping) messages
        }
        elsif ($datastring =~ /^user ([^ ]*).*$/) {
          my $user_name = $1;
          # now check if filter string is also there
          $datastring =~ /^user ([^ ]*).*(filter.*?)$/;
          $result = "# logresp $user_name verified, server N0APRS-1 " . ($2 // "");
        }
        elsif ($datastring =~ /^APRS: (.*)$/) {
          $result = "$1\n";
        }
        else {
          $result = $datastring;
        }
        print "$result\n";
      }
      close($soc_listen);
      exit(0);
    }
    else {
      # Code executed by the child process 1
      my $pingcounter = 0;
      my $doexit = 0;
      local $SIG{HUP} = sub { $doexit = 1 };
      # Code executed by the second child process
      while(not $doexit) {
        $pingcounter = $pingcounter + 1;
        select(undef, undef, undef, 0.25); # sleep 250 ms
        if (not $pingcounter % (60*4)) {
          send($soc_broadcast, $server_ping, 0, $paddr);
        }
      }
      exit(0);
    }
  }
  $child = $child + 1;
}

select(undef, undef, undef, 0.25); # sleep 250 ms
send($soc_broadcast, $server_ping, 0, $paddr);

while(<STDIN>) {
  send($soc_broadcast, $_, 0, $paddr);
}
Basically the same technique is used here as in the shell version:
  1. Listens on the local port 30448 for incoming UDP messages, parses them and prints responses to its STDOUT
  2. Redirects its STDIN to the port 30448
  3. Broadcasts regularly its header on the UDP port 30448
But in this form the script cannot be used with YAAC because YAAC creates a TCP connection to the server and script only read/writes its STDIN/STDOUT. It is implemented this way for simplicity because I can easily connect any process's STDIN/STDOUT to a TCP socket using socat.

So to start the server the following command is used:

socat -d -d TCP-LISTEN:14580,reuseaddr,fork exec:./aprs/aprsfakeserver.pl,pty,stderr

As long as the server is started a new port should be created in YAAC:
Now YAAC will receive and display APRS messages from my fake server.

 

APRS messages


The next problem is that the JSON formatted messages from sondes decoders needs to be converted into APRS messages that YAAC understands. I have taken Zilog's script as the base and rewritten it

Messages from sondes decoders are broadcasted on the UDP port 5678 on my local network. To feed YAAC with them the following command is started:

nc -luk 5678 |
 ./aprs/json2aprs.pl MYCALLSIGN MYLAT MYLON "My comment srting" | 
 socat -u - UDP4-DATAGRAM:0.0.0.0:30448,broadcast,reuseaddr

MYCALLSIGN is supposed to be a HAM operator callsign but can be any alphanumeric string up to 6 symbols in this case. The callsign should be specified without any SSID. MYLAT and MYLON are latitude and longitude of your position in decimal degrees.

Properly configured and started results in the following:



Adding additional information to the map


In the screenshot above I also have HAM operators APRS messages displayed. I receive these messages using another RTL-SDR dongle attached to another computer on my local network. I start the following command on that computer:

./scripts/demodulatenfm.sh -f 144800000 -g 0 -s 2400000 -P 35 -b 6000 | \
sox -t raw -esigned-integer -b 16 -r 48000 - -b 16 -t raw -r 22050 -esigned-integer - | \
tee >(aplay -r 22050 -f S16_LE -t raw -c 1 -B 500000) | \
stdbuf -oL multimon-ng -t raw -a AFSK1200 -A - 2>/dev/null | \
grep --line-buffered 'APRS: ' | tee /dev/stderr | \
socat -u - udp-datagram:192.168.1.255:30448,broadcast

So multimon-ng decodes demodulated signal and broadcasts in on my local network (192.168.1.255 is the broadcast address of my Wi-Fi router) on the same UDP port (30448) which aprsfakeserver script listens to. In this case I don't have to translate messages because they are already in APRS format. multimon-ng project is here and demodulatenfm.sh script is here

The same way I could create weather APRS messages using information from my wireless temperature sensors and inject them into YAAC.


Display weather reports


I have several wireless sensors reporting outside temperature and humidity. Decoded information is broadcasted as JSON messages on UDP port 6666:

{"time" : "2020-09-02 21:42:18", "model" : "pressure sensor", "id" : 242, "pressure" : "1022.22"}
{"time" : "2020-09-02 21:42:08", "model" : "inFactory sensor", "id" : 129, "temperature_C" : 15.444, "humidity" : 71}

Message come asynchronously and with different time intervals. With the following script I convert such JSON messages into APRS form and inject them into YAAC:

nc -luk 6666 | ./weather2aprs.pl N0CALL 47.00 12.00 | socat -u - UDP4-DATAGRAM:0.0.0.0:30448,broadcast,reuseaddr

YAAC shows such APRS messages the following way:




Submitting decoded sondes messages to a real APRS-IS server


Using json2aprs.pl script I can also commit sondes messages to a real server like radiosondy.info . The problem is that one does not simply commit all messages to such servers. First of all messages needs to be filtered to avoid committing of wrongly or not fully decoded ones. Additionally to avoid server overload messages should be sent with a timeout if a sonde is above some predefined altitude. I have written a simple filter script that does this:
#!/usr/bin/env perl
#

use strict;
use warnings;

use JSON;
use threads::shared;

$|=1;

my $homelat; my $homelon;

while (@ARGV) {
  $homelat = shift @ARGV;
  $homelon = shift @ARGV;
}

defined $homelat or $homelat = 0.0;
defined $homelon or $homelon = 0.0;

$homelat != 0.0 or die "Home latitude should not be 0.0";
$homelon != 0.0 or die "Home longitude should not be 0.0";

my $max_distance_m    = 1000000; # 1000 km
my $max_altitude_m    = 50000;   # 50 km
my $min_altitude_m    = -50;     # -50 m
my $min_sats          = 4;       # minimum number of sats

# if a sonde is above $decimation_alt altitude then
# don't commit to the server too often to not overload it
my $decimation_period = 15;      # commit to the server each N seconds
my $decimation_alt    = 3000;    # decimation activation altitude in m

our %messages_dict;

$SIG{ALRM} = sub {
    # reschedule the next signal for N seconds from now
    alarm $decimation_period;
    lock (%messages_dict);
    my $id;
    foreach $id (keys %messages_dict) {
      print $messages_dict{$id};
      delete $messages_dict{$id};
    }
};

# schedule decimation filter
alarm $decimation_period;

sub filter_sonde_by_id {
  my ($id) = @_;

  defined $id or return 0;

  # filter DFM wrong ids out
   not $id =~ /Dxxxxxxxx/ or return 0;
#  re.match(r'DFM-\d{6}', _serial)

  return 1;
}

my $json;

while (<>) {
  # only for json strings
  if ($_ =~ /^{.*}$/) {
    $json = decode_json($_);
    my $datetime = $json->{"datetime"};

    my $lat = $json->{"lat"};
    my $lon = $json->{"lon"};

    my $responsejson = `./calcdistance.sh $homelat $homelon $json->{"lat"} $json->{"lon"}`;
    my $distancejson = decode_json($responsejson);
    my $distance = $distancejson->{"distance"};
    if ($distance > $max_distance_m) {
      print STDERR "Discarded by the distance filter: $_";
    } 

    my $sats= $json->{"sats"};
    if (defined $sats) {
      if ($sats < $min_sats) {
        print STDERR "Discarded by the sats filter: $_";
      } 
    }

    my $altitude = $json->{"alt"};
    if ($altitude > $max_altitude_m or $altitude < $min_altitude_m) {
      print STDERR "Discarded by the altitude filter: $_";
    } 

    my $id = $json->{"id"};

    filter_sonde_by_id($id) or next;

    if ($altitude < $decimation_alt) {
      # anyting below the $decimation_alt is reported immediately
      print $_;
      next;
    }
    else {
      # otherwise put in into the decimation  dictionary 
      lock (%messages_dict);
      $messages_dict{$id} = $_; 
    }

  } 
}

Now I can use the same socat trick to connect to a real APRS-IS server:
socat -d -d exec:./aprs/aprsfakeclient.sh,pty,stderr TCP:radiosondy.info:14590

where aprsfakeclient.sh is:

#!/bin/bash

nc -luk 5678 | ./aprs/json2aprsfilter.pl $MYLAT $MYLON | \
./aprs/json2aprs.pl $MYCALLSIGN $MYLAT $MYLON "My ARPS proxy for receivemultisonde.sh"

Note: sending anything to APRS-IS servers is strictly speaking not allowed if you don't own a proper HAM license because it could happen that this information will also be retransmitted over the air which is illegal without a license. I don't know the details so I only tried committing to the port 14590 of radiosondy.info server. This port is known to be not retransmitting.

No comments: