Adventures in setting up a weather station

So, for the lack of a side project, I decided to set up a weather station, for automatically measuring indoor (and eventually outdoor) temperatures, and whatnot. It.. wasn’t as straightforward as it might have been. Read on!

Approach 1: Alpine Linux

So, considering the station would most likely be a relatively low-maintenance appliance, I first thought: “hey, Alpine Linux could do. It is for embedded systems, right?”.

Well.. turns out it wasn’t nearly as easy. To reveal the conclusion at the beginning:

  • With RPi, generally, you have to run a diskless install (well, with enough bodging perhaps not.. but in any case, full-SD installs like on Raspibian are not officially well supported). In practice, this means that the amount of space available is limited by your memory - and there’s not a whole lot of it. I gave up when I realized I need a whole Python 3 installation, and I couldn’t make it fit.
  • Lots of gotchas. E.g. when you create your own overlay, you are well served to add a /etc/.default_boot_services file, otherwise your system will boot up to a half-set-up state. In addition, you may need to set up a .boot_repository file to make sure packages you want are reinstalled upon reboot (remember, diskless mode!). Don’t forget overlays either!

I set up an open repository with what I’ve done; it might be of interest for those seeking to build Alpine Linux images.

Approach 2: Raspibian

After approach 1 turned out to be unviable without excess work, I decided to go with the trustworthy old classic, Raspibian Lite (at the time of writing, 11/01/2021 Buster image).

Enable 1-wire

First step is to enable the 1-Wire interface - as I have a DS18B20 temperature sensor, I will need this for it to operate correctly. This can be done via raspi-config

Set up InfluxDB

Per vendor instructions:

curl -s https://repos.influxdata.com/influxdb.key | sudo apt-key add -
source /etc/os-release
echo "deb https://repos.influxdata.com/debian $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/influxdb.list

sudo systemctl unmask influxdb.service
sudo systemctl start influxdb

Following script might be useful for configuration adjustments for Pi limitations

sed -i "s/.*reporting-disabled.*/reporting-disabled = true/g" /etc/influxdb/influxdb.conf
sed -i 's/.*index-version.*/index-version = "tsi1"/g' /etc/influxdb/influxdb.conf
sed -i 's/.*cache-max-memory-size.*/cache-max-memory-size = "75m"/g' /etc/influxdb/influxdb.conf
sed -i 's/.*query-timeout.*/query-timeout = "60s"/g' /etc/influxdb/influxdb.conf

Your InfluxDB might now be accessible from outside RPi. Adjust this as well, if necessary from /etc/influxdb/influxdb.conf

Set up Grafana

Per vendor instructions as well. I chose the OSS release, but you may choose Enterprise if you like.

sudo apt-get install -y apt-transport-https
sudo apt-get install -y software-properties-common wget
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -

echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list

sudo apt-get update
sudo apt-get install grafana

sudo systemctl enable grafana-server
sudo systemctl start grafana-server

Prepare InfluxDB

Before rest of the steps, run this:

curl -i -XPOST http://localhost:8086/query --data-urlencode "q=CREATE DATABASE weatherlog"

Remember, your InfluxDB is currently operating without authentication. This is the reason why it should not be accessible from outside RPi in most circumstances

Setting up DS18B20 logging

For this purpose, I drafted a small, simple Ruby script.

# encoding: UTF-8
# frozen_string_literal: true

require 'net/http'
require 'uri'

# User variables
##############

MEASUREMENT_INTERVAL = 15
INFLUXDB_SERVER = "http://127.0.0.1:8086"

W1_SYS_FLDR = "/sys/bus/w1/devices"

DATABASE = "weatherlog"
MEASUREMENT = "climate"

NAME_MAPPINGS = Hash.new {|hash, key| key}
# Define mappings here for different sensor names. Other sensors will receive default names

# Simple logging script for DS18B20 temperature sensors
##############

module DS18B20Logger
  def self.interval()
    loop do
      yield
      sleep MEASUREMENT_INTERVAL
    end
  end

  def self.ingest_and_send(serial_path)
    begin
      read_temperature = File.read("#{W1_SYS_FLDR}/#{serial_path}/temperature").strip.to_f / 1000.0
      
      # Bodge together an InfluxDB line protocol statement
      line = "#{MEASUREMENT},location=#{NAME_MAPPINGS[serial_path]} temperature=#{read_temperature}"

      response = Net::HTTP.post URI("#{INFLUXDB_SERVER}/write?db=#{DATABASE}"), line
      puts "Received response: #{response.inspect}"
    rescue => e
      STDERR.puts "Failed to ingest with device #{serial_path}: #{e.inspect}"
    end
  end

  def self.capture()
    begin
      Dir.foreach(W1_SYS_FLDR) do |fldr|
        # Only accept entries for DS18B20
        if /\A28-(?<serial>[[:xdigit:]]{12})\Z/i =~ fldr
          #puts "Found device, reading from: #{serial}"
          ingest_and_send "28-#{serial}"
        end
      end
    rescue => e
      STDERR.puts "Could not enumerate: #{e.inspect}. Skipping this cycle"
    end
  end

  def self.run()
    interval do
      puts "Capturing temperature"
      capture
    end
  end
end

Signal.trap("INT") { exit }

DS18B20Logger::run()

which can be run with a Systemd definition

[Unit]
Description=DS18B20 logging service

[Service]
Type=simple
Restart=always
RestartSec=1
User=nobody
ExecStart=/usr/bin/env ruby /opt/w1/ds18b20logger.rb

[Install]
WantedBy=multi-user.target

Don’t forget to install Ruby: sudo apt-get install ruby

Setting up Ruuvi tag scans

POST-RELEASE NOTE: The hcidump method mentioned below, whilst functional, is probably deprecated. A newer and possibly better (at least security-wise!) method would be using Bleson, whilst giving necessary setcap privileges to the Python executable. Privilege additions are still needed, but there’s a substantial difference between giving select capabilities and root entirely. Tread with care.

Bluetooth is complicated. There’s a wide variety of methods to read BLE (Bluetooth Low Energy) data, as a casual search engine search will reveal - and in context of Ruuvi tags, users have ran into a wide myriad of issues.

For this simple solution, we will need Python, Bluez, and this incredibly useful library from Tomi Tuhkanen

sudo apt-get install bluez bluez-hcidump python3 python3-pip

Set up libraries for global use (dirty trick, I know - but then again, the kind of operations the library does generally requires root)

sudo pip3 install ruuvitag_sensor influxdb

As adapted from the original script written by the library author above

from influxdb import InfluxDBClient
from ruuvitag_sensor.ruuvi import RuuviTagSensor

client = InfluxDBClient(host='localhost', port=8086, database='weatherlog')

# Write your location mappings here
LOCATION_MAPPINGS = {
  'XX:XX:XX:XX:XX:XX':  'exterior'
}

def write_to_influxdb(received_data):
    """
    Save data to InfluxDB, following roughly the schema given before with W1 measurements
    returns:
        Object to be written to InfluxDB
    """
    mac = received_data[0]
    payload = received_data[1]

    dataFormat = payload['data_format'] if ('data_format' in payload) else None
    fields = {}
    fields['temperature']               = payload['temperature'] if ('temperature' in payload) else None
    fields['humidity']                  = payload['humidity'] if ('humidity' in payload) else None
    fields['pressure']                  = payload['pressure'] if ('pressure' in payload) else None
    fields['acceleration_x']             = payload['acceleration_x'] if ('acceleration_x' in payload) else None
    fields['acceleration_y']             = payload['acceleration_y'] if ('acceleration_y' in payload) else None
    fields['acceleration_z']             = payload['acceleration_z'] if ('acceleration_z' in payload) else None
    fields['battery_voltage']            = payload['battery']/1000.0 if ('battery' in payload) else None
    fields['tx_power']                   = payload['tx_power'] if ('tx_power' in payload) else None
    fields['movement_counter']           = payload['movement_counter'] if ('movement_counter' in payload) else None
    fields['measurement_sequence_number'] = payload['measurement_sequence_number'] if ('measurement_sequence_number' in payload) else None
    fields['tag_id']                     = payload['tagID'] if ('tagID' in payload) else None
    fields['rssi']                      = payload['rssi'] if ('rssi' in payload) else None
    json_body = [
        {
            'measurement': 'climate',
            'tags': {
                'mac': mac,
                'location': LOCATION_MAPPINGS[mac] if (mac in LOCATION_MAPPINGS) else mac,
                'ruuvi_data_format': dataFormat
            },
            'fields': fields
        }
    ]
    client.write_points(json_body)


RuuviTagSensor.get_datas(write_to_influxdb)

It can be run similarly to Ruby

[Unit]
Description=Ruuvi Tag logging service

[Service]
Type=simple
Restart=always
RestartSec=1
User=root
ExecStart=/usr/bin/env python3 /opt/ruuvi/ruuvilog.py

[Install]
WantedBy=multi-user.target

Conclusion

And that’s that! Hindsight is 20/20, but I should have started with Raspibian from the beginning - Alpine Linux took a fair bit of time to tinker with. It did arose my interest for even smaller embedded systems though - if I set up a decent cross-compiling environment and have an use-case that truly doesn’t require much in way of auxillary software (like this does!), Alpine with its diskless-first design could be a very viable choice. Time will tell.. 👀


© Arttu Ylä-Sahra 2016-2021
Page rendered on 2021-08-30T00:16:54.129+00:00