URLwatch qrz.com Pages

For a while I used urlwatch to keep me up to date with some satellite grid activators and QO-100 DXpeditions by their qrz.com pages. The drawback about qrz.com pages is that the content you are looking for is provided base64 encoded. Due to this it is not easy to determine changes between page revisions as it is with plain HTML or text. Luckily there is an option to configure a custom diff tool to analyze the changes (see [1]).

So a little bash script has been made the firstly cuts out the relevant line from the page source which contains the base64 encoded content. This is then base64 decoded before being passed to a standard Unix diff. This way the real changes of the qrz.com page is detected.

The source for the script is:

#!/bin/bash
OLD=$(cat "$1" | grep "jQuery('.action-render-').contents().find('#biodata').html" | sed -e "s/^\s*jQuery('\.action-render-').contents()\.find('#biodata').html( Base64\.decode(\"//" | sed -e "s/\") );$//" | base64 -d)
NEW=$(cat "$2" | grep "jQuery('.action-render-').contents().find('#biodata').html" | sed -e "s/^\s*jQuery('\.action-render-').contents()\.find('#biodata').html( Base64\.decode(\"//" | sed -e "s/\") );$//" | base64 -d)
diff -purN <(echo "$OLD") <( echo "$NEW")
exit 0

Of course this depends on the HTML source of qrz.com pages and may need to be adapted in the future. But for now it works and shows changes made by the user.

References

[1] https://urlwatch.readthedocs.io/en/latest/advanced.html#using-word-based-differences