Author: wbk

Posted on

A new NAS

For my birthday I got parts for a new server/NAS, complete up to a 1-TB-SSD. After adding some HDD’s that where not in use elsewhere, the hardware is like:

  • Intel J5005, 4 single thread cores at 2GHz, give or take
  • 32GB of DDR4 memory
  • 1 TB of SSD storage
  • 11 TB of spinning storage, spread over
    • 6TB SMR
    • 4TB SMR
    • 1TB regular PMR
  • 1GBit ethernet
  • 1 PCIe 2x connection

The total box should run at 5W or less when idle, with HDD’s powered spinned down. At boot, when the disks spin up, peak usage could hit 100W, while active use with spinning disks should cost about 20W.

All is powered by a 90W power brick and a 120W PicoPSU; no software is installed yet, so actual measurements have to wait.

The server comes just in time: the current home server is running out of both disk space and SATA ports. The NAS will have multiple roles:

  • Provide space for photos and videos;
  • Provide a backup target for laptops;
  • Replace a host of ARM-powered Yunohosts
  • Provide reverse proxying for Yunohosts that will not get virtualized and Yunohosts that are in the household temporarily (while being installed or while waiting for an appointment with the host-household)
  • Domotica-related things (suse.AI, sensors, remote switching), VPN, firewall, the list goes on!

I am somewhat nervous about running backups and web servers on the same hardware, but after playing with virtualization ‘on premise’ and at hosting providers, I feel confident it is a viable combination.

As the NAS is a step up from the old one memory wise, virtualization should run fine. Still, 32GB is not a huge amount, so my first thoughts went to LXC and Docker. After diving into stories of others’ installations, I found Proxmox.

There are some caveats; the advanced options of the Debian installer are not available (option: install Debian, add Proxmox), being bound to the decisions the Proxmox team makes with regard to development of the platform and, unhelpful enough: root-login needs to be enabled.

Most things I read about Proxmox are positive. Management of storage space is flexible, LXC hosts and VM’s are easily managed and there is a large community. It goes without saying that the platform is Free/Libre Open Source software. All in all, the benefits made me prepare an installation medium.

Not knowing exactly what I want, what I need and what the result will be in the long run (software maintenance, storage options), made the installation less than straightforward.

Print this entry

Posted on

Fediverse with Nextcloud/nginX on Yunohost deel II

The result of part I was being able to find users on the same system. Nice, but not the goal. Over the past weeks I spent some evenings glancing over the problem, hoping to read a post with a nice oneliner as a solution. No such luck.

Today I spent more time than I hoped, but did find part of the solution. The result is now that the webfinger testtool returns a result:

https://webfinger.net/lookup/?resource=wbk%40online.osba.nl

returns

Request log

18:20:13 Looking up WebFinger data for acct:wbk@online.osba.nl
18:20:13 GET https://online.osba.nl/.well-known/webfinger?resource=acct%3Awbk%40online.osba.nl

JSON Resource Descriptor (JRD)

{
  "subject": "acct:wbk@online.osba.nl",
  "links": [
    {
      "rel": "self",
      "type": "application/activity+json",
      "href": "http://online.osba.nl/nextcloud/nextcloud/index.php/apps/social/@wbk"
    },
    {
      "rel": "http://ostatus.org/schema/1.0/subscribe"
    }
  ]
}

It used to return an error, 404, not found.

More details are at the Yunohost forum, but the important bit is the configuration and where to put it. The config file is the Nginx-configuration at

/etc/nginx/conf.d/online.osba.nl.conf

There are two servers defined, one on port 80 and one on port 443. The first one had a preconfigured redirect, while there was none in the secure section. That was strange, but it didn’t raise any alarms with me. It turns out that the redirect has to be available over TSL as well, and that it has to be a combined redirect/rewrite.

The combined stanza looks like:

location = /.well-known/webfinger {
  rewrite ^ https://online.osba.nl/nextcloud/public.php?service=webfinger&$1 last; # $1 will use the first parameter (?resource=…)
}

Put this bit between the include-stanza’s and the log-definitions.

Even though the test at webfinger.org succeeds, I have no such succes when contacting this Nextcloud from a Nextcloud on another Yunohost.

To be continued…

Print this entry

Posted on

Public IPv4 subnet with Fritz!Box

AVM does not write in large letters on their website that the Fritz!Boxes can handle public IP subnets. The feature is mentioned in the online help; enabling it in the first place happens via Home Network / item Network / tab Network Settings / header IP Adresses / button IPv4 Adresses / header Public IPv4 Subnet, at the bottom of the page.

And then? Then I got stuck. For evenings in a row. After some tries I got lucky at the XS4all helpline and spoke with someone knowledgeable on the subject. Managed to make a device reachable from the outside world on the public IPv4 on a single port. All happy, hung up the phone and tried another device and another port forwarding. Access denied, ‘An error occurred. Error description: A device with this name has already been configured.’

The awful thing is: not any port forwarding can be added or altered anymore. After contacting the AVM helpdesk, we found some hints of causes:

  • Renaming
  • Multiple NIC’s on one system

Renaming: that has to do with renaming the device, and later deleting the device under its new name. Some residual config stays behind under the old name. Since the new devices all came up as ‘Yunohost’ + a number initially, I would rename them to their given host name. Later the next device would come online, initially as Yunohost again. The left over config from the previous round now collides with this Yunohost.

Multiple NIC’s: AVM warned that one machine, connected to the Fritzbox via more than one cable, is not supported and will cause difficult to diagnose problems.

Even after learning some of the possible causes, the situation still has returned. I have a backup of a working configuration now, in which most machines got a basic profile that has most forwardings in place. If something runs amok, I can put that configuration back.

Assigning a public IP

To avoid the situation altogether, I don’t rename temporary devices anymore. The best cure is to be able to assign a device its public IP before connecting it to the FB for the first time. In a headless device that is not always practical.

Specifically for a temporary Yunohost with Armbian on Orange Pi Zero I now do things along these convoluted lines:

  • Just connect in any way to the FB and request an IP via DHCP
  • Leave the name as-is
  • Configure anything needed via SSH on the local IP
  • Make sure an ethernet cable as well as a serial connection are available
  • Use nmtui to configure the ethernet port for a public IP
    • Do not yet activate the new configuration
  • Disconnect the ethernet cable
  • Use the serial connection to reboot
  • Wait for the FB to move the device from active connections to idle connections in the network connections section of the home network.
  • Delete the stale configuration
    • That is only possible when the FB thinks the device is inactive
    • It has only effect when the name of the device has not been changed from what it was when it was first encountered by the FB
    • Sometimes it helps to reboot the FB, but it is not necessary and also does not always speed things up
  • Once deleted, reconnect the ethernet cable
  • After a while (network manager will find a connected cable, and activate the configuration) the host will come up in the FB under its new (public) IPv4.
    • Some chance it’s difficult to spot, because it got a new generic name, based on the IP address.
  • Now the device should be reachable on its public IPv4. Configuration of port forwarding should not pose difficulties.

Troubleshooting

Direct troubleshooting is not really an option via the web interface. AVM asked me to generate support data. After reviewing the detailed logs, I could guess the cause of the problem before sending it to the patient people at AVM.

The support data can be generated by clicking the contents at the left hand bottom, and then FRITZ!Box Support. There are two variants, ‘Generate Support Data’ and ‘Generate Advanced Support Data’. The ‘basic’ version had all that was necessary to point at the cause problem.

The problem, once introduced, is very difficult to resolve from the available functionality in the web interface. There used to be a telnet option, but that was on an older model Fritzbox, with a much earlier version of the OS, at a time when we still had a handset in the house that we could connect to the phone port of the Fritzbox. A quick lookup learns that it needed *7* typed on the connected phone to enable telnet, but lacking such a device I have no known option to enable it anymore. Options might be still there with alternative firmwares, but at the moment things run OK.

Print this entry

Posted on

Local domain name resolution with Fritz!Box

The Yunohosts running in the local network were initially unreachable due to different causes, the cause depending on whether the request was made from the local net or from the public internet.

The local case was easily worked around using the hosts file, by adding a line to /etc/hosts of each device on the local net (and suppressing ensuing headaches):

127.0.0.1      localhost.localdomain localhost
80.127.182.180 online.osba.nl

Once the headaches of trying to edit host files on Android devices promised heavier than those I prospected finding the cause in the local network, I set out finding that cause. It turned out to be DNS rebind protection in the modem/router, an up-to-date (07.01, as of writing) Fritz!Box.

An entry on Marphys blog pointed me to the setting in question:

The list was empty, of course, and after adding online.osba.nl the entries in the hosts files were not needed anymore.

Print this entry

Posted on

Not enough swap

Installing lxml (for Synapse/Matrix) gave problems on one of the Yunohosts ended in an error all the time.

The error indicated not enough memory available at some point. The machines are fairly limited in RAM, so that explains. Extra swap space to the rescue!

Armbian creates a swap partition and resizes the system partition (and the file system) to fit the uSD-card during first boot, so there is no space available to increase swap space.

Unfortunately, shrinking an ext4 file system is not as trivial as growing/extending it. It helps that the uSD-card can easily be put in another system, avoiding the need to resize a (semi-) online root partition. One bit of information made me rethink my shrink-root, grow-swap strategy: ostensibly a lot of writing might go on while resizing:

The filesystems considered here tend to disperse data throughout the underlying block device, so a large amount of copying may be needed even if the filesystem has never been filled beyond the size it is being reduced to. Do not start resize2fs unless you are confident you can allow it to finish. You can request a progress bar using the -p option.

That is not what I expected or had taken in account.

Since I do expect it to be a temporary measure, I decided to create a swap file instead of a swap partition. Arch, as usual, has a very helpful page on their wiki. There are some warnings against using swap files for other file system types especially with non-contigous swap files, but ext4 has no special mention.

The steps for creating and using a swap file:

fallocate -l 512M /swapfile && chmod 600 /swapfile && mkswap /swapfile && swapon /swapfile

I used a 1GB file, and when lxml still wouldn’t compile, I added another gig of swap.

“yunohost app install synapse” is running. If it ends in an error once more, I need to check whether the cause actually is still (out of) memory-related.


Print this entry

Posted on

Fediverse with Nextcloud on Yunohost: cloud ID @localhost :-(

The Nextcloud Social App nags about .well-known/webfinger being not set up properly:

.well-known/webfinger isn't properly set up!
Social needs the .well-known automatic discovery to be properly set up. If Nextcloud is not installed in the root of the domain, it is often the case that Nextcloud can't configure this automatically. To use Social, the admin of this Nextcloud instance needs to manually configure the .well-known redirects: Open documentation ↗

The documentation states that a line in the config is to be uncommented; since that block of configuration is not there in the nginx nextcloud.conf that Yunohost created, I added it just above the .well-known’s for carddav and caldav, ie, on top of the config.

The nextcloud instance does not run in the root of the domain, so the rewrite rule has to account for that:

# The following rule is only needed for the Social app.
# Uncomment it if you're planning to use this app.
rewrite ^/.well-known/webfinger /nextcloud/public.php?service=webfinger last;

After that things still don’t run smoothly, searching the Nextcloud forum pointed me to occ. First check the current config for the social app:

sudo -u nextcloud php ./occ config:list social
{
  "apps": {
    "social": {
      "address": "http:\/\/localhost\/index.php",
      "enabled": "yes",
      "installed_version": "0.1.4",
      "types": ""
    }
  }
}

Then update to the required domain:

sudo -u nextcloud php ./occ config:app:set social address --value="http://online.osba.nl/nextcloud/index.php"
Config value address for app social set to http://online.osba.nl/nextcloud/index.php

and check again

sudo -u nextcloud php ./occ config:list social
{
  "apps": {
    "social": {
      "address": "http:\/\/online.osba.nl\/nextcloud\/index.php",
      "enabled": "yes",
      "installed_version": "0.1.4",
      "types": ""
    }
  }
}

The warning about webfinger seems solved, but is replaced by a popup:

Failed to load account details wbk@online.osba.nl

After /var/www/nextcloud# sudo -u nextcloud php ./occ social:reset and confirming I was sure about resetting anything and everything social, I was able to ‘follow’ and ‘unfollow’ users on the same system.
Also possible, it seems, is following Nextcloud on Mastodon.xyz. No messages yet, though.
Not yet possible: following users on another nextcloud instance near here.

To be continued…

Print this entry

Posted on

Yunohost on Orange Pi Zero with Armbian

Orange Pi has a neat range of microcomputers. The smallest cost less than 10 Euro’s. For that price it is remarkably complete and powerful enough to run a suite of web services. They are easily installed and managed via Yunohost.

The hardware in short, specs copied from the excellent Sunxi-wiki:

SoC H2+
DRAM 256 MiB or 512 MiB DDR3
Power DC 5V DC-IN via µUSB or pin headers or PoE (optional)
Features
Video CVBS (on pin headers)
Audio microphone, stereo line-out on pin headers
Network 10/100Mbps Ethernet and XR819 Wi-Fi
Storage µSD
USB 1 USB 2.0 Host, 1 USB 2.0 OTG, 2 x USB 2.0 on pin headers

The 512 MB-option costs almost 25% more than the 256 MB-option, but spending two Euro’s there saves a lot of headaches later on. Depending on experience with the first handful of Zero-boards I might opt for the OPi PC later. An important factor that made me choose the OPi Zero, is that it draws little enougd power to run of regular USB and that it can be converted to run off power over ethernet. I haven’t tried that yet: it only supports passive PoE, and I am not sure my switches work with anything but 802.3af PoE.

Yunohost does not (yet) have images for the OPi zero with H2+ processor. Armbian does have an image, ready made for flashing to uSD.

I am not the only one running Armbian on this board; there are enough people downloading for that specific board to have the image appear on the Armbian front-page

Armbian advices to flash a uSD card using Etcher. Etcher is relatively heavy for the job on hand, but is a comfortable tool. On some of my computers Etcher refuses to find the uSD card because of the layout of the other disks (perhaps related to software RAID or LVM). In that case I use dd with a block size of 1 and a sync afterwards; 

dd if=/home/wbk/dl/Armbian_....img of=/dev/sdX bs=1M status=progress
1336+0 records in
1336+0 records out
1400897536 bytes (1.4 GB, 1.3 GiB) copied, 135.054 s, 10.4 MB/s

After flashing, on first boot of Armbian, change the default password from 1234 for root to something useful. On those first boots I do use a serial interface. At first I worried I might hook it up incorrectly and break something (which I did not, but have subsequently done more than once). The one thing I did check carefully (using a multimeter) was the working voltage of the serial interface of the USB2TTL / USB2serial convertor: it has to be 3.3V for the OPi Zero. At 5V something might get fried.

Running a serial connection to a headless board is a lot of fun: it seems your computer’s display (or terminal window) is the devices’ screen. Everything from the first boot messages scrolls by.

Using a serial interface saves looking for the IP of the device to get connected. So you have to think of ordering some of those convertors, but they’ll save some hassle later on.

After the first boot of Armbian, reboot (for seeing everything is OK with your new password).

Yunohost has a curl-able script for installing the Yunohost-base-system on top of Debian:

curl https://install.yunohost.org | bash

When it is done, run post install:

yunohost tools postinstall

Edit december 2019, from the Yunohost documentation: alternatively, restore a backup instead of postinstalling. Upload the archive to the server and place it in /home/yunohost.backup/archive; instead of postinstall, run:

yunohost backup restore 'backupname'

After that, get a LetsEncrypt-certificate:

yunohost domain cert-install your.domain.tld

Reading this text probably takes more time than executing those steps. I used 32G uSD cards, at least class 10, preferably with ‘App performance’ label (A1 or higher).

Print this entry