Raspberry Pi pre release testing [1.0.7]

Hello community,
I was talking to Matthias @32c3 and suggested I could take the RPI part off his hands.
Yesterday I have been working on improving the build scripts and I am at a point where it is automated in a way that you just need to invoke one make target, lean back and wait for the image to come out ready to be dumped on a SD card.

Right now I only have an RPI 1 B+ on my hands for testing. I am trying to get hold of a RPI2(or find one at home that is not used) and Zero - although the later may take some time to acquire.

My roadmap for the next few weeks looks like so:

* Finalize the build process of the RPi1 image [DONE]
* Fine tune the build process for RPi2 image [DONE]
* Adapt build process for RPi zero image [DONE]
* Update documentation for all RPi variants [DONE]
* WiFi autodetection [DONE]
* Establish testing checklist [DONE]
* Fix timesave [DONE]
* Fix minidlna permissions [DONE]
* Allow building from a dev branch [DONE]
* Use WPA_supplicant.conf to connect to host network in case no AP capable USB WiFi could be found [DONE]
* Testing, testing, testing

I will also try to hang out more in IRC and the RPI section of the board.

I am also looking for two testers each for all of the images. Special bonus for all testers is a mention in the RPI's projects README on github.

So hands up, who is willing to help out with testing in the near future? Testers will download the image, dump it, run it and follow a checklist to make sure everything is working as it should.

RPI 1

Seksan2
stylesuxx

RPI 2

stylesuxx
TBA

RPI zero

TheExpertNoob
TBA

Edited 9 time(s). Last edit at 01/26/2016 06:43PM by stylesuxx.

C'mon guys, testing is fun and quality assurance for the whole community. I am trying to get hold of an RPi2 and will add myself as 2nd tester for RPi1 and RPi2 if I can get hold of it.

Still for the zero I will need someone for testing. It was impossible for me to get hold of one of those devices. Also willing to take a zero in form of a donation if that is how you are willing to help.

Checklist draft may be viewed here => [github.com] let me know if I missed something.

I have a Pi Zero next to my computer with a console cable attached ready for testing. PM me? I also have a Pi B(not-a-plus) My Pi 2 is sort of my main NAS server and cant really be used for testing at the moment (as its shared among the node/neighbors for "Media" sharing)

Edit: didn't see the Github link. Testing... will report back my testings.



Edited 1 time(s). Last edit at 01/21/2016 04:32PM by TheExpertNoob.

This is just a checklist for future tests. Since I was updating build scripts and Stuff. Before we do ne next release we will do a testing round, so if you could join us with the zero, that would be awesome.

I will join with the zero.

Edit: I had to boot the image on my PiB to install wpa_supplicant.

pacman -U wpa_supplicant

# nano /etc/wpa_supplicant.conf

network={
        ssid="My SSID"
        psk="mypasskey"
}

DONE FROM SERIAL CONSOLE
# ifconfig wlan0 down
# pkill dnsmasq
# pkill hostapd
# pkill lighttpd
# wpa_supplicant -iwlan0 -c/etc/wpa_supplicant.conf -B -D wext
# dhcpcd
(ping 8.8.8.8) to test if you wish

***Do internet stuff***

# ifconfig wlan0 down
# pkill wpa_supplicant
# systemctl restart piratebox

***Back to AP Mode***



Edited 7 time(s). Last edit at 01/22/2016 05:41AM by TheExpertNoob.

I had to do this every boot to get things to work. I'm not sure why

# pkill lighttpd
# pkill hostapd
# pkill dnsmasq
# systemctl restart piratebox

And the Checklist so far.

## Checklist for Pi Zero
* [Y] SSH connection to PirateBox with the username *alarm* and the password *alarm* could be established
* [Y] Message of the day containing information about *First Steps* is displayed correctly
* [Y] Change the password for the *alarm* user, log out and log back in
* [Y] Start the PirateBox (after the above steps)
* [Y] PirateBox' WiFi is available
* [Y] Connection to PirateBox' WiFi could be established (my phone could, PC had some trouble)
* [Y] It is possible to post to the shoutbox
* [Y] It is possible to upload files to the PirateBox
* [Y] Enable Kareha Image and Discussion Board
* [Y] Menu entry for the board is available in the Menu
* [Y] It is possible to post to the board
* [Y] Set some date and enable Fake-timeservice
* [N/A] Enable UPnP media server
* [N] Enable PirateBox on startup
* [Y] Reboot (but have to perform above steps to configure properly)
* [Redundant?] PirateBox' WiFi is available
* [Redundant?] Connection to PirateBox' WiFi could be established
* [Y] Date matches the set date from the Fake-timeservice

Edit: After reviewing systemctl list-units, it appears my timesave.service failed upon boot, which may be why the rest of piratebox wont load on boot. Looking into it.
Log



Edited 3 time(s). Last edit at 01/22/2016 05:15AM by TheExpertNoob.

Can you check if the default units for dnsmasq & hostap are disabled?
That may cause the problem.

The steps aren't redundant, they are for double check, if after a reboot everything is still working as expected- you are doing a (re)configuration of the box, you know

I think, adding wpa_supplicant is a good idea, especially useful for RPi-1 A & Zero

---

This is only my signature.

Alright thank you for the feedback TheExpertNoob.
I added wpa_supplicant to the dependencies and it is now installed when the image is built.

I was wondering if we should add a /etc/wpa_supplicant.conf with a default network name and password, so peopble would only need to change the ESSID of their local WiFi and would be able to connect to the piratebox initially. (on the other hand, people with a Zero will most probably know how to connect to their board,...)

Regarding the timesave service, is the box coming up properly if you disable the timesave service at startup?
Will look into this later today.

@Seksan2 We yesterday hat a spontaneous testing round in IRC. TheExpertNoob built an image from the current dev branch. You can either do the same or I will build one and upload it, let me know which way you want to go. I could use some feedback on the build script itself so if you would build it yourself I would greatly appreciate it.

Matthias,
how do I go about checking/disabling the default units?
Edit: easy as pie, derp.
I assumed piratebox used the default units but used/implemented its own config.

I'm sort of new to Arch, learning as I go.

Stylesuxx,
A default /etc/WPA_supplicant.conf would make it easier.

I don't know if putting it on the boot partition would be easier for the end user to edit it, mount /dev/boot to /boot on startup, and have WPA_supplicant pull config from /boot/WPA_supplicant.conf (fat 32 partition easliy readable on all systems to edit the file before first boot this way, a console cable may not be needed) As most users will probably be windows users.
Also, a script to "toggle" between WPA and hostapd modes for the zero.


Once I figure out how to disable services on boot, I'll report back.



Edited 2 time(s). Last edit at 01/22/2016 12:57PM by TheExpertNoob.

This is on a failed piratebox startup from no power:

# systemctl status hostapd
* hostapd.service - Hostapd IEEE 802.11 AP, IEEE 802.1X/WPA/WPA2/EAP/RADIUS Authenticator
   Loaded: loaded (/usr/lib/systemd/system/hostapd.service; disabled; vendor preset: disabled)
   Active: inactive (dead)

# systemctl status dnsmasq
* dnsmasq.service - A lightweight DHCP and caching DNS server
   Loaded: loaded (/usr/lib/systemd/system/dnsmasq.service; disabled; vendor preset: disabled)
   Active: inactive (dead)
     Docs: man:dnsmasq(8)

# systemctl status piratebox -l
* piratebox.service - PirateBox Service
   Loaded: loaded (/etc/systemd/system/piratebox.service; enabled; vendor preset: disabled)
   Active: active (exited) (Result: exit-code) since Fri 2016-01-22 05:23:43 UTC; 8min ago
  Process: 134 ExecStart=/opt/piratebox/init.d/piratebox_alt start (code=exited, status=255)
 Main PID: 134 (code=exited, status=255)
   CGroup: /system.slice/piratebox.service

Jan 22 05:23:45 alarmpi piratebox_alt[134]: Copy over design files
Jan 22 05:23:46 alarmpi piratebox_alt[134]: Probe wlan
Jan 22 05:23:46 alarmpi piratebox_alt[134]: Starting hostap...
Jan 22 05:23:46 alarmpi piratebox_alt[134]: 0
Jan 22 05:23:46 alarmpi piratebox_alt[134]: Setting up wlan
Jan 22 05:23:46 alarmpi piratebox_alt[134]: Bringing up wifi interface wlan0
Jan 22 05:23:46 alarmpi piratebox_alt[134]: wlan0: ERROR while getting interface flags: No such device
Jan 22 05:23:46 alarmpi piratebox_alt[134]: ..failed
Jan 22 05:23:46 alarmpi systemd[1]: piratebox.service: Main process exited, code=exited, status=255/n/a
Jan 22 05:23:46 alarmpi piratebox_alt[134]: failed setting up Interface

I do have probing enabled. but id doesn't seem to take in to account the time to probe. Its defaulted to 10 seconds and I increased to 30 seconds. hmm....

Just did a reboot and everything worked just fine! Piratebox even started without errors. But it doesn't work every time.

After performing systemctl restart piratebox

systemctl status piratebox -l
* piratebox.service - PirateBox Service
   Loaded: loaded (/etc/systemd/system/piratebox.service; enabled; vendor preset: disabled)
   Active: active (exited) since Fri 2016-01-22 05:24:14 UTC; 3min 17s ago
  Process: 196 ExecStop=/opt/piratebox/init.d/piratebox_alt stop (code=exited, status=0/SUCCESS)
  Process: 208 ExecStart=/opt/piratebox/init.d/piratebox_alt start (code=exited, status=0/SUCCESS)
 Main PID: 208 (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/piratebox.service
           |-224 /usr/sbin/hostapd /opt/piratebox/conf/hostapd.conf
           |-230 /usr/sbin/dnsmasq -x /var/run/piratebox_dnsmasq.pid -C /opt/piratebox/conf/dnsmasq_generated.conf
           |-237 python /opt/piratebox/bin/droopy -d /opt/piratebox/share/Shared -m  --chmod 755 8080
           `-249 /usr/sbin/lighttpd -f /opt/piratebox/conf/lighttpd/lighttpd.conf

Jan 22 05:24:15 alarmpi piratebox_alt[208]: 0
Jan 22 05:24:16 alarmpi piratebox_alt[208]: Generated HTML-Shoutbox File.
Jan 22 05:24:16 alarmpi piratebox_alt[208]: Starting lighttpd...
Jan 22 05:24:16 alarmpi piratebox_alt[208]: 0
Jan 22 05:24:29 alarmpi hostapd[224]: wlan0: STA 58:6d:8f:ee:42:88 IEEE 802.11: associated
Jan 22 05:24:29 alarmpi hostapd[224]: wlan0: STA 58:6d:8f:ee:42:88 RADIUS: starting accounting session 56A1BCFF-00000000
Jan 22 05:24:45 alarmpi hostapd[224]: wlan0: STA 58:6d:8f:ee:42:88 IEEE 802.11: disassociated
Jan 22 05:25:17 alarmpi hostapd[224]: wlan0: STA f8:cf:c5:01:3c:f4 IEEE 802.11: associated
Jan 22 05:25:17 alarmpi hostapd[224]: wlan0: STA f8:cf:c5:01:3c:f4 RADIUS: starting accounting session 56A1BCFF-00000001
Jan 22 05:25:17 alarmpi hostapd[224]: wlan0: STA f8:cf:c5:01:3c:f4 IEEE 802.11: disassociated

I assume this is the culprit and have no idea how to fix it. It loads even after the login promt appears.

alarmpi login: [   15.540133] usbcore: registered new interface driver rtl8192cu

Edited 5 time(s). Last edit at 01/22/2016 01:11PM by TheExpertNoob.

TheExpertNoob Wrote:
-------------------------------------------------------
> I don't know if putting it on the boot partition
> would be easier for the end user to edit it, mount
> /dev/boot to /boot on startup, and have
> WPA_supplicant pull config from
> /boot/WPA_supplicant.conf (fat 32 partition easliy
> readable on all systems to edit the file before
> first boot this way, a console cable may not be
> needed) As most users will probably be windows
> users.
> Also, a script to "toggle" between WPA and hostapd
> modes for the zero.

This is a great idea. Will implement it like this. Need to find a work around for qemu tho, since it is complaining about entries in /etc/fstab, that is why I "cleared" them ;-)

Your WiFi stick is registered after the init scripts run - that is a Problem.
I *think* we can solve this with a udev rule - but this means piratebox is not allowed to be enabled as a service at bootup but rather triggered by the udev rule when the driver is ready - don't know how to do that for *all* WiFi sticks with a single rule :-/

Other, ultra hacky way would be a wrapper script for the piratebox init. It would need to check every n seconds if the wifi device is available and only then run the init script.

TheExpertNoob thank you for all the logs, this helps a lot :-) Almost feels if I was working on a Zero myself.

Github link is dead. Someone changed the branch name. Also I can't sync my clone anymore. N.P., I can re-download.

Started another branch => [github.com]

Another random question..

What is the proper dd count for this image?

Im using:

sudo dd if=/dev/sdb bs=2048 count=1048576 | pv | sudo dd of=testpiratebox-rpi.img bs=2048 count=1048576

for backup purposes. I don't need to backup the whole 8G!

bs=2048 count=1048576

2048 * 1048576 = 2147483648 Byte -> 2097152 KB -> 2048 MB -> 2GB

You are of course right, it still is bigger than it needs to be, but if it is zipped it is ~ 500MB or so. We could of course shrink the partition to it's needed size + some spare.

edit: I misread your post. Yes 2GB is the right size.



Edited 2 time(s). Last edit at 01/22/2016 02:44PM by stylesuxx.

Quote
stylesuxx
This is a great idea. Will implement it like this. Need to find a work around for qemu tho, since it is complaining about entries in /etc/fstab, that is why I "cleared" them ;-)

I don't know what was in fstab, but have you tried the nofail option?

Adding the nofail option, fixed the qemu boot problem - thank you.

I also added a helper script to recognize the WiFi sticks we have support for, install the appropriate hostapd package and adapt the configuration.

This is triggered by a udev rule that triggers when a wlan0 gets available, if everything went well it will start the PirateBox service.

I added it to the build process but did not check if the build is actually working - I'll leave that for tomorrow :-)

hostapd configs dont have a driver listed.

should be:

driver=rtl871xdrv

for both r8188eu.conf and rtl8192cu.conf
at least I think.. We can test it out later.

Edit: tested, wasn't needed for me. So far looks promising! It booted straight up from a fresh build with no errors and straight into piratebox. Running MOTD steps now.

May need this in MOTD to activate the board after setting the board password.

Language: HTML
$ wget http://127.0.0.1/board/kareha.pl

Edited 2 time(s). Last edit at 01/23/2016 08:06AM by TheExpertNoob.

Tested
SSH log
Serial Log

Only followed MOTD and added the wget as second part to step 6
I personally started at step 5 and enabled minidlna this time around.

errors with timesave and minidlna on reboot.

Note: didn't need to use wpa_supplicant this time around.



Edited 3 time(s). Last edit at 01/23/2016 08:35AM by TheExpertNoob.

I think the *cus and *eu sticks do not need the driver in there to work properly, I am trying with my different sticks to verify.

Timesave said when failing:

Nov 27 18:07:31 alarmpi timesave.sh[147]: Sorry, changing timebackward via timesave is not possible

Did you set a date in the past?

I need to edit the MOTD, since "The enable piratebox on startup" is not applicable anymore. Also I maybe should take out "Start piratebox for first time" since it is either started because it has found matching USB WiFi or it will not start anyway.

Oh and mindlna seems to have problems opening its database, need to look into this, at least sqlite seems to be present,...

Edit:
Just confirmed with all 3 WiFi Sticks, the configs work as they are right now.
Also we need to slightly change the timesave functionality I guess. At least do a little bit more do get it working..

sudo timedatectl set-ntp false
sudo timedatectl set-time "2023-05-23 17:42:00"

To disable sync and set a time in arch, but then it still does not work.
Problem seems to be the comparision with current date and saved date. $TIMESAVE_FORMAT does not seem to be defined,..

Draft to new MOTD => [paste.bubblesort.me]



Edited 5 time(s). Last edit at 01/23/2016 04:29PM by stylesuxx.

Things are making a fast progress. Wow, thank you for your time & work.

Quote

how do I go about checking/disabling the default units?
Edit: easy as pie, derp.
I assumed piratebox used the default units but used/implemented its own config.

I'm sort of new to Arch, learning as I go.

Stylesuxx,
A default /etc/WPA_supplicant.conf would make it easier.

VERY Nice idea about the WPA_supplicant.conf
Actually, PirateBox "breaks" the advantages of systemd for being a stand alone script package. :-(
The basic concept of PirateBoxScripts on Laptop was to not harm the host-system too much in terms of configuration changes.


About the TIMESAVE stuff, you find the documentation at the commit & code comments: [github.com]

---

This is only my signature.

OK, found where you are setting the format, only problem seems to be, that setting the date with the openwrt format does not work with arch:

# Does not work
date -s"201705261113"
date: invalid date '201705261113'

# Works, but not comparable
date -s"2017-05-26 11:13"
Fri May 26 11:13:00 UTC 2017

# Works, but not comparable
date -s"20170526 1113"
Fri May 26 11:13:00 UTC 2017

# Works, comparable, but not exact
sudo date -s"20170526"
Fri May 26 00:00:00 UTC 2017

So I will set the timeformat to "+%C%g%m%d", this is not perfect but at least it works to some degree.

Minidlna had a problem because it is not allowed to write to /opt/piratebox/tmp, I now gave that dir 777, that fixes the problem.
Timesave and minidlna fixes are pushed to the testing branch.



Edited 1 time(s). Last edit at 01/24/2016 01:03AM by stylesuxx.

I won't be able to test again till Tuesday. Those 12-hour work shifts are all work and sleep for three days.

@timesave:
ahhh! The problem was not setting the time but a script error during comparison, fuck, I didn't test that. :-/
Please consider fixing PirateBoxScripts beside a RPi-buildscript fix.

The minidlna problem: Change the User in the service file those variables from the piratebox.conf file

LIGHTTPD_USER=nobody
LIGHTTPD_GROUP=nogroup

This is the only reliable way, that

/opt/piratebox/tmp
and access to the shared files

will work.

Matthias

Adjusted the timesave script so it can handle date time strings with spaces, as needed for arch. This *should* not break any other functionality.

stylesuxx Wrote:
-------------------------------------------------------
> Alright thank you for the feedback TheExpertNoob.
> I added wpa_supplicant to the dependencies and it
> is now installed when the image is built.
>
> I was wondering if we should add a
> /etc/wpa_supplicant.conf with a default network
> name and password, so peopble would only need to
> change the ESSID of their local WiFi and would be
> able to connect to the piratebox initially. (on
> the other hand, people with a Zero will most
> probably know how to connect to their board,...)
>
> Regarding the timesave service, is the box coming
> up properly if you disable the timesave service at
> startup?
> Will look into this later today.
>
> @Seksan2 We yesterday hat a spontaneous testing
> round in IRC. TheExpertNoob built an image from
> the current dev branch. You can either do the same
> or I will build one and upload it, let me know
> which way you want to go. I could use some
> feedback on the build script itself so if you
> would build it yourself I would greatly appreciate
> it.

Im unfortunately not in a position to build one myself so it would be great if you made that one for me!

MOTD

Still needs

Language: HTML
wget http://127.0.0.1/board/kareha.pl

at step 3.5

also makefile needs to point to it if we are still building from the feature/test-adaptations branch... or just wait till the final release? Anyway, current makefile points to the old MOTD still with the old timesave stuff. I had to dig to find it.

@Seksan2... PMed you the current build of feature/test-adaptations branch.



Edited 2 time(s). Last edit at 01/25/2016 02:57PM by TheExpertNoob.

Thanks.

Yes, I am currently working on a feature to pass the dev dependencies to the Makefile - Should be done today, can't promise this though.

Also I will upload images as soon as I am done with the above.

Can you briefly explain why wgetting the file is necessary?

Because visiting the forum without doing so just displays the root content of files of the forum/board.

Same as step 4 of post-install instructions.

Language: HTML
Point your browser to http://piratebox.lan/board/kareha.pl for the first initialization.

Edit: I hate bbcode formatting.

Edit2: haven't tested NOT doing this on today's build. I'm still getting errors on time save, probably my fault. Hence why I went digging for the new MOTD for the new date format.



Edited 4 time(s). Last edit at 01/25/2016 07:42PM by TheExpertNoob.

No not your fault. I just pushed the new changes. You can now build an image with the piratebox webserver scripts from a branch of your choosing. To use the current release candidate branch run the build like so:

make BRANCH="release-1.0.7"

Which will pull the git repo, checkout the branch, build the piratebox package, and use this package and all the scripts from the checked out branch. If you do not provide a branch it defaults to master.

I will add the steps for the board to the MOTD.

Timesave should be working if you build like that.
minidlna also starts up - now I only need to find out how to access it to confirm it is indeed working :-)

edit:
I just checked out the imageboard installation, the board installer does as a last step already wget that file, if I wget it manually after enabling and setting the password, I still need to klick the link on the page to enable it.

edit 2:
It does this with a wrong parameter though, -s is not recognized on arch. I'll change it in the script directly to

wget -qO- [127.0.0.1] &> /dev/null

... just pushed it.



Edited 5 time(s). Last edit at 01/25/2016 09:38PM by stylesuxx.