Making PirateBox more modular, better startup tool

Hey everyone,
I already twittered a bit about that a bit, but I want to point out, what I'm working on. While reviewing the implementation of the mesh functions, I came to the conclusion, that it is badly integrated. It read like "it should work somehow" and not like an working "in the concept idea" and while reading the code more and more for overworking it, i came to the point "That's it, you need to change the init scripts from the bottom and rewrite some parts".

What is the current rewrite about:
- Modular start & stop system
- Reduce the hook - concepts (there was the possibility to lay down a file in a folder, which gets executed on specific points during initialization, start & stop)
- Split up piratebox.conf in small configuration files
- Split up the conf folder divided by

   * conf (folder)
   |---- piratebox.conf  ; Contains path variables
   |-----* modules_conf (folder)  (Contains configuration parts)
   |     |---- hostname.conf    Contains the hostname and maybe hostname relevant sutff
   |     |---- network.conf       Contains IP Addresses
   |-----* lighttpd (folder)
         |-*- conf.d     (folder)  Contains small snippets of the lighttpd.conf, that you can easily enable,
                               disable or extend the configuration only with (re)placing a new file here.

As the restructure of the configuration files is already close to a 75% rewrite of the core functions (the magic happens there), the module system is the next step to make the box easier expendable. I created a function library, which takes care of the start and stop process and relies on the files on a folder called "modules.available".
while during the first initialisation a first set of default modules are being enabled, you can later enable and disable those easily, like

/opt/piratebox/bin/piratebox_modules.sh enable droopy
/opt/piratebox/bin/piratebox_modules.sh disable droopy

These states can be asked at the piratebox_modules.sh script:

/opt/piratebox/bin/piratebox_modules.sh enabled droopy

To create and make a module available, I created an example module, which can be used in the following steps:

  - Copy example to a new file "my_new_module"
  - Edit the file
  - find and replace the word "example" with  "my_new_module"
  - Fill out the start and stop section, and choose a point where the module should be started.
  - lay down custom configuration in conf/modules_conf

Later enable it with the commands above.
So you can add you own functions very easily without edditing or extending your code... and adding a comment, which I forgot for the example

I converted 90% of the stuff from the init.d/piratebox scripts to this new version, but it isn't already pushed to the repository.. (I forgot that yesterday evening).

The boot order is based upon the number you define in the module. Those numbers are clustered like the following post:

---

This is only my signature.



Edited 2 time(s). Last edit at 02/14/2015 11:58AM by Matthias.

New start order - numbers
Of course, the user can choose a startup number within th other ranges, but only on the reserved places, it is sure, that nothing intereferes.
As a double number doesn't overwrite the other, the naming of the service will make the order
### Init section with init hooks stays the same !!

    01 - 09   USER ONLY
    10 -----------   prepare
      11 Do cleanups
      12 Generate some config files
      12 do JSON generation
      12 shoutbox init (permissions and text)
      13 distribute constom dirslisting
    20 ------------   Network configuration
        21 do IW
        22 DO Probe interface   CRUCIAl=YES
       23 Do hostap
       24  bridge-Creation  CRUCIAl=YES
       24 DO ifconfig - v4    CRUCIAl=YES
       25 DO ifconfig - v6    CRUCIAl=YES
       29 DO bridge-ADD   
    30 --------   Client Network-Config Serving Part
       31 dnsmasq
       32 radvd
    40-49  ------------    USER ONLY
    50 ------  WWW - Content Serving
       51 DROOPY
       52 Lighttpd
       53 global chat server (broadcast shoutbox)
    70  ----  OTher services
      71 IRC Server
      72 avahi-to-sdns
    80-99  -- USER ONLY

New stop order - numbers

    00 - 09    USER ONLY
    10 ------ Disable new connections
       11  hostapd
       12  dnsmasq
       13  radvd
    20 -----
       21 global shoutbox
       21  avahi-to-sdns
       22 lighttpd
       23 droopy
    30 - 39  ---  USER ONLY
    40 ----
       41 irc_server
    70 --- rest of networking
       71 ifconfig down 
       72 bridge client remove
       73 bridge delete 
       74 iw remove
    80-99  -- USER ONLY

It follows the order we had before, but you have slots where you can add you software for start & stop of piratebox (if it should only work when piratebox is up and running).
CRUCIAL=Yes , means, that there is the flag in the module set, when the startup of that module fails, the complete further startup is canceld.

While structuring the configuration files, I figured out, that I have to rework the way how to use the interface configuration needs to be overworked. Currently everwhere wlan0 is written down by hand, but I assume there is a rule, which interface name should be used for which part of the daemon, like

Quote

When Bridge-Add (add AP-wifi to an existing bridge) is active and dnsmasq is running too, dnsmasq should operate on the bridge interface and not the AP-wifi interface

That should reduce the technical configuration. I'm still working on a good concept or a function library, that will help to identify that. BUT there will be always the option to overwrite the decision of the function with setting a configuration value!



Edited 1 time(s). Last edit at 02/14/2015 12:06PM by Matthias.

The next major change is the mesh integration. I thought a while about it...
First of all, the support for IPv6 on dnsmasq is dropped, as it worked only on notebooks and there is no reason why not install radvd in addition. I assume nobody ever used it, too.

The current situation
You have to enable ipv6, what results that each device has the <prefix>::1 ip address and then later a working radvd is distributing the IPv6s.
On OpenWrt with the mesh package enabled, an additional IPv6 address is generated set to the system configuration (from OpenWrt) and a 2nd configuration file is used to note down that IPv6 address. This IP was used for identify a single node in the mesh network in combination with a generated hostname-prefix.
So in a full working PirateBox, there were:
- two IPv6 configurations
- parts for a hostname prefix
- not easible change for stuff
- complex generation of the hostnames
- mixed up in ipv6 stuff
- you are able to setup the node interface to a different one, then the interface PirateBox is working on.

Future look alike
After the rework, there will be only one IPv6 address of the PirateBox, which is generated (through OpenWrt scripts or) locally by the install_piratebox.sh script.
There will be only one hostname - "piratebox.lan" per default, which gets changed to "<random prefix>.piratebox.lan" when you activate mesh on OpenWrt & others.
If you change the hostname, you can change it to whatever you like and you are reachable by this name though the mesh network.

That makes the configuration less complex and more straight forward... and less confusing in maintaining. If you still want an complex configuration, add you own module+configuration for setting these things up.



Edited 1 time(s). Last edit at 02/14/2015 08:45AM by Matthias.

This will change everything.
So, the forum threads might be come less useful.
LibraryBox needs alot rework, but will have benefit from it later on
Alot of documentation will become obsolete

And alot on the PirateBox OpenWrt package needs to be changed.

It is needed, because of the ongoing development of more and more "offline networks", which needs the same functions to get their hotspot and functions online.


That will be a long and hard way, ending up in a version 2.0 of PirateBox.

Good news, these sound like very needed changes and I cant wait to take a spin with them.

This reorg sounds very cool and I'm looking forward to it. Has any of this work been commited and if so what's the best repo/branch to use? Alternately, if there's a nightly build of the install zip then that would be great.

Hey,
it is commited, but not merged into the development branch, because I'm guessing that some more work on the OpenWrt integration needs to be done. You can find the code on Github PirateBoxScripts_Webserver repository, configuration_n_start_rework branch

you should be able to checkout the branch and use it, e.g. on a raspberry

Don't consider it as stable

best regards
Matthias

---

This is only my signature.