I notice that nowhere on these forums has anyone mentioned IPFS. It's a distributed storage protocol. Obscure, and still un development, but I think it's worth looking in to because it would be the perfect storage backend for a piratebox. I know that mesh development has been proceeding with the pace of a lazy snail, but IPFS could make it easier.
It works by storing all data in a single, consistent content-addressed structure. With IPFS, a file added into a piratebox need only be thrown into the IPFS software - that's it. It returns a hash for retrieving that file. If any other node on the IPFS network is then queried for that hash, it will either return the file (if it has it) or transparently query as much of the network as it can practically reach to see if any other nodes have it, and then return the file. Internally everything is broken into blocks for ease of management and retrieval.
It's perfect for what piratebox needs: Insert a file to get a hash, supply a hash to get a file. No need for piratebox devs to worry about how it works - it's enough just to establish any communications channel at all upon which IPFS can communicate with other nodes.
Thoughts, anyone? I'm just about to set up a testing piratebox myself (I used to have one years ago, but things have advanced since then), and start hacking IPFS into it to see how it handles.
The hacking in of IPFS is actually pretty easy.
1. Install IPFS.
2. Add two lines to the web server config and add mod_proxy.
3. Adjust IPFS config to enable writing.
4. Configure iptables to allow mdns discovery.
5. Add to the HTML greeting page a notice explaining how to use it, and a link to Qmd2jKm99A2pJucn58xg5nGVRtUuNgSpiZ3jjdL39Bw95V.
My piratebox is now functional - mostly a roll-your-own rather than using the 'official' software here. I just need to order a few more components to make it portable. IPFS is able to handle all file upload and storage functions in a very convenient and practical manner.
No need for scripts to handle file transfer: IPFS looks like it could be a very much viable alternative to Droopy. Better than Droopy - it doesn't just let users upload files, it can also handle file exchange on a mesh too - serving, searching and retrieval.
It can't do the message board, though.
Edited 1 time(s). Last edit at 05/20/2018 12:55PM by SuriRaven.
Sounds amazingly easy to me. What kind of mdns does it do? Does it IPv6 mdns as well?
Our mesh interface is IPv6 only.
Do you have found any information about an OpenWrt build of the IPFS client?
edit:
I assume there is no port to OpenWrt and something run-able on those small embedded devices.
But I am pro to implemented the IPFS feature for RPi based PirateBox. I am not sure, if that should be more like an opt-in or an opt-out (enable normal upload again).
In general, I wrote a php based upload to get away from droopy on the a feature branch. There is still some work to be done with themeing :-(
Edited 2 time(s). Last edit at 05/20/2018 04:15PM by Matthias.
It's certainly IPv6 capable, so I can assume it does mdns on IPFS. If not, it should be easy to make it do so. There's an ARM binary that works on my pi, but chances are you'll have to compile it for OpenWrt - it's written in Go. The Pi runs it handily, but I did have to tweak the configuration a bit to lower CPU usage.
Well, that look long enough to repeat and document! My piratebox is a custom job, not one of the images provided, so this is all specific to me - but you should be able to adapt it.
-----
Instructions for converting your piratebox to use IPFS.
1. Repartition the drive (optional).
This is not actually required, but in the case of a piratebox it is wise to separate the IPFS data from the OS. This is to reduce the possibility of serious data corruption in the event of an unexpected loss of power - which, in many pirateboxes, is the most common way of turning off. Actually you've still a good chance of screwing IPFS up with unclean shutdowns, so... don't do that.
So just shrink your OS down to as low as you feel comfortable, and make the rest a bit ext4 partition. Personally I make this my /home/ partition, but you can put it elsewhere and link the .ipfs directory to it.
For those pirateboxes which use a separate device for storage due to limited internal flash memory, this step is obviously not required.
2. Install IPFS.
Detailed instructions will not be provided here. Simply install IPFS as you would on any other computer, and initialise it for whichever user will be running the IPFS service.
3. Configure IPFS.
The default IPFS configuration is quite resource-intensive. Pirateboxes are a resource- and power-constrained environment. They may or may not be connected to the internet, and may or may not be connected to other physically nearby IPFS nodes either as part of a mesh or as client devices. IPFS needs a few small configuration changes to operate optimally under these conditions. So open up ~/.ipfs/config and make the changes:
StorageMax needs to be a little bit less than the available space on your IPFS storge partition.
Under "Access-Control-Allow-Methods" you see only 'GET' by default. Change to 'GET PUT POST' in order to enable write methods. This will allow clients to upload files into IPFS.
Likewise, change the "Writable" field to "true" - those two settings are both needed.
The next two changes are to reduce resource usage, and are only required if you have CPU, memory or power constraints. Which most pirateboxes do.
"Routing" defaults to type "dht". You want "dhtclient"
Finally, look for the "ConnMgr" field - this one sets how many connections IPFS tries to maintain to the network. If your node has no internet connection that won't mean much, but if it does the reducing 'HighWater' and 'LowWater' will greatly reduce resouce usage (at the expense of making it harder to retrieve blocks). I use 100 and 150 respectively.
4. Apache
I use apache on my piratebox, so you will need to change this procedure entirely if you are using another web server. In order for both IPFS and the webserver to use the same port, you need to configure the web server to forward traffic into the /ipfs/ and /ipns/ directories to the go-ipfs daemon as if it were a proxy server. So first, enable the apache module mod_proxy - and don't forget proxy_http goes with it. Then add two lines to httpd.conf or apache2.conf, whichever your disto uses:
ProxyPass "/ipfs" "[127.0.0.1];
ProxyPass "/ipns" "[127.0.0.1];
Note the use of 127.0.0.1 rather than localhost. This matters. By default IPFS only binds the gateway to 127.0.0.1, while localhost will resolve to ::1 - if you use localhost you get ipv6 connectons, you must have ipv4 for this. At least until the IPFS people change the behavior.
5. Firewall.
Almost done! Only three steps left. Configure your firewall so that the interface to which clients connect will allow traffic on UDP port 5353 (IPFS's MDNS discovery) and TCP 4001 (IPFS connections). Don't forget to allow these on IPv6 as well! These are in addition to the usual piratebox requirements of 53, 67/68 and 80.
6. IPFS on startup.
Add to rc.local: su <user> -c "ipfs daemon" &
<user> is whichever used account you have running go-ipfs. If you are using a Pi with rasperian, that's usually the 'pi' user.
7. Add to the page.
Finally, edit your piratebox's HTML page, the part users actually see. Remove any references to the Droopy horror, and replace with a link to the IPFS file uploading utility. This utility is actually stored within IPFS itsself:
/ipfs/Qmd2jKm99A2pJucn58xg5nGVRtUuNgSpiZ3jjdL39Bw95V/
If your piratebox has no internet connectivity at all, you'll need to manually download that on an internet-connected IPFS node and copy it across, then put it in. But if you do have internet connectivity, simply accessing that address will cause IPFS to search the network for a source and retrieve it.