Failure to extend root - slow USB stick?

I just starting using PirateBox to create a small sharing spot for this little corner of my neighborhood. I got it going on an AR150 and it works great...but only when I use an 8GB USB drive. I bought a PNY 64GB stick and I can't get it to work. It successfully mounts (as sda, after editing etc/ext.config) and there is an fstab entry:

/dev/sda1 /mnt/usb auto umask=0,utf8=1,noatime,rw 0 0

but it looks like it comes up really late, very long after loop is loaded (like 40 seconds later).

[ 48.940000] sda: sda1

Is there a way I can insert a pause in the boot sequence after the disk is attached and before the loop module is loaded, like by adding a delay to one of the boot scripts? I'll try to do something like this, but if anyone has encountered this kind of problem I'd appreciate hearing about robust solutions. Thanks!


--------------details:

Here's dmesg excerpt from the *non-working* setup:

[ 10.060000] USB Mass Storage support registered.
[ 10.350000] usb 1-1: new high-speed USB device number 2 using ehci-platform
[ 10.500000] scsi0 : usb-storage 1-1:1.0
[ 11.500000] scsi 0:0:0:0: Direct-Access Specific STORAGE DEVICE 0009 PQ: 0 ANSI: 4
[ 11.510000] sd 0:0:0:0: [sda] 120922112 512-byte logical blocks: (61.9 GB/57.6 Gi
[ 11.520000] sd 0:0:0:0: [sda] Attached SCSI removable disk
[ 31.060000] Loading modules backported from Linux version master-2014-05-22-0-gf2032ea
###skipping stuff
[ 33.160000] loop: module loaded
###skipping stuff
[ 45.540000] device wlan0 entered promiscuous mode
[ 45.550000] br-lan: port 2(wlan0) entered forwarding state
[ 45.550000] br-lan: port 2(wlan0) entered forwarding state
[ 45.560000] ADDRCONF(NETDEV_CHANGE): wlan0: link becomes ready
[ 47.550000] br-lan: port 2(wlan0) entered forwarding state
[ 48.940000] sda: sda1

It looks like it waited for 20 seconds and then moved on, loading loop before sda was mounted at 48.94 seconds at the very end of the log. mount seems to confirm this:

rootfs on / type rootfs (rw)
/dev/root on /rom type squashfs (ro,relatime)
proc on /proc type proc (rw,noatime)
sysfs on /sys type sysfs (rw,noatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noatime,size=30836k)
tmpfs on /dev type tmpfs (rw,noatime,size=512k,mode=755)
devpts on /dev/pts type devpts (rw,noatime,mode=600)
/dev/mtdblock4 on /overlay type jffs2 (rw,noatime)
overlayfs:/overlay on / type overlayfs (rw,relatime,lowerdir=/,upperdir=/overlay)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
none on /proc/bus/usb type usbfs (rw,relatime)
/dev/sda1 on /mnt/usb type vfat (rw,noatime,fmask=0000,dmask=0000,allow_utime=0022,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)

sda1 is mounted as the very last thing. In contrast, my *working* 2GB system mounts sda almost immediately

[ 10.060000] USB Mass Storage support registered.
[ 10.360000] usb 1-1: new high-speed USB device number 2 using ehci-platform
[ 10.510000] scsi0 : usb-storage 1-1:1.0
[ 11.940000] scsi 0:0:0:0: Direct-Access PNY USB 2.0 FD 1100 PQ: 0 ANSI: 4
[ 11.950000] sd 0:0:0:0: [sda] 15950592 512-byte logical blocks: (8.16 GB/7.60 Gi
[ 11.950000] sd 0:0:0:0: [sda] Write Protect is off
[ 11.960000] sd 0:0:0:0: [sda] Mode Sense: 43 00 00 00
[ 11.960000] sd 0:0:0:0: [sda] No Caching mode page present
[ 11.970000] sd 0:0:0:0: [sda] Assuming drive cache: write through
[ 11.980000] sd 0:0:0:0: [sda] No Caching mode page present
[ 11.980000] sd 0:0:0:0: [sda] Assuming drive cache: write through
[ 11.990000] sda: sda1

and mount returns

rootfs on / type rootfs (rw)
/dev/root on /rom type squashfs (ro,relatime)
proc on /proc type proc (rw,noatime)
sysfs on /sys type sysfs (rw,noatime)
tmpfs on /tmp type tmpfs (rw,nosuid,nodev,noatime,size=30836k)
tmpfs on /dev type tmpfs (rw,noatime,size=512k,mode=755)
devpts on /dev/pts type devpts (rw,noatime,mode=600)
/dev/mtdblock4 on /overlay type jffs2 (rw,noatime)
overlayfs:/overlay on / type overlayfs (rw,relatime,lowerdir=/,upperdir=/overlay)
debugfs on /sys/kernel/debug type debugfs (rw,relatime)
/dev/sda1 on /mnt/usb type vfat (rw,noatime,fmask=0000,dmask=0000,allow_utime=0022,codepage=cp437,iocharset=iso8859-1,shortname=mixed,utf8,errors=remount-ro)
/dev/loop0 on /mnt/ext type ext4 (rw,sync,relatime,user_xattr,barrier=1,data=ordered)
/dev/loop1 on /opt/piratebox type ext4 (rw,relatime,user_xattr,barrier=1,data=ordered)
none on /proc/bus/usb type usbfs (rw,relatime)

Hi,
this is a very good summary. Thank you.

But I am still wondering what is happening! That is a crazy issue and I haven't seen such a thing before.
I have a few questions about the new 64GB usb stick:

- Can you confirm it is formatted in MBR mode?
- Did you copy over all the files from the small to the big usb stick?

--

extendRoot deligates the mount of the USB stick to the OS. We are writing configuration elements to UCI ( /etc/config/fstab ) which results into the normal fstab configuration file.
The OS usually mounts the USB stick during bootup and then the loop-filesystems are mounted during further init-phase.

This seems messed up somehow :-/

You can add the following line to /etc/init.d/ext

 ext_usbwait

add here

start() {
  # Wait for slow USB sticks
  ext_usbwait

  ext_mount usb
  # init if we have a fresh USB disk
  if [ ! -d "$ext_usbdir" ]
  then
      echo "WARNING: empty USB disc found. no mount done"
      exit 9
  fi
  ext_mount usb swap ext
}

This might help in your case... but I would prefer to find out the reason behind this issue..


best regards Matthias

Hi Matthias, it seems that the stick is not slow--I was wrong. It does have something to do with the formatting, but I can't tell what is wrong. Maybe it's a leftover superblock at the beginning that is making it look like a device without a partition, even though it has a single partition (see below).

I can start the system manually with this recipe...power up and wait for boot, then ssh in and:
mount /dev/sda /mnt/usb
#ignore any errors here!!!
/etc/init.d/ext restart
mount -o loop /mnt/usb/PirateBox/pbIMG_ws.img /opt/piratebox
/etc/init.d/piratebox start

The first line is key. If I try to do this:
mount /dev/sda1 /mnt/usb
I get a "no such file or directory" error and the USB stick is not mounted. If instead I use the "incorrect" syntax (which I guess is the syntax for no partition at all)
mount /dev/sda /mnt/usb
it returns "mount: mounting /dev/sda on /mnt/usb failed: Invalid argument" *but it actually mounts successfully as sda1* Just ignoring that error message, after that time I can put the loop devices in place and and start all of the services. Interestingly, the trick I've seen you give to set ext_usbdevice='/dev/sda' didn't work for me probably because something was getting tripped up by the error that is returned even though the mount is successful. Forcing the fs type with -t vfat doesn't help.

So all this points to a formatting error for the USB stick, but I have tried everything from using Rufus and guiformat in Win10 to partitioning/formatting 5 different ways from my linux box. The linux tools show nothing that is unusual to me, but I've never had this kind of problem before. FYI, here is the same USB device enumerated as sdc on that machine:

rob@rob-desktop:/media/rob/PBOX64GB$ sudo file -s /dev/sdc
/dev/sdc: DOS/MBR boot sector; partition 1 : ID=0xb, start-CHS (0x0,1,1), end-CHS (0x3ff,254,63), startsector 63, 120921192 sectors

rob@rob-desktop:/media/rob/PBOX64GB$ sudo file -s /dev/sdc1
/dev/sdc1: DOS/MBR boot sector, code offset 0x58+2, OEM-ID "mkfs.fat", sectors/cluster 64, reserved sectors 64, Media descriptor 0xf8, sectors/track 63, heads 255, hidden sectors 63, sectors 120921192 (volumes > 32 M, FAT (32 bit), sectors/FAT 14784, reserved 0x1, serial number 0x2c3d2226, label: "PBOX64GB "

rob@rob-desktop:/media/rob/PBOX64GB$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 931.5G 0 disk
├─sda1 8:1 0 350M 0 part
├─sda2 8:2 0 930.7G 0 part
└─sda3 8:3 0 450M 0 part
sdb 8:16 0 931.5G 0 disk
└─sdb1 8:17 0 931.5G 0 part /
sdc 8:32 1 57.7G 0 disk
└─sdc1 8:33 1 57.7G 0 part /media/rob/PBOX64GB
sr0 11:0 1 1024M 0 rom

The only thing that is a little weird is that the start sector is 63, when I am used to this being a larger number like 2048. I'm not a linux superstar and I'm sort of amazed I got this far--I probably accidentally learned something Do you know of any other tools I can try to make sure there is only the one clean partition and no leftover junk ahead of it that could trick the system? Thanks for your help!

The advice with using /dev/sda in ext.config will only help, if the box isn't already installed because of the generated fstab configuratoin entries.

My USB stick looks pretty much the same:

[matze@shrimpy ~]$ sudo file -s /dev/sdb
/dev/sdb: DOS/MBR boot sector; partition 1 : ID=0xb, active, start-CHS (0x0,1,1), end-CHS (0x150,31,63), startsector 63, 15130017 sectors
[matze@shrimpy ~]$ sudo file -s /dev/sdb1
/dev/sdb1: DOS/MBR boot sector, code offset 0x58+2, OEM-ID "MSDOS5.0", sectors/cluster 8, reserved sectors 3280, Media descriptor 0xf8, sectors/track 63, heads 255, hidden sectors 63, sectors 15130017 (volumes > 32 M, FAT (32 bit), sectors/FAT 14744, serial number 0xdeee0bc3, unlabeled

If you want to test your idea about the superblock or want to wipe out the partition table with some filesystem entries... then you can run the following command (exchange sdX with the representation on you computer like above sdc)

sudo dd if=/dev/zero of=/dev/sdX bs=1024 count=100
sudo sync

This command overwrites the firs 100kB with zeros. This wipes out all partition information.

You can setup the partition again using fstab:

echo "
o
n
p
1


t
c
w

" > /tmp/fstab.config 
sudo fstab /dev/sdX <  /tmp/fstab.config 
sudo sudo  mkfs.vfat  dev/sdX1

We will see if that helps...

best regards Matthias

Hi Matthias,

Overwriting didn't help; it still has that weird behavior after reformatting. I tried some other USB drives and they just work without any extra effort.

I think I'm going to admit defeat on this one I installed a little work-around to make it initialize properly by adding the script below to /etc/init.d and enabling it. Not as pretty as I'd like, but it works and everything comes up perfectly in standalone mode. Thank you again for all of your help, and thanks for your work on PirateBox. I like it so much, I'm going to order a few more GL-AR150s and spread them around this part of the neighborhood!

best regards,

Rob

#!/bin/sh /etc/rc.common

# checks to see if the loop devices successfully mounted and if not,
# tries to recover from failed startup that happens with some USB drives

START=94

patch_boot()	{
		set +e; mount /dev/sda /mnt/usb; set -e
		sleep 5
		/etc/init.d/ext restart
		mount -o loop /mnt/usb/PirateBox/pbIMG_ws.img /opt/piratebox
		/opt/piratebox/bin/timesave.sh /opt/piratebox/conf/piratebox.conf recover
		/etc/init.d/piratebox start
}

start() { 
	mount | grep -q loop0 || patch_boot
}

"$@"

Too bad, but try a different usb stick :-p

Do you want to invest 2 or 3 boxes to play around with development version and do some development for mesh networking ? It seems you are quite good :-)

Perhaps, but I probably will not have enough free time until the summer. My holiday break is ending, and I still have some things to do with this single node. I do want to experiment with a mesh using 2 stationary boxes in my neighborhood and one in my vehicle that will sync content when it is at home. When I start experimenting with the mesh I will probably find something I want to change Happy New Year!

Currently we do not have syncing available, but it would be helpful to have a helping hand for approaching a stable implementation.

Happy new year!