Error: no such device :ostree Endless 3.6.3

[worldpossible]

Hi all -

I’m seeing this screen at boot which I see has been reported a few times on the forums already. I thought I should flag that this occurred very recently on a machine that was running 3.6.3 (unclear to me if it was upgraded to 3.6.4 or 3.7.0 before this happened).

Are there any newer instructions on how to solve this?

[Daniel]

This means that the grub bootloader is starting to run, but is unable to find the endless partition.

This problem could be caused by some kind of disk failure causing data corruption, which unfortunately can happen unexpectedly from time to time.

It could also be caused by a software bug, but I don’t think we have any current understanding of why that would be the case (or what the exact recovery steps would be).

If you need a quick fix (and don’t mind about user data) then reinstalling Endless is your quick way out.

If you have time to help diagnose any potential software bug causing this (& it would be great to have your help here), the next step would be to boot an Endless USB disk, open the disks app, click on the internal hard disk, and post a screenshot.
If you can also identify which device node corresponds to that disk (e.g. /dev/sda), it would be useful to see the output of the Terminal command e.g. sudo fdisk -l /dev/sda.

Alternatively, if convenient and you are OK with us temporarily having remote access to the computer, we could coordinate a remote debugging session with an Endless developer.

[worldpossible]

Thanks @Daniel –

I’ve added some screen shots here:

I’m also happy to do a remote session with an EOS team member or ship the laptop somewhere if it was going to be helpful.

Best,
Jeremy

[egrath]

Theory

At first we need a little lession on how a current Linux system boots with UEFI:

There are normally two or three partitions on your drive:

• The EFI Boot partition, which contains the Bootloader (in our case GRUB-UEFI)
• (optionally) BIOS Boot partition
• The Root-Partition, which contains your installation, including the Kernel to boot from, all the system files and in case of EOS, also your Homedirectory

Secondly, we need to know how a partition can be accessed/identified:

• By it’s device path, e.g. /dev/sdXY, where X is the device number and Y the partition number. /dev/sda3 is the root partition in most cases
• By a UUID (example: e5189710-f245-11e9-aaef-0800200c9a66), which is generated during the filesystem creation
• By a Label (example: ostree) which is generated during the filesystem creation

When GRUB gets installed, the identifier of the partition containing the root filesystem is embedded into the Bootloader executable. When this identifier is for some reasons no longer available, the message “Error: no such device: XXXXXXXX” is displayed when booting.

EOS uses a Label as the identifier for booting wit GRUB.

How to fix it

Boot with a EOS Live CD, open a Terminal and make yourself root:

sudo bash

Then we need to make sure, that the filesystem is clean, by issuing:

e2fsck -f /dev/sda3

As the last step, we rewrite the Label of the root Partition:

tune2fs -L ostree /dev/sda3

Make sure that /dev/sda3 is your root partition, by issuing a blkid after making yourself root.

[worldpossible]

Thanks @egrath - before I fix it, @Daniel asked me to grab a few image chunks to review. I’m sending him those and then will test your fix.

[Daniel]

If you can, please hold off on that experiment for now, and leave the system in broken state while I analyse further. I’d like to try and find the root cause and see how how we can prevent it from happening in more cases.

[worldpossible]

I know this is bizarre, but I came in today and booted this up to continue diagnostics and the machine booted to Endless. It had been about 3 weeks of getting the ostree error at boot and about 8 boot ups attempted over that time. The device was on a network overnight, but not booted to Endless that I am aware of. Here is the latest disks photos: