Xterm shows in 3.10

I noticed (as I do) that xterm and uxterm now show up in the launcher (they don’t in 3.9). I would say that the xterm package could just be removed but it seems some Endless packages depend on it namely:
eos-core eos-core-amd64 eos-factory-tools

While thinking about packages to remove…
ifupdown - networkmanager should really cover this
ntp - at least the default config could be handled easily by the systemd-timesyncd (although I see it’s been removed)

Thanks for your input here! xterm is still used by some factory-support tools we have, so it is staying around at least for now, but we are looking at ways to exclude it’s desktop icon like we did before.

ifupdown is priority important, like rsyslog & logrotate, not planning to take action against that at this time… but would welcome a Debian-upstream discussion on them changing this!

We should probably re-examine migrating ntp to systemd-timesyncd as you suggest… When we looked before (years ago) it did not work as well as ntp for the case where you are not using systemd-networkd for networking.

Will look into the Debian end about those not being priority important.

I’d also be happy to see if they could be removed from the image (or not included) - is there a more recent way to build an endless OS like image then https://github.com/dbnicholson/deb-ostree-builder ? (that’s what ostree repo links to)

systemd-timesyncd
My quick check (XFCE builds) seems to indicate that debian11 defaults to using systemd-timesyncd. Will check again with a fresh build.

If using debootstrap doing --variant=minbase would just exclude all “important” packages and just install the required set.

Yes a minimal debian bullseye image used systemd-timesyncd. Oddly enough it does not include dbus which means querying using timedatectl doesn’t work with a bus error.

It’s too late to look at changing important level packages for bullseye - will look into it for 12.

We use mmdebstrap but there is probably a similar option there.

We are in the process of opening up our tools etc, but we’re still some distance away from publishing the ostree builder. However, we are trying to align with Debian as much as possible, so we might not take any kind of whitelist or blocklist approach even if its quite simple to achieve, unless there’s some tangible link to achieving our mission of increasing access to technology & education.

But there may be an interesting angle of can we drop the installation of all Priority important packages? as you suggest. Would this cause some important stuff to be dropped? I would imagine so, but I admit to not having checked, I’m also not familiar with the guidelines/policy around how Debian tags packages like this.

Indeed mmdebstrap does have equivalent --variant options.

I generated two examples and the default here. Anything that really is required should be a dependency that gets brought in anyway. Checked systemd and dmidecode and I believe they would both be brought in by existing depends. isc-dhcp-client may not be brought in, so that could be an issue - although easily added to one of the meta packages.

Aside: It seems like the Priority field might be getting changes at some point: https://wiki.debian.org/Teams/Dpkg/Spec/ProtectedField

Interesting - indeed my rough understanding was not quite right, perhaps it is not really necessary to preinstall all Priority important packages in the general case. When we have a chance we should experiment with the --variant option and seeing if any truly interesting packages do get dropped.

@dan FYI!

Indeed! Thanks @BryanQuigley for the lists! I hadn’t looked at this in a while.

Installing Priority: Important goes back to the beginning of Endless likely because that’s the default that debootstrap does. However, we did discuss this around roughly 2017. It really just needs someone to analyze what’s there, determine what we want, and ensure it’s in our own metapackage. Just like when we dropped Recommends and you put together a system to make sure we acknowledged what was in there.

You definitely don’t need the important packages to boot. They’re a bit more like Recommends in the sense that they’re things you’d probably expect to be installed. Like, can you boot without udev and systemd? Sure, but that’s definitely not the experience we’re going for in Endless.

BTW, here’s how you can get the same info straight from the apt lists using dctrl-tools:

$ grep-aptavail -n -s Package -F Essential yes
$ grep-aptavail -n -s Package -F Priority required
$ grep-aptavail -n -s Package -F Priority important

I never knew about grep-aptavail - thanks!

Went through all the packages and this is what I found:
In eos-meta explicitly
apt-utils debconf-i18n debian-archive-keyring gpgv iproute2 kmod less netbase whiptail systemd-sysv udev vim-tiny

Brought in by something else:
adduser cpio dmidecode procps libreadline8 readline-common sensible-utils systemd fdisk

Add to eos metapackages IMHO:
iputils-ping
nano
nftables

Need discussion?
isc-dhcp-client (Network Manager has a built-in one that I just switched over too… Seems fine…).
isc-dhcp-common

To be removed:
cron ifupdown logrotate rsyslog tasksel-data init (explicitly depending on systemd makes more sense)

Happy to make the PR if the above looks sane. I’d plan to add the isc-dhcp as that seems like a bigger change.

Sounds good to me, I would be willing to give this a try based on what you have found here.

I see in my logs:
NetworkManager[375]: <info> [1616464740.4540] dhcp-init: Using DHCP client 'internal'
Looks like this is the NM default. So isc-dhcp-client can be removed too, it is not being used.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Sorry for the delay in wrapping this up.
We have pushed this change to use mmdebstrap --variant=required.

The packages now dropped are: cron, ifupdown, init, isc-dhcp-client, isc-dhcp-common, libdns-export1110, libestr0, libfastjson4, libisc-export1105, liblognorm5, logrotate, rsyslog, tasksel, tasksel-data

Dan also pointed out that https://wiki.debian.org/Teams/Dpkg/Spec/ProtectedField does not actually describe the Priority field, it describes something different.

The Priority field is documented here:
https://www.debian.org/doc/debian-policy/ch-archive.html#s-priorities

Important programs, including those which one would expect to find on any Unix-like system. If the expectation is that an experienced Unix person who found it missing would say “What on earth is going on, where is foo ?”, it must be an important package.

That sounds very subjective. So yes it seems like the right decision to ignore that label and decide for ourselves (via our standard metapackage) what we view as important & expected. Thanks for pushing this along!

Awesome! Hope that makes systemd maintenance a bit easier :slight_smile:

I am trying out replacing ntp with systemd-timesyncd per the below hacked setup. Is there anything that would be useful to test before a PR?

Download the debian buster package
Copy /lib/systemd/system/systemd-timesyncd.service from deb to /etc/systemd/system/systemd-timesyncd.service
Copy the binary to /etc/
Modify the service to point to it in /etc/
sudo systemctl mask ntp
sudo systemctl enable systemd-timesyncd.service

Restarting the service does cause the clock to be synced again (I made the time wrong). For whatever reason timedatectl can see that is is synced, but can’t see the details of the method.

Yeah it would be nice to advance there too. Did some digging around the issues and discussions we’ve had in the past. Here’s what I found, would definitely appreciate any research you are interested in doing!

  • Security - does timesyncd only implement SNTP (simplified NTP?) which is a less secure version of the NTP protocol that may be prone to MITM attacks? An attacker could setup a malicious NTP server and fiddle with a target’s clock, which would then expose the target to other attacks (preventing the download of security updates, for example)?
  • Should we consider Chrony instead, which seems to be the RH/Fedora preference?
  • Whenever a network connection is established, we restart ntpd; this is to get an accurate system time as quickly as possible when an intermittently-connected machine gets back online. Last time I looked, timesyncd had code to monitor network connectivity and behave accordingly, but it made the assumption that everyone uses systemd-networkd for network management.
  • We use the ‘iburst’ configuration option, for a similar reason, to update the clock really soon after boot, useful for ARM devices that don’t have a battery-backed RTC.
  • At one point we had to take steps to ensure /var/lib/ntp/ntp.drift file is created after upgrade/reboot (or on a new system), otherwise NTP could not store & benefit from drift info
  • You should download the ntp source code from deb.endlessos.org to see if there’s anything else we’ve changed.
  • Need to test that the time sync option in the GNOME control center works appropriately. We patched this for ntp.

Security

I don’t believe it’s less secure. timesynd does just trust 1 server… NTP checks 3 and correlates them IIRC. But if you can be MITM the attacker could just fake all 3. IMU It can be more accurate with NTP, but that would only help if the time servers have varying RTT.

Should we consider Chrony instead,
Chrony is nice, but likely overkill for 99% of Endless users. The two big use cases for Chrony I see are:

  • Having one endless machine as a local time server for the others in a deployment.
  • Using GPS (or other clock) as a local time source

I’m guessing it’s possible to do that with podman but haven’t looked at in depth.

…Last time I looked, timesyncd had code to monitor network connectivity and behave accordingly, but it made the assumption that everyone uses systemd-networkd for network management.

In my testing it does sync shortly after the network is restored.

useful for ARM devices that don’t have a battery-backed RTC

The Arch wiki indicates this is one of the goals of timesyncd too. I didn’t find info on iburst though.

Will dig in more to the changes later.

Useful links:
Arch Wiki
StackExchange

Need to test that the time sync option in the GNOME control center works appropriately. We patched this for ntp.

I’m not sure that exists in the current systemd.

Also opened https://github.com/endlessm/systemd/pull/132

Yeah, looking closer I don’t think we have security concerns to worry about, and ntpd/chrony may indeed be overkill.

But the network sync thing seems to be a real issue. On Fedora, using NetworkManager as standard but with chrony disabled and timesyncd enabled, I was able to reproduce with:

  1. Go online
  2. Watch timedatectl --monitor timesync-status until the poll interval increases to e.g. 8 minutes. Keep this monitor open.
  3. Go offline
  4. Set date back 2 minutes using date command
  5. Go online

Under this test, systemd will not correct the time for ~8 minutes. If you wait longer in step 2 for the poll interval to increase further, I expect the delay would be larger.

I don’t think that this blocks us moving to timesyncd, but we will need to keep a NM hook around that pokes timesyncd in the appropriate way when network connectivity is established. Any idea if there is anything better than restarting the whole service?