gpg: waiting for lock

[Niklap97]

About every 5 times when starting my Lenovo T430 the message "gpg: waiting for lock (held by x)" comes up and heads does not start. i remain stuck in an endless loop.

gpg: waiting for lock (held by 169) ... 
gpg: waiting for lock (held by 169) ...
gpg: waiting for lock (held by 169) ...
gpg: waiting for lock (held by 169) ...
gpg: waiting for lock (held by 169) ...

[tlaurion]

@Niklas-Nitrokey This is what you get directly at boot? Prior of having HOTP: Success and a TOTP code?
When does that happen exactly?

[Niklap97]

I get this output immediately after starting the laptop.

[tlaurion]

Call tree:

• init calls cbfs-init (which outputs Loading from CBFS...). User keys are exported from ROM and put in ramfs at this point. Is the user public key at fault here? Why some thimes and not the others?
• init then calls key-init which loads gpg user keys and distro keys from /etc/distro

Anything funky in that /etc/distro directory on your side for the rom you are using @Niklas-Nitrokey ?
Something weird with user generated keys fused inside of the ROM that are imported here?

Which platform? Which commit id? Which fork?

[tlaurion]

Wiping the error redirections (2> /dev/null) on your platform/fork should give useful input to troubleshoot this issue I unfortunately cannot replicate on my side on x230 with latest commit ID.

Not sure I understand what is happening here, but it seems that gpg waits a child to exit, which never happens.

@Niklas-Nitrokey : You say it happens randomly 1/5 times?
It would be useful to see the output of when it succeeds (with 2>/dev/null removed above) and when it doesn't under initrd/bin/key-init:

# Import user's keys
gpg --import /.gnupg/keys/*.key  /.gnupg/keys/*.asc  || true

# Import trusted distro keys allowed for ISO signing
gpg --homedir=/etc/distro/ --import /etc/distro/keys/*  || true
#Set distro keys trust level to ultimate (trust anything that was signed with these keys)
gpg --homedir=/etc/distro/ --list-keys --fingerprint --with-colons|sed -E -n -e 's/^fpr:::::::::([0-9A-F]+):$/\1:6:/p' |gpg --homedir=/etc/distro/ --import-ownertrust || true
gpg --homedir=/etc/distro/ --update-trust  || true

# Add user's keys to the list of trusted keys for ISO signing
gpg --export | gpg --homedir=/etc/distro/ --import || true

[Niklap97]

The problem has already occurred for x230s and t430s.
commit: Nitrokey@5e5c36f

How can I change the key-init so that the changes are saved for the reboot?

[tlaurion]

The problem has already occurred for x230s and t430s.
commit: Nitrokey@5e5c36f

How can I change the key-init so that the changes are saved for the reboot?

@Niklas-Nitrokey The simplest solution is to edit the sources directly and build a new rom to be flashed internally with only changes under key-init as shown in previous post to give output of what is going on prior of looping for something locked for what we don't know why (I can't replicate).

A more complicated approach would be to inject modified key-init with cbfs tool from within heads, in a rom previously backed up with flash.sh tool.

Here is an example of how the file /etc/config.user is injected inside of the rom, which is then extracted back in place by cbfs-init at boot.

[tlaurion]

@Niklas-Nitrokey still relevant on master? Please close otherwise.

[tlaurion]

Please reopen otherwise

[Niklap97]

Sorry for the late reply we installed the new version last week on the corresponding laptops with this "lock problem" and it seems to have fixed the problem completely.

[tlaurion]

@Niklap97 I'm having this issue right now on a x230 test laptop without a battery connected for a while and which was disconnected from power adapter in the last days.

That laptop has a clock skew issue (its in year 2145! Welcome to the future) and first message on top of the screen boot (before Heads banner) I get is "rtc_cmos: 00:03: hctosys: unable to read the hardware clock".

If I reboot and hit r to hit recovery console early, and reboot, the same rtc_cmos error above appears but I do not have the "gpg: waiting for lock" issue anymore.

This happened directly after a maximized -> maximized internal flash upgrade keeping settings, so I my gpg pubring and trustdb have been migrated. On firmware upgrade, there is MRC cache that is rewritten on SPI flash.... Unfortunately, I cannot replicate further, since going early recovery shell fixed the issue (why is the unanswered question here... since I am still in 2145 at this point)

But the hint here is that RTC clock is skew, and if the laptop has issues keeping time, RTC battery (button battery under keyboard) should probably be replaced.

Also, before booting, it is recommended (if time skew) to call network-recovery-init from command line with Ethernet cable connected, or to set time manually through documented date -sand hwclock -w which are not so convenient.

network-init-recovery is recommended because automatic:

• It will randomize Ethernet MAC locally (does not survive booting into OS. Maximized builds still have a DE:AD:C0:FF:EE hardcoded MAC by default builds)
• It will get IP from DHCP server
• Attempt to sync time through NTP against obtained DNS server from DHCP response (normally internal router)
  • Otherwise will attempt NTP against ntp.pool.org (with IP of server obtained through name resolution from DNS server obtained through DHCP).

This is less error prone then setting date and proper input format, and will set time in UTC as Heads requires, leaving no room for user error. On reboot, until

Hope that helps.

[githubuseravailable]

Also, before booting, it is recommended (if time skew) to call network-recovery-init from command line with Ethernet cable connected

@tlaurion i also have the same issue,
please inform, how to access command line, to call network-recovery-init, before booting ?
also is it compulsory to use Ethernet cable ? is it okay with Wifi ?

the complete issue is in ticket Starting new kernel takes long time

[tlaurion]

@githubuseravailable

please inform, how to access command line, to call network-recovery-init, before booting ?

Hit 'r' keyboard's key repeatedly and frenetically at boot, until you reach recovery shell.
Then you can call network-recovery-init which will try to obtain DHCP address from router, then try to sync clock through NTP with router and otherwise will try to use router's IP address as DNS server to resolve ntp.pool.org and sync time with IP address returned for that DNS name.

Otherwise, you would have to set time manually. Just searched through osresearch.net with the search tool, and was not able to find any reference to data nor skew, so that got lost somehow ?!?
Would be beneficial to open an issue under heads-wiki github repository so that that is documented properly.

also is it compulsory to use Ethernet cable ? is it okay with Wifi ?

Ethernet cable only. There is nothing to deal with WPA keys (wpa-supplicant depending on mbetls instead of openssl per space constraints) nor anything that fancy under Heads to deal with WIFI card. Could most probably be done though, but an integration project in itself where no use-case actually justifies requiring wifi internet access through Heads for the moment.

[Diboby]

Hi,

This indicates the presence of a lock file within ~/.gnupg or its subfolders, likely located within the ~/.gnupg/public-keys.d directory, which shouldn't be there. It needs to be removed. For example: rm -rf ~/.gnupg/public-keys.d/pubring.db.lock.

If you run cat on the lock file, you'll find the PID that causes the issue. It is the same inside the error message removing stale lockfile (created by 17297) or in my case gpg: Note: database\_open 134217901 waiting for lock (held by 7) ....

Here are the steps to fix the issue

Look the directory content

> ls -al ~/.gnupg/public-keys.d/
total 84
drwxr-x--- 2 root root  4096 Mar 16 02:34 .
-rw-r--r-- 2 root root    27 Mar 16 02:34 .#lk0x00007ff8c7261aa0.buildkitsandbox.7
drwx------ 4 root root  4096 Mar 16 02:34 ..
-rw-r--r-- 1 root root 65536 Mar 16 02:34 pubring.db
-rw-r--r-- 2 root root    27 Mar 16 02:34 pubring.db.lock

Print the lock file content

> cat ~/.gnupg/public-keys.d/pubring.db.lock 
         7
buildkitsandbox

Look the error

> gpg --list-keys
gpg: Note: database_open 134217901 waiting for lock (held by 7) ...
gpg: Note: database_open 134217901 waiting for lock (held by 7) ...
gpg: Note: database_open 134217901 waiting for lock (held by 7) ...
gpg: Note: database_open 134217901 waiting for lock (held by 7) ...
gpg: Note: database_open 134217901 waiting for lock (held by 7) ...
gpg: keydb_search_first failed: Operation timed out

Now check the solution

> rm -f ~/.gnupg/public-keys.d/pubring.db.lock 
> gpg --list-keys
[keyboxd]
---------
pub   rsa**** 2024-03-15 [SC]
      B7244*******5341455F5D5*******1F96D
uid           [ unknown] GitLab ***** (032024) <c*******[email protected]>
sub   rsa**** 2024-03-15 [E]

Here are some other approaches to prevent this error:

• Docker:
  When creating your Docker image, include the public-keys.d folder like this:
  RUN mkdir -p \~/.gnupg/public-keys.d before importing keys.

• On a local machine:
  If the folder ~/.gnupg/public-keys.d doesn't exist, create it before importing keys.

Hope this helps!

[tucq88]

In my case, just kill the .lock works

rm -rf ~/.gnupg/*.lock
rm -rf ~/.gnupg/public-keys.d/*.lock

Reference: https://superuser.com/questions/1811518/gpg-stops-doing-anything-on-mac

[tlaurion]

This was fixed a while ago no? Was linked to cmos system kept time being either way in the past compared to kept under repo public keys, and was thought to be fixed with early boot detection of date being < 2024 recently, and forcing the end user to set time with added helper, before those keys are loaded from firmware.

Is this still an issue for someone here? If so please open a new issue with proper screenshots. This was resolved with PR pointing here see above and this issue is closed for that reason. Was surprised seeing a comment here :)