-
Notifications
You must be signed in to change notification settings - Fork 76
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Resources stuck on DELETING #19
Comments
After restarting it is still stuck, but now another error, lvm timed out: The most interesting, that ErrorReport-5BB322C9-CDA0B-000000.log |
I will look into this, but can you please re-upload ErrorReport-5BAFAC32-CDA0B-000000.log, as it has the same content as ErrorReport-5BAFAC32-CDA0B-000001.log |
@ghernadi, sure, there is logs from another node, with the same problem: ErrorReport-5BAF8C77-12FF9-000000.log |
Alright, the The "Access to deleted resource" is most likely a consequence of the previous Regarding the timeout of |
In the current release?
I like linstor and drbd, and I'm glad to help you, thanks for your work! |
No, sorry, it is fixed internally and will be in the next release... |
Hi, we just faced with the same situation (on 1.7.1): # linstor r l -r pvc-9ed4b88e-3ba3-400c-bf9b-4750a3384b6b pvc-f8aa2cba-8784-4a21-b8c1-fc192272d8d9 pvc-7fbdf685-39b0-43b9-80f3-a2bbe0e144a4 -a
+-------------------------------------------------------------------------------------------------+
| ResourceName | Node | Port | Usage | Conns | State |
|=================================================================================================|
| pvc-7fbdf685-39b0-43b9-80f3-a2bbe0e144a4 | m1c12 | 7426 | Unused | Ok | UpToDate |
| pvc-7fbdf685-39b0-43b9-80f3-a2bbe0e144a4 | m1c6 | 7426 | Unused | Connecting(m1c10) | UpToDate |
| pvc-9ed4b88e-3ba3-400c-bf9b-4750a3384b6b | m1c12 | 7350 | Unused | Ok | UpToDate |
| pvc-9ed4b88e-3ba3-400c-bf9b-4750a3384b6b | m1c6 | 7350 | Unused | Connecting(m1c5) | UpToDate |
| pvc-f8aa2cba-8784-4a21-b8c1-fc192272d8d9 | m1c12 | 7365 | InUse | Ok | UpToDate |
| pvc-f8aa2cba-8784-4a21-b8c1-fc192272d8d9 | m1c4 | 7365 | Unused | Ok | UpToDate |
+-------------------------------------------------------------------------------------------------+ # linstor r d m1c5 pvc-9ed4b88e-3ba3-400c-bf9b-4750a3384b6b
SUCCESS:
Description:
Node: m1c5, Resource: pvc-9ed4b88e-3ba3-400c-bf9b-4750a3384b6b marked for deletion.
Details:
Node: m1c5, Resource: pvc-9ed4b88e-3ba3-400c-bf9b-4750a3384b6b UUID is: 48e29d6f-3f80-4615-97a6-2a81b1af0b64
SUCCESS:
Deleted 'pvc-9ed4b88e-3ba3-400c-bf9b-4750a3384b6b' on 'm1c5'
ERROR:
Description:
(Node: 'm1c6') Creation of the DRBD configuration file for resource 'pvc-9ed4b88e-3ba3-400c-bf9b-4750a3384b6b' failed due to an I/O error
Cause:
Creation of the DRBD configuration file failed due to an I/O error
Correction:
- Check whether enough free space is available for the creation of the file
- Check whether the application has write access to the target directory
- Check whether the storage is operating flawlessly
Details:
The error reported by the runtime environment or operating system is:
No space left on device
Show reports:
linstor error-reports show 5EBE5356-85D84-000080
SUCCESS:
Notified 'm1c12' that 'pvc-9ed4b88e-3ba3-400c-bf9b-4750a3384b6b' is being deleted on Node(s): [m1c5]
command terminated with exit code 10 # linstor error-reports show 5EBE5356-85D84-000080
None # linstor v l -r pvc-9ed4b88e-3ba3-400c-bf9b-4750a3384b6b pvc-f8aa2cba-8784-4a21-b8c1-fc192272d8d9 pvc-7fbdf685-39b0-43b9-80f3-a2bbe0e144a4 -a
+-------------------------------------------------------------------------------------------------------------------------------------------+
| Node | Resource | StoragePool | VolNr | MinorNr | DeviceName | Allocated | InUse | State |
|===========================================================================================================================================|
| m1c10 | pvc-7fbdf685-39b0-43b9-80f3-a2bbe0e144a4 | DfltDisklessStorPool | 0 | 1446 | /dev/drbd1446 | | | Unknown |
| m1c12 | pvc-7fbdf685-39b0-43b9-80f3-a2bbe0e144a4 | thindata | 0 | 1446 | /dev/drbd1446 | 1.41 GiB | Unused | UpToDate |
| m1c6 | pvc-7fbdf685-39b0-43b9-80f3-a2bbe0e144a4 | thindata | 0 | 1446 | /dev/drbd1446 | 1.41 GiB | Unused | UpToDate |
| m1c12 | pvc-9ed4b88e-3ba3-400c-bf9b-4750a3384b6b | thindata | 0 | 1350 | /dev/drbd1350 | 1.40 GiB | Unused | UpToDate |
| m1c5 | pvc-9ed4b88e-3ba3-400c-bf9b-4750a3384b6b | DfltDisklessStorPool | 0 | 1350 | /dev/drbd1350 | | | Unknown |
| m1c6 | pvc-9ed4b88e-3ba3-400c-bf9b-4750a3384b6b | thindata | 0 | 1350 | /dev/drbd1350 | 1.40 GiB | Unused | UpToDate |
| m1c12 | pvc-f8aa2cba-8784-4a21-b8c1-fc192272d8d9 | thindata | 0 | 1365 | /dev/drbd1365 | 3.14 GiB | InUse | UpToDate |
| m1c4 | pvc-f8aa2cba-8784-4a21-b8c1-fc192272d8d9 | thindata | 0 | 1365 | /dev/drbd1365 | 3.15 GiB | Unused | UpToDate |
+-------------------------------------------------------------------------------------------------------------------------------------------+ The error shows |
my bad |
Hehe, But yes, I will think about what we could do in such a situation :) |
It used to write to stderr if writing the error report fails due to an IOException, so if that still works, it might be in a journal entry or syslog entry. Provided that those were not out of storage space as well... |
Hi, I just faced with the similar problem on another installation,
this command just stuck:
tace log:
|
UPD: After the manual removal via
|
Creation of new resource is also stuck:
controller trace log: satellite trace log: |
Probably related stack trace:
|
fixed by swaping java 1.8 to 1.11
|
Hi! The trace logs from #19 (comment) does not really help as that shows only that the controller set the DELETE flags on rsc and vlm, but not why the satellite does not respond to the update.. Regarding "no error report" from #19 (comment), even if Linstor does not tell you "here is the error report", you might still want to check #19 (comment) again - ErrorReports would be helpful here. Regarding Java version 8 -> 11 from the last 2 comments.. Looks like there are some incompatibilities if a Java class was compiled with 9+ and being run with 8... Apparently Linstor was not compiled with Java 8 (or at least with 8 as target) before packaging. we will look into this, thanks for the hint! |
How do you get your packages? self build? |
@rp- this was proxmox-5 cluster upgraded to proxmox-6, linstor was installed directly from linbit's repository: # cat /etc/apt/sources.list.d/linbit.list
deb http://packages.linbit.com/proxmox/ proxmox-6 drbd-9.0 Unfortunately, I've got an access to it already after problem has occurred. |
@ghernadi unfortunately I have not found any additional error reports :-( |
Just faced this again:
|
is the |
It is accessible In pod with linstor-satellite |
While it could be triggered by an installation problem, the fact that it is a NullPointerException means that there must be a programming error somewhere on the code path too. |
Hi checkout #432 |
I have some problem with removing my resources.
I presume it may be same problem connected with long lvm output #8
Any way I can't delete my resources even after correcting
global_filter
for mylvm.conf
satellite's log:
ErrorReport-5BAFAC32-CDA0B-000000.log
ErrorReport-5BAFAC32-CDA0B-000001.log
This problem occurs quite often for my installation.
The text was updated successfully, but these errors were encountered: