QGIS crash when using the python-lxml package (Windows)

[felixvons]

What is the bug or the crash?

Hi,

I ran into an issue with the latest QGIS version when using lxml (e.g. from openpyxl) in a QGIS plugin or from the console.
I can force the crash from the Python console but not every time in my complex plugin setup.

Maybe the issue #55807 is related to this issue.

Python Stack Trace
Windows fatal exception: access violation

Current thread 0x00005718 (most recent call first):
  File "C:\Users/felix/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\test_qgis_plugin_qgis_crash_with_xml\tests\test_qgis.py", line 17 in test_lxml
    el = Element("test")
  File "C:\Users/felix/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\test_qgis_plugin_qgis_crash_with_xml\__init__.py", line 63 in run_lxml
    test_qgis.test_lxml()


Stack Trace


RtlpWaitOnCriticalSection :
RtlpEnterCriticalSectionContended :
RtlEnterCriticalSection :
xmlDictReference :
PyObject_Vectorcall :
PyObject_Vectorcall :
PyEval_EvalFrameDefault :
PyFunction_Vectorcall :
Py_hashtable_compare_direct :
PyObject_Call :
PyObject_Call :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
QObject::qt_static_metacall :
QAbstractButton::clicked :
QAbstractButton::click :
QAbstractButton::mouseReleaseEvent :
PyInit_QtWidgets :
QWidget::event :
PyInit_QtWidgets :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QApplicationPrivate::sendMouseEvent :
QSizePolicy::QSizePolicy :
QSizePolicy::QSizePolicy :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QGuiApplicationPrivate::processMouseEvent :
QWindowSystemInterface::sendWindowSystemEvents :
QEventDispatcherWin32::processEvents :
qt_plugin_query_metadata :
QEventLoop::exec :
QCoreApplication::exec :
main :
BaseThreadInitThunk :
RtlUserThreadStart :




QGIS Info
QGIS Version: 3.34.9-Prizren
QGIS code revision: cbbc7bdf
Compiled against Qt: 5.15.13
Running against Qt: 5.15.13
Compiled against GDAL: 3.9.1
Running against GDAL: 3.9.1



System Info
CPU Type: x86_64
Kernel Type: winnt
Kernel Version: 10.0.22631

Steps to reproduce the issue

You find a test plugin here: https://github.com/felixvons/test_qgis_plugin_qgis_crash_with_xml

1. Open an empty QGIS project
2. Drag & Drop a GeoPackage into the current project (or use the python code)
3. try to create an "lxml.etree.Element" in the python console
4. maybe retry the steps 2 and 3
5. QGIS crash

Versions

<style type="text/css"> p, li { white-space: pre-wrap; } </style>

QGIS-Version	3.34.9-Prizren	QGIS-Codeversion	cbbc7bd
Qt-Version	5.15.13
Python-Version	3.12.4
GDAL-Version	3.9.1
PROJ-Version	9.4.0
EPSG-Registraturdatenbankversion	v11.004 (2024-02-24)
GEOS-Version	3.12.2-CAPI-1.18.2
SQLite-Version	3.45.1
PDAL-Version	2.6.3
PostgreSQL-Client-Version	16.2
SpatiaLite-Version	5.1.0
QWT-Version	6.2.0
QScintilla2-Version	2.14.1
BS-Version	Windows 11 Version 2009
Aktive Python-Erweiterungen
test_qgis_plugin_qgis_crash_with_xml	1.0
db_manager	0.1.20
grassprovider	2.12.99
MetaSearch	0.3.6
processing	2.12.99

QGIS-Version 3.34.9-Prizren QGIS-Codeversion [cbbc7bd](https://github.com/qgis/QGIS/commit/cbbc7bdf) Qt-Version 5.15.13 Python-Version 3.12.4 GDAL-Version 3.9.1 PROJ-Version 9.4.0 EPSG-Registraturdatenbankversion v11.004 (2024-02-24) GEOS-Version 3.12.2-CAPI-1.18.2 SQLite-Version 3.45.1 PDAL-Version 2.6.3 PostgreSQL-Client-Version 16.2 SpatiaLite-Version 5.1.0 QWT-Version 6.2.0 QScintilla2-Version 2.14.1 BS-Version Windows 11 Version 2009

Aktive Python-Erweiterungen
test_qgis_plugin_qgis_crash_with_xml
1.0
db_manager
0.1.20
grassprovider
2.12.99
MetaSearch
0.3.6
processing
2.12.99

Supported QGIS version

• I'm running a supported QGIS version according to the roadmap.

New profile

• I tried with a new QGIS profile

Additional context

The detailed reason for the crash is unknown to me.

I found two following solutions to fix the problem (for me):

1. rebuild lxml 5.2.1 from source on Windows
2. update lxml from 5.2.1 to 5.2.2

I created a small test plugin to force the crash with lxml/openpyxl: https://github.com/felixvons/test_qgis_plugin_qgis_crash_with_xml

[CharlesGag]

Had a similar bug, updated lxml from 5.2.1 to 5.2.2 fixed it !

In my case, I was trying to load a workbook using openpyxl and it was crashing at that moment. Doing it in the QGIS Python console or outside qgis worked fine, but it crashed all the time when running it in my custom processing tool.

Thanks !

[felixvons]

Still persists in QGIS 3.34.10 LTR

Python Stack Trace

Windows fatal exception: access violation

Current thread 0x00003614 (most recent call first):
  File "C:\Users/XX/AppData/Roaming/QGIS/QGIS3\profiles\test/python/plugins\test_qgis_plugin_qgis_crash_with_xml\tests\test_qgis.py", line 17 in test_openpyxl_workbook
    el = Element("test")
  File "C:\Users/XX/AppData/Roaming/QGIS/QGIS3\profiles\test/python/plugins\test_qgis_plugin_qgis_crash_with_xml\__init__.py", line 56 in run
    test_qgis.test_openpyxl_workbook()

Stack Trace

RtlpWaitOnCriticalSection :
RtlpEnterCriticalSectionContended :
RtlEnterCriticalSection :
xmlDictReference :
PyObject_Vectorcall :
PyObject_Vectorcall :
PyEval_EvalFrameDefault :
PyFunction_Vectorcall :
Py_hashtable_compare_direct :
PyObject_Call :
PyObject_Call :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
QObject::qt_static_metacall :
QAbstractButton::clicked :
QAbstractButton::click :
QAbstractButton::mouseReleaseEvent :
PyInit_QtWidgets :
QWidget::event :
PyInit_QtWidgets :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QApplicationPrivate::sendMouseEvent :
QSizePolicy::QSizePolicy :
QSizePolicy::QSizePolicy :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QGuiApplicationPrivate::processMouseEvent :
QWindowSystemInterface::sendWindowSystemEvents :
QEventDispatcherWin32::processEvents :
qt_plugin_query_metadata :
QEventLoop::exec :
QCoreApplication::exec :
main :
BaseThreadInitThunk :
RtlUserThreadStart :

QGIS Info
QGIS Version: 3.34.10-Prizren
QGIS code revision: 113de9e
Compiled against Qt: 5.15.13
Running against Qt: 5.15.13
Compiled against GDAL: 3.9.2
Running against GDAL: 3.9.2

System Info
CPU Type: x86_64
Kernel Type: winnt
Kernel Version: 10.0.22631

[jakosek]

same for me, random crashes with python-docx (internally use lxml), upgrade to lxml-5.3.0 solved crash

[Djedouas]

Hi @jef-n , apparently python3-lxml has to be updated in Windows packaging

[felixvons]

Hi,

still valid in newest LTR 3.34.15

Report Details

Python Stack Trace

Windows fatal exception: access violation

Current thread 0x00003d04 (most recent call first):
  File "C:\Users/XXX/AppData/Roaming/QGIS/QGIS3\profiles\test/python/plugins\test_qgis_plugin_qgis_crash_with_xml\tests\test_qgis.py", line 17 in test_openpyxl_workbook
    el = Element("test")
  File "C:\Users/XXX/AppData/Roaming/QGIS/QGIS3\profiles\test/python/plugins\test_qgis_plugin_qgis_crash_with_xml\__init__.py", line 56 in run
    test_qgis.test_openpyxl_workbook()

Stack Trace

RtlpWaitOnCriticalSection :
RtlpEnterCriticalSectionContended :
RtlEnterCriticalSection :
xmlDictReference :
PyObject_Vectorcall :
PyObject_Vectorcall :
PyEval_EvalFrameDefault :
PyFunction_Vectorcall :
PyArg_CheckPositional :
PyObject_Call :
PyObject_Call :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
PyInit_QtCore :
QObject::qt_static_metacall :
QAbstractButton::clicked :
QAbstractButton::click :
QAbstractButton::mouseReleaseEvent :
PyInit_QtWidgets :
QWidget::event :
PyInit_QtWidgets :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QApplicationPrivate::sendMouseEvent :
QSizePolicy::QSizePolicy :
QSizePolicy::QSizePolicy :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QGuiApplicationPrivate::processMouseEvent :
QWindowSystemInterface::sendWindowSystemEvents :
QEventDispatcherWin32::processEvents :
qt_plugin_query_metadata :
QEventLoop::exec :
QCoreApplication::exec :
main :
BaseThreadInitThunk :
RtlUserThreadStart :

QGIS Info
QGIS Version: 3.34.15-Prizren
QGIS code revision: 386f258
Compiled against Qt: 5.15.13
Running against Qt: 5.15.13
Compiled against GDAL: 3.10.1
Running against GDAL: 3.10.1

System Info
CPU Type: x86_64
Kernel Type: winnt
Kernel Version: 10.0.22631

[Djedouas]

Hi @jef-n , the python3-lxml package version currently in OSGeo4W build crashes QGIS, how could we update the package version?

Previous comment:

upgrade to lxml-5.3.0 solved crash

Thanks

[agiudiceandrea]

@Djedouas, the python3-lxml package version 5.3.0 is now available in the OSGeo4W repository.