Test suite failures when running pytest

[badshah400]

Description

Trying to build Mathics 8.0.0 for openSUSE Tumbleweed and running into many (~170) failing tests when running the test-suite using pytest. Full build log attached for failing build.

How to Reproduce

1. Build Mathics3 8.0.0.
2. Run pytest from inside the test dir.
3. Error with pytest summary: 175 failed, 2738 passed, 287 skipped, 61 deselected, 14 xfailed, 16 xpassed, 3 warnings

Output Given

As shown in this build log.

Expected behavior

Test suite should pass.

Your Environment

OS: openSUSE Tumbleweed (Linux kernel 6.13.0, x86_64)

• GCC 14
• NumPy 1.26.4
• Pillow 11.1
• Python 3.11 and 3.12 (tested on both)
• Pytest 8.3.4
• SymPy 1.13.3
• SciPy 1.14.1
• scikit-image 0.25.0

Workarounds

Skipping a whole bunch of failing tests individually using pytest label filtering.

Additional context

Build tests are part of building Mathics RPM packages for openSUSE. Here is the specfile:

%if "%{flavor}" == "test"
%bcond_without test
%define psuffix -test
%else
%bcond_with test
%define psuffix %{nil}
%endif

# https://github.com/Mathics3/mathics-core/issues/932
%define skip_python313 1
%define pyname Mathics3
Name:           python-Mathics%{psuffix}
Version:        8.0.0
Release:        0
Summary:        A general-purpose computer algebra system
# Mathics itself is licensed as GPL-3.0 but it includes third-party software with MIT, BSD-3-Clause, and Apache-2.0 Licensing; also includes data from wikipedia licensed under CC-BY-SA-3.0 and GFDL-1.3
License:        Apache-2.0 AND BSD-3-Clause AND GPL-3.0-only AND MIT
URL:            https://mathics.github.io/
Source0:        https://github.com/Mathics3/mathics-core/releases/download/%{version}/%{pyname}-%{version}.tar.gz
BuildRequires:  %{python_module colorama}
BuildRequires:  %{python_module devel}
BuildRequires:  %{python_module mpmath >= 0.19}
BuildRequires:  %{python_module numpy < 2}
BuildRequires:  %{python_module pexpect}
BuildRequires:  %{python_module python-dateutil}
BuildRequires:  %{python_module setuptools}
BuildRequires:  %{python_module sympy >= 1.10.1}
BuildRequires:  fdupes
BuildRequires:  python-rpm-macros
Requires:       python-Mathics-Scanner >= 1.4.1
Requires:       python-Pint
Requires:       python-Pympler
Requires:       python-llvmlite
Requires:       python-mpmath >= 0.19
Requires:       python-numpy < 2
Requires:       python-palettable
Requires:       python-python-dateutil
Requires:       python-requests
Requires:       python-scipy
Requires:       python-sympy >= 1.10.1
Requires:       (python-Pillow >= 9.2 if python-base >= 3.7)
Requires(post): update-alternatives
Requires(postun): update-alternatives
Recommends:     python-scikit-image >= 0.17
BuildArch:      noarch
%if %{with test}
# SECTION For tests
BuildRequires:  %{python_module Mathics}
BuildRequires:  %{python_module Mathics-Scanner >= 1.4.1}
BuildRequires:  %{python_module Pillow >= 9.2 if %python-base >= 3.7}
BuildRequires:  %{python_module Pint}
BuildRequires:  %{python_module chardet}
BuildRequires:  %{python_module llvmlite}
BuildRequires:  %{python_module palettable}
BuildRequires:  %{python_module pytest}
BuildRequires:  %{python_module requests}
BuildRequires:  %{python_module scikit-image >= 0.17}
BuildRequires:  %{python_module stopit}
BuildRequires:  %{python_module testsuite}
BuildRequires:  %{python_module typing-extensions}
# /SECTION
%endif
Provides:       python-Mathics3 = %{version}
%python_subpackages

%description
Mathics is a general-purpose computer algebra system (CAS). It is meant to be a
free, lightweight alternative to Mathematica.

%prep
%autosetup -p1 -n mathics3-%{version}

# REMOVE SHEBANGS FROM FILES INSTALLED TO NON-EXEC LOCATIONS
pushd mathics
for d in `find ./ -prune -type d`
do
  find ${d} -name "*.py" -exec sed -i "1,4{/\/usr\/bin\/env/d}" '{}' \;
done
popd

%build
%if %{without test}
export USE_CYTHON=0
%python_build
%endif

%install
%if %{without test}
export USE_CYTHON=0
%python_install
%python_clone -a %{buildroot}%{_bindir}/mathics
%python_expand %fdupes %{buildroot}%{$python_sitelib}
%endif

%if %{with test}
%check
# Home page tests require django server up and running, test_gudermannian needs network access
# test_image: https://github.com/Mathics3/mathics-core/issues/837
pushd test
export USE_CYTHON=0
%pytest -k 'not (test_home_page or test_gudermannian or test_image)'
popd
%endif

%if %{without test}
%post
%python_install_alternative mathics

%postun
%python_uninstall_alternative mathics

%files %{python_files}
%license COPYING.txt
%doc README.rst AUTHORS.txt
%python_alternative %{_bindir}/mathics
%{python_sitelib}/mathics/
%{python_sitelib}/%{pyname}-%{version}-py%{python_version}.egg-info/
%endif

%changelog

[rocky]

Thanks for the report.

It might have something to do with the build tree and the source tree being in different places.

Some JSON tables must be created and used in the Mathics3 kernel. This may be causing the failure, although I don't see how the error message is exactly related to this.

With the information you've given, I'll see if I can reproduce the failure and see what can be done to fix.

Is there a mode where one can build in the source tree? Or make sure that mathics/data/operator-tables.json and mathics/data/op-tables.json appear in both the source and build trees?

[rocky]

Also, thanks for packaging on OpenSUSE.

[badshah400]

Happy to help in whatever small way I can.

I tried to copy the test suite — the two json files in their appropriate location were already there — to inside the build tree and run the tests from there using pytest, but I run into the same issues. Will investigate this further and see if I am setting up the build dir correctly.

Thanks for your very useful app.