Check examples to be standalone

[bsipocz]

It would be nice to add a config option that checks whether the docstring examples are complete standalone ones without relying the namespace of the full file they're being defined.

astropy/astropy#6878 (comment)

[asmeurer]

Does anyone know of a workaround to get this behavior?

[asmeurer]

Getting this to work here is as simple as this

diff --git a/pytest_doctestplus/plugin.py b/pytest_doctestplus/plugin.py
index 6d90d71..a0c3a84 100644
--- a/pytest_doctestplus/plugin.py
+++ b/pytest_doctestplus/plugin.py
@@ -518,6 +518,7 @@ class DocTestFinderPlus(doctest.DocTestFinder):
         return True

     def find(self, obj, name=None, module=None, globs=None, extraglobs=None):
+        globs = {}
         tests = doctest.DocTestFinder.find(self, obj, name, module, globs, extraglobs)
         if hasattr(obj, '__doctest_skip__') or hasattr(obj, '__doctest_requires__'):
             if name is None and hasattr(obj, '__name__'):

So it shouldn't be hard to add a flag for that.

However, I also noticed that enabling pytest-doctestplus changes the output of things like

>>> np.array([1, 2])
array([1, 2])

to

>>> np.array([1, 2])
[1, 2]

How hard will it be to make it so that you can use doctestplus to get this "standalone namespace" behavior, and not have it change any of the output or any of the other things it does?

[asmeurer]

There's also the question of whether this makes more sense to contribute to upstream pytest instead (c.f. #100). CC @nicoddemus

[asmeurer]

(In case it wasn't clear, I'm interesting in contributing a fix for this one way or the other)

[bsipocz]

@asmeurer - contributions are very welcome, either here, or if it's directly upstream in pytest, that's great, too.

As we touched on it in #100, it's not yet clear who will have the capacity to make the transition of this upstream (or what wasn't mentioned there, to straight away make the separation out into one plugin that is being used by pytest by default).

[asmeurer]

Actually I really want this to be in the upstream doctest. But we can start with baby steps.

Presumably since this isn't implemented yet, putting it in pytest itself would be independent of the work in #100, assuming that is the correct path forward.

[nicoddemus]

Indeed this feature can be added to pytest itself independently from #100. What are you folk thinking, a new option to enable this feature?

[bsipocz]

A new option sounds good to me, I'm afraid this cannot be the default as it would break everyone's tests.

[bsipocz]

The other option is to use something similar what the matplotlib plot sphinx directive does with the context. After all, I think it would me most sense to be able to control at the individual file level where to have the standalone examples.

[asmeurer]

For file-level or even docstring-level control I think it would make sense to define something in the file itself, or to put a decorator on the docstring. For my use case I want this behavior universally, so I will implement this as an ini flag. Extending it should not be difficult.

[asmeurer]

Upstream PR over at pytest-dev/pytest#6978.