response.txt

We thank the referee for the close reading of the paper and the insightful
comments.  We provide a response to specific comments from the referee
and the statistical editor below.

> There are some stylistic issues with the text. I have left comments inline
>  where appropriate.

We have addressed all of the stylistic issues that were raised by the referee.
Here are some responses to the more detailed comments:

Comment on health of the project:

We have added a plot of contributors per month along with our other metrics
and some further comments about the health of the project.  A full analysis
following the CHAOSS framework is beyond the scope of the current paper. 

Comment on styles:
As it was not related to the section, we removed the first paragraph 
in the APE section that commented on different styles.  However, we
did add a comment in the affiliated package section indicating that
they may have different requirements and structure than the core package. 

Comment on NDData:
At the time of initial development, the developers were not aware of 
any open source packages that would have fit the requirements of NDData.  Since
then, we have been working with different groups (LSST, sunpy) to coordinate
development of the NDData interface as well as assessing other attempts at 
something similar.  There is currently an APE about this under 
discussion (https://github.com/astropy/astropy-APEs/pull/14)  . 

> More generally, given the technical nature of the work being described here,
> I frequently found myself wanting a small code snippet that demonstrated the
> new/improved functionality of the packages being described. Perhaps the
> authors could find a way to include short code examples where appropriate,
> either inline in the paper body or in the appendix.

We did include code snippets in the first astropy paper from 2013. 
However, we have made a conscious decision to avoid that this time 
due to the following reasons:

- While we aim for stability, many of the newsest features are still
  considered provisional, meaning that their API can still change. The
  paper describes astropy 2.0, but at the time of writing this,
  astropy 3.0 is already released. We hope that this paper is useful
  for years to come. Code examples for a specific version are at risk
  of being outdated much sooner than that.

- Code examples belong in the documentation. In the documentation at
  docs.astropy.org we provide extensive examples with enough comments
  to actually understand what is going on. If readers are curious, we
  want them to read the documentation.

- It takes a lot of space to explain a code example well. Currently,
  the text lists the names of the most important classes or
  functions. To give a meaningful example that shows these classes in
  action and is useful to people not familiar with astropy (or even
  Python), we would have to detail several parameters and the format
  of the results. We think that it is more useful to explain the
  general concept in the paper and not the names and the specific
  implementation of the functions and parameters.

- Most new features would require some setup. For example, showing the
  code for WCS axis plotting is not very useful without loading an
  image with a WCS in it first. More complex features profit more from
  code examples, but they also require much more setup to make such an
  example self-consistent and useful. Again, the documentation is a better
  place for that.

As a compromise, we have added links to the documentation in many places and
there is a link in the introduction that points to the github repository where
this article is written. That repository contains (and the text in the
introduction says this) the scripts that we used to generate the figures.


> Specific issues that need to be corrected: There are some errors in the paper
> (inline comments left where appropriate), for example, Figure 6 has a fairly
> major issue with either the wrong caption or the wrong figure but I'm sure
> this is easily addressed. I have left comments throughout the modified
> 'main.tex' file in the form `\inlinecomment{REVIEWER}{Comment}`

We have updated Figure 6 to have the correct caption. Inline comments
were also addressed throughout the text. 

> More generally I have a few comments/concerns:

> Given the title of this paper, I expected to see more discussion by the
> authors about the inclusivity of their community. An uber-skeptical reader
> could make the assessment that the Astropy community has a code of conduct
> therefore believes it's inclusive. I suspect that the community is doing
> much more than this though... So, has any attempt been made by the authors
> to quantify the inclusivity of the Astropy community? There's a growing
> literature in the quantitative social sciences studying open source
> communities that the authors might want to look into: e.g. the work of
> Gousios http://www.gousios.gr/publications.html and Vasilescu
> https://cmustrudel.github.io/publications/ come to mind. Some discussion
> or attempt to quantify the inclusivity is recommended.

We thank the referee for bringing up this important point. After some discussion
within the Astropy community, it became clear that a section regarding inclusion
is beyond the scope of the current paper. Quantitative analysis of the Astropy
community demographics is warranted. However, this is a serious undertaking
beyond the capabilities of our community, and it may require IRB approval. Based
on the referee's comment, we have established initial contact with the NumFOCUS
Diversity & Inclusion in Scientific Computing (DISC) program. Results will not
be available in time for this paper. We have opted to remove "inclusive" from
the paper title to make room for future study and dialogue.

> The phase 'open development' is used a few times but isn't really defined or
> connected to the broader open source ecosystem.  What other communities are
> working in this way? How novel is 'open development' by their assessment? For
> example, other communities have adopted policies that encourage community
> contribution - most notable is the Node.js Foundation's contribution policies
> designed to promote 'Healthy Open Source'
> https://medium.com/the-node-js-collection/healthy-open-source-967fa8be7951 .
> How does the Astropy community efforts compare to this?

This question is at the intersection of two communities. On the one
side, there is the open source community, which by itself consists of
many different projects with vastly different norms and
processes. Many terms, "open develoment" being one of them, are used
with slightly different meanings in different parts of this
community. The model of astropy - the development happens openly on
github and anyone with an internet connection can comment on issues or
propose a pull request, but only a select set of maintainers can merge
them - is by no means special. Many open source communities work in a
very similar way, although they may use different terminology to
describe it. On the other hand, there is the astrophysics community,
and while there are several open source and open development (in the
way astropy uses the term) projects in astronomy, astropy is probably
the biggest one. Many astronmers are not involved in any open
development (also many make use of open source projects they are not
involved in e.g. GNU/Linux), and the concept is new to many of
them. This article caters mostly to the astrophysics community, so we
decided to add a definition, but we refrain from a comparative
discussion with other open source projects.

> A number of times I found myself wondering how the Astropy projects decides
> whether to build their own package or leverage something available in the
> scientific Python ecosystem? On the face of it it seems like every decision
> made by the Astropy project to build a new core package only increases the
> long term cost/overhead of maintaining this functionality. For example,
> Section 3.5.2 describes `astropy.table` and notes that there's similarities
> to the pandas package and numpy arrays. The functionality described in
> `astropy.nddata` also sounds somewhat generic. Some discussion as to why
> existing community tools (such as pandas) weren't used would be very welcome.
> In addition, if the answer to this question is that these other tools don't
> support all of the functionality the Astropy community needs (e.g. ability to
> handle Quantities with units in tables) are any efforts being made by the
> Astropy community to contribute these changes to packages such as Pandas?

This is an important point, but turns out to be very difficult to generalize
how these decisions are made (ironically!). From a *dependency* point of view,
Astropy deeply intertwines with the existing Python scientific ecosystem.
Where possible we always try to use other tools instead of building our own.
Some major astropy functionality is essentially an astro-relevant "wrapper"
around existing code. For example, modeling makes it easier to use fitting
functions from scipy with models that are primarily of interest in astronomy.
Other bits of functionality are really astro-specific or the work to make it
more "general" is difficult.  The coordinates and nddata subpackages are good
examples of this latter case - despite the general-sounding names, they are
focused on the needs of the astronomy community.  The `nddata` case the
referee mentions is particularly notable because the actual underlying machinery
is built completely on `numpy`, but essentially provides a standard "metadata"
language for N-dimensional data *in astronomy*.  That is, it has attributes like
`wcs`, `flags`, etc, that have a particular meaning and shared "cultural"
understanding in astronomy that is lost if it is made into a more "general" tool.

Other pieces were originally more astro-specific or had no equivalent when they
were started and have grown more generalizable as time goes on - e.g. units and
table. In general, it's considered "best effort" to generalize these things
because the first priority is basic tools for astronomy. Such changes come with
non-trivial added cost because that means it now needs a maintainer to do releases,
review pull requests, maintain the docs, etc. Using the astropy community helps
this, of course, but it is still more work than simply leaving these things in
the core. So how much anything is generalized dependes heavily on the time/interest
of an individual to do the porting. So for example some of the documentation and
testing machinery is now (after v2.0) getting moved from astropy into more
generally-useful independent packages.  Meanwhile, units/Quantity would require a
much larger effort to disentangle from astropy and maintain independently - but no
maintainer with enough time has thus far volunteered.

We have added a new section 2.7 outlining some of the above discussion.

Additionally, as the referee points out, `astropy.table` represents a particularly
unusual case for this.  It falls under the "historical" case in that it was started
before Pandas.  By the time Pandas became a standard tool in the pydata stack,
tables had already developed a fairly large user base in astronomy, so it was not
an option to completely eliminate it and say "use Pandas".  It had also developed
several features missing from Pandas but important for astronomy users (e.g.,
support for units, fully-fledged objects as columns, and support for the FITS
table concept of "multidimensional columns").  The `astropy.table` investigated
this possibility, but it was clear that some of the features would require major
incompatible changes *in Pandas* to do this, and it was clear by then that the
Pandas user base was too large to accept this.  This led to the current solution
of `to_pandas` and `from_pandas`, which allows transformation back-and-forth for
the use cases where this is supported. We have added a briefer summary of this
discussion to the text in the tables subsection (3.5.2).


________________________

> Note from statistical editor.

> In sec 3.11.3, a sentence can be added warning users about difficulties in
> interpreting the statistical significance of Lomb -Scargle periodogram peaks.
> This is important because unreliable interpretation is common, and warnings
> are not given elsewhere (e.g. NASA Exoplanet Archive Periodogram Service).
> A good reference is Vanderplas 2017 (https://arxiv.org/abs/1703.09824)
> soon to appear in the ApJ Suppl.

A short note directing the reader to Vanderplas (2007) was included.