forked from astropy/astropy-v2.0-paper
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathmain.tex
1375 lines (1145 loc) · 81.8 KB
/
main.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
\documentclass[modern]{aastex61}
\usepackage{xspace}
\usepackage[utf8]{inputenc}
\usepackage[T1]{fontenc}
\usepackage{ulem}
\newcommand{\escapecmd}[1]{\texttt{\detokenize{#1}}}
\submitjournal{ApJ}
\shorttitle{Astropy Project II}
\shortauthors{Astropy Project et al.}
% Packages / projects / programming - for consistency!
\newcommand{\package}[1]{\texttt{#1}\xspace}
\newcommand{\github}{\package{GitHub}}
\newcommand{\python}{\package{Python}}
\newcommand{\astropy}{Astropy\xspace}
\newcommand{\astropypkg}{\package{astropy}}
% For consistency:
\newcommand{\sectionname}{Section\xspace}
\renewcommand{\figurename}{Figure\xspace}
\newcommand{\equationname}{Equation\xspace}
\renewcommand{\tablename}{Table\xspace}
% For commenting - can be deleted before submission
\usepackage[colorinlistoftodos]{todonotes}
\newcommand{\inlinecomment}[2]{\todo[inline]{#1: #2}\xspace}
\newcommand{\comment}[2]{\todo{#1: #2}\xspace}
\begin{document}
\draft{\today}
\title{The Astropy Project}
\correspondingauthor{Astropy Coordination Committee}
\email{[email protected]}
\author{Astropy Collaboration}
\begin{abstract}
% I (Adrian) took a first stab at the abstract, but it needs some work. Feel
% free to modify!
The \astropy project supports and fosters the development of open-source and openly-developed
\python packages that provide commonly-needed functionality to the astronomical
community.
A key element of the \astropy project is the core package \astropypkg, which serves as the
foundation for more specialized projects and packages.
In this article, we provide an overview of the organization of the \astropy
project and summarize key features in the core package as of the recent major
release, version 2.0.
We then describe the project infrastructure designed to facilitate and support
development for a broader ecosystem of inter-operable packages.
We conclude with a future outlook of planned new features and directions for the
broader \astropy project.
\end{abstract}
% Adrian PW: how are there still no software keywords in AAS journals?!?
\keywords{%
Astrophysics - Instrumentation and Methods for Astrophysics
---
methods: data analysis
---
methods: miscellaneous
}
\section*{\textit{Notes and guidelines (to be removed)}}
\begin{itemize}
\item The goal is to produce a brief and informative paper that covers major
\astropy principles not mentioned in the first paper, the core v2.0
package, and infrastructure in \astropy project to support development
in \python.
\item We don't plan on including code in this paper, but if you think you
will need to include code in your section, please add it to a separate
\python module (.py file) and include it in this repository.
\item Use \escapecmd{\sectionname} not ``Section,'' \escapecmd{\figurename}
not ``Figure''
\item Use \escapecmd{\astropypkg} for the astropy package,
\escapecmd{\astropy} for the astropy project, \escapecmd{\python} not
``Python''
\item If your subpackage was included in Paper I, then please just include a
note on what the package does, a reference to paper I, and any new major
updates to your package
\item If your subpackage was not included, then please describe the
subpackage on level with what was in the first paper, and highlight any
major features in it. Typical length should be equivalent to one page
(single-column).
\item Please make sure you are logged into Overleaf or pushing a commit with
your information to be able to track the contributors to the paper.
\end{itemize}
\section*{Global TODO items}
\begin{itemize}
\item \sout{subpackage not sub-package!} BMS: checked as done 13/11
\item spell check!
\item check consistent astropy usage
\end{itemize}
\section{Introduction} \label{sec:intro}
% first draft by Moritz (hamogu)
% heavily edited by Adrian PW - let me know if you want to revert anything!
All astronomical research makes use of software in some way. Astronomy as a field has thus long supported the development of software tools
for astronomical tasks: from scripts that enable individual scientific research to software packages for small collaborations to data reduction pipelines for survey operations.
Some software packages are or were supported by large institutions and are intended for a wide range of users. These packages typically provide some level of documentation and user support or
training.
Other packages are developed by individual researchers or research groups and
are then typically used by smaller groups for more domain-specific purposes.
Whether for a package meant for wide distribution or for scripts and programs
for a specific research project, the implementation of astronomical software can
be eased through the use of a library that provides core functionality that is
common to many astronomical tasks.
The users of such software then also benefit from a community and ecosystem
built around a common foundation.
The \astropy project has grown to become this community, and the \astropypkg
core package a feature-rich library for \python-based astronomical software.
\inlinecomment{Tom R}{the 'this community' above is a bit too general - do we mean it is the only community in astronomy, or in the Python astronomy world? I'd change this to either 'to become this community for Python astronomy software' or 'to become such a community'. For instance yt had a community before we came along for the simulation community, so I want to make sure we don't claim we are the only community out there.}
\inlinecomment{Perry G}{Agreed}
The development of the \astropypkg core package began as a largely
community-driven effort to standardize core functionality for astronomical
software in \python.
In this way, its genesis differs from but builds upon many substantial and
% * <[email protected]> 2017-11-13T13:13:08.884Z:
%
% how do astroy builds upon these? or it's more like builds upon the developer experience?
%
%
% ^.
former astronomical software development efforts that were commissioned or
initiated through large institutional support, for example IRAF \citep[developed
at NOAO;][]{IRAF}, MIDAS \citep[developed at ESO;][]{MIDAS}, or Starlink
\citep[originally developed by a consortium of UK institutions and now
maintained by the East Asian Observatory;][]{starlink1982,starlink2013}.
% More recently, community-driven efforts have seen significant success in the astronomical sciences \citep{yt}
\inlinecomment{BMS}{I like the idea to reference yt here}
%
\python\footnote{\url{https://www.python.org/}} is an increasingly popular,
general-purpose programming language that is available under a permissive open
source software license free of charge for all major operating systems.
Stable and well-developed packages provide support for array representation and arithmetic
\citep[\package{numpy};][]{numpy}, a wide variety of functions for scientific
computing \citep[\package{scipy};][]{numpy}, and publication-quality plotting
\citep[\package{matplotlib};][]{matplotlib}. Tens of thousands of other packages
are available which can help with tasks that are not astronomy specific but
might be performed in the course of astronomical research, for example interfacing with databases or statistical inference.
\python has become especially popular in the quantitative sciences,
where researchers must simultaneously produce research, perform data analysis,
and develop software.
A large part of this success owes itself to a vibrant community of developers
and a continuously-growing ecosystem of tools and web services that enable
easier collaboration on software development, easier writing and sharing of
software documentation, and continuous testing and validation of software.
More recently, package managers such as
Anaconda\footnote{\url{https://anaconda.org/}} have streamlined the installation
process for most packages, therefore significantly reducing the barrier to entry
for using many such libraries.
The \astropy project aims to provide an open-source, open-development core
package (\astropypkg) and an ecosystem of \emph{affiliated packages} that
support astronomical functionality in the \python programming language.
The \astropypkg core package is now a feature-rich library of sufficiently
general tools and classes that supports the development of more specialized
code. An example of such functionality is reading and writing FITS files: it would be
time consuming and impractical for multiple groups to implement the FITS
standard \citep{FITS} and maintain software for such a general-purpose need.
Another example of such a common task is in dealing with representations of and
transformations between astronomical coordinate systems.
The \astropy project aims to develop and provide high quality code and
documentation according to the best practices in software development.
The project makes use of these tools to do so without central institutional
oversight.
The first public release of the \astropypkg package is described in
\cite{astropy}. Since then, the \astropypkg package has been
used in hundreds of projects, and the scope of the package has grown
considerably. At the same time, the community of astronomers
contributing to the project has grown tremendously and an ecosystem
of packages supporting or affiliated with the \astropypkg core has
developed.
In this paper, we describe the current status of the \astropy community and the
\astropypkg core package and discuss goals for future development.
We start by describing the way the \astropy project functions and is organized
in \sectionname~\ref{sec:org}. We then describe the main software efforts
developed by the \astropy project itself: a core package called \astropypkg
(\sectionname~\ref{sec:core}) and several separate packages that help maintain
the infrastructure for, e.g., testing and documentation
(\sectionname~\ref{sec:infrastructure}). We end with a short vision for
the future of \astropy in particular and astronomical software in general
in \sectionname~\ref{sec:future}.
\section{Organization and infrastructure}
\label{sec:org}
\subsection{Coordination of Astropy}
% draft by ?? (BMS thinks Erik)
% edited by Adrian PW
\label{sect:coordcom}
From its inception, \astropy has required coordination to ensure the project
as a whole and its coding efforts are consistent and reasonably efficient.
While many \python projects adopt a ``Benevolent Dictator For Life'' (BDFL)
model, \astropy has instead opted for a \emph{coordination committee}. This
is in part due to the nature of the project as a large-scale collaboration
between many contributors with many interests, and in part due to simply the
amount of work that needs to get done. For the latter reason, the
project has expanded the committee from three to four members starting in
2016.
For resolving disagreements about the \astropypkg core package or other \astropy-managed code, the coordination committee primarily acts to work toward consensus, or when consensus is difficult to achieve, generally acts as a ``tie-breaker.''
The committee also oversees affiliated package applications to ensure they are in keeping with \astropy's vision and philosophy, as well as the associated procedures.
Additionally, the committee oversees the assignment of roles (primarily driven by already-existing contributions), and increasingly has acted as the ``face'' of the Project, providing contact with organizations like NumFOCUS (the body that holds any potential funding in trust for \astropy) or the American Astronomical Society (AAS).
\subsection{Astropy development model}
% draft by Adrian PW and Erik T.
Code is contributed to the \astropypkg core package or modified through ``pull
requests'' (via \github) that often contain several \texttt{git} commits.
Pull requests may fix bugs, implement new features, or improve or modify the
infrastructure that supports the development and maintenance of the package.
Individual pull requests are generally limited to a single conceptual addition
or modification to make code review tractable.
Pull requests that modify or add code to a specific subpackage must be reviewed
and approved by one of the subpackage maintainers before it is merged into the
core codebase.
Bugs and feature requests are reported via the \github issue tracker and labeled
with a set of possible labels that help classify and organize the issues.
The development workflow is detailed in the \astropypkg
documentation.\footnote{\emph{How to make a code contribution}, \url{http://docs.
astropy.org/en/stable/development/workflow/development_workflow.html}}
As of version 2.0, \astropypkg contains $212244$ lines of code\footnote{This
line count includes comments, as these are often as important for
maintainability as the code itself. Without comments there are $142197$ lines
of code.} contributed by $232$ unique contributors over $19270$ \texttt{git}
commits.
The commit distribution \inlinecomment{Tom R}{too vague, maybe 'the distribution of number of commits as a function of ...'} follows an approximate power-law (with log-slope of
$\approx -0.47$) \inlinecomment{AG}{round to -0.5? This is astronomy, after all.}, meaning most of the commits were and are contributed by a
relatively small fraction of the contributor pool.
Roughly half of the commits were contributed by the top $4$
contributors, while $\sim 90$ percent of the commits came from the top $24$
contributors.
\subsection{APEs - Astropy Proposals for Enhancement}
% draft by hamogu
% edited by Adrian PW
Central to the success of \astropy is an open environment where anybody can
contribute to the project.
However, this model leads to ``organic'' growth where different features are
implemented by different people with different programming styles and
interfaces.
Thus, \astropy has a mechanism to more formally propose significant changes to
the core package (e.g., re-writing the coordinates subpackage; \citealt{ape5}),
to plan out major new features (e.g., a new file format; \citealt{ape6}), or
institute new organization-wide policies (e.g., adopting a code of conduct;
\citealt{ape8}).
This mechanism is called ``Astropy Proposal for Enhancement'' (APE) and are
modeled after the ``Python Enhancement Proposals'' (PEP) that guide the
development of the \python programming language.
In an APE, one or more authors describe in detail the proposed changes or
additions, including a rationale for the changes, how these changes will be
implemented, and, in the case of code, what the interface will be \citep{ape1}.
The APEs are discussed and refined by the community before much work is invested
into a detailed implementation; anyone is welcome to contribute to these
discussions during the open consideration period. APEs are proposed via pull
requests on a dedicated GitHub repository\footnote{\url{https://github.com/astropy/astropy-APEs}},
and anyone can therefore read the proposed APEs and leave comments in-line.
In previous APEs a community consensus emerged and APEs are accepted and become
the basis for future work at this point.
In cases where consensus cannot be reached, the
\astropy coordination committee can decide to close the discussion and
make an executive decision based on the community input on the APE in question.
\subsection{Concept of affiliated packages}
% it may be described in previous subsection, if not the Brigitta can try to
% write it up
% edited by Adrian PW
A major part of the \astropy project is the concept of
``Affiliated Packages''. An affiliated package is an astronomy-related
\python package that is not part of the \astropypkg core package, but
has requested to be included as part of the \astropy project's
community. These packages support the goals and vision of \astropy of
improving code reuse, interoperability, and embracing good coding
practices such as testing and thorough documentation.
Affiliated packages contain functionality that is more specialized,
have license incompatibilities, or have external dependencies (e.g., GUI
libraries) that make these packages more suitable to be separate from the
\astropypkg core package.
Affiliated packages may also be used to develop substantial new functionality
that will eventually be incorporated into the \astropypkg core package.
New functionality benefits from having a rapid development and release cycle that is not tied to that of the \astropypkg core.
% AG: rephrased this.
%Initially separate packages with typically rapid initial development and new experimental features benefit from having a release cycle that is not tied to the
Affiliated packages are listed on the main \astropy website and advertised to the community through \astropy mailing lists, so becoming an affiliated package is a good way for new and existing packages to gain exposure while at the same time promoting \astropy's high standard for code and documentation quality. This process of listing and promoting affiliated packages is one way in which the \astropy project tries to help with \inlinecomment{AG}{Increase?} code reuse in the astronomical community.
Packages can become affiliated to \astropy by applying for this status on a public mailing list. The coordination committee (\sectionname~\ref{sect:coordcom}) reviews such requests and issues recommendations for the improvement of a package where possible.
%\subsection{Mixin}
%\subsection{Accuracy testing across many different implementation}
\subsection{Release cycle and Long Term Support}
% This could also include information about the development cycle
% Brigitta can writes this up if there's no other taker
% edited by Adrian PW
The \astropypkg package has a regular release schedule consisting of new significant
releases every 6 months, with bugfix releases as needed \citep{ape2}.
The major releases contain new features or any significant changes, whereas
the bugfix releases only contain fixes to code or documentation but no new
features.
Some versions are additionally designated as ``Long-term support'' (LTS)
releases, which continue to receive bugfixes for 2 years following the release
with no changes to the API\@.
The LTS versions are ideal for pipelines and other applications where API
stability is essential.
The latest LTS release (v2.0) is also the last one that supports \python 2, and
will receive bug fixes until the end of 2019 \citep{ape10}.
The version numbering of the \astropypkg core package reflects this release
scheme: the core package version number uses the form x.y.z, where ``x'' is
advanced for LTS releases, ``y'' is advanced for non-LTS feature releases, and
``z'' is advanced for bugfix releases.
\subsection{Distribution of packages}
The \astropypkg core package is available from several of the \python distributions for scientific
% * <[email protected]> 2017-10-27T07:13:51.093Z:
%
% > \astropypkg i
% I changed a few of the macros in here to refer to the package rather than the project because it is the package being discussed here.
%
% ^.
computing, such as Anaconda or Enthought Canopy, which include also some of
\astropy's affiliated packages.
Alternatively, \astropypkg can be readily installed from the Python Package
Index\footnote{\url{https://pypi.python.org/pypi}} (PyPI)
\inlinecomment{Tom R}{make footnote link directly to astropy on PyPI?}
using \texttt{pip}, the standard command-line tool for installing \python packages.
For most recent \astropypkg releases, pre-built wheel distributions are obtainable
from PyPI for Windows, Mac OS X and Linux systems that provide faster
installation compared to the source distribution.
% hamogu: I think that's too detailed for a journal paper.
% It's in the docs if anyone needs it.
%Optionally, the version of
%the package and the dependencies needed to run or build the software are given
%by a specifier to the pip package manager as described in PEP 440\footnote{For
%more detailed installation instructions and optional flags see
%\url{http://docs.astropy.org/en/stable/install.html\#installing-astropy}}.
Packaged versions of \astropy are also available for several
operating systems that are managed by a system's package manager,
including Debian, Ubuntu, Fedora, Gentoo, Arch Linux, Mac OS X (e.g.,
via brew, macports or fink) and FreeBSD\@. However, on occasion these packages
may be outdated compared with the latest stable release of \astropy accessible
from the PyPI\@. Pre-built binary conda packages for all of the affiliated packages are also available via the \texttt{astropy} conda channel for Linux, Mac OS X, and MS Windows.
% * <[email protected]> 2017-10-27T07:16:27.310Z:
%
% > re-built binary conda packages
% Feel free to remove if this is too detailed. I do think the available of the whole ecosystem across all platforms has helped speed its adoption.
%
% ^.
%\subsection{Support of Astropy}
% Removing unless this gets written about in
% a later version
% This section should be a straight forward/factual description of the current
% funding and support provided to astropy with the purpose of explaining the
% current structure to the community
%\subsection{Difficulty of reversing design choices}
%Deciding if a feature should be included
%difficulty to decide where general use ends and "handy feature for some" starts, i.e. how to reject PRs or deal with maintenance burden
\section{Astropy Core Package v2.0}
\label{sec:core}
% Adrian
The \astropy project aims to provide \python-based packages for all tasks that
are commonly needed in a large subset of the astronomical community.
At the foundation is the \astropypkg core package, which provides general
functionality (e.g., coordinate transformations, reading and writing astronomical
files) or base classes for other
packages to utilize for a common interface (e.g., \texttt{NDData}).
In this section, we highlight new features introduced or substantially improved
since version v0.2 (previously described in \citealt{astropy}).
% \subsection{Analytic Functions}
% moved to models and depreciated --
% just here for completeness at the moment
\inlinecomment{BMS}{I suggest to order the subsection lexicographically as reading the draft the current order seemed very random (unlike at the docs page where they the grouping logic is obvious)}
\inlinecomment{Tom R}{the only issue is if we rely on having e.g. units described first if they are used later}
\subsection{Units}\label{sec:units}
% Adrian
The \texttt{astropy.units} subpackage adds support for representing units and
numbers with associated units --- ``quantities'' --- in code.
Historically, quantities in code have often been represented simply as numbers,
with units implied or noted via comments in the code because of considerations
about speed: having units associated with numbers inherently adds overhead to
numerical operations.
In \texttt{astropy.units}, \texttt{Quantity} objects subclass the \texttt{numpy}
array object and have been designed with speed in mind.
\inlinecomment{Perry G}{Agreeing with Tom below, I would avoid using terms like subclass, decorators, mixin, namespace, (and perhaps even class, instead using objects consistently; this is no place to be pendantic about the distinction between classes and objects; even to the extent of adding a qualifying sentence somewhere like: "In this paper we will use object when we mean object or classes; in software there are important distinctions between the two terms, but they are not relevant in this paper."), and the like and replace them with more generic descriptions, e.g., "extended", or "tools to make defining functions that can use quantities simple to write" }
\inlinecomment{Tom R}{I think saying 'subclass' here is too detailed and is mentioned below in 3.1.1 if needed - 'make it possible to attach units to numpy arrays' or something like that?}
As of \astropypkg version 2.0, units and quantities are prevalent in most other
\astropypkg subpackages and are thus a key concept for using the package as a
whole.
Units are intimately entwined in the definition of astronomical coordinates and
thus nearly all functionality in the \texttt{astropy.coordinates} subpackage
(see \sectionname~\ref{sec:coordinates}) depends on this functionality.
For most other subpackages, quantities are at least accepted, and often expected
by default.
The motivation and key concepts behind this subpackage were described in detail
in the previous paper \citep{astropy}, and thus here we primarily highlight new
features and improvements.
\subsubsection{Interaction with \package{numpy} arrays}
The \texttt{Quantity} object
is now a subclass of the \texttt{numpy.ndarray} object and therefore
works well with many of the functions in \texttt{numpy} that support
array operations. For example, \texttt{Quantity} objects with angular
units can be passed in to the trigonometric functions implemented in
\texttt{numpy} (e.g., \texttt{numpy.cos}) and do the right thing: the
units are internally converted to radians (what the \texttt{numpy}
trigonometric functions expect) before being passed to \texttt{numpy}.
\subsubsection{Speed improvements}
\inlinecomment{Tom R}{I wonder if we might just want to mention at the top level that performance has been improved in places? This seems a bit too detailed}
The \texttt{Quantity} object is now significantly
faster than in early versions of \astropypkg. This is in large part due
to the fact that \texttt{Quantity} subclasses the \texttt{numpy.ndarray}
object, but even relative to more recent versions of \astropypkg,
operations with \texttt{Quantity} objects are generally faster. This is
largely thanks to improvements in the way \texttt{numpy} arrays can be
subclassed.
\subsubsection{Logarithmic units and magnitudes}
By default, taking the logarithm of
a \texttt{Quantity} object with non-dimensionless units intentionally
fails.
However, some well-known units are actually logarithmic quantities,
where the logarithm of the value is taken with respect to some reference
value.
Examples include astronomical magnitudes, which are logarithmic fluxes,
and decibels, which are more generic logarithmic ratios of quantities.
Logarithmic, relative units are now supported in \texttt{astropy.units}.
\subsubsection{Using quantities in tables}
\inlinecomment{Tom R}{this is a new feature of table, not units, so I think we don't want to mention this here}
The \texttt{astropy.table} subpackage (see
\sectionname~\ref{sec:table}) now supports using quantities in tables
with the \texttt{QTable} class.
As a result, units specified in FITS tables are supported, and tables
with quantities can be read and written to various file formats.
\subsubsection{Defining functions that require quantities}
When writing code or
functions that expect \texttt{Quantity} objects, we often want to
enforce that the input units have the correct type.
For example, we may want to require only length-type \texttt{Quantity}
objects.
These requirements often lead to implementing repetitive code for
validating \texttt{Quantity} inputs.
\texttt{astropy.units} now provides a \python decorator,
\texttt{quantity\_input()}, that does this verification automatically.
\subsection{Constants}
% David S.
% Edited by Adrian PW
The \texttt{astropy.constants} subpackage provides a selection of physical and
astronomical constants as \texttt{Quantity} objects (see
\sectionname~\ref{sec:units}).
A brief description of this package was given in \citep{astropy}.
In version 2.0, the built-in constants have been organized into modules for
specific versions of the constant values.
For example, physical constants have \texttt{codata2014} \citep{codata2014} and
\texttt{codata2010} versions.
Astronomical constants are organized into \texttt{iau2015} and \texttt{iau2012}
modules to indicate their sources (resolutions from the International
Astronomical Union, IAU).
The \texttt{codata2014} and \texttt{iau2015} versions are combined into the
default constant value version: \texttt{astropyconst20}.
For compatibility with \astropypkg version 1.3 and earlier, the \texttt{astropyconst13}
namespace \inlinecomment{Tom R}{jargon alert: 'namespace'} is still available and provides access to the adopted versions of the
constants from earlier versions of \astropypkg.
To use previous versions of the constants as \emph{units} (e.g., solar masses),
the values from the desired namespace have to be imported directly; with version
2.0, \texttt{astropy.units} uses the \texttt{astropyconst20} versions.
Astronomers using \texttt{astropy.constants} should take particular note of the
constants provided for Earth, Jupiter, and the Sun.
Following IAU 2015 Resolution B3 \citep{iau2015b3}, nominal values are now given
for mass parameters and radii.
The nominal values will not change even as ``current best estimates'' are
updated.
\subsection{Coordinates}
\label{sec:coordinates}
% Adrian, Erik
The \package{astropy.coordinates} subpackage is designed to support representing
and transforming celestial coordinates and, new in version 2.0, velocities.
The framework heavily relies on the \package{astropy.units} subpackage, and most
inputs to objects in this subpackage are expected to be \texttt{Quantity}
objects.
Some of the machinery also relies on the Essential Routines of Fundamental
Astronomy (ERFA) \texttt{C} library for some of the critical underlying
transformation machinery \citep{erfa}, which is based on the Standards Of
Fundamental Astronomy (SOFA) effort \citep{sofa}.
A key concept behind the design of this subpackage is that coordinate
\textit{representations} and \textit{reference systems / frames} are independent
of one another.
For example, a set of coordinates in the International Celestial Reference
System (ICRS) reference frame could be represented as spherical (right
ascension, declination, and distance from solar system barycenter) or Cartesian
coordinates ($x$, $y$, $z$ with the origin at barycenter).
They can therefore change representations independent of being transformed to
other reference frames (e.g., the Galactic coordinate frame).
The classes that handle coordinate representations (the \texttt{Representation}
classes) act like three-dimensional vectors and thus support vector arithmetic.
The classes that represent reference systems and frames (the \texttt{Frame}
classes) internally use \texttt{Representation} objects to store the coordinate
data---that is, the \texttt{Frame} classes accept coordinate data, either as a
specified \texttt{Representation} object, or using short-hand keyword arguments
to specify the components of the coordinates.
These preferred representation and short-hand component names differ between
various astronomical reference systems.
For example, in the ICRS frame, longitude and latitude are right ascension
(\texttt{ra}) and declination (\texttt{dec}), whereas in the Galactic frame, the
spherical angles are Galactic longitude (\texttt{l}) and latitude (\texttt{b}).
Each of the \texttt{Frame} classes define their own component names and
preferred \texttt{Representation} class.
The frame-specific component names map to corresponding components on the
underlying \texttt{Representation} object that internally stores the coordinate
data.
For most frames the preferred representation is spherical, although this is
determined primarily by the common use in the astronomical community.
Many of the \texttt{Frame} classes also have attributes specific to the
corresponding reference system that allow the user to specify the frame.
For example, the Fifth Fundamental Catalogue (FK5) reference system requires
specifying an equinox to determine the reference frame.
If required, these additional frame attributes must be specified along with the
coordinate data when a \texttt{Frame} object is created.
\figurename~\ref{fig:frame-transform-graph} shows the network of possible
reference frame transformations as currently implemented in
\texttt{astropy.coordinates}.
Custom, user-implemented \texttt{Frame} classes that define transformations to
any reference frame in this graph can then be transformed to any of the other
connected frames.
\begin{figure}
\includegraphics[width=\textwidth]{coordinates_graph.pdf}
\caption{%
The full graph of possible reference frame transformations implemented in
\texttt{astropy.coordinates}.
\label{fig:frame-transform-graph}
}
\end{figure}
The typical user doesn't usually have to interact with the \texttt{Frame} or
\texttt{Representation} classes directly.
Instead, \texttt{astropy.coordinates} provides a high-level interface to
representing astronomical coordinates through the \texttt{SkyCoord} class.
The \texttt{SkyCoord} class was designed to provide a single class that
accepts a wide range of possible inputs.
It supports coordinate data in any coordinate frame in any representation by
internally using the \texttt{Frame} and \texttt{Representation} classes.
In what follows, we briefly highlight key new features in
\texttt{astropy.coordinates}.
\subsubsection{Local Earth coordinate frames}
In addition to representing celestial
coordinates, \astropypkg now supports specifying positions on the Earth in
a number of different geocentric systems with the \texttt{EarthLocation}
class.
With this, \astropypkg now supports Earth-location-specific coordinate
systems such as the altitude-azimuth (\texttt{AltAz}) or horizontal system.
Transformations between \texttt{AltAz} and any Barycentric coordinate frame
also requires specifying a time using the \texttt{Time} class from
\texttt{astropy.time}.
With this new functionality, many of the common tasks associated with
observation planning can now be completed with \astropypkg or the
\astropy-affiliated package \package{astroplan}\citep{astroplan_AAS}.
\subsubsection{Proper motion and velocity transformations}
In addition to positional coordinate data, the \texttt{Frame} classes now
also support velocity data.
As the default representation for most frames is spherical, most of the
\texttt{Frame} classes expect proper motion and radial velocity components
to specify the velocity information.
The names of the proper motion components all start with \texttt{pm} and
adopt the same longitude and latitude names as the positional components.
Transforming coordinates with velocity data is also supported, but in some
cases the transformed velocity components have limited accuracy because the
transformations are done numerically.
The visualization of the coordinate frame transform graph highlights which
velocity transformations can be done exactly and which transformations are
done using a finite-difference scheme.
The low-level interface for specifying and transforming velocity data (see
the next point) is currently experimental.
As such, in v2.0, only the \texttt{Frame} classes (and not the
\texttt{SkyCoord} class) support handling velocities.
\subsubsection{Derivatives of coordinate representations}
\inlinecomment{Tom R}{does this really warrant a separate section or is it just a byproduct of the velocity stuff that can be mentioned in 3.3.2?}
As mentioned above, the \texttt{Representation} classes act like
three-dimensional vectors.
\texttt{astropy.coordinates} now also supports handling first derivatives of
vectors / representations through the new \texttt{Differential} classes.
The \texttt{Differential} classes are currently used internally within the
\texttt{Frame} classes to store the velocity data.
\subsubsection{Solar System Ephemerides}
Also new is support for computing ephemerides of major solar system bodies
and outputting the resulting positions as coordinate objects.
These ephemerides can be computed either using analytic approximations from
ERFA, or from downloaded JPL ephemerides (the latter requires the
\package{jplephem}\footnote{\url{https://github.com/brandon-rhodes/python-jplephem}}
optional dependency and an internet connection).
% hamogu: I think the specific names of functions can be in the docs
% and don't need to be repeated here.
%The \texttt{get\_body} and \texttt{get\_body\_barycentric} functions provide
%the gateway to this functionality, yielding \texttt{SkyCoord} objects given
%a specific body and choice of ephemeris source.
\subsubsection{Accuracy of coordinate transformations}
In order to check the accuracy of the coordinate transformations in \package{astropy.coordinates}, we have created a set of benchmarks that we use to compare transformations between a set of coordinate frames for a number of packages\footnote{\url{http://www.astropy.org/coordinates-benchmark/summary.html}}. Since
no package can be guaranteed to implement all transformations to arbitrary precision, and since some transformations are sometimes subject to interpretation of standards (in particular in the case of Galactic coordinates), we do not designate any of the existing packages as the `ground truth' but instead compare each tool to all other tools to find. The benchmarks are thus useful beyond the \astropy project since they allow all of the tools to be compared to all other tools. The tools included in the benchmark at the moment include the \astropypkg core package, Kapteyn \citep{kapteyn}, NOVAS \citep{novas}, PALpy \citep{pal}, PyAST \citep[a wrapper for AST, described in][]{ast}, PyTPM\footnote{\url{https://github.com/phn/pytpm}}, PyEphem \citep{pyephem}, and pySLALIB \citep[a Python wrapper for SLALIB, described in][]{slalib}.
The benchmarks are meant to evolve over time and include an increasing variety of cases. At the moment, the benchmarks are set up as follows - we have generated a standard set of 1000 pairs of random longitudes/latitudes that we use in all benchmarks. Each benchmark is then defined using an input and output coordinate frame, using all combinations of FK4, FK5, Galactic, ICRS and Ecliptic frames. For now we set the epoch of observation to J2000, and the frame to J2000 in the case of the FK5 and ecliptic frames and B1950 for the FK4 frame, but in future we plan to include a larger variety of epochs and equinoxes, as well as tests of conversion to/from Altitude/Azimuth. For each benchmark, we convert the 1000 longitudes/latitudes from the input/output frame with all tools and quantify the comparison by looking at the median, minimum, maximum, and standard deviation of the absolute separation of the output coordinates from each pair of tools. \tablename~\ref{tab:coordinate_benchmarks} gives an example of the relative accuracy of the conversion from FK4 to Galactic coordinates for all pairs of tools. This shows for example that \astropy, Kapteyn and PyTPM agree perfectly (to the precision shown), while PALpy, pySLALIB, and PyAST also agree perfectly amongst themselves, but show an offset of around 0.2\arcsec with the former three packages. Finally, PyEphem disagrees with other packages by 0.4--0.8\arcsec. These values are only meant to be illustrative and will change over time as the benchmarks are refined and packages are updated.
\inlinecomment{Perry G}{Wouldn't the table be better represented as a matrices? Or perhaps as a matrix as a crude image with colors or intensities representing the size of the quantity. Is the difference between median and mean significant enough to warrant showing both? A link to the actual numbers may satisfy readers that want the exact values, but I suspect the majority don't need to see that precision.}
\begin{table}
\begin{center}
\begin{tabular}{llcccc}
\hline
\hline
Package 1 & Package 2 & Median & Mean & Maximum & Std. Dev. \\
& & arcsec & arcsec & arcsec & arcsec \\
\hline
astropy & kapteyn & 0.000 & 0.000 & 0.000 & 0.000 \\
\nodata & palpy & 0.218 & 0.196 & 0.249 & 0.056 \\
\nodata & pyast & 0.218 & 0.196 & 0.249 & 0.056 \\
\nodata & pyephem & 0.459 & 0.458 & 0.860 & 0.231 \\
\nodata & pyslalib & 0.218 & 0.196 & 0.249 & 0.056 \\
\nodata & pytpm & 0.000 & 0.000 & 0.000 & 0.000 \\
kapteyn & palpy & 0.218 & 0.196 & 0.249 & 0.056 \\
\nodata & pyast & 0.218 & 0.196 & 0.249 & 0.056 \\
\nodata & pyephem & 0.459 & 0.458 & 0.860 & 0.231 \\
\nodata & pyslalib & 0.218 & 0.196 & 0.249 & 0.056 \\
\nodata & pytpm & 0.000 & 0.000 & 0.000 & 0.000 \\
palpy & pyast & 0.000 & 0.000 & 0.000 & 0.000 \\
\nodata & pyephem & 0.563 & 0.570 & 1.012 & 0.253 \\
\nodata & pyslalib & 0.000 & 0.000 & 0.000 & 0.000 \\
\nodata & pytpm & 0.218 & 0.196 & 0.249 & 0.056 \\
pyast & pyephem & 0.563 & 0.570 & 1.012 & 0.253 \\
\nodata & pyslalib & 0.000 & 0.000 & 0.000 & 0.000 \\
\nodata & pytpm & 0.218 & 0.196 & 0.249 & 0.056 \\
pyephem & pyslalib & 0.563 & 0.570 & 1.012 & 0.253 \\
\nodata & pytpm & 0.459 & 0.458 & 0.860 & 0.231 \\
pyslalib & pytpm & 0.218 & 0.196 & 0.249 & 0.056 \\
\hline
\end{tabular}
\end{center}
\caption{Comparison of the accuracy of the FK4 to Galactic transformation between different packages.\label{tab:coordinate_benchmarks}}
\end{table}
\subsection{Time}
\label{sec:time}
% Adrian PW
The \package{astropy.time} subpackage focuses on supporting time scales (e.g.,
UTC, TAI, UT1) and time formats (e.g., Julian date, modified Julian date) that
are commonly used in astronomy.
This functionality is needed, for example, to calculate barycentric corrections
or sidereal times.
\package{astropy.time} is currently built on the ERFA (\citealt{erfa}) C
library, which replicates the Standards of Fundamental Astronomy (SOFA;
\citealt{sofa}) but is licensed under a three-clause BSD license.
The package was described in detail in \citet{astropy} and has
stayed stable for the last several versions of \astropypkg.
Thus, in what follows, we only highlight significant changes or new features
since the previous \astropy paper.
\subsubsection{Barycentric and Heliocentric corrections}
Detailed eclipse or transit
timing requires accounting for light travel time differences from the
source to the observatory because of the Earth's motion.
It is therefore common to instead convert times to the Solar System
barycenter or heliocenter where the relative timing of photons is
standardized.
With the location of a source on the sky (i.e. a \texttt{SkyCoord}
object), the location of an observatory on Earth (i.e. an
\texttt{EarthLocation} object), and time values as \texttt{Time}
objects, the time corrections to shift to the solar system barycenter or
heliocenter can now be computed with \package{astropy.time} using the
\texttt{light\_travel\_time} method of a \texttt{Time} object.
\subsubsection{\texttt{Time} objects as \texttt{Table} columns} \texttt{Time} objects
can now be added as columns in \package{astropy.table} \texttt{Table}
objects. This is described in more detail in
\sectionname~\ref{sec:table} with the concept of a ``mixin'' column
type.
\inlinecomment{Tom R}{I would remove this tiny section. As for units, I think we should only mention the whole mixin stuff in the table section, not for every single object that can go in a table}
\subsection{Data containers}
\subsubsection{nddata}
The \package{astropy.nddata} supackage provides three types of functionality: an
abstract interface for representing generic arbitrary-dimensional datasets
intended primarily for subclassing by developers of other packages, concrete
classes building on this interface, and utilities for manipulating these kind of
datasets.
The \texttt{NDDataBase} class provides the abstract interface for gridded data
with attributes for accessing metadata, the world-coordinate system (WCS),
uncertainty arrays matched to the data shape, and other traits.
Building on this interface, the \texttt{NDData} class provides a minimal working implementation for storing \package{numpy} arrays. These classes serve as useful base classes for package authors wishing to develop their own classes for specific use cases and as containers for exchanging gridded data.
The classes \texttt{NDDataRef}, \texttt{NDDataArray}, and \texttt{CCDData} extend the base storing functionality with options to do basic arithmetic (addition, subtraction, multiplication, and division) including error propagation in limited cases and slicing of the dataset based on grid coordinates that appropriately handles masking, errors, and units (if present). Additionally, the \texttt{CCDData} class also provides reading and writing from and to FITS files and uses data structures from \astropypkg, like \texttt{WCS}, to represent the contents of a file abstractly.
The \package{astropy.nddata.utils} module provides utilities that can operate on either plain \package{numpy} arrays or any of the classes in the \package{astropy.nddata} subpackage. It features a class for representing two-dimensional image cutouts, allowing one to easily link pixels in the cutout to pixels in the original image or from the image to the cutout, to convert between world and pixel coordinates in the cutout, and to overlay the cutout on images. Functions to enlarge or reduce an image by doing block replication or reduction are also provided.
\subsubsection{Tables}
\label{sec:table}
% fleshed out / edited by Adrian PW
The \package{astropy.table} subpackage provides functionality for representing
and manipulating heterogeneous data.
\inlinecomment{APW:}{Need more introduction to the table subpackage!}
The package was described in detail in \cite{astropy}.
Next, we summarize key new features or updates to \package{astropy.table}.
\inlinecomment{Perry G}{Some mention that some of these are inspired by Pandas?}
\subsubsection{Support for grouped table operations}
Tables can contain data that naturally form groups, for example monitoring of an object in different bands.
We may then want to split the table or a column into these groups (in this example: observations in different filters) and
compute some statistic on the data, or filter the groups based on some
criteria.
Grouped operations are now supported by \texttt{Table} objects.
%,
%mainly through the \texttt{group\_by()} method and subsequent operations
%on the grouped version of the table or column.
\subsubsection{Support for table concatenation}
Tables can be combined in several
different ways.
If two tables have the same rows, we may want to stack them
``vertically'' to create a new table with the same columns but all rows.
If two tables are row-matched but have distinct columns, we may want to
stack them ``horizontally'' to create a new table with the same rows
but all columns.
For other situations, more general table concatenations or joins are
also possible when two tables share some common columns.
%All three of these operations are now possible with \texttt{Table}
%objects through the \texttt{vstack}, \texttt{hstack}, and \texttt{join}
%functions in \package{astropy.table}.
\subsubsection{Mixin columns for time and coordinates}
The \texttt{Table} object now supports \texttt{Quantity}, sky coordinate (\texttt{SkyCoord}),
astronomical \texttt{Time} objects and and other objects to serve
as array-like containers for column data through the ``mixin'' protocol.
\inlinecomment{Tom R}{this is waaaaay too short and jargony. It's a very cool feature that deserves more reader-friendly text!}
%hamogu: I've commented this out.
% I think it's very specific and not relevant to non-developers.
% If we want to keep it, we need to add at least 102 senteces about what Pandas
% is and I think that's more text than is warrented for this feature.
%\subsubsection{Transform to and from \texttt{Pandas} \texttt{DataFrame} objects}
%\texttt{Table} objects can now be created from a \texttt{DataFrame}
%object (\texttt{from\_pandas()}), or exported to a \texttt{DataFrame}
%object (\texttt{to\_pandas()}).
%\texttt{DataFrame} objects do not support multi-dimensional array
%columns, so this conversion will fail if the source \texttt{Table}
%object contains any such columns.
\subsection{io}
% Simon
The \package{astropy.io} subpackage provides support for reading and writing
data to a variety of ASCII and binary file formats, such as a wide range of
ASCII data table formats, FITS, and VOTable.
It also provides a unified interface for reading and writing data with these
different formats using the \package{astropy.table} subpackage.
For many common cases this simplifies the process of file input and output and
reduces the need to master the separate details of all the I/O packages within
\astropypkg.
%\inlinecomment{SCO}{Pyfits deprecation in favor of io.fits ?}
%\inlinecomment{SCO}{Table and FITS tables}
%\subsubsection{Unified file read/write interface}
%\inlinecomment{Check when it was added (0.3?). Overview of the supported formats (ASCII,FITS, votable, HTML, JSViewer/Datatables).}
We summarize key new features or updates to \package{astropy.io} below,
organized by sub-subpackage.
\inlinecomment{Tom R}{is a sub-subpackage a thing, or is it just 'subpackage' all the way down?}
\subsubsection{Enhanced CSV format}
One of the problems when storing a table in an ASCII format is
preserving table meta-data such as comments, keywords and column data
types, units, and descriptions. Using the newly defined \emph{Enhanced
Character Separated Values} (ECSV, \citet{ape6}) format it is now
possible to write a table to an ASCII-format file and read it back
with no loss of information. The ECSV format has been designed to be
both human-readable and compatible with most simple CSV readers.
\subsubsection{\texttt{.ascii}: fast readers and writers}
The \package{astropy.io.ascii} module now includes a significantly
faster Cython/C engine for reading and writing ASCII files. This is
available for the most common formats.
%following formats: \texttt{basic},
% \texttt{commented\_header}, \texttt{csv}, \texttt{no\_header},
% \texttt{rdb}, and \texttt{tab}.
On average the new engine is about
4 to 5 times faster than the corresponding pure-\python
implementation, and is often comparable to the speed of the
\package{Pandas} \citep{pandas} ASCII file interface. The fast
reader has parallel processing option that allows harnessing
multiple cores for input parsing to achieve even greater speed
gains.
By default, \texttt{read()} and \texttt{write()} will attempt to use
the fast C engine when dealing with compatible formats. Certain
features of the full read / write interface are not available in the
fast version, in which case the pure-\python version will automatically
be used.
\subsubsection{\texttt{.ascii}: HTML tables}
The \package{astropy.io.ascii} subpackage now provides the capability
to read a table within an HTML file or web URL into an astropy
\texttt{Table} object. This requires the \package{BeautifulSoup4}
package to be installed. Conversely a \texttt{Table} object can now
be written out as an HTML table.
\subsubsection{\texttt{.fits}: command-line scripts (\texttt{fitsheader}, \texttt{fitsinfo})}
The \package{astropy.io.fits} subpackage now provides command-line
scripts for printing a summary of the HDUs in a FITS file(s)
(\texttt{fitsinfo file.fits}) and for printing the header information
to the screen in a human-readable format (\texttt{fitsheader
file.fits}).
\subsubsection{\texttt{.fits}: lazy loading }
The \package{astropy.io.fits} subpackage now supports \emph{lazy
loading}, where all HDUs are not loaded until they are requested (or
the file is closed). This should provide substantial speedups for
situations using the convenience functions (e.g., \texttt{getheader()}
or \texttt{getdata()}) to get HDU’s that are near the front of a file
with many HDUs.
\inlinecomment{Tom R}{too detailed? I don't think we need to mention every single performance enhancement, right?}
\subsubsection{\texttt{.misc}: YAML serialization}
The new \package{astropy.io.misc.yaml} module allows converting
astropy objects into a standard YAML format.
This can be used beyond \texttt{astropy.io.misc.yaml}: for example, to
serialize the metadata of tables before saving to other formats like
HDF5.
%\inlinecomment{BMS}{Maybe move the command line tools into a separate subsection to highlight them (even though we mostly only have fits related scripts)? They are supposed to be used more frequently by individual users.}
%\inlinecomment{hamogu}{MY feeling is that this section is too detailed already. If I had written it, I would not have mentioned the scripts at all.}
\subsection{Modeling}
\label{sec:modeling}
% The whole modeling submodule was missing from the previous paper, so everything really, including compound models, unit support etc.
% Nadia, Lim
% Tom R. -- unit support
\subsubsection{Overview}
The \package{astropy.modeling} subpackage provides a framework for representing
analytical models and performing model evaluation and parameter fitting. Models and
fitters are independent of each other: a model can be fit with different
fitters and new fitters can be added without changing existing models. The
framework is designed to be flexible and easily extensible. The goal is to have
a rich set of models but also make it easy to create new ones if necessary. The
modeling framework is used in a variety of data analysis tools and is the basis
for the Generalized World Coordinate System (GWCS)
package.\footnote{\url{https://github.com/spacetelescope/gwcs}}
% hamogu: I comented this out because we don't have prrof that it's
% ``uniquly'' useful. THe developers of e.g. ISIS, Sherpa, for even IRAF
% will probably disagree.
%The recently-added support for \package{astropy.units} in models and fitters
%make it a uniquely powerful package for working with astrophysical models.
\subsubsection{Single Model Definition and Evaluation}
\inlinecomment{hamogu: I tihnk this is too techical. I know it's a complex subpackage but I had trouble understanding this section even as a regular user.}
Most models are defined by parameters and maintain an ordered list of parameter names, \texttt{Model.param\_names}. A model is instantiated by passing in values (scalars or arrays) for its parameters. A parameter is a descriptor that provides a proxy for the value and stores additional information -- default value, default unit, and parameter constraints. The value and constraints can be updated by assignment. Supported parameter constraints include \texttt{fixed}, and \texttt{tied} parameters, and \texttt{bounds} on parameter values. Most models have a fixed parameter set but for some (e.g., polynomials), the number of parameters is defined by another argument (e.g., degree-of-freedom in the case of polynomials). Parameters support arithmetic operations and are combined with the inputs during evaluation using the \package{numpy} broadcasting rules. A model is evaluated by calling it as a function.
Models have a \texttt{Model.inverse} property, which returns the analytical inverse, if available, or raises an exception otherwise. This is a settable property; i.e. a model instance can be assigned as inverse to another model. For example, a polynomial model can be assigned as an inverse to another polynomial model.
Another useful settable property of models is \texttt{Model.bounding\_box}. This attribute sets the domain over which the model is defined. This greatly improves the efficiency of evaluation when the input range is much larger than the characteristic width of the model itself.
\subsubsection{Model Sets}
\package{astropy.modeling} provides an efficient way to instantiate the same type of model with many different sets of parameter values by passing the \texttt{n\_models} argument on instantiation, which sets the number of models to instantiate. This creates a model set that can be efficiently evaluated for example, in PSF (point spread function) photometry, all objects in an image will have a PSF of the same functional form, but with different positions and amplitides.
\subsubsection{Compound Models}
Models can be combined using arithmetic expressions. The result is also a model, which can further be combined with other models. Modeling supports arithmetic (+, -, *, /, and **), join ($\&$), and composition ($|$) operators. The rules for combining models involve matching their inputs and outputs. For example, the composition operator, $|$, requires the number of outputs of the left model to be equal to the number of inputs of the right one. For the join operator, the total number of inputs must equal the sum of number of inputs of both the left and the right models. For all arithmetic operators, the left and the right models must have the same number of inputs and outputs. An example of a compound model could be a spectrum with interstellar absorption. The stellar spectrum and the interstellar extinction are represented by separate models, but the observed spectrum is fitted with a compound model that combines both.
\subsubsection{Fitting Models to Data}
\package{astropy.modeling} provides several fitters which are wrappers around some of the \texttt{numpy} and \texttt{scipy.optimize} functions and provide support for specifying parameter constraints. The fitters take a model and data as input and return a copy of the model with the optimized parameter values set. The goal is to make it easy to extend the fitting framework to create new fitters. The optimizers available in \astropypkg version 2.0 are Levenberg--Marquardt, Simplex, SLSQP, and LinearLSQFitter (which is based on \texttt{numpy.linalg} that provides exact solution for linear models).
\inlinecomment{APW}{Add citations for the different optimizaton algorithms in paragraph above?}
Modeling also supports a plugin system for fitters, which allows using the
\astropypkg models with external fitters. An example of this is
\package{SABA}\footnote{\url{https://github.com/astropy/saba}}, which is a bridge between
Sherpa\footnote{\url{http://cxc.cfa.harvard.edu/contrib/sherpa/}}
and \package{astropy.modeling}, to bring the Sherpa fitters into \astropypkg.
\subsubsection{Creating New Models}
New model classes can be created in two ways:
\begin{enumerate}
\item The simplest way is to use the \texttt{custom\_model} decorator in modeling with a user-defined function that takes the inputs and model parameters as arguments.
\item It is also possible to subclass the \texttt{Model} base class and create an arbitrarily complex custom model.
\end{enumerate}
\subsubsection{Unit Support}
The \package{modeling} package now supports the representation, evaluation, and fitting of models using \texttt{Quantity} objects, which attach units to scalar values or arrays of values. In practice, this means that one can, for example, fit a model to data with units and get parameters that also have units out, or initialize a model with parameters with units and evaluate it using input values with different but equivalent units. Using this, we have implemented a blackbody model (\texttt{BlackBody1D}) that can be used to fit observed fluxes in a variety of units and as a function of different units of spectral coordinates (e.g., wavelength or frequency).
\subsection{Convolution}
% Adam G.
\astropypkg implements `normalized convolution' \citep[e.g.,][]{Knutsson1993}, which is an image reconstruction technique in which missing data are ignored during the convolution and replaced with values interpolated using the kernel. An example of this is given in \figurename~\ref{fig:convolution-example}. In versions $<=1.3$, the direct convolution and Fast Fourier Transform (FFT) convolution approaches were not consistent, with FFT convolution implementing normalized convolution and direct convolution implementing a different approach. As of version 2.0, the two methods are consistent and include a suite of consistency checks.
\begin{figure}
\includegraphics[width=\textwidth]{convolution_example.png}
\caption{%
An example showing different modes of convolution available in the \python
ecosystem. The red x's mark pixels that are set to NaN in the original data
(a). If the data are convolved with a Gaussian kernel on a 9x9 grid using
scipy's direct convolution (b), any pixel within range of the original NaN
pixels is also set to NaN. Panel (c) shows what happens if the NaNs are set
to zero first: the originally NaN regions are depressed relative to their
surroundings. Finally, panel (d) shows \astropypkg's convolution behavior,
where the missing pixels are replaced with values interpolated from their
surroundings using the convolution kernel.
\label{fig:convolution-example}
}
\end{figure}
\subsection{Visualization}
% Larry: Image visualization (stretching, scaling), RGB
The \package{astropy.visualization} subpackage provides functionality that can be helpful when visualizing data. This includes a framework for plotting astronomical images with coordinates with \package{matplotlib} (previously the standalone \package{wcsaxes} package), functionality related to image normalization (including both scaling and stretching), smart histogram plotting, RGB color image creation from separate images, and custom plotting styles for \package{matplotlib}.
\subsubsection{Image Stretching and Normalization}