forked from buildbot/buildbot
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathNEWS
2406 lines (1729 loc) · 103 KB
/
NEWS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
User visible changes in Buildbot. -*- outline -*-
* Release 0.7.9 (15 Sep 2008)
** New Features
*** Configurable public_html directory (#162)
The public_html/ directory, which provides static content for the WebStatus()
HTTP server, is now configurable. The default location is still the
public_html/ subdirectory of the buildmaster's base directory, but you can
change this by passing a suitable argument when creating the WebStatus()
instance in your master.cfg file:
c['status'].append( WebStatus(8080, public_html="/var/www/buildbot") )
*** Lock access modes (#313)
Albert Hofkamp added code to provide two distinct access modes to Locks:
"counting" and "exclusive". Locks can accept a configurable number of
"counting"-mode users, or a single "exclusive"-mode. For example, a Lock is
defined with maxCount=3, and then a 'compile' BuildStep uses this lock in
counting mode, while a 'cleanup' BuildStep uses this lock in exclusive mode.
Then, there can be one, two, or three simultaneous Builds in the compile step
(as long as there are no builds in the cleanup step). Only one build can be
in the cleanup step at a time, and if there is such a build in the cleanup
step, then the compile steps in other builds will wait for it to finish.
Please see the "Interlocks" section of the user's manual for more details.
** Bugs Fixed
*** Buildslave missing_timeout= fired too quickly (#211)
By providing a missing_timeout= argument when creating the BuildSlave
instance, you can ask the buildmaster to send email if a buildslave is
disconnected for too long. A bug in the previous version caused this
notification to be sent too soon, rather than waiting until the timeout
period expired. This should be fixed now.
*** Test command display fixed (#332)
In the previous version, a steps.shell.Test step would display the parsed
test results (in the step's box on the waterfall display) in lieu of any
other descriptive text the step might provide. In this release, these two
pieces of information are combined.
** Minor Changes
The buildmaster's version is logged to its twistd.log file at startup. The
buildslave does the same, to its own logfile.
Remote commands now record how long each command took. The "elapsedTime="
message will appear in the step's main logfile.
The "buildbot restart" command no longer fails if the buildbot wasn't already
running.
The FileUpload and FileDownload steps now create their target directories
(and any missing intermediate directories) before writing to the destination
file.
The per-build and per-step web pages now show the start, finish, and elapsed
time of their build or step.
If a Subversion-based build is started with a mixture of Changes that specify
particular numeric revisions and "HEAD" Changes (which indicate that a trunk
checkout is desired), the build will use a trunk checkout. Previously this
would probably cause an error. It is not clear how this situation might
arise.
** Compability With Other Tools
The mercurial commit hook (buildbot.changes.hgbuildbot) in the previous
version doesn't work with hg-1.0 or later (it uses an API function that was
present in the hg-0.9.5 release, but was removed from hg-1.0). This
incompability has been fixed: the new version of buildbot should be
compatible with hg-1.0 and newer (and it probably retains compability with
hg-0.9.5 and earlier too). (#328)
The Git tool has traditionally provided two ways to run each command, either
as subcommands of /usr/bin/git (like "git checkout"), or as individual tools
(like /usr/bin/git-checkout). The latter form is being removed in the
upcoming 1.6 Git release. Previous versions of Buildbot have used the
git-checkout form, and will break when Git is upgraded to 1.6 or beyond. The
new Buildbot release switches to the subcommand form. Note that this is a
change on the buildslave side.
The Git checkout command will now use the default branch (as set in the
steps.source.Git() step definition) if the changes that it is building do not
specify some other branch to build. (#340)
** Deprecation Schedule
No features have been deprecated in this release, and no deprecated features
have been removed. As a reminder, the following deprecated features are
scheduled for removal in an upcoming release:
c['sources'] (plural) was replaced by c['change_source'] (singular) in 0.7.6,
and will be removed by 0.8.0.
c['bots'] was replaced by c['buildslaves'] in 0.7.6, and will be removed by
0.8.0 . c['bots'] only accepts BuildSlave instances, not name/passwd tuples.
The html.Waterfall status target was replaced by html.WebStatus in 0.7.6, and
will be removed by 0.8.0.
* Release 0.7.8 (24 Jul 2008)
** New features
The IRC bot will respond to three new commands: 'notify' subscribes the
channel (or the sender, if the command is sent as a private "/msg") to hear
about build events. 'join' tells the bot to join some new IRC channel.
'leave' tells it to leave a channel. See the "IRC Bot" section of the User's
Manual for details. (#171)
Build Steps now have "statistics", in addition to logfiles. These are used to
count things like how many tests passed or failed. There are methods to sum
these counters across all steps and display the results in the Build status.
The Waterfall display now shows the count of failed tests on the top-most box
in each column, using this mechanism.
The new buildbot.steps.shell.PerlModuleTest step was added, to run Perl unit
tests. This is a wrapper around the regular ShellCommand that parses the
output of the standard perl unit test system and counts how many tests
passed/failed/etc. The results are put into the step's summary text, and a
count of tests passed/failed/skipped are tracked in the steps's statistics.
The factory.CPAN build factory has been updated to use this, so configuring a
Buildbot to test a perl module available from CPAN should be as easy as:
s = source.CVS(cvsroot, cvsmodule)
f = factory.CPAN(s)
Build Properties have been generalized: they remain associated with a single
Build, but the properties can be set from a variety of sources. In previous
releases, the Build itself would set properties like 'buildername', 'branch',
and 'revision' (the latter two indicating which version of the source code it
was trying to get), and the source-checkout BuildSteps would set a property
named 'got_revision' (to indicate what version of the soruce code it actually
got). In this release, the 'scheduler' property is set to indicate which
Scheduler caused the build to be started. In addition, the config file can
specify properties to be set on all Builds, or on all Builds for a specific
Builder. All these properties are available for interpolation into
ShellCommands and environment variables by using the WithProperties() marker.
It may be easier to implement simple build parameterization (e.g. to upload
generated binaries to a specific directory, or to only perform long-running
tests on a nightly build instead of upon every checkin) by using these Build
Properties than to write custom BuildSteps.
** Other improvements
The /buildslaves web page shows which slaves are currently running builds.
Offline slaves are displayed in bold.
Buildbot's setup.py now provides metadata to setuptools (if installed): an
entry_points script was added, and a dependency upon twisted-2.4.x or newer
was declared. This makes it more likely that 'easy_install buildbot' will
work.
The MailNotifier class acquired a mode="passing" flag: in this mode, the
buildbot will only send mail about passing builds (versus only on failing
builds, or only on builds which failed when the previous build had passed).
** Bugs fixed
Don't display force/stop build buttons when build control is disabled (#246)
When a build is waiting on a lock, don't claim that it has started (#107)
Make SVN mode=copy tolerate symlinks on freebsd, "cp -rp" -> "cp -RPp" (#86)
The svnpoller changesource now ignores branch deletion (#261)
The Git unit tests should run even if the user has not told Git about their
username/email.
The WebStatus /xmlrpc server's getStatus() method was renamed to the
more-accurate getLastBuildResults().
The TinderboxMailNotifier status output acquired an useChangeTime= argument.
The bonsaipoller changesource got some fixes.
** Deprecation Schedule
No features have been deprecated in this release, and no deprecated features
have been removed. As a reminder, the following deprecated features are
scheduled for removal in an upcoming release:
c['sources'] (plural) was replaced by c['change_source'] (singular) in 0.7.6,
and will be removed by 0.8.0.
c['bots'] was replaced by c['buildslaves'] in 0.7.6, and will be removed by
0.8.0 . c['bots'] only accepts BuildSlave instances, not name/passwd tuples.
The html.Waterfall status target was replaced by html.WebStatus in 0.7.6, and
will be removed by 0.8.0.
* Release 0.7.7 (29 Mar 2008)
** Things You Need To Know
*** builder names must not start with an underscore (`_').
These are now reserved for internal buildbot purposes, such as the magic
"_all" pseudo-builder that the web pages use to allow force-build buttons
that start builds on all Builders at once.
** New Features
*** "buildbot checkconfig"
The "buildbot checkconfig" command will look at your master.cfg file and tell
you if there are any problems with it. This can be used to test potential
changes to your config file before submitting them to the running
buildmaster. This is particularly useful to run just before doing "buildbot
restart", since the restart will fail if the config file has an error. By
running "buildbot checkconfig master.cfg && buildbot restart", you'll only
perform the restart if the config file was ok. Many thanks to Ben Hearsum for
the patch.
*** Waterfall "?category=FOO" query-arguments
The Waterfall page now accepts one or more "category=" query arguments in the
URL, to filter the display by categories. These behave a lot like the
"builder=" query argument. Thanks to Jermo Davann for the patch.
** Bugs Fixed
Many bugs were fixed, and many minor features were added. Many thanks to
Dustin Mitchell who fixed and coordinated many of these. Here is a terse
list, for more details, please see the Trac page for the 0.7.7 release, at
http://buildbot.net/trac/query?status=closed&milestone=0.7.7 :
Many of the URLs generated by the buildbot were wrong.
Display of last-heard-from timestamps on the buildslaves web page were wrong.
Asking an IRC bot about a build waiting on a Lock should no longer crash.
Same for the web viewer.
Stop treating the encouraged info/ directory as leftover.
Add more force/stop build buttons.
Timestamps displayed on the waterfall now handle daylight savings properly.
p4poller no longer quits after a single failure.
Improved Git support, including 'try', branch, and revisions.
Buildslaves now use 'git', not 'cogito'.
Make older hg client/servers handle specific-revision builds properly.
Fix twisted.scripts._twistw problem on twisted-2.5.0 and windows.
Fix workdir= and env= on ShellCommands
Fix logfile-watching in 'buildbot start' on OS-X.
Fix ShellCommand crashes when the program emits >640kB of output per chunk.
New WarningCountingShellCommand step.
Fix TreeSize step.
Fix transfer.FileUpload/FileDownload crashes for large files.
Make 'buildbor reconfig' on windows tell you that it doesn't work.
Add a To: header to the mail sent by the slave-missing timeout.
Disable usePTY= for most unit tests, it makes some debian systems flunk tests.
Add 'absolute source stamps'
Add 'triggerable schedulers', and a buildstep to trigger them.
Remove buildbot.changes.freshcvsmail
Add new XMLRPC methods: getAllBuilders, getStatus, getLastBuilds.
Accept WithProperties in more places: env=, workdir=, others.
Use --no-auth-cache with SVN commands to avoid clobbering shared svn state.
Add hours/minutes/seconds in the waterfall's ETA display.
Trial: count Doctest lines too.
ShellCommand: record more info in the headers: stdin closing, PTY usage.
Make it possible to stop builds across reconfig boundaries.
SVN revision numbers are now passed as strings, which was breaking MailNotifier
** Deprecation Schedule
The changes.freshcvsmail change source was replaced by
changes.mail.FCMaildirSource in 0.7.6, and has been removed in 0.7.7 .
c['sources'] (plural) was replaced by c['change_source'] (singular) in 0.7.6,
and will be removed by 0.8.0.
c['bots'] was replaced by c['buildslaves'] in 0.7.6, and will be removed by
0.8.0 . c['bots'] only accepts BuildSlave instances, not name/passwd tuples.
The html.Waterfall status target was replaced by html.WebStatus in 0.7.6, and
will be removed by 0.8.0.
* Release 0.7.6 (30 Sep 2007)
** Things You Need To Know
*** 'buildbot upgrade-master'
Each time you install a new version of Buildbot, you should run the new
'buildbot upgrade-master' command on each of your pre-existing buildmasters.
This will add files and fix (or at least detect) incompatibilities between
your old config and the new code.
*** new WebStatus page
The Waterfall has been replaced by the more general WebStatus display,
described below. WebStatus serves static files from a new public_html/
directory that lives in the buildmaster's basedir. Files like index.html,
buildbot.css, and robots.txt are served directly from that directory, so any
modifications you wish to make should be made to those files. In particular,
any custom CSS you've written should be copied into public_html/buildbot.css.
The 'upgrade-master' command will populate this directory for you.
The old Waterfall page is deprecated, but it should continue to work for
another few releases. It is now a subclass of WebStatus which just replaces
the default root URL with another copy of the /waterfall resource.
*** Compatibility: Python-2.3 or newer, Twisted-2.0 or newer
No compatiblity losses here, buildbot-0.7.6 is compatible with the same
versions of python and twisted that 0.7.5 was.
Buildbot is tested on a regular basis (http://buildbot.buildbot.net) against
nearly a full matrix of Python-(2.3,2.4,2.5) * Twisted-(2.0,2.1,2.2,2.4,2.5).
*** New Buildbot Home Page
Buildbot has moved to a new Trac instance at http://buildbot.net/ , and all
new bugs and tickets should be filed there. The old sourceforge bugs at
http://buildbot.sf.net/ will slowly be migrated over. Mailing lists are still
managed at sourceforge, and downloads are still available there.
*** Changed/Deprecated master.cfg Keys and Classes
c['sources'] (plural) has been replaced by c['change_source'] (singular).
c['bots'] has been replaced by c['buildslaves'], and it expects a list of
BuildSlave instances instead of tuples. See below for more details.
The 'freshcvsmail' change source has been deprecated, and will be removed in
the next release.
The html.Waterfall status target has been deprecated, and replaced by
html.WebStatus .
** New Features
*** WebStatus
The new WebStatus display is a superset of the old Waterfall. It contains a
waterfall as a sub-page, but it also contains pages with more compact
representations of recent build status. The "one_line_per_build" page
contains just that, and "one_box_per_builder" shows just the information from
the top of the waterfall page (last-finished-build and current-activity).
The initial page (when you hit the root of the web site) is served from
index.html, and provides links to the Waterfall as well as the other pages.
Most of these pages can be filtered by adding query arguments to the URL.
Adding "?builder=XYZ" will cause the page to only show results for the given
builder. Adding "?builder=XYZ&builder=ABC" will show results for either
builder. "?branch=trunk" will limit the results to builds that involved code
from the trunk.
The /waterfall page has arguments to hide those annoying "buildslave
connected" messages, to start and and at arbitrary times, and to auto-refresh
at a chosen interval (with a hardcoded minimum of 15 seconds). It also has a
"help" page with forms that will help you add all of these nifty filtering
arguments.
The recommended practice is to modify the index.html file to include links to
the filtered pages that you find most useful.
Note that WebStatus defaults to allowForce=False, meaning that the display
will not offer or accept "Force Build" or "Stop Build" controls. (The old
Waterfall defaults to allowForce=True).
The new WebStatus pages try very hard to use only relative links, making life
better when the Buildbot sits behind an HTTP reverse proxy.
In addition, there is a rudimentary XMLRPC server run by the WebStatus
object. It only has two methods so far, but it will acquire more in the
future. The first customer of this is a project to add a buildbot plugin to
Trac.
*** BuildFactory.addStep(Step(args))
BuildFactories can be set up either with a complete list of steps, or by
calling the .addStep() method repeatedly. The preferred way to provide a step
is by instantiating it, rather than giving a class/kwargs pair. This gives
the BuildStep class a chance to examine the arguments (and complain about
anything it doesn't like) while the config file is being read and problems
are being logged. For example, the old-style:
from buildbot.process.factory import BuildFactory, s
steps = [s(CVS, cvsroot="blah", mode="copy"),
s(Compile, command=["make", "all"]),
s(Test, command=["make", "test"]),
]
f = BuildFactory(steps)
is now:
f = BuildFactory()
f.addStep( CVS(cvsroot="blah", mode="copy") )
f.addStep( Compile(command=["make", "all"]) )
f.addStep( Test(command=["make", "test"]) )
Authors of BuildStep subclasses which override __init__ to add new arguments
must register them with self.addFactoryArguments(**newargs) to make sure that
those classes will work with this new style, otherwise the new arguments will
be lost.
Using class/kwargs pairs is deprecated, and will be removed in a future
release.
*** BuildSlave instances, max_builds=, notify_on_missing=
Buildslave specification has changed a lot in this release. The old config:
c['bots'] = [ ("bot1name", "bot1passwd"),
("bot2name", "bot2passwd") ]
is now:
from buildbot.buildslave import BuildSlave
c['slaves'] = [ BuildSlave("bot1name", "bot1passwd"),
BuildSlave("bot2name", "bot2passwd") ]
This new form gives us the ability to add new controls. The first is
"max_builds=", which imposes a concurrency limit that is like the usual
SlaveLock, but gives the buildmaster the opportunity to find a different
slave to run the build. (the buildslave is chosen before the SlaveLock is
claimed, so pure SlaveLocks don't let you take full advantage of build
farms).
The other addition is "notify_on_missing=", which accepts an email address
(or list of addresses), and sends a message when the buildslave has been
disconnected for more than an hour (configurable with missing_timeout=). This
may be useful when you expect that the buildslave hosts should be available
most of the time, and want to investigate the reasons that it went offline.
** Other Improvements
The IRC bot has been refactored to make it easier to add instant-messaging
status delivery in the future. The IM plugins are not yet written, though.
When multiple buildslaves are available for a given build, one of them will
be picked at random. In previous releases, the first one on the list was
always picked. This helps to add a certain measure of load-balancing. More
improvements will be made in the future.
When the buildslave does a VC checkout step that requires clobbering the
build directory (i.e. in all modes except for 'update'), the buildslave will
first set the permissions on all build files to allow their deletion, before
it attempts to delete them. This should fix some problems in which a build
process left non-user-writable files lying around (frequently a result of
enthusiastic unit tests).
The BuildStep's workdir= argument can now accept a WithProperties()
specification, allowing greater control over the workdir.
Support for the 'Bazaar' version control system (/usr/bin/bzr) has been
added, using the buildbot.steps.source.Bzr class. This is a replacement for
the old 'Arch' (/usr/bin/tla and /usr/bin/baz) systems, which are still
supported by Buildbot with the source.Arch and source.Bazaar classes,
respectively. Unfortunately the old baz system claimed the 'Bazaar' classname
early, so the new system must use source.Bzr instead of the desired
source.Bazaar . A future release might change this.
A rudimentary Gnome Panel applet is provided in contrib/bb_applet.py, which
provides 'buildbot statusgui' -like colored status boxes inside the panel.
Installing it is a bit tricky, though.
The 'buildbot try' command now accepts a '--diff=foo.patch' argument, to let
you provide a pre-computed patch. This makes it easier to test out patches
that you've looked over for safety, without first applying them to your local
source tree.
A new Mercurial change source was added, hg_buildbot.py, which runs as an
in-process post-commit hook. This gives us access to much more information
about the change, as well as being much faster.
The email-based changesource have been refactored, to make it easier to write
new mail parsers. A parser for the SVN "commit-email.pl" script has been
added.
** Bugs Fixed
Far too many to count. Please see
http://buildbot.net/trac/query?status=closed&milestone=0.7.6 for a partial
list of tickets closed for this release, and the ChangeLog for a complete
list of all changes since 0.7.5 .
* Release 0.7.5 (10 Dec 2006)
** Things You Need To Know
*** The Great BuildStep Renaming
All BuildSteps have moved! They used to be classes in buildbot.process.step,
but now they all have separate modules in buildbot.steps.* . They have been
split out into separate categories: for example, the source checkout steps
are now buildbot.steps.source.CVS, buildbot.steps.source.Darcs, etc. The most
commonly used one is probably buildbot.steps.shell.ShellCommand . The
python-specific steps are in buildbot.steps.python, and the Twisted-specific
steps are in buildbot.steps.python_twisted .
You will need to update your master.cfg files to use the new names. The old
names are deprecated and will be removed altogether in the next release.
*** Compatibility
Buildbot now requires python-2.3 or later. Buildbot now requires
Twisted-2.0.0 or later. Support for earlier versions of both has finally been
removed. If you discover it works with unsupported versions, please return
your Buildbot to the factory for repairs :-).
Buildbot has *not* yet been tested against the recent python-2.5 release. It
has been tested against the latest SVN version of Twisted, but only in
conjunction with python-2.4 .
** new features
*** reconfiguring a Builder no longer causes a disconnect/reconnect cycle
This means that sending SIGHUP to the master or running 'buildbot reconfig
MASTERDIR' command no longer interrupts any current builds, nor does it lose
pending builds like it did before. This involved a fairly substantial
refactoring of the various internal BotPerspective/BotMaster/Builder classes.
Note that reconfiguring Schedulers still loses any Changes that were waiting
for the tree to become stable: hopefully this will be fixed in the next
release.
*** 'buildbot start/restart/reconfig' now show logs until startup is complete
These commands now have additional code to follow twistd.log and display all
the lines that are emitted from the beginning of the start/reconfig action
until it has completed. This gives you a chance to see any problems detected
in the config file without needing to manually look in twistd.log or use
another shell to 'tail -f' it. This also makes it clear which config file is
being used. This functionality is not available under windows.
In addition, if any problems are detected during 'start' or 'restart' (but
not reconfig), the buildbot command will terminate with a non-zero exit
status, making it easier to use in scripts. Closes SF#1517975.
*** Locks now take maxCount=N to allow multiple simultaneous owners
This allows Locks to be non-exclusive but still limit maximum concurrency.
Thanks to James Knight for the patch. Closes SF#1434997.
*** filetransfer steps
buildbot.steps.transfer.FileUpload is a buildstep that will move files from
the slave to the master. Likewise, FileDownload will move files from the
master down to the buildslave. Many thanks to Albert Hofkamp for contributing
these classes. Closes SF#1504631.
*** pyflakes step
buildbot.steps.python.PyFlakes will run the simple 'pyflakes' static analysis
tool and parse the results to tell you about undefined names, unused imports,
etc. You'll need to tell it how to run pyflakes, usually with something like
command=["pyflakes", "src/packagedir"] or the like. The default command is
"make pyflakes", which assumes that you have a suitable target in your
top-level Makefile.
*** Monotone support
Nathaniel Smith has contributed initial support for the Monotone version
control system. The code still needs docs and tests, but on the other hand it
has been in use by the Monotone buildbot for a long time now, so it is
probably fairly stable.
*** Tinderbox support
Ben Hearsum and the Mozilla crew have contributed some classes to allow
Buildbot to work with Tinderbox clients. One piece is
buildbot.changes.bonsaipoller.BonsaiPoller, which is a ChangeSource that
polls a Bonsai server (which is a kind of web-vased viewcvs CGI script) to
discover source code changes. The other piece is
buildbot.status.tinderbox.TinderboxMailNotifier, which is a status plugin
that sends email in the same format as Tinderbox does, which allows a number
of Tinderbox tools to be driven by Buildbot instead.
*** SVN Poller
Niklaus Giger contributed a ChangeSource (buildbot.changes.svnpoller) which
polls a remote SVN repository on a periodic basis. This is useful when, for
whatever reason, you cannot add a post-commit hook script to the repository.
This obsoletes the external contrib/svn_watcher.py script.
** notes for plugin developers
*** IStatusLog.readlines()
This new method makes it easier for a status plugin (or a
BuildStep.createSummary method) to walk through a StatusLog one line at a
time. For example, if you wanted to create an extra logfile that just
contained all the GCC warnings from the main log, you could use the
following:
def createSummary(self, log):
warnings = []
for line in log.readlines():
if "warning:" in line:
warnings.append()
self.addCompleteLog('warnings', "".join(warnings))
The "BuildStep LogFiles" section of the user's manual contains more
information. This method is not particularly memory-efficient yet (it reads
the whole logfile into memory first, then splits it into lines); this will be
improved in a future release.
** bug fixes
*** Update source.SVN to work with the new SVN-1.4.0
The latest subversion changed the behavior in an unusual situation which
caused the unit tests to fail. This was unlikely to cause a problem in actual
usage, but the tests have been updated to pass with the new version.
*** update svn_buildbot.py to avoid mangling filenames
Older versions of this script were stripping the wrong number of columns from
the output of 'svnlook changed', and would sometimes mangle filenames. This
has been fixed. Closes SF#1545146.
*** logfiles= caused subsequent build failures under Windows
Earlier versions of buildbot didn't explicitly close any logfiles= file
handles when the build finished. On windows (where you cannot delete a file
that someone else is reading), this could cause the next build to fail as the
source checkout step was unable to delete the old working directory. This has
been fixed. Closes SF#1568415.
*** logfiles= didn't work on OS-X
Macintosh OS-X has a different behavior when reading files that have reached
EOF, the result was that logfiles= sometimes didn't work. Thanks to Mark Rowe
for the patch.
** other changes
The 'buildbot sighup MASTERDIR' command has been replaced with 'buildbot
reconfig MASTERDIR', since that seems to be a slightly more meaningful name.
The 'sighup' form will remain as an alias.
* Release 0.7.4 (23 Aug 2006)
** Things You Need To Know
The PBChangeSource's prefix= argument has changed, you probably need to add a
slash now. This is mostly used by sites which use Subversion and
svn_buildbot.py.
The subcommands that are used to create a buildmaster or a buildslave have
changed. They used to be called 'buildbot master' and 'buildbot slave'. Now
they are called 'buildbot create-master' and 'buildbot create-slave'. Zipf's
Law suggests that these are more appropriate names for these
infrequently-used commands.
The syntax for the c['manhole'] feature has changed.
** new features
*** full Perforce support
SF#1473939: large patch from Scott Lamb, with docs and unit tests! This
includes both the step.P4 source-checkout BuildStep, and the changes.p4poller
ChangeSource you'll want to feed it. P4 is now supported just as well as all
the other VC systems. Thanks Scott!
*** SSH-based Manhole
The 'manhole' feature allows buildbot developers to get access to a python
read/eval/print loop (REPL) inside the buildmaster through a network
connection. Previously, this ran over unencrypted telnet, using a simple
username/password for access control. The new release defaults to encrypted
SSH access, using either username/password or an authorized_keys file (just
like sshd). There also exists an unencrypted telnet form, but its use is
discouraged. The syntax for setting up a manhole has changed, so master.cfg
files that use them must be updated. The "Debug options" section in the
user's manual provides a complete description.
*** Multiple Logfiles
BuildSteps can watch multiple log files in realtime, not just stdout/stderr.
This works in a similar fashion to 'tail -f': the file is polled once per
second, and any new data is sent to the buildmaster.
This requires a buildslave running 0.7.4 or later, and a warning message is
produced if used against an old buildslave (which will otherwise produce no
data). Use "logfiles={'name': 'filename'}" to take advantage of this feature
from master.cfg, and see the "ShellCommand" section of the user's manual for
full documentation.
The 'Trial' buildstep has been updated to use this, to display
_trial_temp/test.log in realtime. It also knows to fall back to the previous
"cat" command if the buildslave is too old.
*** BuildStep URLs
BuildSteps can now add arbitrary URLs which will be displayed on the
Waterfall page in the same place that Logs are presented. This is intended to
provide a link to generated HTML pages, such as the output of a code coverage
tool. The step is responsible for somehow uploading the HTML to a web server:
this feature merely provides an easy way to present the HREF link to the
user. See the "BuildStep URLs" section of the user's manual for details and
examples.
*** LogObservers
BuildSteps can now attach LogObservers to various logfiles, allowing them to
get real-time log output. They can use this to watch for progress-indicating
events (like counting the number of files compiled, or the number of tests
which have run), and update both ETA/progress-tracking and step text. This
allows for more accurate ETA information, and more information passed to the
user about how much of the process has completed.
The 'Trial' buildstep has been updated to use this for progress tracking, by
counting how many test cases have run.
** new documentation
What classes are useful in your master.cfg file? A table of them has been
added to the user's manual, in a section called "Index of Useful Classes".
Want a list of all the keys in master.cfg? Look in the "Index of master.cfg
keys" section.
A number of pretty diagrams have been added to the "System Architecture"
portion of the manual, explaining how all the buildbot pieces fit together.
An HTML form of the user's manual is now shipped in the source tarball. This
makes it a bit bigger: sorry about that. The old PyCon-2003 paper has been
removed from the distribution, as it is mostly supplanted by the user's
manual by this point.
** bugfixes
SF#1217699 + SF#1381867: The prefix= argument to PBChangeSource has been
changed: now it does just a simple string-prefix match and strip. The
previous behavior was buggy and unhelpful. NOTE: if you were using prefix=
before, you probably need to add a slash to the end of it.
SF#1398174: ignore SVN property changes better, fixed by Olivier Bonnet
SF#1452801: don't double-escape the build URL, fixed by Olivier Bonnet
SF#1401121: add support for running py2exe on windows, by Mark Hammond
reloading unchanged config files with WithProperties shouldn't change anything.
All svn commands now include --non-interactive so they won't ask for
passwords. Instead, the command will fail if it cannot be performed without
user input.
Deprecation warnings with newer versions of Twisted have been hushed.
** compatibility
I haven't actually removed support for Twisted-1.3.0 yet, but I'd like to.
The step_twisted default value for --reporter matches modern Twisteds,
though, and won't work under 1.3.0.
ShellCommand.flunkOnFailure now defaults to True, so any shell command which
fails counts as a build failure. Set this to False if you don't want this
behavior.
** minor features
contrib/darcs_buildbot.py contains a new script suitable for use in a darcs
commit-hook.
Hovering a cursor over the yellow "Build #123" box in the Waterfall display
will pop up an HTML tooltip to show the reason for the build. Thanks to Zandr
Milewski for the suggestion.
contrib/CSS/*.css now contains several contributed stylesheets to make the
Waterfall display a bit less ugly. Thanks to John O'Duinn for gathering them.
ShellCommand and its derivatives can now accept either a string or a list of
strings in the description= and descriptionDone= arguments. Thanks to Paul
Winkler for the catch.
* Release 0.7.3 (23 May 2006)
** compatibility
This release is compatible with Twisted-1.3.0, but the next one will not be.
Please upgrade to at least Twisted-2.0.x soon, as the next buildbot release
will require it.
** new features
*** Mercurial support
Support for Mercurial version control system (http://selenic.com/mercurial)
has been added. This adds a buildbot.process.step.Mercurial BuildStep. A
suitable hook script to deliver changes to the buildmaster is still missing.
*** 'buildbot restart' command
The 'buildbot restart BASEDIR' command will perform a 'buildbot stop' and
'buildbot start', and will attempt to wait for the buildbot process to shut
down in between. This is useful when you need to upgrade the code on your
buildmaster or buildslave and want to take it down for a minimum amount of
time.
*** build properties
Each build now has a set of named "Build Properties", which can be set by
steps and interpolated into ShellCommands. The 'revision' and 'got_revision'
properties are the most interesting ones available at this point, and can be
used e.g. to get the VC revision number into the filename of a generated
tarball. See the user's manual section entited "Build Properties" for more
details.
** minor features
*** IRC now takes password= argument
Useful for letting your bot claim a persistent identity.
*** svn_buildbot.py is easier to modify to understand branches
*** BuildFactory has a new .addStep method
*** p4poller has new arguments
*** new contrib scripts: viewcvspoll, svnpoller, svn_watcher
These poll an external VC repository to watch for changes, as opposed to
adding a hook script to the repository that pushes changes into the
buildmaster. This means higher latency but may be easier to configure,
especially if you do not have authority on the repository host.
*** VC build property 'got_revision'
The 'got_revision' property reports what revision a VC step actually
acquired, which may be useful to know when building from HEAD.
*** improved CSS in Waterfall
The Waterfall display has a few new class= tags, which may make it easier to
write custom CSS to make it look prettier.
*** robots_txt= argument in Waterfall
You can now pass a filename to the robots_txt= argument, which will be served
as the "robots.txt" file. This can be used to discourage search engine
spiders from crawling through the numerous build-status pages.
** bugfixes
*** tests more likely to pass on non-English systems
The unit test suite now sets $LANG='C' to make subcommands emit error
messages in english instead of whatever native language is in use on the
host. This improves the chances that the unit tests will pass on such
systems. This affects certain VC-related subcommands too.
test_vc was assuming that the system time was expressed with a numeric
timezone, which is not always the case, especially under windows. This
probably works better now than it did before. This only affects the CVS
tests.
'buildbot try' (for CVS) now uses UTC instead of the local timezone. The
'got_revision' property is also expressed in UTC. Both should help deal with
buggy versions of CVS that don't parse numeric timezones properly.
* Release 0.7.2 (17 Feb 2006)
** new features
*** all TCP port numbers in config file now accept a strports string
Sometimes it is useful to restrict certain TCP ports that the buildmaster
listens on to use specific network interfaces. In particular, if the
buildmaster and SVN repository live on the same machine, you may want to
restrict the PBChangeSource to only listen on the loopback interface,
insuring that no external entities can inject Changes into the buildbot.
Likewise, if you are using something like Apache's reverse-proxy feature to
provide access to the buildmaster's HTML status page, you might want to hide
the real Waterfall port by having it only bind to the loopback interface.
To accomplish this, use a string like "tcp:12345:interface=127.0.0.1" instead
of a number like 12345. These strings are called "strports specification
strings", and are documented in twisted's twisted.application.strports module
(you can probably type 'pydoc twisted.application.strports' to see this
documentation). Pretty much everywhere the buildbot takes a port number will
now accept a strports spec, and any bare numbers are translated into TCP port
numbers (listening on all network interfaces) for compatibility.
*** buildslave --umask control
Twisted's daemonization utility (/usr/bin/twistd) automatically sets the
umask to 077, which means that all files generated by both the buildmaster
and the buildslave will only be readable by the account under which the
respective daemon is running. This makes it unnecessarily difficult to share
build products (e.g. by symlinking ~/public_html/current_docs/ to a directory
within the slave's build directory where each build puts the results of a
"make docs" step).
The 'buildbot slave <PARAMS>' command now accepts a --umask argument, which
can be used to override the umask set by twistd. If you create the buildslave
with '--umask=022', then all build products will be world-readable, making it
easier for other processes (run under other accounts) to access them.
** bug fixes
The 0.7.1 release had a bug whereby reloading the config file could break all
configured Schedulers, causing them to raise an exception when new changes
arrived but not actually schedule a new build. This has been fixed.
Fixed a bug which caused the AnyBranchScheduler to explode when branch==None.
Thanks to Kevin Turner for the catch. I also think I fixed a bug whereby the
TryScheduler would explode when it was given a Change (which it is supposed
to simply ignore).
The Waterfall display now does more quoting of names (including Builder
names, BuildStep names, etc), so it is more likely that these names can
contain unusual characters like spaces, quotes, and slashes. There may still
be some problems with these kinds of names, however.. please report any bugs
to the mailing list.
* Release 0.7.1 (26 Nov 2005)
** new features
*** scheduler.Nightly
Dobes Vandermeer contributed a cron-style 'Nightly' scheduler. Unlike the
more-primitive Periodic class (which only lets you specify the duration
between build attempts), Nightly lets you schedule builds for specific times
of day, week, month, or year. The interface is very much like the crontab(5)
file. See the buildbot.scheduler.Nightly docstring for complete details.
** minor new features
*** step.Trial can work with Trial from Twisted >2.1.0
The 'Trial' step now accepts the trialMode= argument, which should be a list
of strings to be added to trial's argv array. This defaults to ["-to"], which
is appropriate for the Trial that ships in Twisted-2.1.0 and earlier, and
tells Trial to emit non-colorized verbose output. To use this step with
trials from later versions of Twisted, this should be changed to
["--reporter=bwverbose"].
In addition, you can now set other Trial command-line parameters through the
trialArgs= argument. This is a list of strings, and defaults to an empty list.
*** Added a 'resubmit this build' button to the web page
*** Make the VC-checkout step's description more useful
Added the word "[branch]" to the VC step's description (used in the Step's
box on the Waterfall page, among others) when we're checking out a
non-default branch. Also add "rNNN" where appropriate to indicate which
revision is being checked out. Thanks to Brad Hards and Nathaniel Smith for
the suggestion.
** bugs fixed
Several patches from Dobes Vandermeer: Escape the URLs in email, in case they
have spaces and such. Fill otherwise-empty <td> elements, as a workaround for
buggy browsers that might optimize them away. Also use binary mode when
opening status pickle files, to make windows work better. The
AnyBranchScheduler now works even when you don't provide a fileIsImportant=
argument.
Stringify the base revision before stuffing it into a 'try' jobfile, helping
SVN and Arch implement 'try' builds better. Thanks to Steven Walter for the
patch.
Fix the compare_attrs list in PBChangeSource, FreshCVSSource, and Waterfall.
Before this, certain changes to these objects in the master.cfg file were
ignored, such that you would have to stop and re-start the buildmaster to
make them take effect.
The config file is now loaded serially, shutting down old (or replaced)
Status/ChangeSource plugins before starting new ones. This fixes a bug in
which changing an aspect of, say, the Waterfall display would cause an
exception as both old and new instances fight over the same TCP port. This
should also fix a bug whereby new Periodic Schedulers could fire a build
before the Builders have finished being added.
There was a bug in the way Locks were handled when the config file was
reloaded: changing one Builder (but not the others) and reloading master.cfg
would result in multiple instances of the same Lock object, so the Locks
would fail to prevent simultaneous execution of Builds or Steps. This has
been fixed.
** other changes
For a long time, certain StatusReceiver methods (like buildStarted and
stepStarted) have been able to return another StatusReceiver instance
(usually 'self') to indicate that they wish to subscribe to events within the
new object. For example, if the buildStarted() method returns 'self', the