forked from buildbot/buildbot
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathNEWS
2654 lines (1879 loc) · 111 KB
/
NEWS
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
User visible changes in Buildbot. -*- outline -*-
** New buildstep `MTR'
A new class buildbot.process.mtrlogobserver.MTR was added. This buildstep is
used to run test suites using mysql-test-run. It parses the stdio output for
test failures and summarises them on the waterfall page. It also makes server
error logs available for debugging failures, and optionally inserts
information about test runs and test failures into an external database.
* Release 0.7.11p (July 16, 2009)
Fixes a few test failures in 0.7.11, and gives a default value for branchType
if it is not specified by the master.
* Release 0.7.11 (July 5, 2009)
Developers too numerous to mention contributed to this release. Buildbot has
truly become a community-maintained application. Much hard work is not
mentioned here, so please consult the git logs for the detailed changes in this
release.
** Better Memory Performance, Disk Cleanup
Buildbot handles its memory usage a bit better, and can automatically purge old
history to keep memory and disk usage low. Look for eventHorizon,
buildHorizon, logHorizon, and changeHorizon.
** Password Protection for Force Build and Stop actions
It is now possible to require authentication to force build and stop via the
WebStatus interface. To use this, set the 'auth' field of WebStatus to a valid
IAuth implementation. Current implementations are:
BasicAuth with a list of user/passwords
HTPasswdAuth with an .htpasswd file
By default, the unauthenticated behavior will occur.
** Web Status changes
The "Graceful Shutdown" feature, as a kind of "force", now obeys allowForce.
The waterfall and other pages are more deeply interlinked.
Pending builds can be individually cancelled, or cancelled in bulk.
** Fixed Transfer Steps
Transfer step classes are more reliable; DirectoryUpload and DirectoryDownload
use tarfile instead of manually framing files. The DirectoryUpload step also
now supports compression.
** Conditional Steps
Steps now take a doStepIf parameter which can be used to implement simple
conditional execution of a step.
** Colorized Steps
Steps are now hilighted with a color in the build view to indicate their
success or failure.
** Improved build prioritization
Bugfixes and fairer scheduling
** Transposed Grid
Similar to the grid view, but with the axes reversed and showing different
info. Located at /tgrid.
** Trigger steps improvements
Trigger now supports copy_properties, to send selected properties to the
triggered build.
* Release 0.7.10 (25 Feb 2009)
This release is mainly a collection of user-submitted patches since
the last release.
** New Features
*** Environment variables in a builder (#100)
It is useful to be able to pass environment variables to all steps in a
builder. This is now possible by adding { .. 'env': { 'var' : 'value' }, ... }
to the builder specification.
*** IRC status plugin improvements (#330, #357, #378, #280, #381, #411, #368)
*** usePTY specified in master.cfg, defaults to False (#158, #255)
Using a pty has some benefits in terms of supporting "Stop Build", but causes
numerous problems with simpler jobs which can be killed by a SIGHUP when their
standard input is closed. With this change, PTYs are not used by default,
although you can enable them either on slaves (with the --usepty option to
create-slave) or on the master.
*** More information about buildslaves via the web plugin (#110)
A new page, rooted at /buildslave/$SLAVENAME, gives extensive information about
the buildslave.
*** More flexible merging of requests (#415)
The optional c['mergeRequests'] configuration parameter takes a function
which can decide whether two requests are mergeable.
*** Steps can be made to run even if the build has halted (#414)
Adding alwaysRun=True to a step will cause it to run even if some other step
has failed and has haltOnFailure=True.
*** Compress buildstep logfiles (#26)
Logs for each buildstep, which can take a lot of space on a busy buildmaster,
are automatically compressed after the step has finished.
*** Support for "latent" buildslaves
The buildslaves that are started on-demand are called "latent" buildslaves.
Buildbot ships with an abstract base class for building latent buildslaves, and
a concrete implementation for AWS EC2.
*** Customized MailNotifier messages (#175)
MailNotifier now takes an optional function to build the notification message,
allowing ultimate site-level control over the format of buildbot's notification
emails.
*** Nightly scheduler support for building only if changes have occurred
With the addition of onlyIfChanged=True, the Nightly scheduler will not schedule
a new build if no changes have been made since its last scheduled build.
*** Add ATOM/RSS feeds to WebStatus (#372)
Two new pages, /atom and /rss, provide feeds of build events to any feed
reader. These paths take the same "category" and "branch" arguments as the
waterfall and grid.
*** Add categories to Schedulers and Changes (#182)
This allows a moderate amount of support for multiple projects built in a
single buildmaster.
*** Gracefully shut down a buildslave after its build is complete
The /buildslaves/$SLAVENAME pages have a "Gracefully Shutdown" button which
will cause the corresponding slave to shut itself down when it finishes its
current build. This is a good way to do work on a slave without causing a
spurious build failure.
*** SVN source steps can send usernames and passwords (#41)
Adding username="foo" and/or password="bar" to an SVN step will cause
--username and --password arguments to be passed to 'svn' on the slave side.
Passwords are suitably obfuscated in logfiles.
** New Steps
*** DirectoryUpload (#393)
This step uploads an entire directory to the master, and can be useful when a
build creates several products (e.g., a client and server package).
*** MasterShellCommand
This step runs a shell command on the server, and can be useful for
post-processing build products, or performing other maintenance tasks on the
master.
*** PyLint (#259)
A PyLint step is available to complement the existing PyFlakes step.
** Bugs Fixed
*** Process output from new versions of Test::Harness (#346)
*** Fixes to the try client and scheduler
*** Remove redundant loop in MailNotifier (#315)
*** Display correct $PWD in logfiles (#179)
*** Do not assume a particular python version on Windows (#401)
*** Sort files in changes (#402)
*** Sort buildslaves lexically (#416)
*** Send properties to all builds initiated by AnyBranchScheduler
*** Dependent Schedulers are more robust to reconfiguration (#35)
*** Fix properties handling in triggered buidls (#392)
*** Use "call" on Windows to avoid errors (#417)
*** Support setDefaultWorkdir in FileUpload and FileDownload (#209)
*** Support WithProperties in FileUpload and FileDownload (#210)
*** Fix a bug where changes could be lost on a master crash (#202)
*** Remove color settings from non-presentation code (#251)
*** Fix builders which stopped working after a PING (#349, #85)
*** Isolate Python exceptions in status plugins (#388)
*** Notify about slaves missing at master startup (#302)
*** Fix tracebacks in web display after a reconfig (#176)
** Version-Control Changes
*** Many Mercurial fixes
- Inrepo branch support finalized (source step + changegroup hook + test case)
(#65 #185 #187)
- Reduced amount of full clones by separating clone with update into
clone/pull/update steps (#186, #227) (see #412 for future work here)
- Fixed mercurial changegroup hook to work with Mercurial 1.1 API (#181, #380)
*** Many git fixes
*** Add got_revision to Perforce support (#127)
*** Use "git foo" everywhere instead of deprecated "git-foo"
** Minor Changes
*** factory.addSteps (#317)
If you have a common list of steps that are included in multiple factories, you
can use f.addSteps(steplist) to add them all at once.
*** Twisted logfile rotation and cleanup (#108)
By default, Buildbot now rotates and cleans up the (potentially voluminous)
twistd.log files.
*** Prioritize build requests based on the time they wre submitted (#334)
Balancing of load is a bit more fair, although not true load balancing.
* Release 0.7.9 (15 Sep 2008)
** New Features
*** Configurable public_html directory (#162)
The public_html/ directory, which provides static content for the WebStatus()
HTTP server, is now configurable. The default location is still the
public_html/ subdirectory of the buildmaster's base directory, but you can
change this by passing a suitable argument when creating the WebStatus()
instance in your master.cfg file:
c['status'].append( WebStatus(8080, public_html="/var/www/buildbot") )
*** Lock access modes (#313)
Albert Hofkamp added code to provide two distinct access modes to Locks:
"counting" and "exclusive". Locks can accept a configurable number of
"counting"-mode users, or a single "exclusive"-mode. For example, a Lock is
defined with maxCount=3, and then a 'compile' BuildStep uses this lock in
counting mode, while a 'cleanup' BuildStep uses this lock in exclusive mode.
Then, there can be one, two, or three simultaneous Builds in the compile step
(as long as there are no builds in the cleanup step). Only one build can be
in the cleanup step at a time, and if there is such a build in the cleanup
step, then the compile steps in other builds will wait for it to finish.
Please see the "Interlocks" section of the user's manual for more details.
** Bugs Fixed
*** Buildslave missing_timeout= fired too quickly (#211)
By providing a missing_timeout= argument when creating the BuildSlave
instance, you can ask the buildmaster to send email if a buildslave is
disconnected for too long. A bug in the previous version caused this
notification to be sent too soon, rather than waiting until the timeout
period expired. This should be fixed now.
*** Test command display fixed (#332)
In the previous version, a steps.shell.Test step would display the parsed
test results (in the step's box on the waterfall display) in lieu of any
other descriptive text the step might provide. In this release, these two
pieces of information are combined.
** Minor Changes
The buildmaster's version is logged to its twistd.log file at startup. The
buildslave does the same, to its own logfile.
Remote commands now record how long each command took. The "elapsedTime="
message will appear in the step's main logfile.
The "buildbot restart" command no longer fails if the buildbot wasn't already
running.
The FileUpload and FileDownload steps now create their target directories
(and any missing intermediate directories) before writing to the destination
file.
The per-build and per-step web pages now show the start, finish, and elapsed
time of their build or step.
If a Subversion-based build is started with a mixture of Changes that specify
particular numeric revisions and "HEAD" Changes (which indicate that a trunk
checkout is desired), the build will use a trunk checkout. Previously this
would probably cause an error. It is not clear how this situation might
arise.
** Compability With Other Tools
The mercurial commit hook (buildbot.changes.hgbuildbot) in the previous
version doesn't work with hg-1.0 or later (it uses an API function that was
present in the hg-0.9.5 release, but was removed from hg-1.0). This
incompability has been fixed: the new version of buildbot should be
compatible with hg-1.0 and newer (and it probably retains compability with
hg-0.9.5 and earlier too). (#328)
The Git tool has traditionally provided two ways to run each command, either
as subcommands of /usr/bin/git (like "git checkout"), or as individual tools
(like /usr/bin/git-checkout). The latter form is being removed in the
upcoming 1.6 Git release. Previous versions of Buildbot have used the
git-checkout form, and will break when Git is upgraded to 1.6 or beyond. The
new Buildbot release switches to the subcommand form. Note that this is a
change on the buildslave side.
The Git checkout command will now use the default branch (as set in the
steps.source.Git() step definition) if the changes that it is building do not
specify some other branch to build. (#340)
** Deprecation Schedule
No features have been deprecated in this release, and no deprecated features
have been removed. As a reminder, the following deprecated features are
scheduled for removal in an upcoming release:
c['sources'] (plural) was replaced by c['change_source'] (singular) in 0.7.6,
and will be removed by 0.8.0.
c['bots'] was replaced by c['buildslaves'] in 0.7.6, and will be removed by
0.8.0 . c['bots'] only accepts BuildSlave instances, not name/passwd tuples.
The html.Waterfall status target was replaced by html.WebStatus in 0.7.6, and
will be removed by 0.8.0.
* Release 0.7.8 (24 Jul 2008)
** New features
The IRC bot will respond to three new commands: 'notify' subscribes the
channel (or the sender, if the command is sent as a private "/msg") to hear
about build events. 'join' tells the bot to join some new IRC channel.
'leave' tells it to leave a channel. See the "IRC Bot" section of the User's
Manual for details. (#171)
Build Steps now have "statistics", in addition to logfiles. These are used to
count things like how many tests passed or failed. There are methods to sum
these counters across all steps and display the results in the Build status.
The Waterfall display now shows the count of failed tests on the top-most box
in each column, using this mechanism.
The new buildbot.steps.shell.PerlModuleTest step was added, to run Perl unit
tests. This is a wrapper around the regular ShellCommand that parses the
output of the standard perl unit test system and counts how many tests
passed/failed/etc. The results are put into the step's summary text, and a
count of tests passed/failed/skipped are tracked in the steps's statistics.
The factory.CPAN build factory has been updated to use this, so configuring a
Buildbot to test a perl module available from CPAN should be as easy as:
s = source.CVS(cvsroot, cvsmodule)
f = factory.CPAN(s)
Build Properties have been generalized: they remain associated with a single
Build, but the properties can be set from a variety of sources. In previous
releases, the Build itself would set properties like 'buildername', 'branch',
and 'revision' (the latter two indicating which version of the source code it
was trying to get), and the source-checkout BuildSteps would set a property
named 'got_revision' (to indicate what version of the soruce code it actually
got). In this release, the 'scheduler' property is set to indicate which
Scheduler caused the build to be started. In addition, the config file can
specify properties to be set on all Builds, or on all Builds for a specific
Builder. All these properties are available for interpolation into
ShellCommands and environment variables by using the WithProperties() marker.
It may be easier to implement simple build parameterization (e.g. to upload
generated binaries to a specific directory, or to only perform long-running
tests on a nightly build instead of upon every checkin) by using these Build
Properties than to write custom BuildSteps.
** Other improvements
The /buildslaves web page shows which slaves are currently running builds.
Offline slaves are displayed in bold.
Buildbot's setup.py now provides metadata to setuptools (if installed): an
entry_points script was added, and a dependency upon twisted-2.4.x or newer
was declared. This makes it more likely that 'easy_install buildbot' will
work.
The MailNotifier class acquired a mode="passing" flag: in this mode, the
buildbot will only send mail about passing builds (versus only on failing
builds, or only on builds which failed when the previous build had passed).
** Bugs fixed
Don't display force/stop build buttons when build control is disabled (#246)
When a build is waiting on a lock, don't claim that it has started (#107)
Make SVN mode=copy tolerate symlinks on freebsd, "cp -rp" -> "cp -RPp" (#86)
The svnpoller changesource now ignores branch deletion (#261)
The Git unit tests should run even if the user has not told Git about their
username/email.
The WebStatus /xmlrpc server's getStatus() method was renamed to the
more-accurate getLastBuildResults().
The TinderboxMailNotifier status output acquired an useChangeTime= argument.
The bonsaipoller changesource got some fixes.
** Deprecation Schedule
No features have been deprecated in this release, and no deprecated features
have been removed. As a reminder, the following deprecated features are
scheduled for removal in an upcoming release:
c['sources'] (plural) was replaced by c['change_source'] (singular) in 0.7.6,
and will be removed by 0.8.0.
c['bots'] was replaced by c['buildslaves'] in 0.7.6, and will be removed by
0.8.0 . c['bots'] only accepts BuildSlave instances, not name/passwd tuples.
The html.Waterfall status target was replaced by html.WebStatus in 0.7.6, and
will be removed by 0.8.0.
* Release 0.7.7 (29 Mar 2008)
** Things You Need To Know
*** builder names must not start with an underscore (`_').
These are now reserved for internal buildbot purposes, such as the magic
"_all" pseudo-builder that the web pages use to allow force-build buttons
that start builds on all Builders at once.
** New Features
*** "buildbot checkconfig"
The "buildbot checkconfig" command will look at your master.cfg file and tell
you if there are any problems with it. This can be used to test potential
changes to your config file before submitting them to the running
buildmaster. This is particularly useful to run just before doing "buildbot
restart", since the restart will fail if the config file has an error. By
running "buildbot checkconfig master.cfg && buildbot restart", you'll only
perform the restart if the config file was ok. Many thanks to Ben Hearsum for
the patch.
*** Waterfall "?category=FOO" query-arguments
The Waterfall page now accepts one or more "category=" query arguments in the
URL, to filter the display by categories. These behave a lot like the
"builder=" query argument. Thanks to Jermo Davann for the patch.
** Bugs Fixed
Many bugs were fixed, and many minor features were added. Many thanks to
Dustin Mitchell who fixed and coordinated many of these. Here is a terse
list, for more details, please see the Trac page for the 0.7.7 release, at
http://buildbot.net/trac/query?status=closed&milestone=0.7.7 :
Many of the URLs generated by the buildbot were wrong.
Display of last-heard-from timestamps on the buildslaves web page were wrong.
Asking an IRC bot about a build waiting on a Lock should no longer crash.
Same for the web viewer.
Stop treating the encouraged info/ directory as leftover.
Add more force/stop build buttons.
Timestamps displayed on the waterfall now handle daylight savings properly.
p4poller no longer quits after a single failure.
Improved Git support, including 'try', branch, and revisions.
Buildslaves now use 'git', not 'cogito'.
Make older hg client/servers handle specific-revision builds properly.
Fix twisted.scripts._twistw problem on twisted-2.5.0 and windows.
Fix workdir= and env= on ShellCommands
Fix logfile-watching in 'buildbot start' on OS-X.
Fix ShellCommand crashes when the program emits >640kB of output per chunk.
New WarningCountingShellCommand step.
Fix TreeSize step.
Fix transfer.FileUpload/FileDownload crashes for large files.
Make 'buildbor reconfig' on windows tell you that it doesn't work.
Add a To: header to the mail sent by the slave-missing timeout.
Disable usePTY= for most unit tests, it makes some debian systems flunk tests.
Add 'absolute source stamps'
Add 'triggerable schedulers', and a buildstep to trigger them.
Remove buildbot.changes.freshcvsmail
Add new XMLRPC methods: getAllBuilders, getStatus, getLastBuilds.
Accept WithProperties in more places: env=, workdir=, others.
Use --no-auth-cache with SVN commands to avoid clobbering shared svn state.
Add hours/minutes/seconds in the waterfall's ETA display.
Trial: count Doctest lines too.
ShellCommand: record more info in the headers: stdin closing, PTY usage.
Make it possible to stop builds across reconfig boundaries.
SVN revision numbers are now passed as strings, which was breaking MailNotifier
** Deprecation Schedule
The changes.freshcvsmail change source was replaced by
changes.mail.FCMaildirSource in 0.7.6, and has been removed in 0.7.7 .
c['sources'] (plural) was replaced by c['change_source'] (singular) in 0.7.6,
and will be removed by 0.8.0.
c['bots'] was replaced by c['buildslaves'] in 0.7.6, and will be removed by
0.8.0 . c['bots'] only accepts BuildSlave instances, not name/passwd tuples.
The html.Waterfall status target was replaced by html.WebStatus in 0.7.6, and
will be removed by 0.8.0.
* Release 0.7.6 (30 Sep 2007)
** Things You Need To Know
*** 'buildbot upgrade-master'
Each time you install a new version of Buildbot, you should run the new
'buildbot upgrade-master' command on each of your pre-existing buildmasters.
This will add files and fix (or at least detect) incompatibilities between
your old config and the new code.
*** new WebStatus page
The Waterfall has been replaced by the more general WebStatus display,
described below. WebStatus serves static files from a new public_html/
directory that lives in the buildmaster's basedir. Files like index.html,
buildbot.css, and robots.txt are served directly from that directory, so any
modifications you wish to make should be made to those files. In particular,
any custom CSS you've written should be copied into public_html/buildbot.css.
The 'upgrade-master' command will populate this directory for you.
The old Waterfall page is deprecated, but it should continue to work for
another few releases. It is now a subclass of WebStatus which just replaces
the default root URL with another copy of the /waterfall resource.
*** Compatibility: Python-2.3 or newer, Twisted-2.0 or newer
No compatiblity losses here, buildbot-0.7.6 is compatible with the same
versions of python and twisted that 0.7.5 was.
Buildbot is tested on a regular basis (http://buildbot.buildbot.net) against
nearly a full matrix of Python-(2.3,2.4,2.5) * Twisted-(2.0,2.1,2.2,2.4,2.5).
*** New Buildbot Home Page
Buildbot has moved to a new Trac instance at http://buildbot.net/ , and all
new bugs and tickets should be filed there. The old sourceforge bugs at
http://buildbot.sf.net/ will slowly be migrated over. Mailing lists are still
managed at sourceforge, and downloads are still available there.
*** Changed/Deprecated master.cfg Keys and Classes
c['sources'] (plural) has been replaced by c['change_source'] (singular).
c['bots'] has been replaced by c['buildslaves'], and it expects a list of
BuildSlave instances instead of tuples. See below for more details.
The 'freshcvsmail' change source has been deprecated, and will be removed in
the next release.
The html.Waterfall status target has been deprecated, and replaced by
html.WebStatus .
** New Features
*** WebStatus
The new WebStatus display is a superset of the old Waterfall. It contains a
waterfall as a sub-page, but it also contains pages with more compact
representations of recent build status. The "one_line_per_build" page
contains just that, and "one_box_per_builder" shows just the information from
the top of the waterfall page (last-finished-build and current-activity).
The initial page (when you hit the root of the web site) is served from
index.html, and provides links to the Waterfall as well as the other pages.
Most of these pages can be filtered by adding query arguments to the URL.
Adding "?builder=XYZ" will cause the page to only show results for the given
builder. Adding "?builder=XYZ&builder=ABC" will show results for either
builder. "?branch=trunk" will limit the results to builds that involved code
from the trunk.
The /waterfall page has arguments to hide those annoying "buildslave
connected" messages, to start and and at arbitrary times, and to auto-refresh
at a chosen interval (with a hardcoded minimum of 15 seconds). It also has a
"help" page with forms that will help you add all of these nifty filtering
arguments.
The recommended practice is to modify the index.html file to include links to
the filtered pages that you find most useful.
Note that WebStatus defaults to allowForce=False, meaning that the display
will not offer or accept "Force Build" or "Stop Build" controls. (The old
Waterfall defaults to allowForce=True).
The new WebStatus pages try very hard to use only relative links, making life
better when the Buildbot sits behind an HTTP reverse proxy.
In addition, there is a rudimentary XMLRPC server run by the WebStatus
object. It only has two methods so far, but it will acquire more in the
future. The first customer of this is a project to add a buildbot plugin to
Trac.
*** BuildFactory.addStep(Step(args))
BuildFactories can be set up either with a complete list of steps, or by
calling the .addStep() method repeatedly. The preferred way to provide a step
is by instantiating it, rather than giving a class/kwargs pair. This gives
the BuildStep class a chance to examine the arguments (and complain about
anything it doesn't like) while the config file is being read and problems
are being logged. For example, the old-style:
from buildbot.process.factory import BuildFactory, s
steps = [s(CVS, cvsroot="blah", mode="copy"),
s(Compile, command=["make", "all"]),
s(Test, command=["make", "test"]),
]
f = BuildFactory(steps)
is now:
f = BuildFactory()
f.addStep( CVS(cvsroot="blah", mode="copy") )
f.addStep( Compile(command=["make", "all"]) )
f.addStep( Test(command=["make", "test"]) )
Authors of BuildStep subclasses which override __init__ to add new arguments
must register them with self.addFactoryArguments(**newargs) to make sure that
those classes will work with this new style, otherwise the new arguments will
be lost.
Using class/kwargs pairs is deprecated, and will be removed in a future
release.
*** BuildSlave instances, max_builds=, notify_on_missing=
Buildslave specification has changed a lot in this release. The old config:
c['bots'] = [ ("bot1name", "bot1passwd"),
("bot2name", "bot2passwd") ]
is now:
from buildbot.buildslave import BuildSlave
c['slaves'] = [ BuildSlave("bot1name", "bot1passwd"),
BuildSlave("bot2name", "bot2passwd") ]
This new form gives us the ability to add new controls. The first is
"max_builds=", which imposes a concurrency limit that is like the usual
SlaveLock, but gives the buildmaster the opportunity to find a different
slave to run the build. (the buildslave is chosen before the SlaveLock is
claimed, so pure SlaveLocks don't let you take full advantage of build
farms).
The other addition is "notify_on_missing=", which accepts an email address
(or list of addresses), and sends a message when the buildslave has been
disconnected for more than an hour (configurable with missing_timeout=). This
may be useful when you expect that the buildslave hosts should be available
most of the time, and want to investigate the reasons that it went offline.
** Other Improvements
The IRC bot has been refactored to make it easier to add instant-messaging
status delivery in the future. The IM plugins are not yet written, though.
When multiple buildslaves are available for a given build, one of them will
be picked at random. In previous releases, the first one on the list was
always picked. This helps to add a certain measure of load-balancing. More
improvements will be made in the future.
When the buildslave does a VC checkout step that requires clobbering the
build directory (i.e. in all modes except for 'update'), the buildslave will
first set the permissions on all build files to allow their deletion, before
it attempts to delete them. This should fix some problems in which a build
process left non-user-writable files lying around (frequently a result of
enthusiastic unit tests).
The BuildStep's workdir= argument can now accept a WithProperties()
specification, allowing greater control over the workdir.
Support for the 'Bazaar' version control system (/usr/bin/bzr) has been
added, using the buildbot.steps.source.Bzr class. This is a replacement for
the old 'Arch' (/usr/bin/tla and /usr/bin/baz) systems, which are still
supported by Buildbot with the source.Arch and source.Bazaar classes,
respectively. Unfortunately the old baz system claimed the 'Bazaar' classname
early, so the new system must use source.Bzr instead of the desired
source.Bazaar . A future release might change this.
A rudimentary Gnome Panel applet is provided in contrib/bb_applet.py, which
provides 'buildbot statusgui' -like colored status boxes inside the panel.
Installing it is a bit tricky, though.
The 'buildbot try' command now accepts a '--diff=foo.patch' argument, to let
you provide a pre-computed patch. This makes it easier to test out patches
that you've looked over for safety, without first applying them to your local
source tree.
A new Mercurial change source was added, hg_buildbot.py, which runs as an
in-process post-commit hook. This gives us access to much more information
about the change, as well as being much faster.
The email-based changesource have been refactored, to make it easier to write
new mail parsers. A parser for the SVN "commit-email.pl" script has been
added.
** Bugs Fixed
Far too many to count. Please see
http://buildbot.net/trac/query?status=closed&milestone=0.7.6 for a partial
list of tickets closed for this release, and the ChangeLog for a complete
list of all changes since 0.7.5 .
* Release 0.7.5 (10 Dec 2006)
** Things You Need To Know
*** The Great BuildStep Renaming
All BuildSteps have moved! They used to be classes in buildbot.process.step,
but now they all have separate modules in buildbot.steps.* . They have been
split out into separate categories: for example, the source checkout steps
are now buildbot.steps.source.CVS, buildbot.steps.source.Darcs, etc. The most
commonly used one is probably buildbot.steps.shell.ShellCommand . The
python-specific steps are in buildbot.steps.python, and the Twisted-specific
steps are in buildbot.steps.python_twisted .
You will need to update your master.cfg files to use the new names. The old
names are deprecated and will be removed altogether in the next release.
*** Compatibility
Buildbot now requires python-2.3 or later. Buildbot now requires
Twisted-2.0.0 or later. Support for earlier versions of both has finally been
removed. If you discover it works with unsupported versions, please return
your Buildbot to the factory for repairs :-).
Buildbot has *not* yet been tested against the recent python-2.5 release. It
has been tested against the latest SVN version of Twisted, but only in
conjunction with python-2.4 .
** new features
*** reconfiguring a Builder no longer causes a disconnect/reconnect cycle
This means that sending SIGHUP to the master or running 'buildbot reconfig
MASTERDIR' command no longer interrupts any current builds, nor does it lose
pending builds like it did before. This involved a fairly substantial
refactoring of the various internal BotPerspective/BotMaster/Builder classes.
Note that reconfiguring Schedulers still loses any Changes that were waiting
for the tree to become stable: hopefully this will be fixed in the next
release.
*** 'buildbot start/restart/reconfig' now show logs until startup is complete
These commands now have additional code to follow twistd.log and display all
the lines that are emitted from the beginning of the start/reconfig action
until it has completed. This gives you a chance to see any problems detected
in the config file without needing to manually look in twistd.log or use
another shell to 'tail -f' it. This also makes it clear which config file is
being used. This functionality is not available under windows.
In addition, if any problems are detected during 'start' or 'restart' (but
not reconfig), the buildbot command will terminate with a non-zero exit
status, making it easier to use in scripts. Closes SF#1517975.
*** Locks now take maxCount=N to allow multiple simultaneous owners
This allows Locks to be non-exclusive but still limit maximum concurrency.
Thanks to James Knight for the patch. Closes SF#1434997.
*** filetransfer steps
buildbot.steps.transfer.FileUpload is a buildstep that will move files from
the slave to the master. Likewise, FileDownload will move files from the
master down to the buildslave. Many thanks to Albert Hofkamp for contributing
these classes. Closes SF#1504631.
*** pyflakes step
buildbot.steps.python.PyFlakes will run the simple 'pyflakes' static analysis
tool and parse the results to tell you about undefined names, unused imports,
etc. You'll need to tell it how to run pyflakes, usually with something like
command=["pyflakes", "src/packagedir"] or the like. The default command is
"make pyflakes", which assumes that you have a suitable target in your
top-level Makefile.
*** Monotone support
Nathaniel Smith has contributed initial support for the Monotone version
control system. The code still needs docs and tests, but on the other hand it
has been in use by the Monotone buildbot for a long time now, so it is
probably fairly stable.
*** Tinderbox support
Ben Hearsum and the Mozilla crew have contributed some classes to allow
Buildbot to work with Tinderbox clients. One piece is
buildbot.changes.bonsaipoller.BonsaiPoller, which is a ChangeSource that
polls a Bonsai server (which is a kind of web-vased viewcvs CGI script) to
discover source code changes. The other piece is
buildbot.status.tinderbox.TinderboxMailNotifier, which is a status plugin
that sends email in the same format as Tinderbox does, which allows a number
of Tinderbox tools to be driven by Buildbot instead.
*** SVN Poller
Niklaus Giger contributed a ChangeSource (buildbot.changes.svnpoller) which
polls a remote SVN repository on a periodic basis. This is useful when, for
whatever reason, you cannot add a post-commit hook script to the repository.
This obsoletes the external contrib/svn_watcher.py script.
** notes for plugin developers
*** IStatusLog.readlines()
This new method makes it easier for a status plugin (or a
BuildStep.createSummary method) to walk through a StatusLog one line at a
time. For example, if you wanted to create an extra logfile that just
contained all the GCC warnings from the main log, you could use the
following:
def createSummary(self, log):
warnings = []
for line in log.readlines():
if "warning:" in line:
warnings.append()
self.addCompleteLog('warnings', "".join(warnings))
The "BuildStep LogFiles" section of the user's manual contains more
information. This method is not particularly memory-efficient yet (it reads
the whole logfile into memory first, then splits it into lines); this will be
improved in a future release.
** bug fixes
*** Update source.SVN to work with the new SVN-1.4.0
The latest subversion changed the behavior in an unusual situation which
caused the unit tests to fail. This was unlikely to cause a problem in actual
usage, but the tests have been updated to pass with the new version.
*** update svn_buildbot.py to avoid mangling filenames
Older versions of this script were stripping the wrong number of columns from
the output of 'svnlook changed', and would sometimes mangle filenames. This
has been fixed. Closes SF#1545146.
*** logfiles= caused subsequent build failures under Windows
Earlier versions of buildbot didn't explicitly close any logfiles= file
handles when the build finished. On windows (where you cannot delete a file
that someone else is reading), this could cause the next build to fail as the
source checkout step was unable to delete the old working directory. This has
been fixed. Closes SF#1568415.
*** logfiles= didn't work on OS-X
Macintosh OS-X has a different behavior when reading files that have reached
EOF, the result was that logfiles= sometimes didn't work. Thanks to Mark Rowe
for the patch.
** other changes
The 'buildbot sighup MASTERDIR' command has been replaced with 'buildbot
reconfig MASTERDIR', since that seems to be a slightly more meaningful name.
The 'sighup' form will remain as an alias.
* Release 0.7.4 (23 Aug 2006)
** Things You Need To Know
The PBChangeSource's prefix= argument has changed, you probably need to add a
slash now. This is mostly used by sites which use Subversion and
svn_buildbot.py.
The subcommands that are used to create a buildmaster or a buildslave have
changed. They used to be called 'buildbot master' and 'buildbot slave'. Now
they are called 'buildbot create-master' and 'buildbot create-slave'. Zipf's
Law suggests that these are more appropriate names for these
infrequently-used commands.
The syntax for the c['manhole'] feature has changed.
** new features
*** full Perforce support
SF#1473939: large patch from Scott Lamb, with docs and unit tests! This
includes both the step.P4 source-checkout BuildStep, and the changes.p4poller
ChangeSource you'll want to feed it. P4 is now supported just as well as all
the other VC systems. Thanks Scott!
*** SSH-based Manhole
The 'manhole' feature allows buildbot developers to get access to a python
read/eval/print loop (REPL) inside the buildmaster through a network
connection. Previously, this ran over unencrypted telnet, using a simple
username/password for access control. The new release defaults to encrypted
SSH access, using either username/password or an authorized_keys file (just
like sshd). There also exists an unencrypted telnet form, but its use is
discouraged. The syntax for setting up a manhole has changed, so master.cfg
files that use them must be updated. The "Debug options" section in the
user's manual provides a complete description.
*** Multiple Logfiles
BuildSteps can watch multiple log files in realtime, not just stdout/stderr.
This works in a similar fashion to 'tail -f': the file is polled once per
second, and any new data is sent to the buildmaster.
This requires a buildslave running 0.7.4 or later, and a warning message is
produced if used against an old buildslave (which will otherwise produce no
data). Use "logfiles={'name': 'filename'}" to take advantage of this feature
from master.cfg, and see the "ShellCommand" section of the user's manual for
full documentation.
The 'Trial' buildstep has been updated to use this, to display
_trial_temp/test.log in realtime. It also knows to fall back to the previous
"cat" command if the buildslave is too old.
*** BuildStep URLs
BuildSteps can now add arbitrary URLs which will be displayed on the
Waterfall page in the same place that Logs are presented. This is intended to
provide a link to generated HTML pages, such as the output of a code coverage
tool. The step is responsible for somehow uploading the HTML to a web server:
this feature merely provides an easy way to present the HREF link to the
user. See the "BuildStep URLs" section of the user's manual for details and
examples.
*** LogObservers
BuildSteps can now attach LogObservers to various logfiles, allowing them to
get real-time log output. They can use this to watch for progress-indicating
events (like counting the number of files compiled, or the number of tests
which have run), and update both ETA/progress-tracking and step text. This
allows for more accurate ETA information, and more information passed to the
user about how much of the process has completed.
The 'Trial' buildstep has been updated to use this for progress tracking, by
counting how many test cases have run.
** new documentation
What classes are useful in your master.cfg file? A table of them has been
added to the user's manual, in a section called "Index of Useful Classes".
Want a list of all the keys in master.cfg? Look in the "Index of master.cfg
keys" section.
A number of pretty diagrams have been added to the "System Architecture"
portion of the manual, explaining how all the buildbot pieces fit together.
An HTML form of the user's manual is now shipped in the source tarball. This
makes it a bit bigger: sorry about that. The old PyCon-2003 paper has been
removed from the distribution, as it is mostly supplanted by the user's
manual by this point.
** bugfixes
SF#1217699 + SF#1381867: The prefix= argument to PBChangeSource has been
changed: now it does just a simple string-prefix match and strip. The
previous behavior was buggy and unhelpful. NOTE: if you were using prefix=
before, you probably need to add a slash to the end of it.
SF#1398174: ignore SVN property changes better, fixed by Olivier Bonnet
SF#1452801: don't double-escape the build URL, fixed by Olivier Bonnet
SF#1401121: add support for running py2exe on windows, by Mark Hammond
reloading unchanged config files with WithProperties shouldn't change anything.