generated from satcompetition/2020
-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathtrack_main.html
140 lines (121 loc) · 7.66 KB
/
track_main.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<title>SAT Competition</title>
<link rel="stylesheet" href="main.css" type="text/css">
<link rel="stylesheet" href="template.css" type="text/css">
<link rel="icon" type="image/x-icon" href="doge2.ico">
<script src="https://www.w3schools.com/lib/w3.js"></script>
<style>a#track-main { color:#e17861; }</style>
</head>
<body>
<div class="main">
<div class="navigation" w3-include-html="navigation.html"></div>
<script>w3.includeHTML();</script>
<div class="content">
<h1 class='tpl-name'></h1>
<h2>Submission Instructions</h2>
<p>
The Main Track (as well as the CaDiCaL Hack Subtrack) will be run on the <a href="https://www.starexec.org/starexec/public/about.jsp">StarExec-cluster</a>.
The solvers will be executed with a time limit of 5000 seconds and memory limit of <s>24GB</s> <span style="color:#e17861">128GB</span>.
The StarExec system will be also used for submitting the solvers.
</p>
<h2>Usage of StarExec</h2>
<h4>Getting an acount</h4>
<ol>
<li><a href="https://www.starexec.org/starexec/public/registration.jsp">
Register</a> for an account in the <b>SAT community</b> (unless you are already registered).</li>
<li>Wait until one of the SAT community leaders (Marijn Heule and Tomas Balyo) approves your request (should not take more than 24h).</li>
<li>If you register late <q>in the season</q>, it may also be necessary to add your new user account to the respective spaces mentioned below. Please
send an email to the organiziers if you do not seem to be able to find the necessary spaces on StarExec. (Also note that you might need to
log out of StarExec and then log in again to see such changes taking effect.)
</ol>
<h4>Uploading a solver for testing</h4>
<ol>
<li>The solver is submitted by uploading its source code and a build script. The solvers are then compiled by StarExec.</li>
<li>The uploaded file must have a special format (see the
<a href="https://wiki.uiowa.edu/display/stardev/User+Guide#UserGuide-Uploading">
StarExec User Guide</a> for details, here is an <a href="downloads/mini.zip">example with minisat</a>).
Please follow the required directory structure precisely otherwise StarExec will not be able to compile and run your solver.
</li>
<li>Login to StarExec and click Spaces->Explore (on the top) then on the left open root->SAT->Sat2022Testing</li>
<li>At the bottom of page click "upload solver" to open the upload dialog.</li>
<li>Click "Browse" and select the zip file with your solvers code, enter the solver name, change downloadable to "no" (otherwise everyone can see you source code) and click "upload".</li>
</ol>
<h4>Testing your solver</h4>
<p>We put a few testing instances in the Sat2022Testing space which you can use if you uploaded your solver to this space. To run a test:</p>
<ol>
<li>Go to the root->SAT->Sat2022Testing space.</li>
<li>Unfold "n jobs (+)" and click "create job". This opens a page where you don't need to change anything so just click "next".
(Update: currently the testing problems in Sat2022Testing are compressed using bz2. If your solver does not support native uncompression,
just additionally choose "unpact bz (272)" as a preprocessor here.)
</li>
<li>On the next page select the "choose" option (for the solver), on the next page select the "all" option (for the testing instances)</li>
<li>A page with all solvers (and their configurations) comes up, select the solver you want to run and click "submit" at the bottom of the page.</li>
<li>You can check out the status and results of your test job by clicking on Jobs on the top and selecting your job.</li>
</ol>
<h4>Outputing proofs in the Main Track</h4>
<p>The proof for UNSAT instances must be written in a file called <b>proof.out</b> and placed in the folder which your start script gets as the second parameter ($2). Thefore
your starexec_run_default should contain something like
<pre>#!/bin/bash
./solver $1 --proof-file=$2/proof.out</pre>
</p>
<p>
Note, that the answer line ("s SATISFIABLE" and "s UNSATISFIABLE") and the solution in case of a satisfiable instance (lines starting with "v") must still go to stdout (not into "proof.out").
</p>
<h4>Testing proof output validation</h4>
<p>
To test that your proofs are correctly validated by the version of <a href="http://www.cs.utexas.edu/~marijn/drat-trim/">drat-trim</a>
installed on StarExec, it is easiest to download <a href="downloads/jobxml.zip">this job xml description</a>,
unzip it, replace (while keeping the quotes) "YOUR-SOLVER-CONFIGURAION-ID-GOES-HERE" in the xml file there
with your solver's configuration id as found on StarExec (<a href="downloads/confId.png">example here</a>),
zipping again and uploading the result to the testing space Sat2020Testing using Jobs/"upload job xml" button.
This will run your solver on the five testing benchmarks out of which three are unsatisfiable.
If your solver solves at least one of them under 300s, drat-trim will be subsequently run on the produced proofs.
When inspecting the finished job, do not forget to check both "job information" and (the more detailed) "job output".
</p>
<h4>Uploading the final version of your solver</h4>
<p>
The submission of the final version of a solver is essentially the same as uploading the solver for testing (see above).
To submit the final version of your solver upload it to the space root->SAT->SatComp22.
</p>
<p>
Don't forget to select "downloadable:no" when uploading your solver to prevent other participants from seeing your source code before the start of the competition.
After the solver submission deadline we will ask all the participants to update the downloadable option to "downloadable:yes". This will make the source code
visible to the organizers and all the participants. The participants must comply with this request in order to participate in the competition (except for the No Limits Track).
</p>
<p>
For the Parallel Track and the Incremental Library Track the source codes are submitted by email to the organizers.
</p>
<h4>Emailing your System Description Document and Benchmarks to the organizers</h4>
<p>After you have submitted the final version of your solver send an email to the organizers ([email protected]) containing your system description document
and 20 benchmark problems (see <a href="rules.html">General Rules</a> for more information). </p>
<!--
<ol>
<li></li>
<li></li>
</ol>
<ul>
<li>The uploaded file must have a special format (see the
<a href="https://wiki.uiowa.edu/display/stardev/User+Guide#UserGuide-Uploading">
StarExec User Guide</a> for details, here is an <a href="downloads/minisat220.zip">example with minisat</a>).
</li>
<li>The binaries should be statically linked in order to avoid problems with missing libraries.</li>
<li>The binaries must be compatible with the
<a href="https://www.starexec.org/starexec/public/machine-specs.txt">StarExec machine specifications</a>.</li>
<li>We recommend to select "run test job = yes" with "setting profile = SAT" when uploading a solver. This creates a test job that
runs the newly upladed solver on a random 3SAT formula with 100 variables (10 seconds time limit). If the test job fails
your solver binary is probably not compatible with starexec.</li>
<li>This <a href="https://www.starexec.org/vmimage/">virtual machine image</a> that matches StarExec
can be useful for compiling the proper binary.</li>
</ul>
</li>
<li>Send an email to [email protected] with your system description (see <a href="#general">general rules</a>)
and (optionally) your source code.</li>
</ol>
-->
</div>
</div>
</body>
</html>