Skip to content

Commit

Permalink
Update arXiv papers
Browse files Browse the repository at this point in the history
  • Loading branch information
github-actions[bot] committed Oct 14, 2024
1 parent a1d3e82 commit 54fe5fc
Show file tree
Hide file tree
Showing 2 changed files with 525 additions and 3 deletions.
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
<h2 style='font-family: Helvetica, Arial, sans-serif; font-size: 16px; color: #4a4a4a;'>Updated on 2024-10-13</h2>
<h2 style='font-family: Helvetica, Arial, sans-serif; font-size: 16px; color: #4a4a4a;'>Updated on 2024-10-14</h2>

<h3 style='font-family: Helvetica, Arial, sans-serif; font-size: 14px; color: #2a7ae2;'>Brain</h3>

Expand All @@ -14,10 +14,10 @@
</thead>
<tbody>
<tr style='border-bottom: 1px solid #d4d4d4;'>
<td style='padding: 8px;'>2024-10-10</td>
<td style='padding: 8px;'>2024-10-11</td>
<td style='padding: 8px;'>The Computational Complexity of Circuit Discovery for Inner Interpretability</td>
<td style='padding: 6px;'>Federico Adolfi, Martina G. Vilas, Todd Wareham</td>
<td style='padding: 8px;'><a href='http://arxiv.org/abs/2410.08025v1' style='color: #1a73e8;'>Link</a></td>
<td style='padding: 8px;'><a href='http://arxiv.org/abs/2410.08025v2' style='color: #1a73e8;'>Link</a></td>
<td style='padding: 4px;'>Many proposed applications of neural networks in machine learning, cognitive/brain science, and society hinge on the feasibility of inner interpretability via circuit discovery. This calls for empirical and theoretical explorations of viable algorithmic options. Despite advances in the design and testing of heuristics, there are concerns about their scalability and faithfulness at a time when we lack understanding of the complexity properties of the problems they are deployed to solve. To address this, we study circuit discovery with classical and parameterized computational complexity theory: (1) we describe a conceptual scaffolding to reason about circuit finding queries in terms of affordances for description, explanation, prediction and control; (2) we formalize a comprehensive set of queries that capture mechanistic explanation, and propose a formal framework for their analysis; (3) we use it to settle the complexity of many query variants and relaxations of practical interest on multi-layer perceptrons (part of, e.g., transformers). Our findings reveal a challenging complexity landscape. Many queries are intractable (NP-hard, $\Sigma^p_2$-hard), remain fixed-parameter intractable (W[1]-hard) when constraining model/circuit features (e.g., depth), and are inapproximable under additive, multiplicative, and probabilistic approximation schemes. To navigate this landscape, we prove there exist transformations to tackle some of these hard problems (NP- vs. $\Sigma^p_2$-complete) with better-understood heuristics, and prove the tractability (PTIME) or fixed-parameter tractability (FPT) of more modest queries which retain useful affordances. This framework allows us to understand the scope and limits of interpretability queries, explore viable options, and compare their resource demands among existing and future architectures.</td>
</tr>
<tr style='border-bottom: 1px solid #d4d4d4;'>
Expand Down
Loading

0 comments on commit 54fe5fc

Please sign in to comment.