Skip to content

Commit

Permalink
Bug in control.py (#18)
Browse files Browse the repository at this point in the history
If `model` has an attribute called `model`, then you should get the layers via `model.model.layers` not `model.layers`
  • Loading branch information
sam-scale authored Mar 10, 2024
1 parent 4351da0 commit 34e363e
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion repeng/control.py
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ def model_layer_list(model: ControlModel | PreTrainedModel) -> torch.nn.ModuleLi
model = model.model

if hasattr(model, "model"): # mistral-like
return model.layers
return model.model.layers
elif hasattr(model, "transformer"): # gpt-2-like
return model.transformer.h
else:
Expand Down

0 comments on commit 34e363e

Please sign in to comment.