Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG] Add celltypes to CellResponse interface and Network.rename_cell() #970

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

asoplata
Copy link
Collaborator

@asoplata asoplata commented Jan 14, 2025

This PR was originally much more complex, but has been greatly simplified by moving the discussion of hardcoded-celltypes to #972. This PR has now been changed to simply be an attempt to merge in the parts of #702 that can be made to pass our tests.

@asoplata
Copy link
Collaborator Author

It is known that the tests are failing "across the board" after merging in #702 's prior work. These are mostly tied to new cell_types arguments and argument order for CellResponse and Network. I should be able to fix these issues as of tomorrow.

@asoplata
Copy link
Collaborator Author

asoplata commented Jan 16, 2025

Edit: The text formerly in this this comment has been moved to #972 (comment) .

@rythorpe
Copy link
Contributor

I agree, CellResponse's handling of cell_types of very confusing. I like solution (1) where you convert everything to "spike_types" instead, which when compared internally to _cell_type_names can be used to delineate between cell vs. drive spikes.

@jasmainak
Copy link
Collaborator

@asoplata I'll go through this PR ... just wanted to mention that you should avoid making branches on upstream as much as possible. It keeps the upstream repo clean. The only branches you want to be making are for each of the releases. Anyone with admin writes can push to any PR, so you can just start with a normal PR on your fork and anyone with push access to hnn-core can also push to the branch on your fork

@jasmainak
Copy link
Collaborator

I agree cell_types are somewhat of a misnomer. But just be wary of changing things that may already be used in other people's codes. If it's really inconvenient, make sure to deprecate it ... I can help you organize a proper deprecation warning if you have never done it before.

For this PR, I would recommend creating a plan which breaks down the contribution into smaller PRs each of which are easy to review and merge. My initial thinking was to make net._add_cell_type public, then see how much of the pipeline works for a new cell type ... e.g., does the plotting of spikes work etc. Once you have that work, then in a separate PR you can implement more advanced features

len(self.pos_dict['L5_basket'])))
return '<%s | %s>' % (class_name, s)
# Dynamically create the description based on the current cell types
descriptions = []
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
descriptions = []
descriptions = list()

just a stylistic suggestion, it's hard to distinguish [] and {} but list() and dict() are fairly clear and unmistakable.

asoplata added a commit that referenced this pull request Jan 17, 2025
This includes many small changes that either fix test-breaking issues
for this branch, retain consistency, or revert certain changes in order
for us to work on more comprehensive solutions to these hardcoding
 issues.

For `Network._create_cell_coords()`, I reverted the changes, because how
we will handle custom celltype positions needs to be considered very
carefully and comprehensively on its own. For the first version, we may
want to force a user of custom-celltypes to provide their own position
data for all their celltypes, in other words passing all the data we
currently assemble using
https://github.com/jonescompneurolab/hnn-core/blob/master/hnn_core/network.py#L432-L435
. If we did this, we could expose `Network.set_cell_positions()` as a
public function to help users make their own positioning, but they would
not be required to use that method. This would be significantly less
brittle than exposing and maintaining `_create_cell_coords` as an API.

For the `cell_types` -> `self.cell_types` changes, I reverted them
because `cell_types` is already converted into `self.cell_types` in an
organized way in `Network._add_cell_type()`.

In `Network.rename_cell()`, simplified type validation checks.

In `test_cell_response.py`, cleaned up a little bit and removed "print"
debugging from tests. Fixed type tests in `test_network.py` to work with
the more standard type validation.

In `viz.py::plot_spikes_raster()`, removed the additional color
selecting code due to color selection already being present in the
function.

In `viz.py` in general, removed non-hardcoded mapping between cell type
names and colors, due to inconsistency of how colors are set in
different plotting functions. Currently, it appears to me that different
plotting functions use different colors for the same cell type. For the
discussion of `cell_response._cell_type_names` vs
`cell_response.cell_types`, see
#970 (comment)
.
@asoplata
Copy link
Collaborator Author

asoplata commented Jan 21, 2025

@asoplata I'll go through this PR ... just wanted to mention that you should avoid making branches on upstream as much as possible. It keeps the upstream repo clean. The only branches you want to be making are for each of the releases. Anyone with admin writes can push to any PR, so you can just start with a normal PR on your fork and anyone with push access to hnn-core can also push to the branch on your fork

@jasmainak Yeah, I'm not sure why I thought this approach was the best. Apologies! Originally, I thought that for a potentially very-large feature with lots of development on it, it would be good to make a "feature branch" out of #702, and then use this PR as the main place for organizing our hardcoding-removal efforts; other, smaller PRs could go into this feature-branch before the whole branch goes to master. The more I thought about it after your comment, however, the more I realized the disadvantages, including:

  1. devs would have to do things in a nonstandard way (at all)
  2. devs would have to target their own PR-branches to a different branch that is not necessarily in-sync with master, master<->feature<->individualPR,
  3. devs would have to build and rebase onto a potentially-volatile feature-branch instead of master. In contrast, I would argue that helping people rebase their own changes on top of master locally is, itself, at least medium-complexity. Adding more complexity to that is a bad idea.
  4. Even if is difficult to break down a major feature into smaller parts, doing so makes master's history significantly easier to comprehend (i.e. the same reason we use "linear history" in the first place), and allows better and independent analysis of each component of said feature (each individual PR). It also means there is just "one workspace context" so to speak.
  5. Intermingling of the actual code for the PR versus discussion (especially about things not located specifically in this PR) would make the conversation harder to read.
  6. etc.

Instead, I should have fixed and merged #702, then made an Issue unbound to specific PRs. As such, I will move the discussion points over to a new issue #972, and then we'll go about it the usual way: many smaller PRs into master.

I will make one more small commit to make your stylistic change, then this PR should actually be merged, and the branch deleted.

@asoplata asoplata changed the title [WIP] Remove hardcoded celltypes [MRG] Add celltypes to CellResponse interface and Network.rename_cell() Jan 21, 2025
@asoplata
Copy link
Collaborator Author

@jasmainak Okay, once this passes the tests, this PR should be ready for merge then branch deletion. Thanks!

Comment on lines +1180 to +1182
for new_name in new_names:
assert new_name in net.cell_types.keys()
assert new_name in net.pos_dict.keys()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also test that original_name isn't a key anymore

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in latest commit.

asoplata and others added 11 commits January 27, 2025 13:23
This is a "toy" commit solely for the purpose of creating a new upstream
branch.
removing hard coding in _create_cell_coords function and linking this with __init__ in class Network

fixing the _create_cell_coords

refactoring _create_cell_coords

refactoring _create_cell_coords contd

removing cell_name from 466 argument

correcting styling error and fixing self.post_dict in class network

making different corrections in network

fixed line 463

working with line 465 network.py

still working with line 407

adding a space

added self before cell_types

commented line 404 and added self to cell_types to make it an attribute to instance network

de-indenting line 115 for pos_dict to get input from the whole code

de-indenting lines 103 thru 115 in network

corrected names of cells to match cell.name

added a comment on the plan lines 81 thru 86 in network_models

changing cell name to L2_basket

added a rename_cell function and _rename_basket subfunction WIP

making some re-organizing of comments to rename_cell function

rename is working with gid_ranges ordered, still working on visualization

fixing a typo

modified rename function to order cell_types items

visualization raster working and colors matching

fixing extra white spaces etc...

.

Updating plot cell function WIP

maintain original order in net.gid_ranges only

modify net.gid_ranges in loop to simplify rename_cell

cleaning up code

modifying the discription

removing extra lines

fixed color_map in plot_raster

marker set to o for all cells

adding tests

aethetic modification

correct elif or error

correcting a typo

correcting pytests

updating pytest

fixing network tests

cell-repsonse test

_add_cell-type
This includes many small changes that either fix test-breaking issues
for this branch, retain consistency, or revert certain changes in order
for us to work on more comprehensive solutions to these hardcoding
 issues.

For `Network._create_cell_coords()`, I reverted the changes, because how
we will handle custom celltype positions needs to be considered very
carefully and comprehensively on its own. For the first version, we may
want to force a user of custom-celltypes to provide their own position
data for all their celltypes, in other words passing all the data we
currently assemble using
https://github.com/jonescompneurolab/hnn-core/blob/master/hnn_core/network.py#L432-L435
. If we did this, we could expose `Network.set_cell_positions()` as a
public function to help users make their own positioning, but they would
not be required to use that method. This would be significantly less
brittle than exposing and maintaining `_create_cell_coords` as an API.

For the `cell_types` -> `self.cell_types` changes, I reverted them
because `cell_types` is already converted into `self.cell_types` in an
organized way in `Network._add_cell_type()`.

In `Network.rename_cell()`, simplified type validation checks.

In `test_cell_response.py`, cleaned up a little bit and removed "print"
debugging from tests. Fixed type tests in `test_network.py` to work with
the more standard type validation.

In `viz.py::plot_spikes_raster()`, removed the additional color
selecting code due to color selection already being present in the
function.

In `viz.py` in general, removed non-hardcoded mapping between cell type
names and colors, due to inconsistency of how colors are set in
different plotting functions. Currently, it appears to me that different
plotting functions use different colors for the same cell type. For the
discussion of `cell_response._cell_type_names` vs
`cell_response.cell_types`, see
#970 (comment)
.
@asoplata asoplata force-pushed the refactor-hardcoded-celltypes branch from 7028981 to a55779c Compare January 27, 2025 18:28
@jasmainak
Copy link
Collaborator

Sorry I haven't had the time to look yet! Please feel free to ping me again on the relevant PRs if I don't respond back in a few days.

@asoplata
Copy link
Collaborator Author

No rush, @jasmainak ; just so you know, I had to force-push solely due to a conflict in whats_new. Currently, the only differences between the version you reviewed 2 weeks ago are that whats_new change, and the additional test suggestion by Ryan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants