Prototype for storing single-cell data #1020

arteymix · 2024-02-05T19:01:42Z

TODO

add a CLI for loading/reloading single cell data and cell type assignments
extend the GEO loader to detect MEX and other supported formats and apply the appropriate strategy for loading vectors
Add support for HDF5-based single-cell data formats #1039
support saving single cell data to disk (I already added a SingleCellExpresionDataMatrix, we need to finish the work and write it to file). I think MEX is a pretty decent output format for this.

REST API

review which fields should be exposed on the REST API for filtering purposes
add aliases in the REST API to refer to the preferred single cell dimension and cell assignment

gemma-core/src/test/java/ubic/gemma/persistence/util/ListUtilsTest.java

...va/ubic/gemma/persistence/service/expression/experiment/ExpressionExperimentServiceImpl.java

.../main/java/ubic/gemma/persistence/service/expression/experiment/ExpressionExperimentDao.java

gemma-core/src/main/java/ubic/gemma/model/expression/bioAssayData/DataVector.java

...-core/src/main/resources/ubic/gemma/model/expression/designElement/CompositeSequence.hbm.xml

arteymix · 2024-06-11T23:36:41Z

I'm in the process of merging the dev branch to get this work up-to-date.

arteymix · 2024-06-12T21:02:38Z

gemma-core/src/main/resources/ubic/gemma/model/analysis/Investigation.hbm.xml

@@ -130,6 +130,13 @@
 					<!-- cannot be non-null because subsets and generic experiments don't have curation details -->
 					<column name="CURATION_DETAILS_FK" not-null="false" sql-type="BIGINT" unique="true"/>
 				</many-to-one>
+				<set name="singleCellExpressionDataVectors" lazy="true" fetch="select" inverse="true"
+					 cascade="all-delete-orphan">


We should remove the -delete-orphan and manage vectors the same way we do for raw and processed ones.

That include bulk insertion, removal, etc.

Move classes that analyzes single-cell data in ubic.gemma.core.analysis.singleCell. Extract logic that converts data from different scale type.

…nData

… files

Don't require -qtName to be supplied and in the case of no set of vectors matching exactly the one being imported, find one by name.

…disk

Add a new interface BioAssaySetValueObject and use it in the code so that we don't assume that an ExpressionExperimentSubsetValueObject derives from an ExpressionExperimentValueObject. Reintroduce the DoubleVectorValueObject cache. Since we're not caching processed vector in Hibernate anymore, this now make sense to do. Make sure that operations on ProcessedExpressionDataVectorService evict the vectors and vectors by gene caches. Make sure that we create copies when setting a minimum P-value or rank on a vector as to not contaminate the content of the cache. Ideally, those models should be immutable. Regenerate the DWR client code as we're adding a new model.

Add an ExpressionDataPrimitiveDoubleMatrix interface to provide unboxed doubles from bulk and single-cell data matrices. Rename DoubleSingleCellExpressionDataMatrix to SingleCellExpressionDataDoubleMatrix for consistency with other matrix names.

…not set

Make sure that services are transient in tags.

arteymix force-pushed the feature-single-cell branch 6 times, most recently from 9222a95 to 788a61b Compare February 7, 2024 20:13