Error in harmonyObj$cluster_cpp(): element-wise pow()

[fbx233]

Hi!

I'm trying to use Harmony to correct batch effect

But I encounter the following error:

Harmony 1/10  
0%   10   20   30   40   50   60   70   80   90   100%  
[----|----|----|----|----|----|----|----|----|----|
**************************************************|  
Error in harmonyObj$cluster_cpp() :   
  element-wise pow(): incompatible matrix dimensions: 100x29 and 29x1

My configuration is as follows. And I was running Rcpp_1.0.9 and on the server.

platform       x86_64-pc-linux-gnu         
arch           x86_64                      
os             linux-gnu                   
system         x86_64, linux-gnu           
status                                     
major          4                           
minor          1.3                         
year           2022                        
month          03                          
day            10                          
svn rev        81868                       
language       R                           
version.string R version 4.1.3 (2022-03-10)
nickname       One Push-Up
Rcpp version 1.0.13

So,I installing an older version and solve it

devtools::install_version("Rcpp", version = "1.0.8", repos = "http://cran.us.r-project.org/")

[Enchufa2]

Could you please try the current master version? I think this may have been solved in #1335.

devtools::install_github("RcppCore/Rcpp")

[eddelbuettel]

Can you share a link to Harmony? Is it this project: https://github.com/immunogenomics/harmony ?

If so my suspicion is that is could be a recent Armadillo change which makes matrix/vector ops pay closer attention to matching dimensions. There too the version on GitHub (and on CRAN) will work but may requires a small change in the package using it as we have seen in a few other packages. Which RcppArmadillo version are you using?

[eddelbuettel]

Harmony is also still in r2u as a binary for Ubuntu 20.04, 22.04 and 24.04 and works there.

Oh, and once I use correct capitalization I also see it on CRAN in good standing. So as @Enchufa2 suggested, just make sure all your CRAN packages are current, it should then work with the Rcpp version at CRAN too.

[eddelbuettel]

@fbx233 Please help us to help you. Can you provide the output of, say, sessionInfo() ?

[fbx233]

> sessionInfo()
R version 4.3.3 (2024-02-29)
Platform: x86_64-conda-linux-gnu (64-bit)
Running under: Rocky Linux 8.5 (Green Obsidian)

Matrix products: default
BLAS/LAPACK: /my_conda_env/lib/libmkl_rt.so.2;  LAPACK version 3.9.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C             LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

time zone: Asia/Shanghai
tzcode source: system (glibc)

attached base packages:
 [1] grid      splines   stats4    parallel  stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] reticulate_1.37.0           scRNAtoolVis_0.1.0          lubridate_1.9.3             forcats_1.0.0               stringr_1.5.1               readr_2.1.5                
 [7] tibble_3.2.1                tidyverse_2.0.0             dior_0.1.5                  plot1cell_0.0.0.9000        wordcloud_2.6               simplifyEnrichment_1.12.0  
[13] GEOquery_2.70.0             rlang_1.1.4                 cowplot_1.1.3               EnsDb.Hsapiens.v86_2.99.0   ensembldb_2.26.0            AnnotationFilter_1.26.0    
[19] GenomicFeatures_1.54.1      loomR_0.2.0                 itertools_0.1-3             iterators_1.0.14            R6_2.5.1                    hdf5r_1.3.11               
[25] DoubletFinder_2.0.4         ComplexUpset_1.3.3          purrr_1.0.2                 ggbeeswarm_0.7.2            biomaRt_2.58.0              progress_1.2.3             
[31] scales_1.3.0                MASS_7.3-60.0.1             ggh4x_0.2.8                 plotly_4.10.4               scCustomize_2.1.2           harmony_1.2.0              
[37] hrbrthemes_0.8.7            viridis_0.6.5               viridisLite_0.4.2           RColorBrewer_1.1-3          Nebulosa_1.12.0             presto_1.0.0               
[43] Rcpp_1.0.13                  knitr_1.46                  tidyr_1.3.1                 reshape2_1.4.4              data.table_1.15.4           enrichplot_1.22.0          
[49] ggrepel_0.9.5               ReactomePA_1.46.0           treemap_2.4-4               clusterProfiler_4.10.0      org.Mm.eg.db_3.18.0         AnnotationDbi_1.64.1       
[55] wesanderson_0.3.7           circlize_0.4.16             ComplexHeatmap_2.18.0       SingleCellExperiment_1.24.0 SummarizedExperiment_1.32.0 GenomicRanges_1.54.1       
[61] GenomeInfoDb_1.38.1         IRanges_2.36.0              S4Vectors_0.40.2            MatrixGenerics_1.14.0       matrixStats_1.3.0           ggsci_3.1.0                
[67] monocle_2.32.0              DDRTree_0.1.5               irlba_2.3.5.1               VGAM_1.1-11                 Biobase_2.62.0              Matrix_1.6-5               
[73] BiocGenerics_0.48.1         patchwork_1.2.0             ggpubr_0.6.0                ggplot2_3.5.1               Seurat_5.1.0                SeuratObject_5.0.2         
[79] sp_2.1-4                    dplyr_1.1.4                

loaded via a namespace (and not attached):
  [1] igraph_1.5.1             graph_1.80.0             Formula_1.2-5            ica_1.0-3                rematch2_2.1.2           zlibbioc_1.48.0          tidyselect_1.2.1        
  [8] bit_4.0.5                doParallel_1.0.17        clue_0.3-65              tm_0.7-13                lattice_0.22-6           rjson_0.2.21             blob_1.2.4              
 [15] S4Arrays_1.4.1           png_0.1-8                cli_3.6.2                ggplotify_0.1.2          ProtGenerics_1.34.0      goftest_1.2-3            textshaping_0.4.0       
 [22] BiocIO_1.12.0            uwot_0.1.16              shadowtext_0.1.3         curl_5.2.1               evaluate_0.23            mime_0.12                tidytree_0.4.6          
 [29] leiden_0.4.3.1           stringi_1.8.3            backports_1.5.0          XML_3.99-0.16            httpuv_1.6.15            paletteer_1.6.0          magrittr_2.0.3          
 [36] rappdirs_0.3.3           mclust_6.1               ggraph_2.1.0             org.Hs.eg.db_3.18.0      sctransform_0.4.1        DBI_1.2.2                reactome.db_1.86.2      
 [43] withr_3.0.0              systemfonts_1.1.0        lmtest_0.9-40            tidygraph_1.3.0          rtracklayer_1.62.0       splancs_2.01-45          htmlwidgets_1.6.4       
 [50] fs_1.6.4                 labeling_0.4.3           SparseArray_1.2.2        extrafont_0.19           zoo_1.8-12               XVector_0.42.0           RhpcBLASctl_0.23-42     
 [57] timechange_0.3.0         foreach_1.5.2            fansi_1.0.6              ggtree_3.10.0            R.oo_1.26.0              RSpectra_0.16-1          extrafontdb_1.0         
 [64] ggrastr_1.0.2            fastDummies_1.7.3        gridGraphics_0.5-1       lazyeval_0.2.2           yaml_2.3.8               survival_3.7-0           scattermore_1.2         
 [71] crayon_1.5.2             RcppAnnoy_0.0.22         progressr_0.14.0         tweenr_2.0.3             later_1.3.2              base64enc_0.1-3          gfonts_0.2.0            
 [78] ggridges_0.5.6           codetools_0.2-20         GlobalOptions_0.1.2      alphahull_2.5            HSMMSingleCell_1.20.0    KEGGREST_1.42.0          Rtsne_0.17              
 [85] shape_1.4.6.1            fastICA_1.2-4            limma_3.58.1             gdtools_0.3.7            Rsamtools_2.18.0         filelock_1.0.3           foreign_0.8-86          
 [92] pkgconfig_2.0.3          xml2_1.3.6               GenomicAlignments_1.38.0 aplot_0.2.3              spatstat.sparse_3.0-3    ape_5.8                  gridBase_0.4-7          
 [99] xtable_1.8-4             interp_1.1-6             car_3.1-2                plyr_1.8.9               httr_1.4.7               tools_4.3.3              globals_0.16.3          
[106] checkmate_2.3.2          htmlTable_2.4.3          beeswarm_0.4.0           broom_1.0.6              nlme_3.1-164             HDO.db_0.99.1            dbplyr_2.4.0            
[113] ggunchull_1.0.1          digest_0.6.35            farver_2.1.1             tzdb_0.4.0               ks_1.14.2                yulab.utils_0.1.4        rpart_4.1.23            
[120] crul_1.5.0               glue_1.7.0               cachem_1.0.8             BiocFileCache_2.10.1     polyclip_1.10-6          Hmisc_5.1-3              generics_0.1.3          
[127] proxyC_0.3.4             Biostrings_2.70.1        mvtnorm_1.2-5            parallelly_1.37.1        statmod_1.5.0            ragg_1.3.0               RcppHNSW_0.6.0          
[134] fontBitstreamVera_0.1.1  carData_3.0-5            pbapply_1.7-2            spam_2.10-0              gson_0.1.0               utf8_1.2.4               graphlayouts_1.1.0      
[141] httpcode_0.3.0           ggsignif_0.6.4           gridExtra_2.3            shiny_1.8.1.1            GenomeInfoDbData_1.2.11  R.utils_2.12.3           RCurl_1.98-1.14         
[148] memoise_2.0.1            rmarkdown_2.26           pheatmap_1.0.12          R.methodsS3_1.8.2        future_1.33.2            RANN_2.6.1               fontLiberation_0.1.0    
[155] rstudioapi_0.16.0        spatstat.data_3.0-4      cluster_2.1.6            janitor_2.2.0            spatstat.utils_3.0-4     hms_1.1.3                fitdistrplus_1.1-11     
[162] sgeostat_1.0-27          munsell_0.5.1            colorspace_2.1-0         ggdendro_0.2.0           dotCall64_1.1-1          ggforce_0.4.2            dbscan_1.2-0            
[169] xfun_0.43                abind_1.4-5              GOSemSim_2.28.0          treeio_1.26.0            bitops_1.0-7             promises_1.3.0           scatterpie_0.2.3        
[176] RSQLite_2.3.4            leidenbase_0.1.27        qvalue_2.34.0            fgsea_1.28.0             DelayedArray_0.28.0      GO.db_3.18.0             compiler_4.3.3          
[183] prettyunits_1.2.0        graphite_1.48.0          listenv_0.9.1            Rttf2pt1_1.3.12          fontquiver_0.2.1         tensor_1.5               BiocParallel_1.36.0     
[190] spatstat.random_3.2-3    fastmap_1.1.1            fastmatch_1.1-4          rstatix_0.7.2            vipor_0.4.7              ROCR_1.0-11              nnet_7.3-19             
[197] gtable_0.3.5             KernSmooth_2.23-24       miniUI_0.1.1.1           deldir_2.0-4             htmltools_0.5.8.1        RcppParallel_5.1.6       bit64_4.0.5             
[204] spatstat.explore_3.2-6   lifecycle_1.0.4          ggprism_1.0.5            NLP_0.2-1                restfulr_0.0.15          vctrs_0.6.5              slam_0.1-50             
[211] spatstat.geom_3.2-9      snakecase_0.11.1         DOSE_3.28.1              ggfun_0.1.5              future.apply_1.11.2      pracma_2.4.4             pillar_1.9.0            
[218] combinat_0.0-8           jsonlite_1.8.8           GetoptLong_1.0.5 

[eddelbuettel]

Ok, now please compare the versions to what available.packages() shows at CRAN. For example harmony itself is shown as 1.2.0 -- but CRAN is at 1.2.1. So that is on you / on Conda.

As we demonstrated, the package at CRAN tests fine. So in your mix of 200+ packages you have likely more than this one that are behind. You may have to talk to the people behind these Conda packages. The source package is fine, and eg the binaries I provide via r2u are fine.

Do you know how to get current packages, from source if you must?

I see no issue in Rcpp itself here, so I plan to close this. Please feel free to convince me otherwise, ideally with a minimally reproducible example showing an issue in the CRAN versions which, as we demonstrated, appear to be just fine.

[fbx233]

eddelbuettel

Yes it is, harmony is here (https://github.com/immunogenomics/harmony)

[eddelbuettel]

Yes, we know.

Now please upgrade to the current version of it and see if it works.

[fbx233]

OK, thank you

[eddelbuettel]

BTW the github-based mirror of all the CRAN package has it at 1.2.1: https://github.com/cran/harmony

For reference, CRAN package page also at 1.2.1: https://cran.r-project.org/package=harmony

[Enchufa2]

Also, please check whether the current master version (i.e. remotes::install_github("RcppCore/Rcpp")) solves the issue. And if none of this works, then we would need a minimal reproducible example to investigate further.

[eddelbuettel]

@Enchufa2 No, different issue methinks. This was a recent RcppArmadillo change and harmony adjusted for it IIRC, but didn't not account for in the ChangeLog.

Note that the harmony results at CRAN are perfectly clean with the CRAN version of Rcpp. But @fbx233 is for sure one harmony release behind. Which could well be the issue here.

[Enchufa2]

But if it's RcppArmadillo, in principle, it should fail with Rcpp 1.0.8 too, and @fbx233 reported that downgrading Rcpp solves the issue. Anyway, @fbx233 give us please a minimal reproducible example to avoid going in circles here. :)

[eddelbuettel]

But if it's RcppArmadillo, in principle, it should fail with Rcpp 1.0.8 too

Fair point. I thought I had harmony flagged in one of those recent change tickets for RcppArmadillo but apparently not.

[eddelbuettel]

@fbx233 We would still need an actual reproducible code example that fails for you.