Unable to download base data area using downloadIsisData

[AustinSanders]

Hello. I am new to linux and using wsl2 Ubuntu-20.04.
I have scoured the internet for days and tried every command I can come across trying to figure out how to download mission specific data. To those with stronger technical skills, this may seem rudimentary but could you please help me understand how I download data. Since rsync no longer works I am lost as to how to download data. I have ISIS 5.0.0 installed but I cannot figure out how I actually download the base data and subsequent mission data. I have installed rclone but do not understand how to save or run the script and/or download. I have tried wget also but I cannot find any clear example of a command I need to type in to download the base data.

While logged into my isis env in anaconda, I have tried
"downloadIsisData base $ISISDATA"
"wget downloadIsisData base $ISISDATA"
but I no matter what I try I receive command not found.

Is there a simple straightforward command to use for downloading data?
Thank you.

@Kulain619

I moved this question to the discussion board for better visibility and archival.

[AustinSanders]

@Kulain619 It sounds like you may not be in the correct directory. From the root of the repository, you should be in /isis/scripts. If you cloned the repository to ~/kulain619/ISIS3, you'd need to be in ~/kulain619/ISIS3/isis/scripts/ to run the downloadIsisData script

[Kulain619]

Thank you for responding. I assumed the correct download folder would be /Kulain619/anaconda3/envs/isis/data which is the order my directories follow, but I will try
"downloadIsisData base $ISISDATA"
and
"downloadIsisData mro $ISISDATA"
commands into the scripts folder. Am I understanding that correctly?

Alternatively, I tried downloading some data from the PDS SPICE Archives with instructions from the tutorial pdf. It listed this command
"wget -m -nH --cut-dirs=5 -nv (insert the URL of the "Archive
Link" for the SPICE data set here)."

"wget -m -nH --cut-dirs=5 -nv https://naif.jpl.nasa.gov/pub/naif/pds/pds4/mars2020/"
is what I used and it filled up my entire hard drive in no time (which has 330 GB available) even though it was listed as 8.5 GB :(
I am still working on trying to remove all of that and may just have to do a fresh linux install but then I will try what you recommend.

[Kulain619]

I have still not been able to figure out the downloading of base or mission data and I know it is just around the corner. I would greatly appreciate some help. Are the "download data script" as you called it and rclone approaches different or one in the same? I have tried both (as far as I understand them) but cannot figure out where I am going wrong.
I have tried using the command "downloadIsisData mro $ISISDATA" from the readme but I always receive command not found. I have tried this from my isis/scripts dir instead of isis/data as you mentioned but I still get command not found.

I have setup rclone exactly as the directions say and tried many different commands but I keep receiving messages about flags or non flag arguments. A couple for examples are....
"rclone sync remote = asc_s3:asc-isisdata/usgs_data/base/"
"rclone sync asc_s3:asc-isisdata/usgs_data/base/"
"rclone sync remote:asc_s3:asc-isisdata/usgs_data/base/"

What exactly do I need to type as my rclone sync command?

Thank you.

[AustinSanders]

From your scripts directory, can you try ./downloadIsisData base $ISISDATA ? The ./ may be required here

[AustinSanders]

As an update, I had a fundamental misunderstanding about the installation of downloadIsisData (seen on #5181 ), and this should be accessible from the installed binaries. That issue is being tracked, and I'm working on getting the script placed in the correct install directory. In the meantime, ./downloadIsisData may serve as a temporary fix.

[Kulain619]

Great news, I was able to download the base and mro data I was seeking. Thank you very much for giving me some direction.
I uninstalled and started over from scratch. I was using ISIS 5.0.0 before which was probably largely part of the problem.
Nonetheless, I installed ISIS 7.1.0 since other posts you directed me to implied the script was not included until that version. I noticed this install has both the rclone.conf and the downloadIsisData files I was struggling to download and/or implement previously.
It was easier for me to follow the installation instructions this time since it has all become more familiar with trial and error.
In addition to the new version, this time I installed the environment variables with option i. conda env config vars (preferred) instead of using option ii. isisVarInit.py.
The first time I tried to run "downloadIsisData base $ISISDATA" I did receive a message telling me to download rclone, thankfully it even gave me the exact command to use which I just copied and pasted (even though this may seem rudimentary, it would be nice if the installation instructions included this step and command line for novice users. The installation command line from the rclone site probably accomplishes the exact same result but looks slightly different).
Once I had rclone installed, I just went to the path I wanted to download the mission data into (made a directory called data) and used the commands as-is, without ./ and they worked perfectly.

downloadIsisData base $ISISDATA
about 26GB and took less than 10 minutes

downloadIsisData mro $ISISDATA
about 210GB and took just under an hour

More of a convenience, but it would be nice if the approximate size of these data sets was easier to discover beforehand.

THANK YOU @AustinSanders !!

[AustinSanders]

Haha I think I was wrong about 95% of the things that I suggested, but I'm glad that I was able to point you to a discussion that solved the problem!

I've opened #5183 to correct the missing rclone dependency.