Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to clone git repo in pub get due to missing SSH key #99

Open
gwendiagram opened this issue Mar 24, 2020 · 16 comments
Open

Unable to clone git repo in pub get due to missing SSH key #99

gwendiagram opened this issue Mar 24, 2020 · 16 comments
Labels
good first issue Good for newcomers

Comments

@gwendiagram
Copy link

gwendiagram commented Mar 24, 2020

Flutter allows Git repos to be referenced via flutter pub get (https://flutter.dev/docs/development/packages-and-plugins/using-packages) however, when sylph running on AWS tries to flutter pub get it fails due to a missing ssh key. AWS doesn't have a place to put the SSH key (only for CodeCommit) and CodeMagic doesn't pass the ssh key when it pushes to AWS.

Logs:

Starting Flutter app co.glean.app in debug mode...
Running "flutter pub get" in flutter_app...                     
Git error. Command: git clone --mirror [email protected]:{repo}

Cloning into bare repository '/home/device-farm/flutter/.pub-cache/git/cache/glean-audio-library-3a19cf8eeb19f3e3abf0bd839accc81263b56c07'...

ssh_askpass: exec(/usr/local/libexec/ssh-askpass): No such file or directory

Host key verification failed.

fatal: Could not read from remote repository.

Please make sure you have the correct access rights

and the repository exists.

pub get failed (server unavailable) -- attempting retry 1 in 1 second...

Is this something that could be fixed within sylph?

@mmcc007
Copy link
Owner

mmcc007 commented Mar 24, 2020

Are you asking about support for private dart repository?

@mmcc007 mmcc007 added the good first issue Good for newcomers label Mar 24, 2020
@gwendiagram
Copy link
Author

Thanks for the quick reply, appreciated!

We weren't keen on going down the private dart repository route as it would increase complexity. We were more interested in if sylph could handle ssh keys as AWS doesn't have a place that we could pass it in.

@mmcc007
Copy link
Owner

mmcc007 commented Mar 25, 2020

Are you using a private git repository in your pubspec.yaml? Please confirm how you are using ssh.

If understanding correctly, it should be a matter of duplicating your ssh config on the VM in Device Farm. This would involve uploading your private ssh key to the VM and configuring ssh on the VM.

@gwendiagram
Copy link
Author

Yup, we are using a private git repository.

Oh, interesting! OK, marvellous. I'll have a look at how I can upload my private key to the Device Farm VM and let you know how it goes.

Thanks a bunch for your help!

@mmcc007
Copy link
Owner

mmcc007 commented Mar 25, 2020

Uploading to the VM has to be done by Sylph. This requires adding support for passing the path to the ssh key to config file, adding the ssh key file to the uploaded bundle, then configuring ssh on the VM using the ssh key.

@gwendiagram
Copy link
Author

OK - do you suggest that we fork the repo or is this something that could be included in the main sylph codebase?

@mmcc007
Copy link
Owner

mmcc007 commented Mar 25, 2020

Yep, if you get it working, send a PR. Not a great fit for Sylph since appears not much call for this feature. But if PR works, may be good to have.

@gwendiagram
Copy link
Author

Sound, cheers mate! We'll have a look.

@mmcc007
Copy link
Owner

mmcc007 commented Mar 25, 2020

Cool!

BTW: Sylph is using dummy ssh keys to get to a private fastlane match git server when building iOS debug ipa in CI (travis). Haven't looked at it in a while but may be possible to take a similar approach on Device Farm VM.

https://github.com/mmcc007/sylph/tree/master/lib/resources/dummy-ssh-keys
https://github.com/mmcc007/sylph/blob/master/lib/resources/script/local_utils.sh#L48

@tim-fdc
Copy link

tim-fdc commented Mar 28, 2020

Hi, maybe a stupid question. Why do you run flutter pub get in the AWS? Is it due to the test dependencies only?

@mmcc007
Copy link
Owner

mmcc007 commented Mar 29, 2020

@tim-fdc basically yes

@tim-fdc
Copy link

tim-fdc commented Mar 29, 2020

I see. Was just wondering if it is possible to avoid the flutter pub get at all. For us it is a bit of a pain at the moment, because we change to "local" depdencies (via path imports) before running the tests in AWS. Not a very scalable approach in particular since we have many internal packages.

@mmcc007
Copy link
Owner

mmcc007 commented Mar 29, 2020

@tim-fdc May not be fully understanding how you are using local packages (in tests or in app, or both). I think, by default, all local packages are uploaded (probably only need local test packages).

If you are using no local test packages, you could try forking sylph, remove the flutter pub get and see what happens.

@tim-fdc
Copy link

tim-fdc commented Mar 30, 2020

@mmcc007 We will try and come back with the result.

@mmcc007
Copy link
Owner

mmcc007 commented Mar 30, 2020

@gwendiagram btw: as a workaround, if you edit your pubspec.yaml to use 'path' to your private packages, Sylph will copy that package into bundle for upload to Device Farm.

Ultimately, the test running on device farm does not need non-test packages. So the ultimate fix is to create a test-only pubspec.yaml on the fly (#100)

This would likely fix the case for local and private non-test packages

@gwendiagram / @tim-fdc

@ESBA-salman
Copy link

We hit this same issue on Bitrise. We forked the repo and commented the lines here:

cat << EOF > ~/.ssh/config

Then used the following to setup sylph in the runner/CI
pub global activate --source git <git_url>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers
Projects
None yet
Development

No branches or pull requests

4 participants