chore: Remove convert subcommand & dep on SG monorepo

[varungandhi-src]

We no longer need SCIP to LSIF conversion because the Sourcegraph
backend has been able to accept SCIP natively since several major
releases, and we do not have any use for LSIF.

Fixes https://linear.app/sourcegraph/issue/GRAPH-824

Test plan

Covered by CI

[phikai]

@varungandhi-src @antonsviridov-src I understand that Sourcegraph may no longer need the convert functionality, but other providers (like GitLab) don't natively support SCIP. The convert functionality provides a bridge for users to continue to be able to index their code.

Is it possible to have this functionality restored?

[varungandhi-src]

@phikai, for that functionality, you can either:

1. Continue using the old scip CLI release which supports SCIP to LSIF conversion. Since we don't make backwards-incompatible changes to the SCIP schema, existing releases of the SCIP CLI are forwards-compatible with future indexes.

2. Write a small library which handles this conversion yourself, based on the old code here (which is Apache 2.0 licensed, and mostly contains the core logic involved in the conversion). The only external dep that might potentially be problematic was on the Sourcegraph monorepo's types such as reader.Element, but those are very simple (you can still see the code in the public snapshot of the Sourcegraph codebase -- they're directly derived from the LSIF spec):

   type Element struct {
       ID      int
       Type    string
       Label   string
       Payload any
   }

When adding new features to SCIP, we have not been making any attempt at trying to map those to LSIF, so I don't think it makes sense for us to act as-if we're still maintaining that code, since we're actually not. We have limited bandwidth on the team, and if I'm being honest, the LSIF related legacy code in Sourcegraph has been slowing us down, so I hope you understand why we might want to simplify things.

[alsmnn]

@varungandhi-src, you can't use the old scip CLI release anymore. If you try to run go build ./cmd/scip it tries to download something from github.com/sourcegraph/sourcegraph/lib, which can't be found anymore. My guess is that this is now a privat repo?

bindings/go/scip/convert.go:19:2: github.com/sourcegraph/sourcegraph/[email protected]: invalid version: git ls-remote -q origin in /home/alsmnn/go/pkg/mod/cache/vcs/1791346fb05b02940886169dc1f2712c426dfbfdfeec0151bbb9fde71fd48a95: exit status 128:
        remote: Repository not found.
        fatal: repository 'https://github.com/sourcegraph/sourcegraph/' not found
bindings/go/scip/convert.go:20:2: github.com/sourcegraph/sourcegraph/[email protected]: invalid version: git ls-remote -q origin in /home/alsmnn/go/pkg/mod/cache/vcs/1791346fb05b02940886169dc1f2712c426dfbfdfeec0151bbb9fde71fd48a95: exit status 128:
        remote: Repository not found.
        fatal: repository 'https://github.com/sourcegraph/sourcegraph/' not found
bindings/go/scip/convert.go:21:2: github.com/sourcegraph/sourcegraph/[email protected]: invalid version: git ls-remote -q origin in /home/alsmnn/go/pkg/mod/cache/vcs/1791346fb05b02940886169dc1f2712c426dfbfdfeec0151bbb9fde71fd48a95: exit status 128:
        remote: Repository not found.
        fatal: repository 'https://github.com/sourcegraph/sourcegraph/' not found
bindings/go/scip/convert.go:22:2: github.com/sourcegraph/sourcegraph/[email protected]: invalid version: git ls-remote -q origin in /home/alsmnn/go/pkg/mod/cache/vcs/1791346fb05b02940886169dc1f2712c426dfbfdfeec0151bbb9fde71fd48a95: exit status 128:
        remote: Repository not found.
        fatal: repository 'https://github.com/sourcegraph/sourcegraph/' not found

Any other thoughts on how to use the old scip CLI release?

[varungandhi-src]

@alsmnn for older versions, you can download the prebuilt binary releases from https://github.com/sourcegraph/scip/releases

My guess is that this is now a privat repo?

Yes. If you need to build from source, you could fork the scip repo and point the scip commit of interest to the matching commit in https://github.com/sourcegraph/sourcegraph-public-snapshot