[New Feature]: Improve dependency management for MI Connectors

[ShammiL]

Description

The current implementations in MI for adding connectors to an integration project and resolving connector dependencies involve several complex and manual steps from both the connector developer and the user sides. To mitigate the problems in the existing implementation, the current mechanisms need to be reformed.

Describe your problem(s)

1. The VSCode extension uses a static JSON file containing all connector metadata. Developers are required to manually update this list every time a connector is updated or a new one is developed.
2. When a user adds a connector to an integration project, the user is required to manually add all the dependency jars.
3. The connector is downloaded and stored inside the project long before build time, when ideally connector and its dependencies should be resolved at build time.

Describe your solution(s)

1. Come up with a proper API spec for the connector store that allows
   • Listing connectors based on a priority algorithm
   • Searching connectors and operations
   • fetching connector metadata including operations and versions released
2. Implement workflows to automate releasing connectors to Maven and treat connectors as Maven dependencies of a project
3. Implement a dependency resolution mechanism to handle connector dependencies and release the users from the burden of having to manually add jars.

Related issue(s) (optional)

No response

Suggested label(s) (optional)

No response

Suggested assignee(s) (optional)

No response

[ShammiL]

Following initial discussions with the teams overseeing the MI VSCode extension and the Connector Store, an API specification has been drafted outlining the required functionalities from the Connector Store side. The APIs will provide access to the following features:

• Retrieve a list of all connectors along with their metadata.
• Search for specific connectors.
• Publish new connectors or release updates to existing connectors in the store.
• Fetch metadata for previous versions of a connector.

The specification can be found here

A final specification will be prepared after reviewing the above draft