Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Embed version in SDKs #915

Closed
78 tasks done
danielrbradley opened this issue May 8, 2024 · 2 comments
Closed
78 tasks done

Embed version in SDKs #915

danielrbradley opened this issue May 8, 2024 · 2 comments
Assignees
@danielrbradley
Labels
kind/task Work that's part of an ongoing epic resolution/fixed This issue was fixed
Milestone
0.108

Comments

@danielrbradley
Copy link
Member

danielrbradley commented May 8, 2024 •
edited
Loading

Part of https://github.com/pulumi/home/issues/3372

  1. Enable embedding version number in each language's SDK
RespectSchemaVersion: true
  1. Configure ci-mgmt

Versions are stable within the same major version for local development but will be updated during the build in CI.

# These files are updated with the real version number
worktreeAllowedChanges: |-
  sdk/**/pulumi-plugin.json
  sdk/dotnet/Pulumi.*.csproj
  sdk/go/*/internal/pulumiUtilities.go
  sdk/nodejs/package.json
  sdk/python/pyproject.toml

publish:
  goSdk:
    usePush: true

Native Providers

PR for the managed providers: #925

Bridged Providers

PR: #920

Bridged provider mass-update run: https://github.com/pulumi/ci-mgmt/actions/runs/9090224158/job/24992409444

Tier 1

Tier 2
@danielrbradley danielrbradley self-assigned this May 8, 2024
@danielrbradley danielrbradley mentioned this issue May 8, 2024
Merged
@mikhailshilkov mikhailshilkov added the kind/task Work that's part of an ongoing epic label May 8, 2024
danielrbradley added a commit to pulumi/pulumi-kafka that referenced this issue May 9, 2024
@danielrbradley
Embed version in SDKs (#410)
3dad44a 
Part of pulumi/ci-mgmt#915

1. Enable embedding version number in each language.
2. Configure CI to allow changes to files which include the version
number and use new push-based Go SDK publishing.
3. Remove explicit dependencies (defer to the languages's defaults).
@danielrbradley danielrbradley mentioned this issue May 10, 2024
Merged
danielrbradley added a commit that referenced this issue May 14, 2024
@danielrbradley
Embed SDK version (#920)
c34e73d 
1. Change default config to use push-based Go SDK publishing
2. Use gopatch to enable RespectSchemaVersion in all languages
3. Rebuild SDKs if we've updated the resources.go file

Related to #915 and
pulumi/home#3372
@danielrbradley danielrbradley mentioned this issue May 15, 2024
Merged
danielrbradley added a commit to pulumi/pulumi-azure-native that referenced this issue May 15, 2024
@danielrbradley
Embed version in SDKs (#3289)
0d8227f 
Part of pulumi/ci-mgmt#915
- Enable `respectSchemaVersion` for each language.
- Re-generated docs schema.
- Update pulumi version.
- Allow selective changes during SDK builds.
- Update gen test snapshot.
@danielrbradley danielrbradley mentioned this issue May 15, 2024
Merged
danielrbradley added a commit that referenced this issue May 15, 2024
@danielrbradley
Native Go SDK publishing (#925)
2c4960e 
Part of #915

Implement Go SDK publishing for native providers 

- Add allowed changes to CheckWorktreeClean.
- Add PublishGoSdk step & job.
- Give version step an id by default.
- Switch release workflow to use PublishGoSdk instead of TagSdk.

Remove azure-native as it's never been managed by this tool so there's
no reason to generate it here.

Replace `pulumictl get version` for goreleaser step with version action
used everywhere else.
@danielrbradley danielrbradley mentioned this issue May 16, 2024
Merged
danielrbradley added a commit to pulumi/pulumi-aws-native that referenced this issue May 16, 2024
@danielrbradley
Respect schema version (#1539)
48cc36f 
Part of pulumi/ci-mgmt#915
- Enable `respectSchemaVersion` for python, go, nodejs and dotnet.
- Inject version during SDK generation 
- Upgrade pu/pu CLI to ensure we have the extra version argument
available.
- Tidy makefile:
- Remove manual updating of the version number for python and nodejs now
the version number is already set.
- Fix dotnet so the version.txt has the provider version *not* the
dotnet SDK version.
- Remove unneeded shell wrapping in bin/pulumi-java-gen target which
actually causes the download to happen while make is calculating the
target rather than when it runs the target.
This was referenced May 16, 2024
Merged
Merged
danielrbradley added a commit to pulumi/pulumi-command that referenced this issue May 17, 2024
@danielrbradley
Embed SDK versions (#438)
c8823cb 
Part of pulumi/ci-mgmt#915

- Enable respectSchemaVersion
- Upgrade pulumi version. We need to use the latest version to get the
--version argument in the gen-sdk command.
- Inject version during generation 
  - Remove code to inject during the build phase now it's already there.
- Customise Java so it doesn't include the version at build time still.
@mikhailshilkov
Copy link
Member

mikhailshilkov commented May 23, 2024 •
edited
Loading

We have a few newly bridged providers, does it need to land there too?

  • scm
  • dbtcloud
  • sdwan
  • meraki

@danielrbradley danielrbradley mentioned this issue May 28, 2024
Merged
danielrbradley added a commit to pulumi/pulumi-kubernetes that referenced this issue May 28, 2024
@danielrbradley
Enable respectSchemaVersion (#3028)
b6f6eff 
<!--Thanks for your contribution. See [CONTRIBUTING](CONTRIBUTING.md)
    for Pulumi's contribution guidelines.

    Help us merge your changes more quickly by adding more details such
    as labels, milestones, and reviewers.-->

### Proposed changes

Embed provider version in all SDKs. Part of
pulumi/ci-mgmt#915, follow up to:
#3008

1. Enable `respectSchemaVersion` for each language so codegen includes
the version number.
2. Pass the _provider_ version into codegen rather than a
language-specific version - as codegen will do the conversion for us.

### Related issues (optional)

- #3008
- pulumi/ci-mgmt#915
danielrbradley added a commit to pulumi/pulumi-docker-build that referenced this issue May 30, 2024
@danielrbradley @blampe
Embed sdk versions (#68)
c305ea8 
Part of pulumi/ci-mgmt#915

Install pulumi in standard way 

Use the same method as other providers to install the pulumi CLI.
- Lock the version of the CLI and therefore codegen for consistent build
results.
- Ensure the local language plugins are using by disabling ambient
plugins.

Enable respect schema version

Remove setting version at SDK build time

---------

Co-authored-by: Bryce Lampe <[email protected]>
This was referenced Jun 3, 2024
Merged
Closed
danielrbradley added a commit to pulumi/pulumi-awsx that referenced this issue Jun 7, 2024
@danielrbradley
Embed SDK version (#1297)
c94dc01 
Part of pulumi/ci-mgmt#915

Bring workflows in line with the approach used in other providers.

1. Use action for checking for unexpected changes - and allow files with
the version embedded to change once we've enabled
`respectSchemaVersion`.
2. Fix how we calculate the version we're building. Stop relying on the
local git checkout to contain all tags. Use a placeholder locally and
override the placeholder in CI with the version calculated from our
action.
    - Remove fetching all tags.
3. (Pending) enable `respectSchemaVersion`.
@danielrbradley danielrbradley mentioned this issue Jul 9, 2024
Merged
danielrbradley added a commit to pulumi/pulumi-eks that referenced this issue Jul 10, 2024
@danielrbradley
Embed SDK version (#1232)
e00ec0c 
Related to pulumi/ci-mgmt#915

- Rework how we calculate versions
- In the makefile, take an input of PROVIDER_VERSION instead of calling
`pulumictl get version`.
- In CI, use the version action in the prerequisites then set the
PROVIDER_VERSION env in other jobs.
  - Remove unshallow clone for tags.
- Replace the git status check with the dedicated action and allow
specific files that contain the version number to be changed during the
build.
- Switch to the publish Go SDK action rather than tagging in-place.

Master branch CI run:
https://github.com/pulumi/pulumi-eks/actions/runs/9870116213
@mjeffryes mjeffryes modified the milestones: 0.108, 0.107 Jul 24, 2024
@danielrbradley danielrbradley mentioned this issue Jul 24, 2024
Merged
@mjeffryes mjeffryes modified the milestones: 0.107, 0.108 Jul 24, 2024
danielrbradley added a commit to pulumi/pulumi-aws-apigateway that referenced this issue Jul 26, 2024
@danielrbradley
Embed version into the SDKs (#161)
b978236 
Related to pulumi/ci-mgmt#915

## Update local build process

- We already have "respectSchemaVersion" set in the schema, but we don't
have a version set in the schema so it had no effect.
- Set the version in the schema via the generate program - passed in as
a new argument.
- Move generation of the go.mod stubs into the generate step so `make
generate` doesn't leave a dirty working tree.
- Remove sed from nodejs build - just copy the package which now already
has the version set.
- Fix setting the `PACKAGE_VERSION` env var for the Java build -
makefile variables are not env variables.
- Rename `VERSION` to `PROVIDER_VERSION` to match other providers.

## Update CI

- Set `PROVIDER_VERSION` in all workflows.
- Use new working tree clean check to allow for changes to the version
number in the SDK.
- Use new Go SDK publish action to push a new commit & tag

## Additional work

- Switch to using `pulumi package gen-sdk` wherein we also bump up to
the latest version of pulumi.
- Fix broken tests:
  - Fix type change in Go.
  - Remove need for AWS_REGION env.
- Use explicit provider path instead of $PATH now we're specifying the
version in the SDK.
  - Actually run the python test.
- Add release verification - where we test the actual published SDK is
usable in Go, Nodejs and Python.
@danielrbradley danielrbradley added the resolution/fixed This issue was fixed label Jul 26, 2024
@danielrbradley
Copy link
Member Author

This has been completed for all providers

flostadler pushed a commit to pulumi/pulumi-eks that referenced this issue Sep 4, 2024
@danielrbradley
Embed SDK version (#1232)
6386666 
Related to pulumi/ci-mgmt#915

- Rework how we calculate versions
- In the makefile, take an input of PROVIDER_VERSION instead of calling
`pulumictl get version`.
- In CI, use the version action in the prerequisites then set the
PROVIDER_VERSION env in other jobs.
  - Remove unshallow clone for tags.
- Replace the git status check with the dedicated action and allow
specific files that contain the version number to be changed during the
build.
- Switch to the publish Go SDK action rather than tagging in-place.

Master branch CI run:
https://github.com/pulumi/pulumi-eks/actions/runs/9870116213
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@danielrbradley danielrbradley

Labels
kind/task Work that's part of an ongoing epic resolution/fixed This issue was fixed
Projects
None yet
Milestone
0.108
Development

No branches or pull requests
3 participants
@danielrbradley @mjeffryes @mikhailshilkov