Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

simplify responsibility #54

Merged
merged 12 commits into from
Sep 3, 2024
Merged
13 changes: 6 additions & 7 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,19 +36,18 @@ eget golangci/golangci-lint
eget goreleaser/goreleaser
```

## schema - Contribute to the JSON schema
## schema - Generate code due to schema change

The JSON schema of the registry can be found in the [registry.schema.yaml] file, after modification of which the schema in JSON format ([registry.schema.json]) and the golang data model ([registry_gen.go]) must be regenerated.
After modifying registry schema ([registry.schema.json]), the [registry_gen.go] file must be regenerated.

```bash
yq -o=json -P docs/registry.schema.yaml > docs/registry.schema.json
go-jsonschema --capitalization URL --capitalization OSS -p k6registry --only-models -o registry_gen.go docs/registry.schema.yaml
go-jsonschema --capitalization URL --capitalization OSS -p k6registry --only-models -o registry_gen.go docs/registry.schema.json
```

[registry.schema.json]: docs/registry.schema.json
[registry_gen.go]: registry_gen.go

## example - Contribute to the example
## example - Update documentation due to example changes

The example registry can be found in [example.yaml] file, the documentation ([registry.md], [README.md]) must be updated after modification.

Expand Down Expand Up @@ -139,14 +138,14 @@ docker build -t k6registry -f Dockerfile.goreleaser .
rm -rf build
```

## all - Update everything
### update - Update everything

The most robust thing is to update everything (both the schema and the example) after modifying the source.

Requires
: schema, example, readme

## legacy - Convert legacy registry
### legacy - Convert legacy registry

```bash
go run ./cmd/k6registry . --legacy | yq '.[]|= pick(["module","description","tier","products","imports","outputs","repo","categories"])|sort_by(.module)' > ./docs/legacy.yaml
Expand Down
42 changes: 20 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,14 @@
<h1 name="title">k6registry</h1>

**Data model and tooling for the k6 extension registry**
**k6 Extension Registry/Catalog Generator**

This repository contains the [JSON schema](docs/registry.schema.json) of the k6 extension registry and the [`k6registry`](#k6registry) command line tool for generating registry from source. The command line tool can also be used as a [GitHub Action](#github-action).
k6registry is a CLI tool and a GitHub Action that enables the generation of the k6 extension registry. The generation source is a YAML (or JSON) file that contains the most important properties of extensions. The generator generates the missing properties from the repository metadata. Repository metadata is collected using the repository manager APIs. GitHub and GitLab APIs are currently supported.

The generator also performs static analysis of extensions. The result of the analysis is the level of compliance with best practices (0-100%). A compliance grade (A-F) is calculated from the compliance level. The compliance level and grade are stored in the registry for each extension. Based on the compliance grade, an SVG compliance badge is created for each extension. Example badge:

![xk6-sql](https://registry.k6.io/module/github.com/grafana/xk6-sql/badge.svg)

The k6 Extension Catalog is an alternative representation of the k6 Extension Registry. The output of the generation can be in k6 Extension Catalog format. This format is optimized to resolve extensions as dependencies.

Check [k6 Extension Registry Concept](docs/registry.md) for information on design considerations.

Expand Down Expand Up @@ -82,7 +88,7 @@ Registry generated from the source above.
"name": "xk6-dashboard",
"owner": "grafana",
"public": true,
"stars": 327,
"stars": 328,
"timestamp": 1719907965,
"topics": [
"xk6",
Expand Down Expand Up @@ -455,15 +461,7 @@ go install github.com/grafana/k6registry/cmd/k6registry@latest

## GitHub Action

`grafana/k6registry` is a GitHub Action that enables k6 extension registry processing and the generation of customized JSON output for different applications. Processing is based on popular `jq` expressions using an embedded `jq` implementation.

The jq filter expression can be specified in the `filter` parameter.

The extension registry is read from the YAML format file specified in the `in` parameter.

Repository metadata is collected using the repository manager APIs. Currently only the GitHub API is supported.

The output of the processing will be written to the standard output by default. The output can be saved to a file using the `out` parameter.
`grafana/k6registry` is a GitHub Action that enables k6 Extension Registry and Catalog generation.

**Inputs**

Expand All @@ -472,10 +470,11 @@ name | reqired | default | description
in | yes | | input file name
out | no | stdout | output file name
api | no | | output directory name
mute | no | `false` | no output, only validation
quiet | no | `false` | no output, only validation
loose | no | `false` | skip JSON schema validation
lint | no | `false` | enable built-in linter
compact| no | `false` | compact instead of pretty-printed output
catalog| no | `false` | generate catalog instead of registry
ref | no | | reference output URL for change detection

In GitHub action mode, the change can be indicated by comparing the output to a reference output. The reference output URL can be passed in the `ref` action parameter. The `changed` output variable will be `true` or `false` depending on whether the output has changed or not compared to the reference output.
Expand All @@ -491,8 +490,8 @@ changed | `true` if the output has changed compared to `ref`, otherwise `false`
**Example usage**

```yaml
- name: Generate registry in JSON format
uses: grafana/[email protected].10
- name: Generate extension registry
uses: grafana/[email protected].18
with:
in: "registry.yaml"
out: "registry.json"
Expand All @@ -504,22 +503,23 @@ changed | `true` if the output has changed compared to `ref`, otherwise `false`
<!-- #region cli -->
## k6registry

k6 extension registry generator
k6 Extension Registry/Catalog Generator

### Synopsis

Command line k6 extension registry generator.
Generate k6 extension registry/catalog from source.

The source of the extension registry contains only the most important properties of the extensions. The rest of the properties are collected by k6registry using the API of the extensions' git repository managers.
The generation source is a YAML (or JSON) file that contains the most important properties of extensions. The generator generates the missing properties from the repository metadata. Repository metadata is collected using the repository manager APIs. GitHub and GitLab APIs are currently supported.

The source of the extension registry is read from the YAML (or JSON) format file specified as command line argument. If it is missing, the source is read from the standard input.
The generator also performs static analysis of extensions. The result of the analysis is the level of compliance with best practices (0-100%). A compliance grade (A-F) is calculated from the compliance level. The compliance level and grade are stored in the registry for each extension. Based on the compliance grade, an SVG compliance badge is created for each extension.

Repository metadata is collected using the API of the extensions' git repository managers. Currently only the GitHub API is supported.
The source is read from file specified as command line argument. If it is missing, the source is read from the standard input.

The output of the generation will be written to the standard output by default. The output can be saved to a file using the `-o/--out` flag.

The `--api` flag can be used to specify a directory to which the outputs will be written. The `registry.json` file is placed in the root directory. The `extension.json` file and the `badge.svg` file (if the `--lint` flag is used) are placed in a directory with the same name as the extension's go module path.


```
k6registry [flags] [source-file]
```
Expand Down Expand Up @@ -551,9 +551,7 @@ By using the `--api` flag, files are created with relative paths in a base direc
```ascii file=docs/example-api.txt
docs/example-api
├── catalog.json
├── openapi.yaml
├── registry.json
├── registry.schema.json
├── category
│   ├── authentication.json
│   ├── browser.json
Expand Down
6 changes: 5 additions & 1 deletion action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ inputs:
description: api directory name
required: false

mute:
quiet:
description: no output, only validation
required: false

Expand All @@ -35,6 +35,10 @@ inputs:
description: compact instead of pretty-printed output
required: false

catalog:
description: generate catalog instead of registry
required: false

ref:
description: reference output URL for change detection
required: false
Expand Down
8 changes: 0 additions & 8 deletions cmd/api.go
Original file line number Diff line number Diff line change
Expand Up @@ -46,14 +46,6 @@ func writeJSON(filename string, source interface{}) error {
}

func writeAPIGroupGlobal(registry k6registry.Registry, target string) error {
if err := writeData(filepath.Join(target, "registry.schema.json"), k6registry.Schema); err != nil {
return err
}

if err := writeData(filepath.Join(target, "openapi.yaml"), k6registry.OpenAPI); err != nil {
return err
}

if err := writeJSON(filepath.Join(target, "registry.json"), registry); err != nil {
return err
}
Expand Down
2 changes: 1 addition & 1 deletion cmd/cmd.go
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ func New() (*cobra.Command, error) {

root := &cobra.Command{
Use: "k6registry [flags] [source-file]",
Short: "k6 extension registry generator",
Short: "k6 Extension Registry/Catalog Generator",
Long: help,
SilenceUsage: true,
SilenceErrors: true,
Expand Down
10 changes: 5 additions & 5 deletions cmd/help.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
Command line k6 extension registry generator.
Generate k6 extension registry/catalog from source.

The source of the extension registry contains only the most important properties of the extensions. The rest of the properties are collected by k6registry using the API of the extensions' git repository managers.
The generation source is a YAML (or JSON) file that contains the most important properties of extensions. The generator generates the missing properties from the repository metadata. Repository metadata is collected using the repository manager APIs. GitHub and GitLab APIs are currently supported.

The source of the extension registry is read from the YAML (or JSON) format file specified as command line argument. If it is missing, the source is read from the standard input.
The generator also performs static analysis of extensions. The result of the analysis is the level of compliance with best practices (0-100%). A compliance grade (A-F) is calculated from the compliance level. The compliance level and grade are stored in the registry for each extension. Based on the compliance grade, an SVG compliance badge is created for each extension.

Repository metadata is collected using the API of the extensions' git repository managers. Currently only the GitHub API is supported.
The source is read from file specified as command line argument. If it is missing, the source is read from the standard input.

The output of the generation will be written to the standard output by default. The output can be saved to a file using the `-o/--out` flag.

The `--api` flag can be used to specify a directory to which the outputs will be written. The `registry.json` file is placed in the root directory. The `extension.json` file and the `badge.svg` file (if the `--lint` flag is used) are placed in a directory with the same name as the extension's go module path.
The `--api` flag can be used to specify a directory to which the outputs will be written. The `registry.json` file is placed in the root directory. The `extension.json` file and the `badge.svg` file (if the `--lint` flag is used) are placed in a directory with the same name as the extension's go module path.
8 changes: 6 additions & 2 deletions cmd/k6registry/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,8 @@ func getArgs() []string {

var args []string

if getenv("INPUT_MUTE", "false") == "true" {
args = append(args, "--mute")
if getenv("INPUT_QUIET", "false") == "true" {
args = append(args, "--quiet")
}

if getenv("INPUT_LOOSE", "false") == "true" {
Expand All @@ -71,6 +71,10 @@ func getArgs() []string {
args = append(args, "--compact")
}

if getenv("INPUT_CATALOG", "false") == "true" {
args = append(args, "--catalog")
}

if api := getenv("INPUT_API", ""); len(api) != 0 {
args = append(args, "--api", api)
}
Expand Down
3 changes: 2 additions & 1 deletion cmd/legacy.go
Original file line number Diff line number Diff line change
Expand Up @@ -80,13 +80,14 @@ func legacyConvert(ctx context.Context) error {

legacyPatch(ext)

repo, _, err := loadRepository(ctx, ext.Module)
repo, tags, err := loadRepository(ctx, ext.Module)
if err != nil {
return err
}

tmp := *ext
tmp.Repo = repo
tmp.Versions = filterVersions(tags)

if ok, _ := lintExtension(tmp); ok {
reg = append(reg, ext)
Expand Down
2 changes: 0 additions & 2 deletions docs/example-api.txt
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
docs/example-api
├── catalog.json
├── openapi.yaml
├── registry.json
├── registry.schema.json
├── category
│   ├── authentication.json
│   ├── browser.json
Expand Down
2 changes: 1 addition & 1 deletion docs/example-api/catalog.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"name": "xk6-dashboard",
"owner": "grafana",
"public": true,
"stars": 327,
"stars": 328,
"timestamp": 1719907965,
"topics": [
"xk6",
Expand Down
2 changes: 1 addition & 1 deletion docs/example-api/category/observability.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"name": "xk6-dashboard",
"owner": "grafana",
"public": true,
"stars": 327,
"stars": 328,
"timestamp": 1719907965,
"topics": [
"xk6",
Expand Down
2 changes: 1 addition & 1 deletion docs/example-api/category/reporting.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"name": "xk6-dashboard",
"owner": "grafana",
"public": true,
"stars": 327,
"stars": 328,
"timestamp": 1719907965,
"topics": [
"xk6",
Expand Down
2 changes: 1 addition & 1 deletion docs/example-api/grade/C.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"name": "xk6-dashboard",
"owner": "grafana",
"public": true,
"stars": 327,
"stars": 328,
"timestamp": 1719907965,
"topics": [
"xk6",
Expand Down
2 changes: 1 addition & 1 deletion docs/example-api/grade/at-least/C.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"name": "xk6-dashboard",
"owner": "grafana",
"public": true,
"stars": 327,
"stars": 328,
"timestamp": 1719907965,
"topics": [
"xk6",
Expand Down
2 changes: 1 addition & 1 deletion docs/example-api/grade/at-least/D.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"name": "xk6-dashboard",
"owner": "grafana",
"public": true,
"stars": 327,
"stars": 328,
"timestamp": 1719907965,
"topics": [
"xk6",
Expand Down
2 changes: 1 addition & 1 deletion docs/example-api/grade/at-least/E.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"name": "xk6-dashboard",
"owner": "grafana",
"public": true,
"stars": 327,
"stars": 328,
"timestamp": 1719907965,
"topics": [
"xk6",
Expand Down
2 changes: 1 addition & 1 deletion docs/example-api/grade/at-least/F.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"name": "xk6-dashboard",
"owner": "grafana",
"public": true,
"stars": 327,
"stars": 328,
"timestamp": 1719907965,
"topics": [
"xk6",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
"name": "xk6-dashboard",
"owner": "grafana",
"public": true,
"stars": 327,
"stars": 328,
"timestamp": 1719907965,
"topics": [
"xk6",
Expand Down
Loading
Loading