Skip to content

Commit

Permalink
Orion v3.0.3 (#224)
Browse files Browse the repository at this point in the history
* Remove n+1 problem for StorageDataObject (#209)

* Remove n+1 problem for StorageDataObject

* Inform user about incorrect query

* fix: ✨ add global counter migratino to account for 303 (#217)

* Orion 303 final touches (#223)

* feat: ✨ add CORS origins for atlas local testing

* chore: 📝 adds 303 release notes

* Fix typos (#176)

* fix typos

* fix typos

* fix typos

* fix typos

* fix typos

* fix typo

* fix typos

* fix typos

* fix typos

* fix typos

* fix typos

* fix typos

---------

Co-authored-by: WRadoslaw <[email protected]>
Co-authored-by: omahs <[email protected]>
  • Loading branch information
3 people authored Nov 3, 2023
1 parent c1a3ac8 commit 5305427
Show file tree
Hide file tree
Showing 18 changed files with 82 additions and 67 deletions.
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@
# 3.0.3
### Optimisation:
- Solves n+1 query issue on the `AssetResolver` `resolvedUrls` field resolver, by reusing `storageBagId` field if available. Resulting in faster query times and inpacting home page loading speed for front end application significantly. This fix involves only the `orion_graphql-server` microservice
### Misc
- adds `http://localhost:3000` and `http://127.0.0.1:3000` as authorized CORS origins for local front end testing
# 3.0.2
### Bug Fixes:
- Store membership handles both as utf-8 string and raw bytes - [#4950](https://github.com/Joystream/joystream/pull/4950)
Expand Down
6 changes: 3 additions & 3 deletions docs/developer-guide/architecture-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,10 @@ Most of the services are part of the [Subsquid framework](https://subsquid.io/),
- [Polkadot JS](https://polkadot.js.org/docs/) (a set of libraries providing common Substrate utilities)
- [TypeORM](https://typeorm.io/#/) (ORM used for interacting with the PostgreSQL database)
- [PostgreSQL](https://www.postgresql.org/docs/14/index.html) (database management system)
- [TypeGraphQL](https://typegraphql.com/docs/introduction.html) (TypeScript library used for extening Subsquid's GraphQL API with [custom resolvers](https://docs.subsquid.io/develop-a-squid/graphql-api/custom-resolvers/))
- [TypeGraphQL](https://typegraphql.com/docs/introduction.html) (TypeScript library used for extending Subsquid's GraphQL API with [custom resolvers](https://docs.subsquid.io/develop-a-squid/graphql-api/custom-resolvers/))
- [Apollo Server](https://www.apollographql.com/docs/apollo-server/) (GraphQL server library used by Subquid)
- [Express.js](https://expressjs.com/en/4x/api.html) (http server library used by the [auth server](./tutorials/authentication-api.md))
- [OpenAPI](https://swagger.io/specification/) (API speification standard used for describing the auth server API)
- [OpenAPI](https://swagger.io/specification/) (API specification standard used for describing the auth server API)
- [Express OpenAPI validator](https://github.com/cdimascio/express-openapi-validator/wiki/Documentation) (express middleware used by the auth server)
- [Handlebars](https://handlebarsjs.com/guide/) (e-mail templating engine)

Expand Down Expand Up @@ -80,4 +80,4 @@ The services can be divided into 2 main categories:
- `2000000000000-Views.js` - additional migration responsible for splitting the database into a `public` and `admin` schema, where `public` schema only exposes the data that is publically available (through the use of SQL views), while `admin` schema contains the original tables which include private data. For more details see _[Managing entity visibility](./tutorials/entity-visibility.md)_ guide.
- `2100000000000-Indexes.js` - additional PostgreSQL indexes (mostly expression indexes on `jsonb` fields, as those are not supported natively by Subsquid).
- `2200000000000-Operator.js` - a migration which inserts the initial operator (root) user based on the environment config. For more details, check the _[Authentication API](./tutorials/authentication-api.md)_ guide.
- `/postgres.conf` - contains PostgreSQL database service configuration.
- `/postgres.conf` - contains PostgreSQL database service configuration.
6 changes: 3 additions & 3 deletions docs/developer-guide/code-style.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,13 @@ We're using [ESLint](https://eslint.org/) and [Prettier](https://prettier.io/) t
We rely on [GitHub workflows](https://docs.github.com/en/actions/using-workflows/about-workflows) like [this one](../../.github/workflows/checks.yml) to make sure the code is linted and formatted according to the ESLint and Prettier rules when a new pull request is opened.

To make sure your code is always properly formatted and linted you can:
- Run `npm run lint --fix && npm run format && git add .` before comitting your changes and pushing them to GitHub.
- Run `npm run lint --fix && npm run format && git add .` before committing your changes and pushing them to GitHub.
- Add an extension to your IDE which will automatically lint and format your code upon saving a file (see: [VSCode extension](#vscode-extension)).
- Run `npm run checks` to make sure your pull request will pass the linting & formatting check.

## VSCode extension

If you're using VSCode as your IDE, you can install [ESLint](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint) extension and configure it to automatically fromat and lint the code for you each time you save a file inside the Orion workspace.
If you're using VSCode as your IDE, you can install [ESLint](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint) extension and configure it to automatically format and lint the code for you each time you save a file inside the Orion workspace.

Once you install the extension you can add following config to the `.vscode/settings.json` file inside the Orion workspace:
```json
Expand All @@ -20,4 +20,4 @@ Once you install the extension you can add following config to the `.vscode/sett
"source.fixAll.eslint": true
}
}
```
```
6 changes: 3 additions & 3 deletions docs/developer-guide/running-local-instance.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ If you need a full local Joystream playground setup which includes:
- [Storage node](https://github.com/Joystream/joystream/tree/master/storage-node)
- [Distributor node](https://github.com/Joystream/joystream/tree/master/distributor-node)

You can run it in a following way:
You can run it in the following way:

```bash
# Clone the Orion repository (if not already done)
Expand Down Expand Up @@ -50,7 +50,7 @@ yarn start

### Local Orion endpoints

After executing those steps you will be able to access the the local instance of Orion at:
After executing those steps you will be able to access the local instance of Orion at:
- http://localhost:4074/playground ([Auth API](./tutorials/authentication-api.md) playground)
- http://localhost:4350/graphql (GraphQL API playground)

Expand Down Expand Up @@ -140,4 +140,4 @@ docker logs --tail 10 -f orion_processor

Again, the local Orion API endpoints are going to be the same as those described in the first section ([Local Orion endpoints](#local-orion-endpoints))

After you're done with the tests you can use either `make down` or `docker-compose down -v` to bring down all Orion services.
After you're done with the tests you can use either `make down` or `docker-compose down -v` to bring down all Orion services.
12 changes: 6 additions & 6 deletions docs/developer-guide/tutorials/adding-custom-queries.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,7 +130,7 @@ Here's a step by step process of how you could do that:
days!: number
}
```
In this specific case the `ChannelWhereInput` already exists in `src/server-extension/resolvers/baseTypes.ts` and it's definition looks like this:
In this specific case the `ChannelWhereInput` already exists in `src/server-extension/resolvers/baseTypes.ts` and its definition looks like this:
```typescript
export const ChannelWhereInput = new GraphQLScalarType({
name: 'ChannelWhereInput',
Expand All @@ -151,7 +151,7 @@ Here's a step by step process of how you could do that:
}
```
Just like in case of `ChannelWhereInput`, the `Channel` [type placeholder](#placeholder-types) in also already defined inside `src/server-extension/resolvers/baseTypes.ts`.
3. Now you can add a new query to the already existing `ChannelsResolver`. The detailed explaination of how it's done is provided in the comments insde the code snippet below:
3. Now you can add a new query to the already existing `ChannelsResolver`. The detailed explanation of how it's done is provided in the comments inside the code snippet below:
```typescript
// src/server-extension/resolvers/ChannelsResolver/index.ts
import 'reflect-metadata'
Expand All @@ -175,7 +175,7 @@ Here's a step by step process of how you could do that:
@Ctx() ctx: Context
): Promise<RecentlyActiveChannelsResult[]> {
// Parse the GraphQL resolve tree (which describes the shape of the result)
// using an untility exposed by `@subsquid/openreader`
// using a utility exposed by `@subsquid/openreader`
const tree = getResolveTree(info)

// Parse `where` and `limit` args of the query using Subsquid's `parseSqlArguments` utility
Expand Down Expand Up @@ -278,7 +278,7 @@ Here's a step by step process of how you could do that:
}
}
```
4. To test this new query, you can follow a process similar to the one descibed in [Testing new queries](#testing-new-queries) section. For example, you can try executing following queries:
4. To test this new query, you can follow a process similar to the one described in [Testing new queries](#testing-new-queries) section. For example, you can try executing following queries:
```graphql
query Top10VideoUploaders30Days {
recentlyActiveChannels(days: 30, limit: 10) {
Expand Down Expand Up @@ -324,6 +324,6 @@ If you're interested in the details of how Subsquid generates the entity queries

Behind the scenes, Subsquid uses `mergeSchemas` method from `@graphql-tools/schema` (see: _[Schema Merging](https://the-guild.dev/graphql/tools/docs/schema-merging)_) to merge the schema built from the custom TypeGraphQL resolvers (`src/server-extension/resolvers`) with the schema autogenerated by `@subsquid/openreader` from the [input schema](https://docs.subsquid.io/basics/schema-file/). You can see this process [here](https://github.com/subsquid/squid-sdk/blob/%40subsquid/graphql-server_v3.2.4/graphql-server/src/server.ts#L118).

In the original Subsquid implementation, the schema built from the custom resolvers is merged on top of the autogenerated schema, which means that the types defined in the custom resolvers schema take priority. In the Orion implementation however, we use a [patched version of `@subsquid/graphql-server`](../../../assets/patches/@subsquid+graphql-server+3.3.2.patch) which does the merge the other way around. This means that even if we define a type like `ChannelWhereInput` as a simple scalar in our custom resolvers, it will be overriden by the `ChannelWhereInput` type coming from the schema autogenerated by `@subsquid/openreader`.
In the original Subsquid implementation, the schema built from the custom resolvers is merged on top of the autogenerated schema, which means that the types defined in the custom resolvers schema take priority. In the Orion implementation however, we use a [patched version of `@subsquid/graphql-server`](../../../assets/patches/@subsquid+graphql-server+3.3.2.patch) which does the merge the other way around. This means that even if we define a type like `ChannelWhereInput` as a simple scalar in our custom resolvers, it will be overridden by the `ChannelWhereInput` type coming from the schema autogenerated by `@subsquid/openreader`.

Note however, that the definitions of 2 object types with the same name are merged together (not overriden), so if we create a _placeholder type_ with the same name as one of the autogenerated types, but with a property that doesn't exist in the original, autogenerated type, the autogenerated type will not completely override our placeholder type, but instead our new property will be added to the autogenerated type.
Note however, that the definitions of 2 object types with the same name are merged together (not overridden), so if we create a _placeholder type_ with the same name as one of the autogenerated types, but with a property that doesn't exist in the original, autogenerated type, the autogenerated type will not completely override our placeholder type, but instead our new property will be added to the autogenerated type.
8 changes: 4 additions & 4 deletions docs/developer-guide/tutorials/adding-mutations.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
## Adding mutations

Adding new mutations is generally similar to [adding custom queries](./adding-custom-queries.md).
The baisc idea is that you add new method to a [custom resolver](https://docs.subsquid.io/graphql-api/custom-resolvers/), decorated with `@Mutation` [TypeGraphQL](https://typegraphql.com/) decorator.
The basic idea is that you add new method to a [custom resolver](https://docs.subsquid.io/graphql-api/custom-resolvers/), decorated with `@Mutation` [TypeGraphQL](https://typegraphql.com/) decorator.

However, usually when thinking about implementing mutations we need to consider the execution context. Most of the mutations will require some specific permissions, for example, they will only be available for the gateway operator or useres with registered accounts.
However, usually when thinking about implementing mutations we need to consider the execution context. Most of the mutations will require some specific permissions, for example, they will only be available for the gateway operator or users with registered accounts.

Another important thing to keep in mind when implementing mutations is the [visibility of the entities](./entity-visibility.md).

Expand All @@ -25,7 +25,7 @@ import { AccountOnly } from '../middleware'

@Resolver()
export class VideosResolver {
// Set by depenency injection
// Set by dependency injection
constructor(private em: () => Promise<EntityManager>) {}
// ...
// Adding this middleware will prevent anyone who doesn't have an account from accessing
Expand Down Expand Up @@ -54,4 +54,4 @@ export class VideosResolver {
})
}
}
```
```
4 changes: 2 additions & 2 deletions docs/developer-guide/tutorials/adding-new-event-handlers.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@ channelById.title = 'New title'
overlay.getRepository(Channel).remove(channelById)
```

It is imporant to understand that although the `overlay` will always provide access to the latest state, you may recieve stale state if you try to query the database directly:
It is important to understand that although the `overlay` will always provide access to the latest state, you may receive stale state if you try to query the database directly:

```typescript
overlay.getRepository(Channel).new({
Expand All @@ -107,4 +107,4 @@ But this should generally be avoided in event handlers, as it may negate the per

### `MAX_CACHED_ENTITIES` environment variable

You can limit the number of entities that the overlay will store in memory by setting the `MAX_CACHED_ENTITIES` environment variable. Increasing this value too much may cause the processor to run out of memory or result in SQL errors due to queries being too large.
You can limit the number of entities that the overlay will store in memory by setting the `MAX_CACHED_ENTITIES` environment variable. Increasing this value too much may cause the processor to run out of memory or result in SQL errors due to queries being too large.
Loading

0 comments on commit 5305427

Please sign in to comment.