Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Support for using multiple databases in datastore #1090

Merged
merged 75 commits into from
Sep 29, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
11524c1
A simple test with multidb support
danieljbruce Mar 20, 2023
e06bee0
A simple test with multidb support
danieljbruce Mar 20, 2023
1f464a5
Merge branch 'multi-db' of https://github.com/danieljbruce/nodejs-dat…
danieljbruce Mar 20, 2023
10da9c8
Add additional checks with get
danieljbruce Mar 20, 2023
ecfce7d
Add a test for saving with another database
danieljbruce Mar 20, 2023
3b6a42c
other datastore should delete the entity
danieljbruce Mar 20, 2023
e254be5
Add a test for the getDatastore method
danieljbruce Mar 20, 2023
77970c8
Add a unit test for database id
danieljbruce Mar 20, 2023
3b86ce2
Do an additional check for the sake of tests
danieljbruce Mar 20, 2023
ea54946
initialize generated client so it can be mocked
danieljbruce Mar 20, 2023
38d955c
comments and setDatabaseId
danieljbruce Mar 21, 2023
57bc2eb
Refactor of database name
danieljbruce Mar 21, 2023
c9856b2
correction to test against old database
danieljbruce Mar 21, 2023
543e486
getRequestWithDatabaseId function
danieljbruce Mar 22, 2023
dde1eba
Change function name to addDatabaseIdToRequest
danieljbruce Mar 22, 2023
8ba512a
Comments in all the places to add reqOpts
danieljbruce Mar 22, 2023
de946b2
Add a warning to a certain test for easier debug
danieljbruce Mar 22, 2023
dd7b06c
This fixed the bug for the save operation
danieljbruce Mar 22, 2023
f167ce3
datastore request id
danieljbruce Mar 22, 2023
364845b
addDatabaseIdToRequest
danieljbruce Mar 22, 2023
7b5f902
Solve all the test errors for createReadStream
danieljbruce Mar 22, 2023
a3bdb92
addDatabaseIdToRequest mock for runQueryStream
danieljbruce Mar 22, 2023
9d7fab1
By default, mock out addDatabaseIdToRequest
danieljbruce Mar 22, 2023
d9a96bf
Revert "addDatabaseIdToRequest mock for runQueryStream"
danieljbruce Mar 22, 2023
7543f57
Revert "Solve all the test errors for createReadStream"
danieljbruce Mar 22, 2023
3f3e965
linting
danieljbruce Mar 23, 2023
32bb92b
Revert "datastore request id"
danieljbruce Mar 23, 2023
92f8778
Revert "This fixed the bug for the save operation"
danieljbruce Mar 23, 2023
6159882
Revert "Comments in all the places to add reqOpts"
danieljbruce Mar 23, 2023
9410652
Introduce addDatabaseIdRequest in request
danieljbruce Mar 23, 2023
b34bf17
Declare a constant so that tests don’t run
danieljbruce Mar 23, 2023
2e1999a
Remove setDatabaseId
danieljbruce Mar 28, 2023
bd97d00
Add comment for addDatabaseIdToRequest function
danieljbruce Mar 28, 2023
d41ca60
Address header issue
danieljbruce Mar 28, 2023
1519ad3
Merge branch 'main' of https://github.com/googleapis/nodejs-datastore…
danieljbruce Mar 28, 2023
0ebabd5
Add databaseId for encoding and decoding keys
danieljbruce Mar 28, 2023
6449d10
Revert "Add databaseId for encoding and decoding keys"
danieljbruce Mar 29, 2023
e9ee522
Try a test with the namespace
danieljbruce Mar 29, 2023
1ba9756
Use the other datastore in the test
danieljbruce Mar 29, 2023
d35cdee
Create a test that looks at specific details
danieljbruce Mar 29, 2023
769417d
Add additional checks to existing test
danieljbruce Mar 29, 2023
e694341
Revert "Revert "Add databaseId for encoding and decoding keys""
danieljbruce Mar 29, 2023
f307f4d
Revert "Revert "Revert "Add databaseId for encoding and decoding keys"""
danieljbruce Mar 29, 2023
0f27ea4
Merge branch 'main' of https://github.com/googleapis/nodejs-datastore…
danieljbruce Mar 31, 2023
76a3dfc
Rename the second database to secondDatabase
danieljbruce May 10, 2023
3fd19fb
Use the database called multidb-test
danieljbruce May 19, 2023
d4e3f91
Merge branch 'main' into multi-db
danieljbruce May 31, 2023
771390c
Eliminate function on index that adds the id
danieljbruce Jun 9, 2023
ae15bc4
Fix the test so that the assertion error bubbles
danieljbruce Jun 9, 2023
6b5c542
Merge branch 'main' into multi-db
danieljbruce Jun 9, 2023
da64dab
Add types so that purpose of function is clear
danieljbruce Aug 8, 2023
8d170fd
Merge branch 'main' into multi-db
danieljbruce Aug 14, 2023
d49e5fc
Add types and names so that the function works
danieljbruce Aug 14, 2023
e7082ab
Merge branch 'multi-db' of https://github.com/danieljbruce/nodejs-dat…
danieljbruce Aug 14, 2023
b50b065
Eliminate repeated code fragments
danieljbruce Aug 15, 2023
f3aadc2
Better names and code organization
danieljbruce Aug 15, 2023
65605e8
Change the variable name of the default key
danieljbruce Aug 15, 2023
6986d4c
Refactor the post key hierarchy out
danieljbruce Aug 15, 2023
0378dea
Add only to the test we are interested in
danieljbruce Aug 18, 2023
a190512
Revert "Add only to the test we are interested in"
danieljbruce Aug 18, 2023
4b62317
Merge branch 'main' into multi-db
danieljbruce Aug 21, 2023
88e20ed
Merge branch 'main' of https://github.com/googleapis/nodejs-datastore…
danieljbruce Sep 20, 2023
739796f
Add second database id
danieljbruce Sep 20, 2023
87450ef
Remove unnecessary mocks
danieljbruce Sep 20, 2023
28b5aaa
Remove unused imports
danieljbruce Sep 20, 2023
46a5627
Add parameterized testing to the test/index file
danieljbruce Sep 20, 2023
2cf3d98
Add parameterized testing to test/transaction.ts
danieljbruce Sep 20, 2023
9505083
This ensures that the tests still run on the deft
danieljbruce Sep 20, 2023
e34735a
Add parameterized testing for the system tests
danieljbruce Sep 20, 2023
eb55764
Use default datastore variable in new tests
danieljbruce Sep 20, 2023
992117f
inline add database id
danieljbruce Sep 21, 2023
7312f49
Modify the namespace in parameterized tests
danieljbruce Sep 21, 2023
aef80a1
Move databaseId to SharedQueryOptions for alignme
danieljbruce Sep 21, 2023
72255ef
Run the linter
danieljbruce Sep 27, 2023
07dc746
Merge branch 'main' into multi-db
danieljbruce Sep 28, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@
"@types/node": "^20.4.9",
"@types/proxyquire": "^1.3.28",
"@types/sinon": "^10.0.0",
"async": "^3.2.4",
"c8": "^8.0.1",
"gapic-tools": "^0.2.0",
"gts": "^5.0.0",
Expand Down
12 changes: 12 additions & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -700,6 +700,16 @@ class Datastore extends DatastoreRequest {
);
}

/**
* Gets the database id that all requests will be run against.
*
* @returns {string} The database id that the current client is set to that
* requests will run against.
*/
getDatabaseId(): string | undefined {
return this.options.databaseId;
}

getProjectId(): Promise<string> {
return this.auth.getProjectId();
}
Expand Down Expand Up @@ -1818,6 +1828,7 @@ promisifyAll(Datastore, {
'double',
'isDouble',
'geoPoint',
'getDatabaseId',
'getProjectId',
'isGeoPoint',
'index',
Expand Down Expand Up @@ -1898,6 +1909,7 @@ export interface DatastoreOptions extends GoogleAuthOptions {
namespace?: string;
apiEndpoint?: string;
sslCreds?: ChannelCredentials;
databaseId?: string;
}

export interface KeyToLegacyUrlSafeCallback {
Expand Down
5 changes: 5 additions & 0 deletions src/request.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@
KeyProto,
ResponseResult,
Entities,
ValueProto,

Check warning on line 47 in src/request.ts

View workflow job for this annotation

GitHub Actions / lint

'ValueProto' is defined but never used
} from './entity';
import {
Query,
Expand Down Expand Up @@ -596,9 +596,9 @@
const results = res.batch.aggregationResults;
const finalResults = results
.map(
(aggregationResult: any) => aggregationResult.aggregateProperties

Check warning on line 599 in src/request.ts

View workflow job for this annotation

GitHub Actions / lint

Unexpected any. Specify a different type
)
.map((aggregateProperties: any) =>

Check warning on line 601 in src/request.ts

View workflow job for this annotation

GitHub Actions / lint

Unexpected any. Specify a different type
Object.fromEntries(
new Map(
Object.keys(aggregateProperties).map(key => [
Expand Down Expand Up @@ -933,7 +933,7 @@
callback?: SaveCallback
): void | Promise<CommitResponse> {
const transaction = this.datastore.transaction();
transaction.run(async (err: any) => {

Check warning on line 936 in src/request.ts

View workflow job for this annotation

GitHub Actions / lint

Unexpected any. Specify a different type
if (err) {
try {
await transaction.rollback();
Expand Down Expand Up @@ -992,6 +992,10 @@
}
}

if (datastore.options && datastore.options.databaseId) {
reqOpts.databaseId = datastore.options.databaseId;
}

if (method === 'rollback') {
reqOpts.transaction = this.id;
}
Expand Down Expand Up @@ -1143,6 +1147,7 @@
reqOpts?: RequestOptions;
}
export interface SharedQueryOptions {
databaseId?: string;
projectId?: string;
partitionId?: google.datastore.v1.IPartitionId | null;
readOptions?: {
Expand Down
Loading
Loading