Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: v3 api sync call #906

Merged
merged 25 commits into from
Aug 27, 2024
Merged
Show file tree
Hide file tree
Changes from 24 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
a9d84fb
wip: hack job
krpeacock Jul 18, 2024
0315037
v3 happy path functioning
krpeacock Jul 18, 2024
81ad181
fixing watermark / mitm vulnerability
krpeacock Jul 22, 2024
2264d3d
feat: handling for "accepted" but not replied sync call
krpeacock Jul 23, 2024
0b136f1
removing mainnet unit test
krpeacock Jul 23, 2024
2013775
wip: hack job
krpeacock Jul 18, 2024
39d53f0
v3 happy path functioning
krpeacock Jul 18, 2024
b5043fe
fixing watermark / mitm vulnerability
krpeacock Jul 22, 2024
e8ff517
feat: handling for "accepted" but not replied sync call
krpeacock Jul 23, 2024
9b2af1a
removing mainnet unit test
krpeacock Jul 23, 2024
49ad4fb
Merge branch 'kai/SDK-1448-sync-call' of github.com:dfinity/agent-js …
krpeacock Aug 12, 2024
f6a55e9
fix: automatic fallback to v2
krpeacock Aug 12, 2024
e7371af
making e2e tests compatible with dfx 0.22.0 when the agent falls back…
krpeacock Aug 13, 2024
b5523ed
lint: removes unused import
krpeacock Aug 13, 2024
0c0dfee
removes .only from test
krpeacock Aug 13, 2024
769953a
correct logic for usingV2 check
krpeacock Aug 13, 2024
26a71c7
pinning to greater than or equal to expected value for consistency
krpeacock Aug 13, 2024
0fffde9
removing logs
krpeacock Aug 13, 2024
dd3e4c9
changelog
krpeacock Aug 13, 2024
445b6a9
fix: corrects sync vs async log
krpeacock Aug 13, 2024
44b2447
fix: throw error in ActorMethod if agent.rootKey isn't initialized
krpeacock Aug 26, 2024
a123cb3
fix: remove `accepted` case from actor reply as it is not supported
krpeacock Aug 26, 2024
5242c89
Fall back to polling if we recieve an Accepted response code
krpeacock Aug 26, 2024
4b3268a
Merge branch 'main' into kai/SDK-1448-sync-call
krpeacock Aug 26, 2024
787241d
removes default case
krpeacock Aug 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@

### Added

- feat: sync_call support in HttpAgent and Actor
- Skips polling if the sync call succeeds and provides a certificate
- Falls back to v2 api if the v3 endpoint 404's
- Adds certificate to SubmitResponse endpoint
- adds callSync option to `HttpAgent.call`, which defaults to `true`
- feat: management canister interface updates for schnorr signatures
- feat: ensure that identity-secp256k1 seed phrase must produce a 64 byte seed
- docs: documentation and metadata for use-auth-client
Expand Down
48 changes: 36 additions & 12 deletions e2e/node/basic/counter.test.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import counterCanister, { createActor } from '../canisters/counter';
import { Actor, HttpAgent } from '@dfinity/agent';
import counterCanister, { idl } from '../canisters/counter';
import { it, expect, describe, vi } from 'vitest';

describe('counter', () => {
Expand Down Expand Up @@ -37,26 +38,49 @@ describe('counter', () => {
describe('retrytimes', () => {
it('should retry after a failure', async () => {
let count = 0;
const { canisterId } = await counterCanister();
const fetchMock = vi.fn(function (...args) {
if (count <= 1) {
count += 1;
count += 1;
// let the first 3 requests pass, then throw an error on the call
if (count === 3) {
return new Response('Test error - ignore', {
status: 500,
statusText: 'Internal Server Error',
});
}

// eslint-disable-next-line prefer-spread
return fetch.apply(
null,
args as [input: string | Request, init?: RequestInit | CMRequestInit | undefined],
);
return fetch.apply(null, args as [input: string | Request, init?: RequestInit | undefined]);
});

const counter = await createActor({ fetch: fetchMock as typeof fetch, retryTimes: 3 });
try {
expect(await counter.greet('counter')).toEqual('Hello, counter!');
} catch (error) {
console.error(error);
const counter = await Actor.createActor(idl, {
canisterId,
agent: await HttpAgent.create({
fetch: fetchMock as typeof fetch,
retryTimes: 3,
host: 'http://localhost:4943',
shouldFetchRootKey: true,
}),
});

const result = await counter.greet('counter');
expect(result).toEqual('Hello, counter!');

// The number of calls should be 4 or more, depending on whether the test environment is using v3 or v2
if (findV2inCalls(fetchMock.mock.calls as [string, Request][]) === -1) {
// TODO - pin to 4 once dfx v0.23.0 is released
expect(fetchMock.mock.calls.length).toBe(4);
} else {
expect(fetchMock.mock.calls.length).toBeGreaterThanOrEqual(4);
}
}, 40000);
});

const findV2inCalls = (calls: [string, Request][]) => {
for (let i = 0; i < calls.length; i++) {
if (calls[i][0].includes('v2')) {
return i;
}
}
return -1;
};
36 changes: 32 additions & 4 deletions e2e/node/basic/watermark.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -114,16 +114,44 @@ test('replay attack', async () => {
expect(startValue3).toBe(1n);

const queryResponseIndex = indexOfQueryResponse(fetchProxy.history);
console.log(queryResponseIndex);

fetchProxy.replayFromHistory(queryResponseIndex);

// the replayed request should throw an error
expect(fetchProxy.calls).toBe(7);
// The number of calls should be 4 or more, depending on whether the test environment is using v3 or v2
const usingV2 =
findV2inCalls(
fetchProxy.history.map(response => {
return [response.url];
}),
) !== -1;
if (usingV2) {
// TODO - pin to 5 once dfx v0.23.0 is released
// the replayed request should throw an error
expect(fetchProxy.calls).toBe(5);
} else {
expect(fetchProxy.calls).toBeGreaterThanOrEqual(5);
}

await expect(actor.read()).rejects.toThrowError(
'Timestamp failed to pass the watermark after retrying the configured 3 times. We cannot guarantee the integrity of the response since it could be a replay attack.',
);

// The agent should should have made 4 additional requests (3 retries + 1 original request)
expect(fetchProxy.calls).toBe(11);
// TODO - pin to 9 once dfx v0.23.0 is released
if (usingV2) {
// the replayed request should throw an error
// The agent should should have made 4 additional requests (3 retries + 1 original request)
expect(fetchProxy.calls).toBe(9);
} else {
expect(fetchProxy.calls).toBeGreaterThanOrEqual(9);
}
}, 10_000);

const findV2inCalls = (calls: [string][]) => {
for (let i = 0; i < calls.length; i++) {
if (calls[i][0].includes('v2')) {
return i;
}
}
return -1;
};
2 changes: 1 addition & 1 deletion e2e/node/canisters/counter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ let cache: {
actor: any;
} | null = null;

const idl = ({ IDL }) => {
export const idl = ({ IDL }) => {
return IDL.Service({
inc: IDL.Func([], [], []),
inc_read: IDL.Func([], [IDL.Nat], []),
Expand Down
4 changes: 2 additions & 2 deletions e2e/node/integration/actor.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,9 +14,9 @@ test("Legacy Agent interface should be accepted by Actor's createActor", async (
);

// Verify that update calls work
await actor.write(8n); //?
await actor.write(8n);
// Verify that query calls work
const count = await actor.read(); //?
const count = await actor.read();
expect(count).toBe(8n);
}, 15_000);
// TODO: tests for rejected, unknown time out
4 changes: 2 additions & 2 deletions packages/agent/src/actor.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import { CallRequest, SubmitRequestType, UnSigned } from './agent/http/types';
import * as cbor from './cbor';
import { requestIdOf } from './request_id';
import * as pollingImport from './polling';
import { ActorConfig } from './actor';
import { Actor, ActorConfig } from './actor';

const importActor = async (mockUpdatePolling?: () => void) => {
jest.dontMock('./polling');
Expand Down Expand Up @@ -329,7 +329,7 @@ describe('makeActor', () => {
`);
expect(replyUpdateWithHttpDetails.result).toEqual(canisterDecodedReturnValue);

replyUpdateWithHttpDetails.httpDetails['requestDetails']['nonce'] = new Uint8Array(); //?
replyUpdateWithHttpDetails.httpDetails['requestDetails']['nonce'] = new Uint8Array();

expect(replyUpdateWithHttpDetails.httpDetails).toMatchSnapshot();
});
Expand Down
58 changes: 37 additions & 21 deletions packages/agent/src/actor.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@ import {
SubmitResponse,
} from './agent';
import { AgentError } from './errors';
import { IDL } from '@dfinity/candid';
import { bufFromBufLike, IDL } from '@dfinity/candid';
import { pollForResponse, PollStrategyFactory, strategy } from './polling';
import { Principal } from '@dfinity/principal';
import { RequestId } from './request_id';
import { toHex } from './utils/buffer';
import { Certificate, CreateCertificateOptions } from './certificate';
import { Certificate, CreateCertificateOptions, lookupResultToBuffer } from './certificate';
import managementCanisterIdl from './canisters/management_idl';
import _SERVICE, { canister_install_mode, canister_settings } from './canisters/management_service';

Expand Down Expand Up @@ -525,35 +525,51 @@ function _createActorMethod(
const ecid = effectiveCanisterId !== undefined ? Principal.from(effectiveCanisterId) : cid;
const arg = IDL.encode(func.argTypes, args);

if (agent.rootKey == null)
throw new AgentError('Agent root key not initialized before making call');

const { requestId, response, requestDetails } = await agent.call(cid, {
methodName,
arg,
effectiveCanisterId: ecid,
});

requestId;
response;
requestDetails;

if (!response.ok || response.body /* IC-1462 */) {
throw new UpdateCallRejectedError(cid, methodName, requestId, response);
let reply: ArrayBuffer | undefined;
let certificate: Certificate | undefined;
if (response.body && response.body.certificate) {
const cert = response.body.certificate;
certificate = await Certificate.create({
krpeacock marked this conversation as resolved.
Show resolved Hide resolved
certificate: bufFromBufLike(cert),
rootKey: agent.rootKey,
canisterId: Principal.from(canisterId),
blsVerify,
});
const path = [new TextEncoder().encode('request_status'), requestId];
const status = new TextDecoder().decode(
lookupResultToBuffer(certificate.lookup([...path, 'status'])),
);

switch (status) {
case 'replied':
reply = lookupResultToBuffer(certificate.lookup([...path, 'reply']));
break;
case 'rejected':
throw new UpdateCallRejectedError(cid, methodName, requestId, response);
case 'default':
krpeacock marked this conversation as resolved.
Show resolved Hide resolved
throw new ActorCallError(cid, methodName, 'update', { requestId: toHex(requestId) });
}
}
// Fall back to polling if we recieve an Accepted response code
if (response.status === 202) {
const pollStrategy = pollingStrategyFactory();
// Contains the certificate and the reply from the boundary node
const response = await pollForResponse(agent, ecid, requestId, pollStrategy, blsVerify);
certificate = response.certificate;
reply = response.reply;
}

const pollStrategy = pollingStrategyFactory();
// Contains the certificate and the reply from the boundary node
const { certificate, reply } = await pollForResponse(
agent,
ecid,
requestId,
pollStrategy,
blsVerify,
);
reply;
const shouldIncludeHttpDetails = func.annotations.includes(ACTOR_METHOD_WITH_HTTP_DETAILS);
const shouldIncludeCertificate = func.annotations.includes(ACTOR_METHOD_WITH_CERTIFICATE);

const httpDetails = { ...response, requestDetails } as HttpDetailsResponse;

if (reply !== undefined) {
if (shouldIncludeHttpDetails && shouldIncludeCertificate) {
return {
Expand Down
2 changes: 2 additions & 0 deletions packages/agent/src/agent/api.ts
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,8 @@ export interface SubmitResponse {
error_code?: string;
reject_code: number;
reject_message: string;
// Available in a v3 call response
certificate?: ArrayBuffer;
} | null;
headers: HttpHeaderField[];
};
Expand Down
7 changes: 4 additions & 3 deletions packages/agent/src/agent/http/http.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import { Principal } from '@dfinity/principal';
import { requestIdOf } from '../../request_id';

import { JSDOM } from 'jsdom';
import { AnonymousIdentity, SignIdentity, toHex } from '../..';
import { Actor, AnonymousIdentity, SignIdentity, toHex } from '../..';
import { Ed25519KeyIdentity } from '@dfinity/identity';
import { AgentError } from '../../errors';
import { AgentHTTPResponseError } from './errors';
Expand Down Expand Up @@ -106,7 +106,7 @@ test('call', async () => {
expect(requestId).toEqual(expectedRequestId);
const call1 = calls[0][0];
const call2 = calls[0][1];
expect(call1).toBe(`http://127.0.0.1/api/v2/canister/${canisterId.toText()}/call`);
expect(call1).toBe(`http://127.0.0.1/api/v3/canister/${canisterId.toText()}/call`);
expect(call2.method).toEqual('POST');
expect(call2.body).toEqual(cbor.encode(expectedRequest));
expect(call2.headers['Content-Type']).toEqual('application/cbor');
Expand Down Expand Up @@ -320,7 +320,7 @@ test('use anonymous principal if unspecified', async () => {
expect(calls.length).toBe(1);
expect(requestId).toEqual(expectedRequestId);

expect(calls[0][0]).toBe(`http://127.0.0.1/api/v2/canister/${canisterId.toText()}/call`);
expect(calls[0][0]).toBe(`http://127.0.0.1/api/v3/canister/${canisterId.toText()}/call`);
const call2 = calls[0][1];
expect(call2.method).toEqual('POST');
expect(call2.body).toEqual(cbor.encode(expectedRequest));
Expand Down Expand Up @@ -812,3 +812,4 @@ test('it should log errors to console if the option is set', async () => {
const agent = new HttpAgent({ host: HTTP_AGENT_HOST, fetch: jest.fn(), logToConsole: true });
await agent.syncTime();
});

Loading
Loading