Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature request: Cross-entropy error function #175

Open
tvedebrink opened this issue Jun 23, 2022 · 0 comments
Open

Feature request: Cross-entropy error function #175

tvedebrink opened this issue Jun 23, 2022 · 0 comments

Comments

@tvedebrink
Copy link

In nn.ts it seems as only quadratic loss is supported in the playground? The definition of the Error class seems to indicate that one could implement the cross-entropy error function rather easy. With target being $t$ and $o$ for output The cross-entropy is given as
$$f(o,t) = t\log(o) + (1-t)\log(1-o),$$
where the derivative with respect to $o$ is
$$f'(o,t) = \frac{\partial}{\partial o} t\log(o) + (1-t)\log(1-o) = \frac{t}{o} - \frac{1-t}{1-o} = \frac{t-o}{o(1-o)}$$
In this case, is my suggestion below correct?

public static CROSSENTROPY: ErrorFunction = {
    error: (output: number, target: number) =>
               target*log(output) + (1-target)*log(1-output),
    der: (output: number, target: number) => (target - output)/(output*(1-output))
  };

In the playground.ts script I would then change all references to SQUARE to CROSSENTROPY or make it possible to select the error function by drop-down menu?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant