Ollama-compatible AI-assisted code generation

[haesleinhuepf]

This enables ollama support technically. Unfortunately, I'm not aware of LLMs which work well. @pr4deepr suggested there are some. If he or anyone else can show us useful ollama-compatible open-weight LLMs , we might merge this too.

If anyone wants to try this is the compiled jar:
https://cloud.scadsai.uni-leipzig.de/index.php/s/a2bSXY7n2JfGjKn

[pr4deepr]

Hi @haesleinhuepf

I don't think the models are going to perform as well as the commercial models 'yet'. However, the models are improving and having this option enables accessibility as paying for subscriptions is not possible for everyone.

I've been trying to run ollama within Fiji, but I keep getting an error.
I know I can access ollama via the url: http://127.0.0.1:11434/v1 because I can access it from python using your notebook from your preprint.

The error I'm getting:

sun.net.www.protocol.http.HttpURLConnection:http://127.0.0.1:11434/v1
{"messages":[{"role":"user","content":"You are an extremely talented Bio-image Analyst and programmer.\nYou write code in ImageJ Macro.\nWrite concise and high quality code for ImageJ/Fiji.\nPut minimal comments explaining what the code does.\nYour task is the following:\nWrite code to open images in Fiji"}],"model":"mistral-nemo"}
Exception in thread "AWT-EventQueue-0" java.lang.RuntimeException: java.io.FileNotFoundException: http://127.0.0.1:11434/v1
at org.scijava.ui.swing.script.TextEditor.lambda$askLLMServiceProviderToGenerateCode$51(TextEditor.java:3284)
at java.awt.event.InvocationEvent.dispatch(InvocationEvent.java:311)
at java.awt.EventQueue.dispatchEventImpl(EventQueue.java:758)
at java.awt.EventQueue.access$500(EventQueue.java:97)
at java.awt.EventQueue$3.run(EventQueue.java:709)
at java.awt.EventQueue$3.run(EventQueue.java:703)
at java.security.AccessController.doPrivileged(Native Method)
at java.security.ProtectionDomain$JavaSecurityAccessImpl.doIntersectionPrivilege(ProtectionDomain.java:74)
at java.awt.EventQueue.dispatchEvent(EventQueue.java:728)
at java.awt.EventDispatchThread.pumpOneEventForFilters(EventDispatchThread.java:205)
at java.awt.EventDispatchThread.pumpEventsForFilter(EventDispatchThread.java:116)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(EventDispatchThread.java:105)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:101)
at java.awt.EventDispatchThread.pumpEvents(EventDispatchThread.java:93)
at java.awt.EventDispatchThread.run(EventDispatchThread.java:82)
Caused by: java.io.FileNotFoundException: http://127.0.0.1:11434/v1
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1913)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1505)
at org.scijava.ui.swing.script.OpenAIClient.prompt(OpenAIClient.java:57)
at org.scijava.ui.swing.script.TextEditor.lambda$askLLMServiceProviderToGenerateCode$51(TextEditor.java:3277)
... 14 more

I tried using localhost as well in the url.

Cheers
Pradeep

[haesleinhuepf]

Hey @pr4deepr ,

awesome, thanks for the feedback!

Which model are you using and which operating system?

[pr4deepr]

ahh, sorry..
I'm using Windows 10. I tried mistral-nemo and deepseek-coder-v2.

[haesleinhuepf]

I may know the issue. Would you mind trying this Ollama URL?

http://localhost:11434/v1/chat/completions

I will update the default value for this if it works for you.

[pr4deepr]

Can confirm its working.