[WIP] Mini-batch solvers from StochOpt.jl

[tbng]

Objective: adding solver implemented in julia from https://github.com/gowerrobert/StochOpt.jl . Based on Nidham Gazagnadou, Robert M. Gower, Joseph Salmon (2019). Optimal mini-batch and step sizes for SAGA.

TODO list:

• Write a julia solver that calls and returns the fitted weights (solvers/julia_saga_batch.jl)
• Write a python wrapper for the file, including installing the required packages, then run the wrapper function ( solvers/julia_saga_batch.py)
• Benchmarks this solver with multiple other solvers (sklearn l-bfgs + SGD + SAGA, SVRG, etc.) with different datasets from libsvm)

[tbng]

Maybe also pinging @ngazagna

[tomMoral]

some comments.

Also, could you share the result you got? I am curious about what it look like for now.

[tbng]

Subopt curves are definitely not sexy yet:

[tbng]

Some plots on 2 reps simulated data, the delay time in the beginning for StochOpt solvers is probably due to julia wrapper initializing the function