Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release on crates.io? #61

Open
efoerster opened this issue May 4, 2020 · 30 comments
Open

Release on crates.io? #61

efoerster opened this issue May 4, 2020 · 30 comments

Comments

@efoerster
Copy link

efoerster commented May 4, 2020

Thanks for the great project. We are using citeproc-rs in texlab and would like to publish texlab on crates.io soon, see latex-lsp/texlab#152. However, this would require citeproc-rs to be available on crates.io, too. Do you have plans on making a release on crates.io?

@0-wiz-0
Copy link

0-wiz-0 commented Apr 6, 2021

I second this - please provide a crate on crates.io, to allow easier packaging for texlab.

@cormacrelf
Copy link
Collaborator

Yes, I plan to do this very soon. I was holding off because of the messiness of releasing internal crates as well. But a solution emerged when reading a rust_analyzer CI script, which is to add an extra crate name prefix during publishing so people can see very obviously that they’re internal.

@carlosala
Copy link

Hey! I also think would be great to do it, if you need some help to do it just say it!

@carlosala
Copy link

Hi @cormacrelf!
Are you finally planning to release it to crates.io?
It would help a lot to the texlab!
Thanks!

@bdarcus
Copy link

bdarcus commented Oct 16, 2021

I don't use texlab (though looks cool), but seems like them publishing their cargo is blocked by citeproc-rs not being available.

latex-lsp/texlab#399

Just out of curiosity, how does texlab use citeproc-rs?

@Antigravityd
Copy link

Any updates on this? Crates.io support for any library makes packaging on Guix trivial, and I'm really missing my latex completions...texlab appears to use this to parse BibTeX files to correctly complete citation names.

@0-wiz-0
Copy link

0-wiz-0 commented May 29, 2022

I just noticed that texlab stopped using this crate:
latex-lsp/texlab@bfcfff5

@bdarcus
Copy link

bdarcus commented May 29, 2022

This really needs to be a higher priority. It's now been more than two years since the OP opened this issue!

Other developers/projects aren't going to rely on, or contribute to, citeproc-rs if it's not available via crates.io.

@NilsIrl
Copy link

NilsIrl commented May 29, 2022

I just noticed that texlab stopped using this crate: latex-lsp/texlab@bfcfff5

And it is now published on crates.io

@kmaasrud
Copy link

kmaasrud commented Mar 5, 2023

@cormacrelf what is the current status? As soon as the 1.67 lifetime issue is fixed, wouldn't you consider this ready for a release on crates.io now?

@bdarcus
Copy link

bdarcus commented Mar 5, 2023

@kmaasrud - just curious, and I have no insight into the status, but are you wanting to use citeproc-rs with djoc?

https://github.com/kmaasrud/djoc

Cool to see that, BTW!

@kmaasrud
Copy link

kmaasrud commented Mar 5, 2023

@kmaasrud - just curious, and I have no insight into the status, but are you wanting to use citeproc-rs with djoc?

@bdarcus yes exactly! CSL is no doubt the future of citation processing, and while there exists a LaTeX package for it, it is written in Lua and thus not supported by Tectonic/XeTeX (which I'm using as my LaTeX backend.)

I also want the binary to be self-contained, which is not possible when using BibTeX (embedding biber would be a nightmare.)

@bdarcus
Copy link

bdarcus commented Mar 6, 2023

@kmaasrud cool; let me know if and when you want feedback.

@zepinglee
Copy link

... while there exists a LaTeX package for it, it is written in Lua and thus not supported by Tectonic/XeTeX (which I'm using as my LaTeX backend.)

@kmaasrud Technically speaking, it also works with XeTeX by replacing the bibtex (or biber) command-line procedure with citeproc-lua.

@kmaasrud
Copy link

kmaasrud commented Mar 7, 2023

@kmaasrud Technically speaking, it also works with XeTeX by replacing the bibtex (or biber) command-line procedure with citeproc-lua.

@zepinglee Indeed, but then the external dependency only shifts from biber over to citeproc-lua. Lua would definitely be easier to embed than biber though, so I might need to look into that.

However, given that such a comprehensive crate like this exists, it'd be a shame to depend on Lua...

@bdarcus
Copy link

bdarcus commented Mar 24, 2023

I ended up writing up a long comment related to this thread over here, a related Rust-based citation processor (with a crate!):

typst/hayagriva#32 (comment)

@urschrei
Copy link

I'll chime in here and note that I've been keeping an eye on citeproc-rs for a while, as both a long-time contributor to the Zotero ecosystem (I'm the author and maintainer of Pyzotero) and a long-time Rust user (I've been publishing and maintaining widely-used crates since Rust 1.0 in 2015).

I would be interested in contributing / helping to maintain the crate, but a couple of things are holding me back:

  1. As @bdarcus notes it's not really clear to me what needs to happen before someone is willing to cut a release;
  2. Its status as a component of Zotero, and the Zotero team's plans for it – I recall that there was a perf gap, which I found highly surprising, but I couldn't dig into it due to a compilation bug which I gather is now resolved.

In summary: I'm not interested in contributing to something that doesn't have a future as part of Zotero, but if it does, by all means let me know if you're interested in contributions (cc @cormacrelf @dstillman)

@dstillman
Copy link
Member

dstillman commented Mar 25, 2023

So I think this is a bit of a chicken-and-egg problem.

We don't currently have anyone able to work on citeproc-rs or even to assess patches — I could click the merge button on PRs but I wouldn't really know what I was accepting. Given the performance issues we saw and various other blockers (an incomplete list, I'm sure), and the relative completeness and suitability of citeproc-js, we're not particularly inclined to invest more in this project in order to determine if it could ever be a suitable replacement in Zotero. And without a clear future in Zotero, I'm not particularly inclined to publish it to crates.io under our name, nor do I think it would really make sense for anyone to do so without the project being under active development. As far as I know citeproc-rs won't currently even parse many/most current CSL styles, since it doesn't have full CSL 1.0.2 support, so it's likely of pretty limited use to anyone at this point.

If someone from the community was willing to help address the remaining issues and get citeproc-rs to the point where it was clear that there was a path to using it as the default processor in Zotero, we would be open to putting more resources towards its continued development as well as its health as an open-source project. But I don't know if anyone would be willing to do that without a stronger guarantee that it was going to end up in Zotero, and that's just not something we can promise at this point.

So I'm not sure how we get past that. @urschrei's offer to contribute is much appreciated, but I'm not clear if that's just about the crate or the processor more generally, and the latter is obviously a much bigger lift. There's no shortage of love for Rust, but I'm not sure that translates into love for CSL processor development.

As it is, other than the occasional issue (some of which we could work around in Zotero if we had to), citeproc-js mostly just does what we need, even on platforms like iOS where we need to call out to JS.

Sorry I don't have a better answer here.

@urschrei
Copy link

Thanks Dan, that's helpful. Maybe a sensible approach here is a strictly time-bounded attempt on my part to get to grips with the codebase and see whether full CSL 1.0.2 support is possible without a significant time investment, since there seems to be little point in proceeding otherwise.

@bdarcus
Copy link

bdarcus commented Mar 25, 2023 via email

@bdarcus
Copy link

bdarcus commented Mar 25, 2023

Maybe a sensible approach here is a strictly time-bounded attempt on my part to get to grips with the codebase and see whether full CSL 1.0.2 support is possible without a significant time investment ...

On this, I should emphasize that the changes in that release are trivial; mostly things like new variable names.

So if there's a problem doing this:

  • I'd be really surprised!
  • if it were the case, that would indeed signal some fundamental problem

I am guessing the bigger issue is the performance issues Dan mentioned (and which also seems surprising).

@urschrei it might be worth looking at the even newer Haskell citeproc, if you are able to identify any processing bottlenecks and looking for ideas? That was a clean rewrite of an earlier implementation, and supposedly significantly faster.

@dstillman
Copy link
Member

dstillman commented Mar 26, 2023

Actually supporting 1.0.2 terms should be trivial. The larger issue there is citeproc-rs currently not accepting unknown input, which isn't really appropriate for our use case for a number of reasons.

For performance, there are two concerns: pathological cases, which hopefully can be easily addressed with some optimization, and more fundamental problems with WebAssembly performance, which wouldn't reflect on the processor itself but would affect the Zotero desktop use case (though we might be able to just run it as a separate binary). We need to evaluate the WebAssembly performance in the Zotero 7 dev build, but it will be easier for us and others to do that once citeproc-rs can parse 1.0.2 styles.

@bdarcus
Copy link

bdarcus commented Mar 27, 2023

Would #13 be a potential alternative to webassembly, assuming the other details can be sorted out?

Would be cool if such a thing were compatible with the jsons served by https://github.com/jgm/citeproc/blob/master/man/citeproc.1.md; a standard JSON citation/bibliography API (see OpenAPI).

@bdarcus

This comment was marked as outdated.

@bdarcus

This comment was marked as outdated.

@bdarcus
Copy link

bdarcus commented Jun 2, 2023

Another update, very obviously rust-related.

https://github.com/bdarcus/csln

It's a reimplementation of the csl-next draft typescript model in pure Rust, with very tight coupling (thanks to serde) between the JSON schema input and internal model.

I'm pretty confident in that model, though it would need more review, testing, and iteration for me to be fully happy with it.

I'm much less confident in my programming skills, and the fact I'm a complete Rust newbie.

But I'm absolutely serious about building this out. I just need some help.

It should compile fine using the cargo, and I have it licensed under the same terms as citeproc-rs.

It's not quite pare with the typescript processor; here's an example of where I'm at:

target/debug/csln processor/examples/style.csl.yaml processor/examples/ex1.bib.yaml

Example result:

{
  "smith1": {
    "disamb-condition": false,
    "group-index": 1,
    "group-length": 1,
    "group-key": "Smith, Sam:2023-10"
  },

So the core of the processor at this point is a sorted bibliography vector, and this HashMap.

The next step is a function to iterate through the former and template and use the latter to generate the pre-rendered AST.

@bdarcus
Copy link

bdarcus commented Jun 2, 2023

PS - just learned the typst folks are working on a 1.0 processor in Rust.

https://github.com/typst/citationberg

Feels like maybe there needs to be some collaboration across these projects.

@bdarcus
Copy link

bdarcus commented Jun 3, 2023

Now that I've learned a bit of Rust so I can better understand this code base, I do think something like what I'm doing in csln and this could be aligned.

The idea is really a new input model, where the schemas are generated from the code, and so they are tightly-aligned.

Oh and removing a lot of unnecessary logic from the template language.

But it seems to me the processing model is generally pretty sound, with lots of performance optimizations. Not sure from my brief review what the bottlenecks could be, but I suspect they're resolvable.

@bdarcus
Copy link

bdarcus commented Jun 7, 2023

I've forked the repo over at the CSL org, and applied Cormac's 1.67 branch, so at least it (partially) compiles :-)

https://github.com/citation-style-language/citeproc-rs

But there are 125 clippy warnings, and the citeproc-io doesn't build, and I don't myself, with my newbie skills, know how to fix it all.

I still think it'd likely be easier and more future-proof to merge what I'm doing with some of what's in this code base.

@bdarcus
Copy link

bdarcus commented Oct 23, 2023

The typst folks are just about to merge CSL 1.0 support in their Hayagriva library.

typst/hayagriva#66

It relies on their parser library:

https://github.com/typst/citationberg

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests