Support streaming

[Twey]

It seems that a file is missing from the capnpc-rust distribution to support the streaming attribute:

PS> capnp compile -orust --src-prefix=schema\\ schema\point.capnp
schema\\point.capnp:8:35-41: error: A method declaration uses streaming, but '/capnp/stream.capnp' is not found in the import path. This is a standard file that should always be installed with the Cap'n Proto compiler.

point.capnp:

@0xdcaf678ef37b50f2;

interface MyInterface {
  streamingCall @0 (callback :Callback) -> ();

  interface Callback {
    # NEW: This streaming call features flow control!
    sendChunk @0 (chunk :Data) -> stream;
    done @1 ();
  }
}

[dwrensha]

What happens if you do this?

capnp compile -orust --src-prefix=schema\\ schema\point.capnp

I expect that you'll get the same error. I think this means that your base capnproto-c++ installation is broken somehow.

How did you install the capnp executable?

Note, in any case, that capnproto-rust provides no special streaming support yet. You should be able to use methods that have the stream attribute, but the attribute will be ignored.

[dwrensha]

EDIT I meant to ask: what happens if you do this?

capnp compile -ocapnp --src-prefix=schema\\ schema\point.capnp

[Twey]

Yes, it produces the same error.
This Cap'n P was installed using the official Windows instructions for Cap'n Proto installation, not including the C++ bit (because, as the instructions say, if I don't want to use C++ I don't need to). I think the implication is that each language's implementation is responsible for providing these files? I see the stream.capnp in the official repo seems to include a C++-specific file.
My understanding is that streaming doesn't require any particular language support, and the purpose of the stream.capnp is to translate uses of streaming into normal Cap'n Proto that doesn't require special handling?

[dwrensha]

If you get the same error when doing capnp compile -ocapnp, then that means the problem is happening before anything having to do with Rust is involved.

I think probably the best solution will be for you to additionally copy the stream.capnp file (and maybe c++.capnp too) into your local installation, and make sure that capnp compile can find it (possibly through the -I flag). And probably the official Windows instructions should be updated to mention this.

[dwrensha]

I opened capnproto/capnproto#1288 upstream.

[dwrensha]

Upstream documentation was fixed in capnproto/capnproto@ac29766

[tamird]

I'm seeing a related issue. How is one supposed to specify the location of so-called "well known" capnp files? Compiling my schema works:

$ capnpc  -ocapnp schema/ex.capnp  | grep stream
      sendChunk @0 (chunk :Data) -> stream;

but code generation fails here:

capnproto-rust/capnpc/src/lib.rs

Lines 352 to 358 in c702d16

	::capnp::Error::failed(format!(
	"Unable to stat capnp input file `{}` in working directory {}: {}. \
	Please check that the file exists and is accessible for read.",
	file.display(),
	current_dir,
	error
	))

my build.rs:

fn main() {
    capnpc::CompilerCommand::new()
        .src_prefix("schema")
        .file("schema/schema.capnp")
        .file("capnp/stream.capnp")
        .run()
        .expect("schema compiler command");
}

[dwrensha]

@tamird possibly you need to specify an import path:

capnproto-rust/capnpc/src/lib.rs

Lines 153 to 161 in c702d16

	/// Adds an --import_path flag. Adds `dir` to the list of directories searched
	/// for absolute imports.
	pub fn import_path<P>(&mut self, dir: P) -> &mut Self
	where
	P: AsRef<Path>,
	{
	self.import_paths.push(dir.as_ref().to_path_buf());
	self
	}

(I don't expect this to be necessary if the capnp c++ library and headers are installed in a standard location, but maybe yours aren't?)

[tamird]

@dwrensha I'm not sure how that would help. The failure happens in the rust code generator library before capnpc is invoked. How would an import path help?

[dwrensha]

Ah, yes, that will not help. I opened a new issue: #430

I think probably, for core schemas that come with the C++ distribution, we should include their generated Rust code in the base capnp crate.

Until then, you need to generate code for stream.capnp yourself. Because this is not just an import -- it's actually full code generation -- you'll need to provide the full path in the .file() call. You might find it convenient to copy stream.capnp into your own project's repo.