Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add SandboxBuilder image override #50

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions src/cmd/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,14 @@ pub enum CommandError {
#[error("invalid output of `docker inspect`: {0}")]
InvalidDockerInspectOutput(#[source] serde_json::Error),

/// The data received from the `docker manifest inspect` command is not valid.
#[error("invalid output from `docker manifest inspect`: {0}")]
InvalidDockerManifestInspectOutput(#[source] serde_json::Error),

/// The remote image is larger than the specified size limit.
#[error("sandbox image is too large: {0}")]
SandboxImageTooLarge(usize),

/// An I/O error occured while executing the command.
#[error(transparent)]
IO(#[from] std::io::Error),
Expand Down
58 changes: 55 additions & 3 deletions src/cmd/sandbox.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,22 @@ pub struct SandboxImage {
name: String,
}

#[derive(serde::Deserialize)]
struct DockerManifest {
config: DockerManifestConfig,
layers: Vec<DockerManifestLayer>,
}

#[derive(serde::Deserialize)]
struct DockerManifestConfig {
digest: String,
}

#[derive(serde::Deserialize)]
struct DockerManifestLayer {
size: usize,
}

impl SandboxImage {
/// Load a local image present in the host machine.
///
Expand All @@ -27,11 +43,33 @@ impl SandboxImage {
///
/// This will access the network to download the image from the registry. If pulling fails an
/// error will be returned instead.
pub fn remote(name: &str) -> Result<Self, CommandError> {
pub fn remote(name: &str, size_limit: Option<usize>) -> Result<Self, CommandError> {
let mut image = SandboxImage { name: name.into() };
let digest = if let Some(size_limit) = size_limit {
let out = Command::new_workspaceless("docker")
.args(&["manifest", "inspect", name])
.run_capture()?
.stdout_lines()
.join("\n");
let m: DockerManifest = serde_json::from_str(&out)
.map_err(CommandError::InvalidDockerManifestInspectOutput)?;
let size = m.layers.iter().fold(0, |acc, l| acc + l.size);
if size > size_limit {
return Err(CommandError::SandboxImageTooLarge(size));
}
Some(m.config.digest)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see any config or digest field with docker manifest inspect. What version of the docker CLI were you using?

> docker manifest inspect ros:latest | jq .config
null
> docker manifest inspect ros:latest
{
   "schemaVersion": 2,
   "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
   "manifests": [
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 3042,
         "digest": "sha256:ba0d9de623ad01a96d420140fed94cfb8de5e75be31cc33f0592d46e10dcccee",
         "platform": {
            "architecture": "amd64",
            "os": "linux"
         }
      },
      {
         "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
         "size": 3041,
         "digest": "sha256:42152a7c40961b39313cd0268a8d0aff483b56e817c784515a5b57cf2f2d1f28",
         "platform": {
            "architecture": "arm64",
            "os": "linux",
            "variant": "v8"
         }
      }
   ]
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

19.03.6

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, with that version I can't see the config part either.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

> docker --version
Docker version 19.03.8, build afacb8b7f0

So I'm worried this doesn't actually work.

} else {
None
};
info!("pulling image {} from Docker Hub", name);
Command::new_workspaceless("docker")
.args(&["pull", &name])
.args(&[
"pull",
&digest.map_or(name.to_string(), |digest| {
let name = name.split(':').next().unwrap();
format!("{}@{}", name, digest)
}),
])
.run()
.map_err(|e| CommandError::SandboxImagePullFailed(Box::new(e)))?;
if let Some(name_with_hash) = image.get_name_with_hash() {
Expand Down Expand Up @@ -146,6 +184,7 @@ pub struct SandboxBuilder {
user: Option<String>,
cmd: Vec<String>,
enable_networking: bool,
image: Option<String>,
}

impl SandboxBuilder {
Expand All @@ -160,6 +199,7 @@ impl SandboxBuilder {
user: None,
cmd: Vec::new(),
enable_networking: true,
image: None,
}
}

Expand Down Expand Up @@ -203,6 +243,14 @@ impl SandboxBuilder {
self
}

/// Override the image used for this sandbox.
///
/// By default rustwide will use the image configured with [`WorkspaceBuilder::sandbox_image`].
pub fn image(mut self, image: SandboxImage) -> Self {
self.image = Some(image.name);
self
}

pub(super) fn env<S1: Into<String>, S2: Into<String>>(mut self, key: S1, value: S2) -> Self {
self.env.push((key.into(), value.into()));
self
Expand Down Expand Up @@ -274,7 +322,11 @@ impl SandboxBuilder {
args.push("--isolation=process".into());
}

args.push(workspace.sandbox_image().name.clone());
if let Some(image) = self.image {
args.push(image);
} else {
args.push(workspace.sandbox_image().name.clone());
}

for arg in self.cmd {
args.push(arg);
Expand Down
3 changes: 2 additions & 1 deletion src/crates/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,8 @@ impl Crate {
}
}

pub(crate) fn copy_source_to(&self, workspace: &Workspace, dest: &Path) -> Result<(), Error> {
/// Copy this crate's source to `dest`. If `dest` already exists, it will be replaced.
pub fn copy_source_to(&self, workspace: &Workspace, dest: &Path) -> Result<(), Error> {
if dest.exists() {
info!(
"crate source directory {} already exists, cleaning it up",
Expand Down
2 changes: 1 addition & 1 deletion src/workspace.rs
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ impl WorkspaceBuilder {
let sandbox_image = if let Some(img) = self.sandbox_image {
img
} else {
SandboxImage::remote(DEFAULT_SANDBOX_IMAGE)?
SandboxImage::remote(DEFAULT_SANDBOX_IMAGE, None)?
};

let mut agent = attohttpc::Session::new();
Expand Down