Skip to content

A framework for out-of-core and parallel execution

License

Notifications You must be signed in to change notification settings

davidizzle/Dagger.jl

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Dagger.jl


A framework for out-of-core and parallel computing

Documentation Build Status
Build Status

At the core of Dagger.jl is a scheduler heavily inspired by Dask. It can run computations represented as directed-acyclic-graphs (DAGs) efficiently on many Julia worker processes and threads, as well as GPUs via DaggerGPU.jl.

The DTable has been moved out of this repository. You can now find it here.

Installation

Dagger.jl can be installed using the Julia package manager. Enter the Pkg REPL mode by typing "]" in the Julia REPL and then run:

pkg> add Dagger

Or, equivalently, install Dagger via the Pkg API:

julia> import Pkg; Pkg.add("Dagger")

Usage

Once installed, the Dagger package can be loaded with using Dagger, or if you want to use Dagger for distributed computing, it can be loaded as:

using Distributed; addprocs() # Add one Julia worker per CPU core
using Dagger

You can run the following example to see how Dagger exposes easy parallelism:

# This runs first:
a = Dagger.@spawn rand(100, 100)

# These run in parallel:
b = Dagger.@spawn sum(a)
c = Dagger.@spawn prod(a)

# Finally, this runs:
wait(Dagger.@spawn println("b: ", b, ", c: ", c))

Use Cases

Dagger can support a variety of use cases that benefit from easy, automatic parallelism, such as:

This isn't an exhaustive list of the use cases that Dagger supports. There are more examples in the docs, and more use cases examples are welcome (just file an issue or PR).

Contributing Guide

PRs Welcome GitHub issues GitHub contributors

Contributions are encouraged.

There are several ways to contribute to our project:

Reporting Bugs: If you find a bug, please open an issue and describe the problem. Make sure to include steps to reproduce the issue and any error messages you receive regarding that issue.

Fixing Bugs: If you'd like to fix a bug, please create a pull request with your changes. Make sure to include a description of the problem and how your changes will address it.

Additional examples and documentation improvements are also very welcome.

Resources

List of recommended Dagger.jl resources:

Help and Discussion

For help and discussion, we suggest asking in the following places:

Julia Discourse and on the Julia Slack in the #distributed channel.

Acknowledgements

We thank DARPA, Intel, and the NIH for supporting this work at MIT.

About

A framework for out-of-core and parallel execution

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • Julia 94.7%
  • HTML 5.3%