Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RFC: Stabilize the alloc crate #2480

Merged
merged 9 commits into from
Apr 3, 2019
331 changes: 331 additions & 0 deletions text/2480-liballoc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,331 @@
- Feature Name: `liballoc`
- Start Date: 2018-06-14
- RFC PR: [rust-lang/rfcs#2480](https://github.com/rust-lang/rfcs/pull/2480)
- Rust Issue: [rust-lang/rust#27783](https://github.com/rust-lang/rust/issues/27783)

# Summary
[summary]: #summary

Stabilize the `alloc` crate.

This crate provides the subset of the standard library’s functionality that requires
a global allocator (unlike the `core` crate) and an allocation error handler,
but not other operating system capabilities (unlike the `std` crate).


# Motivation
[motivation]: #motivation

## Background: `no_std`

In some environments the `std` crate is not available:
micro-controllers that don’t have an operating system at all, kernel-space code, etc.
The `#![no_std]` attribute allows a crate to not link to `std` implicitly,
using `core` instead with only the subset of functionality that doesn’t have a runtime dependency.

## `no_std` with an allocator

The `core` crate does not assume even the presence of heap memory,
and so it excludes standard library types like `Vec<T>`.
However some environments do have a heap memory allocator
(possibly as `malloc` and `free` C functions),
even if they don’t have files or threads
or something that could be called an operating system or kernel.
Or one could be defined [in a Rust library][wee-alloc]
ultimately backed by fixed-size static byte array.

An intermediate subset of the standard library smaller than “all of `std`”
but larger than “only `core`” can serve such environments.

[wee-alloc]: https://github.com/rustwasm/wee_alloc

## Libraries

In 2018 there is a [coordinated push]
toward making `no_std` application compatible with Stable Rust.
As of this writing not all of that work is completed yet.
For example, [`#[panic_implementation]`][panic-impl] is required for `no_std` but still unstable.
So it may seem that this RFC does not unlock anything new,
as `no_std` application still need to be on Nigthly anyway.

[coordinated push]: https://github.com/rust-lang-nursery/embedded-wg/issues/42
[panic-impl]: https://github.com/rust-lang/rust/issues/44489

The immediate impact can be found in the library ecosystem.
Many general-purpose libraries today are compatible with Stable Rust
and also have potential users who ask for them to be compatible with `no_std` environments.

For a library that is fundamentally about using for example TCP sockets or threads,
this may not be possible.

For a library that happens to only use parts of `std` that are also in `core`
(and are willing to commit to keep doing so), this is relatively easy:
add `#![no_std]` to the crate, and change `std::` in paths to `core::`.

And here again, there is the intermediate case of a library that needs `Vec<T>`
or something else that involves heap memory, but not other parts of `std` that are not in `core`.
Today, in order to not lose compatibility with Stable,
such a library needs to make compatibility with `no_std` an opt-in feature flag:

```rust
#![no_std]

#[cfg(feature = "no_std")] extern crate alloc;
#[cfg(not(feature = "no_std"))] extern crate std as alloc;

use alloc::vec::Vec;
```

But publishing a library that uses unstable features, even optionally,
comes with the expectation that it will be promptly updated whenever those features change.
Some maintainers are not willing to commit to this.

With this RFC, the library’s code can be simplified to:

```rust
#![no_std]

extern crate alloc;

use alloc::vec::Vec;
```

… and perhaps more importantly,
maintainers can rely on the stability promise made by the Rust project.


# Guide-level explanation
[guide-level-explanation]: #guide-level-explanation

## For libraries

When using `#![no_std]` in a crate, that crate does not implicitly depend on `std`
but depends on `core` instead. For example:

```diff
-use std::cell::RefCell;
+use core::cell::RefCell;
```

APIs that require a memory allocator are not available in `core`.
In order to use them, `no_std` Rust code must explicitly depend on the `alloc` crate:

```rust
#[macro_use] extern crate alloc;

use core::cell::RefCell;
use alloc::rc::Rc;
```

Note: `#[macro_use]` imports the [`vec!`] and [`format!`] macros.

[`vec!`]: https://doc.rust-lang.org/alloc/macro.vec.html
[`format!`]: https://doc.rust-lang.org/alloc/macro.format.html

Like `std` and `core`, this dependency does not need to be declared in `Cargo.toml`
since `alloc` is part of the standard library and distributed with Rust.

The implicit prelude (set of items that are automatically in scope) for `#![no_std]` crates
does not assume the presence of the `alloc` crate, unlike the default prelude.
So such crates may need to import either that prelude or specific items explicitly.
For example:

```rust
use alloc::prelude::*;

// Or

use alloc::string::ToString;
use alloc::vec::Vec;
```

## For programs¹

[¹] … and other roots of a dependency graph, such as `staticlib`s.

Compared to `core`, the `alloc` crate makes two additional requirements:

* A global heap memory allocator.

* An allocation error handler (that is not allowed to return).
This is called for example by `Vec::push`, whose own API is infallible,
when the allocator fails to allocate memory.

`std` provides both of these. So as long as it is present in the dependency graph,
nothing else is required even if some crates of the graph use `alloc` without `std`.

If `std` is not present they need to be defined explicitly,
somewhere in the dependency graph (not necessarily in the root crate).

* [The `#[global_allocator]` attribute][global_allocator], on a `static` item
of a type that implements the `GlobalAlloc` trait,
defines the global allocator. It is stable in Rust 1.28.

* [Tracking issue #51540] propose the `#[alloc_error_handler]` attribute
for a function with signature `fn foo(_: Layout) -> !`.
As of this writing this attribute is implemented but unstable.

[global_allocator]: https://doc.rust-lang.org/nightly/std/alloc/#the-global_allocator-attribute
[Tracking issue #51540]: https://github.com/rust-lang/rust/issues/51540


# Reference-level explanation
[reference-level-explanation]: #reference-level-explanation

The `alloc` crate already exists (marked unstable),
and every public API in it is already available in `std`.

Except for the `alloc::prelude` module, since [PR #51569] the module structure is a subset
of that of `std`: every path that starts with `alloc::` is still valid and point to the same item
after replacing that prefix with `std::` (assuming both crates are available).

The concrete changes proposed by this RFC are:

* Stabilize `extern crate alloc;`
(that is, change `#![unstable]` to `#![stable]` near the top of `src/liballoc/lib.rs`).

* Stabilize the `alloc::prelude` module and its contents
(which is only re-exports of items that are themselves already stable).

* Stabilize the fact that the crate makes no more and no less than
the two requirements/assumptions of a global allocator and an allocation error handler
being provided for it, as described above.

The exact mechanism for [providing the allocation error handler][Tracking issue #51540]
is not stabilized by this RFC.

In particular, this RFC proposes that the presence of a source of randomness
is *not* a requirement that the `alloc` crate can make.
This is contrary to what [PR #51846] proposed,
and means that `std::collections::hash_map::RandomState` cannot be moved into `alloc`.

[Tracking issue #27783] tracks “the `std` facade”:
crates whose contents are re-exported in `std` but also exist separately.
Other such crates have already been moved, merged, or stabilized,
such that `alloc` is the only remaining unstable one.
Therefore #27783 can serve as the tracking issue for this RFC
and can be closed once it is implemented.

[PR #51569]: https://github.com/rust-lang/rust/pull/51569
[PR #51846]: https://github.com/rust-lang/rust/pull/51846
[Tracking issue #27783]: https://github.com/rust-lang/rust/issues/27783


The structure of the standard library is therefore:

* `core`: has (almost) no runtime dependency, every Rust crate is expected to depend on this.
* `alloc`: requires a global memory allocator,
either specified through the `#[global_allocator]` attribute
or provided by the `std` crate.
* `std`: re-exports the contents of `core` and `alloc`
so that non-`no_std` crate do not need care about what’s in what crate between these three.
Depends on various operating system features such as files, threads, etc.
* `proc-macro`: depends on parts of the compiler, typically only used at build-time
(in procedural macro crates or Cargo build scripts).



# Drawbacks
[drawbacks]: #drawbacks

[Tracking issue #27783] is the tracking issue for the `alloc` crate and, historically, some other crates.
Although I could not find much discussion of that, I believe it has been kept unstable so far
because of uncertainty of what the eventual desired crate structure
for the standard library is, given infinite time and resources.

In particular, should we have a single crate with some mechanism for selectively disabling
or enabling some of the crate’s components, depending on which runtime dependencies
are available in targeted environments?
In that world, the `no_std` attribute and standard library crates other than `std`
would be unnecessary.

By stabilizing the `alloc` crate, we commit to having it − and its public API − exist “forever”.


# Rationale and alternatives
[alternatives]: #alternatives

## Single-crate standard library

The `core` and the `no_std` attribute are already stable,
so in a sense it’s already too late for the “pure” version of the vision described above
where `std` really is the only standard library crate that exists.

It may still be [desirable] to regroup the standard library into one crate,
and it is probably still possible.
The `core` crate could be replaced with a set of `pub use` reexport
to maintain compatibility with existing users.
Whatever the eventual status for `core` is,
we can do the same for `alloc`.
[PR #51569] mentioned above also hopes to make this easier.

While we want to leave the possibility open for it,
at the time of this writing there are no concrete plans
for implementing such a standard library crates unification any time soon.
So the only alternative to this RFC seems to be
leaving heap allocation for `no_std` in unstable limbo for the foreseeable future.

[desirable]: https://aturon.github.io/2018/02/06/portability-vision/#the-vision

## Require randomness

[PR #51569] proposed adding a source of randomness to the other requirements
made by the `alloc` crate.
This would allow moving `std::collections::hash_map::RandomState`,
and therefore `HashMap` (which has `RandomState` as a default type parameter),
into `alloc`.

This RFC chooses not to do this because it would make it difficult to use for example `Vec<T>`
in environments where a source of randomness is not easily available.

I hope that the language will eventually make it possible to have `HashMap` in `alloc`
without a default hasher type parameter, and have the same type in `std` with its current default.

Although I am not necessarily in favor
of continuing the increase of the number of crates in the standard library,
another solution for `HashMap` in `no_std` might be another intermediate crate
that depends on `alloc` and adds the randomness source requirement.

Additionally, with this RFC it should be possible to make https://crates.io/crates/hashmap_core
compatible with Stable Rust.
The downside of that crate is that although based on a copy of the same code,
it is a different type incompatible in the type system with `std::collections::HashMap`.


# Prior art
[prior-art]: #prior-art

I am not aware of a mechanism similar to `no_std` in another programming language.

[Newlib] is a C library for “embedded” systems that typically don’t have an operating system.
It does provide a memory allocator through `malloc` and related functions, unconditionally.

[Newlib]: https://sourceware.org/newlib/


# Unresolved questions
[unresolved]: #unresolved-questions

* Did I miss something in [PR #51569] that makes `alloc` not a subset of `std`?
A double-check from someone else would be appreciated.

* Should the crate be renamed before stabilization?
It doesn’t have exclusivity for memory-allocation-related APIs,
since the `core::alloc` module exists.
What really characterizes it is the assumption that a global allocator is available.
The name `global_alloc` was proposed.
(Although the crate doesn’t only contain the global allocator itself.)

* ~Should the `alloc::prelude` module be moved to `alloc::prelude::v1`?
This would make the `alloc` module structure a subset of `std` without exception.
However, since this prelude is not inserted automatically,
it is less likely that we’ll ever have a second version of it.
In that sense it is closer to `std::io::prelude` than `std::prelude::v1`.~
Done in [PR #58933].

* In addition to being a subset of `std`, should the `alloc` crate (by itself)
be a super-set of `core`? That is, should it reexport everything that is defined in `core`?
See [PR #58175] which proposes reexporting `core::sync::atomic` in `alloc::sync`.

[PR #58933]: https://github.com/rust-lang/rust/pull/58933
[PR #58175]: https://github.com/rust-lang/rust/pull/58175