Encapsulating Lifetime of the Field
This is a post about an annoying Rust pattern and an annoying workaround, without a good solution :)
Problem Statement
Suppose you have some struct which holds some references inside. Now, you want to store a reference to this structure inside some larger struct. It could look like this:
struct Foo<'a> {
buff: &'a String
}
struct Context<'f> {
foo: &'f Foo
}The code, as written, does not compile:
error[E0106]: missing lifetime specifier
--> src/main.rs:8:14
|
8 | foo: &'f Foo
| ^^^ expected lifetime parameterTo fix it, we need to get Foo an additional lifetime:
struct Foo<'a> {
buff: &'a String
}
struct Context<'f, 'a: 'f> {
foo: &'f Foo<'a>
}And this is the problem which is the subject of this post. Although
Foo is supposed to be an implementation detail, its lifetime, 'a,
bleeds to Context’s interface, so most of the clients of Context
would need to name this lifetime together with 'a: 'f bound. Note
that this effect is transitive: in general, rust struct has to name
lifetimes of contained types, and their contained types, and their
contained types, … But let’s concentrate on this two-level example!
The question is, can we somehow hide this 'a from users of Context? It’s
interesting that I’ve first distilled this problem about half a year ago in this
urlo
post,
and today, while refactoring some of Cargo internals in
#5476 with
@dwijnand, I’ve stumbled upon something, which
could be called a solution, if you squint hard enough.
Extended Example
Let’s create a somewhat longer example to check that lifetime setup actually works out in practice.
struct Foo<'a> {
buff: &'a String
}
impl<'a> Foo<'a> {
fn len(&self) -> usize {
self.buff.len()
}
}
struct Context<'f, 'a: 'f> {
foo: &'f Foo<'a>
}
// Note how we have to repeat ugly `'a: 'f` bound here!
impl<'f, 'a: 'f> Context<'f, 'a> {
fn new(foo: &'f Foo<'a>) -> Self {
Context { foo }
}
fn len(&self) -> usize {
self.foo.len()
}
}
// Check, that we actually can create a `Context`
// from `Foo` and call a method.
fn test<'f, 'a>(foo: &'f Foo<'a>) {
let ctx = Context::new(foo);
ctx.len();
}First fix
The first natural idea is to try to use the same lifetime, 'f for
both & and Foo: it fits syntactically, so why not give it a try?
struct Foo<'a> {
buff: &'a String
}
impl<'a> Foo<'a> {
fn len(&self) -> usize {
self.buff.len()
}
}
struct Context<'f> {
foo: &'f Foo<'f>
}
impl<'f> Context<'f> {
fn new<'a>(foo: &'f Foo<'a>) -> Self {
Context { foo }
}
fn len(&self) -> usize {
self.foo.len()
}
}
fn test<'f, 'a>(foo: &'f Foo<'a>) {
let ctx = Context::new(foo);
ctx.len();
}Surprisingly, it works! I’ll show a case where this approach breaks down
in a moment, but let’s first understand why this works. The magic
happens in the new method, which could be written more explicitly as
fn new<'a: 'f>(foo: &'f Foo<'a>) -> Self {
let foo1: &'f Foo<'f> = foo;
Context { foo: foo1 }
}Here, we assign a &'f Foo<'a> to a variable of a different type &'f
Foo<'f>. Why is this allowed? We use 'a lifetime in Foo only for
a shared reference. That means that Foo is
covariant over
'a. And that means that the compiler can use Foo<'a> instead of
Foo<'f> if 'a: 'f. In other words rustc is allowed to shorten the
lifetime.
It’s interesting to note that the original new function didn’t say
that 'a: 'f, although we had to add this bound to the impl block
explicitly. For functions, the compiler infers such bounds from
parameters.
Hopefully, I’ve mixed polarity an even number of times in this variance discussion :-)
Going invariant
Let’s throw a wrench in the works by adding some unique references:
struct Foo<'a> {
buff: &'a mut String
}
impl<'a> Foo<'a> {
fn push(&mut self, c: char) {
self.buff.push(c)
}
}
struct Context<'f, 'a: 'f> {
foo: &'f mut Foo<'a>
}
impl<'f, 'a: 'f> Context<'f, 'a> {
fn new(foo: &'f mut Foo<'a>) -> Self {
Context { foo }
}
fn push(&mut self, c: char) {
self.foo.push(c)
}
}
fn test<'f, 'a>(foo: &'f mut Foo<'a>) {
let mut ctx = Context::new(foo);
ctx.push('9');
}Foo is now invariant, so the previous solution does not work:
struct Context<'f> {
foo: &'f mut Foo<'f>
}
impl<'f> Context<'f> {
fn new<'a: 'f>(foo: &'f mut Foo<'a>) -> Self {
let foo1: &'f mut Foo<'f> = foo;
Context { foo: foo1 }
}
fn push(&mut self, c: char) {
self.foo.push(c)
}
}error[E0308]: mismatched types
--> src/main.rs:17:37
|
17 | let foo1: &'f mut Foo<'f> = foo;
| ^^^ lifetime mismatch
|
= note: expected type `&'f mut Foo<'f>`
found type `&'f mut Foo<'a>`Unsheathing existentials
Let’s look again at the Context type:
struct Context<'f, 'a: 'f> {
foo: &'f mut Foo<'a>
}What we want to say is that, inside the Context, there is some
lifetime 'a which the consumers of Context need not care about,
because it outlives 'f anyway. I think that the syntax for that
would be something like
struct Context<'f> {
foo: &'f mut for<'a: f> Foo<'a>
}Alas, for is supported only for traits and function pointers, and
there it has the opposite polarity of for all instead of exists,
so using it for a struct gives
error[E0404]: expected trait, found struct `Foo`
--> src/main.rs:12:30
|
12 | foo: &'f mut for<'a: 'f> Foo<'a>
| ^^^^^^^ not a traitA hack
However, and this is what I realized reading the Cargo’s source code, we can use a trait here!
struct Foo<'a> {
buff: &'a mut String
}
impl<'a> Foo<'a> {
fn push(&mut self, c: char) {
self.buff.push(c)
}
}
trait Push {
fn push(&mut self, c: char);
}
impl<'a> Push for Foo<'a> {
fn push(&mut self, c: char) {
self.push(c)
}
}
struct Context<'f> {
foo: &'f mut (Push + 'f)
}
impl<'f> Context<'f> {
fn new<'a>(foo: &'f mut Foo<'a>) -> Self {
let foo: &'f mut Push = foo;
Context { foo }
}
fn push(&mut self, c: char) {
self.foo.push(c)
}
}
fn test<'f, 'a>(foo: &'f mut Foo<'a>) {
let mut ctx = Context::new(foo);
ctx.push('9');
}We’ve added a Push trait, which has the same interface as the Foo
struct, but is not parametrized over the lifetime. This is
possible because Foo’s interface doesn’t actually depend on the 'a
lifetime. And this allows us to magically write foo: &'f mut (Push + 'f).
This + 'f is what hides 'a as “some unknown lifetime, which outlives 'f”.
A hack, refined
There are many problems with the previous solution: it is ugly, complicated and introduces dynamic dispatch. I don’t know how to solve those problems, so let’s talk about something I know how to deal with :-)
The Push trait duplicated the interface of the Foo struct. It
wasn’t that bad, because Foo had only one method. But what if
Bar has a dozen of methods? Could we write a more general trait,
which gives us access to Foo directly? Looks like it is possible, at
least to some extent:
struct Foo<'a> {
buff: &'a mut String
}
impl<'a> Foo<'a> {
fn push(&mut self, c: char) {
self.buff.push(c)
}
}
trait WithFoo {
fn with_foo<'f>(&'f mut self, f: &mut FnMut(&'f mut Foo));
}
impl<'a> WithFoo for Foo<'a> {
fn with_foo<'f>(&'f mut self, f: &mut FnMut(&'f mut Foo)) {
f(self)
}
}
struct Context<'f> {
foo: &'f mut (WithFoo + 'f)
}
impl<'f> Context<'f> {
fn new<'a>(foo: &'f mut Foo<'a>) -> Self {
let foo: &'f mut WithFoo = foo;
Context { foo }
}
fn push(&mut self, c: char) {
self.foo.with_foo(&mut |foo| foo.push(c))
}
}
fn test<'f, 'a>(foo: &'f mut Foo<'a>) {
let mut ctx = Context::new(foo);
ctx.push('9');
}How does this work? Generally, we want to say that “there exists some
lifetime 'a, which we know nothing about except that 'a: 'f”. Rust
supports similar constructions only for functions, where for<'a> fn
foo(&'a i32) means that a function works for all lifetimes 'a. The
trick is to turn one into another! The desugared type of callback f,
is &mut for<'x> FnMut(&'f mut Foo<'x>). That is, it is a function
which accepts Foo with any lifetime. Given that callback, we are
able to feed our Foo with a particular lifetime to it.
Conclusion
While the code examples in the post juggled Foos and Bars, the
core problem is real and greatly affects the design of Rust code. When
you add a lifetime to a struct, you “poison” it, and all structs which
contain it as a member need to declare this lifetime as well. I would
love to know a proper solution for this problem: the described trait
object workaround is closer to code golf than to the practical
approach.
Discussion on /r/rust.