More about `dyn Any`

We’ve taken a lot of care to emphasize that dyn Trait isn’t a supertype nor a form of dynamic typing, but in one of the examples we saw that dyn Any can “downcast” back to the erased base type. Or to quote the official documentation, Any is

A trait to emulate dynamic typing.

Here we take a closer look at dyn Any specifically.

The general idea

The Any trait is implemented for all types which satisfy a 'static bound. It supplies a method type_id, which returns an opaque but unique identifier of the implementing type. We also have TypeId::of::<T>, which lets us look up the TypeId of any 'static type.

Together, this allows fallible downcasting by doing things along these lines:

    pub fn downcast_ref<T: Any>(&self) -> Option<&T> {
        if self.is::<T>() {
            // SAFETY: just checked whether we are pointing to the correct type, and we can rely on
            // that check for memory safety because we have implemented Any for all types; no other
            // impls can exist as they would conflict with our impl.
            unsafe { Some(self.downcast_ref_unchecked()) }
        } else {
            None
        }
    }

And is simply compares TypeId::of::<T>() to self.type_id().

Details of specific downcasts aside, that’s pretty much it! All that was needed is the global identifier (TypeId). The standard library provides the various downcasting methods in order to encapsulate the required unsafety.

As a reminder from before, the vtable pointers themselves are not suitable to use as a global identifier of erased types. The same trait can have multiple vtables due to codegen units and linker implementations, and different traits can have the same vtable due to deduplication optimizations.

Where exactly the language goes with respect to comparing vtable pointers is an open question. It’s not unimaginable that all vtables will gain some lifetime-erased version of TypeId, but related to some discussion below, this may not be as straightforward as it may sound.

Downcasting methods are not trait methods

Note that the only method available in the Any trait is type_id. All of the downcasting methods are implemented on the erased dyn Any directly, or on Box<dyn Any>, or on dyn Any + Send, etc. Downcasting doesn’t generally make sense for a non-erased base type – you already know what it is!

Another good reason for this is that types like Box<dyn Any> implement Any, making easy to accidentally call the Box<dyn Any> implementation instead of the dyn Any implementation in the case of type_id. It would be much more fraught if downcast_ref worked like this, for example.

However, this does mean that having Any as a supertrait does not allow downcasting for your own dyn Traits. Instead you have to first upcast to dyn Any, and then downcast, as we’ve seen before.

But note that a supertrait Any bound is not the only solution for custom downcasting! We explore another approach below.

Some brief examples

Here we look at some simple examples of type erasing and downcasting concrete types directly.

The basics

Getting a dyn Any isn’t any different than any other kind of type erasure:

#![allow(unused)]
fn main() {
use std::any::Any;
let mut i = 0;
let rf: &dyn Any = &();
let mt: &mut dyn Any = &mut i;
let bx: Box<dyn Any> = Box::new(String::new());
}

You have to keep in mind the 'static requirement though:

#![allow(unused)]
fn main() {
use std::any::Any;
let local = ();
let borrow = &local;
// fails because `borrow` is not `'static`
let _: &dyn Any = &borrow;
}

On the upside, dyn Any is always dyn Any + 'static, which makes many trait object related borrow check errors impossible.

Although Any is implemented for unsized types, and unsized types can have TypeIds too, the Sized restriction for type erasure still applies:

#![allow(unused)]
fn main() {
use std::any::Any;
// fails because `str` is not `Sized`
let _: &dyn Any = "";
}

Fallible downcasting is pretty straightforward as well. For references the return is an Option:

#![allow(unused)]
fn main() {
use std::any::Any;
let mut i = 0;
let rf: &dyn Any = &();
let mt: &mut dyn Any = &mut i;
assert_eq!( rf.downcast_ref::<()>(), Some(&()) );
assert_eq!( rf.downcast_ref::<String>(), None );

assert!( mt.downcast_mut::<i32>().is_some() );
assert_eq!( mt.downcast_mut::<String>(), None );
}

For Boxes, the return type is a Result so that the you can keep ownership of the Box<dyn Any> if the downcast is not applicable. The Ok variant is a Box<T> so that you can choose whether it’s appropriate to unbox the type or not.

#![allow(unused)]
fn main() {
use std::any::Any;
let bx: Box<dyn Any> = Box::new(String::new());
if let Err(bx) = bx.downcast::<i32>() {
    println!("Hmm, not an `i32`.");
    if let Ok(bx) = bx.downcast::<String>() {
        let s: String = *bx;
        println!("Yep, it was a `String`.");
    }
}
}

That’s it for the basics!

The `TypeMap` pattern

For an example with a more practical bent, let’s say you wanted to store a distinct value for each distinct type you may encounter, for some reason. Maybe you’re storing callbacks for types which are likewise type erased, say, and the callback for a Dog would be different than that for a Cat, and you might not even have a callback for the Mouse.

One way to do this would be to have a data structure that maps a TypeId to the values. A “type map”, if you will:

#![allow(unused)]
fn main() {
use std::any::Any;
use std::any::TypeId;
use std::collections::HashMap;
pub struct TypeMap<V> {
    map: HashMap<TypeId, V>,
}

impl<V> TypeMap<V> {
    pub fn insert<T: Any>(&mut self, value: V) -> Option<V> {
        let id = TypeId::of::<T>();
        self.map.insert(id, value)
    }
    pub fn get_mut<T: Any>(&mut self) -> Option<&mut V> {
        self.get_mut_of(&TypeId::of::<T>())
    }
    pub fn get_mut_of(&mut self, id: &TypeId) -> Option<&mut V> {
        self.map.get_mut(id)
    }
    // ...
}
}

This could be used for the callback idea:

#![allow(unused)]
fn main() {
use std::any::Any;
use std::any::TypeId;
use std::collections::HashMap;
pub struct TypeMap<V> {
    map: HashMap<TypeId, V>,
}
impl<V> TypeMap<V> {
    pub fn insert<T: Any>(&mut self, value: V) -> Option<V> {
        let id = TypeId::of::<T>();
        self.map.insert(id, value)
    }
    pub fn get_mut<T: Any>(&mut self) -> Option<&mut V> {
        self.get_mut_of(&TypeId::of::<T>())
    }
    pub fn get_mut_of(&mut self, id: &TypeId) -> Option<&mut V> {
        self.map.get_mut(id)
    }
}
pub struct Visitor {
    map: TypeMap<Box<dyn FnMut(&dyn Any)>>,
}

impl Visitor {
    // Because we return closures we have previously created,
    // we should take care to not *assume* that the parameter
    // in the callback is of the correct type.  If we never
    // let our closures escape to the outside world, we could
    // safely assume that the parameter was, in fact, `T`.
    //
    // It would be sound to `panic` if the parameter was not
    // `T` even if we let the closures escape, but it would
    // not be sound to use the unstable `downcast_ref_unchecked`
    // so long as we're letting the closure escape.
    pub fn register<T, F>(&mut self, mut callback: F) -> Option<Box<dyn FnMut(&dyn Any)>>
    where
        T: Any,
        F: 'static + FnMut(&T),
    {
        let callback = Box::new(move |any: &dyn Any| {
            if let Some(t) = any.downcast_ref::<T>() {
                callback(t);
            }
        });

        self.map.insert::<T>(callback)
    }

    pub fn get_callback<T: Any>(&mut self) -> Option<impl FnMut(&T) + '_> {
        self.map
            .get_mut::<T>()
            .map(|f| {
                |t: &T| f(t)
            })
    }

    pub fn visit<T: Any>(&mut self, value: &T) -> bool {
        if let Some(mut callback) = self.get_callback::<T>() {
            callback(value);
            true
        } else {
            false
        }
    }

    pub fn visit_erased(&mut self, value: &dyn Any) -> bool {
        if let Some(callback) = self.map.get_mut_of(&value.type_id()) {
            callback(value);
            true
        } else {
            false
        }
    }
}
}

Above we have also type erased our callback signatures, since we needed a single type for our values. This is somewhat the data structure version of erasing a trait.

For whatever reason you might want to map by types, this is an existing pattern in the ecosystem.

Custom downcasting

In this example, we explore how you can add (emulated) downcasting to your own traits, without an Any supertrait bound and without a 'static bound on your entire trait, either. The implementation does still depend on the TypeId, so the actual downcasting is still limited to types which satisfy a 'static bound.

The approach is to make implementing the general idea possible by having a method in our trait that returns the TypeId of the implementing type.

#![allow(unused)]
fn main() {
use core::any::TypeId;
pub trait Trait {
    // Heads up: We'll need to revisit this definition in just a bit
    fn type_id(&self) -> TypeId
    where
        Self: 'static
    {
        TypeId::of::<Self>()
    }
}
}

Recall that the compiler’s implementation of Trait for dyn Trait will implicitly downcast the receiver and call this method. Therefore, we now have a way to get the TypeId of the base type out of dyn Trait.

Then we can implement our own downcast methods:

#![allow(unused)]
fn main() {
use core::any::TypeId;
pub trait Trait { fn type_id(&self) -> TypeId where Self: 'static { TypeId::of::<Self>() } }
// n.b. this is `dyn Trait + 'static` :-)
impl dyn Trait {
    pub fn is<T: 'static>(&self) -> bool {
        TypeId::of::<T>() == self.type_id()
    }

    pub fn downcast<T: 'static>(self: Box<Self>) -> Result<Box<T>, Box<Self>> {
        if (*self).is::<T>() {
            let ptr = Box::into_raw(self) as *mut T;
            // SAFETY: Keep reading :-)
            unsafe { Ok(Box::from_raw(ptr)) }
        } else {
            Err(self)
        }
    }
}
}

…and similarly for downcast_ref, downcast_mut, and more implementations for dyn Trait + Send, dyn Trait + Send + Sync, and so on. (Yes, it can be a lot of boilerplate.)

However, there’s a large soundness hole in this example. Implementors of Trait can supply their own implementation of the type_id method! They can override the default body and return a TypeId that is not the implementing type. That makes our unsafe produce UB; since type_id isn’t an unsafe method, our implementation is to blame.

We could make the method unsafe. In this example, we will instead make it impossible to override the default body. To do this, we need that method to be final. Well, Rust doesn’t have final yet. However, we can seal the method by giving it a parameter which can only be named within our module.

#![allow(unused)]
fn main() {
// A private module (no `pub`)
mod private {
    // Containing a `pub` type (to avoid errors on our public trait method)
    pub struct Seal;
}
}

 pub trait Trait {
-    fn type_id(&self) -> TypeId
+    #[doc(hidden)]
+    fn type_id(&self, _: private::Seal) {
     where
         Self: 'static
     {
         TypeId::of::<Self>()
     }
 }

 impl dyn Trait {
     pub fn is<T: 'static>(&self) -> bool {
-        TypeId::of::<T>() == self.type_id()
+        TypeId::of::<T>() == self.type_id(private::Seal)
     }

Now if an implementor tries to write out the signature of the type_id method, they’ll get a privacy error.

Here’s a playground with a couple more methods. This example is based on std::error::Error and its own custom downcasting implementation.

The representation of `TypeId`

TypeId is intentionally opaque and subject to change. It was internally represented by a u64 for quite some time; as of Rust 1.72 the representation is a u128. At some future time it could be something more exotic.

Long story short, you’re not meant to rely on the exact representation of TypeId, only it’s type comparing properties.

Why `'static`?

The Any trait is implemented for all types which satisfy a 'static bound, but no other types; in fact, it has a 'static bound and thus cannot be implemented for types that do not meet a 'static bound. This means that emulating dynamic typing with Any cannot be done for borrowing types (except those that borrow for 'static), for example.

Why such a harsh restriction? In short, lifetimes are erased before runtime, types with different lifetimes would have to have the same TypeId identifier, and thus downcasting based on the TypeId would ignore lifetimes and be wildly unsound. Lifetimes are a part of types and certain relationships must be preserved for soundness, but as the lifetimes have been erased before runtime, it’s not possible to preserve the relationships dynamically.

Thus there is just no sound way to use TypeId or any similar lifetime-unaware identifier to perform non-'static downcasts directly.

There is more information in this RFC PR, for the curious. Note that the PR was accepted but then later removed, and was never about non-'static downcasting; it was about a non-'static type_id method. The idea was to get a “type” identifier that ignored lifetimes.

It was withdrawn in large part because if such a thing existed, the chances of it being abused in some wildly unsound way are about 100%.

An alternative (as presented in that thread) is to have some way to dynamically check if two types (which are perhaps generic) are equal without imposing a 'static bound. The check could only be meaningful for types that were “inherently 'static”, that is, types that do not involve any lifetime parameters at all. That would be possible without actually exposing a non-'static TypeId or otherwise enabling downcasting.

The tradeoff results in pretty unintuitive behavior: &'static str cannot be compared to &'static str with this approach, because there is a lifetime parameter involved with &str!

Another alternative is to provide some sort of “type lambda” which is itself 'static, but can soundly map erased lifetimes back to their proper position. A sketch is provided here, but an in-depth exploration is out of scope for this guide.

Higher-ranked type considerations

Let’s take a minute to talk about types that do have a sub and supertype relationship in Rust! Types which are higher-ranked have this relationship. For example:

#![allow(unused)]
fn main() {
// More explicitly, this is a `for<'any> fn(&'any str)` function pointer.
// The type is higher-ranked over the lifetime.
let fp: fn(&str) = |_| {};

// This type is a supertype of the higher-ranked type.
let fp: fn(&'static str) = fp;

// This errors because you can't soundly downcast the types.
// let fp: fn(&str) = fp;
}

And as we mentioned before, this sub/supertype relationship applies to dyn Trait types as well:

#![allow(unused)]
fn main() {
trait Look<'s> {}
type HR = dyn for<'any> Look<'any> + 'static;
type ST = dyn Look<'static> + 'static;
}

Here, HR is a subtype of ST, but not the same type. However, they both satisfy a 'static bound:

#![allow(unused)]
fn main() {
trait Look<'s> {}
type HR = dyn for<'any> Look<'any> + 'static;
type ST = dyn Look<'static> + 'static;
fn assert_static<T: ?Sized + 'static>() {}

assert_static::<HR>();
assert_static::<ST>();
}

As 'static types, they have TypeIds. As distinct types, their TypeIds are different, even though one is a subtype of the other.

And this in turn means that you can’t stop thinking about sub-and-super types by simply applying a 'static bound. If you need to “disable” sub/super type coercions in a generic context for soundness, you must make that context invariant or take other steps to avoid a soundness hole, even if you have a 'static bound.

See this issue for a real-life example of such a soundness hole, and this comment in particular exploring the sub/super type relationships of higher-ranked function pointers.

The fact that the sub-and-super types are still distinct types means that it’s also possible for them to have differing implementations of a trait other than Any, as well:

#![allow(unused)]
fn main() {
trait Look<'s> {}
type HR = dyn for<'any> Look<'any> + 'static;
type ST = dyn Look<'static> + 'static;

trait Trait {
    fn say(&self);
}

impl Trait for HR {
    fn say(&self) {
        println!("Meow");
    }
}

impl Trait for ST {
    fn say(&self) {
        println!("Woof");
    }
}
}

While this does currently give a future compatibility warning, the plan is to keep accepting this pattern.

History around `const` `TypeId::of`

As of Rust 1.91, TypeId::of is a const fn. (Any::type_id is still not const as trait methods in general cannot yet be const.)

The stabilization was a long time coming due to how different parts of the compiler considered type equality. When different parts of the compiler disagreed, the results could be unsound in a const context.

To briefly describe the historical disagreement, consider the following example.

#![allow(unused)]
fn main() {
let one: for<'a    > fn(&'a str, &'a str) = |_, _| {};
let two: for<'a, 'b> fn(&'a str, &'b str) = |_, _| {};
let mut fp = one;
fp = two;
let mut fp = two;
fp = one;
}

Due to covariance, it’s always possible to call either of the functions from the other. They can be considered subtypes of one another. However, similar to what we explored above, they are distinct types with distinct TypeIds.

Before PR 118247, parts of the compiler would consider the two types to be subtypes of each other, and consider types which are subtypes of each other to be equal types – let’s call them semantically equal. However, this was unsound as TypeId is based on syntactic equality.

The PR resolved that disagreement in favor of the syntactic interpretation.

The code example just above still compiles, so internally the types are either still subtypes of each other, or some other coercion is possible even though they are not subtypes of each other. In either case, it is unintuitive that they are nonetheless distinct types.

Built-in `dyn` implementation precedence considerations

We noted before that even if you have a blanket implementation which should cover dyn Trait, the built-in implementation of Trait for dyn Trait usually takes precedence. And we linked to this issue about how this is currently sometimes unsound.

Based on the conversation around that issue, the language developers would prefer to give precedence to the user implementation in most cases. However, this would break the Any trait! Its implementation is a blanket implementation which overlaps with the built-in dyn Any implementation. But returning the TypeId for dyn Any from <dyn Any as Any>::type_id would negate the entire point of the trait – getting the TypeId of the erased type.

In other words, for dyn Any (and the dyn of any subtraits of Any) it is crucial that the built-in implementation continues to take precedence.

And in fact, it even goes beyond that. Consider our (and Error’s) built-in downcasting approach:

pub trait Trait {
    // SAFETY: This method must return the base type's `TypeId` to ensure
    // soundness in the `is` and downcasting methods of this module.  The
    // method is therefore sealed so that the default body below is the
    // only possible body.
    #[doc(hidden)]
    fn type_id(&self, _: private::Seal) -> TypeId
    where
        Self: 'static,
    {
        TypeId::of::<Self>()
    }
}

If there’s any situation where the built-in implementation does not take precedence, <dyn Trait>::type_id will return the TypeId for dyn Trait and not the erased type – even though there is no supertrait relationship here.

There is such a situation under active development as of this writing – final methods. One of the possible implementations for that feature is that final methods could be excluded from the vtable of the trait. But in effect, that would mean that the default method body takes precedence over the built-in vtable based implementation.

Or in other words, if final methods are never in the vtable, making Trait::type_id a final method as an alternative to sealing the method cannot work.

Where the feature goes is still an open question.

Keyboard shortcuts

Learning Rust