How to achieve Compile Time reflection in next ten thousand years

language design
1

I've been postponing this for a while, but wanted to write my thoughts in case a bus hit me.

Compile time is an often asked feature of Rust. But what would you need, to have this implemented by a 3rd party crate?

To get this, you first would need a Reflect trait. Ideally, this trait would define a pub const fn typeinfo() -> TypeInfo (where TypeInfo is an enum that has all the type info necessary)[1] Granted you can work around this via macros, by implementing this for primitive types yourself.

Given that you might need to instantiate an enum or a struct, you're going to need some sort of inverse of mem::discriminant. Basically a function/macro that looks like this

    fn reverse_discriminant<T: DiscriminantKind>(d: Discriminant<T>) -> T;

Finally, to either go with trait generator or the enum typeinfo, you will have to deal with complex types with N arguments or functions with N arguments (e.g. MyCoolType<N1, N2, N3, N4, N5, N6, N7....N123> or fn(A1, A2, A3, A4, A5, A6, A7, .... A451) ), to achieve that you need some sort of variadic generics. Or declaring that Rust types will never have more than thirteen type arguments.

In summary of importance:

  1. Get variadic generics - this will make it easier for any possible implementors.
  2. Get a way to read/write enum variants, or struct/union fields.
  3. Expand what is possible in const. Having an easier way to iterate, have strings rather than numbers and so on in const is a great boon. Also, maybe make it possible for a trait to declare all it has to const implemented.

  1. Why an enum and not some kind of complex trait tree that was in JeanHeyd's original proposal? Because if you want to have logic that decides whether the struct is aligned four or eight bytes aligned, you probably want to use something understandable like info.get_align() > 4 rather than a baroque set of traits. ↩ī¸Ž

1 Like
2

Have you heard of https://facet.rs/ ? I'm very interested in seeing where that experiment with reflection and dynamically creating objects goes.

15 Likes
3

There's something missing from this API, which is the payload. The problem is that each enum variant has different payload types. To properly make this API you need something like dependent types (generally considered unfeasible); or you need to receive some opaque trait object for parameters, and return Option<T> (None if the parameter doesn't match the type of that variant, checked at runtime)

2 Likes
4

fwiw I proposed a goal for experimenting with reflection over the next 6 months.

I am neither convinced we should to reflection nor that we shouldn't. Only that it's possible and lots of folk want it. I want to explore a design that I feel is very Rusty and see where it takes us. If we rip it all out in a year and decide it's creating more problems than it solves, that's ok, too. But there are just too many cool use cases for it not to try imo

14 Likes
5

facet does everything at runtime, and doesn't interact well with the rest of the type system.
The polar opposite to that would be a fully generic API, which does everything at compile time.
I have experimented with that here, and I ended up with something similar to soasis' work. It looks something like this:

struct Cons<A, B>(pub A, pub B);

trait Introspect {
    const NAME: &'static str;
}

trait Struct: Introspect {
    // either UnitShape, TupleShape, or NamedShape
    type Shape: StructShape; 
    // e.g. `Cons<Field0, Cons<Field1, Cons<..., ()>>>`,
    // where `Field0`, `Field1`, .. are dummy types generated by the derive macro 
    // and implement the `Field` trait.
    type Fields; 
}

trait Field {
    const NAME: Option<&'static str>;
    type Type: ?Sized;
    type Root: Introspect;

    fn try_get_ref(p: &Self::Root) -> Option<&Self::Type>;
    fn try_get_mut(p: &mut Self::Root) -> Option<&mut Self::Type>;
}

At the heart of this API is Cons, which represents a list of types without the need for variadic generics. It allows for recursive impls, e.g

impl PrintFields for () {}
impl<Head: Field, Tail: PrintFields> PrintFields for Cons<Head, Tail> {
  fn print_fields() {
    println!("{:?}", Head::NAME);
    Tail::print_fields();
  }
}

With just that (and a lot of atrocious code) it is already possible to replace serde's derive macros (without helper attributes), which I've done here.
That code exposes newtype structs ser::Reflect<T> and de::Reflect<T>, which implement serde::Serialize or serde::Deserialize, given that T: Introspect.
To fully replace the serde derive macros, users would still need to actually implement serde::Serialize and serde::Deserialize for their own types:

impl Serialize for MyStruct {
    fn serialize<S: Serializer>(&self, s: S) -> Result<S::Ok, S::Error> {
        ser::Reflect(self).serialize(s)
    }
}

Unlike a blanket impl<T: Introspect> Serialize for T impl (or the approach facet is pursuing), this allows custom implementations and plays nicely with the type system.

And while I haven't benchmarked anything, this does hold the promise of being zero-cost, unlike facet.

None of this is particularly ergonomic at the moment, but does seem like a promising approach to me. A better API and language features like variadic generics, sealed/closed traits and disjoint impls wrt. associated types would likely make this much more ergonomic.

1 Like
6

That is a bit complicated though. Potentially, it is more efficient in terms of code size to just have tables of field types than to generate code. This could be really important in embedded if you have many types you need to serialise and deserialise.

My understanding as of right now is that facet currently doesn't deliver on that, and I don't k ow enough about it to have any opinion on if it is at all feasible to reach such a point.

But I guess a compile time reflection library could still be used to generate tables for runtime use, so then the question becomes one of compile time, which is quite an important metric.

7

Of interest, C++ just voted in reflection for C++26 with the following proposals (I think this is all of them, but not 100% sure I didn't miss any). There may be some interesting ideas here to use:

2 Likes
8

Daniel Lemire also made a much more easily accessible blog about it: Discover C++26’s compile-time reflection – Daniel Lemire's blog