Extending Moose

Editor's Notes

• #2: Presented YAPC::NA, 2009-06-23. CMU, Pittsburgh, PA.
• #3: People can be shy about asking questions in a room full of strangers. So if you want, you can ask me a question on Twitter and I'll eventually get to it. Just direct your questions at @sartak. That way you don't forget it or worry about interrupting me. Got the idea from Giles Bowkett: http://gilesbowkett.blogspot.com/2009/04/how-to-take-questions-at-tech.html
• #4: Of course, many people dislike Twitter, so I'll check out at IRC as a last resort. I'm Sartak there too. You don't have to use either of these. If you have a burning question or correction, shoot your hand up at any time.
• #5: The title of this talk is Extending Moose for Applications. It means just what it says. We want to extend Moose to be more useful and expressive for the problem at hand.
• #6: But I really wanted to call the talk Domain-specific Metaprogramming. It's more honest. But it's pretentious and a little scary. Hell, with that title I would have been able to sleep in today.
• #7: But these two titles are the same talk.
• #8: You extend Moose through metaprogramming, which is what the talk is all about.
• #9: And an application is a domain-specific system. I want to teach you new concepts and techniques so that you can solve your business needs.
• #10: In March I was at a Lisp conference with jrockway. I'm not much of a Lisper but I do appreciate Lisp for the huge impact it has had on language design. The conference was very enjoyable. What stuck out most was CLOS, which is Common Lisp's version of Moose. It is 15 years older.
• #11: When I was there, quite a few talks began with "I extended CLOS to do X". I was so impressed that immediately after the first day of the conference, I submitted this talk for YAPC. I think Moose is ready to let many more people seriously dig into its guts to make their applications much better.
• #12: What do I actually mean by "metaprogramming"?
• #13: How many of you have asked an object if it performs a particular method?
• #14: Or if an object is an instance of a particular class?
• #15: Or if an object does a particular role? (chromatic?) This is a relatively new one, added in 5.10.
• #16: Who here has programmatically generated code? String eval has a lot of well-deserved stigma, but it's occasionally necessary.
• #17: String eval is interesting because there's an obvious separation of the code that is doing the generating (all these function calls and concatenation) and the code that is generated (the contents of $code). We could say that the first three lines are metaprogramming, since it's code whose domain is other code. The contents of $code would be the "base" level.
• #18: Here's another example of metaprogramming. Many of you have cargo culted this for your Moose classes to make them faster. (Somehow!) Have you ever stopped to just look at this? Obviously the whole expression means to make the current class immutable.
• #19: There are two method calls. What is that first one actually doing?
• #20: In other words: What is $meta? Let's print it!
• #21: It's an object. Its class is Moose::Meta::Class.
• #22: Moose::Meta::Class is a class *for* classes. Like every ordinary class, Moose::Meta::Class has attributes and methods. An instance of Moose::Meta::Class *describes* some class. A class has a name, a list of superclasses, a list of methods. You can also create a new_object of a class, or add an attribute to a class, or get the complete list of roles the class does.
• #23: If we go back to the string eval example, we can see some parallels. Moose itself is like the first section, which is the meta level. Your class is similar to the contents of $code, which is the base level. In Moose, there is far more interaction between the two layers than in this example.
• #24: This is the Point class again, but with Moose. Before I can talk about *extending* Moose, you have to know a bit about how it works internally.
• #25: We now have the class metaobject for Point in $point. We could call make_immutable on it if we wanted to make Point faster. But instead, let's inspect it.
• #26: If we look at the name of the class metaobject, we'll get...
• #27: Point, which is the name of the class that $point describes.
• #28: And the list of attributes?
• #29: y and x. You might've expected x and y, since that was the order we added them to the class. However, the attribute and method lists are unordered. The canonical representation of these lists in Moose is a hash. The list methods just perform "keys" on that hash. This way we get free name collision resolution.
• #30: And finally does Point have a clear method?
• #31: Yes.
• #32: And next up we have the Point3D subclass of Point. It has an additional attribute 'z' and a method modifier on the 'clear' method.
• #33: Let's start by asking the same questions.
• #34: Name?
• #35: Point3D
• #36: What attributes does Point3D have?
• #37: Just one, z! get_attribute_list works based on the *local* class. It does not consider inheritance. When you're using metaclasses you need to know whether inherited entities are included or ignored.
• #38: Clear method?
• #39: Yes, Point3D does have a clear method.
• #40: Even though in this case "after clear" is just a method modifier, that is good enough for has_method. has_method, like has_attribute, checks only the local class, it ignores inherited methods.
• #41: For many metaclass methods, you have to know whether inheritance is considered. Generally, methods that ignore inheritance have shorter names than methods that include inheritance. The word "all" is a good indicator as well. You also need to know whether the method returns names or objects, though that's less easy to screw up. When you run your code the first time, the wrong choice will be immediately evident.
• #42: One of the uses of this sort of introspection could be a REST interface. In addition to the usual CRUD operations, you could dump the class to YAML or JSON as documentation. Another program could use that description to generate classes. You have all this data available when you write classes with Moose, so reusing it in your application is far better than declaring it several times. Don't repeat yourself.
• #43: Don't get the wrong idea about this slide! I'm going to define the Point class again, but without actually using the Moose sugar. I want to demonstrate that metaclasses do more than just describe classes, we can change them too. And Moose users do that all the time when they use the friendly functions like "has"
• #44: We start by creating the metaclass. Note that the name is the first parameter with no hash key. This interface sucks, we know. :(
• #45: Then we set the superclass. If we don't do this then we'll get no "new" method. This is one of the things "use Moose" does transparently for us.
• #46: The "x" attribute. Again we pass the name without a key, but the other options are the same.
• #47: The "y" attribute. It's the same as x but for the name, so maybe a loop would be better.
• #48: And finally the clear method. All of Moose's sugar functions are thin wrappers for metaclass methods. "has" and its friends actually form a very small part of Moose.
• #49: These are all the important entities that Moose provides.
• #50: We've seen that Moose::Meta::Class objects each describe a class.
• #51: We also have classes for all the other interesting things in the program. Moose's devs like OO, so it makes sense that Moose itself is designed using object-oriented programming.
• #52: We can get the attribute metaobject with the "get_attribute" method. get_read_method returns the name of a method that can be used to read the attribute's value. We call the method it returns ("x") on an instance of Point.
• #53: There are also method metaobjects. There's not much to them though. All you can really do with methods is invoke them.
• #54: There are two steps to extend Moose. You extend a particular metaclass with subclassing or role application. Then you just use it. We definitely prefer role application over subclassing. That lets extensions compose. If everything subclassed, then you'd only be able to use one extension at a time.
• #55: We want to create a class that counts the number of instantiations.
• #56: We start by defining a role that has a counter attribute and a method to increment it. This is a role that you could consume in your ordinary classes. It's not special in any way.
• #57: Now we define a new role that we expect to apply to Moose::Meta::Class that hooks up the counter role to the "new_object" method.
• #58: Now all we have to do use tell Moose we want to use this role on the metaclass. After the special -traits option to use Moose, the rest is just the same. The words "trait" and "role" are mostly synonymous. There are vague differences but not worth covering. When you're adding roles to metaclasses in Moose, they're called traits.
• #59: We can inspect the count which is an attribute of "Point->meta".
• #60: Point->new is a Moose::Object method that enters the meta-level, hence the red arrow. The black arrows indicate method calls that don't traverse the base/meta boundary.
• #61: I want to introduce another class, Line. This does NOT use the CountInstances trait.
• #62: The Line class's instances are not counted. The major point is that one class's extensions do not affect other classes. We're not monkeypatching in a method modifier or attribute or anything like that. With monkeypatching you could destroy Moose's own workings by redefining a particular method it uses internally. This is Perl, we love to design modular, robust systems. Monkeypatching is the antithesis of that ideal.
• #63: We can also apply roles to other metaclasses, such as attributes. Many of you have used MooseX::AttributeHelpers; it works by extending the attribute metaobject. "provides" is an attribute of the AttributeHelpers Counter role which ultimately uses "add_method" on your class.
• #64: We're writing a web application so it's important to know how to render every attribute as a form field. We'll use a role to let the application writer (usually you!) declare what type of <input> element each attribute uses.
• #65: We're writing a user class. The name is usually a short string of characters, so a regular text element makes sense for it.
• #66: For a password, we want to obscure the value of the element so that someone looking over your shoulder can't steal your password.
• #67: And finally we let the user explain all about themselves in a long biography textarea.
• #68: That's a lot of redundancy. We know that when we're writing a web application, most classes will eventually be rendered as a form some day.
• #69: Ideally we'd be able to say just how each field is rendered, not that it is going to have a particular field type. This problem is more pronounced when you are using many metaclass extensions together.
• #70: We have a solution that comes in two modules. These modules are the workhorses of extending Moose. They were written by Dave Rolsky after he realized that Moose extensions were not as composable as we wished. These modules let you do the right thing easily, which is the goal of every well-designed module.
• #71: Moose::Exporter is a module that lets you write modules like "Moose" and "Moose::Role". These modules create a class (or role) metaobject, and provide for the user some sugar functions. Moose and Moose::Role are themselves implemented with Moose::Exporter.
• #72: MetaRole is a module that lets you apply roles to the various metaclasses in your class. You use it inside of Moose::Exporter.
• #73: Here we're defining a module MyWeb::OO that people use instead of Moose itself. We have to load a bunch of stuff, including the role we're going to apply to every attribute. We then setup import methods. This particular invocation makes MyWeb::OO provide all of the Moose sugar. You could add new functions to that if you wanted to.
• #74: Here is the good stuff, inside the init_meta method. This is called to construct a metaclass for the user of MyWeb::OO. We let Moose's own init_meta do the heavy lifting. We then change it slightly so that FieldType is automatically applied to attribute metaclasses. This is a pretty decent amount of code, but it's all well documented and could be abstracted further if you wanted to make this common case require less typing.
• #75: Let's revisit the User class now that we have this shiny new exporter. We've extended the attribute in a useful way. The user doesn't need to know the traits invocation. They don't need to know about all this metaprogramming stuff. They just use this new option to "has" as if it were in Moose from the start.
• #76: Another important point of Moose::Util::MetaRole is that it composes with other extensions seamlessly. You could write all of these Moose extensions that inject all sorts of roles into the various metaclasses. For example, Persistent would inject both class and attribute roles, and provide a database_name keyword.
• #77: Though it served as a useful launching-off point, immutability is the most irritating thing about extending Moose. If your extensions affect object construction or accessors, then you will probably need to care about immutability.
• #78: One of the ways we make Moose faster is by string evaling constructors and accessors. That certainly makes Moose faster, but for the .1% of users who want to extend Moose, it sucks. You need to hook the methods called here to add the string of code that you need. You can also turn off immutabilization, but that slows the class down. Damned if you do, damned if you don't.
• #79: These are some projects that extend Moose to great effect. If I've convinced you that domain-specific metaprogramming is awesome, you should read through their source.
• #80: I'd also like to give a shout-out to Pascal Costanza's ContextL project. This is a pretty serious extension of CLOS's metaobject protocol. It provides things like layered classes where layers are dynamically turned on and off which alters the class definition. It's really neat and worth looking at.
• #81: Roles themselves would not have been easy or clean without Class::MOP's metaobject protocol.
• #82: Finally, if you really like this stuff, get this book. Alan Kay, inventor of OO, said "This is the best book anybody has written in ten years". He also says it's a very hard book to read because of its Lisp-centric nature, but hopefully that isn't too big a stumbling block. It's just an abstract syntax tree!