Api Design

Editor's Notes

• #2: Presented YAPC::Asia, 2009-09-10. Tokyo Institute of Technology, Tokyo, Japan.
• #3: There is a good Google Tech Talk on "how to design an API and why it matters". There isn't a whole lot of overlap between this talk and that one. Watch that one. http://www.youtube.com/watch?v=aAb7hSCtvGw
• #4: At YAPC::NA 2009, this guy, Hans Dieter Pearcey aka confound aka hdp, presented a talk about Dist::Zilla. http://www.flickr.com/photos/nuffin/179250512/
• #5: Dist::Zilla was written by this other guy, Ricardo Signes, aka rjbs. http://www.flickr.com/photos/nuffin/179250812/
• #6: Dieter presented this slide about Dist::Zilla's pluggable design. I loved it and I wanted to devote an entire talk to its glory. http://weftsoar.net/~hdp/dzil/
• #7: I'm here to highlight really cool API designs that these projects have. In particular, they design for extensibility and pluggability. Extensibility is really important to the current and future success of these projects.
• #8: If you haven't noticed yet, this talk is going to be very Moose-heavy. All those modules have the Moose nature. http://www.flickr.com/photos/redune/6562798/
• #9: There is a poorly kept secret for designing great APIs. I hope that all of you already do this, but you probably do not do it enough.
• #10: Write tests.
• #11: Write so many tests your ears bleed. I am not joking!
• #12: If you remember nothing else, remember to write tests!
• #13: Write tests so you can tell if your API is painful to use. Which of these would you rather be stuck with? Make it painless for your users. Some of them might be using your module a lot. If it's tedious to use your module...
• #14: ... then you'll piss your users off. They'll leave and use some other module, or worse, find out where you live.
• #15: This is Jesse Vincent, the nicest guy in the world :)
• #18: Moose serves as the foundation for the rest of the talk, so I want to explain what it "got right" in terms of its API. These next few slides are difficult but it will get clearer and less heady, so wake up soon if you space out.
• #19: Moose is built on top of a metaobject protocol. This is Class::MOP. See my "Extending Moose for Applications" talk for a proper introduction to the metaobject protocol http://sartak.org/talks/yapc-na-2009/extending-moose/
• #20: The MOP is vital to Moose's operation. Basically, it means that every part of your class is represented by an object.
• #21: When you say "has" it creates an instance of the Moose::Meta::Attribute class, which holds information like the attribute's name, its type constraint, default value, etc.
• #22: The is => 'ro' option creates a "cache" method in your class. It also creates an object of class Moose::Meta::Method::Accessor to represent that "cache" method.
• #23: This is important because we can subclass Moose's class to add our own special logic, such as making the cache persist across processes. Subclassing and adding logic is ordinary object-oriented programming!
• #24: We can also specify roles to apply to cache's attribute object. This is slightly better because it means a single attribute can have many extensions. Just like how it's better to design with roles than subclasses in ordinary programming.
• #25: The metaobject protocol powers most of the MooseX modules. In my opinion, the metaobject protocol is responsible for a very large part of Moose's popularity. The other reason for Moose's popularity is it enables concise class code.
• #26: Moose also makes a very clean separation between its sugar layer and the rest of the system.
• #27: Say you wanted to get ahold of some class...
• #28: Then add an attribute to it. This doesn't work because "has" is not a method. Its first parameter is supposed to be the attribute name, not the class you're adding the attribute to.
• #29: So we have to call $class's "has" as a function. This kind of thing is ridiculous. Maybe the other class has used "no Moose" so that "has" is deleted. Or perhaps it renamed "has".
• #30: Not to mention how ugly this mess is.
• #31: If we look at the source code of Moose, we can see "has" is basically a wrapper around the "add_attribute" method of the Class::MOP::Class instance.
• #32: Much better. There's no messy syntax. This can be used outside of $class's namespace just fine. This also works if class has cleaned up after Moose with "no Moose" or namespace::clean.
• #33: Having a clean sugar layer means that other people can write better sugar. I like the idea of providing a separate Devel::Declare-powered sugar layer in a separate distribution. It forces you to cleanly separate the pieces.
• #34: Path::Dispatcher is a standalone-URI dispatcher. I wrote it because I wanted Jifty::Dispatcher for Prophet's command-line interface. This is its sugar layer. Like Moose, it has a clean, extensible API if you want the freedom to do unusual things.
• #35: It used to be that Path::Dispatcher::Declarative was implemented as an ordinary Sub::Exporter-using module.
• #36: This is not at all extensible. You can't change the meaning of "on" or "under" because these are hardcoded. Reusing this sugar would be painful as well.
• #37: This was fine for a few weeks, but then Robert Krimen started using Path::Dispatcher. And he wanted to extend it for a module he was writing called Getopt::Chain.
• #38: Path::Dispatcher::Builder makes the sugar layer creation use OOP. This let Robert subclass Path::Dispatcher::Builder and use it for his own modules. He can reuse the regular dispatcher logic, tweak it by overriding methods, and add his own behavior.
• #39: OO sugar is a really neat idea that I haven't seen anywhere else.
• #40: HTTP::Engine abstracts away the various HTTP server interfaces that Perl has accumulated since HTTP was invented. The benefit is in letting the user pick which server interface best fits their particular needs.
• #41: For example, you can use mod_perl if you enjoy pain.
• #42: Or FastCGI if you're a cool dude.
• #43: HTTP::Engine works well because the code you write doesn't have to worry about redirecting I/O streams, making sense of %ENV, or any of the other crap you do when writing against a particular server module.
• #44: HTTP::Engine boils the web server cycle to the least common denominator. You take a request...
• #45: … and return a response.
• #46: Can we please standardize on this? New server modules can implement an HTTP::Engine::Interface, then immediately every existing HTTP::Engine-based application can switch to it by changing only a single line of code.
• #47: Now I want to explain why this is so awesome.
• #48: Here's a list of plugins used by a typical Dist::Zilla-based distribution.
• #49: Dist::Zilla itself occasionally calls methods like this. The key bit is "plugins_with".
• #50: plugins_with takes a role name...
• #51: ...and selects the plugins that "do" the role. These plugins all do the "FileGatherer" role, which means the plugin adds files to a distribution.
• #52: Then, dzil calls gather_files on each of these plugins so it can actually add files to the distribution. "License", "Readme", and "MetaYAML" add the respective files, "AllFiles" adds every file the author wrote. "PodTests" adds pod testing files to the distribution.
• #53: Dist::Zilla uses this architecture for all of the interesting parts of building a CPAN distribution. This is "munging files", which lets plugins edit files to increase the version number, or move tests around.
• #54: It turns out that RT has a very similar extension mechanism.
• #55: This code exists in User/Prefs.html. The callback method selects all plugins that do the "User/Prefs.html" "role".
• #56: Then it calls the FormEnd "method" (template) on these selected plugins.
• #57: And you can pass arbitrary parameters to each method.
• #58: This works extremely well for us! We try to build most customer extensions with callbacks. It's basically the same design as Dist::Zilla's.
• #59: RT has had callbacks since 2002, first released in 3.0.0. This pattern has been the best mechanism for any kind of RT extension for almost seven years now.
• #60: This design gives the user choice over which behavior she wants. And in my experience, users really really want choice.
• #61: This design is also extensible for free. These are some of the modules that have been written to extend Dist::Zilla.
• #62: All they need to do is fulfill the requirements of the roles they "do". I'm going to talk about that more in my (Parameterized) Roles talk. http://sartak.org/talks/yapc-asia-2009/(parameterized)-roles/
• #63: Extensibility is also important for code you can't share. We can't ask Ricardo to include company secrets for Dist::Zilla, and maintaining a fork really sucks.
• #64: So now you know!
• #65: IM::Engine is a project I'm working on. It's basically HTTP::Engine for IM. You can write a bot, once, that will run on any service IM::Engine can talk to, including IRC. IM::Engine smooths over the differences in the protocols.
• #66: I've extended Ricardo's design with a number of helper methods. plugin_collect is the one I like best.
• #67: For each plugin that does the ExtendsObject::User role...
• #68: ...call its "traits" method.
• #69: The return value of this call is the list of all return values of the "traits" methods.
• #70: This is the important part of plugin_collect's implementation. There's not much there. I like very layered APIs because they're easier to understand and reuse, especially by your users, than huge monolithic methods. Each layer does only a little bit of work.
• #71: Here's a piece of design I like a lot. This lets plugins participate in object construction. Each plugin can provide constructor arguments.
• #72: This lets plugins participate even more in object construction. Now plugins can provide roles for the object you're constructing. This lets plugins add attributes and methods to the object. I use this in a plugin to give state management methods to User objects.
• #73: new_with_traits comes from MooseX::Traits. It's a really nice module for designing pluggable and extensible systems. You just pass a list of roles to new_with_traits and it will arrange it so that the object does those roles.
• #74: Other objects of that class are not affected by new_with_traits. The way it works internally is by creating a new subclass of Class. This is vital because it maintains modularity. I don't want my extensions to screw up your extensions.
• #75: In Moose land, roles and traits are basically synonymous. Some people will tell you there are subtle differences, but there's no clear consensus. I just say "roles" except when I have to say "traits" for a module.
• #76: So that is all I have time to cover. There are plenty more nice examples in modules like KiokuDB, Fey, and the now-moosified Catalyst.
• #77: Moose teaches us that extensibility can lead to a great corpus of extensions. Separation of sugar keeps you and your users flexible.
• #78: The OO sugar layer is a new idea that I hope catches on. I'll have to dedicate more time to it.
• #79: If you omit inconsequential details, then your application remains flexible and concise.
• #80: Pluggability does not have to be implicit, as in subclassing. Explicitly controlling pluggability lets you do more interesting things.
• #81: … such as the things IM::Engine does, by letting plugins manipulate system objects. It also provides methods for common plugin operations so you don't have to repeat them everywhere.
• #82: I almost forgot...
• #83: I almost forgot...
• #84: Thank you to Ishigaki-san for translating my slides!