MLUC 2011 XQuery Enigma

XQuery’s Enigmatic Information Architecture Role MarkLogic User Conference 2011Peter O’Kellypeter@okellyassociates.com

BackgroundWhere I’m coming fromIndustry analyst/consultant working in information management and collaboration domains for ~30 yearsBackgroundApplication developer and data architectProduct management and strategy roles at Lotus, IBM, Groove Networks, Macromedia, and MicrosoftIndustry analyst/consultant with the Patricia Seybold Group and Burton Grouppbokelly.blogspot.com4/28/2011© 2011 O’Kelly Associates3

BackgroundMy high-level XQuery perspectiveXQuery truly is awesome…A very well-designed language and standards initiative, optimized for un- or semi-structured informationBut XQuery appears to be somewhat stalled, in terms of overall market momentumIt’s important to understand and address the reasons for the stallBecause a vibrant XQuery standard, along with related techniques and tools, are important for the evolution of information management4/28/2011© 2011 O’Kelly Associates4

Why XQuery is AwesomeA purpose-built XML content manipulation languageGracefully applying the joy of sets to XML contentOffering a sustainably complementary fit with SQLDesigned by experts including SQL co-author Don ChamberlinEvolving to go far beyond queriesWith search, conditional expressions, function libraries, and moreCan replace a kitchen sink of earlier technologiesFewer moving parts means more simplicity and less maintenanceA W3C Recommendation, building on XML Schema, XPath, and other standards4/28/2011© 2011 O’Kelly Associates6

Other Evidence of Non-MainstreamingVendor uncertainty or other hesitationE.g., at Gilbane Boston 2010, few of the exhibitors I spoke with had even heard of XQueryNot a statistically significant survey, but still surprisingFew of the current market-leading collaboration/ content platforms are based on XQueryTangent: this suggests there is a compelling market opportunity for new collaboration/content entrants that are XQuery-based4/28/2011© 2011 O’Kelly Associates14

Exploring the XQuery EnigmaSome issues that have probably limited XQuery market momentumLack of a big-picture frameworkInstalled base inertiaStandards and politicsThe Internet ethosLimited techniques and tools[Don’t panic! We’ll return to the future-optimistic themes in a few minutes...]4/28/2011© 2011 O’Kelly Associates15

A Big-picture FrameworkA digital information item dichotomy ResourcesDigital artifacts optimized for human comprehensionOrganized in terms of narrative, hierarchy, and sequenceExamples: books, magazines, documents (e.g., PDF, Word), Web pages, XBRL documents… RelationsApplication-independent descriptions of real-world things and relationshipsExamples: business domain databases, e.g., customer, sales, HR…4/28/2011© 2011 O’Kelly Associates16

A Big-picture FrameworkComplementary levels of modeling abstractionConceptualTechnology-neutralUsed to establish contextual consensus Also very useful, when done well, for creating logical modelsLogicalCaptures conceptual models in a technology renderingExamples: (beyond-the-basics) hypertext and relationalInformation workers and app developers ideally work at this level of abstractionPhysicalIncludes implementation-level detailsIdeally, activity at this level is limited to system architects and administrators4/28/2011© 2011 O’Kelly Associates18

The Lack of a Big-picture FrameworkWithout a framework, there’s likely to beUncertainty about what to use whenConflict based more on miscommunication and/or misunderstanding than real issuesInsufficient focus on Application/data independenceConceptual/logical/physical model independenceLow probability of appreciation for the sustainable and complementary fit between XQuery and SQL 4/28/2011© 2011 O’Kelly Associates21

Installed Base InertiaIncumbent vendorsDBMS vendorsApplication vendorsLarge organizations usually have distinct “content” and “data” management groups, often with little collaboration between themContent-focused people are often more instance-oriented and care a lot about schema flexibilityDatabase-focused people are generally more type-oriented and care a lot about schema precision4/28/2011© 2011 O’Kelly Associates22

Installed Base InertiaProgrammer preferences can be perniciousObject-oriented frameworks have a lot in common with resources (e.g., hierarchy, sequence, and positional navigation)The object/relational “impedance mismatch” is still irksome, in some tools/frameworksBut that does not mean it’s reasonable to default to resource-oriented approaches for all domains, even if the application is XML-centric, because not all XML content is resource-centricDoing so can dumb-down DBMS usage patterns, with significant consequences4/28/2011© 2011 O’Kelly Associates25

Standards and Politics“The nice thing about standards is that you have so many to choose from” Andrew TanenbaumAs in the development of SQL, there are complex challenges at the intersection of standards groups, vendor agendas, and academic prioritiesThe Open XML/ODF debate is another recent, relevant, and revealing case study4/28/2011© 2011 O’Kelly Associates26

Standards and PoliticsNoSQL“A rhetorically clever and manipulative name … Saying ‘NoSQL’ says what you’re against, not what you’re for” (Joe Maguire)As with the largely failed “object database” wave 20+ years ago, NoSQL extremists appear to underestimate the expressive power and utility of what they propose to displaceWhile there is ample room for database-related innovation, polarizing the debate is unhelpful4/28/2011© 2011 O’Kelly Associates27

The Internet EthosLots to likeOpen, community-driven, vendor-independent…But also some risks; e.g., the InternetDoesn’t complain if your system is inefficient or ineffective Is culturally conducive to cyber-polarizationE.g., there are probably still lively Internet forum debates about the relative merits of DTD, Schematron, RNG, and XML Schema And xBASE versus SQL, and RPG versus COBOL…This creates a key challenge: it’s difficult to get vitality readings on standards and technology alternativesIncluding major initiatives such as XQuery and XHTML 2.04/28/2011© 2011 O’Kelly Associates28

Limited Techniques and ToolsSome SQL reality checks Relatively few people work directly with SQLThe vast majority of information workers and developers who benefit from using SQL do so indirectly, through tools ranging from IDEs to query/reporting applicationsThe development of ODBC was pivotal for software vendors and application developers working with RDBMSsMaking it possible for them to use a single interface model for multiple products4/28/2011© 2011 O’Kelly Associates29

Limited Techniques and ToolsXQuery's market uptake has been constrained by the small number of XQuery-based tools and applications Which is in turn limited in part by the lack of a successful ODBC equivalent for XQueryWhich, in turn, is partly a function of Microsoft’s apparent XQuery ambivalence Many XML-focused developers believe they get most of what they need from XPathWithout tools to promote effective use of XQuery, it’s a difficult value proposition to make4/28/2011© 2011 O’Kelly Associates30

Limited Techniques and ToolsModeling techniques and tools are also pivotalThere are some good options today for physical database modelingBut few choices for logical modelingAnd almost a complete lack of conceptual modeling toolsFor XML information modeling, there are even fewer modeling technique/tool options todayIt’s also a cultural and incentive system challengeIf developers are paid to primarily focus on physical models, that’s what most of them will do4/28/2011© 2011 O’Kelly Associates31

Limited Techniques and ToolsMany XML-focused developers appear to believe they don’t need to invest time and attention in modelingIn part because XML-focused application development often starts with existing XML schemas and/or documents rather than “green field” modelingBut modeling is equally applicable to resource and relation domains, forEstablishing contextual consensusHelping to promoteApplication/information independenceConceptual/logical/physical model independenceFostering the effective application of set theory and maximizing the use of declarative expressions4/28/2011© 2011 O’Kelly Associates32

Projections and RecommendationsXQuery is going to be a mainstream successRDBMSs aren’t going away anytime soonThe standards scene is evolving in subtly significant waysMore and better modelingMarkLogic is very well positioned4/28/2011© 2011 O’Kelly Associates34

XQuery Will be a Mainstream SuccessAnd already is a success, for many progressive IT organizations and software vendorsThe next wave of XQuery momentum will likely come more from content management than traditional database management Providing significant opportunities to have fewer information architecture moving partsE.g., to spend less on specialized enterprise content management, records management, and Web content management servers and tools4/28/2011© 2011 O’Kelly Associates35

XQuery Will be a Mainstream SuccessRecommendationsLearn and fully leverage XQueryGo beyond the basics to master the full XQuery language“Querying XML,” by Jim Melton and Stephen Buxton, is a useful resource in this context Seek to simplify and consolidate, e.g., To do less scripting/programming and more declarative development using XQueryTo migrate content and apps from legacy ECM systems4/28/2011© 2011 O’Kelly Associates36

RDBMSs Aren’t Going AwayResources and relations are complementaryAnd XQuery and SQL offer very strong synergySystems such as Google’s Megastore are important leading indicators, as hybrid models“NoSQL” will rapidly evolve Initially implied “Just say ‘no’ to SQL”Later quietly redefined as “Not Only SQL”What may be next: “New Opportunities for SQL”I.e., some developers may reconsider the value of SQL and RDBMSs, after hitting NoSQL limitations4/28/2011© 2011 O’Kelly Associates37

RDBMSs Aren’t Going AwayRecommendationsDevelop expertise in both (beyond-the-basics) hypertext and relational modelsAnd explore the information flows between themProvide clear customer requirements and feedback to your RDBMS, application, and tool vendorsEncourage them to fully exploit resource/relation synergyEstablish clear developer criteria on what to use when, e.g., for NoSQL alternativesConsider applying the framework presented earlier4/28/2011© 2011 O’Kelly Associates38

Subtly Significant Standards EvolutionThe industry is a very different place compared to when SQL was standardized in the mid-1980sThe Internet ethos is pervasive, and key vendors have learned to productively play the standards game togetherThere have been some major standards changes recently, e.g., the discontinuation of XHTML 2.0But there is also clear market momentum consolidation around standards including XML Schema, XPath 2.0, XSLT, and HTML5And, although not always obviously, XQuery 4/28/2011© 2011 O’Kelly Associates39

Subtly Significant Standards EvolutionRecommendationsPlace well-informed standards bets, regularly check assumptions, and be willing to make course correctionsGet involved Make your standards-related requirements clear to your strategic vendorsActively participate in standards activities4/28/2011© 2011 O’Kelly Associates40

More and Better ModelingConceptual, logical, and physical modeling are critical success factors for both resources and relationsOrganizations that under-invest in modeling are essentially reverting to the obsolete programs-have-files approach, limitingApplication/data (and content) independenceConceptual/logical/physical model independence4/28/2011© 2011 O’Kelly Associates41

More and Better ModelingRecommendationsDevelop modeling expertise Explore resources such as “Mastering Data Modeling” (Carlis/Maguire)Apply the big-picture framework for consensus on (resources + relations) * (conceptual/logical/physical)Build and consistently use model repositories Also ensure modeling and reuse are supported by developer incentive systemsProvide clear modeling-related requirements to your tool and server vendors4/28/2011© 2011 O’Kelly Associates42

MarkLogic is Very Well PositionedMarkLogic Placed an early bet on XQuery, and continued to focus on XQuery while many other vendors balked Has insights from XML information management market leadership in key domains including media, government, and financeIs led by a deeply experienced and strong teamRecommendationsShare your experiences this week and consider proposing a customer case study for MLUC 20124/28/2011© 2011 O’Kelly Associates43

MLUC 2011 XQuery Enigma

More Related Content

What's hot

Viewers also liked

Similar to MLUC 2011 XQuery Enigma

Recently uploaded

MLUC 2011 XQuery Enigma

Editor's Notes