Skip to content

build-your-own-parser #22

Open
0 of 4 issues completed
Open

Description

@dougwilson

If you look at the middleware, after I had refactored it, all they are is a call to typeis and a call to read. We should add something like bodyParser.generic() to let people roll their owner simple body parsers.

Activity

  1. self-assigned this
    on May 27, 2014
  2. jonathanong commented on May 27, 2014

    @jonathanong
    Member

    I think this lib is pretty easy to understand. IMO they should just fork this lib if they wanna do more stuff. Most of the logic should be in separate libs anyways

  3. Fishrock123 commented on May 27, 2014

    @Fishrock123
    Contributor

    I think it would be better to have the interface and people have parsing libs than lots of body-parser forks.

  4. dougwilson commented on May 27, 2014

    @dougwilson
    ContributorAuthor

    I think it would be better to have the interface

    This module should be the interface. json and urlencoded should be the separable libs that use this module ;)

  5. jonathanong commented on May 28, 2014

    @jonathanong
    Member

    You guys are just adding more responsibility to your plate hah

  6. Fishrock123 commented on May 28, 2014

    @Fishrock123
    Contributor

    Yeah, that's the issue, may be too much to be worthwhile to support.

  7. dougwilson commented on May 28, 2014

    @dougwilson
    ContributorAuthor

    i.am.machine :DDD

  8. mikermcneil commented on Jul 29, 2014

    @mikermcneil
    Member

    This module should be the interface. json and urlencoded should be the separable libs that use this module ;)

    I can tackle this if it's helpful

  9. dougwilson commented on Jul 29, 2014

    @dougwilson
    ContributorAuthor

    This is actually nearly done, as it was intended for the goal of bringing body-parser back into express core.

    Out of curiosity, what parser were you planning to build that we don't have already :)?

  10. mikermcneil commented on Jul 29, 2014

    @mikermcneil
    Member

    Out of curiosity, what parser were you planning to build that we don't have already :)?

    good question

  11. mikermcneil commented on Jul 29, 2014

    @mikermcneil
    Member

    xml

  12. mikermcneil commented on Jul 29, 2014

    @mikermcneil
    Member

    lol

  13. dougwilson commented on Jul 29, 2014

    @dougwilson
    ContributorAuthor

    xml

    lol. Somehow I knew that was the answer ;) Currently the best you can do is to use bodyParser.text and then feed the text into a XML parser.

  14. dougwilson commented on Jul 29, 2014

    @dougwilson
    ContributorAuthor

    The reason why I had a feeling is because I use this module to parse XML all the time, but of course using text + parser requires the request body to buffer up instead of feeding it to an incremental parser (CSV is another common one, which I also use!).

    This new stuff will actually be out sooner rather than later since I should not be distracted with express core.

  15. 2 remaining items

  16. dougwilson commented on Aug 1, 2014

    @dougwilson
    ContributorAuthor

    @rlidwka or, looking at your code now, you could technically wrap bodyParser.json and if it errors, check err.body and parse that with the json5 parser :) I'm not actually saying this invalids the issue at had, here, because it's still valid and I'm working on it, haha.

  17. lopugit commented on Jun 12, 2019

    @lopugit

    How do you build may I ask?

  18. jonchurch commented on Feb 14, 2025

    @jonchurch
    Member

    Reviving this issue as a more central place to discuss all the Generic parser work.

    tldr;

    Im in favor of creating a generic parser middleware interface folks can extend to create custom middlewares for their advanced usecases. However, Im against exposing an opts.parser function on existing parsers which users can use as an escape hatch

    The Current PRs

    We have several PRs open right now to land a Generic implementation. I don't want to leave feedback on all of these, or the additional closed ones, so Im writing this comment instead. cc @wesleytodd @UlisesGascon @ctcpip @Phillip9587

    The existing approach here exposes an opts.parser option, which I don't want to accept.

    Flexibility vs Simplicity

    I'm open to offering a Generic interface for custom parsers, but I don’t believe we should expose an opts.parser option in our existing middleware parsers. I prefer we export well known and well tested parsers which are hard to muck up and simpler to reason about for us and users. (We do that already with urlencoded extended, and I've read enough issues to know that confuses people.)

    The goal of body-parser (IMO) is to provide simple, reliable parsers for common HTTP body formats like JSON and urlencoded data, covering common usecases therein. By sticking to common use cases and avoiding overcomplication, the core middleware is easier for users to adopt and maintain long term.

    Extending a generic interface to create a more custom parser is a viable way to provide users flexibility while keeping the simplicity of the core middlewares.

    Customization needs are rare compared to the more common issues reported, which suggests that the current parsers cover most users' needs.

    We’ve seen a small but recurring set of issues where users want to extend or change the behavior of existing parsers. Examples include:

    Most of these requests represent edge cases or specialized needs, which is why I think a Generic parser interface is the best way to address them without overcomplicating the core parsers.

  19. Phillip9587 commented on Feb 18, 2025

    @Phillip9587
    Member

    I agree with @jonchurch that we should focus on maintaining and improving our core parsers while allowing users to create their own custom parsers as needed. Keeping the core parsers simple and reliable should remain our priority.

    Additionally, I think we should start over rather than trying to rebase the existing PRs. A fresh approach would give us a cleaner foundation to work from and ensure we implement this in the best possible way.

    That said, there is #551, which introduces the normalizeOptions function. This PR standardizes and validates common parser options like inflate, limit, type and verify, ensuring consistent defaults and reducing duplication across the parsers. I think we should merge it as foundational work, which would help keep the scope of this effort smaller.

    Looking forward to hearing everyone's thoughts!

  20. constantind commented on Jan 9, 2026

    @constantind

    In my opinion should be enough to provide the custom JSON.parser, because the default JSON.parse is blocking in V8 seems to be getting very slow since nodejs 20 with increased message size and concurrency, such that folks can try BigFriendlyJSON, StreamableJSON, multi-threaded Json parsing, node-rapidjson, yeldable-json and any other way to make parsing non blocking (
    #132 ), run in parallel and overcome the JSON issue: https://nodejs.org/en/learn/asynchronous-work/dont-block-the-event-loop#blocking-the-event-loop-json-dos

  21. Guthib-of-Dan commented on Apr 30, 2026

    @Guthib-of-Dan

    Hello, I should mention JSON.parseBinary here and ArrayBuffer.prototype.detach as well as in “use-non-blocking-parser”, as by integrating them body-parser would not need to implement custom JSON parser
    #132 (comment)

  22. Guthib-of-Dan commented on May 4, 2026

    @Guthib-of-Dan

    @Phillip9587 @dougwilson I would like to know your opinion and revive this discussion (or finish it in the context of json).

  23. dougwilson commented on May 4, 2026

    @dougwilson
    ContributorAuthor

    Hi @Guthib-of-Dan sorry I haven't been involved in this project for many years now. Just wanted to reply so you didn't feel like you were left hanging.

  24. Guthib-of-Dan commented on May 4, 2026

    @Guthib-of-Dan

    That is ok @dougwilson , but idea will benefit JS overall, not only this project. So if you are interested, you can still look) I am looking for support right now to make “final steps” towards delegates, which is why I would appreciate all opinions, be it from body-parser or elsewhere

  25. dougwilson commented on May 4, 2026

    @dougwilson
    ContributorAuthor

    No worries. I haven't done JS in just as long now 😅 . So much has changed idk anymore.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions