line number mapping for debug

[pmuellr]

There's been some email chit-chat regarding support coffeescript in the existing browser debuggers. The basic idea is to include information in the generated JavaScript files that the debuggers can make use of to display the original coffeescript source, and deal with line mapping issues between the CoffeeScript source and generated JavaScript.

Thought I'd go ahead and open up an issue for discussion.

[pmuellr]

So some thoughts:

• CoffeeScript source location - somehow we need the coffeescript source. See Issue add compile option to output a "//@ sourceURL=foo" comment #557 as one possible solution. The debugger would then be responsible for finding the source given that "URL". Java current just uses the basename() of the file as the file name in it's debug info, but of course it also has the full class name available to it as well. Not sure how we might get some "relative" name in there, so it seems like basename() or abspath() of the input file are the choices. abspath() exposes more information than it should, basename() will cause naming conflicts for same-named files in different directories
• line number information - The debugger will eventually want this in hash tables, both directions. Go from a cs line # to js line # when setting breakpoints, and js line # to cs line # when the JS vm is communicating to the debugger (eg, "stopped at line 47"). Presumably the actual data for the line number data could be in JSON format (eg, an object literal mapping cs #'s to an array of js #'s).

[mrblonde]

Might be worth looking at this also:
http://nex-3.com/posts/92-firesass-bridges-the-gap-between-sass-and-firebug

Maybe this same type of strategy could be used so that firebug could step through javascript line by line, but display the coffeescript code instead.

Tough project, but could turn into something pretty killer!

[jashkenas]

I'd be glad to help someone out with this ticket, but I'm not going to be spearheading it myself ... so closing as frozen for the time being. If someone wants to step up and do a port of FireSass for CoffeeScript, do let us know.

[jashkenas]

Alright -- this ticket now has life again, in Firefox at least:

https://bugzilla.mozilla.org/show_bug.cgi?id=618650

Reopening...

[subtleGradient]

Quick helpful workaround:

Include each line of the original coffeescript source inside a block comment before the js code that it generated. You could include the original coffeescript source lune numbers in the comments.

That would at least help simplify the learning and debugging process.

[StanAngeloff]

Mascara has a source-line mapping utility that only works in Firefox. It's quite possible to implement something similar:

try
  console.log 'Hello'
  a for i in [0..10]
catch error
  lines = { 31: 3, 32: 3, 33: 3 }
  error.message = error.message + ' on line ' + lines[error.stack.split('\n').shift().split(':').pop()]
  throw error

Run on Try CoffeeScript. Firefox only, though.

[andrewschaaf]

Related code:

Closure Compiler (JS -> JS)

...can generate source-to-source mappings:

java -jar $CLOSURE_JAR --js example.js --create_source_map ./map --js_output_file compiled.js

SourceMapLegacy.java has comments describing the format.

_PYXC-PJ (PY -> JS)_

...can generate mappings using the same format, but with more file info: [{"sha1":"...",...}] instead of []

commit: "changed emit() return type from a string to list of strings and kids"

commit: "source-to-source mappings! (same format as Closure Compiler)"

** exception-server **

An exception logging/viewing server has been written (in NodeJS) that uses these mappings to convert an exception's stack information into pre-compilation code snippets with TextMate URLs.

(your compilers/scripts send the server (code_sha1, mapping)s and (sha1, code)s at compile-time)

I'll post it [REDACTED TIMEFRAME] when it's more documented/polished.

** common-exception **

...can extract file/line/[col] information from NodeJS, Firefox, Chrome, Safari

[lucaswoj]

Crazy idea: What if when the CoffeeScript was compiled to JavaScript, it was done such that the lines corresponded 1:1 by placing multi-line JavaScript conversions on a single line.

[pmuellr]

I suspect it is crazy, but it would be useful. I actually do that with https://github.com/pmuellr/scooj , but only because the source is largely unprocessed JavaScript anyway. The nice thing is that even though you see some grungy code in the debugger, you do know the exact line number to go to in your editor.

[alexandertrefz]

I suggest a --debug coffee parameter that does something like this:

# Add Core-Utils to Underscore Namespace
_.ducktype = (obj, methods...) ->
  some("code")

to become:

var __slice = Array.prototype.slice;
// Line 2
_.ducktype = function() {
  var methods, obj;
  obj = arguments[0], methods = 2 <= arguments.length ? __slice.call(arguments, 1) : [];
  // Line 3
  return some("code");
};

this would work Browser independent and i think(im not a coffescript compiler expert, though) it could be implement without big hassles.

[andrewschaaf]

Let's define some terms:

Def a "compiler" compiles a non-empty set of "source file"s to exactly one "compiled file"

Defs

• a "(coord to coord)-mapping" is a function from (line, col) in the compiled file to (source_file_ref, line, col)
• a "(line to line)-mapping" is a function from (line) in the compiled file to (source_file_ref, line)
• a compiled file is "{coord,line}-{coord,line}-commented" if it contains comments in some format which provide a ({coord,line} to {coord,line})-mapping
• a "(...mapping...) file" is a byte-string in some format which contains a representation of a (...mapping...)

Notes:

• Don't worry, the representations of these mappings are usually much more space-efficient than the ultra-trivial {coord1:info1, ...}
• source_file_ref should include sha1(source_file) if possible, because that's really useful when used with my not-yet-released exception server

Defs

• "CCMF" is a (coord to coord)-mapping format. It's what Closure Compiler generates
• "CCMFH" is a backward-compatible extension with required hashes and optional other info

See my writeup of CCMF[H]: https://gist.github.com/769845

[andrewschaaf]

.lineno

Recall: adding .lineno to each node instance is a solved problem:

#955

With no side effects, that grammar.coffee change could be merged at any time.

[andrewschaaf]

Let's get this done! (Please? Please? Please?)

CoffeeScript should support both

1. {coord,line}-line-commenting in whatever that format turns out to be (e.g. via --line-commenting)
2. exporting CCMFH files (e.g. via --ccmfh-out=)

(1) might need to wait for some format finalization, but (2) does not.

And the nodes.coffee work required for (2) will apply to (1).

I'm trying to do a FOSS triple-launch in [REDACTED TIMEFRAME]:

• a NodeJS web framework
• a coherent Freedom Box attempt
• an exception server with awesome mapping support and TextMate links to your uncompiled source files
  • Screenshot of a mapping-following representation of an exception that occured while NodeJS was running code compiled by PYXC-PJ from Python syntax

(all three are written in CoffeeScript)

...and it would be really, really awesome to have CoffeeScript CCMFH support before then so I can integrate it.

Imagine being able to hit Command-R and have jashkenas/coffee-script-tmbundle compile, invoke node, and then represent the resulting exception in a beautiful and convenient mapping-following way with CoffeeScript source links and code snippets...

[andrewschaaf]

Some modest proposals

The Good (but more complex and possibly slower)

Something like what PYXC-PJ does.

EDIT: e.g.

Throw
  compileNode: (o) ->
    ...
    something(o, ["throw ", @expression, ";"])

Instead of having the main compilation function return a string, have it return a BAR.

BAR ::= ELEMENT-list

ELEMENT ::= string | FOO | BAR

FOO ::= something which can
    1. provide the code fragment for that subtree
    2. provide the ((line, col) -> (source line))
       mapping information for that fragment

Some function can then use the resulting BAR to generate both

1. the usual JavaScript
2. a CCMFH file

PYXC-PJ commits:

• "changed emit() return type from a string to list of strings and kids"
• "source-to-source mappings! (same format as Closure Compiler)"

I'm not a nodes.coffee expert (yet) so I'm not sure if subnode.compile(o) can be deferred. If so would let FOO be, e.g., {subnode:@expression, o:o}

Would o have been destructively modified by the time foo.subnode(foo.o) gets run?

I think The Good could be done without deferring .compile, but that might be harder.

The Bad (but simple (though verbose) to implement and possibly faster)

Something like what Closure Compiler does.

Instead of the beatifully concise current compileNode methods, do something like:

    something.startNode this
    something.add           "throw {"
    something.addSubnode    @expression, o
    
    # addSubnode: (node, o) ->
    #   ...
    #   node.compile o
    #   ...
    
    something.add           ";"
    something.endNode()

This would invoke the .compiles in the same order as the current implementation.

The Fugly

Have whatever methods invoke compileNode replace ([...].compileNode([...])) with

((() ->
    token = newUniqueToken()
    (
        "/*FUGLY:#{token}:@lineno*/" + 
        [...].compileNode([...]) +
        "/*/FUGLY:#{token}*/"
    )
)())

...and write a crazy function that uses the resulting text monstrosity to generate (comment-free JS, a CCMFH file)

This would be totally insane, but it would work and it would involve the least modification to nodes.coffee

[andrewschaaf]

I'll work on a

something(o, ["throw ", @expression, ";"])

approach