# Improvements to JavaScript analysis

## General improvements

* TypeScript 3.8 is now supported.

* You can now suppress alerts using either single-line block comments (`/* ... */`) or line comments (`// ...`).

* Resolution of imports has improved, leading to more results from the security queries:
    - Imports with the `.js` extension can now be resolved to a TypeScript file,
      when the import refers to a file generated by TypeScript.
    - Imports that rely on path-mappings from a `tsconfig.json` file can now be resolved.
    - Export declarations of the form `export * as ns from "x"` are now analyzed more precisely.

* The analysis of sanitizers has improved, leading to more accurate results from the security queries.
  In particular:
    - Sanitizer guards now act across function boundaries in more cases.
    - Sanitizers can now better distinguish between a tainted value and an object _containing_ a tainted value.

* Call graph construction has been improved, leading to more results from the security queries:
    - Calls can now be resolved to indirectly-defined class members in more cases.
    - Calls through partial invocations such as `.bind` can now be resolved in more cases.

* Support for flow summaries has been more clearly marked as being experimental and moved to the new `experimental` folder.

* Support for the following frameworks and libraries has been improved:
    - [chrome-remote-interface](https://www.npmjs.com/package/chrome-remote-interface)
    - [Electron](https://electronjs.org/)
    - [for-in](https://www.npmjs.com/package/for-in)
    - [for-own](https://www.npmjs.com/package/for-own)
    - [fstream](https://www.npmjs.com/package/fstream)
    - [Handlebars](https://www.npmjs.com/package/handlebars)
    - [http2](https://nodejs.org/api/http2.html)
    - [jQuery](https://jquery.com/)
    - [jsonfile](https://www.npmjs.com/package/jsonfile)
    - [Koa](https://www.npmjs.com/package/koa)
    - [lazy-cache](https://www.npmjs.com/package/lazy-cache)
    - [mongodb](https://www.npmjs.com/package/mongodb)
    - [ncp](https://www.npmjs.com/package/ncp)
    - [Node.js](https://nodejs.org/)
    - [node-dir](https://www.npmjs.com/package/node-dir)
    - [path-exists](https://www.npmjs.com/package/path-exists)
    - [pg](https://www.npmjs.com/package/pg)
    - [react](https://www.npmjs.com/package/react)
    - [recursive-readdir](https://www.npmjs.com/package/recursive-readdir)
    - [request](https://www.npmjs.com/package/request)
    - [rimraf](https://www.npmjs.com/package/rimraf)
    - [send](https://www.npmjs.com/package/send)
    - [Socket.IO](https://socket.io/)
    - [SockJS](https://www.npmjs.com/package/sockjs)
    - [SockJS-client](https://www.npmjs.com/package/sockjs-client)
    - [typeahead.js](https://www.npmjs.com/package/typeahead.js)
    - [vinyl-fs](https://www.npmjs.com/package/vinyl-fs)
    - [WebSocket](https://developer.mozilla.org/en-US/docs/Web/API/WebSockets_API)
    - [write-file-atomic](https://www.npmjs.com/package/write-file-atomic)
    - [ws](https://github.com/websockets/ws)


## New queries

| **Query**                                                                       | **Tags**                                                          | **Purpose**                                                                                                                                                                            |
|---------------------------------------------------------------------------------|-------------------------------------------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Cross-site scripting through exception (`js/xss-through-exception`) | security, external/cwe/cwe-079, external/cwe/cwe-116              | Highlights potential XSS vulnerabilities where an exception is written to the DOM. Results are not shown on LGTM by default. |
| Missing await (`js/missing-await`) | correctness | Highlights expressions that operate directly on a promise object in a nonsensical way, instead of awaiting its result. Results are shown on LGTM by default. |
| Polynomial regular expression used on uncontrolled data (`js/polynomial-redos`) | security, external/cwe/cwe-730, external/cwe/cwe-400 | Highlights expensive regular expressions that may be used on malicious input. Results are shown on LGTM by default. | 
| Prototype pollution in utility function (`js/prototype-pollution-utility`) | security, external/cwe/cwe-400, external/cwe/cwe-471 | Highlights recursive assignment operations that are susceptible to prototype pollution. Results are shown on LGTM by default. |
| Regular expression always matches (`js/regex/always-matches`) | correctness, regular-expressions | Highlights regular expression checks that trivially succeed by matching an empty substring. Results are shown on LGTM by default. |
| Unsafe jQuery plugin (`js/unsafe-jquery-plugin`) | | Highlights potential XSS vulnerabilities in unsafely designed jQuery plugins. Results are shown on LGTM by default. |
| Unnecessary use of `cat` process (`js/unnecessary-use-of-cat`) | correctness, security, maintainability | Highlights command executions of `cat` where the fs API should be used instead. Results are shown on LGTM by default. |


## Changes to existing queries

| **Query**                      | **Expected impact**          | **Change**                                                                |
|--------------------------------|------------------------------|---------------------------------------------------------------------------|
| Clear-text logging of sensitive information (`js/clear-text-logging`) | More results | More results involving `process.env` and indirect calls to logging methods are recognized. |
| Duplicate parameter names (`js/duplicate-parameter-name`) | Fewer results | This query now ignores additional parameters that reasonably can have duplicated names. |
| Expression has no effect (`js/useless-expression`) | Fewer false positive results | The query now recognizes block-level flow type annotations and ignores the first statement of a try block. |
| Identical operands (`js/redundant-operation`) | Fewer results | This query now excludes cases where the operands change a value using ++/-- expressions. |
| Incomplete string escaping or encoding (`js/incomplete-sanitization`) | Fewer false positive results | This query now recognizes and excludes additional cases where a single replacement is likely to be intentional. |
| Incomplete URL scheme check (`js/incomplete-url-scheme-check`) | More results | This query now recognizes additional variations of URL scheme checks. |
| Missing CSRF middleware (`js/missing-token-validation`) | Fewer false positive results | The query reports fewer duplicates and only flags handlers that explicitly access cookie data. |
| Superfluous trailing arguments (`js/superfluous-trailing-arguments`) | Fewer results | This query now excludes cases where a function uses the `Function.arguments` value to process a variable number of parameters. |
| Syntax error (`js/syntax-error`) | Lower severity | This results of this query are now displayed with lower severity. |
| Unbound event handler receiver (`js/unbound-event-handler-receiver`) | Fewer false positive results | This query now recognizes additional ways event handler receivers can be bound. |
| Uncontrolled command line (`js/command-line-injection`) | More results | This query now recognizes additional ways of constructing arguments to `cmd.exe` and `/bin/sh`. |
| Uncontrolled data used in path expression (`js/path-injection`) | More results | This query now recognizes additional ways dangerous paths can be constructed and used. |
| Use of call stack introspection in strict mode (`js/strict-mode-call-stack-introspection`) | Fewer false positive results | The query no longer flags expression statements. |
| Use of password hash with insufficient computational effort (`js/insufficient-password-hash`) | Fewer false positive results | This query now recognizes and excludes additional cases that do not require secure hashing. |
| Useless regular-expression character escape (`js/useless-regexp-character-escape`) | Fewer false positive results | This query now distinguishes between escapes in strings and regular expression literals. |

## Changes to libraries

* The predicates `RegExpTerm.getSuccessor` and `RegExpTerm.getPredecessor` have been changed to reflect textual, not operational, matching order. This only makes a difference in lookbehind assertions, which are operationally matched backwards. Previously, `getSuccessor` would mimic this, so in an assertion `(?<=ab)` the term `b` would be considered the predecessor, not the successor, of `a`. Textually, however, `a` is still matched before `b`, and this is the order we now follow.
* An extensible model of the `EventEmitter` pattern has been implemented.
* Taint-tracking configurations now interact differently with the `data` flow label, which may affect queries
  that combine taint-tracking and flow labels.
    - Sources added by the 1-argument `isSource` predicate are associated with the `taint` label now, instead of the `data` label.
    - Sanitizers now only block the `taint` label. As a result, sanitizers no longer block the flow of tainted values wrapped inside a property of an object.
      To retain the old behavior, instead use a barrier, or block the `data` flow label using a labeled sanitizer.
