Document guidelines to enable language features for embedded languages

[aeschli]

In VSCode each file is associated with a language. Language supports such as code completion, hovers, are contributed to that language. This is ideally through a language server.
When a language allows to embedded snippets of an other language (e.g. CSS in HTML or HTML in PHP) there are various techniques that a language server can use:

• The language server also implements support for the embedded language. It can do that by including libraries that provide that support. For example there are easy to use node modules for css, less, scss, html and json or more basic language supports for typescript
• Forward requests to an other language server as done in intelephense

The first approach has the following advantages

• Full control of the user experience. Completion proposals, hovers... can be tuned to apply to the situation.
• No dependencies on other language servers, self contained server that is easy to embedded also in other editors or IDE.

In either case the embedded content needs to be escaped according to the owner language. E.g > needs to be &gt.

[aeschli]

@aeschli Do you think that an extension can make the html part in php get the same behavior as the standalone html file? including let all the plugin for html work fine in that part?
Extension system is really important as we all know, but not all the features are suitable for plugins. What we actually need is parse one file to multiple language and treat them separately. This should be a built-in ability for a code editor.
And, all the PHP files are actually HTML, with a special HTML tag. Only the content in this tag should not be regarded as HTML. It is just a language embedded in HTML and can't work without it (except CLI). So the question is not about HTML in PHP, HTML is the main container. And also lots of language can be embedded in HTML, such as css javascript vbscript svg perl java, I don't think any extension can do this without official support.

@popcorner PHP files are actually not adhering to HTML syntax: <?php is not valid in pure HTML. Although it seems like PHP is embedded in HTML, the syntax is defined by PHP so it's actually 'HTML' inside PHP.
Same with templating languages like Smarty:

<div>
{escape} 
This is some text I want <> escaped. 
{/escape}
</div>

(snippet from here
Valid Smarty, not HTML. Try it the HTML validator)
Every templating language has their own way of escaping embedded content and why the HTML language server can't just be easily used. It lacks the knowledge of the embedding language.

[jens1o]

The language server also implements support for the embedded language. It can do that by including libraries that provide that support. For example there are easy to use node modules for css, less, scss, html and json or more basic language supports for typescript

The problem is, how can I say I'm the master without maintaining a fork and rule them to only work in specific parts of a file? Why can't the language servers I want to control say what they are able to handle, so I do not need to worry about it and the end-user can install any extension they like, so I do not need to upload an extension douzens of megabytes big to support really any language?

[aeschli]

You don't need to maintain a fork. You can forward the requests. But your Smarty server needs to transform the document to valid HTML and then ask the HTML server for the result on this document. That's approach number 2.
You can also tell the HTML language support to handle Smarty files by associating smarty files to HTML. But it will struggle if the file is not valid HTML as in the example above.

[jens1o]

You can forward the requests.

I'm looking forward to an example code. ;) And somehow you didn't answer on my second question. How can I know, when I want the users to install extensions without any more config to know why parts can be handled by which server?

[aeschli]

• Here's how intelephense does it: https://github.com/bmewburn/vscode-intelephense/blob/cdd5f66f09e8ee26a2d69a0799ba761c90bc8d43/src/embeddedContentDocuments.ts#L306
  The requests to the PHP language server are injected and instead forwarded to VSCode using a command. That command gets a virtual 'html' document. Whoever registered on 'html' will provide proposals.

• Here's how we implemented some time ago for css embedded in html (no longer in use). The server sends the request back the client the client asks other extensions based on a virtual documents that just contain the embedded language content.

I don't understand your last question. Can you rephrase?

[jens1o]

You can use Smarty for everything, it's not dedicated to html, css, js, java, xml, json... So in the current version I need to detect which language it actually is, but that's the downside, because I want the language servers that the user installed to give me examples of what they are able to handle, so I do not need to worry about, and it's completly modular(so I would not need to detect whether it's PHP or Smarty code).

1. User installs an extension, which provides language support(e.g. PHP).
2. User opens a smarty file, and the smarty-extension asks vscode for a list of language servers(content providers) and their samples(so what does a php code pattern look like?).
3. Once the user types in something that (for example) PHP can handle, I know I can delegate the request to the PHP language server and it can handle the request, so I simply wrap it around.

That would be sooo easy and soo extensible.

I hope you understand why that concept would be so awesome and the next step.

[bmewburn]

@aeschli the html language server went with option 2 first for embedded languages then changed to option 1 later. Were there other advantages, in addition to those listed above, that prompted the change in direction?

@jens1o In that situation one solution could be to provide a config setting so the user can declare what the embedded language is. Then your extension can create virtual documents with the appropriate language id and forward them on.

[jens1o]

In that situation one solution could be to provide a config setting so the user can declare what the embedded language is.

I want to make it as simply as possible, prompting the user for this to declare is very likely to include an error or missing an exception.

[aeschli]

@bmewburn The main reason was really to be able to control the user experience. For example in the case of JavaScript embedded in HTML, we want to preconfigure the JavaScript language server with the dom definition files.

[octref]

@jens1o

User opens a smarty file, and the smarty-extension asks vscode for a list of language servers(content providers) and their samples(so what does a php code pattern look like?).

Can you clarify "samples" with examples?

Currently if you want to get HTML completions in your extension, all you need to do in your extensions are three things:

1. Figure out if the completion position is in HTML or your-lang
2. If it's HTML, create a virtual HTML document with the embedded content
3. Call the command vscode.executeCompletionItemProvider.

If you are writing a language server that could handle embedded language X, things such as "getting range and content of embedded regions" (for 1 and 2) should be the language server parser's responsibility.