Make .htaccess usage work for the widest possible configurations without relaxing security and document pitfalls

Same here on a fresh install. Does not happen with upgraded versions. I have to turn off css aggregation and then bartik and seven look nice.

The requests for the aggregated css files create a 500 Internal Server error:

Apache Error Log:

sites/default/files/.htaccess: Option MultiViews not allowed here

The .htaccess in the public files directory generated by earlier D8 versions was different. It contained

Options None
Options +FollowSymLinks

instead of

Options -Indexes -ExecCGI -Includes -MultiViews

This looks like we ran into it once before:

#919596: -MultiViews in .htaccess requires odd AllowOverride Options=All,MultiViews.

If you're experiencing the error please indicate Apache version/platform,

Also the issue title indicates the workaround if you're affected, https://mathiasbynens.be/notes/apache-allowoverride-all has more detail too.

I think there's an easy solution here. Can anyone think of a negative implication from NOT defining Options -MultiViews in the files directory?

MultiViews can cause problems with clean URLs (or so says the Clean URL documentation guide), and that is why we avoid them.

But files directory and vendor directory files are never accessed via Clean URLs. They get called with a direct URL, and there are no RewriteRules in the .htaccess files in either of those directories for MultiViews to fight with.

So, since that directive isn't doing anything for us, an easy fix would be to just remove the -MultiViews from the generated .htaccess files and the vendor .htaccess file. Patch attached.

Took a while but found the php docs for this:
http://httpd.apache.org/docs/current/content-negotiation.html#multiviews

So it's documented that All doesn't cover multiviews.

@Damien_Vancouver what about image derivatives? That needs to be a route with the same path as the files directory.

Hi @catch,

I did some image derivative testing and it doesn't seem that leaving MultiViews enabled causes any serious problems... other than the fact it allows insecure viewing of already existing derivative images via a crafted MultiViews URL.

Here was what I tested:

- applied patch from #5 and did a clean install of latest 8.0.x
- Ensured that Options +MultiViews is set in the VirtualHost and tested it in a test directory to confirm it was working.
- created an article with a large image (image.jpg).
- Accessed the image file directly via the MultiViews Map URL (ie. went directly to its URL path at sites/default/files/2015-11/image, omitting the file extension). Apache shows the image, as is expected to happen with mod_negotiation enabled and Options +MultiViews set.
- Viewed the article in Drupal, which shows a thumbnail of the image using the "medium" style. Clicked to view the article in full mode. Both the "medium" and "large" styles worked as expected.
- Took one of the derivative URLs and removed everything after the file extension (ie. the .jpg?itok=XXXXXXX). The image derivative displayed as one would expect with MultiViews on. (This is a security side effect of MultiViews)
- Deleted the image derivative files from both style directories and verified that they are regenerated and load correctly when going to their direct URLs or refreshing the page in Drupal. (mod_rewrite rules from the top level .htaccess firing for a nonexistant file or directory and running as expected).
- Removed the image derivative files again and tested that the files would not load with the full URL and the token missing (sites/default/files/styles/large/public/2015-11/image.jpg). I received access denied as expected.
- With the derivative file still removed, I tried accessing its filename using a constructed MultiViews URL (the filename with no extension or token). I also received an Access Denied on this request..

These last two tests show that the DoS protection for secure image derivatives is still functioning with MultiViews on. While an anonymous attacker can get around needing a token to view an image derivative file that already exists, (s)he cannot force image derivative generation for arbitrary files, which was the DoS vulnerability that secure image derivatives fixed.

So, as far as I can tell, the only problem with not having Options -MultiViews, is: if someone has MultiViews turned on, then they are effectively allowing an attacker who constructs MultiViews-style URLs to view already-generated image derivatives, despite not having the correct token.

@Damien_vancouver I'd expect to be able to view already generated files without the token already, so not sure even that is an issue?

@catch,

I'd expect to be able to view already generated files without the token already, so not sure even that is an issue?

You're right, I must have been trying to load it without the file present when I got the Access Denied. Already existing images are returned without a token.

Tagging 8.0.1 target, it's not clear how many people are actually affected by this, or whether it's less or more than were affected by the original bug that was fixed, but would be good to get it fixed in the next patch release so we don't need to find out either way.

I think just removing this option is probably OK, my assumption is that the original -MultiViews when added to core was probably a copy/paste rather than in response to a specific vulnerability. However I'll try to find the history of the original issue that added it, and also we should add a note to drupal.org/requirements recommending that it's set in vhosts configuration, as well as a change record update to https://www.drupal.org/node/2608772 saying the same.

I added mention of setting Options -MultiViews if it's enabled by default to https://www.drupal.org/requirements/webserver , and updated the change record at https://www.drupal.org/node/2608772.

There was a typo in the issue description so I've updated that as well.

Hello.
I just installed 8.0.1 on my host server and I am having an issue in which css and js files are not loading and I'm getting a error code 500. It affects the toolbar and doesn't load enough css for me to properly access all of the administrative options when I'm logged in as the first user.
I don't have access to the Apache config files, so I was wondering if there was another way to fix the issue than the method prescribed by earlier comments or if a patch was in the works. The host I'm using is running PHP 5.6.16

#2620220: CSS/JS aggregates do not load: HTTP 500 errors when mod_lsapi is installed

Patch update with latest 8.1.x-dev

Patch with latest 8.1.x-dev drupal version.

Agreed, why patch failed testing. Uploading patch with excluding vendor directory changes.

Patch updated for drupal-8.2.x-dev

I think this issue needs an expanded scope. The problem we have is that it appears impossible to generate an .htaccess that offers the maximum security for everyone. The change that caused this issue to be created was #1269780: Remove symlinks option from .htaccess which we did because hosters where moving to the more secure module of only allowing symlink to be followed if the owners matched. If the webserver is configured to allow mutliviews we certainly don't want our files directory to behave the way that multiviews works. However it is a totally reasonable webserver configuration to not allow multiviews to be overridden. Also it appears that #1269780: Remove symlinks option from .htaccess broke OpenSuse.

So what's the best way forward? I think:

• We require AllowOverride All but as pointed out that doesn't include multiviews - so we need to remove this from the auto-generated htaccess files.
• We should add something to .htaccess that disables multiviews and suggests commenting this out if you webserver doesn't allow this. The advantage of doing this is the site would brake the moment it is running on a server and not fail less obviously later. And no one loses any protection - there are numerous reports on the web about the vulnerabilities from having multiviews enabled... https://hackerone.com/reports/25382 -
  

  This behaviour can help an attacker to learn more about his target, for example, generate a list of base names, generate a list of interesting extensions, look for backup files and so on.

• We should add something to .htaccess that documents how to fix the issue caused by the opensuse configuration - this might need to be a followup because https://www.drupal.org/docs/7/system-requirements/web-server is not exactly clear on what to do.

I'm not convinced this is a major because there is a workaround by updating your apache config, we are doing the secure thing for most people, and we haven't had reports of this affecting any major hosters. But that doesn't mean we shouldn't try to make life easier for more custom webserver configurations.

Regarding the OpenSUSE breakage,

@alexpott wrote

We should add something to .htaccess that documents how to fix the issue caused by the opensuse configuration - this might need to be a followup because https://www.drupal.org/docs/7/system-requirements/web-server is not exactly clear on what to do.

As a result of the reports in #1269780: Remove symlinks option from .htaccess , I filed a bug with SuSE explaining the problem with their configuration. The problem is not Drupal specific and can occur with any use of mod_rewrite if one don't explicitly set a Symlink option. https://bugzilla.suse.com/show_bug.cgi?id=955701 and as a result they modified their apache configuration file to have some hints on how to properly enable mod_rewrite: https://build.opensuse.org/request/show/347918 ("apache2-default-server.conf" changes).

That change should now be pushed to anyone running the stable Leap should have it from last November's 42.2 release. However, it's just two lines of text that could easily be overlooked - it has no mention of "Clean URLs":

# NOTE: For directories where RewriteRule is used, FollowSymLinks
# or SymLinksIfOwnerMatch needs to be set in Options directive.

Extra documentation in Drupal's .htaccess will definitely be helpful.

Here is a first draft of something short:

# For OpenSUSE and some other custom Apache configurations, one of
# the following directives must be set to use Clean URLs (mod_rewrite)
# Only enable one of the following two, and only if seeing a Server Error.
# Options +FollowSymLinks
# Options +SymLinksIfOwnerMatch

For the documentation page, Here is a draft of some new text to replace the existing "Symlinks" paragraph at https://www.drupal.org/docs/7/system-requirements/web-server.

SymLinks: Drupal versions prior to 8.0 set "Options +FollowSymLinks" in the .htaccess file. Some shared hosts require a different directive for security reasons: "Options +SymLinksIfOwnerMatch".
If the site shows a Server Error on every page, changing the "FollowSymLinks" to "SymLinksIfOwnerMatch" in .htaccess may fix the problem. In this case there will also be an entry in the Apache error log that says: ".htaccess: Option FollowSymLinks not allowed here"

Drupal 8.0 does not set either Symlink option, leaving the configuration up to the Apache webserver. This avoids the problem described above on shared hosts, but it can cause installation to fail on OpenSUSE distributions and some secured Apache configurations. The default OpenSuse configuration does not come with mod_rewrite enabled, which is required in order to use Clean URLs in Drupal. Due to OpenSUSE's secure default configuration, one of either "Options +FollowSymLinks" or "Options +SymLinksIfOwnerMatch" must also be explicitly added for mod_rewrite to work properly. Without that change, enabling mod_rewrite will cause a Server Error on every Drupal page, and the Apache error log will have an error similar to: "AH00670: Options FollowSymLinks and SymLinksIfOwnerMatch are both off, so the RewriteRule directive is also forbidden due to its similar ability to circumvent directory restrictions : /srv/www/htdocs/drupal/core/install.php". To fix the problem, add either "Options +FollowSymLinks" or "Options +SymLinksIfOwnerMatch" to /etc/apache2/default-server.conf. If you cannot edit that file, you can also make the change to your site's .htaccess file.

Note: Editing .htaccess is the fastest way to quickly troubleshoot and fix these types of Apache configuration problems. But, if you edit the .htaccess file remember that you have to re-edit your changes each new Drupal version. That is because Drupal ships with a new .htaccess file which will overwrite your changes. It's therefore best to make the changes permanent by moving them to Apache's configuration instead once you have fixed the problem.

If that looks OK I can add it as a documentation edit. Anything missing? Too wordy?

The MultiViews stuff proposed in #26 seems sensible and safe as well.

@damien_vancouver thanks for the information - that looks really good. I've updated the issue title and issue summary.

re-roll for 8.8.x and 8.9.x

It looks like the reroll for 8.9.x in #35 is exactly the same as the one for 8.8.x and for me it didn't work, uploading a working version.