Use redcarpet (Github Flavored Markdown) because Maruku breaks pages

[eed3si9n]

background

• In Set と Seq のページが見れなくなっている scalajp/scala.github.com#29 @xuwei-k reported that Sets and Seqs are no longer visible in Japanese translation of the Collections overview.
• In 4bfc706c3f7f6 @heathermiller confirmed that after gh-pages upgrading its jekyll the site no longer built, so the files were renamed to *.disable.

what this changes

First, I am restoring the files to the original state.

Second, I'm changing the markdown processor to redcarpet (Github Flavored Markdown). The bug seems to be coming from Maruku's handling of non-latin characters, and this is not the only time I've seen it break. I pretty much avoid any use of list or tables because Maruku is so bad. (If you or @xeno-by was wondering why I'm always using html in the translations, this is the reason)

With GFM comes fensed code block and all the goodies that we're used to in the rest of github ecosystem.
If it's so happy and shiny, why isn't it turned on by default? I don't know, but maybe because they've been developing redcarpet in more recent Ruby environment, and you have to take a few steps to set it up locally.

github/pages-gem

You can see what gh-pages is using with github-pages.gemspec.

• Ruby 2.0.0
• jekyll 1.1.2
• redcarpet 2.2.2

Or gem install github-pages says Using Jekyll with Pages.

demo

I merged this into scalajp/scala.github.com's gh-pages branch to make sure it works:

• http://scalajp.github.io/scala.github.com/ja/overviews/collections/seqs.html
• http://scalajp.github.io/scala.github.com/ja/overviews/collections/sets.html

Most of the links are broken since they assume toplevel /, but you can check the renderings by putting in scala.github.com/ manually.

[heathermiller]

First off, thanks a lot for figuring all of this out. And thanks for generating/uploading a sample.
And most of all, thanks for selecting a Maruku alternative – Maruku has been an absolute nightmare from day one.

I see the list of dependencies, jekyll 1.1.2, redcarpet 2.2.2., and Ruby 2.0.0.

My only concern is that using redcarpet requires most folks to upgrade their ruby/jekyll/etc toolchain. I'm completely for folks upgrading ruby/jekyll/etc. The only thing I worry about is the potential burden upgrading might have on peoples' will to contribute.

My experience trying to generate this redcarpet branch (without thinking too much about the versions I have of anything, except for jekyll), for example:

• To myself: "I have Jekyll 1.0.3. I've not witnessed any discrepancies between what the Github Pages version of Jekyll (1.1.2) generates and what I generate locally. So why bother upgrading Jekyll? I'll just install redcarpet"
• I do: sudo gem install redcarpet. Errors abound. eg ./redcarpet.h:8:10: fatal error: 'ruby/encoding.h' file not found
• To myself: "Hmm, ok, nevermind. I should just upgrade Jekyll I guess. Oh yeah, it's probably easier to just install this github-pages thing"
• I do: sudo gem install github-pages. Errors abound. ERROR: Error installing github-pages: github-pages requires Ruby version ~> 2.0.0.
• To myself: "Oh yeah, duh, that's right, there was that Ruby 2.0.0 dependency listed in the PR. Duh. Ok."
• [Googles for upgrading Ruby on OSX] "Ok, I have to install Ruby using RVM"
• [Realizes I don't have RVM installed (not installed out of the box on OSX)]

That's all fine and good and I can do all of this. Though I worry that using recarpet is going to cause most potential contributors to have to go through this laundry list of installations, when before it was a simple gem install jekyll and that was it. I'm concerned that by adding like 6 more steps to the process, it will scare people away. At the least, it'll annoy people like Martin, etc, who like as little process as possible when writing docs.

For example, after one installs RVM (via: curl -L https://get.rvm.io | bash -s stable --ruby)
One must then go through these (handful of) steps: https://coderwall.com/p/tptocq
Before finally being able to run gem install github-pages

WDYT? Am I being paranoid, or is this a valid concern?

[eed3si9n]

WDYT? Am I being paranoid, or is this a valid concern?

That's why I too wrote:

If it's so happy and shiny, why isn't it turned on by default? I don't know, but maybe because they've been developing redcarpet in more recent Ruby environment, and you have to take a few steps to set it up locally.

The page Google landed me recommended Rbenv, so I upgraded Xcode and its Command Line Tool yesterday, sitting in a waiting area of a hospital, just to reproduce the issue. If we were to take this route, we should specify exact route to upgrade to Ruby 2.0 on Mac. It's not just Martin, we're all annoyed. It's a valid concern.

On the flip side, it's gh-pages that updated its Ruby/jekyll version, not me. It's like the production server has gone to Scala 2.10 and a library broke some code. Not taking actions would be equivalent of continuing to use Scala 2.9 locally because Apple installed it by default, and commenting out the code. Having the identical setup is the right thing to do, as long as we use gh-pages. At least, that's how I see it as a person who spent weekends translating. Given a valid technical solution, removing published documents that people have bookmarked, is not a viable course of action. 404 because Martin won't fly.

[heathermiller]

Ok, so good news :-)

We discussed upgrading the toolchain today at the Scala core meeting, and decided to go for it. So in the next week or two, I'll be updating the toolchain and also migrating away from Github Pages to a custom self-hosted Jenkins build.

[eed3si9n]

Thanks for making it happen!

[heathermiller]

Thanks for taking an active role, testing it out, and pushing it :) If it wasn't for you, it'd probably not have happened for a while!

[retronym]

I take it this PR has served its purpose, and we can close this now.

[eed3si9n]

@retronym This issue after nearly an year since the report is still on-going. I just got a report on twitter in Japanese that Seq page has "md.disabled" extension and it downloads the file on the user's browser.

See "列トレイト Seq、IndexedSeq、および LinearSeq" link from http://docs.scala-lang.org/ja/overviews/collections/introduction.html.

/cc @xuwei-k for scalajp#29.

[retronym]

I closed this thinking of this recent change: 0354155

I'm not an expert with markdown and friends, so maybe I got that wrong.

[eed3si9n]

Understood. I also just realized master is using kramdown, which may have fixed the original Japanese issue. I'll asking the original reporter to send in a new pull req with md.disabled file rename.