encoding/base64: wrongly decodes bad base64 as valid

[harshavardhana]

Please answer these questions before submitting your issue. Thanks!

1. What version of Go are you using (go version)?

go1.6

1. What operating system and processor architecture are you using (go env)?
   All operating systems.
2. What did you do?
   https://play.golang.org/p/DiwWTDmGyC

package main

import (
    "encoding/base64"
    "encoding/hex"
    "fmt"
)

func main() {
    // Good base64.
    goodBytes, err := base64.StdEncoding.DecodeString("WvLTlMrX9NpYDQlEIFlnDA==")
    if err != nil {
        fmt.Println(err)
        return
    }
    // Bad base64.
    badBytes, err := base64.StdEncoding.DecodeString("WvLTlMrX9NpYDQlEIFlnDB==")
    if err != nil {
        fmt.Println(err)
        return
    }
    goodHex := hex.EncodeToString(goodBytes)
    badHex := hex.EncodeToString(badBytes)
    fmt.Println(goodHex)
    fmt.Println(badHex)

    goodBytes, err = hex.DecodeString(goodHex)
    if err != nil {
        fmt.Println(err)
        return
    }

    badBytes, err = hex.DecodeString(badHex)
    if err != nil {
        fmt.Println(err)
        return
    }

    goodBase64 := base64.StdEncoding.EncodeToString(goodBytes)
    badBase64 := base64.StdEncoding.EncodeToString(badBytes)
    fmt.Println(goodBase64)
    fmt.Println(badBase64)
}

1. What did you expect to see?

Error for the second bad base64

    badBytes, err := base64.StdEncoding.DecodeString("WvLTlMrX9NpYDQlEIFlnDB==")

Here err should be non 'nil'.

An example of ruby decoder throwing ArgumentError for similar conditions under RFC4648.

require "base64"

good = Base64.strict_decode64("WvLTlMrX9NpYDQlEIFlnDA==") ## RFC4648
puts good.length

begin
  bad = Base64.strict_decode64("WvLTlMrX9NpYDQlEIFlnDB==") ## RFC4648
  puts bad.length
rescue ArgumentError
  puts "Invalid base64"
end

1. What did you see instead?

Success for the second bad base64.

[kostya-sh]

Python 2.7.8 base64 module implements decoding the same way as encoding/base64 in Go:

>>> base64.b64decode("WvLTlMrX9NpYDQlEIFlnDB==")
'Z\xf2\xd3\x94\xca\xd7\xf4\xdaX\r\tD Yg\x0c'
>>> base64.b64decode("WvLTlMrX9NpYDQlEIFlnDA==")
'Z\xf2\xd3\x94\xca\xd7\xf4\xdaX\r\tD Yg\x0c'

RFC4648 allows this. The relevant section is 3.5 (https://www.ietf.org/rfc/rfc4648.txt):

The padding step in base 64 and base 32 encoding can, if improperly
implemented, lead to non-significant alterations of the encoded data.
For example, if the input is only one octet for a base 64 encoding,
then all six bits of the first symbol are used, but only the first
two bits of the next symbol are used. These pad bits MUST be set to
zero by conforming encoders, which is described in the descriptions
on padding below. If this property do not hold, there is no
canonical representation of base-encoded data, and multiple base-
encoded strings can be decoded to the same binary data. If this
property (and others discussed in this document) holds, a canonical
encoding is guaranteed.

In some environments, the alteration is critical and therefore
decoders MAY chose to reject an encoding if the pad bits have not
been set to zero. The specification referring to this may mandate a
specific behaviour.

[bradfitz]

I suppose we could have some opt-in mechanism to be stricter, per the spec's "The specification referring to this may mandate a specific behaviour." I could imagine security applications demanding more strictness, for example.

Maybe we add a method to base64.Encoding like:

// Strict returns a strict encoding of e. It does not modify e.
func (e *Encoding) Strict() *Encoding {
   ....
}

[xuyangkang]

This sounds interesting to me. May I take over it?

[bradfitz]

@xuyangkang, go for it.

[gopherbot]

CL https://golang.org/cl/24964 mentions this issue.