find UnicodeEncodeError when run command:DATA_TYPE=reverse ./bin/data/toy.sh

[nmaac]

$ DATA_TYPE=reverse ./bin/data/toy.sh
Using type=reverse. To change this set DATA_TYPE to 'copy' or 'reverse'
Writing to /home/user/nmt_data/toy_reverse. To change this, set the OUTPUT_DIR environment variable.
Traceback (most recent call last):
File "/home/user/workspace/tsinghua/seq2seq/seq2seq/bin/tools/generate_toy_data.py", line 146, in
main()
File "/home/user/workspace/tsinghua/seq2seq/seq2seq/bin/tools/generate_toy_data.py", line 142, in main
write_parallel_text(train_sources, train_targets, ARGS.output_dir)
File "/home/user/workspace/tsinghua/seq2seq/seq2seq/bin/tools/generate_toy_data.py", line 115, in write_parallel_text
source_file.write(record + "\n")
UnicodeEncodeError: 'ascii' codec can't encode character u'\u7b11' in position 0: ordinal not in range(128)

using:
python 2.7.5
centos 7

[dennybritz]

Thanks for reporting! Should be fixed in #93

[nmaac]

Hi, @dennybritz
when run command: python -m bin.infer
I also met the UnicodeEncodeError:

File "seq2seq/tasks/decode_text.py", line 188, in after_run
print(sent)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u7b11' in position 14: ordinal not in range(128)

[dennybritz]

Does it work if you set export PYTHONIOENCODING=UTF-8 as an environment variable?

You could also try the fix discussed in #96.

[nmaac]

Yes, it works