<- Back to Tasks

Standardize UTF8 / UTF16L encoding for MQL code

Hello,

Looks like custom indicators MQL code is a mixture of UTF8 and UTF16 LE.

Would it be possible to standardize to UTF8? It's supported very nicely by MT4/5 and is VCS friendly. UTF-16 is for masochists.

That's really easy fix.

Thank you,

Enyx


find custom_indicators -name *.mq? | xargs file custom_indicators/MetaTrader5/Indicators/SqQQE.mq5: Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqStochastic.mq5: C source, Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqHighestInRange.mq5: ASCII text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqParabolicSAR.mq5: ASCII text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqKeltnerChannel.mq5: Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqLowest.mq5: UTF-8 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqLowestInRange.mq5: ASCII text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqLowestIndex.mq5: UTF-8 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqBullsPower.mq5: ASCII text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqBBWidthRatio.mq5: Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqLinReg.mq5: Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqAroon.mq5: Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqADX.mq5: C source, UTF-8 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqHighest.mq5: UTF-8 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqBearsPower.mq5: ASCII text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqFibo.mq5: UTF-8 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqHeikenAshi.mq5: ASCII text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqIchimoku.mq5: UTF-8 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqHighestIndex.mq5: UTF-8 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqPivots.mq5: UTF-8 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqCCI.mq5: C source, UTF-8 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqAvgVolume.mq5: UTF-8 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqATR.mq5: UTF-8 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqMTKeltnerChannel.mq5: C source, Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Experts/SQ_TickDataExportEA.mq5: Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Experts/SQ_IndicatorValuesExportEA.mq5: C source, Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/scripts/ExportProperties.mq4: ASCII text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqADX.mq4: C source, UTF-8 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqQQE.mq4: C source, Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqHeikenAshi.mq4: ASCII text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqLowestInRange.mq4: C source, ASCII text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqAroon.mq4: C source, ASCII text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqFibo.mq4: C source, Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqBBWidthRatio.mq4: C source, Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqParabolicSAR.mq4: C source, ASCII text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqHighest.mq4: ASCII text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqAvgVolume.mq4: C source, ISO-8859 text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqIchimoku.mq4: Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqLinReg.mq4: C source, ISO-8859 text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqMTKeltnerChannel.mq4: C source, ISO-8859 text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqLowestIndex.mq4: ASCII text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqKeltnerChannel.mq4: C source, Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqHighestInRange.mq4: C source, ASCII text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqStochastic.mq4: C source, Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqLowest.mq4: ASCII text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqATR.mq4: UTF-8 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqPivots.mq4: Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqTEMA.mq4: C source, ISO-8859 text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqHighestIndex.mq4: ASCII text, with CRLF line terminators custom_indicators/MetaTrader4/Experts/SQ_TickDataExportEA.mq4: Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Experts/SQ_IndicatorValuesExportEA.mq4: UTF-8 Unicode text, with CRLF line terminators





Attachments
No attachments
  • Votes 0
  • Project StrategyQuant X
  • Type Feature
  • Status Fixed
  • Priority Urgent

History

DB
#1

Enyx

14.07.2020 14:03

Task created

DB
#2

Enyx

28.07.2020 23:35

Updated list for 129RC1:



find custom_indicators -name *.mq? | xargs file | grep UTF-16 custom_indicators/MetaTrader5/Indicators/SqQQE.mq5: Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqStochastic.mq5: C source, Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqKeltnerChannel.mq5: Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqBBWidthRatio.mq5: Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqLinReg.mq5: Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqAroon.mq5: Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Indicators/SqMTKeltnerChannel.mq5: C source, Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Experts/SQ_TickDataExportEA.mq5: Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader5/Experts/SQ_IndicatorValuesExportEA.mq5: C source, Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqQQE.mq4: C source, Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqFibo.mq4: C source, Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqBBWidthRatio.mq4: C source, Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqIchimoku.mq4: Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqKeltnerChannel.mq4: C source, Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqStochastic.mq4: C source, Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqFractal.mq4: C source, Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Indicators/SqPivots.mq4: Little-endian UTF-16 Unicode text, with CRLF line terminators custom_indicators/MetaTrader4/Experts/SQ_TickDataExportEA.mq4: Little-endian UTF-16 Unicode text, with CRLF line terminators


And here is quick fix on Linux


find custom_indicators -type f -name "*.mq?" | \
 while read file; \
 do [ -n "$(file -bi "$file"|grep 'utf-16le')" ] && \
 echo "$file" && \
 iconv -f utf-16le -t utf-8 "$file" > "$file.utf8" && \
 mv "$file.utf8" "$file"; \
 done


Thank you,


Enyx

TT
#3

Tamas

29.07.2020 13:49

Status changed from New to Fixed

DB
#4

Enyx

05.08.2020 13:08
Thank you. I confirm is fixed in 129RC2+
Votes: 0

Drop files to upload

or

choose files

Max size: 5MB

Not allowed: exe, msi, application, reg, php, js, htaccess, htpasswd, gitignore

...
Wait please