unexpected character " U+1B parsing .xmq

[msulyaev]

Strange bytes at the beginning of an .xmq prevent parsing it back by xmq.

xmq --version
xmq: 2.8.0

xmq simple.xml > simple.xmq

cat simple.xmq
root

file simple.xmq
simple.xmq: ASCII text, with escape sequences

hexdump simple.xmq
0000000 5b1b 3b30 3833 353b 313b 3636 726d 6f6f
0000010 0a74
0000012

xmq simple.xmq to-xml
simple.xmq:1:1: error: unexpected character " U+1B
root
 ^

Binary from https://github.com/libxmq/xmq/releases/download/2.8.0/xmq-macosx-release.gz .
Running OSX 14.4.1.

[msulyaev]

... forgot to mention that the source simple.xml is of course nothing strange:

cat simple.xml
<?xml version="1.0" encoding="UTF-8"?>
<root/>

hexdump simple.xml
0000000 3f3c 6d78 206c 6576 7372 6f69 3d6e 3122
0000010 302e 2022 6e65 6f63 6964 676e 223d 5455
0000020 2d46 2238 3e3f 3c0a 6f72 746f 3e2f 000a
000002f

[weetmuts]

I think you got some ansi color escape sequences stored in the simple.xmq file.
It is way to loong for the word root, and the file command detects escape sequences.

Normally "xmq simple.xml > simple.xmq" should autodetect that it is not printing on a terminal and not use ansi.

Can you run "xmq --debug simple.xml > simple.xmq"
then "od -t x1 simple.xmq"

and post the results here.

[msulyaev]

Hi! Here's the debug:

xmq --debug simple.xml > simple.xmq
[xmq] detected terminal xterm-256color
(xmq) COLORFGBG means dark
[xmq] option --debug
(xmq) added to-xmq command
(xmq) added print command
[xmq] performing load
(xmq) cmd-load simple.xml
[XMQ] file size 47
[XMQ] read 47 bytes total 0
[XMQ] read total 47 bytes fsize 47 bytes
[XMQ] content detected as xml since <?xml found
[XMQ] fixup comments after readin
[XMQ] fixup comments |root element
[XMQ] trim element
(xmq) cmd-load 47 bytes from simple.xml
[xmq] performing to-xmq
(xmq) cmd-to xmq render terminal
[xmq] performing print
(xmq) cmd-print output
(xmq) cmd-unload document
[XMQ] freeing source name
[XMQ] freeing xml doc
[XMQ] freeing xmq doc

od -t x1 simple.xmq
0000000    1b  5b  30  3b  33  38  3b  35  3b  31  36  36  6d  72  6f  6f
0000020    74  0a
0000022

[weetmuts]

Thanks, it was the COLORFGBG that forced color even when writing to a file! Bug. This is now fixed. Release 2.8.1

I also changed the ordering of background color lookup, if you are running in an xterm which can report
the actual background color, then it uses that information before using COLORFGBG.

You can force with XMQ_THEME=darkbg XMQ_THEME=lightbg XMQ_THEME=mono

Please test!

[msulyaev]

Thanks, Fredrik!
2.8.1 pulled, built, and tested. Works fine!

As a side note, I was seriously impressed how gracefully xmq handles namespaces:

cat xmlnshell.xml
<?xml version="1.0" encoding="UTF-8"?>
<root>
  <hello xmlns="http://a" xmlns:x="http://x" x:attn="attv">
    <world xmlns="http://b" xmlns:x="http://y" x:attn="attv"/>
  </hello>
</root>

xmq xmlnshell.xml > xmlnshell.xmq

cat xmlnshell.xmq
root {
    hello(x:attn  = attv
          xmlns   = http://a
          xmlns:x = http://x)
    {
        world(x:attn  = attv
              xmlns   = http://b
              xmlns:x = http://y)
    }
}

xmq xmlnshell.xmq to-xml
<?xml version="1.0" encoding="utf8"?>
<root><hello xmlns:x="http://x" xmlns="http://a" x:attn="attv"><world xmlns="http://b" xmlns:x="http://y" x:attn="attv"/></hello></root>