Null Char in XML result document

[geoffmcl]

See sf905 created: 2009-03-11 by christophe chenon.

I suggested a patch back then, but for whatever reason it never made it into the code so the bug remained.

Input: cfg -xml

<entry>
<edef>A string of the form &#xdddd or &#dddd</edef>
</entry>

Output: where the tilde, ~, represents a null character.

line 2 column 28 - Warning: unescaped & or unknown entity "&#xdddd"
line 2 column 39 - Warning: entity "&#" doesn't end in ';'
Tidy found 2 warnings and 0 errors!
<entry>
<edef>A string of the form &amp;#xdddd or ~dddd</edef>
</entry>

Strangely, if you add --preserve-entities yes to the config, the output is better -

<entry>
<edef>A string of the form &#xdddd or &#;dddd</edef>
</entry>

Presently working on an improved patch...

[geoffmcl]

Have placed a fix in the issue-373 branch...

The modified patch is -

diff --git a/src/entities.c b/src/entities.c
index 935b567..7dd55eb 100644
--- a/src/entities.c
+++ b/src/entities.c
@@ -352,6 +352,7 @@ uint EntityCode( ctmbstr name, uint versions )
 Bool TY_(EntityInfo)( ctmbstr name, Bool isXml, uint* code, uint* versions )
 {
     const entity* np;
+    int res;
     assert( name && name[0] == '&' );
     assert( code != NULL );
     assert( versions != NULL );
@@ -363,13 +364,23 @@ Bool TY_(EntityInfo)( ctmbstr name, Bool isXml, uint* code, uint* versions )

         /* 'x' prefix denotes hexadecimal number format */
         if ( name[2] == 'x' || (!isXml && name[2] == 'X') )
-            sscanf( name+3, "%x", &c );
+            res = sscanf( name+3, "%x", &c );
         else
-            sscanf( name+2, "%u", &c );
+            res = sscanf( name+2, "%u", &c );

-        *code = c;
-        *versions = VERS_ALL;
-        return yes;
+        /*  Issue #373 - Null Char in XML result doc - sf905 2009 */
+        if ( res == 1 )
+        {
+            *code = c;
+            *versions = VERS_ALL;
+            return yes;
+        }
+        else
+        {
+            *code = 0;
+            *versions = ( isXml ? VERS_XML : VERS_PROPRIETARY );
+            return no;
+        }
     }

     /* Named entity: name ="&" followed by a name */

Now using the above xml sample with a config -

input-xml: yes
output-xml: yes
preserve-entities: yes

Get output as follows, which seems perfect, given that these are not valid entities -

<entry>
<edef>A string of the form &#xdddd or &#dddd</edef>
</entry>

Without adding the --preserve-entities yes, will of course get an escaped & output -

<entry>
<edef>A string of the form &amp;#xdddd or &amp;#dddd</edef>
</entry>

But in both case now no null in the output.

I hope others with get a chance to $ git checkout issue-373, build and confirm this is indeed fixed... and it will be merged to master... thanks...

[geoffmcl]

Test binaries for various OS have been added to tagged Pre-Release.

Your help in testing tidy would be most appreciated.

[geoffmcl]

branch 373 now merged to master... seems to have fixed the issue...