<article> proprietary attribute "itemprop"

[zanona]

Hey guys, just wondering why itemprop can only be applied to div and applying it to article would result in warnings?

<div itemprop=employee itemscope itemtype=http://schema.org/Person>...</div>

=> OK

<article itemprop=employee itemscope itemtype=http://schema.org/Person>...</article>

=> <article> proprietary attribute "itemprop"

[geoffmcl]

@zanona thanks for the report...

Well this simple answer is TidyAttr_ITEMPROP is included in the table W3CAttrsFor_DIV[], along with some other ITEMxxxx attributes, while it is not included in W3CAttrsFor_ARTICLE[].

Now is that deliberate? Or just an oversight? We have to check the W3C HTML5 specificaltions...

It would help if you could research and find some links to decide this... after that it is just a copy-paste operation... thanks...

[zanona]

Thanks @geoffmcl,

I have done some really light research here and found the following:

From W3C I could spot this in regards the itemprop attribute

Every HTML element may have an itemprop attribute specified, if doing so adds one or more properties to one or more items

As well as for itemscope and, I presume, the remaining itemxxx attributes:

Every HTML element may have an itemscope attribute specified.

Under MDN itemprop is also listed as a Global attribute with the observation:

Global attributes are attributes common to all HTML elements; they can be used on all elements, though the attributes may have no effect on some elements.

Please let me know if these observations are valid.

Thanks again for your help.

[geoffmcl]

@zanona thanks for commencing the research into this... it seems in the 'microdata' section, which is relatively new to tidy in the first place... And I would not look at references other than the W3C... Tidy is a W3C utiltiy...

See #209, and maybe others... you will find a note in there to the effect "that microdata is NOT an approved W3C Recommendation, but it is supported by the validator so..."

That implementation was for RDFa - https://www.w3.org/TR/html-rdfa/ - it also notes "RDFa attributes are technically legal on ALL elements."...

I do not have a full understanding of the different between microdata and RDFa... except what I can read in the Abstract section... just different data formats...

But just looking at the facts that we have. That is the tables in that giant file attrdict.c - 1.5MB - by far the largest file in tidy source...

A quick perl analysis shows it contains some 135 elements, from A to XMP. Of that 135 elements, only 45 presently have ITEMPROP, usually together with the 4 other ITEMxxxx atrributes - that is ITEMID ITEMPROP ITEMREF ITEMSCOPE ITEMTYPE...

The list with: A ABBR ADDRESS BLOCKQUOTE BODY BR CAPTION CITE CODE DD DFN DIV DL DT EM FORM H1 H2 H3 H4 H5 H6 IMG INPUT KBD LABEL LI LINK OBJECT OL OPTION P PRE Q SAMP SELECT SPAN STRONG TABLE TD TEXTAREA TH TR UL VAR.

Why, how was this list chosen?

So some 90 elements, the majority, do not have the ITEMPROP attribute listed, and will thus not have any other ITEMxxxx attributes...

But that is a BIG missing list, not just the ARTICLE you mentioned, but of course it is there too...

The missing list: ACRONYM APPLET AREA ARTICLE ASIDE AUDIO B BASE BASEFONT BDI BDO BIG BUTTON CANVAS CENTER COL COLGROUP COMMAND DATALIST DEL DETAILS DIALOG DIR EMBED FIELDSET FIGCAPTION FIGURE FONT FOOTER FRAME FRAMESET HEAD HEADER HGROUP HR HTML I IFRAME INS ISINDEX KEYGEN LEGEND LISTING MAIN MAP MARK MATHML MENU MENUITEM META METER NAV NEXTID NOFRAMES NOSCRIPT OPTGROUP OUTPUT PARAM PICTURE PLAINTEXT PROGRESS RB RBC RP RT RTC RUBY S SCRIPT SECTION SMALL SOURCE STRIKE STYLE SUB SUMMARY SUP SVG TBODY TEMPLATE TFOOT THEAD TIME TITLE TRACK TT U VIDEO WBR XMP

If they are allowed on every one of these element, then it seems we should have another code way to verify such universal attributes, rather than just a massive copy-paste to each element in the list, again massively increasing the size of attrdict.c... Will try to look for a way to accept these 5 ITEMxxx attributes on all elements, in case this becomes the decided right way...

But it is way above my pay grade to decide this alone ;=)) I am just a coder, when others have decided, voted the way... established clearly the right thing to do...

So need lots more comments on this, more ideas... and as always, patches, PR always welcome for consideration...

[balthisar]

@goeff, you're at the top of the scale in our small organization.

Do we have any other attributes that are universal? They might be candidates for moving to a "universal" structure, too. Maybe attrdict.c could actually shrink in size.

I can try to look into this during the weekend, unless someone else beats me to it.

I suppose the only real question is whether or not it's agreed that these attributes are meant to be universal.

[geoffmcl]

@balthisar you ask another important question... sort of are there any attributes that are universally allowed on every element?

Tweeking my perl attrdict.c analyser a little, to show which attributes appear in all 135 element listed?

Well the answer is none at present, per our current table layout. This is not addressing whether they should be in all, just looking at what we presently have, to see if that sheds any light...

The attribute with the highest hits is ID, present in 130 of the 135 elements, but it is closely followed by CLASS, STYLE, and TITLE, present in 129 of the 135 element... and next is DIR in 126 of 135. Must remember to also try to look at the exception... is this a special element set? See below...

Next is 125:135: 10 attributes of the form Onxxxxxx, but not all On! - OnCLICK OnDBLCLICK OnKEYDOWN OnKEYPRESS OnKEYUP OnMOUSEDOWN OnMOUSEMOVE OnMOUSEOUT OnMOUSEOVER OnMOUSEUP (10). It looks like the other Onxxxxx appear down at 114-113:135, with 45 attributes - looks like the balance 43 more Onxxxx attributes, plus ACCESSKEY. Plus LANG 123, XML_LANG 119...

Now a big block of attributes 124:135: 35 attributes of the form ARIA_xxxxx. This appears to list all the ARIA_ attributes, so will not repeat that big list here.

Then the numbers drop away quickly

cnt: attribute list... (cnt)
============================
112 CONTENTEDITABLE CONTEXTMENU DRAGGABLE DROPZONE HIDDEN SPELLCHECK TABINDEX (7)
91 ROLE (1)
86 XMLNS (1)
46 ITEMPROP (1)
45 ITEMID ITEMREF ITEMSCOPE ITEMTYPE (4)
36 SDAFORM (1)
27 ALIGN (1)
18 NAME (1)
17 SDAPREF TYPE WIDTH (3)
12 FORM HEIGHT (2)
10 DISABLED SRC (2)
8 CHAR CHAROFF VALIGN (3)
7 VALUE (1)
6 AUTOFOCUS LABEL MEDIA (3)
5 BGCOLOR COMPACT SIZE TARGET (4)
4 ALT CHARSET CITE HREF (4)

Finally there are some 147 attributes that just appear in 3=13, 2=49, and 1=85 elements... the rare earths...

So while there are none that are currently universal there are many that appears in most, and maybe could be better handled with an exception list, especially big blocks like the ARIA_xxxx set... rather than just duplicating them in each element table...

So maybe there is a way to code this differently, and thus reduce the bulk of attrdict.c... sort of having a single table structure of attriubtes - these are allowed just about every where except for specific elements...

And that led me to examining the reverse - for each attribute, show what it is not present in. Maybe this will give some clue, patterns, ...

Like for the top 6 attributes, the closest to universal that we presently have, what elements do not include this attribute -

Attribute: Elements that do not include the attribute
ID       : LISTING NEXTID PICTURE PLAINTEXT XMP  (5)
CLASS    : BASEFONT LISTING NEXTID PICTURE PLAINTEXT XMP  (6)
STYLE    : BASEFONT LISTING NEXTID PICTURE PLAINTEXT XMP  (6)
TITLE    : BASEFONT LISTING NEXTID PICTURE PLAINTEXT XMP  (6)
DIR      : APPLET BASEFONT FRAME FRAMESET LISTING NEXTID PICTURE PLAINTEXT XMP  (9)
OnCLICK  : APPLET BASEFONT FONT FRAME FRAMESET ISINDEX LISTING NEXTID PLAINTEXT XMP  (10)

Hmmm, see glimming of a pattern, but...

The full perl script log file is here - http://geoffair.org/tmp/attrdict.txt - most should be self explanitory... and would share the script, but requires a little setup...

But yes, the only real question is whether or not it's agreed that these attributes are meant to be universal? Then, how universal is universal?

I remember reading a phrase in one of the references viewed that said like - "while these attributes may be added to every element, they may have no meaning..." - sort of like syaing you can add them to all without a problem... And I do not think browsers would care, since in many cases they have no effect on what is displayed.

But this is a difficult, open question... help needed... thanks...

[balthisar]

The current code is highly readable, and once compiled the structs are fairly efficient. And because they're defined at compile time we avoid all sorts of potential memory issues. Generally, I like things pretty much as they are if there aren't enough universal attributes.

If we're worried about the source file size, though, a pragmatic approach would be to define a lot of the common stuff in macros, and let them be expanded into the structs. If the itemprop stuff is truly universal, it's simpler to add that macro to every item than to add all of the itemprop family. Given your analysis above, there are probably whole families of properties that we can treat this way.

Decreasing the file size isn't a high priority, obviously, but what sense of priority do we have over all of the itemprop? If it's agreed that they are universal, I can at least do that much.

[balthisar]

I'll take a look at some of your analysis again, @geoffmcl. In the meantime I implemented a couple of macros in attrdict.c: INCLUDE_MICRODATA and INCLUDE_ARIA, and this knocked a third off the file size and eliminated 4258 lines. I went ahead and added INCLUDE_MICRODATA to every element that should address this bug report (still not sure if it's the "correct" choice or not).

Not issuing a PR as it's based on the localize_flat branch, but it's pushed here to ensmallen_attrdict.

[balthisar]

A little bit more searching provides some additional source code optimization opportunities that don't change program structure:

https://www.w3.org/TR/html-markup/global-attributes.html

It would be trivial add create another pair of macros -- say, INCLUDE_CORE_ATTRIBS and INCLUDE_CORE_EVENTS -- and include these in every element that we have defined. However this obviously doesn't necessary apply to pre-HTML5, so we would have to do one of two things:

• Assume that if an attribute isn't currently in an AttrVersion structure for a particular tag, then it only applies for (X)HTML5, or
• Try to find legacy documentation to see if Tidy's been missing something forever.

Whichever the case is, it's probably a much more manual editing process than the simple regex substitutions I used for the current push.

[balthisar]

accesskey is a good example of the above. It's valid on every element for HTML5, but only certain elements of HTML4.

Perhaps Tidy's attribute lookup logic will allow us to specify overrides. Supposing we have an attribute defined twice in the struct, does Tidy guarantee that the first instance will be found and used? This would allow us to define the "correct" accesskey, while also including it as part of the macro-expanded definition without hurting anything. For example INCLUDE_CORE_ATTRIBS would mark it as not available for HTML4 and older, but we would manually allow it for the form-related items.

Seeing that each element's attribute list ends with TidyAttr_UNKNOWN, I'm guessing we stop at the first hit... let's have a look. attrs.c:483 and nearby indicate that we seem to stop at the first hit. I guess I will put that to a test.

[balthisar]

Hmmm... it seems as if Tidy is ignoring attributes and HTML version for some reason. Given

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<head>
<title>Test</title>
</head>
<body>
<p accesskey="JEHLL" align="left" contextmenu="hello">hello world
</body>
</html>

and
./tidy --doctype transitional /Users/jderry/Desktop/test.html

I'm given no warnings about the invalid tags. Going back to 5.1.29 results in the same behavior. I would expect a warning if I'm using attributes that aren't part of the HTML version being Tidy'd.

[balthisar]

I know I'm highjacking this issue a bit -- it's still open but should be fixed in ensmallen_attrdict -- but the attrdict is now a comfortable 428K/4559 lines versus the start at 1.5Mb/13926 ,all just by moving all of the common stuff into macros (which actually isn't complete yet).

I'm not posting as a PR because I based off the wrong branch. The only changed file is attrdict.c which can be cherry picked. I'll put up a master-based branch before issuing a PR.