Thought experiment: using schema.org + JSON-LD instead

[darobin]

First: don't shoot me. This is just a thought experiment, but one that I believe may prove useful.

My reasoning is as follows:

• Many content producers already have to produce schema.org+JSON-LD for their content. If they could reuse it, it would be better than having to produce a second JSON representation with essentially the exact same information but a different syntax. (For people who don't already produce this, the differences for the most part boil down to taste.)
• Many consumers already understand it, notably all major search engines.
• The extensibility model is already covered: no need to think about it.
• Several issues are already covered (I'm just listing a few that are obvious, there are certainly more): Support link relations #44, Author email field #26, MIME-type should be more specific #22, Use more specific MIME-type application/feed+json #23, consider not using _ to prefix extensions #19, Title-only items? #18, Split home_page_url into two separate fields #38, Are URLs URLs? #39, "mime_type" should be "media_type" #13, Unique ID for feed #10, DurationInSeconds and SizeInBytes are bad names #15, Consider a distinct media type ("MIME type") #7.
• It's already documented and already has a process for change and evolution, a lively community, a successful repo on GitHub, etc. All we need to define is the vernacular.

Downside:

• It's a bit more verbose (but not that much).

This is just a translation of the example from the site's front page to give a feel for the result. If there's interest we can dig through more details.

{
  "@type": "Blog",
  "name": "My Example Feed",
  "mainEntity": "https://example.org/",
  "url": "https://example.org/feed.json",
  "blogPost": [
    {
      "@type": "BlogPosting",
      "@id": "https://example.org/second-item",
      "articleBody": "This is a second item.",
      "url": "https://example.org/second-item"
    },
    {
      "@type": "BlogPosting",
      "@id": "https://example.org/initial-post",
      "articleBody": { "@type": "rdf:HTML", "@value": "<p>Hello, world!</p>" },
      "url": "https://example.org/initial-post"
    }
  ]
}

[gruber]

"articleBody": "This is a second item.",
"articleBody": { "@type": "rdf:HTML", "@value": "<p>Hello, world!</p>" },

That's more than just "move verbose" -- it's more complex. A feed reader wouldn't be able to make any assumptions about the content of articleBody, they'd have to inspect each one.

[darobin]

@gruber I'm not sure what you mean about "inspecting". If the articleBody is a string, then it's text. If it's an object then the @type will tell you what it is — an option we could (and probably should) limit to HTML. That's no more inspection than checking to see if there is a content_text or a content_html.

[hsivonen]

This isn't the first time RDF is pushed onto feeds. It might be worthwhile to reflect on how well the RSS 1.0 effort did compared to RSS 0.92 and Atom and how often RSS 1.0 was actually consumed using RDF tooling.

Several issues are already covered

JSON-LD answers issue #39 by saying RFC 3986 even though the WHATWG URL spec is the one that's more useful to implement (actually says how to handle errors), so at least that point doesn't suggest that inheriting the choices of JSON-LD would be a positive thing.

[darobin]

Whoa there, let's not FUD Henri. JSON-LD != RDF. It's RDF-compatible, but most people who use it completely ignore that because it was designed that way. We have a complete and complex publishing system built atop JSON-LD and there is nothing in it RDFy, even though we do manipulate graphs. Worst-case scenario "RDF" shows up in the names of some things.

[frivoal]

I totally support that. json-ld-feed is generic enough I'd be happy to store the the source of my blog/site in that, generate the HTML from it, and serve it in a feed either as is or lightly filtered.

You could theoretically do the same with json-feed, but you'd constantly run into things that it can't quite express yet, and either not do it, or come up with your own non-standard way of doing it, which means hell for clients trying to figure out what (if anything) it can do with the various bits of non standard data they find in your feed. Not all feed readers need to react to everything you find in a feed, but having well understood ways of including all sorts of information makes life easier for anyone who wants to.

Here are a handful of examples of things json-ld would know how to do, but json feed doesn't, all taken from a poetry site/blog I'm running (https://namino.rivoal.net/poems/):

• say which language various things are in (author name, content of the post...) (relates to Language & Internationalization #40)
• Non-english text in titles that need markup to be correctly represented, like 猫（ねこ）と太陽（たいよう）と
• some items have an author, some items have a translator, some have both, some have several of each (relates to Behavior for multiple authors #6)
• I store the author's name in multiple languages, also some times their place of birth, date of birth and date of death.
• I have various metadata about the book poems come from. This is easily expressed using json-ld + schema.org.

And a bunch more like this. Aside from the international aspects, arguably, you do not need all of these in a feed. But given the choice between a format that already has answer to all of these, and one that doesn't, I think it'd be a lot less pain to go with the one that does.

Having not just the json syntax in common with the rest of the world, but also the json-ld semantics makes for much easier interactions with all sorts of tools out there.

[hsivonen]

Whoa there, let's not FUD Henri. JSON-LD != RDF

"LD" is a rebranding of the "Semantic Web", so I think its fair to ask how things would be better this time round. XML got namespaces due to RDF, so clearly, the SemWeb influence wasn't positive on XML and contributes in significant part to why JSON feels simpler than XML even now that the SemWeb/LD community has moved on from XML. Now JSON-LD adds that complexity to JSON for largely the same reason.

More specifically in relation to feeds, history has RSS 1.0 as a data point. It seems fair to ask why mixing "LD" to JSON-based feeds would be better this time round compared to adding "RDF" or "Semantic Web" to XML-based feeds earlier.

[frivoal]

It seems fair to ask why mixing "LD" to JSON-based feeds would be better this time round

1. While LD brings some complexity to JSON, the result is still way less complex that XML.

2. Using JSON-LD may not be the only way to solve all the relevant problems, but relevant problems need solving. As currently proposed, JSONFeed falls short on of a number of things that JSON-LD already solves. (See my earlier comment, or many of the open issues, including those quoted by @darobin ).

3. There's no evidence that all these problems will be solved for JSONFeed, or that if they are solved, the result will be less complex than JSON-LD.

[darobin]

To be candid Henri, it isn't clear to me why you feel compelled to import the nasty parts from the W3C/WHATWG playbook here. Just in that last short comment I count castigation by association (RDF, namespaces…), negative reframing ("rebranding"), and slippery sloping ("ZOMG you're going to bring namespaces"). There's a world outside standards and an awful lot of people are nice there. I see no need to spread that sort of tactic.

If your concern is the risk of grandfathering in complexity, that's a legitimate concern — certainly one that I share at any rate — but in that case why not just say it and discuss solutions instead of reaching for the negative?

The avowed purpose of this proposal is to maintain the simplicity of JSON Feed while reaping the benefits from an existing vocabulary that is already widely deployed, is already consumed by several major implementers, and has already solved many of the problems — including some that haven't been filed yet (@frivoal's ruby text example is a great one: it's a common Western-centric error to assume that things like title can be conveyed solely with a string). Grandfathering in external complexity would defeat the purpose, so I reckon it would be pretty stupid.

Thankfully there's a lot that can be done to retain the full simplicity of JSON Feed while not reinventing the wheel, not rediscovering solved problems, and benefiting from implementations. That's why I mentioned creating a vernacular, the implication being that it's just a subset of the syntax that processes like JSON-LD — enough to inherit the solved problems — but restricts it in a way that makes it usable by anyone who doesn't care what other things you can do there.

There are some technicalities involved obviously; I'm happy to help with those.

[manton]

This is an interesting proposal, and it's given me the opportunity to become more familiar with JSON-LD and how it might fit with JSON Feed. Thanks! But it's really a different solution. JSON Feed prioritizes clarity and ease of implementation over some flexibility. JSON-LD is much larger in scope. If JSON-LD is solving problems for people, that's great. I'm glad it's there for developers to use if they need it.

Also, since RDF and RSS 1.0 was brought up... Many of us remember those early debates and we don't want to recreate that kind of "us vs. them" split. The web is bigger now, and there are a bunch of JSON formats. Just something to keep in mind.

I think we can let this thread run its course and then close it. If anyone has related blog posts or other articles, feel free to post a link so there's a record.

[hsivonen]

To be candid Henri, it isn't clear to me why you feel compelled to import the nasty parts from the W3C/WHATWG playbook here.

I find it interesting that you accuse me of a nasty playbook here, when you suggested resolving issue #39 by the proxy decision of buying into a large bundle of decisions when you must be aware that of the specs mentioned over at issue #39 the WHATWG URL spec best serves practical needs and that a decision to delegate the choice to JSON-LD would effective mean buying into the W3C playbook of pretending WHATWG non-existence by refusing to make normative references to WHATWG specs even when it would make the most sense technically.

I thought that suggesting this kind of roundabout way of resolving issue #39 was particularly uncool, which is what prompted me to comment here in the first place.

Other than that, I spent quite a bit of time participating in the Atom standardization process, so I have some nostalgic interest in what happens in the next generation of feeds. Back then, there was debate of RDF or not to RDF and Atom chose not to RDF. Now that someone is trying to simplify further, it seems backwards to import the complexity back then avoided especially when there's the RSS 1.0 case to study. (It's telling that you seem to take association with RDF as an offense.)

Further, over the years, I've had a lot of bad time with Namespaces both implementation-wise and standards committee-wise, so I was particularly unhappy to discover that the bit of RDF history that should have been thoroughly recognized as a bad design has been imported into JSON-LD. I would hope that the next generation of specs could be spared from the badness of the prefix-based indirection of Namespaces in XML.

[darobin]

@manton I think maybe I was not entirely clear about the proposal. The idea is emphatically not to bring in all of JSON-LD. That would seem neither useful nor practical. The proposal is little more than to keep JSON Feed as it is with just 1) different field names, 2) a different mechanism for text/HTML distinction, and 3) @type. Item (3) is the one that annoys me most, but I can show examples of the sort of things that it enables (without touching the complexity of processors one bit).

Simplicity and clarity are unaffected, but we get a format that has already solved the problems faced here, that is already understood by major consumers, and that is already compatible with the content produced by a few million sites. I'm not entirely sure what's not to like…

FWIW I too recall many of the horrors surrounding the earlier RSS discussions, which is why I'm sad to see that universe spill over here, into what I hoped would be a constructive discussion.

[darobin]

@hsivonen Again, if it's #39 you have a problem with why not simply say so instead of being aggressive over it? There are very few people who care which URL spec is referenced and I am not one of them — like the vast, vast majority of developers I just use libraries for that crap. If you think that JSON-LD should reference WHATWG URL that is perfectly fine by me. I'm pretty sure that if you open an issue there then @gkellogg or whoever handles it will be happy to consider it.

I'm not sure what you're referring to about "pretending WHATWG non-existence"; I left that world a couple of years ago precisely because I feel life is too short to put up with the kind of toxic comment you are bringing here.

I would like to have a friendly, constructive discussion about this issue — that's why I came here. I have seen the (sadly, unofficial) numbers for the number of pages that have the type of schema.org content we're talking about here from a major processor and they're very impressive. Building on what is actually used out there strikes me as a plus over publishing the same content mostly using different key names. That's all I care about here.

I know you're technically very competent Henri, so if you have constructive input then I very much welcome it. But I don't think technical competence can justify negative behaviour, so if it's just to grind whatever old axes you have with whatever organisation, I would very kindly, and very respectfully, ask that you please take your anger somewhere else.

[manton]

@darobin Thanks for clarifying the proposal. Even those 3 changes would turn JSON Feed into something very different, though. I'm happy to learn from the problems that JSON-LD has already solved, but we can't rename the fields, or it becomes a different format.

[frivoal]

but we can't rename the fields, or it becomes a different format.

That's true, but JSONFeed is very new and hasn't been deployed much yet (you can point a some deployments, but not at some tens of thousands of deployments yet), and regardless of whether you adopt the JSON-LD proposal or not, I think it's premature to consider it frozen. I know you've promised it would be and that you'd be compatible forever going forward, but nobody gets something this subtle right on the first attempt, and not all problems can be solved by strict additions.

Even if you end up rejecting the approach proposed here, please reconsider your stance on rejecting all breaking changes this early.

[manton]

@frivoal Thanks for the feedback. We've been trying to take the approach of keeping most of the issues open for a while, including this one, so that people have a chance to comment. This proposal is for a completely different format, though. It's not so much that it's a "breaking change" as it throws everything out and starts over. :-)