[Node] Arbitrary embedding .meta
info into a tiddler file, i.e. single-file tiddlers of any type
#8156
Replies: 5 comments 9 replies
-
Great idea. I'd also need to look through the specifics of both the utils filesystem code & the filesystem adaptor plugin. I think we would also need to hook into the boot code that loads tiddlers & tracks filesystem paths. Possibly add an MIME type there as well. That all seems like we could abstract it better to support custom metadata. |
Beta Was this translation helpful? Give feedback.
-
Thanks @CyberFoxar. As you note, this has been discussed a few times in the past which is a clear sign that it is an enduring issue. In TiddlyWiki terms, I think the proposal is to introduce a new file format for representing tiddlers with embedded YAML metadata, and then to use that format in the Node.js configuration in preference to There are a couple of issues:
So, where I am at the moment is that I think we should indeed introduce a new file format that is similar to Could you give some examples of the metadata you'd like to use, and the effect you'd expect them to have? |
Beta Was this translation helpful? Give feedback.
-
I'm OK with your example of listing JPG-files and extract the EXIF data as TW meta data. But I'm against the possibility to handle arbitrary binary files with unknown content for security reasons. |
Beta Was this translation helpful? Give feedback.
-
Alright, an update. My goal is to create a new kind of plugin, that I'm tentatively naming Proposed Mechanism: deferred tiddler loadingI'm trying to work out how a deferred loading of Tiddlers might work. The goal is to add a new, optional property to {
"directories": [
{
"isDeferred": true // {boolean}
"deferredContentType": "bin/test-tiddler" // {string} - defines which plugin to defer to
// Rest of the fields as-is
}
]
} This optional property would signal to I'm a bit iffy on the details, but I believe that registering another This would basically be a lite version of the content of Tiddlers loaded via this method need to be marked in some way (so that TiddlerLoader API proposalMy JS is a bit rusty, but that's what I'm working with for now. Please do tell me improvements ! The most obvious one, that I'm on the fence about is whether or not we should try to make the API more flexible, to allow passing a bit more info that just the filename to the plugin. Maybe /*\
title: Proposed API for deferred tiddler loader module
type: application/javascript
module-type: tiddlerLoader
Example API
\*/
(function(){
/**
* When given binary data, tries to deserialize it into a tiddler.
*
* @param {string} filename - original filename, does not include path
* @param {Buffer} fileBuffer - file as read by NodeJS
* @returns {$tw.Tiddler} - decoded Tiddler object to be added to the store
*/
function loadTiddlerFromBinary(filename, fileBuffer){
throw Error("not implemented")
}
/**
* When given a Tiddler, binarize it however we like and gives
* back a temporary object holding the data.
*
* @param {string} filename
* @param {$tw.Tiddler} tiddler - tiddler to be binarized
* @returns {
* {
* buffer: Buffer,
* filename: string
* fileOptions: {fs.WriteFileOptions | undefined}
* }
* }
*/
function makeBinaryFromTiddler(filename, tiddler){
throw Error("not implemented")
}
TiddlerLoaderPlugin.prototype.load = loadTiddlerFromBinary
TiddlerLoaderPlugin.prototype.save = makeBinaryFromTiddler
function TiddlerLoaderPlugin(){
// init stuff
}
exports["bin/test-tiddler"] = TiddlerLoaderPlugin
})() |
Beta Was this translation helpful? Give feedback.
-
More news ! I've made a PR ( #8201 ) to implement the proposed feature ! |
Beta Was this translation helpful? Give feedback.
-
Idea for arbitrary-embedding metadata info inside a tiddler.
The gist
Would it be possible to have the information that's currently in a
.meta
file embedded in the tiddler itself, without resorting to a.json
or.tid
tiddler ?Why
I am in the process of migrating a good bunch of my markdown notes from dendron (think obsidian, but more open) to tiddlywiki.
One feature I like about dendron is the use of YAML Frontmatter, which is a block of yaml (or anything, really), delimited by fences, that defines meta-information about the rest of the document. It works more or less exactly how a
.tid
file is constructed, except it's a tad more versatile.Anyhow, my issue stems from the fact that I'd love to reuse/keep the frontmatter that is already in my markdown notes, without having to convert them to json/tid or live with a bunch of
.meta
files. I have a personal gripe with the use of.meta
files for text-based content, as I feel this will create undue clutter in my notes as well as risk me losing even more info next time I try switching systems.How?
I have taken a look as to if a plugin might fix that, but it seems to be a root issue with how the nodeJS filesystem code is implemented, taking a look at this line and this one from
/core/modules/utils/filesystem.js
it seems this behavior is not (currently) overridable via plugins.A tentative fix would be to either have a hook to change the behavior at this place, or allow plugins to inject themselves there. Say, allow a plugin to hook into
filesystem.js.saveTiddlerToFile
to handle either the actual writing of the file, or pass on the "compiled" binary data to the call tofs.writeFile
. This would also require to patchboot.js
to handle loading tiddler with meta in them, specifically patching/boot/boot.js.[$tw.loadTiddlersFromSpecification, $tw.loadTiddlersFromFile]
to be able to read and load meta-in-tiddlers.This would enable a plugin developer like me to use any arbitrary file format to serialize (and deserialize) single-file tiddlers in the filesystem. It'd allow me to call a plugin such as remark-frontmatter to parse frontmatter of text files and use that as the metadata for the tiddler, but it would also enable more enterprising people to use any kind of field for meta info. One could imagine storing the metadata inside of the EXIF data of a picture (see XMP:Description) or the "comment" field of JPEG files.
Similar subjects
The subject of tiddlywiki's handling of
.meta
files has been lightly touched on in the past, see this discussion on Discourse of someone whom a frontmatter-enabled tiddlywiki would've helped.Interest for storing and accessing metadata within EXIF has been mentionned here in the Discourse.
I am not sure if there are more specific examples.
Offer for help
I am really liking TiddlyWiki so far, and I'd be glad to help / contribute to the above feature. I am still learning the ropes a bit, but I was already looking into patching this behavior into my own version of tiddlywiki, but I figure that some of you guys might have a better idea of how best it should be done.
Beta Was this translation helpful? Give feedback.
All reactions