Why do personal knowledge base applications like Obsidian have all these bells and whistles for querying and parsing metadata/frontmatter but nothing similar for the actual content of notes?
I've so far seen this exact problem in 4 different apps; Obsidian, Trillium, SilverBullet, and TiddlyWiki.
All of these let you write in some sort of lightweight markup (markdown for most, but TiddlyWiki has its own markup language). All of these also let you add frontmatter or metadata to your notes, and have barrels and barrels of features for querying, parsing, cataloging, and tabulating this frontmatter.
However, markdown is itself structured data. It has headings and tables and lists. That's structure. Why not have similar facilities for querying the markdown itself?
I got into this on the TiddlyWiki forums. A lot of users encouraged me to put everything into metadata fields, practically leaving nothing in the body of notes. Then why have bodies? Why not just have a vault full of YAML files?
Why can't I, for example, get a list of all notes where a certain heading exists, or extract data from a table in a note, or count the number of words in a note, or in a section of a note?
Bluetreefrog @lemmy.world - 13hr
Some years ago I used an indexing tool called DocFetcher. It was amazing. I would love that kind of search functionality in Obsidian.
3
damnthefilibuster - 12hr
A little confused here. Some of the examples you mentioned are already features of obsidian (if not others).
Search all notes that have a certain text as heading? Obsidian’s general search has that. Count number of words in a note or a text selection? That’s generally available in the app.
Do you mean - I would like to reference a note in another note and in the übernote, it should tell me how many words there are in the former note? Maybe a plugin exists for that.
Or the table example - do you want to extract the text and automatically show it in another note?
Isn’t that what note embedding does?
![[OtherNote#^mytable]]
Or, if you want to actually use the data and not just show it, then I believe you need to use DataViews.
Please let us know other examples of use cases that might not be present in Obsidian. You’ve raised an interesting topic!
Also, the people who said that you should put everything in the metadata are idiots or trolls. While you can build metadata over time, it should not be your main focus. The focus should always be your notes.
3
sramder @lemmy.world - 15hr
Sounds like a great well researched question to me. Tried obsidian for a bit but never really got into it… seemed like just installing whatever stack powers WiKipenis would be simpler, just have not had the time to mess around.
2
litchralee @sh.itjust.works - 14hr
I recently learned about Obsidian from a friend, but haven't started using it yet, so perhaps I can offer a perspective that differs from current users of Obsidian or any of the other apps you listed.
To start, I currently keep a hodge-podge of personal notes, some digitally and some in handwriting, covering different topics, using different formats, and there's not really much that is common between any of these, except that I am the author. For example, I keep a financial diary, where I intermittently document the thinking behind certain medium/long-term financial decisions, which are retained only as PDFs. I also keep README.md files for each of the code repos that I have for electronics and Kubernetes-related projects. Some of my legacy notes are in plain-text .txt file format, where I'm free-form record what I've working on, relevant links, and lists of items outstanding or that are TODOs. And then there is the handwritten TODO and receivables list that I keep on my fridge.
Amongst all of this chaos, what I would really like to have the most is the ability to organize each "entry" in each of their respective domains, and then cross-reference them. That is, I'm not looking to perform processing on this data, but I need to organize this data so that it is more easily referenced. For example, if I outline a plan to buy myself a new server to last 10 years, then that's a financial diary entry, but it would also manifest itself with TODO list items like "search for cheap DDR5 DIMMs" (heaven help me) and "find 10 GbE NIC in pile". It may also spawn an entry in my infrastructure-as-code repo for my network, because I track my home network router and switch configurations in Git and will need to add new addresses for this server. What I really need is to be able to refer to each of these separate documents, not unlike how DOIs uniquely identify research papers in academic journals.
It is precisely because my notes are near-totally unstructured and disparate that I want a powerful organization system to help sort it, even if it cannot process or ingest those notes. I look at Obsidian -- based on what little I know of it -- like a "super filing cabinet" -- or maybe even a "card catalog" but that might be too old of a concept lol -- or like a librarian. After all, one asks the librarian for help finding some sort of book or novel. One does not ask the librarian to rehash or summarize or extract quotes from those books; that's on me.
2
numpty @sh.itjust.works - 14hr
Check out Amplenote. Great for sorting loose thoughts into notes, then into tasks, then schedule them if you want.
2
masterspace @lemmy.ca - 12hr
Because indexing a structured field with limited values is different from indexing a "structured" document with fields that can be anything.
early_riser in nostupidquestions @lemmy.world
Why do personal knowledge base applications like Obsidian have all these bells and whistles for querying and parsing metadata/frontmatter but nothing similar for the actual content of notes?
I've so far seen this exact problem in 4 different apps; Obsidian, Trillium, SilverBullet, and TiddlyWiki.
All of these let you write in some sort of lightweight markup (markdown for most, but TiddlyWiki has its own markup language). All of these also let you add frontmatter or metadata to your notes, and have barrels and barrels of features for querying, parsing, cataloging, and tabulating this frontmatter.
However, markdown is itself structured data. It has headings and tables and lists. That's structure. Why not have similar facilities for querying the markdown itself?
I got into this on the TiddlyWiki forums. A lot of users encouraged me to put everything into metadata fields, practically leaving nothing in the body of notes. Then why have bodies? Why not just have a vault full of YAML files?
Why can't I, for example, get a list of all notes where a certain heading exists, or extract data from a table in a note, or count the number of words in a note, or in a section of a note?
Some years ago I used an indexing tool called DocFetcher. It was amazing. I would love that kind of search functionality in Obsidian.
A little confused here. Some of the examples you mentioned are already features of obsidian (if not others).
Search all notes that have a certain text as heading? Obsidian’s general search has that. Count number of words in a note or a text selection? That’s generally available in the app.
Do you mean - I would like to reference a note in another note and in the übernote, it should tell me how many words there are in the former note? Maybe a plugin exists for that.
Or the table example - do you want to extract the text and automatically show it in another note?
Isn’t that what note embedding does?
Or, if you want to actually use the data and not just show it, then I believe you need to use DataViews.
Please let us know other examples of use cases that might not be present in Obsidian. You’ve raised an interesting topic!
Also, the people who said that you should put everything in the metadata are idiots or trolls. While you can build metadata over time, it should not be your main focus. The focus should always be your notes.
Sounds like a great well researched question to me. Tried obsidian for a bit but never really got into it… seemed like just installing whatever stack powers WiKipenis would be simpler, just have not had the time to mess around.
I recently learned about Obsidian from a friend, but haven't started using it yet, so perhaps I can offer a perspective that differs from current users of Obsidian or any of the other apps you listed.
To start, I currently keep a hodge-podge of personal notes, some digitally and some in handwriting, covering different topics, using different formats, and there's not really much that is common between any of these, except that I am the author. For example, I keep a financial diary, where I intermittently document the thinking behind certain medium/long-term financial decisions, which are retained only as PDFs. I also keep README.md files for each of the code repos that I have for electronics and Kubernetes-related projects. Some of my legacy notes are in plain-text .txt file format, where I'm free-form record what I've working on, relevant links, and lists of items outstanding or that are TODOs. And then there is the handwritten TODO and receivables list that I keep on my fridge.
Amongst all of this chaos, what I would really like to have the most is the ability to organize each "entry" in each of their respective domains, and then cross-reference them. That is, I'm not looking to perform processing on this data, but I need to organize this data so that it is more easily referenced. For example, if I outline a plan to buy myself a new server to last 10 years, then that's a financial diary entry, but it would also manifest itself with TODO list items like "search for cheap DDR5 DIMMs" (heaven help me) and "find 10 GbE NIC in pile". It may also spawn an entry in my infrastructure-as-code repo for my network, because I track my home network router and switch configurations in Git and will need to add new addresses for this server. What I really need is to be able to refer to each of these separate documents, not unlike how DOIs uniquely identify research papers in academic journals.
It is precisely because my notes are near-totally unstructured and disparate that I want a powerful organization system to help sort it, even if it cannot process or ingest those notes. I look at Obsidian -- based on what little I know of it -- like a "super filing cabinet" -- or maybe even a "card catalog" but that might be too old of a concept lol -- or like a librarian. After all, one asks the librarian for help finding some sort of book or novel. One does not ask the librarian to rehash or summarize or extract quotes from those books; that's on me.
Check out Amplenote. Great for sorting loose thoughts into notes, then into tasks, then schedule them if you want.
Because indexing a structured field with limited values is different from indexing a "structured" document with fields that can be anything.
Just use a text editor that can search.