Assembling Cassiopaean Concepts

[Chris]

In this regard, I think we should rely on the amazing work done earlier and the methods used to build the Cassiopaean Glossary through this network.

It'll be great if we can get some input from the members who developed the glossary and their insights on how they solved this problem. So much of the basic groundwork has been laid already and we should make use of it. Because of the scale of this project, it will behoove us to take all the help that is available, and "Network, Network, Network!"

 

[mcb]

...
Are there clever ways of searching through the C's transcripts conceptually? For instance, searching "of thought" might bring up this:
...

That is pretty much what I was trying to do with my database, although it was limited to nouns and noun phrases so "thought" would be tracked but not "of thought."

Apart from the technical problems of downloading and refreshing the data -- the issues that consumed me -- there is the problem of how to present and filter the information in a way that makes it useful. If anyone wants to pursue that approach, what I would need is people that can apply visualization and query tools to the task, that operate on XML (or similar) graphs.

I had reached the point with my database where I could bulk download a particular topic, but I have never gone on to be able to do the same for an entire board. Since the transcripts are a board, not just a topic, the database isn't too useful just yet for this application.

Given the necessary source data, visualization tools can can reveal major patterns in the information, and serve as a kind of index, while query tools might sometimes be more useful for exploring particular concepts. Both would be needed, I think.

 

[darksai]

...
Are there clever ways of searching through the C's transcripts conceptually? For instance, searching "of thought" might bring up this:
...

That is pretty much what I was trying to do with my database, although it was limited to nouns and noun phrases so "thought" would be tracked but not "of thought."

Apart from the technical problems of downloading and refreshing the data -- the issues that consumed me -- there is the problem of how to present and filter the information in a way that makes it useful. If anyone wants to pursue that approach, what I would need is people that can apply visualization and query tools to the task, that operate on XML (or similar) graphs.

I had reached the point with my database where I could bulk download a particular topic, but I have never gone on to be able to do the same for an entire board. Since the transcripts are a board, not just a topic, the database isn't too useful just yet for this application.

Given the necessary source data, visualization tools can can reveal major patterns in the information, and serve as a kind of index, while query tools might sometimes be more useful for exploring particular concepts. Both would be needed, I think.

I've posted my design for a session searching tool here. It could be extended to do the kinds of things you've mentioned, though the big difference is that mine uses a fresh database and necessitates a manual filtering approach of useful data that would then automatically be sorted in a practical way. How this would work for general posts and threads I can't be sure (unless they're basically broken down by hand into passages by type (eg informative, anecdotal, research, etc..), though the cassiopedia and esoteric glossary would work really well (since like the transcripts, they have a neatly contained structure.)