Sea of thoughts

The computer currently gives us three advantages - the advantages of computing power, information storage and global connectivity. But as yet, the computer has not extended our intellect. It has not made man smarter.

scenario i

Bjørn Tennøe. FMH/ Design Fiction

the need

the new extension of mind

Our society is developing, and there has been and will be many shifts in the way people spend their working day. In what way changes the outcome of the work people do?

abstraction of work tasks

Let us assume that the trend of abstraction in work tasks and work output continues. Clearly, if the development of the information society sustains (as assumed in the FMH scenarios), it's a fact as good as any. This logically leads to the hypothesis that work tasks, output and even assignments will become more abstract. Thus, workers may receive such tasks as contemplating over subjective feelings or to make philosophical discussions over a freely chosen subject. Most white-collar workers will be educated somehow in what they do, and that they themselves view the output of their work as concrete - a report, an analysis, a passing or a sorting of information.

But our mind is not made for manipulating abstracts. Discussions and group work become tough when the goals are not set and the form of the end result is unknown. We have a better time off thinking about physical objects or familiar and experienced situations. A further abstraction of our workday therefore creates a demand for a new illustration tool - to make our thoughts, opinions, ideas and concepts viewable and discussible. This illustration tool might very well be a computer application.

Likewise, understanding concept others have developed and searching for and arranging information are tasks that would benefit from better visualisation. Imagine how much easier classes would become if the teacher's words and ideas were continuously illustrated as he or she spoke. And what if the pupils could reach out and change the illustration, seeing it from a different perspective, add their thought and comments to it. What if they even could bring it home to their parents, to show how they learned and discussed at the school today.
If the computer could mirror our mind, so we could experience our ideas like we experience the world around us, our comprehension of abstract concepts would greatly ease. In fact, the concepts would no longer be abstract - they would be objects and illustrations to see and understand.

emerging collaborative tools

When the work tasks become more abstract it is also apparent that people need to work together, both because one must share knowledge between professions and because of the individuals' diversity.
For the last 15 years, several North American firms have experimented with electronic collaboration tools (To narrow down the definition a bit, I refer to those kinds of tools that support real-time communication, especially meetings. E-mail and newsgroups fall out of this definition). Many have found the tools greatly beneficial and promote them with great enthusiasm. The paperback "No More Teams!" briefly describes some of the installations and how people felt about them .

new tools through innovation and new technology

Up to now, though, the set-ups of these tools have been very demanding on physical and financial resources. The meetings are arranged in special rooms, and effective real-time collaboration with other groups can only happen if they too are situated in a similar room . But as the need for effective groupwork and idea sharing is more clearly expressed, tools for the similar need will be accessible to everybody. Clearly it would be a critical step forward if attending these meetings could be done regardless of location and physical set-ups.

the vision

The vision, therefore, is to create (outlines for) a tool that:

visualises our ideas
lets us collaborate on these ideas together
facilitates rather than obstruct the communication of the users
supports natural language

Tied to this application must be modules, possibly from an operating system, that:
provides factual feedback and suggests who to get in touch with guides us through the collaborative process
keeps track of the creation process
supports presentations visually
captures user actions

All applications are server-driven supporting wearable devices entity building - wandering in landscapes - illustrating thoughts

The application is supposed to build an illustration that describes a concept or idea and our opinions about it. The application tries to break up and organise the content of the discussion. It will seek to make a hierarchical structure where main outlines open up to display more subtle details. Whatever we create with this application it will be fragmented, so that it can be rearranged and so that small pieces can be identified and annotated. This makes the result quite different from what we expect from a classical document, which has a narration built into it. We return to this topic in the paragraph document generation.

The illustration contains all kinds of media elements. The media elements can easily be imported, but equally important is the creation of elements inside the application.

Words make up the fundament of the system. Our words. Whatever we relate and associate to a subject or idea. Between these words are relations, connecting the word together in a theoretically infinitely complex web. Through this application, though, the structure of the web must become comprehensible.
Groups of words form entities that themselves form entities. Structures join to form superstructures. The combinations of the major entities are landscapes.
These relations between the words and entities are a basic and very important feature. It is the relations that give the idea a structure.
Viewed at a distance, one major landscape emerges. Removing ourselves further, even this becomes an entity to join other entities. Perhaps at one point it takes the shape of a word - and so the system is perfectly scalable, fractally, like the universe.

People have different views and thoughts. Their views of the expressed ideas are different too. The Sea of Thought remembers the peoples' views. A view describes a relation between entities - arrangements fluctuate but are not random. People can grab entities and move them. This creates the landscapes. New landscapes describing different aspects of the subject.
When someone sees a new landscape, it can be hard to understand what the landscape is about. But after an introduction, it should be easy to navigate into the theme. As a thought gets a volume, it is easier to identify and discuss it.
When the computer knows about your preferences and knowledge, it can modify the entities and the landscape so it suits you upon introduction. Of course, seeing someone else's view of an idea can be confusing. But this is also the way we learn more about the idea.
One person's view is one out of infinitely many. How can a view be shared? Perhaps by giving out abstract glasses. A collection of glasses is a group's collective view of an idea, a project, a problem. Perhaps a solution can be viewed through the glasses - or there might be no answer.

case study: travel planning (illustrated in the video)


identification of the two major problems

relation generation

Relations between words and entities should be generated smoothly, so that when the user speaks about or points at a relation, this relation should visualise by itself in the user interface. Thus the application must have some recognition technology to understand the syntax of the users' language.
Even when the technology works quite smoothly, the user sometimes has to reach out and create, alter or destroy relations between words. This operation has to be performed with gestural actions. They do not have to break down the discussion. For instance, words that are mentioned will appear near the surface of the interface, and they will more easily trigger for manipulation than other objects.
entity generation and the universal icon language
Essential to the thought pond is the natural generation of visible entities or icons. When a user navigates, visual symbols appear at all levels of scale. It is crucial that the entities are instantly recognisable or understandable, at least to the experienced user. Here the project might touch with Tuomas Karttunen's idea of the universal icon language.
Ideally, the entities should be generated automatically or semi-automatically in interaction with one or more user. The automated generation of entities must base themselves on an understanding of human perception and cognitive ability.
This task seems to overflow with 'magic', but some approaches are possible. It is imaginable that the computer, based on all the information it can gather from the structure and the discussion about it, can perform a deep search for similar structures in other documents available. It then presents the users a palette of icons from these structures. The users can select one or more objects and modify them to suit their needs. When millions of documents are created the search-based suggestions needn't be too bad, and a significant effort on icon building has to be performed only when conceptually new areas are researched.
Creating theories / laws / rules for these icons would be a major goal and would solve major problems in the field of using symbols to identify abstracts. This would be beneficial for far more than the applications proposed here. The vision is challenging, though: A universal, immediately understandable multimedia-Esperanto.

minor subjects

working environment, co-operation and level of submersion
"Gestural actions are a prominent and productive aspect of the group's activity" - John Tang, Xerox PARC with Stanford University.
The application should be viewable in all modes of visualisation of and benefit from the properties of these. Basically a small, framed display is good enough for personal use, but use of more sophisticated augmented realities will be good for collaborating purposes. It is the collaboration method that most strongly suggests the level of electronic visualisation in the working environment. As a suggestion I have provided the following interfaces.
Navigating the structure will not be very difficult. Tuomas Karttunen's reality layers allow us look at different aspects of the structure. An important aspect is to clearly point out which element are made or manipulated by the active projects and which are collected elsewhere.
document generation

When sufficient information has been given to the system, and the total concept is built, the landscape has been shaped. As previously mentioned, the concept is modular, giving it a different nature than documents. It is no predefined beginning or end, no narration, no storyline. In most cases such a document is the desired format of the work output, and so a document has to be created.
If the development has been somewhat linear, and especially if there originally existed a thorough work assignment, this is the point where the application should be able to generate an outline - a disposition - of the idea contained in the landscape. If the contained ideas are not so simple, the user will make the storyline himself, telling the system how to order the entities.
The disposition would serve as the basis for document creation in the future version of the word processor - the report creation tool. When a document or presentation has been made, it will be stored as a special entity in the landscape. Of course, many storylines and many corresponding documents can be stored in the same landscape. Different viewpoints and different purposes need different documents. They all benefit from a common knowledge base.
The landscape will have a specific digital format whereas the documents will not, since they can be anything from a letter to a song to an interactive movie. It is therefore easy to believe that the landscape will be stored in a database, while the documents will be stored in a file directory structure. The data objects will point to each other.

The application as a presentation tool - Preparation of presentations - Characteristics of the presentation

submission of information to search systems

All information stored digitally must have some kind of format - in this case, a database structure is needed. A document derived from an abstract landscape will be extremely suitable for search system registration. If the documents of the net would have such headers, not only would it be possible finding documents of all kinds about particular subjects - searching for documents with a specific viewpoint would also be possible.

accessibility for all

The abstract landscape also makes an excellent reference for applications that deal with translation, either to other languages or other media formats.
sources of inspiration for further development

visual programming languages

Visual programming languages like Director and Max provides the iconisation of ideas. At least Max is also zoomable - one icon can contain more icons of the same kind. But they can only build applications - their intention is not to let ideas grow.


Thinkmap from Plumbdesign is an experimental thesaurus browser. The words float in a space, continuously moving, almost as if they were floating in a body of calm water. Words with relations are connected together by lines. The application is written in Java and is viewable through a modern browser.

how to move forward?

This project is not straightforward to illustrate. Although the relation generation will be tough to program, it can be faked for the exhibition in May. Therefore the entity generation seems the most difficult problem to solve. It will be difficult to find icons and graphics good enough to give the user a real idea about how the application would behave in real life.
To move forward in this field would probably involve getting help from such people as linguists, psychologists, graphic designers and even mathematicians.