Knowledge-based Programming to Knowledge Representation: The Basic Tasks

We continue from the earlier post. As in the earlier post, much of this post content are taken from Franz Baader's lecture notes (freely available on the web) of which I claim no authorship whatsoever.

Knowledge-based programming can be generalized into an area called knowledge representation. More precisely, a prerequisite for knowledge-based programming is that the relevant knowledge needs, in some way, to be represented symbolically, and thus stored in a machine/computer. Obviously, the way knowledge is represented should enable a machine, or in this case, a general problem solver to process it and answer the queries given to it. Again, following Franz Baader's lecture notes, the following are the important subtasks in knowledge representation:

1. knowledge acquisition;
2. symbolic representation;
3. working with and applying the represented knowledge.

Knowledge acquisition

Knowledge acquisition is a difficult task because essentially, knowledge that we would like to represent is human's knowledge. After all, the notion of knowledge makes much more sense when discussed within human context; it is arguably absurd to talk about knowledge of a table, knowledge of a stone, knowledge of a bird (well it may have something that resembles knowledge, but strictly speaking, we would call it instinct, rather than knowledge), or knowledge of a book (a book may contain knowledge, though in the end what it contains is human's knowledge).

There are various ways to acquire knowledge about the application domain which is needed to solve the relevant problem, for example, by interviewing experts, mining documents, etc. Obviously, this implies that we have to start from a very informal, imprecise, and incomplete description of the domain to a more formal, machine processable specification. This topic is a very broad research area that overlaps with many other research area including learning and knowledge engineering. For our discussion, we will not deal with this issue explicitly.  The reader are recommended to look at other literatures which can be easily found online. From now on, we simply assume that we have a way to acquire knowledge in an effective manner.

Symbolic representation

There are several meaning of the word "representation" if we look into dictionary. We do not intend to discuss them in details, but rather, we understand that representation indicates a connection between things in the real world and things that we use to depict them. For our discussion, symbolic representation is actually a depiction or description of some real world entity that is symbolic in nature. In the research area of knowledge representation and reasoning, the following questions are discussed [Baader, 2005].

• What is the exact connection between the symbolic representation and the real world entities it is supposed to describe?
• What formalism is used to represent the knowledge?
• What is the meaning (semantics) of the chosen formalism?

These questions give rise to a plethora of formalisms, methods, techniques, approaches, whatever --- you name it, to represent knowledge. In a more general sense, knowledge representation can be classified into ones which are symbolic and ones which are not. The latter includes artificial neural network or connectionist systems and evolutionary computation. The other approaches like fuzzy systems and statistical-based systems, although heavily non-symbolic in nature, can be integrated with symbolic approaches in a rather seamless manner. The third question above is, in particular, frequently not applicable (at least in a rather straightforward way) to non-symbolic approaches like artificial neural network. It is undeniable that such systems do work and solve problems in applications robustly and efficiently. However, such systems typically lack semantics. For example, take any artificial neural network system which has been trained for some period of time to classify documents according to their topics. Presumably, the weights of connections between neurons in that system represent the knowledge necessary to perform the classification correctly. Unfortunately, there is no way for us to understand what is the relationship between certain configuration of numerical weights of connections between neurons and the fact that document A has topic X or document B has topic Y.  Such a gap is very difficult to close, despite the efforts that have been made so far, e.g., research in neural-symbolic integration (see e.g., the  7th International Workshop on Neural-Symbolic Learning and Reasoning (NeSy'11), or the "Perspective of Neural Symbolic Integration" book).

For the purpose of our discussion here, we would only concern ourselves with symbolic based systems, more specifically, logic-based knowledge representation (LBKR) systems.  Aside from the above three questions, research in LBKR also investigates the expressivity aspect, i.e., the adequacy of the expressive power of the chosen formalism. The question of expressive power essentially asks what statements can be formulated and what cannot within a given formalism which is restricted to the representational means defined for it. The more expressive a formalism is, the more knowledge can be presented. Of course, such a formalism can be considered adequate only when all the relevant knowledge can be presented. As an example, the sentence

There exists no prime number that is greater than every other prime number

cannot be adequately expressed using propositional Boolean logic that only uses connectives like conjunction (AND), disjunction (OR) and negation (NOT). This example illustrates that the expressive power of propositional logic formalism is not adequate to express the above sentence. On the other hand, it is also a common question to ask whether all of the representational means attributed to the formalism are really needed, that is, whether some of the representational means is redundant. For example, in propositional Boolean logic, it is sufficient to use only conjunction and negation to express all statements that can be expressed using propositional Boolean logic. In other words, the disjunction operator is redundant in the presence of conjunction and negation.

On the semantics, LBKR systems are expected not to depend solely on any procedural semantics. This means that the semantics should never depend on particular programs that are built to do whatever tasks which are assigned to the systems. If this is the case, then the semantics, i.e., the meaning, of knowledge would change if the program is changed. Instead, they should have declarative semantics, hence separating the representation and the programs that are used to process it. A declarative semantics essentially consists of an abstraction of the world that we want to represent together with a mapping from any symbolic expression allowed in the formalism to an element in the abstraction. Such an abstraction is often stated in a form of a set or other algebraic structures. Representational means of the formalism usually allows us to build more complex symbolic expressions from simpler ones. Thus, the mapping given by the semantics typically associates the representational means with some operations on the abstraction. In addition, declarative semantics specifies a notion of "truth" which is used to determine whether a symbolic expression is true in the world induced by the abstraction. Moreover, the relationship between representational means and operations on the abstraction allows for the "truth" of a symbolic expression to actually be computed.

Working with and applying the represented knowledge

LBKR systems, in addition to the above requirements, should allow users to do some kind of manipulation in the form of [Baader, 2005]:

• adding new knowledge to existing knowledge bases;
• removing knowledge from a knowledge base;
• accessing the explicitly stored knowledge;
• accessing the implicitly stored knowledge

The above activities can be viewed as knowledge management activities which similarly exist for database systems. In fact, in some sense, a (relational) database system can be considered as a form of knowledge-based system. This fact can be easily seen if we see a database as a set of relation which can be expressed as simple first-order logic statements. Also, queries to a database, say stated using SQL, are actually first-order logic statements whose truth are tested against the relations in the database. The feature that a standard database system lacks is the ability to access implicitly stored knowledge. This feature implies that a LBKR system should be able not only to retrieve knowledge that is stored explicitly, but also to deduce implicit knowledge that is implied by the explicitly stored knowledge. Such a feature is called inference mechanism or inference procedure. Referring to my earlier post on "Classical Programming vs. Knowledge-based Programming: Example", the general problem solver mentioned there would be realized here as inference mechanisms. As an example, we use the one from that post; consider the following knowledge base:

[sourcecode language="ruby" light="true" wraplines="false"]

directly-connected(a,b).
directly-connected(a,c).
directly-connected(b,d).
directly-connected(b,e).
directly-connected(c,e).

connected(x,y) if directly-connected(x,y).
connected(x,y) if directly-connected(x,z) and connected(z,y).

[/sourcecode]

Here, directly-connected(a,b) is  explicit knowledge because it is explicitly stated in the knowledge base above, whereas directly-connected(a,c) is implicit knowledge because it is implied and not explicitly stated by the knowledge base.

Related articles

• Classical Programming vs. Knowledge-based Programming (krisnadhi.wordpress.com)
• Classical Programming vs. Knowledge-based Programming: Example (krisnadhi.wordpress.com)

Classical Programming vs. Knowledge-based Programming: Example

Continuing from my last post, let me give you an example on how classical programming differs from knowledge-based programming (the example is taken from Prof. Franz Baader's lecture notes). Consider the acyclic graph in the picture below.

With regards to the above acyclic graph, we would like to solve the following problem: given to node $latex x$ and $latex y$ in the graph, determine whether these two nodes are connected.

In the classical programming, this problem can be solved by giving a procedure that gives you precisely the desired answer, namely "yes" if  both nodes are connected, and "no" otherwise. This procedure might be something like this:

[sourcecode language="ruby" light="true" wraplines="false"]

procedure connected(x,y)
if x = a then
if y = b or y = c then
return true
else
return (connected(b,y) or connected(c,y))
else
if x = b then
if y = d or y = e then
return true
else
return false
else
if x = c then
if y = e then
return true
else
return false
else
return false
[/sourcecode]

It is straightforward to see that if we ask whether $latex e$ is connected to any node, the answer would be false. Now, what if the graph is changed, say by adding a new node $latex f$ and an edge from $latex e$ to $latex f$? Of course, the above procedure will return the wrong answer when we ask whether $latex e$ is connected to $latex f$. We might have to modify the whole program to incorporate this new case.

On the other hand, the knowledge-based programming will solve the above problem in a different way. First, the graph, i.e., the problem description and its restrictions are specified explicitly as the knowledge about graph and the meaning of "connected". For example, the following knowledge base gives a correct description of the graph and the "connectedness" between nodes in the graph.

[sourcecode language="ruby" light="true" wraplines="false"]

directly-connected(a,b).
directly-connected(a,c).
directly-connected(b,d).
directly-connected(b,e).
directly-connected(c,e).

connected(x,y) if directly-connected(x,y).
connected(x,y) if directly-connected(x,z) and connected(z,y).

[/sourcecode]

Next, against the above knowledge base, we give the following query:

[sourcecode language="ruby" light="true" wraplines="false"]

connected(a,b)?

[/sourcecode]

To answer the above query, we simply run a general problem solver and get the answer. Thus, classical programming focuses more on "how" to obtain the solution, while knowledge-based programming would focus more on "what" the solution would be like.

In practice, this distinction may not be as strict as what the example illustrated. Classical programming typically also allows parts of the program to contain the data structures that describe the problem specification, whereas some knowledge-based programming applications encode the knowledge about application in the program using procedural rules.

In knowledge representation and reasoning, one of the main research theme is about formulating and specifying languages that allow users to express knowledge symbolically and developing algorithms which can perform as a general problem solver for queries by users to the knowledge base.

Related articles

• Classical Programming vs. Knowledge-based Programming

Classical Programming vs. Knowledge-based Programming

I just read again Prof. Franz Baader's old lecture manuscript of Logic Based Knowledge Representation course. He taught this course in the International Master Programme in Computational Logic at Technische Universität Dresden during 2005's Summer Semester. There is an interesting passage that compares the so-called classical programming against knowledge-based programming. Classical programming is essentially the computer programming that you usually learn the first time you enter a computer science undergraduate program. On the other hand, knowledge-based programming is the programming that you typically do when you want to build an "intelligent" application. These are the differences between them according to him:

Classical programming is characterized with the following properties:

• a specialized program that is tailored to a specific application;
• knowledge about the problem domain and additional constraints are implicit in the structure of the program;
• therefore, knowledge is "introduced" by the programmer;
• if there are changes in the application, the program has to be rewritten.

On the other hand, knowledge-based programming has the following properties:

• use of general problem solvers that are independent of the specific domain;
• knowledge about the application domain is represented explicitly in a knowledge base;
• this knowledge can be acquired by a knowledge engineer, independent of the problem solvers;
• if there are changes in the application, only the knowledge base need to be changed, not the problem solver.

So in essence, there is a clear separation between the problem solver and the problem specification itself. The logic that leads to a solution to a problem in a specific domain is not encoded in the program, but rather, it is encoded as logical inferences which uncover implicit knowledge within the problem specification. This is what drives research in (logic-based) knowledge representation, the field of which is where I am working on. There are plethora of methods and formalisms on how to actually represent knowledge. One of them is Description Logic, which has found its way to applications in the Semantic Web. I am excited to be part of it, and hopefully this research area will also be picked up by more Indonesian researchers in the future.

Quotes from Edgar W. Dijkstra

The other day, I stumbled upon the Memorial Resolution about Edgar Wybe Dijkstra, whom you may have known as the creator of the now-famous Dijkstra shortest path algorithm. Dijkstra was a professor at the Eindhoven University of Technology, and later at University of Texas at Austin (1984 - 2002). He was the winner of 1972's ACM Turing Award, the "Nobel prize of computer science". The full text of the Memorial Resolution containing a short but very interesting and inspiring biography of his life can be accessed at: http://www.utexas.edu/faculty/council/2002-2003/memorials/Dijkstra/dijkstra.html. I just want to quote some of the things there which I really found inspiring.

On his project of "Streamlining Mathematical Arguments", he said:

"As a matter of fact, the challenges of designing high-quality programs and of designing high-quality proofs are very similar, so similar that I am no longer able to distinguish between the two: I see no meaningful difference between programming methodology and mathematical methodology in general. The long and short of it is that the computer’s ubiquity has made the ability to apply mathematical method[s] more important than ever."

On teaching formal methods (which was considered radical novelties) in the university, he said:

"Teaching to unsuspecting youngsters the effective use of formal methods is one of the joys of life because it is so extremely rewarding. Within a few months, they find their way in a new world with a justified degree of confidence that is radically novel for them; within a few months, their concept of intellectual culture has acquired a radically novel dimension. To my taste and style that is what education is about. Universities should not be afraid of teaching radical novelties; on the contrary, it is their calling to welcome the opportunity to do so. Their willingness to do so is our main safeguard against dictatorships, be they of the proletariat, of the scientific establishment, or of the corporate elite."

He consider creating a teachable material must be one of the main objectives of a research:

"For me, the first challenge for computing science is to discover how to maintain order in a finite, but very large, discrete universe that is intricately intertwined. And a second, but not less important challenge is how to mould what you have achieved in solving the first problem, into a teachable discipline: it does not suffice to hone your own intellect (that will join you in your grave), you must teach others how to hone theirs. The more you concentrate on these two challenges, the clearer you will see that they are only two sides of the same coin: teaching yourself is discovering what is teachable."

An interesting fact about him:

Dijkstra enjoyed the natural beauty of Austin and the surrounding hill country. He and his wife had a fondness for exploring state and national parks in their Volkswagen bus, dubbed the Touring Machine, in which he wrote many of his technical papers. They had a constant stream of visitors in their home, for whom he enjoyed playing Mozart on his Bösendorfer piano.

The introduction given for his 1972's ACM Turing Award:

The working vocabulary of programmers is studded with words originated or forcefully promulgated by E. W. Dijkstra: display, deadly embrace, semaphore, go-to-less programming, structured programming. But his influence on programming is more pervasive than any glossary can possibly indicate. The precious gift that this Turing Award acknowledges is Dijkstra’s style: his approach to programming as a high, intellectual challenge; his eloquent insistence and practical demonstration that programs should be composed correctly, not just debugged into correctness; and his illuminating perception of problems at the foundations of program design. He has published about a dozen papers, both technical and reflective, among which are especially to be noted his philosophical address at IFIP, his already classic papers on cooperating sequential processes, and his memorable indictment of the go-to statement. An influential series of letters by Dijkstra have recently surfaced as a polished monograph on the art of composing programs. We have come to value good programs in much the same way as we value good literature. And at the center of this movement, creating and reflecting patterns no less beautiful than useful, stands E. W. Dijkstra.

His thought about communication problems through writing:

At a given moment, the concept of polite mathematics emerged, the underlying idea of which is that, even if you have only 60 readers, it pays to spend an hour if by doing so you can save your average reader a minute. By inventing an idealized “average reader”, we could translate most of the lofty human goal of politeness into more or less formal criteria we could apply to our texts.

Other quotes from him:

Computer Science is no more about computers than astronomy is about telescopes.

A formula is worth a thousand pictures.

Always design your programs as a member of a whole family of programs, including those that are likely to succeed it.

Separate Concerns.

A Programming Language is a tool that has profound influence on our thinking habits.

The competent programmer is fully aware of the strictly limited size of his own skill; therefore he approaches the programming task in full humility, and among other things he avoids clever tricks like the plague.

Progress is possible only if we train ourselves to think about programs without thinking of them as pieces of executable code.

Program testing can at best show the presence of errors but never their absence.

... if 10 years from now, when you are doing something quick and dirty, you suddenly visualize that I am looking over your shoulders and say to yourself, Dijkstra would not have liked this, well that would be enough immortality for me.

As a matter of fact the still often repeated requirement that axioms should be self evident strikes me as a medieval relic: to the extent that they take philosophy seriously, it is impossible for me to take logicians seriously. (Again this may be a cultural difference: it seems there are societies in which philosophers still have some intellectual standing.)

The use of COBOL cripples the mind; its teaching should, therefore, be regarded as a criminal offense.

Being abstract is something profoundly different from being vague.

The problems of the real world are those that remain when you ignore their known solutions.

The prisoner falls in love with his chains. (In reference to programmers using inadequate tools.)

I pray daily that more of my fellow programmers may find the means of freeing themselves from the curse of compatibility.

Brainpower is by far our scarcest resource.

My daughter and I taking a shower with equal frequency is a frightening thought for both of us. (February 21,1984; commenting on a mutual exclusion algorithm in which the two components are alternately granted access to the critical section.)

Nothing is as expensive as making mistakes.

If you carefully read its literature and analyze what its devotees actually do, you will discover that software engineering has accepted as its charter, “How to program if you cannot.”

We must give industry not what it wants, but what it needs.

Waiting is a very funny activity: you can’t wait twice as fast.

Do not try to change the world. Give the world the opportunity to change itself.

So-called natural language is wonderful for the purposes it was created for, such as to be rude in, to tell jokes in, to cheat or to make love in (and Theorists of Literary Criticism can even be content-free in it), but it is hopelessly inadequate when we have to deal unambiguously with situations of great intricacy, situations which unavoidably arise in such activities as legislation, arbitration, mathematics or programming. (Foreword to Teaching and Learning Formal Methods, edited by C. N. Dean and M. G. Hinchey, Academic Press, 1996.)

The traditional mathematician recognizes and appreciates mathematical elegance when he sees it. I propose to go one step further, and to consider elegance an essential ingredient of mathematics: if it is clumsy, it is not mathematics.

Don’t compete with me: firstly, I have more experience, and secondly, I have chosen the weapons. (During first lecture in Capita Selecta, August 29, 1996.)

Maintaining a large range of agilities mental and physical requires regular exercise [...]. That is why the capable are always busy. (Lecture, Capita Selecta, October 10, 1996.)

Mathematicians are like managers; they want improvement without change. (During a meeting of the Austin Tuesday Afternoon Club, Fall 1996.)

... I had already come to the conclusion that in the practice of computing, where we have so much latitude for making a mess of it, mathematical elegance is not a dispensable luxury, but a matter of life and death.

Hans Rosling's Washing Machine and World's Energy Consumption

http://www.youtube.com/watch?v=BZoKfap4g4w

Another interesting video on washing machine and the world's energy consumption. And it's funny too!

Like money, out of all the world's energy, the richest people consume the most. Thus, until they have the same energy consumption per person, they shouldn't give an advice to others, what to do and what to not to do.

So.... who does not need a washing machine?

Microsoft's corporate culture from an open-source backer's point of view

http://www.zdnet.com/blog/microsoft/can-an-open-source-backer-thrive-inside-microsoft-this-one-says-no/9545?alertspromo=&tag=nl.rSINGLE

While I do agree that some competition is needed based on a meritocracy system, but too much competition may indeed be destructive, especially if this is within the scope of one organization.

It's an interesting corporate culture, as some of their products are #1 used worldwide (which of course also due to intensive marketing and an aggressive methods of selling, bordering on monopoly). Don't know how many of them that were considered a failure, though.

Rules of Brainstorming

Finally, an update after a while...

I just read an interesting article on how can an innovation be killed easily in any organizations. One interesting point in the article is that brainstorming is good and recommended, but often times, brainstorming just ends up killing good ideas. The reason is that because we often do brainstorming while ignoring the rules behind it. So what are the rules?

1. There is no such thing as a bad idea. Brainstorming is a place to throw in ideas. As such, any ideas should be regarded as valuable as the best idea out there. No matter how far-fetched an idea is, you never know that idea will precisely lead you to the one that is best for your problem.
2. Don't talk yet about "why not". Often times, a good idea is abandoned simply because we start thinking on why that idea might be not good for the problem and not fit the reality. Discussions on whether an idea is bad should be done in another time, not during brainstorming. Brainstorming should only be used to throw in ideas, not ruling them out.
3. Nothing should stifle the flow of ideas. Ideas have to flow freely. There should be no "buts". What we want to hear is the words such as "and", "or",  and "what if?".
4. Again, just to emphasize: there's no such thing as a bad idea.