Showing posts with label Study and research. Show all posts
Showing posts with label Study and research. Show all posts

Thursday, June 23, 2011

Knowledge-based Programming to Knowledge Representation: The Basic Tasks

We continue from the earlier post. As in the earlier post, much of this post content are taken from Franz Baader's lecture notes (freely available on the web) of which I claim no authorship whatsoever.

Knowledge-based programming can be generalized into an area called knowledge representation. More precisely, a prerequisite for knowledge-based programming is that the relevant knowledge needs, in some way, to be represented symbolically, and thus stored in a machine/computer. Obviously, the way knowledge is represented should enable a machine, or in this case, a general problem solver to process it and answer the queries given to it. Again, following Franz Baader's lecture notes, the following are the important subtasks in knowledge representation:

  1. knowledge acquisition;
  2. symbolic representation;
  3. working with and applying the represented knowledge.

Knowledge acquisition

Knowledge acquisition is a difficult task because essentially, knowledge that we would like to represent is human's knowledge. After all, the notion of knowledge makes much more sense when discussed within human context; it is arguably absurd to talk about knowledge of a table, knowledge of a stone, knowledge of a bird (well it may have something that resembles knowledge, but strictly speaking, we would call it instinct, rather than knowledge), or knowledge of a book (a book may contain knowledge, though in the end what it contains is human's knowledge).

There are various ways to acquire knowledge about the application domain which is needed to solve the relevant problem, for example, by interviewing experts, mining documents, etc. Obviously, this implies that we have to start from a very informal, imprecise, and incomplete description of the domain to a more formal, machine processable specification. This topic is a very broad research area that overlaps with many other research area including learning and knowledge engineering. For our discussion, we will not deal with this issue explicitly.  The reader are recommended to look at other literatures which can be easily found online. From now on, we simply assume that we have a way to acquire knowledge in an effective manner.

Symbolic representation

There are several meaning of the word "representation" if we look into dictionary. We do not intend to discuss them in details, but rather, we understand that representation indicates a connection between things in the real world and things that we use to depict them. For our discussion, symbolic representation is actually a depiction or description of some real world entity that is symbolic in nature. In the research area of knowledge representation and reasoning, the following questions are discussed [Baader, 2005].

  • What is the exact connection between the symbolic representation and the real world entities it is supposed to describe?
  • What formalism is used to represent the knowledge?
  • What is the meaning (semantics) of the chosen formalism?

These questions give rise to a plethora of formalisms, methods, techniques, approaches, whatever --- you name it, to represent knowledge. In a more general sense, knowledge representation can be classified into ones which are symbolic and ones which are not. The latter includes artificial neural network or connectionist systems and evolutionary computation. The other approaches like fuzzy systems and statistical-based systems, although heavily non-symbolic in nature, can be integrated with symbolic approaches in a rather seamless manner. The third question above is, in particular, frequently not applicable (at least in a rather straightforward way) to non-symbolic approaches like artificial neural network. It is undeniable that such systems do work and solve problems in applications robustly and efficiently. However, such systems typically lack semantics. For example, take any artificial neural network system which has been trained for some period of time to classify documents according to their topics. Presumably, the weights of connections between neurons in that system represent the knowledge necessary to perform the classification correctly. Unfortunately, there is no way for us to understand what is the relationship between certain configuration of numerical weights of connections between neurons and the fact that document A has topic X or document B has topic Y.  Such a gap is very difficult to close, despite the efforts that have been made so far, e.g., research in neural-symbolic integration (see e.g., the  7th International Workshop on Neural-Symbolic Learning and Reasoning (NeSy'11), or the "Perspective of Neural Symbolic Integration" book).

For the purpose of our discussion here, we would only concern ourselves with symbolic based systems, more specifically, logic-based knowledge representation (LBKR) systems.  Aside from the above three questions, research in LBKR also investigates the expressivity aspect, i.e., the adequacy of the expressive power of the chosen formalism. The question of expressive power essentially asks what statements can be formulated and what cannot within a given formalism which is restricted to the representational means defined for it. The more expressive a formalism is, the more knowledge can be presented. Of course, such a formalism can be considered adequate only when all the relevant knowledge can be presented. As an example, the sentence

There exists no prime number that is greater than every other prime number

cannot be adequately expressed using propositional Boolean logic that only uses connectives like conjunction (AND), disjunction (OR) and negation (NOT). This example illustrates that the expressive power of propositional logic formalism is not adequate to express the above sentence. On the other hand, it is also a common question to ask whether all of the representational means attributed to the formalism are really needed, that is, whether some of the representational means is redundant. For example, in propositional Boolean logic, it is sufficient to use only conjunction and negation to express all statements that can be expressed using propositional Boolean logic. In other words, the disjunction operator is redundant in the presence of conjunction and negation.

On the semantics, LBKR systems are expected not to depend solely on any procedural semantics. This means that the semantics should never depend on particular programs that are built to do whatever tasks which are assigned to the systems. If this is the case, then the semantics, i.e., the meaning, of knowledge would change if the program is changed. Instead, they should have declarative semantics, hence separating the representation and the programs that are used to process it. A declarative semantics essentially consists of an abstraction of the world that we want to represent together with a mapping from any symbolic expression allowed in the formalism to an element in the abstraction. Such an abstraction is often stated in a form of a set or other algebraic structures. Representational means of the formalism usually allows us to build more complex symbolic expressions from simpler ones. Thus, the mapping given by the semantics typically associates the representational means with some operations on the abstraction. In addition, declarative semantics specifies a notion of "truth" which is used to determine whether a symbolic expression is true in the world induced by the abstraction. Moreover, the relationship between representational means and operations on the abstraction allows for the "truth" of a symbolic expression to actually be computed.

Working with and applying the represented knowledge

LBKR systems, in addition to the above requirements, should allow users to do some kind of manipulation in the form of [Baader, 2005]:

  • adding new knowledge to existing knowledge bases;
  • removing knowledge from a knowledge base;
  • accessing the explicitly stored knowledge;
  • accessing the implicitly stored knowledge

The above activities can be viewed as knowledge management activities which similarly exist for database systems. In fact, in some sense, a (relational) database system can be considered as a form of knowledge-based system. This fact can be easily seen if we see a database as a set of relation which can be expressed as simple first-order logic statements. Also, queries to a database, say stated using SQL, are actually first-order logic statements whose truth are tested against the relations in the database. The feature that a standard database system lacks is the ability to access implicitly stored knowledge. This feature implies that a LBKR system should be able not only to retrieve knowledge that is stored explicitly, but also to deduce implicit knowledge that is implied by the explicitly stored knowledge. Such a feature is called inference mechanism or inference procedure. Referring to my earlier post on "Classical Programming vs. Knowledge-based Programming: Example", the general problem solver mentioned there would be realized here as inference mechanisms. As an example, we use the one from that post; consider the following knowledge base:

[sourcecode language="ruby" light="true" wraplines="false"]

directly-connected(a,b).
directly-connected(a,c).
directly-connected(b,d).
directly-connected(b,e).
directly-connected(c,e).

connected(x,y) if directly-connected(x,y).
connected(x,y) if directly-connected(x,z) and connected(z,y).

[/sourcecode]

Here, directly-connected(a,b) is  explicit knowledge because it is explicitly stated in the knowledge base above, whereas directly-connected(a,c) is implicit knowledge because it is implied and not explicitly stated by the knowledge base.

Wednesday, June 22, 2011

Classical Programming vs. Knowledge-based Programming: Example

Continuing from my last post, let me give you an example on how classical programming differs from knowledge-based programming (the example is taken from Prof. Franz Baader's lecture notes). Consider the acyclic graph in the picture below.

With regards to the above acyclic graph, we would like to solve the following problem: given to node $latex x$ and $latex y$ in the graph, determine whether these two nodes are connected.

In the classical programming, this problem can be solved by giving a procedure that gives you precisely the desired answer, namely "yes" if  both nodes are connected, and "no" otherwise. This procedure might be something like this:

[sourcecode language="ruby" light="true" wraplines="false"]

procedure connected(x,y)
if x = a then
if y = b or y = c then
return true
else
return (connected(b,y) or connected(c,y))
else
if x = b then
if y = d or y = e then
return true
else
return false
else
if x = c then
if y = e then
return true
else
return false
else
return false
[/sourcecode]

It is straightforward to see that if we ask whether $latex e$ is connected to any node, the answer would be false. Now, what if the graph is changed, say by adding a new node $latex f$ and an edge from $latex e$ to $latex f$? Of course, the above procedure will return the wrong answer when we ask whether $latex e$ is connected to $latex f$. We might have to modify the whole program to incorporate this new case.

On the other hand, the knowledge-based programming will solve the above problem in a different way. First, the graph, i.e., the problem description and its restrictions are specified explicitly as the knowledge about graph and the meaning of "connected". For example, the following knowledge base gives a correct description of the graph and the "connectedness" between nodes in the graph.

[sourcecode language="ruby" light="true" wraplines="false"]

directly-connected(a,b).
directly-connected(a,c).
directly-connected(b,d).
directly-connected(b,e).
directly-connected(c,e).

connected(x,y) if directly-connected(x,y).
connected(x,y) if directly-connected(x,z) and connected(z,y).

[/sourcecode]

Next, against the above knowledge base, we give the following query:

[sourcecode language="ruby" light="true" wraplines="false"]

connected(a,b)?

[/sourcecode]

To answer the above query, we simply run a general problem solver and get the answer. Thus, classical programming focuses more on "how" to obtain the solution, while knowledge-based programming would focus more on "what" the solution would be like.

In practice, this distinction may not be as strict as what the example illustrated. Classical programming typically also allows parts of the program to contain the data structures that describe the problem specification, whereas some knowledge-based programming applications encode the knowledge about application in the program using procedural rules.

In knowledge representation and reasoning, one of the main research theme is about formulating and specifying languages that allow users to express knowledge symbolically and developing algorithms which can perform as a general problem solver for queries by users to the knowledge base.

Tuesday, June 21, 2011

Classical Programming vs. Knowledge-based Programming

I just read again Prof. Franz Baader's old lecture manuscript of Logic Based Knowledge Representation course. He taught this course in the International Master Programme in Computational Logic at Technische Universität Dresden during 2005's Summer Semester. There is an interesting passage that compares the so-called classical programming against knowledge-based programming. Classical programming is essentially the computer programming that you usually learn the first time you enter a computer science undergraduate program. On the other hand, knowledge-based programming is the programming that you typically do when you want to build an "intelligent" application. These are the differences between them according to him:

Classical programming is characterized with the following properties:

  • a specialized program that is tailored to a specific application;
  • knowledge about the problem domain and additional constraints are implicit in the structure of the program;
  • therefore, knowledge is "introduced" by the programmer;
  • if there are changes in the application, the program has to be rewritten.

On the other hand, knowledge-based programming has the following properties:

  • use of general problem solvers that are independent of the specific domain;
  • knowledge about the application domain is represented explicitly in a knowledge base;
  • this knowledge can be acquired by a knowledge engineer, independent of the problem solvers;
  • if there are changes in the application, only the knowledge base need to be changed, not the problem solver.

So in essence, there is a clear separation between the problem solver and the problem specification itself. The logic that leads to a solution to a problem in a specific domain is not encoded in the program, but rather, it is encoded as logical inferences which uncover implicit knowledge within the problem specification. This is what drives research in (logic-based) knowledge representation, the field of which is where I am working on. There are plethora of methods and formalisms on how to actually represent knowledge. One of them is Description Logic, which has found its way to applications in the Semantic Web. I am excited to be part of it, and hopefully this research area will also be picked up by more Indonesian researchers in the future.

Tuesday, January 11, 2011

Indonesia's Fulbright scholarships: AMINEF & Selection Process

There has been some emails to me asking about Fulbright scholarship for Indonesian. It seemed that some of the questions asked more about how the selection process for this scholarship is done in Indonesia. I think it would be useful if some of the things I know based on my own personal experience going through the selection process myself are recorded here. [Disclaimer: I will try to be as accurate as possible, but mistakes are possible. Any corrections are thus welcome].

Fulbright Programs and AMINEF

Fulbright programs in Indonesia are administered through a bi-national commission called AMINEF, which is an abbreviation of American Indonesian Exchange Foundation. In some other countries, Fulbright programs can also be administered by a similar commission or directly by the U.S. embassy or consular offices.

As expected from its name, AMINEF board consists of representatives from Indonesia and the U.S. with Indonesian Minister of National Education and U.S. Ambassador as the joint chairmen. AMINEF was established in 1992 and, based on what I've heard, historically has its root from USAID programs for Indonesia since 1950s. AMINEF's basic missions are, quoting from its official website, to administer various scholarship programs for both Indonesians and Americans, and to give information about education in the U.S. as well as what opportunities are available to study there. Currently, AMINEF administers the following programs for Indonesians:

  1. Fulbright Master's Degree Program
  2. Fulbright Freeport Master's Degree Program
  3. Fulbright - DIKTI Master's Program
  4. Fulbright Science and Technology Award for PhD
  5. Fulbright Presidential PhD Program
  6. Fulbright - DIKTI PhD Program
  7. Fulbright Doctoral Dissertation Research Program
  8. Fulbright - DIKTI Doctoral Dissertation Research Program
  9. Fulbright Senior Research Program
  10. Fulbright - DIKTI Senior Research Award Program
  11. Fulbright Foreign Language (Bahasa Indonesia) Teaching Assistant (FLTA) Program
  12. Hubert H. Humphrey Fellowship Program for Mid-Career Professionals
  13. Fulbright Visiting Specialist Program
  14. Fulbright Tsunami Relief Initiative Master's Degree Program
  15. Community College Summit Initiative Program
  16. International Leadership in Education Program (ILEP) - Teacher Exchange Program
  17. Global Undergraduate Exchange Program
  18. Fulbright KEMLU Program: designed for employee/diplomats of Indonesian Ministry of Foreign Affairs.

Besides those programs, AMINEF also administers several programs for Americans for study and research in Indonesia. You can find details of those programs including their requirements in AMINEF's website. What I am going to write in more detail here is related to the Fulbright Master's and PhD scholarships selection process done in Indonesia (program #1 and #5 in the list above). Other countries may have different schemes for the selection, though some of the things may be applicable as well.

Selection Process for Fulbright Master's and PhD Programs

As seen in the list above, Indonesian Fulbright scholarships for study at Master's and PhD level are designed in several schemes.  Although I've obviously never gone through all those schemes by myself, I believe the selection process is similar. The difference between those schemes lies more on the eligibility criteria and the source of funding. Every Fulbright scholarships usually covers tuition fees, monthly stipends, health insurance, and airfares. In addition, for successful candidates, costs for TOEFL and GRE tests for placement in U.S. universities are covered.

Call for application is normally published in September - November of the previous year. AMINEF makes regular roadshow to several major Indonesian cities. These roadshow are typically held in major universities in the area. The call for application is also broadcasted via mailing lists, brochures, etc.

The deadline for submission of application is typically around April, from which the selection process starts. The selection process itself is divided into three phases:

  1. Document evaluation
  2. Interview
  3. Placement in the U.S. universities

In general, applicants are required to submit a completed application form (downloadable from AMINEF's website), two essays: personal motivation and study/research plan, and reference letter. Other documents may also be required, e.g., curriculum vitae, copy of ID card, degree certificate and academic transcript from earlier level of education, etc. In the document evaluation, all the documents submitted are evaluated based on the completeness, eligibility, as well as applicant's profile. The competition in this phase is particularly high. I've heard that the number of applicants may reach more than 800 for the master's level. On the other hand, the number of applicants who will be invited for interview is possibly in between 60-100 applicants for Master's level of which 20-30 Fulbright scholarship's principal candidates will be selected. For PhD level, the competition is slightly less where around 300 applicants submit their applications, 60-80 are invited for interview and 30-40 are selected as principal candidates. Note that though, this number is an estimate based on what I've gathered from discussions with fellow Fulbrighters and AMINEF's officers.

After the document evaluation, selected applicants will be invited for an interview session, usually in September or October. Since the applicants may come from any location in Indonesia, depending on the number of applicants in a particular region, AMINEF may hold interviews in several cities. This is done to lower the cost and logistics as they are all taken care of by AMINEF. Each interview will be for about 1 hour, and there will be 4-5 interviewers. In my case, I was interviewed by 3 Americans and 2 Indonesians. Judging from the questions by the interviewer, the interview generally aims to have an in-depth exploration about the applicant's academic and research capability as well as applicant's potential to have future impact in the society, scientific community and the increased understanding between people of both countries.

A couple of weeks after the interview, some of the applicants will be informed by AMINEF whether he/she is selected as a principal candidate or an alternate candidate for Fulbright scholarships. The corresponding applicant must confirm his/her selection as a candidate for Fulbright scholarship and whether he/she will continue to the next phase, i.e., placement in a U.S. university. A principal status means that the scholarship is basically granted, pending the placement in a U.S. university, whereas an alternate status means that the scholarship may be granted if there is a principal candidate that resigns from the selection, or more funding is available later.

After the interview phase, the selection process is continued with placement in U.S. universities. In this phase, an application plan is drawn for each candidate. This phase consists of preparing the required test scores: TOEFL, GMAT/GRE General, and GRE Subject (for specific subjects), improving essays for application (personal motivation and study/research plan), and adding more reference/recommendation letters (may need up to 4 letters). The TOEFL & GRE tests will be conducted, usually in October or November to catch up with the application deadline of most U.S. universities in December. International Institute of Education (IIE) that manages most Fulbright programs all over the world coordinates the placement with the help of AMINEF. Based on the test scores, the candidate's academic interest and availability of suitable degree programs in the U.S., IIE suggests a list of universities to which a candidate's application will be submitted. The list also takes into account the candidate's personal preference and IIE will ask for the candidate's approval before they go ahead with submitting the application. The good thing with this process is that the candidate needs not to pay the application fees which are typically charged by U.S. universities for the application. Such fees may reach USD 75 or more for each application which is quite expensive especially if we apply to more than one universities. For example, in my case, my application was sent to 6 universities.

After submitting the applications, the candidate only needs to wait for the results whether he/she is accepted to a U.S. university. This may take a few months because some U.S. universities sometimes decide the acceptance as late as June. If a U.S. university decides that the candidate can be admitted, it will send an admission offer, through IIE. Otherwise, the candidate is usually informed that he/she is not admitted to the university. After receiving some admission offers, a candidate can decide which offer that he/she would like to take. Around this time, AMINEF will also organize a Pre-departure Orientation as a preparation before leaving to the U.S. The candidate will also be asked to prepare for his/her passport, medical checkup and other documents related to his/her departure. IIE will also inform whether a pre-academic preparation is required/offered for the candidate in the U.S. This preparation can be either a couple of weeks of language preparation or a one-week gateway program, if the candidate's English is already very good.

The candidate will also be asked to apply for a U.S. visa. There will be some help from AMINEF on visa application (preparation for DS-2019 form, guidelines, appointment for visa interview), but in general, the candidate will have to go through the application process by him/herself. This consists of document preparation, filling DS-160 online form, and visa interview. The visa application itself is free of charge for Fulbright scholars (and his/her spouse and children, if they come along), which is very nice because otherwise,  the candidate must pay around US$ 140, which is a lot. If there's no problem with the visa, then usually in August, the candidate will leave to the U.S., starting the Master's or PhD program from the Fall quarter/semester of the year.

Wednesday, November 17, 2010

Fulbright Master's scholarship 2011 (to study in the U.S.)

This one is an advertising... :-)

Somebody posted this info on milis beasiswa (beasiswa (at) yahoogroups.com), and as a Fulbrighter myself, I feel obliged to spread this too... :-) Preference will be given to applicants serving as faculty members of private institutions of higher education in Indonesia, but that does not mean that those who're not a faculty member are not eligible or have no chance at all. It's alright if you're not a full time faculty yet (which is possibly because you only have bachelor's degree). The definition of "faculty member" here also applies if you're only a research or teaching assistant at a higher education institution.

For those who's interested in pursuing a PhD degree, Fulbright also has separate scholarship programs for PhD.

This scholarship is very prestigious and competitive, even more so for the Master's level. Good luck to you who want to try...

===========================

Fulbright Master Fulbright Master Scholarships, American Indonesian Exchange Foundation (AMINEF), USA

Preference will be given to applicants who serve as faculty members of state and private institutions of higher education in Indonesia. Applicants will possess:

  • a Bachelor (S1) degree with a minimum GPA of 3.0 (4.00 
  • leadership 
  • a good understanding of Indonesian and international 
  • demonstrated commitment to the chosen field of study
  • a willingness to return to Indonesia upon completion of the Fulbright program
  • a minimum institutional TOEFL score of 550 or IELTS score of 6.0

DISCIPLINES
All disciplines and fields of study are eligible for a Fulbright award. The fields of study covered by the Fulbright scholarship program in the past were limited to the Social Sciences and Humanities.  Since 2004, however, the policy has been changed to include also other disciplines with the exception of fields related to patient care or medical training nor to continue their study at medical school.

HOW TO APPLY
Candidates should complete the appropriate application forms. Forms are available either by mail or in person at the AMINEF Office, Gedung Balai Pustaka, 6th floor, Jl. Gunung Sahari Raya 4, Jakarta 10720. Please return to AMINEF your complete application package by the application deadline that includes:

  • Completed application form. This includes a clearly written and concise study objective.
  • Copy of your most recent, less than two years old, institutional TOEFL or IELTS score report.
  • One letter of reference, either from your current employer or previous lecturer.
  • Copy of academic transcript (English translation).
  • Copy of identity document (KTP or passport).

CONTACT INFORMATION
Specific questions regarding the application process may be addressed via e-mail to the following address: infofulbright_ind (at) aminef.or.id. We do not accept email applications. Hard copies must be sent or delivered to American Indonesian Exchange Foundation.

DEADLINE
The deadline for the submission of application materials for all programs is April 30, 2011.
Note: Program requirements are subject to change without notice.
The form can be downloaded here.

http://www.aminef.or.id/file/fulbright/2010%20FULBRIGHT%20APPLICATION%20FORM.doc

Friday, October 29, 2010

For PhD students who are getting started …

I just came across an interesting post by Matt Welsh from Harvard. He advised a few things to those PhD students whom are getting started their long endeavor towards a successful PhD research, including myself of course.

  • Don't let school get in your way of your education.
  • Just dive in and have no fear.
  • Don't read too much (at first).
  • Keep track of the papers that you read.
  • Take a lot of notes.

Reflecting the advice to my (current) situation, I found that none of the items checked off. I guess I am not yet in full shape of doing the PhD research. :'(