Arteology: Planning an Empirical Study

Planning an Empirical Study

En Español   In Finnish   Contents

Context of a Research Project

Each research project has important relations to the existing theory of the pertinent field of research, and usually also to empiria, the tangible world of people, objects and events. These relations determine which methods of research are possible and effectual, and they also prescribe the general character of the project. Indeed, these relations are used as a basis of the traditional classification of research activities:

The four diagrams below illustrate the above mentioned four modes of study.

Project of Formal ScienceProject of Basic ResearchProject
of Applied ResearchProject of Development

The above presented traditional division of factual science into three genres: basic research, applied research, and development, is not optimal when discussing methods and processes of research projects, because the essential difference in methodology is simply, whether the project shall aim at practical application, or not. The division is therefore not used on the following presentation of arteology, and it is given here only because of its historical interest, and because many published statistics on research activities are still using it.

On the basis of whether the project principally aims at either theoretical or practical benefits, research approaches can be categorized into two broad classes:

Another dichotomy which influences the selection of research method is based on the degree of universality of the information that the project is searching for. The principal alternatives in this respect are:

When combining the two dichotomies we get the following table:

  Informative research of facts. It gives knowledge about the object but avoids making changes to it: Normative study of needs and goals of people, of how to fulfil them or how to remove practical problems by modifying the object of study:
Idiographic study of single cases (persons, objects or events): Case studies of notable artists, works of arts or historical events. Also histories of products where these are studied as unique discrete items. Research and development projects which aim at improving the present state of things by developing an existing activity or developing a new product.
Nomothetic study which gathers generally valid knowledge: Exploring invariances, knowledge which is true for all the studied objects, sometimes anywhere in the world as a 'law of nature'. Composing universally applicable theory of design, i.e. governmental regulations, standards, algorithms, handbooks and other tools for designers.

Beside the above named two fundamental decisions about approach of study, there are many minor practical questions to be considered when selecting a suitable method for investigating any empirical question, either in informative or normative manner. The following outline can help you in solving these details. It consists of three appoaches:

Models for Research Process

Planning a research process means defining the logical chain of operations which starts from the available inputs of theory and data and produces the desired informative or normative output. Usually you can choose the process among many alternatives, of which a few typical ones are given below, separately for informative and normative projects. Beside these typical model processes you can sometimes adopt and modify the approach of an earlier investigation, if a suitable one is at hand.

The Process of Informative Research

Informative Research Informative research aims at gathering objective knowledge about the object of study. Objectivity means that even in the case that several people take part in the project, there will not be much disagreement about data or results. The research process need thus include no arbitration between contrasting opinions which often complicate normative projects. In the best case, the project can advance in a simple linear succession of collecting data, analyzing and reporting them. Nevertheless, it happens quite often that some procedures have to be redone several times before a satisfactory result is found, and the process thus becomes iterative. Both these alternatives are explained below.

The Linear Process

Research project Linear process where each operation is done only once is the speediest one and therefore when possible you will usually prefer it to the iterative one explained later on. It is common in technological research which deal with unequivocally measurable physical things, not with opinions and personal appraisals. The linear process is simply a sequence of distinct tasks, typically the following:

  1. Defining the problem and the hypotheses (if any)
  2. Study of literature
  3. Planning the empirical study. Defining the population to be studied, refining the concept definitions and the methods of measurement
  4. Gathering data
  5. Analysing the data. It can consist, for example, of expressing the data as a model, or of verifying a hypothesis with them, or of predicting the future of the object of study.
  6. Assessing the validity and reliability of the results
  7. Reporting

PERT diagram The linear research process can often be planned in advance quite precisely. For example, the tasks in a research project are often linked in such a way that it is possible to start a task only when some other task is completed. In a project diagram, you can indicate such a link with an arrow between the tasks, and thus create a PERT graph (abbreviation from Program Evaluating and Review Technique). An example is on the left. You can also include such "tasks" (thin black arrows in the graph) which are just logical dependences between the stages of work and involve no working time. The graph can be drawn on a calendar grid, or just on plain paper. It is usually made in a horizontal position.

If you refine the PERT graph by adding the estimated duration of each task, it becomes possible to specify the critical path of your project. It means the sequence of those tasks which dictate the shortest possible duration of your project (assuming that you have enough resources at your disposal). In the PERT graph on the left, this succession of tasks (the red arrows) contains 2+1+2+2+3+2 = 12 working days.
In reality, few research projects have unlimited man-power resources; nevertheless a PERT graph may prove useful in clarifying the logical chain of tasks.

For defining the exact content in each of the tasks in the linear process you can make use of the checklist about the options of procedure presented later on.

The Iterative Process

Viewpoints Sometimes you want to study an object that you do not know so well that you could plan all the phases of the investigation exactly. It can even difficult to decide which facts are to be collected, and it becomes clear first after some data have been analyzed. You must be prepared to change your plans as soon as the investigation deepens your understanding of the issue. Linear advancement is thus impossible, though often it is possible first to try it and if it does not work, shift to the iterative approach.

One variant of iterative study is to begin with a pilot study, i.e. a preliminary study with a small amount of material, where you can test the method that is intended to be used later in the final study with complete material. In this case the iterative cycle will be completed twice.

More common is that the iterative process is repeated as many times as necessary to reach a satisfactory result, or until the resources are exhausted.

The process normally starts at studying the object from several different viewpoints, either from the angles of various established sciences (like in the diagram on the right) or just from miscellaneous practical points of view. As soon as possible, you should specify the viewpoint of your study and explain how you "understand" the object. Once you have defined your point of view and your problem, you will need to gather only such empirical knowledge that is related to the problem; that will enable you to minimize the material you will have to analyse.

Analysis in exploratory research is essentially abstraction and generalization. Abstraction means that you translate the empirical observations, measurements etc. into concepts; generalization means arranging the material so that it disengages from single persons, occurrences etc. and focuses on those structures (invariances) that are common to all or most of the cases.

When planning the content of the various phases of the iterative process you can make use of the list of standard questions presented later on.

Iterative approach is common when studying the actions and accomplishments of people, especially historical ones. The approach is discussed on other pages, under the titles of Exploratory Research and Exploratory Case Study. A similar approach in the study of old texts is the Hermeneutic Research method.

The Process of Normative Research

Normative studyThe target of normative research is to improve the object of study or to create a new, better object. In its simplest layout, the normative process of research and development might consist of a linear series of simple decisions, for example as follows:

  1. Defining the target. It could be e.g. removing an existing inconvenience or creating a new product.
  2. Defining which factors in the context can be modified and which not. You might think, for example, that the quickest way to achieve the target could be to change the political system. However, the project cannot do it and therefore it must be taken as "given".
  3. Planning how to reach the target, preferably as a few alternatives.
  4. Selecting the best alternative (which is either the one that fulfils best the target, or the one that gives a satisfactory result with least expenses).
  5. Making a detailed plan of action.
  6. Submitting the practical proposals to the people that can decide on them (e.g. the management of the company or a governmental agency) which may require redoing any of the preceding stages.
  7. The operations in practice.

Indeed, the process of industrial new product development is often planned as a linear procedure.

However, many normative projects deal with complex practical problems, and it is often impossible to proceed straight to the synthesis and proposal. Some usual reasons for complications are:

Complications like these can compel redoing a part of the work, in other words iteration, returning to an earlier stage of the process. If there are many such backward returns the process begins to resemble more a circle than a linear succession of decisions. Indeed, a spiral like the one on the right is a very typical model of a development project.
Normative spiralNormal phases in the iterative "spiral of development" are as follows.

  1. evaluative description of the initial state (perhaps including its earlier development) and defining the need for improvements
  2. analysis of relationships and possibilities to change things
  3. synthesis: proposal for improvement
  4. evaluation of the proposal.

By repeating the sequence from 2 to 4, and by gradually improving the proposal, an acceptable result is usually found.

When planning the descriptive and analytic phases, you can benefit from the list of standard questions presented later on. Besides, typical normative research and development processes are described on the pages about Developing an Activity, of Developing an Industrial Product, of Theory of Design and of Scientific Art.

Modifying the Method of Another Investigation

The general models of process given above are not the only possible starting point in selecting the method of investigation for your particular problem. A working set of methods can often be adopted from an earlier published research project, thus saving much time othervise spent in planning and testing a fresh tailor-made method.

Where to find suitable archetypes of research?
If you are working in a research institution, its earlier published projects already provide you some obvious candidates from which you can try to adopt methods.
Another source of material is a bibliographical study which normally gives to you numerous potential models of research method even when you do not explicitly give "method" as a key word. Instead, you will often want to locate reports of earlier studies of a comparable issue, i.e. those which have either

or, in the best case, both of the above.

Note that when duplicating the methods of an earlier project you have to take care of not copying those procedures that are unsuited to your special problem of study. When imitating the method of an earlier project, you cannot avoid of tacitly accepting many components of its paradigm like its approach, definitions of concepts, and even tacit evaluations. This may be advantageous because it promotes "normal science" i.e. the steady growth of the field of study where the scientists base their work on the results achieved earlier.

One potential disadvantage of relying on an existing paradigm is that it tends to restrict the area of new studies. A strong paradigm invites neophyte scientists to study such problems that are firmly related to the existing theory and which already have been studied to some extent, and for which there are well-tried methods. This is the reason behind the fact that many research institutions today are specializing in either "qualitative" or "quantitative" studies (the Two Cultures of Research), which often unnecessarily restricts their work.

Imitating earlier methods may be convenient when you study problems that have emerged inside the scientific community. This is the case often if your goal is just to make research, for example a thesis. However, when the problem originates from the practical world, from the recent evolution in society and in industry, you should think twice before imitating the methods of earlier published studies. Often your chances of finding a solution to the problem improve if you use the more tortuous approach that is outlined below.

Options for Procedure Details

The context of the investigation gets projected onto the research process, and affects its methods, from three sides:

  1. The input material. Shall we study objective facts or subjective opinions, or both? If facts, is their mode of presentation explicit, or should we include even tacit knowledge like the skill in the use of an artifact, of the professional skills of its makers?
  2. ExampleTheoretical input. Can we start from earlier known models which describe the object of study, or have we to start from scratch?
  3. Expected output. Is there a client who needs our findings? Possible applications of research findings include forecasting, and development of products and activities. Beside such practical uses, your findings may have scientific value, if they enlarge the scope of general knowledge in the field.

Exactly how these three points of view affect your future project and its methods, is shown in greater detail in the tables below. You could use these tables for producing a tentative logical map of your project.

You may want to start planning your project by first printing a copy of this page you are reading, and then by drawing a pencil line which goes through one or more of the boxes on each row, like in the figure on the right. The order of the tables and questions follows a customary process of research, but in planning your project you do not need to start from top - you can start anywhere, for example from the lowest boxes if you know the purpose to which your findings are going to be used.

The logical map that you get with this method gives, of course, just the basis for your plan of work. Besides, you perhaps want to set targets for the use of resources and time. These questions are discussed later on.

Type of information to be studied, and its origin:
Facts, i.e. objective, explicit assertions about existing people, objects and processes... Tacit know-how, artesan's skill... Subjective opinions...
...collected from empiria: Study of Static Objects. / Observation. / Experimentation ...from public documents: Literature study. / Study of existing statistics. / Ex Post Facto Research. ...from people: Interrogation (Interview or Questionnaire.) / Study of letters and private documents. / Action Research. / Collective Design. / Evaluating a Design Proposal.
 
Demarcating the group to be studied
Only one specimen or case is of interest: Case Study The interesting group is clearly demarcated, and all its members will be studied The target group is so large that all its members cannot be studied.
One case at a time will be studied, until knowledge gets saturated The researcher picks some cases and examines them A random sample will be studied
 
Is there already a model of the phenomenon?
No; the project is hoped to produce such a model: exploratory research Yes, but the original model is incomplete and it shall be improved during the study Yes; there is also a hypothesis which shall be tested during the project.
 
Logic of analysis
Synchronic, or cross sectional study: Diachronic, or historical, study:
One case, or a few similar cases: Case Study A few different cases: Comparison Large number of cases: Classification Quantitative analysis of variables Evolution of individuals or specimens Time series of variables
 
Use of the obtained knowledge
Informative study which does not aim at improving the object of study. The project ends at the finished report: Normative study which aims at practical application of the findings:
Description (or explanation) of the object of study Forecasting the future of the object of study Developing the activity that was studied Developing an industrial product Creating art with the help of science

Planning the Use of Resources

The target of the project - what you are expected to accomplish - has been discussed above, and when well defined it will then give the basis for scheduling your work and planning the resources necessary for the work. Targets can make the work easier and faster: it is easier to proceed when you know what you are aiming at.

Beside the goals for what shall be achieved, it may be useful to plan those resources which are critical or scarce, like e.g.

Scarcity of resources can compel you to revise the project plan, as there are great differences in the costs of methods. Savings can be attained by e.g. the following strategies:

Of course, such savings often result in a lower level in the reliability, validity and in the practical usefulness of the outcome.

Time Schedule

Timing of the research project is often governed by outside requirements and restrictions, like:

Gantt diagram If you can divide your project into separate tasks with their individual targets, it also becomes possible to plan each task in advance.
If you present each task as a bar on a calendar based grid, the result is a Gantt diagram, sometimes called "road map", an example of which can be seen in the upper figure on the right. Such a diagram may help defining the most effective sequence for the jobs and allocating resources to all the various tasks. It also helps in budgeting your incomes and expenses, or your own weekly hours and those of your research assistants.

Follow-up The calendar based Gantt diagram is also an effective tool in the follow-up of the progress of the project. If you weekly mark in red the real progress of each task, like in the lower figure, you will get a good overview of the general situation of your project. In the example, the interviews are well ahead of their timetable, while reporting lags behind and might require assistance.

For managing a large project, a computer with a project planning program is often used. If you feed the durations and the logical relationships of the various tasks into the computer, it then prints out the general plan of the project as a Gantt or PERT graph, whichever you choose. The project planning program can also help you in the follow-up of the project.

En Español  In Finnish   Contents

April 8, 2005. Original location: http://www2.uiah.fi/projects/metodi
Comments to the author: