| |
According to the idea of scientific realism, the object of study exists in the empirical (tangible) world but theory belongs to the conceptual world of thinking. The goal of research is to create a theoretical picture of the object of study which resides in the empirical world. The reason behind the researcher's efforts can be pure inquisitiveness (informative study), but it is also possible that this picture shall be later used for improving the object of study or other similar objects (normative study). All the theoretical knowledge that we have concerning empirical things make up a more or less complete picture of the empirical world.
In the diagram on the right, you can see on green background a fragment of the empirical world which includes an object of a study, and on the right half of the diagram the corresponding theoretical model.
The notion of theory as an image of empiria is generally a fruitful one when studying artifacts, their creators and users. A similar vista is also used by designers though there is the difference that the designers proceed in the opposite direction, i.e. from theory to empiria, starting their work in the world of concepts and then proceeding to realize their images in the empiria.
A researcher (and a designer as well) normally constructs the image of his object of study by using two types of elements:
Concepts serve not only as building blocks of theory, but also as connecting points between an empirical object and its theoretical image. It is the researcher's task to take care that the links from his concepts to the corresponding empirical objects or phenomena are as unequivocal as possible. It will help in reducing errors when registering data, as well as later errors when a user of the report applies the researcher's findings to his own purposes and problems.
The most usual and elementary indication of a link between a theoretical concept and its empirical counterpart is given by the name of the concept. Moreover, there is the more precise method of presenting a definition for the concept, but normally it can be used only for a few key concepts of the project, because we cannot count on the user of the report remembering more than a handful unique definitions, nor can we expect him to peruse the list of definitions repeatedly.
The researcher has to select for his task appropriate concepts, their names and definitions, but this does not mean that he personally should create all these things. On the contrary, it is usually advantageous to adopt as many concepts as possible, with names and definitions, from earlier treatises in the field. In fact, every established branch of science has an ample assortment of cardinal concepts which reiterate in numerous studies. Handbooks and textbooks in the field of science usually include lists of such concepts together with their definitions. If these established concepts can be used in a novel research project, unnecessary work can be avoided and the new project can profit from existing theory including tested methods of measurement and analysis and data about previous comparable cases. Besides, using generally known vocabulary brings the findings of the new project into close contact with already existing theory and it thus enlarges effectively the range of knowledge in the field.
Though it thus seems advisable to avoid creating novel concepts when launching a new research project, it must be admitted that the researcher not always has free choice in the matter because of the lack of suitable established concepts. Such is the case in exploratory research where the project deals with questions that have not been studied earlier.
Moreover, it often happens that the researcher, after doing intensive studies on a topic, concludes that one or more concepts around the topic have traditionally been ill defined and need revision.
In the case that a novel concept has to be created, two aspects of the concept need consideration: its name, and its definition.
Naming a novel concept. Usual methods for creating new scientific terms are, among others:
Definitions. The name of a concept can normally give just an approximate idea for the content, just as the title does for a book. To give a more precise account, definitions are used so that both the researcher and the reader-user of the study report can have the same notion of the object. There are two types of definitions:
In the diagram below, yellow backgroud marks concepts for which empirical definitions can be given easily or are self-evident, and nominally defined concepts are shown in red color.
For example, when studying the productivity of a given workshop the 'output' can be measured as number of finished items per month, and 'work input' can be measured in man-hours per month. Now we can use these operationally defined variables for defining nominally a third concept:
'productivity' = 'output of products' / 'work input'.
Other nominally defined variables in the diagram are cost per unit, profitability etc.
In many fields of research the concepts and theory can be arranged in layers so that there is a layer of empirically defined concepts and on this basis one or more layers which consist of successively nominally defined, more and more abstract concepts. An example is shown below.
The empirically measurable concepts are on the right: 'speed of operating' an instrument, and 'errors in operation'. They are then used on a higher level of abstraction for defining 'effectiveness' of operation. Effectiveness and 'learnability' are used on a third level to define 'usability', and so on. Rest of the figure, made by Shackel, is explained under Usability of Interactive Products.
It is by no means necessary to construct an all-encompassing model which contains every concept that the researcher has worked with, with definitions. A useful rule of thumb says that the researcher should try to operate with as few theoretical variables or concepts as possible. This recommendation goes by the name of "Occam's razor" dating back to the 14th century and attributed to Wilhelm Occam.
Besides, it would be impossible to give a definition for each word in the report. It is not necessary to define everyday words like 'human being', 'day' and 'buy' if these are used in their usual meanings given by basic dictionaries.
If devising new definitions is necessary, the following four requirements are often given as a target, though they are usually easier to meet with when studying physical objects than in the study of people or cultural artifacts:
A vicious circle usually consists of two nominal definitions referring to each other while the concepts used in these definitions have no real definitions to link them to empiria. In the following, there is such a link between two nominal definitions:
- a "need" = what makes the organism act
- "action" = the actions for satisfying needs
Note that it is all right if you use just one of the definitions above. The vicious circle is created first when both are used simultaneously with no additional connection to empiria.
However, the mark of a real master is his/her ability to break any rule. Thus, MacIntyre (1985, p.219) defines in cold blood: "The good life for man is the life spent in seeking for the good life for man." This is formally a vicious circle but no less it transmits a message.
Validity and reliability can be illustrated by imagining a study whose purpose was to
measure the length of objects. A faulty yard stick which had shrunk by 10 percent,
was used. The real length of the stick was thus 90 cm although it had a normal
looking scale from 0 to 100 cm.
The measurements made with this stick were
reliable: the measured objects could be placed in an exact order of length and
objects of the same length were properly recognized; if the measurements were
repeated, the same results were always obtained. The measurements were not,
however, valid
in spite of their good repeatability: they did not indicate the real lengths but something
else.
Other examples:
Phenomena that are studied in natural sciences and technology can usually be measured and expressed as quantitative variables and models. In such research, it is usually easy to fulfil all the four requirements we discussed above. However, the definitions of concepts that interest researchers studying people and cultures are not as easy. Often the most natural way to describe them in qualitative words, and it is often more difficult to meet all the four requirements. The researcher could consider which requirement he could compromise with in such research. What is often compromised on is validity. If a concept is very difficult to be measured, you could consider replacing it by another close concept which is easier to measure. "Intelligence tests" in psychology are one example of this: different tests yield different results when testing the same person, which fact indicates that they are measuring different things.
Can definitions be altered as the work proceeds? Yes, and in fact, in qualitative research this is usual because the researcher's understanding of what he is studying usually deepens when the research proceeds. When studying human culture it sometimes happens that an entire study contains nothing more than an attempt to define a few stubborn concepts. Especially when studying works of art you have to take into account that their content quite often varies depending on the context where they are studied, and often the work is intentionally made to elude fixed definitions.
On the other hand, if the definitions which have served as a basis for measurements or registration are changed after gathering empirical data, the gathered data which corresponds to the old definition is wasted.
In the final report definitions are often listed in an appendix or at the beginning of text. Another possibility is to give the definition in the text when starting to discuss a new concept.
Every word you can think of belongs to a language, and similarly every definition for a theoretical concept belongs to a specific language or system of coding. This language identifies the phenomenon or the object and its relevant attributes in two quite different "worlds" each of which have distinct requirements that the definition has to fulfil:
in empirical work, when gathering data from the phenomenon or the object of
study, the language should support reliable registering (see the above discussion on validity and reliability). Later, when the project is finished and the research findings are being applied to the benefit of people, the same language can once more be used.1. Individual naming of specific objects or events does not much promote the building of general theory, but in case studies and documentation it is sometimes used: "The Paris Exhibition 1900", "Parthenon", "Le Corbusier".
2. Presenting the visual form or pattern can be important when
studying artifacts. It is easy to make a photograph, but another thing is how you
analyze the patterns in the picture. Earlier, it was a little difficult to analyse images
and the normal method was first to introduce each pattern in one picture, give this
pattern a name (e.g. "schooner" and "yawl" are the type names of the upper boats on
the right) and then continue the analysis verbally.
It is also possible to analyse forms and patterns directly, with no translation to words. Examples of simple visual recordings of individual objects can be found in Measuring Physical Objects, and examples of general conceptual presentations of patterns under the title Icon
Models.
3. Classification or "coding" compresses information effectively when only one or only a few attributes of the phenomenon are of interest. What is of interest, is a thing that you must conclude on the basis of your problem and the proposed model. If the interesting attribute is present in the object or phenomenon, you mark it with the predefined code, usually a number or a letter. Thus, if you are studying boats of different types, but their exact shape is not of great interest to you, you could translate the boat types to codes, for example:
| schooner | = 1 |
| yawl | = 2 |
| cutter | = 3 |
| gaff sloop | = 4 |
| Bermuda sloop | = 5 |
Classification is also called nominal scale if one wants to emphasize that it
is a kind of measurement.
Many classifications of everyday life are suited to research purposes as such, for
example the distinction between men and women. More examples of classifications on
a qualitative basis are found under the title Classification.
Classification is discontinuous in nature, which means that the variable has only certain discrete values, but no intermediate values between them.
After classification, the number of the individuals or measurements designed to each class will be known, and this is how qualitative data can be made quantitative. On the other hand, if the researcher is mostly interested in qualitative concepts, transforming them into quantities would only be detrimental because that would make just those qualitative features the researcher is interested in disappear. Thus, there would not be much point in classifying works of art on a nominal scale into just two classes: "beautiful" and "ugly".
4. Verbal description is suited to almost any phenomenon of human culture. This type of study is often called qualitative because adjectives are usually prominent in the description. Examples are presented in Case Study.
5. Ordinal scale puts the individuals in a line, for example in the order of size, without paying any attention to how large the individual differences are. On this scale, we could for example indicate preferences of taste: "I'd rather live in a terraced house than in a block of flats." "This fabric is more beautiful than that one." Worded scales used in opinion polls must also be seen as some type of ordinal scale:
beautiful / - / - / - / - / - / ugly
In the scale above, a qualitative concept (beauty) which is usually evaluated verbally has been operationalized to be measured and analysed numerically. It is up to the researcher to decide if this procedure is fair toward the concept or if it discards some important points.
Most (but not all) ordinal scales are discontinuous in nature, which means that they contain only discrete values, but no intermediates.
6. Arithmetical scale. Its spacing is uniform, that is, the intervals between the markings are equal. These scales are often continuums, so the number of possible values is infinite. Hence these scales allow very precise measurements of the variables. Most physical quantities are measured this way. The following subcategories exist, though in practice there is little difference between them:
Operationality of the scales in analysis. When selecting the language of presentation, the researcher should consider not only the facility and accuracy of registering facts in the empiria, but also the practical procedures of analysis and the requirements of data processing, if that is used. In that, arithmetic scales are unbeatable: the relations between numbers are exact and known to everyone, it is easy to feed them into the computer and there are excellent mathematical methods available for analysing them. Some of the most powerful methods of analysis include divisions and can consequently only be applied to data measured with a ratio scale. From the point of view of analysis, the ratio scale would thus be preferable. Indeed, it often happens that a researcher "operationalizes" one of the attributes of his object of study: he deliberately constructs a never-heard-of ratio scale for measuring it because he wants to use statistical analysis.
Remember, on the other hand, that the success of analysis depends also on validity: how faithfully the language can present the thing that is of interest in the study? In this respect verbal or other non mathematical languages can often be superior after all. There is a discussion on the format of data in the various stages of analysis under Tools for Analysis.
When preparing definitions of concepts it is worth the while to deliberate how their empirical recording shall be made, and which kind of difficulties it might entail. Sometimes a slight adjustment of the concept definition can help to avoid unnecessary field work, errors and disturbing variation of data.
A researcher is usually searching invariable structures in the object of study, which structures usually become visible as some kind of systematic variation in the measurements and recordings. Beside this interesting variation, the data usually contain other, disturbing variation which the researcher usually wants to minimize. Its most usual sources are:
Each of these three types of error will be discussed in the following.
A meter includes the apparatus and the arrangement with which the researcher registers whatever interests him in the object. Also non-mechanical methods for gathering data can function as meters, such as psychological tests and the scales used in questionnaires, e.g.:
pleasant 
unpleasant
The worst defect that a meter can have is that it measures the wrong thing. In other words, a good meter must be valid. Moreover, it should be reliable, giving always the same results even when used by different people. Finally, a sufficient resolution will be needed. Resolution (or least count) is equal to the smallest measurable difference between the observations.
In practice, no meter is totally reliable; measuring always includes a slight error
of measurement, i.e., a difference between the measurement and the real value.
No absolutely correct value can be ascertained in practice, nor is there any way of
knowing exactly the error of measurement.
The error can, however, be minimized, and to this end, you have to look at the two components of the error.
Let us say that we have an object that is difficult to measure, and we have made not less than 172 measurements of it. These are represented by small squares in the figure below. The smallest measured value is =5, the largest =34, and the average of all the results is 17.
The random error equals the difference between one measurement and the mean of all the measurements. For example, the random error of the measurement at the value=32 is 32-17=15. The random error can be measured
in the same way as any other dispersion of values, for example by calculating the standard deviation. A small dispersion also means good
reliability.
The random error is easily eliminated by calculating the arithmetic mean of the results.
A systematic error equals the difference of the mean of all measurements from the real value of the variable (which is normally unknown in the study). In our figure, it is 24-17=7. The systematic error will normally remain the same when the measurement is repeated. Hence, it is difficult to detect it in a study. A systematic error also indicates that the meter is not quite valid.
Sometimes it is possible to detect a systematic error if the same object is measured with two different methods. If a systematic error is discovered, it is eliminated by correcting the measurements (e.g. by normalizing them) or by calibrating the scale of the meter.
Manufacturers of meters often guarantee that the total error (random + systematic) of their product will be lower than a certain limit, provided that the instrument is used properly.
When studying a phenomenon, the researcher usually wishes to find out how it is caused or influenced by certain "explaining factors". However, normally empirical objects are influenced by many factors beside those that the researcher is interested in.
On the right is an example of a typical research model (an experimental design, in this case) for studying the relationship of a certain stimulus and the expected reaction to it. In this example the research project gets spoiled because the object of study receives, instead of the intended stimulus, unexpected "noise" or disturbance which causes the wrong type of reaction.
Disturbances can occur in any type of study, such as observation, interview, filling in a questionnaire, etc.
Obviously data which are corrupted by noise are of small value in research. To avoid them, there are two possible approaches: either minimizing the disturbances themselves, or eliminating their effect from recordings.
Preventing or diminishing the disturbances that are present when recording data can be accomplished, for example, by choosing a quiet and secluded environment for observations or interviews.
Note that even the activities of research itself, like posing questions, measuring etc. can cause some disturbance. For eliminate it, special "unobtrusive" research methods have been devised.
Sometimes it is possible to shield the research object, thus keeping out the disturbing influences. This is common practice in experiments set up in laboratories.
It is to be noted, however, that shielding, arranging the recording to an environment other than where the activity normally takes place, or any other manipulation of the genuine circumstances of the phenomenon can alter the object or procedure in unexpected ways. In other words, it can lessen the validity of the recorded data.
Eliminating the effects of disturbances after the data have been recorded is usually an uncertain method, because these effects are seldom exactly known. The procedures are in principle similar to the corrections that are used for erroneous measurements:
If the beliefs and attitudes of the researcher affect the observations, these are called subjective; if not, objective. Subjectivity can play a role even later in the project, when analysing and interpreting data, but that problem is not discussed on this page which regards the questions of registering facts.
Because subjectivity is caused by personal beliefs and attitudes which are relatively constant, it causes a systematic error in recording. It can affect both informative and normative data.
In informative research which aims at recording the state of the object as it is (or as it was, in historical study), subjectivity often causes errors when studying people's or animals' actions and their motives. The researcher often relies too much on his facility of empathy, looks at the activity from his own point of view only, and fails to perceive the real purpose of the endeavor.
One more example of informative studies which often suffer from subjectivity is when people who produce works of art and other phenomena of human culture are studying their own or their colleagues' works. To study one's own activity was earlier wholly disapproved by many scientists, but today many think instead that the maker of an artifact is the best expert on it and can perhaps do useful work as a researcher as well.
As a contrast, the ideal of objectivity is often easy to achieve in natural sciences, when studying objects to which the researcher has no personal attachment nor sympathy.
In normative research projects which are aimed at improving the object of study (or other similar objects) the preferences which shall guide the project should normally be collected from those people who are going to encounter the result of the project or use it. These groups of interest are sometimes classified as 'agents', 'beneficiaries' and 'victims' of the project, cfr. Point of view. Subjectivity when surveying these people's preferences concerning the desirable improvements to the object means that the researcher mixes in his own opinions and interprets other people's suggestions on the basis of his personal preferences. Another usual example of subjectivity is that the researcher simply forgets some interest groups, for example those people that are going to suffer from the secondary consequences of development, or he forgets the environmental aspects.
In normative projects of research and development the target of impartiality is sometimes difficult to achieve completely. The researchers that have been hired to the project are often more or less obliged to share its values and promote its goals, i.e. to improve the object. What "improving" means, is usually preliminarily defined in the initial plans for the project. Besides, the research team often includes one or more planners or designers who are expected to continue the project by preparing proposals for accomplishing the necessary improvements in reality, and these people often have strong opinions on the future development already at the outset.
Despite of difficulties, in most cases there is no reason to abandon the traditional general rule to scientists which says simply that they should aim at objectivity, that is, they should not let their opinions affect the recording of facts (and of opinions of interest groups, when applicable). Using the metaphor of the "two worlds" of empiria and theory, researchers should thus try to stay, as personally not interested spectators, outside of both of them.
The methods available for a researcher for deliberately counteracting his subjectivity are slightly different depending on whether the researcher is working alone, or as a member of a team.
A researcher working alone can try to unearth his personal attitudes by putting to himself questions such as the following:
Besides, the researcher can try to detect outside forces which could bias his observations, like:
Attitudes and outside influences which may be revealed with the help of above questions normally belong to human life and are often quite legitime and understandable. Nevertheless, the researcher should try to make them explicit in order to be able to analyze them and decide whether they are likely to influence his work too much. It would be no bad idea to put a few words about your analysis in the report as well.
For a team of researchers, a collective discussion is an efficient method to clarify everyone's personal evaluations and perhaps arbitrate between them.
Also such trivial reasons as slightly different methods of collecting data can lead to interpreting the observations differently, which can cause some systematic variation in the data. This annoying variation can be minimized by giving exact instructions to the assistants, by rehearsing the procedures, and by assigning the cases randomly to the assistants.
Note, finally, that there are approaches of study which do not condemn empathy and co-operation between researchers and the people that are being studied but instead make use of mutual compassion because it helps the researcher to understand the situation and actions of people. Methods which take advantage from it include action research and participating observation.
| |
September 15, 2005. Original location:
http://www2.uiah.fi/projects/metodi
Comments to the author:
