It would be hard to argue that education at all levels has not changed in the past three decades. Indeed, a great deal has changed, and there are some who say for the better, while others disagree. Perhaps the most pervasive and expensive of these changes has been the infusion of technology into practically everything connected to education. In addition, there have been, and continue to be almost daily proposals concerning everything from school vouchers and charter schools to increased emphases in the curriculum on such content as conflict resolution, multicultural education and violence prevention education. There is little doubt that advances in distance education technology seemingly move at near-light speed. New instructional schema have come along to replace other schema that a few scant years ago were thought to be "better." Indeed, the recent history of education is replete with cycles of new, better programs constantly replacing older, formerly better programs. Yes, there has been a great deal of change in education over the past three decades, but have these changes actually improved anything? What does "better" actually mean? What does "improvement" mean? Perhaps most important is the question of whether government can mandate improvement and actually get it.

Ends, as reflected in various curricula, have changed and so have the means for attaining them, but how many of the changes have been improvements? It is anybody's guess. Without criteria for improvement, it is impossible, first, to identify those programs that offer the greatest hope for improvement and, second, to evaluate programs in progress to determine how effective they actually are.

It is not difficult to find statements in the literature to the effect that this process or that is more effective, more productive, or just better than some other. But, using such terms only puts off until another day the task of defining effectiveness, production, and better.

A belief in, and the allocation of resources for systematic planning in education have helped little, if at all. A fundamental premise of systematic education planning is that resources will be allocated on the basis of expected effectiveness of courses of action that are selected from among alternative courses of action. Fundamentally, this requires the education planner to say that one course of action is an improvement over another that might be taken to reach a goal.

OBJECTIVES MORE ALIGNED WITH GOALS

A program will be improved when the day-to-day operational objectives are changed so that their contribution to the overall goals of the program is increased.

For instructional programs this means that the instructional objectives of specific programs will be improved when they are changed so that they are more relevant to expected outcomes. A number of newly developed nationally known programs are claiming success in this area. For example, the developers of some mathematics programs claim that their objectives contribute more toward understanding of mathematics as expressed in various state mathematics standards than the traditional mathematics objectives. Similar claims are made for science, foreign language, English language, and other programs. Since increased student understanding of the process involved in the subject areas has long been a goal of education, these new objectives will result in improvement of instructional programs if it can be shown that they actually do make a greater contribution to the achievement of the long term goals than the objectives of the old programs. However, because goals are long-range and objectives are short-range, it is unlikely that the effects of altered objectives can be determined except through the carefully designed longitudinal studies that are typical of medicine and industry.

If systematic planning in education is to have any real meaning, the objectives that are accomplished as a result of the allocation of resources must contribute to the long-range goals of the people and institutions that the planner serves. It is these long-range goals that must be kept in mind.

Few would argue that educational technology has resulted in significant increases in efficiency. If time alone were the sole criterion, technology has permitted teachers to keep records more efficiently, make presentations in a variety of multimedia formats, and spend less time searching for information. All of this is putatively to increase student learning.

In instructional programs, if a change in instructional method leads students to learn the same amount with less effort or time than they did previously, or if they learn more with the same effort or time, then the change is an improvement. For example, if all the students in a given class are assigned the same learning activities, it is probable that some of them will already have learned the objectives of the assignment that is given. Therefore, they will be expending time and effort to accomplish little in the way of learning. If the process were to be changed so that the students who had already met an objective were given assignments more appropriate to an objective that they had not yet accomplished, then their efforts would probably pay off in increased learning. The amount of effort expended by the students would remain the same, but learning would increase. Therefore, the efficiency of the instructional process would also increase.

Taking another example, if an instructional program utilized computer assisted instruction or other instructional technology in such a way that a number of students taught by a given teacher could be increased by 10 percent without a decrease in the other areas of program effectiveness, then it could be said that the efficiency of the instructional process increased. This, too, would be an improvement.

This particular criterion of improvement - efficiency - has often been maligned by educators who assume that efficiency is the concern only of persons who are interested in cutting the cost of education regardless of the results.

However, efficiency cannot be judged accurately except in terms of two factors - effort and the product that it produces. Waste can and does occur when reducing costs reduces product output disproportionately. Waste can also occur when increases in effort do not significantly increase output.

Also, the question arises: Is increasing the efficiency of a process an improvement if the objectives of the process are wrong? If you are going to do the wrong things, why try to do them efficiently? The answers to these questions seem to be obvious. If resources are going to be expended doing the wrong things, then it is better to dissipate as few of those resources as possible. Efficiency then becomes equally important whether or not the objectives of a process are judged appropriate by the criteria expressed in the first improvement category.

REDUCTION OF UNDESIRABLE SIDE-EFFECTS

Other things being equal, an instructional or administrative procedure will be improved when a change of procedure results in the generation of fewer undesirable side-effects.

Virtually every action taken in instruction and administration, no matter how effectively it accomplishes the objective of the action, results in the generation of unplanned side-effects. Sometimes these side-effects are so seriously negative that they prevent implementation of programs that are otherwise satisfactory. If the undesirable side-effects of a program or action can be reduced or eliminated, then the program has been improved, although it may accomplish no more objectives than it did before the change. As such, reduction of undesirable side-effects may well be the target for the allocation of resources by the cost-effectiveness motivated education planner.

Many of the undesirable side-effects of the standard programs in public education are not difficult to identify. For example, report cards containing student grades are sent home to parents so that they will have some idea how their children are doing in school. For some this means that their parents will punish them severely-an undesirable side-effect of the report card method of reporting to parents.

Access to the Internet in schools and libraries permits students and teachers to retrieve information rapidly. However, an unintended side effect is that some students may use this technology for unacceptable purposes (Internet games, pornography, etc.)

Other examples of undesirable side-effects of educational practices could be easily cited. Yet, this aspect of improvement has received little attention from educators. Undesirable side-effects and their reduction are usually overlooked in the rush to have students learn more, regardless of other effects. It is not uncommon for an innovative program to be branded a failure because it could not be shown that the students did learn more. Some changes in programs can be justified as improvements if it can be shown that students would learn less of those things that are unwanted.

INCREASED RELIABILITY

Other things being equal, a program will be improved when the reliability of the program is increased in the context of use for which it was intended.

Reliability is a term used to express the consistency with which a program achieves its objectives. The term has been used widely in industry to describe process and product performance, but for some reason has been limited mainly to the field of testing in education. Yet, there seems to be no reason why the term should not be used in its broader context in the field of education.

Because of the possibility of potential confusion resulting from its application only to the field of testing in education, the improvement category of reliability will require fuller explanation.

Suppose, for example, a goal of an elementary school is to have all students who have received six years of reading instruction to test at or above the 3.0 grade level as measured by a standardized reading achievement test. The school accepts all students for the program except those who have severe visual, auditory, or mental handicaps. Suppose further that the school draws its student population from a middle-class community and that all measurable characteristics of these students remain constant over a period of years. Achievement testing indicates that the reading goal has been achieved by 80 percent of the six-year students each year for the past six years. Thus, a degree of reliability, using a particular reading program with these students, has been established.

Now, suppose a program modification results in 85 percent of the students meeting the objective for the next three years. Suppose also that there were no increase in resource allocation or undesirable side-effects. The increase of five percent in reliability of the new program of reading instruction over the old program would be an improvement.

Any procedure or process operates with a degree of uncertainty. Processes of collecting attendance data, keeping financial records, test scoring and bus transportation function with varying degrees of uncertainty no matter how carefully structured and supervised. To decrease the degree of uncertainty, i.e., increase the reliability, is to improve the process.

Sometimes apparent increases in reliability really aren't increases at all. One must exercise caution when comparing the reliability of processes to avoid coming to erroneous conclusions. Suppose, for example, that the school described previously were a school of controlled enrollment-a private school. Suppose further that attendance were to be denied to any student who had reached the third grade level and was more than one year below grade level in tested reading achievement. There is little doubt that this would increase the percent of students reaching the objective of reading for the six-year program. Yet, the process of teaching reading had not changed. The apparent increase in reliability of reading instruction in this example would be the result of eliminating (concealing) those students with whom the process did not work. It would be a mistake to adopt this process in another situation to improve instruction on the basis of inflated reliability obtained in this fashion.

Actually, the category of reliability is the one most often dealt with by educators who are attempting to improve instruction, although they may not have recognized it as such. Generally, when pretests and post-tests are administered to students to see if they have accomplished the learning objectives of an instructional process, the purpose is to check the reliability of the instructional process. When a school system administrator says that the achievement test scores of the students in this system are not high enough, he is really saying that the reliability of the instructional processes that are in use are not reliable enough to suit him. That is assuming, of course, that a goal of the system is to have the students score high on achievement tests and that the objectives of the instructional processes are those that are measured by the test.

If a program can be designed to accomplish objectives that were not obtainable at all with previous programs, then that program is an improvement.

More than three decades ago, several newspapers carried the account of a machine that had been invented that made it possible for people who were blind to learn to read printed materials. Before invention of the machine, instruction of students who were blind was limited to teaching them to read Braille. It was literally impossible to teach them to read printed materials.

The invention of the machine made it possible for instructors of the blind to attain a long-range goal of which they only dreamed for many decades. Thus, instruction for the blind was improved through the achievement of objectives that were simply impossible before. New technologies such as voice-assisted computers, and access to vast repositories of information via the Internet provide teachers with tools that enable them to achieve objectives with their students that were impossible a few short years ago.

Instructional strategies have been devised over the past few years that make it possible to accomplish learning objectives in mathematics and science that were not possible a few years ago. Technology has played a key role in this instructional revolution, but it is ultimately the teacher who mediates the instructional processes to accommodate change.

Whenever a new procedure is devised that makes it possible for a person or institution to accomplish something that he has wanted to accomplish but which was formerly impossible, this is an improvement.

Applications of these categories of educational improvement are numerous. Many will immediately occur to those who are faced with the task of selecting some one process from among alternative courses of action. There is no doubt that any proposed program or course of action will involve trade offs between various emphases on the criteria described in this article. Such trade offs have been, and will continue to be made in the arena of values and economics. Technology in general, and educational technology in particular will have enormous impacts on every aspect of education. It is at once the goals of simplicity and the bewildering complexity of much of the new technology that will be in the eye of the storm. Among many, there is an increasing awareness that we perhaps depend on technology more than we are willing to trust it. The fundamental question is, of course, whether we are willing to pay the price that such dependency demands.

The criteria described above have application to areas other than education, but it is the intent of ADPRIMA that they be helpful to teachers and administrators in their professional work.