Full opinion text
MEMORANDUM DECISION AND ORDER SIFTON, District Judge. This is an action brought pursuant to Title VII of the Civil Rights Act of 1964, as amended, 42 U.S.C. § 2000e et seq.; the Civil Rights Act of 1871, 42 U.S.C. § 1983; the fourteenth amendment to the United States Constitution; and Section 296 of the New York Human Rights Law (Executive Law), seeking declaratory and injunctive relief and damages to redress alleged sex-based discrimination against plaintiff and the class she represents in connection with the physical test portion of New York City’s Examination 3040 (“Exam 3040”) for the entry level position of firefighter in New York City. Plaintiff, Brenda Berkman, is a twenty-nine-year-old woman who passed the written portion of Exam 3040, but failed the physical test portion, which she took on February 22,1978. The class she represents consists of 410 women who took the written portion of Exam 3040 and are alleged either to have taken the physical portion of Exam 3040 and failed it or to have been deterred from taking it as a result of sex discrimination by defendants. Defendants are the City of New York, its Mayor, the City’s Fire Department and its Commissioner, the City’s Personnel Department, its Director and former Director, and the Civil Service Commission of the City of New York. Appearing as Intervenors pursuant to this Court’s Order of April 10, 1981, are the Uniform Firefighters Association (“UFA”) and the Uniform Fire Officers Association (“UFOA”), both employee organizations representing current job incumbents. The trial of this case was conducted before the undersigned, sitting without a jury, over several weeks between September and December 1981. For the reasons set forth herein, I conclude that the physical portion of Exam 3040 discriminated against plaintiff and the class she represents on the basis of their sex and that injunctive relief is appropriate prohibiting further use of the eligibility list established pursuant to Exam 3040 except on a showing of compelling necessity, directing the preparation of a new physical exam which does not discriminate against women, and awarding plaintiff interim and other relief to secure compliance with the requirements of the Civil Rights Act of 1964, as amended. What follows sets forth the findings of fact and conclusions of law on which these determinations are based, as required by Rule 52(a) of the Federal Rules of Civil Procedure. Jurisdiction over plaintiff’s Title VII action exists under 42 U.S.C. § 2000e-5. Plaintiff, by filing her complaint with the Equal Employment Opportunity Commission on May 16,1978, has complied with the time limitations imposed by that section with respect to all of the named defendants except the defendant Fire and Personnel Departments and the Director of the latter Department, who were not named in plaintiff’s administrative complaint. As to those defendants not named in plaintiff’s administrative complaint jurisdiction exists since there is substantial identity between those defendants who were named in the conciliation proceeding and those not named and since those not named had notice of the pendency of the conciliation proceedings. Vulcan Society of Westchester County v. City of White Plains, 82 F.R.D. 379 (S.D.N.Y.1979). Jurisdiction exists over plaintiff’s claim under 42 U.S.C. § 1983 and the fourteenth amendment by virtue of 28 U.S.C. § 1343. Pendant jurisdiction exists over plaintiff’s claim under New York State’s Human Rights Law. Background As of the end of 1980, New York City employed in excess of 285,000 persons. Of these, approximately 175,000 (including those in the City’s Fire Department) were hired by the Department of Personnel. The balance were employed by independent agencies such as the Board of Education and the Off-track Betting Corporation. Of the approximately 175,000 employed by the Department of Personnel, approximately 168,000 (including those in the City’s Fire Department) were in the competitive class, a class including all positions for which it is practicable to determine the merit and fitness of candidates by competitive examination. Employment statistics for the uniformed force of the New York City Fire Department for the period 1973 through 1981 were as follows: Year Firemen Officers Total 1973 10,720 2,548 13,394 1974 10,426 2,538 13,091 1975 9,089 2,347 11,548 1976 8,304 2,267 10,662 1977 8,847 2,329 11,271 1978 8,513 2,375 10,979 1979 8,966 2,404 11,466 1980 8,765 2,460 11,048 1981 9,042 2,563 11,616 The Fire Department of the City of New York is charged with responsibility for ex-tinguishment, prevention and investigation of fires occurring in the City. The most recent available statistics for operational firefighting incidents (1979) indicate a total of close to 350,000 incidents annually requiring Fire Department attention. The largest number of these incidents have been false alarms (162,529). A total of 43,072 of these incidents involved structural fires, of which the largest number occurred in residential buildings (31,504), the next largest, in vacant buildings (5,698), followed by fires in commercial (4,227) and public structures (1,643). Non-structural fires (71,298) exceeded structural fires. In addition to its ordinary firefighting duties, the Fire Department was, in 1979, called upon to respond to close to 72,250 other emergencies of various types. The Fire Commissioner is the head of the Department, responsible for policy decisions. The Chief of Department is charged with operational responsibility. Under the Chief of Department the uniformed force of the Department consists of the assistant chiefs, deputy assistant chiefs, deputy chiefs, battalion chiefs, chief medical officer, medical officers, chaplains, captains, lieutenants, marine engineers, pilots, and firemen. The uniformed force is organized into divisions, battalions, companies, and other operational units. A division is composed of one or more battalions and is commanded by a battalion chief. A company is composed of its captain, one or more lieutenants, and the firefighters assigned to it. The command structure of Department operations consists of borough commands in each of the five boroughs of New York City. Each borough command (with the exception of Staten Island) is composed of several divisions, the number varying in each borough. The borough commanders report to the Chief of Operations. Companies are divided into engine companies, ladder companies, rescue companies, and marine companies. An engine company consists of a pumper apparatus and an assigned complement of personnel (typically, one captain, three lieutenants, and 20 or 25 firefighters). One platoon (typically, one officer and four or five firefighters) is assigned to the apparatus on each tour of duty. The pumper apparatus consists of pumping equipment and carries lengths of hoseline, nozzles, and other equipment. The engine company is primarily responsible for the extinguishment of fires. A ladder company consists of a ladder truck apparatus and an assigned complement of personnel (typically, one captain, three lieutenants, and 25 firefighters). One platoon (typically, one officer and five firefighters) is assigned to the apparatus on each tour of duty. The ladder truck apparatus includes an extension ladder, portable ladders, tools and equipment for entry and ventilation of burning buildings, and a variety of other equipment. The members of the ladder company perform operations at a fire scene exclusive of extinguishment, including forcible entry, ventilation, search and rescue, and overhauling, the process of seeking hidden sources of fire after the visible fire is extinguished. There are at present 208 engine companies and 138 ladder companies in the Department. These are currently housed in 233 firehouses, most of which quarter one ladder company and one engine company. In addition to engine companies and ladder companies, there are rescue and marine companies. A rescue company consists of the rescue apparatus and an assigned complement of personnel (one captain, three lieutenants and 25 firefighters). A platoon (one officer and five firefighters) is assigned to the apparatus on each tour of duty. The members of a rescue company are assigned on an ad hoc basis at fires to augment ladder or engine company personnel. They also perform specialized search and extrication tasks, utilizing special tools carried on the apparatus for use in emergencies. There are four rescue companies in the City, one assigned to each borough, except Staten Island. Rescue companies respond to multiple alarm fires and unusual emergencies. A marine company is responsible for ex-tinguishment of fires on the waterfront and in New York Harbor. A marine company consists of a captain, three lieutenants, four pilots, eight marine engineers, one wiper, and either ten to twenty firefighters. Forty-five firefighters are assigned to the marine division. Under Section 487a-3.0a. of the New York City Administrative Code (the “Administrative Code”), a member of the Fire Department must be a citizen of the United States, must be able to read and write the English language with understanding, and must have passed his or her eighteenth, but not twenty-ninth, birthday on the date of the filing of an application for civil service examination. No person will be appointed unless twenty-one years of age at the time of appointment. There is at present no restriction on the sex of applicants. There are four grades of firefighter. Section 487a-7.0 of the Administrative Code provides in this connection: “Members of the uniformed force, upon appointment shall be assigned to the fourth grade; after one year of service in the fourth grade, they shall be advanced to the third grade; after one year in the third grade, they shall be advanced to the second grade; after one year in service in the second grade, they shall be advanced to the first grade; and they shall in each instance receive the annual pay or compensation of the grade to which they belong.” The basic starting salary for firefighter fourth grade is currently $17,949. Under Section 487a-4.0 of the Administrative Code, preliminary to a permanent appointment as a firefighter, there is a period of probation of one year. Probationary firefighters undergo a six-week training program in the fundamentals of firefighting at the beginning of their probationary period. The Department of Personnel of New York City has responsibility for the hiring of Fire Department personnel. When staffing needs arise for the position of entry-level firefighter, the Fire Department advises the Department of Personnel and requests the scheduling of an entrance examination. The examination is prepared by the Department of Personnel with the assistance of the Fire Department. Scheduling, notification, receipt of applications, and administration of the examination are the responsibility of the Department of Personnel. The examination produces an eligible list of individuals who have passed the examination. Thereafter, the Department of Personnel and the Fire Department conduct further investigations, including background investigation, medical examination, and interviews, of sufficient members of the eligible list to satisfy projected hiring requirements. Candidates who satisfactorily complete the post-examination investigations are deemed qualified. In order to fill vacancies, the Fire Department will request certification of qualified eligible candidates from the Department of Personnel. Upon issuance of such certification, the Fire Department appoints candidates from the qualified eligible list. Examinations for the Position of Fireman Prior to Exam 3040 From 1960 to 1971 the Department of Personnel administered five examinations for the position of fireman. None of these was open to women. These were the five examinations which immediately preceded Exam 3040, which is at issue in this lawsuit. From October 5, 1960, to January 31, 1961, pursuant to Notice of Examination 9010, the Department of Personnel accepted applications for the position of fireman. A written test for 9010 was administered March 3, 1961, and the eligible list was promulgated January 1, 1962. No physical test appears to have been administered. From July 5, 1962, to October 2, 1962, pursuant to Notice of Examination 9606, the Department of Personnel accepted applications for the position of fireman. A written test for 9606 was administered December 1, 1962, followed by a qualifying physical test. The physical test was graded on a pass/fail basis; the written test had a passing score of 70%. The eligible list resulting from this process was promulgated June 26, 1963. From December 3, 1963, to January 27, 1964, pursuant to Notice of Examination 9984, the Department of Personnel accepted applications for the position of fireman. A written test for 9984 was administered on May 19, 1964, followed by a competitive physical test from which an eligible list was promulgated. The written examination and the competitive physical examination were given equal weight. An average of 70% was required on the results of both written and physical tests in order to qualify for appointment. From April 3, 1968, to May 31, 1968, pursuant to Notice of Examination 7060, the Department of Personnel again accepted applications for the position of fireman. The competitive written test for 7060 was administered June 15, 1968, followed by a qualifying (pass/fail) physical test from which an eligible list was promulgated August 20, 1968. A 70% score on the written test was required to pass. From February 3, 1971, to August 18, 1971, pursuant to Notice of Examination 0159, the Department of Personnel accepted applications for the position of fireman. The written test for 0159 was administered on September 18, 1971, followed by a qualifying (pass/fail) physical test, and an eligible list was promulgated January 18, 1973. Again, a 70% score on the written test was required to pass. The Vulcan Society Litigation On January 12,1973, immediately preceding the promulgation of the eligible list resulting from Exam 0159, the Vulcan and Hispanic Societies, representing black and Hispanic firefighters, commenced an action in the United States District Court for the Southern District of New York pursuant to 42 U.S.C. §§ 1981 and 1983 and sought a preliminary injunction to restrain the New York City Personnel Department, Civil Service Commission, and Fire Department from appointing firemen from List No. 0159. Vulcan Society v. Civil Service Commission, 73 Civ. 201 (Weinfeld, J.). Thereafter, on June 12, 1973, Judge Weinfeld issued an opinion declaring Examination 0159 unconstitutional under the equal protection clause because of its discriminatory impact on blacks and Hispanics and enjoining defendants from making further appointments based on the examination’s results, except upon a showing of compelling necessity. 360 F.Supp. 1265, aff’d in part and remanded, 490 F.2d 387 (2d Cir. 1973). Subsequently, on August 2, 1973, Judge Weinfeld issued an order directing defendants, inter alia, to proceed expeditiously to prepare a new examination for the position of fireman which did not discriminate against blacks and Hispanics in accordance with professionally accepted methods of test preparation. Retention of AIR The City had previously placed itself in a position from which it was able to commence compliance with Judge Weinfeld’s direction with some rapidity. In 1970 Congress had enacted Public Law 91-648, known as the Intergovernmental Personnel Act (“IPA”), 84 Stat. 1909 (1970), which described itself as follows: “An Act to reinforce the federal system by strengthening the personnel resources of State and local governments, to improve intergovernmental cooperation in the administration of grant-in-aid programs, to provide grants for improvement of State and local personnel administration, to authorize Federal assistance in training State and local employees, to provide grants to State and local governments for training of their employees, to authorize interstate compacts for personnel and training activities, to facilitate the temporary assignment of personnel between the Federal Government, the State and local governments, and for other purposes.” Pursuant to Title II of the IPA, “Strengthening State and Local Personnel Administration,” the United States Civil Service Commission was authorized to make grants to state and local governments to enable them to strengthen their staffs by improving personnel administration. Pub. L.No.91-648, § 201, 84 Stat. 1911 (1970). In January 1973, the City’s Personnel Department had determined to take advantage of this federal program by applying for and receiving IPA funding for development of personnel tests and personnel test validation procedures. With the promise of such funding, the City, in January 1973, entered into two contracts with a Washington, D.C. consulting firm named American Institutes for Research (“AIR”) — one directed towards developing a method of constructing valid physical selection tests for physically demanding city jobs, the other directed towards the development of suitable written examinations. Since IPA grants are awarded to support broad research projects and not to pay for the development of an operational selection procedure to select employees for a particular job, the 1973 City contracts looked to the development of generalizable approaches to job analysis, test development, and validation which could be applied to a wide variety of city jobs. In accordance with IPA requirements, both contracts were for one year only. However, each contract was thereafter renewed annually for the following two years. Although general in their overall objectives, both contracts provided that AIR would study three job titles in particular in order to develop more broadly applicable methods. In March 1973 the decision was made that the three jobs to be studied would be fireman, sanitationman, and parking enforcement agent. Specifically, as of January 1, 1973, the City contracted with AIR for the “Development of Physical Test Validation Procedures” to be applied to jobs with physical demands. The contract was authorized under IPA Grant No. 72-14. The full amount of the contract ($37,476) was to be paid with funds received from the U.S. Civil Service Commission. The term of the contract was January 1, 1973, through December 31, 1973, later extended to March 31, 1974. The second year of the City’s contract with AIR for the “Development of Physical Test Validation Procedures” was carried out under IPA Grant No. 73-NY-27c. The full amount of the contract ($22,-500) was to be payable with funds received from the U.S. Civil Service Commission. The term of the contract was April 1974, through December 31, 1974, later extended to February 15, 1975. Also, as of January 1, 1973, the City entered into a contract with AIR to develop written test procedures. Programs were to be developed which, with a minimum number of modifications, might be utilized for most entry-level City positions and some promotional positions. The contract was authorized under IPA Grant No. 72-5, and the full amount of the contract ($59,992) was to be payable with funds received from the U.S. Civil Service Commission. The term of the contract was January 1,1973, to December 31, 1973, but was extended to March 31, 1974. The second year of the City’s contract with AIR for the development of written test validation procedures was carried out under IPA Grant No. 73-NY-28c. The full amount of the contract ($19,956) was to be payable with funds received from the U.S. Civil Service Commission. The term of the contract was April 1, 1974, through December 31, 1974, later extended to February 15, 1975. While the parties to the contracts had contemplated that the overall term of the projects undertaken by AIR would last three years and while IPA grants (No. 75-NY-04c and No. 75-NY-05c in the amounts of $19,922 and $15,000 for the physical and written projects, respectively) to be paid in full by the U.S. Civil Service Commission had been issued, both contracts were can-celled by the City, effective September 30, 1976. These cancellations were explained by witnesses for the City at trial as attributable to the City’s fiscal crisis. Whatever the merits of this explanation the cancellations, as will be seen, had the unfortunate consequence of depriving the City of an opportunity, at least as part of AIR’s IPA-funded work, to produce a criterion-based validation study of the physical test for fireman which is at issue in this litigation— that is, a follow-up study of those who passed the physical test to determine how well they performed as firefighters in order to decide if the test accurately predicted job performance. In all events, in the period immediately following the June 1973 decision in the Vulcan case, with AIR already performing pursuant to these two contracts and, coincidentally, investigating the position of City firefighter as one of the three job titles from which a generalizable approach was to be developed, the City understandably turned to AIR for the specific purpose of developing a new entrance level examination for firefighters as required by Judge Weinfeld’s decision. Accordingly, on October 25,1973, the City entered into a contract with AIR for the purpose of preparing an examination for firemen in the New York City Fire Department. The contract was not to exceed $32,-500, and the work was to be completed by April 12, 1974, although this date too was later extended. In the contract AIR agreed, inter alia, to comply in the course of its work with EEOC Guidelines on employee selection pertaining to fair employment, to conduct a job analysis of the position of fireman, using material and expertise it had acquired on its two other contracts with the City, and to prepare, based on the job analysis, a notice of examination, test plan, written test, medical standards, and minimum standards and procedures to be followed in administration of the physical test. Precisely what portions of its earlier work in connection with the 1972 contracts were, in fact, drawn upon by AIR in developing Exam 3040 pursuant to this contract is the subject of considerable debate between the parties. However, since all parties are agreed that at least the physical abilities analysis (“PAA”) conducted by AIR pursuant to the 1972 physical test contract formed a central part of the job analysis eventually relied upon to develop the firefighters’ exam, it is appropriate to discuss that aspect of the job analysis first. Physical Abilities Analysis The 1972 proposals for the IPA contracts specified that AIR’s work for the City would build on previous research and methodology developed by Dr. Edwin Fleishman, an industrial psychologist, then the head of AIR’s principal office. Dr. Fleishman’s previous research in physical fitness measurement is reported in his book, The Structure and Measurement of Physical Fitness (1964). The book describes a method of factor analysis by which Dr. Fleishman identified eight basic physical fitness “factors” as comprising the domain of human physical performance. Dr. Fleishman isolated these factors by applying a factor-analysis methodology described in the book to test scores on various physical fitness tests — all, coincidentally, administered to male subjects. Factor analysis is a statistical technique for determining the minimum number of factors necessary to account for the inter-correlations among a set of variables. Through factor analysis, Dr. Fleishman isolated clusters of physical tests that appeared to measure the same underlying ability. Each cluster represents a factor. The correlation between a test and a factor is called a factor loading. The eight factors identified by Dr. Fleishman in his 1964 book are static strength, explosive strength, gross body equilibrium, extent flexibility, gross body coordination, dynamic strength, dynamic flexibility, and speed of limb movement. A ninth physical fitness factor, stamina, was subsequently adopted by Dr. Fleishman as an additional physical ability from tests reported by other researchers in the field of industrial psychology. Dr. Fleishman’s 1964 book also identifies the factor loadings of each of the physical fitness tests on each factor and contains recommendations as to which tests best measure each hypothesized underlying ability. Tests were recommended on the basis of being “factorially pure,” i.e., loading highest on the factor tested and not loading significantly on other factors. The book also contains data on the reliability of the various tests recommended. In 1970 and 1971 Dr. Fleishman undertook further research to identify the full range of all human abilities. Based on this work, he reported that there are 37 human abilities: 14 cognitive abilities, nine psycho-motor abilities, five perceptual abilities, and the nine physical abilities enumerated above. Having identified the 37 human abilities, Dr. Fleishman and his colleagues developed a system of rating scales, known as the abilities analysis, which could be used by raters to indicate on a scale of one to seven how much of a given ability is needed to perform a particular task. The rating scales include references to particular concrete instances of human behavior thought to exemplify the ability under consideration and a particular degree of that ability. These concrete examples were intended to serve as “anchors” at intervals along the scale to assist the rater in using the instrument. These behavior anchors were developed based on research with groups of psychologists who devised the task anchors in group sessions and later participated in trying out the scales by using them to rate various tasks. The anchors are descriptions of various kinds of human activity and do not reflect tasks commonly performed by any particular occupational group. For example, anchors under the physical ability of stamina include, at the top of the seven-point scale, “swim the English Channel”; at a point mid-way between five and six on the scale of seven, “climb the stairs to the top of the Statue of Liberty”; and at the bottom of the scale, “walk to the corner grocery.” Under gross body coordination the anchors are “do a skilled ballet dance like Swan Lake”; “a runner jumps a series of ten three-foot hurdles”; and “make a lay up shot in the basketball game.” Under extent flexibility the anchors are “side show lady bends herself into a pretzel shape and other unusual positions”; “do a modern dance”; and, at the bottom of the scale, “reach for the control lever of a drill press.” Using these scales the person completing the abilities analysis is asked to state “how much” of each ability is needed to do the task under study. In addition to the task anchors, the test instrument gives a definition of the ability, e.g.: “Static Strength. This ability involves the amount of muscle force (power) used against an object to lift, push or pull the object. Force is used without stopping up to the amount needed to move the object. This ability can involve different muscle groups like the hand, arm, back, shoulder, and leg. This ability does not involve the use of force over a long time. This ability does not deal with the number of times the act is repeated.” (Emphasis in original) Further, an effort is made in each instance to differentiate the ability under consideration from other abilities with which it might be confused. Thus, static strength is contrasted with explosive and dynamic strength and with stamina as follows: “HOW STATIC STRENGTH IS DIFFERENT FROM OTHER ABILITIES: Use force without stopping to lift, push or pull objects vs. Explosive Strength (2): Gather energy to move the body’s own weight or objects with short bursts of force. Use force to lift, push or pull objects vs. Dynamic Strength (3): Use muscle power (force) repeatedly or without stopping to hold up or move the body’s own weight. Does not involve the use of force over a long time vs. Stamina (4): Do physical work over a long time; involves heart and blood vessels resisting becoming tired.” Further, the persons performing the abilities analysis are asked to complete a sheet entitled “comments” on which they are to state separately with respect to “usual tasks” (described as those performed every day or almost every day on the job) and with respect to “special tasks” — “what the worker does” and “under what circumstances the worker does it.” It is principally based on these comments that defendants contend that the PAA was content validated as grounded in the important observable behaviors of firefighters. The Job Analysis of the Physical Requirements of the Firefighter In November 1973, AIR administered the PAA to 23 firemen and 12 fire officers. This use of the PAA was the first time that the instrument had ever been used in an actual analysis of an entire job as opposed to the analysis of a specific task. The firemen and officers were asked to rate the amount of each ability required by all of the usual tasks of their job and by all of the special tasks of their job on a scale of 0 to 7. Usual tasks were, as noted above, defined as tasks performed every day or every other day. Special tasks were defined as tasks performed less often. The PAA questionnaire also asked the raters to provide written comments, giving examples of both usual and special tasks which required each ability. These comments provide some idea of the various specific tasks which the raters were considering in answering the questions directed to the rather abstract abilities they were asked to rate. For some reason, most probably because of the difficulties presented by the complex analysis instrument, relatively few of the 35 firemen who did the PAA were able to think of special tasks — that is, ones performed on a less frequent basis than every day or almost every day on the job. As a result, noting that the rank orderings for special tasks would be influenced by very small differences in mean ratings and calling them “secondary ratings,” AIR determined to rely on the ratings given for every day tasks and noted that the results for the special tasks analysis “should be viewed with caution and considered merely suggestive.” Using the responses given for usual tasks, the abilities were then ranked as follows, based on the mean ratings for both officers and firemen: Mean Score on Ability Scale of 7 Stamina 5.41 Static Strength 4.64 Explosive Strength 4.41 Gross Body Equilibrium 4.27 Extent Flexibility 4.26 Gross Body Coordination 4.04 Dynamic Strength 4.03 Dynamic Flexibility 3.74 Speed of Limb Movement 3.41 The AIR effort was not only the first occasion on which an entire job was analyzed using the PAA; it was also the first time job incumbents, as opposed to professional scientists, were asked to perform the rather abstract abilities analysis. In all earlier uses of the test instruments to rate specific tasks, the ratings had been made by trained psychologists, familiar with the mechanisms of the analysis, who had at least some acquaintance with the definitions they were being asked to apply and some of whom had, in fact, participated in the preparation of the test anchors used in an effort to make the analysis concrete. Given the complexity of the rating scales and the nature of the task anchors, the analysis is clearly capable of producing a different result when used by different individuals with different backgrounds. Thus, the significance of an anchor for static strength (which is set at a level of 5.8 on a scale of 7) described as loading five full fifty-gallon drums into a truck may be quite different for a person who has done the task described than for a person who has not. In fact, there is considerable evidence that actual confusion and misuse of the instrument did occur. As already noted, a large number of respondents failed to find the selected abilities present at all in special firefighting tasks, i.e., those occurring less frequently than once every one or two days. Moreover, many of the same tasks were considered usual tasks by some respondents and special tasks by others. Thus, “overhauling using ladders,” “ventilation,” “lifting people to stretchers,” “moving large refrigerator or furniture in fire building,” “making rescue of person or persons of varying weights under adverse conditions,” and “forcible entry” are all listed as special tasks by some respondents. Other respondents describe essentially the same tasks, e.g., “moving usual household items such as refrigerator in a time of heavy fire conditions,” “overhauling at a fire,” “needed when forcing entry through doors or walls under fire conditions,” “moving heavy refrigerators, furniture, unconscious persons,” “to remove person from smoke-filled room or apartment,” “force doors, vent,” as a part of usual operations. Moreover, many of the comments list the same activity or job behavior as an example of all or almost all of the abilities inquired about in the PAA questionnaire. For example, “functions at fire operations” and “stretching a hose” is given as an example of each of the given abilities in one fireman’s comments. While this may reflect a correct understanding of the abilities being asked about (it is possible, for example, given the complexity of body movements involved in stretching a hose, to find at one point or another in the evolution an example of static strength, explosive strength, dynamic strength, stamina, extent flexibility, and gross body coordination), it is difficult to say, given the complexity of the example of job behavior offered in the comment, whether the example reflects a correct understanding of the ability or not. Nor is it clear that one has given concrete support for the degree of importance attributed to the ability when one has cited a complex job behavior involving to some unspecified degree the ability inquired about, but also involving other abilities. This point may be made clearer by examining the definition of dynamic strength given in the PAA, along with the contrasting definitions provided, and then considering a number of examples given for that ability in the comments of firemen. Dynamic strength is defined in the PAA instrument as follows: “This ability involves the power of arm and trunk muscles to hold up or move the body’s own weight repeatedly or at one time without stopping. This ability involves muscles resisting getting tired when having to repeatedly or without stopping to hold up or move the body’s own weight.” (Emphasis in original) Contrasts are then provided between dynamic strength and static strength (involving the use of force to lift, push or pull objects), explosive strength (involving short bursts of muscle force to move the body’s own weight or objects), and stamina (defined as work over a long time, involving “heart and blood vessels resisting becoming tired”). The following comments by firemen, listing job behaviors thought to exemplify dynamic strength, are taken in order from a list prepared by AIR from the firemen’s comments: “Pull hose up six flights in smoke, pull ceilings; overhauling operations; carry persons up or down stairs; to and from places. “Advancing a line into a building, in heat and smoke. Rescue operations. [Underlined comments represent a special task.] “Advancing a line when charged up two or three floors or when pulling a charged line in a window off the fire escape four floors up. “Firefighting; climbing stairs with tools and Air Paks on backs; climbing ladders; leaning over roofs to break windows. “Under heavy fire conditions in rescues, this is pertinent. “Pulling and running at a fire. “Act of fighting a fire — carrying in line, movement from place to place with line while extinguishing fire. “Needed when advancing charged line at fire. Needed for opening ceilings and walls to allow access to fire to engine company. “Pulling your own body up to the top of tall bulkheads. Pulling yourself over fences and walls to reach a certain objective (rear office building, adjoining roof tops). Pulling yourself up outside of fire escape to pass fire.” What is noteworthy about this list, when it is compared with the definition for dynamic strength given above, is that none of the groups of examples, except the last, appears to involve the defined ability as anything like its most noticeable characteristic. Indeed, each of the first eight activities described appear to involve first and foremost the contrasted ability of static strength in which force is used to lift, push or pull objects. The contrasted ability of stamina and other unmentioned abilities such as extent flexibility appear as much involved in the examples listed as the defined ability of dynamic strength. When these examples are placed alongside others in which the defined ability is almost certainly misunderstood, e.g., “moving Multi-Yersal Nozzle from apparatus” as an example of explosive strength, it appears that the PAA and its ranking of abilities are not so well grounded in the observable behaviors of firefighters that one can predict with any confidence from the degree of which the abilities are possessed the capacity of an individual to perform well as a firefighter. Moreover, despite the small size of the group used to perform the analysis and the failure to select that group on a random basis, no effort appears to have been made — either on a subjective basis, by weeding out responses that evinced misunderstanding or inappropriate use of the scales, or on the more objective and systematic basis — to compute inter-rater correlation coefficients. Inter-rater correlation coefficients are statistical expressions of the degree to which raters agree with each other. Without inter-rater correlation coefficients, a researcher can have only a rough idea with regard to the consistency with which raters ranked the rated abilities. The only inter-rater correlations which do appear to have been calculated — those comparing the rank ordering by firemen with the rank ordering by officers — supply further troublesome data, since there appears to have been a significantly greater degree of agreement between the officers and men in the more concrete physical demands analysis (“PDA”) than that found with respect to the PAA. Finally, AIR failed to reconcile the results of the PAA with data from other job analysis techniques. In this connection, it appears appropriate to consider, in particular, the PDA in some detail in part because of the contrast between the PDA and the PAA in the degree to which the analysis is concretely grounded in observable work behaviors. The Physical Demands Analysis The PDA is a job analysis method, designed by the U.S. Department of Labor, which describes systematically the physical demands of a job in terms of 11 different activities: strength, climbing, balancing, stooping, kneeling, crouching, crawling, reaching, handling, fingering, and feeling. On November 13, 1973, a panel of 19 firemen and 12 officers under the supervision of AIR used the PDA instrument to describe the job of fireman. The panel was first asked to divide the job into six categories — sitting, standing, crawling, walking, running, and other — and to state the percentage of time spent in each activity. The results showed that 71.7% of the firefighter’s time was spent in the more physically taxing work positions of standing, walking, running, and crawling. Next, the panel was asked to analyze the strength demands of the job by stating in terms of “sometimes,” “often,” and “very often” the need to lift, carry, push or pull five categories of weights from the very light (0-10 lbs.) to the very heavy (100 + lbs.). The results of this exercise showed that firefighters sometimes have to lift weights on the average of 110 lbs., carry weights on the same average, push weights of 161 lbs., and pull weights of 125 lbs. At the other end of the scale, firemen “very often” lift, carry, push or pull weights averaging 63 lbs. Each of these activities — lifting, carrying, pushing, and pulling — were then particularized by questions as to what was required to be moved, the circumstances, the tools to be used, and the nature of any special job or task requiring the ability. In addition, respondents were asked to rate the importance of strength in general on a scale of no importance, important, and high importance. Although the answers to these questions would appear to have been particularly valuable in establishing a content valid test for strength, little use appears to have been made of them in fact, since the PAA provided another system of rating strength as a required ability in terms of three separate components, static strength, explosive strength, and dynamic strength, and since it was the PAA which, as noted below, was principally relied upon. After dealing with strength in this fashion, the PDA further investigated each of the remaining activities (e.g., climbing, crawling, balancing, fingering, etc.) by asking whether it is present in the job, its importance in terms of the same importance scale referred to above and what climbing, balancing, etc. the worker did, (and, in the case of climbing, how far the firefighter had to climb), the circumstances under which the activity was performed, and the name of any special task requiring the activity. While AIR made some effort to compare the results of the PDA with the results of the PAA, some of the comparisons smack of an effort at rationalization of apparent conflicts in the results of the two analyses to support use of the PAA. Thus, although climbing and crawling were rated first and fourth out of the eleven behaviors on the PDA, the ability of dynamic strength with which AIR associated them is given a rank of seven out of the nine abilities on the PAA. The differences are to be reconciled, according to AIR, because dynamic strength was ranked third with respect to special tasks on the PAA. Elsewhere, however, as noted above, AIR had cautioned that the general lack of response to the special tasks portion of the PAA made it not a very useful part of the job analysis. As a result, the contrast between the ratings of the PDA and the PAA for dynamic strength remains troubling. So too, AIR noted that reaching is ranked considerably lower on the PDA than the ability with which AIR associated it on the PAA (extent flexibility). AIR explained this difference as “most likely” reflecting the difference of definition given for the two physical activities. In other areas no comparison at all is made between the ratings achieved on the PAA and on the PDA because the differences of definition between the physical demands involved in the PDA and the physical abilities involved in the PAA make clear that different matters are being inquired about. Thus, as firmly grounded as the PDA appears to be in the observable behaviors of the job of firefighter, it does not appear that those observable behaviors provide much support for the analysis accomplished by the PAA. The Test Development As Dr. Fleishman testified and as AIR’s reports to the City confirm, the principal reason for selecting the PAA method of job analysis for use in connection with Exam 3040 over the other techniques explored pursuant to the 1972 contracts was that the appropriate tests to determine applicants’ abilities identified as necessary to the fireman’s job had already been the subject of extensive study chiefly by Dr. Fleishman himself in his 1964 book and later work. As already noted, the appropriate test for each ability, Dr. Fleishman had determined, was, all else being equal, the one most factorially pure for the abilities determined by the PAA to be important for the job, that is to say, tests with relatively high loadings on those factors and relative low loadings on other factors. In order to assure that the tests did not unduly overlap in measuring the same abilities, and also as the beginning of an effort, which could not draw to any substantial degree on Dr. Fleishman’s prior work, namely, an effort to develop an appropriate scoring and weighting system, AIR, in December 1973, tried out an experimental battery of 11 factorially “pure” tests of eight physical abilities selected on the basis of the PAA, as well as a test of discrimination reaction time, a psyehomotor ability. The 11 physical tests tried out in December 1973 and the eight abilities (in the rank order established by the PAA) they were intended follows: to measure were as The tests were administered at the Baruch Recreational Center, which contains a small gym, on the Lower East Side of Manhattan, to 68 men then in training to become firefighters and to 32 firemen who had recently completed their training and were in the first probationary year. All of the persons taking the trial tests had, thus, taken and passed the previous physical Exam 0159, administered by the Fire Department two years before, in 1971. The reason for using trainees and probationers as test subjects was to try out the tests to be used on a sample closely approximating the applicant population expected to take Exam 3040. (The sample was not confined exclusively to trainees because there were not enough at the time to produce a test sample of 100.) Also, to this same end, and mindful of its obligation to the litigants in the Vulcan case, the City selected a sample designed to include a sufficient number of white, black, and Hispanic men to permit evaluation of the relative performance of each group on the various tests. Since, however, it was anticipated at this point that women would again not be permitted to apply for the firefighters position, no women were included in the December 1973 tryout sample. After the tryout battery was given, AIR computed correlations among the various tests to determine the degree of overlap between them and, also, correlated the tryout scores of 95 of the men in the sample with their scores on the five physical tests of Exam 0159 (the physical test previously used by the City), one of which was an agility test. The use to which the results of these correlations were put in determining which tests were to be included in Exam 3040 and, most importantly, in establishing the scoring of the exam will be discussed infra. Before turning to the content of the battery of tests recommended by AIR to the City for inclusion in Exam 3040, however, it is appropriate to note at this point the occurrence of another event which was, in its implications, at least, to have profound effects on the test development of Exam 3040. In early 1974, because of concern about the impact of a height requirement on Hispanic persons, then-Fire Commissioner O’Hagan asked AIR to consider the appropriate height requirement for the fireman job. The Height-Related Tests On January 14, 1974, 12 individuals of varying heights (six firemen and six non-firemen from the Model Cities Program) performed a number of actual job sample tasks, thought to be height- and job-related, to determine what relationship, if any, existed between the height of an individual and his ability to perform firefighting tasks. Subsequently, on January 29, and February 2, 1974, the following tasks and one test, again thought to be height-related, were administered to a group of 100 firemen and 100 non-firemen, all male, including persons shorter than the City’s then current 5'7" minimum height requirement for firefighters: (1) Ladder Climb — Subject climbs 35-foot ladder wearing a Scott air pack, coat, boots, and hat; (2) Ladder Raise and Positioning— Subject raises non-extended 20-foot length of 35-foot extension ladder from floor to wall and back down; (3) Window Ventilation — Subject swings six-foot hook to a designated target area in upper right quadrant of a wall; (4) Hose Unclamping — Subject removes one end of a hose section from a 74"-high clamp (with an examiner’s assistant unclamping and steadying the other end), lowers the hose to the ground, and then raises and re-clamps; (5) Victim Rescue — Subject carries a 120-pound dummy up and down one flight of stairs one time; (6) Vertical Hose Stretch — Subject hauls over the shoulder the nozzle section of a 2V2" hose up and down one flight of stairs; (7) Static Strength — Hand grip test. On February 22, 1974, immediately following these tests, AIR sent to the City a recommended test plan for both the written and physical portions of Exam 3040, including recommended tests and scoring tables. For the physical test battery, AIR recommended eight tests to test the parenthetically indicated abilities: the one-mile run/walk (stamina), hand grip preferred (static strength), dummy carry (static strength, explosive strength, dynamic strength, and gross body equilibrium), agility (explosive strength, stamina, dynamic strength and gross body coordination), freestyle broad jump (explosive strength), balance (gross body equilibrium), twist and touch (extent flexibility), and push-ups (dynamic strength). Of these eight recommended tests, five were factorially pure tests discussed in Dr. Fleishman’s 1964 book, and one, the mile run, was a factorially pure test for stamina not included in his book but recognized in the literature. However, as indicated by the descriptions given above, the agility test and the dummy carry were also recommended although they were not factorially pure tests. Their inclusion in the test battery requires separate discussion. The “Face Valid” Tests The agility test recommended in February 1974 — consisting of an obstacle course with two walls, one 5' and one 8' — appears to have been drawn directly from the prior Fire Department Exam 0159, administered on a pass/fail basis in 1971. Why it was substituted, as it appears to have been, for the more factorially pure cable jump test is not at all clear. Whereas the cable jump test is said by Dr. Fleishman to measure the sixth-ranked ability of gross body coordination, the dodge-and-run aspect of the agility test recommended is said by Fleishman in his 1964 book to measure the third-ranked ability of explosive strength — an ability already tested for by two of the recommended subtests (free-style broad jump and dummy carry). Moreover, in calculating intercorrelations between scores for the agility test on the December 1973 tryout and on Exam 0159, AIR determined that the agility test had a moderate correlation with tests for explosive strength and stamina, a somewhat lower correlation with gross body equilibrium, and the lowest correlation of all with gross body coordination — the ability tested by the cable jump test for which it was substituted. This substitution became of crucial importance for plaintiff and the class she represents because the height-related eight-foot wall which formed part of the agility test became not just a stumbling block, but a literal barrier for almost all women taking the exam. In January 1974, immediately before submission of the proposed test battery to the City, an effort was made by AIR to justify the inclusion of the eight-foot wall in the agility test. The AIR letter reporting the results of an effort to justify the height chosen, however, explains its rationale not in terms of the height needed to establish the ability (explosive strength) sought to be tested, but rather in terms of the nature of the fireman’s job. Moreover, rather than refer to the results of any of the job analyses already performed in order to determine if the analyses had established the relative importance of scaling walls as a task performed by firemen, AIR concluded, apparently on the basis of a single unrecorded interview between an AIR representative and representatives of the Fire Department, that firemen in New York have occasion to encounter eight-foot parapets between roof tops which must be scaled in the course of their work. The relative importance of this work behavior to the job as a whole is nowhere addressed in AIR’s job analysis. The dummy test (as it came to be known) appears to have had its origin in the height study requested by then-Commissioner O’Hagan. Again the reason for inclusion of the test in the February recommendations is unclear. Whatever the explanation for the inclusion — whether a desire to aid minorities or a recognition, which later proved accurate, that the City and the Department of Personnel would not accept too many tests with which they were unfamiliar either from earlier exams, the athletic field, or notions of what the fireman’s job entailed — inclusion of the test presented the same difficulties, given AIR’s overall approach, as the inclusion of the agility test. Again, the complexity of the test assured that more than one ability was, in all likelihood, being tested for, thus increasing the problem, sought to be avoided by Fleishman’s factorially pure approach, that scoring would be distorted by measuring more than once for the same ability. In addition, since neither the dummy carry nor the agility test arose out of any systematic analysis of observable work behaviors (although appearing to owe their justification to such an origin), the addition of the tests reinforced the need for a criterion-related validity study. The same movement away from the measurement of the relatively abstract abilities required for the job based on Fleishman’s abilities analysis towards so-called “face valid” tests continued thereafter as City officials began to criticize and question the proposed test battery, in some cases on quite subjective grounds. Thus, both the Chief of the Fire Department and the Chief of the Department of Personnel complained of the scoring plan for the mile run (which permitted a candidate to complete the mile in 12 minutes and still not fail), not on the basis of its methodology, but simply because they or some close relative could run the mile in less time. Similarly, the so-called twist-and-touch test for the ability of extent flexibility and the balance test for gross body equilibrium were the subjects of complaints by then-Commissioner O’Hagan to representatives of the City Department of Personnel and through the Department to AIR on the ground that what was wanted was a test “for firefighters, not for ballet dancers.” In response to this criticism of the mile run, AIR appears to have initially reacted quite appropriately by re-examining the data on the basis of which its scoring of the mile run had been determined. This re-examination revealed that AIR had developed the March 1974 scoring norms based on the distribution of scores in the December 1973 tryouts, using a 31-lap track at the Baruch gym. Obviously, the effect of having so many short laps was to increase the average time of the runners. When City officials complained that the scoring system was too lenient and insisted it be made more stringent, AIR asked to try out the mile run again on a four-lap track. This request was, however, rejected. Instead, the representatives of the City’s Department of Personnel directed AIR’s attention to a standard reference work by T. K. Cureton, Jr., which set out suggested scores for the mile run based on norms developed from runs performed by an all-male sample population consisting of high school, college, and military men involved in a program of daily physical training in preparation for military service in World War II. AIR accepted the suggestion that this more stringent scoring norm be used with little if any attention to the logic of using such norms in its overall scoring system, either in terms of job requirements or norms in the applicant pool. A substitute for the rejected twist-and-touch test appears to have emerged from a visit by AIR, Fire Department personnel, and a representative of the Personnel Department to a physical examination for firefighters being administered by the City of Chicago in August 1974, made at the specific request of then-Commissioner O’Hagan who had already expressed his dissatisfaction with AIR’s test battery. The Chicago battery consisted of five tests: flexed arm hang, obstacle run, stair climb, man lift and carry, and hose coupling. The AIR representative summoned to Chicago was asked to observe and evaluate the Chicago battery as compared with AIR’s. Following this visit, AIR continued to recommend the tests it had originally proposed, based primarily on differences in the job analysis and testing techniques being used, with Chicago testing for multiple abilities in tests resembling actual job tasks and New York testing for single abilities by means of factorially pure tests highly loaded for a single factor deemed important to the job. In addition, AIR opposed use of the one factorially pure test used in Chicago, the flexed arm hang, because of the low degree of reliability of that test from one administration of the test to another — so called test/re-test reliability. Not satisfied with this response, a representative of the City’s Department of Personnel then contacted, apparently without consulting AIR, the consultants used by the City of Chicago in preparing that City’s physical exam and asked those consultants to criticize AIR’s proposed test battery. This criticism served to confirm the impression of the City officials that the mile run scoring was too lenient and that the twist- and-touch and balance tests were “inappropriate” for firemen. In addition, the Chicago experts pointed out that at least one of the tests, push-ups, would have an adverse impact on women because of their lower average upper-body strength. Without reaching any professionally acceptable resolution of these differences, AIR thereafter acceded to the City’s requests by substituting a test dubbed a “ledge walk” test for Fleishman’s balance test of gross body equilibrium, adding a new “obstacle” to the Exam 0159 obstacle test called “window-ladder-window” as a test for extent flexibility and replacing push-ups with what AIR had previously termed a relatively unreliable test of dynamic strength: the flexed arm hang. Both the “window-ladder-window” obstacle and the flexed arm hang were drawn directly from the Chicago battery. The “ledge walk” appears to be entirely unstudied and unprecedented as a test of the ability of gross body equilibrium which it purported to measure either in terms of factor analysis or reliability. The “window-ladder-window” test, by being incorporated into the agility test, came to measure extent flexibility (to the extent it can be said to have reliably measured it at all) in terms of time rather than in terms of the inches to which one could extend oneself, as under the original twist and touch test. The ledge walk, performed with eyes open, no longer eliminated that type of balance produced by visual eye contacts, as was the case with Fleishman’s balance test. In fact, there appears no basis for saying that any of the three tests reliably measured the abstract abilities measured by the Fleishman tests for which they were substituted. As the result of these additions to the proposed test battery, it became necessary to conduct another tryout test to establish new scoring norms. This tryout was administered in early November 1974 to a group of 75 incumbent firemen, only six of whom were probationers. The November tryout battery consisted of the new tests: the flexed arm hang, ledge walk, and a window-ladder-window test (consisting of crawling through a window, across a horizontal ladder and through another window). In addition, the dummy carry was tried out again apparently in order to get new scoring norms. A final proposed test battery for Exam 3040 was forwarded to the Department of Personnel on November 29, 1974. The physical portion of the exam contained seven physical subte