Harden v. Department of Professional Regulation (18 Fla. Supp. 2d 212, 1986)

OPINION

MICHAEL M. PARRISH, Hearing Officer.

FINAL ORDER

Pursuant to notice, a formal hearing was held in this rule challenge case at Miami, Florida, before Michael M. Parrish, a duly designated Hearing Officer of the Division of Administrative Hearings, on the following days: October 24, 29, 30, 31 and November 1, 13, 14, 20, 26, and 27, 1984. The parties were represented at the hearing by the following counsel:

At the hearing numerous witnesses were called by both parties and voluminous exhibits were offered into evidence. The ten days of hearing in this case were reported by court reporters, but no transcript of the proceedings has been prepared. Following the completion of the hearing all parties submitted post-hearing materials to the Hearing Officer in the form of Proposal Final Orders containing proposed findings of fact, proposed conclusions of law, and argument in support of their respective positions. All of these post-hearing submissions have been given careful consideration in the formulation of this Final Order. Specific rulings on all proposed findings of fact submitted by all parties are contained in the Appendix which is attached to and incorporated in this Final Order. The parties waived the requirement in Section 120.56(3), Florida Statutes, providing for rendition of this Final Order within 30 days of the hearing.

INTRODUCTION

By Petition filed on January 25, 1984, the Petitioners have challenged the validity of numerous rules and alleged rules of the Department of Professional Regulation and the Electrical Contractors’ Licensing Board, which are described as follows in the Petition.

(A) Rule 21GG-6.01(c) of the Rules of the Electrical Contractors’ Licensing Board;

(B) Rule 21GG-6.01(1),(2),(3) of the Rules of the Electrical Contractors’ Licensing Board;

(C) Rule 21GG-6.01(4) of the Rules of the Electrical Contractors’ Licensing Board;

(D) Rule 21.11.11(3) of the Rules of the Department of Professioal Regulation, which was subsequently amended in October 1982;

(E) Rule 21-11.11(13) of the Rules of the Department of Professional Regulation, which was subsequently renumbered in October 1982, and appears verbatim as Rule 21-11.11(3)(1);

(F) Unpublished Rule of the Electrical Contractors’ Licensing Board enacted on November 9, 1982, which provides for reassigning point credits to examination questions on the July 1982 Electrical Contractors’ Licensing Examination;

(G) Unpublished Rule of the Department of Professional Regulation which provides that a candidate who requests an examination review of his own examination papers may be provided with a copy of a master examination and designed the right to review his own examination papers or true and correct copies of his own examination papers;

(H) Rule 21-11.11(3) of the Rules of the Department of Professional Regulation as amended in October 1982;

(I) Rule 21-11.14 of the Rules of the Department of Professional Regulation;

(J) Unpublished Rule of the Department of Professional Regulation that the examination booklets of a candidate for licensure by the Electrical Contractors’ Licensing Board can be destroyed if a specific request to review the same in addition to a request for examination review is not made;

(K) Unpublished Rule of the Department of Professional Regulation that the actual examination booklet of a candidate for licensure by the Electrical Contractors’ Licensing Board or a photocopy or exact duplicate of said examination papers need not be retained for a period of two years;

(L) Unpublished Rule of the Department of Professional Regulation that they shall not consider supplemental material or provide a copy of the same to the Electrical Contractors’ Licensing Board in connection with the review of a licensing examination, where the examination of the candidate providing the supplemental information is under review;

(M) Unpublished Rules of the Department of Professional Regulation and the Electrical Contractors’ Licensing Board during the review of a candidate’s licensing examination that the materials being reviewed do not have to be furnished to the candidate whose examination is being reviewed;

(N) Unpublished Rules of the Department of Professional Regulation and the Electrical Contractors’ Licensing Board that a question which requires a candidate to chose “the correct answer” may have more than one correct answer and remain a valid question for testing competency so that credit for the two “incorrect” answers are not given to candidates, while credit for the two “incorrect” answers is given to candidates;

(O) Unpublished Rule of the Electrical Contractors’ Licensing Board that the Board does not have to consider all supplemental material provided by candidates whose examinations are being reviewed and may refuse to consider part or all of the supplemental information provided by some candidates;

(P) Unpublished Rule of the Electrical Contractors’ Licensing Board that at the review of the candidate’s licensing examination of the candidate does not have an opportunity to be heard prior to the vote of the Board;

(Q) Unwritten Rule of the Department of Professional Regulation that the package of documents which they prepare and submit to the Electrical Contractors’ Licensing Board during the review of candidates’ examinations is not maintained for a period of two years and cannot be produced intact pursuant to a Request for Production of same in a Chapter 120.57 examination challenge hearing;

(R) Unwritten Rule of the Department of Professional Regulation that consultants in technical areas are not consulted and asked to review challenged examination questions even when the challenged questions are not in the area of expertise of Department of Professional Regulation staff members who formulate recommendations to the Electrical Contractors’ Licensing Board regarding the validity and reliability of such questions;

(S) Unpublished Rules of the Board and the Department of Professional Regulation that consultants in technical areas may be consulted and asked to review challenged examination questions in a given area, but they are not asked to review all challenged questions in said area even where the Department of Professional Regulation staff members who formulate recommendations to the Electrical Contractors’ Licensing Board regarding the validity and reliability of such questions do not have such technical expertise to make such recommendations;

(T) Unwritten Rules of the Department of Professional Regulation and the Electrical Contractors’ Licensing Board that a question on the Electrical Contractors’ Licensing Examination is valid and reliable solely because that question can be found in an approved reference, and, therefore need not be reviewed;

(U) Unwritten Rules of the Department of Professional Regulation and Electrical Contractors’ Licensing Board that a question on the Electrical Contractors’ Licensing Examination is valid for testing minimum competency when the question cannot be answered from any information contained in any of the approved reference materials which candidates are advised to study in preparation for the licensing examination;

(V) Unwritten Rule of the Electrical Contractors’ Licensing Board that they will not permit a candidate to present any expert testimony at the examination review of that candidate’s examination by the Board;

(W) Rule 21GG-6.03 of the Electrical Contractors’ Licensing Board;

(X) Unpublished Rule of the Electrical Contractors’ Licensing Board requiring more than minimum competency in order for a candidate to receive state certification and licensure as an electrical contractor;

(Y) Unpublished Rule of the Electrical Contractors’ Licensing Board that each Board member did not vote on each and every matter affecting the Petitioners’ examination reviews, their licensing, or issues related thereto;

(Z) Unpublished Rule of the Electrical Contractors’ Licensing Board that the votes of each member are not recorded on each and every matter on which action is taken, even when a specific request is made therefore;

(AA) Unpublished Rules of the Department of Professional Regulation and the Electrical Contractors’ Licensing board that exceptions to the National Electrical Code are not to be applied by candidates unless they are directed to apply exceptions in the stem of examination questions;

(BB) Unpublished Rules of the Department of Professional Regulation and the Electrical Contractors’ Licensing Board that no attorney or agent of any examinee is entitled to see the examinee’s examination papers, notes of the Department or the Board, or any documents or information contained by the Department or the Board with respect to said examinee’s examination unless and until the candidate commences a Chapter 120.57 hearing and makes a request for production of same and until the hearing officer orders production of same;

(CC) Unpublished Rule of the Electrical Contractors’ Licensing Board and the Department of Professional Regulation that the examination review by the Board is conducted without a complete discussion of the questions under review, and that every effort is made to refer to any item under discussion in terms of its number or in such terms so that the contents thereof are not disclosed to the public;

(DD) Unpublished Rule of the Department of Professional Regulation and the Electrical Contractors’ Licensing Board that Board examination review is delegated to the Department of Professional Regulation and the Examination Committee of the Board, and that if the Department of Professional Regulation or the Examination Committee indicate that they have verified a question and found the question to be valid, no effort is made by the Electrical Contractors’ Licensing Board to review the validity of reliability of the challenged examination question;

(EE) Unpublished Rules of the Electrical Contractors’ Licensing Board and the Department of Professional Regulation that candidates were not permitted during the administration of the subject examinations to use any applicable code reference books or any approved reference materials which contained notes or additions thereto;

(FF) Unpublished Rule of the Electrical Contractors’ Licensing Board that the examination review is “anonymous” and the identity of a candidate who has requested an examination review shall not be disclosed.

The Petitioners contend that all of the promulgated rules, unpublished rules, and unwritten rules described above are invalid rules for a number of different reasons that are set forth in their Petition. The basic contentions of the Respondents are that the promulgated rules are valid rules and that the rules described in the Petition as “unpublished” or “unwritten” rules of the Department and Board are, in fact, not rules at all.

AMENDMENTS TO THE PETITION

During the course of the hearing the parties stipulated to the following amendments to the Petition.

At paragraph 9, first line, “1982” is changed to read “1981.”

At paragraph 28, first line, “January” is changed to read “July.”

At paragraph 41, fourth line, “it” is changed to read “its.”

At paragraph 75, delete the entire paragraph.

At paragraph 79, fourth line, “March 20” is changed to read “March 19 and 20.”

At paragraph 110, the entire paragraph is amended to read as follows: “The amendment to Rule 21GG-6.01(4) required that electrical contractors pass the certification examination with a score of 75 percent, whereas all other construction industry licensing board contractors licensed by the Department of Professional Regulation (13 other categories) are merely required to achieve a 70 percent pass score.”

At paragraph 131, the following is added at the end of the paragraph: “and Rule 21GG-6.03.”

At paragraph 135, the last sentence of the paragraph is changed to read: “Accordingly, if it is determined that Rule 21-11.11(13) and Rule 21GG-6.03 shall in any way preclude Petitioners from achieving their licensure, then Rule 21-11.11(13) and Rule 21GG-6.03 are invalid, arbitrary and lacking in rational basis.”

At paragraph 136, the first line is changed to read as follows: “That RUle 21.11.11(13) and Rule 21GG-6.03 are invalid as applied if they mandate.”

At paragraph 137, the first line is changed to read the same as the changed first line of paragraph 136.

At paragraph 138, the first sentence is changed to read as follows: “That there are no standards within Rule 21-11.11(13) and Rule 21GG-6.03 wherein those affected by their application may know how they are to be applied.

At paragraph 211, second line, “the” is changed to “their.”

POST-HEARING MOTIONS

The Petitioners have filed three post-hearing motions in this case which remained to be ruled upon. They are (1) a motion to amend petition to request costs as part of the Petitioners’ request for relief filed on July 8, 1985, (2) a motion for payment of expert witness fees to certain of the Petitioners’ expert witnesses filed on July 8, 1985, and (3) a motion for expert witness fees, legal fees and costs filed on January 28, 1986. The July 8, 1985 motion to amend the petition and the portion of the January 28, 1986, motion dealing with legal fees and costs are addressed in the Conclusions of Law portion of this Final Order. The motion for payment of expert witness fees is addressed in a separate order issued simultaneously with this Final Order (Similar motions filed in related examination challenge proceedings under Section 120.57, Florida Statutes, will be addressed in the Recommended Order to be issued in those cases.)

FINDINGS OF FACT

Based on the stipulations and admissions of the parties, the exhibits received in evidence, and the testimony of the witnesses at the hearing in this case, I make the following findings of fact.

Findings based on stipulations and admissions of the parties

1. The Petitioners, John Eugene Harden and Dova Cauthen, qualified for and were administered the January 1982 Electrical Contractors’ Licensing Examination. After their examinations were graded and regraded, Petitioners were notified that they had not received a “Passing” score of 75 or more.

2. The Petitioners, John Eugene Harden and Dova Cauthen, qualified for and were administered the July 1982 Electrical Contractors’ Licensing Examination. After their examinations were graded and regraded, Petitioners were notified that they had not received a “Passing” score of 75 or more.

3. Both Petitioners reside in Dade County, Florida.

4. The Respondent Department of Professional Regulation (hereafter DPR or the “Department”) is an agency of the State of Florida.

5. The Respondent Electrical Contractors’ Licensing Board (hereafter referred to as the “Board”) is an agency of the State of Florida statutorily responsible, along with DPR, for licensing certified electrical contractors. The Office of the Executive Director of the Board is located at 130 North Monroe Street, Tallahassee, Florida.

6. The business address of Petitioner Harden is Harden Electric, 311 N.E. 8th Street, Homestead, Florida 33030.

7. The business address of Petitioner Cauthen is 959 N.E. 79th Street, Miami, Florida 33138.

8. On March 20, 1981, Rules 21GG-6.01(2), 21GG-6.01(3), 21GG-6.01(4), of the Rules of Electrical Contractors’ Licensing Board, were amended. Rule 21GG-6.01(2) altered the format of the licensing examination so that the formerly entirely open book examination became a part open book, part closed book examination. Rule 21GG-6.01(4) raised the passing grade on the licensing examination from 70 to 75. Rule 21GG-6.01(3) altered the technical format.

9. On December 3, 1981, Susan Tully, counsel to the Electrical Contractors’ Licensing Board, caused a change to be made in Rule 21Gg-6.01(l)(c) of the Rules of the Electrical Contractors’ Licensing Board. The changes was made without formal notice or informal notice to the public and without a vote of the Electrical Contractors’ Licensing Board, although the change was discussed at a Board meeting. None of the procedures set forth in Section 120.54 of the Florida Statutes were followed in amending this Rule. Liz Cloud was the Bureau Chief of the Division of Elections, Bureau of Administrative Code, Department of State. The address of the Bureau is Room 1802, The Capitol Building, Tallahassee, Florida.

10. The change in Rule 21GG-6.01(l)(c) eliminated parts 72 A, B, C and D and indicated that the entire Fire Safety Code (and not just the aforementioned parts) would be a subject of the Electrical Contractors’ Licensing Examinations. In fact, the reason for the alleged “technical change” was that the Board intended and tested examinees in the January and July 1982 Licensing Examinations on materials in parts of the Fire Safety Code in addition to those contained in 72 A, B, C and D of the Fire Safety Code.

11. Petitioner Harden specifically requested to review his January and July 1982 Electrical Contractors’ Licensing Examinations.

12. On March 25, 1982, Petitioner Harden went to Tallahassee, Florida to review his January 1982 Licensing Examination papers, but although he requested an “examination review” he was never provided with his own examination booklet or a copy thereof. During this alleged “examination review” Mr. Harden wrote written objections to the January 1982 examination on the forms provided to him based on the master copy of the examination provided to him.

13. On August 31, 1982, Petitioner Harden went to Tallahassee, Florida to review his July 1982 Licensing Examination papers, but although he requested an “examination review” he was never provided with his own examination booklet or a copy thereof. During this alleged “examination review” Mr. Harden wrote written objections to the July 1982 examination on the forms provided to him based on the master copy of the examination provided to him.

14. Petitioner Cauthen specifically requested an “examination review” with respect to her January and July 1982 Licensing Examination papers.

15. On March 17, 1982, Petitioner Cauthen went to Tallahassee, Florida to review her January 1982 Licensing Examination papers, but although she requested an “examination review” she was never provided with her own examination booklet or a copy thereof. During this alleged “examination review” Ms. Cauthen wrote written objections to the January 1982 examination on the forms provided to her based on the master copy of the examination provided to her.

16. On August 19, 1982 Petitioner Cauthen went to Tallahassee, Florida to review her July 1982 Licensing Examination papers, but although she requested an “examination review” she was never provided with her own examination booklet or a copy thereof. During this alleged “examination review” Ms. Cauthen wrote written objections to the July 1982 examination on the forms provided to her based on the master copy of the examination provided to her.

17. Petitioners Harden and Cauthen sought Board review of their January and July 1982 Electrical Contractors’ Licensing Examinations. Petitioners were advised in July 1983 that no copies of their actual booklets exist; Petitioners were advised in October 1983 that their actual booklets were shredded.

18. At the November 8, 1982, meeting of the Board, Ms. Ida Cameron representing DPR, presented to the Board a package of information (assembled in package form) for the Board to consider during the examination review.

19. With regard to Question Number 71 on the afternoon portion of the July 1982 Licensing Examination, two of the four possible responses, “A” and “C” were credited. Candidates like Petitioner Cauthen who answered “B” received no credit for the question.

20. Petitioners Harden and Cauthen have requested Chapter 120.57 hearings with respect to their January and July 1982 licensing examinations and the review, grading and agency action with respect to same. Although request for production was specifically made for said packages of information with respect to the January 1982 and July 1982 licensing examinations in February, 1983, no packages have been produced to date.

21. DPR destroyed the Petitioners’ examination booklets before the end of the two year period immediately following each of the 1982 examination.

22. That Section 455.217 of the Florida Statutes requires the Board “by rule” to designate areas of competency to be covered by each licensing examination.

23. That Section 455.217 states that the Board shall “by rule specify the general areas of competency to be covered by each examination, the relative weight to be assigned in grading each area tested, and the score necessary to achieve a passing grade.”

24. That the amendment of Rule 21GG-6.01(2) required candidates to commit to memory portions of the electrical code, accounting, law, worker’s compensation rules, federal employer’s tax guide, AIA General Conditions, business practices, legal and insurance requirements.

25. In the July 1982 Electrical Contractors’ Licensing Examination candidates were given one hour more time in which to take the examination than was given during the January 1982 examination.

26. Rule 21GG-6.01(4) was amended in conjunction with Rules 21Gg-6.01(l)(a), (2) and (3) at a board meeting in March, 1981.

27. Rule 21GG-6.01(4) raised the passing score on the January 1982 Electrical Contractors’ Examination and the July 1982 Electrical Contractors’ Examination from 70 to 75.

28. The amendment to Rule 21GG-6.01(4) required that electrical contractors pass the certification examination with a score of 75 percent, whereas all other construction industry licensing board contractors licensed by the Department of Professional Regulation (13 other categories) were merely required to achieve a 70 percent pass score.

29. That Section 455.217 of the Florida Statutes provides that “The board shall make rules providing for re-examination of any applicants who have failed the examination.”

30. That DPR did not and has never provided Petitioners, their attorneys or agents with their own examination booklets for the January and July 1982 Electrical Contractors’ Licensing Examinations. DPR has provided Petitioners only with copies of “master” examinations for their review.

31. That Section 455.217 of the Florida Statutes provides that the Board shall make available an examination review procedure for applicants.

32. That the Respondents produced Notice of Destruction of Examination Booklets and Other Examination Materials dated April 14, 1982, which allegedly evidences destruction of Petitioners’ examination booklets for the January 7, 1982, Examination on April 14, 1982.

33. That the Respondents produced Notice of Destruction of Examination Booklets and Other Examination Materials dated October 15, 1982, which allegedly evidences destruction of Petitioners’ examination booklets for the July 13, 1982, Examination of October 15, 1982.

34. The findings in paragraph . 1 through 33, immediately above, are based directly on the stipulations of the parties, most of which stipulations were memorialized at the beginning of the hearing. The findings in the following paragraphs are based primarily on testimony and exhibits, but some of them are also based in whole or in part on stipulations. In the findings which follow there are certain to be at least some repetitious findings in the course of putting matters into context and making additional findings which are related to some of the stipulated findings. I have tried to avoid all unnecessary repetition, but a certain amount is necessary for clarity and a certain amount is unavoidable due to the sheer size of the task at hand.

Findings on background matters and on matters relating to more than one rule

35. Both of the Petitioners in this rule challenge proceeding are individuals who have applied to the Electrical Contractors’ Licensing Board for licensure as certified electrical contractors. Both of them have been approved to sit for the licensure examination. Petitioner Harden took the Board’s licensure examination on each of the following occasions: July 1981, January 1982, July 1982, and January 1983. The Board has not given him a passing grade on any of those four examinations. Petitioner Cauthen took the Board’s licensure examination on two occasions: January 1982 and July 1982. The Board has not given her a passing grade on either of those two examinations. The grade notifications received by these Petitioners show, inter alia, that Petitioner Cauthen received a grade of 73 on the July 1982 examination. Both of these Petitioners have presently pending formal proceedings under Section 120.57(1), Florida Statutes, in which they are challenging various matters related to the preparation of, administration of, scoring of, and inherent validity of the Board’s January 1982 and July 1982 licensure examinations.

36. The Electrical Contractors’ Licensing Board certification examination is different from a master electrician examination. However, the two examinations are in many ways similar because of the overlap in the nature of the subject matter to be tested on both examinations.

37. The passing score or cut score on the local master electrician examination administered by Dade County is 705. Most of the questions on that examination are about the National Electric Code. Part of the Dade County master electrician exam is closed book.

38. Prior to 1972, persons wishing to engage in electrical contracting in the state of Florida were required to be licensed by the local governments in the areas in which they sought to operate. Since 1972, persons wishing to engage in electrical contracting in the state of Florida must be licensed by a unit of local government or by the Electrical Contractors’ Licensing Board. Persons who are licensed by the Electrical Contractors’ Licensing Board are known as certified electrical contractors. A certified electrical contractor can engage in electrical contracting anywhere in the state of Florida without local licensure. Persons who are licensed by one or more local governments (typically a municipality or a county) are known as registered electrical contractors. Registered electrical contractors are licensed to engage in electrical contracting only in the geographic areas encompassed by the boundaries of the local government entities that issued their local licenses. There is, however, a certain amount of reciprocity from one local government to another.

39. Certification as a state certified electrical contractor does not authorize the electrical contractor to work as an electrician, although a certified electrical contractor can pull permits. A person does not have to be a licensed electrician in order to become a state certified electrical contractor, although many electrical contractors are also licensed as journeyman or master electricians. Candidates for the electrical contractor licensure examination have a great variety in the nature and scope of their background and experience. This variety in background and experience is among the reasons which cause testing for minimum competence as an electrical contractor not to be an exact science.

40. As between certification and registration, certification by the Electrical Contractors’ Licensing Board is not a major factor in the ability to compete for business. A person who has a statewide certification can save some money by avoiding the expense of obtaining local competency cards, but as a general rule a registered electrical contractor can work just about anywhere in the state due to reciprocity. Especially, a person who has passed a Block master electrical examination or a Block local electrical contractor’s examination can work just about anywhere in the state because most counties in Florida accept the Block examination.

41. Registered electrical contractors regularly compete for business with certified electrical contractors. For example, Petitioner Cauthen’s company is presently licensed in Monroe, Dade, and Broward counties, where it does a considerable amount of business. Petitioner Cauthen’s company bids on lots of contracts and submits bids in competition with both registered and certified electrical contractors. Thus, the company is already in competition with certified electrical contractors. Similarly, Board Member Isaacs, whose certified company does 90% of its work in Duval County, has regular competition from registered electrical contractors. Board Member Isaacs competes with approximately 250 electrical contractors. Of that 250, approximately 175 are registered and the others are certified. A much bigger factor in competition than the registration versus certification issue is the cost of doing business in more than one area or the cost of doing business at a location that is distant from one’s primary base of operations. Only about fifty electrical contracting firms regularly compete for business over the entire state of Florida. Accordingly, a restriction on the number of persons licensed by the Board as certified electrical contractors would have minimal, if any, limitation on the competition faced by those members of the Board who are certified electrical contractors.

42. The Electrical Contractors’ Licensing Board administered its own certification licensure examination from 1972 through 1980. During that period of time the Board administered the examination a total of twenty-two times. During those twenty-two sessions of Board administered examinations, a total of 824 candidates sat for the examination, of which 392 were successful.

43. Although the percentage of candidates who were successful on a particular Board administered examination ranged from a low of 20.7% to a high of 78.7%, the average passing rate of all candidates on all twenty-two of the Board administered examinations was 47.575. (By way of comparison, it is interesting to note that the percentage of candidates who are successful on the Block master electrician examination is between 30% and 35% of those who take the examination.) Further, on fifteen of the twenty-two occasions on which the board administered the examination, the percentage of candidates who were successful was 50% or less.

44. In 1979, the regulation of professions and occupations was reorganized. The Department of Professional Regulation (the Department) was created an an umbrella agency over numerous boards, including the Electrical Contractors’ Licensing Board and the Construction Industry Licensing Board. The responsibility for examination applicants, which in the case of the Electrical Contractors’ Licensing Board had been handled by the Board members themselves, was taken over by the Office of Examination Services (O.E.S.).

45. Toward the end of 1980 and the beginning of 1981, O.E.S. did not like what the Board was doing and the Board did not like what O.E.S. was doing. There was a general lack of trust between the two entities, there was poor communication between the two entities, and the relationship between them at that time might best be described as estranged. Although both entities made efforts at cooperation with the other, such efforts were not always effective or well received. As a result of the poor communication between the two entities, certain personal concerns over Board actions by individuals within the O.E.S. were never communicated to the Board (and therefore could not be answered.)

46. The first examination administered by the O.E.S. was in January 1981. Of the fifty candidates who took the January 1981 examination, forty-six were successful. This was a passing rate of 925. In view of the Board’s experience when it was administering its own examination, the Board was understandably surprised and concerned when the percentage of candidates passing the first O.E.S. administered examination was almost twice the average passing percentage on the Board administered examinations.

47. The Board had no evidence that the overall qualifications or capabilities of the applicants examined by O.E.S. at the January 1981 examination were any higher than the capabilities of those examined previously. To the contrary, a review of their qualifications based on the information in their applications indicated that their qualifications were substantially the same as those of previous candidates tested by the Board. Also, the admission standards for the January 1981 examination were the same as those for prior examinations.

48. The Board concerns about the O.E.S. administered January 1981 examination included the following matters, among others, memorialized at a Board meeting discussing the examinations:

The candidates had been allowed to take any reference material desired into the examination. Previously the Board had allowed only reference books listed as part of the application form.

There was a very high pass rate in comparison to all past examinations.

There appeared to be too few calculation questions.

The questions had been placed on the paper in sequence with the reference book materials.

There was a possibility that notes were taken into the examination which would have given advantage to the candidates.

The [Board’s] Examination Committee had been under the assumption that the entire examination would be presented for review of the questions. Instead the only questions presented were ones questionable under Department criteria for measuring competency.

It was explained, from past experience, that persons conducting the seminars for exams had obtained the entire exam content from persons taking the examination in the past. It had been done with a camera which was taken into the examination. The Board, when administering the examination, had been very strict on what was used in performing the examination. In the last exam the security was very poor.

49. The Board felt there were so few calculation questions on the O.E.S. administered examination in January of 1981 that the examination was not a proper or sufficient examination. This was due at least in part to the fact that O.E.S. had difficulty covering all areas of the examination with the desired number of questions because the O.E.S. bank of questions was very limited at that time.

50. Previous Board examinations were open book with certain reference material allowed. When the Office of Examination Services took over the administration of the January 1981 exams, it sent to the candidates a form letter adapted from the Construction Industry Licensing Board. This form letter instructed the candidates that they would be able to bring into the examination any notes and other materials desired. This was contrary to the past practice of the Board and contrary to the Board’s wishes.

51. The Board was very frustrated and concerned about the circumstances which led up to the January 1981 examination because the O.E.S. had refused to allow the Board to have any input into the examination and the Board thought the O.E.S. had come up with an apprentice level examination. In the Board’s opinion, the January 1981 examination prepared by O.E.S. clearly tested at a level less than that of minimum competency. And although O.E.S. did not agree with the Board that the January examination was invalid, O.E.S. did agree that the examination was a lot easier than they had thought it would be.

52. The Electrical Contractors’ Licensing Board did not conduct any formal studies or formal analysis prior to making the 1981 amendments to Rule 21GG-6.01 changing the format of the examination and the cut score. However, the Board’s Rules Committee did a great deal of work on the matter and the matter was discussed extensively at Board meetings. In this regard it is important to keep in mind that the Board members (with the exception of the two lay members) were all experienced practicing electrical contractors who were familiar with the requirements of day-to-day electrical contracting.

53. The Board’s decision to change part of the examination to closed book was due to a concern that people with no electrical knowledge could become licensed if the examination was all open book and had a low percentage of technical questions. The Board felt that more than half of the examination should be technical questions. The closed book National Electrical Code questions were for the purpose of testing what a person with experience in the field of electrical contracting should know about everyday matters.

54. The primary, if not the sole, motivation for the Board’s 1981 amendments to Rule 21Gg-6.01 was the Board’s concern about the quality of the O.E.S. administered examination and the Board’s feeling that, due to its estranged relationship with O.E.S., the only vehicle through which it could effectively influence the quality of future examinations was through rule-making. Those amendments were not motivated by any desire on the part of the Board to restrict competition in the field of electrical contracting.

55. During the past few years the job requirements of an electrical contractor have remained substantially the same.

56. Block and Associates (hereinafter “Block”) is a company based in Gainesville, Florida, that writes licensure examinations for contractors in various trades. Block writes such examinations for cities, counties, and states. Block has prepared electrical contracting licensure examinations for the states of Florida, Georgia, and Oklahoma, as well as for St. Johns County, Florida. Block prepares licensure examinations for just about anything that involves electricity, including such things as electrical contracting, electrical journeyman, electrical master, low voltage, elevator, burglar alarm and fire alarm. Block prepares master electrician examinations for over one hundred governmental entities. The Block electrical master exams and the Block electrical contractor exams use some of the same questions.

57. Subsequent to the 1982 examinations which for the gravamen of these Petitioners’ related proceedings under Section 120.57, Florida Statutes, the ECLB has contracted with Block and Associates to prepare the Board’s certification licensure examination. The Board has never said anything to Block about wanting to achieve any particular passing rate or wanting to have any particular percentage of the candidates taking the examination achieve a passing score. The Board has never interfered with Block’s autonomy in the preparation of the electrical contractor exam.

58. All of Block’s electrical electrical contractor examinations include business questions. It is important to test for business skills because an electrical contractor who gets into business trouble may start cutting corners and cutting corners is a hazard to the public. Some of Block’s electrical contractor examinations also have specific sections on safety.

59. Block always does a post-administration statistical analysis of the examination questions it uses. The reason for this is to find out if anything is wrong with the questions; in other words, to determine whether the questions are valid. The computer program used by Block to check the validity of its tests was prepared for Block by a professor in the College of Education at the University of Florida. Block uses a computer to conduct a validity analysis of each test it administers.

Findings regarding Rule 21GG-6.01(l)(c) of the Electrical Contractors Licensing Board

60. Rule 21GG-6.01(l)(c) of the Electrical Contractors’ Licensing Board reads as follows (with the portion challenged by the Petitioners underscored):

(1) The areas of competency to be covered by the certification examination shall be as follows:

(a) * * *

(b) * * *

(c) Safety, which shall include but not be

limited to, OSHA regulations, life safety codes, and the Fire Safety Code (NFPA).

61. Prior to the December 1981 amendment which is challenged here, Rule 21GG-6.01(l)(c) read as follows:

(1) The areas of competency to be covered by the certification examination shall be as follows:

(a) ***

(b) * * *

(c) Safety, which shall include but not be limited to, OSHA regulations, life safety codes, and the fire safety code (NFPA 72A, B, C and D).

62. The Board’s purpose in amending the rule to delete the reference to “72A, B, C and D” was to make it clear to candidates for examination that the Board intended to include in the safety portion of its examinations questions from portions of Volume Seven of the NFPA other than Chapters 72A, B, C, and D of that volume. Volume Seven of the NFPA contains quite a bit of material in addition to the material included in Chapters 72A, B, C, and D of that volume. In view of the modifying phrase “shall include but not to be limited to,” the amendment to delete the reference to Chapters 72A, B, C, and D was not a necessary prerequisite to the use of examination questions based on other portions of Volume Seven of the NFPA. The Board could ask the same examination questions before and after the amendment.

63. Several weeks before each examination, the Department of Professional Regulation sent all candidates for examination, including these Petitioners, a notice to appear for the examination. The notice to appear included a list of reference books on which the examination was to be based. That list of reference books contained a specific reference to Volume Seven of the NFPA. All sixteen volumes which comprise the entire NFPA were not listed as reference books.

64. Neither of these Petitioners demonstrated that he or she was genuinely confused or misdirected by the amendment to Rule 21GG-6.01(l)(c). Neither Petitioner contended he or she had studied all sixteen volumes of the NFPA. Instead, both Petitioners studied from the books on the reference list.

65. Neither Petitioner has an application for the examination pending; neither claims an intention to take the examination again.

Findings regarding Rule 21GG-6.01 (1), (2), (3), and (4) of the Electrical Contractors’ Licensing Board

66. At the time of the examinations which are the subject of these Petitioners’ related cases under Section 120.57, Florida Statutes, Rule 21GG-6.01(1), (2), (3), and (4) of the Electrical Contractors’ Licensing Board read as follows, in pertinent part:

(1) The areas of competency to be covered by the certification examination shall be as follows:

(a) Technical, which shall include, but not be limited to, electrical calculations, estimating, designs, and electrical schematics; and which shall be divided into two (2) separate areas, one containing electrical calculations, the other containing Code-related questions not requiring calculations.

(b) General business, which shall include but not be limited to, problems relating to accounting, law, insurance, workers’ compensation and social security.

(c) * * *

(2) The format of the examination shall be as follows:

(a) The portions of the examinations containing the electrical calculations and safety questions shall be open book. The applicant is responsible for bringing and may use during these portions the applicable code books, reference materials as approved by the Board, and noiseless mechanical or non-mechanical instruments he wishes to use.

(b) The portions of the examination containing the non-calculation technical questions and all business questions shall be closed-book. The application will not be permitted to use the reference materials listed in 2(a) or any other reference materials.

(c) Security measures as set forth by the Department shall be followed during both portions of the examination.

(3) The relative grading weight to be assigned to each area of competency shall be approximately as follows:

(a) Technical — 60%

1. Electrical Calculations — 40% of the Test

2. Code-related questions — 20% of the Test

(b) General business — 30%

(c) Safety — 10%

(4) An applicant shall be required to achieve a score of a general average of not less than seventy-five percent (75%) in order to pass the examination and be certified for licensure. There shall not be a practical or clinical examination.

67. The language quoted above is the language of the relevant portions of the rule as it read after the March 1981 amendments to the rule. The principal amendments in March of 1981 may be summarized as follows: (1) changes which divided technical questions on the examination into two separate areas, (2) changes which made part of the examination open book and part closed book [it was previously all open book], and (3) a change in the minimum passing score from 70% to 75%. These Petitioners challenge the validity of all three of the principal amendments.

68. As originally proposed, the March 1981 amendments to subsection (2)(a) of Rule 21GG-6.01 would have permitted a candidate to bring “any notes” to the open book portion of the examination. During the adoption process the words “and any notes” were deleted from the amendment by the Board. The words “and any notes” did not appear in Rule 21GG-6.01 prior to the 1981 amendments. Nor was it the practice of the Board prior to 1981 to permit candidates for examination to take any notes into the examination when it was administered by the Board.

Findings regarding examination content and relative grading weights

69. In November of 1983 the Electrical Contractors’ Licensing Board again amended Rule 21GG-6.01(3) to change the relative grading weights of the areas of competency to be tested. After November of 1983 the relative grading weights of the areas of competency were as follows: Technical increased from 60% to 65%; electrical calculations increased from 40% to 45%; code-related questions remained 20% of the examination; general business was decreased from 30% to 25% of the examination; and safety remained 105 of the examination.

70. The licensing examination given by the Board has historically contained materials testing knowledge of electrical work and also materials testing the business qualifications of the applicants. Since the inception of the Board’s licensing exam, the electrical work portions have contained electrical calculations.

71. Throughout the time the Board has been administering licensure examinations for certification, the percentages of the examination devoted to particular subject areas have been based upon the Board’s belief as to the appropriate levels of knowledge needed by a certified electrical contractor. The weighting of the examination was equitable when it was weighted 60% technical, 30% business, and 10% safety, and it is still equitable now that it is weighted 65% technical, 25% business, and 10% safety. The Board changed to the current percentages at the suggestion of Block and Associates. The Board’s examination has had similar ratios for a long time, at least as far back as 1971.

72. The proportions of the examination devoted to each particular subject matter did not change due to the March 1981 amendments. With the exception of the January 1981 examination put together by O.E.S., electrical calculations had historically been 40% of the test. Since the Board felt this percentage should be maintained, and since it was necessary to divide those calculations from the other portion of the technical section (since one would be tested in the closed book portion and the other in the open book portion), the division of the technical part of the exam was set out in the rule.

73. On an electrical contractor licensure examination it is appropriate to include in the area of “technical questions, questions dealing with the following subjects: electrical calculations, parts of estimating, and designs and electrical schematics. In the “business portion of an electrical contractor examination, it is appropriate to include questions dealing with the following: accounting, worker compensation and social security.

Findings regarding closed book versus open book examination

74. Local licensing examinations had, in 1981, included a closed book portion. Local licensing examinations and state licensing examinations given in other states still include such a closed book portion. The basic Block examination has always included a closed book portion.

75. Closed book examinations are not all that unusual in occupational and professional licensure testing. The CPA and Nursing examinations are closed book. Also, virtually all of the electrical examinations administered by Block and Associates include a closed book portion. The advantage of a closed book examination technique is that it is easier to identify the candidates who have had practical experience in the subject matter being tested. A closed book examination is a better test for field experience and for general knowledge of the subject matter being tested.

76. Certain questions known as “anchor” questions were given both in the open book examination before the format change and in the closed book section after the format changed. The relative performance by candidates on the same questions in the two formats shows the format change not to have prejudiced the candidates.

77. Further, an examination analysis of the results of the closed book portions of the ECLB examinations reveals that, based on candidate performance, the closed book portion of the examinations was easier than the open book portion. This also shows that the format change did not prejudice the candidates.

Findings regarding “cut scores”

78. From 1972 until the rule changes in 1981, the minimum passing score, or “cut score,” set by the Board for the certification licensure examination was 705.

79. Cut scores of both 70% and 75% are very common cut scores for licensure examinations in the fields of electrical contracting, journeyman electrician, and master electrician, as well as licensure examinations in other occupations and professions. Many local licensure examinations have a 75% cut score.

80. Standing alone, the cut score to be applied to a future examination is somewhat lacking in specific significance. A cut score must be considered in relation to the content of the examination to which it is to be applied.

81. All of the testing experts who testified at the hearing agreed that setting an examination cut score is an inherently arbitrary decision. In the final analysis it is a judgment call that cannot be made with scientific exactitude and is unavoidably arbitrary to a certain extent. As explained by one of the Petitioners’ testing experts, even though statistical methodology is available for the determination of cut scores, to a large extend the cut score is a random decision, the validity of which can be assessed only in relationship to the specific testing instrument to which it is applied.

82. For example, during the course of Block’s preparation of its first licensure examination in 1958 for the City of Ormond Beach, when it came time to decide what cut score to use, the five experts who were preparing the examination talked it over and decided that 15% was the equivalent of a good “C” when compared to academic grades, and they decided to use 75% as their cut score. It is not possible to be much more scientific or precise than that in setting cut scores, especially when one has a group of candidates of varied experience and background taking the examination.

83. A skilled test preparer can construct an examination of equal validity within a range of cut scores from 655 to 80% if the test preparer knows at the time of constructing the examination what the cut score will be. With any cut score in that range, a skilled test preparer can raise or lower the percentage of candidates who will actually pass the examination by careful selection of the questions to be used on the examination. By using easy questions, the person preparing the examination can insure that more people will pass a test with a high cut score. Similarly, by using hard questions, the person preparing the examination can insure that fewer people will pass an examination with a low cut score.

84. On some standardized licensure examinations the cut score is determined after the examination results are examined. In other words, the examination results are “curved” based on the performance of the group of candidates who take a particular sessions of the examination and the actual cut score is determined in relationship to the raw scores achieved by each particular group of candidates. This flexible method of setting cut scores seems to work well on standardized examinations which are administered to large numbers of candidates, but it is questionable whether it would work very well with small groups of candidates. The Board’s examinations have historically been administered to small groups of candidates.

Findings regarding the effects of changes on testing instruments

85. In order for a licensure examination to accomplish the purpose for which it is administered, it should, of course, be valid and reliable.

86. Several of the Petitioners’ expert witnesses described what might be characterized as the best way to make changes to an examination or as the ideal way to make changes to an examination. Nevertheless, the manner in which the Electrical Contractors’ Licensing Board went about making the changes to the examination which were incorporated in the March 1981 rule changes was a reasonable and logical response to the circumstances faced by the Board. It was perhaps not the best way to have resolved matters, but it was a reasonable response. The Board’s actions in this regard was not arbitrary or capricious. Even the Petitioners’ most persuasive expert in the field of testing conceded that if the content experts (in this case the Board members) felt that given a specific content domain and given a specific item bank the testing instrument measured less than minimum competency, a logical response would be to raise the cut score. Another logical reaction to that situation would be to change a portion of the examination from open book to closed book. Both actions would be reasonable and logical responses to that type of problem.

87. Where there has been a modification in the format of a testing instrument, an accepted methodology for analysis of the effects of format modification is the use of repetitive questions from prior testing instruments. These repetitive questions are known as “anchor” questions. If a statistically significant deviation in performance on the anchor questions is demonstrated, the format modification requires further analysis through other statistical tools. However, if an analysis of the anchor questions shows inconclusive or mixed results on the anchor questions (such as equal or improved candidate performance on those questions), it may then be concluded that the format change is not significant to candidate performance and may be an appropriate format change relative to the profession or occupation being examined.

88. Content validity is the most practical measure of the validity of a licensure examination. The content validity of an examination depends on the extent to which the questions on an examination may be accepted as representative of performance within a specifically defined content domain of which the examination instrument is a sample. Content validation would be the best way to determine the validity of the examination which were administered in 1982 pursuant to the March 1981 rule amendments. One method of establishing content validity is to have a group of experts identify the domains of knowledge that are applicable to the profession or occupation being tested. The establishment of content validity relies heavily on the opinions and judgments of people who are experts in the relevant profession or occupation. Expert judgment plays an intergral part in developing the definition of the content domain to be tested.

89. To the extent that the content domain of Board examinations was established by the March 1981 amendments to Rule 21GG06.01(3), the content domain was established on the basis of the expert judgment of persons who were experts in the field of electrical contracting, namely, the members of the Electrical Contractors’ Licensing Board. The relative grading weights established in the 1981 amendments to Rule 21GG-6.01(3) have at least a possible correlation to electrical contractor competence. In fact, the persuasive expert testimony establishes that those relative grading weights did bear a reasonable relationship to electrical contractor competence even though there has been no recent change in the nature of the responsibilities of electrical contractors. In this regard it must be remembered that testing is not an exact science and no test for minimum competency can ever be expected to be an exact model of actual work experience. The best that can be expected is a reasonable model.

90. There are various accepted methodologies for the creation of standardized examinations, adherence to which tends to enhance the likelihood that the examination instrument finally produced will be valid and reliable. However, adherence to those accepted procedures is not a necessary prerequisite to creation of a valid and reliable examination. This is particularly true when one is going to be testing small groups of people with varied backgrounds instead of the massive groups of people with similar backgrounds for whom standardized tests are more typically designed. In the final analysis the only practical and reliable measure of the validity of an examination is by statistical analysis of the examination after it has been administered.

91. Given the nature of the circumstances faced by the Board at the time of the March 1981 amendments to the examination format and cut score, given the nature of the pool of candidates to be examined, given the nature of the changes contemplated by the Board, and given the very nature of the process of testing for minimum competency— which involves perhaps as much art as it does science — there is no study or data which would have been particularly useful to the Board in helping to determine exactly what the effect of their changes would be. Such effects can only be determined or measured with any degree of accuracy after the administration of an examination that incorporated the changes. Following the administration of such an examination, it is possible to perform a statistical analysis of all questions used on the examination and to eliminate or give credit for any questions which are shown by statistical analysis to be invalid or unreliable. This is precisely the process that is used by Block in the validation of their examinations and is an accépted testing procedure.

92. It would have served no useful purpose to have conducted a trial run of an examination using licensed certified electrical contractors as a test group for the new examination format. First, it would be virtually impossible to try to put together an accurate cross-section of certified electrical contractors to use as a test group. Second, one would expect them all to pass the examination, so when they did so nothing of value would have been learned. Finally, the administration of such a trial run would risk the possibility of compromising examination question security.

93. One aspect of accepted methodology for the preparation of standardized examinations is the definition of the content domain of the examination, i.e., a determination of what knowledge is essential to demonstrate that the candidates for licensure are minimally competent. This aspect of examination preparation is often accomplished by performing a formal job analysis, which is, in essence, a study of all of the usual tasks performed by a person engaged in the occupation or profession to be tested, including an evaluation of the relative importance of each of those tasks to minimum competence. The content domain can also be defined on the basis of the judgment of a group of experts in the occupation or profession to be tested.

94. With the exception of the lay members who were added in recent years, all of the members of the Electrica