HRH Development Tool (Update: September 1, 2006)

I. Status Overview
  1. A draft version of the HRH Policy Development Tool is ready. An explanation of the organization of the questions and flow of control through the survey is provided in Section II below.

  2. A draft version of the evaluation function is ready. An explanation of how the evaluation function operates is provided in Section III below.

  3. A proposed method of evaluating the Situational Analysis questions and incorporating the evaluation in the recommendation ratio is presented in Section IV below.


II. Survey
  1. To access the survey, click on the link below:

    http://camss.clemson.edu/HRH_Pilot_Test

  2. Visual aids

    1. To help understand the organization of the survey, we provide a graphical view of a major portion of the survey questions. This image will help to understand the flow of control through the survey. The image will open in a new window and is quite large; it will be necessary to scroll left and right, up and down to view different parts of the survey. To see the image, please click on the link below:

      graphical view of survey.

    2. We also provide a question block table showing which questions are contained in different question blocks. This table will also help to understand the survey organization. Two copies of this table are available:

      an outline view    and     a detailed spreadsheet view.

  3. Notes: Please start the survey in one window, open the graph in another window, and open the outline view of the question block table in a third window.

    1. The question block table shows that the survey now has 91 question blocks and over 680 questions.

    2. The graph illustrates the flow of control through the survey. The survey starts with QB1 (Question Block 1). The table shows that QB1 contains only one question (#2) which allows the client to choose which module to work on. Clicking through the survey and referring to the graph and table, we see that QB2 (Capacity to develop and implement policy) and QB3 (Policy environment) are the two Situational Analysis modules. QB5 is the Entry module, QB6 is the Exit module, and QB7 is the beginning of the Existing Workforce module.

    3. Note that QB4 contains only one question (#25). This is the final question that provides the client with the option to return to QB1 (by responding with a "Yes") in order to work on another module. If the client answers "No" to question #25, control moves to the END block and terminates.

    4. Note that blocks QB2, QB3, QB5, and QB6 all have a sequential flow of control. The client sees all of the questions contained in these blocks in predetermined order. When the questions have been presented, control proceeds to the penultimate block QB4. Contrast this with the skip patterns illustrated by blocks QB7-QB12.

    5. QB7 is the head of a "family" of question blocks which also include blocks QB8 through QB12. Certain responses to questions in QB7 (spefically "Disagree" or "Strongly disagree") will trigger transitions to follow-up questions which attempt to extract from the client additional detail regarding the concerns of the client discovered in QB7. After the follow-up questions have been answered, control proceeds to the next set of questions found in QB13. Note that for simplicity, only the skip patterns for the QB7 family are shown in this early version of the graph. Similar skip patterns now exist for each family of question blocks headed by blocks Q13 through Q91.

      This demonstrates a significant advantage of online surveys over paper-and-pencil surveys: complex skip patterns such as illustrated by QB7-QB12 would be very difficult and impractical to implement using a paper survey.



III. Evaluation Function
  1. To access the evaluation function, click on the "EVAL" link found in the lower lefthand corner of any question page of the survey. The window that pops up shows the calculations that the evaluation function has performed on the client's responses so far. After viewing the contents, please close evaluation function window before continuing with the survey. This ensures that when the "EVAL" link is clicked again, the new window will contain the most recent client responses available.

  2. Each client will have a unique ID number which is displayed at the top of an evaluation window. The date and time of the evaluation report will be stamped below the client window. The client's responses to the survey given in (question_id, answer_id) format is listed next. This is purely for debugging purposes and will disappear in the final version.

  3. The questions that contribute to a policy for a certain group (e.g., Increase production capacity for Nurses) are listed below the group name. In this example, questions 26, 31, 46, etc. contribute to this policy-group pair. As questions are answered by the client, they are highlighted in red. If a question is skipped by a client or if the client answers "Don't Know", the question is not highlighted.

  4. The recommendation ratio is calculated thus:

    ratio = (sum of weights of questions answered) / (number of questions answered so far)

    where weights are between -2 (Strongly Disagree) and 2 (Strongly Agree). As relevant questions are answered, the weights which are used to calculate the ratio are listed below the question numbers so that we can verify that the calculations are being carried out as intended.

  5. The confidence measure is calculated thus:

    confidence = (number of questions answered) / (total number of questions in set)

    where a "Don't Know" is considered to be the same as an unanswered question.

  6. The weights used for this sample are described in:

    Policy Development Logic

    and are used as examples only. These weights should be replaced by weights developed by WHO HRH staff.



IV. (Proposed) Evaluation of Situational Analysis Questions
  1. More detail later.