- 1 Project Proposal
- 2 User and Task Analysis
- 3 Design Sketches
- 4 Paper Prototype
- 5 Computer Prototype
- 6 Experimental Design
- 7 Implementation
- 8 Final Implementation File & Instructions on Use:
- 9 User Testing and Analysis
Pediatric Diagnosis Manager (PDM)
Users and tasks
Our users are female, minimally trained, medical staff around 16-20 years of age that work to diagnose children for illnesses in developing 3rd world countries.
The tasks we wish to support with this web application are
- Digitization of the IMCI booklet, allowing for diagnosis lookup based on symptoms
- Updateable information, in the case of new symptoms, diagnoses, and treatments
- Calculator widget, for certain conversions such as Celsius to Fahrenheit, pounds to kilograms, and such
- Integration with patient records, so certain cases can be handled without going through the same standardized processes
Existing solutions and their limitations
- At the moment, our users diagnose patients by looking up a paper manual of flow-charts by hand. The user finds the desired page, and then walks through, step-by-step, a flowchart of instructions. In the event of other queries, the user would then flip back and forth between the table of contents and his next desired page, and then read the instructions and follow through with what is written.
- Our web application would be advantageous in terms of efficiency. Flipping through a manual by hand is a very poor method of indexing, as it relies on sight and hand movements. For a computer, the time it takes to look up an appropriate page, given a query, can be near instantaneous. Also, depending on results or queries, the application can be more decisive with what information to display, while the pages have to be cluttered with varying degrees of information, making it difficult to read. Using automated searching and dynamic displays in a web application can save a lot of time and stress over the user's current methods.
Throughout our meeting, we came up with many ideas for a health-related web application, such as
- A diagnosis assistant, where symptoms would be plugged in and a suitable diagnosis and treatment would be outputted.
- A patient records keeper, where a patient's medical history would be stored and displayed in an efficient manner.
- A workout assistant, which would help you get a total body workout by recommending and displaying certain stretches or exercises that work on parts of your bodies that you're not focusing on.
- An encyclopedia of medicine, where certain medicine can be looked up by patients for their own information.
- A calorie tracker, where a user can keep track of his caloric intake to manage a healthy lifestyle.
- A fitness social networking program, where people can log on to see events like marathons they might be interesting in nearby, or groups nearby that they could join
- A rare disease tracker, where you could view your family history and possible genetic problems and such
User and Task Analysis
Data Gathering Techniques
- Interview - for an interview, we'd arrange a meeting with Prabu Selvam and have him act as a user representative. Questions asked would numerous, but a few would be "how literate is the typical user; are they able to interpret numbers and symbols?", "how much experience do our users have with computer programs, or technology in general?", "what kind of machines are available to the user base that we could program for?", and "around what age are the users generally clustered around?"
- Researching Similar Products - for research, we'd probably examine the current solution, a PDA-based tool made by D-tree. In examination, we'd test it against the principles of user design as learned in this class, and get a feel for what is needed in general (what they did right), and how we can improve upon that.
- Direct Observation - for direct observation, it'd be best if we could arrange a trip to one of the clinics in a targeted area for distribution of this program, such as Bangladesh. There, we'd use the staff as a sample group to try to log complaints from the user base about the current solution, get demographics on users which can be statically stretched over the user population, and really get a direct feel for how they approach their job: their mood, their tasks, etc.
- Subject - Prabu Selvam
- Type - User Representative
We, and the rest of the health teams, met with Prabu Selvam. He gave out a sheet detailing the user base and tasks, as well as answers to questions he felt would be asked. Additionally, he answered questions we posed (as stated above).
- User Class 1 - Trainee
- Around 18 years old
- No to minimal experience with electronic user interfaces
- Low literacy level; reads slowly, can interpret numbers
- New to whole experience, can be intimidated with sheer amount of information
- Sunita is scared, and sometimes wonders how she got into all of this in the first place, but she takes her duty very seriously. So seriously, in fact, that she gets a little too nervous when there are so many steps and instructions to check. Sometimes she makes a wrong decision and wishes she could go back, sometimes she just gets too stressed on vague symptoms. If things just weren't so cluttered and verbose she could get straight to work, and maybe gain some confidence.
- User Class 2 - Doctor
- Around 20 years old
- Has significantly more experience than a trainee
- Could be a trainee with experience, or a supervisor from outside
- Anika has been around long enough to see patients with a calm attitude. She's already memorized several symptom-diagnosis pairs that keep coming up in her check-ups; enough that she can 'eyeball' some symptoms. With this, she'd like it if there was some way to skip over the unnecessary '20 questions' game of symptom matching and get instructions for treatment. She's got a lot of patients to get to, after all.
- User Class 3 - Technician
- Around 18 years old
- Minimal to average experience with computers
- Has to maintain upgrades to the diagnosis interface
- Rupa may not use the diagnosis tool directly, but she knows that things change over time: new symptoms arise, new diseases, and so on. She knows that we need the system needs to be kept up-to-date, but she doesn't have the skill to say, reprogram the whole thing when something changes. If the tool was easy to edit or updates were automated in some way, she'd have a much easier time.
- Other Stakeholders - Prabu Selvam, D-Tree, other NGOs
- Task 1
- Goal - Diagnose a patient
- Does the user know the patient's ID, or is he new?
- Does the patient have a previous history?
- Does the user already know the diagnosis or not?
- Log in either by entering the patient's ID or by logging in as a guest
- If the diagnosis is already know, skip to the 'possible diagnoses' page
- If he has a history and it's relevant, look it up by hitting the appropriate button
- Select a protocol (Child, infant, high HIV area, etc.)
- Select a symptom group (fever, breathing trouble, dehydration...)
- Answer all the symptom questions by hitting the Yes/No radio buttons (is his temperature above 100, is his breathing about 45 per sec, etc.)
- Select a possible diagnosis (the most likely will be highlighted)
- Follow the treatment steps
- Complexity - medium
- Frequency - Very often
- Potential Errors
- Wrong diagnosis is selected
- Incorrect measurements leading to wrong answers to symptom questions
- Cannot remember patient ID
- Skips to diagnosis on incorrect judgment
- Patient lies about some information
- Task 2
- Goal - Look up a patient's record of vaccinations or diagnosis history
- Does the user know the patient's ID?
- Enter the patient's ID and log in
- Select vaccination history or diagnosis history
- Look up appropriate information
- Complexity - low
- Frequency - Often
- Potential Errors
- Patient's ID is forgotten
- Wrong ID is inputted
- Database is tampered with
- Task 3
- Goal - Update the program with new data
- Does the user know how to edit?
- Does the user know whether the data requires an overhaul or just an update?
- Login as an admin
- Edit the appropriate database, link new symptoms, diagnoses and treatments
- Complexity - high
- Frequency - once a month, maybe less
- Potential Errors
- Admin password is forgotten
- Faulty updates that damage the data, like an accidental erase
- Routine Diagnosis
- Sunita opens the door and meets with her first patient of the day(FREQUENCY: very often). After checking general danger signs and seeing that he is in stable condition, she asks if it is his first time here(PRECONDITION). He says it isn't, so Sunita clicks on Existing Patient and, upon asking for his name, searches for him(SUBTASK). After finding the patient, she sees that he has been sick with diarrhea before and asks if this is a followup(PRECONDITION). The patient states that he feels hot. Sunita clicks on new diagnosis instead of followup, picks the symptom group fever, and gets out a thermometer to check if this is valid(SUBASK). Unfortunately, after letting it sit under the patient's tongue for a few minutes, she sees that the thermometer is degrees Fahrenheit instead of Celsius(ERROR). Fortunately, the page either lists the temperature to check for fever in both degrees or has a conversion calculator in the toolbar(ERROR RECOVERY). After converting, she sees that indeed, the patient has a temperature of 37.8 degrees. Sunita goes down the list of questions, asking the patient how long he's been feeling badly, and feeling if he has a stiff neck or runny nose and if he lives in a area where malaria is a high risk(SUBTASK). Once the questions are all answered, the interface states that it is a normal fever and, after confirming, lists the steps for treatment. After administering a dose of paracetamol, she advises the mother to visit again in 2 days if the fever doesn't go away. Feeling relieved that it's over, Sunita clicks the home button to reset the interface, instead of clicking OK(ERROR). Luckily, the interface automatically logs diagnoses(ERROR RECOVERY), so she breathes a sigh of relief for the small reprieve to the beginning of another stressful day.
- Danger Diagnosis
- Anika rushes to the room and sees the patient is in a bad way: convulsions and difficulties breathing(PRECONDITION). She knows that it most definitely is pneumonia, but she can't recall the antibiotic that's needed for treatment(ERROR). The emergency button, clearly labeled to be clicked when the patient is showing danger signs, is immediately clicked by Anika(ERROR RECOVERY). The page now lists all possible diagnoses, and Anika clicks pneumonia(SUBTASK). As if in warning, the page doesn't accept it readily and lists pneumonia's related symptoms(ERROR). Seeing that the patient has somewhat stabilized, Anika decides to err on the side of caution and tests how fast he is breathing(ERROR RECOVERY). Instead of scrabbling for a stopwatch(ERROR), Anika uses the toolbars stopwatch to help count the number of breaths in a minute(ERROR REVOERY). Seeing that it is about 48, Anika feels confident in her answer and clicks to accept the diagnosis of pneumonia(SUBTASK). Upon seeing the treatment page, Anika eyeballs Co-trimoxazole: that was the name of the antibiotic. After administering a dose of it, Anika immediately refers the mother and patient to a hospital for further treatment.
- History Update
- Because of a dangerous situation, Anika had to quickly acquire a diagnosis without inputting all of the patient's information(PRECONDITION). Now that the patient has been stabilized, she needs to input all of the information so the records can be kept on track. She asks the mother if her child has been to this clinic before: she answers no(SUBTASK). Anika then selects New Patient and fills out all the forms with answers supplied from the mother (name, date of birth, address and the like) (SUBTASK). Being a bit too relaxed after a strenuous situation, she clicks next without filling out all the forms(ERROR). Noticing her error, she hits the ever-present back button after hitting her forehead and fills out the rest of the form fields(ERROR RECOVERY). She then selects the History option instead of the Diagnosis option, where she decides to add a new event that corresponds to today's diagnosis(SUBTASK). She also takes the opportunity to ask the mother about vaccinations(PRECONDITION), and fills out the rest of the patient's information. With this new information, follow-up visits will be a breeze and one less headache.
Preliminary Design Sketches
- Conceptual Storyboard
- UI Storyboard
- Selection of Regular, Administrative or Emergency Protocol
- Tested during the login phase, where hopefully the user will instinctively know what to pick, or information will be supplied on the page to help facilitate the appropriate decision
- Determining whether a patient is new or existing
- Tested during the select patient phase, where a "search" option will be present so, in case you don't know whether you patient is new or has already been, you can search and eliminate risk
- Selecting appropriate symptoms/diagnoses/treatments
- Tested during the diagnosis phase, information about the symptoms/diagnoses/treatments will always be present to help facilitate the appropriate decisions, as well as a back button to reduce errors
- Maintaining the patient information in the database
- Tested during the report phase, generation of reports will be, for the most part, automatic, so hopefully a report will never be lost or never created
When pilot users engaged with our interface, the facilitator presented himself as a "dummy" patient who would answer the medical questions about his symptoms, while the pilot user would act like the nurse. The pilot user was then given instructions to attempt to use our interface to diagnose our patient and supply him with a treatment, while maintaining the patient's information for future reference. Essentially, they would do a simple go-through of our interface for it's most basic purpose: diagnose a patient.
- Login and select a correct login type
- Find out whether the patient is new or existing, and do the appropriate actions to log them in
- Diagnose by selecting correct protocol, symptom group, diagnosis, and treatment
- Generate a report of the diagnosis and save for future reference
Observations on Testing Day
- In general, some of the pages were a bit too cluttered and inconsistent with their layouts, leading to slow operation and confusion. Splitting them up or hiding information until an action is taken would help facilitate the flow.
- Button consistency confusion, going to the next page sometimes involved a next button in the lower corner, and sometimes involved a button stuffed in a column somewhere. With this, the flow was a little confusing.
- Some information was lacking to make appropriate decisions, such as the difference between regular, administrative, and emergency login types. More information should be presented (but avoid cluttering the page).
- Users were unsure of what to do on the final "generate report" page; perhaps it should be automated or more information and direction should be supplied.
- In our interface, the first diagnosis in the list is always the best, but the users were not aware of this fact. We should make it apparent with visual signals or information that the first diagnosis is the best.
- More consistency is needed with the layout, as objects like the video panel keep getting swapped around in positioning.
- Users unsure of the purpose of the toolbar; perhaps information should be displayed about it on the first page, or we should highlight buttons on pages where their use might be needed.
- Treatment page was missing, should be added.
- Video confusion, some users thought that movies would take too long to watch and an alternative should be given, like simple pictures and text.
- In general, the facilitator felt the need to supply too many hints or explanations for directional purposes. From this, we must make the actual interface contain these crucial hints and explanations instead, since the facilitator will not be supplied with each release of the interface.
We updated our prototype for the real users by...
- Adding a treatment page
- Covering information with post-its and peeling them off as users proceed
- Videos replaced by pictures
- Generate report layout page changed, buttons moved for helping the flow
- Video/Picture panel standardized for all pages
Observations on the Real Users
- Had difficulty navigating the interface, due to the sheer amount of information on the screen at a time
- Difficulty in making decisions for diagnosis and treatment, perhaps our interface should make the best decision for them
- Confusing flow for some buttons, like Immunization Schedule, which should be attached to a person, not a protocol
- Follow-up appointments aren't handled, how will the user know? Need a notification system or a protocol
- Overall, the interface needs to be a bit more simple and straightforward, in terms of clutter and amount of choices made
- The interface is too cluttered to be manageable for portable devices (and in general)...
- We should separate parts of pages out to their own pages, and make them as naked as possible
- There are some special cases of symptoms that ruin the flow for the rest of the flowchart...
- Keep the flow the same, since it works practically 99% of the time. For the special cases, just give information for this cases as necessary
- There are times when there are too many words on screen, given the low literacy of our users...
- Replace some of the text with icons to help the language flow
- Immunization schedule feels odd when placed with protocols...
- Place the immunization schedule button at the patient screen
- Users overwhelmed with the amount of choices for treatment and diagnosis...
- Get rid of the choices through automation, which will automatically pick the best choice. We should reduce the users discretion as much as possible and not worry about prioritizing, just go for exact answers
- Follow-up appointments need to be handled and notified to the users...
- There should be a clinic-wide calendar in the toolbar that lists follow-up appointments, as well as a protocol for follow-ups (and perhaps a list of follow-ups when you log-in)
Our Prototype AIR application: https://wiki.cs.umd.edu/cmsc434_s10/index.php?title=Image:PDM.zip
- System Requirements
- You will need Eclipse and it's Flex Builder
- Your screen resolution should be at least 1024x768
- Windows or Mac should be fine
- Starting up instructions
- Currently you download our code at https://wiki.cs.umd.edu/cmsc434_s10/index.php?title=Image:PDM.zip and save it on your computer
- Go into Eclipse and import a Flex Project, selecting our zip file as the archive folder
- Run Template.mxml as a Air Application
- Have fun adding and selecting patients!
- Adding a patient and searching for a patient has been fully implemented so far
- The home button is fully implemented, and the back and forward buttons are sometimes implemented (depends on the page)
- The toolbar has not been implemented yet
- Vaccinations and Medical History can be viewed, but not manipulated
- Diagnosis and Followup has not been implemented yet
- Patient Selection Page
- Add a New Patient Page
- Patient Information Page
- Vaccination History Page
- Medical History Page
- Search for Existing Patient Page
Competing Systems: Compare our PDM to the diagnostic tool by D-Tree that is currently used by clinics
Hypothesis: If nurses use PDM over the D-Tree diagnostic tool, they should experience better usability in terms of faster diagnoses, less errors, and overall satisfaction
Independent Vars: Diagnosis tool used
Dependent Vars: Speed of diagnosis, errors occurred during diagnosis, satisfaction after diagnosis
- Patient and patient's symptoms
- Computer platform (screen size, OS, desktop/portable)
- Years of experience of nurses in clinic
Task: Have one nurse diagnose a chosen patient with the D-Tree tool, then have another nurse diagnose the same patient with the PDM (within subject - for every 2 nurses there is one patient)
User Population: Who:Nurses and related emergency personnel How:Find pairs of nurses with similar experience from clinics in Bangladesh
Null Hypothesis: Both tools offer similar speed of diagnosis, number of errors, and overall satisfaction
Alternative Hypothesis: PDM offers quicker diagnoses, less errors during diagnosis, and better overall satisfaction
Competing Systems: I want to compete against the way they use IMCI currently, which is looking up information in a paper flowchart.
Hypothesis: PDM is a much better system in that it allows nurses less stress and more satisfaction.
Independent Vars: The systems used for the experiment
Dependent Vars: The overall satisfaction of the nurses after completing the task
- The amount of experience of a nurse
- Time of day when we run experiments
- Computer literacy of a nurse
- Location when we run experiments
Task: A patient comes in with a need to get diagnosed. First, diagnose them the old way, with the paper flowchart. Then, upon acquiring another patient with similar symptoms, diagnose him with our tool. (Between subject)
User Population: Who: The young nurses who diagnose patients How: Use statistics to find a large enough group of nurses that have worked similar amounts of time, and also test their computer literacy before choosing them
Null Hypothesis: Nurses that use PDM or the paper flowchart feel there is no difference in ease of use or satisfaction
Alternative Hypothesis: Nurses feel that PDM is much easier and satisfying to use
Competing Systems: Compare our PDM with other team's IMCI designs.
Hypothesis: Which program generates a correct diagnosis most quickly amongst nurses?
Independent Vars: The program used to diagnose a disease
Dependent Vars: The time required to complete a diagnosis
- Setting (location)
- User (identification)
- Protocol (same information to give the same diagnosis within the program)
Task: Diagnose a child known to have dehydration, jaundice, etc. Each nurse diagnoses one, one program per nurse. (Between Subject)
User population: Who: Nurses in Bangladesh How To Sample?: Use a random number generator to pick the first 20 unique positions on a staff list
Null Hypothesis: There is no statistical difference between the two programs.
Alternative Hypothesis: Our program is faster at spitting out diagnoses.
Competing Systems: We would compare PDM with the current IMCI flowchart system in use.
Hypothesis: PDM will give a more accurate diagnosis as compared to the existing IMCI flowchart system.
Independent Vars: The diagnosing system used.
Dependent Vars: The accuracy of the diagnosis/treatment.
- Age - The subjects will be in the age group of 16-20.
- Clinician experience - with 0-1 year of clinical experience.
- Diagnosis difficult: similar illness attempted to diagnose
Task: Diagnose a patient (Between Subject -- testing the diagnosis with the same patient)
User population: Who: clinicians who are comparatively new to the whole IMCI experience. How: Select nurse trainees who are getting trained in using the IMCI system.
Null Hypothesis: The two systems help the subjects in diagnosing the patients with equal accuracy.
Alternative Hypothesis: The PDM system helps the subjects in diagnosing patients with a significantly greater accuracy as compared to the existing IMCI flowchart.
Competing Systems: We can compare our PDM to other groups' Diagnostic Interfaces
Hypothesis:If speed of diagnoses is related to the usability of the interface, our interface should offer faster diagnoses with better usability.
Independent Vars: The interface chosen to diagnose patients.
Dependent Vars: How much time it takes to complete a diagnosis correctly.
- The testing area (room location)
- The complexity of symptoms (each patient should have at most one major symptom, like coughing, but not coughing and diarrhea)
Task:A nurse should diagnose a patient with another group's Diagnostic Interface, and then another nurse should diagnose the same patient. Each pair of nurses should diagnose one patient, but the patient can be different (in terms of symptoms) between pairs. (Within subject)
User Population: Who: Nurses in general, with varying experience levels, so we can see if usability suffers in the hands of not only nurses with low experience, but capable nurses. How:Pure random. Visit a clinic, pick a random letter and get all nurses with last names starting with that letter. Repeat.
Null Hypothesis: All systems offer such a similar speed of diagnosis that, when choosing an interface, picking one at random will offer equal usability no matter what.
Alternative Hypothesis: Our PDM offers faster diagnoses when compared to other group's Diagnostic Interfaces
UI: Must have
1. Login page with functioning login 2. Administrator panel to import/export database, add users, etc. 3. Page to select protocol 4. Page to select symptom group 5. Pages to select symptoms 6. Page showing diagnosis from symptoms 7. Page showing treatment for selected diagnosis 8. Add new patient page 9. Search/Edit for existing patient
UI: Nice to have
1. Calculator 2. Calender 3. Unit Converter 4. Videos to show what symptoms/symptom groups are 5. Style, eyecandy and Pizazz
Server: Must have
1. Working database to store accounts and patient data
Server: Nice to have
1. XML IMCI import/export feature 2. Advanced searching on database 3. Advanced error handling (for example user typing in wrong data type into fields etc.)
Final Implementation File & Instructions on Use:
1. To see the product for our final implementation, you need the Adobe Air Plugin, which can be downloaded from a link off of http://www.adobe.com 2. Download the following file, unzip, and install. Go to the folder it installed into and run "Template.exe". If you have a Mac, it should be named something else, but still run correctly.
The username / password to log in is admin1 // password1.