Team 6

From Cmsc434_s10
Jump to: navigation, search

RefeReScheduler

A webapp to help intramural referees coordinate who is going to referee which games and when.

Team members:



David, Wen, Kelly, Mark

Project Proposal

Users and tasks

  • Target Audience: Intramural Sports referees and administrators.
  • Tasks
Tasks small.jpg
    • Add sports schedules and locations
    • Add/remove referees, their availability, their sports
    • Swap shifts between referees, via a coordination website and email notifications based on roles
    • Validate administrator's ref choices based on availability and schedule


Existing solutions and their limitations

  • Currently performed by
    • Just lots of email to a general email list
    • Only master schedule is kept by a single administrator and is not available to others
    • Availability is done by hand written cards
  • The current methods are hard to coordinate and hard to verify. They are confusing and less available than a webapp. The current methods involve referees swapping by email and the paper trail is hard to follow, a coordinated way of dealing with this would be ideal.


Meeting notes

BS 3 small.jpg
  • Revamp the Current IM website: For anyone who has attempted to find information on the intramural website, they know that it is hard to navigate and unattractive.
  • The Accent Database: We thought the accent database proposed in class sounded interesting but ended up being unsure how to go about it. It could be a database containing references to outside sources as well as internal source material that has videos/audio/IPA pronunciations of various accents by reigon.
  • Pick-Up Sports: Sometimes people who want to play a pick-up game decide not to because they don't want to trek out there and have no one to play with. Designing a website that would help notify people that people are looking to play.
  • Golf club Scorecard: When you golf you can take use the app on your iphone to keep track of your score. And afterwards will calculate your handicap based on the course.
  • Campus Traffic: When is the best time to try and get where you want to go on campus. You could use an app or website that would warn you of special events and general traffic patterns on campus.
  • Getting to Class: There are a lots of ways to go from one point on campus to the other, but which is best. A web app could be used to calculate a walking route based on any number of things (avoiding hills, time, traffic of walkers and cars).
  • Frisbee Team Trip Planner: The frisbee team plans trips to other schools for games, this website would help coordinate who is driving to the game and who is in the car, it would also expedite the forms that the school requires for these trips.
BS 1 small.jpg
BS 2 small.jpg

User and Task Analysis

Applicable Techniques

3 Applicable techniques:

  • Interview - There are only two people in the Intramural system that are responsible for scheduling referees. Because of that fact it makes most sense to interview these people about their system.
    • Possible questions:
      • Can you explain the current process of scheduling referees.
      • Explain how referees currently find subs for their shifts.
      • What do you like about your current system of scheduling and substitution?
      • What are some of the things that generally go wrong with the current system?
  • Questionnaire - Because of the number of referees that work for Intramurals it would make the most sense to give some of them a questionare to fill out information. Many of the questions can be similar to the questions used in the interview because this would give us two perspectives of the same process.
  • Examining existing solutions
    • The method in which refs currently find substitutes for their time slots is via multiple emails to various people. We could examine a few of these strings of emails to see how the interactions work and the effectiveness of various techniques such as offering to trade time slots. We would also be able to note who these emails are being sent out to and how the substitute is chosen.

Questionare Technique Details

1. What class-standing are you in school and how many semesters have you worked as a referee?

2. On a scale from 1 to 10 identify your level of computer knowledge (1 being no idea-- 10 being computer genius)

3. Do you submit an availability schedule to your manager prior to schedules being created? If yes, what is the process for submitting your availability?

4. In what format do you receive your schedule? (i.e. Is it sent through email? Is it personalized?)

5. How often do you sub for other refs? How often do you get other refs to sub for you?

6. Describe the process of obtaining a substitute.

7. What do you like and/or dislike about the current scheduling process? Do you like the substitution system? How could it be made better?

  • Findings
    • Most referees are working in their first or second semester as a ref.
    • Referees rarely submit a weekly availability schedule to their manager
    • Most referees have very few problems with the current system, however they found it "annoying" that there is no way of knowing if a shift has been taken after a ref submitted a sub request. It equals a lot of extra emailing.

User Classes

  • Administrator
    • Age: 25-40
    • Education: Undergraduate-Graduate degree
    • Computer skills: Average
    • Motivation: An administrator is mainly looking to schedule referees once a week, as well as contact referees with announcements.
  • Referee
    • Age: 18-23
    • Education: Current Undergraduate Student
    • Computer Skills: Average
    • Motivation: Referees have two main motivations: find out when they are scheduled to work, and find a substitute for the shifts they can't work.
  • Supervisor
    • Age: 19-23
    • Education: Current Undergraduate/Graduate Student
    • Computer Skills: Above Average
    • Motivation: Know who is scheduled to work each night.

Personas

  • Administrator
    • Mary Katherine is full-time employee administrator who is in charge of all of intramurals at the University of Maryland. She is the only one who receives the latest most up-to-date schedule of refs working their shifts. It is up to her to let her employees know when they are scheduled to work. Sometimes there is confusion revolving that. When there is a no-show ref at the game, she has the latest copy to figure out who was supposed to be there.
  • Referee
    • Wesley Fasay is a computer science student at the University. After classes are over, he tries to ref as much as possible when he does not have projects to code in order to make money. When there is a deadline coming up, he likes to swap with work with other refs in order to get work done. In exchange, he is willing to take other shifts when there aren't projects due soon to make up for the money that is lost. He emails the refs he is comfortable with emailing to work out a trading in schedule, and then sends a copy of the change to an administrator.
  • Supervisor
    • Heather Smith is a junior in college working as a supervising ref. She has has been a ref for a few semesters and knows how it works. Because only the admin get the latest copy of a modified schedule, she is not always aware of who should be working. When there is a no-show at the game, she is sometimes not informed as to who is supposed to be there.

Other Potential Stakeholders

  • Sports Team Players
    • Students at the University of Maryland who participate in the intramural sports need referees to distinguish points/fouls and other rules that are apart of the sport they are taking part in in order to establish fair play. Without a referee, the game would not be able to take place. It is up to the school to put forth the referees and the students are inconvenienced when they do not do so.

Tasks

Administrator

    • Goal: Create data of which refs are doing what
    • Preconditions: Have a list of desired schedules by the employees
    • Complexity: Medium
    • Frequency: Once a semester, and whenever there is a modification
    • Potential Errors:
      • Forgetting to create an account
      • Creating a normal ref account for a supervisor
      • Inputing wrong information into the form
      • Adds ref to wrong sport
    • Subtasks:

1. Add sport

  • Log in
  • Click “Add sport”
  • Input information into form
    • start/end dates
    • days/times of play
    • shift start/end times
    • number of refs/shift
    • refs to include with sport
  • Click “Add”

2. Add new user

  • Log in
  • Click “Add user”
  • Input information into form
    • name
    • email
    • initial password
    • initial sports to be added to
  • Click Radio Button (Ref/Supervisor/Admin)
  • Click “Add”

3. Schedule referees

  • Log in
  • Click on sport
  • Click on specific sport
  • Click on week to update
  • Click drop-down for day/shift
  • Click name of referee
  • Click save if not done
  • Click post when completed

4. Add referee to sport

  • Log in
  • Click on “Sports”
  • Click on specific sport
  • Click “Add Referee to Sport”
  • Click drop down menu
  • Click referee name
  • Click “Add”

Referee

    • Goal: To see when to work and reschedule when needed
    • Preconditions: Have an account and schedule made by the admin
    • Complexity: Low
    • Frequency: Once a semester, and whenever a modification is needed
    • Potential Errors:
      • Request the wrong shift
      • Taking an shift that they are not available for
    • Subtasks:

1. View Shifts

  • Log in
  • Click specific sport
  • Click week
  • name/shift is highlighted
  • Click specific shift for more information

2. Request Sub

  • Log in
  • Click sport
  • Click day of month
  • Click “Request Sub”
  • Input reason (Text Box)
  • Click “Submit request”

3. Take Sub Shift

  • View email request
  • Log in
  • Click specific sport
  • Click red highlighted days (indicates sub shift needed)
  • Click name of ref who needs sub
  • Click “Take Shift”

4. Update availability

  • Log in
  • Click “Availability”
  • View current availability
    • Red Blocks = availability that remains constant
    • Yellow Blocks = availability for specific week
    • Disclosure at top “Availability will take affect for week…”
  • Click on shift
  • Click if availability is permanent or for week
  • Click “Update availability”

5. Update information

  • Log in
  • Click “User Information”
  • Click “Edit”
  • Edit information
  • Click “Update/Save”

6. View other’s Contact info

  • Log in
  • Click “Contact List”
  • Click name
  • View

Supervisor

    • Goal: Same as ref, but with every sport
    • Preconditions: Have a supervisor account and schedule made by the admin
    • Complexity: Low
    • Frequency: Once a semester, and whenever a modification is needed
    • Potential Errors:
      • Prints schedule prior to all the adjustments are made
    • Subtasks:

1. Print Daily ref schedule

  • Log in
  • Click “Sports”
  • Click specific sport
  • Click day
  • Click “Print schedule”

Design Sketches

Task Scenarios

It’s that time of year again, 9x9 soccer is just around the corner(kick). MK has hired her referees and needs to schedule them(PRECONDITIONS). She logs into her administrator account on RefReScheduler(SUBTASK). She clicks on the Sports page(SUBTASK), and then clicks the icon to add a sport(SUBTASK). She sees before her an empty sports form, she is a little intimidated but knows what must be done. She begins filling out the form with the soccer information, including how many refs she would need for each shift(SUBTASK). Finally she finishes, but WAIT!! There’s a mistake in the Location: (you never play soccer at the armory)(ERROR). Clicking the “edit” next to “Location” she types in “Turf.” Much better, now to just schedule the refs…(FREQUENCY:multiple times a semester)

Wesley Facey woke up one afternoon after a LONG night. BLAH!!! He was not feeling well(PRECONDITIONS) and remembered that he had to referee soccer that night. He knew that he would need to find someone to take his shift(HIGHLY MOTIVATED). So he logged onto his RefReScheduler account(SUBTASK), clicked on the Calendar Page(SUBTASK), and found the shift he didn’t want to work(SUBTASK). He clicked on his name to fill out the Sub Request Form(SUBTASKS), and then submitted it.(FREQUENCY:weekly) As he returned to bed he hears a melodic ring from his Blackberry indicating he had one new email…(SUCCESS?!)

Heather Smith is supervising the intramural games tonight. The start time for one of the games has come but Wesley Facé still hasn't shown up(HIGHLY MOTIVATED). Wesley is often asleep during important appointments, so Heather goes to RefReSchedule to find out what Wesley Facé's phone number is(NEEDS). She goes to the website, and logs in using her username and password(PRECONDITION). She clicks on people, and selects Wesley from the list, and can see Wesley's Contact Info (TASKS). She calls Wesley, waking him up from his obviously alcohol induced nap(TASK). She passive agressively asks Wesley what time it is, but Wesley, although still drunk and half asleep, is quick on the draw and tells Heather that he will be there ASAP. Wesley runs out the door as fast as he can and arrives 20 minutes later at his scheduled and late soccer game, only slightly needing to vomit(SUCCESS).

Design 1

Design 2

Story Board

UI Board

Design 1

Design 2

Paper Prototype

Initial Tests

Risk Assessment

  • Things we were worried about
    • Knowing which things were buttons (Schedule Page)
      • Specifically clicking knowing that the user's name was a button
      • We tested this with Scenario 2 (see below)
    • Knowing which header button to click for each task
      • This was tested through all of the Scenarios
    • On the Sports page we wanted to test whether a user would know to click the edit button for each field
      • We tested this with Scenario 1 (see below)
    • On the Contacts page we wanted to see what info users expected to see.
      • We used Scenario 3 to test this (see below)

Prototype Pictures

Briefing

  • We told the user, "Our application would be used by the intramural sports department for scheduling and rescheduling sports and referees. It is designed to allow refs to view their own schedule and request shift changes as well as accept new shifts from other refs."

Scenario 1

You are an administrator. The Outdoor Soccer season is about to start and you must add information about that sport to the website. Please complete the following tasks.

  • Create a sport called: Outdoor Soccer
  • Edit the new sports Details
    • Location
    • Sport Start Date
    • Sport End Date
    • Divisions that will compete in the sport
    • Game Times
    • Shift Times
    • Number of refs required per shift

Scenario 2

You are a referee and need to find a sub for one of your shift next week, but you don't want to lose out on that money so you decide to also take a shift from a ref who needs a sub.

Please complete the following tasks.

  • Submit a sub request for your Sunday shift
    • Edit the reason for the request
  • Take a sub shift on Tuesday.
    • Confirm the switch.

Scenario 3

You are a supervisor and one of your referees hasn't shown up for their shift! You need to get someone here, so you decide to call around and ask if people can come referee.

Complete the following tasks:

  • Find Amy's contact information
    • Call Amy!
  • Find Wesley's contact information
    • Call Wesley

Observations on Test Day

  • Our login page did not have a login button.
  • The header buttons weren't as self-explanatory as we thought they would be
    • Drop-down Menus was suggested
  • It was not obvious that a user should click on their own name to request a sub.
    • We think it will be more obvious when implemented on the computer because we can make use of button details and user interface consistency.
  • On the confirmation page people did not realize there was a "reason for request" text box.
    • We could add color (red asterisk) to identify that it is required.
  • On the Sports Information page users had trouble with hitting the "edit" buttons.
    • We thought that when an administrator adds a sport all of the text boxes could become editable for convenience.
  • Adding game time (table information) on the Sports info page, some users were confused about the +/- button to edit the field.
    • Making the buttons more obvious we think would help this.
  • Users were wondering if each field was saved automatically or whether there was a button they had to click.
    • Even though we think it would be cool to save automatically, we will add a save all button to adhere to interface consistency and the users peace of mind.
  • On the Contact Info page people expected a filter by sport in order to find refs for a specific sport.
    • This is a feature that can be added, as well as other specific filters.

User Test

Prototype Iteration

  • We redrew the Schedule Page to attempt to make things look more like buttons.
  • We added a Save button to the Sport Info Page.
  • We also changed the functionality of the Sport Info Page.
    • When you "Add a new sport" all of the text fields become editable.

Observations on real users

  • User 1
    • For our first task the user was a little hesitant about what to click to request a sub. After the initial hesitation, he breezed through the rest of the task without any confusion.
    • The user initially missed our intended button to add a sport for our second task. He commented, however, that the design for our second task was intuitive.
    • The user easily completed the third task without any errors.
  • User 2 - Kurt
    • For task 1, the user did not know to click his name to request a sub. The rest of the task was completed successfully without any errors.
    • For task 2, the user was slow to find the 'Add Sport' button. He completed the task without error.
    • The user completed task 3 without any errors. He suggested a filter button so he could more quickly look up referees of a specific sport.
    • This user was overall very impressed with our idea and design. He felt that our project could become marketable if done properly.
  • User 3 - Phil
    • The user completed task 1 without error. He commented that he was a little hesitant to click on his name to request a sub. But he added the design would be very easy to learn after using it once.
    • The user completed task 2 correctly. He was a little confused about our 'add sport' button.
    • Task 3 was completed correctly without error or confusion. The user suggested a few additional features to our interface for task 3. He would have liked to see a filter that sorted by sport and availability. He also would like a drop-down menu from the main tab bar.

Risk resolution

  • Many users had took some time to realize what buttons to push on the Schedule Page. (The things that are buttons don't seem like buttons on paper)
    • What this means is that we are going to have to be careful about how we create this page, and make sure that it is obvious what is clickable. We can do this by adding animation (color change, bolding, etc) to the page.
  • Every user had trouble with our 'Add Sport' button for task 2.
    • This confusion occurred because our paper prototype was not able to dynamically change the way our computer prototype will be able to do. For instance, the edit buttons that were drawn onto our paper prototype were a source a confusion, but these buttons will not be visible on the computer prototype until after the user initially enters a sport.

Computer Prototype

System Requirements

Flash player plugin needs to be installed.

Startup Instructions

Click on the link under "Implementation" below.

Fidelity

All of the screens are implemented. Currently, users can log in using any name and password, go to the people page to view the contact info of various referees. The admin (right now, it's anyone) is able to edit and save the announcements on the homepage. The user can also go to the schedule page, change over to week view and click on a red shift (indicates that someone is requesting a sub) and take that shift or click on a yellow shift (indicates that the user currently owns that shift) and request a sub.

Comparison

  • Login page

Login.JPG Ref login.png

  • Homepage

HomePage.JPG Ref home.png

  • Request a sub

RequestSub.JPG Ref request.png

  • Week view of the schedule

Schedule(Original).JPG Ref week.png

  • Contact info page

ContactInfo(Pre).JPG Ref people.png

  • After clicking on a name on contact page

ContactInfo(Person).JPG Ref people2.png

  • Sports page

SportInfo(Pre).JPG Ref sports.png

Experimental Design

Mark Heneks: Comparing E-Mail and Our App

What competing system or design alternative do you want to compare against?

Email based shift rescheduling

Part 1: Design

State a lucid and testabe hypothesis

RefeReScheudler have a lower rate of unfulfilled substitute requests.


Identify independent and dependent variables

  • Independent: Shift rescheduling system
  • Dependent: Unfulfilled requests for substitute referees.
  • Controls:
    1. Same group of referees will participate in both systems
    2. Length of study, preferably accounting for semester variability.
    3. Sports being played during period of study.

Design the Experimental Protocol

  • Task: Collect information on which substitute requests get filled and which do not for each user of the system. This data can be collected over the course of a semester while the users perform their substitute requests with the different systems.
  • Within Subject

Choose the user population

  • Who? The entire referee staff
  • How to sample? The application applies to a small group of people who all already work together. Collecting data on the whole population is easy, effective, and allows for the best possible analysis.

Part 2: Run Experiments

Apply for human Subjects protocol review (IRB)

Run some pilot participants

Modify and finalize the experimental protocol

Run the experiment.

Part 3: Analyze Reults

Perform statistical analysis

  • Suppose you are going to run t-test on your data. State the null and alternative hypotheses.
  • Null Hypothesis (H0): The rate of filled substitute requests will remain the same.
  • Alternative Hypothesis (H1): The rate of filled substitute requests will decrease using RefeReScheduler.

Draw conclusions

Communicate results

David Moore

What competing system or design alternative do you want to compare against?

The current scheduling system consists of the administrator creating a master schedule and then sending the schedule to each individual ref. The administrator must schedule refs based on paper based availability sheets. Changes are made solely through email, making them difficult to track. Our new system will allow administrators to schedule referees online. The finished system would keep track of which referees were available at each time slot, so the administrator would not have to check paper availability sheets. Also, changes would be made and tracked online.

Part 1: Design

State a lucid and testabe hypothesis The RefeRescheduler will make the IM Administrator's task of creating and adjusting referee schedules easier and more efficient.

Identify independent and dependent variables

  • Independent: Referee availability
  • Dependent: The time it takes to create the master schedule.
  • Controls:
    1. Number of available referees
    2. Number of shifts to schedule
    3. Changes in referee availability

Design the Experimental Protocol

  • Task: Create a master schedule for one week of one sport. Then repeat for additional weeks. We will record the time it takes to complete each week.
  • Within Subject

Choose the user population

  • Who?
    • Intramural Sports administrators
  • How to sample?
    • Have the current campus IM administrators test our system.

Part 2: Run Experiments

Apply for human Subjects protocol review (IRB)

Run some pilot participants

Modify and finalize the experimental protocol

Run the experiment.

Part 3: Analyze Reults

Perform statistical analysis

  • Suppose you are going to run t-test on your data. State the null and alternative hypotheses.
  • Null Hypothesis (H0):
    • It is not faster to schedule referees using our new scheduling application.
  • Alternative Hypothesis (H1):
    • It is faster to schedule referees using our new scheduling application.

Draw conclusions

Communicate results

Wen Li

What competing system or design alternative do you want to compare against?

The current system of finding a replacement for a no-show at a game is for the supervisor to call up referees who are also refs for that sport. The supervisors may or may not have an updated version of the final schedule, know availabilities, or perhaps may not be able to recall all the refs for one sport. The design alternative is for the supervisors to use the people page as well as looking at the most updated schedule.

Part 1: Design

State a lucid and testabe hypothesis The RefeRescheduler will make getting a last minute ref at a game more efficient than the current method.

Identify independent and dependent variables

  • Independent: Referee availability
  • Dependent: The time it takes to get a replacement ref
  • Controls:
    1. Availability of referees
    2. Shift that needs a referee

Design the Experimental Protocol

  • Task: Pretend that it's almost game time and there are no referees for a specific game. The supervisor needs to find a suitable ref for that game.
  • Within Subject

Choose the user population

  • Who?
    • Intramural Sports supervisors
  • How to sample?
    • Have the current campus supervisors test our system.

Part 2: Run Experiments

Apply for human Subjects protocol review (IRB)

Run some pilot participants

Modify and finalize the experimental protocol

Run the experiment.

Part 3: Analyze Reults

Perform statistical analysis

  • Suppose you are going to run t-test on your data. State the null and alternative hypotheses.
  • Null Hypothesis (H0):
    • It is not faster to get last-minute referees using our new scheduling application.
  • Alternative Hypothesis (H1):
    • It is faster to get last-minute referees using our new scheduling application.

Draw conclusions

Communicate results

Kelly O'Neil

  • Currently Refs are emailed a schedule a week prior to working. The schedule is set up in a basic excel grid where each ref is listed under the shift they are working.

Part 1: Design

  1. State a lucid and testable hypothesis
    • Referees are less likely to miss a shift because they did not realize they were scheduled using RefReScheduler.
  2. Identify independent and dependent variables
    • Independent:
      • Refs, their availability
    • Dependent:
      • Number of shifts a Ref is scheduled for, When they are scheduled
    • Controls:
      1. ability to differentiate colors/names
      2. reading a basic grid schedule
  3. Design the experimental protocol
    • Task:
      • Look at a schedule and say what shifts they are scheduled for
    • How?
      • Within subject.
  4. Choose the user population
    • Who?
      • Referees currently working for IM
    • How to sample?
      • Ask a number of refs from each sport to participate.

Part 2: Run Experiments

  1. Apply for human subjects protocol review (IRB)
  2. Run some pilot participants
  3. Modify and finalize the experimental protocol
  4. Run the experiment

Part 3: Analyze Results

  1. Perform statistical analysis
    • Null Hypothesis:
      • Refs are less likely to miss a shift because they did not realize they were scheduled using the RefReScheduler.
    • Alternative Hypothesis:
      • Refs are not less likely to miss a shift because they did not realize they were scheduled using the RefReScheduler.
  2. Draw conclusion
    • Using color to highlight a Ref's shifts make it easier for them to find the shifts they are scheduled for.
  3. Communicate results

Implementation

Prototype!
Final!

The task implemented is to look at your schedule and either request a sub on the days you work, or take a shift from another ref.

  • Potentially we would like to implement a few more functionalities on the Ref Schedule Page, (cancelling a sub request, changing listeners when a shift is changed).
  • Then we would like to get the schedule working with the back-end so that the schedule is automatically created and updated as users use it.

User Testing and Analysis