Kristen Reay

Week 7 — Human Computer Interaction — Running Web Experiments

A/B Tests, Split testing, ex. — Splitting the traffic of who comes to the site. Collect metrics (conversions, click throughs, etc.) 3 versions of a site - # of columns - getting more content “above the fold”. Page A / Page B / Page C — calculate % traffic (same for all 3) vs. new sales and see the change. Page C had a drop in sales! 

Ways design makes a difference! Small changes in a design can make a real difference in the effectiveness of the site. 

• The position and color of the primary call to action.
• Position on the page of testimonials, if used.
• Whether linked elements are in test or as images.
• The amount of white space on a page, giving the content space to breathe.
• The position and prominence of the main heading
• The number of columns used on the page.
• The number of visual elements competing for attention.
• The age, sex, and appearance of someone in a photo.

Content above courtesy of A List Apart http://alistpart.com/articles/designcancripple

Question: How do we measure the outcome and if it’s what we want? With tests!

“Our expectations are often wrong.” - Scott Klemmer

“You should follow me on twitter here” had the highest number of click through on Dustin Curtis’ blog. 

Large Scale changes design: Make smal but consequential differences detectable. Small differences accumulate. Beware of anomalies: investigate further. Some of the changes you may see may be anomalies. Important to run real manipulations to avoid data anomalies. 

Principles for Effective Inline Experiments

1. Run experiments with equal number of people in each of the two conditions. This is the fastest way to detect an effect if your design is causing a change that matters.
2. Initially, ramp things up. If your deign is too drastic, and it’s negative, then you don’t want to hit 50%. Ramp it up to 50% once you make changes along the way. If you don’t see a positive effect, then roll it back.
3. Figure out what you want to measure. “Pick a meaningful yardstick”
4. Run your experiment long enough for people to get accustomed to it. Sometimes furst use is not the same as what people are familiar with. 
5. Rules for random assignment: assignment should be consistent (same person should see the same interface every time they logon). It’s important that the assignment is random. 
6. By running controlled experiments you can be sure it’s your changes that are causing the effect. Run online tests, and combine it with in person studies.

Designer on the online age!

• Role has shifted to being about creating multiple alternatives.
• People are often too sure of themselves.
• Rapid experimentation can help you fail fast to succeed sooner. 

Analyzing Experiments - Rate Comparison

3 Questions that you can ask and answer by analyzing your data:

1. What doe smy data look like - graphically. Look for paterns. 
2. What are the overall numbers? Aggregate stats for quick summary. Usually mean and standard deviation.
3. Are the differences “real”? Is the number significant enough?

Chi- Square Test

(Observed - expected)^2 / expected

• Critical values for chi-squared - our P value. 
• As the chi-square number gets bigger, our the difference between expected and actual grows, then it is unlikely that this information could have been created by a unbiased coin. 
• Degrees of freedom - number of choices you have minus 1. (ex. A 6 sided die degree of freedom is 5.)

The Null Hypothesis - our opening bid, is “we dont think there’s a different between our two conditions.” Ex. The coin is not loaded. 

• Check whether the data falsifies the null hypothesis.

“To get a feel for your data, graph it all!” - Scott Klemmer

Week 7 — HCI — Running In-Person Experiments

Don’t miss out on watching people!

• What are they doing?
• What are the confused about?
• What did they miss?

Watch People: The guidelines:

1. Make clear goals. Lay out the scope. Make it narrow so it is guided by “What do you hope to learn?” Make sure you test it in an appropriate setting. Make scenarios for realistic use. Task completion time is useful, also ask yourself if the user successfully completed the task.

2. Plan out the steps for your test. Get at least 2 people, the facilitator and the note taker. 

3. Create concrete instructions. Write down the steps so the experiment is sound — everyone gets the same instructions.

4. Ethical considerations. People feel responsible for “doing well”. Make consent volitary, avoid pressuring people to participate, and make sure they know they can stop at any time. Remind people, multiple times, that you are testing the frame, the system, the user interface, NOT THEM! Repeat this many times to reduce stress.

5. Start with simple task then move to more complex tasks. Are you going to provide training? Depends on what your testing. If someone hits a stumbling block, you can help them along. Pilot your study! A pilot study runs through your study before an actual participant does. Good idea!

6. record video! Capture success or failure. Screen or person. Either is great.

7. Interruption? Sure, if you have questions, need to help them, etc.

8. Think aloud. This is helpful if you need to know what users are thinking, not just what they are doing. “Tell us what you are thinking”, :Tell us what they are trying to do?”,  ”Tell us questions that arise” — Tell people you’d like them to think aloud in the beginning  and throughout the study prod them for details! Don’t take what people say as the end-all. People are usually talking to help you out and generate answers. But it may not be exactly what’s going on in their noodle :) - Scott

9. Greet people well!

10. Collect process data and bottom-line data. Bottom-line numbers are good for task completion time (don’t include with think aloud). 

11. Debrief the user. So you can tell them what you were trying to get to and explain why you asked them to think aloud and partake in the study. 

UX Webinar w/ Tomer Sharon

Notes from today’s MINDBODY webinar with Tomer Sharon

There is a strong relationship between attitude and behavior. FALSE. What people think is correct or how they feel about something has little to do with their actual behavior.

Factors of attitude to behavior relationship

1. 1. A general attitude predicts a wide set of behaviors.How social are you? Use the test, number of share, likes, and posts on facebook. 
2. 2. Time between measurements. There is a positive correlation between attitude and behavior increases in measurement time-difference is shorter. Because people’s attitudes are volatile. We change our minds!!
3. Way of acquiring attitude. Past experience clarifies attitude and increases its positive correlation with behavior. 
4. Clarity of attitude. When attitude is clearer, their opinion will be more sound. Sometimes people don’t know how they feel. With past experience we can predict the future behavior.
5. How accessible the attitude is to your conscience. Conscience accessible
6. The users personality affects the behavior and attitude correlation. High self monitoring people, those concerned with their self presentation  are hard to predict their behavior. There is a self monitoring scale that you can use to find out the score of self monitoring people. 
7. Don’t ask what people need. Instead observe what they do. Go to your customers and ask to watch them do what they do. Just watch. 
8. Don’t ask for feedback. Here’s my idea. What do you think? No information will be collected. Instead watch people use it. Draw the thing if you don’t have it. Ask them to do something with the drawing. 

UX Research

• Don’t ask about opinions only after users experience the product.
• Ask about past experiences rather than predictions of the future. 

Quiz

What is the expected strength of relationship between attitude and behavior? Strong, weak, no idea.

Week 7 — Designing Experiments

Designing Studies You Can Learn From

“Do you like my interface?” How do we get beyond this question?

Questions:

1. What are we comparing it to?
2. What constitutes good? 
3. What fraction of people click on a link?
4. Is there a relationship between the time of day a class is offered and the number of students that attend the class?

Independent vs. Dependent Variables

• Task completion time.
• Accuracy - how many mistakes did people make? Could they recover?
• Recall - how much do they remember?
• Emotional - how confident was the user? Where they stressed? Would they recommend it to a friend?

Some definitions:

• Dependent variables are variables that the experimenter measures in the experiment.
• Independent variables are the things that you manipulate? 
• External and Internal Validity. - Bring it out into the world!

Strategies for fair comparisons:

1. Insert your new approach into the production setting
2. Recreate the production approach in your new setting
3. Scale things down so you’re just looking at a apiece of a larger system.
4. When expertise is relevant, train people up.

Controlled Comparison enables Causal Inference - you can learn stuff. 

HCI Week 6 — Visual Design

White space conveys grouping. 

Use size contrasts to indicate hierarchy. 

Good visual design goes unnoticed because the content comes through so seamlessly. Make th einformation that matters come through clearly.

3 goals for visual design:

1. Guide people - convey structure, relative importance, and relationships. 
2. Pace - draw people in, help them orient themselves, and provide hooks to dive deep.
3. Message - express meaning and style, breathe life into the content.

People don’t read online. :) - Jakob Nielsen

Books to read!

Designing Interfaces - Jennifer Tidwell. 

The New Typography, Jan Tschichold

HCI - Week 5 cont. — Distributing Cognition

In Week 5, Scott Klemmer addresses Distributing Cognition.

Distributing Cognition can:

• Encourage experimentation - trying out solutions to improve. (tetris)
• Scaffold learning and reduce errors through redundancy (Montessouri blocks)
• Show only the differences that matter (London Undeerground)
• Convert slow calculation into fast perception (Map coloring)
• Support chunking, especially by experts (Chess and gestures)
• Increase efficiency (diagrams)
• Facilitate collaboration (cockpits)

Show the information you need and nothing else. Example: The London Underground.

• Task specific representation is very important. 
• Hue is not an additive representation. There is not a strong ordering we can give to the colors. There is not a more than less than representation. 
• Edward Tufte’s land and see map. All the things above sea level are brown. We are leveraging our understanding of the color of land to understand the map. Brown is land. And blues are ocean. 
• Can we make interaction for chunkable. Place a lower load on our memory. 
• Buxton: Chunking in interfaces. Copy/Paste example.
• Show we represent information visually or texturally. 

Informational Equivalence: If all the information is inferrable in one place can it be in another place as well. 

Poor Representation Examples:

• Wrong password entered, please change it. But what is the problem with the one I entered? 
• BB Edit - Document encoding mismatch. Cancel or save anyway. Option I want: make everything Latin-1 or make everything UTF-8. Help me make the right decision.

Dialogue boxes should be action oriented, guiding user towards the next step. 

• eBay- Pay with bank account versis pay with credit card. Dimmed out button to make it less likely to be clicked. Yikes!

Books!! Suggested by Scott.

Don Norman’s, Things that Make Us Smart

Ed Hutchins, Cognition in the Wild

Herbert Simon, Sciences of the Artificial

HCI — Week 5: Manipulation and Representations — Part I

“The key innovation of a graphical user interface, is that input is performed directly on top of output.” - Scott Klemmer

• Simply asking people what they wants may cause you to miss opportunities.

Questions to use when looking for where challenges arise.

1. How easily can someone determine the function of the device?
2. How easily can someone tell what actions are possible?
3. How easily can someone determine mapping from intention to physical movement?
4. How easily can someone perform the action?
5. How easily can someone tell what state the system is in? / If its in desired state?
6. How easily can someone determine mapping from system state to interpretation?

Command Line vs. GUI

What is better? It depends! But what makes them different. Command lines require knowledge, they don’t give immediate feedback, and they don’t leverage metaphors for things humans already know. The GUI does a better job in terms of visibility, feedback, and consistency. GUI’s shine in discoverability. 

When is the command line better? When the indirection that it offers is a benefit rather than a draw back. This is powerful when you can express stuff more abstractly and therefor do things more efficiently. 

Mental Models - What makes an interface learnable and what leads to errors in user interfaces. 

“There’s a big pitfall in being a designer, you’ve spent so much time with the system that you know how it works under the hood and how you imagine other people will think about it. Your expertise can be crippling, the mental model that you expect users to have, you expect it to be the same as yours and it doesn’t play out in practice.” - Klemmer.

That’s why it’s important to get other users in front of your design as soon as possible. 

Slips vs. Mistakes

Slip - the right model for the system and how it works, but you accidentally do the wrong thing. A motor error.

Mistakes - when you do what you intend to do, but you have the wrong model. Driving, you think you should take an exit, but it turns out to be wrong. 

• Designers try to prevent these errors. Slips can be prevented to avoid slips. Designers need to provide better feedback to avoid mistakes.

Presentation Matters

“The ways in which we and the world organize and represent ideas can have a drastic impact on our cognitive abilities.” - Scott Klemmer

“Solving a problem simply means representing it so as to make the solution transparent.” - Herbert Simon, The Science of the Artificial

Naturalness Principles

• Experiential cognition is aided when the properties of the representation match the properties of the thing being represented.
• Integrating the necessary step with the natural step makes you not forget the necessary step. 

Book suggestion: Don Norman’s, The Design of Everyday Things. 

HCI - Week 4 - 10 Design Heuristics: Part I & II

Scott Klemmer introduces design heuristics to identify design interface errors -and ways to address them. He will give us examples and talk through them with us! Yay! Sounds fun!

Focus on narrow issues in the user interfaces using the Ten Design Heuristics:

• Show system status - provide feedback on where the system is and where the user is in time and flow. Especially when the user waits longer than 1 second for response.

• Familiar metaphors and language - provide a match in the interface to something the user is familiar with in the world around them. Ex. print dialog box shows print preview. Use terms and language that is familiar to the user. 

• Control and freedom - Making sure that the amount of control and freedom is approapriate for the user and the tasks. User can undo, not get stuck in workflows. Can continue or go back. Give the user freedom to explore. 

• Consistency & standards - consistent layout, the same buttons will always be in the same place. Consistent names throughout the product.  

• Error prevention - Prevent people from data loss. “This file already exists.” Prevent clutter with a message like, “You already have this paper.” A bad example is “Click cancel to continue.” Prevent bad input. 

• Recognition over recall - Create interfaces that make actions, objects, directions, easily retrievable  Rule of thumb: if there’s a post-it note then there is something missing. Avoid codes! Setup defaults. Lead with these defaults and provide previews. “Minimize the user’s memory load by making objects, actions, and options visible  The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.”

• Flexibility & Efficiency - Provide keyboard shortcuts! View other information to provide heightened productivity. Provide recommendations for items the user might also like. Keep it relevant! Ex. the remote control that was hacked! 
• Aesthetic & minimalist design - is the core information “above the fold”? visual design: have colors mean something. Be a minimalist, is the signal-to-noise ratio appropriate? 

• Recognize, diagnose, and recover from errors - Make the problem clear! And can the information also offer you some sort of view of ways you can fix the problem. Offering one dialog box for all errors is not good. When a search yields no results, one of the most powerful things you can do it make it more workable by “intelligently relaxing restraints.” 

• Help - Give help choices with previews. Tell the user the problem and help them to a solution. Good example, a subscriber wants to unsubscribe to an email distribution list. The software asks for a reason why. If you choose, “I get too many emails” then, you can have the options to receive emails on a weekly, monthly, or quarterly basis! Cool. Help people have fun! 

HCI - Week 4 - Lecture 4.1 - Heuristic Evaluation

Ways to Evaluate your Designs

• Critique based approaches - incredibly effective. Critique is important to get before user testing because then your don’t waste your users on finding the simple quick flaws in your design. Structured peer critqie is important before release because it helps you smooth out the rough edges. 

Heuristic Evaluation - Jakob Nielson developed this method. The goal is to find usability problems in a design. Quick feedback, high bang for buck strategy.

• Provide a set of people, other stakeholders, or outside design experts, a set of principle, or heuristics. Then the will find problems on their own. Then they get together and talk about what they found, This is “the wisdom of crowds.”
• Can be used on working UI or sketches. 

HCI - Week 3 - Paper and Video Prototyping & Mock-ups

Week 3 lectures cover: 

• Paper Prototypes & Mock-ups
• Wizard-Of-Oz Prototyping
• Video Prototyping
• Creating & Comparing Alternatives

Story boarding starts with star people and no one should care about their drawing skills. The biggest mistake one can make is by starting the design with the interface. Start with a scene.

Start with a storyboard! Story boards let your think through how one may complete a task! It’s more about the scene. 

Setting

• People Involved
• Environment
• Task being accomplished

Sequence

• What steps are involved?
• What leads someone to use the app?
• What task is being illustrated?

Satisfaction

• What’s motivates people to use this system?
• What does it enable people to accomplish?
• What need does the system fill?

Wizard-Of-Oz Prototyping fakes it to make it! Quick and dirty paper prototypes are the most powerful. The Wizard-Of-Oz technique is “very useful to mimic some kind of Artificial Intelligence in a cheap and quick way.”

• Advantages of Wizards: Quick, cheap, iterative, more “real” than paper, multiple variations, “places the user at the center of development”, fail fast, build vision for application early. 

Making a Wizard-Powered Prototype according to Scott Klemmer.

1. Map out scenarios and application flow - What should happen in response to user behavior?
2. Put together interface “skeletons”
3. Develop “hooks” for wizard input
4. Where and how the wizard will provide input - selecting the next screen, entering text, entering a zone, recognizing speech, etc. Remember that later you’ll need to replace with computer.
5. Rehearse wizard role with a colleague - practice with a friend! What scenarios are most productive, what questions need to be re-written? Get rid of the easy bugs early. 

Types of feedback: Think aloud (speak freely as performing tasks), retrospective (best when think aloud distracts), heuristic evaluation (works with experts too).

Video Prototyping!

• Cheap, fast, communication tool, sets a “spec” for the developers, ties interface design decisions towards user tasks!
• Used throughout design life-cycle.
• Example: Apple’s Knowledge Navigator. 
• Show the whole task : motivation to success outcome at the end.
• Use need finding to set up scenarios in video prototyping.
• Wendy Mackay’s materials on Video Prototyping is helpful. 

Quantity vs. Qaulity

An art class was divided in half. One half was told that the quantity of pieces they create is important; the more you make the better your grade. The other half was told that the quality of one piece is what they will get graded on. 

“While the quantity group was busily churning out piles of work-and learning from their mistakes-the qaulity group had sat theorizing about perfection, and in the end had little more to show for their efforts than grandiose theories and a pile of dead clay.” - Bayles and Orland

People pick one idea early on and stick with it. - Functional Fixation Dunker, 1945

• Parallel prototyping yields better results. Separates ego from artifact. 
• Sharing multiple designs creates a more positive environmental with designers and those who critique their work. 
• Alternative designs also help create a common vocabulary for all those involved.

** Note: These are my notes from the online lecture series, Human Computer Interaction thanks to Stanford University and Scott Klemmer. 

Studying Human Computer Interaction - Week 1 Notes

Prototyping

• Design, implementation, and evaluation.
• “Fail fast so you can succeed sooner”
• Good design = joy!
• Bad design = causes problems, and degrades quality of life.

Design for people

• People’s tasks, goals, values drive development
• Talk to actual experts
• Work with users throughout the process

Prototyping is valuable.

• Embody design hypothesis.
• A reflective conversation with materials to get on the same page with everyone in the room.
• Gain insights
• Goal = feedback
• Question rendered as an artifact. somehting you make to communicate
• Serves as a common ground to serve as something that everyone can physically see and understand together
• “Prototypes nearly always are and ought to be incomplete”
• “Prototyping is a strategy for efficiently dealing with thing that hard to predict”
• Types of prototypes : feel, implementation, role
• Maximize the learning and communication you will get out of a prototype, minimize the time it takes to make it

“The best way to have a good idea is to have a lot of ideas.” -Linus Pauling

Focus on the goals of the design

• Don’t come up with just one deisgn
• What do you hope to achieve with your design idea?
• Set goals early!
• “Sketching user Experiences” - Buxton

How can we measure success?
Why evaluate designs with people?
Can people figure out how to use it?
How does people’s behavior change when we change the interface?
Watch someone use my interface.

Usability Studies

• Surveys
• Feedback from experts
• Users use prototype
• Comparative feedback 
• Participant observation

HCI

• July 1945 - As We May Think
• Memex - all of the world’s information would be available on a deskto
• Grace Hopper - compiler
• Graphical uder interface > the mouse and hypertext