1
Assignments Details
In light of the University guidelines this session the internal assessment will cover 70% of the course marks. As per the controller of exam and dean faculty of social sciences every students need to submit 04 assignments (01 from each unit) for a course to be hand written and then send it through email (scanned or photographed copy of the hand written assignments) to the concerned teacher for evaluation.
For the course code PEB-4003 (Research Methodology and Statistics) every student need to submit four assignments covering one topic from each unit. It should be hand written and reach me through email. The last date for the submission of all the four assignments will be 3rd June, Wednesday, 2020. No assignments will be considered after the due date.
Materials that are copied from the provided learning materials will not be considered.
Quick Look-
Assignments- Total 04 Assignments need to submit Assignments and Course Units- 01 Assignments from each unit
Mode of writing- Hand written
Conversion of assignment- Scanned or click the photo
Mode of submission- Through email (s.khurram.nisar@gmail.com)
2
Compiled Learning Material
Research Methodology and Statistics
Course Code: PEB-4003
Department of Physical Education
A.M.U. 2020
Compiled By: Sayed Khurram Nisar (Ph.D.)
3
Research Methodology Learning Materials
Compiled by: Sayed Khurram Nisar (Ph.D.) Video Links:
Research Methods
http://study.com/academy/lesson/research-methodologies-quantitative-qualitative-mixed-method.html Measures of variability
http://study.com/academy/lesson/measures-of-variability-range-variance-standard-deviation.html Types of sampling techniques
http://study.com/academy/lesson/what-is-sampling-in-research-definition-methods-importance.html http://study.com/academy/lesson/sampling-techniques-in-scientific-investigations.html
Referencing
https://www.youtube.com/watch?v=6RVrhlMUaS8
http://www.powershow.com/view/26a409-NzYzM/The_Harvard_Reference_System_powerpoint_ppt_presentation https://www.youtube.com/watch?v=RdG91lhDseA
Types of data
https://www.youtube.com/watch?v=7bsNWq2A5gI https://www.youtube.com/watch?v=hZxnzfnt5v8 Scales of measurements
https://www.youtube.com/watch?v=KIBZUk39ncI https://www.youtube.com/watch?v=yJpiUHbLKLU Hypothesis
http://study.com/academy/lesson/what-is-a-null-hypothesis-definition-examples.html http://study.com/academy/lesson/alternative-hypothesis-definition-example.html
http://study.com/academy/lesson/formulating-the-research-hypothesis-and-null-hypothesis.html Variables
http://study.com/academy/lesson/research-variables-dependent-independent-control-extraneous-moderator.html Chi-square test
http://study.com/academy/lesson/what-is-a-chi-square-test-definition-example.html https://www.youtube.com/watch?v=ODxEoDyF6RI
z-test
https://www.youtube.com/watch?v=McISiEiXgfE https://www.youtube.com/watch?v=AvjJuCKUIq4 t-test
http://study.com/academy/lesson/what-is-a-t-test-procedure-interpretation-examples.html https://www.youtube.com/watch?v=BlS11D2VL_U
4 https://www.youtube.com/watch?v=3azuAaOJack
Mixed Method
https://slideplayer.com/slide/6250241/
Qualitative research
https://www.youtube.com/watch?v=4PI4h2tmBkw#action=share
https://www.youtube.com/watch?time_continue=5&v=Qq3OiHQ-HCU&feature=emb_title Data Understanding
https://www.youtube.com/watch?v=NOj3eH-qHqk&feature=emb_rel_pause
5
2.1 Advantages, limitations, usages and techniques of qualitative, quantitative and mixed methods of research
-What Are the Advantages of Qualitative Research?
1. Subject materials can be evaluated with greater detail.
There are many time restrictions that are placed on research methods. The goal of a time restriction is to create a measurable outcome so that metrics can be in place. Qualitative research focuses less on the metrics of the data that is being collected and more on the subtleties of what can be found in that information. This allows for the data to have an enhanced level of detail to it, which can provide more opportunities to glean insights from it during examination.
2. Research frameworks can be fluid and based on incoming or available data.
Many research opportunities must follow a specific pattern of questioning, data collection, and information reporting. Qualitative research offers a different approach. It can adapt to the quality of information that is being gathered. If the available data does not seem to be providing any results, the research can immediately shift gears and seek to gather data in a new direction. This offers more opportunities to gather important clues about any subject instead of being confined to a limited and often self-fulfilling perspective.
3. Qualitative research data is based on human experiences and observations.
Humans have two very different operating systems. One is a subconscious method of operation, which is the fast and instinctual observations that are made when data is present. The other operating system is slower and more methodical, wanting to evaluate all sources of data before deciding. Many forms of research rely on the second operating system while ignoring the instinctual nature of the human mind. Qualitative research doesn‟t ignore the gut instinct. It embraces it and the data that can be collected is often better for it.
4. Gathered data has a predictive quality to it.
One of the common mistakes that occurs with qualitative research is an assumption that a personal perspective can be extrapolated into a group perspective. This is only possible when individuals grow up in similar circumstances, have similar perspectives about the world, and operate with similar goals. When these groups can be identified, however, the gathered individualistic data can have a predictive quality for those who are in a like- minded group. At the very least, the data has a predictive quality for the individual from whom it was gathered.
6
5. Qualitative research operates within structures that are fluid.
Because the data being gathered through this type of research is based on observations and experiences, an experienced researcher can follow-up interesting answers with additional questions. Unlike other forms of research that require a specific framework with zero deviation, researchers can follow any data tangent which makes itself known and enhance the overall database of information that is being collected.
6. Data complexities can be incorporated into generated conclusions.
Although our modern world tends to prefer statistics and verifiable facts, we cannot simply remove the human experience from the equation. Different people will have remarkably different perceptions about any statistic, fact, or event. This is because our unique experiences generate a different perspective of the data that we see. These complexities, when gathered into a singular database, can generate conclusions with more depth and accuracy, which benefits everyone.
7. Qualitative research is an open-ended process.
When a researcher is properly prepared, the open-ended structures of qualitative research make it possible to get underneath superficial responses and rational thoughts to gather information from an individual‟s emotional response. This is critically important to this form of researcher because it is an emotional response which often drives a person‟s decisions or influences their behavior.
8. Creativity becomes a desirable quality within qualitative research.
It can be difficult to analyze data that is obtained from individual sources because many people subconsciously answer in a way that they think someone wants. This desire to
“please” another reduces the accuracy of the data and suppresses individual creativity. By embracing the qualitative research method, it becomes possible to encourage respondent creativity, allowing people to express themselves with authenticity. In return, the data collected becomes more accurate and can lead to predictable outcomes.
9. Qualitative research can create industry-specific insights.
Brands and businesses today need to build relationships with their core demographics to survive. The terminology, vocabulary, and jargon that consumers use when looking at products or services is just as important as the reputation of the brand that is offering them. If consumers are receiving one context, but the intention of the brand is a different context, then the miscommunication can artificially restrict sales opportunities.
Qualitative research gives brands access to these insights so they can accurately communicate their value propositions.
10. Smaller sample sizes are used in qualitative research, which can save on costs.
7
Many qualitative research projects can be completed quickly and on a limited budget because they typically use smaller sample sizes that other research methods. This allows for faster results to be obtained so that projects can move forward with confidence that only good data is able to provide.
11. Qualitative research provides more content for creatives and marketing teams.
When your job involves marketing, or creating new campaigns that target a specific demographic, then knowing what makes those people can be quite challenging. By going through the qualitative research approach, it becomes possible to congregate authentic ideas that can be used for marketing and other creative purposes. This makes communication between the two parties to be handled with more accuracy, leading to greater level of happiness for all parties involved.
12. Attitude explanations become possible with qualitative research.
Consumer patterns can change on a dime sometimes, leaving a brand out in the cold as to what just happened. Qualitative research allows for a greater understanding of consumer attitudes, providing an explanation for events that occur outside of the predictive matrix that was developed through previous research. This allows the optimal brand/consumer relationship to be maintained.
What Are the Disadvantages of Qualitative Research?
1. The quality of the data gathered in qualitative research is highly subjective.
This is where the personal nature of data gathering in qualitative research can also be a negative component of the process. What one researcher might feel is important and necessary to gather can be data that another researcher feels is pointless and won‟t spend time pursuing it. Having individual perspectives and including instinctual decisions can lead to incredibly detailed data. It can also lead to data that is generalized or even inaccurate because of its reliance on researcher subjectivisms.
2. Data rigidity is more difficult to assess and demonstrate.
Because individual perspectives are often the foundation of the data that is gathered in qualitative research, it is more difficult to prove that there is rigidity in the information that is collective. The human mind tends to remember things in the way it wants to remember them. That is why memories are often looked at fondly, even if the actual events that occurred may have been somewhat disturbing at the time. This innate desire to look at the good in things makes it difficult for researchers to demonstrate data validity.
3. Mining data gathered by qualitative research can be time consuming.
8
The number of details that are often collected while performing qualitative research are often overwhelming. Sorting through that data to pull out the key points can be a time- consuming effort. It is also a subjective effort because what one researcher feels is important may not be pulled out by another researcher. Unless there are some standards in place that cannot be overridden, data mining through a massive number of details can almost be more trouble than it is worth in some instances.
4. Qualitative research creates findings that are valuable, but difficult to present.
Presenting the findings which come out of qualitative research is a bit like listening to an interview on CNN. The interviewer will ask a question to the interviewee, but the goal is to receive an answer that will help present a database which presents a specific outcome to the viewer. The goal might be to have a viewer watch an interview and think, “That‟s terrible. We need to pass a law to change that.” The subjective nature of the information, however, can cause the viewer to think, “That‟s wonderful. Let‟s keep things the way they are right now.” That is why findings from qualitative research are difficult to present. What a research gleans from the data can be very different from what an outside observer gleans from the data.
5. Data created through qualitative research is not always accepted.
Because of the subjective nature of the data that is collected in qualitative research, findings are not always accepted by the scientific community. A second independent qualitative research effort which can produce similar findings is often necessary to begin the process of community acceptance.
6. Researcher influence can have a negative effect on the collected data.
The quality of the data that is collected through qualitative research is highly dependent on the skills and observation of the researcher. If a researcher has a biased point of view, then their perspective will be included with the data collected and influence the outcome.
There must be controls in place to help remove the potential for bias so the data collected can be reviewed with integrity. Otherwise, it would be possible for a researcher to make any claim and then use their bias through qualitative research to prove their point.
7. Replicating results can be very difficult with qualitative research.
The scientific community wants to see results that can be verified and duplicated to accept research as factual. In the world of qualitative research, this can be very difficult to accomplish. Not only do you have the variability of researcher bias for which to account within the data, but there is also the informational bias that is built into the data itself from the provider. This means the scope of data gathering can be extremely limited, even if the structure of gathering information is fluid, because of each unique perspective.
8. Difficult decisions may require repetitive qualitative research periods.
9
The smaller sample sizes of qualitative research may be an advantage, but they can also be a disadvantage for brands and businesses which are facing a difficult or potentially controversial decision. A small sample is not always representative of a larger population demographic, even if there are deep similarities with the individuals involve. This means a follow-up with a larger quantitative sample may be necessary so that data points can be tracked with more accuracy, allowing for a better overall decision to be made.
9. Unseen data can disappear during the qualitative research process.
The amount of trust that is placed on the researcher to gather, and then draw together, the unseen data that is offered by a provider is enormous. The research is dependent upon the skill of the researcher being able to connect all the dots. If the researcher can do this, then the data can be meaningful and help brands and progress forward with their mission. If not, there is no way to alter course until after the first results are received. Then a new qualitative process must begin.
10. Researchers must have industry-related expertise.
You can have an excellent researcher on-board for a project, but if they are not familiar with the subject matter, they will have a difficult time gathering accurate data. For qualitative research to be accurate, the interviewer involved must have specific skills, experiences, and expertise in the subject matter being studied. They must also be familiar with the material being evaluated and have the knowledge to interpret responses that are received. If any piece of this skill set is missing, the quality of the data being gathered can be open to interpretation.
11. Qualitative research is not statistically representative.
The one disadvantage of qualitative research which is always present is its lack of statistical representation. It is a perspective-based method of research only, which means the responses given are not measured. Comparisons can be made and this can lead toward the duplication which may be required, but for the most part, quantitative data is required for circumstances which need statistical representation and that is not part of the qualitative research process.
The advantages and disadvantages of qualitative research make it possible to gather and analyze individualistic data on deeper levels. This makes it possible to gain new insights into consumer thoughts, demographic behavioral patterns, and emotional reasoning processes. When a research can connect the dots of each information point that is gathered, the information can lead to personalized experiences, better value in products and services, and ongoing brand development.
https://www.youtube.com/watch?time_continue=31&v=sQ8_P69uMjg https://vittana.org/23-advantages-and-disadvantages-of-qualitative-research
https://www.youtube.com/watch?v=7m0LVHK8a94&index=6&list=PLqHnHG5X2PXCsCMyN3_Ezug AF7GKN2poQ
10
What are the Advantages of Quantitative Research?
Advantages
 Can be used when large quantities of data need to be collected.  The result is usually numerical (quantifiable) and hence considered more “objective”.  The data is considered quantifiable and usually generalizable to a larger population.  It can allow SFCG to see changes overtime and help develop quantitative indicators.  It can provide a clear, quantitative measure to be used for grants and proposals.
Advantages of Quantitative Research
1. Provides estimates of populations at large. 2. Indicates the extensiveness of attitudes held by people. 3. Provides results which can be condensed to statistics. 4. Allows for statistical comparison between various groups. 5. Has precision, is definitive and standardized. 6.
Measures level of occurrence, actions, trends, etc. 7. Can answer such questions as "How many?" and "How often?"
What are the Disadvantages of Quantitative Research?
Disadvantages
ď‚· Results need to be calculated using Excel, Access, or data analysis software (such as SPSS), which may not always be accessible to a country program. ď‚· Time consuming, as the researcher or SFCG team member needs to enter, clean and then analyse the data. ď‚· The larger the sample, the more time it takes to analyse the data and analyse results. ď‚· The larger the sample the more time it takes to collect data. ď‚· The quantitative data ignores a very important human element.
Improper representation of the target population
As mentioned in the article, improper representation of the target population might hinder the researcher for achieving its desired aims and objectives. Despite of applying appropriate sampling plan representation of the subjects is dependent on the probability distribution of observed data. This may led to miscalculation of probability distribution and lead to falsity in proposition.
For example, a study purports to check the proportion of female aged between 20-30 years are applying make-up ranges of international brands. The target population in this case is the women belonging to the said age group, with both professional and non-professional backgrounds, residing in Delhi. The sampled population based on the probability distribution has to be calculated against the total females residing in the city (e.g. 400 sampled out 7,800,615 female populations). However, there is a scope of getting partial information about the range of makeup products from the sampled, owing to its meagre form against the total population.
Hence, the results of the study cannot be generalised in context to a larger population, but rather be suggested.
11 Lack of resources for data collection
Quantitative research methodology usually requires a large sample size. However due to the lack of resources this large-scale research becomes impossible. In many developing countries, interested parties (e.g., government or non-government organisations, public service providers, educational institutions, etc.) may lack knowledge and especially the resources needed to conduct a thorough quantitative research (Science 2001).
Inability to control the environment
Sometimes researchers face problems to control the environment where the respondents provide answers to the questions in the survey (Baxter 2008). Responses often depend on particular time which again is dependent on the conditions occurring during that particular time frame.
For example, if data for a study is collected on residents’ perception of development works conducted by the municipality, the results presented for a specific year (say, 2009), will be held redundant or of limited value in 2015. Reasons being, either the officials have changed or the development scenario have changed (from too effective to minimal effective or vice versa).
Limited outcomes in a quantitative research
Quantitative research method involves structured questionnaire with close ended questions. It leads to limited outcomes outlined in the research proposal. So the results cannot always represent the actual occurring, in a generalised form. Also, the respondents have limited options of responses, based on the selection made by the researcher.
For example, answer to a question– “Does your manager motivates you to take up challenges”;
can be yes/no/can’t say or Strongly Agree to strongly disagree. But to know what are the strategies applied by the manager to motivate the employee or on what parameters the employee does not feel motivated (if responded no), the researcher has to ask broader questions which somewhat has limited scope in close-ended questionnaires
Expensive and time consuming
Quantitative research is difficult, expensive and requires a lot of time to be perform the analysis.
This type of research is planned carefully in order to ensure complete randomization and correct designation of control groups (Morgan 1980). A large proportion of respondents is appropriate for the representation of the target population. So, as to achieve in-depth responses on an issue, data collection in quantitative research methodology is often too expensive as against qualitative approach.
For example, to understand the influence of advertising on the propensity of purchase decision of baby foods parents of 5-year old and below of Bangalore, the researcher needs collect data from 200 respondents. This is time consuming and expensive, given the approach needed to each of these parents to explain the study purpose.
12 Difficultly in data analysis
Quantitative study requires extensive statistical analysis, which can be difficult to perform for researchers from non- statistical backgrounds. Statistical analysis is based on scientific discipline and hence difficult for non-mathematicians to perform.
Quantitative research is a lot more complex for social sciences, education, anthropology and psychology. Effective response should depend on the research problem rather than just a simple yes or no response.
For example, to understand the level of motivation perceived by Grade 5 students from the teaching approach taken by their class teachers, mere yes and no might lead to ambiguity in data collection and hence improper results. Instead a detailed interview or focus group technique might develop in-depth views and perspectives of both the teachers and children.
Requirement of extra resources to analyse the results
The requirements for the successful statistical confirmation of result is very tough in a quantitative research. Hypothesis is proven with few experiments due to which there is ambiguity in the results. Results are retested and refined several times for an unambiguous conclusion (Ong 2003). So it requires extra time, investment and resources to refine the results.
13
What are the Advantages and Disadvantages of Mixed methods of Research?
Strengths
The third and more preferred method of research has an array of strengths that appeal to many researchers. To begin with, the combined strengths of both quantitative and qualitative research can be found when using this method of research. Further, terms, pictures and narratives can be used to add connotation to numbers. In addition, while using mixed methods of research, researchers have the advantage of using numbers to add precision to words, pictures, and narratives. Another advantage of applying the mixed method in research is that researchers can generate and actually test a grounded theory.
Applying the mixed method of research allows the researcher to tackle a broader and a more complete range of research questions owing to the fact that the researcher is not confined within the tenets of a particular method of research. In addition, researchers have the ability to use the strength of one method of research to counter or overcome the weaknesses in another method. In other words it incorporates the concept of complementarity.
In the advent of a researcher conjuring up a conclusion under this method of research, they are in a better position to provide stronger evidence in the conclusion bit through convergence and collaboration of findings. Furthermore, the method of research allows the researcher to add insights and methods that might be omitted when only a single method is adopted. Similarly, the method allows the researcher to simplify to increase the simplicity of the results. Finally, since the mixed methods of research is all about the incorporation of both qualitative and quantitative methods of research, the researcher can produce more complete knowledge necessary to inform theory and practice.
Weaknesses
Unfortunately, this method of research also has a few shortcomings despite its overwhelming support from researchers. Firstly, owing to its duplicity content, the application of the mixed methodology in one study can prove difficult to handle by any one single researcher. This is the case especially when the researcher has two apply two or more approaches concurrently.
Furthermore, a researcher choosing to rely on this method of research has to learn about multiple methods and approaches and understand how to appropriately mix them. Similarly, a lot of researchers are of the view that any one researcher should work within either the qualitative or the quantitative method. Moreover, the mixed method of research is more expensive and time consuming than any other method of research due to its duplicity content. Finally, since it is a mixture of two relatively different methods of research, a lot of researchers and methodologists have as yet to fully workout problems of interpreting conflicting results, quantitative data and the paradigm mixing.
14 Strengths of the Mixed Methods Research
1. Mixed methods research can be used to increase the generalising of the results.
2. Researcher can provide stronger evidence for a conclusion through convergence and corroboration of findings.
3. Researcher can add insights and understanding that might be missed only a single method is used.
4. Researcher can answer a broader and more complete range of research question because he is not confined to single method or approach.
5. Words, pictures and narrative can be used to add meaning to numbers and numbers can be used to add precision to words and narrative.
6. Provides the strengths of both qualitative and quantitative research.
7. Lastly, qualitative and quantitative research used together produce more complete knowledge necessary to inform theory and practice.
Weaknesses Of Mixed Method Research
1. Researcher has to learn about multiple methods and approaches and understand how to mix them appropriately.
2. It can be difficult for a single researcher to carry out both qualitative and quantitative research, especially if two or more approaches are expected to be used concurrently, it may require a research team.
3. It is more time consuming.
4. Methodological purists contend that one should always work within either a qualitative or a quantitative paradigm.
15
16
Which method is used for what?
17
18
A rule of thumb for deciding whether to use qualitative or quantitative data is:
ď‚· Use quantitative research if you want to confirm or test something (a theory or hypothesis)
19
ď‚· Use qualitative research if you want to understand something (concepts, thoughts, experiences)
For most research topics you can choose a qualitative, quantitative or mixed methods approach.
Which type you choose depends on, among other things, whether you‟re taking an inductive vs.
deductive research approach; your research question(s); whether you‟re doing experimental, correlational, or descriptive research; and practical considerations such as time, money, availability of data, and access to respondents.
Research question How satisfied are students with their studies?
Quantitative research approach
You survey 300 students at your university and ask them questions such as: “on a scale from 1-5, how satisfied are your with your professors?”
You can perform statistical analysis on the data and draw conclusions such as: “on average students rated their professors 4.4”.
Qualitative research approach
You conduct in-depth interviews with 15 students and ask them open-ended questions such as:
“How satisfied are you with your studies?”, “What is the most positive aspect of your study program?” and “What can be done to improve the study program?”
Based on the answers you get you can ask follow-up questions to clarify things. You transcribe all interviews using transcription software and try to find commonalities and patterns.
Mixed methods approach
You conduct interviews to find out how satisfied students are with their studies. Through open- ended questions you learn things you never thought about before and gain new insights. Later, you use a survey to test these insights on a larger scale.
It‟s also possible to start with a survey to find out the overall trends, followed by interviews to better understand the reasons behind the trends.
How to analyze qualitative and quantitative data
Qualitative or quantitative data by itself can‟t prove or demonstrate anything, but has to be analyzed to show its meaning in relation to the research questions. The method of analysis differs for each type of data.
20 Analyzing quantitative data
Quantitative data is based on numbers. Simple math or more advanced statistical analysis is used to discover commonalities or patterns in the data. The results are often reported in graphs and tables.
Applications such as Excel, SPSS, or R can be used to calculate things like:
ď‚· Average scores
ď‚· The number of times a particular answer was given
ď‚· The correlation or causation between two or more variables
ď‚· The reliability and validity of the results Analyzing qualitative data
Qualitative data is more difficult to analyze than quantitative data. It consists of text, images or videos instead of numbers.
Some common approaches to analyzing qualitative data include:
ď‚· Qualitative content analysis: Tracking the occurrence, position and meaning of words or phrases
 Thematic analysis: Closely examining the data to identify the main themes Qualitative research can help you…
ď‚· Develop hypotheses for further testing and for quantitative questionnaire development,
ď‚· Understand the feelings, values, and perceptions that underlie and influence behavior
ď‚· Identify customer needs
ď‚· Capture the language and imagery customers use to describe and relate to a product, service, brand, etc.
ď‚· Perceptions of marketing/communication messages
ď‚· Information obtained in quantitative study and to better understand the context/meaning of the data
ď‚· Generate ideas for improvements and/or extensions of a product, line, or brand
ď‚· Uncover potential strategic directions for branding or communications programs
ď‚· Understand how people perceive a marketing message or communication piece
ď‚· Develop parameters (i.e., relevant questions, range of responses) for a quantitative study Situations where qualitative research is often used:
ď‚· New product idea generation and development
21
ď‚· Investigating current or potential product/service/brand positioning and marketing strategy
ď‚· Strengths and weaknesses of products/brands
ď‚· Understanding dynamics of purchase decision dynamics
ď‚· Studying reactions to advertising and public relations campaigns, other marketing communications, graphic identity/branding, package design, etc.
ď‚· Exploring market segments, such as demographic and customer groups
ď‚· Studying emotions and attitudes on societal and public affairs issues
ď‚· Assessing the usability of websites or other interactive products or services
ď‚· Understanding perceptions of a company, brand, category and product
 Determining consumer language as a preliminary step to develop a quantitative survey Do not expect qualitative research to…
ď‚· Count, measure or offer statistical validation
ď‚· Determine the best product concept or price point; or establish the importance of specific customer needs or satisfaction criteria
ď‚· Be a substitute for quantitative research because of time and/or budgetary constraints when quantitative evaluation is critical
Choosing the right method to inform design decisions
Research methods are not interchangeable and have specific objectives which they are most suitable for. Choosing the most applicable method will depend on what design decisions need to be made for the creation or refinement of a design.
As a rule of thumb, use qualitative research methods for exploring ideas, designs, or processes which do not require concrete hypotheses or structured, statistically valid feedback. I have found that qualitative methods are particularly useful for serendipitous discovery, and often provide more in-depth insights.
You can use quantitative research methods such as A/B testing for validating or choosing a design based on user satisfaction scores, perceived usability measures, and/or task performance.
You will find the insight will be more superficial, but the data is statistically valid and can be generalised to the entire user population.
A combination of qualitative and quantitative research is typically best for most design projects if budget allows. By using both methods you can achieve a deeper level of insight through the exploratory nature of the research in addition to statistical evidence to support your design decisions. Using qualitative and quantitative research together answers to the; Who, What, When, Where, Why, and How, of user behaviours and experiences.
22
Major 5 Qualitative Methods
23 METHODS OF DATA COLLECTION
We‟ll now take a look at the different methods or tools used to collect data, and some of their pros (+) and cons (-). You may notice some methods falling under both categories, which means that they can be used in gathering both types of data.
I. Qualitative Data Collection Methods
Exploratory in nature, these methods are mainly concerned at gaining insights and understanding on underlying reasons and motivations, so they tend to dig deeper. Since they cannot be quantified, measurability becomes an issue. This lack of measurability leads to the preference for methods or tools that are largely unstructured or, in some cases, maybe structured but only to a very small, limited extent.
Generally, qualitative methods are time-consuming and expensive to conduct, and so researchers try to lower the costs incurred by decreasing the sample size or number of respondents.
Face-to-Face Personal Interviews
This is considered to be the most common data collection instrument for qualitative research, primarily because of its personal approach. The interviewer will collect data directly from the subject (the interviewee), on a one-on-one and face-to-face interaction. This is ideal for when data to be obtained must be highly personalized.
24
The interview may be informal and unstructured – conversational, even – as if taking place between two casual to close friends. The questions asked are mostly unplanned and spontaneous, with the interviewer letting the flow of the interview dictate the next questions to be asked.
However, if the interviewer still wants the data to be standardized to a certain extent for easier analysis, he could conduct a semi-structured interview where he asks the same series of open- ended questions to all the respondents. But if they let the subject choose her answer from a set of options, what just took place is a closed, structured and fixed-response interview.
ď‚· (+) This allows the interviewer to probe further, by asking follow-up questions and getting more information in the process.
ď‚· (+) The data will be highly personalized (particularly when using the informal approach).
ď‚· (-) This method is subject to certain limitations, such as language barriers, cultural differences, and geographical distances.
ď‚· (-) The person conducting the interview must have very good interviewing skills in order to elicit responses.
Qualitative Surveys
 Paper surveys or questionnaires. Questionnaires often utilize a structure comprised of short questions and, in the case of qualitative questionnaires, they are usually open- ended, with the respondents asked to provide detailed answers, in their own words. It‟s almost like answering essay questions.
o (+) Since questionnaires are designed to collect standardized data, they are ideal for use in large populations or sample sizes of respondents.
o (+) The high amount of detail provided will aid analysis of data.
o (-) On the other hand, the large number of respondents (and data), combined with the high level and amount of detail provided in the answers, will make data analysis quite tedious and time-consuming.
ď‚· Web-based questionnaires. This is basically a web-based or internet-based survey, involving a questionnaire uploaded to a site, where the respondents will log into and accomplish electronically. Instead of a paper and a pen, they will be using a computer screen and the mouse.
o (+) Data collection is definitely quicker. This is often due to the questions being shorter, requiring less detail than in, say, a personal interview or a paper questionnaire.
o (+) It is also uncomplicated, since the respondents can be invited to answer the questionnaire by simply sending them an email containing the URL of the site where the online questionnaire is available for answering.
o (-) There is a limitation on the respondents, since the only ones to be able to answer are those who own a computer, have internet connection, and know their way around answering online surverys.
o (-) The lesser amount of detail provided means the researcher may end up with mostly surface data, and no depth or meaning, especially when the data is processed.
25 Focus Groups
Focus groups method is basically an interview method, but done in a group discussion setting.
When the object of the data is behaviors and attitudes, particularly in social situations, and resources for one-on-one interviews are limited, using the focus group approach is highly recommended. Ideally, the focus group should have at least 3 people and a moderator to around 10 to 13 people maximum, plus a moderator.
Depending on the data being sought, the members of the group should have something in common. For example, a researcher conducting a study on the recovery of married mothers from alcoholism will choose women who are (1) married, (2) have kids, and (3) recovering alcoholics.
Other parameters such as the age, employment status, and income bracketdo not have to be similar across the members of the focus group.
The topic that data will be collected about will be presented to the group, and the moderator will open the floor for a debate.
ď‚· (+) There may be a small group of respondents, but the setup or framework of data being delivered and shared makes it possible to come up with a wide variety of answers.
ď‚· (+) The data collector may also get highly detailed and descriptive data by using a focus group.
ď‚· (-) Much of the success of the discussion within the focus group lies in the hands of the moderator. He must be highly capable and experienced in controlling these types of interactions.
Documental Revision
This method involves the use of previously existing and reliable documents and other sources of information as a source of data to be used in a new research or investigation. This is likened to how the data collector will go to a library and go over the books and other references for information relevant to what he is currently researching on.
ď‚· (+) The researcher will gain better understanding of the field or subject being looked into, thanks to the reliable and high quality documents used as data sources.
ď‚· (+) Taking a look into other documents or researches as a source will provide a glimpse of the subject being looked into from different perspectives or points of view, allowing comparisons and contrasts to be made.
ď‚· (-) Unfortunately, this relies heavily on the quality of the document that will be used, and the ability of the data collector to choose the right and reliable documents. If he chooses wrong, then the quality of the data he will collect later on will be compromised.
Observation
In this method, the researcher takes a participatory stance, immersing himself in the setting where his respondents are, and generally taking a look at everything, while taking down notes.
26
Aside from note-taking, other documentation methods may be used, such as video and audio recording, photography, and the use of tangible items such as artifacts, mementoes, and other tools.
ď‚· (+) The participatory nature may lead to the researcher getting more reliable information.
ď‚· (+) Data is more reliable and representative of what is actually happening, since they took place and were observed under normal circumstances.
ď‚· (-) The participation may end up influencing the opinions and attitudes of the researcher, so he will end up having difficulty being objective and impartial as soon as the data he is looking for comes in.
 (-) Validity may arise due to the risk that the researcher‟s participation may have an impact on the naturalness of the setting. The observed may become reactive to the idea of being watched and observed. If he planned to observe recovering alcoholic mothers in their natural environment (e.g. at their homes with their kids), their presence may cause the subjects to react differently, knowing that they are being observed. This may lead to the results becoming impaired.
Longitudinal studies
This is a research or data collection method that is performed repeatedly, on the same data sources, over an extended period of time. It is an observational research method that could even cover a span of years and, in some cases, even decades. The goal is to find correlations through an empirical or observational study of subjects with a common trait or characteristic.
An example of this is the Terman Study of the Gifted conducted by Lewis Terman at Stanford University. The study aimed to gather data on the characteristics of gifted children – and how they grow and develop – over their lifetime. Terman started in 1921, and it extended over the lifespan of the subjects, more than 1,500 boys and girls aged 3 to 19 years old, and with IQs higher than 135. To this day, this study is the world‟s “oldest and longest-running” longitudinal study.
 (+) This is ideal when seeking data meant to establish a variable‟s pattern over a period of time, particularly over an extended period of time.
ď‚· (+) As a method to find correlations, it is effective in finding connections and relationships of cause and effect.
ď‚· (-) The long period may become a setback, considering how the probability of the subjects at the beginning of the research will still be complete 10, 20, or 30 years down the road is very low.
ď‚· (-) Over the extended period, attitudes and opinions of the subjects are likely to change, which can lead to the dilution of data, reducing their reliability in the process.
Case Studies
In this qualitative method, data is gathered by taking a close look and an in-depth analysis of a
“case study” or “case studies” – the unit or units of research that may be an individual, a group of
27
individuals, or an entire organization. This methodology‟s versatility is demonstrated in how it can be used to analyze both simple and complex subjects.
However, the strength of a case study as a data collection method is attributed to how it utilizes other data collection methods, and captures more variables than when a single methodology is used. In analyzing the case study, the researcher may employ other methods such as interviewing, floating questionnaires, or conducting group discussions in order to gather data.
ď‚· (+) It is flexible and versatile, analyzing both simple and complex units and occurrence, even over a long period of time.
ď‚· (+) Case studies provide in-depth and detailed information, thanks to how it captures as many variables as it can.
ď‚· (-) Reliability of the data may be put at risk when the case study or studies chosen are not representative of the sample or population.
Types of Quantitative Methods
28
29
30
31 Quantitative Data Collection Methods
Data can be readily quantified and generated into numerical form, which will then be converted and processed into useful information mathematically. The result is often in the form of statistics that is meaningful and, therefore, useful. Unlike qualitative methods, these quantitative techniques usually make use of larger sample sizes because its measurable nature makes that possible and easier.
Quantitative Surveys
Unlike the open-ended questions asked in qualitative questionnaires, quantitative paper surveys pose closed questions, with the answer options provided. The respondents will only have to choose their answer among the choices provided on the questionnaire.
ď‚· (+) Similarly, these are ideal for use when surveying large numbers of respondents.
ď‚· (+) The standardized nature of questionnaires enable researchers to make generalizations out of the results.
ď‚· (-) This can be very limiting to the respondents, since it is possible that his actual answer to the question may not be in the list of options provided on the questionnaire.
ď‚· (-) While data analysis is still possible, it will be restricted by the lack of details.
Interviews
Personal one-on-one interviews may also be used for gathering quantitative data. In collecting quantitative data, the interview is more structured than when gathering qualitative data, comprised of a prepared set of standard questions.
These interviews can take the following forms:
ď‚· Face-to-face interviews: Much like when conducting interviews to gather qualitative data, this can also yield quantitative data when standard questions are asked.
o (+) The face-to-face setup allows the researcher to make clarifications on any answer given by the interviewee.
o (-) This can be quite a challenge when dealing with a large sample size or group of interviewees. If the plan is to interview everyone, it is bound to take a lot of time, not to mention a significant amount of money.
ď‚· Telephone and/or online, web-based interviews. Conducting interviews over the telephone is no longer a new concept. Rapidly rising to take the place of telephone interviews is the video interview via internet connection and web-based applications, such as Skype.
o (+) The net for data collection may be cast wider, since there is no need to travel through distances to get the data. All it takes is to pick up the phone and dial a number, or connect to the internet and log on to Skype for a video call or video conference.
o (-) Quality of the data may be questionable, especially in terms of impartiality.
The net may be cast wide, but it will only be targeting a specific group of
32
subjects: those with telephones and internet connections and are knowledgeable about using such technologies.
ď‚· Computer-assisted interviews. This is called CAPI, or Computer-Assisted Personal Interviewing where, in a face-to-face interview, the data obtained from the interviewee will be entered directly into a database through the use of a computer.
o (+) The direct input of data saves a lot of time and other resources in converting them into information later on, because the processing will take place immediately after the data has been obtained from the source and entered into the database.
o (-) The use of computers, databases and related devices and technologies does not come cheap. It also requires a certain degree of being tech-savvy on the part of the data gatherer.
Quantitative Observation
This is straightforward enough. Data may be collected through systematic observation by, say, counting the number of users present and currently accessing services in a specific area, or the number of services being used within a designated vicinity.
When quantitative data is being sought, the approach is naturalistic observation, which mostly involves using the senses and keen observation skills to get data about the “what”, and not really about the “why” and “how”.
ď‚· (+) It is a quite simple way of collecting data, and not as expensive as the other methods.
ď‚· (-) The problem is that senses are not infallible. Unwittingly, the observer may have an unconscious grasp on his senses, and how they perceive situations and people around.
Bias on the part of the observer is very possible.
Experiments
Have you ever wondered where clinical trials fall? They are considered to be a form of experiment, and are quantitative in nature. These methods involve manipulation of an independent variable, while maintaining varying degrees of control over other variables, most likely the dependent ones. Usually, this is employed to obtain data that will be used later on for analysis of relationships and correlations.
Quantitative researches often make use of experiments to gather data, and the types of experiments are:
ď‚· Laboratory experiments. This is your typical scientific experiment setup, taking place within a confined, closed and controlled environment (the laboratory), with the data collector being able to have strict control over all the variables. This level of control also implies that he can fully and deliberately manipulate the independent variable.
 Field experiments. This takes place in a natural environment, “on field” where, although the data collector may not be in full control of the variables, he is still able to do so up to
33
a certain extent. Manipulation is still possible, although not as deliberate as in a laboratory setting.
ď‚· Natural experiments. This time, the data collector has no control over the independent variable whatsoever, which means it cannot be manipulated. Therefore, what can only be done is to gather data by letting the independent variable occur naturally, and observe its effects.
You can probably name several other data collection methods, but the ones discussed are the most commonly used approaches. At the end of the day, the choice of a collection method is only 50% of the whole process. The correct usage of these methods will also have a bearing on the quality and integrity of the data being sought.
34 Types of Mixed Methods
35 Mixed Methods Research Designs
The field of mixed methods has only been widely accepted for the last decade, though researchers have long been using multiple methods, just not calling them “mixed.” Mixed methods research takes advantage of using multiple ways to explore a research problem.
Basic Characteristics
ď‚· Design can be based on either or both perspectives.
ď‚· Research problems can become research questions and/or hypotheses based on prior literature, knowledge, experience, or the research process.
ď‚· Sample sizes vary based on methods used.
ď‚· Data collection can involve any technique available to researchers.
ď‚· Interpretation is continual and can influence stages in the research process.
Why Use Mixed Methods?
The simple answer is to overcome the limitations of a single design. A detailed answer involves:
ď‚· To explain and interpret.
ď‚· To explore a phenomenon.
ď‚· To develop and test a new instrument.
ď‚· To serve a theoretical perspective.
ď‚· To complement the strengths of a single design.
ď‚· To overcome the weaknesses of a single design.
ď‚· To address a question at different levels.
ď‚· To address a theoretical perspective at different levels.
What are some strengths?
ď‚· Can be easy to describe and to report.
ď‚· Can be useful when unexpected results arise from a prior study.
ď‚· Can help generalize, to a degree, qualitative data.
ď‚· Helpful in designing and validating an instrument.
ď‚· Can position research in a transformative framework.
What are some weaknesses?
ď‚· Time required.
ď‚· Resolving discrepancies between different types of data.
ď‚· Some designs generate unequal evidence.
ď‚· Can be difficult to decide when to proceed in sequential designs.
ď‚· Little guidance on transformative methods.
Methodologist John Creswell suggested a systematic framework for approaching mixed methods research. His framework involves four decisions to consider and six strategies.
36
Four Decisions for Mixed Method Designs (Creswell, 2003, p. 211) 1. What is the implementation sequence of data collection?
2. What method takes priority during data collection and analysis?
3. What does the integration stage of finding involve?
4. Will a theoretical perspective be used?
Six Mixed Methods Design Strategies (Creswell, 2003) 1. Sequential Explanatory
ď‚· Characterized by: Collection and analysis of quantitative data followed by a collection and analysis of qualitative data.
ď‚· Purpose: To use qualitative results to assist in explaining and interpreting the findings of a quantitative study.
2. Sequential Exploratory
ď‚· Characterized by: An initial phase of qualitative data collection and analysis followed by a phase of quantitative data collection and analysis.
ď‚· Purpose: To explore a phenomenon. This strategy may also be useful when developing and testing a new instrument
3. Sequential Transformative
ď‚· Characterized by: Collection and analysis of either quantitative or qualitative data first.
The results are integrated in the interpretation phase.
ď‚· Purpose: To employ the methods that best serve a theoretical perspective.
4. Concurrent Triangulation
ď‚· Characterized by: Two or more methods used to confirm, cross-validate, or corroborate findings within a study. Data collection is concurrent.
ď‚· Purpose: Generally, both methods are used to overcome a weakness in using one method with the strengths of another.
5. Concurrent Nested
 Characterized by: A nested approach that gives priority to one of the methods and guides the project, while another is embedded or “nested.”
ď‚· Purpose: The purpose of the nested method is to address a different question than the dominant or to seek information from different levels.
37 6. Concurrent Transformative
ď‚· Characterized by: The use of a theoretical perspective reflected in the purpose or research questions of the study to guide all methodological choices.
ď‚· Purpose: To evaluate a theoretical perspective at different levels of analysis.
38
2.2 Literature Sources
39
40
Hypothesis
41
42
43
2.3 Sampling Techniques
Probability sampling techniques- Probability sampling is based on the fact that every member of a population has a known and equal chance of being selected. For example, if you had a population of 100 people, each person would have odds of 1 out of 100 of being chosen.
Probability sampling gives you the best chance to create a sample that is truly representative of the population.
44 https://www.youtube.com/watch?v=eIZD1BFfw8E https://www.youtube.com/watch?v=emJK9nE4rFg https://www.youtube.com/watch?v=pTuj57uXWlk
https://www.surveygizmo.com/resources/blog/probability-sampling/
https://www.statisticshowto.com/probability-sampling/
https://www.youtube.com/watch?v=saO1yLxd1p8 Non- probability sampling techniques-
With non-probability sampling, those odds are not equal. For example, a person might have a better chance of being chosen if they live close to the researcher or have access to a computer.
It's the opposite of probability sampling, where you can calculate the odds. In addition, probability sampling involves random selection, while non-probability sampling does not–it relies on the subjective judgement of the researcher. The odds do not have to be equal for a method to be considered probability sampling.
45
https://www.statisticshowto.com/non-probability-sampling/
https://www.researchgate.net/publication/319066480_Non- Probability_and_Probability_Sampling
See chapter 3 Choosing Between Nonprobability Sampling and Probability Sampling
Sampling Essentials: Practical Guidelines for Making Sampling Choices By: Johnnie Daniel
46 https://www.youtube.com/watch?v=c99B-xO_SiE
47
2.4 Ethical principles for research and designing informed consent (International and Indian- policies and procedures)
The five general principles
Complementing the numerous ethical standards are APA‟s five General Principles of Ethics for Psychologists. Prescriptive/non-enforceable in nature, the general principles are there not to limit and impose on us, but instead to “guide and inspire psychologists toward the very highest ethical ideals of the profession” (p. 3) – be it in their clinical practice, while conducting a study, consulting a company, etc. Here is a concise overview of how we can translate them to research, how respecting them enriches and elevates our practice and how dismissing them may result in tainting an otherwise brilliant and illuminating research:
Principle A: Beneficence and Nonmaleficence – The first principle states that “In their professional actions, psychologists seek to safeguard the welfare and rights of those with whom they interact professionally and other affected persons and the welfare of animal subjects of research”`(p.3) , among other. Many ethical standards are already in place to ensure that externally. In terms of personal consideration, the first principle stresses out the need for researchers to work independently of biases (itself a vast, multifaceted topic that poses an obstacle to quality science making), prejudices, and malignant affiliations and with a clear sense that what they are doing has very often impact on the lives of others. It is thus important for us to have an understanding that biased research affects the public negatively not only through the wide-reaching reports by media, but also by its usage by policymakers and lawmakers and always to stay critical and alert for such possibility.
Principle B: Fidelity and Responsibility – Outlining the value of conscientiousness in the psychological practice and research, the second principle somewhat overlaps with the first one. It differs in the focus it has, moving into an overview of what to mind when working with our colleagues and within our work network. While responsibility is a universally understood value, the principle also states that “.. [psychologists] are concerned about the ethical compliance of their colleagues‟ scientific and professional conduct. Psychologists strive to contribute a portion of their professional time for little or no compensation or personal advantage” (p.3). In research his might translate to us as taking part of the peer-review process, striving to help fellow scientists improve the quality of their work before it enters into circulation. Ethical misconduct should be pointed out whenever we can spot it, but it is always to be done with respect to the researcher who conducted it, as decision-making in relation to ethics is fairly complex and influenced by factors that may lie beyond one‟s control. (Trevino & Youngblood, 1990)
Principle C: Integrity – The third principle summarizes what we are supposed not to do in our practice as researchers. Cases of manipulation, fraud, fabricating results and general scientific misconduct are not unheard of, affecting tremendously the field. A somewhat recent widely publicized case of such lack of integrity is that of Mr. Diederik Stapel, a Dutch social psychologist who fixed the results in over 30 of his papers, some of which were published in prestigious and esteemed journals. (Callaway, 2011) Even though fraud is controlled for and strict sanctions are enforced against it (Mr. Stapel lost not only his reputation, but also his job)
