1 Ethics for Data Science

1.1 Ethical Issues in Data Science

1.1.1 Introduction

Sara Baase, in her classic textbook A Gift of Fire, defines ethics as “the study of what it means to do the right thing” [1]. Michael Quinn [2] further specifies that this “right thing” relates to various situations within a particular society. Here we include a list of broad categories of these various situations and call them ethical issues. We present our working definitions of these ethical issues in the context of the computer and data sciences. Each was created to be a one sentence definition that captures the spirit of what we’ve seen throughout a wide range of texts and conversations.

1.1.2 What is an ethical dilemma?

An ethical dilemma may occur whenever there is a conflict between two or more ethical issues or aspects of the same issue. For example, if your company employs cheap labor overseas they may profit economically, but lack in dignifying and promoting physical/mental health of the workers. As a result of these conflicting issues, the company is forced to make a decision: Should they continue to hire international workers for poor wages to keep a high profit, or should they invest more in their employees at the expense of losing revenue.

1.1.3 Some ethical issues in Data Science

Media Literacy How do we tell whether an article, video, etc. is factual, reliable, and unbiased?

Accessibility Who can use the products and analyses we produce? Does this include people with economic, physical, mental, and emotional disadvantages?

Data Visualization Do our visualizations represent data in a clear, accessible way that accurately reflects the meaning of underlying data or model?

Transparency Do we represent data, algorithms, or any information that affects a stakeholder in an open, explicit, and easily comprehensible format?

Diversity Are a variety of people (race and ethnicity, age, gender, skills and education level, sexual orientation, religion, etc.) contributing to the development of an idea or product in an effort to avoid bias and increase the breadth of insight into product needs?

Sustainability Do we employ practices that protect the environment, including living creatures and other naturally occurring resources and features?

Intellectual Property Who owns the work? Who controls development, dissemination, and pricing? Are all those who worked on the project fairly acknowledged and compensated? Are non-owners also treated fairly?

Decision Making for AI Who is responsible for how an artificially intelligent machine makes (ethical) decisions, especially those involving life-altering actions?

Freedom of Speech What are the rights and limitations of the public to express themselves and correspond through electronic communication? Who enforces the rights and limitations?

Dignity How do we value human and animal life, well-being, and connection relative to other goals and desires?

Physical and Mental Health How do we proactively take care to protect the health of employees (including oneself) and consumers of one’s product?

Professional Citizenship How do we use our professional expertice wisely both among others “in the guild” and with those outside it? What do we say? How must we listen? What guidelines does our profession have for this? Should we follow them?

Reliability What is our obligation to make reliable products (and to measure and assess that reliability)?

Hospitality What is our obligation to make our work easily understandable and usable by peers in a similar field? To those with different expertise and experience?

Privacy and Security How do we balancing the need for protecting personal and group data with the use of that data for the welfare of the public? Who “owns” data about people? How may it be used?

Algorithmic Bias Might our algorithms represent bias (intential or unintential) arising from ignorance, assumptions, or past societal patterns? What do we do about that?

Data Integrity How does they way we handle data affect potential for bias or false results?

Professional Ethics What are our obligations in the workplace toward our co-workers, supervisors, subordinates, the company, shareholders, and stakeholders? What do we do when these are in conflict?

1.2 Simulating Loan Thresholds

1.2.1 A simplified situation

Imagine a machine learning algorithm that takes information about loan applicants and assigns them a credit score from 0 to 100. The credit scores are not perfect, however. Applicants with higher credit scores are more likely to repay their loan, but some people with lower credit scores will repay loans, and some with higher scores will not.

Your task is to devise a system that uses these credit scores to decide who gets a loan.

  1. How will you measure how “good” your system is? What sorts of things could you measure/quantify as indication of how well your system works? We’ll call those things metrics. Come up with at least 3 metrics.
  1. Which of your metrics do you like better? Why?

Now let’s experiment a bit with some simulated data. Go to the “Simulating Loan Thresholds” page at http://research.google.com/bigpicture/attacking-discrimination-in-ml/. The graphs and figures provide metrics, and you can slide the credit score threshold and see how these metrics change. It is based on issues raised Hardt et al. (2016).

  1. What metrics are included on this page? How many of them are the same (or equivalent to)1 the ones you came up with?

  1. What credit score threshold maximizes the number of correct decisions? Does that same threshold also minimize the number of incorrect decisions?

  2. What threshold maximizes profit?

    Note: The app assumes a marginal gain of $300 for each loan repaid and a loss of $700 for each loan default. This is mentioned later in the text, but you can also figure it from the interactive graphic. How?

  3. Are your answers to #4 and #5 the same? Why or why not?

  4. The app doesn’t let you experiment with this, but suppose repaid loans had a marginal benefit of $400 and defaulted loans a marginal cost of only $600.

    1. Would that lead a profit-maximizer to increase or decrease the threshold? Why?

    2. How would the threshold change if you are maximizing correct decisions.

1.2.2 Different groups

Don’t click the red buttons until directed. Instead, just drag the threshold sliders.

  1. The text says: “In this case, the distributions of the two groups are slightly different, even though blue and orange people are equally likely to pay off a loan.”

    1. In what way are the distributions (of credit scores) different for blue people and orange people? (Remember your statistics? You might think about things like center, spread, shape, etc. But remeber that we are comparing distributions of two variables: credit score and whether or not they repay the loan.)

    2. How might you improve the graphical display to make it easier to see how the two groups differ?

    3. Does the credit score do a better job for one group than for the other? What do you mean by better?

  2. Adjust the thresholds to maximize the profit that the bank gets for each group individually. (a) What are the resulting thresholds for the two groups?

  1. At those thresholds, which group is offered more loans? (c) Of the loans that were offered, which group repays more of those loans? (d) What is the resulting profit for the bank? (e) Is that fair? (explain.) (Check your answer by clicking on the red “Max Profit” button.)

  1. Adjust the thresholds to maximize profit under the constraint that blue and orange must have the same threshold. (This is tricky in the UI; hint: it must be between the two thresholds you found in the previous exercise.) Answer the same five questions (a-e) as in the previous exercise. Then check your answer by clicking the “Group Unaware” button.

  2. Starting at the “Group Unaware” thresholds, change the threshold on the orange group to increase its Positive Rate to match the Positive Rate of the blue group. (a) What happens to the True Positive Rate? (b) What happens to the bank’s profit? (c) Is that good? Explain.

  3. Explore the “Demographic Parity” and “Equal Opportunity” objectives by clicking their buttons. The authors of the interactive visualization seem to have a favorite among these objectives. What reasons might someone have to agree or disagree with their assessment?

1.2.3 Thinking about your thinking

There are a lot of value-laden words in the questions above: words like good, better, fair, etc. Compare two ways of awarding loans (from the ones above or from other way you come up with), one of which you think is “better” than the other. This will be more interesting if you choose two that are close rather than one that you think is great and one that you think is terrible – make the decision difficult.

  1. What makes one better than the other? Imagine you had to defend the choice, what would be the main components of your argument?

  2. Are there general principles that would apply to other situations that have nothing to do with loans and credit scores?

  3. If you could wish for any feature(s) to be added to this simulation, what would it/they be?

1.2.4 The article

This simple simulation is a by-product of a paper: Hardt et al. (2016).

Here is the abstract:

We propose a criterion for discrimination against a specified sensitive attribute in supervised learning, where the goal is to predict some target based on available features. Assuming data about the predictor, target, and membership in the protected group are available, we show how to optimally adjust any learned predictor so as to remove discrimination according to our definition. Our framework also improves incentives by shifting the cost of poor classification from disadvantaged groups to the decision maker, who can respond by improving the classification accuracy. We enourage readers to consult the more complete manuscript on the arXiv.

  1. For each of the following words/phrases, say what the correspond to in the simulation we just looked at.

    1. specified sensitive attribute
    2. some target
    3. available features
    4. predictor
    5. membership in the protected group
    6. adjust (any) learned predictor

1.3 Ethical Frameworks and Tests

When making ethical decisions, what do we base them upon? Below are brief descriptions of four frameworks for making ethical decisions. As you have ethical discussions, see if you can identify when you and your conversation partners are using these (or other) frameworks. Keep in mind that it is possilbe for more than one framework to be invovled at the same time. When this happens, see if you can identify which framework has the stronger influence or how the different frameworks are shaping each other.

Bottom line for all frameworks: How would you honestly and rationally defend your decision?

The descriptions below are based on draft materials for ethics in data science being developed by authors at Point Loma Nazarene University.

Some of the test questions were derived from material on the following websites (accessed 6/2019):

1.3.1 Utilitarian Ethics

Basic idea: The utilitarian framework determines the best solution by choosing the course of action which seems the best for the most people affected by the solution (for the most stakeholders).

Ethical tests based on the framework:

  • Who will be affected by this decision? (who are the stakeholders?)
  • Who benefits from this decision?
  • Who will be harmed by this decision?
  • Do the benefits outweigh the harms?
  • What are all the long-term and short-term consequences?
  • Does this option do less harm than the alternatives?

Example: There are five ill patients in a hospital who desperately need organ transplants; however, there are no available organs. By chance, a healthy traveler comes in for a check-up and happens to be a match for each of the five patients. Assuming that there would be no legal repercussions, the doctor could kill the traveler and use his organs to save the 5 lives. The stakeholders would be each of the patients and their families. From a utilitarian perspective, the right thing to do could be to kill the one healthy person in order to save the five, because this option brings about the greatest utility (or happiness) despite causing instrumental harm.

1.3.2 Virtue Ethics

Basic idea: The virtue ethics framework determines the best solution by choosing the solution that is most in agreement with our social virtues (moral values such as honesty, compassion, diversity.)

Ethical tests based on the framework:

  • Does this decision agree or disagree with my moral values?
  • Are any of my virtues in conflict with each other because of this dilemma?
  • Think of someone whose moral judgement you respect, would this decision line up with his or her values?
  • Would a child or your parental figure be proud of your decision?
  • Is this decision in line with my company’s core values?

Example: If you are unsure of whether or not it is right to punish a poor person for stealing food, your virtue of compassion may say that it isn’t correct while your virtue of justice may say that it is.

1.3.3 Ethical Analogies

Basic idea: The analogy approach compares two completely different things that might have similarities in concepts. Generally the concept in the analogy is better understood, and might help in understanding the concept in the other and determining its ethicality.

Ethical tests based on the framework:

  • Can I think of a simpler scenario to compare this situation to?
  • Is this situation similar to anything I have seen before?
  • Can I use these similar scenarios to help clarify or make decisions about the current dilemma?

Example: If you are unsure if it is ethical to hack someone without stealing/manipulating any of their information, you may compare it to breaking into someone’s home and only looking around.

1.3.4 Deontologcal Ethics

Basic idea: The deontological framework is built on a set of rules or principles. While more rigid, it can sometimes clarify the gray areas and subjectivity left by the other frameworks.

Ethical tests based on the framework:

  • Does this decision break any laws?
  • Would I be going against any codes of ethics?
  • What would my profession’s governing body think of this decision?
  • Does my decision go against any implied, unspoken moral laws?
  • If I am setting a standard, would I want it to be followed by other people (including my competitors)?

Example: If you are unsure of whether or not it is ethical to murder someone, you can refer to the US law which says that it is wrong.

1.3.5 Identifying Frameworkds

  1. Return to your discussion of using credit scores to decide who gets a loan. Identify places where the four ethical frameworks were used to evaluate the situation.
  1. The ASA (American Statistical Association) and ACM (Association for Computing Machinery) have each published ethical guidelines for their professions.

Identify the ethical framework(s) being used in these statements.

1.3.6 Additional Resources

1.4 Values Survey

1.4.1 What is a value?

“A value is something you are willing to act on” – Ayn Rand, philosopher

“Values are a matter of what guides you through every day, every task, every encounter with another human being.” – Richard Bolles, What Color is Your Parachute?

“a preference that is felt and/or considered to be justified – morally, by reasoning, or by aesthetic judgments, usually by two or all three of these” – Sarah Spiekermann, Ethical IT

  1. Write the definition of “value” in your own words. (You might find it handy to look at the list of values below to see how well your definition works compared to that list.)

1.4.2 Most Important Values

Circling the values in the list below that are most important to you. Try to choose 20-25 top values. Think a value is missing from the list? Feel free to add additional values.

Achievement Fun Power
Advancement Generosity Privacy
Adventure Gratitude Problemsolving
Appreciation Growth Professionalism
Attentiveness Happiness Punctuality
Authenticity Harmony Quality
Autonomy Health Recognition
Balance Honesty Relationships
Beauty Hospitality Reliability
Beingthebest Humility Respect
Belonging Humor Risk-taking
Challenge Inclusivity Safety
Compassion Individuality Security
Competition Independence Self-expression
Consistency Influence Service
Contentment Innovation Simplicity
Creativity Integrity Spirituality
Decisiveness Intelligence Stability
Dignity Kindness Status
Diversity Leadership Success
Education Leisure Teamwork
Entrepreneurship Listening Tenacity
EnvironmentalProtection Love Transparency
Equality Loyalty Trustworthiness
Fame Mastery Usefulness
Family Mercy Vision
Financialsecurity Open-mindedness Wealth
Flexibility Peace
Freedom Physical activity
  1. After you have circled your 20-25 top values, put a W next to the circled values that might inform where you would choose to work when you graduate.
  1. When everyone in your group has finished, each person should choose two of the values that they have put W’s next to and describe to the group how that value would influence a job choice for you.

  1. How are values related to ethical frameworks? Look over your circled values, and especially the two you selected in the previous question.

    1. If you were asked why you value those things, how would explain? Does your explanation sound like it is coming from one of our ethical frameworks (utilitarianisms/consequentialism, deontology, or virtue ethics)?

    2. Does your set of values lean toward one of these more than the others?

  1. Compare your answers to the previous item with others in your group. Are there notable similarities? notable differences?

    In a workplace setting which do you think matters more: which values are most important or why those values are most important?

References

Hardt, Moritz, Eric Price, Eric Price, and Nati Srebro. 2016. “Equality of Opportunity in Supervised Learning.” In Advances in Neural Information Processing Systems 29, edited by D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, 3315–23. Curran Associates, Inc. http://papers.nips.cc/paper/6374-equality-of-opportunity-in-supervised-learning.pdf.

  1. For our purposes, two metrics are equivalent if they rank any two things the same – that is, they always agree about which option is the better one.↩︎