Part 7 - More Tech Ethics Issues: Surveillance, Anthropomorphism, Attention

(Part 7 of the University of Hertfordshire Tech Ethics Course. << Part 6 | Part 8 >>)

Surveillance

“Have people inadvertently given away data that will harm themselves, others or society in general?”

A 2019 report from tech research company Comparitech ranked the most surveilled cities in the world. Unsurprisingly, China won eight of the top 10 spots but coming in at number six, and sporting around 630,000 cameras, was London. The British capital has approximately one recording device for every 14 inhabitants. According to cctv.co.uk: “Anyone going about their business in London will be caught on camera around 300 times per day.”

In the past decade, China’s internal security spending (much of which is widely believed to be spent on surveillance tech) increased tenfold. It now significantly outstrips their external defence budget. Closer to home, by 2025 cctv.co.uk expects the number of cameras in London will top one million. Pair all of this data with the growing sophistication of facial recognition and we have a new world.

The arguments for automated surveillance using facial recognition are usually made by governments or multinationals:

• Improved security (a common argument in the UK).
• Better social cohesion, control, and trust; especially when coupled with social scoring (China).
• More convenient grocery shopping (Amazon in the US).

The last might seem a rather trivial benefit, but it appears popular.

The arguments against tend to be made by academics and the EU:

• Facial recognition technology isn’t good enough for what's it's being used for yet, which will lead to miscarriages of justice.
• It is an invasion of privacy.
• It is a threat to personal liberty.
• Although the data is recorded in public, it is not publicly available. When it remains in the hands of MNCs, we're all working for them for nothing.

As engineers, we should be aware that the EU considered banning public surveillance in 2020. It has not yet done so, but may do in future. Even without a ban, the legal floor remains that recorded video data may fall under a county's rules for data protection.

Anthropomorphism

“Who is human and who only appears (masquerades) as human? Unless we can individually and collectively be certain of the answer to this question, we face what is, in my view, the most serious problem possible” - Philip K Dick

Anthropomorphism is defined as “the attribution of human characteristics or behaviour to a god, animal, or object.” Humans are very good at it.

We don’t yet have chatbots that can pass the Turing test (which is about the imitation of intelligence, not AGI. Essentially, it's anthropomorphism). In most cases, however, they don’t need to be all that convincing to fool us.

We aren’t expecting the “person” we’re talking to over webchat support to be anything other than a human and the last thing we're going to ask is, “Can you eat a chair?” (which, according to Steve Warswick, winner of the 2019 Loebner prize for the world's most convincing human emulation, is the kind of question often asked to trip up chatbots).

To be fooled into believing a chatbot is a human is only inadvertent anthropomorphism - it's being deliberately mislead. True anthropomorphism requires us to be convinced something we know is a machine is as intelligent, caring, and compassionate as us. That is sadly easy to do when it has a human-looking form.

Ben Goertzel, the creator of the performance chatbot “Sophia” actively aims to achieve this. “For most of my career as a researcher," he said, "people believed that it was hopeless, that we’d never achieve human-level AI. Now, half the public thinks we’re already there.” That's hardly surprising, given that's what he tells them.

In reality, we are a long way off artificial general intelligence (which we can't currently even define). If we were closer, we'd have some ethical discussions on that point but for the moment it is largely an irrelevant distraction. We'll instead concentrate on anthropomorphism as it currently applies in tech.

The main arguments in favour are:

• Anthropomorphic robots are a cheap way to provide (the imitation of) care, love, and companionship to people. 
• It’s a cheaper way to provide customer service (and recent research suggests that’s more effective if people believe they are talking to a human rather than merely a chatbot).

The arguments against human emulation are:

• It’s a lie.
• It undermines real relationships (which are more challenging than talking to a robot that has no reciprocal needs).
• Most dangerously, anthropomorphism leads to a misleading impression of competence and predictability. It can cause users to over-attribute accuracy and safety to “AI” algorithms.

As an engineer, it isn’t against the law for products to pretend to be human, but you must consider the ethical and safety implications.

Attention

Where does your attention go? In our attention economy, that matters. Marketing firms don’t only pay for clicks, they often pay for impressions (the number of people who see an ad on their screen). That means old school media and new social media firms want to keep you online, with ads in front of you, for as long as possible. Some of them are very good that. Is it a problem?

On the positive side, media is entertainment. If people want to look at it, surely that’s a good thing - it's the whole point.

It’s your choice how you spend your time. Arguments about “amusing ourselves to death” (according to philosopher Neil Postman) were made about TV in the last century and novels before that, both of which may have done society good by encouraging empathy and social cohesion. Modern social media causes people to become more personally and viscerally involved in the issues of the day and that has upsides and downsides, but should lead to a more engaged populace in the longer term. Surely that's a good thing?

The counter argument is that modern media is more deliberately compulsive than TV or books and is significantly more superficial. That it is reducing our attention spans and removing nuance, leading to anxiety (or possibly just correlated with it), increasing polarization, and reducing the quality of social discourse and day-to-day family life. For employers, there is also a fear it may be reducing productivity by distracting the workforce with trivial Facebook updates and pointless clickbait when they should be getting on with their jobs.

10 years ago, 7% of the US population used one or more social networking sites. Now that figure has increased to 65% and the global average time spent daily in social media is 2 hours 23 minutes. Note that's still less than TV (3 h 35 mins in the US)

To a certain extent, social media usage is a matter of choice. Unlike smoking, it doesn’t provenly cause harm. However, time is a precious commodity and social media is designed to be addictive. One potential ethical approach might be to allow users to audit themselves (easily find out how much time they have spent on your app) enabling them to make informed choices.

In this post we have looked at surveillance, anthropomorphism, and attention. In the next one we'll consider: open v closed code and data; social scoring; accessibility and exclusion.

(Part 7 of the University of Hertfordshire Tech Ethics Course. << Part 6 | Part 8 >>)

Part 6 - The Tech Ethical Issues To Talk About

(Part 6 of the University of Hertfordshire Tech Ethics Course. << Part 5 | Part 7 >>)

In our last few posts, we have discussed tech ethics and responsible technology in general terms and left it to the practitioner (you) to decide how to apply it to your own future products. In the next posts, we are going to briefly look at some specific areas that get a great deal of press coverage. Some get more than others - ethics shouldn’t be subject to fashion, but inevitably it is.

In each case, I am not going to define the rights and wrongs - other than pointing out where the law already does so - but I will try to outline some of the main arguments on both sides, and where the ethical floor might be, no matter which side of the argument you come down on.

Over the next few blog posts I’ll cover:

• Energy use in the tech sector.
• AI and Big Data.
• Cyberwarfare, propaganda and killer robots.
• Surveillance.
• Anthropomorphism.
• Attention.
• Social Media influence.
• Open v closed code and data.
• The role of social scoring and civil order (mass surveillance apps).
• Accessibility.
• Exclusion.
• Privacy.
• Security.
• Future of trust.
• Changing behaviours and social norms.

I’ll also discuss different definitions of social good (e.g. individualistic vs community) which vary from country to country and person to person.

That’s a lot! Inevitably, it will only be an overview.

Energy Use

Tech is one of the most successful industries worldwide. That makes it one of the fastest-growing users of fossil fuels.

The UK’s electricity was just over 40% hydrocarbon generated in 2019. Globally, fossil fuels generated ~65% of electricity in 2017. Data centres alone are currently estimated to use 2% of the world’s electricity and if you add in devices, the % gets much higher (according to Greenpeace, 10-20%). Machine Learning is particularly energy intensive.

Does this make the tech industry a force for good or evil when it comes to climate change?

On one hand, we could assert that communications tech cuts down on travel (very green); technology often increases efficiency (green again); and there are societal benefits to tech that make it worth some climate cost.

On the other, we might say tech is electricity-powered and there are better, more sustainable ways to generate that than by burning fossil fuels. Many people argue that the tech industry is rich and powerful, and therefore the right thing is for the industry to lead the way in using clean electricity.

Those views are not conflicting.

As engineers, our responsibility is to stay informed; make active choices; and pay attention to our energy usage. For example, it is vastly cleaner to host instances in AWS’s Dublin region (100% renewable or offset) than in US East (only 50% offset). Google Cloud and Azure are both 100% offset everywhere. This information is usually available on cloud provider’s sustainability web pages. Read it. If data isn’t available for your provider, that is not a good sign. Ask for it.

AI, Machine Learning and Big Data

I’m not talking about general AI and whether to build Skynet here. That’s a rather long way off. What I am focusing on is data analytics, and the automation of physical and intellectual tasks.

Data Analysis 

When we discuss Machine Learning (ML), what we're usually talking about is machine-enhanced statistical analysis of existing data sets. Sometimes that analysis uses something like deep learning, but surprisingly often it is still just a SQL query!

ML is an astonishingly useful tool for humanity. Using digital techniques, huge pools of high quality, often high density, data can be analysed. That information couldn’t be processed manually in any realistic timeframe.

For example, medical, astronomical, agricultural or other scientific photographs can be scanned and automatically studied, potentially unearthing radical new hypotheses in those fields. Similarly, huge quantities of public domain text data is already being analysed, leading to breakthroughs in automatic translation. Major leaps are being made in medicine by ML.

But it's not all good news. There are also significant ethical concerns about ML:

• Is the source data accurate, or does it contain false information? Historic data may include beliefs that are incorrect but may have been, or continue to be, widely held. Products based on such biased data might be unfair or cause unlawful discrimination.
• Has the data been sourced in a responsible manner or have people inadvertently given away information that will harm themselves, others or society in general?
• Is the analysis bugged in a way that is hard to detect? Has it been sufficiently tested?
• How do we handle false positives or negatives? These will happen even if there are no bugs because that is how statistics works. 
• Does restricted access to the source data lead to monopolies that are not in the public interest?

The EU’s GDPR legislation attempts to address some of these concerns via its transparency and right of challenge rules.

As engineers, our responsibility is to obey the law; check the provenance of our data; be aware that data quality impacts every conclusion we reach; test thoroughly; and rigorously document and account for all our decisions.

We also need to understand that error is baked in to statistics: data = model + error  - i.e. some error will always happen. The inevitable mistakes therefore need to be handled compassionately and thoroughly accounted for.

Automation

One of the common uses of AI or machine learning is in job automation. Again, this is neither good nor bad but has benefits and risks.

Task automation means many dull, dangerous, or rote tasks don't need to be performed by humans any more. That increases productivity and accuracy, and reduces cost. One example is self-driving cars, which are anticipated will reduce congestion and accidents. That’s all good.

So, what are the arguments against automation?

One concern is that where humans are directly controlled by algorithms, the result can be heartless. For example, where gig economy workers are given automatically calculated and distributed work schedules, they sometimes allow no time for family life or illness. Let’s take the real life case of the Kronos scheduling software. In 2014, The New York Times revealed the way it optimised for efficiency screwed up the lives of employees. Presumably the developers hadn’t intended that, they just hadn’t foreseen the problems and had no existing guidelines to work from.

Of course, humans can also be cruel, but here the responsibility for mercy lies with the software developer. He or she has to code it in. The risk is, they might not (probably won't?) have a good understanding of the human factors in the situation. If they mess it up, they could make people's lives hell.

Automation can also lead to significant labour disruption. For example, Uber's stated business plan is to replace all its human drivers with those lovely self-driving cars. The oil industry has also introduced significant automation to oil wells, leading to a huge fall in the number of human employees.

Finally, automation directly links business productivity to capital (money to spend on robots or software), which advantages those who already have capital. That can lead to wealth concentration, which may not be in the public interest.

As engineers, the last two points are probably beyond our scope, but we do need to consider how our code is written and tested so it doesn’t harm people who are controlled by it and provides mechanisms for problems to be detected and reported. You may also want to consider whether you are happy with the existence of the product you’re building. That is your personal choice.

The Future of Warfare

Even if you don’t go into the defence industry, you need to be aware of the direction warfare is headed in because the wars of the future will not only be fought with physical weaponry - they may be fought on your software.

Killer Robots

The most obvious ethical dilemma around tech and war today is the use of killer robots.

Remotely controlled, unmanned aerial vehicles (UAVs), aka drones, are already widely used on the battlefield, particularly in the Middle East. Also under development are machines that can target and shoot without human intervention: so-called killer robots (an accurate, if literally loaded, term).

The main argument in favour of these is they could be more accurate and reduce battlefield loss of life.

The primary argument against them is that the technology is still not good enough to use anywhere near civilian populations. They often kill the wrong person and targeting mistakes are frequently swept under the carpet by the military, rather than properly addressed. Note that the same arguments are also applied to UAVs with human triggers, where the automatically-generated hit list may contain mistakes (see AI and Machine learning above).

More philosophical reasons against autonomous weapons centre around whether humans should ever be killed automatically. You might argue landmines do it, but to kill that way with a landmine is against Geneva and UN conventions - most of us have already decided that’s wrong.

Another argument is that killer robots make warfare cheaper (once the tech has been created) and therefore more of it can be waged. Whether that is a "for" or "against" depends on your viewpoint and current context.

Cyberwarfare

Cyberwarfare is a new weapons frontier. In the Ukraine in 2015, the power grid was hacked and brought down by a sophisticated cyber attack. From 2005 to 2010, the US's Stuxnet virus attacked Iran's uranium enrichment program. In the first case, the target was the control code for a power facility. In the second, it was possibly every Windows machine in the world (it only triggered if the PC was in an Iranian nuclear plant).

In future, the target could be your system. It is hard to write code or support systems that are proof against a state actor, but as an engineer it is vital your systems are proof against everyone they can be resilient to. Don’t get taken down and cause the deaths of thousands because you didn’t apply a security patch.

Propaganda and civil disorder

“Destabilizing an adversary society by creating conflict in it and creating doubt, uncertainty, distrust in institutions” - Keir Giles, senior consulting fellow on Russia at Chatham House.

Creating killer robots is expensive up-front. A lower Capex alternative is propaganda: eroding trust in a government using targeting advertising, misinformation, deep fakes or just fake news. The US may even be using popular games.

As an engineer, it is your responsibility to consider if your new social media platform, or product (e.g. a game), or tool (like a video editor) could be used as a weapon of destabilisation and how you would detect that and stop it.

What Else?

In this post, we have very briefly covered some of the ethical issues around climate change (energy use), AI and Machine Learning, and Cyber warfare. It is part of being a professional to balance up these benefits and risks.

In the next post in this series, we’ll look at surveillance, anthropomorphism and attention…

(Part 6 of the University of Hertfordshire Tech Ethics Course. << Part 5 | Part 7 >>)

Part 5 - Why do Humans do Bad Things?

(Part 5 of the University of Hertfordshire Tech Ethics Course. << Part 4 | Part 6 >>)

People do bad things because they’re evil. If you’re a good person, you’ll never do anything wrong.

Hurray!! You can stop reading here.

Hang on a Minute!

Unfortunately, as we discussed in the last article, humans don’t appear to work like that. The study of social psychology suggests our behaviour is highly influenced by our environment. Your individual (usually good) nature is less critical than you might hope.

Most of us want to be ethical. This post is about what psychology tells us stands in our way, and what we can do about that. 

I’m a technologist not a psychologist, so these are mostly the judgments and investigations of my colleague and co-author, the registered psychologist Andrea Dobson. Many thanks Andrea!

Obedience

“More hideous crimes have been committed in the name of obedience than in the name of rebellion.” - C.P. Snow

After the second world war, psychologists started looking at why seemingly-normal people could do very bad things. The trigger was the Nuremberg trials. The world was stunned as, over and over, individuals justified mass murder on the grounds that “Befehl ist Befehl” - an order is an order.

In 1963, Yale psychologist Stanley Milgram decided to investigate further. He wanted to know how powerful the desire to be obedient was and how far it could change people’s behaviour. He devised a set of infamous electric shock experiments and what he found was extraordinarily disturbing. 65% of ordinary Americans would electrocute a stranger, provided the order came from an authority figure.

Some of the studies that followed have reported obedience rates of over 80% (from Italy, Germany, Austria, Spain, and Holland). It is now well accepted that obedience is a powerful driver in human behaviour.

Is that all? Do we merely follow orders or is there anything else as powerful that affects us?

Conformity

Would you contradict your colleagues? I’d like to think I would, but the evidence suggests I’m kidding myself. Most of us go along with the group consensus, whatever it might be. In fact, psychology tells me I’m more likely to deny the facts than risk being the odd one out.

In the 1950’s, Polish psychologist Solomon Asch ran a series of experiments to investigate how much an individual’s judgments were affected by those of the folk around them. He discovered most of us (nearly 75% in his tests) conform: we will lie or deceive ourselves, at least some of the time, to publicly fit in with an overwhelming majority.

We’re Doomed!

Does this mean we’re the slaves of our environment? Fortunately not. Or not completely.

• 35% of Milgram’s experimental subjects disobeyed orders and wouldn’t “electrocute” their victim, even under extreme social pressure. 
• 95% of Asch’s subjects went against the group at least once, even if they mostly complied. Rebellion was more common if they had an ally or if voting was secret. 

Obedience and Conformity are not insurmountable, they are merely a strong influence that we should be aware of.

Riven with Guilt?

Experiments suggest most of us want to be good but we will often act badly if either those around us are, or we’re told to.

Does that mean we all live in a constant state of guilt and remorse? The answer is kind-of. We’re very good at ignoring our own guilt, or at least rationalising it away, using a process called Moral Disengagement.

Moral Disengagement is the process of convincing ourselves normal ethical standards don’t apply to us in the situation we’re in. We thus avoid the “self-sanction” that would normally stop us doing something wrong.

According to Albert Bandura of Stanford University: “Moral disengagement functions [..] through moral justification, euphemistic labelling, advantageous comparison, displacing or diffusing responsibility, disregarding or misrepresenting injurious consequences, and dehumanising the victim.”

A common way to diffuse moral responsibility, for example, is through group decision-making:

“People act more cruelly under group responsibility than when they hold themselves personally accountable for their actions” - Bandura

Again, it is something we need to be aware of. Moral disengagement doesn’t work in every case but it does appear to work. Remember that any action you take is an action you are personally ethically and legally responsible for, no matter what moral disengagement may tell you.

Unethical Amnesia

If you can’t quite explain away what you did, psychology suggests you have another option: forget all about it.

Psychologists Francesca Gino and Maryam Kouchaki from Northwestern and Harvard Universities conducted a series of experiments on whether people remembered themselves doing good things better than they recalled doing bad ones. Their studies of over 2100 participants demonstrated people recall times they acted ethically, like playing a game fairly, more clearly than times they cheated. Again, this is something to watch out for - we appear to be hardwired to believe we are better behaved than we are. When we behave less well we literally forget it.

We Seem to be Good at Doing Bad Things. How do we Fix That? 

If we want everyone to act more ethically, there are several approaches we could take.

Top-down change of behavior throughout an entire organisation. 

The trouble is, top down change is hard. Even if the CEO really means it, folk probably won’t believe it - at least not for a long time. Top down changes can take years to permeate, and any authority-based approach can also lead to moral disengagement, which is risky in an ethically unclear situation (“The disappearance of a sense of responsibility is the most far-reaching consequence of submission to authority” - Stanley Milgram).

Bottom up, individual-driven change. 

Bottom-up change could be quicker - people have a strong desire to see themselves as the goodies and will generally act well if left alone. However, people’s desire to do good is easily derailed by Obedience, Conformity and Moral Disengagement. As Bandura puts it: “Given the many psychological devices for disengaging moral control, societies cannot rely entirely on individuals”.

So what could we do?

Some researchers have suggested bad behaviour in companies often comes from bad incentives. For example:

• Too many business transformation programs can warp a company’s own ethical climate by pushing too much change from the top, too quickly and too frequently. People who are rushed or flustered are more likely to become morally disengaged and act unethically.
• Incentives and pressure to inflate achievement of targets can also cause issues. People do what they are rewarded to do, and most are rewarded for hitting KPIs, not following their principles. Again, this leads to moral disengagement.

The best way to combat disengagement is with engagement. So consider:

• What are people paid and promoted for? Does it incentivise dodgy behaviour?
• Are people punished for speaking up and questioning a decision or the accepted way of doing things?
• Do people feel like they work for an amoral company? If they do, they’ll behave that way too.
• Do leaders acknowledge dilemmas or sweep them under the carpet? Are problems discussed openly and frankly? Are diverse or conflicting views heard? 

Speak up!

“In a true learning organisation, employees are able to speak up, express concern and make mistakes without fearing negative consequences like punishment or ridicule.” - Andrea Dobson

Psychological Safety is a management concept that has become popular in the past few years. The idea is to create a team culture that promotes learning by making any question safe to ask, from “I don’t understand, how does that work?” to “isn’t that going to get someone killed?”

It’s a way of working that makes asking difficult, potentially ethical, questions part of your job (obedient) and expected (compliant) and has been suggested as a bulwark against moral disengagement. It is therefore one possible way to promote a more ethical work environment.

“Life in society requires consensus as an indispensable condition. But consensus, to be productive, requires that each individual contribute independently out of his experience and insight.” - Solomon Asch

Psychological safety is just one aspect of a learning organisation and tools are now around to help companies implement it (which, according to Google’s Aristotle project, has productivity advantages beyond just ethics).

The previous posts in this series talked about why you should act ethically in order to do your job professionally and legally. In this post, we discussed the psychological reasons why you, or your colleagues, might not do so even if you want to. The processes and behavioural norms around us can drive us via obedience, conformity, and moral disengagement. In the next post, we will look at some specific sectors of the industry and examine their ethical pros and cons.

(Part 5 of the University of Hertfordshire Tech Ethics Course. << Part 4 | Part 6 >>)

Authors 

Andrea Dobson-Kock is a Registered Psychologist (HPCP) and a Cognitive Behavioural therapist. As a practicing psychologist, Dobson-Kock specialised in depression and anxiety disorders, complex grief and worked for over a decade in mental health.

Anne Currie is an engineer of 25 years, a speaker, writer and science fiction author. She also teaches Tech Ethics at the University of Hertfordshire.

References

S.E. Asch (1956). Studies of independence and conformity: I. A minority of one against a unanimous majority. Psychological Monographs: General and Applied, Vol 70(9),, 1-70.
T.C. McLaverty (2016). The influence of culture on senior leaders as they seek to resolve ethical dilemmas at work
Klass, E. T. (1978). Psychological effects of immoral actions: The experimental evidence. Psychological Bulletin, 85(4), 756
Festinger, L. (1957). A Theory of cognitive dissonance. Stanford, CA: Stanford University Press.
M. Kouchaki & F. Gino (2016). Memories of unethical actions become obfuscated over time. PNAS May 31, 2016. 113 (22) 6166-6171
Hofmann W., Wisneski DC., Brandt, M.J. and Skitka, L.J. (2014). Morality in everyday life. Science 345(6202):1340–1343.
Goodwin, G.P., Piazza, J., Rozin, P. (2014). Moral character predominates in person perception and evaluation. J Pers Soc Psychol 106(1):148–168.
Festinger & Carlsmith (1959). Cognitive consequences of forced compliance. Journal of Abnormal and Social Psychology, 58, 203 – 210
Milgram, S. (1963). Behavioral study of obedience. Journal of Abnormal and Social Psychology, 67, 371-378    .
W. Weiten (2010). Psychology: themes and variations

Sponsored by Container Solutions

The Panopticon Series on Amazon US and UK

Photo by Stefano Pollio on Unsplash