Foreword
Informative Page as part of the master’s thesis at the TUM School of Management at the Chair of Strategy and Organization - by Prof. Dr. Isabell M. Welpe and Nadja Born. Written in cooperation with the Leverhulme Centre for the Future of Intelligence at the Cambridge University, advised by Rune Nyrup.
If you have any questions feel free to reach out to me personally, via: philipp.engel@tum.de
Shortcut
You do not have much time and you only want to have the key facts? Then have a look at the Executive Summary or the Infographic.
Overview: Ethics & AI
Preview: One of the most central elements in western societies is the right of every individual to develop in a free and equal manner (United Nations, 1948, 1986). This right is without restrictions, and people can invoke fair treatment…
ProblemPreview: There has been a significant shift in the Ethics & AI community from designing and publishing ethical AI principles to introducing technical tools and procedural methods to translate principles into practice. This shift was triggered…
Solution Approaches
Preview: Investigating what developer teams assess as useful, the most common definition characterizes ‘perceived usefulness’ in the context of information systems as “the degree to which a person believes that using…
Usefulness Assessment & Research ScopeThe Current Landscape of Procedural Methods: A Literature Review
To properly address the research objective of this thesis, a systematic literature review was conducted, using the above-described scope of procedural methodologies, which can be used proactively during AI system development in an agile environment. In doing so, three suitable approaches were identified:
Preview: The first approach is an internal audit, proposed by Raji et al. (2020), which supports the AI system development during the complete development life cycle. The researchers introduce the SMACTR framework, which divides…
Internal Audit (Raji et al., 2020)Preview: The second approach, ECCOLA from Vakkuri et al. (2021), is an iterative add-on process for the complete development process, with the goal to increase ethical awareness during the development process and …
ECCOLA by Vakkuri et al. (2021)
Preview: The third approach, RE4AI from De Azevedo et al. (2022), is essentially a refined version of the ECCOLA method. Thus, it is also an iterative add-on process, which is applicable for the complete development …
RE4AI by De Azevedo et al. (2022)Best Practices on How to Develop More Useful Procedural Methods
After having reviewed the current body of scientific literature which matches the research scope and having derived hypothesis regarding the perceived usefulness by developer teams, I testes those using a mixed-method approach consisting of semi-structured expert interviews and the system usability scale (SUS). Generally speaking, I could show a relatively low overall SUS score, indicating a strong improvement potential (only two hypothesis, H2 and H5, can be considered to be useful, while also those only just exceeded the threshold). Combining the SUS score with the interview results, I was able to derive seven best practices on how to design more useful procedural methodologies in the future.
Reduce the Abstract Nature through Tensions and Atomic Aspects
This thesis has shown a consensus regarding the positive effects of tensions and atomic aspects on the reduction of the abstract nature of principles. Since ethical principles are likely to be unavoidable as a first step in every ethical-aligned AI project, these two tools provide a powerful and generally applicable method to reduce the inherent abstract nature
and can be used in combination or separately. Based on the overall strong positive interview findings, one could even go further and argue that there is no significant reason to not use these tools and, thus, encourage their use in every AI project.
Positively Change the Working Style through Documentation
This thesis has described the positive effect of documentation on changing the working style to be ethically aligned. This occurs because documentation requires the development team to pause and engage with the ethical dimension from the outset. Furthermore, it provides the possibility to establish a learning process from past AI projects and thus provides assistance on how to deal with the current situation.
Moreover, the process of arguing for a certain decision is one of the most prominent positive aspects. This is of significant importance since one major scientific disagreement relates to the use of quantification elements for ethical principles (Saltelli et al., 2020), such as the one used in the first concept that focuses on tensions. The element of logically arguing for an ethical decision may resolve this disagreement since it encourages, perhaps even requires, an extensive engagement with the various ethical aspects. The argument with the highest level of approval from a specific audience is the strongest contender. Consequently, the approval rate can be interpreted as a ‘lightweight metrics’, and optimising this rate requires an optimisation of the argumentation logic, which, thus, increases the level of ethical alignment. Therefore, this thesis strongly encourages the use of logically arguing for ethical decision making by the developers and the documentation of this process.
Enable Actionable Mitigation through Technical Tools
One of the most prominent shortcomings of the concepts in general is the limited help with regard to mitigating ethical issues. This thesis is not only able to show this limitation but also to showcase that referring developers to technical tools is one possible solution. This not only increases awareness of
these tools, which is currently considered to be low (Siqueira de Cerqueira et al., 2022), but also provides developers with actionable tools to provide real-world change. In generalising this best practice, this thesis encourages scientists who design new processes to ask ‘what’s next’ after every process stage, to view the process from the perspective of developers, and to design the process to be as tangible as possible. However, this thesis does not advocate a general application of referring to technical tools, but only in combination with previously executed divergent elements, something we discuss in detail in the next best practice: combining divergent and convergent elements.
Combining Divergent and Convergent Elements
On a meta level, another prominent shortcoming is the use of open-ended questions to trigger a team discussion. While most stakeholders do not regard discussions as generally useless, they emphasised that the use of discussions as the last element in the concept is ineffective. This is because discussions are regarded as divergent elements that open the solution space and trigger creativity. While this is an important element, it should not be the only one since it may lead to an intangible outcome and, thus, will not be perceived as useful. Consequently, divergent elements should be combined with convergent elements, such as referrals to technical tools in or to specific ethical tensions. In addition, my findings indicate that this
relationship is bidirectional. Thus, convergent tools should also only be used in combination with divergent tools, a recommendation that is directly incorporated in the last best practice: enable actionable mitigation through technical tools. However, while combining both elements is certainly critical, this thesis further advises the combination of divergent and convergent elements, but to always end on a convergent element. This is important since, otherwise, no tangible outcome will be generated and, thus, this will not be perceived as useful.
Guidelines on When to Adopt the Process
Combining the survey and interview findings indicates a convergent need for clear guidelines on when a specific concept should be adopted. This is particularly important since AI development is diverse in almost every aspect, including the team composition, the general organisational structure, and the nature of the AI projects. The interview
findings of this thesis show that these components have a significant influence on perceived usefulness, which is why interviewees demanded clear guidelines on when the concept should be implemented or which aspects need to be in place for the concept to work.
Best Practices on How to Adopt the Process
In addition to demanding guidelines on when to implement the concepts, the interviewees clearly expressed a need for best practices, mostly formulated as the need for case studies on how to implement and execute the concepts in the best way. While this shortcoming can be traced back to the novelty of the research field in general, this thesis argues that clearly
stated best practices should play a more important role in the design and publishing process of papers. Consequently, this thesis advices either the inclusion of an additional paragraph that states important best practices on the implementation of the process or the use of a case study methodology for every publication that wants a high adoption rate in the real world.
Match Work Packages with Developer Roles
As mentioned in a paper by Beckert (2021), developers have a tendency to not view the implementation of ethical considerations as an integral part of their responsibility. This is also one of the findings in this thesis. However, the interviews indicate that this could be solved by designing processes from not only a work-package perspective but also
by directly matching tasks with the roles in a development team, with strong arguments for why each match is correct. This will affirm the awareness of the diverse responsibilities of every role and intra-team dissentions can be prevented from the outset. This excludes one prominent reason stated by developers on why ethical principles are not implemented, despite the acknowledgment of their importance.
Conclusion & Further Remarks
Translating between principles and practice is a difficult venture and this thesis clearly outlines that a large amount of work still needs to be done. However, the author has laid a cornerstone in this thesis for future researchers to develop more useful procedural methods that practitioners can adopt. In addition, by describing a possible modular approach for this challenge, this thesis has further provided a blueprint for future researchers on how to continue.
If you would like to dive deeper in the field of Ethics & AI, feel free to have a look at the original papers I used for my research. Just shoot me an e-mail (philipp.engel@tum.de), if you have further questions or remarks!
Aghajani, E., Nagy, C., Vega-Márquez, O. L., Linares-Vásquez, M., Moreno, L., Bavota, G., & Lanza, M. (2019). Software documentation issues unveiled. IEEE (Ed.), 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE) (pp. 1199–1210). IEEE. Retrieved from https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=881193
Arisholm, E., Briand, L. C., Hove, S. E., & Labiche, Y. (2006). The impact of UML documentation on software maintenance: An experimental evaluation. IEEE Transactions on Software Engineering, 32(6), 365–381. doi: https://doi.org/10.1109/TSE.2006.59
Ayling, J., & Chapman, A. (2022). Putting AI ethics to work: are the tools fit for purpose? AI and Ethics, 2(3), 405–429. doi: 10.1007/s43681-021-00084-x
Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the system usability scale. Intl. Journal of Human--Computer Interaction, 24(6), 574–594. doi: https://doi.org/10.1080/10447310802205776
Barocas, S., Hardt, M., & Narayanan, A. (2019). Fairness in Machine Learning. fairmlbook.org. Retrieved from https://fairmlbook.org
BBC. (2019). Apple’s “sexist” credit card investigated by US regulator. BBC. Retrieved from https://www.bbc.com/news/business-50365609
Beckert, B. (2021). The European way of doing Artificial Intelligence: The state of play implementing Trustworthy AI. IEEE (Ed.), 2021 60th FITCE Communication Days Congress for ICT Professionals: Industrial Data--Cloud, Low Latency and Privacy (FITCE) (pp. 1–8). IEEE. doi: 10.1109/FITCE53297.2021.9588560
Bellamy, R., Dey, K., Hind, M., Hoffman, S., Houde, S., Kannan, K., Lohia, P., Martino, J., Mehta, S., Mojsilovic, A., Nagar, S., Ramamurthy, K., Richards, J., Saha, D., Sattigeri, P., Singh, M., Varshney, K., & Zhang, Y. (2019). AI Fairness 360: An Extensible Toolkit for Detecting and Mitigating Algorithmic Bias. IBM Journal of Research and Development, 63(4), 1–15. doi: 10.1147/JRD.2019.2942287
Cervone, H. F. (2011). Understanding agile project management methods using Scrum. OCLC Systems & Services: International Digital Library Perspectives, 27(1), 18–22. doi: 10.1108/10650751111106528
Cihon, P., Kleinaltenkamp, M. J., Schuett, J., & Baum, S. D. (2021). AI certification: Advancing ethical practice by reducing information asymmetries. IEEE Transactions on Technology and Society, 2(4), 200–209. doi: 10.1109/TTS.2021.3077595
Davis, F. D. (1985). A technology acceptance model for empirically testing new end-user information systems: Theory and results (Massachusetts Institute of Technology). Retrieved from https://www.researchgate.net/profile/Fred-Davis-3/publication/35465050_A_Technology_Acceptance_Model_for_Empirically_Testing_New207_End-User_Information_Systems/links/0c960519fbaddf3ba7000000/A-Technology-Acceptance-Model-for-Empirically-Testing-New-End-User
De Fine Licht, K., & De Fine Licht, J. (2020). Artificial intelligence, transparency, and public decision-making. AI & SOCIETY, 35(4), 917–926. doi: 10.1007/s00146-020-00960-w
Fu, Y., Li, M., & Chen, F. (2012). Impact propagation and risk assessment of requirement changes for software development projects based on design structure matrix. International Journal of Project Management, 30(3), 363–373. doi: 10.1016/j.ijproman.2011.08.004
Garousi, G., Garousi, V., Moussavi, M., Ruhe, G., Smith, B., & da Silva, F. (2013). Evaluating usage and quality of technical software documentation: an empirical study. Proceedings of the 17th International Conference on Evaluation and Assessment in Software Engineering, 24–35. doi: 10.1145/2460999.2461003
Hoda, R., Salleh, N., & Grundy, J. (2018). The rise and evolution of agile software development. IEEE Software, 35(5), 58–63. doi: https://doi.org/10.1109/MS.2018.290111318
Hussain, W., Shahin, M., Hoda, R., Whittle, J., Perera, H., Nurwidyantoro, A., Shams, R. A., & Oliver, G. (2022). How Can Human Values Be Addressed in Agile Methods A Case Study on SAFe. IEEE Transactions on Software Engineering, 1–18. doi: https://doi.org/10.1109/TSE.2022.3140230
Jobin, A., Ienca, M., & Vayena, E. (2019). The global landscape of AI ethics guidelines. Nature Machine Intelligence, 1(9), 389–399. doi: 10.1038/s42256-019-0088-2
Kazim, E., & Koshiyama, A. (2021). A high-level overview of AI ethics. Patterns, 2(9), 1–12. doi: 10.1016/j.patter.2021.100314
Kleinberg, J., Mullainathan, S., & Raghavan, M. (2016). Inherent trade-offs in the fair determination of risk scores. ArXiv Preprint. Retrieved from https://arxiv.org/abs/1609.05807
Lee, M. S. A., Floridi, L., & Singh, J. (2021). Formalising trade-offs beyond algorithmic fairness: lessons from ethical philosophy and welfare economics. AI and Ethics, 1(4), 529-544. doi: 10.1007/s43681-021-00067-y
Lepri, B., Oliver, N., Letouzé, E., Pentland, A., & Vinck, P. (2018). Fair, Transparent, and Accountable Algorithmic Decision-making Processes: The Premise, the Proposed Solutions, and the Open Challenges. Philosophy & Technology, 31(4), 611–627. doi: 10.1007/s13347- 017-0279-x
Li, J., & Chignell, M. (2022). FMEA-AI: AI fairness impact assessment using failure mode and effects analysis. AI and Ethics, 1–14. doi: 10.1007/s43681-022-00145-9
Lütge, C., Hohma, E., Boch, A., Poszler, F., & Corrigan, C. (2022). On a Risk-Based Assessment Approach to AI Ethics Governance. Retrieved from https://www.ieai.sot.tum.de/wp-content/uploads/2022/06/IEAI-White-Paper-on-Risk-Management-Approach_2022-FINAL.pdf
Meyer, T., & Reniers, G. (2013). Engineering risk management. In Engineering Risk Management. De Gruyter.
Morley, J., Floridi, L., Kinsey, L., & Elhalal, A. (2021). From What to How: An Initial Review of Publicly Available AI Ethics Tools, Methods and Research to Translate Principles into Practices. Science and Engineering Ethics, 26(4), 2141–2168. doi: 10.1007/s11948-019- 00165-5
Morley, J., Kinsey, L., Elhalal, A., Garcia, F., Ziosi, M., & Floridi, L. (2021). Operationalising AI ethics: barriers, enablers and next steps. AI & SOCIETY, 1–13. doi: 10.1007/s00146- 021-01308-8
Raji, I. D., Smart, A., White, R. N., Mitchell, M., Gebru, T., Hutchinson, B., Smith-Loud, J., Theron, D., & Barnes, P. (2020). Closing the AI Accountability Gap: Defining an End-to- End Framework for Internal Algorithmic Auditing. Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency, 33–44. doi: 10.1145/3351095.3372873
Rismani, S., & Moon, A. (2021). How do AI systems fail socially?: an engineering risk analysis approach. 2021 IEEE International Symposium on Ethics in Engineering, Science and Technology (ETHICS), 1–8. doi: 10.1109/ETHICS53270.2021.9632769
Ryan, M., & Stahl, B. (2020). Artificial intelligence ethics guidelines for developers and users: clarifying their content and normative implications. Journal of Information, Communication and Ethics in Society, 19(1), 61–86. doi: 10.1108/JICES-12-2019-0138
Saltelli, A., & Di Fiore, M. (2020). From sociology of quantification to ethics of quantification. Humanities and Social Sciences Communications, 7(1), 1–8. doi: 10.1057/s41599-020- 00557-0
Sharp, H., Finkelstein, A., & Galal, G. (1999). Stakeholder identification in the requirements engineering process. Proceedings. Tenth International Workshop on Database and Expert Systems Applications. DEXA 99, 387–391. doi: 10.1109/DEXA.1999.795198
Siqueira de Cerqueira, J., Azevedo, A., Tives, H., & Canedo, E. D. (2022). Guide for Artificial Intelligence Ethical Requirements Elicitation--RE4AI Ethical Guide. 55th Hawaii International Conference on System Sciences, 1–10. doi: 10.24251/HICSS.2022.677
Sonnentag, S., Volmer, J., & Spychala, A. (2008). Job performance. The Sage Handbook of Organizational Behavior, 1, 427–447. Retrieved from http://kops.uni-konstanz.de/bitstream/handle/123456789/10386/sonnentag.volmer.spychala.2008_1.pdf?se quence=1&isAllowed=y
United Nations. (1948). Universal Declaration of Human Rights. Retrieved from https://www.un.org/en/about-us/universal-declaration-of-human-rights
United Nations. (1986). Declaration on the Right to Development. Retrieved from https://www.ohchr.org/en/instruments-mechanisms/instruments/declaration-right- development
Vakkuri, V., Kemell, K.-K., & Abrahamsson, P. (2021). ECCOLA-a method for implementing ethically aligned AI systems. Journal of Systems and Software, 182, 195–204. doi: 0.1016/j.jss.2021.111067
Whittlestone, J., Nyrup, R., Alexandrova, A., & Cave, S. (2019). The Role and Limits of Principles in AI Ethics: Towards a Focus on Tensions. 2019 AAAI/ACM Conference on AI, Ethics, and Society, 195–200. doi: 10.1145/3306618.3314289
Zetzsche, D. A., Arner, D. W., Buckley, R. P., & Tang, B. W. (2021). Regulating Artificial Intelligence in Finance: Putting the Human in the Loop. Sydney Law Review, 43(1), 43–81. Retrieved from https://ssrn.com/abstract=3831758
Zuber, N., Kacianka, S., Gogoll, J., Pretschner, A., & Nida-Rümelin, J. (2022). Empowered and embedded: ethics and agile processes. Humanities and Social Sciences Communications, 9(1), 1–13. doi: 10.1057/s41599-022-01206-4