Computer Science

Building trust in autonomous technologies

Abstract

This research placement examined the barriers to widespread adoption of smart technologies by the elderly. There were three major considerations: our dependence on automation, how we can improve lives and its trustworthiness. With an ageing population, automation will reduce economic and infrastructural pressures hence increasing the need to develop and deploy such technologies. The researchers in this research placement hosted a website, which is no longer available, to inform and encourage members of the public to support the technology. The website contained articles about how automation has improved society and will continue to do so. The website also contained documentation of a simulation conducted by the research group which demonstrated a technique that can detect errors in data. Furthermore, this report accounts one of the researchers’ personal development through contributing to the group.

Introduction

Smart technologies have been improving and in recent times, experienced exponential growth in the market. Their capabilities have grown and are increasingly affordable. Despite their convenience, only the younger generation are readily using these technologies. The elderly are unsure about the revolution: the young have interacted with digital devices since childhood while many from the older generation may only have used these devices decades into their life; they may feel disconnected from the technology due to their lack of exposure – some believe that it is too complicated for them and so will never use it. Much of the research has been on improving usability and the technology will only become easier to use. Then another roadblock to adoption arises: trust.

There were other research in making artificial intelligence more trustworthy such as by providing tools and guidelines to developers[1] of the technology so that these algorithms are built robustly and are transparent, or by trialling technologies in low-risk applications like logistics first before use in high-stakes governmental applications[2]. However, this research project proposed to directly inform consumers instead of recommending improvements to developers or organisations.

This research group (Gavin Choy, Mohammed Shikha and Sahir Ahmed) aimed to inform members of the public that automation has and will likely improve safety for society. By aiming the website at middle-aged audiences, the team hoped to be able to leverage their links to their parents (who stand to benefit significantly due to living alone) for informing and persuading them to try the technology. In the following paragraphs, I will provide arguments that were researched and explained to readers on the website.

Demonstrating that autonomous systems have always improved safety, convenience and cost justify the reasoning that current and future autonomous technologies will also provide the same benefits. The current news of driverless cars having higher accident rates caused many to believe that the technology should not be allowed to be used on public roads. It is important to acknowledge that driverless cars are new, and like all innovations, they will improve; for instance, aeroplanes were unsafe when first developed but with continued developments, they are currently one of the safest modes of transportation – Norman, D. A.[3] attributes autopilot as one of the factors in this improvement. Not supporting and inhibiting the development of autonomous technologies, such as driverless vehicles, will only end the potential for a society with better safety, convenience and productivity.

It is common for the media to focus on the ethical issues of programming autonomous technologies such as decision making and bias. For instance, the case of the driverless car in an unavoidable accident where whichever decision is taken will always lead to at least one person suffering the consequences[4]. Society should consider the ethics of programming these systems but focusing too much on extreme scenarios, however, will only lead to overlooking the real moral dilemma. Derek Alexander Muller[5] argued in his video that the real moral dilemma of self-driving vehicles is that many people are injured or killed on roads due to human error and bad driving (like furious driving and distractions). By supporting research and development, autonomous cars can be deployed earlier and save many lives.

For some, forgetting that a cooker is still on is a severe hazard and smart technologies could prevent the fire department from having to show up. Automatically turning off appliances, water taps and lights can be some of the most helpful functionalities a smart home could offer. With the smart car, not having to drive is safer and substantially more convenient. These technologies could also free up time, so the users could engage in productive, entertainment or social activities.

Making incorrect decisions may be rarer with robust algorithms that are researched, developed and tested thoroughly. There are risks that data corruption, and not the programmed logic, will cause a computer to make the wrong decision (and become a safety hazard such as by increasing room temperature to dangerous levels) or outright fail. Data corruption could occur due to hardware faults, or other factors such as a cosmic ray particle flipping binary digits in data stored on a computer component[6]. To mitigate this risk, there are countermeasures such as checksums to detect corrupted data and algorithms to repair from redundant data. Checksum algorithms are mathematical functions that calculate a unique result, called a checksum, for some input of data. If a checksum of some data is known, and one calculates the checksum of the data at a later time and the result is different, the data must have changed within that time interval. Further, authenticating the source of software updates and data will prevent cybercriminals from compromising the Internet of Things (IoT) – preventing widespread damage, privacy issues and risk to life if a critical system is attacked. Moreover, encryption of data will help maintain privacy for sensitive information including activity and medical records, for example.

I will, in the following sections, detail the development and final product of the research team before evaluating the success of the project and potential points of improvement that could be implemented in future works. Additionally, I will explain how my contributions to the project developed my insight, knowledge and skills.

Development

First Week

The team had discussions with their supervisor about what the project aims are. It was decided that a website and a simulation will be made to demonstrate the concept of smart technology helping the elderly.

Considering that in the eldest age bracket, there is less use of a computer for browsing, the website would not be successful by solely aiming at the elderly. Instead, the site would target middle-aged people. They are a group of people who may share similar views with the eldest about the capabilities and trustworthiness of technology – for instance, research[7] conducted by AARP revealed that only 18% of 1520 American adults in mid-life or elderly (aged 50 or above) are extremely or very confident that wireless devices have good privacy. The youngest are generally more open to these technologies and so their differing views may not allow for their parents and grandparents to be convinced as easily – maintaining the mentality that technology is only for the youth or leading seniors to fear new technologies[8]. Further, the youngest will have an additional generation gap which reduces the likelihood of the project succeeding. While it is easiest to convince the youngest, these reasons will make this impractical as the team aimed to improve the lives of the eldest.

As most of the readers to the website should be the general public, it had to appeal to them and use layman’s terms. Many adults will read news, journalism and editorials from publishers like The Guardian[9], thus including articles for reading on the website may be appealing. It was through these articles that the researchers inform and persuade people to be more accepting of advances in automation technology as well as encouraging the use of them. As a result, three articles were planned.

Having a device prototype would be very useful for demonstrating the help technology could offer. It was thought that a smart device could log and send data to an authority such as recording room temperatures for a general practitioner; research conducted by the Association for the Conservation of Energy[10] revealed that many perished due to a lack of heating in their homes – a device to monitor temperatures could alarm authorities to assist those in danger.

However, it was noted that producing a prototype could be difficult for a research placement of six weeks: not only would the algorithm need to be coded, but also to ensure hardware compatibility. Therefore, it was agreed that a simulation will be more suitable as there would be no hardware to contend with. A simulation assuring the reader that the logic of smart devices is robust would be sufficient for this purpose. The team continued researching for possible simulation ideas.

Second Week

A timetable had been written and as it was based on estimates of how long each task should take to complete, it served as a guide rather than a rigid timeframe. This flexibility benefited a team with little experience in building websites and simulations. Specialisation was implemented to increase the productivity of the team; one was responsible for coding (website and simulation), one for writing two articles and one for writing an article plus designing the logo. All other tasks such as research, design and testing were conducted by the whole team.

Legend:

Red: Ahmed, Choy, Shikha

Blue: Shikha

Green: Choy

Yellow: Ahmed

Figure 1: Gantt chart of the estimated schedule and tasks for each researcher. Source: Gavin Choy

The team discussed potential website designs and viewed websites such as from the World Trade Organization[11] (WTO), Tracker[12] and The Guardian. The WTO’s website was modern and accessible – news and reports were easy to find, and the colour scheme was simple. Tracker was a website for an open-source physics program and was very simple and easy to use. The Guardian was familiar to many and dedicated to providing content for reading. While an uncomplicated design like that of Tracker would be significantly less difficult to implement, maintain and test, a modern user interface targeted for providing research topics and articles for reading would likely appeal to a larger audience. The hosting and coding of the website had begun during this week.

Having elements of an elderly person and technology would best represent the project. The Wi-Fi symbol was commonplace and so combining it with an elderly person suggests the use of networked smart technology. The logo and website colours were matching. Additionally, the team had thought about their project name.

Research and planning of the three articles were completed and ready for writing. The logo creation was done alongside writing the three articles over the week.

Figure 2: The logo and name of the website (the name was finalised during the third week). Source: Gavin Choy

Third Week

The homepage was hosted with placeholder images and text. An introductory text was added to the slideshow and was the first statement read by viewers to the website; this contained brief information about the aims of the project and led to the next two slides that link to the articles and simulation. An About Us page was written and included details about the team members and the supervisors.

During this week, the logo was finalised, and it was decided that the project name would be SAFE – Smart, Affordable and Friendly Electronics. The first thought that should come to mind was safety for both consumers and developers. Perhaps, this may provide the best first impressions for those who were less trusting of autonomous systems. The theme of safety was critical for the entire website hence, the name was representative of the work.

The first two articles were completed and added to the website. Article 1 (Essentials in life: automation) was about society’s dependence on automation to improve safety and convenience while reducing costs. It provides the context to autonomous systems such as robotic manufacture and autopilot for aeroplanes and how they have improved safety significantly – arguing that there were no reasons as to why other autonomous technologies like self-driving vehicles will not follow the same trend. Article 2 (The real moral dilemma) highlighted the ethical considerations of developing self-driving vehicles and extended the argument for use of the technology in the medical industry; this set the tone for Article 3 which was about the use of smart devices to help the elderly and improve their safety by limiting hazards in the home and ensuring the correct medicine consumption. Article 3 continued being written over this week.

To demonstrate that there has been substantial research into the safety of artificial intelligence and autonomous systems, the homepage needed to link to articles, websites and a database of research. A table of sources was created: it included the title of the website and a description of its contents.

Fourth Week

Article 3 was completed. Images from the public domain and other royalty-free sources were added to the articles so that they mimic those from large presses, and attract readers. As always, the website experienced continual improvements with more pages, links and an updated design. Social media accounts were created for the project to serve updates to the public. The addition of background imaging increased appeal to the site.

The team considered using an interactive Flash animation to showcase different scenarios when a smart device could mitigate hazards within the home; the user would be able to compare the scenarios with and without a smart device by clicking buttons in the Flash player. They began experimenting with animation software and creating storylines for the simulation.

Fifth Week

Images were found for the table of research sources to improve the chances of visitors opening links to view other sources as evidence to support the work the team had done. The research sources and their respective thumbnails have been included in the homepage.

Instead of using an interactive flash animation for the simulation, writing a program to demonstrate that there were methods for ensuring data integrity in smart devices would be more appropriate as a demonstration of a working system would be more effective than showing hypothetical scenarios. Users of the technology can be assured that erroneous data would not be used such that the smart device could make potentially fatal incorrect decisions. For the simulation, a mock filesystem of a device was made, and a program was coded in Python, a programming language, so that it periodically checks the integrity of the data and filesystem using checksums.

Sixth Week

There were formatting of the website and adding the sources used in the articles with their thumbnails. The program was finalised and the documentation for the program was written. The documentation included details of what erroneous data could do to a smart device and its implications for a critical device such as a pacemaker. There were brief details about existing countermeasures and how they prevent issues from occurring. Finally, it explained the workings and features of the program. This demonstrated that a smart device and an authority such as the general practitioner could use tools like checksums to verify the correct storage, usage and transmission of data.

Final Product

Website

Figure 3: The homepage (partially shown). Source: Gavin Choy

The landing page presented the visitor with an introductory body of text in a slideshow and a navigation bar. The slideshow was on a timer to display the articles and the simulation. Below were links to websites and articles to show that the research was extensive in the field and so the quality and reliability of products and services should be continually improving. There were also links to the sources used in each of the articles.

Figure 4: Article 1 page (partially shown). Source: Gavin Choy

The theme was inherited from the main page to keep consistency which ensures a good user experience. There was an image after the sub-heading, mimicking those one would find on a typical publisher’s article. Despite the inspiration from online publishers, the columns mimic those of print newspapers and was a cosmetic decision (the three columns scale with the page zoom) – this might be unique to visitors to the project webpages. There was a download link to a PDF for those who wish to read the article later when offline or in paper print (the PDF version did not have columns).

However, not all pages were implemented: the About Us, Simulation and Contact Us pages. Their texts have been written but were to be added later to the site.

Figure 5: The About Us page text. Source: Gavin Choy

Figure 6: The Simulation documentation (partially shown). Source: Gavin Choy

Simulation

Figure 7: Summary flowchart of the simulation program. Source: Gavin Choy

Filesystems are structures a computer uses to store information – it contains folders and files. The simulated filesystem had the following directories (folders):

STORAGE

├── incoming

├── outgoing

├── sent

├── store

└── system

The program was coded in Python[13] version 3.6.5 which is a scripting programming language. For a holistic system, the algorithm had 5 threads (of which 4 were actively doing work). A single thread is analogous to having one person complete all tasks by themself, but having multiple threads is equivalent to having multiple persons working simultaneously. This allows for concurrent activities to take place and prevents one activity from causing slowdowns and hangs (such as when the program is stuck on one task) as would have been the case with a single-threaded implementation.

The ZFS[14] (2018) filesystem uses scrubbing to verify data integrity through comparing data and metadata with their checksum (SHA-256, fletcher2 or fletcher4 are the checksum algorithms supported by ZFS). When scrubbing, all data stored is read and the checksums of these data are calculated – if the calculated checksum is different from the recorded checksum, the data must have changed and so the filesystem attempts to restore the original data from redundant information (such as a mirror, essentially a backup, of the original data). Data is stored in blocks to form a Merkle tree which each block validates its children blocks – this is a cryptographically strong authentication algorithm that ensures data integrity whether it is data rot or a cryptographic attack.

The simulation program employed similar techniques to ensure data integrity: all directories, file paths and each file’s SHA1 checksum were recorded in a text file. Periodically, the program compares the directories, file paths and checksums of the files with the data in the text file. If the current filesystem and the record matches, the program concludes that there is no data corruption. Otherwise, it raises an alarm. To prevent false positives when the text file is corrupt but not the filesystem, the contents of the text file are also verified using a checksum. Therefore, if there is a corrupt text file, the program will flag it and skip scrubbing the filesystem.

For a smart device implementation, the program was extended to verify any incoming files and create a text file containing the structure and hashes of the file when sending to another device or authority. Thus, this demonstrated to the public that data integrity is a thoroughly researched field and that developers can implement these features, especially for critical systems such as a pacemaker. These features help prevent hacking, alert authorities of any problems and shut down or failover to a backup circuit. Henceforth, this assured the reader that there are mechanisms in place to prevent a smart device from malfunctioning and harming the user.

Project Evaluation

This research placement produced work that demonstrated and persuaded readers, especially those who were not trusting of autonomous technologies, that they are to benefit from these innovations. There were links to other research on the website which illustrate the large investments, developments and testing for artificial intelligence. Articles written by the researchers have been optimised for those who read online news, journals and editorials or traditional newspapers. Furthermore, a program demonstrated one of the techniques manufacturers employ to ensure data integrity and proper functioning of devices, particularly those critical to the safety of people.

By demonstrating the potential of technology to improve lives, this could encourage readers to research more about this field, and to possibly encourage and help their parents (the elderly who were the main target for this research project) to use these technologies. As the eldest generally have the least trust in technology, the research group hoped that by leveraging the elderly’s trust in their children, they can begin to use smart devices.

To continue the development of the website, it should be tested by some members of the target audience to ensure usability and appeal. For instance, information such as instructions to enlarge (zoom) the page could help those who are new to computers. These are simple implementations that will significantly improve usability and user satisfaction but have not been included yet.

After the initial testing of the website, the researchers will then need to market this site on the internet; this will likely include partnerships with other people or groups with an interest in the success of a website that promotes the use of smart technologies, such as companies that develop and sell smart devices. Additionally, the researchers will need to learn about search engine optimisation (SEO) to increase the traffic to and the awareness of the website.

Regarding the simulation, a smart device can be prototyped using a single-board computer like the Raspberry Pi[15]. A specific function such as monitoring the ambient temperature to control a heating system can be programmed onto the device. The same, or similar, program to ensure data integrity can be running on the device itself. To demonstrate that the device will not behave unexpectedly in the event of corrupt data or a malicious attempt to change the data, the researchers can modify the data in real-time and the device should either cease all activity or failover to a back-up. This will serve as a proof of concept which can be extended to demonstrate more security techniques such as cryptographic signatures.

While there has been extensive research in helping autonomous technology developers in making robust and transparent systems, and use of the technology in organisations, this investigation aimed to improve public trust especially from the elderly. There will be a greater proportion of elderly people in the future as life expectancy increases which increases the strain on the workforce to provide care; robotic carers are being developed but to deploy them, we need to “tear down public trust barriers”[16].

Personal Development

This was my first cooperative project that lasted six weeks. This required me to coordinate and manage time with others for a longer period. I have completed an Extended Project Qualification (EPQ): To what Extent will Autonomous Vehicles Form the Future of Travel and Urban Planning. While it was a three-month project, I only had to adjust the schedule for oneself. I had to adjust to the changing schedule in this research placement due to modifications in ideas and other delays. All this coordination required good communication between the researchers; we updated our peers and supervisors of our progress through WhatsApp so that we would be briefed for the next time we met. Being flexible made decision making easier as different ideas were suggested by our supervisors and peers which tended to modify the plan. Hence, decisions were swift, and the team managed to adapt which led to improved quality of the work.

Having some experience in writing essays, articles and an EPQ, I was delegated to write two articles. As my EPQ was about autonomous vehicles, the existing research aided me in finding new ideas and sources appropriate for justifying the consistent improvements in AI safety. These writing skills helped me finish both articles within a week as agreed which allowed me to find other research sources the next week. My previous works have been aimed at a technical audience and so I had learnt to adjust to use layman’s terms and to explain with simplified ideas.

Due to the large amount of research in my EPQ, I developed a sources table that had all the necessary information such as the author, date, title and description for each entry; I utilised a similar table design for this research placement which was easy to decipher by my peers and to include on the website.

Figure 8: Sources table for this research placement (partially shown). Source: Gavin Choy

Following the original plan to create an interactive animation, I installed and learnt how to use Blender[17], an open-source 3D rendering software. I decided to use Blender because it is a powerful tool and I have very little experience with other animation software. Should I have animated with the software, Adobe Animate[18], that I had experimented years ago, the result would be 2D. Blender allows for both 2D and 3D rendering by adjusting the camera perspective. Either way, the resulting animation would have been unprofessional due to inexperience.

Since the beginning, our supervisors had suggested that having a mathematical method to prove that the logic of autonomous systems is safe would be ideal. I only recalled a personal project (that I did two years prior) after one week of learning to animate. I suggested that checksums could be used to ensure data integrity which in turn ensures proper functioning of smart devices. As I had experience in building a program to verify data integrity, I was assigned the role of coding the simulation and writing its documentation. Likewise, the documentation explained why data integrity is necessary and uses layman’s terms so that the target audience understands the principles.

The program for my personal project was single-threaded and accepted user input. I enter the directory of the folder and instruct the program to either create a text file with the checksums or to verify its integrity. The scrubbing algorithm was reused in the research placement’s program whereas all other functionality such as automatic scrubbing and file transfers were new additions. Furthermore, learning how to use multiple threads improved the performance of the program and allows for a single program to execute all the functions required – again, proving to our visitors that a holistic system can be achieved.

Figure 9: A trial graphical user interface (GUI) for my personal project. Source: Gavin Choy

My personal project program was coded in Python and had some limitations: due to the global interpreter lock (GIL) in Python, the program cannot use multiple threads available from a multi-thread processor. The GIL is necessary for Python because its memory management is not thread-safe – meaning that running multiple threads could cause errors. As the program utilises checksum algorithms, it is computationally heavy, thus some tasks may benefit from a multi-threaded implementation – also, a multi-threaded program will allow the user interface to remain responsive during computation.

The trial GUI from Figure 9 was created with the Qt[19] framework. Qt is a cross-platform development tool that has a GUI designer and can compile C++ code. C++ will likely be faster at runtime due to the lack of overhead from an interpreter (Python is an interpreted language) and especially for multi-threaded applications. Further, the cross-platform compiler will allow me to write once and compile anywhere for operating systems and hardware platforms that Qt supports.

To continue with my personal project, I need to learn coding with cross-platform frameworks like Qt or wxWidgets[20]. With this, I hope to be able to produce a more versatile and powerful program for my uses. Moreover, these programming skills will be useful for other projects such as data collection, storage and analysis for scientific studies which is a field I am hoping to pursue.

Acknowledgements and Final Comments

I must thank my two supervisors, Dr. Alexander Bolotov and Mr. Ayman El Hajjar, for supporting and helping us with our research placement such as by guidance, suggestions and the hosting of the website. I thank my peers who have worked alongside me and enabled this project to be assembled together. I also thank the Nuffield Foundation for organising and providing us with this opportunity to explore and engage in novel research in securing trust in autonomous systems.

This research placement has been an enjoyable experience for us as we learnt and tested our abilities with each other. This will prepare us for future projects we may undertake in university or our careers with the knowledge, skills and insight developed from this placement.

Works Cited and Further Reading

  1. Rossi, Francesca. “BUILDING TRUST IN ARTIFICIAL INTELLIGENCE.” Journal of International Affairs 72, no. 1 (2018): 127-34. Accessed August 15, 2020. DOI:10.2307/26588348.
  2. Hunter, Andrew P., Lindsey R. Sheppard, Robert Karlén, and Leonardo Balieiro. ARTIFICIAL INTELLIGENCE AND NATIONAL SECURITY: THE IMPORTANCE OF THE AI ECOSYSTEM. Report. Center for Strategic and International Studies (CSIS), 2018. 24-34. Accessed August 15, 2020. DOI:10.2307/resrep22492.7.
  3. Norman, D. A. “The ‘Problem’ with Automation: Inappropriate Feedback and Interaction, Not ‘Over-Automation’.” Philosophical Transactions of the Royal Society of London. Series B, Biological Sciences 327, no. 1241 (1990): 585-93. Accessed August 29, 2018. www.jstor.org/stable/55330.
  4. Scalable Cooperation. “Moral Machine”. MIT Media Lab. Accessed: June 25, 2019. http://moralmachine.mit.edu/
  5. Muller, Derek Alexander. “The Real Moral Dilemma of Self-Driving Cars”. January 19, 2017. YouTube, accessed: August 29, 2018.
  6. Elhage, Nelson. “Attack of the Cosmic Rays!”. Oracle. Accessed: June 25, 2019. https://blogs.oracle.com/linux/attack-of-the-cosmic-rays-v2
  7. Anderson, G. O.. “Technology Use and Attitudes among Mid-Life and Older Americans”. Presentation, AARP Research, December 2017). https://www.aarp.org/content/dam/aarp/research/surveys_statistics/technology/info-2018/atom-nov-2017-tech-module.doi.10.26419%252Fres.00210.001.pdf, accessed: June 26, 2019.
  8. Roupa et al.. “The use of technology by the elderly”. Health Science Journal (n.d). https://www.hsj.gr/medicine/the-use-of-technology-by-the-elderly.php?aid=3614, accessed: July 8, 2020.
  9. The Guardian. Accessed: September 1, 2018. https://www.theguardian.com/uk
  10. Association for the Conservation of Energy. “Chilled to Death: The human cost of cold homes”. PDF presentation, Association for the Conservation of Energy, London, 2015. https://www.ukace.org/wp-content/uploads/2015/03/ACE-and-EBR-fact-file-2015-03-Chilled-to-death.pdf, accessed: August 30, 2018.
  11. World Trade Organization. Accessed: September 1, 2018. https://www.wto.org/
  12. Tracker. Accessed: September 1, 2018. https://physlets.org/tracker/
  13. Python. “About”. Accessed: 3 September 3, 2018. https://www.python.org/about/
  14. Sun Microsystems. “ZFS Administration Guide”. PDF presentation, Sun Microsystems, Santa Clara, May 2007. http://www.dubeiko.com/development/FileSystems/ZFS/zfsadmin.pdf, accessed: September 3, 2018.
  15. Raspberry Pi Foundation. Accessed: June 27, 2019. https://www.raspberrypi.org/
  16. United Kingdom Government. 2020. Care Robots Could Revolutionise UK Care System And Provide Staff Extra Support. Accessed: August 16, 2020. https://www.gov.uk/government/news/care-robots-could-revolutionise-uk-care-system-and-provide-staff-extra-support
  17. Blender. Accessed: September 4, 2018. https://www.blender.org/
  18. Adobe. “Adobe Animate”. Accessed: September 4, 2018. https://www.adobe.com/uk/products/animate.html
  19. Qt. Accessed: September 7, 2018. https://www.qt.io/
  20. wxWidgets. Accessed: June 27, 2019. https://www.wxwidgets.org/

About the Author

This image has an empty alt attribute; its file name is Gavin-Choys-Profile-Picture-1024x683.jpeg

A student at the University of Cambridge studying Natural Sciences (Computer Science, Physics, Mathematics and Materials Science). As Gavin has a strong interest in physics and computing, he regularly researches topics in these fields.

Leave a Reply

Your email address will not be published. Required fields are marked *