Automatic Assistance to Cognitive Disabled Web Users via Reinforcement Learning on the Browser

Tomas Murillo Morales; Peter Heumader; Klaus Miesenberger

doi:10.1007/978-3-030-58805-2_8

Automatic Assistance to Cognitive Disabled Web Users via Reinforcement Learning on the Browser

Tomas Murillo Morales, Peter Heumader, Klaus Miesenberger (Editor)

Institute Integrated Studies

Research output: Chapter in Book/Report/Conference proceeding › Conference proceedings › peer-review

Abstract

This paper introduces a proof of concept software reasoner that aims to detect whether an individual user is in need of cognitive assistance during a typical Web browsing session. The implemented reasoner is part of the Easy Reading browser extension for Firefox. It aims to infer the user’s current cognitive state by collecting and analyzing user’s physiological data in real time, such as eye tracking, heart beat rate and variability, and blink rate. In addition, when the reasoner determines that the user is in need of help it automatically triggers a support tool appropriate for the individual user and Web content being consumed. By framing the problem as a Markov Decision Process, typical policy control methods found in the Reinforcement Learning literature, such as Q-learning, can be employed to tackle the learning problem.

Original language	English
Title of host publication	Computers Helping People with Special Needs - 17th International Conference, ICCHP 2020, Proceedings
Editors	Klaus Miesenberger, Roberto Manduchi, Mario Covarrubias Rodriguez, Petr Penáz
Publisher	Springer
Pages	61-72
Number of pages	12
Volume	12377
ISBN (Print)	9783030588045
DOIs	https://doi.org/10.1007/978-3-030-58805-2_8
Publication status	Published - Sept 2020

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	12377 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Fields of science

102 Computer Sciences
102013 Human-computer interaction
102014 Information design
102015 Information systems
102021 Pervasive computing
102022 Software development
102024 Usability research
102026 Virtual reality
102027 Web engineering
102036 Digital accessibility
502007 E-commerce
506002 E-government
509002 Disability studies
602013 Sign language research
202004 Brain-computer interface
211902 Assistive technologies
503008 E-learning
302027 Hearing, voice and language disorders

JKU Focus areas

Digital Transformation
Sustainable Development: Responsible Technologies and Management

Access to Document

10.1007/978-3-030-58805-2_8

Cite this

Murillo Morales, T., Heumader, P., & Miesenberger, K. (Ed.) (2020). Automatic Assistance to Cognitive Disabled Web Users via Reinforcement Learning on the Browser. In K. Miesenberger, R. Manduchi, M. Covarrubias Rodriguez, & P. Penáz (Eds.), Computers Helping People with Special Needs - 17th International Conference, ICCHP 2020, Proceedings (Vol. 12377, pp. 61-72). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12377 LNCS). Springer. https://doi.org/10.1007/978-3-030-58805-2_8

Murillo Morales, Tomas ; Heumader, Peter ; Miesenberger, Klaus (Editor). / Automatic Assistance to Cognitive Disabled Web Users via Reinforcement Learning on the Browser. Computers Helping People with Special Needs - 17th International Conference, ICCHP 2020, Proceedings. editor / Klaus Miesenberger ; Roberto Manduchi ; Mario Covarrubias Rodriguez ; Petr Penáz. Vol. 12377 Springer, 2020. pp. 61-72 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{009d6dcff5de431ca1acbea419e61643,

title = "Automatic Assistance to Cognitive Disabled Web Users via Reinforcement Learning on the Browser",

abstract = "This paper introduces a proof of concept software reasoner that aims to detect whether an individual user is in need of cognitive assistance during a typical Web browsing session. The implemented reasoner is part of the Easy Reading browser extension for Firefox. It aims to infer the user{\textquoteright}s current cognitive state by collecting and analyzing user{\textquoteright}s physiological data in real time, such as eye tracking, heart beat rate and variability, and blink rate. In addition, when the reasoner determines that the user is in need of help it automatically triggers a support tool appropriate for the individual user and Web content being consumed. By framing the problem as a Markov Decision Process, typical policy control methods found in the Reinforcement Learning literature, such as Q-learning, can be employed to tackle the learning problem.",

author = "\{Murillo Morales\}, Tomas and Peter Heumader and Klaus Miesenberger",

year = "2020",

month = sep,

doi = "10.1007/978-3-030-58805-2\_8",

language = "English",

isbn = "9783030588045",

volume = "12377",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "61--72",

editor = "Klaus Miesenberger and Roberto Manduchi and \{Covarrubias Rodriguez\}, Mario and Petr Pen{\'a}z",

booktitle = "Computers Helping People with Special Needs - 17th International Conference, ICCHP 2020, Proceedings",

}

Murillo Morales, T, Heumader, P & Miesenberger, K (ed.) 2020, Automatic Assistance to Cognitive Disabled Web Users via Reinforcement Learning on the Browser. in K Miesenberger, R Manduchi, M Covarrubias Rodriguez & P Penáz (eds), Computers Helping People with Special Needs - 17th International Conference, ICCHP 2020, Proceedings. vol. 12377, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 12377 LNCS, Springer, pp. 61-72. https://doi.org/10.1007/978-3-030-58805-2_8

Automatic Assistance to Cognitive Disabled Web Users via Reinforcement Learning on the Browser. / Murillo Morales, Tomas; Heumader, Peter; Miesenberger, Klaus (Editor).
Computers Helping People with Special Needs - 17th International Conference, ICCHP 2020, Proceedings. ed. / Klaus Miesenberger; Roberto Manduchi; Mario Covarrubias Rodriguez; Petr Penáz. Vol. 12377 Springer, 2020. p. 61-72 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 12377 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference proceedings › peer-review

TY - GEN

T1 - Automatic Assistance to Cognitive Disabled Web Users via Reinforcement Learning on the Browser

AU - Murillo Morales, Tomas

AU - Heumader, Peter

A2 - Miesenberger, Klaus

A2 - Manduchi, Roberto

A2 - Covarrubias Rodriguez, Mario

A2 - Penáz, Petr

PY - 2020/9

Y1 - 2020/9

N2 - This paper introduces a proof of concept software reasoner that aims to detect whether an individual user is in need of cognitive assistance during a typical Web browsing session. The implemented reasoner is part of the Easy Reading browser extension for Firefox. It aims to infer the user’s current cognitive state by collecting and analyzing user’s physiological data in real time, such as eye tracking, heart beat rate and variability, and blink rate. In addition, when the reasoner determines that the user is in need of help it automatically triggers a support tool appropriate for the individual user and Web content being consumed. By framing the problem as a Markov Decision Process, typical policy control methods found in the Reinforcement Learning literature, such as Q-learning, can be employed to tackle the learning problem.

AB - This paper introduces a proof of concept software reasoner that aims to detect whether an individual user is in need of cognitive assistance during a typical Web browsing session. The implemented reasoner is part of the Easy Reading browser extension for Firefox. It aims to infer the user’s current cognitive state by collecting and analyzing user’s physiological data in real time, such as eye tracking, heart beat rate and variability, and blink rate. In addition, when the reasoner determines that the user is in need of help it automatically triggers a support tool appropriate for the individual user and Web content being consumed. By framing the problem as a Markov Decision Process, typical policy control methods found in the Reinforcement Learning literature, such as Q-learning, can be employed to tackle the learning problem.

UR - https://www.scopus.com/pages/publications/85091535619

U2 - 10.1007/978-3-030-58805-2_8

DO - 10.1007/978-3-030-58805-2_8

M3 - Conference proceedings

SN - 9783030588045

VL - 12377

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 61

EP - 72

BT - Computers Helping People with Special Needs - 17th International Conference, ICCHP 2020, Proceedings

PB - Springer

ER -

Murillo Morales T, Heumader P, Miesenberger K, (ed.). Automatic Assistance to Cognitive Disabled Web Users via Reinforcement Learning on the Browser. In Miesenberger K, Manduchi R, Covarrubias Rodriguez M, Penáz P, editors, Computers Helping People with Special Needs - 17th International Conference, ICCHP 2020, Proceedings. Vol. 12377. Springer. 2020. p. 61-72. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-58805-2_8

Automatic Assistance to Cognitive Disabled Web Users via Reinforcement Learning on the Browser

Abstract

Publication series

Fields of science

JKU Focus areas

Access to Document

Other files and links

Cite this