The final OpenTox Evaluation report summarises the testing and evaluation carried out on OpenTox in the final year of the project. The evaluation included assessment of OpenTox from the following points of view a) framework, b) standards, c) ontology, d) data resources, e) services, and f) applications.
In 2011 an OpenTox Virtual Machine was developed. This allows the user to download and install a virtual machine on their computer where OpenTox applications can be run in a local environment. Hence although OpenTox supports a distributed set of resources, it may also be collapsed to run as a local stand-alone application. We brought the virtual machine software to SETAC Africa and successfully installed it on ca. 40 laptops of workshop participants.
Online tutorials have also been created supporting usage of the applications on an ongoing basis. International use in 2011 has been broadened and demonstrated by the coordinator who for example has worked with risk assessors at Health Canada and African scientists at SETAC Africa in the use of the applications in workshops, in addition to responding to a number of presentation invitations from industry and international conference organisers.
These developments were disseminated and evaluated through a number of means, ranging from internal testing and evaluation by non-developers within the OpenTox project, to provision of online tutorials and exercises, to a hands-on workshop involving 9 exercises carried out in the final month of the project, which was subsequently made available online for further evaluation, and to dissemination at a variety of international conferences. This report summarises the results, feedback and evaluation of these developments of the OpenTox framework and its services and applications. We also provide recommendations for next steps and future directions.
Internal testing and evaluation supported the development of a number of OpenTox resources, services and applications so that they reached the stage of prototypes that could be provided to users to carry out exercises in a workshop setting or on their own. Underlying these is the OpenTox Application Programming Interface (API), which is an important output of the FP7 research project. It provides a specification against which development may continue by both OpenTox partners and the broader development community. In addition to further development of existing resources and applications, new ones may be created compliant with the specifications, thus supporting a growing set of interoperable linked resources for the field of toxicology. As the specification is an open standard it may also continue to develop and improve. As the latest version of the API (1.2) supports Authorisation and Authentication, such infrastructure can support the integrated use of both open and commercial resources. Policies for controlled access to resources can also be implemented which supports user access to resources based on licences, data sharing agreements and legal contracts. The use of OpenTox API-compliant REST services to communicate instructions between linked resources with URI addresses supports the use of a wide variety of commands to carry out operations such as data integration, algorithm use, model building and validation. OpenTox thus provides a “lower level programming language” to the more technically-oriented user for carrying our research, executing tasks or application development.
The OpenTox ontology development made good progress in the last year of the project and included development of an ontology to support endpoints and organ effects, and the capability to convert from the existing ToxML data standard into a semantic web format. From the toxicology ontology workshop held at the EBI resulted a perspectives review and a toxicology ontology roadmap to provide guidance to the field. As OpenTox creates a semantic web for toxicology, it should be an ideal framework for incorporating such future ontology developments, thus supporting both a mechanistic framework for toxicology in addition to best practices in statistical analysis, a combination that the field will surely profit from.
A number of (Q)SAR models were built with OpenTox for a number of REACH endpoints (carcinogenicity, mutagenicity, aquatic/fish toxicity, LogP), and a representative subset is presented in the report. Several of these models have been made available with validation reports through the ToxPredict application. We propose a “challenge” should be organised in which the community could develop different models using OpenTox services applied to different REACH endpoints.
The OpenTox 2011 conference in Munich, held at the end of the FP7 project in August 2011, attracted a diverse and knowledgeable audience including many principle investigators in the field, working in different areas. Hence, the group of ca. 80 scientists was strongly cross-disciplinary, cross-sector and international. With this backdrop we also ran a final workshop in which 9 different OpenTox exercises and applications were run by participants in groups in a hands-on format. It is difficult to communicate the intensity of the day as many participants engaged in a variety of activities all of which had been made possible by the project. This report summarises those experiences extended through additional experiences that went before and were continued afterwards through testing and evaluation and through the provisions of online tutorials.
The evaluation of OpenTox by these users provided a clear response that the project has produced very valuable and concrete results. Moreover, although still at the prototyping stage, many users were able to carry out activities with applications of significant scientific interest, including activities that had not been previously possible for them. The majority of evaluation answers to our questions produced answers in the excellent and very good categories, providing a sound endorsement by the user community that we have created something really valuable and worthwhile from the project. OpenTox should therefore be seen as a special success story the European Commission has supported through its research program.
At the end of every research project there are always issues that the work has surfaced which provide input into future work. Here we summarise some of the main issues which we currently have discovered or recognised: 1. OpenTox is a brand new framework, hence developers need to learn about the new approaches before being able to implement; 2. It is difficult to maintain the performance of a distributed set of resources and to ensure the ongoing evaluation and stability of core distributed services; 3. Certain users may prefer a local application, which is an area requiring future development by application developers; 4. Additional resources are needed to ensure the quality of a resource e.g., to curate a data source and to filter and label for quality; 5. Applications requiring integration of resources from multiple partners require a significant integrated testing methodology and effort; 6. A well-defined API is insufficient on its own to support developers in their development of resources to be accessed by other applications; 7. A significant effort is required to incorporate support for a broad range of datasets, protocols and models in the expanding field of predictive toxicology. Even more important is communication between different disciplines who do not understand each other very well; 8. The “21st century” mechanistic approach to predictive toxicology requires a framework and methods that place numerous demands beyond the building of a “simple” (Q)SAR for a single endpoint; 9. A substantial amount of work is needed to organise and harmonise the set of ontologies needed for predictive toxicology. This will take a significant coordinated effort and investment; 10. Further application development, communication skills and learning methodologies are needed to provide easy-to-use fit-for-purpose tools for end users not specialised in computer science techniques.
We provide here recommendations for the future development and sustainability of OpenTox and its benefit and impact:
1) OpenTox should be developed further to support creation of a global interoperable infrastructure for Predictive Toxicology to aid achievement of the goals of the 21st century vision. This could be supported through a dedicated EC infrastructure project supporting the goals of a number of programs including European activities under REACH, FP7 Health, FP7 Environment, EC JRC, and Innovative Medicines Initiative and international activities such as the US EPA ToxCast program and Tox21. The unique benefit of OpenTox lies in its core commitment to open standards which can support in a neutral way the goals of all stakeholders; 2) A program should be put in place to implement the Toxicology Ontology Roadmap. OpenTox can play a leadership role in collaboration with other stakeholders; 3) In consultation with industry, CEFIC and ECHA, an analysis should be carried out promptly to identify steps that could be taken to optimise OpenTox for supporting the activities of industry in meeting the 2013 REACH deadlines; 4) A business ecosystem approach providing value to industry and SMEs should be developed for OpenTox in support of its economic exploitation; 5) Consideration should be given to the formation of a legal structure for OpenTox e.g., as an international organisation or foundation; 6) The potential of OpenTox to be exploited in the development of safer medicines should be pursued; 7) OpenTox has evolved a community of interested parties beyond the initial FP7 partners. This should be developed further as an active and vibrant network; 8) OpenTox partners committed at their final project meeting to a combined interest and resolve to continue to develop OpenTox beyond the end of the project. Doing so will be rewarding!
OpenTox 2011 Technical Reports
You can also access and download the OpenTox 2011 technical reports on the framework, database, algorithms, validation and confidential data. See links at:
I presented on OpenTox applications at the BIO-IT 2011 conference this week (Boston, 12 -14 April 2011) in the workshop session on Current Methods for Computational Toxicology and Chemogenomics. Presenting in this productive and interactive session:
•Josef Scheiber, Ph.D., Pharma Research & Early Development Informatics, F. Hoffmann-La Roche Ltd. •Jeremy Jenkins, Ph.D., Senior Investigator I, Developmental & Molecular Pathways, Quantitative Biology, Novartis Institutes for BioMedical Research •Christopher Southan, Ph.D., ChrisDS Consulting •Luis Tari, Ph.D., Postdoc Research Fellow, Biomedical Informatics, F. Hoffmann-La Roche Ltd. •Antony Williams, Ph.D., Vice President, Strategic Development, ChemSpider, Royal Society of Chemistry •Yuriy Gankin, Ph.D., Co-Founder and Chief Scientific Officer, GGA Software Services •Barry Hardy, Ph.D.,Director, Community of Practice & Research Activities and OpenTox Project Coordinator, Douglas Connect
The conference showed the growing acceptance and indeed demand by industry for Open Source and Open Standards-oriented solutions which are increasingly providing both superior technical solutions and lower cost end user service-oriented business cases.
The OpenTox presentation includes developments on numerous end user applications (built upon OpenTox APIs and web services) including ToxCreate, ToxPredict, Bioclipse, Qedit, Ches-Mapper, and integration examples with other platforms and initiatives such as CDK, ToxML, and ToxCast. You can download a copy of the slides from:
Abstract submissions should be completed by 30 April for consideration for the conference program and bursary awards. Poster abstracts will continue to be accepted through 30 June, to be eligible for meeting-based review and innovation merit awards.
I gave a talk at at the IQPC Predictive Toxicology Conference in London, UK on 24 February. In this presentation I discuss the critical role of interoperability in collaboration situations in drug design and predictive toxicology. I reflect on the inspiration of the Tamboti Tree Use Case from our recent conservation work trip to South Africa. The role of intereroperability and ontology development in the OpenTox predictive toxicology framework is discussed. The evolution of development and prototyping within OpenTox has demonstrated its potential as a Semantic Web for Predictive Toxicology, alternative testing methods and risk assessment. Several recent OpenTox interoperations have been achieved including with Bioclipse, CDK, ToxCast, and Leadscope. OpenTox will provide a valuable open standards based foundation on which to create services and link resources for the new ToxBank infrastructure project.
I provide an overview of the approach used in the FP7 Synergy project where we have established a Scientists Against Malaria virtual organisation which is carrying out a number of modelling and experimental drug design activities against a kinase target in Plasmodium, the parasite that causes malaria. We use an ontological approach to interoperate between a collaborative electronic notebook, Synergy collaboration services and event-driven recommendation rules for consensus formation and traffic light discussion and decision making on conflicting predictions or experimental results.
This book provides a very useful reference and comprehensive summary of all recent/current significant Alternative Testing R&D FP6/7 projects in Europe, plus additional perspectives on other initiatives e.g., in US, Japan etc. It includes a chapter on #OpenTox.
Barry Hardy PhD Director, Community of Practice & Research Activities and OpenTox Project Coordinator (www.opentox.org)
I will deliver an OpenTox seminar at the IQPC (Pharma IQ) Predictive Toxicology conference taking place in London 23-24 Feb 2011, in which I will present on recent developments with regards to OpenTox interoperability advances, and particularly to support of collaborative drug discovery projects. Also of interest to the increasing support by OpenTox of in vitro methods is a significant program on stem cell technology. The conference will address the accuracy of drug safety testing through the integration of advanced prediction methods, including stem cell based assays and screening into safety evaluation strategies. The main conference will take place in London on 23rd-24th February 2011 and new for this year, there will be a pre-conference focus day on 22nd February 2011 dedicated to the use of stem cells as a tool to enhance predictive toxicology. http://www.predictivetoxicology.co.uk/Event.aspx?id=394530