Archive for the ‘Data quality’ Category

ChemSpider data cleanup

In previous posts, we have discussed the automated workflow we use to check new incoming data for structure and synonym errors. These checks allow us to remove the most common types of errors before they are added to the site. However, these filters do not apply to data already in ChemSpider.

Manual curation is an important part of our work. We periodically review the data on our most accessed records, in addition to ad-hoc removal or correction of erroneous data that we or our users notice when using the site. However, there are far too many records and far too much data to clean up using manual curation alone.

Recently we have focused on bulk identification and removal of erroneous data. This work has covered mapping errors and other clearly incorrect values in our experimental property data, correction or removal of malformed synonyms, correction of incorrectly labelled synonyms, and resolution of structure/synonym clashes.

Experimental Properties

We retrieved all 6.3 million experimental properties, text properties, and associated annotations from the ChemSpider database. We then compared the original text of the property as it was written in the original file to how that text was parsed and mapped by our deposition system. This enabled us to identify and correct several types of errors affecting around 2% of the properties in our database:

  • 35,774 experimental property values had been assigned the incorrect unit (e.g. g/L instead of g/mL, °C instead of °F)
  • 2,591 boiling points measured under non-standard pressure did not have this pressure displayed
  • 4,292 densities had their density and temperature values swapped
  •  79,252 miscellaneous erroneous properties and associated annotations were deleted. For example, “white crystals” mapped as melting point, impossibly high melting points or densities, etc.

Synonyms

Synonyms, chemical names, and identifiers are the most abundant type of data on ChemSpider, with a total of more than 446 million synonyms. These synonyms have additional metadata including language labels and flags identifying what type of synonym they are (e.g. CAS number, UNII, INN, trade name).

Simple Checks

We ran a series of regular expression string searches to identify synonyms with incorrect metadata, as well as malformed or otherwise erroneous synonyms.

  • 200,007 synonym type flags added, and 4,766 incorrect flags removed
  • 9,170 synonyms with an incorrect language label identified.
  • 631,697 erroneous synonyms identified, including scrambled characters, properties/units, molecular formulae as synonyms, purity information, or invalid CAS numbers or EC numbers (formerly called EINECS).
  • 922,334 instances of these erroneous synonyms deleted from ChemSpider records.

Structure/Synonym comparison

After identifying and removing these synonym-level errors, we then cross-checked ChemSpider records and their synonyms to identify mismatches. This work included amino acids, nucleic acids, and pharmaceutically acceptable salts.

As a first pass, we compared synonyms to molecular formulae to identify records missing key elements. Examples include synonyms describing a sodium salt when the molecular formula does not contain sodium, or describing an amino acid when the molecular formula contains no nitrogen. A total of 28,194 of these synonym/formula clashes were identified and removed.

For records that passed this initial molecular formula check, we performed a SMARTS comparison to identify chemical structures missing key structural features described in the synonym.  These SMARTS strings were written broadly, with common substitutions allowed to prevent unnecessary removal of valid synonyms from derivative compounds.

In the following examples, the mismatched part of the synonym is highlighted in bold.

Structure

Removed synonym

Chemical structure of sulfur dioxide Sulfate ion
Chemical structure of zolpidem Zolpidem tartrate
Chemical structure of Sodium S-sulfocysteine Sodium S-sulfocysteine hydrate

After identifying these clashes, we manually spot-checked the output to weed out false positives and iterate the SMARTS filters. 101,257 synonym/structure clashes were identified and removed.

These checks included the following categories:

  • Amino acids and their derivatives: 6 formula clashes, 56 structure clashes
  • Nucleic acids, nucleosides, nucleotides: 977 formula clashes, 1,870 structure clashes
  • Halogens: 13,437 formula clashes, 1,256 structure clashes
  • Alkali and alkaline earth metals, and aluminium: 3,586 formula clashes, 56 structure clashes
  • Carboxylic acids and their derivatives: 5,002 formula clashes, 88,501 structure clashes
  • Other pharmaceutically acceptable acids: 3,534 formula clashes, 1,529 structure clashes
  • Amides and amines: 190 formula clashes, 304 structure clashes
  • Deuterates, hydrates, methylbromides: 1,462 formula clashes, 7,685 structure clashes

Get involved

You are the expert in your area of chemistry, so if you see something that doesn’t look quite right please let us know. If the error is confined to a single ChemSpider record, click the “Comment On This Record” box at the top of the affected record and let us know what the problem is. All we need is a sentence describing the error, however the more information you can provide, the better.

For more systemic errors, or in cases where you want to attach supplementary information or corrected chemical structures, please get in touch via email (chemspider@rsc.org).

ChemSpider Pre-Deposition Filters

Written by Mark Archibald.

In a previous post (Behind the Scenes at ChemSpider) we discussed some of the challenges in upholding data quality across one of the largest chemical databases in the world. We identified automated filtering as a key tool when dealing with far more records than a human could reasonably handle. In this post we’ll go into more detail about how that filtering works, what the challenges are, and the role played by human intervention.

To perform this filtering we use KNIME, an open-source data processing platform. The wide range of KNIME nodes developed by the active cheminformatics community allows us to ask chemistry-specific questions of the data we process. In simple terms, input chemical structures that match our criteria are passed on to the next node, while those that don’t are written out to an error file. After processing all structures, the result is a file of structures that have successfully passed through all the filters and several (usually smaller) files of structures rejected for various reasons.

Structures are filtered. Flagged structures are reviewed, and passed structures are added to ChemSpider.

It’s not possible to review all of the generated files in full, as this would eliminate the time-saving advantages of automated processing. However, output files of all types are spot checked for accuracy and to iteratively improve the filtering criteria. Certain output files have high potential for false positives and so we review them in full.

Formats and identifiers

Submitted files can be in one of several different formats. The most common is SDF (structure data file, a chemical structure format containing multiple structures with associated data fields). The advantage of this format is that it contains 2- or 3-dimensional structures, so we can immediately start processing the file without having to convert an identifier to a structure. This means that the final structure we deposit is more likely to exactly match the original. The disadvantage of the SDF format is that it is specialised – many users will be unfamiliar with it or won’t have software to create and display the files.

We also receive different spreadsheet formats (excel, csv, tsv) with structures encoded in text-based notation systems like SMILES  or InChI. The advantage of this format is that it doesn’t require specialised software (provided the submitter has SMILES or InChIs for the compounds).The disadvantage is that the structures require conversion to SDF before processing and deposition to ChemSpider. Additionally, these formats contain information about atoms and their connectivity but lack layout information. This can introduce errors as different structure drawing packages can parse these structures slightly differently, resulting in alterations to the final deposited structure.

Filtering criteria

The criteria by which we judge chemical structures are a mixture of definitive chemical rules and less well-defined ‘rules of thumb’ based on our experience and chemical knowledge. Examples of both follow.

Empty structures, query atoms and incorrect valences

The first filter is the simplest – ChemSpider is a structure-centric database, so it’s not possible to deposit any input entries that lack a structure.

Similarly, each ChemSpider record requires a single defined chemical structure, so we exclude anything using a query atom to represent a variable atom or attachment point.

Another simple filter is to exclude structures in which atoms have invalid valences.

Charge imbalance

In general, entries in ChemSpider should represent a real-world, isolable compound. This means that we filter out structures with a non-zero overall charge. However, we make exceptions for certain examples where a counterion is generally unimportant and it’s useful to consider the charged species alone, such as choline (ChemSpider record).

Structures containing undefined stereocentres

Undefined stereocentres alone don’t represent a chemical error. However, structures like that shown below (cholesterol without any defined stereocentres) occur frequently and, although chemically valid, it’s extremely unlikely that they represent the intended structure.

Cholesterol skeleton with no defined stereochemistry

Cholesterol skeleton without stereochemistry

As a result we have a rule of thumb that excludes structures containing more than two undefined stereocentres. This is not a hard-and-fast rule, but rather an attempt to strike a balance between excluding structures like the one above and including structures where the undefined stereocentres are intentional and correct.

The count of undefined stereocentres (as determined by examining the InChI) sometimes includes cases where it is conventional to exclude stereochemical wedges. Examples include nucleic acids with no wedges on the phosphate and adamantyl groups without explicit stereochemistry – it’s unusual to draw these compounds with wedges, and users will rarely use wedges in their search. These potential false positives are filtered out and reviewed manually. A curator can then decide whether to include them in the deposition, improving the overall accuracy of the filter.

Structures containing many components

This is another rule of thumb – there’s no upper limit on how many separate components a correctly depicted chemical substance can have. However, from experience we find that excluding structures with more than four separate components removes most obviously nonsensical entries (e.g.  attempts to depict alloys) while retaining the majority of correct entries.

When applying this rule, pharmaceutical molecules represent a major source of false positives because they are often multiple hydrates and/or salts with multiple counterions (e.g. Irinotecan hydrochloride trihydrate). Excluded structures that are hydrates or contain common pharmaceutical salts are flagged for human review.

Synonym filter

This filter compares the synonyms assigned to a given structure with its molecular formula and performs some ‘common sense’ checks. For example, a relatively frequent error is associating the name of a salt form (e.g., mozavaptan hydrochloride) with the structure of the free base (mozavaptan). In this case, the filter removes synonyms containing ‘hydrochloride’ because the molecular formula does not contain Cl.

SMARTS

SMARTS (Wikipedia page) is a way of describing general chemical structures. It’s based on SMILES, but has additional features allowing the specification of variable chain lengths, number of bonds, number of hydrogens, variable bond orders, or more than one potential element at a site.

We use SMARTS to identify common erroneous features in a structure. These include:

  • Azides and diazo groups depicted with a pentavalent nitrogen
  • A ‘floating’ alkane unconnected to the main structure (probably caused by an accidental click in a drawing program)
  • Metal carboxylates depicted as a protonated carboxylic acid with an elemental metal atom
  • Hexafluorophosphates (and similar species) depicted as phosphorous pentafluoride and a separate fluoride ion

SMIRKS

SMIRKS is a further extension of SMILES to depict reactions. We don’t use it to represent real reactions, but to define structural transformations – allowing us to fix simple structural errors that can be resolved by breaking and creating bonds.

One example is connecting charge-separated Grignard reagents to give a more accurate depiction:

Reconnecting disconnected Grignard reagents

Reconnecting Grignards

Organometallics

The difficulties of encoding organometallic structures in machine-readable formats are well documented (J. Chem. Inf. Model. 51, 12, 3149-3157). There is an ongoing IUPAC project to extend the InChI’s functionality, but for now, the challenges remain.

Every ChemSpider record is fundamentally based on an InChI, and so we are bound by the current limitations. This means that we can’t depict coordination bonds or bonds with non-integer order – any bond drawn is interpreted as a standard covalent bond with one electron contributed by each atom.

Although we generally can’t represent organometallic structures in the manner a human chemist would prefer, we still attempt to choose the ‘least wrong’ structure from various possible compromises.

Ferrocene is a classic example of this problem and illustrates several of the issues we have to consider. A few common ways to draw ferrocene are shown below (there are many more).

Common depictions of ferrocene lose bonding information when converted to mol files

Converting ferrocene structures to mol format can introduce errors in molecular formula, bond orders or valence

 

Most of the structures shown take advantage of extended features of chemical drawing packages in order to represent ferrocene’s bonding in a way that’s attractive and easily understandable to a human chemist. Unfortunately, once transferred to the simplified but universal mol format, some of those features are lost, resulting in nonsensical structures. Although structure D is unchanged, this representation has other problems: incorrect valence on Fe and no representation of the aromaticity of the cyclopentadienyl ligands.

We have a limited number of ways in which we can depict ferrocene and related structures in ChemSpider, none of which give an accurate representation of the bonding or a view that would satisfy an inorganic chemist. However, we can choose the ‘least bad’ of the possible compromises and allow machine readability:

Fe2+ and (C5H5-)2

Our compromise

Although this structure (ChemSpider record) doesn’t capture the hapticity of ferrocene and the charge localisation on a single carbon is inaccurate, it retains correct overall charges and valences and doesn’t show the ligands as sigma-bonded.

More generally, we apply some rules and transformations to standardise representations of organometallic structures. Many of these rules involve choosing whether to depict a metal–carbon (or metal–heteroatom) as covalent or ionic, depending on the nature of the metal and the ligand. Again, compromises are necessary when working within the limitations of machine-readable structures, but we attempt to classify ‘more ionic’ and ‘more covalent’ bonds. Some examples follow:

  • Disconnect oxygen from group 1 and 2 metals
  • Connect oxygen to all other metals
  • Disconnect carbon from sodium, potassium and calcium
  • Connect carbon to group 11 and 12 metals, p-block metals and some metalloids

As expected, general rules like these fail in certain cases. Therefore we have additional, more specific rules to cover exceptions, which we iteratively refine.

But these errors still appear in ChemSpider!

At present the filtering described only applies to new data coming into ChemSpider. The full ChemSpider database, built up over many years, certainly contains examples of every error described here. To fix these legacy errors, we intend to run the entire database through the same quality filters. This is a significant task with some specific challenges: the files requiring human review become orders of magnitude larger, the processing time and memory/CPU overhead is high, and the larger the data set the more likely we will run into false positives. In order to manage these challenges, we are taking the time to refine our processes on new depositions, and periodically checking our progress by running subsets of the full ChemSpider database through our filters. We know you need access to data you can trust, so we want to make sure we get this right. We’ll continue to update you as this project progresses, so stay tuned!

Behind the Scenes at ChemSpider

A peek at who we are, how we run the site, and how we manage data quality.

What is ChemSpider and who runs the service?

ChemSpider is one of the largest chemical databases in the world, containing data on over 65 million chemical structures. This data is freely available to the public at ChemSpider.com, a website published by the Royal Society of Chemistry.

How does the Royal Society of Chemistry support ChemSpider?

ChemSpider.com is an independent service that does not rely on direct or research grant funding. The Royal Society of Chemistry supports the website using the surplus generated by our publishing activities, allowing us to provide a sustainable and reliable service. We also generate revenue from advertising and by providing paid for web services, such as our APIs, for non-academic users. These activities help keep ChemSpider financially sustainable and help support our server costs, staff hours and development.

These services enable us to make the site available free anyone in the world, and we reached over six million unique users in 2017. These users range from school students looking for help with their homework, to researchers working in academia and industry, to general users who want to keep their chemical knowledge up to date. They come from every continent except Antarctica, and just about every country on Earth.

What goes into ChemSpider?

ChemSpider data comes from the chemical sciences community itself – submitted by researchers, databases, publishers, chemical vendors and many more.

We have two main inclusion criteria for ChemSpider data:

  1. Machine readability – Depositors must provide structures in a machine-readable format, typically a .mol file that is interpretable by InChI – the open-source chemical structure representation algorithm.The .mol format describes how a compound is arranged, atom-by-atom and bond-by-bond. This means that it can only accurately depict small molecules with defined structures. For ChemSpider, “small” means structures up to 4000 daltons, including short peptides, oligonucleotides, and other structures. Large proteins, extended crystal lattices or long nucleotides are too big to describe sensibly in ChemSpider, but are available from other databases suited for larger molecules.

    We also only accept ‘defined structures’ – compounds with exact chain lengths, fully expressed functional groups, and integer bond orders – due to the requirement to describe every heavy atom in a molecule. This means we can only accept structures for which we can generate a valid InChI.

    Most ChemSpider structures are organic molecules. However, we do accept some inorganic and organometallic compounds, with specific methods for curating these.

  2. Real compounds – We do not accept virtual or prophetic compounds.

As far as possible, we only accept compounds that have been synthesised or isolated in physical form. This means we do not accept transition states, theoretically predicted compounds, virtual compounds from vendors or prophetic compounds from patents.

Who are our data sources?

We have received data from almost 250 unique data sources, including data from chemical vendors, specialist databases, individuals, research groups and publishers. These sources cross the breadth of the chemical sciences – including biochemistry, pharmacology and toxicology, natural products, spectroscopy and crystallography. Each ChemSpider record includes links to all of the data sources for the compound, enabling users to find and to check the provenance of the data.

Our data source list is continually changing, as we find new sources of data to add and remove outdated or low-quality data sources.

We no longer accept data from other data aggregators. We have taken this step to match our quality requirements with other databases and reduce the propagation of algorithmically generated errors that can arise from prophetic sources. One example of this is Chessboardane, which originated from an optical structure recognition program interpreting a data table contained within a patent as a chemical structure. The result was an 81-carbon grid structure, erroneously identified as a complex cyclic alkane, which was deposited in a public repository and shared between multiple aggregators.

Because of this, we only seek data directly from the original sources, where we have greater certainty about the data’s provenance and accuracy, and are working to curate legacy data still within ChemSpider.

Because of examples like Chessboardane, we are cautious about accepting data from text-and-data-mined sources that depositors have programmatically extracted from text or encoded images in patents or scientific literature. After review, we have added some of the highest quality data mined sources. We will continue to review potential new data-mined sources on a case-by-case basis to ensure that their data meet our quality standards.

Automated filters

A manual check of every one the 65 million records in ChemSpider would take an individual more than 600 years to complete working round the clock – even if we only invested five minutes of curation time per record.

Instead, we run each deposition through a series of automated filters to pick out unsuitable structures, such as those with incorrect valences, unbalanced charges, or missing stereochemistry. In addition to structure filters, we also apply basic name and synonym filtering and regularly review the processed files so that we can improve our filters.

We have provided a simplified overview of this process below, and will provide a more detailed description of our filters in a separate blog post:

Structures are run through filters in KNIME. Those that fail the filters are removed and reviewed. Passed structures are deposited to ChemSpider

Curation by ChemSpider staff

ChemSpider is run by a small team of full-time curators, who work to add new compounds, remove errors, and respond to user feedback. Our staff have extensive experience of both chemical data and practical chemistry, with backgrounds in fields such as organic synthesis and art conservation, and a wealth of experience working on other Royal Society of Chemistry databases, such as The Merck Index* Online and Analytical Abstracts.

Community curation

Because we cannot review every record ourselves, we really appreciate comments or corrections from our users.  The easiest way to help us improve ChemSpider is to leave feedback or email us when you spot an error. We try to act on user feedback within a few days – sooner for simpler queries. Please let us know if you find an error by leaving a comment on the relevant ChemSpider record, or by emailing us (chemspider@rsc.org).

Users wishing to get more involved can directly deposit structures and curate synonyms related to their research or work, without having to email the ChemSpider team.

We are extremely grateful for all the contributions our community curators have made over the years.

Keep using and contributing to ChemSpider

To access information on over 65 million chemical structures, go to ChemSpider.com, which is fully searchable by structure, name, or advanced query, from any device, anywhere, for free.

To deposit data, tell us about an error, become a curator, or for any other query, please do not hesitate to email us at chemspider@rsc.org

*The name THE MERCK INDEX is owned by Merck Sharp & Dohme Corp., a subsidiary of Merck & Co., Inc., Whitehouse Station, N.J., U.S.A., and is licensed to The Royal Society of Chemistry for use in the U.S.A. and Canada.