Archive for the ‘Behind the scenes’ Category

See the future of ChemSpider

18 Jan 2024

For several months now we have been working hard to rebuild ChemSpider from the ground up. This is redeveloping the technical implementation of the site, as well as a complete reassessment of the website; how it looks, how it works and the data that we present.

We’ve been careful to retain all the familiar features of the site and particularly the records pages, but by looking at the way visitors to the site use the functionality, surveys of users, and user interviews we have made changes to provide a cleaner and simpler interface. We hope this new ChemSpider site, while not a complete copy of the features in the old site, will provide access to all the information that you need in a more intuitive and user-friendly way.

While we continue to refine the new site we have made a preview version of the new interface available as Beta.ChemSpider.com alongside the existing ChemSpider site. Beta.ChemSpider only contains the data for the first 5 million ChemSpider records for the moment. Once we are happy that the new site is ready it will replace the current ChemSpider site.

For the moment the new website is running in parallel to the existing https://chemspider.com site. This allows us to provide you with the ability to try improved design, provide feedback, and should you need to still switch back to the familiar interface. We are still adding functionality and tweaking the data, but your feedback will help us to validate where we have made improvements and where we still need to do more.

To try our new site go to: https://beta.chemspider.com/

FAQs

1. Where is Feature X/Data Y on beta.chemspider.com?

Some record tabs which were based on out-of-date or limited availability data have been retired. Other features like structure searching are yet to be added to the new website as we look for a solution to replace the one currently used on ChemSpider.com.

2. Should I use Beta.ChemSpider?

This a decision for you to make after you’ve tried the site. While we don’t have ChemSpider IDs from 5,000,001 to 129,000,000 in the beta site we have data for many common compounds. Please try it out and decide for yourself.

3. What does beta mean?

It means that things might change on the beta.chemspider.com site, features might be added, removed or temporarily break. We are also working on how we process the data that is loaded in the site and so sometimes the data might change as we reload the data.

4. Can you add Feature A/Data B?

We are always interested in new sources of data and features and welcome any suggestions, it might take some time for us to get to reviewing/implementing these.

5. What about ChemSpider accounts?

For the moment the new website doesn’t need any sign-in to use it, as ChemSpider accounts were not well used. We may have new features in the future that will require a login, but for now we won’t have one.

6. How do I provide feedback?

We want to know what you like or dislike about the new site, please do fill in the in-site surveys/feedback forms, or send an email to chemspider@rsc.org.

Comments Off on See the future of ChemSpider

ChemSpider data cleanup

15 Dec 2023

In previous posts, we have discussed the automated workflow we use to check new incoming data for structure and synonym errors. These checks allow us to remove the most common types of errors before they are added to the site. However, these filters do not apply to data already in ChemSpider.

Manual curation is an important part of our work. We periodically review the data on our most accessed records, in addition to ad-hoc removal or correction of erroneous data that we or our users notice when using the site. However, there are far too many records and far too much data to clean up using manual curation alone.

Recently we have focused on bulk identification and removal of erroneous data. This work has covered mapping errors and other clearly incorrect values in our experimental property data, correction or removal of malformed synonyms, correction of incorrectly labelled synonyms, and resolution of structure/synonym clashes.

Experimental Properties

We retrieved all 6.3 million experimental properties, text properties, and associated annotations from the ChemSpider database. We then compared the original text of the property as it was written in the original file to how that text was parsed and mapped by our deposition system. This enabled us to identify and correct several types of errors affecting around 2% of the properties in our database:

35,774 experimental property values had been assigned the incorrect unit (e.g. g/L instead of g/mL, °C instead of °F)
2,591 boiling points measured under non-standard pressure did not have this pressure displayed
4,292 densities had their density and temperature values swapped
79,252 miscellaneous erroneous properties and associated annotations were deleted. For example, “white crystals” mapped as melting point, impossibly high melting points or densities, etc.

Synonyms

Synonyms, chemical names, and identifiers are the most abundant type of data on ChemSpider, with a total of more than 446 million synonyms. These synonyms have additional metadata including language labels and flags identifying what type of synonym they are (e.g. CAS number, UNII, INN, trade name).

Simple Checks

We ran a series of regular expression string searches to identify synonyms with incorrect metadata, as well as malformed or otherwise erroneous synonyms.

200,007 synonym type flags added, and 4,766 incorrect flags removed
9,170 synonyms with an incorrect language label identified.
631,697 erroneous synonyms identified, including scrambled characters, properties/units, molecular formulae as synonyms, purity information, or invalid CAS numbers or EC numbers (formerly called EINECS).
922,334 instances of these erroneous synonyms deleted from ChemSpider records.

Structure/Synonym comparison

After identifying and removing these synonym-level errors, we then cross-checked ChemSpider records and their synonyms to identify mismatches. This work included amino acids, nucleic acids, and pharmaceutically acceptable salts.

As a first pass, we compared synonyms to molecular formulae to identify records missing key elements. Examples include synonyms describing a sodium salt when the molecular formula does not contain sodium, or describing an amino acid when the molecular formula contains no nitrogen. A total of 28,194 of these synonym/formula clashes were identified and removed.

For records that passed this initial molecular formula check, we performed a SMARTS comparison to identify chemical structures missing key structural features described in the synonym. These SMARTS strings were written broadly, with common substitutions allowed to prevent unnecessary removal of valid synonyms from derivative compounds.

In the following examples, the mismatched part of the synonym is highlighted in bold.

Structure	Removed synonym
	Sulfate ion
	Zolpidem tartrate
	Sodium S-sulfocysteine hydrate

After identifying these clashes, we manually spot-checked the output to weed out false positives and iterate the SMARTS filters. 101,257 synonym/structure clashes were identified and removed.

These checks included the following categories:

Amino acids and their derivatives: 6 formula clashes, 56 structure clashes
Nucleic acids, nucleosides, nucleotides: 977 formula clashes, 1,870 structure clashes
Halogens: 13,437 formula clashes, 1,256 structure clashes
Alkali and alkaline earth metals, and aluminium: 3,586 formula clashes, 56 structure clashes
Carboxylic acids and their derivatives: 5,002 formula clashes, 88,501 structure clashes
Other pharmaceutically acceptable acids: 3,534 formula clashes, 1,529 structure clashes
Amides and amines: 190 formula clashes, 304 structure clashes
Deuterates, hydrates, methylbromides: 1,462 formula clashes, 7,685 structure clashes

Get involved

You are the expert in your area of chemistry, so if you see something that doesn’t look quite right please let us know. If the error is confined to a single ChemSpider record, click the “Comment On This Record” box at the top of the affected record and let us know what the problem is. All we need is a sentence describing the error, however the more information you can provide, the better.

For more systemic errors, or in cases where you want to attach supplementary information or corrected chemical structures, please get in touch via email (chemspider@rsc.org).

Comments Off on ChemSpider data cleanup

ChemSpider Pre-Deposition Filters

18 Sep 2018

Written by Mark Archibald.

In a previous post (Behind the Scenes at ChemSpider) we discussed some of the challenges in upholding data quality across one of the largest chemical databases in the world. We identified automated filtering as a key tool when dealing with far more records than a human could reasonably handle. In this post we’ll go into more detail about how that filtering works, what the challenges are, and the role played by human intervention.

To perform this filtering we use KNIME, an open-source data processing platform. The wide range of KNIME nodes developed by the active cheminformatics community allows us to ask chemistry-specific questions of the data we process. In simple terms, input chemical structures that match our criteria are passed on to the next node, while those that don’t are written out to an error file. After processing all structures, the result is a file of structures that have successfully passed through all the filters and several (usually smaller) files of structures rejected for various reasons.

It’s not possible to review all of the generated files in full, as this would eliminate the time-saving advantages of automated processing. However, output files of all types are spot checked for accuracy and to iteratively improve the filtering criteria. Certain output files have high potential for false positives and so we review them in full.

Formats and identifiers

Submitted files can be in one of several different formats. The most common is SDF (structure data file, a chemical structure format containing multiple structures with associated data fields). The advantage of this format is that it contains 2- or 3-dimensional structures, so we can immediately start processing the file without having to convert an identifier to a structure. This means that the final structure we deposit is more likely to exactly match the original. The disadvantage of the SDF format is that it is specialised – many users will be unfamiliar with it or won’t have software to create and display the files.

We also receive different spreadsheet formats (excel, csv, tsv) with structures encoded in text-based notation systems like SMILES or InChI. The advantage of this format is that it doesn’t require specialised software (provided the submitter has SMILES or InChIs for the compounds).The disadvantage is that the structures require conversion to SDF before processing and deposition to ChemSpider. Additionally, these formats contain information about atoms and their connectivity but lack layout information. This can introduce errors as different structure drawing packages can parse these structures slightly differently, resulting in alterations to the final deposited structure.

Filtering criteria

The criteria by which we judge chemical structures are a mixture of definitive chemical rules and less well-defined ‘rules of thumb’ based on our experience and chemical knowledge. Examples of both follow.

Empty structures, query atoms and incorrect valences

The first filter is the simplest – ChemSpider is a structure-centric database, so it’s not possible to deposit any input entries that lack a structure.

Similarly, each ChemSpider record requires a single defined chemical structure, so we exclude anything using a query atom to represent a variable atom or attachment point.

Another simple filter is to exclude structures in which atoms have invalid valences.

Charge imbalance

In general, entries in ChemSpider should represent a real-world, isolable compound. This means that we filter out structures with a non-zero overall charge. However, we make exceptions for certain examples where a counterion is generally unimportant and it’s useful to consider the charged species alone, such as choline (ChemSpider record).

Structures containing undefined stereocentres

Undefined stereocentres alone don’t represent a chemical error. However, structures like that shown below (cholesterol without any defined stereocentres) occur frequently and, although chemically valid, it’s extremely unlikely that they represent the intended structure.

Cholesterol skeleton with no defined stereochemistry

Cholesterol skeleton without stereochemistry

As a result we have a rule of thumb that excludes structures containing more than two undefined stereocentres. This is not a hard-and-fast rule, but rather an attempt to strike a balance between excluding structures like the one above and including structures where the undefined stereocentres are intentional and correct.

The count of undefined stereocentres (as determined by examining the InChI) sometimes includes cases where it is conventional to exclude stereochemical wedges. Examples include nucleic acids with no wedges on the phosphate and adamantyl groups without explicit stereochemistry – it’s unusual to draw these compounds with wedges, and users will rarely use wedges in their search. These potential false positives are filtered out and reviewed manually. A curator can then decide whether to include them in the deposition, improving the overall accuracy of the filter.

Structures containing many components

This is another rule of thumb – there’s no upper limit on how many separate components a correctly depicted chemical substance can have. However, from experience we find that excluding structures with more than four separate components removes most obviously nonsensical entries (e.g. attempts to depict alloys) while retaining the majority of correct entries.

When applying this rule, pharmaceutical molecules represent a major source of false positives because they are often multiple hydrates and/or salts with multiple counterions (e.g. Irinotecan hydrochloride trihydrate). Excluded structures that are hydrates or contain common pharmaceutical salts are flagged for human review.

Synonym filter

This filter compares the synonyms assigned to a given structure with its molecular formula and performs some ‘common sense’ checks. For example, a relatively frequent error is associating the name of a salt form (e.g., mozavaptan hydrochloride) with the structure of the free base (mozavaptan). In this case, the filter removes synonyms containing ‘hydrochloride’ because the molecular formula does not contain Cl.

SMARTS

SMARTS (Wikipedia page) is a way of describing general chemical structures. It’s based on SMILES, but has additional features allowing the specification of variable chain lengths, number of bonds, number of hydrogens, variable bond orders, or more than one potential element at a site.

We use SMARTS to identify common erroneous features in a structure. These include:

Azides and diazo groups depicted with a pentavalent nitrogen
A ‘floating’ alkane unconnected to the main structure (probably caused by an accidental click in a drawing program)
Metal carboxylates depicted as a protonated carboxylic acid with an elemental metal atom
Hexafluorophosphates (and similar species) depicted as phosphorous pentafluoride and a separate fluoride ion

SMIRKS

SMIRKS is a further extension of SMILES to depict reactions. We don’t use it to represent real reactions, but to define structural transformations – allowing us to fix simple structural errors that can be resolved by breaking and creating bonds.

One example is connecting charge-separated Grignard reagents to give a more accurate depiction:

Reconnecting Grignards

Organometallics

The difficulties of encoding organometallic structures in machine-readable formats are well documented (J. Chem. Inf. Model. 51, 12, 3149-3157). There is an ongoing IUPAC project to extend the InChI’s functionality, but for now, the challenges remain.

Every ChemSpider record is fundamentally based on an InChI, and so we are bound by the current limitations. This means that we can’t depict coordination bonds or bonds with non-integer order – any bond drawn is interpreted as a standard covalent bond with one electron contributed by each atom.

Although we generally can’t represent organometallic structures in the manner a human chemist would prefer, we still attempt to choose the ‘least wrong’ structure from various possible compromises.

Ferrocene is a classic example of this problem and illustrates several of the issues we have to consider. A few common ways to draw ferrocene are shown below (there are many more).

Common depictions of ferrocene lose bonding information when converted to mol files

Converting ferrocene structures to mol format can introduce errors in molecular formula, bond orders or valence

Most of the structures shown take advantage of extended features of chemical drawing packages in order to represent ferrocene’s bonding in a way that’s attractive and easily understandable to a human chemist. Unfortunately, once transferred to the simplified but universal mol format, some of those features are lost, resulting in nonsensical structures. Although structure D is unchanged, this representation has other problems: incorrect valence on Fe and no representation of the aromaticity of the cyclopentadienyl ligands.

We have a limited number of ways in which we can depict ferrocene and related structures in ChemSpider, none of which give an accurate representation of the bonding or a view that would satisfy an inorganic chemist. However, we can choose the ‘least bad’ of the possible compromises and allow machine readability:

Our compromise

Although this structure (ChemSpider record) doesn’t capture the hapticity of ferrocene and the charge localisation on a single carbon is inaccurate, it retains correct overall charges and valences and doesn’t show the ligands as sigma-bonded.

More generally, we apply some rules and transformations to standardise representations of organometallic structures. Many of these rules involve choosing whether to depict a metal–carbon (or metal–heteroatom) as covalent or ionic, depending on the nature of the metal and the ligand. Again, compromises are necessary when working within the limitations of machine-readable structures, but we attempt to classify ‘more ionic’ and ‘more covalent’ bonds. Some examples follow:

Disconnect oxygen from group 1 and 2 metals
Connect oxygen to all other metals
Disconnect carbon from sodium, potassium and calcium
Connect carbon to group 11 and 12 metals, p-block metals and some metalloids

As expected, general rules like these fail in certain cases. Therefore we have additional, more specific rules to cover exceptions, which we iteratively refine.

But these errors still appear in ChemSpider!

At present the filtering described only applies to new data coming into ChemSpider. The full ChemSpider database, built up over many years, certainly contains examples of every error described here. To fix these legacy errors, we intend to run the entire database through the same quality filters. This is a significant task with some specific challenges: the files requiring human review become orders of magnitude larger, the processing time and memory/CPU overhead is high, and the larger the data set the more likely we will run into false positives. In order to manage these challenges, we are taking the time to refine our processes on new depositions, and periodically checking our progress by running subsets of the full ChemSpider database through our filters. We know you need access to data you can trust, so we want to make sure we get this right. We’ll continue to update you as this project progresses, so stay tuned!

Comments Off on ChemSpider Pre-Deposition Filters

Behind the Scenes at ChemSpider

26 Jun 2018

A peek at who we are, how we run the site, and how we manage data quality.

What is ChemSpider and who runs the service?

ChemSpider is one of the largest chemical databases in the world, containing data on over 65 million chemical structures. This data is freely available to the public at ChemSpider.com, a website published by the Royal Society of Chemistry.

How does the Royal Society of Chemistry support ChemSpider?

ChemSpider.com is an independent service that does not rely on direct or research grant funding. The Royal Society of Chemistry supports the website using the surplus generated by our publishing activities, allowing us to provide a sustainable and reliable service. We also generate revenue from advertising and by providing paid for web services, such as our APIs, for non-academic users. These activities help keep ChemSpider financially sustainable and help support our server costs, staff hours and development.

These services enable us to make the site available free anyone in the world, and we reached over six million unique users in 2017. These users range from school students looking for help with their homework, to researchers working in academia and industry, to general users who want to keep their chemical knowledge up to date. They come from every continent except Antarctica, and just about every country on Earth.

What goes into ChemSpider?

ChemSpider data comes from the chemical sciences community itself – submitted by researchers, databases, publishers, chemical vendors and many more.

We have two main inclusion criteria for ChemSpider data:

Machine readability – Depositors must provide structures in a machine-readable format, typically a .mol file that is interpretable by InChI – the open-source chemical structure representation algorithm.The .mol format describes how a compound is arranged, atom-by-atom and bond-by-bond. This means that it can only accurately depict small molecules with defined structures. For ChemSpider, “small” means structures up to 4000 daltons, including short peptides, oligonucleotides, and other structures. Large proteins, extended crystal lattices or long nucleotides are too big to describe sensibly in ChemSpider, but are available from other databases suited for larger molecules.
We also only accept ‘defined structures’ – compounds with exact chain lengths, fully expressed functional groups, and integer bond orders – due to the requirement to describe every heavy atom in a molecule. This means we can only accept structures for which we can generate a valid InChI.

Most ChemSpider structures are organic molecules. However, we do accept some inorganic and organometallic compounds, with specific methods for curating these.
Real compounds – We do not accept virtual or prophetic compounds.

As far as possible, we only accept compounds that have been synthesised or isolated in physical form. This means we do not accept transition states, theoretically predicted compounds, virtual compounds from vendors or prophetic compounds from patents.

Who are our data sources?

We have received data from almost 250 unique data sources, including data from chemical vendors, specialist databases, individuals, research groups and publishers. These sources cross the breadth of the chemical sciences – including biochemistry, pharmacology and toxicology, natural products, spectroscopy and crystallography. Each ChemSpider record includes links to all of the data sources for the compound, enabling users to find and to check the provenance of the data.

Our data source list is continually changing, as we find new sources of data to add and remove outdated or low-quality data sources.

We no longer accept data from other data aggregators. We have taken this step to match our quality requirements with other databases and reduce the propagation of algorithmically generated errors that can arise from prophetic sources. One example of this is Chessboardane, which originated from an optical structure recognition program interpreting a data table contained within a patent as a chemical structure. The result was an 81-carbon grid structure, erroneously identified as a complex cyclic alkane, which was deposited in a public repository and shared between multiple aggregators.

Because of this, we only seek data directly from the original sources, where we have greater certainty about the data’s provenance and accuracy, and are working to curate legacy data still within ChemSpider.

Because of examples like Chessboardane, we are cautious about accepting data from text-and-data-mined sources that depositors have programmatically extracted from text or encoded images in patents or scientific literature. After review, we have added some of the highest quality data mined sources. We will continue to review potential new data-mined sources on a case-by-case basis to ensure that their data meet our quality standards.

Automated filters

A manual check of every one the 65 million records in ChemSpider would take an individual more than 600 years to complete working round the clock – even if we only invested five minutes of curation time per record.

Instead, we run each deposition through a series of automated filters to pick out unsuitable structures, such as those with incorrect valences, unbalanced charges, or missing stereochemistry. In addition to structure filters, we also apply basic name and synonym filtering and regularly review the processed files so that we can improve our filters.

We have provided a simplified overview of this process below, and will provide a more detailed description of our filters in a separate blog post:

Structures are run through filters in KNIME. Those that fail the filters are removed and reviewed. Passed structures are deposited to ChemSpider

Curation by ChemSpider staff

ChemSpider is run by a small team of full-time curators, who work to add new compounds, remove errors, and respond to user feedback. Our staff have extensive experience of both chemical data and practical chemistry, with backgrounds in fields such as organic synthesis and art conservation, and a wealth of experience working on other Royal Society of Chemistry databases, such as The Merck Index* Online and Analytical Abstracts.

Community curation

Because we cannot review every record ourselves, we really appreciate comments or corrections from our users. The easiest way to help us improve ChemSpider is to leave feedback or email us when you spot an error. We try to act on user feedback within a few days – sooner for simpler queries. Please let us know if you find an error by leaving a comment on the relevant ChemSpider record, or by emailing us (chemspider@rsc.org).

Users wishing to get more involved can directly deposit structures and curate synonyms related to their research or work, without having to email the ChemSpider team.

We are extremely grateful for all the contributions our community curators have made over the years.

Keep using and contributing to ChemSpider

To access information on over 65 million chemical structures, go to ChemSpider.com, which is fully searchable by structure, name, or advanced query, from any device, anywhere, for free.

To deposit data, tell us about an error, become a curator, or for any other query, please do not hesitate to email us at chemspider@rsc.org

*The name THE MERCK INDEX is owned by Merck Sharp & Dohme Corp., a subsidiary of Merck & Co., Inc., Whitehouse Station, N.J., U.S.A., and is licensed to The Royal Society of Chemistry for use in the U.S.A. and Canada.

Comments Off on Behind the Scenes at ChemSpider

Introduction to the new ChemSpider website

21 May 2015

Blog post written by David Sharpe.

The ChemSpider team at the Royal Society of Chemistry is proud to announce that our new look ChemSpider website has been launched. As discussed in our last post one of the key features of this new design is to make ChemSpider work on as many devices (from desktops to mobile phones).

The ChemSpider homepage as it might appear on a desktop computer (left) and a mobile phone (right)

As the screenshots above illustrate, the difference in size, shape and the method of interacting with the page means the view of the website that you need is very different between devices. The nature of a responsive website design also means that some of the screenshots that we provide might be a little different from the view that you see when accessing the site, however the variances should be clear. We hope this results in an experience where usability and readability are not sacrificed for functionality.

What has changed? … and what has stayed the same?

To start with the things that have stayed the same: ChemSpider is still based on the same quality-data and provides mechanisms for users to supply and curate data. We also haven’t changed how the search queries work, so searches that you ran previously should still return the same results.

The key changes

1. The new page header

We’ve moved all of the old menu items into a bar at the very top of every page (1), we also display a search bar just above the main page content (2). On smaller displays you will see icons for the Quick search box, Sign In and Help items, all other options can be found under the ‘hamburger’ symbol (3).

Comparison of the ChemSpider page header on large and small screens

2. Shorter record pages

One of the biggest challenges of making ChemSpider work on a mobile is how to display all of the information that we have on a much smaller screen. I think that our solution will actually make ChemSpider better for everyone – regardless of how they view the site.

Previously, a ChemSpider record was one big long page that had basic details about the chemical structure at the top of the page, followed by a number of infoboxes that could be opened or closed and also re-ordered. This worked fine in most cases but led to situations where you had to do lots of scrolling up and down, and might not be able to spot the infobox that you were looking for. Now, we still show some information about the chemical structure at the top of the page (1) but, below that there is a single pane (2) which contains tabs (3) that allow you to select the section of the record that you wish to display. This means that it is always easy to look at some information and see the structure to which it relates.

The new page layout consists of a Compound header (1) and a Pane (2) diplaying the contents of the infotabs (3)

3. No Java, No worries

Many browsers no longer support Java applets. Good Java-free versions of chemistry tools have really started to take off in the last 18 months and the time was right to start the switch over. This means that the site now incorporates JSmol – enabling 3D structure view, CIF viewer and NMR/IR/MS spectra display as well as Ketcher and Elemental for structure input/editing.

4. Structure searches simplified

Previously, creating a structure search was a bit of a pain as you had to: open the structure editor in a pop-up, draw your structure, and then save it back into the searches pages – now our structure editors are embedded into the interface, cutting down the number of steps needed to get to your results and making it easier to tweak searches.

One particularly useful feature for anyone accessing the site on a tablet or mobile phone is the Convert Structure tab which can be used to load in a complex structure as a basis for a search, for instance using “dibenzylamine” in the structure conversion gives a structure that can be quickly elaborated to the Simpkins’ chiral base precursor amine shown in the the screenshot.

What’s next?

Hold on a moment there! We’ve only just got all of these great features into the site! I’m joking, but we will be spending time tweaking and perfecting the new design. We will then be able to focus on further development, if I were to speculate – I’d suggest that we will look at more (non-Java) tools that can be incorporated into the site to give a better experience, and new methods of improving the quality of data in our records.

In the meantime, please explore the site and do email us at chemspider-at-rsc.org to let us know what you think of the new site.

Comments Off on Introduction to the new ChemSpider website

What’s new with ChemSpider?

15 May 2015

Blog post written by David Sharpe.

Subscribers to this blog might have noticed that we’ve been a bit quiet of late. I want to assure that this doesn’t mean that we have been resting on our laurels. In fact we have been working on a whole host of improvements to ChemSpider – improving our infrastructure, developing ways to increase data quality and designing a new layout for our records.

We will discuss both the data quality work and the website redesign work in more detail in separate posts but ahead of the release of the new website design I want to provide some insight into what to expect when the changes go live.

Why are we changing the site now?

Well there are quite a few reasons:

Primarily, we need to have a site that meets the standards of the modern internet. This means that the site needs to be usable not only on a desktop computer but also on a tablet or a mobile phone. This is often referred to as responsive web design
ChemSpider has always had records that are full of lots of rich and varied types of information – which poses a challenge when it comes to presenting that information so that it is discoverable and easy to understand once found. We hope that the new layout will present data in an intuitive and clear way that will provide a better experience for everyone.
We need to move away from technologies that are not supported by the widely used browsers. Java-based tools have been an issue for users on certain platforms for a while and this is only going to get worse. For a long time we have provided non-Java structure editors alongside the Java tools (the current version of the site incorporates Elemental and Ketcher for structure drawing. This release will see the adoption of JSmol to enable 3D structure view and Spectra display widgets for devices that don’t support Java . At this time we are providing both Java and non-Java solutions but expect to phase out Java applets in the near future.
Improve the integration of ChemSpider with the wider Royal Society of Chemistry web family

Will there be any more changes to how the site works?

There will certainly be some changes to some aspects of the site due to: user feedback and bug fixes. We also what to look at how we can make more complex interfaces such as Advanced Search more usable, but we hope that there won’t be any major changes to the site.

Will all of the features that you use still be accessible?

In the main, the answer to this is; Yes! It might be that they now appear slightly differently or be accessible through a different interface. There are 2 caveats:

When accessing the site on mobile devices

The layout of a page on the smaller screens and tablets often needs to be different – wherever possible this is achieved by rearranging the elements of the page and adding new controls. But for some parts of the ChemSpider interface we realised that there wasn’t a good way to display all of the data and the only solution was not to show that part of the page on these smaller screens.
Removed features

There are a couple of features (such as the Print button) which we felt were no longer relevant in the new design or need to be redesigned to make them more usable.

When will the new site be launched?

We hope that the new site will be ready to release within the next week.

How will the changes affect you?

We hope that the transition will be smooth for everyone. Once the new design goes live you might need to refresh/clear your Browser Cache. The new design does require a modern browser with a good support of the HTML 5 specification. We will try to ensure that the site is usable on as wide a range of browsers and platforms as possible but expect that the site will not work well in older browsers such as IE7.

Will it still be possible to access the site using the old interface?

Unfortunately, the old interface will not be available alongside the new one.

How will you be able to provide feedback on the new design?

The best way to provide feedback will be to email us at chemspider-at-rsc.org

Keep an eye out for the new design – when it is made live we will write a blog post about the changes.

Comments Off on What’s new with ChemSpider?

Southampton University internships to transfer thesis data into LabTrove and ChemSpider

25 Sep 2013

Written by Aileen Day.

This summer there have been a number of students from the University of Southampton doing internships on joint projects between the university and the Royal Society of Chemistry and ChemSpider. Three of these students have been sifting through theses from past members of Richard Whitby’s research group in order to extract the compound, spectra and reaction data in it (and linked lab note books, and archive spectra files) and share these in LabTrove, ChemSpider, and CSSP. The students – Alex Hartke, Yet Wai Lee and Josh Whittam (all 2nd year undergraduates) – are shown below together with the boxes of thesis data, lab notebooks and spectra print outs that they digitised.

Southampton University interns

Between them they digitised 7 theses, by A.Henderson, L. Sayer, D. Owen, D.Macfarlane, F. Giustiniano, G. Saluste, J. Stec, which resulted in 1035 LabTrove pages being published to the Whitby Group’s LabTrove blog.

The theses were a rich source of compound information – including compound structures, names, properties and spectra, all of which were also deposited into ChemSpider resulting in 208 new compound pages, and about 600 spectra.

For this project the students manually deposited the compound information into LabTrove and then deposited the compounds and spectra to ChemSpider. However, we are currently developing a range of ChemSpider jquery widgets which can be integrated into web-based ELNs such as LabTrove which will make it easier to enter compound information from ChemSpider into experiments, and also to publish compound and reaction data from the ELNs to ChemSpider, CSSP and ChemSpider Reactions. This will follow on from the initial proof of concept to retreive ChemSpider information and enter it into LabTrove pages.

With this long-term aim in view, the LabTrove pages that the interns stored the compound and reaction data were structured using LabTrove templates, and this structuring will make it easier for publishing widgets to understand the data and process it the correct way. In this way, the project was partly a test to ensure that the templates were suitable for storing compound data in LabTrove. As well as the ChemSpider compound and associated data template (with corresponding help page, templates were also written to store reaction data in a formatted way, since the theses were primarily focused on the synthesis of compounds. At their simplest, basic reaction data can be stored in LabTrove using the ChemSpider Reactions template (and corresponding help page, and eventually posts written in this format will be easily publishable to ChemSpider Reactions. More detailed reaction data can be stored using the ChemSpider SyntheticPages style reaction template (and corresponding help page. The initial aim was to deposit all of this reaction data into ChemSpider SyntheticPages but it became clear that it was difficult for anyone other than the researcher who conducted the reaction, or their superviser to supply the necessary level of detail for CSSP submissions, and in particular couldn’t easily be reached by retrospectively abstracting theses. As a result, only a handful of reactions were submitted to CSSP, and the majority (over 500) were stored in LabTrove for future submission to ChemSpider Reactions.

If reactions can be published easily from ELNs to ChemSpider Reactions and that is easily queryable by other researchers and their applications when performing new reactions this will be a major step towards the aims of the Dial-a-molecule (an EPSRC Grand Challenge network). An important part of the reaction data which needs to be captured is the stoichiometry table of substances used and produced in a reaction. However, these stoichiometry tables are too complicated to incorporate into a LabTrove template, so the LabTrove reaction templates will be used in conjunction with a new ChemSpider jquery widget which is currently in the process of being integrated with LabTrove (more details to follow on this blog shortly!) which will construct them. The widget performs ChemSpider lookups to retrieve compound information, and will calculate equivalents, thereby saving the researcher time when working out the amounts of reactants needed or yields of products obtained. An example of a reaction post which was initially created using the ChemSpider Reactions template and then supplemented by adding a stoichiometry table to it using the ChemSpider Edit Stoichiometry Table widget is shown here.

If you are a LabTrove user and wish to use the ChemSpider templates, their source is available via their links above, and instructions for using templates in Labtrove are documented here.

Comments Off on Southampton University internships to transfer thesis data into LabTrove and ChemSpider

Recent Improvements to ChemSpider Search (part 2)

21 Jun 2013

Last time we told you about a number of improvements we have added to ChemSpider in the recent site updates, including combined substructure and properties search and searching by molecular formula ranges. As promised, this time we will cover how to search by properties like melting point or appearance.

Searching by Supplementary Information

Until now, although you could view properties when you were already on a record, there was no way to search by melting point, refractive index, appearance or bioactivity. This update has implemented a new search interface which allows you to search this data. You can now find compounds that are reported as being isolated from yeast, or compounds with a melting point of 32-35 °C.

There are 2 main parts to our Supplementary search interface.

Text Properties Search

Text properties include appearance, chemical class, drug status, or safety data. You can search any of these properties by using key words. When you start typing, a number of suggested search terms will appear, which can help you narrow down what search term to use.

You can also use wild cards by entering *, which can give you a little more flexibility in your search term – so if your unknown is a blue, crystalline material a search for “Blue crystal*” will turn up all records which mention the word “blue”, as well as any word beginning with “crystal” (such as crystals or crystalline).

Numeric Properties Search

Numeric properties include physical properties like experimental or predicted boiling point, optical rotation, or LogP. Since we draw data from a wide range of data sources, not all of this information is sent to us in the same format or with the units depicted the same way. In order to make it possible for you to search across all the properties in our database no matter how it was supplied to us, we have done a lot of background work on tidying up and standardizing this data.

All numeric properties can be searched using min/max or with a +/- range and the search term can be entered in a variety of units – eg. Fahrenheit or Celsius for temperature, or psi or mmHg for pressure. Because the boiling point of a material is dependent at the pressure at which the measurement is made and not all boiling points are measured at atmospheric pressure we have created a feature that attempts to compensate for this. It uses the Clausius-Clapeyron equation to create estimated (standardised) boiling points for searching, please remember this when looking at your results.

As you can see, you are able to search on a wide variety of experimental properties, including boiling point, LogP, melting point, specific gravity and solubility. Please note that although many of the more common compounds have some properties, these properties are only available on a subset of our records – so if you do not get a result on a property search, it might be that we haven’t added that information yet.

Hopefully this gives you a good idea of the improvements we’ve made to ChemSpider search, and how these new features make it easier than ever to find what you are looking for. See the following post for a case study that showcases several of the new features covered in these posts.

Comments Off on Recent Improvements to ChemSpider Search (part 2)

Recent Improvements to ChemSpider Search (part 1)

13 Jun 2013

We recently published an update to the ChemSpider website which, in addition to fixing a number of bugs, has added some useful new features. Three of these features are highlighted in this post – one which you might have noticed already, and two which you may not have discovered yet.

Auto-Complete

We have reinstated the auto-complete feature on the ChemSpider homepage. Now, when you begin typing in the search box, ChemSpider makes suggestions based on what you have typed. This makes it easier than ever to find what you are looking for – even if you aren’t quite sure how to spell it.

Combined Structure/Property Searches

People frequently ask if there is a way to search substructure and other properties like molecular weight or molecular formula at the same time. This update now makes it possible to perform this kind of combined search from our improved Advanced Search page.

E.g. If you are interested in finding compounds which are structurally similar to Valium, you can enter a benzodiazepinone substructure and restrict it to compounds with a molecular weight of 275-325.

This search then returns Valium along with other similar drugs like clonazepam, nitrazepam and lorazepam.

There are many other search options that can be combined with a substructure/similarity search so look at the Advanced Search page and have a play.

Molecular Formula Range Searching

You can also search a range of molecular formulae at once. To specify the range for a given element, put the range in parentheses after the element. E.g. C7H(10-12)O(0-1) would return all compounds containing exactly 7 carbons and between 10 to 12 hydrogens and which may or may not contain an oxygen. This type of search can be performed from the Simple Search page, as part of an Advanced Search or from the ChemSpider homepage.

Best of all, this can be combined with any of the other search parameters on the Advanced Search page including the substructure search. For example, if you wanted to find polychlorinated biphenyls containing at least three Chlorines you could perform a substructure search for a biphenyl with a molecular formula of C12H(0-7)Cl(3-10).

In our next post, we will cover some new ways you can search by properties that are stored in our records such as melting point, density, etc.

Comments Off on Recent Improvements to ChemSpider Search (part 1)