This publication is free to access through this site. Learn More
Letter
Estimating the Confidence of Peptide Identifications without Decoy DatabasesClick to copy article linkArticle link copied!
View Author Information
Interdisciplinary Center for Scientific Computing, University of Heidelberg, Speyerer Strasse 6, 69115 Heidelberg, Germany, Proteomics Center at Children’s Hospital Boston, 320 Longwood Avenue, Boston, Massachusetts 02115, Department of Pathology, Children’s Hospital Boston and Harvard Medical School, 320 Longwood Avenue, Boston, Massachusetts 02115, and Department of Neurobiology, Harvard Medical School and F. M. Kirby Neurobiology Center, Children’s Hospital Boston, 3 Blackfan Circle, Boston, Massachusetts 02115
* Corresponding author. Hanno Steen, Children’s Hospital Boston, Department of Pathology, Enders 1130, 320 Longwood Avenue, Boston, MA 02115. Fax: +1-617-730-0168. E-mail: [email protected]
†University of Heidelberg.
‡Proteomics Center at Children’s Hospital Boston.
§Department of Pathology, Children’s Hospital Boston and Harvard Medical School.
∥Department of Neurobiology, Harvard Medical School and F. M. Kirby Neurobiology Center, Children’s Hospital Boston.
Abstract
Click to copy section linkSection link copied!
Using decoy databases to compute the confidence of peptide identifications has become the standard procedure for mass spectrometry driven proteomics. While decoy databases have numerous advantages, they double the run time and are not applicable to all peptide identification problems such as error-tolerant or de novo searches or the large-scale identification of cross-linked peptides. Instead, we propose a fast, simple and robust mixture modeling approach to estimate the confidence of peptide identifications without the need for decoy database searches, which automatically checks whether its underlying assumptions are fulfilled. This approach is then evaluated on 41 LC/MS data sets of varying complexity and origin. The results are very similar to those of the decoy database strategy at a negligible computational cost. Our approach is applicable not only to standard protein identification workflows, but also to proteomics problems for which meaningful decoy databases cannot be constructed.
This publication is licensed for personal use by The American Chemical Society.
Copyright © 2010 American Chemical Society
Peptide identifications must be accompanied with confidence statements such as false discovery rates (FDRs) to ensure the reliability of results. Peptides identified with scores below the corresponding cutoffs have a higher likelihood of being random matches than deemed acceptable and thus are disregarded. (1) The standard procedure of obtaining false discovery rates, as suggested by guidelines of major proteomics journals (see, e.g., ref 1), is through decoy databases searches. (4, 16) Databases of nontarget proteins are generated by, e.g., randomizing or reversing the amino acid sequence of the target proteins. The percentage of MS/MS spectra matching these artificial sequences rather than to the original sequence database is regarded as a measure of the (global) FDR. Numerous extensions have been proposed using more elaborate classifiers (9, 7, 8, 2) or semisupervised approaches (11, 3) that only need a subset of decoy hits.
While the decoy database-driven approach and its extensions have clear advantages, there are two major limitations: (i) The run time is increased by a factor of 2 since every MS/MS spectrum needs to be evaluated against all candidates in the decoy database, (10) which should have at least the same size as the original database. Thus, the database search can become a bottleneck for large-scale experiments, which is particularly true for data sets acquired on low-resolution/low-accuracy instruments for which database searches take much longer than for high accuracy data sets. Furthermore, decoy database search strategies are suboptimal for specialized applications such as the identification of cross-linked peptides where both, the target as well as the decoy database, are quadratic in size compared to standard databases. (13) (ii) A suitable decoy database cannot be generated for all applications. For instance, for error-tolerant searches, the actual, correct protein sequence is unknown and amino acid substitutions are allowed. Thus, it cannot be assured that the decoy database contains no potential target peptides since it cannot be filtered a priori against the actual correct sequences. As a consequence, the decoy FDR might be suboptimal since correct identifications could be counted as decoy hits (see the Supporting Information for a simulation). Similarly, for de novo identification strategies, a sequence database is not even required and thus a decoy database cannot be constructed.
Several approaches have been proposed, which do not require a decoy database to estimate a FDR. Kim et al. (10) propose the development of scoring schemes with a realistic probabilistic output for each single spectrum. Current search engines, however, give an unreliable probabilistic output for an individual spectrum since they only have a small number of data points available for the estimate. (10) Other approaches focus on estimating the FDR from all spectra: PeptideProphet (9) applies a parametric two-component mixture modeling approach, which is optimized to incorporate decoy database search results but does not require them. While very powerful when at least a subset of a decoy database is available, its reliability suffers without decoy information: The estimate is based on parametric assumptions for the distributions of the incorrect and correct identifications that are not always fulfilled. Further, PeptideProphet is currently only applicable to Mascot, Sequest, and X!Tandem search results and automatically checks whether the sample size is sufficient to fit a model but does not test whether the parametric model assumptions are fulfilled. An extension proposed by Choi et al. (2) utilizes a mixture model with an unlimited number of components which can be fitted to an arbitrary distribution using a reversible jump Markov chain Monte Carlo approach. While the model is very powerful and has valuable theoretical properties, the authors themselves state that important weaknesses from a practical point of view are its major computational requirements as well as its dependency on the proper setting of priors, which can be data set dependent.
The goal of this contribution is to bridge the gap between the two mixture models proposed by Nesvizhskii et al. (9) and Choi et al. (2) We present a method which combines the high reliability of the latter model (2) achieved by allowing a mixture of distributions for the incorrect identifications with the simplicity and quick run time, (9) making it applicable to a wider range of proteomic applications without the need for decoy databases. We outline the methodology and show that the results of our approach applied to 41 proteomics data sets randomly chosen from ongoing experiments closely follow the results obtained by decoy searches and are closer to the decoy results than other methods not based on decoy database searches. Our proposed procedure has the additional advantage that it is not limited to specific search engines, but generally applicable to various scoring schemes, and it automatically tests whether the model assumptions are fulfilled.
Methods
Click to copy section linkSection link copied!
Overview
We propose an adaptive two- or three-component Gaussian mixture model for the score distribution. With the use of an automated heuristic, the more appropriate model is chosen and the null distribution stemming from random hits to the database is extracted. The confidence of peptide identifications is then computed based on this distribution. In a final quality control step, a χ2-test is used to check the model assumptions. A flowchart of the approach is given in the Supporting Information.
Mixture Model
We follow the idea of Keller et al. (9) which begins with a mixture model of two distributions. One normal distribution represents the random matches to the database, i.e., incorrect peptide identifications. A second normal distribution represents correct identifications (Figure 1a). To enhance the model flexibility and to account for low-quality spectra which cannot reliably be matched to any peptide sequence and thus show lower scores, we also fit a three component model (Figure 1b).
Figure 1
Figure 1. Schematic for the two- (a) and three-components (b) model. The orange, dashed−dotted distribution corresponds to the incorrect peptides in the two-component model, which is separated into the red dotted distribution of low-quality spectra and the yellow dashed distribution of random hits in the three-component model. The green solid distribution corresponds to the correct peptides. The blue line shows the 5% confidence cutoff as indicated by our approach.
Number of Component Decision Criterion
Let μ = (μ1,μ2) and σ2 = (σ12,σ22) denote the parameters of the two-component mixture model and η = (η1,η2,η3) and τ2 = (τ12,τ22,τ32) denote the parameters of the three-component mixture model, sorted by location, with μ1 and η1 corresponding to the lowest average score. Our approach then applies a simple heuristic to decide between the models: When application of the three-component model leads to the distribution of the low-quality spectra being used to explain random hits, and the random hits distribution itself is shifted to partially explain correct hits, the three-component model does not fulfill its purpose. Since there is no need to precisely model the correct scores with more than one distribution, the two-component model is then used. In other words, the two component model is selected if μ2 < η2 + 2.57τ2 where μ2 is the mean of the high score component of the two component approach and η2 and τ2 correspond to the mean and standard deviation of the central component of the three component mixture model. The value of 2.57 corresponds to the 99.5% quantile of the normal distribution.
False Discovery Proportion
We let f0 = N(ν0,ξ0) denote the normal distribution of the incorrect hits and f1 = N(ν1,ξ1) denote the normal distribution of correct hits with (ν0,ν1,ζ0,ζ1) = (μ1,μ2,σ1,σ2) or (ν0,ν1,ζ0,ζ1) = (η2,η3,τ2,τ3) as indicated by the number of component decision criterion. We estimate the score significance cutoff inspired by the idea of the false discovery proportion (FDP), (12) which is closely related to the concept of the global false discovery rate (FDR). The FDR is interpreted as the expected value of the FDP: FDR = IE(FDP). (14) While the FDR estimate depends on reliable estimates of both distributions, i.e., f0 and f1, (9) we can estimate the FDP from f0 alone (12) and do not have to rely on accurately modeling f1. A FDP of α % can be understood as having at maximum a probability of α of a false discovery under the null distribution (f0). Since we have obtained f0, which is the null distribution, we can use it to estimate the likelihood that any score stems from the null distribution: 
Accordingly, for a given confidence level of (100 − α) %, we can estimate the first score to be rejected using the appropriate quantile. Consequently, the FDP can also be used to estimate the expected number of false positives above the cutoff.
χ2-Test
The proposed procedure assumes normal distributions for the mixture model. As a final quality control step, we test whether this assumption is supported by the data with a χ2-test and obtain a p-value (see the Supporting Information).
Implementation
Our approach is implemented in R using the mixtools package (17) and is available as open source code from http://software.steenlab.org/curveFDP and http://hci.iwr.uni-heidelberg.de/software.php. Analysis CPU time on a data set of approximately 5000 spectra was 3 s on a 2 GHz AMD Opteron machine.
Experiments
We conducted four sets of experiments. In the first set, we compare the performance of our approach on four previously described data sets (15) to PeptideProphet (with and without decoy information) and to the original decoy approach (see the supplementary data in the Supporting Information). The data sets “Yeast1” and “Human 1” were analyzed on an LTQ-Orbitrap (Thermo Scientific) equipped with a nanoflow HPLC system (Eksigent). For the analysis of “Mouse1” and “Human 2”, an LTQ equipped with a split-based microscale capillary HPLC system was used (both Thermo Scientific). Mascot (version 2.2.04) was used for database searching and scoring. A list of the data acquisition details and search parameters is given in the Supporting Information.
In a second set of experiments, we used Mascot search results to evaluate the performance of our approach and the original decoy approach on 37 additional data sets. “Human3” to “Human21” are low-accuracy data acquired on an LTQ, whereas “Human22” and “Rat1” to “Rat17” are high accuracy data acquired on an LTQ-Orbitrap (see the Supporting Information for details and data availability). All data sets were searched using Mascot and the confidence of peptide identifications was estimated using the original decoy FDR approach, the Mascot E-value FDR, (6) and our procedure, which did not use the decoy database information.
In a third experiment, we study the influence of the sample size on the confidence level estimation as well as on its variance. Using the “Human 2” data set of the first experiment, we randomly sampled between 100 and 5000 spectra from the original data set and computed the decoy FDR as well as the curve FDP.
The fourth experiment investigates the influence of the good/bad spectra ratio on the confidence level estimation. Using the largest data set from the second experiment, “Rat 1”, we applied our curve FDP approach to estimate for each spectrum the probability of belonging to the distribution of random matches or correct identifications. We then sampled spectra from both distributions according to these probabilities and with ratios of correct identifications ranging from 0% to 100%. The resulting sets of spectra were used to estimate confidence levels for the decoy FDR and our curve FDP approach.
To analyze the variation of each estimate, we used a bootstrap resampling procedure to obtain 90% empirical confidence intervals (see the Supporting Information for details).
Results
Click to copy section linkSection link copied!
Comparison with PeptideProphet without Decoy Information
While the existing nondecoy database search strategy of PeptideProphet is very conservative when evaluated against the original decoy approach and identifies on average 500 peptides or a quarter less than decoy FDRs (original decoy FDR or PeptideProphet with decoy information), our curve based approach closely follows the original decoy FDR (see Figure 2). While our curve FDP approach does show significantly more variation in the bootstrap estimates for some of the data sets (“Mouse” and “Human2”), the confidence intervals overlap for all data sets and hence indicate similar performance.
Figure 2
Figure 2. Comparison of the number of identified peptides at the 5% confidence level using PeptideProphet with decoy information (dotted line) and without decoy information (dashed−dotted line), our curve FDP approach (dashed line), and the decoy FDR (straight line); connecting lines are drawn for visual clarity only. Our approach and the decoy FDR show very similar behavior on all four data sets, (15) whereas PeptideProphet without decoy information is significantly more conservative.
Comparison with Original Decoy FDR
Results for experiment 2 where we study 37 data sets are given in Figure 3. Overall, these results confirm the observations from the first experiment: The numbers of peptides identified by our curve FDR approach closely follow those of the decoy-based FDR. We see a slightly less conservative estimate than observed with the decoy approach for the LTQ-data sets and a slightly more conservative approach for the LTQ-Orbitrap experiments. The Mascot E-value FDR is by far the most conservative approach identifying on average more than 1000 peptides or 40% less than the other two approaches. The respective cutoff scores and the average precursor charge values are available in the Supporting Information. With the exception of two outliers for the “Human4” and the “Human8” data set which show unusually large variation, the variation of our curve FDP estimate is larger, but in the same overall range as the variation of the decoy FDR.
Figure 3
Figure 3. The performance with regard to the number of identified peptides at the 5% confidence level of our curve FDP approach (dashed line) is compared to the decoy FDR approach (solid line) and the Mascot E-value FDR (dotted line). Data sets are ordered by the number of identified peptides using the decoy FDR; connecting lines are drawn for visual clarity only. 90% confidence intervals based on bootstrap sampling are drawn but too narrow to see in some cases. Our approach closely follows the decoy FDR; it is less conservative with regard to the low-accuracy LTQ-data (upper plot) and more conservative with regard to the high-accuracy Orbitrap data (lower plot), while the E-value FDR is much more conservative.
Influence of Sample Size
For the decoy FDR, we see a decrease in the variance for an increasing sample sizes whereas our approach shows constant variation (see Figure 4). For small sample sizes, the two approaches show similar variation, for larger ones, our approach has more variation. Overall, it becomes evident that the estimation of confidence levels contains uncertainty which is commonly ignored at the moment. This uncertainty should thus be accounted for in subsequent analysis steps, e.g., for the inference of identified proteins.
Figure 4
Figure 4. Analysis of the influence of the sample size on the cutoff score estimation and its variation. At the 5% confidence level, our curve FDP approach (dashed line) is compared to the decoy FDR approach (solid line). The overall number of spectra (divided by 100) is shown by the dotted line. Both approaches show a rather constant behavior with regard to the sample size, a slight downward movement can be observed for the curve FDP for smaller sample sizes and a slight upward movement for the decoy FDR. While the variation of our approach is rather constant, we see an increase of the variation for the decoy FDR for smaller sample sizes where it shows larger variation than the curve FDP approach.
Influence of the Ratio of Good to Bad Spectra
Both the decoy FDR and our curve FDP approach show an overall similar stability over varying ratios of good spectra. The curve FDP estimate exhibits more overall variation whereas the decoy FDR shows more bias with respect to the percentage of good spectra. Only for the extreme case of 0% good spectra, we see large deviations: the decoy FDR overestimates the cutoff score and the curve FDP underestimates it (see Figure 5), when comparing the data set with the original ratio of good and bad spectra.
Figure 5
Figure 5. Influence of the ratio of good to bad spectra. Both the curve FDP (dashed line) as well as the decoy FDR (solid line) show a rather stable behavior over varying ratios of good spectra. The curve FDP estimates show more overall variation, while the decoy FDR shows a downward trend with less strict estimates for higher ratios of good spectra. Both approaches show the most extreme behavior for the case of 0% good spectra with an increase in the cutoff score for the decoy FDR and a decrease for the curve FDP.
Discussion
Click to copy section linkSection link copied!
Our curve FDP estimation method is built on two key assumptions. The first one is the decomposition of the score values into an appropriate number of components and the second one is assumption of normal distributions.
Determination of the Number of Components
As illustrated in Figure 8 in the Supporting Information, we see a strong difference in the final number of fitted components between score distributions derived from LTQ and LTQ-Orbitrap data: for the former, there is a pronounced tendency toward three-distribution-fits, whereas the latter can mostly be approximated with two-component fits. While we generally note differing characteristics of the score distributions for the LTQ and LTQ-Orbitrap data (e.g., indicated by the differing cutoff scores in Figure 7 in the Supporting Information), a possible explanation for the need of an additional component might be that fragmentation on the LTQ is not limited to peptides since no charge-state screening is performed resulting in a higher number of fragmentation spectra of nonpeptide origin equivalent to low-quality spectra.
Assumption of Normal Distributions
For the majority of data sets in this analysis, there was no evidence of a departure from normality. However, departure from this normality assumption led to stronger differences between our approach and the decoy FDR. This is underscored by the sample set which shows the largest difference between our approach and the decoy FDR approach (“Human22”): the normality assumption was rejected in 91 of 100 bootstrap resamples. However, such unreliable results are detected by the incorporated automated check of the underlying assumptions. In contrast to existing approaches, this allows specialized treatment for these cases. From our experience, results usually tend to become more conservative when the assumptions are not met. Since our FDP-based approach only relies on the distribution of the incorrect hits to be estimated correctly, it is particularly robust to departures from the normal distribution for the correct identifications. For low scores, the mixture of two normal distributions, separating overall bad spectra and the false identifications, increase the flexibility and robustness of the model. The flexibility of the proposed model is also illustrated by application to other scoring schemes (see the Supporting Information, Figure 10, for an application to PepNovo (5) scores).
Conclusions
Click to copy section linkSection link copied!
We describe a fast, simple, and robust alternative to estimating confidence levels of peptide identifications from any protein identification algorithm. Our approach combines the simplicity of the model of PeptideProphet (9) with the flexibility provided by Choi’s approach. (2) The main advantage of our approach is that confidence levels can be computed without the need for a decoy database, thus resulting (i) in a time advantage for standard protein identification workflows and (ii) applicability to proteomics problems for which no meaningful decoy database search strategies can be formulated. An automated test for the appropriateness of the model assumptions ensures the quality of the results. Using a bootstrap approach, we demonstrated that confidence estimates for peptide identification contain significant variation, which should be accounted for in the subsequent analysis steps.
Supporting Information
Click to copy section linkSection link copied!
Additional information as noted in text. This material is available free of charge via the Internet at http://pubs.acs.org.
Terms & Conditions
Most electronic Supporting Information files are available without a subscription to ACS Web Editions. Such files may be downloaded by article for research use (if there is a public use license linked to the relevant article, that license may permit other uses). Permission may be obtained from ACS for other uses through requests via the RightsLink permission system: http://pubs.acs.org/page/copyright/permissions.html.
Acknowledgment
Click to copy section linkSection link copied!
The authors would like to thank Michael Hanselmann and Anna Kreshuk (Interdisciplinary Center for Scientific Computing (IWR), University of Heidelberg, Germany) for comments, suggestions, and fruitful discussions. We gratefully acknowledge financial support by the DFG under Grant No. HA4364/2-1 (B.Y.R., F.A.H.), the Alexander von Humboldt-Foundation (Grant 3.1-DEU/1134241 to M.K.), Robert Bosch GmbH (F.A.H.), as well as the Helmholtz Initiative for Systems Biology (F.A.H.).
References
Click to copy section linkSection link copied!
This article references 17 other publications.
- 1Bradshaw, R. A., Burlingame, A. L., Carr, S., and Aebersold, R. Mol. Cell. Proteomics 2006, 5, 787– 788Google Scholar1https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BD28XkslClsr4%253D&md5=a0b5774e683a220d733ee4b07eb830e4Reporting protein identification data the next generation of guidelinesBradshaw, Ralph A.; Burlingame, Alma L.; Carr, Steven; Aebersold, RuediMolecular and Cellular Proteomics (2006), 5 (5), 787CODEN: MCPOBS; ISSN:1535-9476. (American Society for Biochemistry and Molecular Biology)There is no expanded citation for this reference.
- 2Choi, H., Ghosh, D., and Nesvizhskii, A. I. J. Proteome Res. 2008, 7, 286– 292Google Scholar2https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BD2sXhsVejtbbO&md5=c3f146e9befac3cbdd963098077a516aStatistical Validation of Peptide Identifications in Large-Scale Proteomics Using the Target-Decoy Database Search Strategy and Flexible Mixture ModelingChoi, Hyungwon; Ghosh, Debashis; Nesvizhskii, Alexey I.Journal of Proteome Research (2008), 7 (1), 286-292CODEN: JPROBS; ISSN:1535-3893. (American Chemical Society)Reliable statistical validation of peptide and protein identifications is a top priority in large-scale mass spectrometry based proteomics. PeptideProphet is one of the computational tools commonly used for assessing the statistical confidence in peptide assignments to tandem mass spectra obtained using database search programs such as SEQUEST, MASCOT, or X! TANDEM. The authors present two flexible methods, the variable component mixt. model and the semiparametric mixt. model, that remove the restrictive parametric assumptions in the mixt. modeling approach of PeptideProphet. Using a control protein mixt. data set generated on an linear ion trap Fourier transform (LTQ-FT) mass spectrometer, the authors demonstrate that both methods improve parametric models in terms of the accuracy of probability ests. and the power to detect correct identifications controlling the false discovery rate to the same degree. The statistical approaches presented here require that the data set contain a sufficient no. of decoy (known to be incorrect) peptide identifications, which can be obtained using the target-decoy database search strategy.
- 3Choi, H. and Nesvizhskii, A. I. J. Proteome Res. 2008, 7, 254– 265Google Scholar3https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BD2sXhsVOntbfM&md5=bb0ea1f340c4ba7d099f655c203f49edSemisupervised Model-Based Validation of Peptide Identifications in Mass Spectrometry-Based ProteomicsChoi, Hyungwon; Nesvizhskii, Alexey I.Journal of Proteome Research (2008), 7 (1), 254-265CODEN: JPROBS; ISSN:1535-3893. (American Chemical Society)Development of robust statistical methods for validation of peptide assignments to tandem mass (MS/MS) spectra obtained using database searching remains an important problem. PeptideProphet is one of the commonly used computational tools available for that purpose. An alternative simple approach for validation of peptide assignments is based on addn. of decoy (reversed, randomized, or shuffled) sequences to the searched protein sequence database. The probabilistic modeling approach of PeptideProphet and the decoy strategy can be combined within a single semisupervised framework, leading to improved robustness and higher accuracy of computed probabilities even in the case of most challenging data sets. The authors present a semisupervised expectation-maximization (EM) algorithm for constructing a Bayes classifier for peptide identification using the probability mixt. model, extending PeptideProphet to incorporate decoy peptide matches. Using several data sets of varying complexity, from control protein mixts. to a human plasma sample, and using three commonly used database search programs, SEQUEST, MASCOT, and TANDEM/k-score, the authors illustrate that more accurate mixt. estn. leads to an improved control of the false discovery rate in the classification of peptide assignments.
- 4Elias, J. E. and Gygi, S. P. Nat. Methods 2007, 4, 207– 214Google Scholar4https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BD2sXitFChtrs%253D&md5=4336d04ea53dc7a161d83de1fa8249d3Target-decoy search strategy for increased confidence in large-scale protein identifications by mass spectrometryElias, Joshua E.; Gygi, Steven P.Nature Methods (2007), 4 (3), 207-214CODEN: NMAEA3; ISSN:1548-7091. (Nature Publishing Group)Liq. chromatog. and tandem mass spectrometry (LC-MS/MS) has become the preferred method for conducting large-scale surveys of proteomes. Automated interpretation of tandem mass spectrometry (MS/MS) spectra can be problematic, however, for a variety of reasons. As most sequence search engines return results even for 'unmatchable' spectra, proteome researchers must devise ways to distinguish correct from incorrect peptide identifications. The target-decoy search strategy represents a straightforward and effective way to manage this effort. Despite the apparent simplicity of this method, some controversy surrounds its successful application. Here the authors clarify their preferred methodol. by addressing 4 issues based on obsd. decoy hit frequencies: (i) the major assumptions made with this database search strategy are reasonable; (ii) concatenated target-decoy database searches are preferable to sep. target and decoy database searches; (iii) the theor. error assocd. with target-decoy false pos. (FP) rate measurements can be estd.; and (iv) alternate methods for constructing decoy databases are similarly effective once certain considerations are taken into account.
- 5Frank, A. and Pevzner, P. Anal. Chem. 2005, 77, 964– 973Google ScholarThere is no corresponding record for this reference.
- 6Goloborodko, A. A., Mayerhofer, C., Zubarev, A. R., Tarasova, I. A., Gorshkov, A. V., Zubarev, R. A., and Gorshkov, M. V. Rapid Commun. Mass Spectrom. 2010, 24, 454– 462Google Scholar6https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BC3cXos1aguw%253D%253D&md5=44c4b6c350db0531c802f85b734bd59eEmpirical approach to false discovery rate estimation in shotgun proteomicsGoloborodko, Anton A.; Mayerhofer, Corina; Zubarev, Alexander R.; Tarasova, Irina A.; Gorshkov, Alexander V.; Zubarev, Roman A.; Gorshkov, Mikhail V.Rapid Communications in Mass Spectrometry (2010), 24 (4), 454-462CODEN: RCMSEF; ISSN:0951-4198. (John Wiley & Sons Ltd.)Estn. of false discovery rate (FDR) for identified peptides is an important step in large-scale proteomic studies. We introduced an empirical approach to the problem that is based on the FDR-like functions of sets of peptide spectral matches (PSMs). These functions have close values for equal-sized sets with the same FDR and depend monotonically on the FDR of a set. We have found three of them, based on three complementary sources of data: chromatog., mass spectrometry, and sequences of identified peptides. Using a calibration on a set of putative correct PSMs these functions were converted into the FDR scale. The approach was tested on a set of ∼2800 PSMs obtained from rat kidney tissue. The ests. based on all three data sources were rather consistent with each other as well as with one made using the target-decoy strategy. Copyright © 2010 John Wiley & Sons, Ltd.
- 7Higgs, R. E., Knierman, M. D., Freeman, A. B., Gelbert, L. M., Patil, S. T., and Hale, J. E. J. Proteome Res. 2007, 6, 1758– 1767Google Scholar7https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BD2sXjs1Srsbs%253D&md5=d9275935349ed354ae7e086cacabcd16Estimating the Statistical Significance of Peptide Identifications from Shotgun Proteomics ExperimentsHiggs, Richard E.; Knierman, Michael D.; Freeman, Angela Bonner; Gelbert, Lawrence M.; Patil, Sandeep T.; Hale, John E.Journal of Proteome Research (2007), 6 (5), 1758-1767CODEN: JPROBS; ISSN:1535-3893. (American Chemical Society)The authors present a wrapper-based approach to est. and control the false discovery rate for peptide identifications using the outputs from multiple com. available MS/MS search engines. Features of the approach include the flexibility to combine output from multiple search engines with sequence and spectral derived features in a flexible classification model to produce a score assocd. with correct peptide identifications. This classification model score from a reversed database search is taken as the null distribution for estg. p-values and false discovery rates using a simple and established statistical procedure. Results from 10 analyses of rat sera on an LTQ-FT mass spectrometer indicate that the method is well calibrated for controlling the proportion of false positives in a set of reported peptide identifications while correctly identifying more peptides than rule-based methods using one search engine alone.
- 8Jiang, X., Jiang, X., Han, G., Ye, M., and Zou, H. BMC Bioinf. 2007, 8, 323Google Scholar8https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A280%3ADC%252BD2snitlKhsg%253D%253D&md5=ec311c61ae9dd370fedff21ddf1180e1Optimization of filtering criterion for SEQUEST database searching to improve proteome coverage in shotgun proteomicsJiang Xinning; Jiang Xiaogang; Han Guanghui; Ye Mingliang; Zou HanfaBMC bioinformatics (2007), 8 (), 323 ISSN:.BACKGROUND: In proteomic analysis, MS/MS spectra acquired by mass spectrometer are assigned to peptides by database searching algorithms such as SEQUEST. The assignations of peptides to MS/MS spectra by SEQUEST searching algorithm are defined by several scores including Xcorr, Delta Cn, Sp, Rsp, matched ion count and so on. Filtering criterion using several above scores is used to isolate correct identifications from random assignments. However, the filtering criterion was not favorably optimized up to now. RESULTS: In this study, we implemented a machine learning approach known as predictive genetic algorithm (GA) for the optimization of filtering criteria to maximize the number of identified peptides at fixed false-discovery rate (FDR) for SEQUEST database searching. As the FDR was directly determined by decoy database search scheme, the GA based optimization approach did not require any pre-knowledge on the characteristics of the data set, which represented significant advantages over statistical approaches such as PeptideProphet. Compared with PeptideProphet, the GA based approach can achieve similar performance in distinguishing true from false assignment with only 1/10 of the processing time. Moreover, the GA based approach can be easily extended to process other database search results as it did not rely on any assumption on the data. CONCLUSION: Our results indicated that filtering criteria should be optimized individually for different samples. The new developed software using GA provides a convenient and fast way to create tailored optimal criteria for different proteome samples to improve proteome coverage.
- 9Keller, A., Nesvizhskii, A. I., Kolker, E., and Aaebersold, R. Anal. Chem. 2002, 74, 5383– 5392Google Scholar9https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BD38XmvVyktL4%253D&md5=fad3d58f90b1ff57439ea6ca2d76f13fEmpirical Statistical Model To Estimate the Accuracy of Peptide Identifications Made by MS/MS and Database SearchKeller, Andrew; Nesvizhskii, Alexey I.; Kolker, Eugene; Aebersold, RuediAnalytical Chemistry (2002), 74 (20), 5383-5392CODEN: ANCHAM; ISSN:0003-2700. (American Chemical Society)We present a statistical model to est. the accuracy of peptide assignments to tandem mass (MS/MS) spectra made by database search applications such as SEQUEST. Employing the expectation maximization algorithm, the anal. learns to distinguish correct from incorrect database search results, computing probabilities that peptide assignments to spectra are correct based upon database search scores and the no. of tryptic termini of peptides. Using SEQUEST search results for spectra generated from a sample of known protein components, we demonstrate that the computed probabilities are accurate and have high power to discriminate between correctly and incorrectly assigned peptides. This anal. makes it possible to filter large vols. of MS/MS database search results with predictable false identification error rates and can serve as a common std. by which the results of different research groups are compared.
- 10Kim, S., Gupta, N., and Pevzner, P. A. J. Proteome Res. 2008, 7, 3354– 3363Google Scholar10https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BD1cXnvF2gs74%253D&md5=c0770ec96d5046fc6cc52d7454da3708Spectral Probabilities and Generating Functions of Tandem Mass Spectra: A Strike against Decoy DatabasesKim, Sangtae; Gupta, Nitin; Pevzner, Pavel A.Journal of Proteome Research (2008), 7 (8), 3354-3363CODEN: JPROBS; ISSN:1535-3893. (American Chemical Society)A key problem in computational proteomics is distinguishing between correct and false peptide identifications. The authors argue that evaluating the error rates of peptide identifications is not unlike computing generating functions in combinatorics. The authors show that the generating functions and their derivs. (spectral energy and spectral probability) represent new features of tandem mass spectra that, similarly to Δ-scores, significantly improve peptide identifications. Furthermore, the spectral probability provides a rigorous soln. to the problem of computing statistical significance of spectral identifications. The spectral energy/probability approach improves the sensitivity-specificity tradeoff of existing MS/MS search tools, addresses the notoriously difficult problem of "one-hit-wonders" in mass spectrometry, and often eliminates the need for decoy database searches. The authors therefore argue that the generating function approach has the potential to increase the no. of peptide identifications in MS/MS searches.
- 11Käll, L., Canterbury, J. D., Weston, J., Noble, W. S., and Maccoss, M. J. Nat. Methods 2007, 4, 923– 925Google Scholar11https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A280%3ADC%252BD2snksFaltQ%253D%253D&md5=d9f40d025046fd771e274313fd526d12Semi-supervised learning for peptide identification from shotgun proteomics datasetsKall Lukas; Canterbury Jesse D; Weston Jason; Noble William Stafford; MacCoss Michael JNature methods (2007), 4 (11), 923-5 ISSN:1548-7091.Shotgun proteomics uses liquid chromatography-tandem mass spectrometry to identify proteins in complex biological samples. We describe an algorithm, called Percolator, for improving the rate of confident peptide identifications from a collection of tandem mass spectra. Percolator uses semi-supervised machine learning to discriminate between correct and decoy spectrum identifications, correctly assigning peptides to 17% more spectra from a tryptic Saccharomyces cerevisiae dataset, and up to 77% more spectra from non-tryptic digests, relative to a fully supervised approach.
- 12Korn, E. L., Troendle, J. F., Mcshane, L. M., and Simon, R. J. Stat. Plann. Inference 2004, 124 (2) 379– 398Google ScholarThere is no corresponding record for this reference.
- 13Maiolica, A., Cittaro, D., Borsotti, D., Sennels, L., Ciferri, C., Tarricone, C., Musacchio, A., and Rappsilber, J. Mol. Cell. Proteomics 2007, 6, 2200– 2211Google Scholar13https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BD1cXksF2hsg%253D%253D&md5=e344924a84221dd43f7682a36b667c73Structural analysis of multiprotein complexes by cross-linking, mass spectrometry, and database searchingMaiolica, Alessio; Cittaro, Davide; Borsotti, Dario; Sennels, Lau; Ciferri, Claudio; Tarricone, Cataldo; Musacchio, Andrea; Rappsilber, JuriMolecular and Cellular Proteomics (2007), 6 (12), 2200-2211CODEN: MCPOBS; ISSN:1535-9476. (American Society for Biochemistry and Molecular Biology)Most protein complexes are inaccessible to high resoln. structural anal. The authors report the results of a combined approach of crosslinking, mass spectrometry, and bioinformatics to two human complexes contg. large coiled-coil segments, the NDEL1 homodimer and the NDC80 heterotetramer. An important limitation of the crosslinking approach, so far, was the identification of cross-linked peptides from fragmentation spectra. The authors' novel approach overcomes the data anal. bottleneck of crosslinking and mass spectrometry. The authors constructed a purpose-built database to match spectra with cross-linked peptides, define a score that expresses the quality of the authors' identification, and est. false pos. rates. The authors show that their anal. sheds light on crit. structural parameters such as the directionality of the homodimeric coiled coil of NDEL1, the register of the heterodimeric coiled coils of the NDC80 complex, and the organization of a tetramerization region in the NDC80 complex. The authors' approach is esp. useful to address complexes that are difficult in addressing by std. structural methods.
- 14Pawitan, Y., Calza, S., and Ploner, A. Bioinformatics 2006, 22, 3025– 3031Google ScholarThere is no corresponding record for this reference.
- 15Renard, B. Y., Kirchner, M., Monigatti, F., Ivanov, A. R., Rappsilber, J., Winter, D., Steen, J. A. J., Hamprecht, F. A., and Steen, H. Proteomics 2009, 9, 4979– 4984Google ScholarThere is no corresponding record for this reference.
- 16Weatherly, D. B., Atwood, J. A., Minning, T. A., Cavola, C., Tarleton, R. L., and Orlando, R. Mol. Cell. Proteomics 2005, 4, 762– 772Google Scholar16https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BD2MXlsV2nsrs%253D&md5=846ef30ff62411a2305416987a71c42dA heuristic method for assigning a false-discovery rate for protein identifications from Mascot database search resultsWeatherly, D. Brent; Atwood, James A., III; Minning, Todd A.; Cavola, Cameron; Tarleton, Rick L.; Orlando, RonMolecular and Cellular Proteomics (2005), 4 (6), 762-772CODEN: MCPOBS; ISSN:1535-9476. (American Society for Biochemistry and Molecular Biology)MS/MS and database searching has emerged as a valuable technol. for rapidly analyzing protein expression, localization, and post-translational modifications. The probability-based search engine Mascot has found wide-spread use as a tool to correlate tandem mass spectra with peptides in a sequence database. Although the Mascot scoring algorithm provides a probability-based model for peptide identification, the independent peptide scores do not correlate with the significance of the proteins to which they match. Herein, the authors describe a heuristic method for organizing proteins identified at a specified false-discovery rate using Mascot-matched peptides. The authors call this method PROVALT, and it uses peptide matches from a random database to calc. false-discovery rates for protein identifications and reduces a complex list of peptide matches to a nonredundant list of homologous protein groups. This method was evaluated using Mascot-identified peptides from a Trypanosoma cruzi epimastigote whole-cell lysate, which was sepd. by multidimensional LC and analyzed by MS/MS. PROVALT was then compared with the two traditional methods of protein identification when using Mascot, the single peptide score and cumulative protein score methods, and was shown to be superior to both in regards to the no. of proteins identified and the inclusion of lower scoring nonrandom peptide matches.
- 17Young, D., Benaglia, T., Chauveau, D., Elmore, R., Hettmansperger, T., Hunter, D., Thomas, H., and Xuan, F.mixtools: Tools for analyzing finite mixture models, R package, version 0.3.2; 2008.Google ScholarThere is no corresponding record for this reference.
Cited By
Click to copy section linkSection link copied!
This article is cited by 21 publications.
- Dominik Madej, Henry Lam. Modeling Lower-Order Statistics to Enable Decoy-Free FDR Estimation in Proteomics. Journal of Proteome Research 2023, 22
(4)
, 1159-1171. https://doi.org/10.1021/acs.jproteome.2c00604
- Dominik Madej, Long Wu, Henry Lam. Common Decoy Distributions Simplify False Discovery Rate Estimation in Shotgun Proteomics. Journal of Proteome Research 2022, 21
(2)
, 339-348. https://doi.org/10.1021/acs.jproteome.1c00600
- Chengjian Tu, Quanhu Sheng, Jun Li, Danjun Ma, Xiaomeng Shen, Xue Wang, Yu Shyr, Zhengping Yi, and Jun Qu . Optimization of Search Engines and Postprocessing Approaches to Maximize Peptide and Protein Identification for High-Resolution Mass Data. Journal of Proteome Research 2015, 14
(11)
, 4662-4673. https://doi.org/10.1021/acs.jproteome.5b00536
- Giulia Gonnelli, Michiel Stock, Jan Verwaeren, Davy Maddelein, Bernard De Baets, Lennart Martens, and Sven Degroeve . A Decoy-Free Approach to the Identification of Peptides. Journal of Proteome Research 2015, 14
(4)
, 1792-1798. https://doi.org/10.1021/pr501164r
- Nai-ping Dong, Yi-Zeng Liang, Qing-song Xu, Daniel K. W. Mok, Lun-zhao Yi, Hong-mei Lu, Min He, and Wei Fan . Prediction of Peptide Fragment Ion Mass Spectra by Data Mining Techniques. Analytical Chemistry 2014, 86
(15)
, 7446-7454. https://doi.org/10.1021/ac501094m
- Martina Fischer, Susann Zilkenat, Roman G. Gerlach, Samuel Wagner, and Bernhard Y. Renard . Pre- and Post-Processing Workflow for Affinity Purification Mass Spectrometry Data. Journal of Proteome Research 2014, 13
(5)
, 2239-2249. https://doi.org/10.1021/pr401249b
- Suruchi Aggarwal, Anurag Raj, Dhirendra Kumar, Debasis Dash, Amit Kumar Yadav. False discovery rate: the Achilles’ heel of proteogenomics. Briefings in Bioinformatics 2022, 23
(5)
https://doi.org/10.1093/bib/bbac163
- James C. Wright, Jyoti S. Choudhary. PSM Scoring and Validation. 2016, 69-92. https://doi.org/10.1039/9781782626732-00069
- Thilo Muth, Bernhard Y. Renard, Lennart Martens. Metaproteomic data analysis at a glance: advances in computational microbial community proteomics. Expert Review of Proteomics 2016, 13
(8)
, 757-769. https://doi.org/10.1080/14789450.2016.1209418
- Sven H. Giese, Franziska Zickmann, Bernhard Y. Renard. Detection of Unknown Amino Acid Substitutions Using Error-Tolerant Database Search. 2016, 247-264. https://doi.org/10.1007/978-1-4939-3106-4_16
- Franziska Zickmann, Bernhard Y. Renard. MSProGene: integrative proteogenomics beyond six-frames and single nucleotide polymorphisms. Bioinformatics 2015, 31
(12)
, i106-i115. https://doi.org/10.1093/bioinformatics/btv236
- Anke Penzlin, Martin S. Lindner, Joerg Doellinger, Piotr Wojtek Dabrowski, Andreas Nitsche, Bernhard Y. Renard. Pipasic: similarity and expression correction for strain-level identification and quantification in metaproteomics. Bioinformatics 2014, 30
(12)
, i149-i156. https://doi.org/10.1093/bioinformatics/btu267
- Alessandro Tanca, Antonio Palomba, Massimo Deligios, Tiziana Cubeddu, Cristina Fraumene, Grazia Biosa, Daniela Pagnozzi, Maria Filippa Addis, Sergio Uzzau, . Evaluating the Impact of Different Sequence Databases on Metaproteome Analysis: Insights from a Lab-Assembled Microbial Mixture. PLoS ONE 2013, 8
(12)
, e82981. https://doi.org/10.1371/journal.pone.0082981
- Thilo Muth, Dirk Benndorf, Udo Reichl, Erdmann Rapp, Lennart Martens. Searching for a needle in a stack of needles: challenges in metaproteomics data analysis. Mol. BioSyst. 2013, 9
(4)
, 578-585. https://doi.org/10.1039/C2MB25415H
- Mathias Kuhring, Bernhard Y. Renard, . iPiG: Integrating Peptide Spectrum Matches into Genome Browser Visualizations. PLoS ONE 2012, 7
(12)
, e50246. https://doi.org/10.1371/journal.pone.0050246
- Francesco Mancuso, Jakob Bunkenborg, Michael Wierer, Henrik Molina. Data extraction from proteomics raw data: An evaluation of nine tandem MS tools using a large Orbitrap data set. Journal of Proteomics 2012, 75
(17)
, 5293-5303. https://doi.org/10.1016/j.jprot.2012.06.012
- Bernhard Y. Renard, Buote Xu, Marc Kirchner, Franziska Zickmann, Dominic Winter, Simone Korten, Norbert W. Brattig, Amit Tzur, Fred A. Hamprecht, Hanno Steen. Overcoming Species Boundaries in Peptide Identification with Bayesian Information Criterion-driven Error-tolerant Peptide Search (BICEPS). Molecular & Cellular Proteomics 2012, 11
(7)
, M111.014167-1-M111.014167-12. https://doi.org/10.1074/mcp.M111.014167
- Hanns Soblik, Abuelhassan Elshazly Younis, Makedonka Mitreva, Bernhard Y. Renard, Marc Kirchner, Frank Geisinger, Hanno Steen, Norbert W. Brattig. Life Cycle Stage-resolved Proteomic Analysis of the Excretome/Secretome from Strongyloides ratti—Identification of Stage-specific Proteases. Molecular & Cellular Proteomics 2011, 10
(12)
, M111.010157. https://doi.org/10.1074/mcp.M111.010157
- Bernhard Y Renard, Martin Löwer, Yvonne Kühne, Ulf Reimer, Andrée Rothermel, Özlem Türeci, John C Castle, Ugur Sahin. rapmad: Robust analysis of peptide microarray data. BMC Bioinformatics 2011, 12
(1)
https://doi.org/10.1186/1471-2105-12-324
- Wei ZHANG, Ji-Yang ZHANG, Hui LIU, Han-Chang SUN, Chang-Ming XU, Hai-Bin MA, Yun-Ping ZHU, Hong-Wei XIE. Development of Algorithms for Mass Spectrometry-based Label-free Quantitative Proteomics*. PROGRESS IN BIOCHEMISTRY AND BIOPHYSICS 2011, 38
(6)
, 506-518. https://doi.org/10.3724/SP.J.1206.2010.00560
- Alexey I. Nesvizhskii. A survey of computational methods and error rate estimation procedures for peptide and protein identification in shotgun proteomics. Journal of Proteomics 2010, 73
(11)
, 2092-2123. https://doi.org/10.1016/j.jprot.2010.08.009
Learn about these metrics
Article Views are the COUNTER-compliant sum of full text article downloads since November 2008 (both PDF and HTML) across all institutions and individuals. These metrics are regularly updated to reflect usage leading up to the last few days.
Citations are the number of other articles citing this article, calculated by Crossref and updated daily. Find more information about Crossref citation counts.
The Altmetric Attention Score is a quantitative measure of the attention that a research article has received online. Clicking on the donut icon will load a page at altmetric.com with additional details about the score and the social media presence for the given article. Find more information on the Altmetric Attention Score and how the score is calculated.
Recommended Articles
Figure 1
Figure 1. Schematic for the two- (a) and three-components (b) model. The orange, dashed−dotted distribution corresponds to the incorrect peptides in the two-component model, which is separated into the red dotted distribution of low-quality spectra and the yellow dashed distribution of random hits in the three-component model. The green solid distribution corresponds to the correct peptides. The blue line shows the 5% confidence cutoff as indicated by our approach.
Figure 2
Figure 2. Comparison of the number of identified peptides at the 5% confidence level using PeptideProphet with decoy information (dotted line) and without decoy information (dashed−dotted line), our curve FDP approach (dashed line), and the decoy FDR (straight line); connecting lines are drawn for visual clarity only. Our approach and the decoy FDR show very similar behavior on all four data sets, (15) whereas PeptideProphet without decoy information is significantly more conservative.
Figure 3
Figure 3. The performance with regard to the number of identified peptides at the 5% confidence level of our curve FDP approach (dashed line) is compared to the decoy FDR approach (solid line) and the Mascot E-value FDR (dotted line). Data sets are ordered by the number of identified peptides using the decoy FDR; connecting lines are drawn for visual clarity only. 90% confidence intervals based on bootstrap sampling are drawn but too narrow to see in some cases. Our approach closely follows the decoy FDR; it is less conservative with regard to the low-accuracy LTQ-data (upper plot) and more conservative with regard to the high-accuracy Orbitrap data (lower plot), while the E-value FDR is much more conservative.
Figure 4
Figure 4. Analysis of the influence of the sample size on the cutoff score estimation and its variation. At the 5% confidence level, our curve FDP approach (dashed line) is compared to the decoy FDR approach (solid line). The overall number of spectra (divided by 100) is shown by the dotted line. Both approaches show a rather constant behavior with regard to the sample size, a slight downward movement can be observed for the curve FDP for smaller sample sizes and a slight upward movement for the decoy FDR. While the variation of our approach is rather constant, we see an increase of the variation for the decoy FDR for smaller sample sizes where it shows larger variation than the curve FDP approach.
Figure 5
Figure 5. Influence of the ratio of good to bad spectra. Both the curve FDP (dashed line) as well as the decoy FDR (solid line) show a rather stable behavior over varying ratios of good spectra. The curve FDP estimates show more overall variation, while the decoy FDR shows a downward trend with less strict estimates for higher ratios of good spectra. Both approaches show the most extreme behavior for the case of 0% good spectra with an increase in the cutoff score for the decoy FDR and a decrease for the curve FDP.
References
This article references 17 other publications.
- 1Bradshaw, R. A., Burlingame, A. L., Carr, S., and Aebersold, R. Mol. Cell. Proteomics 2006, 5, 787– 7881https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BD28XkslClsr4%253D&md5=a0b5774e683a220d733ee4b07eb830e4Reporting protein identification data the next generation of guidelinesBradshaw, Ralph A.; Burlingame, Alma L.; Carr, Steven; Aebersold, RuediMolecular and Cellular Proteomics (2006), 5 (5), 787CODEN: MCPOBS; ISSN:1535-9476. (American Society for Biochemistry and Molecular Biology)There is no expanded citation for this reference.
- 2Choi, H., Ghosh, D., and Nesvizhskii, A. I. J. Proteome Res. 2008, 7, 286– 2922https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BD2sXhsVejtbbO&md5=c3f146e9befac3cbdd963098077a516aStatistical Validation of Peptide Identifications in Large-Scale Proteomics Using the Target-Decoy Database Search Strategy and Flexible Mixture ModelingChoi, Hyungwon; Ghosh, Debashis; Nesvizhskii, Alexey I.Journal of Proteome Research (2008), 7 (1), 286-292CODEN: JPROBS; ISSN:1535-3893. (American Chemical Society)Reliable statistical validation of peptide and protein identifications is a top priority in large-scale mass spectrometry based proteomics. PeptideProphet is one of the computational tools commonly used for assessing the statistical confidence in peptide assignments to tandem mass spectra obtained using database search programs such as SEQUEST, MASCOT, or X! TANDEM. The authors present two flexible methods, the variable component mixt. model and the semiparametric mixt. model, that remove the restrictive parametric assumptions in the mixt. modeling approach of PeptideProphet. Using a control protein mixt. data set generated on an linear ion trap Fourier transform (LTQ-FT) mass spectrometer, the authors demonstrate that both methods improve parametric models in terms of the accuracy of probability ests. and the power to detect correct identifications controlling the false discovery rate to the same degree. The statistical approaches presented here require that the data set contain a sufficient no. of decoy (known to be incorrect) peptide identifications, which can be obtained using the target-decoy database search strategy.
- 3Choi, H. and Nesvizhskii, A. I. J. Proteome Res. 2008, 7, 254– 2653https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BD2sXhsVOntbfM&md5=bb0ea1f340c4ba7d099f655c203f49edSemisupervised Model-Based Validation of Peptide Identifications in Mass Spectrometry-Based ProteomicsChoi, Hyungwon; Nesvizhskii, Alexey I.Journal of Proteome Research (2008), 7 (1), 254-265CODEN: JPROBS; ISSN:1535-3893. (American Chemical Society)Development of robust statistical methods for validation of peptide assignments to tandem mass (MS/MS) spectra obtained using database searching remains an important problem. PeptideProphet is one of the commonly used computational tools available for that purpose. An alternative simple approach for validation of peptide assignments is based on addn. of decoy (reversed, randomized, or shuffled) sequences to the searched protein sequence database. The probabilistic modeling approach of PeptideProphet and the decoy strategy can be combined within a single semisupervised framework, leading to improved robustness and higher accuracy of computed probabilities even in the case of most challenging data sets. The authors present a semisupervised expectation-maximization (EM) algorithm for constructing a Bayes classifier for peptide identification using the probability mixt. model, extending PeptideProphet to incorporate decoy peptide matches. Using several data sets of varying complexity, from control protein mixts. to a human plasma sample, and using three commonly used database search programs, SEQUEST, MASCOT, and TANDEM/k-score, the authors illustrate that more accurate mixt. estn. leads to an improved control of the false discovery rate in the classification of peptide assignments.
- 4Elias, J. E. and Gygi, S. P. Nat. Methods 2007, 4, 207– 2144https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BD2sXitFChtrs%253D&md5=4336d04ea53dc7a161d83de1fa8249d3Target-decoy search strategy for increased confidence in large-scale protein identifications by mass spectrometryElias, Joshua E.; Gygi, Steven P.Nature Methods (2007), 4 (3), 207-214CODEN: NMAEA3; ISSN:1548-7091. (Nature Publishing Group)Liq. chromatog. and tandem mass spectrometry (LC-MS/MS) has become the preferred method for conducting large-scale surveys of proteomes. Automated interpretation of tandem mass spectrometry (MS/MS) spectra can be problematic, however, for a variety of reasons. As most sequence search engines return results even for 'unmatchable' spectra, proteome researchers must devise ways to distinguish correct from incorrect peptide identifications. The target-decoy search strategy represents a straightforward and effective way to manage this effort. Despite the apparent simplicity of this method, some controversy surrounds its successful application. Here the authors clarify their preferred methodol. by addressing 4 issues based on obsd. decoy hit frequencies: (i) the major assumptions made with this database search strategy are reasonable; (ii) concatenated target-decoy database searches are preferable to sep. target and decoy database searches; (iii) the theor. error assocd. with target-decoy false pos. (FP) rate measurements can be estd.; and (iv) alternate methods for constructing decoy databases are similarly effective once certain considerations are taken into account.
- 5Frank, A. and Pevzner, P. Anal. Chem. 2005, 77, 964– 973There is no corresponding record for this reference.
- 6Goloborodko, A. A., Mayerhofer, C., Zubarev, A. R., Tarasova, I. A., Gorshkov, A. V., Zubarev, R. A., and Gorshkov, M. V. Rapid Commun. Mass Spectrom. 2010, 24, 454– 4626https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BC3cXos1aguw%253D%253D&md5=44c4b6c350db0531c802f85b734bd59eEmpirical approach to false discovery rate estimation in shotgun proteomicsGoloborodko, Anton A.; Mayerhofer, Corina; Zubarev, Alexander R.; Tarasova, Irina A.; Gorshkov, Alexander V.; Zubarev, Roman A.; Gorshkov, Mikhail V.Rapid Communications in Mass Spectrometry (2010), 24 (4), 454-462CODEN: RCMSEF; ISSN:0951-4198. (John Wiley & Sons Ltd.)Estn. of false discovery rate (FDR) for identified peptides is an important step in large-scale proteomic studies. We introduced an empirical approach to the problem that is based on the FDR-like functions of sets of peptide spectral matches (PSMs). These functions have close values for equal-sized sets with the same FDR and depend monotonically on the FDR of a set. We have found three of them, based on three complementary sources of data: chromatog., mass spectrometry, and sequences of identified peptides. Using a calibration on a set of putative correct PSMs these functions were converted into the FDR scale. The approach was tested on a set of ∼2800 PSMs obtained from rat kidney tissue. The ests. based on all three data sources were rather consistent with each other as well as with one made using the target-decoy strategy. Copyright © 2010 John Wiley & Sons, Ltd.
- 7Higgs, R. E., Knierman, M. D., Freeman, A. B., Gelbert, L. M., Patil, S. T., and Hale, J. E. J. Proteome Res. 2007, 6, 1758– 17677https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BD2sXjs1Srsbs%253D&md5=d9275935349ed354ae7e086cacabcd16Estimating the Statistical Significance of Peptide Identifications from Shotgun Proteomics ExperimentsHiggs, Richard E.; Knierman, Michael D.; Freeman, Angela Bonner; Gelbert, Lawrence M.; Patil, Sandeep T.; Hale, John E.Journal of Proteome Research (2007), 6 (5), 1758-1767CODEN: JPROBS; ISSN:1535-3893. (American Chemical Society)The authors present a wrapper-based approach to est. and control the false discovery rate for peptide identifications using the outputs from multiple com. available MS/MS search engines. Features of the approach include the flexibility to combine output from multiple search engines with sequence and spectral derived features in a flexible classification model to produce a score assocd. with correct peptide identifications. This classification model score from a reversed database search is taken as the null distribution for estg. p-values and false discovery rates using a simple and established statistical procedure. Results from 10 analyses of rat sera on an LTQ-FT mass spectrometer indicate that the method is well calibrated for controlling the proportion of false positives in a set of reported peptide identifications while correctly identifying more peptides than rule-based methods using one search engine alone.
- 8Jiang, X., Jiang, X., Han, G., Ye, M., and Zou, H. BMC Bioinf. 2007, 8, 3238https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A280%3ADC%252BD2snitlKhsg%253D%253D&md5=ec311c61ae9dd370fedff21ddf1180e1Optimization of filtering criterion for SEQUEST database searching to improve proteome coverage in shotgun proteomicsJiang Xinning; Jiang Xiaogang; Han Guanghui; Ye Mingliang; Zou HanfaBMC bioinformatics (2007), 8 (), 323 ISSN:.BACKGROUND: In proteomic analysis, MS/MS spectra acquired by mass spectrometer are assigned to peptides by database searching algorithms such as SEQUEST. The assignations of peptides to MS/MS spectra by SEQUEST searching algorithm are defined by several scores including Xcorr, Delta Cn, Sp, Rsp, matched ion count and so on. Filtering criterion using several above scores is used to isolate correct identifications from random assignments. However, the filtering criterion was not favorably optimized up to now. RESULTS: In this study, we implemented a machine learning approach known as predictive genetic algorithm (GA) for the optimization of filtering criteria to maximize the number of identified peptides at fixed false-discovery rate (FDR) for SEQUEST database searching. As the FDR was directly determined by decoy database search scheme, the GA based optimization approach did not require any pre-knowledge on the characteristics of the data set, which represented significant advantages over statistical approaches such as PeptideProphet. Compared with PeptideProphet, the GA based approach can achieve similar performance in distinguishing true from false assignment with only 1/10 of the processing time. Moreover, the GA based approach can be easily extended to process other database search results as it did not rely on any assumption on the data. CONCLUSION: Our results indicated that filtering criteria should be optimized individually for different samples. The new developed software using GA provides a convenient and fast way to create tailored optimal criteria for different proteome samples to improve proteome coverage.
- 9Keller, A., Nesvizhskii, A. I., Kolker, E., and Aaebersold, R. Anal. Chem. 2002, 74, 5383– 53929https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BD38XmvVyktL4%253D&md5=fad3d58f90b1ff57439ea6ca2d76f13fEmpirical Statistical Model To Estimate the Accuracy of Peptide Identifications Made by MS/MS and Database SearchKeller, Andrew; Nesvizhskii, Alexey I.; Kolker, Eugene; Aebersold, RuediAnalytical Chemistry (2002), 74 (20), 5383-5392CODEN: ANCHAM; ISSN:0003-2700. (American Chemical Society)We present a statistical model to est. the accuracy of peptide assignments to tandem mass (MS/MS) spectra made by database search applications such as SEQUEST. Employing the expectation maximization algorithm, the anal. learns to distinguish correct from incorrect database search results, computing probabilities that peptide assignments to spectra are correct based upon database search scores and the no. of tryptic termini of peptides. Using SEQUEST search results for spectra generated from a sample of known protein components, we demonstrate that the computed probabilities are accurate and have high power to discriminate between correctly and incorrectly assigned peptides. This anal. makes it possible to filter large vols. of MS/MS database search results with predictable false identification error rates and can serve as a common std. by which the results of different research groups are compared.
- 10Kim, S., Gupta, N., and Pevzner, P. A. J. Proteome Res. 2008, 7, 3354– 336310https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BD1cXnvF2gs74%253D&md5=c0770ec96d5046fc6cc52d7454da3708Spectral Probabilities and Generating Functions of Tandem Mass Spectra: A Strike against Decoy DatabasesKim, Sangtae; Gupta, Nitin; Pevzner, Pavel A.Journal of Proteome Research (2008), 7 (8), 3354-3363CODEN: JPROBS; ISSN:1535-3893. (American Chemical Society)A key problem in computational proteomics is distinguishing between correct and false peptide identifications. The authors argue that evaluating the error rates of peptide identifications is not unlike computing generating functions in combinatorics. The authors show that the generating functions and their derivs. (spectral energy and spectral probability) represent new features of tandem mass spectra that, similarly to Δ-scores, significantly improve peptide identifications. Furthermore, the spectral probability provides a rigorous soln. to the problem of computing statistical significance of spectral identifications. The spectral energy/probability approach improves the sensitivity-specificity tradeoff of existing MS/MS search tools, addresses the notoriously difficult problem of "one-hit-wonders" in mass spectrometry, and often eliminates the need for decoy database searches. The authors therefore argue that the generating function approach has the potential to increase the no. of peptide identifications in MS/MS searches.
- 11Käll, L., Canterbury, J. D., Weston, J., Noble, W. S., and Maccoss, M. J. Nat. Methods 2007, 4, 923– 92511https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A280%3ADC%252BD2snksFaltQ%253D%253D&md5=d9f40d025046fd771e274313fd526d12Semi-supervised learning for peptide identification from shotgun proteomics datasetsKall Lukas; Canterbury Jesse D; Weston Jason; Noble William Stafford; MacCoss Michael JNature methods (2007), 4 (11), 923-5 ISSN:1548-7091.Shotgun proteomics uses liquid chromatography-tandem mass spectrometry to identify proteins in complex biological samples. We describe an algorithm, called Percolator, for improving the rate of confident peptide identifications from a collection of tandem mass spectra. Percolator uses semi-supervised machine learning to discriminate between correct and decoy spectrum identifications, correctly assigning peptides to 17% more spectra from a tryptic Saccharomyces cerevisiae dataset, and up to 77% more spectra from non-tryptic digests, relative to a fully supervised approach.
- 12Korn, E. L., Troendle, J. F., Mcshane, L. M., and Simon, R. J. Stat. Plann. Inference 2004, 124 (2) 379– 398There is no corresponding record for this reference.
- 13Maiolica, A., Cittaro, D., Borsotti, D., Sennels, L., Ciferri, C., Tarricone, C., Musacchio, A., and Rappsilber, J. Mol. Cell. Proteomics 2007, 6, 2200– 221113https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BD1cXksF2hsg%253D%253D&md5=e344924a84221dd43f7682a36b667c73Structural analysis of multiprotein complexes by cross-linking, mass spectrometry, and database searchingMaiolica, Alessio; Cittaro, Davide; Borsotti, Dario; Sennels, Lau; Ciferri, Claudio; Tarricone, Cataldo; Musacchio, Andrea; Rappsilber, JuriMolecular and Cellular Proteomics (2007), 6 (12), 2200-2211CODEN: MCPOBS; ISSN:1535-9476. (American Society for Biochemistry and Molecular Biology)Most protein complexes are inaccessible to high resoln. structural anal. The authors report the results of a combined approach of crosslinking, mass spectrometry, and bioinformatics to two human complexes contg. large coiled-coil segments, the NDEL1 homodimer and the NDC80 heterotetramer. An important limitation of the crosslinking approach, so far, was the identification of cross-linked peptides from fragmentation spectra. The authors' novel approach overcomes the data anal. bottleneck of crosslinking and mass spectrometry. The authors constructed a purpose-built database to match spectra with cross-linked peptides, define a score that expresses the quality of the authors' identification, and est. false pos. rates. The authors show that their anal. sheds light on crit. structural parameters such as the directionality of the homodimeric coiled coil of NDEL1, the register of the heterodimeric coiled coils of the NDC80 complex, and the organization of a tetramerization region in the NDC80 complex. The authors' approach is esp. useful to address complexes that are difficult in addressing by std. structural methods.
- 14Pawitan, Y., Calza, S., and Ploner, A. Bioinformatics 2006, 22, 3025– 3031There is no corresponding record for this reference.
- 15Renard, B. Y., Kirchner, M., Monigatti, F., Ivanov, A. R., Rappsilber, J., Winter, D., Steen, J. A. J., Hamprecht, F. A., and Steen, H. Proteomics 2009, 9, 4979– 4984There is no corresponding record for this reference.
- 16Weatherly, D. B., Atwood, J. A., Minning, T. A., Cavola, C., Tarleton, R. L., and Orlando, R. Mol. Cell. Proteomics 2005, 4, 762– 77216https://chemport.cas.org/services/resolver?origin=ACS&resolution=options&coi=1%3ACAS%3A528%3ADC%252BD2MXlsV2nsrs%253D&md5=846ef30ff62411a2305416987a71c42dA heuristic method for assigning a false-discovery rate for protein identifications from Mascot database search resultsWeatherly, D. Brent; Atwood, James A., III; Minning, Todd A.; Cavola, Cameron; Tarleton, Rick L.; Orlando, RonMolecular and Cellular Proteomics (2005), 4 (6), 762-772CODEN: MCPOBS; ISSN:1535-9476. (American Society for Biochemistry and Molecular Biology)MS/MS and database searching has emerged as a valuable technol. for rapidly analyzing protein expression, localization, and post-translational modifications. The probability-based search engine Mascot has found wide-spread use as a tool to correlate tandem mass spectra with peptides in a sequence database. Although the Mascot scoring algorithm provides a probability-based model for peptide identification, the independent peptide scores do not correlate with the significance of the proteins to which they match. Herein, the authors describe a heuristic method for organizing proteins identified at a specified false-discovery rate using Mascot-matched peptides. The authors call this method PROVALT, and it uses peptide matches from a random database to calc. false-discovery rates for protein identifications and reduces a complex list of peptide matches to a nonredundant list of homologous protein groups. This method was evaluated using Mascot-identified peptides from a Trypanosoma cruzi epimastigote whole-cell lysate, which was sepd. by multidimensional LC and analyzed by MS/MS. PROVALT was then compared with the two traditional methods of protein identification when using Mascot, the single peptide score and cumulative protein score methods, and was shown to be superior to both in regards to the no. of proteins identified and the inclusion of lower scoring nonrandom peptide matches.
- 17Young, D., Benaglia, T., Chauveau, D., Elmore, R., Hettmansperger, T., Hunter, D., Thomas, H., and Xuan, F.mixtools: Tools for analyzing finite mixture models, R package, version 0.3.2; 2008.There is no corresponding record for this reference.
Supporting Information
Supporting Information
Additional information as noted in text. This material is available free of charge via the Internet at http://pubs.acs.org.
Terms & Conditions
Most electronic Supporting Information files are available without a subscription to ACS Web Editions. Such files may be downloaded by article for research use (if there is a public use license linked to the relevant article, that license may permit other uses). Permission may be obtained from ACS for other uses through requests via the RightsLink permission system: http://pubs.acs.org/page/copyright/permissions.html.