Cytokine receptor signalling and design of MAPPIT experiments

An overview of signalling through the leptin receptor (LR) is shown in Fig. 1A, and is described in more detail in the introductory section. With MAPPIT we developed a new method to analyse protein interactions in mammalian cells (Eyckerman et al., 2001). MAPPIT bait constructs were originally designed as chimeric receptors, consisting of the extracellular part of the erythropoietin receptor (EpoR) fused to the transmembrane and intracellular regions of a STAT3 recruitment-deficient LR, with a C-terminally attached bait. MAPPIT prey constructs are composed of a prey polypeptide fused to a part of the gp130 chain carrying 4 STAT3 recruitment sites. Co-expression of interacting bait and prey leads to functional complementation of STAT3 activity that can be measured with the STAT3-responsive rat pancreatitis-associated protein I (rPAPI) promoter-luciferase reporter (Fig. 1B). Intrinsic to this strategy, both modification-independent and tyrosine phosphorylation-dependent interactions can be detected.

To monitor interactions with isolated tyrosine motifs of the LR, we developed a MAPPIT configuration whereby the cytosolic domain of the LR is replaced by a large array of Gly-Gly-Ser (GGS) repeats (Fig. 1C). The MAPPIT technique also allows the analysis of interactions with the LR itself by simple mutation of the Y1138 STAT3-recruitment motif to phenylalanine (Fig. 1D). LRs with different combinations of Y to F mutations of the two other conserved tyrosine motifs (located at positions Y985 and Y1077) were used. This allows the study of protein associations with the LR in its normal oligomeric configuration.

MAPPIT analysis of CIS and SOCS2 interactions with the LR

To determine interaction with the LR, the CISprey fusion protein was transiently co-expressed with the LR(YYF) mutant and the luciferase reporter construct (Fig. 2A). Clear induction of luciferase activity indicated that CIS interacts with the LR. MAPPIT experiments using LR(YFF), LR(FYF) or LR(F3) showed that CIS can interact with both Y985 and Y1077 motifs, whereas no interaction was detected with the LR lacking tyrosines. In a similar way we also tested the SOCS2-LR interaction (Fig. 2A). SOCS2 clearly associates with the LR, but only at position Y1077. Expression of the LR mutants was analysed using a leptin-SEAP binding assay (Fig. 2B), and expression of the FLAG-tagged CIS and SOCS2 preys was revealed by immunoblotting using an anti-FLAG antibody (Fig. 2C).

Phosphopeptide binding analysis

We confirmed the specific interaction of SOCS2 with pY1077 of the LR using a biochemical strategy (Fig. 3). FLAG-tagged SOCS2 or CIS proteins were expressed in HEK293T cells and total cell lysates were incubated with the biotinylated peptides encompassing the LR phosphorylated or non-phosphorylated Y1077 or Y985 motifs to verify (phospho)tyrosine-specific association. SOCS2 clearly interacted with the phosphorylated Y1077 motif but not with the phosphorylated Y985 motif, confirming its specific phosphorylation-dependent interaction with the LR at position Y1077. Association of CIS was found with neither pY985 nor pY1077 indicating that these interactions may be to weak or short-lived to be detected by phosphopeptide affinity chromatography (data not shown).

Relative binding affinities of the CIS and SOCS2 interactions with the LR

To gain further insight into their relative binding affinities for the LR, the CISprey or SOCS2prey were co-expressed with wild-type CIS. Although CIS expression markedly reduced the CISprey signal through both LR(YFF) and LR(FYF), it did not lead to any inhibition of the SOCS2prey signal through Y1077. Conversely, co-expression of wild-type SOCS2 with the CISprey protein clearly diminished the MAPPIT signal at the Y1077 position in the LR whereas the SOCS2prey signal is only partially reduced (Fig. 4A). These results confirm that CIS interactions with Y985 and Y1077 of the LR are weak or transient, whereas the association of SOCS2 with Y1077 is more stable and therefore not easy to compete. It is quite surprising that SOCS2 can inhibit the MAPPIT signal of the CISprey protein through Y985 since SOCS2 is not interacting with this position. Expression levels of the FLAG-tagged proteins were confirmed by immunoblotting using an anti-FLAG antibody (Fig. 4B).

Analysis of CIS and SOCS2 interactions with the LR using GGS-MAPPIT

A new adaptation of the classic MAPPIT method, called GGS-MAPPIT (Fig. 1C), was used to confirm the interaction of CIS and SOCS2 with the LR. In this configuration the cytosolic domain of the LR, following the JAK2 interaction site, is replaced by 60 GGS repeats. GGS triplet repeats are often used as hinge sequences for their known structural flexibility. By using this GGS-MAPPIT strategy any background prey association with the LR-F3 is prevented. The bait constructs containing the LR motifs surrounding Y985 or Y1077 were transiently co-transfected with the prey construct and the rPAP luciferase reporter construct in HEK293T cells. Using GGS MAPPIT we were again able to detect the interaction of CIS with both the Y985 and Y1077 motifs, whereas SOCS2 only interacts with the Y1077 motif, but not with the pY985 motif (Fig. 5A). We tested this GGS-MAPPIT strategy further in erythroleukaemic TF-1 cells and obtained similar results as those found in HEK293T cells (Fig. 5B). A full-length FKBP12 bait was used to evaluate non-specific binding of the CIS and SOCS2 preys. FACS analysis, using antibodies against the extracellular domain of the EpoR, allowed monitoring of the expression of the different GGS baits (Fig. 5C).

SOCS2 interferes with STAT5a recruitment

We previously showed that STAT5 can be activated by the LR upon recruitment to the LR Y1077 and Y1138 motifs (Hekerman et al., 2005). Given the strong interaction of SOCS2 at position Y1077 we examined whether SOCS2 can interfere with STAT5 association at this position. The SH2 domain of STAT5a was inserted in a prey construct and used in MAPPIT experiments using the Y1077 motif as bait in GGS-MAPPIT. In HEK293T cells, co-expression of SOCS2 or a SOCS2 mutant lacking the entire SOCS box completely abolished the MAPPIT signal. Similarly, co-expression of SOCS2 Δbox in TF-1 cells also abrogated the MAPPIT signal, thus excluding a role for elongin B/C recruitment in this suppressive effect (Fig. 6A,B). Similar data were obtained using the LR(FYF) as bait (data not shown). We conclude that SOCS2 can compete with STAT5a association at the pY1077 motif.

SOCS2 interacts with the SOCS box of CIS

Given the discrepancy between binding experiments at position Y985, i.e. SOCS2 interferes with CIS binding without interacting with this recruitment motif itself, we examined whether SOCS2 can directly associate with CIS. We first used a MAPPIT configuration with CIS as bait. Here, SOCS2 clearly interacted specifically with full-length CIS (Fig. 7A). In Fig. 7B, we confirmed this interaction by co-immunoprecipitation. Next we looked at association of SOCS2 with the SOCS box of CIS in a MAPPIT experiment and also observed clear interaction (Fig. 7C).

Elongin B/C recruitment is involved in SOCS2 interference with receptor-binding of CIS

We developed a mutant of SOCS2, SOCS2(LC-QQ), in which elongin B/C recruitment is abrogated by mutating leucine 163 and cysteine 167, analogous to an elongin B/C recruitment-deficient SOCS1 mutant reported before (Kamura et al., 1998). We mutated both residues to glutamines to minimise structural alterations. Elongin B/C association was analysed using a two-step purification method, TAP2, based on the classic TAP method (Puig et al., 2001). This sequential purification procedure involves a first protein A tag-based step, followed by TEV protease cleavage to remove the protein A part of the tag and followed by a FLAG-tag-based immunoprecipitation step. Clearly, this SOCS2(LC-QQ) mutant no longer interacted with elongin B or C (Fig. 8A). Furthermore, this SOCS2 mutant as a prey protein still bound CIS in a MAPPIT experiment (Fig. 8B).

We next examined the interference of SOCS2 with CIS binding in more detail. At position Y985, the inhibitory effect by co-expression of SOCS2 was completely lost when using the SOCS2(LC-QQ) mutant. Recruitment of elongin B/C to the SOCS box of SOCS2 thus appeared essential for interference with CIS interaction at this position. By contrast, no difference was observed for the SOCS2(LC-QQ) mutant at the Y1077 position, clearly in line with a direct competition with CIS binding at this site (Fig. 8C).

Constructs

Generation of the mutant mouse LR(YYF), LR(FYF), LR(YFF), LR(F3) in the pMET7 expression vector was described elsewhere (Eyckerman et al., 1999). A pSEL-hEpoR-Y480 bait vector was derived from the described pSEL-hEpoR-Y402 bait (Eyckerman et al., 2001). In this pSEL-hEpoR-Y480 bait construct most of the intracellular part of the LR was replaced by a flexible GGS_N linker. An unique EcoRI restriction site was introduced immediately following the JAK2 binding site through site-directed mutagenesis (Stratagene) using the primer pair 5′-GCTTGGAAAAATAAAGATGAATTCGTCCCAGCAGCTATGGTC-3′ and 5′-GACCATAGCTGCTGGGACGAATTCATCTTTATTTTTCCAAGC-3′. Phosphorylated oligonucleotides encoding two GGS repeats (5′-TCTGGTGGCAGTGGAGGG-3′ and 5′-AGACCCTCCACTGCCACC-3′) were annealed and head to tail ligated. Ligation was stopped by addition of two other annealed oligonucleotide couples: 5′-AATTCGGAGGGAGTGGTGGC-3′ and 5′-AGAGCCACCACTCCCTCCG-3′; 5′-TCTGGAGGGAGTGGTGGGAGCT-3′ and 5′-CCCACCACTCCCTCC-3′. Annealing of these oligonucleotide couples generated respectively an EcoRI and a SacII (both underlined) sticky end. The final product was ligated in EcoRI-SacII opened pSEL-hEpoR-Y480 vector. Sequence analysis showed that using this procedure 20 GGS repeats were introduced in the bait construct. The linker was further amplified using oligonucletides 5′-CTTCTTCTGGAGCCTGAACC-3′ and 5′-CGCCGCCAATTGCGAACTCCCACCACTCCC-3′. The product was digested with EcoRI-MfeI and ligated in the EcoRI digested pSEL-20GGS-hEpoR-Y480 vector yielding the pSEL-40GGS-hEpoR-Y480 vector. This ligation step was repeated once more to generate pSEL-60GGS-hEpoR-Y480 vector. The pSEL(+2L)60GGS-Y480 construct was generated by site-directed mutagenesis on the pSEL-60GGS-hEpoR-Y480 vector using primers 5′-CCCATAATTATTTCCAGCTGTCTCCTCGTCCTACTGCTCGGAAC-3′ and 5′-GTTCCGAGCAGTAGGACGAGGAGACAGCTGGAAATAATTATGGG-3′ and Y480 was removed by SacI-NotI. The mLR Y985 motif was amplified with forward primer 5′-GCCGAGCTCATGGAAAAATAAAGATGAG-3′ containing a SacI site and reverse primer 5′-CGGGCGGCCGCTCAACAGACAGACTTCTCCCTGTG-3′ containing a NotI site and a stop codon, allowing in-frame coupling to the hEpoR-60GGS chimeric receptor. The same strategy was used for the mLR Y1077 motif, using primers 5′-GCCGAGCTCAGCAACTCTGGTCAGCAAC-3′ and 5′-GGGCGGCCGCTCAAGGTACAAAGTTCTCACC-3′. The final constructs were called pSEL(+2L)60GGS-mLR-Y985 and pSEL(+2L)60GGS-mLR Y1077, respectively. For the pSEL(+2L)60GGS-FKBP12 construct, FKPB12 was cut from the pSELFFY-FKBP12 construct described earlier (Eyckerman et al., 2005) using SacI and NotI, and cloned in the pSEL(+2L)60GGS vector.

The pXP2d2-rPAPI-luciferase reporter, originating from the rPAPI (rat pancreatitis-associated protein I) promoter, is used as previously described (Eyckerman et al., 2001). Expression plasmid vectors pEF-FLAG-I/mSOCS2 and pEF-FLAG-I/mCIS were a gift from R. Starr (The Walter and Eliza Hall Institute of Medical Research, Parkville, Australia). Generation of the prey constructs pMG2-CIS and pMG2-SOCS2 both containing part of the gp130 chain (aa 905-918) in duplicate was described elsewhere (Montoye et al., 2005).

The pMET7-FLAG-CIS construct was created by a three-point ligation. The N-terminal FLAG-tag of CIS was cut from the pEF-FLAG-I/mCIS construct using EcoRI-PvuII and the C-terminal part of CIS was cut from the pMG1-CIS construct using KpnI-PvuII as described elsewhere (Eyckerman et al., 2001). Both parts were ligated in the pMet7 vector digested with EcoRI-PvuII. To create the pMET7-Etag-CIS construct, CIS was amplified from the pMET7-FLAG-CIS using primers 5′-CGTCCGCGGCCGCGGTCCTCTGCGTACAGGGATC-3′ and 5′-GCTGGCTCGAGTCAGAGTTGGAAGGGGTACTGTC-3′ and was ligated in the NotI-XhoI digested pCAGGSE-mMyD88 construct, which was a gift from Rudy Beyaert (Ghent University, Ghent, Belgium). Etag-CIS was then digested with EcoRI-XhoI and ligated in the pMET7 vector.

pMET7-FLAG-SOCS2 was created by cutting SOCS2 from the pMG2-SOCS2 construct using EcoRI-XbaI and ligating it into the EcoRI-XbaI digested pMET7-FLAG-SOCS3 expression vector which was described elsewhere (Eyckerman et al., 2000). SOCS2 Δbox was amplified using primers 5′-TGCCTTTACTTCTAGGCCTG-3′ and 5′-GCAGGTCTAGATTATGATGTATACAGAGGTTTG-3′ from the templates pMET7-FLAG-SOCS2 cloned in the NotI-XbaI opened pMET7-FLAG-SOCS2 to create pMET7-FLAG-SOCS2 Δbox. The pMET7-FLAG-SOCS2(LC-QQ) mutant was created by site-directed mutagenesis of the pMET7-FLAG-SOCS2 construct using primers 5′-GTATACATCAGCACCCACTCAGCAGCATTTCCAACGACTCGCCATTAAC-3′ and 5′-GTTAATGGCGAGTCGTTGGAAATGCTGCTGAGTGGGTGCTGATGTATAC-3′.

Generation of the chimeric receptors containing the extracellular part of the EpoR and the transmembrane and intracellular parts of the leptin receptor, such as EpoR-LR(F3), were described elsewhere (Eyckerman et al., 2001). A full-length CIS bait construct was generated by digesting the pCEL(2L)-Y480 bait construct with SstI and NotI and swapping the EpoR Y402 motif with a PCR product containing full-length CIS and the cloning sites SstI and NotI (primers 5′-GCGCGAGCTCAATGGTCCTCTGCGTACAGGG-3′ and 5′-GCTCGCGGCCGCTCAGAGTTGGAAGGGGTACTGTCGG-3′). The CIS SOCS box bait was made using the same strategy and the PCR amplification of the CIS SOCS box was done with primers 5′ GCGAGAGCTCCGGATCCGCCCGCAGCTTACAACATC and 5′ CGCTGCGGCCGCTTAGAGTTGGAAGGGGTACTG. The SOCS2 (LC-QQ) prey was generated by mutating L163 and C167 of the Wild Type SOCS2 prey using primers 5′-GTATACATCAGCACCCACTCAGCAGCATTTCCAACGACTCGCCATTAAC-3′ and 5′-GTTAATGGCGAGTCGTTGGAAATGCTGCTGAGTGGGTGCTGATGTATAC-3′. pMG2-STAT5aSH2 was created by amplifying the SH2 domain of STAT5a using primers 5′-GCGAGAATTCTCCGGACCCCACTGGAATGATGGGGC-3′ and 5′-CGCTTCTAGATTAACTCGAGGAGAAGACCTCATCCTTGG-3′ and EcoRI-XbaI cloning in the pMG2 vector.

To generate the pMET7TAP2 construct we used the primers 5′-GCGAGGGCCCGCCACCATGGCCCAGCACGACGAGATCTC-3′ and 5′-CGCTCTCGAGGCCCTGGAAGTACAGGTTCTCGCTGGCGGTGGTGGGGATGTCGCTGTTGGCGTCCACGCTG-3′ to amplify the proteinA domain from the pMA-SpaI construct obtained from the BCCM/LMBP plasmid collection and to add a TEV cleavage site and we cloned the PCR product in the pMET7 vector using ApaI-XhoI. The FLAGtag was introduced using primers 5′-GCGAGGGCCCGCCACCATGGCCCAGCACGACGAGATCTC-3′ and 5′-GCGAGAATTCCCCGCTGCCCTTGTCATCGTCGTCCTTGTAGTCCTGGCGCGCGCCCTGGAAGTACAGGTTC-3′ and cloned in the same vector using ApaI-EcoRI.

pMET7TAP2-SOCS2 was constructed by cutting SOCS2 from the pMG2-SOCS2 construct using EcoRI-NotI and ligating it in the pMET7TAP2 construct.

This construct was then used to create the pMET7TAP2-SOCS2(LC-QQ) by site-directed mutagenesis using the primers 5′-GTATACATCAGCACCCACTCAGCAGCATTTCCAACGACTCGCCATTAAC-3′ and 5′-GTTAATGGCGAGTCGTTGGAAATGCTGCTGAGTGGGTGCTGATGTATAC-3′. All constructs were verified by DNA sequence analysis.

Cell culture, transfection and reporter assays

Culture conditions, transfection procedures and luciferase assays for HEK293T cells were as previously described (Eyckerman et al., 2000). For a typical luciferase experiment, HEK293T cells were seeded in six-well plates 24 hours before overnight transfection with the desired constructs together with the luciferase reporter gene. Two days after transfection cells were left untreated (not stimulated NS) or were stimulated with 100 ng/ml leptin overnight. The luciferase activity of the transfected cells were measured by chemiluminescence. The factor-dependent TF-1 erythroleukaemia cell line was grown in RPMI medium supplemented with 10% foetal calf serum and 1 ng/ml GM-CSF. After electroporation (300 V, 1500 μF), cells were starved (removal of GM-CSF) for 24 hours and were subsequently stimulated with 5 ng/ml hEpo overnight. After 24 hours, luciferase activity was measured as described before.

Leptin binding assay

LR expression on the surface of HEK293T cells was measured using a binding assay with a mouse leptin-secreted alkaline phosphatase (SEAP) chimeric protein (Tartaglia et al., 1995). 48 hours after transfection, cells were incubated for 2 hours with a 1:50 dilution of conditioned Cos1 medium containing the chimeric protein with or without an excess of leptin. After two washing steps, cells were lysed in a buffer (1% Triton X-100, 20 mM Tris-HCl pH 7.4) and alkaline phosphatase activity was measured by chemiluminescence, using CSPD substrate (PhosphaLight, Tropix) according to the protocol provided by the manufacturer.

FACS analysis

The expression of the chimeric hEpoR-mLR GGS baits was monitored using goat anti-human EpoR polyclonal IgG (R&D Systems) at 2 μg/ml and Alexa Fluor 488-conjugated donkey anti-goat IgG (Molecular Probes) at 4 μg/ml. Fluorescence-activated cell sorting (FACS) was performed on a FACSCalibur (Becton Dickinson).

Western blot analysis

Expression of the gp130-fusion proteins, CIS and SOCS2, all flag-tagged, were verified by western blot analysis. Transfected HEK293T or TF-1 cells were lysed in RIPA buffer: 200 mM NaCl, 50 mM Tris-HCl pH 8, 0.05% SDS, 2 mM EDTA, 1% NP40, 0.5% DOC, Complete™ Protease Inhibitor Cocktail (Roche). 4× loading buffer (125 mM Tris-HCl pH 6.8, 6% SDS, 20% glycerol, 0.02% BFB, 10% β-mercaptoethanol) was added to the lysates which were then loaded on a 10% polyacrylamide gel and blotted overnight. Blotting efficiency was checked using PonceauS staining (Sigma). Flag-tagged proteins were revealed using monoclonal anti-Flag antibody M2 (Sigma) and anti-mouse-HRP (horseradish peroxidase) (Amersham Biosciences).

(Phospho)peptide affinity chromatography

Approximately 35×10⁶ HEK293T cells were transfected with either pEF-FLAG-1/mSOCS2 or pEF-FLAG-1/mCIS and were lysed in lysis buffer (20 mM HEPES pH 7, 1 mM MgCl₂, 10 mM KCl, 0.5 mM DTT, 150 mM NaCl, 0.5% NP40, 20% glycerol, 1 mM NaVO₄, Complete™ Protease Inhibitor Cocktail). The lysates were centrifuged for 5 minutes at 10,000 g and loaded on a pre-column with Sepharose 4B beads and streptavidin-agarose to prevent nonspecific interactions. Pre-cleared lysates were then incubated for 2 hours at 4°C with the (phospho)-tyrosine peptides as indicated coupled to streptavidin-agarose beads through their biotin group. The beads were then washed twice with lysis buffer and resuspended in 2× loading buffer (62.5 mM Tris-HCl pH 6.8, 3% SDS, 10% glycerol, 0.01% BFB, 5% β-mercaptoethanol). Specific protein binding was revealed by SDS-PAGE and immunoblotting using the anti-flag antibody and anti-mouse-HRP. The sequences of the used peptides were biotin-QRQPSVK_(P)Y₉₈₅ATLVSNDK and biotin-NHREKSVC_(P)Y₁₀₇₇LGVTSVNR. Synthesis and purification of the biotinylated (phospho)tyrosine peptides and coupling to streptavidin-agarose beads was described before (Eyckerman et al., 2000).

Co-immunoprecipitation

Approximately 2×10⁶ HEK293T cells were transfected with pMet7-Flag-SOCS2 and pMet7-Etag-CIS. Cleared lysates (modified RIPA lysis buffer) were incubated with 4 μg/ml anti-FLAG mouse monoclonal antibody (Sigma) and protein-G-Sepharose (Amersham Biosciences). After immunoprecipitation, SDS-PAGE and western blotting, interactions were detected using anti-E-Tag antibody (Amersham Biosciences).

TAP2 purification and mass spectrometry

HEK293T cells were transfected with the appropriate constructs and lysed in cell lysis buffer (50 mM Tris-HCl pH 8, 10% glycerol, 1% NP40, 150 mM NaCl, 5 mM NaF, 5 μM ZnCl₂, 1 mM Na₃VO₄, 10 mM EGTA, Complete™ Protease Inhibitor Cocktail). The insoluble fraction was centrifuged and the supernatant was incubated with IgG Sepharose (Amersham Biosciences) overnight. The beads were washed three times with washing buffer (20 mM Tris-HCl pH 7.5, 5% glycerol, 0.1% NP40, 150 mM NaCl) and twice with TEV (Tobacco Etch Virus) protease cleavage buffer 1 (10 mM Tris-HCl pH 8, 150 mM NaCl, 0.1% NP40, 0.5 mM EDTA) and were then incubated with TEV protease in TEV protease cleavage buffer 1 for 2 to 4 hours. The beads were then centrifuged and the supernatant was incubated with anti-FLAG agarose (Sigma) in TEV protease cleavage buffer 2 (10 mM Tris-HCl pH 8, 150 mM NaCl, 0.1% NP40) for 2 to 4 hours. The anti-FLAG agarose beads were washed three times with washing buffer and incubated with FLAG peptide for 10 minutes. The beads were spun down and 4× loading buffer was added to the supernatants before loading on a polyacrylamide gel. Proteins were either visualised by silver staining, or for mass spectrometry analysis with Sypro Ruby protein gel stain according to the manufacturer's instructions (Molecular Probes). Proteins of interest were excised and in-gel digested with trypsin as described. The resulting peptide mixture was dried, re-dissolved in 20 μl of 0.1% formic acid in 2/98 (v/v) acetonitrile/water and half of it was applied for nano-LC-MS/MS analysis on an Ultimate (Dionex, Amsterdam, The Netherlands) in-line connected to an Esquire HCT ion trap (Bruker Daltonics, Bremen, Germany). The sample was first trapped on a trapping column (PepMap™ C18 column, 0.3 mm ID ×5 mm, Dionex) and after back-flushing, the sample was loaded on a 75 μm ID ×150 mm reverse-phase column (PepMap™ C18, Dionex). The peptides were eluted with a linear solvent gradient over 50 minutes of 0.1% formic acid in acetonitrile/water (7/3, v/v). Using data-dependent acquisition, only multiple charged ions with intensities above threshold 100,000 were selected for fragmentation. For MS/MS analysis, an MS/MS fragmentation amplitude of 0.7 V and a scan time of 40 milliseconds was used. The fragmentation spectra were converted to Mascot generic files (mgf) using the Automation Engine software (version 3.2, Bruker) and searched using the MASCOT database search engine (http://www.matrixscience.com) against the SwissProt and the NCBInr Database (taxonomy mammalia). Only spectra that exceeded Mascot's threshold score for identify (set at the 95% confidence level) were retained for further manual validation.