Skip to content
Snippets Groups Projects
Commit 008b193e authored by pabvald's avatar pabvald
Browse files

Reset

parent 7482a681
Branches
No related tags found
No related merge requests found
Expand all Show whitespace changes
Inline Side-by-side
Showing
with 0 additions and 111362 deletions
LICENSE deleted 100644 → 0
+ 0
− 30
View file @ 7482a681
BSD License
For SentEval software
Copyright (c) 2017-present, Facebook, Inc. All rights reserved.
Redistribution and use in source and binary forms, with or without modification,
are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.
* Neither the name Facebook nor the names of its contributors may be used to
endorse or promote products derived from this software without specific
prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
README.md deleted 100755 → 0
+ 0
− 244
View file @ 7482a681
# SentEval: evaluation toolkit for sentence embeddings
SentEval is a library for evaluating the quality of sentence embeddings. We assess their generalization power by using them as features on a broad and diverse set of "transfer" tasks. **SentEval currently includes 17 downstream tasks**. We also include a suite of **10 probing tasks** which evaluate what linguistic properties are encoded in sentence embeddings. Our goal is to ease the study and the development of general-purpose fixed-size sentence representations.
**(04/22) SentEval new tasks: Added probing tasks for evaluating what linguistic properties are encoded in sentence embeddings**
**(10/04) SentEval example scripts for three sentence encoders: [SkipThought-LN](https://github.com/ryankiros/layer-norm#skip-thoughts)/[GenSen](https://github.com/Maluuba/gensen)/[Google-USE](https://tfhub.dev/google/universal-sentence-encoder/1)**
## Dependencies
This code is written in python. The dependencies are:
* Python 2/3 with [NumPy](http://www.numpy.org/)/[SciPy](http://www.scipy.org/)
* [Pytorch](http://pytorch.org/)>=0.4
* [scikit-learn](http://scikit-learn.org/stable/index.html)>=0.18.0
## Transfer tasks
### Downstream tasks
SentEval allows you to evaluate your sentence embeddings as features for the following *downstream* tasks:
| Task | Type | #train | #test | needs_train | set_classifier |
|---------- |------------------------------ |-----------:|----------:|:-----------:|:----------:|
| [MR](https://nlp.stanford.edu/~sidaw/home/projects:nbsvm) | movie review | 11k | 11k | 1 | 1 |
| [CR](https://nlp.stanford.edu/~sidaw/home/projects:nbsvm) | product review | 4k | 4k | 1 | 1 |
| [SUBJ](https://nlp.stanford.edu/~sidaw/home/projects:nbsvm) | subjectivity status | 10k | 10k | 1 | 1 |
| [MPQA](https://nlp.stanford.edu/~sidaw/home/projects:nbsvm) | opinion-polarity | 11k | 11k | 1 | 1 |
| [SST](https://nlp.stanford.edu/sentiment/index.html) | binary sentiment analysis | 67k | 1.8k | 1 | 1 |
| **[SST](https://nlp.stanford.edu/sentiment/index.html)** | **fine-grained sentiment analysis** | 8.5k | 2.2k | 1 | 1 |
| [TREC](http://cogcomp.cs.illinois.edu/Data/QA/QC/) | question-type classification | 6k | 0.5k | 1 | 1 |
| [SICK-E](http://clic.cimec.unitn.it/composes/sick.html) | natural language inference | 4.5k | 4.9k | 1 | 1 |
| [SNLI](https://nlp.stanford.edu/projects/snli/) | natural language inference | 550k | 9.8k | 1 | 1 |
| [MRPC](https://aclweb.org/aclwiki/Paraphrase_Identification_(State_of_the_art)) | paraphrase detection | 4.1k | 1.7k | 1 | 1 |
| [STS 2012](https://www.cs.york.ac.uk/semeval-2012/task6/) | semantic textual similarity | N/A | 3.1k | 0 | 0 |
| [STS 2013](http://ixa2.si.ehu.es/sts/) | semantic textual similarity | N/A | 1.5k | 0 | 0 |
| [STS 2014](http://alt.qcri.org/semeval2014/task10/) | semantic textual similarity | N/A | 3.7k | 0 | 0 |
| [STS 2015](http://alt.qcri.org/semeval2015/task2/) | semantic textual similarity | N/A | 8.5k | 0 | 0 |
| [STS 2016](http://alt.qcri.org/semeval2016/task1/) | semantic textual similarity | N/A | 9.2k | 0 | 0 |
| [STS B](http://ixa2.si.ehu.es/stswiki/index.php/STSbenchmark#Results) | semantic textual similarity | 5.7k | 1.4k | 1 | 0 |
| [SICK-R](http://clic.cimec.unitn.it/composes/sick.html) | semantic textual similarity | 4.5k | 4.9k | 1 | 0 |
| [COCO](http://mscoco.org/) | image-caption retrieval | 567k | 5*1k | 1 | 0 |
where **needs_train** means a model with parameters is learned on top of the sentence embeddings, and **set_classifier** means you can define the parameters of the classifier in the case of a classification task (see below).
Note: COCO comes with ResNet-101 2048d image embeddings. [More details on the tasks.](https://arxiv.org/pdf/1705.02364.pdf)
### Probing tasks
SentEval also includes a series of [*probing* tasks](https://github.com/facebookresearch/SentEval/tree/master/data/probing) to evaluate what linguistic properties are encoded in your sentence embeddings:
| Task | Type | #train | #test | needs_train | set_classifier |
|---------- |------------------------------ |-----------:|----------:|:-----------:|:----------:|
| [SentLen](https://github.com/facebookresearch/SentEval/tree/master/data/probing) | Length prediction | 100k | 10k | 1 | 1 |
| [WC](https://github.com/facebookresearch/SentEval/tree/master/data/probing) | Word Content analysis | 100k | 10k | 1 | 1 |
| [TreeDepth](https://github.com/facebookresearch/SentEval/tree/master/data/probing) | Tree depth prediction | 100k | 10k | 1 | 1 |
| [TopConst](https://github.com/facebookresearch/SentEval/tree/master/data/probing) | Top Constituents prediction | 100k | 10k | 1 | 1 |
| [BShift](https://github.com/facebookresearch/SentEval/tree/master/data/probing) | Word order analysis | 100k | 10k | 1 | 1 |
| [Tense](https://github.com/facebookresearch/SentEval/tree/master/data/probing) | Verb tense prediction | 100k | 10k | 1 | 1 |
| [SubjNum](https://github.com/facebookresearch/SentEval/tree/master/data/probing) | Subject number prediction | 100k | 10k | 1 | 1 |
| [ObjNum](https://github.com/facebookresearch/SentEval/tree/master/data/probing) | Object number prediction | 100k | 10k | 1 | 1 |
| [SOMO](https://github.com/facebookresearch/SentEval/tree/master/data/probing) | Semantic odd man out | 100k | 10k | 1 | 1 |
| [CoordInv](https://github.com/facebookresearch/SentEval/tree/master/data/probing) | Coordination Inversion | 100k | 10k | 1 | 1 |
## Download datasets
To get all the transfer tasks datasets, run (in data/downstream/):
```bash
./get_transfer_data.bash
```
This will automatically download and preprocess the downstream datasets, and store them in data/downstream (warning: for MacOS users, you may have to use p7zip instead of unzip). The probing tasks are already in data/probing by default.
## How to use SentEval: examples
### examples/bow.py
In examples/bow.py, we evaluate the quality of the average of word embeddings.
To download state-of-the-art fastText embeddings:
```bash
curl -Lo glove.840B.300d.zip http://nlp.stanford.edu/data/glove.840B.300d.zip
curl -Lo crawl-300d-2M.vec.zip https://s3-us-west-1.amazonaws.com/fasttext-vectors/crawl-300d-2M.vec.zip
```
To reproduce the results for bag-of-vectors, run (in examples/):
```bash
python bow.py
```
As required by SentEval, this script implements two functions: **prepare** (optional) and **batcher** (required) that turn text sentences into sentence embeddings. Then SentEval takes care of the evaluation on the transfer tasks using the embeddings as features.
### examples/infersent.py
To get the **[InferSent](https://www.github.com/facebookresearch/InferSent)** model and reproduce our results, download our best models and run infersent.py (in examples/):
```bash
curl -Lo examples/infersent1.pkl https://dl.fbaipublicfiles.com/senteval/infersent/infersent1.pkl
curl -Lo examples/infersent2.pkl https://dl.fbaipublicfiles.com/senteval/infersent/infersent2.pkl
```
### examples/skipthought.py - examples/gensen.py - examples/googleuse.py
We also provide example scripts for three other encoders:
* [SkipThought with Layer-Normalization](https://github.com/ryankiros/layer-norm#skip-thoughts) in Theano
* [GenSen encoder](https://github.com/Maluuba/gensen) in Pytorch
* [Google encoder](https://tfhub.dev/google/universal-sentence-encoder/1) in TensorFlow
Note that for SkipThought and GenSen, following the steps of the associated githubs is necessary.
The Google encoder script should work as-is.
## How to use SentEval
To evaluate your sentence embeddings, SentEval requires that you implement two functions:
1. **prepare** (sees the whole dataset of each task and can thus construct the word vocabulary, the dictionary of word vectors etc)
2. **batcher** (transforms a batch of text sentences into sentence embeddings)
### 1.) prepare(params, samples) (optional)
*batcher* only sees one batch at a time while the *samples* argument of *prepare* contains all the sentences of a task.
```
prepare(params, samples)
```
* *params*: senteval parameters.
* *samples*: list of all sentences from the tranfer task.
* *output*: No output. Arguments stored in "params" can further be used by *batcher*.
*Example*: in bow.py, prepare is is used to build the vocabulary of words and construct the "params.word_vect* dictionary of word vectors.
### 2.) batcher(params, batch)
```
batcher(params, batch)
```
* *params*: senteval parameters.
* *batch*: numpy array of text sentences (of size params.batch_size)
* *output*: numpy array of sentence embeddings (of size params.batch_size)
*Example*: in bow.py, batcher is used to compute the mean of the word vectors for each sentence in the batch using params.word_vec. Use your own encoder in that function to encode sentences.
### 3.) evaluation on transfer tasks
After having implemented the batch and prepare function for your own sentence encoder,
1) to perform the actual evaluation, first import senteval and set its parameters:
```python
import senteval
params = {'task_path': PATH_TO_DATA, 'usepytorch': True, 'kfold': 10}
```
2) (optional) set the parameters of the classifier (when applicable):
```python
params['classifier'] = {'nhid': 0, 'optim': 'adam', 'batch_size': 64,
'tenacity': 5, 'epoch_size': 4}
```
You can choose **nhid=0** (Logistic Regression) or **nhid>0** (MLP) and define the parameters for training.
3) Create an instance of the class SE:
```python
se = senteval.engine.SE(params, batcher, prepare)
```
4) define the set of transfer tasks and run the evaluation:
```python
transfer_tasks = ['MR', 'SICKEntailment', 'STS14', 'STSBenchmark']
results = se.eval(transfer_tasks)
```
The current list of available tasks is:
```python
['CR', 'MR', 'MPQA', 'SUBJ', 'SST2', 'SST5', 'TREC', 'MRPC', 'SNLI',
'SICKEntailment', 'SICKRelatedness', 'STSBenchmark', 'ImageCaptionRetrieval',
'STS12', 'STS13', 'STS14', 'STS15', 'STS16',
'Length', 'WordContent', 'Depth', 'TopConstituents','BigramShift', 'Tense',
'SubjNumber', 'ObjNumber', 'OddManOut', 'CoordinationInversion']
```
## SentEval parameters
Global parameters of SentEval:
```bash
# senteval parameters
task_path # path to SentEval datasets (required)
seed # seed
usepytorch # use cuda-pytorch (else scikit-learn) where possible
kfold # k-fold validation for MR/CR/SUB/MPQA.
```
Parameters of the classifier:
```bash
nhid: # number of hidden units (0: Logistic Regression, >0: MLP); Default nonlinearity: Tanh
optim: # optimizer ("sgd,lr=0.1", "adam", "rmsprop" ..)
tenacity: # how many times dev acc does not increase before training stops
epoch_size: # each epoch corresponds to epoch_size pass on the train set
max_epoch: # max number of epoches
dropout: # dropout for MLP
```
Note that to get a proxy of the results while **dramatically reducing computation time**,
we suggest the **prototyping config**:
```python
params = {'task_path': PATH_TO_DATA, 'usepytorch': True, 'kfold': 5}
params['classifier'] = {'nhid': 0, 'optim': 'rmsprop', 'batch_size': 128,
'tenacity': 3, 'epoch_size': 2}
```
which will results in a 5 times speedup for classification tasks.
To produce results that are **comparable to the literature**, use the **default config**:
```python
params = {'task_path': PATH_TO_DATA, 'usepytorch': True, 'kfold': 10}
params['classifier'] = {'nhid': 0, 'optim': 'adam', 'batch_size': 64,
'tenacity': 5, 'epoch_size': 4}
```
which takes longer but will produce better and comparable results.
For probing tasks, we used an MLP with a Sigmoid nonlinearity and and tuned the nhid (in [50, 100, 200]) and dropout (in [0.0, 0.1, 0.2]) on the dev set.
## References
Please considering citing [[1]](https://arxiv.org/abs/1803.05449) if using this code for evaluating sentence embedding methods.
### SentEval: An Evaluation Toolkit for Universal Sentence Representations
[1] A. Conneau, D. Kiela, [*SentEval: An Evaluation Toolkit for Universal Sentence Representations*](https://arxiv.org/abs/1803.05449)
```
@article{conneau2018senteval,
title={SentEval: An Evaluation Toolkit for Universal Sentence Representations},
author={Conneau, Alexis and Kiela, Douwe},
journal={arXiv preprint arXiv:1803.05449},
year={2018}
}
```
Contact: [aconneau@fb.com](mailto:aconneau@fb.com), [dkiela@fb.com](mailto:dkiela@fb.com)
### Related work
* [J. R Kiros, Y. Zhu, R. Salakhutdinov, R. S. Zemel, A. Torralba, R. Urtasun, S. Fidler - SkipThought Vectors, NIPS 2015](https://arxiv.org/abs/1506.06726)
* [S. Arora, Y. Liang, T. Ma - A Simple but Tough-to-Beat Baseline for Sentence Embeddings, ICLR 2017](https://openreview.net/pdf?id=SyK00v5xx)
* [Y. Adi, E. Kermany, Y. Belinkov, O. Lavi, Y. Goldberg - Fine-grained analysis of sentence embeddings using auxiliary prediction tasks, ICLR 2017](https://arxiv.org/abs/1608.04207)
* [A. Conneau, D. Kiela, L. Barrault, H. Schwenk, A. Bordes - Supervised Learning of Universal Sentence Representations from Natural Language Inference Data, EMNLP 2017](https://arxiv.org/abs/1705.02364)
* [S. Subramanian, A. Trischler, Y. Bengio, C. J Pal - Learning General Purpose Distributed Sentence Representations via Large Scale Multi-task Learning, ICLR 2018](https://arxiv.org/abs/1804.00079)
* [A. Nie, E. D. Bennett, N. D. Goodman - DisSent: Sentence Representation Learning from Explicit Discourse Relations, 2018](https://arxiv.org/abs/1710.04334)
* [D. Cer, Y. Yang, S. Kong, N. Hua, N. Limtiaco, R. St. John, N. Constant, M. Guajardo-Cespedes, S. Yuan, C. Tar, Y. Sung, B. Strope, R. Kurzweil - Universal Sentence Encoder, 2018](https://arxiv.org/abs/1803.11175)
* [A. Conneau, G. Kruszewski, G. Lample, L. Barrault, M. Baroni - What you can cram into a single vector: Probing sentence embeddings for linguistic properties, ACL 2018](https://arxiv.org/abs/1805.01070)
data/downstream/MRPC/MSR Paraphrase Corpus.lnk deleted 100755 → 0
+ 0
− 0
View file @ 7482a681
File deleted
data/downstream/MRPC/Microsoft Shared Source License.htm deleted 100755 → 0
+ 0
− 199
View file @ 7482a681
<html>
<head>
<meta http-equiv=Content-Type content="text/html; charset=shift_jis">
<meta name=Generator content="Microsoft Word 11 (filtered)">
<title>Microsoft Shared Source License</title>
<style>
<!--
/* Font Definitions */
@font-face
{font-family:"‚l‚r –¾’©";
panose-1:2 2 6 9 4 2 5 8 3 4;}
@font-face
{font-family:Century;
panose-1:2 4 6 4 5 5 5 2 3 4;}
@font-face
{font-family:"\@‚l‚r –¾’©";
panose-1:2 2 6 9 4 2 5 8 3 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
text-align:justify;
text-justify:inter-ideograph;
font-size:10.5pt;
font-family:Century;}
p.MsoPlainText, li.MsoPlainText, div.MsoPlainText
{margin:0cm;
margin-bottom:.0001pt;
text-align:justify;
text-justify:inter-ideograph;
font-size:10.5pt;
font-family:"‚l‚r –¾’©";}
/* Page Definitions */
@page Section1
{size:595.3pt 841.9pt;
margin:99.25pt 87.65pt 3.0cm 87.65pt;
layout-grid:18.0pt;}
div.Section1
{page:Section1;}
-->
</style>
</head>
<body lang=JA style='text-justify-trim:punctuation'>
<div class=Section1 style='layout-grid:18.0pt'>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>Microsoft
Research Paraphrase Corpus</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>&nbsp;</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>This
Microsoft Research Shared Source license agreement (&quot;MSR-SSLA&quot;) is a
legal agreement between you and Microsoft Corporation (&quot;Microsoft&quot; or
&quot;we&quot;) for the software or data identified above, which may include
source code, and any associated materials, text or speech files, associated
media and &quot;online&quot; or electronic documentation and any updates we
provide in our discretion (together, the &quot;Software&quot;). </span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>&nbsp;</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>By
installing, copying, or otherwise using this Software, found at
http://research.microsoft.com/downloads, you agree to be bound by the terms of
this MSR-SSLA. If you do not agree, do not install copy or use the Software.
The Software is protected by copyright and other intellectual property laws and
is licensed, not sold.&nbsp;&nbsp;&nbsp; </span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>&nbsp;</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>SCOPE
OF RIGHTS:</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>You
may use, copy, reproduce, and distribute this Software for any non-commercial
purpose, subject to the restrictions in this MSR-SSLA. Some purposes which can
be non-commercial are teaching, academic research, public demonstrations and
personal experimentation. You may also distribute this Software with books or
other teaching materials, or publish the Software on websites, that are
intended to teach the use of the Software for academic or other non-commercial
purposes.</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>You
may not use or distribute this Software or any derivative works in any form for
commercial purposes. Examples of commercial purposes would be running business
operations, licensing, leasing, or selling the Software, distributing the
Software for use with commercial products, using the Software in the creation
or use of commercial products or any other activity which purpose is to procure
a commercial gain to you or others.</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>If the
Software includes source code or data, you may create derivative works of such
portions of the Software and distribute the modified Software for
non-commercial purposes, as provided herein.</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>&nbsp;</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>In
return, we simply require that you agree: </span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>1.
That you will not remove any copyright or other notices from the Software.</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>2.
That if any of the Software is in binary format, you will not attempt to modify
such portions of the Software, or to reverse engineer or decompile them, except
and only to the extent authorized by applicable law. </span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>3.
That if you distribute the Software or any derivative works of the Software,
you will distribute them under the same terms and conditions as in this
license, and you will not grant other rights to the Software or derivative works
that are different from those provided by this MSR-SSLA. </span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>4.
That if you have created derivative works of the Software, and distribute such
derivative works, you will cause the modified files to carry prominent notices
so that recipients know that they are not receiving the original Software. Such
notices must state: (i) that you have changed the Software; and (ii) the date
of any changes.</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>5.
That Microsoft is granted back, without any restrictions or limitations, a
non-exclusive, perpetual, irrevocable, royalty-free, assignable and
sub-licensable license, to reproduce, publicly perform or display, install,
use, modify, distribute, make and have made, sell and transfer your
modifications to and/or derivative works of the Software source code or data,
for any purpose.&nbsp; </span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>6.
That any feedback about the Software provided by you to us is voluntarily
given, and Microsoft shall be free to use the feedback as it sees fit without
obligation or restriction of any kind, even if the feedback is designated by
you as confidential. </span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>7.
THAT THE SOFTWARE COMES &quot;AS IS&quot;, WITH NO WARRANTIES. THIS MEANS NO
EXPRESS, IMPLIED OR STATUTORY WARRANTY, INCLUDING WITHOUT LIMITATION,
WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, ANY WARRANTY
AGAINST INTERFERENCE WITH YOUR ENJOYMENT OF THE SOFTWARE OR ANY WARRANTY OF
TITLE OR NON-INFRINGEMENT. THERE IS NO WARRANTY THAT THIS SOFTWARE WILL FULFILL
ANY OF YOUR PARTICULAR PURPOSES OR NEEDS. ALSO, YOU MUST PASS THIS DISCLAIMER
ON WHENEVER YOU DISTRIBUTE THE SOFTWARE OR DERIVATIVE WORKS.</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>8.
THAT NEITHER MICROSOFT NOR ANY CONTRIBUTOR TO THE SOFTWARE WILL BE LIABLE FOR
ANY DAMAGES RELATED TO THE SOFTWARE OR THIS MSR-SSLA, INCLUDING DIRECT,
INDIRECT, SPECIAL, CONSEQUENTIAL OR INCIDENTAL DAMAGES, TO THE MAXIMUM EXTENT
THE LAW PERMITS, NO MATTER WHAT LEGAL THEORY IT IS BASED ON. ALSO, YOU MUST
PASS THIS LIMITATION OF LIABILITY ON WHENEVER YOU DISTRIBUTE THE SOFTWARE OR
DERIVATIVE WORKS.</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>9.
That we have no duty of reasonable care or lack of negligence, and we are not
obligated to (and will not) provide technical support for the Software.</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>10.
That if you breach this MSR-SSLA or if you sue anyone over patents that you
think may apply to or read on the Software or anyone's use of the Software,
this MSR-SSLA (and your license and rights obtained herein) terminate
automatically.&nbsp; Upon any such termination, you shall destroy all of your
copies of the Software immediately.&nbsp; Sections 5, 6, 7, 8, 9, 10, 13 and 14
of this MSR-SSLA shall survive any termination of this MSR-SSLA.</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>11.
That the patent rights, if any, granted to you in this MSR-SSLA only apply to
the Software, not to any derivative works you make.</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>12.
That the Software may be subject to U.S. export jurisdiction at the time it is
licensed to you, and it may be subject to additional export or import laws in
other places. You agree to comply with all such laws and regulations that may
apply to the Software after delivery of the software to you.</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>13.
That all rights not expressly granted to you in this MSR-SSLA are reserved.</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>14.
That this MSR-SSLA shall be construed and controlled by the laws of the State
of Washington, USA, without regard to conflicts of law.&nbsp; If any provision
of this MSR-SSLA shall be deemed unenforceable or contrary to law, the rest of
this MSR-SSLA shall remain in full effect and interpreted in an enforceable
manner that most nearly captures the intent of the original language.</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>&nbsp;</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>&nbsp;</span></p>
<p class=MsoPlainText><span lang=EN-US style='font-family:"Courier New"'>Copyright
&copy; Microsoft Corporation. All rights reserved.</span></p>
</div>
</body>
</html>
data/downstream/MRPC/Microsoft Shared Source License.rtf deleted 100755 → 0
+ 0
− 85
View file @ 7482a681
{\rtf1\ansi\ansicpg932\uc2\deff0\stshfdbch11\stshfloch21\stshfhich21\stshfbi0\deflang1033\deflangfe1041{\fonttbl{\f0\froman\fcharset0\fprq2{\*\panose 02020603050405020304}Times New Roman;}
{\f2\fmodern\fcharset0\fprq1{\*\panose 02070309020205020404}Courier New;}{\f11\froman\fcharset128\fprq1{\*\panose 02020609040205080304}\'82\'6c\'82\'72 \'96\'be\'92\'a9{\*\falt MS Mincho};}
{\f21\froman\fcharset0\fprq2{\*\panose 02040604050505020304}Century;}{\f38\froman\fcharset128\fprq1{\*\panose 02020609040205080304}@\'82\'6c\'82\'72 \'96\'be\'92\'a9;}{\f39\froman\fcharset238\fprq2 Times New Roman CE;}
{\f40\froman\fcharset204\fprq2 Times New Roman Cyr;}{\f42\froman\fcharset161\fprq2 Times New Roman Greek;}{\f43\froman\fcharset162\fprq2 Times New Roman Tur;}{\f44\froman\fcharset177\fprq2 Times New Roman (Hebrew);}
{\f45\froman\fcharset178\fprq2 Times New Roman (Arabic);}{\f46\froman\fcharset186\fprq2 Times New Roman Baltic;}{\f47\froman\fcharset163\fprq2 Times New Roman (Vietnamese);}{\f59\fmodern\fcharset238\fprq1 Courier New CE;}
{\f60\fmodern\fcharset204\fprq1 Courier New Cyr;}{\f62\fmodern\fcharset161\fprq1 Courier New Greek;}{\f63\fmodern\fcharset162\fprq1 Courier New Tur;}{\f64\fmodern\fcharset177\fprq1 Courier New (Hebrew);}
{\f65\fmodern\fcharset178\fprq1 Courier New (Arabic);}{\f66\fmodern\fcharset186\fprq1 Courier New Baltic;}{\f67\fmodern\fcharset163\fprq1 Courier New (Vietnamese);}{\f151\froman\fcharset0\fprq1 MS Mincho Western{\*\falt MS Mincho};}
{\f149\froman\fcharset238\fprq1 MS Mincho CE{\*\falt MS Mincho};}{\f150\froman\fcharset204\fprq1 MS Mincho Cyr{\*\falt MS Mincho};}{\f152\froman\fcharset161\fprq1 MS Mincho Greek{\*\falt MS Mincho};}
{\f153\froman\fcharset162\fprq1 MS Mincho Tur{\*\falt MS Mincho};}{\f156\froman\fcharset186\fprq1 MS Mincho Baltic{\*\falt MS Mincho};}{\f249\froman\fcharset238\fprq2 Century CE;}{\f250\froman\fcharset204\fprq2 Century Cyr;}
{\f252\froman\fcharset161\fprq2 Century Greek;}{\f253\froman\fcharset162\fprq2 Century Tur;}{\f256\froman\fcharset186\fprq2 Century Baltic;}{\f421\froman\fcharset0\fprq1 @\'82\'6c\'82\'72 \'96\'be\'92\'a9 Western;}
{\f419\froman\fcharset238\fprq1 @\'82\'6c\'82\'72 \'96\'be\'92\'a9 CE;}{\f420\froman\fcharset204\fprq1 @\'82\'6c\'82\'72 \'96\'be\'92\'a9 Cyr;}{\f422\froman\fcharset161\fprq1 @\'82\'6c\'82\'72 \'96\'be\'92\'a9 Greek;}
{\f423\froman\fcharset162\fprq1 @\'82\'6c\'82\'72 \'96\'be\'92\'a9 Tur;}{\f426\froman\fcharset186\fprq1 @\'82\'6c\'82\'72 \'96\'be\'92\'a9 Baltic;}}{\colortbl;\red0\green0\blue0;\red0\green0\blue255;\red0\green255\blue255;\red0\green255\blue0;
\red255\green0\blue255;\red255\green0\blue0;\red255\green255\blue0;\red255\green255\blue255;\red0\green0\blue128;\red0\green128\blue128;\red0\green128\blue0;\red128\green0\blue128;\red128\green0\blue0;\red128\green128\blue0;\red128\green128\blue128;
\red192\green192\blue192;}{\stylesheet{\qj \li0\ri0\nowidctlpar\aspalpha\aspnum\faauto\adjustright\rin0\lin0\itap0 \fs21\lang1033\langfe1041\kerning2\loch\f21\hich\af21\dbch\af11\cgrid\langnp1033\langfenp1041 \snext0 Normal;}{\*\cs10 \additive
\ssemihidden Default Paragraph Font;}{\*\ts11\tsrowd\trftsWidthB3\trpaddl108\trpaddr108\trpaddfl3\trpaddft3\trpaddfb3\trpaddfr3\trcbpat1\trcfpat1\tscellwidthfts0\tsvertalt\tsbrdrt\tsbrdrl\tsbrdrb\tsbrdrr\tsbrdrdgl\tsbrdrdgr\tsbrdrh\tsbrdrv
\ql \li0\ri0\widctlpar\aspalpha\aspnum\faauto\adjustright\rin0\lin0\itap0 \fs20\lang1024\langfe1024\loch\f21\hich\af21\dbch\af11\cgrid\langnp1024\langfenp1024 \snext11 \ssemihidden Normal Table;}{
\s15\qj \li0\ri0\nowidctlpar\aspalpha\aspnum\faauto\adjustright\rin0\lin0\itap0 \fs21\lang1033\langfe1041\kerning2\loch\f11\hich\af2\dbch\af11\cgrid\langnp1033\langfenp1041 \sbasedon0 \snext15 Plain Text;}}{\*\latentstyles\lsdstimax156\lsdlockeddef0}
{\*\rsidtbl \rsid6188942\rsid14507857\rsid16326027}{\*\generator Microsoft Word 11.0.6359;}{\info{\title [Add Name and Version of Software here]}{\author chrisbkt}{\operator chrisbkt}{\creatim\yr2005\mo3\dy1\hr16\min37}{\revtim\yr2005\mo3\dy1\hr16\min38}
{\version3}{\edmins1}{\nofpages3}{\nofwords895}{\nofchars5104}{\*\company Microsoft Corporation}{\nofcharsws5988}{\vern24703}}\paperw11906\paperh16838\margl1753\margr1753\margt1985\margb1701\gutter0 \deftab851\ftnbj\aenddoc\formshade\horzdoc\dgmargin
\dghspace180\dgvspace180\dghorigin1701\dgvorigin1984\dghshow0\dgvshow2\jcompress\lnongrid
\viewkind4\viewscale100\splytwnine\ftnlytwnine\htmautsp\useltbaln\alntblind\lytcalctblwd\lyttblrtgr\lnbrkrule\nobrkwrptbl\snaptogridincell\allowfieldendsel\wrppunct\asianbrkrule\rsidroot6188942\newtblstyruls\nogrowautofit {\upr{\*\fchars
!%),.:\'3b?]\'7d\'81\'91\'81\'8b\'81\'66\'81\'68\'81\'f1\'81\'8c\'81\'8d\'81\'8e\'81\'41\'81\'42\'81\'58\'81\'72\'81\'74\'81\'76\'81\'78\'81\'7a\'81\'6c\'81\'4a\'81\'4b\'81\'54\'81\'55\'81\'45\'81\'52\'81\'53\'81\'49\'81\'93\'81\'6a\'81\'43\'81\'44\'81\'46\'81\'47\'81\'48\'81\'6e\'81\'70\'a1\'a3\'a4\'a5\'de\'df\'81\'91
}{\*\ud\uc0{\*\fchars
!%),.:\'3b?]\'7d{\uc2\u162 \'81\'91\'81\'8b\'81f\'81h\'81\'f1\'81\'8c\'81\'8d\'81\'8e\'81A\'81B\'81X\'81r\'81t\'81v\'81x\'81z\'81l\'81J\'81K\'81T\'81U\'81E\'81R\'81S\'81I\'81\'93\'81j\'81C\'81D\'81F\'81G\'81H\'81n\'81p\'a1\'a3\'a4\'a5\'de\'df\'81\'91}}}}
{\upr{\*\lchars $([\'5c\'7b\'81\'92\'5c\'81\'65\'81\'67\'81\'71\'81\'73\'81\'75\'81\'77\'81\'79\'81\'6b\'81\'90\'81\'69\'81\'6d\'81\'6f\'a2\'81\'92\'81\'8f}{\*\ud\uc0{\*\lchars
$([\'5c\'7b{\uc2\u163 \'81\'92}{\uc1\u165 \'5c\'81e\'81g\'81q\'81s\'81u\'81w\'81y\'81k\'81\'90\'81i\'81m\'81o\'a2\'81\'92\'81\'8f}}}}\fet0\sectd \linex0\endnhere\sectlinegrid360\sectspecifyl\sectrsid12744622\sftnbj {\*\pnseclvl1
\pnucrm\pnstart1\pnindent720\pnhang {\pntxta \dbch .}}{\*\pnseclvl2\pnucltr\pnstart1\pnindent720\pnhang {\pntxta \dbch .}}{\*\pnseclvl3\pndec\pnstart1\pnindent720\pnhang {\pntxta \dbch .}}{\*\pnseclvl4\pnlcltr\pnstart1\pnindent720\pnhang
{\pntxta \dbch )}}{\*\pnseclvl5\pndec\pnstart1\pnindent720\pnhang {\pntxtb \dbch (}{\pntxta \dbch )}}{\*\pnseclvl6\pnlcltr\pnstart1\pnindent720\pnhang {\pntxtb \dbch (}{\pntxta \dbch )}}{\*\pnseclvl7\pnlcrm\pnstart1\pnindent720\pnhang {\pntxtb \dbch (}
{\pntxta \dbch )}}{\*\pnseclvl8\pnlcltr\pnstart1\pnindent720\pnhang {\pntxtb \dbch (}{\pntxta \dbch )}}{\*\pnseclvl9\pnlcrm\pnstart1\pnindent720\pnhang {\pntxtb \dbch (}{\pntxta \dbch )}}\pard\plain
\s15\qj \li0\ri0\nowidctlpar\aspalpha\aspnum\faauto\adjustright\rin0\lin0\itap0\pararsid12744622 \fs21\lang1033\langfe1041\kerning2\loch\af11\hich\af2\dbch\af11\cgrid\langnp1033\langfenp1041 {\loch\af2\insrsid16326027 \hich\af2\dbch\af11\loch\f2 Microsoft
\hich\af2\dbch\af11\loch\f2 \hich\af2\dbch\af11\loch\f2 Research\hich\af2\dbch\af11\loch\f2 \hich\af2\dbch\af11\loch\f2 Paraphrase\hich\af2\dbch\af11\loch\f2 \hich\af2\dbch\af11\loch\f2 Corpus}{\loch\af2\insrsid6188942\charrsid6188942
\par }{\loch\af2\insrsid6188942\charrsid6188942
\par \hich\af2\dbch\af11\loch\f2 This Microsoft Research Shared Source license agreement ("MSR-SSLA") is a legal agreement between you and Microsoft Corporation ("Microsoft" or "we") for the software or data identified above, which may include so
\hich\af2\dbch\af11\loch\f2 urce code, and any associated materials, text or speech files, associated media and "online" or electronic documentation and any updates we provide in our discretion (together, the "Software").
\par
\par \hich\af2\dbch\af11\loch\f2 By installing, copying, or otherwise using this Software, \hich\af2\dbch\af11\loch\f2
found at http://research.microsoft.com/downloads, you agree to be bound by the terms of this MSR-SSLA. If you do not agree, do not install copy or use the Software. The Software is protected by copyright and other intellectual property laws and is license
\hich\af2\dbch\af11\loch\f2 d\hich\af2\dbch\af11\loch\f2 , not sold.
\par
\par \hich\af2\dbch\af11\loch\f2 SCOPE OF RIGHTS:
\par \hich\af2\dbch\af11\loch\f2 You may use, copy, reproduce, and distribute this Software for any non-commercial purpose, subject to the restrictions in this MSR-SSLA. Some purposes which can be non-commercial are teaching, academic research, public
\hich\af2\dbch\af11\loch\f2
demonstrations and personal experimentation. You may also distribute this Software with books or other teaching materials, or publish the Software on websites, that are intended to teach the use of the Software for academic or other non-commercial purpose
\hich\af2\dbch\af11\loch\f2 s\hich\af2\dbch\af11\loch\f2 .
\par \hich\af2\dbch\af11\loch\f2
You may not use or distribute this Software or any derivative works in any form for commercial purposes. Examples of commercial purposes would be running business operations, licensing, leasing, or selling the Software, distributing the Software for use
\hich\af2\dbch\af11\loch\f2 with commercial products, using the Software in the creation or use of commercial products or any other activity which purpose is to procure a commercial gain to you or others.
\par \hich\af2\dbch\af11\loch\f2 If the Software includes source code or data, you may create derivative works\hich\af2\dbch\af11\loch\f2 of such portions of the Software and distribute the modified Software for non-commercial purposes, as provided herein.
\par
\par \hich\af2\dbch\af11\loch\f2 In return, we simply require that you agree:
\par \hich\af2\dbch\af11\loch\f2 1. That you will not remove any copyright or other notices from the Software.
\par \hich\af2\dbch\af11\loch\f2 2. That\hich\af2\dbch\af11\loch\f2
if any of the Software is in binary format, you will not attempt to modify such portions of the Software, or to reverse engineer or decompile them, except and only to the extent authorized by applicable law.
\par \hich\af2\dbch\af11\loch\f2 3. That if you distribute the Software or any\hich\af2\dbch\af11\loch\f2
derivative works of the Software, you will distribute them under the same terms and conditions as in this license, and you will not grant other rights to the Software or derivative works that are different from those provided by this MSR-SSLA.
\par \hich\af2\dbch\af11\loch\f2 4. That i\hich\af2\dbch\af11\loch\f2
f you have created derivative works of the Software, and distribute such derivative works, you will cause the modified files to carry prominent notices so that recipients know that they are not receiving the original Software. Such notices must state: (i)
\hich\af2\dbch\af11\loch\f2 \hich\af2\dbch\af11\loch\f2 that you have changed the Software; and (ii) the date of any changes.
\par \hich\af2\dbch\af11\loch\f2 5. That Microsoft is granted back, without any restrictions or limitations, a non-exclusive, perpetual, irrevocable, royalty-free, assignable and sub-licensable license, to reproduce, p\hich\af2\dbch\af11\loch\f2
ublicly perform or display, install, use, modify, distribute, make and have made, sell and transfer your modifications to and/or derivative works of the Software source code or data, for any purpose.
\par \hich\af2\dbch\af11\loch\f2 6. That any feedback about the Software provided by y\hich\af2\dbch\af11\loch\f2
ou to us is voluntarily given, and Microsoft shall be free to use the feedback as it sees fit without obligation or restriction of any kind, even if the feedback is designated by you as confidential.
\par \hich\af2\dbch\af11\loch\f2 7. THAT THE SOFTWARE COMES "AS IS", WITH NO WARRANTIES\hich\af2\dbch\af11\loch\f2
. THIS MEANS NO EXPRESS, IMPLIED OR STATUTORY WARRANTY, INCLUDING WITHOUT LIMITATION, WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE, ANY WARRANTY AGAINST INTERFERENCE WITH YOUR ENJOYMENT OF THE SOFTWARE OR ANY WARRANTY OF TITLE OR NON-
\hich\af2\dbch\af11\loch\f2 I\hich\af2\dbch\af11\loch\f2
NFRINGEMENT. THERE IS NO WARRANTY THAT THIS SOFTWARE WILL FULFILL ANY OF YOUR PARTICULAR PURPOSES OR NEEDS. ALSO, YOU MUST PASS THIS DISCLAIMER ON WHENEVER YOU DISTRIBUTE THE SOFTWARE OR DERIVATIVE WORKS.
\par \hich\af2\dbch\af11\loch\f2 8. THAT NEITHER MICROSOFT NOR ANY CONTRIBUTOR TO T\hich\af2\dbch\af11\loch\f2
HE SOFTWARE WILL BE LIABLE FOR ANY DAMAGES RELATED TO THE SOFTWARE OR THIS MSR-SSLA, INCLUDING DIRECT, INDIRECT, SPECIAL, CONSEQUENTIAL OR INCIDENTAL DAMAGES, TO THE MAXIMUM EXTENT THE LAW PERMITS, NO MATTER WHAT LEGAL THEORY IT IS BASED ON. ALSO, YOU MUS
\hich\af2\dbch\af11\loch\f2 T\hich\af2\dbch\af11\loch\f2 PASS THIS LIMITATION OF LIABILITY ON WHENEVER YOU DISTRIBUTE THE SOFTWARE OR DERIVATIVE WORKS.
\par \hich\af2\dbch\af11\loch\f2 9. That we have no duty of reasonable care or lack of negligence, and we are not obligated to (and will not) provide technical support for the Software.
\par \hich\af2\dbch\af11\loch\f2 10. T\hich\af2\dbch\af11\loch\f2
hat if you breach this MSR-SSLA or if you sue anyone over patents that you think may apply to or read on the Software or anyone's use of the Software, this MSR-SSLA (and your license and rights obtained herein) terminate automatically. Upon any such term
\hich\af2\dbch\af11\loch\f2 i\hich\af2\dbch\af11\loch\f2 nation, you shall destroy all of your copies of the Software immediately. Sections 5, 6, 7, 8, 9, 10, 13 and 14 of this MSR-SSLA shall survive any termination of this MSR-SSLA.
\par \hich\af2\dbch\af11\loch\f2 11. That the patent rights, if any, granted to you in this MSR-SSLA only appl\hich\af2\dbch\af11\loch\f2 y to the Software, not to any derivative works you make.
\par \hich\af2\dbch\af11\loch\f2 12. That the Software may be subject to U.S. export jurisdiction at the time it is licensed to you, and it may be subject to additional export or import laws in other places. You agree to comply wit\hich\af2\dbch\af11\loch\f2
h all such laws and regulations that may apply to the Software after delivery of the software to you.
\par \hich\af2\dbch\af11\loch\f2 13. That all rights not expressly granted to you in this MSR-SSLA are reserved.
\par \hich\af2\dbch\af11\loch\f2 14. That this MSR-SSLA shall be construed and controlled by the laws of \hich\af2\dbch\af11\loch\f2
the State of Washington, USA, without regard to conflicts of law. If any provision of this MSR-SSLA shall be deemed unenforceable or contrary to law, the rest of this MSR-SSLA shall remain in full effect and interpreted in an enforceable manner that most
\hich\af2\dbch\af11\loch\f2 \hich\af2\dbch\af11\loch\f2 nearly captures the intent of the original language.
\par
\par
\par \hich\af2\dbch\af11\loch\f2 \hich\f2 Copyright \'a9\loch\f2 Microsoft Corporation. All rights reserved\hich\af2\dbch\af11\loch\f2 .}{\loch\af2\insrsid6188942
\par }}
\ No newline at end of file
data/downstream/MRPC/msr_paraphrase_README.htm deleted 100755 → 0
+ 0
− 1331
View file @ 7482a681
This diff is collapsed.
data/downstream/MRPC/msr_paraphrase_README.rtf deleted 100755 → 0
+ 0
− 894
View file @ 7482a681
This diff is collapsed.
data/downstream/MRPC/msr_paraphrase_data.txt deleted 100755 → 0
+ 0
− 10949
View file @ 7482a681
This diff is collapsed.
data/downstream/MRPC/msr_paraphrase_test.txt deleted 100755 → 0
+ 0
− 1726
View file @ 7482a681
This diff is collapsed.
data/downstream/MRPC/msr_paraphrase_train.txt deleted 100755 → 0
+ 0
− 4077
View file @ 7482a681
This diff is collapsed.
data/downstream/QUORA/quora_duplicate_questions.csv deleted 100644 → 0
NaN GiB (NaN%)
View file @ 7482a681
File suppressed by a .gitattributes entry, the file's encoding is unsupported, or the file size exceeds the limit.
data/downstream/SICK/SICK_test_annotated.txt deleted 100644 → 0
+ 0
− 4928
View file @ 7482a681
This diff is collapsed.
data/downstream/SICK/SICK_train.txt deleted 100644 → 0
+ 0
− 4501
View file @ 7482a681
This diff is collapsed.
data/downstream/SICK/SICK_trial.txt deleted 100644 → 0
+ 0
− 501
View file @ 7482a681
This diff is collapsed.
data/downstream/SST/binary/sentiment-dev deleted 100644 → 0
+ 0
− 872
View file @ 7482a681
This diff is collapsed.
data/downstream/SST/binary/sentiment-test deleted 100644 → 0
+ 0
− 1821
View file @ 7482a681
This diff is collapsed.
data/downstream/SST/binary/sentiment-train deleted 100644 → 0
+ 0
− 67349
View file @ 7482a681
This diff is collapsed.
data/downstream/SST/fine/sentiment-dev deleted 100644 → 0
+ 0
− 1101
View file @ 7482a681
This diff is collapsed.
data/downstream/SST/fine/sentiment-test deleted 100644 → 0
+ 0
− 2210
View file @ 7482a681
This diff is collapsed.
data/downstream/SST/fine/sentiment-train deleted 100644 → 0
+ 0
− 8544
View file @ 7482a681
This diff is collapsed.
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment