sileod/Discovery
Mining Discourse Markers for Unsupervised Sentence Representation Learning
This project provides large, high-quality datasets of adjacent sentence pairs, each linked by a specific 'discourse marker' (like "therefore," "personally," or "sadly"). These datasets help train systems to understand how sentences relate to each other, improving tasks like identifying sentiment or determining if one statement implies another. It's for researchers and developers building advanced natural language understanding models.
No commits in the last 6 months.
Use this if you need extensive, diverse, and clean data to train machine learning models for understanding the relationship and flow between sentences in text.
Not ideal if you're looking for an out-of-the-box solution for a specific application without any machine learning development.
Stars
61
Forks
3
Language
Jupyter Notebook
License
Apache-2.0
Category
Last pushed
May 31, 2023
Commits (30d)
0
Get this data via API
curl "https://pt-edge.onrender.com/api/v1/quality/nlp/sileod/Discovery"
Open to everyone — 100 requests/day, no key needed. Get a free key for 1,000/day.
Higher-rated alternatives
google/langfun
OO for LLMs
tanaos/artifex
Small Language Model Inference, Fine-Tuning and Observability. No GPU, no labeled data needed.
preligens-lab/textnoisr
Adding random noise to a text dataset, and controlling very accurately the quality of the result
vulnerability-lookup/VulnTrain
A tool to generate datasets and models based on vulnerabilities descriptions from @Vulnerability-Lookup.
masakhane-io/masakhane-mt
Machine Translation for Africa