PsyProxy
Datasets·Sentiment·10__social_media_sentiment__sentiment140__binary

Sentiment140 Twitter sentiment

Sentiment140 is a public corpus of 1.6M tweets automatically labeled positive or negative based on emoticon presence at collection time. The texts in this corpus predominantly reflect personal experiences and social interactions, often revolving around everyday life events and emotions. Common themes include expressions of excitement about upcoming vacations and family gatherings , as well as sentiments of boredom and illness , such as mentions of migraines and flu symptoms . Additionally, there are instances of sharing compliments and encouragement among friends, alongside casual discussions about media consumption like TV shows and music. Overall, the texts convey a mix of personal reflections, social connections, and light-hearted banter. [Summary on 50 random texts by ChatGPT 4o Mini].

Distribution of Sentiment (positive vs negative)
1
2
100,000 at floor100,000 at ceiling
200,000
items
37,064
holdout n
Sentiment (positive vs negative)
target
Binary
kind
26
systems compared
Criterion validity

Reported holdout systems from the verified card

Binary classification uses FVE as the task-primary metric. Secondary columns keep the companion metrics visible so binary, ordinal, regression, and multiclass cards are not compared through one flattened score.

Source podium · FVE · 10 families
Gold
OpenAI (Rathje)
0.435
Silver
PsyProxy
0.303
Bronze
LIWC
0.195
Model-family mix
OpenAI / LLM · 3PsyProxy · 4Lexicon · 2Baseline · 15Topic model · 2

Best PsyProxy row is #2 overall among all model families on this card.

SystemFamilyVariantFVEAUCF1Primary scale
llmOpenAI Model gpt-4o-mini
OpenAI / LLMpermissive0.4350.9000.843
llmOpenAI Model gpt-4.1-nano
OpenAI / LLMpermissive0.3420.8630.821
psyproxyPsyProxy — Social Economics Lens v0.5 · 1000d
PsyProxypermissive0.3030.8480.776
llmOpenAI Model gpt-5-nano
OpenAI / LLMpermissive0.2860.8450.774
psyproxyPsyProxy — Technology Lens v0.5 · 800d
PsyProxypermissive0.2750.8320.765
psyproxyPsyProxy — Behavioral Sciences Lens v0.5 · 1000d
PsyProxypermissive0.2270.8040.736
psyproxyPsyProxy — Health Lens v0.9 · 1100d
PsyProxypermissive0.2220.8030.727
lexLinguistic Inquiry and Word Count (LIWC)
Lexiconpermissive0.1950.7870.716
lexValence Aware Dictionary and sEntiment Reasoner (VADER)
Lexiconpermissive0.1550.7520.636
baselineTool for the Automatic Analysis of Syntactic Sophistication and Complexity (TAASSC)
Baselinepermissive0.0680.6760.612
baselineTextDescriptives
Baselinepermissive0.0560.6610.617
topicBERTopic
Topic modelpermissive0.0540.6440.675
baselineEmpath
Baselinepermissive0.0300.5920.661
baselineTool for the Automatic Analysis of Lexical Sophistication (TAALES)
Baselinepermissive0.0200.5970.554
baselineTool for the Automatic Analysis of Cohesion (TAACO)
Baselinepermissive0.0170.5840.528
topicHierarchical Dirichlet Process (tomotopy HDP)
Topic modelpermissive0.0080.5500.628
baselineAmazon Video-Games reviews ordinal · via best lens
Baselinepermissive
baselineAmazon Video-Games reviews continuous · via best lens
Baselinepermissive
baselineDisneyland TripAdvisor reviews continuous · via best lens
Baselinepermissive
baselineIMDB movie reviews (ACL) binary · via best lens
Baselinepermissive
baselineDouban movie reviews (Chinese) ordinal · via best lens
Baselinepermissive
baselineDisneyland TripAdvisor reviews binary · via best lens
Baselinepermissive
baselineDruglib drug reviews regression · via best lens
Baselinepermissive
baselineAmazon Video-Games reviews binary · via best lens
Baselinepermissive
baselineDruglib drug reviews ordinal · via best lens
Baselinepermissive
baselineLIAR fact-check statements ordinal · via best lens
Baselinepermissive