Abstract:
Social science-related NLP tasks, such as emotion or humor detection,
capture important language semantics to add to the implicit pragmatics from
text. Instruction tuning has been shown to improve the many capabilities of
large language models (LLMs) such as commonsense reasoning, reading comprehension,
and computer programming. However, little is known about the
effectiveness of instruction tuning on the social domain where implicit pragmatic
cues are often needed to be captured. We explore the use of instruction
tuning for social science NLP tasks and introduce Socialite-llama--
an open-source, instruction-tuned Llama2. On a suite of 20 social science
tasks, Socialite-llama improves upon the performance of Llama2 as well
as matches or improves upon the performance of a state-of-the-art, multitask
finetuned model on a majority of them. Further, Socialite-llama
also leads to improvement on 5 out of 6 related social tasks as compared to
Llama2, suggesting instruction tuning can lead to generalized social understanding.
All resources including our code, model and dataset can be found
throughbit.ly/socialitellama.