HW 6 — Words and Meanings with spaCy & WordNet

LIN 301: Computation for Linguists

Overview

In this assignment, you’ll combine spaCy and WordNet to explore how computers represent word meaning.
You’ll tokenize and lemmatize text, examine how WordNet structures semantic relations, and test similarity across words and semantic fields.

By the end, you should be able to:

• Use spaCy to process and analyze text.
• Access WordNet synsets, definitions, hypernyms, and hyponyms.
• Compute semantic similarity between words.
• Build a semantic field lexicon using WordNet and evaluate its coverage.

Part 0 — Setting Up Your Document

• Create a new file called hw6_lastname_firstname.qmd
• Create an executable code block loading the following packages:

import spacy
from nltk.corpus import wordnet as wn
import pandas as pd
import matplotlib.pyplot as plt
import nltk

nltk.download('wordnet')

# Load spaCy model
nlp = spacy.load("en_core_web_sm")

Part 1 — Tokenizing & Lemmatizing Text

• Visit Project Gutenberg and download a book of your choice as .txt file.
• Delete any information about Project Gutenberg at the beginning and end of the text.
• Use spaCy to:
  • Tokenize and lemmatize it using spaCy.
  • Filter for content words (nouns, verbs, adjectives, adverbs).
  • Count frequency of lemmas.
• Then, create a pandas DataFrame showing:
  • lemma
  • POS
  • frequency
• Finally, plot the 10 most frequent content lemmas in a bar chart.

Part 2 — Exploring Meaning with WordNet

• Next, pick the five most frequent lemmas from Part 1.
• For each lemma, print all WordNet synsets.
• For each synset, print:
  • Definition
  • Example sentence (if available)
  • Hypernyms
  • Hyponyms
• Note: when listing hypernyms/hyponyms, you’re printing synset names (e.g., move.v.02) or their lemma names—either is fine, but you should be consistent.

Part 3 — Semantic Similarity

• Using your 10 most frequent content lemmas, construct 5 pairs of words
• Compute Wu-Palmer similarity scores for the first sense of each pair.
• Discuss: which word pairs are most similar? Do you agree with the scores that you arrived at using WordNet?

Part 4 — Building a Semantic Field

• Using the “verbs of violence” as your model, you’re going to create a new semantic field, such as “verbs of motion”, “verbs of cooking”, “emotion adjectives”, or “kinship nouns”.
• First, write 1-2 sentences defining the category.
• Second, create a preliminary word list of at least 10 words that you believe to be part of that category. For example, for “verbs of violence”, we would create something like:

violence_words = ["hit","strike","attack","fight","kill","punch","shoot","stab","murder","slap"]

• These will be your “seed_words”. Note that if your words in questions are not verbs, you will need to change the code a bit. -Use wn.synsets(word, pos='n') for nouns, 'a' for adjectives, 'v' for verbs, and 'r' for adverbs.
• Discuss:
  • How many new items did WordNet add beyond your seeds?
  • Which obvious ones are missing, if any?
  • Are there any weird additions to the list?
• Finally, run wup similarity scores on all of the words in your list. What is the average overall similarity score?

Submission

1. Submit your .qmd file with all code and outputs.
   
2. Be sure to replace “lastname_firstname” with your actual last name and first name.

Grading Rubric (100 pts total)

Part 0 — Setup (5 pts)

Criteria	Points
Correctly imports spacy, wordnet, pandas, matplotlib, and downloads WordNet data	3
Loads the model en_core_web_sm without errors	2

Part 1 — Tokenizing & Lemmatizing (20 pts)

Criteria	Points
Text successfully loaded and cleaned (Gutenberg headers/footers removed)	6
Correctly tokenizes and lemmatizes; filters for NOUN, VERB, ADJ, ADV	6
Creates a DataFrame showing lemma / POS / frequency	4
Bar chart of top 10 lemmas, labeled and readable	4

Part 2 — Exploring Meaning with WordNet (20 pts)

Criteria	Points
Prints all synsets for each of the five lemmas	6
Includes definition, example, hypernyms, and hyponyms for each	8
Output is clear and consistently formatted	6

Part 3 — Semantic Similarity (20 pts)

Criteria	Points
Constructs five word pairs and computes Wu–Palmer similarity for first senses	10
Displays results clearly (table or printout)	5
Discussion of which pairs are most similar and whether results make sense	5

Part 4 — Building a Semantic Field (30 pts)

Criteria	Points
Defines the category in 1–2 sentences	5
Creates an appropriate seed list (≥10 items)	5
Uses WordNet to expand or explore the field; documents new items	10
Discusses coverage (missing/odd items, outliers)	5
Computes pairwise Wu–Palmer similarities and reports average score	5

Presentation & Reproducibility (5 pts)

Criteria	Points
File named correctly (hw6_lastname_firstname.qmd); all code runs top-to-bottom; results organized	5