Computation for Linguists

Writing Functions

Dr. Andrew M. Byrd

2025-11-14

Review

What did you learn last time?

Recap from Last Time

spaCy
WordNet
analyzing non-English languages
transcribing to Latin

Functions

What are functions?

blocks of code that run only when called
return data as a result
make your code simpler by allowing you to avoid code repetition

Avoiding repetition

In the past, when we had to run the same task across multiple objects, we often just copy and pasted our code

book_path = "austen/pride_and_prejudice.txt"
with open(book_path, mode="r", encoding="utf-8") as book_file:
  book_lines = book_file.readlines()
clean_lines = [line.strip().lower() for line in book_lines]
pride_and_prejudice = [word for line in clean_lines for word in line.split() if word]
    
book_path = "austen/sense_and_sensibility.txt"
with open(book_path, mode="r", encoding="utf-8") as book_file:
  book_lines = book_file.readlines()
clean_lines = [line.strip().lower() for line in book_lines]
sense_and_sensibility = [word for line in clean_lines for word in line.split() if word]

Wouldn’t it be great if we could only do this once?

We’ve Seen Functions Before

from collections import Counter
import spacy

nlp_en = spacy.load("en_core_web_sm")
nlp_es = spacy.load("es_core_news_sm")

def is_preposition(tok):
  # True for tokens that are prepositions or postpositions
  return tok.pos_ == "ADP"

def prep_stats(nlp, text, lang_label):
  doc = nlp(text)
  # word-like tokens only for the denominator
  word_tokens = [t for t in doc if t.is_alpha]
  total = len(word_tokens)
  preps = [t.text for t in doc if is_preposition(t)]
  n = len(preps)
  rate = (n / total) if total else 0.0
  print(f"{lang_label}: {n} prepositions / {total} word tokens ({rate:.2%})")
  print("Top forms:", Counter(w.lower() for w in preps).most_common())

# Run on English and Spanish samples
prep_stats(nlp_en, text_en, "English")
prep_stats(nlp_es, text_es, "Spanish")

And Here

import spacy
from spacy.matcher import PhraseMatcher
from nltk.corpus import wordnet as wn

nlp = spacy.load("en_core_web_sm")

# WordNet helpers (via NLTK) – returns only verb synsets
def verb_senses(word):
    return wn.synsets(word, pos='v')  # verb synsets only

# Collects all (recursive) hyponyms for a verb synset (troponyms in WN terms).
# This function recursively collects all *hyponyms* (a.k.a. “troponyms” for verbs)
# of a given synset. A hyponym is a more *specific* instance of an action.
# Example: the verb “attack.v.01” has hyponyms like “bomb.v.01”, “invade.v.01”, etc.
def all_verb_hyponyms(root):
      seen, stack = set(), [root]
    while stack:
        cur = stack.pop()
        if cur in seen:
            continue
        seen.add(cur)
        # For verbs, .hyponyms() are the troponyms
        stack.extend(cur.hyponyms())
    return {s for s in seen if s.pos() == 'v'}

# This function extracts lemma names (the “canonical” word forms)
# from a set of synsets, e.g. 'strike.v.01' → ['strike', 'hit', 'smite', ...].
# Optionally filters out multiword expressions like "shoot_down".
def lemmas_from_synsets(synsets, keep_multiword=False):
    out = set()
    for s in synsets:
        for lem in s.lemmas():
            name = lem.name().lower()
            if not keep_multiword and "_" in name:
                continue
            out.add(name.replace("_", " "))
    return out

# Build a violence lexicon from WordNet using a few intuitive seeds
seed_verbs = ["attack", "assault", "hit", "strike", "punch", "kick", "stab", "shoot", "beat"]
base_synsets = []
for w in seed_verbs:
    ss = verb_senses(w)
    if ss:
        # take the most “central” sense by picking the one with most hyponyms
        ss_scored = sorted(ss, key=lambda s: len(s.hyponyms()), reverse=True)
        base_synsets.append(ss_scored[0])

# expand via hyponyms (troponyms)
expanded = set()
for s in base_synsets:
    expanded |= all_verb_hyponyms(s)

# collect lemmas (single-token by default)
violent_verb_lemmas = sorted(lemmas_from_synsets(expanded, keep_multiword=False) | set(seed_verbs))
print(f"{len(violent_verb_lemmas)} violent verb lemmas (sample):", violent_verb_lemmas[:25])

# 4) spaCy PhraseMatcher by lemma — IMPORTANT: run full pipeline on patterns
matcher = PhraseMatcher(nlp.vocab, attr="LEMMA")
patterns = list(nlp.pipe(violent_verb_lemmas))  # not make_doc: we need lemmas
matcher.add("VIOLENCE", patterns)

Writing Functions

Let’s write a simple function

def plus_two(number):
    '''
    This function adds 2 to the input.
    '''
    output = number + 2
    return(output)

How does a function work?
- What is return()?

Loading Your Function

plus_two(1)

Writing a more complex function

def add_and_mult(a, b):
    '''
    This function adds a to b,
    and multiplies a and b.
    '''
    add = a + b
    mult = a * b
    return(add, mult)

Loading Our Function

add_and_mult(4, 6)

Not just math

def initialize(full_name):
    '''
    Return the person's initials.
    '''
    name_parts = full_name.split(" ")
    initial_list = [name[0]+"." for name in name_parts]

    output = ""
    for initial in initial_list:
        output = output + initial

    return(output)

Running our Function

initialize("Louisa May Alcott")
# 'L.M.A.'

initialize("Edgar Allan Poe")
# 'E.A.P.'

initialize("Jane Austen")
# 'J.A.'

We can put it in a separate `.py` script

Let’s copy this code and put it in a separate script called initialize.py

from initializer import initialize

initialize("Louisa May Alcott")
# 'L.M.A.'

initialize("Edgar Allan Poe")
# 'E.A.P.'

initialize("Jane Austen")
# 'J.A.'

Writing an “Read Book” Function

Earlier, we saw duplicated code, how might we simplify this into a single function?

book_path = "austen/pride_and_prejudice.txt"
book_file = open(book_path, mode = 'r')
book_lines = book_file.readlines()
clean_lines = [line.rstrip().lower() for line in book_lines]
pride_and_prejudice = [w for w in clean_lines if w]

book_path = "austen/sense_and_sensibility.txt"
book_file = open(book_path, mode = 'r')
book_lines = book_file.readlines()
clean_lines = [line.rstrip().lower() for line in book_lines]
sense_and_sensibility = [w for w in clean_lines if w]

Writing an “Read Book” Function

Our script needs to:

load a path that connects to a text file
open the file
read in all of the lines, then
remove all of the newlines and convert everything to lower case, then
filter out all of the empty lines, then
assign the end result to a variable.

Activity: Write a Function

Write a function that loads up a file
Assign that function’s output (return()!) to a variable name
Confirm that your function works by loading up a file we’ve analyzed before (like alice.txt)

def read_book(path):
    with open(path, mode="r", encoding="utf-8") as book_file:
        book_lines = book_file.readlines()
    clean_lines = [line.strip().lower() for line in book_lines]
    
    # Split each line into words and flatten the list
    words = [word for line in clean_lines for word in line.split() if word]
    return words

alice = read_book("alice.txt")
print(alice[:200])

Final Activity

Final Activity: Review

import spacy

content_lemmas = [t.lemma_.lower() for t in doc
                  if not (t.is_stop or t.is_punct or t.is_space or t.like_num)]

Final Activity: Review

import spacy
from nltk.corpus import wordnet as wn
from nltk.corpus.reader.wordnet import NOUN, VERB, ADJ, ADV

# Load spaCy
nlp = spacy.load("en_core_web_sm")

# Sentence with both noun and verb "bear"
text = "The bears bear their burdens bravely."
doc = nlp(text)

# Map spaCy POS tags to WordNet POS tags -- this is a **function**, we'll get to these soon
def get_wordnet_pos(spacy_pos):
    if spacy_pos.startswith("N"):
        return NOUN
    elif spacy_pos.startswith("V"):
        return VERB
    elif spacy_pos.startswith("J"):
        return ADJ
    elif spacy_pos.startswith("R"):
        return ADV
    return None

# Loop through tokens and look up WordNet entries
for token in doc:
    wn_pos = get_wordnet_pos(token.tag_)
    lemma = token.lemma_.lower()

    if wn_pos and not token.is_stop and not token.is_punct:
        synsets = wn.synsets(lemma, pos=wn_pos)
        print(f"\n{token.text.upper()} ({token.pos_}) → lemma: {lemma}")
        for s in synsets[:3]:  # show just the first 3 senses
            print(f"  - {s.definition()}  [examples: {s.examples()}]")

Final Activity

In this function (called nlp_it) we are going to:
- Take in a English string as an argument
- Process that string using nlp()
- Eliminate stopwords, punctuation, etc.
- Then process each lemma using WordNet, printing up the synset, definition, and examples
- Note: you will note need to return() anything here

Computation for Linguists

Review

Recap from Last Time

Functions

What are functions?

Avoiding repetition

We’ve Seen Functions Before

And Here

Writing Functions

Writing Functions

Loading Your Function

Writing a more complex function

Loading Our Function

Not just math

Running our Function

We can put it in a separate .py script

Writing an “Read Book” Function

Writing an “Read Book” Function

Activity: Write a Function

Final Activity

Final Activity: Review

Final Activity: Review

Final Activity

We can put it in a separate `.py` script