Computation for Linguists

Learning the Command Line: File Extensions, Beginning Grep

Dr. Andrew M. Byrd

2025-09-24

Review

  • What did we learn last time about the Terminal?

Review of Terminal Commands

  • mkdir
  • ls
  • rmdir, rm
  • touch
  • curl -o file.txt
  • cd
  • source script.sh

Review Activity

  • Navigate to your LIN_301 folder
  • Using Terminal:

File Extensions

File Extensions

When programming, there are lots of different file types:

  • txt - a (raw) text file
  • html - an html file, to be viewed in a web browser
  • xml - an Exstensible Markup Language file
  • md - a Markdown file (can be converted into html)
  • tex - a LaTeX file (can be converted into a pdf)
  • py - a Python script
  • r - an R script

Converting Files

All of these can be easily converted, as they’re all just text!

# create md file
touch test.md

# start with a md file
echo "# Hello World" > test.md
echo "This is *italic* and **bold**." >> test.md

# convert markdown → html
pandoc test.md -o test.html

# convert html → plain text
pandoc test.html -t plain -o test.txt
  • You can download pandoc if you’d like; if not, you can just follow along for this section.

Converting to docx and odt

  • Let’s create a .sh file called convert.sh
  • We’ll paste in this code:
# 1) Make a tiny Markdown source
echo "# Hello!" > demo.md
echo "This is a *line* and **text**." >> demo.md

# 2) Convert MD → HTML
pandoc demo.md -o demo.html

# 3) Convert MD → DOCX and → ODT
pandoc demo.md -o demo.docx
pandoc demo.md -o demo.odt
  • Run the code:
source convert.sh

Identifying What’s Inside: file

  • Let’s see what the Terminal thinks is inside these files:
file demo.html demo.docx demo.odt

What are these things?

  • Let’s use a command head, to print out what’s inside the file itself.
head -n 20 demo.md
head -n 20 demo.html
head -n 20 demo.odt
head -n 20 demo.docx

What are these things?

  • Word & Google Docs aren’t text formats at all, they’re zip files!
unzip -l demo.docx | head
unzip -l demo.odt  | head
  • Contrast that with:
unzip -l demo.md | head
unzip -l demo.html | head

Flattening Files

  • We can use Pandoc to flatten all three to plain text:
pandoc demo.html -t plain -o from_html.txt
pandoc demo.docx -t plain -o from_docx.txt
pandoc demo.odt  -t plain -o from_odt.txt

File Extensions

  • How much data does each file type take up?
ls -lh demo.* from_*.txt

The Takeaway?

wc, grep, and piping

Pride & Prejudice

  • Earlier you:
  • Using cd, navigate to LIN_301/p_and_p and using ls confirm that the file is there

Pride & Prejudice

  • Confirm that you indeed have Pride and Prejudice downloaded:
less p_and_p.txt
  • You can scroll up and down in the document using the up/down arrows
  • Hit spacebar to jump down a page
  • Press q to return to the command prompt

Let’s Analyze this Thing

In Terminal, type in the following code:

wc p_and_p.txt
  • What is this?
    • Lines, Words, Characters
    • How many of each?

wc Flags

  • Lines only:
wc -l p_and_p.txt
  • Words only:
wc -w p_and_p.txt
  • How about characters only?

grep

  • Super useful & common command used to search within documents
  • Here’s how it works:
grep "search_string" file_name.txt
  • This code gives all of the lines where “search_string” is found in “file_name.txt”

grep

  • Using this syntax, search for all instances of “yes” in Pride and Prejudice
grep "search_string" file_name.txt
  • Look at the output in the Terminal – do you notice anything funny?

Piping Commands together

  • So far we can:
    • count how many lines, words, and characters there are in a file
    • see which lines have instances of a specific string there are in a file
  • But what if we wanted to count how many lines have that specific string?
    • We use the pipe < | > (just above the enter key)

Piping Commands together

  • Type in this code:
grep "yes" p_and_p.txt | wc
  • This tells us the number of words, lines, and characters are within lines that contain “yes”
  • How could we narrow this to just tell us how many lines contain “yes”?

Activity: Frankenstein

Using the Terminal:

  • Create a new folder called frank
  • Download Frankenstein from https://www.gutenberg.org/cache/epub/84/pg84.txt
  • Name the file frank.txt and move it to the frank folder
  • Using grep, wc, and |, count how many lines have the word monster in them