Solved exercise: count, filter and summarize a log file in Bash

Bash script with getopts and awk that validates a lab usage log, filters by room and threshold, and totals minutes per user. Explained solution.

Updated on November 2, 2026

Exam Code Bash Programming fundamentals Exam level Tested code

Exam Code: Solved exercise: count, filter and summarize a log file in Bash

Programming Fundamentals (CS1) · Lab exam (shell)Question 2 · 2.5 points · 30 min

The department's computer labs keep a usage log of their workstations. Each useful line has the form user:room:minutes. The user consists only of lowercase letters and digits ([a-z0-9]+). The room consists of letters and digits ([A-Za-z0-9]+). The minutes field is a non-negative integer ([0-9]+) and may have leading zeros. Spaces are not allowed inside a data line.

Write a Bash script main.sh that reads the log from standard input and accepts two options, in any order. -r ROOM restricts the summary to the records of that room, using an exact, case-sensitive comparison. -m MINUTES is a non-negative integer: the minimum total of minutes a user needs to appear in the listing. By default there is no room filter and the threshold is 0. You may use only Bash, coreutils, grep, sed and awk.

Blank lines, including lines made only of spaces and tabs, are ignored and not counted. Lines starting with # are also ignored and not counted. Any other line that does not have exactly three fields in the format above is invalid. Invalid lines are always counted, whatever room was requested with -r.

Output starts with one line per user whose total minutes, within the room filter, are greater than or equal to the threshold. Each line has the form user total sessions, with single spaces. Users are sorted by name in byte order, the order of LC_ALL=C. Then come exactly four summary lines, in this order. records: R counts the valid records that pass the room filter. invalid: I counts the invalid lines. total: T is the sum of minutes of those R records, including users hidden by the threshold. users: U counts the users listed.

If the input is empty, only the four summary lines are printed, all with zeros. The invocation is incorrect if it has an unknown option, an option without a value, extra arguments, or a non-numeric -m. In that case the script writes usage: main.sh [-r ROOM] [-m MINUTES] to standard error and exits with status 2, writing nothing to standard output. The example below runs the script with threshold 50.

Sample input

# lab log, week 12
alice:L1:45
bob:L2:30
alice:L2:50
carol:L1:20

bob:L1:40
alice-smith:L1:15
dave:L3:abc
carol:L1:25

Arguments

-m 50

Expected output

alice 95 2
bob 70 2
records: 6
invalid: 2
total: 210
users: 2

Option parsing with getopts and default values. Rejects unknown options, missing values, extra arguments and a non-numeric minimum: 0.75
Classifies each line (ignored, invalid or valid) in the correct order. The invalid count does not depend on the room filter: 0.75
Accumulates minutes and sessions per user. The room filter applies to the data and the threshold only to the user lines: 0.5
Exact output, also with empty input: users in byte order, followed by the four-line summary: 0.5

Hints

Hint 1 · How do I keep options and data apart?

The options arrive as arguments and the data arrives on standard input. Treat them as two separate problems. Handle the options first with the getopts builtin and an option string that starts with :. That way errors reach your case and Bash does not print its default message. Afterwards, discard the parsed options with shift $((OPTIND - 1)) and check that nothing is left. Validate -m with a regular expression before you use it in a numeric comparison.

Hint 2 · A Bash loop or a single awk pass?

Counting and summing by key is exactly what awk associative arrays are for. Pass the room and the threshold with -v and split fields with -F:. The order of the rules matters. First drop what must be ignored. Then detect and count what is invalid. Apply the room filter only at the end. If you filter before validating, the number of invalid lines depends on -r, and the statement forbids that.

Hint 3 · How do I sort the users without scrambling the summary?

An awk for (u in ...) loop guarantees no order at all. You can hand the sorting to sort, but the four summary lines must stay at the end and in their own order. A classic trick is decorate, sort, undecorate. Prefix each line with a key and a tab that force the position you want, sort the whole output, then strip the key with cut.

Solution

Explained solution

The solution has two clearly separated parts. Bash handles the interface: it reads the options, validates them and decides whether the call is correct. A single awk pass handles the data: it classifies each line, accumulates per user and prints the result. A sort followed by a cut then puts every line in its place.

Setting LC_ALL=C at the top is not decoration. It makes the [a-z] ranges in the regular expressions and the order used by sort behave the same on every machine. The statement asks for byte order, and this setting is how you get it.

#!/usr/bin/env bash
# Lab usage summary: main.sh [-r ROOM] [-m MINUTES] < log
set -u
export LC_ALL=C

usage() {
    echo "usage: main.sh [-r ROOM] [-m MINUTES]" >&2
    exit 2
}

room=""
minimum=0
while getopts ":r:m:" opt; do
    case "$opt" in
        r) room="$OPTARG" ;;
        m) minimum="$OPTARG" ;;
        *) usage ;;
    esac
done
shift $((OPTIND - 1))

[ "$#" -eq 0 ] || usage
[[ "$minimum" =~ ^[0-9]+$ ]] || usage

awk -F: -v room="$room" -v minimum="$minimum" '
BEGIN { threshold = minimum + 0 }
/^[ \t]*$/ || /^#/ { next }
NF != 3 || $1 !~ /^[a-z0-9]+$/ || $2 !~ /^[A-Za-z0-9]+$/ || $3 !~ /^[0-9]+$/ {
    invalid++
    next
}
room != "" && $2 != room { next }
{
    records++
    total += $3
    minutes[$1] += $3
    sessions[$1]++
}
END {
    for (u in minutes) {
        if (minutes[u] >= threshold) {
            printf "1\t%s %d %d\n", u, minutes[u], sessions[u]
            shown++
        }
    }
    printf "2a\trecords: %d\n", records
    printf "2b\tinvalid: %d\n", invalid
    printf "2c\ttotal: %d\n", total
    printf "2d\tusers: %d\n", shown
}' | sort | cut -f2-

Options. The string ":r:m:" declares that -r and -m take a value. The leading colon turns on silent mode. On an unknown option getopts stores ? in opt, and on an option with no value it stores :. Both cases fall into the *) branch and call usage, which writes to standard error and exits with status 2. Since getopts imposes no order, the call -m 45 -r L1 in the fourth test works just as well as the reverse order.

Up-front validation. After shift $((OPTIND - 1)), $@ holds only the non-option arguments. This script accepts none, because the data comes on standard input, so [ "$#" -eq 0 ] rejects any extras. The test [[ "$minimum" =~ ^[0-9]+$ ]] stops a value such as -m ten from reaching awk. If it did, awk would silently turn it into 0, and the script would print a wrong result without any warning.

Classifying lines. In awk, each line goes through the rules from top to bottom, and next stops that walk. The first rule drops blank lines and comments. The second checks for exactly three fields and matches each field against its pattern. On any failure it increments invalid. In the example, alice-smith fails because of the hyphen and dave:L3:abc fails on the minutes, hence invalid: 2. The room filter runs only after that, which is why the -r L1 tests still report 2 invalid lines.

Accumulation. Every record that gets this far adds to the global counters records and total. It also adds to the associative arrays minutes and sessions, which are indexed by user. Adding $3 treats it as a number, so 007 counts as 7, as the last test shows. The threshold is converted to a number in BEGIN with threshold = minimum + 0, so that the comparison is numeric rather than a string comparison.

Ordered output. In END the script walks over the users, prints those with minutes[u] >= threshold and counts them in shown. Each user line is decorated with key 1 and a tab, and the summary lines get keys 2a to 2d. sort then orders the whole output. Lines starting with 1 come before those starting with 2, and within the user lines the name decides. In the C locale a space sorts before any digit, so amy comes before amy2. Finally, cut -f2- removes the key.

Edge cases. With empty input, awk processes no lines but still runs END. Uninitialised variables print as 0 with %d, so the four summary lines come out as zeros, as the empty-input test requires. Also note that total does not depend on the threshold. In the example, carol is not listed because she has only 45 minutes, but those 45 minutes are still part of total: 210.

Code and tests on GitLab

Test cases

Case	Input	Expected output	Actual output	Result
Statement example with threshold 50	`-m 50 # lab log, week 12 alice:L1:45 bob:L2:30 alice:L2:50 carol:L1:20 bob:L1:40 alice-smith:L1:15 dave:L3:abc carol:L1:25`	`alice 95 2 bob 70 2 records: 6 invalid: 2 total: 210 users: 2`	`alice 95 2 bob 70 2 records: 6 invalid: 2 total: 210 users: 2`	OK
No options	`# lab log, week 12 alice:L1:45 bob:L2:30 alice:L2:50 carol:L1:20 bob:L1:40 alice-smith:L1:15 dave:L3:abc carol:L1:25`	`alice 95 2 bob 70 2 carol 45 2 records: 6 invalid: 2 total: 210 users: 3`	`alice 95 2 bob 70 2 carol 45 2 records: 6 invalid: 2 total: 210 users: 3`	OK
Room filter L1	`-r L1 # lab log, week 12 alice:L1:45 bob:L2:30 alice:L2:50 carol:L1:20 bob:L1:40 alice-smith:L1:15 dave:L3:abc carol:L1:25`	`alice 45 1 bob 40 1 carol 45 2 records: 4 invalid: 2 total: 130 users: 3`	`alice 45 1 bob 40 1 carol 45 2 records: 4 invalid: 2 total: 130 users: 3`	OK
Both options in the other order	`-m 45 -r L1 # lab log, week 12 alice:L1:45 bob:L2:30 alice:L2:50 carol:L1:20 bob:L1:40 alice-smith:L1:15 dave:L3:abc carol:L1:25`	`alice 45 1 carol 45 2 records: 4 invalid: 2 total: 130 users: 2`	`alice 45 1 carol 45 2 records: 4 invalid: 2 total: 130 users: 2`	OK
Empty input	(empty)	`records: 0 invalid: 0 total: 0 users: 0`	`records: 0 invalid: 0 total: 0 users: 0`	OK
Leading zeros, name prefixes and a short line	`zoe:B7:007 amy2:B7:5 amy:B7:0 zoe:B7`	`amy 0 1 amy2 5 1 zoe 7 1 records: 3 invalid: 1 total: 12 users: 3`	`amy 0 1 amy2 5 1 zoe 7 1 records: 3 invalid: 1 total: 12 users: 3`	OK

Actual outputs: code compiled with Bash 5.2 and run in an isolated container on 2 November 2026.

Complexity

Time. Let N be the number of lines and U the number of distinct users. The awk pass is O(N) on average, because each line does a constant number of regex matches and hash-table accesses. sort works on U + 4 lines, which costs O(U log U). The total is O(N + U log U). When each user has many records, reading the input dominates in practice.

Memory. The file is never stored. awk keeps only two entries per user, in minutes and sessions, so memory is O(U). A log of millions of lines with a few dozen users fits easily in the memory limit. A solution that did data=$(cat) and scanned the variable several times would use O(N) memory for no reason.

Common mistakes

Applying the room filter before validating the line. The invalid count then depends on -r, and the summary no longer meets the specification.
Forgetting shift $((OPTIND - 1)) or not checking for extra arguments. A call such as main.sh -r L1 L2 is then accepted silently, and the user believes two rooms were filtered.
Reading with cat | while read and accumulating in Bash variables. The loop runs in a subshell, so the counters are lost when the pipeline ends and the summary prints zeros.
Assuming that for (u in minutes) visits the keys alphabetically. It may happen to work on small data and then fail on other data or with another awk implementation.
Sorting the whole output with sort without decorating it first. The records: and total: lines get mixed in with the user names and the summary is scrambled.
Applying the threshold to records and total as well. A related slip is comparing against -m without validating it, so a non-numeric value silently becomes 0.

Variants

Summarize by room instead of by user

Add a -g option that switches the grouping key. In awk, compute key = (byroom ? $2 : $1) and use key as the index of both arrays. The rest of the script, including the decoration for sort, stays the same. Think about whether -r and -g together make sense: with both, the listing would have a single line.

List users by minutes, highest first

Keep key 1 for the user lines and add a second field with the zero-padded minutes, for example printf "1\t%010d\t%s %d %d\n". Give the summary lines an empty second field, as in "2a\t\trecords: %d\n". Sort by fields with sort -t $'\t' -k1,1 -k2,2r -k3,3. The third key breaks ties by name. All lines now have the same shape, so cut -f3- undecorates them in one step.

Read from a file or from standard input

If the professor wants an optional file name, accept at most one argument after the options. Relax the check [ "$#" -eq 0 ] to [ "$#" -le 1 ]. Check that the file is readable with [ -r "$1" ] and pass it to awk as an operand. With no argument, awk reads standard input and nothing else changes.

What the professor will ask

“Suppose that instead of awk you had written "sort log | while IFS=: read u r m; do records=$((records+1)); ...; done" followed by "echo records: $records". Why could it print records: 0 even though the file has data?”

In Bash every element of a pipeline runs in a subshell, the last one included, unless shopt -s lastpipe is set with job control off. The loop increments its own copy of records, and that copy disappears when the pipeline ends. The echo that follows sees the parent shell's variable, which is still unset or 0. There are three fixes. You can do all the work inside the subshell, for example by grouping the loop and the echo in braces. You can feed the loop with a redirection or a process substitution instead of a pipe. Or, as in the solution, you can let awk do the accumulation and the summary in a single process.

On the syllabus

Using the command interpreter is part of the syllabus of CC3301 Systems Software Programming at Universidad de Chile, whose programme lists the shell alongside processes, pipes and the Linux file API. That is the natural setting for this exercise, where an awk | sort | cut pipeline does the work and subshells explain the professor's question. The Programming Fundamentals course at Instituto Politécnico Nacional covers control statements, functions and files, which appear here from the scripting side.

Grouping by key connects with the structured data topic in Programming (1236) at UNAM and with the arrays unit of Algorithms and Programming I (95.11) at Universidad de Buenos Aires. The awk associative array is the high-level version of those structures.

Article generated with AI.larebelion